LiteLLM, an open-source LLM Gateway with 27,000+ GitHub stars, is seeking a founding backend engineer to help scale their platform that allows companies to call 100+ LLM APIs in OpenAI format. The Y Combinator-backed startup has raised $1.6M in seed funding and serves enterprise clients including NASA, Adobe, and Rocket Money who need unified access to multiple AI models.
What you should know: LiteLLM provides both a Python SDK and FastAPI server that standardizes calls across major AI providers like OpenAI, Anthropic, Bedrock, Azure, and VertexAI.
- The role offers $160K-$220K salary plus 0.50%-3.00% equity for candidates with 1+ years of backend experience.
- The position requires working closely with the CEO and CTO to maintain API transformations that convert OpenAI-format requests to provider-specific formats.
- Companies upgrade to LiteLLM Enterprise when they need production monitoring through Prometheus metrics and secure access management via SSO or JWT.
Key technical challenges: The founding engineer will tackle critical infrastructure improvements to handle enterprise-scale deployments.
- Migrate key systems from httpx to aiohttp to achieve 10x higher throughput.
- Add support for new features like Anthropic’s thinking parameter on Bedrock.
- Handle provider-specific limitations, such as OpenAI o-1’s lack of streaming support.
- Ensure the platform can compute aggregate spending with 1M+ database logs.
- Implement cost tracking and logging for Anthropic’s /v1/messages API.
The tech stack: The platform runs on Python and FastAPI for backend services, with JavaScript/TypeScript handling frontend components.
- Infrastructure includes Redis, Postgres, S3, Google Cloud Storage, Datadog, and Slack API integrations.
- The team prioritizes open-source development and direct user engagement to understand enterprise requirements.
Interview process: Candidates face a compressed one-day virtual interview spanning 3.5 hours total.
- Two 60-minute coding sessions work through real LiteLLM GitHub issues, evaluating problem-solving and code quality.
- A 45-minute architecture interview covers AI infrastructure and scalability scenarios.
- A 30-minute founder chat discusses working style and company direction.
Company background: Founded in 2023 by Krrish Dholakia and Ishaan Jaffer, LiteLLM emerged from Y Combinator’s W23 batch as a two-person team.
- The startup addresses the complexity enterprises face when integrating multiple AI providers into their workflows.
- Their open-source approach has attracted significant developer adoption, with enterprise clients upgrading for production-grade features.
Recent Stories
DOE fusion roadmap targets 2030s commercial deployment as AI drives $9B investment
The Department of Energy has released a new roadmap targeting commercial-scale fusion power deployment by the mid-2030s, though the plan lacks specific funding commitments and relies on scientific breakthroughs that have eluded researchers for decades. The strategy emphasizes public-private partnerships and positions AI as both a research tool and motivation for developing fusion energy to meet data centers' growing electricity demands. The big picture: The DOE's roadmap aims to "deliver the public infrastructure that supports the fusion private sector scale up in the 2030s," but acknowledges it cannot commit to specific funding levels and remains subject to Congressional appropriations. Why...
Oct 17, 2025Tying it all together: Credo’s purple cables power the $4B AI data center boom
Credo, a Silicon Valley semiconductor company specializing in data center cables and chips, has seen its stock price more than double this year to $143.61, following a 245% surge in 2024. The company's signature purple cables, which cost between $300-$500 each, have become essential infrastructure for AI data centers, positioning Credo to capitalize on the trillion-dollar AI infrastructure expansion as hyperscalers like Amazon, Microsoft, and Elon Musk's xAI rapidly build out massive computing facilities. What you should know: Credo's active electrical cables (AECs) are becoming indispensable for connecting the massive GPU clusters required for AI training and inference. The company...
Oct 17, 2025Vatican launches Latin American AI network for human development
The Vatican hosted a two-day conference bringing together 50 global experts to explore how artificial intelligence can advance peace, social justice, and human development. The event launched the Latin American AI Network for Integral Human Development and established principles for ethical AI governance that prioritize human dignity over technological advancement. What you should know: The Pontifical Academy of Social Sciences, the Vatican's research body for social issues, organized the "Digital Rerum Novarum" conference on October 16-17, combining academic research with practical AI applications. Participants included leading experts from MIT, Microsoft, Columbia University, the UN, and major European institutions. The conference...