Backend LLM Engineer

Backend LLM Engineer at LiteLLM | Y Combinator's Work at a Startup Backend LLM Engineer at LiteLLM(W23) https://www.workatastartup.com/companies/litellm $120K - $180K • 0.25% - 0.75% Call every LLM API like it's OpenAI 100+ LLMs San Francisco, CA, US / Remote (MX) Full-time US citizenship/visa not required 1+ years About LiteLLM LiteLLM (https://github.com/BerriAI/litellm) is a Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - Bedrock, Azure, OpenAI, VertexAI, Cohere and is used by companies like Rocket Money, Adobe, Twilio, and Siemens. About the role TLDR LiteLLM is an open-source LLM Gateway with 34K+ stars on GitHub and trusted by companies like NASA, Rocket Money, Samsara, Lemonade, and Adobe. We’re rapidly expanding and seeking our 6th Engineer focused on owning ‘excellence’ for unified API’s across Core LLM’s (openai/gemini/anthropic models). What is LiteLLM LiteLLM provides an open source Python SDK and Python FastAPI Server that allows calling 100+ LLM APIs (Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic) in the OpenAI format We just hit $6M ARR and have raised a $1.6M seed round from Y Combinator, Gravity Fund and Pioneer Fund. You can find more information on our website, Github and Technical Documentation. Why do companies use LiteLLM enterprise Companies use LiteLLM Enterprise once they put LiteLLM into production and need enterprise features like Prometheus metrics (production monitoring) and need to give LLM access to a large number of people with SSO (secure sign on) or JWT (JSON Web Tokens) What you will be working on Skills: Python, LLM APIs, FastAPI, High-throughput/low-latency As the Backend LLM Engineer, you'll be responsible for ensuring LiteLLM unifies the format for calling LLM APIs in the broader OpenAI + Anthropic spec. This involves writing transformations to convert API requests from OpenAI/Anthropic spec to various LLM provider formats, building provider-agnostic u