←
LiteLLM (YC W23)1,324 posts
LiteLLM (YC W23) avatar✓

LiteLLM (YC W23) ✓

@LiteLLM

The industry standard AI Gateway for models, mcps and agents github.com/BerriAI/litellm

◷ Joined December 2022▧ 136 Photos and videos
Moe Khalil avatar
⟳ LiteLLM (YC W23) retweeted
Moe Khalil@Mokhalil01·5h

I’ve tried a bunch of swarm frameworks and none of them work that well We want to build a good swarm framework into our SDK at @LiteLLM To do this, we want to work with devs that are actually building swarms in prod If this is you please reply or dm me, would love to chat

LiteLLM (YC W23) avatar
LiteLLM (YC W23)@LiteLLM·8h

LiteLLM can now stream text-to-speech audio on /v1/audio/speech. Your app can play the clip while the provider is still sending chunks. Here is how to get started with streamed speech: - Send the request: call POST /v1/audio/speech with "stream": true, or use stream=True in the SDK - Play the audio: LiteLLM sends each chunk as the provider makes it. A long @ElevenLabs clip starts to play after 0.72 seconds. - Track the request: when the audio ends, LiteLLM writes the spend row and releases the key’s max_parallel_requests slot. Merged PR: github.com/BerriAI/litellm/p…

0:10
Ishaan avatar
⟳ LiteLLM (YC W23) retweeted
Ishaan@ishaan_jaff·13h

We’re launching agent evaluations in the @LiteLLM SDK for developers who want to compare models and harnesses on their own tasks. Before you had to write the eval loop, create temporary workspaces and collect the results in a spreadsheet. Now you can reuse your agent, define typed tasks with a success check, and call litellm.agent_eval(). You get a scorecard with pass rate, cost, tokens and duration.

Post attachment
Moe Khalil avatar
⟳ LiteLLM (YC W23) retweeted
Moe Khalil@Mokhalil01·16h

We're launching agent evals in the LiteLLM SDK. You can now compare models and harnesses on your own tasks with litellm.agent_eval(). Read more below ↓

Post attachment
Moe Khalil avatar
⟳ LiteLLM (YC W23) retweeted
Moe Khalil@Mokhalil01·Nov 17

You can now use your Cursor subscription through the @LiteLLM Gateway! Read more below ↓

Post attachment
LiteLLM (YC W23) avatar
LiteLLM (YC W23)@LiteLLM·Nov 17

LiteLLM now supports currencies other than USD for prices and spend. Here is how to use your currency in 3 simple steps: 1 - Set prices in your currency: send pricing_currency: "EUR" with input_cost_per_token and output_cost_per_token on a chat request. LiteLLM converts the prices to USD for spend tracking. 2- Use daily ECB rates: LiteLLM gets the reference rates from the European Central Bank and caches them. You do not set exchange rates yourself. 3- See spend in your currency: on the Usage dashboard, select a currency. The card shows an estimate next to the USD total, with the date of the rate. More details in the PRs: github.com/BerriAI/litellm/p… github.com/BerriAI/litellm/p… github.com/BerriAI/litellm/p…

0:09
Moe Khalil avatar
⟳ LiteLLM (YC W23) retweeted
Moe Khalil@Mokhalil01·Nov 16

Founders building agents with usage-based pricing: how do you handle cost-tracking, spend limits, and charging users? Is it all custom code? Would you be open to having a gateway handle the complexities here? For context, we're working on a customer portal for people using LiteLLM in prod, and would love to hear from people who are living and breathing these problems

LiteLLM (YC W23) avatar
LiteLLM (YC W23)@LiteLLM·Nov 15

What shipped in LiteLLM this week: - Customer Portal (beta): serve your own customers through LiteLLM and attribute traffic and inference spend to each customer, team and member. - /v1/messages is fully in Rust. - Day 0 model support: Claude Fable 5.2, Claude Mythos 5.2, Gemini 4 Carbon, Gemini 4 Hydrogen and Gemini Decision 1, across Anthropic, Vertex AI, Bedrock, Azure and GitHub Copilot. - @replicate : video, image and speech models on /v1/videos, /v1/images/generations and /v1/audio/speech. Join us live for office hours Fridays at noon Pacific to learn more.

3:19
Moe Khalil avatar
⟳ LiteLLM (YC W23) retweeted
Moe Khalil@Mokhalil01·Nov 15

FYI if you’re struggling with this we built the @LiteLLM customer portal for exactly this problem. Would love to chat :)

LiteLLM (YC W23) avatar
LiteLLM (YC W23)@LiteLLM·Nov 14

Generate videos on @fal through the LiteLLM Gateway, with the cost tracked per clip. Two simple steps to generate a video on Fal via LiteLLM: 1 - Add the model: put the @fal video model in config.yaml or add in the Admin UI 2 - Send the request: call POST /v1/videos with a prompt, a size and a length in seconds. LiteLLM records the price of the clip based on the length and resolution. Learn about the models supported: models.litellm.ai/providers

0:10
Moe Khalil avatar
⟳ LiteLLM (YC W23) retweeted
Moe Khalil@Mokhalil01·Nov 14

If your AI product is usage based, you’ve probably struggled with setting budgets on your customers, capping them accurately, and load balancing between them The new @LiteLLM Customer Portal handles it all for you Out now, read more below

Post attachment
LiteLLM (YC W23) avatar
LiteLLM (YC W23)@LiteLLM·Nov 13

Great walkthrough from Amazon Web Services (AWS) team on Building a self-service LLM gateway with Amazon Bedrock and LiteLLM Blog: aws.amazon.com/blogs/machine… S/o to Sitaraman Vijay Krishna, Eric Christensen, Saurav Bhattacharya for the deep dive!

Yassin avatar
⟳ LiteLLM (YC W23) retweeted
Yassin@YassinKortam·Nov 13

LiteLLM Private Registry is now available to customers with an existing LiteLLM confidentiality agreement. It provides early fixes for the most severe vulnerabilities. All security fixes still go public. OSS access policy is in development. litellm.ai/blog?post=litellm…

Post attachment