We moved DeepSeek V4.1 Flash on AMD MI355X from vLLM to SGLang.
Median time to first token on live traffic went from 388 ms to 193 ms. No failed requests during the switch, and nothing changed for anyone calling the API.
Dedicated endpoints: openrelay.inc/inference?utm_…
Excited to share that we're leading @OpenRelayInc's $10M seed round!
OpenRelay builds inference-native infrastructure, matching each workload to the hardware that suits it best. In just five months, they've grown from $0 to $10M in annualized run rate, processing over 5 trillion tokens a month across 20+ regions and 10 hardware types.
I first met @jadenswang and @prashant8124 this summer during @ycombinator's S26 batch and was immediately struck by how complementary their backgrounds were. Jaden started a data center business in college before joining Voltage Park, while Prashant was one of the founding engineers on AWS Bedrock.
As open models become increasingly capable and the hardware landscape diversifies, developers have more opportunities to improve the performance and economics of inference, but few have the infrastructure or expertise to take full advantage of them.
We're thrilled to be partnering with the OpenRelay team, alongside @EniacVC and @Standard_Cap.
More on why we invested: uncorkcapital.com/blog/inves…
The future of inference isn't everyone renting the same GPUs from neocloud training clusters. We're building a vertically integrated inference delivery network to change that.
Just raised $10M led by @uncorkcap with participation from @EniacVC and @Standard_Cap . Time to build.
Call one provider and you get their prices, their rate limits and their 3 a.m. outage.
OpenRelay puts one endpoint in front of GPUs across providers and routes every request to the best silicon. Provider down? We reroute.
Change one line in your SDK.
openrelay.inc/inference-deli…
OpenRelay is a delivery network for AI inference.
We pool GPUs across providers and locations and route every request to the best silicon. Provider down? We reroute.
We bill flat rate, not per token. Just create an API key and change one line.
openrelay.inc/inference-deli…
Reserved GPUs sit idle most of the month.
Bring Your Own Capacity: onboard your fleet with one command, no open ports. We serve inference on the idle hours and pay you for them. You want a node back, it's yours again.
And we run it for free.
openrelay.inc/bring-your-own…
Your agent can manage its own compute on OpenRelay.
Ask for an H100 and it deploys the VM. Ask for a batch and it runs it overnight. It watches what it spends, live, and stops at the budget you set.
Give your agent an OpenRelay key, then let it cook.
docs.openrelay.inc/docs/cli/…
Laya makes decisions instead of writing text. Use a few hundred labeled examples to set thresholds before relying on it. The playground has four presets.
Docs: docs.openrelay.inc/docs/infe…
Model by Convai Innovations: huggingface.co/convaiinnovat…
Same three primitives: choice, score, yes/no. Up to 64 questions per request, one forward pass, a probability distribution and a confidence on every answer.
A four-question ticket triage is 345 tokens, or $0.000007.
Decision models are a new category: Jev from TypeSafe launched last week, typed answers with probabilities instead of text.
Laya is the open-source one. It is live on OpenRelay: POST /v1/decisions, $0.02 per 1M input tokens, nothing else billed.
we've partnered with @OpenRelayInc as a compute partner to provide private AI inference
hmu if you need provably end-to-end encrypted tokens
register for capacity here: openrelay.inc/inference-cata…
The essays that shaped how I think were written by someone I assumed I would never meet. Today he spent an afternoon helping two founders (@jadenswang ) get better. Thank you @paulg and @jesslivingston for having us! This was the most memorable afternoon of my career