←
e6data103 posts
e6data avatar

e6data

@e6data

Agent-native data foundations to scale your AI apps, analytics, and agents. Get more intelligence per dollar, keep your leverage, and account for every action.

◷ Joined December 2023▧ 41 Photos and videos
e6data avatar
e6data@e6data·Nov 12

Our very own Vivek Garg spoke recently about building the data platform for an agent-native world, and the downsampling barriers to retaining agent event history which is increasingly important to overcome for regulatory needs. The talk covers an approach to building data foundations that can accept more data without losing control, including ingestion at 1.5M events/sec at peak into a single Apache Iceberg table with new data queryable in about 30 seconds, and these three takeaways: 1️⃣Write files scales with workers. Committing to the table does not, because every commit has to advance the same table state. 2️⃣Put one committer in front of the table. Workers keep writing Parquet in parallel, and the committer batches their files into one commit. 3️⃣Remove the commit path conflicts doesn’t make commits free. Each commit adds metadata, so pick an interval that meets your freshness target and keep snapshots and manifests trimmed. You can watch the full talk here: youtube.com/watch?v=dLRiNA0O… A big thank you to @streamnativeio for hosting!

Prakalp Choubey avatar
⟳ e6data retweeted
Prakalp Choubey@ChoubeyPrakalp·Nov 10

we might gradually see top frontier LLMs (Mythos, Astra etc) getting benchmarked more on hardware engineering with new releases

e6data avatar
e6data@e6data·Nov 5

We’ve built a new query engine from the ground up to handle agent demand, and it’s already live in production with our customers. “1. The workload. A database query is predictable. An AI query could run for a week, and the systems to manage those requests don't exist yet.”

Ben Horowitz on what he expects in AI over the next 8 to 12 months. He thinks the infrastructure for AI is wrong, and it reminds him of the early days of cloud computing. Back then, everyone thought they could put their existing software on the other side of the wire. It didn't work, and everything got rebuilt: storage, networking, operating systems, virtualization, containers. This is why he expects the same with AI: 1. The workload. A database query is predictable. An AI query could run for a week, and the systems to manage those requests don't exist yet. 2. The chips. Nvidia's GPUs were built for video games. Ben calls them power-hungry and low-yield for what they do today. 3. The models. Training them means sucking down all the data on the internet, and he doubts that is where we end up. His conclusion: from the model on down, everything is going to change, and there will be a lot of opportunity in that. Full breakdown here: theaiopportunities.com/p/ben… If you build or invest in AI, I go deeper on this in AI Market Fit: theaiopportunities.com/

1:40
e6data avatar
e6data@e6data·Nov 3

The worker-sizing problems and scaling challenges are only going to get worse with agents creating new databases at an exponential rate Very excited to see what the team at PostHog cooks up

PostHog @posthog Nov 2 Article Why we're re-rebuilding our warehouse product Five months ago, we rebuilt PostHog's warehouse product on DuckDB instead of ClickHouse. We did it because a shared ClickHouse cluster couldn't handle hours-long modeling queries, and it lacked a

e6data avatar
e6data@e6data·Nov 2

...and we’re building them from the ground up. If you're struggling to control your data platform costs at scale, reach out for a conversation.

e6data avatar
e6data@e6data·Nov 2

“The energy per query is falling incredibly dramatically.” - @sama, ADIPEC Opening Ceremony 2026 He’s right. Cheaper AI queries means more demand for more queries, and the difference lands on your lakehouse compute bill. ROI from AI now depends on agent-ready data foundations.

Post attachment
e6data avatar
e6data@e6data·Oct 13

Another example of model training optimizations solving token costs As the dust settles, we’re building the data foundations with the price-performance capabilities agent-scale velocity will demand Query, ingest, ETL Agent workflows -> agent traces Reach out if you’d like to learn more

We were able to fine tune an open weight model (Qwen3.5-9B) on proprietary internal data at @coinbase, and it out performed frontier models, at much lower cost and speed, for a specialized task (fraud prevention in this case). I think we'll see more of this pattern inside companies. The frontier models are amazing generalists. But increasingly you'll see specialist models out perform on more narrow tasks, either working on teams, or by themselves. We're making it easy to fine tune models internally for our employees. We even have a skill for it in our internal harness that makes it easy to spin these up and start training them: "/fine-tune-and-serve-llm I want to fine-tune Qwen for payments risk review. help me get started" I imagine we'll have some backend services that could be replaced entirely by this new architecture.

Post attachmentPost attachment
e6data avatar
e6data@e6data·Sep 24

We love the builders who challenge assumptions because when your team starts saying, “Yep, all good” to basically everything, eventually you develop trust issues, so why are we trusting AI agents who grade their own work this way too? This is why Jev is getting so much attention. Our teammate Prakalp’s post breaks down how Jev grades itself with more confidence. Jev returns a probability you can test against your own data. You decide how sure the agent needs to be before it acts, and that bar can differ from one action to the next. In one benchmark, putting Jev in front of the agent’s tool choice cut wrong picks from 16.8% to 7.3%. His advice for anyone building agents is to stop asking yours if it’s sure. Like a ‘yes man’ coworker, it will always say yes.

@typesafeai's Jev returns decisions with probabilities you can actually test. Here's how building agents might just have changed forever

e6data avatar
e6data@e6data·Sep 17

Can AI data readiness be avoided? Actually, yes. We’ve seen this before. You’re eager to bring AI into your enterprise, and ROI sounds incredible… until your tech diligence uncovers the overhaul your data platform needs to achieve the outcomes you want. The reasons are common: - Context is scattered. - Freshness varies. - Access gets complicated. - Governance adds constraints. ... and more. The reality is, most data platforms have blockers for you to get AI right. If this sounds familiar, and you or someone you know has taken on AI work that created infrastructure modernization and possibly migration requirements that will take quarters to deliver, we have the playbook to skip the delay and leapfrog your readiness to faster ROI. If you start with one workflow, the one you think matters most. Our stack gives it the evidence, context, and controls it actually needs, on your existing environment. We work around the gaps blocking that outcome, and leave your data platform alone. Join us at Gartner Data & Analytics Summit, Mumbai, where Vignesh Ganesan is hosting a session to pull back the curtain on how enterprises can take a lean approach to leapfrog unnecessary stages that block you, and prove ROI from your agents in production in weeks. ‘ROI from agents despite gaps and imperfections in data platforms.’ 21 September | 3:50 PM Theatre, Exhibit Showcase And if you can’t make it, you can find us at Booth 305. See you there!

Post attachment
e6data avatar
e6data@e6data·Sep 15

Agent querying lives in a different league. If tomorrow, your CEO required every team to run agentic analytics on your data, how much of your infrastructure would break? Find us at Booth H68 at Big Data LDN next week for the playbook to scale without breaking. If you ask ten people how to get AI agents to query your data directly, you’ll get ten different answers involving caches, indexes, and a warehouse that struggles to perform under concurrency spikes. Most data infrastructure was built for analyst teams running dashboards, not agent swarms firing off hundreds of queries a minute, at all hours, with no pattern to predict. We built e6data so you can handle 1,000+ queries per second with under one-second latency, bring streaming data into queryable Iceberg tables in about 15 seconds, and run it all in your existing cloud or on-prem environment, so your data never leaves your systems. Can’t make it? Feel free to schedule a time with us earlier here: calendly.com/d/dz84-xmy-bsr/…

Post attachment
e6data avatar
e6data@e6data·Aug 20

Congrats to Ramp! This will make the token layer boring (in the best possible way). Once everyone stops bleeding token cost, the next storm is going to hit: agent query volume. We’ve been quietly building for it at e6data. Router solves tokens, we're solving the compute engine.

Ramp@tryrampAug 19

We bought router.com. Now we're building the best one. Our customers buy quadrillions of tokens every month through Ramp, and we've run our own AI on this router for 3 years to keep costs down. Today it's yours at router.com

e6data avatar
e6data@e6data·Aug 13

Nobody files a ticket for a slow dashboard. They just stop opening it and start asking you in Slack. That's how you lose a data product without a single alert firing. Read more here: e6data.com/problems/customer…

0:10
e6data avatar
e6data@e6data·Aug 12

Your compaction job fails nightly with a commit conflict. Losing the race to the streaming writer isn't the first fix; reconciling delete files at scan time is. Rewrite those first, short job, commits before the stream can invalidate it, split compute and then fix the race.

Post attachment
e6data avatar
e6data@e6data·Jun 24

We’re thrilled to introduce e6data’s native MCP. Connect MCP-compatible agents like Claude Code, Codex, Cursor, and custom AI workflows directly to your lakehouse, without extra infra or custom integrations. Huge shoutout to Arnav and Tanay for bringing this to life.

1:47
e6data avatar
e6data@e6data·Jun 11

There’s more hiding inside open lakehouse architectures than most teams realize. Catch @vishnuv9248's talk on June 18 at 11:30 AM PDT, then drop by booth 548 to meet the e6data team. Cost, governance, portability, and the freedom to run anywhere.

0:08