←
Adam Prout407 posts
Adam Prout avatar

Adam Prout

@a_prout

Distinguished Engineer @ Microsoft. Ex-MemSQL/SingleStoreDB cofounder. Database builder. One trick pony.

◷ Joined June 2012▧ 11 Photos and videos
Adam Prout avatar
Adam Prout@a_prout·Nov 17

HorizonDB is the second SQL database service I've been involved in building from initial design to GA and beyond. Folks usually brag about features or performance of new software, but for HorizonDB I'm proud that we've kept a high quality and reliability bar from day one. HorizonDB has matured much faster than past projects I've worked on by leveraging everything we know from decades of designing, building, testing and running database services combined with agents that can execute on our ideas much faster than we can. Teams that care about reliable software and focus their agents and energy on it are going to start to stand out more clearly now than in the past. A 10x reliability advantage is more achievable today (even if it's still hard to benchmark it across services and workloads).

We are thrilled to introduce the fastest cloud-native Postgres Database, and the best way to build operational applications in the One Lake!

Adam Prout avatar
Adam Prout@a_prout·Nov 5

Great Post. Since returning to MSFT a few years ago, I've also been pondering why the company has challenges with product fit-and-finish or with warts as Slava calls them (Azure Portal, Teams, Windows UX, Word's web UI, Azure Devops, even VSCode has gotten a few of them recently). Hangs or random annoying UX bugs don't get priority. As a database guy with very little patience for tooling that doesn't work well, this drives me a bit crazy. The database teams obviously work hard on reliability because it's a core part of what our customers expect from us. Random database hangs are never acceptable. Not all products have the same market pressures (from buyers) put directly on product reliability/usability is one way to look at how/why this happened.

Some things I learned working at Microsoft: 1/ large numbers everywhere. $100M ARR service isn't even a footnote on executive slide decks. Leadership time is too valuable, come back when you're $1B+. Msft ecosystem is so rich that $1B+ sales are common (it's ludicrous how much work goes into closing these). Azure has ~250 services, some are a few mil and stagnant, others in hypergrowth, some are cash cows, you see every quadrant of the growth-share matrix. Imagine the imperial navy with ships of vastly different sizes as far the eye can see 2/ more on immense scale-- the smallest of Azure services run on 10K+ VMs. Large services run on 1M+ (some maybe 10M+ VMs, idk). Msft is stamping one new datacenter into the ground every four days (!!!!), a bewildering rate. Demand is so high, even with all this buildout we're forever capacity constrained. Azure brings in $100B in revenue and is growing 40%+ annually (at this size!!) There are ~225K msft employees. You almost have to think in scientific notation, 10^4 is a tiny number here 3/ msft is totalitarian re serving the customer. It's counterintuitive to outsiders, but the warts in msft products fall out of this culture. For example, it would be easy to improve the Azure Portal. But in all my time working with customers and partners here, I haven't heard even one person complain about the portal UI. Msft invests $0.00 dollars in anything that doesn't serve the customer and is completely unapologetic about downstream implications of this mindset 4/ msft can reshape itself overnight (when it has the will). People argued whether we waited too long to adapt after Storm-0558/Midnight Blizzard hacks (with the benefit of hindsight, we definitely did). But once the leadership team privileged security, everyone's job changed in two weeks. LT tied every engineer's comp to security, centralized reporting of embedded security teams, every org reshuffled its roadmap. Within days everyone was pushing PRs with security fixes. I would have thought it inconceivable to mobilize 200K+ employees this fast 5/ Azure is complicated. Postgres depends on compute, networking, AKS (or EKS for you aws people), secrets, etc. AKS depends on compute, networking, some datastore (probably Cosmos but idk), a dozen other things. Cosmos depends on... you get the idea. When anything breaks, everything above can break too. Some construction crew accidentally cuts a fiber cable in Brazil South, or a dedicated hardware compression chip browns out in storage, or compute failure rate in west eu spikes for some reason, or a million other failure modes. Msft is relentless about improving this, idk exactly but fully loaded probably 50% of engineer time goes into resilience 6/ re lifers who work 20+ years here, SV sneers at this but I now see it differently. I was once on the bridge debugging some IO issue, we call storage, the guy diagnoses it in about a minute, we chitchat, turns out he built NT asyncio support. Or same story but with networking, the guy built NT tcp/ip stack. I used to have a boss here, we were bumping into some problem, she casually goes oh yeah, we hit that when I was working on .NET bytecode. The ex-DEC VAX/VMS database engineers built SQL Server, some are still here. I love bumping into these people 7/ I never *really* understood distribution until I got here. As an example, our team had a customer in Seoul w/ a deep performance problem. We had to fix it to unlock one of these giant contracts. We don't speak Korean, they don't speak English. Of course msft has specialists in Seoul who get to the customer site before you could even get on a plane. There is an office in every major city, a relationship with every major entity, legal experts in every jurisdiction. You can reach these ppl instantly. Claude tells me msft has ~180 offices in ~100 countries (!!) U.S. government has ~270 consulates in ~170 countries. So msft outposts aren't quite as vast as U.S. diplomatic network, but pretty vast 8/ in the 90s everyone was afraid of msft. The lore is that BillG was ruthless. He'd clone you, beat you via ecosystem/distribution strength, and moreover make the ecosystem stronger with the new product. I wasn't there to compare, but msft of today is ruthless about winning, has the will to adapt, and has the wherewithal to undertake colossal projects. Ppl see Teams bugs or whatever and completely misread the signals. Teams is less polished than Slack because msft doesn't care one iota about polish. It cares about winning, and is frighteningly good at it. Underestimate at your own risk

Adam Prout avatar
Adam Prout@a_prout·Oct 8

PostgreSQL famously does "double buffering" of data pages. It's one of the few SQL databases that uses buffered IO so some of its pages end up cached in both the kernel page cache and in PostgreSQL's own page cache (shared buffers). A nice side-effect of swapping out PostgreSQL's storage layer is that the double buffer disappears improving memory efficiency. This is why Azure HorizonDB uses a larger shared_buffers then typical PostgreSQL deployments.

Post attachment
Adam Prout avatar
Adam Prout@a_prout·Oct 2

We've done some work on HorizonDB to simplify PostgreSQL's interactions with cloud storage. Using storage designed for PostgreSQL lets PostgreSQL focus on running queries and transactions while letting the storage layer deal with everything else (i.e., hardening the log across AZs, replicating data to read replica's and standbys, handling dirty page writing, handling torn pages, etc.). The result is simpler algorithms and simpler code. This is the best way to make a system more reliable and more performant.

Post attachment
Adam Prout avatar
Adam Prout@a_prout·Oct 1

hmm... The pg_duckdb extension has been around for a few years with the same capabilities (in open source). This seems like a private fork of it based on the description? github.com/duckdb/pg_duckdb

A painful part of working with your data has always been that your live data and historical data are stuck in separate systems: the order a customer just placed lives in your database, while their last five years of orders sit in a data lake in S3. And answering a real question usually needs both at once (is this a normal purchase for them, or should we flag it?), and to do that you had to move the data together first, copying history out of the data lake into your database (or the other way around), because the database couldn’t read it where it lived. That meant guessing ahead of time which data you’d want, keeping a second copy of it all, building pipelines to move it, and constantly syncing so the two didn’t drift apart. A lot of plumbing, and slow going, all before you could answer one question. And even then, the answers were only as fresh as your last sync. That now changes with Aurora PostgreSQL, which can call your live data and historical data in S3 together, in a single query. No copying, no pipelines to keep in sync. And it’s fast, because we’ve built in DuckDB, a popular open source engine that’s really good at reading and analyzing data right where it’s stored. DuckDB reads the open formats like Parquet and Iceberg already sitting in your data lake, so there’s nothing to convert or move. As folks build AI agents into their apps, the data their agent needs will depend on the task in front of it. Being able to query that specific data live, instead of copying it over just in case, is gonna be a big help for builders. aws.amazon.com/blogs/aws/ama…

Post attachment
Dave Eyler avatar
⟳ Adam Prout retweeted
Dave Eyler@dreyler·Sep 29

We asked an agent to tune the same slow query on 3 Postgres MCP servers: Supabase, Neon and Postgres MCP Pro. Baton let us see why the agent ran extra queries and built unnecessary indexes, and once we knew why, we knew how to fix it. goodtiming.ai/blog/agent-tas…

Adam Prout avatar
Adam Prout@a_prout·Aug 29

Redis if implemented by experienced systems developers that know what they're doing :) (MongoDB went through a similar arc a decade ago with the shift to WiredTiger's storage engine)

The Garnet research paper has received the best paper award at VLDB 2026! The work will be presented in Boston on Thursday, Sep 3. Thanks to all our collaborators, users, partners, & OSS contributors! Links: vldb.org/pvldb/vol19/p224-ch… github.com/microsoft/garnet vldb.org/2026/conference-awa…

Adam Prout avatar
Adam Prout@a_prout·Aug 27

Good paper. transaction log\WAL storage for OLTP databases doesn't belong in S3 (for datalakes... maybe?). The design proposed in the paper is very similar to the HorizonDB WAL server.

There is a lot of hype around putting the WAL on object storage. However, for low-latency systems, e.g. OLTP databases, object storage is a suboptimal primitive for the WAL: (1) PUTs are expensive for small WAL writes [bytes/$]. (2) Object stores are optimized for throughput, not latency, with write latencies often around 10-50 ms. Because every transaction must wait for its WAL record to become durable, every transaction inherits that latency. This points to a missing storage primitive. In our (@maxikuschewski @melhindiCS @viktor_leis) latest accepted research paper at VLDB, we looked at how to close that gap.

Post attachment
Adam Prout avatar
Adam Prout@a_prout·Aug 26

Has anyone tallied up the amount of revenue generated in some way on the Postgres code base (via companies selling forks, services, extensions)? I made an attempt to list some of the services/databases built on its code here: techcommunity.microsoft.com/…

Adam Prout avatar
Adam Prout@a_prout·May 15

Broadly agree that there is some pressure to go faster via agents and hope your tests are good enough to catch problems. That said, I still have hope that the overall quality of software will go up with agents debugging, testing and fixing bugs. They are good at this.

I strongly believe there are entire companies right now under heavy AI psychosis and its impossible to have rational conversations about it with them. I can't name any specific people because they include personal friends I deeply respect, but I worry about how this plays out. I lived through the great MTBF vs MTTR (mean-time-between-failure vs. mean-time-to-recovery) reckoning of infrastructure during the transition to cloud and cloud automation. All those arguments are rearing their ugly heads again but now its... the whole software development industry (maybe the whole world, really). It's frightening, because the psychosis folks operate under an almost absolute "MTTR is all you need" mentality: "its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!" We learned in infrastructure that MTTR is great but you can't yeet resilient systems entirely. The main issue is I don't even know how to bring this up to people I know personally, because bringing this topic up leads to immediately dismissals like "no no, it has full test coverage" or "bug reports are going down" or something, which just don't paint the whole picture. We already learned this lesson once in infrastructure: you can automate yourself into a very resilient catastrophe machine. Systems can appear healthy by local metrics while globally becoming incomprehensible. Bug reports can go down while latent risk explodes. Test coverage can rise while semantic understanding falls. Changes happens so fast that nobody notices the underlying architecture decaying. I worry.

Adam Prout avatar
Adam Prout@a_prout·May 8

Agreed, shared storage (Aurora, AlloyDB, HorizonDB, Neon, etc.) is the dominate design for cloud OTLP. It's not the best in every scenario, but it is for most use cases. You can push a lot of work (replication, full page writes, dirty page writes, etc.) into the storage layer

There is a new era of data tech that is effectively "__ on object storage": Turbopuffer is "vector search on object storage" Warpstream is "kafka on object storage" Neon is "Postgres on Object Storage" or “Postgres on S3”. It doesn’t mean that every read and write goes directly to S3. That would be incredibly slow. I’m saying that a Postgres database in the "Postgres on object storage" category can be faster than one in the "Postgres on a cluster of servers with NVMe disks" category. No one is claiming that S3 is faster than NVMe but Postgres on S3 (with low latency storage in between) can be faster than Postgres running on NVMe with HA on. HA is important here, without HA you don’t do durable writes so it would be an unfair comparison. While neon runs on s3, calls into s3 are almost never on the transaction reads or writes. Writes are sent into a consensus service and streamed into s3 asynchronously. So the claim can be expanded to Postgres running on a disaggregated storage which implements low latency tier on top of s3 is faster then Postgres with HA running on NVMe. We are not the only ones making this claim. For example AWS Aurora says "Aurora has 5x the throughput of MySQL and 3x of PostgreSQL with full PostgreSQL and MySQL compatibility." So why does disaggregating compute allow for higher throughput on Postgres and potentially lower latency as well? The reason is that we can offload a number of CPU and IO operations down to storage. We just published a blog post on how we can turn off full page writes which dramatically reduces WAL volume and saves on CPU cycles on the Postgres node. In many scenarios this may be a wash because for many workloads you might not be write throughout bound and therefore Postgres checkpoints and full page writes don’t impact overall throughput. However this is general purpose enough to impact a large swath of workloads. It’s also important to mention that scaling write throughput is more important since Postgres is a single write system and you can’t scale writes with read replicas. So is Postgres on S3 faster than Postgres on NVMe? We believe it can and will be. Postgres with disaggregated storage and several kernel performance optimizations has higher throughput than stock Postgres on NVMe with HA implemented via sync replication. We'll share more including latency impact as we gather insights after rollout. Lots of things to learn here if saving CPU on full page writes can have a material impact on latency under high throughput. The idea is that if CPU is all used up, freeing up some CPU will impact both latency and throughput - but we'll see! The statement is indeed provocative, but far from “shock value marketing” as some of the responses claim.

Adam Prout avatar
Adam Prout@a_prout·May 8

If I had known database twitter was going to blow up about NVMe vs blob store for OLTP today, we would have hit on that in a little more detail! Still, if you're interested in a career working on databases hopefully this podcast will be of some interest to you.

🎙️ New #TalkingPostgres podcast Ep39 is out! @a_prout, distinguished engineer at Microsoft, talked about his engineering journey from MemSQL to HorizonDB, shared-storage, & why good systems programmers are paranoid 🎧 talkingpostgres.com/episodes… 📺 youtu.be/L_2kyfL9LN0?si=6o5o…

Post attachment