
Check out the paper here: rm.cab/gremlin Generated queries for PG and DuckDB: github.com/Speculative/adver… Jeff Tao will present this work at the AIDB workshop today at VLDB! Check out Jeff's other work: speculative.tech/

Assistant prof @CIS_Penn. Machine learning for systems, databases. Also discuss.systems/@ryanmarcus

Check out the paper here: rm.cab/gremlin Generated queries for PG and DuckDB: github.com/Speculative/adver… Jeff Tao will present this work at the AIDB workshop today at VLDB! Check out Jeff's other work: speculative.tech/

Query optimization benchmarks too easy? Gremlin generates simple select-project-join queries that are up to 800x slower on DuckDB and PostgreSQL than a known plan. Gremlin simultaneously searches for both an adversarial input and matching fast plan to reveal perf bugs in QOs.

"Global Hash Tables Strike Back!" analyzes partitioned vs. fully concurrent GROUP BY implementation strategies, finding that a simple, custom-tailored hash table may be the missing ingredient to effective fully concurrent GROUP BY implementations. vldb.org/pvldb/vol19/p523-ma…

Ultron, in collaboration with @databricks , is an async push-style history engine used by the Databricks fleet. An example Ultron application -- selecting shuffle vs. braodcast hash joins -- reduced the median latency of applicable queries by 25%. rm.cab/ultron

Our group is part of two @VLDBconf papers on query opt. and execution: 1) Ultron, a system for building history-based optimization rules, deployed at @databricks 2) Global hash tables strike back!(?), an experimental analysis of fully concurrent group by aggregation. ⬇️

We conclude with a discussion about how database researchers should use industrial traces, and how we might begin to build systems that optimize for "the query the user never sends." 📄Paper: rm.cab/survivorshipbias

For researchers, databases traces are a MAJOR upgrade compared to synthetic benchmarks (or simply making something up, which is shockingly common). We argue we need more of these workload traces to build a complete picture, and, perhaps more importantly, see what is missing.

Data is limited, so we don't claim a strong and statistically robust result, but we think these case studies are enough to seriously question the prevailing model of "a user submits the queries they want, and it's the vendor's job to run them as fast as possible."

We test this behavior in two observational case studies. #1: database vendors with higher startup costs for ad-hoc queries see fewer ad-hoc queries on average #2: when database vendors optimize a particular code path, usage jumped from 2% to 6% of customers.

Most database teams optimize what they see in workload logs. But those very optimizations change what users choose to run! In our CIDR paper, we argue that industrial workloads exhibit 𝐬𝐮𝐫𝐯𝐢𝐯𝐨𝐫𝐬𝐡𝐢𝐩 𝐛𝐢𝐚𝐬: logs reflect a negotiation between users and the platform.

For that one query that must go 𝑟𝑒𝑎𝑙𝑙𝑦 𝑓𝑎𝑠𝑡, BayesQO (by Jeff Tao) finds superoptimized plans using Bayesian optimization in a learned plan space. It’s costly, but the results can train an LLM to speed things up next time. 📄rm.cab/bayesqo

LimeQO (by @yi_zixuan), a 𝑤𝑜𝑟𝑘𝑙𝑜𝑎𝑑-𝑙𝑒𝑣𝑒𝑙 approach to query optimization, can use neural networks or simple linear methods to find good query hints significantly faster than a random or brute force search. 📄rm.cab/limeqo

OLAP workloads are dominated by repetitive queries -- how can we optimize them? A promising direction is to do 𝗼𝗳𝗳𝗹𝗶𝗻𝗲 query optimization, allowing for a much more thorough plan search. Two new SIGMOD papers! 🧵


📝 Check out the paper: rm.cab/limeqo 📄 Zixuan's NEDB poster: bu-disc.github.io/nedbday/20… 🌐 Zixuan's website: zixy17.github.io/

At aiDM@SIGMOD, PhD student Zixuan Yi will present LimeQO, the first *workload-level* learned query optimizer: simultaneously learning to optimize an entire query workload at once! By casting the problem as low rank matrix completion, we show that linear methods are all you need.