←
Matthias Fey95 posts
Matthias Fey avatar

Matthias Fey

@rusty1s

GPU Software @ NVIDIA - Creator of PyG (PyTorch Geometric) - Founding Engineer @ Kumo - PhD @ TU Dortmund University - Graph Representation Learning

◷ Joined November 2017▧ 20 Photos and videos
Matthias Fey avatar
Matthias Fey@rusty1s·Sep 29

It's good to be back in open-source! 🚀 Alongside Kumo Tabular, we are releasing our NVIDIA Kumo engine, a GPU-accelerated library of foundation models for structured data (TabICL, TabFM, etc). Code: github.com/NVIDIA/structured… Blog: huggingface.co/blog/nvidia/k… A thread 🧵

I’m very excited to share @nvidia Kumo Tabular, a new family of foundation models for tabular data. Kumo Tabular establishes the new Pareto frontier across the entire accuracy–inference-time tradeoff. Just as importantly, we are releasing it openly: open weights, open-source software, and a permissive license for commercial use. HuggingFace: huggingface.co/nvidia/Kumo-T… GitHub: github.com/NVIDIA/structured…

2:14
Matthias Fey avatar
Matthias Fey@rusty1s·Sep 29

We have included reusable neural network building blocks, shared across model implementations whenever possible. This helps us to not only improve efficiency and peak memory of Kumo Tabular but all tabular foundation models (e.g., via Transformer chunking or output buffer reuse).

Matthias Fey avatar
Matthias Fey@rusty1s·Sep 29

For forecasting, we can attach past and future covariates, or multivariate targets. Unlike common scikit-learn interfaces, we expose both an in-context learning forward pass, and a fit/predict interface that caches processor state+key/values for repeated inference.

Post attachment
Matthias Fey avatar
Matthias Fey@rusty1s·Sep 29

We wanted an in-context learning interface that can go beyond single tabular data and support diverse model families (relational, forecasting, natural language). For example, for multi-table workloads, we can attach additional related tables to context and prediction examples.

Matthias Fey avatar
Matthias Fey@rusty1s·Sep 29

The engine allows us to demystify preprocessing of structured data models via generic recipes. TabICL or TabFM may not support datetime or text features. Here, they are just a line of code away. In ensembling, recipes get executed in parallel for maximum efficiency.

Post attachment
Matthias Fey avatar
Matthias Fey@rusty1s·Sep 29

We put a lot of thought into its design: from ingestion and preprocessing to model interfaces and execution. Every data modality (numerical, categorical, text, datetime, ID, etc) is supported via its own tensor subclass, allowing for end-to-end GPU processing.

Post attachment
Jure Leskovec avatar
⟳ Matthias Fey retweeted
Jure Leskovec@jure·Jan 12

🚀 Announcing RelBench V2, a major update to our benchmark for foundation models on relational data! With V2, we are significantly expanding the benchmark’s scope to catalyze further research in Relational Deep Learning (RDL) and Relational Foundation Models (RFMs). Key features: 🍺 4 new databases, spanning domains like e-commerce and beer reviews to scientific research and clinical healthcare. 🧩 40 new predictive tasks, including 28 autocomplete tasks, across new and existing databases. 🔌 External data integrations: 70+ datasets from CTU, 7 datasets from 4DBInfer, and your own data via SQL connector, all in RelBench format. 🛠️ Bug fixes and performance improvements. 🔥 Introducing autocomplete tasks: As opposed to forecasting tasks, autocomplete tasks predict existing columns in the database. We found that models need to deeply understand the relational context to autocomplete database fields, a critical capability that expands the scope of real-world RDL applications. Learn more: 🌐 Website: relbench.stanford.edu 💻 GitHub: github.com/snap-stanford/rel… Huge thanks to @justingu32 @_rishabhranjan_ @jakub_peleska @VHudovernik @CKanatsoulis @fengyuli607, Tang Haiming, Alistiq and everyone else who contributed to our GitHub for making this possible!

Post attachment
rishabh ranjan avatar
⟳ Matthias Fey retweeted
rishabh ranjan@_rishabhranjan_·Jan 12

Although relational databases are everywhere, there is no equivalent of the public internet for pretraining Relational Foundation Models (RFMs). Excited to see RelBench bridging that gap, growing from 7 datasets in v1 to 88+ datasets in v2. Deeply grateful to the numerous community contributions for helping RelBench serve as the central data repository for RFM research. ❤️

🚀 Announcing RelBench V2, a major update to our benchmark for foundation models on relational data! With V2, we are significantly expanding the benchmark’s scope to catalyze further research in Relational Deep Learning (RDL) and Relational Foundation Models (RFMs). Key features: 🍺 4 new databases, spanning domains like e-commerce and beer reviews to scientific research and clinical healthcare. 🧩 40 new predictive tasks, including 28 autocomplete tasks, across new and existing databases. 🔌 External data integrations: 70+ datasets from CTU, 7 datasets from 4DBInfer, and your own data via SQL connector, all in RelBench format. 🛠️ Bug fixes and performance improvements. 🔥 Introducing autocomplete tasks: As opposed to forecasting tasks, autocomplete tasks predict existing columns in the database. We found that models need to deeply understand the relational context to autocomplete database fields, a critical capability that expands the scope of real-world RDL applications. Learn more: 🌐 Website: relbench.stanford.edu 💻 GitHub: github.com/snap-stanford/rel… Huge thanks to @justingu32 @_rishabhranjan_ @jakub_peleska @VHudovernik @CKanatsoulis @fengyuli607, Tang Haiming, Alistiq and everyone else who contributed to our GitHub for making this possible!

Post attachment
rishabh ranjan avatar
⟳ Matthias Fey retweeted
rishabh ranjan@_rishabhranjan_·30 Oct 2025

Transformers are great for sequences, but most business-critical predictions (e.g. product sales, customer churn, ad CTR, in-hospital mortality) rely on highly-structured relational data where signal is scattered across rows, columns, linked tables and time. Excited to finally share what I have been working on over the last year: a Foundation Model architecture which brings the power of Transformers to relational domains, enabling large-scale pretraining and zero-shot generalization in enterprise settings. 🧵1/n

Post attachment
Matthias Fey avatar
Matthias Fey@rusty1s·20 May 2025

Excited to announce Relational Foundation Models, moving supervised Relational Deep Learning into the in-context learning setting: ✅On-the-fly training-free predictions ✅Impressive out-of-the-box performance Paper: shorturl.at/M6ciN Try For Free: kumorfm.ai

Post attachment
Jure Leskovec avatar
⟳ Matthias Fey retweeted
Jure Leskovec@jure·20 May 2025

🚀 Introducing KumoRFM — the world’s first Relational Foundation Model purpose-built for enterprise prediction tasks! KumoRFM reasons over complex relational data to deliver instant, accurate, in-context predictions — no task-specific model training required. A true game-changer for solving key business problems like: ✅ Product recommendations ✅ Fraud detection ✅ Customer retention 🔗 Explore KumoRFM: kumorfm.ai 📄 Read the paper: kumo.ai/research/kumo_relati… 💡 Learn more: kumo.ai/company/news/kumo-re… #AI #EnterpriseAI #RelationalAI #FoundationModels #MachineLearning #KumoRFM #PredictiveAI4o

0:59
Kumo avatar
⟳ Matthias Fey retweeted
Kumo@Kumo_ai_team·6 Feb 2025

Introducing the new Kumo platform: Build predictive and embedding models directly on your relational data — without feature engineering. Go from predictive modeling to predictive AI. hubs.ly/Q035xfgP0

Yiwen Yuan avatar
⟳ Matthias Fey retweeted
Yiwen Yuan@yiwenyuan98·5 Dec 2024

🎉 Excited to announce the release of ContextGNN, our state-of-art and scalable recommendation model from Kumo! 🔗 ContextGNN paper: arxiv.org/abs/2411.19513 What makes ContextGNN unique? It combines pair-wise and two-tower representations into a single architecture, enabling GNN-based recommendation systems to capture both repeated patterns and exploratory user preferences. 💻 Source Code on GitHub: github.com/kumo-ai/ContextGN… This milestone wouldn’t have been possible without the incredible collaboration of my amazing colleagues. Thank you all for being part of this journey! 🙌 (Shoutout to: @zechengzh Xinwei He @aki_bayes @weihua916 @yaoyaowd @manans99 @shenyangHuang @BlazStojanovic Alan Krumholz @janericlenssen @jure @rusty1s @Kumo_ai_team)

Post attachment
Jure Leskovec avatar
⟳ Matthias Fey retweeted
Jure Leskovec@jure·7 Oct 2024

💠 Stanford Graph Learning Workshop 2024! Join leaders from academia and industry to explore the latest in Machine Learning and AI. Topics include Relational domains, Foundation Models, Agents and more. Save the date: Tuesday, Nov 5, 2024, 09:00 - 18:00 PT. The event will be held at Stanford University and live-streamed online. Register and/or submit a talk/poster: snap.stanford.edu/graphlearn…

Post attachment