←
Viacheslav Borovitskiy117 posts
Viacheslav Borovitskiy avatar

Viacheslav Borovitskiy

@vabor112

Assistant Professor in Machine Learning @EdinburghUni. PhD in Mathematics. Ex @ETH_en. See vab.im/vacancies/

◷ Joined February 2023▧ 17 Photos and videos
Viacheslav Borovitskiy avatar
Viacheslav Borovitskiy@vabor112·Oct 22

Deadlines: 🔹 ELLIS PhD Program: 31 Oct 🔹 School of Informatics scholarships (fully funded; home & international): 30 Nov 🔹 CDTs: Nov–Jan Home-fee applicants especially encouraged: more funding available. Email me before applying, and please help spread the word! 2/2

Alexander Terenin avatar
⟳ Viacheslav Borovitskiy retweeted
Alexander Terenin@avt_im·Aug 26

Announcing: my final Cornell-era academic project - launching today! It's called "Bayesian Decision-making Algorithms" - a book which tries to write down everything important I've learned in ten years of research. Chapters 1 and 2 are now online! 👇 Link and more below! 👇 URL: bayesianalgorithms.com Decision-making is ubiquitous in life - taking actions, seeing what happens, learning, doing a better job next time. In doing so, we plan using our mind's model of reality. Bayesian decision-making casts this paradigm into math - by using a Bayesian model to quantify uncertainty about reality. The question becomes: how do we use this uncertainty to take better actions? This is not as easy as it looks. Due to the presence of explore-exploit tradeoffs, maximizing expected value is often bad: strategies that do so can get stuck in bad actions because they don't try enough new things to correctly learn about the world. The fix is to consider more sophisticated strategies for exploration. In this book, I'll show you the major known algorithm classes which balance tradeoffs like the above, and will teach you how they work. This book represents the conclusion of an intellectual journey that began at the start of my twenties. Its primary goal is to inspire the next generation to learn about Bayesian thinking, to enable others to do bigger and more creative things with it. Especially in realistic situations where we do not assume the Bayesian model exactly matches reality. In particular, I think there is a chance Bayesian decision-making is one of the "right abstractions" for understanding how large language models actually work, including how they handle exploration. You should consider working on this! I'm writing the book build-in-public style - I'll be posting other chapters when they're done, which is hopefully soon. I know exactly what I want to write, the remaining work is to do so. Stay tuned for that. I don't know how long it will take me to finish. Among my ambitions, the book is number two on the priority list - nonetheless, writing it is possible, because priority one is bottlenecked by the number of days in a year, not the number of hours in a day. More on that another time. In the meantime, I'd love your comments! Please have a look, and tell me what you think!

Post attachment
Viacheslav Borovitskiy avatar
Viacheslav Borovitskiy@vabor112·May 25

🔹 Convexity? Message passing GNNs find different weights but learn remarkably similar functions—even where they fail. We hypothesize the architecture induces function-space convexity, making them behave like linear models despite their depth. Formalizing this is an open problem.

Viacheslav Borovitskiy avatar
Viacheslav Borovitskiy@vabor112·May 25

🔹 Epistemic collapse: Deep ensembles rely on model disagreement to improve uncertainty (no diversity = no improvement). Across 6 highly diverse datasets, a severe lack of diversity is exactly what we observed. This is a stark contrast to foundational computer vision papers.

Post attachment
Viacheslav Borovitskiy avatar
Viacheslav Borovitskiy@vabor112·May 25

Since Bayesian GCNs did much better, standard ensembling seemed to be the culprit. Pedro dug in to see if this was an isolated anomaly or a general phenomenon pertaining to message passing GNNs (like GCNs and GATs). Here is what we found: 👇

Viacheslav Borovitskiy avatar
Viacheslav Borovitskiy@vabor112·May 25

The origin story: This started from a tutorial benchmark (see img). I was surprised to find deep ensembles of GCNs failing miserably there. Point predictions were fine, but they yielded wildly overconfident uncertainty maps and an NLL much worse than a trivial N(µ, σ²) baseline!

Post attachment
Viacheslav Borovitskiy avatar
Viacheslav Borovitskiy@vabor112·May 25

My first paper with my first PhD student @PedroCVieira0 just landed on arXiv! “Do Deep Ensembles Actually Capture Uncertainty in Graph Neural Networks?” Spoiler alert: the answer is largely no—at least, not much more than a single model does. 🧵👇 📄 arxiv.org/abs/2605.22593

Viacheslav Borovitskiy avatar
Viacheslav Borovitskiy@vabor112·Mar 12

Bonus: a concrete success story. In the traffic prediction benchmark below, a Geometric GP (powered by GeometricKernels) significantly outperforms GNN Ensembles and Bayesian GNNs in both prediction (RMSE) and uncertainty (NLL) quality.

Post attachment
Viacheslav Borovitskiy avatar
Viacheslav Borovitskiy@vabor112·Mar 12

✅ Spaces: Graphs, Meshes, Hyperspheres, Tori, Hyperbolic spaces, SPD matrices, & Lie Groups (SO(n), SU(n)). ✅ Infrastructure: Run seamlessly on PyTorch, JAX, TensorFlow, or NumPy. ✅ Integrations: Plug-and-play wrappers for GPyTorch & GPJax.

Viacheslav Borovitskiy avatar
Viacheslav Borovitskiy@vabor112·Mar 12

What's in the library? GeometricKernels gives you the principled Heat (Diffusion) and Matérn kernels needed to build these models out-of-the-box.

Viacheslav Borovitskiy avatar
Viacheslav Borovitskiy@vabor112·Mar 12

Why care about kernels in 2026? To build probabilistic models on geometric domains! Kernels drive Gaussian processes. While they famously struggle in high dim-s (images/text), many geometric domains are intrinsically low-dim, making these models shine (road networks/3D surfaces).