←
Explainable AI1,185 posts
Explainable AI avatar

Explainable AI

@XAI_Research

Moved to 🦋! Explainable/Interpretable AI researchers and enthusiasts - DM to join the XAI Slack! Twitter and Slack maintained by @NickKroeger1

◷ Joined March 2022▧ 14 Photos and videos
Explainable AI avatar
Explainable AI@XAI_Research·28 Mar 2022

There's a new XAI Slack! Connect with XAI/IML researchers and enthusiasts from around the world. Discuss interpretability methods, get help on challenging problems, and meet experts in your field! DM to join 🥳

Suraj Srinivas avatar
⟳ Explainable AI retweeted
Suraj Srinivas@Suuraj·4 Feb 2025

Our Theory of Interpretable AI (tverven.github.io/tiai-semin…) will soon celebrate its one-year anniversary! 🥳 As we step into our second year, we’d love to hear from you! What papers would you like to see discussed in our seminar in the future? 📚🔍 @tverven @ML_Theorist

Archiki Prasad avatar
⟳ Explainable AI retweeted
Archiki Prasad@ArchikiPrasad·4 Feb 2025

🚨 Excited to share: "Learning to Generate Unit Tests for Automated Debugging" 🚨 which introduces ✨UTGen and UTDebug✨ for teaching LLMs to generate unit tests (UTs) and debugging code from generated tests. UTGen+UTDebug improve LLM-based code debugging by addressing 3 key questions: 1⃣ What are desirable properties of unit test generators? (A: high output acc and rate of uncovering errors) 2⃣ How good are models at 0-shot unit test generation (A: they are not great) ... so how do we improve LLMs' UT generation abilities? (A: bootstrapping from code-generation data via UTGen) 3⃣ How can we use potentially noisy feedback from generated tests for debugging? (A: via test-time scaling and validation + backtracking in UTDebug) 🧵👇

Post attachment
Gianluigi Lopardo avatar
⟳ Explainable AI retweeted
Gianluigi Lopardo@gigilopardo·29 Jan 2025

Hot off the press: my PhD thesis "Foundations of machine learning interpretability" is officially published! Enjoy it at theses.hal.science/tel-04917… @XAI_Research @trustworthy_ml

Chirag Agarwal avatar
⟳ Explainable AI retweeted
Chirag Agarwal@_cagarwal·22 Jan 2025

Exciting opportunity at the intersection of climate science and XAI to work on groundbreaking research in attributing extreme precipitation events with multimodal models. Check out the details and help spread the word! #ClimateAI #Postdoc #UVA #Hiring Job description: shorturl.at/uP5fq Tagging @XAI_Research @trustworthy_ml @uvadatascience @ClimateChangeAI to spread the word!

Dear Climate and AI community! We are hiring 😀 a postdoc to join @UVAEnvironment at @UVA and work with @_cagarwal and myself, on using multimodal AI models and explainable AI to attribute extreme precipitation events! Fascinating stuff! Link below. Please RT! jobs.virginia.edu/us/en/job/…

Rudy Gilman avatar
⟳ Explainable AI retweeted
Rudy Gilman@rgilman33·17 Jan 2025

The later features in DINO-v2 are more abstract and semantically meaningful than I'd expected from the training objectives. This neuron responds only to hugs. Nothing else, just hugs.

0:19
Giang Nguyen avatar
⟳ Explainable AI retweeted
Giang Nguyen@giangnguyen2412·16 Jan 2025

Hi Dylan, it reminds me of our paper where we also train a model (model 2) on the output of another black-box model (model 1). ultimately we find that combining the outputs of model 2 and model 1 helps improve the perf significantly. openreview.net/forum?id=OcFj…

Rohan Paul avatar
⟳ Explainable AI retweeted
Rohan Paul@rohanpaul_ai·15 Jan 2025

LLMs are all circuits and patterns Nice Paper for a long weekend read - "A Primer on the Inner Workings of Transformer-based Language Models" 📌 Provides a concise intro focusing on the generative decoder-only architecture. 📌 Introduces the Transformer layer components, including the attention block (QK and OV circuits) and feedforward network block, and explains the residual stream perspective. It then categorizes LM interpretability approaches into two dimensions: localizing inputs or model components responsible for a prediction (behavior localization) and decoding information stored in learned representations to understand its usage across network components (information decoding). 📌 For behavior localization, the paper covers input attribution methods (gradient-based, perturbation-based, context mixing) and model component importance techniques (logit attribution, causal interventions, circuits analysis). Causal interventions involve patching activations during the forward pass to estimate component influence, while circuits analysis aims to reverse-engineer neural networks into human-understandable algorithms by uncovering subsets of model components interacting together to solve a task. 📌 Information decoding methods aim to understand what features are represented in the network. Probing trains supervised models to predict input properties from representations, while the linear representation hypothesis states that features are encoded as linear subspaces. Sparse autoencoders (SAEs) can disentangle superimposed features by learning overcomplete feature bases. Decoding in vocabulary space involves projecting intermediate representations and model weights using the unembedding matrix. 📌 Then summarizes discovered inner behaviors in Transformers, including interpretable attention patterns (positional, subword joiner, syntactic heads) and circuits (copying, induction, copy suppression, successor heads), neuron input/output behaviors (concept-specific, language-specific neurons), and the high-level structure mirroring sensory/motor neurons. Emergent multi-component behaviors are exemplified by the IOI task circuit in GPT2-Small. Insights on factuality and hallucinations highlight the competition between grounded and memorized recall mechanisms.

Post attachment
Michal Moshkovitz avatar
⟳ Explainable AI retweeted
Michal Moshkovitz@ML_Theorist·13 Jan 2025

This Thursday (in 3 days), @YishayMansour will discuss interpretable approximations — learning with interpretable models. Is it the same as regular learning? Attend the lecture to find out! 💻 Website: tverven.github.io/tiai-semin… @Suuraj @tverven

The theory of interpretable AI seminar is back after the holiday season! 🎅🤶 Our next talk is next Thursday by Yishay Mansour who will talk about interpretable approximations 💻 Website: tverven.github.io/tiai-semin… ⏰Date: 16 Jan @Suuraj @tverven @YishayMansour

Post attachment