←
F.Mackenzie 约克.小汽车. 嘟嘟39,017 posts
F.Mackenzie 约克.小汽车. 嘟嘟 avatar

F.Mackenzie 约克.小汽车. 嘟嘟

@FMackenzie7

🇬🇧 Focused on: AI | Mech Interp | Maths | Info Geometry | Manifold Hyp | EV Battery Safety

◷ Joined November 2019▧ 2,851 Photos and videos
F.Mackenzie 约克.小汽车. 嘟嘟 avatar
F.Mackenzie 约克.小汽车. 嘟嘟@FMackenzie7·Aug 6

Maths list: Algebra (Linear Algebra and Operator Theory) Real Analysis and Multivariable Calculus Harmonic Analysis High-Dimensional Geometry and its Probabilistic Analogues Combinatorics (Graph Theory) Theoretical Computer Science Probability and Mathematical Statistics

F.Mackenzie 约克.小汽车. 嘟嘟 avatar
F.Mackenzie 约克.小汽车. 嘟嘟@FMackenzie7·2h

What drives the emergence of symbolic mechanisms in large language models? 🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️

🧵 How do LLMs develop symbolic reasoning mechanisms? Is it the architecture, or the training data? Excited to share our #NeurIPS2026 paper! 📄 🔑 Key takeaway: Data diversity is a primary driver of symbolic mechanisms. Surprisingly, even basic MLPs can develop the same signatures! 🔗 arxiv.org/abs/2611.13270👇

Post attachment
F.Mackenzie 约克.小汽车. 嘟嘟 avatar
F.Mackenzie 约克.小汽车. 嘟嘟@FMackenzie7·Nov 18

Measuring Combinatorial Creativity in Large Language Models and Humans 🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️

One of the reasons analogy is so pervasive as a creativity mechanism is that it sidesteps this "creative paradox." Though analogy, one can use knowledge from a familiar domain to infer or generate knowledge about a different, but somehow connected domain. As we found in our recent paper (arxiv.org/abs/2509.21043), LLMs are quite good at making analogies, outperforming the average human. Slide from a forthcoming talk.

Post attachment
F.Mackenzie 约克.小汽车. 嘟嘟 avatar
F.Mackenzie 约克.小汽车. 嘟嘟@FMackenzie7·Nov 17

Energy-Based Reasoners: Your Recurrent Reasoner Should Be an Energy-Based Model 🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️

Bye bye looped reasoning… and hello Energy-Based Reasoners (EBRs)⚡! Your looped reasoning model would be better off as an energy-based model (EBM)... TLDR: EBRs uses 345x less inference compute and 2.6x fewer training FLOPs than the best baselines EBRs are 7x-43x more sample efficient than the best looped baselines at inference. EBRs achieve 99.9% on Sudoku-Extreme EBRs achieve near 100% accuracy on multi-solution benchmarks 🧵Thread:

F.Mackenzie 约克.小汽车. 嘟嘟 avatar
F.Mackenzie 约克.小汽车. 嘟嘟@FMackenzie7·Nov 17

Discovering Neuron Communities in Language Models 🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️

New neurons research: introducing Neuron Communities. We discovered communities of neurons that work together inside LLMs. They’re surprisingly small, but powerful – just a few neurons work together to perform common behaviors, like arithmetic, ALL CAPS, or recalling a date. Work led by research fellow @_igorshilov:

0:08
F.Mackenzie 约克.小汽车. 嘟嘟 avatar
F.Mackenzie 约克.小汽车. 嘟嘟@FMackenzie7·Nov 17

V-JEPA-LM: Video-Language Models That Read Dynamics Beyond Static, Text-Aligned Representations 🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️ 👾👾👾👾👾 🥰🥰🥰🥰🥰

V-JEPA-LM: Video-Language Models That Read Dynamics Beyond Static, Text-Aligned Representations Counterfactual supervision helps video-language models read scene dynamics from frozen encoders, topping eleven benchmarks while using far fewer samples. arxivb.org/abs/2611.17831

0:09
F.Mackenzie 约克.小汽车. 嘟嘟 avatar
F.Mackenzie 约克.小汽车. 嘟嘟@FMackenzie7·Nov 17

The multi-scale geometry of neural encoding 🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️

Representational geometry describes what a neural population encodes. But it can be built in many ways, each with its own assumptions. In our new preprint, we derive one from a few simple requirements. Its distances turn out to be exactly tied to mutual information. 🧵 1/13

Post attachment
F.Mackenzie 约克.小汽车. 嘟嘟 avatar
F.Mackenzie 约克.小汽车. 嘟嘟@FMackenzie7·Nov 17

Loopback Transformer: Towards a Unified View of Residual Looped Transformers 🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️

Do loop models need a new residual design? Sharing our new work: Loopback Transformer 🔁 With nearly the same training FLOPs, can a model use fewer Transformer-layer parameters and still outperform a Standard Transformer? Sharing layers saves parameters. But running the same layers again raises another question: which information should survive across passes, and which updates should fade? Inspired by recurrent models, we introduce a unified view of residual connections as memory across depth: write, transport, and read. It brings ordinary residuals, Attention Residuals, Hyper-Connections, and linear attention readers into one framework. Loopback puts this view into practice: store each block's update, geometrically discount older passes, and use attention to retrieve useful information—even from earlier loops. The original input stays on its own path. ⚡ Matched training tokens and executed depth: • 1.4B: 68% of Standard's Transformer-layer parameters, lower validation loss and higher 13-task mean. • 3B: 75% of the layer parameters, 11/13 task wins, mean accuracy 48.56% → 49.45%. • At 191M, 374M, and 778M, the main comparisons give Loopback the lowest validation loss and highest 13-task mean among Standard and seven looped residual baselines. If you find the work useful, an upvote on Daily Papers and a GitHub star would help others discover it! 🤗 Paper on Daily Papers: huggingface.co/papers/2611.0… ⭐ Code: github.com/yancyou/Loopback The idea, the video, and the results below 🧵 1/11

Post attachment
F.Mackenzie 约克.小汽车. 嘟嘟 avatar
F.Mackenzie 约克.小汽车. 嘟嘟@FMackenzie7·Nov 17

Self-Improving Memory Meta-Optimization and Harness Distillation for Continual Learning Agents 🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️

Introducing: Self-Improving Memory 🧠✨📈 We let the model discover its own memory skills, then train them into its weights: - Meta-optimizes memory from the model's own runs with large gains. - Distilling these gains works too. Blog post: 📝 facebookresearch.github.io/R… 🧵1/6

Post attachment
F.Mackenzie 约克.小汽车. 嘟嘟 avatar
F.Mackenzie 约克.小汽车. 嘟嘟@FMackenzie7·Nov 17

∇ Guide: Looped Transformers: when is another pass worth the cost? 🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️

Looped Transformers are low-key fascinating. First of all, you need to understand what travels around the loop and the 3 core things you set for the architecture. A Looped Transformer runs the same Transformer block (or stack) for multiple passes, feeding each output back in as the next input. ▪️ What travels around this loop is the hidden state: - Token embeddings form the initial hidden representation. - The shared stack updates that representation via attention and feed-forward operations. - Its output goes back to the stack for another pass, using an updated hidden state. Some designs also reuse the input, add time-step embeddings, or run multiple residual streams. - After the chosen number of passes, the output head converts the selected representation into token scores. That's all to win additional computation without an additional set of block weights for every pass. Although the weights stay the same during inference, changing inputs let the same block produce different updates. During decoding it attends via the KV cache, during prefill it updates masked tokens in parallel. A model can loop its entire stack or repeat only a middle stack between ordinary input and output blocks. Note that repeating the stack doesn't require generating a new text token between passes. ▪️ So there are 3 basic things you need to identify for the specific Looped Transformer architecture: 1. The repeated blocks 2. The state they receive 3. The rule that controls their execution - a fixed number of passes, a user-selected count, or an adaptive rule If you want to nerd out on the rest, like adaptive depth, what happens to the KV-cache and hardware, 2026 breakthroughs, and when another pass is worth the cost - we put together a full guide over here: turingpost.com/p/looped-tran…

Post attachment
F.Mackenzie 约克.小汽车. 嘟嘟 avatar
F.Mackenzie 约克.小汽车. 嘟嘟@FMackenzie7·Nov 17

▪️ V-JEPA-LM ▪️ Learning Heterogeneous Tactile Representation for General Dexterous Manipulation 🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️

Must-read papers of the week ▪️ World In-Context Learning for Agentic Robot Manipulation ▪️ Don’t Just Trust Your Memory ▪️ Is Your Robot’s Memory Just a Clock? Temporal Shortcuts in Memory-Augmented VLA Policies ▪️ Unbundling Multimodal Vision Encoding ▪️ V-JEPA-LM ▪️ Learning Heterogeneous Tactile Representation for General Dexterous Manipulation ▪️ Training Reasoning Models in Simulation to Natively Use Real Robots ▪️ Embodied First Exam ▪️ Continual-ARC: Measuring Skill Acquisition over a Working Lifetime ▪️ Reframe: Observation-Aligned Multi-Teacher On-Policy Distillation for Agentic Visual Reasoning ▪️ MemInStream: Memorizing or Merely Recognizing? Rethinking Streaming Video Memory Evaluation ▪️ Infrathon: Benchmarking Language Agents for Real-World, Long-Horizon AI Infra Optimization ▪️ Actor, Judge, or Teacher? Understanding Decision Models in Long-Horizon Agents ▪️ GTAGym: AAA Game Tasks Environment for Evaluation and Reinforcement Learning Explore these to keep up with main AI trends. Here’s also the full list of outstanding papers + links, and our weekly AI news digest: turingpost.com/p/who-are-we-…

Post attachment
F.Mackenzie 约克.小汽车. 嘟嘟 avatar
F.Mackenzie 约克.小汽车. 嘟嘟@FMackenzie7·Nov 17

Multivariate Statistical Machine Learning Methods for Genomic Prediction 🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️

BURKOV@burkovNov 16

A free book Multivariate Statistical Machine Learning Methods for Genomic Prediction by Osval Antonio Montesinos López, Abelardo Montesinos López, José Crossa. The book’s focus is how to build, prepare, compare, and use prediction models for genomic selection. It begins with the training-population and testing-population framework, distinguishes prediction from inference, and introduces data types, learning paradigms, and fixed, random, and mixed effects. It then connects those foundations to practical preparation of genomic data: marker cleaning, genomic relationship matrices, breeding-value estimation, normalization, predictor choices, and dimension reduction. he progression from multiple regression and penalized regression to mixed models, Bayesian methods, kernels, support vector machines, neural networks, functional regression, and random forests gives readers a broad view of model families and the assumptions and tasks they address. Read the book online on @ChapterPal: chapterpal.com/book/745baf14…

Post attachment
F.Mackenzie 约克.小汽车. 嘟嘟 avatar
F.Mackenzie 约克.小汽车. 嘟嘟@FMackenzie7·Nov 16

Self-improving agent papers 🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️

Lots of self-improving agent papers this week. A good chunk of the gains turn out to be agents getting better at the benchmark, not the task. So we put the papers that actually check first. 1. Beneath the Surface: What We Overlook When We Talk About Agent Evolution Evolving the harness can lift benchmark scores with no weight updates, but the gains are unreliable, expensive and transfer poorly. Read this one first. arxiv.org/abs/2611.07624 2. Learning the Task or Learning the Verifier? Show a self-improving embodied coding agent the benchmark's success criteria and its score goes up. Its actual task completion doesn't. arxiv.org/abs/2611.08318 3. Evidence-Governed Adoption of Self-Generated Updates in Autonomous AI Agents Separates "the agent wrote a change" from "the agent ships it". Every change has to pass fresh paired audits and a non-inferiority check first. arxiv.org/abs/2611.16378 4. Harness^2: Recursive Agent Harnessing for an Open World A harness that improves itself on new tasks without an answer key, by test-running edits and keeping what helps. arxiv.org/abs/2610.14224 5. Experience-Handoff Reinforcement Learning for Recursive Self-Improvement Agents trained with RL learn to write and curate notes for their future runs, not only to pick better actions. arxiv.org/abs/2611.16409 6. InfraRSI: Harnessing Recursive Self-Improvement for Training Infrastructure Self-improvement where it's hardest: distributed training, where one bad edit means an OOM or silent numerical drift. Every optimization gets verified. arxiv.org/abs/2611.11558 7. Incentivizing Self-Learning in Large Language Models via Reinforcement Learning Give a model long reference material and train it to look things up and study while it solves the problem, instead of relying on what it already knows. arxiv.org/abs/2610.39671 8. PhysLoop: Harnessing LLMs' Physics Awareness for Self-Improving Video Generation Models An LLM spots physics mistakes in generated videos and guides the fixes, so the video model improves on cleaner data. arxiv.org/abs/2611.16748 9. A Cross-Scenario Survey and Analysis of Self-Evolving Multimodal Agents Why a self-improvement trick that works for reasoning often breaks for navigation or manipulation. A good map of the field. arxiv.org/abs/2611.14256 If you're letting an agent edit its own setup: who decides which edits stay? Tests, a human, another model?

F.Mackenzie 约克.小汽车. 嘟嘟 avatar
F.Mackenzie 约克.小汽车. 嘟嘟@FMackenzie7·Nov 16

Weight-Space Reading 🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️

"Weight-Space Reading" Can an LLM predict how changing another model's weights will affect its behavior, without ever running that model? This paper investigates Weight-Space Reading, where LLMs inspect the raw weights of 9B-30B parameter models using Python tools, then predict how small weight modifications will affect capabilities like math accuracy, reasoning length, and refusal behavior, without performing a single forward pass. On subtle weight edits, Qwen3.8-27B achieves 73% balanced accuracy, compared to 57.8% when given only text descriptions of the edits. However, the models also frequently predict changes that never happen. When 40 edits had no measurable effect on Muse-30B, the model incorrectly predicted changes for 85% of them. The researchers also found no conclusive evidence that LLMs are better at reading their own weights than those of other models. alphaxiv.org/abs/2611.weight…

Post attachment
F.Mackenzie 约克.小汽车. 嘟嘟 avatar
F.Mackenzie 约克.小汽车. 嘟嘟@FMackenzie7·Nov 16

Algebraic Topology for Data Scientists by Michael S. Postol (2023). 🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️🙇‍♂️

BURKOV@burkovNov 15

A free online book: Algebraic Topology for Data Scientists by Michael S. Postol (2023). Algebraic Topology for Data Scientists is a textbook for readers in data science, computer science, analytics, and related fields who want to understand how topology can describe data, including readers without prior training in algebraic topology. The central subject is topological data analysis (TDA): using algebraic descriptions of shape and connectivity to complement conventional data and machine-learning methods. The book first develops the language of spaces, continuous maps, algebraic structures, simplicial complexes, and homology, then uses that groundwork to explain persistent homology. Readers encounter ways to represent persistence, assess stability, and turn topological summaries into numerical features for tasks such as clustering and classification. The treatment connects these ideas to point clouds, sublevel sets, graphs, and time series, with examples that include multivariate time-series analysis and classification of internet-of-things data. It also surveys related methods and applications, including Q-analysis, sensor coverage, Mapper, simplicial sets, and UMAP. Read the book online on @ChapterPal: chapterpal.com/ebook/6cb1774…

Post attachment