←
ayan sengupta37 posts
ayan sengupta avatar

ayan sengupta

@ayans007

PhD student at IIT Delhi

◷ Joined July 2010
ayan sengupta avatar
ayan sengupta@ayans007·19 Dec 2025

🔬 Parmanu (Hindi for Atom) is live. Parmanu is part of the Computational Social Systems (LCS2) @lcs2lab at IIT Delhi, led by Prof. Tanmoy Chakraborty @Tanmoy_Chak , and is our dedicated home for Efficient Large Language Models (LLMs) and Small Language Models (SLMs). We’re at a turning point in AI. The future won’t be defined by scaling alone - it will be shaped by efficiency, accessibility, and real-world deployability. Parmanu is our effort to push this efficiency-first vision forward. ✨🤖 🔗 Explore the project page: parmanu.lcs2.in/ Why Parmanu matters 🔥 • 📚 A centralized hub for our research, with papers accepted at ICLR, ICML, NeurIPS, TACL, ACL, and TMLR • 🛠️ Open access to tools, code, and artifacts spanning model compression, KV efficiency, PEFT, inference optimization, knowledge distillation, and model coordination • 🧠 A growing ecosystem focused on making strong language models smarter per parameter, not just larger What this means for the community For researchers 👩‍🔬👨‍🔬 A curated, evolving resource tied to top-tier venues Reproducible artifacts and principled problem formulations A shared space to advance efficiency-centric LLM research For practitioners 👩‍💻👨‍💻 Practical techniques to deploy LLMs under tight latency and memory budgets Faster paths from paper → production Tools that actually work under real deployment constraints What’s coming in 2026 🚀🔮 • 📊 Efficient LLM/SLM leaderboards • 🧪 Open-sourced efficient LLM artifacts • ⚙️ More tools for compression, distillation, and inference • 🤝 Deep integration with Hugging Face and other popular libraries If you’re excited about efficient, sustainable, and scalable AI, check out Parmanu, share feedback, and collaborate with us. The next wave of LLMs won’t just be bigger - they’ll be leaner, faster, and more impactful. 🌟 #EfficientLLMs #SLMs #ModelCompression #InferenceOptimization #KnowledgeDistillation #AIResearch #NLP #ICLR #ICML #NeurIPS #ACL #TACL #TMLR #IITDelhi #LCS2 #Parmanu

ayan sengupta avatar
ayan sengupta@ayans007·Oct 11

🚨 New article: From Scaling Laws to Downscaling Laws x.com/ayans007/status/210927… In 2025 we argued at ICML that the field needs downscaling laws. In 2026 we deliver one at EMNLP. A 10% lower loss costs 4–8× the carbon emission. Pruning 43% of a model keeps 70% of its performance, with zero retraining. Scaling up is expensive. Scaling down is predictable. @Tanmoy_Chak @lcs2lab #downscaling #efficientllms #modelcompression

ayan sengupta @ayans007 Oct 11 Article From Scaling Laws to Downscaling Laws: What Pruning Ten LLMs Taught Us Scaling laws tell you what a bigger model buys. Nobody had a law for what a smaller one costs. In our ICML 2025 position paper we argued the field needs downscaling laws. Our EMNLP 2026 paper delivers

ayan sengupta avatar
ayan sengupta@ayans007·Aug 30

The fix is to remove the knob. PruneNet (@ICLR 2025) learns what to prune from the model's own weights -- no calibration data, nothing for an attacker to poison. Calibration-free by design.

ayan sengupta avatar
ayan sengupta@ayans007·Aug 30

We changed only the calibration set of a structured pruner and a pruned Llama-3.1-8B went from refusing 96% of harmful prompts to 13%. Unstructured pruning? Unaffected. Calibration-free pruning? Nothing to attack. Read the full article at - x.com/ayans007/status/209402… Work done at LCS2, IIT Delhi (Parmanu project) @Tanmoy_Chak @lcs2lab

ayan sengupta @ayans007 Aug 30 Article The Calibration Trap: How Pruning Data Could Silently Break an LLM's Safety TL;DR. Almost every modern structured-pruning method decides which weights to delete using a small calibration set. We show that this innocuous-looking knob is a control surface over the pruned

Tanmoy Chakraborty avatar
⟳ ayan sengupta retweeted
Tanmoy Chakraborty@Tanmoy_Chak·Aug 13

** 𝐀𝐝𝐚𝐩𝐭𝐢𝐯𝐞 𝐀𝐠𝐞𝐧𝐭 𝐂𝐨𝐨𝐫𝐝𝐢𝐧𝐚𝐭𝐢𝐨𝐧 ** Multi-agent systems are powerful, but they can drastically multiply inference costs. Many existing systems rely on fixed or densely activated agent pipelines without adapting computation to each query: Which agents actually need to be consulted? How deep should the reasoning go? And when is communication worth its compute cost? Presenting 𝐆𝐑𝐀𝐃𝐄 -- Gated Routing and Adaptive Depth for Efficient Reasoning 🔗 Preprint: arxiv.org/abs/2607.10836 GRADE optimises multi-agent reasoning by: 🧠 𝐋𝐞𝐚𝐫𝐧𝐞𝐝 𝐂𝐨𝐨𝐫𝐝𝐢𝐧𝐚𝐭𝐢𝐨𝐧: We built a hierarchical system governed by lightweight gates that jointly manage agent selection, routing depth, communication, and pruning dynamically per query. ⚙️ 𝐂𝐨𝐆𝐑𝐏𝐎 𝐓𝐫𝐚𝐢𝐧𝐢𝐧𝐠: We adapt GRPO for collaborative settings with a novel, critic-free RL recipe that assigns a shared advantage signal to all participating agents and gates during a rollout. 🔄 𝐇𝐨𝐭-𝐒𝐰𝐚𝐩𝐩𝐚𝐛𝐥𝐞 𝐄𝐱𝐩𝐞𝐫𝐭𝐬: GRADE features an Expert Registry with per-agent calibration maps. You can swap out expert models at inference time using just 64 anchor queries, without retraining the gates. 🏆 At ~17B average active parameters, GRADE outperforms all baselines on GSM8K, GPQA, and MMLUPro -- beating the strongest baseline on MMLUPro by 4.8 points while using ~39% fewer active parameters. w/ @ScientificGhosh Do check out many more exciting works on small models and agentic coordination being developed as part of our mega project -- 𝐏𝐚𝐫𝐚𝐦𝐚𝐧𝐮 parmanu.lcs2.in/ @lcs2lab @iitdelhi #LLMEfficiency #MultiagentRounting #AgenticAI

ayan sengupta avatar
ayan sengupta@ayans007·Jul 5

Knowledge distillation has a paradox: a student can match its teacher's generalization (right answers) without its fidelity (right reasoning). Turns out that's an off-policy problem. On-policy distillation couples generalization and fidelity - the student learns to reason like its teacher, not just answer like it. Read the full article at - x.com/ayans007/status/207372…. @lcs2lab @Tanmoy_Chak

ayan sengupta @ayans007 Jul 5 Article On-policy Distillation fixes the Fidelity-Generalization Paradox The field has quietly switched from off-policy to on-policy distillation for LLMs. Here's a reason that doesn't usually get stated: on-policy doesn't just improve accuracy - it makes the student

ayan sengupta avatar
ayan sengupta@ayans007·Jun 12

What makes a good teacher? On-policy distillation has spent the year reinventing loss functions to fix problems that come from one source: the teacher doesn't know the student. New article on why every popular OPD loss has an unbounded advantage and why the fix isn't another loss. Read the full article at - x.com/ayans007/status/206541… w/ @Tanmoy_Chak @lcs2lab #LLM #KnowledgeDistillation

ayan sengupta @ayans007 Jun 12 Article On-policy distillation needs a tutor, not a lecturer Distillation works when the teacher knows where the student is. A short article on why student-awareness is the principle, and why solely focusing on the appropiate distillation loss is not the right

ayan sengupta avatar
ayan sengupta@ayans007·Jun 1

Three 2026 papers (AOPD, SOD, OPSD) are patching on-policy distillation one failure mode at a time. Our 2023 work, MPDistil (ICLR 2024 poster), already had structural answers to all four. A short article on the convergence. @Tanmoy_Chak @lcs2lab