←
Tsinghua KEG (THUDM)238 posts
Tsinghua KEG (THUDM) avatar

Tsinghua KEG (THUDM)

@thukeg

#ChatGLM #GLM130B #CodeGeeX #CogVLM #CogView #AMiner The Knowledge Engineering Group (KEG) and THUDM at @Tsinghua_Uni @jietang @ericdongyx

◷ Joined July 2022▧ 32 Photos and videos
Tsinghua CS avatar
⟳ Tsinghua KEG (THUDM) retweeted
Tsinghua CS@thudcst·11 Apr 2025

🏆Congrats to the Storage Research Group from #Tsinghua DCST for winning the#ASPLOS2025/#EuroSys2025 Large-Scale Model Inference Optimization Contest in Rotterdam! They outperformed global competitors, boosting inference performance by 1.1x using AWS NKI framework optimizations.

Post attachment
Paul Vicol avatar
⟳ Tsinghua KEG (THUDM) retweeted
Paul Vicol@PaulVicol·14 Dec 2024

Ruslan Salakhutdinov at the Adaptive Foundation Models Workshop!

Post attachment
Richard Socher avatar
⟳ Tsinghua KEG (THUDM) retweeted
Richard Socher@RichardSocher·20 Nov 2024

AI has a "last-mile problem" similar to self-driving cars. With self-driving cars, early demos impressed, but real-world deployment took years. It's easy to hack up a prototype, but making it work reliably at scale is hard. If each step of an AI agent is only 95% accurate, none of the 30-step workflows will work reliably. Going from 95% to 99.9% accuracy is the real challenge.

1:34
Z.ai avatar
⟳ Tsinghua KEG (THUDM) retweeted
Z.ai@Zai_org·18 Sep 2024

Thank you to the passionate developers for your continued support and patience. CogVideoX-5B-I2V, release!😀 Github: github.com/THUDM/CogVideo CogVideoX-5B-I2V model: huggingface.co/THUDM/CogVide… Gradio space: huggingface.co/spaces/THUDM/…

Gradio avatar
⟳ Tsinghua KEG (THUDM) retweeted
Gradio@Gradio·14 Aug 2024

LongWriter-glm4-9b from @thukeg is capable of generating 10,000+ words at once!🚀 Paper identifies a problem with current long context LLMs -- they can process inputs up to 100,000 tokens, yet struggle to generate outputs exceeding lengths of 2,000 words. Paper proposes that an LLM's effective generation length is inherently bounded by the sample it has seen during supervised fine-tuning😮 Demonstrates that existing long context LLMs already possess the potential for a larger output window--all you need is data with extended output during model alignment to unlock this capability. Code & models are released under Apache License 2.0🧡

0:12
Yushi Bai avatar
⟳ Tsinghua KEG (THUDM) retweeted
Yushi Bai@realYushiBai·14 Aug 2024

Thanks @_akhaliq! We find that your long context LLM is secretly a LongWriter💡All you need is data with extended output during model alignment to unlock this capability. Our code, data, and models: github.com/THUDM/LongWriter

AK@_akhaliq14 Aug 2024

LongWriter Unleashing 10,000+ Word Generation from Long Context LLMs discuss: huggingface.co/papers/2408.0… Current long context large language models (LLMs) can process inputs up to 100,000 tokens, yet struggle to generate outputs exceeding even a modest length of 2,000 words. Through controlled experiments, we find that the model's effective generation length is inherently bounded by the sample it has seen during supervised fine-tuning (SFT). In other words, their output limitation is due to the scarcity of long-output examples in existing SFT datasets. To address this, we introduce AgentWrite, an agent-based pipeline that decomposes ultra-long generation tasks into subtasks, enabling off-the-shelf LLMs to generate coherent outputs exceeding 20,000 words. Leveraging AgentWrite, we construct LongWriter-6k, a dataset containing 6,000 SFT data with output lengths ranging from 2k to 32k words. By incorporating this dataset into model training, we successfully scale the output length of existing models to over 10,000 words while maintaining output quality. We also develop LongBench-Write, a comprehensive benchmark for evaluating ultra-long generation capabilities. Our 9B parameter model, further improved through DPO, achieves state-of-the-art performance on this benchmark, surpassing even much larger proprietary models. In general, our work demonstrates that existing long context LLM already possesses the potential for a larger output window--all you need is data with extended output during model alignment to unlock this capability.

0:12
AK avatar
⟳ Tsinghua KEG (THUDM) retweeted
AK@_akhaliq·14 Aug 2024

LongWriter Unleashing 10,000+ Word Generation from Long Context LLMs discuss: huggingface.co/papers/2408.0… Current long context large language models (LLMs) can process inputs up to 100,000 tokens, yet struggle to generate outputs exceeding even a modest length of 2,000 words. Through controlled experiments, we find that the model's effective generation length is inherently bounded by the sample it has seen during supervised fine-tuning (SFT). In other words, their output limitation is due to the scarcity of long-output examples in existing SFT datasets. To address this, we introduce AgentWrite, an agent-based pipeline that decomposes ultra-long generation tasks into subtasks, enabling off-the-shelf LLMs to generate coherent outputs exceeding 20,000 words. Leveraging AgentWrite, we construct LongWriter-6k, a dataset containing 6,000 SFT data with output lengths ranging from 2k to 32k words. By incorporating this dataset into model training, we successfully scale the output length of existing models to over 10,000 words while maintaining output quality. We also develop LongBench-Write, a comprehensive benchmark for evaluating ultra-long generation capabilities. Our 9B parameter model, further improved through DPO, achieves state-of-the-art performance on this benchmark, surpassing even much larger proprietary models. In general, our work demonstrates that existing long context LLM already possesses the potential for a larger output window--all you need is data with extended output during model alignment to unlock this capability.

0:12
Tsinghua KEG (THUDM) avatar
Tsinghua KEG (THUDM)@thukeg·14 Aug 2024

#VisualAgentBench: proprietary models (4o, 4o-mini, 3.5-sonnet) currently have an edge as visual foundation agents for now, but open models InternVL & GLM-4V are catching up fast, a similar story to LLMs as agents as revealed in #AgentBench back in Aug 2023. arxiv.org/pdf/2408.06327 github.com/THUDM/VisualAgent…

🚨Thrilled to present VisualAgentBench (VAB) with @yugu_nlp and Tianjie, where we enable both TRAINING & TESTING of visual foundation agents across 5 different environments! In all 17 large multimodal models (LMMs) are tested. Find our paper, data, and more insights below 👇 Paper: arxiv.org/abs/2408.06327 Code & Data: github.com/THUDM/VisualAgent… Thanks @_akhaliq for sharing on today’s arxiv on HF!

2:41
Z.ai avatar
⟳ Tsinghua KEG (THUDM) retweeted
Z.ai@Zai_org·26 Jul 2024

We are not just doing “demo only” for video generation. Ying, we are bringing a video generation AI that everyone can use. Create a 6-second video in just 30 seconds. Try our new product now. YING:chatglm.cn/video medium.com/@ChatGLM/zhipuai-…

0:06
0:06
0:06
0:06
Tsinghua CS avatar
⟳ Tsinghua KEG (THUDM) retweeted
Tsinghua CS@thudcst·17 Jul 2024

🏆Proud moment for us! Our paper on 'Explicit factor models for explainable recommendation'(u6v.cn/5OxPGm) has won the Test of Time Award at #SIGIR2024, leading the way in 'explainable recommendation' since 2014. Congrats to outstanding THUIR group from #DCST, #Tsinghua

Post attachment