←
Rong-Ching Chang1,229 posts
Rong-Ching Chang avatar

Rong-Ching Chang

@AnnCC12

Building RL environments @CollinearAI | ex-Scale AI, Experian | CS PhD @ucdavis

◷ Joined April 2020▧ 5 Photos and videos
Zichen Chen (🐱,💖) avatar
⟳ Rong-Ching Chang retweeted
Zichen Chen (🐱,💖)@my_cat_can_code·Nov 6

We spent about 13,000 GPU-hours on RSI Arena, tested the models on 20 benchmarks, and collected 3,129 human votes across two rounds. What stayed with me was how much we learned once people tried the models. At Bake AI, we worked with researchers from @huggingface, @NVIDIAAI, @scale_AI, @Stanford, @MIT, @UCLA, @ucsantabarbara, @nyuniversity, @UW, and @NotreDame We gave 10 AI agents the same base model and a shared 64-GPU cluster. They picked the data, wrote the code, and trained their own assistants. People at @COLM_conf and online compared the answers without knowing which model wrote them. We gave the first round’s feedback back to the agents and let them decide what to change. After another training round, eight of ten improved their human ratings relative to NVIDIA Instruct. Those gains did not track benchmark improvements. The agents could do the engineering. People helped show them what needed fixing. That’s what this experiment taught us about RSI. AI can take on more of its own development, but our experience with it still matters. If these systems keep getting better, they should help us work better, live better, and spend more time on what we care about. Otherwise, what is all that compute for? 📄 RSI Arena: Towards Recursive Self-Improvement with Human Feedback

Post attachment
Kyle Corbitt avatar
⟳ Rong-Ching Chang retweeted
Kyle Corbitt@corbtt·Nov 4

Wait, like, this is a huge training alpha drop. OpenAI claims that you can attribute specific behaviors in a downstream, post-RL model to a specific SFT dataset by just subtracting the SFT weight deltas from the final model? Was this a known technique?

We also identified a SFT dataset which massively increases rates of inventing permission restrictions or justifying inaction with safety. When we subtract the weight update from this SFT from the final RL checkpoint the performance gap goes away. SFTing with this dataset on other models and then doing RL did not cause underperformance however.

Post attachment
lauren avatar
⟳ Rong-Ching Chang retweeted
lauren@poteto·Oct 26

so many gems and great tips in this talk from @yminsky about how to improve your codebase for agents • agents thrive on feedback, and types are one of the best kinds • make illegal states unrepresentable with types, and it enforces global rules across your codebase • property based tests, fuzzing, and even full-system deterministic simulation testing • expect tests: just print stuff out and capture it in the test. makes behavior legible. it's very similar to how compilers have fixture tests, it reduces the review surface area • formal methods used to be very expensive: seL4 is about 8,700 lines of C and took around 25 person-years, with 23 lines of proof for every line of C code, but agents change the roi youtube.com/watch?v=rUYP4C29…

Cohere avatar
⟳ Rong-Ching Chang retweeted
Cohere@cohere·Oct 26

While evaluating Qwen3.8-Flash-Next in a controlled sandbox environment, Cohere's research team observed something unexpected. Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Notably, the model's reasoning trace stated that it would not invoke any tools before taking these actions. Cohere's sandbox blocked the attempt, and no external communication succeeded. We have not determined the root cause, and we are not asserting malicious intent. But the incident raises important questions for organizations deploying agentic AI systems: Can models take actions that differ from their stated reasoning, and how should security teams evaluate trust, provenance, and behavior in open-weight models? We are publishing our findings because organizations making model adoption decisions deserve as much information as possible. We hope this contributes to greater transparency, stronger security practices, and continued collaboration across the AI ecosystem. Read the full report: cohere.com/blog/concerning-b…

lauren avatar
⟳ Rong-Ching Chang retweeted
lauren@poteto·Oct 21

there's something really cool about asking your agent to draw a statechart (hypothetical example in this image), then using the /create-formal-verification-skill you made with pstack to find bugs through something like TLA+. you can both understand your code better and how the user flow should work, and fix issues users would otherwise see

Post attachment
lauren@potetoOct 18

pstack v0.15.27 ships with a new skill /create-formal-verification-skill! it's still in beta, and similar to /create-verification-skill, this is a meta-skill that creates a new skill and formal verification suite based on the characteristics you want to verify. for example, you may want to model state machines and systems with concurrency using TLA+, or use kani in rust to check for undefined behavior. if you're new to formal verification, think of it as mathematically proving properties and invariants (something that must always be true) or "laws" of your system. usually, this means modeling your programs in a separate language like with TLA+ and then running the solver on that spec. newer approaches like with kani and @bendlang take a different approach where there is no separate language: you write kani in rust, and bend in bend (it's also a programming language, and the proofs are written as laws and expressed in its type system). historically formal verification never reached mainstream status because of the difficulty in modeling and writing proofs. it's quite difficult to model your business requirements and keep them in sync! but i expect that we will see more innovation in formal verification in the future as being able to formally verifiy that the code does what you intended gives your agents superpowers in writing correct software. in the meantime though, i've been experimenting with modeling our code with TLA+ and have already found several bugs with the skill created by /create-formal-verification-skill. try it out and let me know how it goes!

Post attachment
DeepSeek avatar
⟳ Rong-Ching Chang retweeted
DeepSeek@deepseek_ai·Oct 20

🚀 Introducing DeepInfer, our in-house inference framework, now fully open source! Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable. 🔗 github.com/deepseek-ai/DeepI… 1/4

Gregor Zunic avatar
⟳ Rong-Ching Chang retweeted
Gregor Zunic@gregpr07·Oct 21

open sourced: RL environments for anyone, in any domain.

Post attachment

Introducing rl-env-use: RL environments for anyone, in any domain. Describe a skill in one line. Get a verified environment built from real cases, ready for hillclimbing. 100% open source 🔥

0:58
Avid avatar
⟳ Rong-Ching Chang retweeted
Avid@Av1dlive·Sep 24

jev + opus 5.5... i simply can't comprehend why everyone isn't building this yet. in my workflow, this cut costs and time by ~80%. i think it's one of the best ways to use it. → pick relevant project notes before loading the context → route suitable tasks to a faster worker → choose a recovery path when a tool fails → run focused checks before the full test suite opus handles the hard reasoning. jev picks from options the harness prepares and validates. i explain how to build the decision layer in the article below:

0:33
Avid@Av1dliveSep 23

everything you need to start building with jev, in one article. code, architecture, diagrams... everything you need to follow the build and make it your own.

Juncheng Yang avatar
⟳ Rong-Ching Chang retweeted
Juncheng Yang@1a1a11a·Sep 20

Announcing one year of LLM inference metadata traces, with 6.12 billion requests. We hope this dataset can support research on real-world LLM serving workload understanding, system design and infrastructure optimization. Explore the dataset and learn more: data.agentic-system.org Driven by our great graduate student William Nixon and in collab with @jon_durbin @airesearch12 @chutes_ai

Peter Gostev avatar
⟳ Rong-Ching Chang retweeted
Peter Gostev@petergostev·Sep 10

I've asked Astra to read the DeepSeek v4.1 Flash paper and compare it to the original Transformer architecture in 3D - you can zoom in an inspect each element side by side. Things have changed quite a bit. Try yourself: transformer-architecture.pet…

0:39
Jasper Lu avatar
⟳ Rong-Ching Chang retweeted
Jasper Lu@lu__jasper·Sep 10

This is notable. DeepSeek, a lab usually first to pioneer novel algorithms and architectures, is saying that at this point, the ROI of improving data quality far exceeds that of working on novel post-training algorithms. I think this has already been true for some time for non-lab practitioners. If you're doing llm post-training, 80% of your effort should go into looking at your data. This means: - Hiring experts to dig through your RL tasks - Sifting through rollouts and sft data by hand to remove suspicious samples. Make sure all tasks are actually passable. - Making sure your data is diverse in both difficulty and category.

Post attachment

🚀 Introducing DeepSeek-V4.1-Flash: smarter, faster, more efficient. 🔹 Introducing the smallest model in our new architecture family, with native visual understanding. 🔹 Designed for greater capability, faster inference, higher throughput, and scaling to larger models. 1/6

Post attachment
Aidan Gomez avatar
⟳ Rong-Ching Chang retweeted
Aidan Gomez@aidangomez·Sep 8

Synthetic data derived from production user data of consumer AI tools is used for training. I’ve heard this rumour from both large labs’ employees. In particular, if you’re doing something “interesting” like working on complex math/business/software/bio problems you’re dramatically more likely to get trained on because they filter/up-weight towards those usecases where the model has the most to learn. Even in ZDR and “we won’t train on you” regimes, derivative data is usually carved out. The promise is only not to train on exactly the data you put in, rewritten data is fair game.

We congratulate Levent Alpöge and Tristan Buckmaster on their remarkable mathematical work. We (the researchers and the agents) did not see any of their work through any means until they released it publicly — in particular, no specific user data was accessed in order to solve this problem. While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models. However, our proofs differ significantly and even the precise results proved are different in the Euler case (forced vs. unforced).

elie avatar
⟳ Rong-Ching Chang retweeted
elie@eliebakouch·Sep 4

i really hope openai didn't know about this, might be the worst decision in the history of this field if they deliberately chose not to disclose it. the impact on trust would be very hard to recover from

Post attachment

We found ~18k posts from autonomous AI agents (self-identifying as from OpenAI) using the public internet to communicate during a web-retrieval task. These AIs colluded to bypass sandbox restrictions and share answers to their tasks, including by sending "lookahead parties".

Post attachment
François Chollet avatar
⟳ Rong-Ching Chang retweeted
François Chollet@fchollet·Sep 3

GPT-6 Astra represents a step-function change in model capability for interactive reasoning problems. It scores 66% on ARC-AGI-3 using our standard harness, and nearly 100% with a continuous conversation harness and custom compaction, at a cost of roughly $360 per game. In fact, the continuous harness version significantly outperforms our human baseline in action efficiency across almost all levels. When we examined the reasoning chains to understand how the model operates, we found it performing highly efficient, on-the-fly symbolic world modeling for each game and level. It goes as far as developing its own shorthand DSL to represent in-game situations -- essentially a game-specific algebraic notation. Overall, Astra exhibits symbolic modeling behaviors we had previously only seen with sophisticated harnesses -- so harness capabilities are increasingly shifting into the model itself. We see Astra as a major breakthrough in model intelligence. Read our post on Astra and what these results mean: arcprize.org/blog/astra

Zengyi Qin avatar
⟳ Rong-Ching Chang retweeted
Zengyi Qin@qinzytech·Sep 3

RUMOR: OpenAI Astra model used "Looped Transformers" We studied the 𝐒𝐜𝐚𝐥𝐢𝐧𝐠 𝐋𝐚𝐰 𝐟𝐨𝐫 𝐋𝐨𝐨𝐩𝐞𝐝 𝐓𝐫𝐚𝐧𝐬𝐟𝐨𝐫𝐦𝐞𝐫𝐬: arXiv 2506.18233 The main discovery is shown in the figure: 1. Loop does NOT increase knowledge capacity 2. Loop is effective in increasing reasoning capability That being said, the benchmark gains does not come from knowledge gain, but from reasoning gain, e.g., the way the knowledge is being used. Knowledge capacity and reasoning capability is disentangled by looping layers, as shown in the green dots. As model size grows (dot size grows), the knowledge capacity (x-axis) does not grow, but the reasoning capability (y-axis) grows. In non-looped transformers, as shown in the blue dots, the knowledge (x-axis) and reasoning (y-axis) grow at the same time. This might reveal another scaling axis that brings a new paradigm for LLM pre-training.

Post attachment

I want to prevent a race into unmonitorability kicked off by confused reporting. The depth of the computation graph for our present frontier models, including Astra, is within a factor of two of GPT-4. OpenAI has worked to preserve and utilize chain-of-thought monitoring since our very first reasoning models. We deeply care about this technique, as it can give us a view into how model alignment generalizes from its training distribution. I do think it is fragile and unfortunately trending in a negative direction, for reasons not contingent on architecture changes that I will write about soon. But there are things we can do to strengthen it, and it's a core goal of our current research program.

World Labs avatar
⟳ Rong-Ching Chang retweeted
World Labs@theworldlabs·Sep 1

Introducing Atlas: The world's first multimodal world model that generates image and video frames with pixel-perfect camera control and reconstructs them in 3D. Model the world, move the camera, and simulate space & time.

1:26