←
rewon101 posts
rewon avatar

rewon

@rewonfc

Here for the research papers

◷ Joined November 2012▧ 4 Photos and videos
rewon avatar
rewon@rewonfc·31 Oct 2025

some news: i quit MSFT last week after 3.5 years with the inflection/microsoft AI crew -- lots of memorable times. i'm taking some time to reconnect w/ old colleagues and friends and its been great. if you are reading this and want to catch up, DM me!!

Inflection AI avatar
⟳ rewon retweeted
Inflection AI@inflectionAI·29 Jun 2023

It’s a big week! We’ve raised $1.3 billion and are building the world’s largest AI cluster (22k H100s). We’re grateful for our investors and new funding that will help us accelerate our mission to make personal AI available to every person in the world. inflection.ai/inflection-ai-…

Jamie Townsend avatar
⟳ rewon retweeted
Jamie Townsend@_j_towns·10 Mar 2021

I've ported @rewonfc's very deep VAE (arxiv.org/abs/2011.10650) from PyTorch to JAX/Flax! Hope other JAX users find this SOTA VAE useful as a forkable baseline... github.com/j-towns/vdvae-jax.

Aran Komatsuzaki avatar
⟳ rewon retweeted
Aran Komatsuzaki@arankomatsuzaki·5 Mar 2021

My post on SotA image generative models was released 🥳 Featured 7 notable recent papers with emphasis on: - VD-VAE - VAE + discriminator (e.g. VQGAN, DC-VAE) - Diffusion models (e.g. DDPMv2) Plus some notes on scaling (e.g. DALL-E) and evaluation. arankomatsuzaki.wordpress.co…

Durk Kingma avatar
⟳ rewon retweeted
Durk Kingma@dpkingma·5 Jan 2021

IMO, best empirical proof to date that AI can be creative. After this sinks in, will there be any naysayers left?

"The images are preprocessed to 256x256 resolution during training. [...] each image is compressed to a 32x32 grid of discrete latent codes using a discrete VAE that we pre-trained using a continuous relaxation." GPT + VAE + scale = impressive results! openai.com/blog/dall-e/

rewon avatar
rewon@rewonfc·17 Dec 2020

It is easy to write a program but it is difficult to create a machine that will read those lines. (Was looking through my journal, found this gpt-3 generation conditioned on haikus)

rewon avatar
rewon@rewonfc·1 Dec 2020

Really thought-provoking work -- congrats to the authors @poolio @YSongStanford @dpkingma and more!

Happy to announce our new work on score-based generative modeling: high quality samples, exact log-likelihoods, and controllable generation, all available through score matching and Stochastic Differential Equations (SDEs)! Paper: arxiv.org/abs/2011.13456

Post attachment
Arash Vahdat avatar
⟳ rewon retweeted
Arash Vahdat@ArashVahdat·6 Oct 2020

It breaks my 💚 when researchers tell me that VAEs don't work. My first typical question is "did you try hierarchial VAE or vanilla VAE?", the answer is usually vanilla VAE. VAEs work much better with hierarchical structures. NVAEs and this work take this to the extreme!

hardmaru@hardmaru6 Oct 2020

Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images “Very Deep VAEs” achieve higher likelihoods, use fewer parameters, generate samples 1000x faster, and are more easily applied to hi-res images, compared to PixelCNN. openreview.net/forum?id=RLRX…

Post attachmentPost attachmentPost attachment
Stanislas Polu avatar
⟳ rewon retweeted
Stanislas Polu@spolu·9 Sep 2020

Posted my first paper on arXiv💥🙌 GPT-f is a Transformer-based automated theorem prover. We show that Transformer + Search is suitable to formal reasoning and continuous self-improvement 🦾 arxiv.org/abs/2009.03393

Post attachmentPost attachmentPost attachmentPost attachment
rewon avatar
rewon@rewonfc·17 Jul 2020

Our base model is a Sparse Transformer. If we make it bigger and train for a while with this augmentation, it results in both very high likelihoods (2.55-2.65 bpd on CIFAR-10) and also samples equal/better than most GANs (as measured by FID). Code here: github.com/openai/distributi…

rewon avatar
rewon@rewonfc·17 Jul 2020

Thanks if you came to our ICML poster on Distribution Augmentation. The zoom discussion was way more fun/interesting than I expected! TLDR of our work: use powerful data aug in your generative model by conditioning it on the aug. Improves samples + likelihoods considerably.

Post attachment
Kaj Sotala avatar
⟳ rewon retweeted
Kaj Sotala@xuenay·15 Jul 2020

I keep seeing all kinds of crazy reports about people's experiences with GPT-3, so I figured that I'd collect a thread of them.