←
Shunsuke Saito952 posts
Shunsuke Saito avatar

Shunsuke Saito

@psyth91

Director, Research Scientist @AIatMeta Digital Human/Vision/Graphics デジタルヒューマンの研究してます。 #pifuhd #sapiens All opinions are my own.

◷ Joined April 2020▧ 96 Photos and videos
Shunsuke Saito avatar
Shunsuke Saito@psyth91·11 Aug 2025

Relightable Codec Avatars is now extended to full-body! At #SIGGRAPH2025, we will present Relightable Full-body Gaussian Codec Avatars. Key contributions include learnable Zonal Harmonics and deferred learnable radiance transfer for specular! Check it out! neuralbodies.github.io/RFGCA…

0:18
Sadao Tokuyama avatar
⟳ Shunsuke Saito retweeted
Sadao Tokuyama@tokufxug·Apr 9

Facebookやインスタ、VRのQuestやAIグラスのRay-Ban MetaのMetaによる新研究「LCA」。 100万本の動画学習により、スマホ撮影から表情や指の動きまで精密な3Dアバターを一瞬で生成可能に。 服の揺れや照明変化も自然に再現され、将来的にスタジオ級の分身がスマホで作れる世界を目指してる模様。

0:36
A.I.Warper avatar
⟳ Shunsuke Saito retweeted
A.I.Warper@AIWarper·Apr 27

Sapiens2 test. All things considered, probably the best pose I have used. Still heavily reliant on a good bbox detection though. I am out of the loop and do not know what the "Best" bbox detection is these days.

0:05
merve avatar
⟳ Shunsuke Saito retweeted
merve@mervenoyann·May 12

Meta silently dropped Sapiens2 last week 🔥 a family of high-res models trained on 1B human images > for pose estimation, body-part segmentation, surface normals, pointmaps (sota) > 6 sizes: 0.1B → 5B params (all ViT patch 16) > high-res: 1024×768 and 4K

0:05
Massimiliano Viola avatar
⟳ Shunsuke Saito retweeted
Massimiliano Viola@massiviola01·Jun 4

There used to be a time when a novel architecture would do it... But today, data is the one dictating the rules!😬 @Meta Reality Labs solved dense human-centric tasks with specialized foundation models, proving that they still hold a massive advantage over generalist ones.

Post attachment
Wojciech Zielonka avatar
⟳ Shunsuke Saito retweeted
Wojciech Zielonka@w_zielonka·Apr 29

I am happy to share that our STAR has been accepted to Eurographics 2026: “How to Build Digital Humans?” It introduces a novel taxonomy and a concise overview of the full creation pipeline, from face and body to hands, garments, and hair. tinyurl.com/5f6u7rks

Post attachment
Astrid Wilde 🌞 avatar
⟳ Shunsuke Saito retweeted
Astrid Wilde 🌞@astridwilde·Apr 24

Sapiens2 is the highest quality ViT backbone that now exists in the public domain. It was pretrained on the equivalent of 1/2 of all human images on Flickr. First public release by a large lab that is non-trivial to replicate. Huge public service. Well done.

Post attachmentPost attachmentPost attachmentPost attachment
Rawal Khirodkar avatar
⟳ Shunsuke Saito retweeted
Rawal Khirodkar@rawal_khirodkar·Apr 24

Introducing Sapiens2 — the next generation of our human-centric vision models Pretrained at scale and at high resolution, Sapiens2 learns human semantics more effectively without losing fidelity, and generalizes strongly across human vision tasks. Paper: arxiv.org/pdf/2604.21681 Accepted at ICLR 2026 Code: github.com/facebookresearch/… Demo: huggingface.co/collections/f…

2:27
Junxuan Li avatar
⟳ Shunsuke Saito retweeted
Junxuan Li@JunxuanL·Apr 3

LCA is accepted at CVPR 2026! 🚀 We introduce a pre/post-training paradigm for 3D avatars (1M in-the-wild videos ➡️ studio data). The result? High-fidelity full-body avatars with emergent relightability and zero-shot stylization. Project: junxuan-li.github.io/lca/ #CVPR2026

0:46
Pablo Vela avatar
⟳ Shunsuke Saito retweeted
Pablo Vela@pablovelagomez1·Jan 19

I've been working a lot with SAM3 and the Momentum Human Rig (MHR). I finally integrated it into the data I'm working with @rerundotio. The progression I've taken looks as follows SAM3 + SAM3D-body on 1. a single image 2. a set of multiple images 3. a single video 4. A multiview video capture I took inspiration from the SAM3D-body paper and built a multiview fitting optimization pipeline. This pipeline involves using the 2D keypoints from the single-view pipeline, triangulating them, and employing an L1 loss between the 2D/3D keypoints. The temporal stability isn't great, so that's the next portion I'm going to focus on. One really frustrating thing about SAM3D-body is the lack of per-joint confidence values. It makes it harder to deal with occlusions. I'm probably going to need to use a separate model, or maybe add a confidence head.

0:11

Back to working on exo + ego in @rerundotio. This is a big jump in progress! One of the main painpoints I've had is getting the ego and exo views aligned in the same coordinate system, but I finally managed to get it all working. This means that now I have 1. Slam working for ego 2. Calibrated exo views 3. 3D keypoints for the full human body 4. 6DoF wrist poses 5. Temporally aligned videos 6. Spatially aligned multi cameras Now it's time to scale it up 🙂

0:14
ゴメパパ avatar
⟳ Shunsuke Saito retweeted
ゴメパパ@gomessdegomess·18 Dec 2025

毎年お馴染みlevelsfyiの年度末レポートがやってきたので気になるところだけまとめてく メリカのトップ給与動向のまとめ

Post attachment
AI at Meta avatar
⟳ Shunsuke Saito retweeted
AI at Meta@AIatMeta·25 Nov 2025

SAM 3D is helping advance the future of rehabilitation. See how researchers at @CarnegieMellon are using SAM 3D to capture and analyze human movement in clinical settings, opening the doors to personalized, data-driven insights in the recovery process. 🔗 Learn more about SAM 3D: go.meta.me/305985

1:33
AI at Meta avatar
⟳ Shunsuke Saito retweeted
AI at Meta@AIatMeta·19 Nov 2025

Introducing SAM 3D, the newest addition to the SAM collection, bringing common sense 3D understanding of everyday images. SAM 3D includes two models: 🛋️ SAM 3D Objects for object and scene reconstruction 🧑‍🤝‍🧑 SAM 3D Body for human pose and shape estimation Both models achieve state-of-the-art performance transforming static 2D images into vivid, accurate reconstructions. 🔗 Learn more: go.meta.me/305985

0:50