
Mixed Reality + Diffusion prototype as a tool for exploring concepts, styles, and moods by transforming real-world surroundings into alternate realities. #MixedReality #MR #AI #StableDiffusion #Quest3

Try / Share / Iterate — Creative Technologist / Chief of Innovation @bonjourdpt

Mixed Reality + Diffusion prototype as a tool for exploring concepts, styles, and moods by transforming real-world surroundings into alternate realities. #MixedReality #MR #AI #StableDiffusion #Quest3

Revisited a 3DGS self-portrait using Atlas from @theworldlabs via its API and Ailleurs, a custom tool I've been developing.

Quick test with Frameboy, the real-time version of Atlas from @theworldlabs . This scene started with a few photos I captured at the Montreal Museum of Fine Arts of a cathedral model by Antoine Polisois. Here, I'm using Atlas + Frameboy to step inside it, generating entirely new views in real time as I explore.

Vitre Experiment #8... The renovations are complete! Once again, I used Atlas from @theworldlabs to generate the alternate version of our hallway, starting from a 3DGS capture of the actual physical space. This time, the entire process was done using Ailleurs, the tool I've been developing on top of the World Labs API. It allows me to generate new versions of a real captured space while preserving its metric scale and spatial alignment. The resulting 3DGS is then brought into Vitre, where it lines up with the physical hallway. I'll share more about Ailleurs in a separate post. #3DGS #WorldModels #GaussianSplatting

I've been building a tool called Ailleurs, using the @theworldlabs API. This has been in the making since I started my experiments with Vitre and Dormant Memories, where I've been using world models to generate alternate versions of real places captured as 3D Gaussian Splats. One of the main challenges was keeping the generated world spatially aligned with the original capture. I wanted a workflow where I could start directly from an existing 3DGS, use its metric scale, cameras and depth information, and generate completely new versions without losing that spatial relationship. Here's the workflow: - Import an existing 3DGS capture, along with its cameras and images. - Explore different looks using an image-to-image model for initial edits. - Propagate the style across the captured space using Atlas. - Generate new views with Atlas, using the original scan's depth and camera positions to preserve the geometry. - Export a dataset with known camera poses and train a new 3DGS using Spirula Studio, or directly with Atlas now that it supports 3DGS export. - Compare and blend the original and generated 3DGS directly in Ailleurs. Because the generated dataset already shares the original capture's coordinate system, there's no need to run SfM again or manually realign the resulting 3DGS. In the latest experiment, I generated 146 views of our hallway, trained a new 3DGS, and ended up with a median distance of around 3 cm from the original capture. It's still very much a prototype, but the entire pipeline is now working end to end, from a physical capture to an alternate 3D world that can be brought back into the real space.

A closer look at the alignment between the original 3DGS of our hallway and the alternate version generated with Atlas from @theworldlabs. Using Ailleurs, the tool I built on top of the Atlas API, I can start directly from the original capture, preserving its metric scale and spatial alignment throughout the process. In the end, both 3DGS end up almost perfectly aligned with each other and with the actual physical hallway. ...and a little glimpse of the Ailleurs UI at the end of the video.
Vitre Experiment #7 For this experiment, I turned our office hallway into an aging biotechnology research facility. This time, the alternate reality was created using Ailleurs, a new tool I've been building on top of the @theworldlabs Atlas / Marble 2 API. The idea behind Ailleurs is to start from an existing 3DGS capture of a real location and use Atlas to generate entirely new versions of that space, while preserving its original geometry, scale, and alignment. #3DGS #WorldModels #GaussianSplatting

Vitre Experiment #7 For this experiment, I turned our office hallway into an aging biotechnology research facility. This time, the alternate reality was created using Ailleurs, a new tool I've been building on top of the @theworldlabs Atlas / Marble 2 API. The idea behind Ailleurs is to start from an existing 3DGS capture of a real location and use Atlas to generate entirely new versions of that space, while preserving its original geometry, scale, and alignment. #3DGS #WorldModels #GaussianSplatting

Vitre Experiment #6… my favorite one so far! Same hallway, new portal effect opening onto a completely different world. Here again, I used Atlas and Marble 2 from @theworldlabs to reimagine the hallway from my original 3D capture. I’m having a lot of fun exploring what these tools can do, and I’m really impressed with the results. As with my previous experiments, the idea behind Vitre is to make the monitor disappear by aligning a virtual scene with the physical world. And of course… everything is 3DGS, rendered in real time. #3DGS #WorldModels #GaussianSplatting

Look where your message got me, @kcimc

I love the point cloud and the detailed renders generated by Atlas... Here's a bit of the process to create the alternate version of the hallway... I used Atlas from @theworldlabs to create the look and generate a series of consistent views, then trained my own 3DGS from those views.
Vitre Experiment #5: Opening a portal… I’ve been experimenting with Atlas and Marble 2 from @theworldlabs , and seriously, the results are impressive! I generated an alternate version of the hallway I’ve been using for my experiments, then used Vitre to align it with the physical space and create the illusion of a portal opening into this alternate reality. #GaussianSplatting #3DGS #WorldModels

Vitre Experiment #5: Opening a portal… I’ve been experimenting with Atlas and Marble 2 from @theworldlabs , and seriously, the results are impressive! I generated an alternate version of the hallway I’ve been using for my experiments, then used Vitre to align it with the physical space and create the illusion of a portal opening into this alternate reality. #GaussianSplatting #3DGS #WorldModels

I’ve been exploring what can be done with the developer data exported from @XGRIDS_OFFICIAL LCC Studio: combining the LiDAR data from a PortalCam capture with images captured separately, in this case using a @DJIGlobal Osmo 360. Left: 3DGS generated directly in LCC Studio from the PortalCam capture. Right: 3DGS trained in Spirula Studio (which is amazing) using the PortalCam LiDAR data for the geometry and the Osmo 360 images for the pixels. Still a WIP... The hybrid version (trained with Spirula) has better colour and a different overall feel, while the LCC version is still noticeably sharper and preserves more fine detail. Next test: adding some of the PortalCam’s own high-resolution images back into the Spirula dataset (the forward-facing rectilinear frames to start), while keeping the Osmo images as the majority of the dataset. #GaussianSplatting #3DGS #PortalCam #SpirulaStudio #Osmo360