Recovering and Creating 3D Experiences from Casual Data

Ethan Weber

EECS Department
University of California, Berkeley
Technical Report No. UCB/EECS-2025-75
May 15, 2025

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2025/EECS-2025-75.pdf

In today's world, we are surrounded by casually captured visual data—photos and videos from our phones, TV shows, cartoons, social media clips, and more. These formats depict rich, physical 3D spatiotemporal worlds, even though, when you look closely, the frames themselves are often not necessarily geometrically consistent or explicitly 3D. Yet, as viewers, we effortlessly perceive the underlying structure, intuitively reconstructing the spaces and stories they represent. We could even imagine what might lie in the space behind the camera, where the photographer is standing. Why is it that humans can so easily make sense of these visual experiences, while machines still struggle to recover or create 3D from such data? This thesis aims to bridge that gap, bringing the human-like ability to recover and create 3D experiences from casual data to machines. We develop new methods that robustly reconstruct 3D environments from unstructured, in-the-wild imagery, such as videos you took with your smartphone, and introduce generative techniques to complete missing regions and hallucinate plausible content where data is sparse or absent. Our work advances the state of the art in neural rendering, scene completion, and generative modeling, with contributions including open-source frameworks, new methods for artifact removal and generative scene completion, and the first large-scale 3D reconstruction of television shows and hand-drawn cartoons. By bridging the gap between 3D reconstruction and generation, this thesis explores new possibilities for experiencing and understanding the visual world—no matter how casual or unconventional the data may be.

Advisor: Angjoo Kanazawa

\"Edit"; ?>

BibTeX citation:

@phdthesis{Weber:EECS-2025-75,
    Author = {Weber, Ethan},
    Title = {Recovering and Creating 3D Experiences from Casual Data},
    School = {EECS Department, University of California, Berkeley},
    Year = {2025},
    Month = {May},
    URL = {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2025/EECS-2025-75.html},
    Number = {UCB/EECS-2025-75},
    Abstract = {In today's world, we are surrounded by casually captured visual data—photos and videos from our phones, TV shows, cartoons, social media clips, and more. These formats depict rich, physical 3D spatiotemporal worlds, even though, when you look closely, the frames themselves are often not necessarily geometrically consistent or explicitly 3D. Yet, as viewers, we effortlessly perceive the underlying structure, intuitively reconstructing the spaces and stories they represent. We could even imagine what might lie in the space behind the camera, where the photographer is standing. Why is it that humans can so easily make sense of these visual experiences, while machines still struggle to recover or create 3D from such data? This thesis aims to bridge that gap, bringing the human-like ability to recover and create 3D experiences from casual data to machines. We develop new methods that robustly reconstruct 3D environments from unstructured, in-the-wild imagery, such as videos you took with your smartphone, and introduce generative techniques to complete missing regions and hallucinate plausible content where data is sparse or absent. Our work advances the state of the art in neural rendering, scene completion, and generative modeling, with contributions including open-source frameworks, new methods for artifact removal and generative scene completion, and the first large-scale 3D reconstruction of television shows and hand-drawn cartoons. By bridging the gap between 3D reconstruction and generation, this thesis explores new possibilities for experiencing and understanding the visual world—no matter how casual or unconventional the data may be.}
}

EndNote citation:

%0 Thesis
%A Weber, Ethan
%T Recovering and Creating 3D Experiences from Casual Data
%I EECS Department, University of California, Berkeley
%D 2025
%8 May 15
%@ UCB/EECS-2025-75
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2025/EECS-2025-75.html
%F Weber:EECS-2025-75