Shizhan Zhu

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2023-266

December 11, 2023

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2023/EECS-2023-266.pdf

Implicit model has emerged as a promising representation for numerous applications, including signal processing, image representation as well as 3D modeling. In this thesis, we will showcase its applications in three domains. First, we show that implicit representation could aid single view surface reconstruction of scenes by learning from a large collections of indoor scene data, and generalize well to unseen indoor scenes. Second, we show that implicit modeling can facilitate grounding of the language query in the 3D space. Lastly, we show that neural relighting, a type of algorithm that incorporates the neural field for modeling the light field, could achieve promising results. On one hand, by incorporating the pre-computed radiance transfer into the neural radiance modeling, we can enable handling of the type of materials with subsurface scattering effects. A hypernet-based representation could further facilitate fast image-based relighting. On the other hand, by carefully modeling the local micro-geometry with the surface normal modeling, as well as incorporating various hints for lighting reflection and self reflection, we could faithfully recover reflection highlights with varying material roughnesses.

Advisors: Trevor Darrell

---

BibTeX citation:

@phdthesis{Zhu:EECS-2023-266,
    Author= {Zhu, Shizhan},
    Title= {Implicit Modeling for 3D Applications},
    School= {EECS Department, University of California, Berkeley},
    Year= {2023},
    Month= {Dec},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2023/EECS-2023-266.html},
    Number= {UCB/EECS-2023-266},
    Abstract= {Implicit model has emerged as a promising representation for numerous applications, including signal processing, image representation as well as 3D modeling. In this thesis, we will showcase its applications in three domains. First, we show that implicit representation could aid single view surface reconstruction of scenes by learning from a large collections of indoor scene data, and generalize well to unseen indoor scenes. Second, we show that implicit modeling can facilitate grounding of the language query in the 3D space. Lastly, we show that neural relighting, a type of algorithm that incorporates the neural field for modeling the light field, could achieve promising results. On one hand, by incorporating the pre-computed radiance transfer into the neural radiance modeling, we can enable handling of the type of materials with subsurface scattering effects. A hypernet-based representation could further facilitate fast image-based relighting. On the other hand, by carefully modeling the local micro-geometry with the surface normal modeling, as well as incorporating various hints for lighting reflection and self reflection, we could faithfully recover reflection highlights with varying material roughnesses.},
}

EndNote citation:

%0 Thesis
%A Zhu, Shizhan 
%T Implicit Modeling for 3D Applications
%I EECS Department, University of California, Berkeley
%D 2023
%8 December 11
%@ UCB/EECS-2023-266
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2023/EECS-2023-266.html
%F Zhu:EECS-2023-266