Zhe Cao

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2022-28

May 1, 2022

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2022/EECS-2022-28.pdf

We exist in a 3D world, where we accomplish everyday tasks by perceiving and interacting with other people and objects in dynamic scenes. Could we develop a perception system to understand such rich interactions? This is crucial for future intelligent systems to collaborate with humans and to create immersive AR/VR experiences. While great progress has been achieved in the individual perception tasks of 3D humans, objects, and scenes, the connections between these components have not been explored much. In this thesis, we attempt to build the connections between these three components to understand their rich interactions. We start by bridging the scene and object component in Chapter 2, where we present an end-to-end learning system to perceive the 3D scene and independent object motions. We next show how 3D scenes influence human motion in Chapter 3, where we design a framework to predict future 3D human motion considering the scene context. In Chapter 4, we study the interaction between human hands and objects, where we introduce an optimization-based method to reconstruct the interaction in the wild. Finally, we conclude with several interesting future directions.

Advisors: Jitendra Malik

---

BibTeX citation:

@phdthesis{Cao:EECS-2022-28,
    Author= {Cao, Zhe},
    Title= {Perceiving 3D Humans and Objects in Motion},
    School= {EECS Department, University of California, Berkeley},
    Year= {2022},
    Month= {May},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2022/EECS-2022-28.html},
    Number= {UCB/EECS-2022-28},
    Abstract= {We exist in a 3D world, where we accomplish everyday tasks by perceiving and interacting with other people and objects in dynamic scenes. Could we develop a perception system to understand such rich interactions? This is crucial for future intelligent systems to collaborate with humans and to create immersive AR/VR experiences. While great progress has been achieved in the individual perception tasks of 3D humans, objects, and scenes, the connections between these components have not been explored much. In this thesis, we attempt to build the connections between these three components to understand their rich interactions. 
	
We start by bridging the scene and object component in Chapter 2, where we present an end-to-end learning system to perceive the 3D scene and independent object motions. We next show how 3D scenes influence human motion in Chapter 3, where we design a framework to predict future 3D human motion considering the scene context. In Chapter 4, we study the interaction between human hands and objects, where we introduce an optimization-based method to reconstruct the interaction in the wild. Finally, we conclude with several interesting future directions.},
}

EndNote citation:

%0 Thesis
%A Cao, Zhe 
%T Perceiving 3D Humans and Objects in Motion
%I EECS Department, University of California, Berkeley
%D 2022
%8 May 1
%@ UCB/EECS-2022-28
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2022/EECS-2022-28.html
%F Cao:EECS-2022-28