Abstract

In this work, we address the problem of scene-aware 3D human avatar generation based on human-scene interactions. In particular, we pay attention to the fact that physical contact between a 3D human and a scene (i.e., physical human-scene interactions) requires a geometrical alignment to generate natural 3D human avatar. Motivated by this fact, we present a new 3D human generation framework that considers geometric alignment on potential contact areas between 3D human avatars and their surroundings. In addition, we introduce a compact yet effective human pose classifier that classifies the human pose and provides potential contact areas of the 3D human avatar. It allows us to adaptively use geometric alignment loss according to the classified human pose. Compared to state-of-the-art method, our method can generate physically and semantically plausible 3D humans that interact naturally with 3D scenes without additional post-processing. In our evaluations, we achieve the improvements with more plausible interactions and more variety of poses than prior research in qualitative and quantitative analysis. Project page: https://bupyeonghealer.github.io/phin/ .

Intuition of Geometric Alignment Loss

Left: The overall appearance of the human avatar and the object for each geometric alignment loss (i.e., distance and normal losses). Right: Illustration of the before/after effect of minimizing each geometric loss. Distance loss computes the point-wise distance between the human avatar and the object of the scene (gradient from red to blue) and operates in the direction of minimizing it. Normal loss operates in the direction of minimizing the cosine similarity between the normal vector of human avatars (blue arrow) and the normal vector of objects (red arrow).

Results

Video

Acknowledgements

This work was supported by Institute of Information & communications Technology Planning & Evaluation(IITP) grant funded by the Korea government(MSIT)(No.2020-0-01336, Artificial Intelligence Graduate School Program(UNIST)).
The website template was borrowed from Michaël Gharbi and Jon Barron.