In this project, we will explore generative AI with a focus on image, video, and 3D content creation and editing. You will build up your background in computer graphics and computer vision and conduct research in the following themes including latent diffusion models and neural radiance fields.
1. Generative models, i.e., latent diffusion models
We will explore image, video, 3D content creation using latent diffusion models with multi-modality input and output including text-to-image, text-to-3D content, and text-to-video, image-to-3D, image-to-video tasks, focusing on high-quality and efficient asset generation.
2. Neural radiance fields
a. Scene repsentation. We will be interested in designing lightweight scene representation, rendering, and training procedures for different applications with real-time rendering performance.
b. Relightable scenes. We will explore using radiance fields for physically based rendering effects.
c. Immersive radiance fields. We will explore and develop robust representation and rendering techniques for panoramic 360-degree neural radiance fields that provide immersive experience.
3. Application projects
a. Interactive 3D content creation tools. We will build a next-generation 3D authoring tool that adopts state-of-the-art generative 2D and 3D models for interactive content creation.
b. Dataset creation. We will build the next large-scale 3D datasets for generative modeling.
Eligibility
It is recommended that you have either taken computer graphics, computer vision, or machine learning modules with a strong coding background in Python or C++. Some experience with Pytorch, Unity, Unreal Engine, or Blender is a good plus.
I am super-interested in working on projects that can potentially submitted to top-tier computer vision and computer graphics venues such as CVPR, ECCV, ICCV, SIGGRAPH, and Eurographics. You are a perfect match if you are highly interested in pursuing research, either in academia or industry, as your long-term career.
If you have your own idea in 3D computer graphics and 3D computer vision in mind, you are welcome to discuss with me to develop this idea into a research project. You can find my latest research work on my Google Scholar.