2024 年度三维视觉前沿趋势与十大进展

As an interdisciplinary field that integrates computer vision，graphics，artificial intelligence（AI），and optical imaging，three-dimensional（3D）vision serves as the foundational cornerstone for building embodied general intelligence and the metaverse ecosystem. In 2024，differentiable representation technologies，exemplified by neural raddiance field （NeRF）and Gaussian splatting，continued to evolve and refine，gradually transcending the boundaries of traditional 3D reconstruction. From microscopic cellular structures to macroscopic physical celestial bodies，and from static scenes to dynamic human bodies，significant improvements in accuracy have been achieved across a spectrum. Propelled by advancements in generative AI technologies and the scaling laws of large models，the field of 3D vision has witnessed a paradigm shift from optimization to generalizable feedforward generation，marking important progress and breakthroughs in the direction of controllable digital content generation. Embodied intelligence remains a focal point of interest，with researchers increasingly recognizing the capture and generation of 3D virtual simulation data and 3D human motion data as central elements of training embodied intelligence. As the concepts of world models and spatial intelligence become hot top⁃ ics among technology researchers，modeling the physical world，understanding spatial relationships，and predicting future states have emerged as crucial research directions，all of which rely on the support of 3D vision technologies. Furthermore，through nontraditional visual sensors and novel reconstruction algorithms，innovations in computational imaging technology have exceeded the physical limitations and performance bottlenecks of traditional 3D reconstruction. These technological breakthroughs are propelling 3D vision into a new era characterized by an intelligent，large-scale learning process that encompasses the processes of perception，modeling，generation，and interaction. Specifically，the development trends in 3D vision primarily manifested in the following aspects：1）Controllable and physics-aware generation of the visual elements of AI-generated content（AIGC）. With the rapid development of AIGC technology，visual content generation has evolved from simple two-dimensional（2D）image creation toward more controllable and physics-aware approaches. This trend requires the combination of physical prior knowledge and multidimensional control parameters，such as 3D viewpoints，lighting conditions，and 3D character motion，to achieve higher-quality content generation. 3D vision technology plays a pivotal role in this process，providing essential spatial-temporal and physical constraints for AIGC. 2）4D spatial intelli⁃ gence：bridging virtual and physical worlds. 4D（3D space + time dimension）spatial intelligence has emerged as a core technology connecting virtual worlds（e. g. ，the metaverse）and physical realities（e. g. ，embodied intelligent robots）. This technology focuses on establishing a digital mapping of dynamic physical environments. By leveraging 3D vision and multimodal large model technologies，AI systems can construct 4D spatial models to understand spatial relationships，pre⁃ dict motion trajectories，and simulate future evolutions. In turn，intelligent agents can interactively learn within physical or virtual 4D environments to acquire intelligence. 3）Data-driven embodied intelligence：3D virtual simulation and human motion capture. The advancement of embodied intelligence relies heavily on high-quality 3D virtual simulation data and the capture/generation of human 3D motion data. These datasets fuel the training of embodied intelligent robots to achieve sophisticated behavioral control. Through high-precision 3D vision technologies，robots can better understand and simulate human actions，resulting in their enhanced intelligence in complex tasks. 4）Differentiable 3D representation and integra⁃ tion with large model technologies. From microscopic cellular structures to indoor environments，human/animal modeling，autonomous driving/city modeling，and even astronomical black hole reconstruction，novel 3D representations，such as NeRF and 3D Gaussian splatting（3DGS），drive performance improvements in scene generation and reconstruction across scales. Their efficiency and flexibility have opened up new possibilities for 3D vision applications. Furthermore，by inte⁃ grating large-scale 3D data with transformer-based architectures and advanced generative methods，such as diffusion mod⁃ els，fundamental 3D vision tasks are being unified into efficient end-to-end frameworks，resulting in the scaled-up learning of core 3D vision paradigms. The breakthroughs in 3D vision in 2024 have injected new momentum into technological devel⁃ opment，with future trends focused on several key directions. Spatiotemporal-consistent world models that integrate 4D spa⁃ cetime and physical laws provide dynamic prediction and interaction support for complex scenarios，such as autonomous driving and embodied intelligence. It is expected that the deep integration of generative AI and 3D content generation tech⁃ nologies will overcome data bottlenecks，enabling the automated creation of high-fidelity and controllable 3D content. Enhanced cross-modal generalization capabilities will also strengthen the fusion of vision，language，and motion modali⁃ ties，thus improving the adaptability and robustness of robotic strategies. Physics-driven dynamic reconstruction，com⁃ bined with physical engines，will achieve high-precision modeling and interactive editing of dynamic scenes，thus advanc⁃ ing digital twins and virtual reality. Meanwhile，3D imaging technologies will accelerate scientific exploration in astrophys⁃ ics and cell biology. Efficient real-time processing and lightweight solutions will also boost the reconstruction and rendering efficiency of large-scale dynamic scenes，promoting edge device applications. Furthermore，ethical and privacy protection will emerge as critical concerns，balancing innovation and security through encryption technologies and regulatory frame⁃ works that govern 3D data acquisition and generation. In summary，this article reveals that，in the future，3D vision will evolve toward an era of more intelligent and universal“spatial intelligence”in which technological breakthroughs will reshape human-computer interaction，scientific exploration，and industrial ecosystems. Therefore，to foster academic exchange，this article analyzed cutting-edge trends in 3D vision and highlights the top ten research breakthroughs of the year，thus offering insights and references for both academia and industry.