Source: 3D Vision Workshop
Add a small assistant: dddvision, note: direction + school/company + nickname, pull you into the group. At the end of the article, industry subdivisions are attached
标题：Neural Radiance Field-based Visual Rendering: A Comprehensive Review
作者:Mingyuan Yao, Yukang Huo, Yang Ran, Qingbin Tian, Ruifeng Wang, Haihua Wang
Institution: China Agricultural University
Original link: https://arxiv.org/abs/2404.00714
In recent years, Neural Radiance Field (NeRF) has made remarkable progress in the field of computer vision and graphics, providing strong technical support for solving key tasks including 3D scene understanding, new perspective synthesis, human reconstruction, robotics, etc., and the academic community has paid more and more attention to this research achievement. As a revolutionary neuroimplicit field representation, NeRF has sparked a sustained research boom in the academic community. Therefore, the purpose of this review is to provide an in-depth analysis of the research literature on NeRF in the past two years and to provide a comprehensive academic perspective for new researchers. This paper first elaborates on the core architecture of NeRF, then discusses various strategies to improve NeRF, and provides case studies of NeRF in different application scenarios to demonstrate its practical utility in different fields. In terms of datasets and evaluation metrics, this article details the key resources required for NeRF model training. Finally, this paper provides an outlook on the future development trend and potential challenges of NeRF, aiming to provide research inspiration for researchers in this field and promote the further development of related technologies.
With the advent of NeRF, neural volume representations based on NeRF and other nerve volume representations have become a compelling technique for learning how to represent 3D scenes from images to render realistic scene images from never-before-observed viewpoints, and related articles have grown exponentially.
NeRF is now widely used in scenarios such as new perspective synthesis, 3D reconstruction, neural rendering, depth estimation, and pose estimation. Considering the rapid progress of NERF-based methods, it is becoming increasingly challenging to keep track of new research developments. Therefore, it is essential to conduct a comprehensive review of the latest advances in this field of research, which will have a positive impact on researchers in the field.
This article details the latest developments in NERF. The main contributions are as follows:
(1) Firstly, a comprehensive review of the existing literature related to NeRF was conducted, including a summary of the early work and an analysis of recent research trends.
(2) The individual elements of the original NeRF model are described in detail, including its network structure, loss function, and rendering method.
(3) A number of datasets were collected and analyzed in detail, and the commonly used NeRF evaluation indicators were summarized.
(4) Variants of NeRF are classified, and their innovations in improving rendering quality, accelerating computing, and applications in indoor, outdoor, human, interactive scenes, etc., are described in detail. The performance of different models in terms of speed, accuracy, and other key performance metrics such as rendering quality, memory usage, and generalization ability was also compared.
and (5) identified the main obstacles in the current research, such as the need for computing resources, the scalability of the model, and the ability to handle complex scenarios. Possible solutions to these challenges are further explored and potential directions for future research are proposed.
(6) The main contributions and impacts of NeRF are summarized, as well as the prospects for future development in this field.
A. Synthetic datasets
NeRF Synthesis Dataset (Blender Dataset): Proposed in the original NeRF paper, this dataset consists of complex 3D scenes crafted using Blender software, including various items such as chairs, drums, plants, etc. In addition, high-resolution images with resolutions up to 800x800 pixels are provided, and each scene is equipped with an appropriate collection of images for training, validation, and testing. In addition, the dataset includes depth and normal plots, as well as comprehensive camera transformation data, which provides important geometric and illumination details for training the NeRF model.
Local Light Field Fusion (LLFF) Dataset: This is a tool for innovative view synthesis research that combines artificial and physical images to facilitate the depiction of complex scenes in virtual exploration. The dataset includes artificial images created using SUNCG and UnrealCV, as well as 24 real-world scene photos taken from portable phones. The LLFF dataset is well suited for a wide range of new view synthesis activities, and is well suited for the training and evaluation of deep learning models, especially in managing the synthesis of new views in real-world scenarios. In addition, LLFF provides an efficient view synthesis algorithm that extends the traditional light field sampling theory by combining the multi-plane image (MPI) scene depiction with the local light scene.
Mip-NeRF Synthetic 360° Dataset (NeRF-360-V2 Dataset): This dataset is an extended synthetic dataset from Mip-NeRF designed to solve 3D reconstruction challenges in scenarios with endless possibilities. The dataset solves the difficulties of infinite scenarios by using nonlinear scene parameterization, real-time distillation, and innovative orientation distortion normalization techniques. Mip-NeRF 360 has the ability to create realistic artificial perspectives and complex depth maps for very complex, infinite real-life scenarios. In the dataset, there are 9 scenes, evenly spaced between indoor and outdoor environments, each with a complex main object or space, and a complex background.
NVS-RGBD Dataset: Includes a rough depth map of real-world scenes recorded by consumer-grade depth sensors. The goal of this dataset is to establish a new NeRF evaluation criterion to evaluate the effectiveness of creating new perspectives using a limited set of views. The NVS-RGBD dataset includes 8 scenes, with coarse depth maps collected from consumer-grade sensors such as Azure Kinect, ZED 2, and iPhone 13 Pro. Artifacts in these depth maps may differ from artifacts in sensor noise.
DONeRF Dataset: This dataset covers a variety of 3D scenarios, including bulldozers, woodland, educational spaces, San Miguel, kiosks, and hairdressers, among others. A range of developers have created these scenarios using Blender, providing a useful basis for studying neural radiance fields and resources, especially for instant rendering and interactive use.
B. Actual data set
Tank & Temple Dataset: This dataset includes standard sequences collected from outside the lab, providing high-definition video footage for both indoor and outdoor environments. Video sequences help create innovative pipelines that leverage video input to improve reconstruction accuracy. Industrial laser scanners are used to collect real-world data from datasets, covering scenarios in both indoor and outdoor environments. In addition, the dataset provides training and test datasets that divide the test data into intermediate and advanced categories to accommodate reconstruction activities of different complexity.
DTU Dataset: This dataset uses a multi-view stereo format with a tenfold increase in scenes and a significant increase in diversity compared to its predecessor. More precisely, it includes 80 scenes with a wide variety. Each scene consists of 49 or 64 accurate camera placements and structured light reference scans, producing RGB images of 1200×1600 pixels.
Euroc Dataset: This dataset covers both indoor and outdoor data and includes a variety of sensor information such as camera and IMU readings. The dataset is widely used in a variety of research areas, including robot vision, determining camera angles, calibrating cameras, and position and navigation. The main feature of this method is the ability to provide sensor data and the real indoor environment with high accuracy, and to evaluate the reconstruction and positioning accuracy of our method through grayscale images and tightly integrated IMU measurements.
Replica Dataset: This dataset represents a high-quality 3D reconstruction of an interior scene created by Facebook. The collection includes 18 detailed and realistic interior settings, each meticulously crafted and depicted to maintain visual realism. Each dataset scene contains a compact 3D mesh, detailed high dynamic range (HDR) textures, data for glass and specular surfaces, as well as semantic classification and instance segmentation.
BlendedMVS Dataset: This massive dataset is tailored for multi-view stereo matching (MVS) networks and provides a large number of training instances for algorithms based on learning MVS. The BlendedMVS collection contains more than 17,000 detailed images covering a variety of landscapes such as urban areas, structures, sculptures, and miniature objects. The breadth and diversity of this dataset makes it an important asset for MVS research.
Amazon Berkeley Object Dataset (ABO Dataset): This dataset is an extensive collection of 3D object understanding designed to bridge the realm of reality and virtual 3D. The dataset includes approximately 147,702 product listings, each associated with 398,212 different images in the catalog, and each product has up to 18 unique metadata characteristics, including category, color, material, weight, and size, among others. The ABO dataset includes artist-made 3D mesh representations of 8,222 items and artist-made representations of 7,953 products. This dataset is ideal for 3D reconstruction, material estimation, and multi-view object retrieval across domains, as these 3D models have complex geometrical designs and contain materials based on physical properties.
Common Objects in 3D Dataset (CO3Dv2 Dataset): This dataset includes 1.5 million multi-view image frames across 50 MS-COCO categories, providing rich image resources, precise camera positions, and 3D point cloud annotation. The breadth and diversity of CO3Dv2 makes it ideally suited to evaluate innovative view synthesis and 3D reconstruction techniques to advance 3D computer vision research.
3D-FRONT Dataset: This is a large-scale artificial indoor scene dataset jointly created by Alibaba Taobao Technology Department, Simon Fraser University, and the Institute of Computing Technology of the Chinese Academy of Sciences. The dataset provides well-designed room designs as well as a large number of style-compatible and high-quality 3D models. The 3D-FRONT facility features 18,797 rooms, each equipped with unique 3D elements, as well as 7,302 pieces of furniture with premium textures. The dataset features a wide range from layout semantics to complex textures for each object, and is designed to assist research in areas such as 3D scene understanding, SLAM, and reconstruction and segmentation of 3D scenes. In addition, the dataset includes Trescope, a simplified rendering tool used to facilitate the basic rendering of 2D images and their annotations.
SceneNet RGB-D Dataset: This dataset is a collection of 5 million real-world images of a synthetic indoor scene, with corresponding ground truth data. The scenarios in the dataset are randomly generated and contain 255 different categories, which are usually recombined into 13 categories, similar to the NYUv2 dataset. These synthetic scenes provide rich perspectives and lighting variations, making the dataset ideal for indoor scene understanding tasks such as semantic segmentation, instance segmentation, object detection, and geometric computer vision tasks such as optical flow, depth estimation, camera pose estimation, and 3D reconstruction.
C. Face datasets
CelebV-HQ Dataset: An extensive, high-quality, and diverse collection of videos with carefully labeled facial features, containing 35,666 clips at a minimum resolution of 512x512, covering 15,653 different identities. Each video clip is manually tagged with 83 different facial features, including appearance, movement, and emotion, which can be used in research areas such as facial recognition, expression research, and video understanding.
CelebAMask-HQ Dataset: The dataset is an extensive collection of high-definition facial images, comprising 30,000 images selected from the CelebA dataset. Each image is equipped with a 512 x 512 pixel split mask. By manually labeling these masks, the researchers obtained detailed data on facial areas, including 19 facial features such as skin, eyes, nose, mouth, etc.
VoxCeleb Dataset: This dataset is a large-scale speaker recognition dataset developed by researchers at the University of Oxford. It contains about 100,000 voice clips of 1,251 celebrities from YouTube videos. Designed to support research on speaker identification and validation, the VoxCeleb dataset provides a real, diverse, and large-scale data resource. The voice clips in the dataset cover different ages, genders, accents, and occupations, as well as a variety of different recording environments and background noise. VoxCeleb is divided into two subsets: VoxCeleb1 and VoxCeleb2. The audio sample rate of the dataset is 16kHz, 16bit, mono, PCM-WAV format.
Faces Labeled in the Field (LFW) Dataset: The dataset is publicly accessible and widely used in facial recognition research. It was compiled by the Computer Vision Lab at the University of Massachusetts Amherst and collected more than 13,000 images of faces from the internet. The images cover 1,680 different individuals, each with at least two images. The purpose of the LFW dataset is to improve the accuracy of face recognition under natural conditions, so it contains face images taken in a variety of different environments, such as different lighting, expressions, poses, and occlusions.
MPIIGaze Dataset: This dataset was collected by 15 users over several months of daily laptop use and contains 213,659 full-face images and their corresponding real-world gaze locations. Experienced sampling techniques ensure consistency in gaze and head position, as well as realistic changes in eye appearance and lighting. To facilitate cross-dataset evaluation, the corners of the eyes, mouths, and pupil centers of 37,667 images were manually annotated. The dataset stands out for its diversity of personal appearance, environment, and photographic equipment, as well as the extended period of data collection, providing an important asset for studying the broad applicability of gaze estimation techniques.
GazeCapture Dataset: This dataset is a large dataset for eye tracking technology, containing approximately 2.5 million images from more than 1,450 volunteers. Collected via mobile devices, the dataset is designed to aid eye tracking research and train relevant convolutional neural networks (CNNs) such as iTracker. Features of the GazeCapture dataset include scalability, trustworthiness, and variability, ensuring the diversity and quality of the data.
Flickr-Faces-HQ (FFHQ) Dataset: This collection of facial images is of high quality and includes 70,000 images in PNG format, each with a resolution of 1024*1024. The FFHQ encompasses a variety of age groups, ethnicities, and cultural heritage, as well as a variety of accessories such as eyeglasses, sunglasses, hats, and more, offering a wide range of diversity.
D. Human datasets
Thuman Dataset: This dataset represents an extensive public collection for 3D human reconstructions and contains approximately 7,000 data points. Each data item includes a surface mesh model with material, RGBD images, and the corresponding SMPL model. Mannequins containing a variety of poses and costumes, captured and reconstructed using DoubleFusion technology. The release of the dataset provides a valuable resource for research in the fields of 3D human modeling, virtual reality, augmented reality, and more.
HuMMan Dataset: The HuMMan dataset is a large-scale, multimodal 4D human body dataset containing 1,000 human subjects, 400,000 sequences, and 60 million frames of data. Features of the dataset include multimodal data and annotations (such as color images, point clouds, keypoints, SMPL parameters, and textured mesh models), a sensor suite that includes popular mobile devices, and a sensor suite designed to cover basic motion. 500 action sets that support a variety of tasks such as motion recognition, pose estimation, parametric human body repair, and textured mesh reconstruction. The HuMMan dataset is designed to support diverse perception and modeling studies, including challenges such as fine-grained motion recognition, dynamic human mesh sequence reconstruction, point cloud-based parametric human body estimation, and cross-device domain gaps.
H36M Dataset: The Human 3.6M dataset is a widely used 3D human pose estimation research dataset. The dataset includes approximately 3.6 million images showing 11 artists (6 men and 5 women) engaging in 15 standard activities in seven different contexts, such as walking, eating, and talking. At the same time, the data was recorded using 4 high-resolution cameras and a fast motion capture system, providing accurate information about the position and angle of the 3D joints. Each actor has a BMI ranging from 17 to 29, ensuring a variety of body types.
Multi-Garment Dataset: The dataset used to reconstruct 3D garments includes 356 images, each showing individuals with different body types, postures, and clothing styles. Derived from real scans, it provides 2078 reconstructed models based on real garments, covering 10 categories and 563 garment instances. Each garment in the dataset is richly annotated, including 3D feature lines (e.g., neckline, cuff profile, hem, etc.), 3D body pose, and corresponding multi-view real-world images.
MARS Dataset: This dataset is a comprehensive video-based Person Re-identification (ReID) compilation containing 1,261 unique pedestrians, captured by six cameras operating nearly simultaneously, each with at least two cameras. Features of the MARS dataset include variations in walking posture, clothing color, and lighting, as well as less-than-ideal image clarity, making it more challenging to identify. In addition, the dataset contains 3,248 jammers to simulate the complexity of real-world scenarios.
E. Other datasets
InterHand2.6M Dataset: This dataset is a large-scale gesture recognition dataset containing more than 2.6 million instances of gestures captured by 21 different people in a controlled environment. The dataset provides annotations for 21 gesture categories, including common gestures such as fist, palm outstretched, thumbs up, etc. Each gesture has multiple variations, such as different gesture poses, backgrounds, and lighting conditions. The InterHand2.6M dataset is designed to support the development and evaluation of gesture recognition algorithms, especially in complex scenarios and diverse gesture expressions.
TartanAir Dataset: This dataset was developed by Carnegie Mellon University to challenge and push the limits of visual SLAM technology. The dataset is generated in a highly realistic simulated environment, with a diverse range of lighting, weather conditions, and moving objects to simulate real-world complexity. TartanAir provides rich multi-modal sensor data, including RGB stereo images, depth images, segmentation labels, optical flow, and camera attitude information. This data helps researchers develop and test SLAM algorithms, especially when dealing with challenging scenarios.
SUN3D Dataset: This dataset contains a wide range of RGB-D videos showing scenes from a variety of venues and structures. The dataset includes 415 sequences across 254 different locations and 41 unique structural records, with each frame detailing the semantic division of objects in the scene and the position of the camera.
Since the advent of NeRF technology, it has driven technological advancements in various fields such as computer vision, virtual reality (VR), augmented reality (AR), and more. In addition, NeRF has demonstrated significant potential and application value in the fields of robotics, urban planning, autonomous navigation, and more.
As an emerging method of 3D scene representation, neural radiance field has attracted extensive attention in the field of computer vision and graphics. However, despite its remarkable achievements in rendering quality and detail, NeRF still faces a number of challenges that point in the direction of the future.
Discussion on computational efficiency
With the development of deep learning methods, it is expected that future research will focus on improving the computational efficiency of NeRF and similar technologies. Such research can explore innovative sampling methods, enhance network configurations, integrate existing geometric understanding, and create more efficient rendering algorithms. In the future, the pursuit of computing efficiency will focus on improving rendering speed and reducing NeRF resource usage.
Researchers may explore improved sampling and integration techniques to reduce the computational requirements for each image rendering. For example, NerfAcc integrates a variety of sampling techniques to achieve faster sampling speeds and lower rendering quality using a unified transmittance estimator. Instead, it is expected that further research will focus on improving network configurations, such as MIMO-NeRF [98], by applying multiple-input, multiple-output (MIMO), multi-layer perceptrons (MLPs), aiming to reduce the frequency of MLP operations during rendering, thereby improving the overall rendering speed. In addition, the integration of recent developments in deep learning, including Transformer architectures and unsupervised learning methods, may pave the way for efficiency gains in NeRF.
B. Discussion of Less View Rendering
Currently, the field of combining fewer and single views is rapidly expanding and becoming the focus of computer vision and graphics research. The advent of methods such as NeRF has enabled scientists to create excellent 3D images from a constrained set of viewpoints. Even with NeRF's impressive multi-view compositing capabilities, its effectiveness is limited due to insufficient training data, which can lead to overfitting and geometry reconstruction errors.
When data are scarce, contemporary research is exploring various regularization techniques to improve the quality of synthesis. For example, the generalization ability of the model can be improved by implementing studies such as geometric prior (GeoNeRF), using generative adversarial networks (GANs) (PixelNeRF), or augmented rendering methods (ViP-NeRF). While these approaches have made progress in reducing training time and improving rendering quality, they still face hurdles such as sparse views, managing occlusion, and restoring geometric detail. Subsequent research may focus on creating more efficient training methods, enhancing network structures to capture better scene details, and investigating unsupervised and self-supervised learning techniques to reduce reliance on large amounts of labeled data. In addition, hybrid approaches that combine physical simulation with scenario understanding may introduce new advances in areas such as virtual reality, augmented reality, and autonomous vehicles.
C. Discussion on Render Quality
With regard to rendering quality, contemporary research has focused on two main categories, namely high-resolution rendering and the generalization potential of models. Processing large amounts of data and computational tasks and maintaining complex details remains a significant obstacle when creating high-resolution, high-quality images (e.g., images over 4K) through model optimization. UHDNeRF and RefSR-NeRF improve their network structure to improve the detection accuracy of the model. However, UHDNeRF improves the detail efficiency of 4K UHD resolution by merging explicit and implicit scene descriptions to improve the rendering of the model, while RefSR-NeRF amplifies the high-frequency details of NeRF by merging high-resolution reference images into the creation of super-resolution views. In terms of its generalization capabilities, NeRF's proficiency in handling unfamiliar scenarios and data is limited and needs to be enhanced by improved network design and training methods. NeRFSR enhances the efficiency of the model for new views through oversampling and joint optimization techniques, while NeRF enhances the generalization ability of the model by incorporating an adaptive neural radiation field into a dynamic scene.
D. Discussion of imaging barriers
Regarding the enhancement of imaging barriers, the main focus of researchers is to solve the challenges of object processing with reflective and transparent properties. Whereas, NeRF often results in blurry or distorted images when interacting with objects with reflective or transparent properties. In response to this challenge, MS-NeRF and Ref-NeRF address this issue by addressing the multi-view consistency issue. MS-NeRF handles reflective and transparent elements by describing the scene as a feature field with multiple parallel regions, while Ref-NeRF produces more accurate rendering results by combining NeRF's ability to process reflective surfaces into a view-based structured and parametric reflection representation. Further research and methods may be needed to improve the efficiency of NeRF in order to address a wider range of rendering problems in complex lighting scenes, including dynamic range lighting, shadows, and overall lighting effects. Subsequent research should explore techniques for combining accurate physical lighting models with NeRF and create new datasets and evaluation criteria to evaluate and confirm the effectiveness of these methods in complex lighting scenarios.
E. Discussion of Application Scenarios
Regarding practical applications, recent research has focused on interactive rendering, making portraits and faces, and realistic reconstruction of scenes, as described below:
1) Interactive rendering technology: The current research on interactive rendering methods focuses on improving rendering efficiency, enriching the user's editing process, and expanding the scope of multimodal interaction features. However, there are still some obstacles and limitations in these areas. There is still a need to improve the intuitiveness and adaptability of the user's editing interface to enable the average user to perform proficient editing tasks without complex training. When it comes to multimodal interactions, it is essential to improve the integration of various inputs such as text, images, and audio for a more intuitive and natural editing process. In addition, the current approach still has difficulties in terms of broad applicability, which may reduce the flexibility of the model and the quality of editing for unfamiliar scenes and objects. Future research could explore these avenues to address these issues. Initially, real-time rendering and efficiency improvements were achieved through optimization algorithms like NerfAcc and the adoption of more efficient hardware acceleration methods, including GPUs and TPUs. In addition, improving the user interface design to improve intuitiveness and ease of use can reduce the difficulty of user editing, thereby improving the accuracy and satisfaction of editing, as achieved with ICE-NeRF and NaviNeRF. It is feasible to enhance the multimodal fusion characteristics of the model to make it more effective in understanding and responding to various inputs. Ultimately, in order to enhance the generalization capabilities of the model and maintain excellent rendering and editing in a variety of applications, it may be necessary to build datasets in different domains, implement meta-learning methods, and innovate the regularization techniques of the model. Through these efforts, future interactive rendering technologies will be able to better meet user needs and provide more powerful and flexible tools for a variety of applications.
2) Portrait reconstruction: Facial synthesis technology has great potential in the future, especially to improve the sense of reality and user interaction experience. The advent of technologies such as FaceCLIPNeRF highlights the ability to accurately process 3D facial expressions and features based on textual descriptions. This method not only retrieves data from still pictures, but also retains consistency from different angles, paving the way for producing customized media content. Instead, the NeRFInvertor approach showcases advanced animation of creating a real identity from a single image, offering great potential for the use of games, movies, and virtual reality. In addition, the creation of GazeNeRF demonstrates the ability to leverage 3D perception methods to alter facial attributes, such as eye position, to improve the interactivity and realism of virtual characters. Finally, the RODIN framework presents innovative opportunities for generating and modifying digital avatars through a 3D diffusion network, enhancing the efficiency of custom and high-precision 3D character production. These technological advancements herald the future of facial synthesis technology, with a focus on real-time processing, diversity, and customization for the user, but also introduced new challenges regarding privacy and ethical considerations.
3) Human Rendering: Currently, the field of human body rendering is experiencing a double growth, covering technological advancements and the expansion of the scope of applications. From a technical point of view, new findings such as TransHuman and GM-NeRF demonstrate a superior new view synthesis framework by educating conditional NeRF using multi-perspective video in the context of limited data. Not only do these methods improve the immediacy and broad applicability of rendering, but they also provide powerful technical support for applications such as virtual reality (VR) and augmented reality (AR). In addition, methods like PersonNeRF allow visualizations to be customized from different angles, poses, and appearances by creating personalized 3D models using a set of individual photos, providing a novel approach to personalization for social media, digital entertainment, and e-commerce.
Secondly, regarding the expansion of the scope of applications, advances in human rendering technology are triggering changes in various fields. For example, the SAILOR framework not only provides superior rendering effects, but also gives users the freedom to edit and create, providing content creators with more creative space and the ability to produce more diverse and detailed visual content. In addition, with advances in data compression and transmission technologies, it is expected that future human rendering will facilitate efficient data transfer in environments with limited network bandwidth, ensuring the smooth running of more advanced VR and AR experiences on mobile devices. This advancement demonstrates the growing importance of human rendering technology in delivering engaging experiences and customized content, introducing new application areas such as entertainment, education, and healthcare.
Despite the many obstacles faced by the NeRF space, it has great growth prospects. As technology continues to advance, NeRF will continue to grow in importance in shaping the future of 3D scene modeling and rendering.
After Mildenhall et al. proposed the NeRF framework. This model has significantly improved aspects of processing speed, output integrity, and training data requirements in its groundbreaking research, thus surpassing many of the limitations of its original form. The success of the NeRF method is attributed to its ability to reconstruct continuous 3D landscapes from a limited perspective and produce high-quality images from different perspectives. The advent of this technology has brought new aspects to the field of computer vision. This innovation paves the way for innovative approaches in opinion synthesis, 3D reconstruction, and neural rendering in computer vision, and NeRF technology has shown great potential in various fields such as style transfer, image editing, avatar development, and 3D urban environment modeling. With the increasing interest of NeRF modeling in academia and industry, a large number of researchers have invested a lot of research resources, which has led to the publication of various preprints and academic works. This paper systematically examines the latest advances in technology and practical applications of NeRF technology, providing a comprehensive review and perspective on its future path and challenges. The focus of this paper is to motivate scholars in the field, with the aim of promoting continuous progress and innovation in NeRF-related technologies.
Readers who are interested in more experimental results and details of the article can read the original paper
This article is only for academic sharing, if there is any infringement, please contact to delete the article.
3D Vision Workshop Exchange Group
At present, we have established multiple communities in the direction of 3D vision, including 2D computer vision, large models, industrial 3D vision, SLAM, autonomous driving, 3D reconstruction, drones, etc., and the subdivisions include:
2D Computer Vision: Image Classification/Segmentation, Target/Detection, Medical Imaging, GAN, OCR, 2D Defect Detection, Remote Sensing Mapping, Super-Resolution, Face Detection, Behavior Recognition, Model Quantification Pruning, Transfer Learning, Human Pose Estimation, etc
Large models: NLP, CV, ASR, generative adversarial models, reinforcement learning models, dialogue models, etc
Industrial 3D vision: camera calibration, stereo matching, 3D point cloud, structured light, robotic arm grasping, defect detection, 6D pose estimation, phase deflection, Halcon, photogrammetry, array camera, photometric stereo vision, etc.
SLAM: visual SLAM, laser SLAM, semantic SLAM, filtering algorithm, multi-sensor fusion, multi-sensor calibration, dynamic SLAM, MOT SLAM, NeRF SLAM, robot navigation, etc.
Autonomous driving: depth estimation, Transformer, millimeter-wave, lidar, visual camera sensors, multi-sensor calibration, multi-sensor fusion, autonomous driving integrated group, etc., 3D object detection, path planning, trajectory prediction, 3D point cloud segmentation, model deployment, lane line detection, Occupancy, target tracking, etc.
3D reconstruction: 3DGS, NeRF, multi-view geometry, OpenMVS, MVSNet, colmap, texture mapping, etc
Unmanned aerial vehicles: quadrotor modeling, unmanned aerial vehicle flight control, etc
In addition to these, there are also exchange groups such as job search, hardware selection, visual product landing, the latest papers, the latest 3D vision products, and 3D vision industry news
Add a small assistant: dddvision, note: research direction + school/company + nickname (such as 3D point cloud + Tsinghua + Little Strawberry), pull you into the group.
3D Vision Workshop Knowledge Planet
3DGS, NeRF, Structured Light, Phase Deflection, Robotic Arm Grabbing, Point Cloud Practice, Open3D, Defect Detection, BEV Perception, Occupancy, Transformer, Model Deployment, 3D Object Detection, Depth Estimation, Multi-Sensor Calibration, Planning and Control, UAV Simulation, 3D Vision C++, 3D Vision python, dToF, Camera Calibration, ROS2, Robot Control Planning, LeGo-LAOM, Multimodal fusion SLAM, LOAM-SLAM, indoor and outdoor SLAM, VINS-Fusion, ORB-SLAM3, MVSNet 3D reconstruction, colmap, linear and surface structured light, hardware structured light scanners, drones, etc.