Current studentsExisting staffContact us
 

CBARF: Cascaded Bundle-Adjusting Neural Radiance Fields From Imperfect Camera Poses

Simple item page
dc.contributor.authorFu, H.
dc.contributor.authorYu, X.
dc.contributor.authorLi, L.
dc.contributor.authorZhang, L.
dc.date.issued2024
dc.description.abstractExisting volumetric neural rendering techniques, such as Neural Radiance Fields (NeRF), face limitations in synthesizing high-quality novel views when the camera poses of input images are imperfect. To address this issue, we propose a novel 3D reconstruction framework that enables simultaneous optimization of camera poses, dubbed CBARF (Cascaded Bundle-Adjusting NeRF). In a nutshell, our framework optimizes camera poses in a coarse-to-fine manner and then reconstructs scenes based on the rectified poses. It is observed that the initialization of camera poses has a significant impact on the performance of bundle-adjustment (BA). Therefore, we cascade multiple BA modules at different scales to progressively improve the camera poses. Meanwhile, we develop a neighbor-replacement strategy to further optimize the results of BA in each stage. In this step, we introduce a novel criterion to effectively identify poorly estimated camera poses. Then we replace them with the poses of neighboring cameras, thus further eliminating the impact of inaccurate camera poses. Once camera poses have been optimized, we employ a density voxel grid to generate high-quality 3D reconstructed scenes and images in novel views. Experimental results demonstrate that our CBARF model achieves state-of-the-art performance in both pose optimization and novel view synthesis, especially in the existence of large camera pose noise.
dc.description.statementofresponsibilityHongyu Fu, Xin Yu, Lincheng Li, Li Zhang
dc.identifier.citationIEEE Transactions on Multimedia, 2024; 26:9304-9315
dc.identifier.doi10.1109/TMM.2024.3388929
dc.identifier.issn1520-9210
dc.identifier.issn1941-0077
dc.identifier.orcidYu, X. [0000-0001-9890-5489] [0000-0002-0269-5649] [0000-0002-3388-9606] [0000-0002-6265-9519]
dc.identifier.urihttps://hdl.handle.net/2440/148645
dc.language.isoen
dc.publisherInstitute of Electrical and Electronics Engineers
dc.relation.granthttp://purl.org/au-research/grants/arc/DP220100800
dc.relation.granthttp://purl.org/au-research/grants/arc/DE230100477
dc.rights© 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
dc.source.urihttps://doi.org/10.1109/tmm.2024.3388929
dc.subject3D Reconstruction; novel view synthesis; neural radiance fields; bundle-adjustment; camera pose registration
dc.titleCBARF: Cascaded Bundle-Adjusting Neural Radiance Fields From Imperfect Camera Poses
dc.typeJournal article
pubs.publication-statusPublished

Files

Collections

Research Outputs