vMAP

Vectorised Object Mapping for Neural Field SLAM

CVPR 2023


Xin Kong    Shikun Liu    Marwan Taher   Andrew Davison  
Dyson Robotics Lab   Imperial College London
Paper Video Poster Code


vMAP can map each object into a separate MLP and then recompose them.


Video


Abstract

TL;DR: We present vMAP, an object-level real-time mapping system, with each object represented by a separate MLP neural field model, and object models are optimised in parallel via vectorised training.

We present vMAP, an object-level dense SLAM system using neural field representations. Each object is represented by a small MLP, enabling efficient, watertight object modelling without the need for 3D priors. As an RGB-D camera browses a scene with no prior information, vMAP detects object instances on-the-fly, and dynamically adds them to its map. Specifically, thanks to the power of vectorised training, vMAP can optimise as 50 individual objects in a single scene, with an extremely efficient training speed of 5Hz map update. We experimentally demonstrate significantly improved scene-level and object-level reconstruction quality compared to prior neural field SLAM systems.

Method


vMAP takes an RGB-D image stream as input, detects objects on-the-fly, and dynamically adds them to its map. Each object is represented by a tiny MLP and the 3D bound is continually updated via the data association across frames. vMAP achieves highly efficient vectorised training by stacking the batch of MLPs together and optimised in a single training step. Thanks to the disentangled object-level mapping design, each object model can be stopped and resumed without any inter-object interference, and can be further recomposed into new 3D scene.

Results

Replica Dataset


3D Reconstruction

Visualisation of Scene Reconstructed by Method at Object-level.


Scene Recomposition

Background




2D Novel View Synthesis

Visualisation of Scene from a Novel View.





ScanNet Dataset

Visualisation at Scene-level.

vMAP

NICE-SLAM



Visualisation of Object .


vMAP

objSDF

TSDF




Live Table Sequence

Visualisation of Method .




BibTeX

@inproceedings{kong2023vmap,
  title={vmap: Vectorised object mapping for neural field slam},
  author={Kong, Xin and Liu, Shikun and Taher, Marwan and Davison, Andrew J},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={952--961},
  year={2023}
}