COLMAP Explained: Building 3D Models from Images
- Anvita Shrivastava
- 6 minutes ago
- 7 min read
The ability to produce reliable 3D models from common photos is important in areas like photogrammetry, computer vision, robotics, etc. Rather than creating the model manually, modern photogrammetry applications can process many overlapping photos and automatically reconstruct the scene geometry.
COLMAP is one of the most popular open-source software tools used for this type of reconstruction work. It offers an excellent pipeline that turns images into camera poses, sparse and dense point clouds, and high-quality 3D models.
What Is COLMAP?
COLMAP is an open-source pipeline for SfM and MVS, which allows users to create 3D models based on images.
The system detects shared features between several photos, calculates the pose of each camera, and reconstructs the 3D geometry of the scene. It also allows for the creation of dense point clouds and surface models.
Some applications for COLMAP include:
3D Reconstruction
Digitization of objects
Processing of drone imagery
Documentation of cultural heritage
Research in computer vision
Robotics and autonomy
Applications of AR and VR
Camera pose estimation
Creation of point clouds
The key benefit of COLMAP is its ability to create 3D models from regular RGB images without the use of special 3D scanners.
How Does COLMAP Build a 3D Model?
COLMAP makes use of a multi-stage approach to turn a two-dimensional image into a three-dimensional format.
Stages involved in the process include:
Images → Feature Extraction → Feature Matching → Camera Reconstruction → Sparse Point Cloud → Dense Reconstruction → 3D Model
Each of these stages plays an essential role in the process of creating a three-dimensional model.
Step 1: Take Overlapping Photographs
Reconstruction requires a series of photographs taken of the same object/environment from various angles.
To achieve accurate reconstruction, each photograph must overlap with others in the sense that it contains all features.
When photographing an object, the following aspects come in handy:
Take images from various perspectives.
Consistent lighting.
Prevent any motion blur.
Image sharpness.
Overlapping images.
Avoid large parts with no texture.
It works the same way in drone photogrammetry. The aerial photographs have overlapping areas with common features for reconstruction purposes.
The geometry of images used in the process influences the outcome significantly.
Step 2: Feature Extraction
The second step involves locating unique visual features in each image.
COLMAP finds keypoints that could be identified from multiple photos. They might include:
Corners
Edges
Texture
Details of the building
Surface markings
Anything that is visually distinctive
These features are described in mathematical terms through their descriptors.
With a descriptor, COLMAP is able to match a feature in one photo with features identified in other photos.
A distinctive corner of a building present in five photos would be identified as the same point in all five photos.
Feature extraction forms the basis of image matching and 3D reconstruction.
Step 3: Feature Matching
After feature extraction, COLMAP looks for feature correspondences among images.
This is known as feature matching.
In case the same real-world location is found in different photos, COLMAP tries to match these locations together.
For instance:
Image 1: Building Corner A
Image 2: Building Corner A
Image 3: Building Corner A
With enough matches found, the software may start figuring out the camera positions used to take each photo.
COLMAP offers several types of matching methods based on the type of the image set and requirements.
Among these:
Exhaustive matching
Sequential matching
Spatial matching
Vocabulary tree matching
Custom matching schemes
The matching method will depend on the number and connections between the images.
Step 4: Structure-from-Motion Reconstruction
Structure-from-Motion is one of the most crucial stages in the COLMAP pipeline.
At this stage, COLMAP will estimate:
Position of cameras
Orientation of cameras
Camera parameters
3D position of feature matches
The output is a sparse point cloud.
Sparse point clouds comprise thousands or even millions of reconstructed 3D points which represent distinct features within the scene.
Even though the reconstruction might seem unfinished at this stage, it still has everything necessary for further reconstruction.
SFM solves two critical problems at once:
Where did each camera stand?
What is the position of observed points in 3D space?
This stage of reconstruction is known as bundle adjustment.
Camera Pose Estimation in COLMAP
A camera pose is the position and orientation of the camera during an image acquisition.
For each successful reconstruction of an image, COLMAP calculates parameters including but not limited to:
Camera location
Rotation
Orientation
Intrinsic camera parameters
Camera lens
Camera pose estimation is critical because 3D reconstruction requires understanding of the geometric relation between two images.
Knowing the camera poses, rays can be cast from various camera poses towards feature matches.
Intersecting viewing rays enable COLMAP to determine the 3D location of a point.
It is called triangulation.
Step 5: Sparse Point Cloud Construction
Structure from motion produces a sparse reconstruction.
A sparse point cloud is not a representation of all surfaces of an object.
Instead, it is a set of reliable 3D points constructed based on matched image features.
A sparse point cloud may show:
General object shape
Building geometry
Terrain structure
Camera locations
Scene orientation
Though sparse models are helpful in visualizations and camera reconstructions, they may lack necessary details to produce realistic 3D surfaces.
That is why Multi-View Stereo is needed.
Step 6: Dense Reconstruction using Multi-View Stereo
Following the sparse model creation, COLMAP is capable of conducting dense reconstruction using Multi-View Stereo.
Multi-View Stereo uses the images and known camera positions to compute the depth information for many more pixels.
In the process, dense geometry information is computed for all visible surfaces.
The output is a dense point cloud consisting of far more points than the initial sparse reconstruction.
Dense reconstruction may consist of:
Building surfaces
Terrain
Statues and other objects
Infrastructure
Architectural elements
Complex environmental elements
The success of the dense point cloud will depend on various parameters such as image resolution, overlap, texture, camera geometry, and processing parameters.
From Point Clouds to 3D Meshes
A point cloud is a set of distinct 3D coordinates.
For modeling a continuous 3D surface, the point cloud may be transformed into a mesh.
The mesh includes:
Vertices
Edges
Faces
This creates a continuous model of the object or scene reconstruction.
With mesh generation, one can make 3D models applicable for:
Visualization
Animation
CAD projects
Digital twins
Virtual reality environments
3D printing
Game creation
Cultural heritage conservation
It is also possible to texture the mesh by applying photographs.
COLMAP and Photogrammetry
The two are closely related since both utilize overlapping images to reconstruct real-life geometry.
Various similarities between photogrammetry and computer vision include:
Camera calibration
Image orientation
Feature matching
Triangulation
Bundle adjustment
Point cloud generation
Surface reconstruction from dense data
One of the most useful qualities of COLMAP is the ability to give the user more control in the process of reconstruction.
In drone photogrammetry tasks, one could apply the same concept to overlapping images taken by drones to reconstruct:
Buildings
Construction sites
Terrains
Infrastructure
Mines
Historical sites
Industrial complexes
Unfortunately, traditional COLMAP reconstruction cannot cover the full geospatial process of photogrammetric mapping in surveying operations.
Other data such as GPS coordinates, RTK/PPK measurements, GCPs, CRS, and georeferencing processes may be necessary in cases where real-world positioning is important.
Advantages of Using COLMAP
There are various benefits that come with using COLMAP for 3D reconstruction and computer vision processes.
Open Source Software
COLMAP is open source software, which means it is freely available for researchers and developers, among other users.
Quality Reconstructions
It supports structure from motion and multi-view stereo to generate quality reconstructions from appropriate image datasets.
Flexible Process
It allows the user to configure the software according to their needs, be it experimental or any other purpose like robotics or any 3D reconstructions.
GPU Accelerated Processes
Some of the processes supported by the software can be accelerated using GPUs, especially when working with large image datasets.
Popular Computer Vision Tool
COLMAP is one of the most popular software used in computer vision research due to its capability in camera reconstruction, feature matching, and 3D modeling.
Challenges and Limitations of COLMAP
However, despite all of its abilities, COLMAP might not be the most convenient option in every case of 3D modeling.
Processing Time
The large number of images can make the reconstruction process take some time, especially feature matching and dense reconstruction.
Hardware Requirements
Dense reconstruction can be computationally intensive and requires an efficient GPU and enough RAM.
Dependence on Image Quality
Images of poor quality can negatively impact the accuracy of reconstruction.
The issues may include:
Motion blur
Inadequate overlap
Textureless scenes
Reflections
Repeated textures
Dynamic Lighting Conditions
Difficulties With Dynamic Objects
Any moving vehicles, humans, trees, or any other dynamic object can lead to incorrect results, as the reconstruction is based on the assumption that the scenery doesn't change much.
Scale and Georeferencing
COLMAP's model may lack real-world scale or georeferencing information.
Additional data about location may be required for surveying and GIS projects.
COLMAP and Modern 3D Reconstruction
Demand for photogrammetric 3D reconstruction keeps growing due to rapid development of drones, smartphones, cameras, and computer vision.
In modern pipeline processes, the integration of photogrammetry with:
LiDAR
Artificial intelligence
Neural rendering
Gaussian splatting
NeRFs
Digital twins
can be observed.
At the same time, COLMAP can contribute to these pipelines by providing camera poses and either sparse or dense geometric reconstruction of the object.
Moreover, in some modern pipelines of computer vision, COLMAP acts as a pre-processing module or camera reconstruction module before the data goes through neural rendering or a machine learning pipeline.
Thus, COLMAP becomes especially interesting for modern researchers who develop new approaches to 3D visualization and reconstruction.
COLMAP is an open-source solution that is used to generate 3D reconstructions using normal images. The software combines methods such as feature extraction, feature matching, Structure-from-Motion, triangulation, bundle adjustment, and Multi-View Stereo in order to create a three-dimensional reconstruction based on overlapping photos.
The standard process of using COLMAP starts from a set of images and proceeds through estimating the camera poses, sparse reconstruction, point cloud creation, and surface modeling.
COLMAP can be used as the base for obtaining three-dimensional information from two-dimensional images in applications like photogrammetry, drone mapping, computer vision, robotics, and 3D digitization.
COLMAP is one of the key solutions for learning how 3D reconstruction works alongside the development of AI, neural rendering, and digital twins.
To learn more about COLMAP and its geospatial capabilities, click here.
For more information or any questions regarding the LizardTech suite of products, please don't hesitate to contact us at:
Email: info@geowgs84.com
USA (HQ): (720) 702–4849
(A GeoWGS84 Corp Company)
Comments