OpenMVG: Open-Source Photogrammetry & Vision Library
- Anvita Shrivastava
- 1 day ago
- 6 min read
Photogrammetry has emerged as a vital tool for the generation of 3D models, point clouds, camera poses, and geospatial data from overlapping photographs. From drones to 3D reconstructions and computer vision, all forms of photogrammetry processes rely on the availability of software packages that can extract geometry from photographs. OpenMVG (Open Multiple View Geometry) is an open-source computer vision library that aims at providing the fundamental algorithms for Multiple View Geometry (MVG) and image-based 3D reconstruction.
Whereas commercial photogrammetric tools offer a full-blown environment for the end-to-end workflow, OpenMVG offers algorithms that help us understand relationships between multiple photographs.
What Is OpenMVG?
OpenMVG is an open-source software library dealing with Multiple View Geometry, which is a subfield in computer vision that aims to recover 3D information from multiple 2D images.
As different images are taken of the same object/landscapes from different perspectives, there are visual characteristics that are common across all the images. OpenMVG utilizes the correspondences to estimate the cameras’ parameters, calculate their poses and orientations, as well as the relations between the visual features.
Some of the steps in the process of using OpenMVG may consist of:
Image Feature Extraction
Feature Description & Matching
Image Pair Selection
Camera Intrinsic Estimation
Relative Pose Estimation
Structure From Motion
Camera Pose Estimation
Sparse 3D Reconstruction
Outputting of Reconstruction Results
All these abilities serve as a basis for many modern photogrammetric and 3D Computer Vision pipelines.
How OpenMVG Works
Photogrammetry reconstruction involves a set of overlapping images.
Image Feature Detection
OpenMVG starts by detecting features in the input photographs. The features refer to points in the images that can be detected in varying viewpoints: corners, edges, textures, or other visually distinctive points.
Image feature detection is necessary because the program has to rely on identifiable points that can be matched in the images.
Feature Description
Once the features have been detected, the program then creates descriptors of the features to be able to recognize whether a feature in one image matches that of the other.
Reliable descriptors are especially necessary when images show changes in viewpoints, scaling, and illumination, among others.
Feature Matching
Feature matching by OpenMVG is the stage at which the common features among images are detected.
This means that if there is a building corner that is seen in some images taken from overlapping drone shots, it can be determined as a common feature. This step is very important because of the geometric information that will be used later in the process of reconstruction.
Sometimes the quality of feature matching can significantly affect the result. Problems such as lack of overlap, repetitive textures, motion blur, vegetation movements, and illumination differences can cause some difficulties in the process of matching.
Camera Pose Estimation
After the number of correspondences becomes sufficient, OpenMVG starts estimating the relative pose of cameras.
The camera pose estimation is also called structure from motion (SfM). The process of SfM consists of estimating camera poses and reconstructing the scene from the set of overlapping photographs.
After that, the parameters of the cameras become clear.
Sparse 3D Reconstruction
OpenMVG can perform 3D reconstruction by using matched observations in images along with the geometry of cameras.
A sparse point cloud will be generated from this 3D reconstruction process that will comprise all those feature points that correspond to recognizable structures in the scene but might not provide dense surface data for mapping.
OpenMVG can be used along with other tools for dense reconstruction of scenes.
OpenMVG and Structure from Motion
One of the main applications of OpenMVG is Structure from Motion.
Structure from Motion is a computer vision algorithm that estimates:
The location of cameras
The orientation of cameras
The intrinsic parameters of the cameras
The locations of 3D features
The relations between several pictures
Imagine a drone that is taking hundreds of images over a construction site. All of those pictures are a two-dimensional projection of the environment. By locating the same physical features on several images, SfM is able to estimate the 3D structure of the scene and the camera motion.
OpenMVG is thus especially relevant as an automatic 3D reconstruction and photogrammetry pipeline for drones.
OpenMVG in Drone Photogrammetry
Standard practices for drone photogrammetry generally include taking hundreds or even thousands of overlapping aerial images to create geospatial data from them.
In this regard, OpenMVG can aid in some of the initial steps of such a pipeline by determining relationships between images and extracting camera parameters.
One possible drone photogrammetry pipeline is as follows:
Drone Images -> Feature Detection -> Feature Matching -> SfM -> Camera Poses -> Sparse Point Cloud -> Dense Reconstruction -> Orthomosaic/3D Model
It should be noted that OpenMVG focuses on computer vision and SfM tasks, and not on being an application for commercial drone mapping.
Additional processes needed for professional surveying may include:
Dense point cloud creation
Ground Control Points integration
Coordinate Reference System transformation.
Digital Elevation Model creation
Orthomosaic creation
Accuracy assessment
OpenMVG vs. OpenMVS
OpenMVG and OpenMVS (Open Multi-View Stereo) are typically used in tandem, yet they handle different processes in the workflow of photogrammetry.
OpenMVG is mostly concerned with camera reconstruction and Structure from Motion. OpenMVS deals with creating a dense geometry from the scene reconstructed by OpenMVG.
The common sequence of the workflow:
Images → OpenMVG → Camera Poses & Sparse Reconstruction → OpenMVS → Dense Point Cloud / Mesh / Surface
Such a combination forms an open-source pipeline for 3D reconstruction using images without having to rely exclusively on commercial photogrammetry software.
OpenMVG for GIS and Mapping
OpenMVG may also be applied to geospatial workflows, especially when there is a need for image-based reconstruction before GIS analysis.
Possible areas of application may include:
Drone surveying
3D terrain modeling
Heritage documentation
Construction surveys
Infrastructure inspection
Robotics
Autonomous navigation
Computer vision research
Object modeling
Indoor surveys
Creation of digital twins
Still, one should not regard OpenMVG automatically as an alternative to GIS or surveying software packages. Geospatial workflow requires further processing of the data obtained through reconstruction into a known coordinate system and verifying its positional accuracy.
OpenMVG and Georeferencing
One of the distinguishing features of photogrammetry is the difference between reconstruction and georeferenced reconstruction.
A typical SfM process would be able to produce a relative reconstruction of a scene; however, the output would not necessarily be tied to the real-world coordinate system.
In the case of applications involving mapping, georeferencing could include the following:
GNSS positions of cameras
RTK/PPK positions
checkpoints
Coordinate reference systems
Calibration of cameras
Having accurate GNSS data for an aerial platform could allow establishing the relationship between the reconstructed model and the real-world coordinate system.
Survey-grade applications would still require independent checkpoints.
OpenMVG Features
Instead of a dedicated photogrammetry software tool, OpenMVG is a modular package that includes such photogrammetry-related functionality as image matching, camera calibration, pose computation, and Structure from Motion.
There are some technical benefits of the open-source code:
Open Source
The software is accessible for modification and incorporation in one's own applications and research projects.
Modularity
It allows using different modules in a customized workflow without the need to use a single monolithic software package.
Computer Vision
Algorithms related to multi-view reconstruction and camera geometry are provided by OpenMVG. It is good for experimentation and research purposes.
Automation
OpenMVG could be incorporated into the automated pipeline for processing large sets of images.
Integration
It could be integrated into other software tools dealing with computer vision, photogrammetry, point clouds, and geospatial analysis.
Advantages of OpenMVG
There are several pros of using OpenMVG for technical photogrammetry and computer vision purposes.
Modularity: It enables the creation of flexible custom workflows by developers.
Open source code: One can study and change the underlying implementation if needed.
Good for research: This library is great for playing around with multi-view geometry.
Automation: OpenMVG can be used as a component of scripted pipelines.
Interoperability: The results of reconstructions can be further processed by other photogrammetry, computer vision, and geospatial products.
Limitations to Consider
OpenMVG is a great tool, but one needs to know its limitations.
One might want to look for another software product to replace the existing mapping solution entirely. OpenMVG does not provide all functionalities that can be found in dedicated platforms, such as dense reconstruction, orthomosaics generation, terrain modeling, georeferencing, editing, visualization, and GIS analysis.
The success of the reconstructions largely depends on image quality and conditions of capture. Lack of image overlap, motion blur, lack of texture, repetitive surfaces, bad camera calibration, and uneven exposures can negatively affect the results.
For surveys with high accuracy requirements, a photogrammetric reconstruction also requires some GNSS/RTK/PPK data, GCPs (when applicable), and an independent accuracy check.
OpenMVG is a very significant library for the open-source community when dealing with photogrammetry, Structure from Motion, and Multi-View Geometry. As opposed to attempting to provide all functionality available in commercial photogrammetry software, it is centered around addressing the basic computer vision problems necessary for camera geometry and 3D structure recovery from images.
OpenMVG can serve as a basis for several custom applications that utilize photogrammetry, such as drone mapping, 3D reconstruction, robotics, and research projects. Combined with other software libraries capable of performing a dense reconstruction, such as OpenMVS, it becomes a key component in an entire pipeline of an open-source photogrammetry project.
In case an organization is developing automated geospatial systems, the most significant aspect about OpenMVG is its openness, modularity, and focus on computer vision techniques.
To learn more about OpenMVG and its geospatial capabilities, click here.
For more information or any questions regarding the LizardTech suite of products, please don't hesitate to contact us at:
Email: info@geowgs84.com
USA (HQ): (720) 702–4849
(A GeoWGS84 Corp Company)
Comments