Common LiDAR Compression Mistakes (and How to Avoid Them)
- Anvita Shrivastava
- 1 day ago
- 4 min read
Updated: 3 hours ago
The volume of LiDAR data is increasing rapidly. Through larger pulse densities, more extensive coverage, and more regular data capture, organisations are now handling Terabyte (TB) or even Petabyte (PB) point cloud datasets. Thus, compression is not an option any longer; it is required. Unfortunately, improper compression of LiDAR Data can often lead to performance bottlenecks, interoperability problems, and even permanent loss of data.
In the following article, we shall examine the most common types of compression mistakes made with LiDAR data; as well as offer practical information about how to avoid making those same mistakes, thus allowing you to lower your data storage costs; without compromising on the integrity or usability of your data. We will also discuss how newer generation LiDAR compression formats such as MrSID®, have been specifically designed to overcome many of these challenges.
Mistake 1: Treating LiDAR Like Generic Binary Data
Using generic file compression formats such as ZIP, GZIP, or typically used file system compression on large LiDAR datasets is a common mistake. These generic compression formats might provide some minimal reduction in original file sizes but do not realize the organization of LiDAR point cloud data.
The following are the issues with using generic file formats for compressing LiDAR:
Limited compression ratios
Lack of knowledge regarding spatial association of point data
Requires complete file un-compression before access (slower access)
To avoid these problems, one needs to use purpose-built, compression formats specifically designed to compress LiDAR point cloud datasets. Purpose-built solutions utilize attributes of each point, spatial coherence, and the hierarchical structure of your eye's data compared to other points found within the same data set so that greater compression ratios will be realized as well as faster access to the uncompressed data.
Mistake 2: Sacrificing Accuracy Without Understanding the Impact
Lossy compression is a highly effective way to shrink the size of files, but the risks of using it indiscriminately in workflows (like engineering, surveying and regulation) can be quite severe.
The major problems with this practice include:
Precision issues resulting from subtle inaccuracies may compound in downstream processes
Some regulatory and contractual obligations necessitate having the data maintained at full Precision
If an original file becomes irretrievable due to reprocessing data, that information may never be available again
What can you do to avoid these types of issues? You will want to utilize a solution that offers both lossless and controlled lossy compression options, enabling you to find a balance between preserving precision and maximizing storage efficiency. Formats such as MrSID LiDAR provide the ability to utilize lossy compression while maintaining an acceptable level of accuracy within established project tolerances.
Mistake 3: Ignoring Random Access and Streaming Performance
A number of compression techniques are developed specifically to decrease the size of a file, but they do not consider how data will be accessed.
What is the reason that this is a problem?
You will have to decompress complete datasets to see or analyze only a small portion of the entire dataset.
Web-based and cloud workflows work slowly.
There are many added processing hours to perform analytic and visual display types of computations.
How to avoid it: Look for LiDAR compression formats that support random access and streaming, enabling users to work with only the points they need. MrSID’s multi-resolution architecture allows applications to quickly access subsets of data without decompressing the entire file.
Mistake 4: Locking Data into Poorly Supported Formats
Having limited compatibility of a compression format may negatively affect the viability of your LiDAR data over time.
Why is this an issue:
Reduced ability to share data between GIS, CAD, and other analytical tools
Complicated ability to share your data with partners/clients
Potential for the format to become obsolete
How to prevent it: Use compression methods that have been proven over time, and that have a large amount of associated developers/integration. MrSID has been utilized successfully for many years within production geospatial workflows, and it integrates well with many of the premier LiDAR and GIS applications.
Mistake 5: Overlooking Metadata and Attributes
The key to understanding LiDAR datasets goes beyond just the XYZ coordinates; this includes classifications, intensities, return numbers, scan angles, and timestamps, and all have unique meaning.
Why this is a concern:
If attributes are lost or corrupted, the ability to perform analysis will be impaired
Inconsistency in metadata provides added difficulty when trying to automate processes or conduct quality control
Reclassification or extracting features from the dataset will therefore become less reliable
How to solve this: use a compression solution that retains point attributes and metadata; the new formats for LiDAR include MrSID and are designed to help you preserve the full richness of your LiDAR data.
Mistake 6: Compressing Once and Forgetting Future Workflows
LiDAR data can be repurposed for many different applications (flood modelling, urban development, AI model training) that are different from the original purpose of the data.
The issues with this are many-fold:
The compression options you choose will affect future analysis.
Reprocessing raw data is both time-consuming and costly.
If you use inflexible formats, you will not be able to adapt to new tools/platforms.
How to deal with these issues is to think about the long-term. Pick a LiDAR compression technique that is scalable, therefore flexible, useable in cloud environments, and able to accommodate changing analytics needs. The use of MrSID as a compressed format will allow you to store the data efficiently now, but still allow those same datasets to be flexible for future use.
Best Practices for Effective LiDAR Compression
In summary, these are some of the best practices to consider:
Use LiDAR-optimized file compression formats
Know when to use lossless vs. lossy compression
Prioritize random access and speed
Retain point attributes and send your associated metadata together
Select well established and future compatible technology options
Compressing LiDAR data is not merely about reducing file size, but rather it involves retaining the integrity of the data, delivering on performance, and creating a long-term value proposition. Organizations that use a proven solution such as LizardTech’s MrSID LiDAR Compression will be able to effectively manage their large datasets of points created from LiDAR without hybridizing the quality and usability of their datasets.
Choosing the appropriate method of compression can affect how effectively an organization is able to streamline their LiDAR workflows and further reduce their storage and delivery costs.
For more information or any questions regarding the LizardTech suite of products, please don't hesitate to contact us at:
Email: info@geowgs84.com
USA (HQ): (720) 702–4849
(A GeoWGS84 Corp Company)