high-order byte of LAS/RGB with 16 bit color

[David Swanson]

Below is a Cut and Paste of Martin Isenburg's comment on our Lizardtech Facebook page.

 here some details for your engineers:

in that email from Feb 4, 2011 called "LAZ / LAS extraction
performance" to the liblas list one of your employeees reported the following results:

01/28/2011 03:50 PM 61,301,311 2398_400.las
01/28/2011 04:28 PM 8,906,275 2398_400.laz
01/28/2011 04:25 PM 4,650,992 2398_400.sid

I immediately was puzzled by the odd compression behavior. This was
the only case where MG4 blatantly outperformed LAZ ... inconsistent
with all other examples.

i finally got around to investigate these puzzling numbers in great detail and have come to the only possible conclusion: either the SID format does either not compress the RBG values at all or (more likely) it does compress only the low order byte of the 16 bit R/G/B values (which is always zero in this particular file). could you check the colors in your 2398_400.sid file after decompression?

how do i arrive at this conclusion? it seems that 2398_400.sid is about 4,250,000 bytes more compact than 2398_400.laz. however ... when i remove the 6 byte RGB records from each point in the LAS file (e.g. go from point type 2 to point type 0) then 2398_400_no_rgb.laz is also about 4,300,000 bytes more compact than 2398_400.laz.

8,906,275 2398_400.laz
4,615,538 2398_400_no_rgb.laz

i then went ahead and extracted the RBG values of all 2,357,734 points and put them in binary form (three unsigned shorts per RGB) into a file. I then compressed this file with various compression schemes:

14,146,404 rgb.rgb
4,194,547 rgb.rgb.7z
4,232,032 rgb.rgb.bz2
6,578,047 rgb.rgb.gz
4,760,187 rgb.rgb.rar

Again ... we find the number for compressed RGB in the ballpark of 4,300,000 bytes. Especially here I started to convince myself that the
4,650,992 bytes of the SID file cannot possibly contain all these RGB
values plus the 20 byte POINT10 data of all 2,357,734 points.

So unless the LT compressor is doing some true entropy magic there is something fishy about the 2398_400.sid file. Can you run the LT compressor on the 2398_400_no_rgb.las file? If my - is correct (that the colors don't get compressed or only the lower-order byte) then you will get a similar performance to 2398_400.las. To create the same " 2398_400_no_rgb.las" that i have been using, simply run

las2las -i 2398_400.laz -o 2398_400_no_rgb.las -point_type 0

using
h_t_t_p://w_w_w.cs.unc.edu/~isenburg/lastool ... as2las.exe
h_t_t_p://w_w_w.cs.unc.edu/~isenburg/lastool ... README.txt

That said ... the results of autzen-colorized (see below) suggest that
LT does compress the color. So that's even more puzzling. Can you shed
some light on what;s going on here, Michael?

Cheers,

Martin
3 hours ago • LikeUnlike


Martin Isenburg and:

i decided to investigate the "(truly) incredible LT performance on 2398_400.las" by installing the LT LiDAR Compressor on my dad's laptop and re-running the experiments. As I suspected ... the LT LiDAR Compressor does not compress the high-order byte of the 16-bit unsigned short RGB color field. Below the proof via lasinfo outputs on the original 2398_400.las and the LT LiDAR Compressor decompressed version. The colors are all zeros. The low-order RGB bytes get compressed correctly which I verified on the autzen data set.

the mystery is solved. (-:

martin

lasinfo.exe -i 2398_400_LT_decompressed.las
reporting all LAS header entries:
file signature: 'LASF'
file source ID: 0
reserved (global_encoding):0
project ID GUID data 1-4: 0 0 0 ''
version major.minor: 1.2
system identifier: ''
generating software: 'LizardTech 1.1.0.2802'
file creation day/year: 103/2011
header size 227
offset to point data 227
number var. length records 0
point data format 2
point data record length 26
number of point records 2357734
number of points by return 0 0 0 0 0
scale factor x y z 0.001 0.001 0.001
offset x y z 0 4000000 0
min x y z 338510.165 4096566.104 1672.827
max x y z 338702.523 4096801.276 1714.060
reporting minimum and maximum for all LAS point record entries ...
x 338510165 338702523
y 96566104 96801276
z 1672827 1714060
intensity 0 0
edge_of_flight_line 0 0
scan_direction_flag 0 0
number_of_returns_of_given_pulse 0 0
return_number 0 0
classification 0 0
scan_angle_rank 0 0
user_data 0 0
point_source_ID 0 0
Color R 0 0



G 0 0



B 0 0
WARNING: there are 2357734 points with return number 0
WARNING: there are 2357734 points with a number of returns of given pulse of 0
histogram of classification of points:
2357734 Created, never classified (0)

lasinfo.exe -i 2398_400.las
reporting all LAS header entries:
file signature: 'LASF'
file source ID: 0
reserved (global_encoding):0
project ID GUID data 1-4: 0 0 0 ''
version major.minor: 1.1
system identifier: ''
generating software: ''
file creation day/year: 0/0
header size 227
offset to point data 227
number var. length records 0
point data format 2
point data record length 26
number of point records 2357734
number of points by return 0 0 0 0 0
scale factor x y z 0.001 0.001 0.001
offset x y z 0 4000000 0
min x y z 338510.165 4096566.104 1672.827
max x y z 338702.523 4096801.276 1714.060
reporting minimum and maximum for all LAS point record entries ...
x 338510165 338702523
y 96566104 96801276
z 1672827 1714060
intensity 0 0
edge_of_flight_line 0 0
scan_direction_flag 0 0
number_of_returns_of_given_pulse 0 0
return_number 0 0
classification 0 0
scan_angle_rank 0 0
user_data 0 0
point_source_ID 0 0
Color R 768 65024



G 256 65024



B 0 64768
WARNING: there are 2357734 points with return number 0
WARNING: there are 2357734 points with a number of returns of given pulse of 0
histogram of classification of points:
2357734 Created, never classified (0)

[David Swanson]

Hello Martin,

Thank you for your comments and information on LiDAR Compressor. This has been - to our engineering team and they are looking in to the issue.

David Swanson
Lizardtech Support