This morning, July 13, 2009, LizardTech unveiled LiDAR Compressor. This application allows consumers of LiDAR data the same benefits LT has provided raster consumers for over 10 years. The press release describes some of the product’s overall capabilities and benefits. I was privileged to be the project manager for this effort and in this post will describe the product from an engineering perspective.

At the lowest level, the application leverages our experience doing wavelet-compression. Our specific approach is the subject of a patent application and I won’t attempt to describe it in detail. However, a concise summary is that we do a 1D wavelet transform on each of the channels of interest (x, y, z, intensity, etc) and then use the same compression techniques we use in our MrSID and JP2 implementations (bitplaning, entropy encoding, appropriate quality-weighting of the bitplanes). 

At the SDK implementation level, we depend on several packages from the OSGeo stack. We use liblas to read data out of the LAS files. LT is proud to be the first (to my knowledge) commercial application of this industrial-strength library. (Regular readers of our blog may recall that last March we supported the OSGeo Code Sprint in Toronto; liblas was certainly part of our interest there!). Liblas, in turn, uses libgeotiff to decode the CRS information in the LAS files and the now venerable GDAL to translate that into a WKT that the rest of us can understand. The result is a C++ SDK available to our application … and soon to yours too (decode only).  We’ve got distributions for both Windows and Linux which will be available on our developer’s site any day now.
 

At the application level, the underlying technology has changed as well. Our flagship product, GeoExpress, is written in managed C++ using the Windows Forms library. Our most recent products (GeoViewer and Lidar Compressor) are built in C# using WPF. Our experience was that this change allowed much greater developer productivity and the creation of more flexible (not to mention prettier) applications. For example, I suspect we would not even have attempted a ListView with dropdown boxes in the header (all dynamically populated based on the selected input text file — see Figure 1 below) if we had to implement it without XAML support:
 

Figure 1: Importing a text file.

Additionally, the application makes extensive use of the WPF/.NET threading support. This allows the application to achieve two design goals: (a)it provides first class support for text-based LiDAR files and (b) it can run several jobs at once. 

Before we can do any kind of processing (e.g. compressing, viewing) we need to tell the file’s reader how many points there are and what the overall extent is. For LAS files (and MG4 files) this is no big deal because everything we need to know is stored explicitly in a compact header. For text files, it’s not stored at all and must be calculated which involves reading the entire input file. This “initialization” (see figures 2 and 3 below) can take a long time and needs to happen in the background.

Figure 2: Text file initializing (in the background, so the application is still responsive.)

A few minutes later …

Figure 3: We’ve read the 321 Mb text file and now know the extent and number of points.

Running several jobs concurrently in and of itself is not a hard problem. The difficulty comes when we consider how to provide feedback (progress bar, log messages), the ability to cancel and deal with failures. For historical reasons, GeoExpress runs its compression work in a separate process. This provides excellent isolation (uh,… that’s part of the ‘history’) but complicates tight integration with the parent application. This is part of the reason we could not include the ability to run concurrent jobs in GeoExpress 7. Lidar Compressor uses WPF’s built in BackgroundWorker class to manage the worker threads.  This vastly simplifies the integration tasks. The result is an application that leverages the multi-processor capabilities of modern hardware as well as a UI that is simple to understand and use (see Figure 4 below).
 

Figure 4: Five jobs running simultaneously.

Everything the user sees is pure WPF except for one thing:  the viewer. The viewer is a WPF wrapper around DirectX. There’s a lot of math that goes into manipulating 3D objects. DirectX provides an API to do this for us and, very significantly, runs it on the graphics card (i.e. no burden on the CPU). So, the good news is that rendering is blazing fast. The bad news is that DirectX 9 D3D (which is what we use) gets very finicky when it comes to older OSs (that is, XP), drivers and hardware. We were not able to get a really satisfying resolution to when DX would fail and why, but we were able to guard against it so that the application handles the failure gracefully.

Checkout the free trial and let us know what you think.

You may have noticed a new product from the Lizard Labs a couple of months ago: GeoViewer 3.0.  What you may not have noticed, however, is that we’ve started using some new technologies to build this soon-to-be-award-winning app.

Installation
GeoViewer is a web download: you click on the link on our website and the app is downloaded and installed right quick.

That’s a nice improvement over the way things used to be.  But the really good bit is that the application is also able to detect when we’ve posted a newer version, perhaps with bug fixes or new features, and automatically update itself.  This dramatically simplifies life for you and us: we don’t have to force-feed you new CDs and you don’t have to wonder if you’ve got the latest version.

Case in point: shortly after posting 3.0.0 on the web, we found a small bug we needed to fix.  Nothing major, really, but annoying enough to justify getting a fix to our customers.  And we’d already had thousands of downloads.  What to do, what to do?  In the old days we’d have tossed out hundreds of 3.0.0 CDs and burned a bunch of new ones.  Now, though, we just post 3.0.1 to the web and within a week all the deployed copies out there will notify their users that a new version is available and would you like to go ahead and download it?  (This feature can be disabled, of course: go to Options, then GeoViewer update preferences.)

This functionality comes to us courtesy of a relatively new Microsoft .NET feature called ClickOnce, one of the new technologies we’re starting to use here.

Look-and-Feel
GeoViewer doesn’t look much like it’s big brother, GeoExpress. 

Notice the soothing blue-grey color tones and those gentle gradient fills?  Goodbye, battleship gray!

We’re not building consumer-facing Twitter clients out here, and we’re not certainly professional UI designers, but we do realize that there is something to be said for working towards a visually pleasing app, even in this relatively staid world of GIS apps.  Look for all our products to start looking and feeling better in future releases.

Oh, and it’s not just the look-and-feel that have gotten the reboot.  You can’t really see it, but under the covers we’re using another new Microsoft .NET technology called WPF (Windows Presentation Foundation) to build GeoViewer.  The WPF Framework gives us programmers a much better way of expressing and writing the user interface logic – gone are the days of event loops and MFC and WM_PAINT messages: we now use XML to control the application’s appearance and elegant property bindings to control the behavior.

And C#
Since we’re embracing .NET technologies for our Windows applications, we’ve also had to make the switch from C++ to C#.  It’s a happy new world to be in – expressive syntax, rich API, and a garbage collector you can trust.  Some time ago we tried programming under .NET using Managed C++, but it was pretty painful.

Of course, our underlying SDKs will remain as C++ libraries.  We’ve learned enough P/Invoke to be able to write our GUI front ends in C# and have them interop over to the C++ layer for the hard stuff.

Upcoming
We’re pretty jazzed out here about being able to break out of the old InstallShield-MFC-C++ world and use some new technologies that allow us to churn out better products faster.  Look for more goodness as we post more releases in the coming months.

We’ve just released our new viewer, LizardTech GeoViewer 3.0, and so far it’s flying off the shelf, or rather across the Internet. It’s free to download here (http://www.lizardtech.com) and we’ve had thousands of takers already. We issued a press release yesterday, and I was going to follow up with an announcement here, but the way things are going it’ll be old news by the time I hit “publish”.

Geospatial professionals have been asking us for a new viewer for a long time, and we were finally able to devote some time to making one. We didn’t want to short-change it, and I think you’ll love the results if you’re a person who uses imagery every day.

For input GeoViewer supports too many raster formats to list here, plus ESRI Shapefiles. It exports to GeoTIFF, JPEG, and PNG. What’s more, you can add layers (images and vector overlays) from local repositories, from a LizardTech Express Server catalog, or from WMS or JPIP servers.

Best of all, you can group your layers and hide or show them, so you can visually compare years, or regions — or any other organizational units you care to create — with a click.

We put a lot of care into making GeoViewer easy to install and get started with. If you haven’t already grabbed yours, why not try it out? And let us know what you think.  

I attended the American Wind Energy Association’s WINDPOWER 2009 in Chicago May 4-7. AWEA WINDPOWER is the premier wind energy conference and exhibition in North America. The North American wind market has been in a steady growth mode for well over five years with a tremendous boom for the past three years. Wind power generating capacity increased by 32% in 2007 and projections for 2009 are trending to be even greater.

The AWEA WINDPOWER Conference and Exhibition is the largest annual wind conference and exhibition in the world, this year featuring over 15,000 attendees and over 1,200 exhibitors. Each year, wind energy professionals gather at this event to learn about the latest industry developments and technologies, review new products and services in the expansive exhibit hall, and network with leading industry decision makers.

LizardTech did not have a booth at this show. As this was our first year at AWEA WINDPOWER we decided to save a little money by only attending rather than exhibiting and I was able to cruise the floor and scout out good leads. We did not spring for the sessions’ access – that would have been an additional $800.00. If you missed the sessions, too, they’re available on DVD for $500.00 at http://www.awea3.org/source/Orders/index.cfm?Section=Store .

I feel that it was better (and cheaper) to be an attendee at this show rather than exhibitor. It was too huge to be stuck in a booth all day. Walking the floor each day and talking with other exhibitors about their GIS needs (most didn’t know what GIS was) I was able to get better leads than I would have at a booth. There is something for everybody at this show and I hope to be back in 2010, when the conference will be held in Dallas. Details about next year’s show are at http://www.windpowerexpo.org/2010/coming_soon.cfm .

Photo courtesy of the American Wind Energy Association.

Glen Thompson and I were fortunate to attend the ESRI Developer’s Summit in Palm Springs last month. In addition to being a great excuse to get out of the rain here in Seattle, the Dev Summit provides a great opportunity to catch up with our friends from Redlands and see what’s new with ArcGIS.

Those who missed it can look over the presentations.

One way to think about an event like this is in terms of what’s cool and exciting and what’s getting the “business as usual” treatment. This year what’s cool and exciting is most definitely client-side web mapping. We’re talking Flex, Javascript and the newly released Silverlight API for ArcGIS Server. These were accompanied by cheering, contests, prizes and “Playful” (?) references to glitzy, “silverlighty” (get it?) applications.

There were no contests or prizes for the backend stuff on which Glen and I spent most of our time. Of particular note, GDAL is definitely on the rise here. GDAL is the open source project that (among other things) provides the ability to convert between raster formats. Principal maintainer Frank Warmerdam presented a technical session on “Custom Raster Format Support in ArcGIS through GDAL”. Guys, start thinking about migrating those Erdas Imagine / RDO extensions to GDAL. The writing is on the wall.

And, speaking of items that will eventually need migrating, I gave a short talk on “Implementing a Custom Image Server Raster Format.” It describes the technical architecture of the project and includes some server benchmarking work that we did in Seattle highlighting the impact of using Express Server with ArcGIS Image Server Extension.

ESRI is even hosting a video of the presentation.

Thanks, guys!