LizardTech.com

Archive for the ‘Open Source’ Category

Extending Express Server Functionality with GeoServer

Monday, September 23rd, 2013

Got five minutes? That’s all it takes to add GeoServer to an existing installation of Express Server.

GeoServer is an open source server for geospatial data and imagery. It includes support for Shapefiles, GeoTIFF, PostGIS, and much more.  You can use GeoServer to distribute imagery via standard protocols, including WMS, WFS, and WCS.

In the example below, LizardTech’s GeoViewer software displays imagery from Express Server and GeoServer. The basemap layer is a MrSID image hosted by Express Server. The vector layer is a shapefile of all the roads in Washington hosted by GeoServer.

GeoServer_imagery_in_GeoViewer

To install GeoServer on the machine that hosts the Express Server, complete the following steps:

  1. Open a web browser and navigate to the GeoServer Stable Download page:
    http://geoserver.org/display/GEOS/Stable
  2. Download the GeoServer Web Archive.
  3. Navigate to the directory where you downloaded the web archive, and extract the contents of the zip file.
  4. Copy the geoserver.war file to the following directory:<Express Server Installation Directory>\ImageServer\Tomcat\webapps
  5. Restart Tomcat.
    • On Windows, open the Services utility, right-click the LizardTech Express Server Tomcat Service, and click Restart.
    • On UNIX, open a terminal and enter the following command to stop Tomcat:
      sudo /etc/init.d/lttomd stop

      Then, enter the following command to start Tomcat:

      sudo /etc/init.d/lttomd start

To open the GeoServer Web Administration Interface, navigate to the following URL:

https://localhost:8443/geoserver

The GeoServer Welcome page appears:

GeoServer_ Welcome

Note that the URL uses https. Because Express Server’s configuration of Tomcat uses https for the Express Server Manager, GeoServer also uses https. To access imagery via WMS with GeoServer, use the following URL in your GIS software:

https://localhost:8443/geoserver/wms?

Note that GeoViewer currently only supports http connections.

To allow connections to GeoServer over http, complete the following steps:

  1. Open the following Tomcat configuration file:
    <Express Server Installation Directory>\ImageServer\Tomcat\conf\server.xml
  2. Search for the following line:
    <!--<Connector port="8080" protocol="HTTP/1.1"
                   connectionTimeout="20000"
                   redirectPort="8443" />-->
  3. Remove the opening and closing comment brackets (<!-- and -->.)
  4. Save the file and restart Tomcat.

Now, when you access GeoServer imagery via WMS, you can use the following URL:

http://localhost:8080/geoserver/wms?

For more information on how to use and configure GeoServer,  refer to the GeoServer documentation.

Viewing Express Server Imagery with Leaflet

Tuesday, September 10th, 2013

Did you know that you can view Express Server imagery with Leaflet via WMS? Leaflet is an open-source JavaScript library for creating rich, interactive web maps.

In the example below, we’ve used Leaflet to display a raster layer of Washington state, a raster basemap layer, and a marker icon that identifies the location of the LizardTech office. The raster layers are stored in the MrSID format on the LizardTech demo Express Server and displayed via WMS. Note that out of the box, Leaflet includes keyboard controls, smooth zoom, support for mobile interactions, and many other features. Additionally, because Leaflet is styled with CSS3, it’s easy to customize pop-ups, controls, and even transitions.

To create the above map, complete the following steps:

  1. Download Leaflet and extract the files.
  2. Create an html file in the same directory where you extracted the Leaflet files.
  3. Link to the Leaflet stylesheet and JavaScript file in the document head.
  4. Paste the following code in the document body:
<div id="div_map" style="width: 800px; height: 600px;">
</div>
	
<script>
						
var washington = new L.tileLayer.wms(
'http://demo.lizardtech.com/lizardtech/iserv/ows', {
    layers: 'Washington',
    maxZoom: 18,
    format: 'image/png',
    transparent: true
});		
		
var modis = new L.tileLayer.wms(
'http://demo.lizardtech.com/lizardtech/iserv/ows', {
    layers: 'MODIS',
    maxZoom: 18,
    format: 'image/png',
    transparent: true
});
		
var map = new L.map('div_map', {
    center: new L.LatLng(47.60489, -122.33695), 
    zoom: 12,
    layers: [modis,washington],
    zoomControl: true
});
		
var marker_LT_office = new L.marker(
[47.60489, -122.33695]).addTo(map);
marker_LT_office.bindPopup("The LizardTech office!");
		
</script>

2011 Code Sprint

Wednesday, March 23rd, 2011

Fellow LizardTech developer Kirk and I just got back from the third annual OSGeo Code Sprint; this time in Montreal, Canada. Daniel Morissette did a great job of organizing this event and has already posted an excellent review of the sprint (here) including pictures (here), so I’ll simply add that that it was great fun to meet personally with the guys who build great libraries like PostGIS, libLAS/libPC and GDAL.

Kirk and I worked on adding MG4 (MrSID Generation 4) support to the new libPC. The “PC” stands for “point cloud”. libPC is the next evolution of libLAS and aims to be the GDAL of point cloud formats. We didn’t finish, but once this work is done it will enable decode support in libPC for compressed LiDAR data in MrSID format. For more info, see http://libpc.org/notes/goals.html.

Fun fact:  The pronunciation of GDAL is the subject of some disagreement. It seems obvious to me and Kirk that “GeeDAL” is correct (like “Vi”, the editor or “Unix”, the OS).  I was amused to hear that in French-speaking Montreal it’s sometimes “J’DAL.” For reasons that are not clear to me some of the inner circle (Frank W and Hobu) usually pronounce it “GuuDAL.”

MrSID in Linux

Enjoying a break after sprinting. Image courtesy of OSGeo Code Sprint 2011.

On a more serious note, much of the GIS industry’s foundation code depends on some of these libraries. That’s good for all of us: quality libraries allow us all to write valuable applications quickly with high confidence in the underlying stability and reliability. They allow us to focus on what we’re good at and (hopefully) what is important to our users. That’s really valuable and it doesn’t cost very much to give a little back.

  1. If you work around a limitation, return the patch to the community. You don’t need to be a committer, just send it in. It’s in everyone’s interest to consider your patch.
  2. If you can’t work around it, offer to pay the community to fix it. Some of these guys make a living doing this and our experience is that the costs are often quite reasonable.
  3. Consider direct financial support of key projects and/or the OSGeo Foundation (http://www.osgeo.org).

Not walking, mapping!

Thursday, March 25th, 2010

You’ve heard of Open Street Map, right? You can think of it as Wikipedia for maps: people from all over the world map their own neighborhoods, and the results are made available for free, to all, with no copyright restrictions. For most of the US the road network has been pretty well mapped already, via free data available from the government, but it can get out of date surprisingly quickly and of course it doesn’t list the really important stuff like where to get the best ice cream.

CUGOS Bainbridge mapping outing

These brave souls are about to redefine the leisurely walk.

On March 13th, nineteen people from CUGOS (including three Lizards) and one dog met up over on sunny Bainbridge Island for an Open Street Map mapping party. It was a diverse group — academics, kids, hackers, spouses. We met in the morning at the Bainbridge ferry dock, took a picture to commemorate the event, and divided up into four groups. Each group then spent the next few hours walking around a different part of Bainbridge’s downtown core, recording the coordinates of any “features of interest”:

  • roads that had changed or been renamed
  • “street furniture”: bus stops, park benches, fire hydrants
  • points of interest: pharmacies, libraries, museums, pubs, ice cream parlors
  • walking trails
  • and much, MUCH more

walking orders

Holly reviews her route while Dane gets “walking orders” from Michael.

Altogether we collected a few hundred different way points, using everything from handheld GPS units to iPhones to laser range finders. Over a leisurely late lunch by the waterfront, and in a subsequent CUGOS meeting, we worked to get all our GPS tracks and waypoints uploaded to the OSM databases.

amigos

Leaving no avenue uncharted, no point of interest unheralded, Michael, Walter and Roger investigate.

A walk through Bainbridge on a sunny day with friends — and GPS gear. What more could a GIS geek want?

A fine day out

Roger and Tifani display the elements of their GPS walking kit (bus stop sold separately).

Photos courtesy of CUGOS’ own Roger and Tifani.

LizardTech Engineering’s introduction to LiDAR Compressor

Monday, July 13th, 2009

This morning, July 13, 2009, LizardTech unveiled LiDAR Compressor. This application allows consumers of LiDAR data the same benefits LT has provided raster consumers for over 10 years. The press release describes some of the product’s overall capabilities and benefits. I was privileged to be the project manager for this effort and in this post will describe the product from an engineering perspective.

At the lowest level, the application leverages our experience doing wavelet-compression. Our specific approach is the subject of a patent application and I won’t attempt to describe it in detail. However, a concise summary is that we do a 1D wavelet transform on each of the channels of interest (x, y, z, intensity, etc) and then use the same compression techniques we use in our MrSID and JP2 implementations (bitplaning, entropy encoding, appropriate quality-weighting of the bitplanes). 

At the SDK implementation level, we depend on several packages from the OSGeo stack. We use liblas to read data out of the LAS files. LT is proud to be the first (to my knowledge) commercial application of this industrial-strength library. (Regular readers of our blog may recall that last March we supported the OSGeo Code Sprint in Toronto; liblas was certainly part of our interest there!). Liblas, in turn, uses libgeotiff to decode the CRS information in the LAS files and the now venerable GDAL to translate that into a WKT that the rest of us can understand. The result is a C++ SDK available to our application … and soon to yours too (decode only).  We’ve got distributions for both Windows and Linux which will be available on our developer’s site any day now.
 
liblas and GDAL in LiDAR Compressor

At the application level, the underlying technology has changed as well. Our flagship product, GeoExpress, is written in managed C++ using the Windows Forms library. Our most recent products (GeoViewer and Lidar Compressor) are built in C# using WPF. Our experience was that this change allowed much greater developer productivity and the creation of more flexible (not to mention prettier) applications. For example, I suspect we would not even have attempted a ListView with dropdown boxes in the header (all dynamically populated based on the selected input text file — see Figure 1 below) if we had to implement it without XAML support:
 
Importing a text file

Figure 1: Importing a text file.

Additionally, the application makes extensive use of the WPF/.NET threading support. This allows the application to achieve two design goals: (a)it provides first class support for text-based LiDAR files and (b) it can run several jobs at once. 

Before we can do any kind of processing (e.g. compressing, viewing) we need to tell the file’s reader how many points there are and what the overall extent is. For LAS files (and MG4 files) this is no big deal because everything we need to know is stored explicitly in a compact header. For text files, it’s not stored at all and must be calculated which involves reading the entire input file. This “initialization” (see figures 2 and 3 below) can take a long time and needs to happen in the background.

Initializing

Figure 2: Text file initializing (in the background, so the application is still responsive.)

A few minutes later …

Extents

Figure 3: We’ve read the 321 Mb text file and now know the extent and number of points.

Running several jobs concurrently in and of itself is not a hard problem. The difficulty comes when we consider how to provide feedback (progress bar, log messages), the ability to cancel and deal with failures. For historical reasons, GeoExpress runs its compression work in a separate process. This provides excellent isolation (uh,… that’s part of the ‘history’) but complicates tight integration with the parent application. This is part of the reason we could not include the ability to run concurrent jobs in GeoExpress 7. Lidar Compressor uses WPF’s built in BackgroundWorker class to manage the worker threads.  This vastly simplifies the integration tasks. The result is an application that leverages the multi-processor capabilities of modern hardware as well as a UI that is simple to understand and use (see Figure 4 below).
 
Five jobs running simultaneously

Figure 4: Five jobs running simultaneously.

Everything the user sees is pure WPF except for one thing:  the viewer. The viewer is a WPF wrapper around DirectX. There’s a lot of math that goes into manipulating 3D objects. DirectX provides an API to do this for us and, very significantly, runs it on the graphics card (i.e. no burden on the CPU). So, the good news is that rendering is blazing fast. The bad news is that DirectX 9 D3D (which is what we use) gets very finicky when it comes to older OSs (that is, XP), drivers and hardware. We were not able to get a really satisfying resolution to when DX would fail and why, but we were able to guard against it so that the application handles the failure gracefully.

3D_sid_o

Checkout the free trial and let us know what you think.