LizardTech was one of the sponsors of an OSGeo code sprint in Toronto last week. Mike Rosen and I were both fortunate enough to attend and spend some quality time with twenty of the brightest minds of the open source geo world, folks working on everything from GDAL to MapServer to PostGIS to OpenLayers.

Perhaps the most visible result from the LizardTech side of things was some performance analysis that Frank Warmerdam, Chris Schmidt, and I did on GDAL’s use of MrSID – which should eventually help everyone downstream of GDAL, notably MapServer.

Paul Ramsey (mine host) and Chris both blogged daily about the event. Perry Nacionales and Paul have also posted some pix on flickr, including some of us all unwinding at Le Hockey (Hey, look! Real live Canadians, in their natural habitat!).

And, though we might have the grace to blush just a little, we’re not too modest to nonetheless link to Chris’ stirring post on using GDAL with our D-SDK.

Image courtesy of Perry Nacionales

The “Call for Presentations and Workshops” for GeoWeb 2009 has just been announced. The theme of this year’s conference is “Cityscapes”: the geography of urban environments, BIM-CAD-GIS convergence, 3D modeling.

LizardTech has been fortunate to be involved with this conference for a number of years now, and so we’re once again looking forward to July and another fun week with our neighbors to the north.

This sort of email comes across our desks pretty regularly:

I’ve got a GeoTIFF image that has some transparent background areas. When I view it with the Acme GIS Viewer, those transparent areas look black, as they should. But when I compress the image with MrSID, those areas aren’t all black anymore – they look all dotty, spotty and mottled! What am I doing wrong?!

This is not a bug, and you’re not doing anything wrong either: it is, alas, a part of the compression process. Long ago when your mother told you that you would have some good days and some bad days, this is what she meant.

Allow me to explain.

Consider Exhibit A:

We see an image that has been rotated several degrees and the exposed triangular areas in the corners are a solid color. These corner pixels are intended to be treated as transparent – the term of art here is “NODATA” – and it just so happens that I chose to present the image to you against a black background. If I chose to present the exact same image to you against a fuchsia background, you’d get Exhibit B:

NODATA pixels are an essential part of the mosaicking process. Were we to construct a mosaic by placing the Exhibit A image on top of another adjoining second image, which was similarly rotated, our transparent areas allow the underlying (“real”) pixels from the second image to show through.

All pixels in an image have to be assigned numerical color values – even the NODATA ones. In Exhibit A, the guy who encoded the image happened to choose black, i.e. (0,0,0), which is pretty typical, but he could have also chosen mauve or puce. The NODATA value is encoded in the image’s header somewhere, so that when a program wants to display the image, it knows that any pixel with that special NODATA color should not be shown and instead let whatever pixel is “underneath” shine through. Nine times out of ten, that will be black – but it could be a fuchsia background or the pixels from another image tile.

(There’s a smart guy in the back of the room mumbling about alpha channels and using shapefiles for masks and stuff like that. Thank you, yes, thank you, those are also ways to mosaic images too, but we’re not using that method today. Let’s move on, please, we have a lot to cover today.)

Right. So far so good.

Now for the tricky bit.

Let’s say you compress Exhibit A to MrSID with a compression ratio of 30:1. The output image looks almost identical to the input image – great. But look at Exhibit C, in which that same output image is displayed against a white background:

“Yikes!” you say. “What am I doing wrong?!” you ask.

Because we compressed the image at 30:1 we’re doing lossy compression, meaning that each pixel’s color values in the output image will be close to, but not exactly equal to, the corresponding pixel’s color values in the input image. The compression process will change some pixels by just a few units, such as changing a (0,0,0) pixel to a (1,0,2) pixel. We won’t get into the algorithmic details of how and why the compression does this, we’ll just assert that it does and tell you that it happens most often at the “edges” of images or when you’re looking at zoomed-out, reduced-resolution versions of images. Fortunately, this change is typically imperceptible to the casual observer.

Unfortunately, the replacer of NODATA pixels is not a casual observer. The replacer is rather tetchy, actually. Keen eye for detail. Type A, O/C, that sort of thing. Which means that it’s gonna treat only the (0,0,0) pixels as transparent and none others. Which means that those pixels that got nudged up to being (1,0,2) and (0,1,1) and such are gonna stay right where they are. Which means that our putative, erstwhile transparent layer is gonna look funny. Especially at the edges of the images or when you’re looking at zoomed-out, reduced-resolution versions of images.

It’s not a bug in MrSID, and you’re not doing anything wrong: it is an unfortunate but necessary side-effect of the compression algorithms we use. Such side-effects are called “artifacts”, and we call this kind of artifact “speckling”.

All is not completely lost, however.

One way to alleviate the problem is to get the NODATA replacer to relax a little – loosen its tie, roll its shoulders a few times, take a couple deep cleansing breaths. Instead of only looking at the (0,0,0) NODATA pixels, it can use a “fuzzy” replacement algorithm, meaning that those pixel values relatively close to (0,0,0) will also be replaced, like (1,0,2) and (0,1,1). Our Express Server, for example, allows you to use this feature when mosaicking layers and even control the “degree” of fuzziness to use. (See the manual for details.) This works great, but keep in mind that you’re replacing more pixels now, and you might have pixels with value (0,1,1) that are supposed to be actual data, such as a dark shadow or black rooftop: these will get treated as NODATA and allow the underlying background data to show through.

Another approach is to use the “despeckle tool” in GeoExpress 7, as described in an earlier blog post. This new feature allows you to handle the whole issue a little more cleanly at encode time, or to clean up existing imagery you’ve already encoded. It works by attempting to find the “real” edges of images like those in Exhibit A, or more complexly polygonal mosaics, and then inhibiting the compression algorithms from touching pixels outside of those edges. Essentially what the propeller-head in the back of the room was going on about earlier. It is not always perfect, but it works for most cases. Again, see the manual for details.

Next time, perhaps, we’ll talk about some other cool artifacts like “ringing” and “grout”. Thanks for tuning in.

-mpg

Because we support multiple platforms (Windows, Solaris, Linux, Mac), our Decode SDK is written in C++.  Some years ago, our C++ APIs used to regularly lead to the question, “do you support Java?”.  The answer was always sorry, no, we’re not a Java shop and we don’t have any Java bindings… But we’ve always provided a relatively simple C API which we claimed could be wrapped using JNI, and for the most part that made people happy.

For the past year or so, though, all the Java requests seem to have disappeared, only to be replaced by the question, “do you support .NET?”  This usually means “do you have any C# bindings?”, although we do get the occasional VB.NET request.  Our response has been sorry, no, we’re not a C# shop and we don’t have any .NET bindings… But, again, we’ve told people that “it ought to be fairly easy to call out to our C API using P/Invoke, .NET’s Interop functionality”.

Well, recently some of the engineers here at LizardTech HQ have started programming in C# for reals, and so now we actually have just enough in-house expertise on the question to be able to provide some additional help on this one.  While the current DSDK release doesn’t provide any C# bindings, we have put together a very simple example app that shows how to use Interop to access the C API.

The code is, at heart, remarkably simple.  First, you declare your C functions so they can be accessed from within your C# class, like so:

      [DllImport("lti_dsdk_cdll.dll")]
      static extern int ltic_openMrSIDImageFile(out
            IntPtr image, string fileName); 

      [DllImport("lti_dsdk_cdll.dll")]
      static extern uint ltic_getWidth(IntPtr image);

Then, you close your eyes, tap your ruby slippers together three times, and innocently call the functions just like they were real functions:

      // this is essentially our void* pointer
      IntPtr image = IntPtr.Zero;
      string infile = "..."; 

      sts = ltic_openMrSIDImageFile(out image, infile);
      ...
      uint width = ltic_getWidth(image);
      ...

You can download the full example from here.

Yes, we know, it’d be nice to provide interop support for the C++ classes so as to give access to the whole SDK… but quite frankly, we’re not sure the market demand is really there yet.  At the very least, though, we’ll try to include interop support for the C API in the next release of the SDK.

Keep those cards and letters coming.

-mpg

Last week I had the pleasure of attending GeoWeb 2008 on behalf of both LizardTech and OSGeo. The conference was once again in Vancouver BC, at my favorite business hotel and conference venue. I’ve attended this conference for a number of years now, and it gets better every passing year.

Just a few highlights:

• The underlying theme running through the week was the integration (confluence? convergence?) of the GIS world with the worlds of CAD and BIM (building information model). Architects typically operate at a different scale than we’re used to, but increasingly they want to be able to envision and model their buildings in the larger urban landscape that we can provide for them. Kimon Onuma and his BIMStorm work demonstrated this integration very well. Going the other direction, traditional GIS folks are looking to things like CityGML to be able to improve the fidelity and add that 3rd dimension to their own models.

• I moderated a one hour discussion on Open Source Servers, ably assisted by panelists Paul Ramsey of Clever Elephant, Justin Deoliveira of OpenGeo, and Bob Bray of Autodesk. The attendance was good, and we had some good questions and discussions about the pros (and sometimes cons) of working in and with open source software.
• On behalf of Cody Benkelman of Mission Mountain Technology, I also presented a cool paper on using Amazon’s Mechanical Turk web service and Google Earth to solve a real problem for a real customer. I tried to get across two main ideas. First, Turks and Turk-like things can be seen as “outsourcing for the Web 2.0 generation”. Secondly, and possibly disconcertingly to some, complete “automation” is not always the best answer – us geeks think of it first, and yes, it’s usually the right move – but not always. Contact us for reprints.

• Dr. Michael Goodchild gave a great workshop on Data Quality – a topic which quite honestly sounded pretty dry and uninspiring, but which turned out to be both educational and interesting. He convinced me that LizardTech’s viewers are displaying lat/long incorrectly, at least from a data quality perspective.
• Michael Jones of Google keynoted again this year, and he once again made everyone stop and think deeply about the human impact the geo community can – and does – have on the world. Not the kind of talk you can summarize easily, you just had to be there.
• This year the conference held its first Student Competition. The competition required use of open source software for the projects; OSGeo was one of the sponsors and as such I was one of the judges. First prize went to Tobias Fleischmann (Paris Lodron University Salzburg, Germany) for “Web Processing Service for Moving Objects Analysis”, which was built using deegree. Second prize went to Tran Tho Ha and Nguyen Thi Thanh Thuy (Politecnico di Milano, Italy) for “e-Collaboration for DGPS/GPS data distribution and receiver device evaluation”, which used PostGIS, MapScript, and OpenLayers. Congratulations to both winners!

• GeoWeb is also famous for being scheduled during Vancouver’s annual “Celebration of Light“, an international

  

  fireworks competition held several evenings high above English Bay. As in previous years, the conference’s evening reception was turned into a sunset dinner cruise, after which we all went up on deck to oooh and aaah at the pyrotechnic ballet.

Finally, just for kicks, I’ll offer the following bits of geotrivia I collected during the conference:

• “A GPS with a bullet hole in it is a paperweight. A paper map with a bullet hole in it is a paper map with a bullet hole in it.” (attributed to the US Marine Corps)
• Tobler’s First Law of Geography: “Nearby things are more similar than distant things.”
• city furniture (noun): features of the urban landscape such as park benches, bus shelters, street lamps, etc
• “The amount of metadata needed for a piece of data varies with the ’social distance’ from me to my data consumer.” (Michael Goodchild)
• For Amazon’s web services, 85% use the REST API and 15% use the SOAP API. (quoted by Satish Sankaran, ESRI)
• On the Vancouver transit system today, 100 of 144 bus routes are under detours, due to 2010 Olympics work. (Peter Ladner, Vancouver deputy mayor)
• Thirty percent of all 911 calls are not associated with a street address. (Talbot Brooks)

GeoWeb 2009 is already being planned, and will include special emphasis on both cityscapes and 3-D modeling.