You’ve heard of Open Street Map, right? You can think of it as Wikipedia for maps: people from all over the world map their own neighborhoods, and the results are made available for free, to all, with no copyright restrictions. For most of the US the road network has been pretty well mapped already, via free data available from the government, but it can get out of date surprisingly quickly and of course it doesn’t list the really important stuff like where to get the best ice cream.

These brave souls are about to redefine the leisurely walk.

On March 13th, nineteen people from CUGOS (including three Lizards) and one dog met up over on sunny Bainbridge Island for an Open Street Map mapping party. It was a diverse group — academics, kids, hackers, spouses. We met in the morning at the Bainbridge ferry dock, took a picture to commemorate the event, and divided up into four groups. Each group then spent the next few hours walking around a different part of Bainbridge’s downtown core, recording the coordinates of any “features of interest”:

• roads that had changed or been renamed
• “street furniture”: bus stops, park benches, fire hydrants
• points of interest: pharmacies, libraries, museums, pubs, ice cream parlors
• walking trails
• and much, MUCH more

Holly reviews her route while Dane gets “walking orders” from Michael.

Altogether we collected a few hundred different way points, using everything from handheld GPS units to iPhones to laser range finders. Over a leisurely late lunch by the waterfront, and in a subsequent CUGOS meeting, we worked to get all our GPS tracks and waypoints uploaded to the OSM databases.

Leaving no avenue uncharted, no point of interest unheralded, Michael, Walter and Roger investigate.

A walk through Bainbridge on a sunny day with friends — and GPS gear. What more could a GIS geek want?

Roger and Tifani display the elements of their GPS walking kit (bus stop sold separately).

Photos courtesy of CUGOS’ own Roger and Tifani.

You may have noticed a new product from the Lizard Labs a couple of months ago: GeoViewer 3.0.  What you may not have noticed, however, is that we’ve started using some new technologies to build this soon-to-be-award-winning app.

Installation
GeoViewer is a web download: you click on the link on our website and the app is downloaded and installed right quick.

That’s a nice improvement over the way things used to be.  But the really good bit is that the application is also able to detect when we’ve posted a newer version, perhaps with bug fixes or new features, and automatically update itself.  This dramatically simplifies life for you and us: we don’t have to force-feed you new CDs and you don’t have to wonder if you’ve got the latest version.

Case in point: shortly after posting 3.0.0 on the web, we found a small bug we needed to fix.  Nothing major, really, but annoying enough to justify getting a fix to our customers.  And we’d already had thousands of downloads.  What to do, what to do?  In the old days we’d have tossed out hundreds of 3.0.0 CDs and burned a bunch of new ones.  Now, though, we just post 3.0.1 to the web and within a week all the deployed copies out there will notify their users that a new version is available and would you like to go ahead and download it?  (This feature can be disabled, of course: go to Options, then GeoViewer update preferences.)

This functionality comes to us courtesy of a relatively new Microsoft .NET feature called ClickOnce, one of the new technologies we’re starting to use here.

Look-and-Feel
GeoViewer doesn’t look much like it’s big brother, GeoExpress. 

Notice the soothing blue-grey color tones and those gentle gradient fills?  Goodbye, battleship gray!

We’re not building consumer-facing Twitter clients out here, and we’re not certainly professional UI designers, but we do realize that there is something to be said for working towards a visually pleasing app, even in this relatively staid world of GIS apps.  Look for all our products to start looking and feeling better in future releases.

Oh, and it’s not just the look-and-feel that have gotten the reboot.  You can’t really see it, but under the covers we’re using another new Microsoft .NET technology called WPF (Windows Presentation Foundation) to build GeoViewer.  The WPF Framework gives us programmers a much better way of expressing and writing the user interface logic – gone are the days of event loops and MFC and WM_PAINT messages: we now use XML to control the application’s appearance and elegant property bindings to control the behavior.

And C#
Since we’re embracing .NET technologies for our Windows applications, we’ve also had to make the switch from C++ to C#.  It’s a happy new world to be in – expressive syntax, rich API, and a garbage collector you can trust.  Some time ago we tried programming under .NET using Managed C++, but it was pretty painful.

Of course, our underlying SDKs will remain as C++ libraries.  We’ve learned enough P/Invoke to be able to write our GUI front ends in C# and have them interop over to the C++ layer for the hard stuff.

Upcoming
We’re pretty jazzed out here about being able to break out of the old InstallShield-MFC-C++ world and use some new technologies that allow us to churn out better products faster.  Look for more goodness as we post more releases in the coming months.

LizardTech was one of the sponsors of an OSGeo code sprint in Toronto last week. Mike Rosen and I were both fortunate enough to attend and spend some quality time with twenty of the brightest minds of the open source geo world, folks working on everything from GDAL to MapServer to PostGIS to OpenLayers.

Perhaps the most visible result from the LizardTech side of things was some performance analysis that Frank Warmerdam, Chris Schmidt, and I did on GDAL’s use of MrSID – which should eventually help everyone downstream of GDAL, notably MapServer.

Paul Ramsey (mine host) and Chris both blogged daily about the event. Perry Nacionales and Paul have also posted some pix on flickr, including some of us all unwinding at Le Hockey (Hey, look! Real live Canadians, in their natural habitat!).

And, though we might have the grace to blush just a little, we’re not too modest to nonetheless link to Chris’ stirring post on using GDAL with our D-SDK.

Update: 13 July 2009. Read an article about the OSGeo code sprint in Toronto by Michael Gerlek here. – mdf

Image courtesy of Perry Nacionales

The “Call for Presentations and Workshops” for GeoWeb 2009 has just been announced. The theme of this year’s conference is “Cityscapes”: the geography of urban environments, BIM-CAD-GIS convergence, 3D modeling.

LizardTech has been fortunate to be involved with this conference for a number of years now, and so we’re once again looking forward to July and another fun week with our neighbors to the north.

This sort of email comes across our desks pretty regularly:

I’ve got a GeoTIFF image that has some transparent background areas. When I view it with the Acme GIS Viewer, those transparent areas look black, as they should. But when I compress the image with MrSID, those areas aren’t all black anymore – they look all dotty, spotty and mottled! What am I doing wrong?!

This is not a bug, and you’re not doing anything wrong either: it is, alas, a part of the compression process. Long ago when your mother told you that you would have some good days and some bad days, this is what she meant.

Allow me to explain.

Consider Exhibit A:

We see an image that has been rotated several degrees and the exposed triangular areas in the corners are a solid color. These corner pixels are intended to be treated as transparent – the term of art here is “NODATA” – and it just so happens that I chose to present the image to you against a black background. If I chose to present the exact same image to you against a fuchsia background, you’d get Exhibit B:

NODATA pixels are an essential part of the mosaicking process. Were we to construct a mosaic by placing the Exhibit A image on top of another adjoining second image, which was similarly rotated, our transparent areas allow the underlying (“real”) pixels from the second image to show through.

All pixels in an image have to be assigned numerical color values – even the NODATA ones. In Exhibit A, the guy who encoded the image happened to choose black, i.e. (0,0,0), which is pretty typical, but he could have also chosen mauve or puce. The NODATA value is encoded in the image’s header somewhere, so that when a program wants to display the image, it knows that any pixel with that special NODATA color should not be shown and instead let whatever pixel is “underneath” shine through. Nine times out of ten, that will be black – but it could be a fuchsia background or the pixels from another image tile.

(There’s a smart guy in the back of the room mumbling about alpha channels and using shapefiles for masks and stuff like that. Thank you, yes, thank you, those are also ways to mosaic images too, but we’re not using that method today. Let’s move on, please, we have a lot to cover today.)

Right. So far so good.

Now for the tricky bit.

Let’s say you compress Exhibit A to MrSID with a compression ratio of 30:1. The output image looks almost identical to the input image – great. But look at Exhibit C, in which that same output image is displayed against a white background:

“Yikes!” you say. “What am I doing wrong?!” you ask.

Because we compressed the image at 30:1 we’re doing lossy compression, meaning that each pixel’s color values in the output image will be close to, but not exactly equal to, the corresponding pixel’s color values in the input image. The compression process will change some pixels by just a few units, such as changing a (0,0,0) pixel to a (1,0,2) pixel. We won’t get into the algorithmic details of how and why the compression does this, we’ll just assert that it does and tell you that it happens most often at the “edges” of images or when you’re looking at zoomed-out, reduced-resolution versions of images. Fortunately, this change is typically imperceptible to the casual observer.

Unfortunately, the replacer of NODATA pixels is not a casual observer. The replacer is rather tetchy, actually. Keen eye for detail. Type A, O/C, that sort of thing. Which means that it’s gonna treat only the (0,0,0) pixels as transparent and none others. Which means that those pixels that got nudged up to being (1,0,2) and (0,1,1) and such are gonna stay right where they are. Which means that our putative, erstwhile transparent layer is gonna look funny. Especially at the edges of the images or when you’re looking at zoomed-out, reduced-resolution versions of images.

It’s not a bug in MrSID, and you’re not doing anything wrong: it is an unfortunate but necessary side-effect of the compression algorithms we use. Such side-effects are called “artifacts”, and we call this kind of artifact “speckling”.

All is not completely lost, however.

One way to alleviate the problem is to get the NODATA replacer to relax a little – loosen its tie, roll its shoulders a few times, take a couple deep cleansing breaths. Instead of only looking at the (0,0,0) NODATA pixels, it can use a “fuzzy” replacement algorithm, meaning that those pixel values relatively close to (0,0,0) will also be replaced, like (1,0,2) and (0,1,1). Our Express Server, for example, allows you to use this feature when mosaicking layers and even control the “degree” of fuzziness to use. (See the manual for details.) This works great, but keep in mind that you’re replacing more pixels now, and you might have pixels with value (0,1,1) that are supposed to be actual data, such as a dark shadow or black rooftop: these will get treated as NODATA and allow the underlying background data to show through.

Another approach is to use the “despeckle tool” in GeoExpress 7, as described in an earlier blog post. This new feature allows you to handle the whole issue a little more cleanly at encode time, or to clean up existing imagery you’ve already encoded. It works by attempting to find the “real” edges of images like those in Exhibit A, or more complexly polygonal mosaics, and then inhibiting the compression algorithms from touching pixels outside of those edges. Essentially what the propeller-head in the back of the room was going on about earlier. It is not always perfect, but it works for most cases. Again, see the manual for details.

Next time, perhaps, we’ll talk about some other cool artifacts like “ringing” and “grout”. Thanks for tuning in.

-mpg