Decode speed decreasing when moving from 6 to 7 SDK version

[jet55]

Hello all)

I made a dll that decodes given area (rectangular) with given scale from SID map and save it as bitmap. I used Geo DSDK 6 version (don't remember exact build, I lost this distributive :( ).
Then I updated to latest SDK (7). I made neccesary changes and now my dll works with new SDK.

But it takes several times more time(!!) to do the same operation with SDK v7!

I am using pipeline with MrSIDImageReader, LTIColorTransformer, LTIMultiResFilter and LTIViewerImageFilter.
My code is based (almost equal :) ) on viewerWIN32 example from SDK.

Thanks in advance for any help!

[gat]

Could we see your source code?
Is this the case for all of your sid files or just some?
Is this with release libs?

[jet55]

Sorry for late answer.
I have uploaded code of my library: [url]h_t_t_p://w_w_w.sendspace.com/file/sl5dsd[/url]
This case happens with all my SID maps, with release and debug versions.

[kirkoman]

Hello jet55.

I've reviewed your code and nothing obvious is standing out. Looks pretty simple and, as you said, similar to the sample viewer code we ship with the SDK.

We do not expect performance to be slower by "several times" with Geo DSDK 7, in fact it should not be slower at all. So I would like to help you figure this issue out if I can.

Can you confirm that the DLL is the only component that you are changing in your software? Is it possible this may be caused by changes in how the DLL is being called?

Can you confirm that in Release mode you are linking to the Release version of our libraries?

How large are the images you are working with?

Does performance of the viewerWIN32 sample seem comparable with the performance of your application on a given image?

Thanks,

Kirk

[jet55]

Hello Kirk.
Thank you for helping!

- I didn't change software or way of calling my dll, only changes in dll have been made to feet v7 SDK.

- Yes, I'm using release versions of libs.

- My maps quite small: ~4mb, 10000 * 10000

- viewerWIN32 uses only (power of 2) magnifications; so it is fast. But I use different magnification(depends on input scene parameters) so I can't compare.

The main time loss occurs in this line:
sts = mReader->read( clippedScene, bufferData ) ;

[kirkoman]

If you are viewing grayscale data, you should add the colortransform to the pipeline *after* the multires filter. No point interpolating 3 identical bands. Also you will transform fewer pixels.

As an additional enhancement, you could take off the colortransform altogether and handle the 1-banded case in CreateDIB(). The Viewer filter already coerces the pipeline into RGB or Greyscale.

Those changes would make the read() call faster in certain situations. But I'm still confused as to why it is slower than your previous DLL. Do you have the code for it so we could do a side-by-side comparison?

Maybe more info on your data... Can you call mrsidinfo (a binary shipped with the SDK) on one of your files and post the results?

[jet55]

The code of previous lib is the same except for changes in initialization:

/////////
LTFileSpec * fileSpec = new LTFileSpec(mFileName);
mReader = new MrSIDImageReader(*fileSpec);
sts = mReader->initialize();
/////////

and filters creation:

/////////
LTIColorTransformer *clrtrans = new LTIColorTransformer( mReader, LTI_COLORSPACE_RGB, 3, true );
sts = clrtrans->initialize();
/////////
LTIMultiResFilter *resFilter = new LTIMultiResFilter( mReader, true );
sts = resFilter->initialize();
/////////
LTIViewerImageFilter *viewerFilter = new LTIViewerImageFilter(mReader, true, true, true);
sts = viewerFilter->initialize();
/////////

My maps are colored (aerophotography)
basic image info:
format: MrSID/MG2
width: 10000
height: 10000
nband: 3
color space: RGB
datatype: uint8
precision: 8
dyn range min: (natural)
dyn range max: (natural)
background color: (none)
nodata color: (none)
nominal size: 300000000 (286 MB)
physical size: 3732283 (4 MB)
compression ratio: 80,4 : 1
mag range: 0,003906 to 1048576,000000
metadata records: 19
num AOI regions: 0

geo coordinate info:
X UL: 1475000,250000
Y UL: 6849999,750000
X res: 0,500000
Y res: -0,500000
X rot: 0,000000
Y rot: 0,000000

geo points (pixel centers):
upper left: (1475000,250000, 6849999,750000)
upper right: (1479999,750000, 6849999,750000)
lower left: (1475000,250000, 6845000,250000)
lower right: (1479999,750000, 6845000,250000)
center: (1477500,000000, 6847500,000000)

MrSID image info:
number of levels: 8
is locked: false
file version: 1.0.1.

MrSID/MG2-specific image info:
is dithered: true
block size: 512

[kirkoman]

jet55,

I have built a driver for your dll and backported the dll code to v6 per the changes you described. Then I did a side-by-side comparison of a 5k x 5k MrSID file (Gen 2 like yours) and did not find a discernible difference in decode speed (if anything it was faster).

So at this point the evidence is pointing to code outside of the DLL and our SDK. If you can send me a concise programmatic repro of the issue I'll be happy to look into it further.

Good luck.

Kirk.


/////// output from my tests

C:\src\Support-Code>Release\DecodeTime.exe 10tet279047.sid 4500
start times are 129162701075073277, 0, 156250
opening 10tet279047.sid... SUCCEEDED
decoding full extent to 4500 pixel bitmap... SUCCEEDED
end times are 129162701147275099, 17187500, 55156250

elapsed real: 7.220182s
elapsed syst: 1.718750s
elapsed user: 5.500000s

C:\src\Support-Code>Release-v7\DecodeTime.exe 10tet279047.sid 4500
start times are 129162701278394858, 0, 0
opening 10tet279047.sid... SUCCEEDED
decoding full extent to 4500 pixel bitmap... SUCCEEDED
end times are 129162701347939903, 15468750, 54062500

elapsed real: 6.954504s
elapsed syst: 1.546875s
elapsed user: 5.406250s

C:\src\Support-Code>

[jet55]

Hello Kirk,

I have made some measurements, results are below:

////////////////////
full scene --> 4500 x 4500 bmp
v6 : 14,547313s
v7 : 14,823814s

full scene --> 3500 x 3500 bmp (!!!)
v6 : 4,341540s
v7 : 14,482481s

full scene --> 1800 x 1800 bmp (!!!)
v6 : 1,275267s
v7 : 3,428841s

full scene --> 1500 x 1500 bmp (!!!)
v6 : 1,168568s
v7 : 3,327256s

full scene --> 1400 x 1400 bmp
v6 : 1,136614s
v7 : 1,195086s

full scene --> 1000 x 1000 bmp
v6 : 1,053794s
v7 : 1,064034s

full scene --> 900 x 900 bmp (!!!)
v6 : 0,372789s
v7 : 1,071960s
////////////////////////////

"(!!!)" marks the differences in times. Resulting bitmap also were different when binary compare them.

Time was measured using QueryPerformanceCounter function, averaged out of 10 tries.

My config is: Pentium 4 3.00Ghz, 1.49GB RAM, Windows XP Professional SP3

[kirkoman]

Aha. Thanks - those numbers shed some light. With your scene sizes I can reproduce the issue now. Comparing the code between SDK releases, it looks like the behavior of the LTIMultiResFilter has changed.

In v6, it would resample the scene from whatever pyramid level the desired magnification was closest to.

In v7, it always chooses the higher-resolution pyramid level.

Thus, in v6 the 3500-pixel scene was getting upsampled from the 2500-pixel pyramid level, whereas now (in v7) it is getting downsampled from the 5000-pixel pyramid level.

This implies that the old way was yielding lower-quality, pixelated decodes at certain magnifications. So a speed/quality tradeoff has been made in the v7 release.

[jet55]

I see... Can it be controlled somehow by means of SDK?
High quality is good, but the difference in quality is quite a little and so less important than 3-times slower decoding speed :(

[kirkoman]

I need to revise my previous post after more research. It turns out v7 does not *always* choose the higher-res level... you just need to be much closer to the lower-res level before it will get used. That threshold is at about the 20% mark instead of the 50% mark.

So as an example, the v6 code will start up-sampling from the 2500-pixel pyramid level once the requested scene gets below 3750. The v7 code does not start up-sampling from that same level until the scene gets below 2973.

For random scene sizes, then, you will see the difference about 30% of the time.

Unfortunately this is hardwired and cannot be controlled via the API. I'm sorry this change turned out to be problematic for you. What you could do is implement your own resampling filter. That would even give you control to dial back that speed/quality threshold further than it was in v6 if that would be desirable.

[jet55]

Thanks for clarification of behavior!
Manual resampling is a good (and obvious) solution for me.