ZLIB_UNCOMPRESS limited to 4GB?

[superchromix]

hi,

I am trying to access data that is stored in a zlib compressed format in a set of existing files. The data size in some of the files is > 4GB.

For the smaller files, I can use ZLIB_UNCOMPRESS to access the data successfully. However, when I try ZLIB_UNCOMPRESS on the larger datasets, IDL crashes hard (the program just closes).

I've tested ZLIB_COMPRESS with large datasets, and this throws an error when the input data is > 4GB.

So, are both of these routines limited to 4GB data? If so, why?

IDL Version 8.3, Microsoft Windows (Win32 x86_64 m64). (c) 2013, Exelis Visual Information Solutions, Inc.

My machine has 32GB of RAM... and I have also tested this on IDL 8.4

Any ideas on how to decompress this data in IDL?

thanks
Mark

[superchromix]

hi,

The 4GB limit is inherent to the ZLIB compress() and uncompress() routines. Instead, in order to access data this large I would need to use ZLIB inflate().

Does anyone have IDL code or a DLM which uses ZLIB inflate() ?

thanks
Mark

[Heinz Stege]

Post by superchromix
The 4GB limit is inherent to the ZLIB compress() and uncompress() routines. Instead, in order to access data this large I would need to use ZLIB inflate().
Does anyone have IDL code or a DLM which uses ZLIB inflate() ?

I don't know, if this helps: 7z from http://www.7-zip.org/ has a
command line version that can be used together with IDLs spawn
procedure. The 7z extract command can extract files from a zip-file
greater than 4 GB.

I did a test with a 0.7 GB file in a 6.6 GB zip file on my Windows PC.

Cheers, Heinz

[superchromix]

Thanks, but unfortunately this is not a ZIP file - it is compressed data stored within another file format.
Mark