When reading hdf5 files compressed with gzip, error occured

[CaptainDuke]

HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
#000: external/hdf5/src/H5Dio.c line 199 in H5Dread(): can't read data
major: Dataset
minor: Read failed
#1: external/hdf5/src/H5Dio.c line 601 in H5D__read(): can't read data
major: Dataset
minor: Read failed
#2: external/hdf5/src/H5Dchunk.c line 2259 in H5D__chunk_read(): unable to read raw data chunk
major: Low-level I/O
minor: Read failed
#3: external/hdf5/src/H5Dchunk.c line 3624 in H5D__chunk_lock(): data pipeline read failed
major: Dataset
minor: Filter operation failed
#4: external/hdf5/src/H5Z.c line 1301 in H5Z_pipeline(): required filter 'deflate' is not registered
major: Data filters
minor: Read failed

[CaptainDuke]

However, I can use command line h5dump to read such compressed h5files successfully.

[CaptainDuke]

I think it's because hdf5.BUILD and WORKSPACE files doesn't compile hdf5 together with 'deflate' filter, but only orign hdf5-1.10.5.tar.gz.

The commmand line should be like this:
./configure --enable-cxx --enable-fortran --with-zlib=/path/to/zlib/include,/path/to/zlib/lib

[yongtang]

@CaptainDuke Do you have a sample file I can test? I think it could an easy fix but I may need a sample file to test it.

[CaptainDuke]

@CaptainDuke Do you have a sample file I can test? I think it could an easy fix but I may need a sample file to test it.

Here's a test file, I just compress the /io/tests/test_hdf5/tdset.h5 with gzip
with h5py.File('compressed_h5.h5', 'w') as output_f: output_f.create_dataset('/dset1', data=h5f['/dset1'][()], compression='gzip')

compressed_h5.zip

[CaptainDuke]

@yongtang
Thanks for your contribution, I will compile new version of this code by myself.

By the way, what's the meaning of output_shape? I found that no matter how I set this value, the output results remain unchanged. Only 'batch' matters.

[yongtang]

@CaptainDuke PR #311 has been merged, you give tensorflow-io-nightly a try. Also new version (v0.7.0) of tensorflow-io will be rebased soon.

For output_shape, it used in DatasetV1, in DatasetV2 there are some changes that makes it less strict. If you can share the code you are trying to run (maybe open another issue?) then we may take a look.