[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #LRS-412716]: alignment problem with compound types

Howdy Jeff!

How're things at NOAA these days?

I don't yet know the answers to your questions, but I am working on it!

I can say that netCDF-4 was not designed or tested to use other than the 
default C compiler layout. However, that doesn't mean I can't change it to do 
so. I just need to wrap my head around it a bit more.

Since your first email on the topic of nested types in cross-platform 
situations, I have extensively refactored the code dealing with HDF5 types. Not 
only have I removed the nested type bug you initially encountered, but also 
provided a significant performance improvement for those using user-defined 
types, and also removed many (but perhaps not all) memory leaks relating to 
HDF5 typeids not being closed.

If you take a look in the file libsrc4/nc4file, in the function commit_type(), 
you can see how I commit types in netCDF-4. As you can see I don't use H5Tpack. 
(Should I? I am about to try to figure that out...)

FYI: the code that reads in a file, including its user-defined types, is in 
libsrc4/nc4file.c.The type information is read by function read_type(). 
Comments on the code are also most welcome. So no need to guess what mistakes 
I've made - you can look at the code and read them directly! ;-)

But let me ask you a question: why do you want to do this anyway? That is: why 
change the default compiler packing? Just curious. I agree that it should work, 
I just wonder why...

Aside from the packing issue, I believe your first bug report is resolved, 
correct? That is, without changing the packing, it should now handle 
cross-platform reads of the nested structure you are using. I have a test that 
does this, which used to fail and now passes everywhere.

While testing the recent changes I also came across a HDF5 typeid bug, which 
Quincey, one of the HDF5 team, is now working on. It occurs for me in a 
compound containing an array of vlen of compound. I am working with Quincey to 
see if there is a way for netCDF-4 to avoid this bug - until then at least some 
nested types will not be readable by some platforms. We're working to determine 
what nested types, and what platforms are involved.

I have still more tests to add on this issue, and Russ has also undertaken to 
add a bunch of complex, nested type tests to ncdump before the next release, 
and I will then turn some or all of them into xplatform tests too, by running 
the on a solaris system, distributing the resulting netcdf-4/HDF5 file, and 
adding a test to ensure that every build platform can correctly read such a 
file.

Please be patient with us - we are all coding as fast as we can. I know that 
because yesterday we all had a "retreat." (Not nearly as interesting as it 
sounds a first. I was thinking retreats like Napoleon from Moscow, or the 
British from Afghanistan. But turns out this kind of retreat is different. You 
just sit in a room and people talk. All day long.)

Anyhoo, at the retreat I learned that we were all programming as hard as we 
can, and I guess I already suspected that. But I guess if it was supposed to be 
fun they wouldn't call it a "retreat."

Thanks,

Ed

PS - Are you coming to the netCDF workshop in August?

Ticket Details
===================
Ticket ID: LRS-412716
Department: Support netCDF
Priority: Normal
Status: Open