Hi Dr. Panitz, > with great interest I read about the problems with the short read > described in the ticket #PDZ-683250. But I have to admit that I did > not understand everything written there due to "missing background". > > However, I would like to point out that I encountered the same problem > when running a regional climate model on a CRAY XC40. The model reads > data from NetCDF files that are stored on a Lustre Filesystem. > > The problem is identical to that described in the ticket. > The content of the files is not corrupted. > The typical size of a file is about 40 Mbyte. > > I use the following module that had been installed by CRAY staff: > cray-netcdf/4.3.2 > > Unfortunately, I cannot say how NetCDF had been installed (which options > had been used). > > Of course, I am in contact with a member of the CRAY staff. His assumption > is that the problems only occurs if "there is lot of traffic on the > filesystem". This assumption seems to be consistent with my experience > that the problem does not occur with every read but in an unsystematic > manner. > > Thus, I would be very interested in any response to the ticket mentioned > above and in any solution. Furthermore, I have the questions: > > - does any option exist for NetCDF installation that could solve the > "short read" problem? No, sorry, there is currently no such option. Although we don't currently test each release on a Cray platform, it's possible that an old configure option we used to support, "--enable-ffio", might work for platforms that support FFIO library when --disable-netcdf-4 is also specified , but our test platforms don't support FFIO and get compile-time errors for ---enable-ffio.. > - would the definition of the "bufrsize" parameter in the call of > "nf90_open" help? If yes, which size would be appropriate, > respectively, how can it be determined? I don't know if it would help, but you might try a smaller size than you're currently using. I know of no way to determine whether a smaller size would work or what it should be other than testing with an example that demonstrates the problem, and that may be impractical if it requires generating a high I/O load to see the problem. We would like to have a solution for this problem, but have no expertise in tuning netCDF on the Cray for Lustre file systems. I see that Googling "cray lustre netcdf" turns up lots of web sites and even an HDF5 paper on this subject: "Tuning HDF5 for Lustre File Systems" from The HDF Group. > Thank your very much for any help. Sorry I couldn't provide any useful help. --Russ > Best regards > > Hans-Juergen Panitz > > > -- > Karlsruher Institut fÃr Technologie (KIT) > Institut fÃr Meteorologie und Klimaforschung > Bereich TroposphÃre (IMK-TRO) > > Dr. Hans-JÃrgen Panitz > > Hermann-von-Helmholtz-Platz 1 > D-76344 Eggenstein-Leopoldshafen > > Phone: xx49-(0)721-608 22802 > Fax : xx49-(0)721-608 24377 > E-Mail: address@hidden > > www.kit.edu > www-fzk.imk.uni-karlsruhe.de > > KIT - UniversitÃt des Landes Baden-WÃrttemberg und > nationales GroÃforschungszentrum in der Helmholtz-Gemeinschaft > > Russ Rew UCAR Unidata Program address@hidden http://www.unidata.ucar.edu Ticket Details =================== Ticket ID: KZJ-320086 Department: Support netCDF Priority: High Status: Closed
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.