Yes, I missed it. Fortran is not my forte, so it will take me a while to understand. Did you try the nf_sync? > Just double checking the pseudo code is not attached? > > > Am 06.04.2018 um 21:22 schrieb Unidata netCDF Support: > > It is hard to figure out why c and d are so much slower without > > seeing at least some form of pseudo-code of exactly what is being > > performed. > > > > Let me take a shot in the dark and suggest that you use the > > nc_sync function to force data to be pushed to the file. This may > > change the memory usage patterns. > > > > Also, if this is a netcdf-4 file, then changing the chunking parameters > > may have an effect. > > > >> Full Name: Tobias Zolles > >> Email Address: address@hidden > >> Organization: University Bergen > >> Package Version: > >> Operating System: > >> Hardware: > >> Description of problem: > >> > >> I have a net cdf file with multiple variables which have 4 dimensions > >> x,y,z and time. The model is calculating each horizontal grid point over a > >> whole year. So it calculates first a z*time matrix. I would like to save > >> this for each grid cell in a net-cdf file instead of first creating a > >> whole x*y*z*time array which i save once. But i realize apparently that > >> this is three orders of magnitude slower than the old approach of just > >> creating the big array and saving it as once. This won't be applicable > >> due to memory constraints in the future. > >> > >> I did some testing now and created a x*y*z*time array and then saved > >> subsets of the data > >> > >> a) via a loop over time and save 365 times the x*y*z array > >> b) loop over time and z to save the x*y array > >> c) a loop over all horizontal grid points saving a time*z > >> array (analog to what we really want) > >> d) for all points *x,y,z (so loop over those) saving a vector > >> with the time dimension. > >> > >> a and b are fast and c and d are about 3-4 orders of magnitude slower. > >> > >> What can be the case for it? I found some information > >> (https://www.unidata.ucar.edu/support/help/MailArchives/netcdf/msg10905.html) > >> which didn't help. I removed the NF_unlimited statement of my time > >> dimension/variable, it is now fixed length of 365. But it didn't change > >> anything performance wise. It feels like a certain hierarchy over > >> my variables. > >> > >> I attached the fortran code of the subroutine initializing and writing > >> the net_cdfs for simplicity I removed the "normal non-dimension" variables > >> and a pseudo code of my writing function. > >> > >> I am using net-cdf-4. Use of unlimited time dimension could be beneficial, > >> but at the current state with daily data it is not necessary. > >> > >> > > =Dennis Heimbigner > > Unidata > > > > > > Ticket Details > > =================== > > Ticket ID: HCQ-327445 > > Department: Support netCDF > > Priority: Normal > > Status: Open > > =================== > > NOTE: All email exchanges with Unidata User Support are recorded in the > > Unidata inquiry tracking system and then made publicly available through > > the web. If you do not want to have your interactions made available in > > this way, you must let us know in each email you send to us. > > > > > > =Dennis Heimbigner Unidata Ticket Details =================== Ticket ID: HCQ-327445 Department: Support netCDF Priority: Normal Status: Open =================== NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.