[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #DEE-999161]: An question on NetCDF large file support



Hi,

> I'm a meteorologist from China. I used the NetCDF C interface to create
> a large NetCDF file and meet a question, I need some help.
> 
> My NetCDF libery version is 3.6.2, OS is Win7(32-bit).  The structure(head
> file) of my NetCDF file is:
> 
> lon=1650
> lat=1100
> level =1
> time=240
> short qpf_XXX (time, level,lat, lon)  ( XXX from 1 to 7, total 7 Vars)
> 
> So, the total size of NC file is about 5.67 GB.  When I create a nc
> file using the "classical" mode , I meet the -62 error code, but when
> I create the file using the "64-bit offset" mode, I succeed.
> 
> I viewed the following page
> 
> http://www.unidata.ucar.edu/software/netcdf/docs/netcdf/Large-File-Support.html
> 
> and got some helpful messages, but I still have some puzzles on the last
> form ( the red words) of the website.
> 
> My Question is:
> 
> In my understanding, the max size of one variable in my NC file is
> 1650*1100*240*2= 871200000 byte( about 0.81 GB ) , which is less than 2
> GiB , why can't I create the NC file by "classical" mode? Is it related
> to the items "Max Number of Fixed Vars > 2Gib" and "Max Record Vars w/
> Rec Size > 2 GiB"?

That page also has a link to a more detailed explanation of size limitations
of classic-model files:

  http://www.unidata.ucar.edu/netcdf/docs/netcdf/Classic-Limitations.html

which includes this:

  If you don't use the unlimited dimension, only one variable can exceed
  2 GiB in size, but it can be as large as the underlying file system
  permits. It must be the last variable in the dataset, and the offset
  to the beginning of this variable must be less than about 2 GiB.

You data is not following this last rule, because the offset from the
beginning of the file is more than 2 GiB for the variables qpf_004,
qpf_005, etc.  Specifically, the offset of qpf_004 from the beginning 
of the non-record data is at least 3*1650*1100*240*2 = 2.6136 GB.

Please let us know if this doesn't answer your question ...

--Russ

> =========Below is the form on the website===========================
> 
> 
> | Limit | No LFS | v3.5 | v3.6/classic | v3.6/64-bit offset | v4.0/netCDF-4
> |
> | Max File Size | 2 GiB | 8 EiB | 8 EiB | 8 EiB | ??
> |
> | Max Number of Fixed Vars > 2 GiB | 0 | 1 (last) | 1 (last) | 2^32 | ??
> |
> | Max Record Vars w/ Rec Size > 2 GiB | 0 | 1 (last) | 1 (last) | 2^32 | ??
> 
> |
> | Max Size of Fixed/Record Size of Record Var | 2 GiB | 2 GiB | 2 GiB | 4 GiB 
> | ??
> |
> | Max Record Size | 2 GiB/nrecs | 4 GiB | 8 EiB/nrecs | 8 EiB/nrecs | ??
> |
> 
> Looking forward to your reply and thank you very much!
> 
> Best Wishes!
> 
> B.H. Chen
> 2013-7-4
> 
> 
Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: DEE-999161
Department: Support netCDF
Priority: Normal
Status: Closed