[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #SIY-529931]: Merging Netcdf files



Hi Olumide,

Since I responded to a separate emailed version of this question, I'm including
that response here, so the question and answer can be found with a search of
our support archives.  Below that, Dennis Shea's detailed response is included.

> I have weather datasets from the TRMM 3-hourly data in Netcdf format. I am
> looking for ways to merge these multiple netcdf files that have same
> variables and are in sequential time periods. Such as 3B42.199801.3hr_V7.nc,
> 3B42.199803.3hr_V7.nc, 3B42.199802.3hr_V7.nc.
> 
> I would like to merge these files together with time component added in the
> file. I would appreciate your suggestions.
> 
> I would also like you to help me CDO scripts to merge these Netcdf files
> together.
> 
> Thanks!

Although CDO scripts might be able to do what you want, the NCO tool named 
ncrcat should work, 
assuming each input file has a time dimension and time is a record dimension.

Dennis Shea, an expert at processing TRMM data, might be able to provide more 
advice about using 
NCL or NCO tools to merge the 3-hourly TRMM files.  I'm Cc:ing him in this 
reply.

--Russ

Dennis Shea's reply:

[1] Assuming that the ' 3B42.199801.3hr_V7.nc ' has an unlimited time
    dimension, the NCO operator 'ncrcat' is the recommended tool to
    concatenate multiple files.  I am not an expert on the CDO but, in
    my opinion, the NCO are better suited for concatenating files than
    the CDO.

     %> ncrcat -O -h   3B42.1998*.3hr_V7.nc    V7_3hr.3B42.1998.nc

    would efficiently create one file with all variables. Note ... a
    file containing all 3hrly data for one year would be *huge*. There
    would be 8*365 time steps on a 400x1440 grid for 6 variables!
    Figure it out for yourself.

[2] One month of 3-hrly TRMM precip contains (say) 248 (8*31) time
    steps and is about 571MB. This includes *only* precipitation and
    no other variables.  NO: 'err', 'source', 'hqp' , 'irp',
    'satobstime' variables.
     
[3] How do I know? I created monthly files containing the
    precipitation variable *only*.  A modeling project I am working on
    required some additional information so I used an NCL script to
    perform some tasks and then invoked an NCO operator from with ine
    NCL script to create the files. A sample dump follows:


    %> ncdump -h 3B42.200301.3hr_V7

netcdf \3B42.200301.3hr_V7 {
dimensions:
        time = UNLIMITED ; // (248 currently)
        lat = 400 ;
        lon = 1440 ;
variables:
        double time(time) ;
                time:calendar = "gregorian" ;
                time:units = "hours since 1997-01-01 00:00:0.0" ;
        double lat(lat) ;
                lat:long_name = "Latitude" ;
                lat:units = "degrees_north" ;
        double lon(lon) ;
                lon:long_name = "Longitude" ;
                lon:units = "degrees_east" ;
        int date(time) ;
                date:units = "yyyymmdd" ;
                date:long_name = "current date" ;
        int datesec(time) ;
                datesec:units = "seconds" ;
                datesec:long_name = "current seconds of current date" ;
        int yyyymmddhh(time) ;
                yyyymmddhh:units = "current date" ;
                yyyymmddhh:long_name = "current date with hh" ;
        float precip(time, lat, lon) ;
                precip:avg_period = "0000-00-00 01:00:00" ;
                precip:delta_t = "0000-00-00 03:00:00" ;
                precip:comments = "Unknown1 variable comment" ;
                precip:long_name = "precipitation" ;
                precip:units = "" ;
                precip:time_statistic = "instantaneous" ;
                precip:missing_value = -9999.9f ;
                precip:_FillValue = -9999.9f ;

// global attributes:
                :creation_date = "Mon Mar  3 09:28:42 MST 2014" ;
                :Conventions = "None" ;
                :time_info = "\n",
                        "NASA:                                                  
                  \n",
                        "The 0Z in the filename is the mid point of a 3 hour    
                  \n",
                        "interval beginning at 22:30Z (1/14/1998) and ending at 
1:30Z (1/15/1998).\n",
                        "The precipitation is the average rain rate for this 
time period.         \n",
                        "" ;
                :source_URL = "\n",
                        
"http://mirador.gsfc.nasa.gov/cgi-bin/mirador/collectionlist.pl?keyword=TRMM\n";,
                        "" ;
                :source_data = "\n",
                        "Download netCDF from NASA MIRADOR: Search for:\n",
                        "TRMM 3-Hourly 0.25 deg. TRMM and Other-GPI Calibration 
Rainfall Data (TRMM_3B42)\n",
                        "" ;
                :title = "TRMM: 3B42: V7" ;
}

[4] gzip'd versions of these files are available via ftp. Each file is
    about 51MB 

ftp ftp.cgd.ucar.edu
anonymous
email
cd /archive/PRECIP
prompt
mget 3B42.*.3hr_V7.nc.gz
quit

If you want only specific years and or months. you can use linux/unix
'wild' notation


There is also a simgle file which contans *monthly mean* precip

TRMM_3B43V7.19980101-20140201.nc.gz

============================================

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: SIY-529931
Department: Support netCDF
Priority: Normal
Status: Closed


NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.