[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDFJava #MIY-264031]: gfs 384 data

Hi Alex:

1) The first time will be slow because it needs to write an index. Subsequent 
opens should be fast. Version 4.3 will be significantly faster on subsequent 
opens.

2) Still, 2 secs for 35K is way too slow. Are you sure of that timing?

3) The process is I/O bound, not CPU. So if those timings are real, you should 
look there.

John

> Good afternoon Sean,
> 
> Thanks again for the source and the advise.  It is much appreciated and got
> me up and running.
> 
> Just want to ask you, based on your experiencem  My grib files are about
> 35K, only have lat/long/time/temp, I am running on the 3.7GHz quad core
> 64bit box with plenty on memory, but it still takes about 2 seconds to open
> the file and get a data set per file.
> 
> Any advise on how to expedite the process?  Is there an option to utilize
> the processor more?
> 
> Thank in advance,
> 
> Alex.
> 
> address@hidden> wrote:
> 
> > Greetings Alex,
> >
> > I've attached a sample java program that goes to our publicly accessible
> > data server and grabs temperature value from the CONUS GFS 80KM grid, and
> > then prints that value to the screen. To extend this simple program to data
> > into an array, which is what you would want to do, check out the following
> > site:
> >
> > https://www.unidata.ucar.edu/software/netcdf-java/tutorial/NetcdfFile.html(particularly
> >  the section on "Manipulating data in Arrays")
> >
> > and
> >
> >
> > https://www.unidata.ucar.edu/software/netcdf-java/tutorial/NetcdfDataset.html(NetcdfDataset.openFile
> >  is what the example java file uses)
> >
> > as well as the general info in the tutorial here:
> >
> > https://www.unidata.ucar.edu/software/netcdf-java/tutorial/index.html
> >
> > Cheers!
> >
> > Sean
> >
> > > Hey,
> > >
> > > Thanks for the links will check it out.
> > >
> > > I am downloading files from the website.  Raw data is in grib/grib2
> > format.
> > > Now that we on the topic, can you point me to the java api for random
> > > access directly on the server, so I can only reap out the fields I need,
> > if
> > > that is possible at all.
> > >
> > > Thanks in advance, Alex.
> > >
> > >
> > > address@hidden> wrote:
> > >
> > > > Greetings Alex,
> > > >
> > > > > Hi Sean,
> > > > >
> > > > > Thank you very much for the reply.  To answer your questions.
> > > > >
> > > > > Yes, I am reading gfs data.
> > > >
> > > > Where do you get the data? From a website, your own model run? Is the
> > raw
> > > > data format in netCDF, grib?
> > > >
> > > > > I am planning to compare the runs, once the data extracted.
> > > > >
> > > > > I would like to keep the program 100% java if possible, which is
> > reading
> > > > > the files, processing and saving.
> > > > >
> > > >
> > > > Keeping things 100% java should not be a problem. It may help to read
> > > > through some example programs (
> > > > http://www.unidata.ucar.edu/software/netcdf/examples/programs/) using
> > > > netCDF-Java to read files and access data, especially the following
> > example
> > > > (
> > > >
> > http://www.unidata.ucar.edu/software/netcdf/examples/programs/Sfc_pres_temp_rd.java
> > ).
> > > > If you are able to open your files and read data from the variables, it
> > > > should be relatively straightforward to dump the data you need to a
> > format
> > > > of your choice.
> > > >
> > > > Cheers,
> > > >
> > > > Sean
> > > >
> > > > >
> > > > >
> > > > > Regards,
> > > > >
> > > > > Alex Friedman
> > > > >
> > > > > address@hidden> wrote:
> > > > >
> > > > > > Greetings Alex,
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I am very new to the api.
> > > > > >
> > > > > > Welcome to netCDF-Java!
> > > > > >
> > > > > > >  My ultimate goal is to translate raw gfs data
> > > > > > > into a table in memory with 3 columns long, lat and temp.
> > > > > > > Is this doable using java api?  Could you point me to the right
> > > > > > direction?
> > > > > > >
> > > > > > > I am parsing the file fine and creating the dataset, I am having
> > > > problems
> > > > > > > understanding how I can output the table and what classes should
> > I
> > > > use.
> > > > > >
> > > > > > If I understand correctly, you wish to extract long, lat, and
> > temp, and
> > > > > > save the data in something like a comma delimited ASCII file? This
> > is
> > > > > > indeed possible; however, for clarification purposes, what is your
> > > > planned
> > > > > > use for the data once you have extracted long, lat, and temp? Also,
> > > > where
> > > > > > are you obtaining the GFS data? I ask because their may be a very
> > easy
> > > > way
> > > > > > to get what you desire.
> > > > > >
> > > > > > Cheers!
> > > > > >
> > > > > > Sean
> > > > > >
> > > > > > >
> > > > > > > Thank you in advance,
> > > > > > >
> > > > > > > Alex.
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > Ticket Details
> > > > > > ===================
> > > > > > Ticket ID: MIY-264031
> > > > > > Department: Support netCDF Java
> > > > > > Priority: High
> > > > > > Status: Open
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > Ticket Details
> > > > ===================
> > > > Ticket ID: MIY-264031
> > > > Department: Support netCDF Java
> > > > Priority: High
> > > > Status: Open
> > > >
> > > >
> > >
> > >
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: MIY-264031
> > Department: Support netCDF Java
> > Priority: High
> > Status: Open
> >
> >
> 
> 


Ticket Details
===================
Ticket ID: MIY-264031
Department: Support netCDF Java
Priority: High
Status: Closed