[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDFJava #OSY-357654]: delays in opening remote NCML file

My speculation is that this just a very big file and
the cost is transmission + read time + time to convert to
CDM on the client. Does that seem reasonable,
or am I missing something. 

> I get ~7 seconds in the browser. Same with CURL
> 
> -Jeff
> 
> 
> On Sun, Dec 3, 2017 at 12:54 PM, Mapes, Brian Earle <address@hidden>
> wrote:
> 
> > A few seconds to return this, for me
> >
> > Error {
> >     code = 403;
> >     message = "Request too big=58988.094912 Mbytes, max=500.0";
> > };
> >
> >
> >
> >
> > address@hidden> wrote:
> >
> > How long does the browser take if you access the .dods e.g.
> > https://na01.safelinks.protection.outlook.com/?url=
> > https%3A%2F%2Fdataserver.nccs.nasa.gov%2Fthredds%2FdodsC%
> > 2Fbypass%2FCREATE-IP%2Freanalysis%2FCFSR%2F6hr%
> > 2Fatmos%2Fpsl.ncml.dods&data=02%7C01%7Cbmapes%40rsmas.miami.edu%
> > 7Cb669f01390d34fe8a83108d53a871659%7C2a144b72f23942d48c0e6f0f17c4
> > 8e33%7C0%7C0%7C636479274232120145&sdata=TJcc4icYvzL6%
> > 2Fb59dRDbNTtWQdx14zuCUJBoQciV42Q%3D&reserved=0
> >
> >
> > I'm tracking a performance problem down for Brian Mapes (cc'ed). He has an
> > ncml file that aggregates a number of ncml files and it takes a very very
> > long time to open the file.
> >
> > Just looking at one of the NCML links that is being accessed:
> >
> > https://na01.safelinks.protection.outlook.com/?url=
> > https%3A%2F%2Fdataserver.nccs.nasa.gov%2Fthredds%2FdodsC%
> > 2Fbypass%2FCREATE-IP%2Freanalysis%2FCFSR%2F6hr%2Fatmos%2Fpsl.ncml&data=02%
> > 7C01%7Cbmapes%40rsmas.miami.edu%7Cb669f01390d34fe8a83108d53a871659%
> > 7C2a144b72f23942d48c0e6f0f17c48e33%7C0%7C0%7C636479274232120145&sdata=
> > 5EBQlAWULuqQldFMKTm%2FJGYOTIKIvSAdVfL%2Fip6pDRA%3D&reserved=0
> >
> > In the browser it is sub-second to access the .das of this file. However,
> > running CURL or the IDV's IOUtil.readcontents it takes ~10 seconds to
> > fetch.  Running ucar.nc2.dt.grid.GridDataset.open it takes ~ 80 seconds to
> > open the link
> >
> > Any ideas?
> >
> > -Jeff
> >
> >
> >
> > =Dennis Heimbigner
> >  Unidata
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: OSY-357654
> > Department: Support netCDF Java
> > Priority: Normal
> > Status: Open
> > ===================
> > NOTE: All email exchanges with Unidata User Support are recorded in the
> > Unidata inquiry tracking system and then made publicly available through
> > the web.  If you do not want to have your interactions made available in
> > this way, you must let us know in each email you send to us.
> >
> >
> >
> > *********************************************
> > Brian Mapes, Professor
> > Department of Atmospheric Sciences
> > Meteorology and Physical Oceanography Program
> > RSMAS, University of Miami
> > 4600 Rickenbacker Causeway
> > <https://maps.google.com/?q=4600+Rickenbacker+Causeway+%0D+Miami,+FL+33149&entry=gmail&source=g>
> > Miami, FL 33149-1098
> >
> > phone: (305) 421-4275
> > fax: (305) 421-4696
> > email: address@hidden
> > Web: http://www.rsmas.miami.edu/users/bmapes/
> > **********************************************
> >
> >
> >
> >
> >
> >
> >
> 
> 

=Dennis Heimbigner
  Unidata


Ticket Details
===================
Ticket ID: OSY-357654
Department: Support netCDF Java
Priority: Normal
Status: Open
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.