Re: [thredds] NetCDF-4 DataReadingStrategy

  • To: "Signell, Richard" <rsignell@xxxxxxxx>
  • Subject: Re: [thredds] NetCDF-4 DataReadingStrategy
  • From: Sean Arms <sarms@xxxxxxxx>
  • Date: Tue, 6 Sep 2016 16:55:26 -0600
Hi Pete,

For versions of TDS 4.x, ncwms would be the appropriate place for the PR.
However, in the next version of TDS (5.x), we are using the next iteration
of ncwms, which comes from edal-java. The github edal-java repository,
which is where I assume the PR would go to get something in 5.0), is
managed by Jon Blower and his group at the University of Reading, so they
would likely have better input on a change like the one you propose. Just a
note that we plan on having a beta version of 5.0 ready by the beginning of
November.

Cheers,

Sean

On Thu, Sep 1, 2016 at 6:16 AM, Signell, Richard <rsignell@xxxxxxxx> wrote:

> Pete,
> I guess the first step should be raising the issue here
> https://github.com/Unidata/thredds/issues
> and referencing @lesserwhirls (Sean Arms at Unidata) for his comment?
>
> -Rich
>
> On Wed, Aug 31, 2016 at 9:04 PM,  <Peter.Warren@xxxxxxxx> wrote:
> > Hi Rich,
> >
> > Thanks for the response.
> >
> > Assuming the Unidata guys are ok with the proposed solution, I'll setup
> a pull request for ncwms and edal-java.
> >
> > I notice in edal-java code there is a TODO stating that the default
> reading strategy chooser should take into account the size of the source
> grid.  I had a brief look at implementing this but it's non-trivial for
> complicated grids.
> >
> > Cheers
> > Pete
> >
> > -----Original Message-----
> > From: Signell, Richard [mailto:rsignell@xxxxxxxx]
> > Sent: Thursday, 1 September 2016 12:07 AM
> > To: Warren, Peter (Mineral Resources, North Ryde) <Peter.Warren@xxxxxxxx
> >
> > Cc: THREDDS community <thredds@xxxxxxxxxxxxxxxx>
> > Subject: Re: [thredds] NetCDF-4 DataReadingStrategy
> >
> > Peter,
> > The THREDDS Data Server code is now here:
> > https://github.com/Unidata/thredds/
> > so you could fork it and submit a pull request for this change if it
> hasn't already been handled in ncWMS2, which will be coming out with TDS
> 5.0, I think in October?
> >
> > Having said this, I'm sure the Unidata guys will now chime in here to
> > correct me.   ;-)
> >
> > -RIch
> >
> > On Wed, Aug 31, 2016 at 1:56 AM,  <Peter.Warren@xxxxxxxx> wrote:
> >> Hi Thredds List,
> >>
> >>
> >>
> >> My colleagues and I (at CSIRO and Geoscience Australia) have been
> >> using a patched version of thredds 4.2 for over 5 years to deliver
> >> satellite earth observation data.  We offer the data in as many
> >> formats as possible but our most used protocol is WMS for
> >> visualisation as overlays on maps.  A the time (around 2011) we had to
> >> patch thredds because we use relatively large (64gb)
> >> netCDF-4 source files which are too large to load into memory using
> >> the default BOUNDING_BOX reading strategy.  Xiangtan Lin modified the
> >> getOptimumDataReadingStrategy method in ncwms to make NetCDF-4 type
> >> datasets default to SCANLINE reading strategy.  This reduced the
> >> memory footprint sufficiently to get the wms layers working.  See
> discussion here:
> >> http://www.unidata.ucar.edu/mailing_lists/archives/thredds/2011/msg003
> >> 28.html
> >>
> >>
> >>
> >> We are now consolidating this thredds instance into a large corporate
> >> system and were hoping we could get this change included in the master
> >> source so we can keep up with the current releases without having patch
> each new one.
> >> Comments in the source seem to suggest that SCANLINE is the intended
> >> default for NetCDF files but the existing check only catches versions
> prior to 4.
> >>
> >>
> >>
> >> So, would it be possible to have this change made to the ncwms master?
> >> I can make the changes and setup a GitHub pull request if that helps.
> >>
> >>
> >>
> >> Thanks
> >>
> >> Peter Warren
> >>
> >>
> >> _______________________________________________
> >> NOTE: All exchanges posted to Unidata maintained email lists are
> >> recorded in the Unidata inquiry tracking system and made publicly
> >> available through the web.  Users who post to any of the lists we
> >> maintain are reminded to remove any personal information that they do
> >> not want to be made public.
> >>
> >>
> >> thredds mailing list
> >> thredds@xxxxxxxxxxxxxxxx
> >> For list information or to unsubscribe,  visit:
> >> http://www.unidata.ucar.edu/mailing_lists/
> >
> >
> >
> > --
> > Dr. Richard P. Signell   (508) 457-2229
> > USGS, 384 Woods Hole Rd.
> > Woods Hole, MA 02543-1598
>
>
>
> --
> Dr. Richard P. Signell   (508) 457-2229
> USGS, 384 Woods Hole Rd.
> Woods Hole, MA 02543-1598
>
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web.  Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> http://www.unidata.ucar.edu/mailing_lists/
>
  • 2016 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: