Re: [thredds] get remote TDS binLimit?

Greetings John!

On Wed, Apr 29, 2020 at 3:42 PM John Maurer <jmaurer@xxxxxxxxxx> wrote:

> Is there an easy way to determine the binLimit setting for a remote TDS?
Currently there isn't a way to get those limits a priori. We could probably
add something that exposes those limits without the need to make a request,
but even then, interpreting those limits can get tricky when compression is
involved (see below).

> Also, when binLimit is exceed in OPeNDAP, does it return an error message?
> I know an ASCII request spits out a 403 message like the following:
>> Error {
>>     code = 403;
>>     message = "Request too big=1275173.181384 Mbytes, max=1000.0";
>> };
>> I guess how that message gets conveyed depends on the client, too (e.g.,
> Python, Matlab, etc.)?
A binary request over the max limit will contain a similar message body.
It's up to the client to check the message and expose it to the user. There
was a recent github issue over on the netCDF-C repository which I believe
led to having the C library at least return the message body, so now that
kind of message should be visible:

That issue also outlines the trickiness of compression I mention above.

What would be even better is to fix the underlying issue that causes the
need for having a limit on the server side. The performance section of the
TDS docs mentions

"The OPeNDAP-Java layer of the server currently has to read the entire data
request into memory before sending it to the client (we hope to get a
streaming I/O solution working eventually). Generally clients only request
subsets of large files, but if you need to support large data requests,
make sure that the -Xmx parameter above is set accordingly." (

It's on my long list of fires, but I need to get TDS 5 out the door before
trying to address that one. On a related note, Unidata is hiring if you
know of anyone looking!

Certainly a candidate with the right skillset could dive into
this...although we might have to reopen the position shortly after (the
code gets pretty complicated).

On a related note, we're grappling with a TDS that returns all 0's when
> binLimit is exceeded, rather than returning an error status. Anybody seen
> this? I've seen mention online of a netcdf-c timeout issue?

I'd be interested in knowing more about this. Exceeding the binLimit should
always result in a 403.

I hope all i s well in Hawaii!


> Thanks for any insights!,
> John Maurer
> Data System Engineer
> Pacific Islands Ocean Observing System (PacIOOS)
> University of Hawaii at Manoa
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web.  Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
  • 2020 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: