While I haven't seen the original file, my point is that you are violating
the CF convention (assuming I am correct in thinking that the TDS matches
the netcdf structure): variables that have the same name as a dimension
need to be one dimensional.
Benno
On Thu, Feb 23, 2012 at 3:57 PM, Claude DEROGNAT <cderognat@xxxxxxx> wrote:
> Hi Benno,****
>
> ** **
>
> Thank you for your answer. The corresponding name between grid coordinates
> and array dimension are driven by the NetCdf CF convention…****
>
> What about my real time constrain, so the require continuous update of the
> provides data ? How can I reach the property from my TDS catalog ?****
>
> ** **
>
> Regards,****
>
> ** **
>
> Claude****
>
> ** **
>
> Ing. Claude DEROGNAT, PhD*
> **ARIA** **TECHNOLOGIES *SA
> 8-10, rue de la Ferme* - *92100 Boulogne-Billancourt * *- France
> tél. / *phone:* 33 (0)1 46 08 68 60
> ligne directe / *direct line: *33 (0)1 46 08 68 65
> fax: 33 (0)1 41 41 93 17
> e-mail: *cderognat@xxxxxxx** *- http : www.aria.fr ****
>
> *P** **Avant d'imprimer ce courriel**, réfléchissez à l'impact sur
> l'environnement,
> et dans la mesure du possible éditez les documents en recto verso, merci!*
> ****
>
> ** **
>
> ** **
>
> *De :* bennoblumenthal@xxxxxxxxx [mailto:bennoblumenthal@xxxxxxxxx] *De
> la part de* Benno Blumenthal
> *Envoyé :* jeudi 23 février 2012 19:45
> *À :* Nathan Potter
> *Cc :* Claude DEROGNAT; support@xxxxxxxxxxx Support; Patrick West; Nathan
> Potter; thredds@xxxxxxxxxxxxxxxx
> *Objet :* Re: [thredds] OLFS and/or BES configuration to obtain a dynamic
> refresh of existing files ...****
>
> ** **
>
> As for the two different structures -- the TDS generated a pure flat
> structure, Hyrax made Grid structures, from the netcdf API point of view
> they are very similar, though there is a significant difference w.r.t.
> height.
>
> I wonder if the height variable is the reason TDS did not try to generate
> Grid structures -- It is a netcdf convention that the height dimension
> should have a height variable that is also one-dimensional. In this case,
> the height variable is four-dimensional. If the height dimension were
> renamed slightly, e.g. heightdim, you might get very different (and more
> consistent) results.
>
> Benno****
>
> On Thu, Feb 23, 2012 at 1:09 PM, Nathan Potter <ndp@xxxxxxxxxxx> wrote:***
> *
>
> ** **
>
> ** **
>
> ** **
>
> On Feb 23, 2012, at 4:24 AM, Claude DEROGNAT wrote:****
>
>
>
> ****
>
> Hi,
>
> My IT said that the file I send you is an old one. My problem is probably
> link to my THREEDS catalog.
> The question is so probably to threeds mailing list. I send them my
> question on Monday just after your message and I still have no response
> back ...
> So, I send you my catalog if you can have a look to it you may find why
> there a mismatch between
>
> http://web.aria.fr:443/thredds/dodsC/CHIMERE@CAMAC@reference@p02/CF_CHIMERE_20120115_d03.nc.html
> and
>
> http://web.aria.fr:443/LENVIS/CHIMERE/CAMAC/reference/p02/CF_CHIMERE_20120115_d03.nc.html
> ****
>
> ** **
>
> ** **
>
> Claude,
>
> I look at the first URL, saw that it was a instance of THREDDS Data Server
> (a fairly recent version 4.0.26) and I forwarded you message to their
> support list.
>
> It was only later that I realized the second URL is what appears to be a
> VERY old (circa version 1.5.0), locally compiled, version of our Hyrax
> server.
>
> So you are running two entirely different implementations of an OPeNDAP
> server.
>
> You can see this clearly when you look at one of the collections with a
> browser:
>
> *TDS*:
> http://web.aria.fr:443/thredds/dodsC/CHIMERE@CAMAC@reference@p02/
>
> *Hyrax*: http://web.aria.fr:443/LENVIS/CHIMERE/CAMAC/reference/p02/***
> *
>
> ** **
>
> why the second is continuously updated but doesn't allow the access to the
> whole set of variables stored in NetCdf file.
> Why the first one are not continuously updated even if the date in the
> threeds catalogs presentation always mentioned the right modified time .. .
> ****
>
>
> So with regards to how often datasets are "updated" - this is a function
> of how the servers are implemented. I don't know what is going on in the
> TDS, but in the Hyrax server each new request gets the file information
> regarding file size and last modified times from the files system on which
> the dataset resides. Hyrax also doesn't cache responses, so it will
> re-interrogate the file every time you ask for something in or about the
> file.****
>
> ** **
>
> The TDS folks are better informed to comment on what's going on in the TDS
> with regards to this.
>
> ****
>
> Now I think you are also saying that these two URLs:****
>
> ** **
>
>
> http://web.aria.fr:443/thredds/dodsC/CHIMERE@CAMAC@reference@p02/CF_CHIMERE_20120115_d03.nc
> ****
>
>
> http://web.aria.fr:443/LENVIS/CHIMERE/CAMAC/reference/p02/CF_CHIMERE_20120115_d03.nc
> ****
>
> ** **
>
> Reference the exact same file on the exact same file system. Is that the
> case?****
>
> ** **
>
> If so, the fact that the data structure representations generated by each
> server for this file are so different is somewhat surprising:****
>
> ** **
>
> ** **
>
> From the TDS:****
>
> ** **
>
>
> http://web.aria.fr:443/thredds/dodsC/CHIMERE@CAMAC@reference@p02/CF_CHIMERE_20120115_d03.nc.dds
> ****
>
> ** **
>
> Dataset {****
>
> String transverse_mercator;****
>
> Float32 x[lon = 101];****
>
> Float32 y[lat = 101];****
>
> Float32 lat[time = 48][lat = 101][lon = 101];****
>
> Float32 lon[time = 48][lat = 101][lon = 101];****
>
> String projectionCoordSys;****
>
> Float32 height[time = 48][height = 25][lat = 101][lon = 101];****
>
> Float32 time[time = 48];****
>
> Float32 Time[Time = 48];****
>
> Float32 REL[time = 48][lat = 101][lon = 101];****
>
> Float32 PHI[time = 48][height = 25][lat = 101][lon = 101];****
>
> Float32 M001S001[time = 48][height = 25][lat = 101][lon = 101];****
>
> } CHIMERE%40CAMAC%40reference%40p02/CF_CHIMERE_20120115_d03.nc;****
>
> ** **
>
> ** **
>
> ** **
>
> From (very old) Hyrax:****
>
> ** **
>
>
> http://web.aria.fr:443/LENVIS/CHIMERE/CAMAC/reference/p02/CF_CHIMERE_20120115_d03.nc.dds
> ****
>
> ** **
>
> Dataset {****
>
> String transverse_mercator;****
>
> Grid {****
>
> Array:****
>
> Float32 x[lon = 101];****
>
> Maps:****
>
> Float32 lon[lon = 101];****
>
> } x;****
>
> Grid {****
>
> Array:****
>
> Float32 y[lat = 101];****
>
> Maps:****
>
> Float32 lat[lat = 101];****
>
> } y;****
>
> String projectionCoordSys;****
>
> Grid {****
>
> Array:****
>
> Float32 REL[time = 48][lat = 101][lon = 101];****
>
> Maps:****
>
> Float32 time[time = 48];****
>
> Float32 lat[lat = 101];****
>
> Float32 lon[lon = 101];****
>
> } REL;****
>
> Grid {****
>
> Array:****
>
> Float32 PHI[time = 48][height = 25][lat = 101][lon = 101];****
>
> Maps:****
>
> Float32 time[time = 48];****
>
> Float32 height[height = 25];****
>
> Float32 lat[lat = 101];****
>
> Float32 lon[lon = 101];****
>
> } PHI;****
>
> Grid {****
>
> Array:****
>
> Float32 M001S001[time = 48][height = 25][lat = 101][lon = 101];***
> *
>
> Maps:****
>
> Float32 time[time = 48];****
>
> Float32 height[height = 25];****
>
> Float32 lat[lat = 101];****
>
> Float32 lon[lon = 101];****
>
> } M001S001;****
>
> Float32 Time[Time = 48];****
>
> } CF_CHIMERE_20120115_d03.nc;****
>
> ** **
>
> ** **
>
> This may be expected behavior, or not. I am tempted to say that you should
> upgrade both the TDS Hyrax to the latest releases before we dig to deeply...
> ****
>
> ** **
>
> ** **
>
> Anyone else care to comment?****
>
> ** **
>
> ** **
>
> ** **
>
> Thanks,****
>
> ** **
>
> Nathan****
>
> ** **
>
> ** **
>
> ** **
>
> ** **
>
> ** **
>
> ** **
>
> ** **
>
> ** **
>
> ** **
>
> ** **
>
>
>
> ****
>
> For instance,
>
> why the second is continuously updated but doesn't allow the access to the
> whole set of variables stored in NetCdf file.
> Why the first one are not continuously updated even if the date in the
> threeds catalogs presentation always mentioned the right modified time .. .
>
> Regards
>
> Ing. Claude DEROGNAT, PhD
> ARIA TECHNOLOGIES SA
> 8-10, rue de la Ferme - 92100 Boulogne-Billancourt - France
> tél. / phone: 33 (0)1 46 08 68 60
> ligne directe / direct line: 33 (0)1 46 08 68 65
> fax: 33 (0)1 41 41 93 17
> e-mail: cderognat@xxxxxxx - http : www.aria.fr
> P Avant d'imprimer ce courriel, réfléchissez à l'impact sur
> l'environnement, et dans la mesure du possible éditez les documents en
> recto verso, merci!
>
>
>
> -----Message d'origine-----
> De : Nathan Potter [mailto:ndp@xxxxxxxxxxx]
> Envoyé : mardi 21 février 2012 17:02
> À : Claude DEROGNAT
> Cc : Nathan Potter; Nathan Potter; support@xxxxxxxxxxx; Patrick West
> Objet : Re: OLFS and/or BES configuration to obtain a dynamic refresh of
> existing files ...
>
>
>
>
> Claude,
>
> It would appear that you are using the THREDDS Data Server (TDS). I say
> this because the configuration file that you attached is unique to the TDS
> and is not a part of Hyrax. The TDS is a UNIDATA product and I think that
> they would be better positioned to answer your questions regarding this,
> you xan reach them here: thredds@xxxxxxxxxxxxxxxx
>
> If you are pretty sure that you are trying to use Hyrax then I think we
> may have entered a twilight zone where parts of the two servers are being
> mashed up into one...
>
> Can you confirm which server software you are attempting to configure and
> use?
>
>
> Thanks,
>
>
> Nathan
>
>
>
> On Feb 21, 2012, at 5:03 AM, Claude DEROGNAT wrote:
>
>
> ****
>
> HI, ****
>
> ** **
>
> Sorry for the delay of my answer but I had to deal with another huge
> project.****
>
> ** **
>
> My system will be dedicated to real time plume modeling.****
>
> The model runs every 30 minutes. At the beginning of each day, it creates
> a result file then every 30 minutes the result file is overwritten with a
> new one containing the additional time frame.****
>
> ** **
>
> I observed à strange behavior between Opendap and thredds.****
>
> The file is continuously updated in the OpenDap server.****
>
> The threeds catalog notices in the 'Dates' field that the file are updates
> but the Access/OpenDap target file are not updated. The available time
> doesn't be continuously updated. May I have to perform any modification in
> the thredds configuration ? A attached you my threddsConfig.xml file...**
> **
>
> ** **
>
> Regards ****
>
> ** **
>
> Ing. Claude DEROGNAT, PhD****
>
> ARIA TECHNOLOGIES SA****
>
> 8-10, rue de la Ferme - 92100 Boulogne-Billancourt - France****
>
> tél. / phone: 33 (0)1 46 08 68 60****
>
> ligne directe / direct line: 33 (0)1 46 08 68 65****
>
> fax: 33 (0)1 41 41 93 17****
>
> e-mail: cderognat@xxxxxxx - http : www.aria.fr ****
>
> P Avant d'imprimer ce courriel, réfléchissez à l'impact sur
> l'environnement, et dans la mesure du possible éditez les documents en
> recto verso, merci!****
>
> ** **
>
> ** **
>
> ** **
>
> -----Message d'origine-----****
>
> De : Nathan Potter [mailto:npotter@xxxxxxxxxxx] ****
>
> Envoyé : mardi 11 octobre 2011 19:31****
>
> À : Claude DEROGNAT; support@xxxxxxxxxxx support****
>
> Cc : Nathan Potter; Patrick West****
>
> Objet : Re: OLFS and/or BES configuration to obtain a dynamic refresh of
> existing files ...****
>
> ** **
>
> ** **
>
> On Oct 11, 2011, at 9:22 AM, Claude DEROGNAT wrote:****
>
> ** **
>
> Dear all,****
>
> ** **
>
> With a correct title it should be better.****
>
> ** **
>
> Regards,****
>
> ...****
>
> ** **
>
> Dear all,****
>
> ** **
>
> I have a question about OpenDap functionality.****
>
> Is-it possible to configure OLFS and/or BES to obtain a dynamic ****
>
> refresh of existing files.****
>
> ** **
>
> If your model is writing to files that are within the scope of the
> BES.Catalog.catalog.RootDirectory, then Hyrax will just pick up the changes
> in the filesystem and refresh it's output. Little of that is cached, and if
> it is, it's triggered to refresh by a change in a files last modified time.
> ****
>
> ** **
>
> If writing your model output into the****
>
> BES.Catalog.catalog.RootDirectory doesn't work you might be able to use
> disk mounts and soft links to achieve what you want.****
>
> ** **
>
> But I have a feeling James and I haven't answered quite the question you
> were asking. The fact that James's earlier reply didn't help makes me think
> that repeating his point in more detail probably didn't either.****
>
> ** **
>
> - Are you perhaps wanting to define aggregations for a periodically
> changing file set? So that the aggregation definition always reflects the
> current state of the model?****
>
> ** **
>
> - Are you talking about automatically updating thredds catalog metadata
> for aggregations?****
>
> ** **
>
> - Updating NcML definitions of aggregations to reflect the new changing
> model results?****
>
> ** **
>
> ** **
>
> ** **
>
> I performed daily forecast of 48h and used Opendap to provide model ****
>
> results to partners.****
>
> Thank to assimilation each 30 minutes, I want to****
>
> - add a new time frame on an existing file****
>
> ** **
>
> Your model can just write to the file within the scope of
> BES.Catalog.catalog.RootDirectory. Hyrax will see that the last- modified
> time of the file has changed.****
>
> ** **
>
> - replace (modified) some fields stored few hours ago.****
>
> ** **
>
> Again, I think this is just your model writing to files that are within
> the scope of BES.Catalog.catalog.RootDirectory****
>
> ** **
>
> ** **
>
> Is-is possible ?****
>
> ** **
>
> Regards,****
>
> ** **
>
> Ing. Claude DEROGNAT, PhD****
>
> ARIA TECHNOLOGIES SA****
>
> 8-10, rue de la Ferme - 92100 Boulogne-Billancourt - France****
>
> tél. / phone: 33 (0)1 46 08 68 60****
>
> ligne directe / direct line: 33 (0)1 46 08 68 65****
>
> fax: 33 (0)1 41 41 93 17****
>
> e-mail: cderognat@xxxxxxx - http : www.aria.fr****
>
> P Avant d'imprimer ce courriel, réfléchissez à l'impact sur ****
>
> l'environnement, et dans la mesure du possible éditez les documents ****
>
> en recto verso, merci!****
>
> ** **
>
> ** **
>
> ** **
>
> -----Message d'origine-----****
>
> De : Nathan Potter [mailto:ndp@xxxxxxxxxxx]****
>
> Envoyé : vendredi 18 septembre 2009 18:14****
>
> À : Patrick West****
>
> Cc : Nathan Potter; Claude DEROGNAT; support@xxxxxxxxxxx****
>
> Objet : [SPAM?] Re: [Support] Bad Request 400 after a complete re- ****
>
> installation ....****
>
> ** **
>
> ** **
>
> ** **
>
> Claude,****
>
> ** **
>
> The OLFS relies on Tomcat's built in authentication mechanism. REad****
>
> this section:****
>
> ** **
>
>
> http://docs.opendap.org/index.php/Hyrax_-_OLFS_Configuration#Authentication_.26_Authorization
> ****
>
> ** **
>
> Of our documents for more information on that, although you will need****
>
> to read the Tomcat documentation to get it sorted out. You can connect****
>
> the authentication service to an LDAP server (using the JNDIRealm) if****
>
> you wish, or you could use the example "MemoryRealm" if all you want****
>
> is something simple but not as secure as LDAP.****
>
> ** **
>
> ** **
>
> Nathan****
>
> ** **
>
> ** **
>
> ** **
>
> On Sep 18, 2009, at 8:26 AM, Patrick West wrote:****
>
> ** **
>
> ** **
>
> On Sep 18, 2009, at 5:38 AM, Claude DEROGNAT wrote:****
>
> ** **
>
> Patrick,****
>
> ** **
>
> Thank you very much for your help ... Thank also Nathan and all the****
>
> opendap support !!!!****
>
> I never success without your help !!!!****
>
> ** **
>
> ** **
>
> Glad we could help. We are always working to make the installs****
>
> easier, and have formed a working group around the issue.****
>
> ** **
>
> the configuration similar to http://opendap.rpi.edu is perfect for****
>
> my application.****
>
> ** **
>
> A last question ... the last I promise ;-))).****
>
> I didn't find simple way to introduce "authentication" to restrict****
>
> access to the opendap (OLFS) server.****
>
> You look very familiar with tomcat. Do you know simple way to only****
>
> introduce one (or more) login/password in the tomcat server access.****
>
> I have in mind something similar to ftp access : http://login:password@IP*
> ***
>
> :port .****
>
> ** **
>
> I'm not sure about this last part, whether the OLFS supports this,****
>
> can get the username and password and do anything with it. Nathan?****
>
> ** **
>
> Patrick****
>
> ** **
>
> ** **
>
> Regards,****
>
> ** **
>
> Claude Derognat • Ingénieur R&D****
>
> ARIA TECHNOLOGIES SA****
>
> 8-10, rue de la Ferme - 92100 Boulogne-Billancourt - France****
>
> tél. / phone: 33 (0)1 46 08 68 60****
>
> ligne directe / direct line: 33 (0)1 46 08 68 65****
>
> fax: 33 (0)1 41 41 93 17****
>
> e-mail: cderognat@xxxxxxx - http : www.aria.fr****
>
> P Avant d'imprimer ce courriel, réfléchissez à l'impact sur****
>
> l'environnement, et dans la mesure du possible éditez les documents****
>
> en recto verso, merci!****
>
> ** **
>
> ** **
>
> -----Message d'origine-----****
>
> De : Patrick West [mailto:pwest@xxxxxxxxxxx]****
>
> Envoyé : jeudi 17 septembre 2009 19:38****
>
> À : Nathan Potter****
>
> Cc : Claude DEROGNAT; support@xxxxxxxxxxx****
>
> Objet : Re: [Support] Bad Request 400 after a complete re-****
>
> installation ....****
>
> ** **
>
> For one of our projects here, actually two of them, we have a ****
>
> default****
>
> context. We set this in the server.xml file, before the closing Host****
>
> tag:****
>
> ** **
>
> <Context docBase="opendap" path="" reloadable="true"****
>
> source="org.eclipse.j st.j2ee.server:opendap"/>****
>
> </Host>****
>
> ** **
>
> And, if you don't want to have to use :8080 when going to the ****
>
> server,****
>
> and you don't already have an apache server running, you can use ****
>
> this****
>
> in your server.xml file.****
>
> ** **
>
> <Connector port="80" address="128.113.28.182" protocol="HTTP/1.1"****
>
> connectionTimeout="20000"****
>
> useIPVHosts="true" redirectPort="443" />****
>
> ** **
>
> This configuration gives us http://opendap.rpi.edu, where we serve****
>
> some sample data.****
>
> ** **
>
> Might need to change this in other locations as well (port 80 for ****
>
> the****
>
> port tomcat listens on, and 443 for ssl connections.)****
>
> ** **
>
> But if you have apache running, then this last part won't work,****
>
> unless****
>
> you change the apache port to something other than 80. There is a ****
>
> way****
>
> to redirect in httpd.conf, but I'm not sure how to do that.****
>
> ** **
>
> In our case, we have one machine with three different IP addresses,****
>
> each one with either an apache server or tomcat server running, each****
>
> listening on port 80 for that IP address, which is why we specify ****
>
> the****
>
> address in the Connector.****
>
> ** **
>
> Patrick****
>
> ** **
>
> On Sep 17, 2009, at 12:58 PM, Nathan Potter wrote:****
>
> ** **
>
> ** **
>
> On Sep 17, 2009, at 9:23 AM, Claude DEROGNAT wrote:****
>
> ** **
>
> Nathan,****
>
> ** **
>
> Thank you for your help.****
>
> But your mail induces additional questions ...****
>
> ** **
>
> How do you configure Tomcat to use Hyrax (running in****
>
> it's context of "opendap") as the "default" servlet ?****
>
> ** **
>
> At this point I would have to go read through the Tomcat****
>
> documentation in order to answer your question.****
>
> ** **
>
> I think the answer is on this page:****
>
> ** **
>
> http://tomcat.apache.org/tomcat-6.0-doc/default-servlet.html****
>
> ** **
>
> ** **
>
> But I haven't tried this and I have no idea if it's actually****
>
> possible (or even advisable...)****
>
> ** **
>
> ** **
>
> ** **
>
> ** **
>
> I tried to couple Tomcat with Apache:****
>
> NC Services (link to BES) work properly but I lost the most part ****
>
> of****
>
> the layout: (no logo, no blue horizontal bar, not link to****
>
> documentation...).****
>
> ** **
>
> Did you read the section in the Hyrax documentation on integrating****
>
> Hyrax with Apache??****
>
> ** **
>
> http://docs.opendap.org/index.php/Hyrax_-_Apache_Integration****
>
> ** **
>
> I know people have done this and they have been able to make all of****
>
> it work.****
>
> ** **
>
> ** **
>
> ** **
>
> What's why, I tried to deploy OLFS on the "default" Tomcat URL (
> http://IP:8080****
>
> ) and NAT the external IP adresse to the LAN IP(:8080) of my****
>
> OpenDap sever. Any idea about that ?****
>
> ** **
>
> That's always worked for me.****
>
> ** **
>
> I just set up the firewall so that all requests sent to http://****
>
> WAN_ADDRESS:80 are forwarded to http://NAT_ADDRESS:8080****
>
> ** **
>
> I hope that helps,****
>
> ** **
>
> ** **
>
> Nathan****
>
> ** **
>
> ...
>
> [Message clipped]
--
Dr. M. Benno Blumenthal benno@xxxxxxxxxxxxxxxx
International Research Institute for climate and society
The Earth Institute at Columbia University
Lamont Campus, Palisades NY 10964-8000 (845) 680-4450
