Re: [idvusers] Important changes regarding the Unidata IDV

Greetings IDV community!

This is a message to let you know that the transition of the official Unidata TDS server, thredds.ucar.edu, to TDS version 4.3 is complete. Also note that the main IDV catalog,

http://www.unidata.ucar.edu/georesources/idvcatalog.xml

has been updated to reflect this change. You will notice that there are now two entries under "Unidata IDD Model Data":

"THREDDS Model Data" and "IDV Model Data (Available until 1 June 2013)".

The "THREDDS Model Data" catalogs point to thredds.ucar.edu (the latest TDS server); the "IDV Model Data" catalogs point to motherlode.ucar.edu, which is the older TDS.

The remaining timeline is as follows:

Saturday, June 1st, 2013 - the previous data access URL, motherlode.ucar.edu, will be changed to use the TDS 4.3 server. This marks the end of TDS 4.2 access on Unidata systems.

Thursday, August 1st, 2013 - the data access URL motherlode.ucar.edu will no longer function. All users of the Unidata TDS should transition to using thredds.ucar.edu by this time. All users will need to upgrade to at least IDV 4.0 by this date.

Please see the message below for the gory details. Please note that we remain committed to making this process go as smoothly as possible. If at any point you have questions, please direct them to support-idv@xxxxxxxxxxxxxxxxx

Cheers,

The IDV and THREDDS Development Teams

On 3/19/13 1:09 PM, Sean Arms wrote:
Greetings IDV community!

The IDV and THREDDS teams have been working hard on transitioning the IDV to the latest version of netCDF-Java, as well as making modifications to allow the IDV to take advantage of the newest version of the THREDDS Data Server (TDS version 4.3).

The time has arrived for the official Unidata TDS to transition to version 4.3. There are very important improvements in this version, especially fixing incorrect GRIB data handling, affecting ~25% of all GRIB parameters from NCEP models. However, we understand that moving to a new version of the IDV and accessing data from a new data server during the school semester may cause some uneasiness. We've come up with a transition plan that we hope will address your concerns for the remainder of the Spring 2013 semester, while at the same time allowing us to make the necessary upgrades to the Unidata TDS.

Why are we doing this? Why is an upgrade necessary?
---------------------------------------------------

For more information on why the move to netCDF-Java and TDS 4.3 is necessary (including fixing incorrect handling of GRIB-1/2 data formats), please browse a very small selection of the discussion that has taken place over the past year or so:

http://www.unidata.ucar.edu/blogs/news/entry/netcdf_java_library_and_tds
http://www.unidata.ucar.edu/blogs/developer/en/entry/grib_renaming
http://www.unidata.ucar.edu/blogs/developer/en/entry/on_the_suitability_of_grib http://www.unidata.ucar.edu/blogs/developer/en/entry/grib_table_case_studies http://www.unidata.ucar.edu/mailing_lists/archives/netcdf-java/2012/msg00048.html http://www.unidata.ucar.edu/mailing_lists/archives/netcdf-java/2012/msg00060.html http://www.unidata.ucar.edu/mailing_lists/archives/netcdf-java/2012/msg00123.html http://www.unidata.ucar.edu/mailing_lists/archives/netcdf-java/2012/msg00240.html

(yes, this is a 'very small' portion of the discussion!)

The bottom line is that you WANT to make this transition, and we strongly encourage you to do so now. If you need to delay this until the semester is over, then we have ways for you to do that, detailed below.

Ok, so when will this happen?
-----------------------------

The timeline for transitioning from the TDS 4.2 to the 4.3 server at Unidata is as follows:

Friday, March 22nd, 2013 - the official data access URL for the Unidata TDS will be thredds.ucar.edu. This address will resolve to a TDS 4.3 server (the latest version).

Saturday, June 1st, 2013 - the previous data access URL, motherlode.ucar.edu, will be changed to use the TDS 4.3 server. This marks the end of TDS 4.2 access on Unidata systems.

Thursday, August 1st, 2013 - the data access URL motherlode.ucar.edu will no longer function. All users of the Unidata TDS should transition to using thredds.ucar.edu by this time.

What does this mean for IDV users?
----------------------------------

Well, that depends on which version of the IDV you are using.

1) If you or your lab are currently using the IDV version 3.1 (or earlier), then you will not need to do anything special for the remainder of the Spring Semester. By June 1st, 2013, you must upgrade to the latest version of the IDV (currently 4.0).

2) If you are currently using IDV 4.0 and are ready to make the transition, simply start accessing the Unidata server at thredds.ucar.edu by typing the following URL into the Catalog address box in the Data Choosers tab:

http://thredds.ucar.edu/thredds/catalog.xml

Note that the default catalog used by the IDV, http://www.unidata.ucar.edu/georesources/idvcatalog.xml, will include a link to the thredds.ucar.edu under "Unidata IDD Model Data" once the transition to TDS 4.3 has been made this Friday.

3) If you are currently using IDV 4.0, use bundles, and would like to continue using the older version of the Unidata TDS (version 4.2), you will need to install a plugin to allow IDV 4.0 to keep using the older TDS. The required plugin is available in the IDV plugin manager under the plugin category:

"Data Sources">"Motherlode TDS 4.2 persistence (for IDV 4.0)"

Using this plugin should ensure that the IDV continues to interface with the TDS 4.2 server as it did in previous versions, at least until June 1st 2013. After June 1st, you should uninstall this plugin.

Will all of my stuff break?
---------------------------

We don't believe so. Many of the changes between TDS 4.2 and 4.3, such as new GRIB variable names and new access URLs, are handled 'under the hood' of the IDV and netCDF-Java libraries, which is why older bundles should continue to work. However, there are some features of the TDS 4.2 that are not yet implemented in TDS 4.3. This decision was based on initial feedback that these features were labeled confusing or unnecessary - specifically, the "constant forecast offset" and "constant forecast hour" features of the TDS Forecast Model Run Feature collections (FMRC), which have been replaced by the GRIB feature collections. We know this will be missed by some, but we hope to have these features back in the TDS in the future.

Want to test your bundles with TDS 4.3 and the IDV 4.0?
-------------------------------------------------------

If you are using IDV version 4.0 and would like to test how the new TDS 4.3 server works with previous bundles, you can install the "Test Unidata TDS 4.2 -> 4.3 Upgrade" plugin, also found in the IDV plugin manager. If you've downloaded this plugin previously, you should remove it and install it again, as we needed to make some changes needed to the plugin to accommodate the timeline above. This plugin will work once the official Unidata TDS, thredds.ucar.edu, moves to 4.3 (scheduled for this Friday, March 22). Note: if you install this plugin, you will need to remove the TDS 4.2 persistence plugin - you may use one plugin or the other, but not both.

Lastly, we know that many IDV users are not subscribed to this list. If you know of someone who uses the IDV, please contact them to make sure they are aware of these very important upcoming changes. As always, feel free to contact us if you have questions, comments, or concerns, at support-idv@xxxxxxxxxxxxxxxxx

Cheers,

The IDV and THREDDS Development Teams