Greetings Michael,
Do lon and lat change between each granule of the aggregation in the
daily files? If so, there will be issues with tools using netCDF-Java
going through the part of the stack that needs to understand things at
a feature type level, such as GRID, as netCDF-Java does not deal with
time varying spatial coordinate systems (this includes WMS and the
netCDF Subset Service in the TDS, for example)...there is some code
that makes me think it might partially deal with it, but I'm guessing
it's nowhere near complete (not sure how much swath feature type was
developed).
That said, I was able to get the following to read in toolsUI as a
GRID by assuming lat/lon to not vary in time (likely an incorrect
assumption):
<?xml version="1.0" encoding="UTF-8"?>
<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
<aggregation dimName="aggTime" type="joinNew">
<variableAgg name="wind_speed"/>
<scan location="C:/Users/sarms/Desktop/qsat/qscat_data/"
suffix=".nc" dateFormatMark="QSCAT_12_#yyyyMMdd" />
</aggregation>
<variable name="wind_speed">
<attribute name="coordinates" value="aggTime lon lat" />
</variable>
</netcdf>
again, this assumes that lat/lon are not time dependent across the
aggregation files. Also note that I've updated the coordinates
attribute to reference the new aggTime coordinate variable that's
automatically created as part of the joinNew aggregation.
Sean
On Mon, May 11, 2020 at 4:15 PM Michael McDonald <mcdonald@xxxxxxxxxxxxx> wrote:
>
> Antonio,
> While the "along_track" is the UNLIMITED dimension in all of these
> swath files. Our attempts to just use this as the dimension for the
> joinExisting aggregation does not throw any issues wrt THREDDS at
> catalog init time, but when you actually try to load/use this dataset
> it does not work and fails with the message in Panoply,
>
> "There was an error preparing the data: Failed creating the data
> handler: Could not read lon axis array."
>
>
>
> e.g., joinExisting aggregation on along_track (fails with: Could not
> read lon axis array.)
> http://tds.coaps.fsu.edu/thredds/dodsC/coaps/qscat/nc_L2B_v4p0_12/nc_L2B_12_daily_TESTjoinExisting-along_track.html
>
> <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
> <aggregation type="joinExisting" dimName="along_track">
> <scan
> location="/Net/data/qscat/data/nc_L2B_v4p0_12/nc_L2B_12_daily/2009"
> suffix=".nc" subdirs="false" />
> </aggregation>
> </netcdf>
>
>
> This *simplified* product is a collection of multiple quikscat swath
> passes in a 24 hour period to assemble a full coverage of the globe.
>
> orbit_file_source:
> https://podaac-opendap.jpl.nasa.gov/opendap/allData/quikscat/L2B12/v4.0/
>
> Which looks like this for a single day.
> http://tds.coaps.fsu.edu/thredds/wms/coaps/qscat/nc_L2B_v4p0_12/nc_L2B_12_daily/2009/QSCAT_12_2009325.nc?STYLES=&SERVICE=WMS&FORMAT=image%2Fpng&REQUEST=GetMap&HEIGHT=512&WIDTH=1024&VERSION=1.1.1&BBOX=-180.0%2C-80.0%2C180.0%2C90.0&LAYERS=wind&SRS=EPSG%3A4326&TRANSPARENT=TRUE
>
>
> Our *end goal* is to have a virtually aggregated collection of all
> this simplified daily swath data from 1999 to 2009 for ingestion as an
> OPeNDAP/WMS dataset into a web-based data visualizer like OWGIS
> (https://owgis.org/), so that users can pick the date(s) they want,
> and view this in a web GUI.
>
>
> We tried to use joinNew to aggregate all these like files on a new
> dimension "T" and populate the time info using the filename
> (dateFormatMark="QSCAT_#HH_yyyyDDD") and get the same error (fails
> with: Could not read lon axis array.)
>
>
> e.g., joinNew aggregation on new dimension T
> http://tds.coaps.fsu.edu/thredds/dodsC/coaps/qscat/nc_L2B_v4p0_12/nc_L2B_12_daily_TESTjoinNew-time-lon-lat.html
>
> <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
> <aggregation dimName="T" type="joinNew">
> <variableAgg name="time"/>
> <variableAgg name="lon"/>
> <variableAgg name="lat"/>
> <variableAgg name="eastward_wind"/>
> <variableAgg name="northward_wind"/>
> <variableAgg name="wind_speed"/>
> <variableAgg name="wind_to_direction"/>
> <variableAgg name="simplified_wvc_quality_flag"/>
> <variableAgg name="rain_impact"/>
> <variableAgg name="distance_from_coast"/>
> <scan dateFormatMark="QSCAT_#HH_yyyyDDD"
> location="/Net/data/qscat/data/nc_L2B_v4p0_12/nc_L2B_12_daily/2009"
> suffix=".nc" subdirs="false" />
> </aggregation>
> </netcdf>
>
>
> Any ideas what we are doing wrong in THREDDS, or is the issue in the
> data files we simplified?
>
>
> >
> > Ian,,
> >
> > What aggregation are you expecting? just on the 'along_track' dimension?
> >
> > Antonio
> > On 5/5/20 19:46, Ian Dimitri wrote:
> >
> > Hello All,
> >
> > Adding onto Michael's existing thread, I am working closely with him
> > attempting to figure out the issue we are having with the aggregation of
> > swath data that has a two-dimensional grid with track and cross-track
> > coordinates. We have tried using FMRC, joinExisting, and joinNew file
> > aggregation methods within THREDDS, all with no success.
> >
> > We tried using many different aggregations methods
> > that exist within the ncWMS2 interface. We have tried doing the simple
> > wildcard aggregation attempt by using the wildcard character (e.g.
> > /ASCAT/nc_L2B_25_daily/2020/*.nc).
> >
> > We have also tried to use the "Dynamic Services" that is also available
> > through ncWMS2, but that also did not seem to work with the
> > multi-dimensional swath data that we are
> > trying to aggregate.
> >
> > Finally another method we tried was through geoserver, where we attempted
> > to use the image mosaic option that it has but that option also proved to
> > be a dead end.
> >
> > None of the methods mentioned above worked properly, and we are wondering
> > if anyone has had any success aggregating swath data with the methods, and
> > would like to share any
> > special steps that had to be taken to make said method work. Are there any
> > other methods I did not mention above that could possibly be able to
> > aggregate the multi-demensional swath data?
> >
> > Any insight or help would be appreciated
> >
> > unaggregated data
> > http://tds.coaps.fsu.edu/thredds/catalog/coaps/qscat/nc_L2B_v4p0_12/nc_L2B_12_daily/catalog.html
> >
> > attempted aggregation with joinNew
> > http://tds.coaps.fsu.edu/thredds/satellite_testing.html?dataset=joinnew-TEST-coaps-qscat-nc_L2B_v4p0_12-nc_L2B_12_daily
>
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web. Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> https://www.unidata.ucar.edu/mailing_lists/
