CONDUIT/CRAFT (C2) Steering Meeting
Kelvin Droegemeier, Univ of
Bob Gall, UCAR, USWRP
Louis Uccellini, NCEP,
Allan Darling, NWS
Fred Branski, NWS
Janelle Janish, NSSL
Kevin Kelleher, NSSL
Bill Callahan, WSI
John Ward, NCEP
Chris MacDermaid, FSL
Univ of Utah
Steve Chiswell, Unidata
Linda Miller, Unidata
Kelvin Droegemeier and Bob Gall led a discussion of the history
of CONDUIT and CRAFT. Both use Unidata's LDM for data distribution.
The project needs a change of strategy by creating a formal advisory
committee comprising university, government and private sector
members. The group would be responsible for discussion of CONDUIT
and CRAFT planning, and implementation strategies, etc. This group
could also facilitate entraining the private sector into the USWRP
planning process. CRAFT can inform CONDUIT through "practices
The merged group should:
- be loosely organized
- be a grass roots effort with broad community participation;
participation by the private sector can be facilitated by working
with the AMS Private Sector Board and the CWSA
- meet twice annually, once at the AMS Annual Meeting.
- the meetings will be open to the public and announced in
advance, though C2 may hold a closed meeting in advance in order
to conduct its business.
Fred Branski said that the NWS is looking into a Family of
Services "operational" solution for delivery of the Level II data,
along with the four hubs being used in conjunction with
Internet2/Abilene and universities. When finalized, the FOS delivery
will be one of the data delivery solutions available, which will
include a charge for the cost recovery. The NWS will offer 24/7
support, customary for FOS services.
Jim Steenburgh, representing the Unidata Users Committee,
presented the idea of providing the National Digital Forecast
Database (NDFD) in the CONDUIT data stream. Discussion among the
group revealed that the NDFD includes about 50 MB/hr (about 255
files), but plans are in place to continue expansion of the data
stream. The data products are released in one lump (about 26-50
MB/hr) with the process taking about 30-90 seconds. This would
increase the queing delay of products (such as the GFS) already seen
The CONDUIT data stream is sending approximately 1.0 GB/hr, peaks
at 1.8 GB/hr.
Louis Uccellini emphasized the need to get NCEP grids to the
community. He endorses the C2 effort and would like to see it become
a main vehicle for achieving this goal for the broad community of
users. C2 is a good forum to bring diverse groups together.
The WRF component is important, but must not impede operations at
NWS. The 3D Var is an important aspect of WRF that the community
would be of interest to the university community. Geoff DiMego
pointed out one of the differences for WRF is the BUFR, versus GRIB
format used for CONDUIT. NetCDF will not used for data
Some of the issues to address for the CONDUIT feed from NCEP
- restricted data, e.g. ECMWF
- defining data sets to be included in the data stream
- procedures for proposing data sets, etc.
CRAFT radars now number 102, comprised primarily of NWS radars,
with a few DOD sites. By the end of February, there should be 124.
The total radars will be 142, including NWS and DOD radars. Kevin
Kelleher reported that the 4 NWS Regional HQ collection hubs might
share data among one another such that each hub would contain data
from all radars. A final decision about this issue has not been
Level II Distribution:
A "Call for Participation" asked for University proposals to
provide top tier distribution of freely available Level II data to
university sites on a cost-recovery basis for private industry sites
were submitted to the UPC. There will be four top-tier sites, MOAs
with the four will be signed and the names simultaneously released
with a 1 March 2004 target date. The four top-tiers must provide
24x7 support. University of Oklahoma will continue providing the
data via CRAFT until the NWS announces the selection of four sites
for data distribution. NOAA is investigating the idea of providing a
Family of Service solution for distribution of the Level II data.
This service would be provided through a cost recovery method for
Model Data Distribution:
- We need some procedures to determine priorities on datasets to
include and which to delete for the CONDUIT feed.
- What is the target latency?
- Many of the models are on a six-hour schedule, so that needs
to be considered.
- Unidata has been working on development of statistics
reporting. Data can get to Brazil in 1-2 seconds. There is no
longer a problem of long delays due to long distances.
- NOMADS, THREDDS and NCEP FTP servers need to be considered and
included in information for the community, as options for
obtaining the data.
- A hybrid system of push-pull could be the appropriate answer
for making the data useful for the diverse community.
- The question of opening the project to the private sector and
government users was discussed. Due to the use of
Internet2-Abilene, private sector and government parties could
work closely with university institutions to access the data.
- Do not duplicate grids data stream.
- GFS being set to 384 (4 times per day)
- GFS ensemble
- Need to revisit the restricted data issue, ECMWF
- AVN is migrating to GFS file names
- Need to explore the threshold for delay in survey
- Coalesce needs
- What is the "shelf-life" of a model? 21/2 hrs?
- Deletions should be performed immediately
- Need to define a process
- Partnerships between the private sector and academia are
Action Item: NDFD not presently possible without
queing delays of other CONDUIT datasets. CONDUIT will explore
alternatives to making NDFD distribution possible in the future.
Action Item: Research queing delay issue: 30
minutes is unacceptable.
Action Item: Remove the RUC 40km grids (RUC 20km
is being sent) and Eta 215 (20km) when ETA 218 (12km) is made
available (pending research on queuing delay issue,
Action Item: A survey could be conducted with
the community for guidance on products to be included in data
Action Item: Give community 30-day notice prior
Action Item: Develop a schematic of CONDUIT
similar to that for CRAFT - so that possible linkages can be better
planned and understood.
Action Item: C2 needs to explore ways in which
CONDUIT can benefit from the NWS version of CRAFT (i.e., the one
used for operations).
Linda Miller - firstname.lastname@example.org
University Corporation for Atmospheric Research
Boulder, CO 80307-3000