[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Support #WGL-271476]: Creating a Bundle in IDV for Exporting Multiple Variables

> > Dear Yuan,
> >
> > First of all, thank you very much for your response and consideration
> > towards our work.
> >
> > We are an institution dedicated to safeguarding the use of hydrological
> > resources in the Argentine Patagonia. Currently, we utilize software (IHMS)
> > for calculating river flow that incorporates parameters such as
> > precipitation and average temperature to estimate snowmelt and snow
> > formation.
> >
> > We have over a hundred meteorological stations installed across the
> > mountain range at various altitudes and on slopes facing different cardinal
> > points. The data from these stations was originally used to calibrate the
> > previously mentioned runoff software, by analyzing different snow and rain
> > events that occurred in recent years.
> >
> > Because the mean temperature threshold is so crucial for determining
> > snowmelt, we strive to adjust the models as accurately as possible to avoid
> > introducing errors into the equations, which could then lead to
> > inaccuracies in the resulting flow calculations. Thus, we aim to select
> > temperature variables that best reflect the actual mean temperature
> > measured by each station.
> >
> > Initially, our process involved extracting a specific set of variables from
> > the GFS via the TDS catalog. We defined the spatial area, extracted all the
> > data for that specific area, and then performed interpolation to find the
> > pixel whose coordinates best matched the real coordinates of each station.
> >
> > For instance:
> >
> > [image: image.png]
> > *Extraction of a variable for a whole spatial area.*
> >
> > [image: image.png]
> > *Extracted .xls file. One value for each pixel contained in that area.*
> >
> > [image: image.png]
> > *For each meteorological station we select the pixel that best approaches
> > its actual location.*
> >
> > However, this method presented two issues: For stations located in valleys
> > (low altitudes) within areas surrounded by high mountains, the temperatures
> > provided tended to be much lower compared to the actual temperatures
> > recorded by the station. We believe, although we may be mistaken, that this
> > is because, with such heterogeneous heights, the pixel averages the
> > altitudes and tends to provide low temperatures based on that average. As a
> > result, the station, being at a lower point, generally records a higher
> > temperature.
> >
> > This issue is greatly resolved when each variable extraction is specific,
> > meaning when the exact coordinate of the station is inputted (even with the
> > limitation of the IDV of a single digit after the comma). To achieve this,
> > we plot the variable in question, locate the exact coordinate, and then
> > extract the data. These extracted data perform much better than those
> > representing the entire pixel where the station is located.
> >
> >
> > [image: image.png]
> > *First we graph all the needed variables for one exact geographical point*
> >
> > [image: image.png]
> > Then we extract those variables from that specific point.
> >
> > [image: image.png]
> > *The exported data in .xls*

Dear Fernando,

I appreciate your follow-up email and now I have no trouble viewing the images. 
It seems that our support software may 
have inadvertently removed the images.

Your first approach to data extraction seems promising. However, I'd like to 
draw your attention to a couple of settings 
that could enhance the accuracy of your results:

Edit > Preferences > Formats & Data
1) Lat/Lon format as: ##0.000
2) Sampling Mode: Weighted Average

These adjustments should provide you with more precise location data. Since the 
model output resolution is on a coarse grid, 
utilizing the weighted average sampling or exploring a downscale algorithm will 
likely yield better results.


> >
> > The second issue we encountered is that temperature variables with a
> > specific altitude (surface, 2m, etc.) provide values that do not correspond
> > accurately to those recorded by the stations. We have been experimenting
> > with isobaric temperatures for some time now, which, although they respond
> > much better, exhibit significant isolated variations during major
> > precipitation events. We have observed that these variations may be related
> > to the estimation of surface pressure. We have observed that when the
> > surface pressure is very low, the corresponding temperature becomes that of
> > an isobar corresponding to a lower pressure as well.

I am totally surprised that isobar temperature yields better result, I wonder if
the selection of weighted average sampling can improve the result extracting 
from
surface temperature.

> >
> > We are trying to find a relationship that allows us to determine which
> > isobaric temperature or specific height temperature best reflects the
> > actual temperature recorded by the station, in order to feed the runoff
> > model with the forecast that best reflects reality.
> >
> > In conclusion, we must extract isobaric temperatures, as well as other
> > meteorological parameters, manually for each of the more than one hundred
> > points corresponding to the stations. This becomes a limitation when
> > calculating the necessary data to model a rainfall event that could have
> > serious consequences, and for which time is crucial. Hence, my inquiry
> > about the method of extracting numerous variables corresponding to a
> > specific coordinate to an excel file.

There's no need for manual handling of a hundred stations. The IDV offers a 
scripting method (ISL) precisely 
for such tasks, making it much simpler. Before I can offer a sample, I'll need 
to revisit the ISL syntax.

On another note, have you ever explored NCAR's hydro model, WRF-Hydro, for your 
research purposes? It could 
potentially offer valuable insights.

Best regards,
Yuan
> >
> > Lastly, I would like to ask if you have encountered a similar adjustment as
> > the one we are attempting and what strategies you have employed to address
> > it. Any information would be incredibly valuable to us.
> >
> > I hope I haven't over-extended this explanation, and I apologize in advance
> > for taking up so much of your time.
> >
> > I look forward to your response.
> >
> > Sincerely and at your disposal,
> >
> > Fernando Frassetto.
> >
> 
> Dear Fernando,
> 
> Thanks for detailed explanation of the workflow of your research work. 
> However, I can not see any images in
> your reply, could you send the original email with images again and CC to my 
> personal email: address@hidden.
> 
> Thanks,
> 
> Yuan
> >
> > El mar, 23 abr 2024 a la(s) 3:06?p.m., Unidata IDV Support (
> > address@hidden) escribió:
> >
> > > > Dears,
> > > >
> > > > I am writing to request assistance in creating a bundle within IDV that
> > > > would facilitate the export of multiple variables from a single
> > > coordinate
> > > > into a microsoft excel file.
> > > >
> > > > Currently, following the download of GFS threads data, our process
> > > involves
> > > > graphing multiple variables for a given coordinate. Subsequently, we are
> > > > compelled to manually export each representation onto separate Excel
> > > files.
> > > >
> > > > Regrettably, we are unable to directly utilize a file that encompasses a
> > > > single variable for a whole given area (to then extract the data for 
> > > > each
> > > > required coordinate), this is due to the fact that exporting an area
> > > > currently presents the variables in a pixel resolution, while we need 
> > > > the
> > > > extraction to be from a specific point. One contributing factor to this
> > > > constraint is our current study on a region characterized by complex
> > > > terrain. Utilizing the entire pixel resolution typically introduces
> > > > significant errors, particularly concerning parameters such as pressure
> > > and
> > > > temperature, because we think it tends to calculate the average height
> > > in a
> > > > whole pixel and with that value calculate the variable for that
> > > coordinate.
> > > >
> > > > Could you kindly provide guidance or support on configuring an IDV 
> > > > bundle
> > > > to facilitate the export of multiple variables from either a single
> > > > coordinate or multiple coordinates into an Excel file? Any instructions
> > > or
> > > > insights you could offer would be immensely appreciated.
> > > >
> > > > Thank you sincerely for your attention to this matter. I eagerly await
> > > your
> > > > response and assistance.
> > > >
> > > > Best regards,
> > > > Fernando Frassetto
> > > >
> > > >
> > > Dear Fernando,
> > >
> > > I found your application of the IDV quite intriguing, especially how it
> > > complements my aim to enhance data preprocessing for various applications
> > > like machine learning.
> > >
> > > While we're in the midst of developing several functions, they're not
> > > ready for public use just yet. I can make them available to you asap. I'd
> > > greatly appreciate your feedback to fine-tune the function interface.
> > >
> > > Regarding GFS data, could you please clarify how multiple variables are
> > > outputted to the Excel file? What's the structure of the Excel file, and
> > > what functions are necessary for each variable?
> > >
> > > Looking forward to your insights.
> > >
> > > Best regards,
> > > Yuan
> > >
> > > Ticket Details
> > > ===================
> > > Ticket ID: WGL-271476
> > > Department: Support IDV
> > > Priority: Normal
> > > Status: Open
> > > ===================
> > > NOTE: All email exchanges with NSF Unidata User Support are recorded in
> > > the Unidata inquiry tracking system and then made publicly available
> > > through the web.  If you do not want to have your interactions made
> > > available in this way, you must let us know in each email you send to us.
> > >
> > >
> > >
> >
> >
> 


Ticket Details
===================
Ticket ID: WGL-271476
Department: Support IDV
Priority: Normal
Status: Open
===================
NOTE: All email exchanges with NSF Unidata User Support are recorded in the 
Unidata inquiry tracking system and then made publicly available through the 
web.  If you do not want to have your interactions made available in this way, 
you must let us know in each email you send to us.