[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[IDV #WRY-586577]: IDV - Error loading MADIS METARS



Hey Bill-

> I’ve attached am Excel file with the LAT/LON of the wind tower locations. 
> Please let me know if you need anything else. As I mentioned in my other 
> e-mail, if I can reformat my ASCII text files to fit an IDV format let me 
> know. Otherwise if there is a way you can read the files so I can plot them 
> that would be great.

Okay, this was not easy, but I'm attaching a sample file with some
of your data and some notes:

- right now we don't support fixed column spacing like you have
in your file.  So, I needed to add commas to make it a CSV file.
- However, this uncovered (a yet unfixed bug) where I need to have
at least one space between commas.   I need to do a lot of testing
before that gets fixed.
- We need to have the date/time combined, but to do that with your
file, I needed to pad the times with 00's.
- I needed to add in the lat/lon for the stations.  Some of the
stations in the data didn't have entries in the Excel file (20,21)
so I just faked it.  We don't currently have a way to have the
locations in a separate file.
- We don't have a way to handle the different heights for the
same station/time.  I got around this by munging the TIDN, SIDE
and Z into a station id.

But, in the end, I was able to display the attached file in the
IDV.  It might give you some ideas.

In the nightly build, we just added the ability to read in the XLS
station file directly and we are going to look into doing this for
the  data as well.   The caveat of the location info still being in
the file will apply until we do a major reworking to read the point
data in through the netCDF Common Data Model.  That work is not
on our short term horizon though.  Is the data originally in Excel
or is it in these text files?

Don

> From: Bauman.Bill
> Sent: Monday, November 05, 2007 3:07 PM
> To: address@hidden
> Subject: RE: [IDV #WRY-586577]: IDV - Error loading MADIS METARS
> 
> 
> 
> Okay, now here's a more complex issue. I want to plot the KSC wind tower 
> network data in IDV if I can. The files are in ASCII text format and look 
> like the partial one attached. The header includes:
> 
> 
> 
> DAY[YMD]   TIME TIDN SIDE   Z  DIR  SPD[MPS] PDIR PSPD[MPS] DDEV  T[C] TD[C]  
> RH
> 
> 
> 
> DAY[YMD] (Date with format YYYYMMDD)
> 
> TIME (Minutes, every 5 minutes)
> 
> TIDN (Tower ID Number)
> 
> SIDE (Location of the sensors on the tower)
> 
> Z (Sensor height on the tower)
> 
> DIR (Wind direction)
> 
> SPD[MPS] (Wind speed in m/sec)
> 
> PDIR (Peak direction)
> 
> PSPD[MPS] (Peak speed in m/sec)
> 
> DDEV
> 
> T[C] (Temperature in deg C)
> 
> TD[C] (Dew Point temperature in deg C)
> 
> RH (Relative Humidity)
> 
> 
> 
> Obviously IDV doesn’t know the format of this data. Just for fun, I tried to 
> open a file using an IDV Data Source Type of “Text Point Data Files” but IDV 
> couldn’t decode the file. Is there some way I can manually tell IDV what type 
> of data is in each column or reformat my files into an IDV format so I can 
> plot at least wind speed & direction plus T and Td? I have the lat/lon of 
> each tower, in a separate file of course. My goal is to overlay the wind 
> tower plots on the model data for model verification.
> 
> 
> 
> -----Original Message-----
> From: Don Murray [mailto:address@hidden]
> Sent: Monday, November 05, 2007 2:22 PM
> To: Bauman.Bill
> Subject: Re: [IDV #WRY-586577]: IDV - Error loading MADIS METARS
> 
> 
> 
> Hi Bill-
> 
> 
> 
> Bauman.Bill wrote:
> 
> > It works. Thanks for the help - the MADIS METAR and MESONET files
> 
> > plot now Some of the stations appear to me missing in east-central
> 
> > Florida but that could be a MADIS issue...I'll start checking
> 
> > different days/times.
> 
> 
> 
> I think that's because of the problem in the file.
> 
> 
> 
> > The only issue, which is minor, in order to display the plots in the
> 
> > Map View, I first need to click the "Apply"  button for the Scale in
> 
> > the Dashboard. One would assume that upon loading the data, the
> 
> > default scale of "1" would automatically be applied. Until I clicked
> 
> > "Apply" all that was visible in the Map View were what looked like
> 
> > small pink dots or dashes at the station locations.
> 
> 
> 
> We have some issues with scaling that have been a pain to track
> 
> down.  It's particularly a problem if you have zoomed in before
> 
> adding in the plots.  Like you say, it "should" work the way
> 
> you expect, but it doesn't always.  At least you figured out
> 
> a work around.
> 
> 
> 
> As for GSD, Mike Barth said he'd fix it today and he's the
> 
> guy who would.  We'll see!
> 
> 
> 
> Don
> 
> *************************************************************
> 
> Don Murray                               UCAR Unidata Program
> 
> address@hidden                        P.O. Box 3000
> 
> (303) 497-8628                              Boulder, CO 80307
> 
> http://www.unidata.ucar.edu/staff/donm
> 
> *************************************************************
> 
> 
> 
> 
> 
> 
> ......................................................................
> The information contained in this email message is intended only for the use 
> of the individuals to whom it is
> addressed and may contain information that is privileged and sensitive. If 
> the reader of this message is not
> the intended recipient, you are hereby notified that any dissemination, 
> distribution or copying of this
> communication is strictly prohibited. If you have received this communication 
> in error, please notify the
> sender immediately by email at the above referenced address. Thank you.
> 


Ticket Details
===================
Ticket ID: WRY-586577
Department: Support IDV
Priority: Normal
Status: Open

Attachment: bill.csv
Description: Binary data