[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Datastream #QOK-427508]: NPS: New "manager", Date problem

Hi Ryan,

address@hidden> wrote:

> New Client Reply: NPS: New "manager", Date problem
> Thanks for the response and no problem about the delay. I realized the
> same thing about the reverse lookup last night. I’ll have to put in a
> “special” request to AWS so that the assigned Elastic IP can have a reverse
> lookup resolve to idd.foggy.nps.edu.


Very good.

re:

> With that said, can you please add idd.foggy.nps.edu anyways while I put
> in the request to AWS?
>

I guess you missed the last part of my previous reply:

"In the interim, you should be ready to go, but you should check as follows:

<as 'ldm' on your machine>
notifyme -vl- -f ANY -h idd.unidata.ucar.edu"

Cheers,

Tom

Thanks!
> Ryan
>
> --
> Ryan Yamaguchi
> Faculty Associate - Research
> Department of Meteorology
> Naval Postgraduate School
> 589 Dyer Road, Room 259
> Monterey, CA 93943
> Office: (831) 656-7645 DSN: 756-7645
> Cell: (831) 682-5159
> ________________________________
> From: Unidata Datastream Support <address@hidden>
> Sent: Thursday, September 12, 2019 7:12:29 AM
> To: Boothe, Mark (CIV) <address@hidden>
> address@hidden>; Boothe, Mark (CIV) <address@hidden>
> Subject: [Datastream #QOK-427508]: NPS: New "manager", Date problem
>
> Hi Ryan,
>
> My apologies for the tardy reply...
>
> address@hidden> wrote:
>
> > New Client Reply: NPS: New "manager", Date problem
> >
> > Hi Tom,
> > I wanted to follow-up, perhaps with the IDD support email…Not sure if it
> > went through on the address@hidden email.
> >
>
> It did, but there have been a LOT of things going on in the past several
> days...
>
> re:
>
> > See questions below.
>
>
> These will be answered from our inquiry tracking system.
>
> re:
>
> > Also, can you please add idd.foggy.nps.edu to your ALLOW list on
> > idd.unidata.ucar.edu?
> >
>
> As far as I can tell, this is a VM in AWS, correct?  Here is what I found:
>
>  $ nslookup idd.foggy.nps.edu
> Server: 127.0.1.1
> Address: 127.0.1.1#53
>
> Non-authoritative answer:
> Name: idd.foggy.nps.edu
> Address: 18.236.13.104
> Name: idd.foggy.nps.edu
> Address: 52.38.107.188
>
> mcidas@odroidU3:~/Documents$ nslookup 52.38.107.188
> Server: 127.0.1.1
> Address: 127.0.1.1#53
>
> Non-authoritative answer:
> 188.107.38.52.in-addr.arpa name =
> ec2-52-38-107-188.us-west-2.compute.amazonaws.com.
>
> Authoritative answers can be found from:
>
> So, there is forward DNS (name -> IP), but no reverse DNS (IP -> name).  In
> order to
> ALLOW you we added an entry to our hosts file that defines your machine's
> IP address
> as the name you specified.  This approach is "brittle" as it will stop
> working if the IP
> address of your VM changes.  It would be much better if reverse DNS was
> setup for
> your machine such that the IP address is globally known as
> idd.foggy.nps.edu.
> In
> the interim, you should be ready to go, but you should check as follows:
>
> <as 'ldm' on your machine>
> notifyme -vl- -f ANY -h idd.unidata.ucar.edu
>
> Cheers,
>
> Tom
>
> > . Thanks!
> > Best regards,
> > Ryan/Mark
> >
> > --
> > Ryan Yamaguchi
> > Faculty Associate - Research
> > Department of Meteorology
> > Naval Postgraduate School
> > 589 Dyer Road, Room 259
> > Monterey, CA 93943
> > Office: (831) 656-7645 DSN: 756-7645
> > Cell: (831) 682-5159
> >
> >
> > From: "Boothe, Mark (CIV)" <address@hidden>
> > Date: Friday, September 6, 2019 at 2:02 PM
> > To: "Yamaguchi, Ryan (CIV)" <address@hidden>, "
> address@hidden>
> > Subject: Re: [Datastream #QOK-427508]: NPS: New "manager", Date problem
> >
> > Hello Tom,
> >
> > Here at the Naval Postgraduate School, Ryan and I have been coordinating
> > with our local IT folks as we prepare the transition of our LDM feed
> from a
> > set of on-site linux machines to AWS.
> >
> > Thanks for your previous message pointing out the RTSTATS link.  That
> > gives a good depiction of the data stream.
> >
> > Do you have any suggestions as we spin up an AWS instance?  For instance:
> >
> > Is there a preferred OS (i.e.: Amazon linux, Red Hat, Suse, Ubuntu,
> etc.)?
> > What other specifications (i.e. memory, computing power, etc.)?
> > How do we inform you of our new AWS instance?
> > We currently receive our data via the University of Washington. Would
> this
> > continue with a new AWS instance, or would we connect directly with the
> IDD?
> > Is there a particular data stream that serves as a good initial test?
> >
> > Thanks,
> > Mark Boothe and Ryan Yamaguchi
> > Naval Postgraduate School
> > Monterey CA 93943
> > ________________________________
> > From: Yamaguchi, Ryan (CIV) <address@hidden>
> > Sent: Friday, June 21, 2019 9:48 AM
> address@hidden
> > >
> > Subject: Re: [Datastream #QOK-427508]: NPS: New "manager", Date problem
> >
> > Hi Tom,
> >
> > Attached is a basic diagram and list of requirements I've sent to our IT
> > folks.  At a minimum, we are trying to get an LDM server and AWIPS EDEX
> > ingest/decoder server spun in the AWS cloud.  We would like to archive
> case
> > studies with Amazon S3.
> >
> > As for the visualization suite, I am planning to host that software on
> > Amazon WorkSpaces (i.e. bring your own thin client).  The students can
> > login to that system and complete all of their analyses there, without
> > pulling out data to their local machine (trying to save costs).
> >
> > Sorry, I do not have an explicit roadmap for cloud computing.  However,
> it
> > seems we have full support from our IT to transition our aging systems to
> > the cloud.
> >
> > Thanks!
> > Ryan
> >
> > address@hidden> wrote:
> >
> >     Hi,
> >
> >     re:
> >     > Thanks for your information this last week as I begin to log into
> > machines here at NPS
> >     > as an LDM manager for the first time.
> >
> >     No worries.
> >
> >     re:
> >     > I'd like to shift gears a bit and help get the ball rolling faster
> > on our transition
> >     > here at NPS from our relatively limited ingest into linux machines
> > on campus to a larger
> >     > ingest on a new Amazon platform.
> >
> >     OK.
> >
> >     re:
> >     > The IT/cybersecurity department here on campus requires,
> >     > of course, specifics on the technical aspects of what this Amazon
> > instance/storage would
> >     > require before we can spin it up.
> >
> >     Of course.
> >
> >     re:
> >     > Do you have, or can you direct us toward a person or
> >     > website, that has specifics that we would need in order to ingest
> > and store a useful
> >     > (maybe a year's worth?) archive?
> >
> >     The volume of data flowing in the IDD is a moving target, so any
> > estimate made today
> >     will be wrong tomorrow.  That being said, you could do some "back of
> > the envelope"
> >     calculations by extrapolating the average volume of data available in
> > the datastreams
> >     desired per hour and then multiplying by the number of hours in a
> > year.  For
> >     instance, a randomly chosen real-server back end of our
> > idd.unidata.ucar.edu
> >     top level IDD relay cluster can be found in:
> >
> >
> >
> http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/siteindex?cluster2.unidata.ucar.edu
> >
> >     The link at the lower left of this page will generate a snapshot list
> > of
> >     the volume of data flowing in each of the datastreams that this node
> is
> >     handling:
> >
> >     Cumulative volume summary
> >
> >
> http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/rtstats_summary_volume?cluster2.unidata.ucar.edu
> >
> >     This looks like:
> >
> >     Data Volume Summary for cluster2.unidata.ucar.edu
> >
> >     Maximum hourly volume 100803.397 M bytes/hour
> >     Average hourly volume  65493.578 M bytes/hour
> >
> >     Average products per hour     523876 prods/hour
> >
> >     Feed                           Average             Maximum
>  Products
> >                          (M byte/hour)            (M byte/hour)
> >  number/hour
> >     SATELLITE             14892.385    [ 22.739%]    19959.534
>  6659.762
> >     CONDUIT               11783.612    [ 17.992%]    33345.747
>  105400.619
> >     NGRID                 10437.259    [ 15.936%]    14778.178
> 64434.857
> >     NEXRAD2                9293.592    [ 14.190%]    11192.046
>  102743.286
> >     NOTHER                 6984.060    [ 10.664%]    10042.353
> 12000.310
> >     NIMAGE                 3973.837    [  6.068%]     9144.665
>  3507.000
> >     FNMOC                  3181.508    [  4.858%]    11549.530
>  8347.071
> >     NEXRAD3                2975.614    [  4.543%]     3511.048
>  127587.762
> >     HDS                    1369.725    [  2.091%]     1754.030
> 44231.143
> >     GEM                     306.029    [  0.467%]     3307.336
>  1779.452
> >     FNEXRAD                 119.299    [  0.182%]      134.965
> 104.548
> >     UNIWISC                  85.293    [  0.130%]      142.226
>  49.857
> >     IDS|DDPLUS               71.118    [  0.109%]       84.169
> 46548.738
> >     EXP                      12.109    [  0.018%]       18.775
> 126.310
> >     LIGHTNING                 8.033    [  0.012%]       15.598
> 354.429
> >     GPS                       0.105    [  0.000%]        1.102
> 0.929
> >
> >     As you can see, there are hourly average and hourly peak volumes
> > listed.
> >
> >     re:
> >     > We realize that the recent transition to GOES-16/17 is
> >     > changing all of the calculations, so making this transition now
> > seems like good timing.
> >
> >     The volume of data flowing in the IDD changes all of the time as new
> > model output
> >     is added, new satellite imagery is added, higher resolution radar
> data
> > becomes
> >     available, etc.  The IDD truly is a fire hose, and its stream is
> > continually growing.
> >
> >     re:
> >     > By the way, Ryan xxxxx has been here a few years and has already
> > made some headway
> >     > in getting this transition going. You may recall earlier
> > correspondence with him, and
> >     > I've cc'ed him. As of now, the two of us are working together on
> > this project to help
> >     > cover this transition to Amazon during any other temporary projects
> > that require weeks
> >     > worth of field work out of the office.
> >
> >     OK, sounds good.
> >
> >     re:
> >     > Thanks for any technical advice you have,
> >
> >     No worries.
> >
> >     Do you/NPS have a road map for where you are heading wrt cloud
> > computing?  If yes, are
> >     you willing to share it?
> >
> >     Cheers,
> >
> >     Tom
> >     --
> >
> >
> ****************************************************************************
> >     Unidata User Support                                    UCAR Unidata
> > Program
> >     (303) 497-8642                                                 P.O.
> > Box 3000
> >     address@hidden                                   Boulder,
> > CO 80307
> >
> >
> ----------------------------------------------------------------------------
> >     Unidata HomePage                       http://www.unidata.ucar.edu
> >
> >
> ****************************************************************************
> >
> >
> >     Ticket Details
> >     ===================
> >     Ticket ID: QOK-427508
> >     Department: Support Datastream
> >     Priority: Normal
> >     Status: Closed
> >     ===================
> >     NOTE: All email exchanges with Unidata User Support are recorded in
> > the Unidata inquiry tracking system and then made publicly available
> > through the web.  If you do not want to have your interactions made
> > available in this way, you must let us know in each email you send to us.
> >
> >
> >
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: QOK-427508
> > Department: Support Datastream
> > Priority: Normal
> > Status: Open
> > Link:
> >
> https://andy.unidata.ucar.edu/staff/index.php?_m=tickets&_a=viewticket&ticketid=30459
> >
> >
>
> --
> +----------------------------------------------------------------------+
> * Tom Yoksas                                      UCAR Unidata Program *
> * (303) 497-8642 (last resort)                           P.O. Box 3000 *
> * address@hidden                            Boulder, CO 80307 *
> * Unidata WWW Service                     http://www.unidata.ucar.edu/ *
> +----------------------------------------------------------------------+
>
>
>
> Ticket Details
> ===================
> Ticket ID: QOK-427508
> Department: Support Datastream
> Priority: Normal
> Status: Open
> ===================
> NOTE: All email exchanges with Unidata User Support are recorded in the
> Unidata inquiry tracking system and then made publicly available through
> the web.  If you do not want to have your interactions made available in
> this way, you must let us know in each email you send to us.
>
>
>
>
>
> Ticket Details
> ===================
> Ticket ID: QOK-427508
> Department: Support Datastream
> Priority: Normal
> Status: Open
> Link:
> https://andy.unidata.ucar.edu/staff/index.php?_m=tickets&_a=viewticket&ticketid=30459
>
>

-- 
+----------------------------------------------------------------------+
* Tom Yoksas                                      UCAR Unidata Program *
* (303) 497-8642 (last resort)                           P.O. Box 3000 *
* address@hidden                            Boulder, CO 80307 *
* Unidata WWW Service                     http://www.unidata.ucar.edu/ *
+----------------------------------------------------------------------+



Ticket Details
===================
Ticket ID: QOK-427508
Department: Support Datastream
Priority: Normal
Status: Open
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.