[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[LDM #MIW-275261]: some questions on hardware configs for our LDM server
- Subject: [LDM #MIW-275261]: some questions on hardware configs for our LDM server
- Date: Wed, 18 Jul 2018 13:51:17 -0600
Greg,
That was helpful. Thanks.
In order to speed things up, would it be possible for us to log onto the system
in question as the LDM user? If so, would you please add the following ssh(1)
credential to the file "~ldm/.ssh/authorized_keys" (NB: the entry has to be one
line only):
from="128.117.144.*" ssh-rsa
AAAAB3NzaC1yc2EAAAADAQABAAACAQCsVY8X6SQ8v4yXyWE7pD4jtn1HznYUlsOZm9AjGOSR5Vi0oFDCjM3Ip/5dm8sMoYPe+DvAOAbbxk0HhdIq3tRjzVNkndM4c4NxvnNrbM0IQMnWPPTEeWMAkXWXLrHQyI+qJJvz3yc7ivX/UJg5BSIzrsMjjEvquHTDpx/iuS0GnCmadk/847hzo8AagPp3gGA5t7iCqh4Riv+obbRYA1foWZApGpF+LPP0QV22W+CllOMAJC+eXY/9zX3wN6hsUJfS+4CANCN6WzaY4w3HgYiFfPoEAYO/I8B/JHphT1VFyc8TjO0fc3+0TFzLYp0M8CH8R7+iA7jlStjDpdZFDWT7idK0afHJaa+gdos+YyIUcIHa5vdh1oiF0Ho8cab7v/HQMhnKitEmtXMEfwU85mSegXxwZt4bl/3MyvOg6OAEo0g0riKVUPUJSXameacGKwalkDFY0nGb0V0UoWmOheNEuBtvOSpWgU24EHWCuC3qWN5TrVgJZxHWugoAi7wIfzEMXRAWq0tBUZtIzBRB9dXSknyaLkSAXn89Mud4b6fR20iQW+bltHHhod6jzVegqNfPI+roIYREDHD7m2P2WQvZtG9SBMo+qv+d/uRYEZPJFxYWZIvUOU6eMiLNstQTJXE6Ov+mYiOGdkGceleMJQUaa471Q/ZiGMmh0KzwdPTLEw==
address@hidden
One thought: if the system is I/O bound, then you might think about offloading
the filing and decoding of data-products to back-end LDM systems that request
from the original system, which would then be relegated to requesting and
relaying only.
> Here's what I've been given ...
> =====================================================================================
> Sen Chiao Tue, Jul 17, 11:21 AM (21 hours ago) to me
>
> We haven’t stream GOES 16 (high def data). IT’s a huge amount and higher
> frequency.
>
> Sen
>
> > On Jul 17, 2018, at 11:19 AM, Greg Bulman <address@hidden> wrote:
> >
> > And this includes what you intend to process in the near future ... the
> high def data?
> >
> > On Tue, Jul 17, 2018 at 11:10 AM Sen Chiao <address@hidden> wrote:
> > I don't exactly know the amount, but you can see everything (data coming
> in) is under /ldm_data
> > We process mainly models (GFS, RUC, HRRR & NAM), satellite (GOES West and
> GOES East) and radio sondes (4 sites around the northern California).
> >
> > Sen
> >
> > On Tue, Jul 17, 2018 at 11:04 AM Greg Bulman <address@hidden> wrote:
> > Sen,
> >
> > ?In case they ask ... do you have a canned answer to ?
> > "amount or type of data we download and then how its processed"
> > ?
> > -Greg?
> [ldm@titan ~]$ /usr/local/ldm/bin/ldmadmin plotmetrics
> No matching data
>
> TITAN:
> (4) 2.0TB SATA 7.2K RPM in a RAID 5 /dev/sdb1 5.4T 2.4T 2.8T 47%
> /data
> Processor = Xeon(R) X5660 2.80GHz
> NumCPU = 24 (probably hyperthreading on)
> RAM = 24576
> load average: 3.65, 4.26, 4.34
>
> DSK |sdb | busy 101% | read 278 | write 3765 | KiB/r 17 ||
> KiB/w 11 | MBr/s 0.46 | MBw/s 4.20 | avq 152.78 | avio 2.47 ms
> |
>
> 3370 - 0K 120.1M 0K
> 49% pqact
> 3368 - 0K 69316K 0K
> 28% pqact
> 29553 - 25376K 0K 0K
> 10% clamscan
> 3378 - 0K 7344K 0K
> 3% pqact
> 1645 - 0K 6772K 0K
> 3% jbd2/sdb1-8
> 9677 - 0K 5608K 0K
> 2% dcgrib2
> 13547 - 5032K 0K 0K
> 2% find
> 3377 - 284K 4016K 20K
> 2% pqact
> 9202 - 1512K 8K 4K
> 2% gdplot2
>
> Everything seems to run mainly on cpu01
> PRC | sys 7.80s | user 17.56s | | #proc 649 | #trun
> 2 | #tslpi 764 | #tslpu 0 | #zombie 0 | clones 37 |
> | #exit 36 |
> CPU | sys 62% | user 161% | irq 4% | | idle
> 2118% | wait 56% | | steal 0% | guest 0% | curf
> 1.75GHz | curscal 62% |
> cpu | sys 3% | user 44% | irq 0% | | idle
> 19% | cpu000 w 34% | | steal 0% | guest 0% | curf
> 1.60GHz | curscal 57% |
> cpu | sys 6% | user 28% | irq 0% | | idle
> 59% | cpu001 w 7% | | steal 0% | guest 0% | curf
> 2.53GHz | curscal 90% |
> cpu | sys 9% | user 12% | irq 0% | | idle
> 78% | cpu004 w 0% | | steal 0% | guest 0% | curf
> 2.00GHz | curscal 71% |
> cpu | sys 6% | user 11% | irq 1% | | idle
> 82% | cpu003 w 1% | | steal 0% | guest 0% | curf
> 1.60GHz | curscal 57% |
> cpu | sys 6% | user 9% | irq 0% | | idle
> 85% | cpu016 w 0% | | steal 0% | guest 0% | curf
> 2.79GHz | curscal 100% |
> cpu | sys 4% | user 10% | irq 1% | | idle
> 83% | cpu002 w 2% | | steal 0% | guest 0% | curf
> 1.60GHz | curscal 57% |
> cpu | sys 4% | user 10% | irq 1% | | idle
> 84% | cpu013 w 1% | | steal 0% | guest 0% | curf
> 1.60GHz | curscal 57% |
> cpu | sys 1% | user 9% | irq 0% | | idle
> 85% | cpu012 w 5% | | steal 0% | guest 0% | curf
> 2.79GHz | curscal 100% |
> cpu | sys 3% | user 3% | irq 1% | | idle
> 93% | cpu007 w 0% | | steal 0% | guest 0% | curf
> 1.60GHz | curscal 57% |
> cpu | sys 3% | user 4% | irq 0% | | idle
> 93% | cpu005 w 0% | | steal 0% | guest 0% | curf
> 1.60GHz | curscal 57% |
> cpu | sys 3% | user 2% | irq 0% | | idle
> 95% | cpu006 w 0% | | steal 0% | guest 0% | curf
> 1.60GHz | curscal 57% |
> cpu | sys 3% | user 4% | irq 0% | | idle
> 93% | cpu009 w 0% | | steal 0% | guest 0% | curf
> 1.60GHz | curscal 57% |
> cpu | sys 2% | user 3% | irq 0% | | idle
> 95% | cpu021 w 0% | | steal 0% | guest 0% | curf
> 1.60GHz | curscal 57% |
> cpu | sys 3% | user 2% | irq 0% | | idle
> 95% | cpu008 w 1% | | steal 0% | guest 0% | curf
> 1.60GHz | curscal 57% |
> cpu | sys 2% | user 3% | irq 0% | | idle
> 95% | cpu011 w 1% | | steal 0% | guest 0% | curf
> 1.60GHz | curscal 57% |
> cpu | sys 1% | user 1% | irq 0% | | idle
> 96% | cpu015 w 1% | | steal 0% | guest 0% | curf
> 1.60GHz | curscal 57% |
> cpu | sys 1% | user 1% | irq 0% | | idle
> 97% | cpu010 w 0% | | steal 0% | guest 0% | curf
> 1.60GHz | curscal 57% |
> cpu | sys 1% | user 2% | irq 0% | | idle
> 97% | cpu017 w 0% | | steal 0% | guest 0% | curf
> 1.60GHz | curscal 57% |
> cpu | sys 1% | user 1% | irq 0% | | idle
> 97% | cpu014 w 1% | | steal 0% | guest 0% | curf
> 1.60GHz | curscal 57% |
> cpu | sys 1% | user 1% | irq 0% | | idle
> 98% | cpu019 w 1% | | steal 0% | guest 0% | curf
> 1.60GHz | curscal 57% |
> cpu | sys 1% | user 1% | irq 0% | | idle
> 98% | cpu022 w 0% | | steal 0% | guest 0% | curf
> 1.60GHz | curscal 57% |
> cpu | sys 1% | user 1% | irq 0% | | idle
> 98% | cpu018 w 0% | | steal 0% | guest 0% | curf
> 1.60GHz | curscal 57% |
> cpu | sys 0% | user 0% | irq 0% | | idle
> 100% | cpu020 w 0% | | steal 0% | guest 0% | curf
> 1.60GHz | curscal 57% |
> cpu | sys 0% | user 0% | irq 0% | | idle
> 100% | cpu023 w 0% | | steal 0% | guest 0% | curf
> 1.60GHz | curscal 57% |
> CPL | avg1 2.04 | avg5 2.10 | | avg15 2.20 |
> | | csw 3068954 | intr 228250 | | |
> numcpu 24
>
> ?If you need more info please let me know. I can bring in our researcher
> too who
> hopefully can add info.?
Regards,
Steve Emmerson
Ticket Details
===================
Ticket ID: MIW-275261
Department: Support LDM
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata
inquiry tracking system and then made publicly available through the web. If
you do not want to have your interactions made available in this way, you must
let us know in each email you send to us.
