Showing entries tagged [multithreading]

Multithreaded TDS Clients (1)

Jeff McWhirter was studying whether he could speed up IDV data access by multithreading on the client. He put together a test which fetched the same data over and over, with 1-10 client threads.

This is hitting motherlode TDS, which has 2 quad-core processors, running solaris x86 and ZFS. It asks for one field from an NCEP model file, stored in GRIB1 compression, so theres some amount of computation. Actual URL:

http://motherlode.ucar.edu:8081/thredds/dodsC/model/NCEP/NAM/CONUS_80km/NAM_CONUS_80km_20090313_1200.grib1.dods?Temperature[0:1:0][0:1:18][0:1:64][0:1:92]

19 * 65 *93  = 114855 floats = 459K bytes.

The result shows that you can't get more than 2.5x speedup in this particular case. I will try this on netcdf files, which should be only pure IO. Also interesting to see if the opendap stack causes significant overhead.

A more realistic test is to ask for successive time slices of non-cached data. These may be better opportunities for overlapping IO.

Unidata Developer's Blog
A weblog about software development by Unidata developers*
Unidata Developer's Blog
A weblog about software development by Unidata developers*

Welcome

FAQs

News@Unidata blog

Take a poll!

What if we had an ongoing user poll in here?

Browse By Topic
Browse by Topic
« August 2019
SunMonTueWedThuFriSat
    
1
2
3
4
6
7
8
9
10
11
13
14
15
17
18
20
21
22
23
24
25
26
27
28
29
30
31
       
Today