[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[THREDDS #ZIZ-818863]: Thredds inflation of data

Ok, I have added a "fix" to thredds/netcdf-Java.
Re: github pull request https://github.com/Unidata/thredds/pull/955

The change relevant to this support question is
that I modified H5tiledLayoutBB.java to allow the
java command to take an argument:
-Dthredds.h5iosp.inflate.buffersize=<integer>
This defines the buffer size to be used by
java.util.zip.InflateStream.
The size defaults to the current 512 bytes.
In order to set this for a server, there is, I think,
a setenv file or some such in which -D flags can be placed.
A snapshot should be available shortly on our nexus server.


> That apparently allocates 512Kbyte as the input buffer (IBM recommend either 
> 256Kbytes or 512Kbytes for best deflate performance).  Note that my example 
> below (particularly having to create an instance of java.util.zip.Inflater) 
> might not be spot on, they key is to be creating the 
> java.util.zip.InflaterInputStream with the 3 parameters (described in the 
> Oracle JDK doc linked below), rather than just the one parameter as it is 
> currently.
> 
> https://docs.oracle.com/javase/8/docs/api/java/util/zip/InflaterInputStream.html#InflaterInputStream-java.io.InputStream-java.util.zip.Inflater-int-
> 
> 
> Thanks
> 
> Martyn
> 
> Martyn Hunt  
> Technical Lead, Mainframe
> Met Office  FitzRoy Road  Exeter  Devon  EX1 3PB  United Kingdom
> Tel: +44 (0)1392 884897  
> Email: address@hidden  Website: www.metoffice.gov.uk
> 
> 
> -----Original Message-----
> From: Unidata THREDDS Support [mailto:address@hidden]
> Sent: 15 November 2017 15:53
> To: Hunt, Martyn <address@hidden>
> Cc: address@hidden; Hunt, Martyn <address@hidden>
> Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> 
> We can do what you ask, except I need to know where the number 512k 
> (512*1024) came from. Is there anything special about it?
> 
> > I've heard back from IBM, and while there is a possibility something can be 
> > done in the JDK in the long term, the alternative solution would be for the 
> > line below from H5tiledLayoutBB.java:
> >
> > java.util.zip.InflaterInputStream inflater = new
> > java.util.zip.InflaterInputStream(in);
> >
> > to be replaced with something like:
> >
> > java.util.zip.Inflater inf = new java.util.zip.Inflater();
> > java.util.zip.InflaterInputStream inflater = new
> > java.util.zip.InflaterInputStream(in, inf, 512*1024);
> >
> > Although I'm sure there will be something else needed to setup the 
> > java.util.zip.Inflater.
> >
> > Would you be able to look to have the functionality added, all of this is 
> > doc'ed in the Oracle and IBM JDK's so it shouldn't break any 
> > implementations.  It would be great to get a test version we could try of 
> > Thredds 4.6 with this in.
> >
> > Thanks
> >
> > Martyn
> >
> > Martyn Hunt
> > Technical Lead, Mainframe
> > Met Office  FitzRoy Road  Exeter  Devon  EX1 3PB  United Kingdom
> > Tel: +44 (0)1392 884897
> > Email: address@hidden  Website: www.metoffice.gov.uk
> >
> > -----Original Message-----
> > From: Unidata THREDDS Support
> > [mailto:address@hidden]
> > Sent: 08 November 2017 20:56
> > To: Hunt, Martyn <address@hidden>
> > Cc: address@hidden; Hunt, Martyn
> > <address@hidden>
> > Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> >
> > Ok. If you need us to add some kind of functionality, let us know.
> >
> > > I'm waiting on IBM java support to look at it, I now suspect that to
> > > make best use of the hardware, there need to be some extra
> > > parameters set on the "in.read" (on line 104 of IO.java)
> > >
> > > From the IBM JDK doc:
> > >
> > > "The GZIPInputStream, DeflaterInputStream, and InflaterInputStream 
> > > classes provide a constructor that enables the input buffer size for the 
> > > deflate or inflate operation to be specified".
> > >
> > > Feel free to close/park this for now, I'm hoping IBM (as the vendor of 
> > > the hardware we are running on and the Compression Hardware as well as 
> > > the owners of the JDK) can  confirm my thinking and give us some 
> > > specifics on how to best get around this.
> > >
> > > Thanks
> > >
> > > Martyn
> > >
> > > Martyn Hunt
> > > Technical Lead, Mainframe
> > > Met Office  FitzRoy Road  Exeter  Devon  EX1 3PB  United Kingdom
> > > Tel: +44 (0)1392 884897
> > > Email: address@hidden  Website: www.metoffice.gov.uk
> > >
> > > -----Original Message-----
> > > From: Unidata THREDDS Support
> > > [mailto:address@hidden]
> > > Sent: 07 November 2017 18:48
> > > To: Hunt, Martyn <address@hidden>
> > > Cc: address@hidden; Hunt, Martyn
> > > <address@hidden>
> > > Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> > >
> > > Martyn- are you making any progress on this?
> > >
> > >
> > > > Yes I had to modify the code. I have attached a zip containing the
> > > > relevant .class and .java files.
> > > > If you want to do it some other way, let me know.
> > > > Basically now if you set the debug flag H5iosp.filter thru the
> > > > GUI, it will also dump the incoming and outgoing inflation sizes.
> > > >
> > > > > Did you have to rebuild the JAR files to get the flag on, or did you 
> > > > > manage to do it through ToolsUI?  If you had to rebuild it, any 
> > > > > chance you could share that build with me so I can retest here with 
> > > > > the flag on?
> > > > >
> > > > > Thanks
> > > > >
> > > > > Martyn
> > > > >
> > > > > Martyn Hunt
> > > > > Technical Lead, Mainframe
> > > > > Met Office  FitzRoy Road  Exeter  Devon  EX1 3PB  United Kingdom
> > > > > Tel: +44 (0)1392 884897
> > > > > Email: address@hidden  Website:
> > > > > www.metoffice.gov.uk
> > > > >
> > > > > -----Original Message-----
> > > > > From: Unidata THREDDS Support
> > > > > [mailto:address@hidden]
> > > > > Sent: 02 November 2017 20:09
> > > > > To: Hunt, Martyn <address@hidden>
> > > > > Cc: address@hidden; Hunt, Martyn
> > > > > <address@hidden>
> > > > > Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> > > > >
> > > > > Ok, got it. I turned on the debug flag in H5tiledLayoutBB and got the 
> > > > > attached output. Looks like everything is as it should be.
> > > > > inflate bytes in= 2296850 bytes out= 7077888  inflate bytes in=
> > > > > 2292229 bytes out= 7077888  inflate bytes in= 2286027 bytes out=
> > > > > 7077888  inflate bytes in= 2281751 bytes out= 7077888  inflate
> > > > > bytes in= 2278732 bytes out= 7077888  inflate bytes in= 2270775
> > > > > bytes out=
> > > > > 7077888  inflate bytes in= 2265708 bytes out= 7077888  inflate
> > > > > bytes in= 2267066 bytes out= 7077888  inflate bytes in= 2280367
> > > > > bytes out=
> > > > > 7077888  inflate bytes in= 2292362 bytes out= 7077888  inflate
> > > > > bytes in= 2300321 bytes out= 7077888  inflate bytes in= 2299787
> > > > > bytes out=
> > > > > 7077888  inflate bytes in= 2295357 bytes out= 7077888  inflate
> > > > > bytes in= 2278990 bytes out= 7077888  inflate bytes in= 2258863
> > > > > bytes out=
> > > > > 7077888  inflate bytes in= 2236726 bytes out= 7077888  inflate
> > > > > bytes in= 2219994 bytes out= 7077888  inflate bytes in= 2197604
> > > > > bytes out=
> > > > > 7077888  inflate bytes in= 2178159 bytes out= 7077888  inflate
> > > > > bytes in= 2164433 bytes out= 7077888  inflate bytes in= 2151000
> > > > > bytes out=
> > > > > 7077888  inflate bytes in= 21
> > > 40669 bytes out= 7077888  inflate bytes in= 2136334 bytes out=
> > > 7077888 inflate bytes in= 2129073 bytes out= 7077888  inflate bytes
> > > in=
> > > 2125608 bytes out= 7077888  inflate bytes in= 2115306 bytes out=
> > > 7077888  inflate bytes in= 2105754 bytes out= 7077888  inflate bytes
> > > in= 2091580 bytes out= 7077888  inflate bytes in= 2075651 bytes out=
> > > 7077888  inflate bytes in= 2054752 bytes out= 7077888  inflate bytes
> > > in= 2034731 bytes out= 7077888  inflate bytes in= 2016893 bytes out=
> > > 7077888  inflate bytes in= 1998545 bytes out= 7077888  inflate bytes
> > > in= 1971600 bytes out= 7077888  inflate bytes in= 1955350 bytes out=
> > > 7077888  inflate bytes in= 1950053 bytes out= 7077888  inflate bytes
> > > in= 1943446 bytes out= 7077888  inflate bytes in= 1935437 bytes out=
> > > 7077888  inflate bytes in= 1920268 bytes out= 7077888  inflate bytes
> > > in= 1901586 bytes out= 7077888  inflate bytes in= 1886620 bytes out=
> > > 7077888  inflate bytes in= 1881664 bytes out= 7077888  inflate bytes
> > > in= 1884698 bytes out= 70778
> > 8
> > > 8  inflate bytes in= 1895758 bytes out= 7077888  inflate bytes in=
> > > 1907741 bytes out= 7077888  inflate bytes in= 1927959 bytes out=
> > > 7077888  inflate bytes in= 1952942 bytes out= 7077888  inflate bytes
> > > in= 1984151 bytes out= 7077888  inflate bytes in= 2003863 bytes out=
> > > 7077888  inflate bytes in= 2001965 bytes out= 7077888  inflate bytes
> > > in= 1976532 bytes out= 7077888  inflate bytes in= 1944348 bytes out=
> > > 7077888  inflate bytes in= 1915409 bytes out= 7077888  inflate bytes
> > > in= 1887227 bytes out= 7077888  inflate bytes in= 1868649 bytes out=
> > > 7077888  inflate bytes in= 1865444 bytes out= 7077888  inflate bytes
> > > in= 1870893 bytes out= 7077888  inflate bytes in= 1883281 bytes out=
> > > 7077888  inflate bytes in= 1899309 bytes out= 7077888  inflate bytes
> > > in= 30905 bytes out= 7077888  inflate bytes in= 30905 bytes out=
> > > 7077888  inflate bytes in= 30905 bytes out= 7077888  inflate bytes
> > > in=
> > > 30905 bytes out= 7077888  inflate bytes in= 30905 bytes out= 7077888
> > > inflate bytes in= 30905 bytes
> > out= 7077888  inflate bytes in= 30905 bytes out= 7077888  inflate
> > bytes in= 30905 bytes out= 7077888  inflate bytes in= 30905 bytes out=
> > 7077888  inflate bytes in= 30905 bytes out= 7077888  inflate bytes in=
> > 30905 bytes out= 7077888
> > > > >
> > > > >
> > > > > > You should be able to pull it from here
> > > > > >
> > > > > > https://drive.google.com/file/d/0B5gLZiYwJyg_YTY0QllISlZwaVU/view?
> > > > > > usp=
> > > > > > sharing
> > > > > >
> > > > > > Thanks
> > > > > >
> > > > > > Martyn
> > > > > >
> > > > > > Martyn Hunt
> > > > > > Technical Lead, Mainframe
> > > > > > Met Office  FitzRoy Road  Exeter  Devon  EX1 3PB  United
> > > > > > Kingdom
> > > > > > Tel: +44 (0)1392 884897
> > > > > > Email: address@hidden  Website:
> > > > > > www.metoffice.gov.uk
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Unidata THREDDS Support
> > > > > > [mailto:address@hidden]
> > > > > > Sent: 01 November 2017 18:05
> > > > > > To: Hunt, Martyn <address@hidden>
> > > > > > Cc: address@hidden; Hunt, Martyn
> > > > > > <address@hidden>
> > > > > > Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> > > > > >
> > > > > > Sure, go ahead and send me the file.
> > > > > >
> > > > > > > No, but I'd not set that up for the ToolsUI, but it definitely is 
> > > > > > > inflating the data.  Could I send you an example NetCDF file, as 
> > > > > > > you are more familiar with this you may get a better result.
> > > > > > >
> > > > > > > Thanks
> > > > > > >
> > > > > > > Martyn
> > > > > > >
> > > > > > > Martyn Hunt
> > > > > > > Technical Lead, Mainframe
> > > > > > > Met Office  FitzRoy Road  Exeter  Devon  EX1 3PB  United
> > > > > > > Kingdom
> > > > > > > Tel: +44 (0)1392 884897
> > > > > > > Email: address@hidden  Website:
> > > > > > > www.metoffice.gov.uk
> > > > > > >
> > > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Unidata THREDDS Support
> > > > > > > [mailto:address@hidden]
> > > > > > > Sent: 01 November 2017 15:53
> > > > > > > To: Hunt, Martyn <address@hidden>
> > > > > > > Cc: address@hidden; Hunt, Martyn
> > > > > > > <address@hidden>
> > > > > > > Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> > > > > > >
> > > > > > > Odd. Did you see any log output from java.util.zip?
> > > > > > >
> > > > > > > > No extra output appears in the console when following those 
> > > > > > > > instructions.  I've tried turning on some other debug flags and 
> > > > > > > > they do produce data, so not sure why the H5iosp one isn't 
> > > > > > > > producing any extra output on the console.  Where can we go 
> > > > > > > > next?
> > > > > > > >
> > > > > > > > Thanks
> > > > > > > >
> > > > > > > > Martyn
> > > > > > > >
> > > > > > > > Martyn Hunt
> > > > > > > > Technical Lead, Mainframe
> > > > > > > > Met Office  FitzRoy Road  Exeter  Devon  EX1 3PB  United
> > > > > > > > Kingdom
> > > > > > > > Tel: +44 (0)1392 884897
> > > > > > > > Email: address@hidden  Website:
> > > > > > > > www.metoffice.gov.uk
> > > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: Unidata THREDDS Support
> > > > > > > > [mailto:address@hidden]
> > > > > > > > Sent: 30 October 2017 17:47
> > > > > > > > To: Hunt, Martyn <address@hidden>
> > > > > > > > Cc: address@hidden
> > > > > > > > Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> > > > > > > >
> > > > > > > > You can run ToolsUI from the command line locally.
> > > > > > > > You will need to download the ToolsUI jar at
> > > > > > > > http://artifacts.unidata.ucar.edu/content/repositories/uni
> > > > > > > > da ta -releases/edu/ucar/toolsUI/4.6.10/toolsUI-4.6.10.jar
> > > > > > > >
> > > > > > > > Once you have it, you should be able to run the command:
> > > > > > > > java -jar toolsUI-4.6.10.jar This will start up ToolsUI.
> > > > > > > > You then need to do the following:
> > > > > > > >
> > > > > > > > 1. Follow the menu Debug->Debug Flags->H5iosp and click on the 
> > > > > > > > "filter" entry.
> > > > > > > >
> > > > > > > > 2. You will then need to put the path to the file in the file 
> > > > > > > > window at the top and hit return.
> > > > > > > >
> > > > > > > > 3. The simplest way to invoke the inflation code is to right 
> > > > > > > > click on one of the compressed variables listed in the 
> > > > > > > > right-hand window. Then click the write binary data entry; it 
> > > > > > > > should pick a default directory and default output file name. 
> > > > > > > > Click close.
> > > > > > > >
> > > > > > > > The debug output should appear in the console window in which 
> > > > > > > > you did the jar command.
> > > > > > > >
> > > > > > > > It does not look very useful to me, but you should also be able 
> > > > > > > > to get the java.util.zip log information and that may tell you 
> > > > > > > > more since it should only contain the inflation calls from 
> > > > > > > > H5tiledLayoutBB.
> > > > > > > >
> > > > > > > >
> > > > > > > > > Do I need to run ToolsUI on the server that is running 
> > > > > > > > > thredds, or can I run it locally on my PC?  I've had a play 
> > > > > > > > > with ToolsUI and I can't see where it is writing the debug 
> > > > > > > > > info to?
> > > > > > > > >
> > > > > > > > > Thanks
> > > > > > > > >
> > > > > > > > > Martyn
> > > > > > > > >
> > > > > > > > > Martyn Hunt
> > > > > > > > > Technical Lead, Mainframe Met Office  FitzRoy Road  
> > > > > > > > > Exeter  Devon  EX1 3PB  United Kingdom
> > > > > > > > > Tel: +44 (0)1392 884897
> > > > > > > > > Email: address@hidden  Website:
> > > > > > > > > www.metoffice.gov.uk
> > > > > > > > >
> > > > > > > > > -----Original Message-----
> > > > > > > > > From: Unidata THREDDS Support
> > > > > > > > > [mailto:address@hidden]
> > > > > > > > > Sent: 27 October 2017 23:00
> > > > > > > > > To: Hunt, Martyn <address@hidden>
> > > > > > > > > Cc: address@hidden
> > > > > > > > > Subject: [THREDDS #ZIZ-818863]: Thredds inflation of
> > > > > > > > > data
> > > > > > > > >
> > > > > > > > > Actually, ToolsUI supports enabling/disabling debug flags 
> > > > > > > > > from within the running program. No need to create your own 
> > > > > > > > > program that wraps it.
> > > > > > > > >
> > > > > > > > > In the ToolsUI main menu, go to "Debug"->"Debug 
> > > > > > > > > Flags"->"H5iosp" and select the "filter" menu item.
> > > > > > > > >
> > > > > > > > > Ticket Details
> > > > > > > > > ===================
> > > > > > > > > Ticket ID: ZIZ-818863
> > > > > > > > > Department: Support THREDDS
> > > > > > > > > Priority: Normal
> > > > > > > > > Status: Closed
> > > > > > > > > ===================
> > > > > > > > > NOTE: All email exchanges with Unidata User Support are 
> > > > > > > > > recorded in the Unidata inquiry tracking system and then made 
> > > > > > > > > publicly available through the web.  If you do not want to 
> > > > > > > > > have your interactions made available in this way, you must 
> > > > > > > > > let us know in each email you send to us.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > =Dennis Heimbigner
> > > > > > > > Unidata
> > > > > > > >
> > > > > > > >
> > > > > > > > Ticket Details
> > > > > > > > ===================
> > > > > > > > Ticket ID: ZIZ-818863
> > > > > > > > Department: Support THREDDS
> > > > > > > > Priority: Normal
> > > > > > > > Status: Open
> > > > > > > > ===================
> > > > > > > > NOTE: All email exchanges with Unidata User Support are 
> > > > > > > > recorded in the Unidata inquiry tracking system and then made 
> > > > > > > > publicly available through the web.  If you do not want to have 
> > > > > > > > your interactions made available in this way, you must let us 
> > > > > > > > know in each email you send to us.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > =Dennis Heimbigner
> > > > > > > Unidata
> > > > > > >
> > > > > > >
> > > > > > > Ticket Details
> > > > > > > ===================
> > > > > > > Ticket ID: ZIZ-818863
> > > > > > > Department: Support THREDDS
> > > > > > > Priority: Normal
> > > > > > > Status: Open
> > > > > > > ===================
> > > > > > > NOTE: All email exchanges with Unidata User Support are recorded 
> > > > > > > in the Unidata inquiry tracking system and then made publicly 
> > > > > > > available through the web.  If you do not want to have your 
> > > > > > > interactions made available in this way, you must let us know in 
> > > > > > > each email you send to us.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > =Dennis Heimbigner
> > > > > > Unidata
> > > > > >
> > > > > >
> > > > > > Ticket Details
> > > > > > ===================
> > > > > > Ticket ID: ZIZ-818863
> > > > > > Department: Support THREDDS
> > > > > > Priority: Normal
> > > > > > Status: Open
> > > > > > ===================
> > > > > > NOTE: All email exchanges with Unidata User Support are recorded in 
> > > > > > the Unidata inquiry tracking system and then made publicly 
> > > > > > available through the web.  If you do not want to have your 
> > > > > > interactions made available in this way, you must let us know in 
> > > > > > each email you send to us.
> > > > > >
> > > > > >
> > > > > >
> > > > > > You should be able to pull it from here
> > > > > >
> > > > > > https://drive.google.com/file/d/0B5gLZiYwJyg_YTY0QllISlZwaVU/view?
> > > > > > usp=sharing
> > > > > >
> > > > > > Thanks
> > > > > >
> > > > > > Martyn
> > > > > >
> > > > > > Martyn Hunt
> > > > > > Technical Lead, Mainframe
> > > > > > Met Office  FitzRoy Road  Exeter  Devon  EX1 3PB  United
> > > > > > Kingdom
> > > > > > Tel: +44 (0)1392 884897
> > > > > > Email: address@hidden  Website:
> > > > > > www.metoffice.gov.uk
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Unidata THREDDS Support
> > > > > > [mailto:address@hidden]
> > > > > > Sent: 01 November 2017 18:05
> > > > > > To: Hunt, Martyn <address@hidden>
> > > > > > Cc: address@hidden; Hunt, Martyn
> > > > > > <address@hidden>
> > > > > > Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> > > > > >
> > > > > > Sure, go ahead and send me the file.
> > > > > >
> > > > > > > No, but I'd not set that up for the ToolsUI, but it definitely is 
> > > > > > > inflating the data.  Could I send you an example NetCDF file, as 
> > > > > > > you are more familiar with this you may get a better result.
> > > > > > >
> > > > > > > Thanks
> > > > > > >
> > > > > > > Martyn
> > > > > > >
> > > > > > > Martyn Hunt
> > > > > > > Technical Lead, Mainframe
> > > > > > > Met Office  FitzRoy Road  Exeter  Devon  EX1 3PB  United
> > > > > > > Kingdom
> > > > > > > Tel: +44 (0)1392 884897
> > > > > > > Email: address@hidden  Website:
> > > > > > > www.metoffice.gov.uk
> > > > > > >
> > > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Unidata THREDDS Support
> > > > > > > [mailto:address@hidden]
> > > > > > > Sent: 01 November 2017 15:53
> > > > > > > To: Hunt, Martyn <address@hidden>
> > > > > > > Cc: address@hidden; Hunt, Martyn
> > > > > > > <address@hidden>
> > > > > > > Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> > > > > > >
> > > > > > > Odd. Did you see any log output from java.util.zip?
> > > > > > >
> > > > > > > > No extra output appears in the console when following those 
> > > > > > > > instructions.  I've tried turning on some other debug flags and 
> > > > > > > > they do produce data, so not sure why the H5iosp one isn't 
> > > > > > > > producing any extra output on the console.  Where can we go 
> > > > > > > > next?
> > > > > > > >
> > > > > > > > Thanks
> > > > > > > >
> > > > > > > > Martyn
> > > > > > > >
> > > > > > > > Martyn Hunt
> > > > > > > > Technical Lead, Mainframe
> > > > > > > > Met Office  FitzRoy Road  Exeter  Devon  EX1 3PB  United
> > > > > > > > Kingdom
> > > > > > > > Tel: +44 (0)1392 884897
> > > > > > > > Email: address@hidden  Website:
> > > > > > > > www.metoffice.gov.uk
> > > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: Unidata THREDDS Support
> > > > > > > > [mailto:address@hidden]
> > > > > > > > Sent: 30 October 2017 17:47
> > > > > > > > To: Hunt, Martyn <address@hidden>
> > > > > > > > Cc: address@hidden
> > > > > > > > Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> > > > > > > >
> > > > > > > > You can run ToolsUI from the command line locally.
> > > > > > > > You will need to download the ToolsUI jar at
> > > > > > > > http://artifacts.unidata.ucar.edu/content/repositories/uni
> > > > > > > > da ta -releases/edu/ucar/toolsUI/4.6.10/toolsUI-4.6.10.jar
> > > > > > > >
> > > > > > > > Once you have it, you should be able to run the command:
> > > > > > > > java -jar toolsUI-4.6.10.jar This will start up ToolsUI.
> > > > > > > > You then need to do the following:
> > > > > > > >
> > > > > > > > 1. Follow the menu Debug->Debug Flags->H5iosp and click on the 
> > > > > > > > "filter" entry.
> > > > > > > >
> > > > > > > > 2. You will then need to put the path to the file in the file 
> > > > > > > > window at the top and hit return.
> > > > > > > >
> > > > > > > > 3. The simplest way to invoke the inflation code is to right 
> > > > > > > > click on one of the compressed variables listed in the 
> > > > > > > > right-hand window. Then click the write binary data entry; it 
> > > > > > > > should pick a default directory and default output file name. 
> > > > > > > > Click close.
> > > > > > > >
> > > > > > > > The debug output should appear in the console window in which 
> > > > > > > > you did the jar command.
> > > > > > > >
> > > > > > > > It does not look very useful to me, but you should also be able 
> > > > > > > > to get the java.util.zip log information and that may tell you 
> > > > > > > > more since it should only contain the inflation calls from 
> > > > > > > > H5tiledLayoutBB.
> > > > > > > >
> > > > > > > >
> > > > > > > > > Do I need to run ToolsUI on the server that is running 
> > > > > > > > > thredds, or can I run it locally on my PC?  I've had a play 
> > > > > > > > > with ToolsUI and I can't see where it is writing the debug 
> > > > > > > > > info to?
> > > > > > > > >
> > > > > > > > > Thanks
> > > > > > > > >
> > > > > > > > > Martyn
> > > > > > > > >
> > > > > > > > > Martyn Hunt
> > > > > > > > > Technical Lead, Mainframe Met Office  FitzRoy Road  
> > > > > > > > > Exeter  Devon  EX1 3PB  United Kingdom
> > > > > > > > > Tel: +44 (0)1392 884897
> > > > > > > > > Email: address@hidden  Website:
> > > > > > > > > www.metoffice.gov.uk
> > > > > > > > >
> > > > > > > > > -----Original Message-----
> > > > > > > > > From: Unidata THREDDS Support
> > > > > > > > > [mailto:address@hidden]
> > > > > > > > > Sent: 27 October 2017 23:00
> > > > > > > > > To: Hunt, Martyn <address@hidden>
> > > > > > > > > Cc: address@hidden
> > > > > > > > > Subject: [THREDDS #ZIZ-818863]: Thredds inflation of
> > > > > > > > > data
> > > > > > > > >
> > > > > > > > > Actually, ToolsUI supports enabling/disabling debug flags 
> > > > > > > > > from within the running program. No need to create your own 
> > > > > > > > > program that wraps it.
> > > > > > > > >
> > > > > > > > > In the ToolsUI main menu, go to "Debug"->"Debug 
> > > > > > > > > Flags"->"H5iosp" and select the "filter" menu item.
> > > > > > > > >
> > > > > > > > > Ticket Details
> > > > > > > > > ===================
> > > > > > > > > Ticket ID: ZIZ-818863
> > > > > > > > > Department: Support THREDDS
> > > > > > > > > Priority: Normal
> > > > > > > > > Status: Closed
> > > > > > > > > ===================
> > > > > > > > > NOTE: All email exchanges with Unidata User Support are 
> > > > > > > > > recorded in the Unidata inquiry tracking system and then made 
> > > > > > > > > publicly available through the web.  If you do not want to 
> > > > > > > > > have your interactions made available in this way, you must 
> > > > > > > > > let us know in each email you send to us.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > =Dennis Heimbigner
> > > > > > > > Unidata
> > > > > > > >
> > > > > > > >
> > > > > > > > Ticket Details
> > > > > > > > ===================
> > > > > > > > Ticket ID: ZIZ-818863
> > > > > > > > Department: Support THREDDS
> > > > > > > > Priority: Normal
> > > > > > > > Status: Open
> > > > > > > > ===================
> > > > > > > > NOTE: All email exchanges with Unidata User Support are 
> > > > > > > > recorded in the Unidata inquiry tracking system and then made 
> > > > > > > > publicly available through the web.  If you do not want to have 
> > > > > > > > your interactions made available in this way, you must let us 
> > > > > > > > know in each email you send to us.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > =Dennis Heimbigner
> > > > > > > Unidata
> > > > > > >
> > > > > > >
> > > > > > > Ticket Details
> > > > > > > ===================
> > > > > > > Ticket ID: ZIZ-818863
> > > > > > > Department: Support THREDDS
> > > > > > > Priority: Normal
> > > > > > > Status: Open
> > > > > > > ===================
> > > > > > > NOTE: All email exchanges with Unidata User Support are recorded 
> > > > > > > in the Unidata inquiry tracking system and then made publicly 
> > > > > > > available through the web.  If you do not want to have your 
> > > > > > > interactions made available in this way, you must let us know in 
> > > > > > > each email you send to us.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > =Dennis Heimbigner
> > > > > > Unidata
> > > > > >
> > > > > >
> > > > > > Ticket Details
> > > > > > ===================
> > > > > > Ticket ID: ZIZ-818863
> > > > > > Department: Support THREDDS
> > > > > > Priority: Normal
> > > > > > Status: Open
> > > > > > ===================
> > > > > > NOTE: All email exchanges with Unidata User Support are recorded in 
> > > > > > the Unidata inquiry tracking system and then made publicly 
> > > > > > available through the web.  If you do not want to have your 
> > > > > > interactions made available in this way, you must let us know in 
> > > > > > each email you send to us.
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > > =Dennis Heimbigner
> > > > > Unidata
> > > > >
> > > > >
> > > > > Ticket Details
> > > > > ===================
> > > > > Ticket ID: ZIZ-818863
> > > > > Department: Support THREDDS
> > > > > Priority: Normal
> > > > > Status: Open
> > > > > ===================
> > > > > NOTE: All email exchanges with Unidata User Support are recorded in 
> > > > > the Unidata inquiry tracking system and then made publicly available 
> > > > > through the web.  If you do not want to have your interactions made 
> > > > > available in this way, you must let us know in each email you send to 
> > > > > us.
> > > > >
> > > > >
> > > > >
> > > >
> > > > =Dennis Heimbigner
> > > > Unidata
> > > >
> > >
> > > =Dennis Heimbigner
> > > Unidata
> > >
> > >
> > > Ticket Details
> > > ===================
> > > Ticket ID: ZIZ-818863
> > > Department: Support THREDDS
> > > Priority: Normal
> > > Status: Closed
> > > ===================
> > > NOTE: All email exchanges with Unidata User Support are recorded in the 
> > > Unidata inquiry tracking system and then made publicly available through 
> > > the web.  If you do not want to have your interactions made available in 
> > > this way, you must let us know in each email you send to us.
> > >
> > >
> > >
> >
> > =Dennis Heimbigner
> > Unidata
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: ZIZ-818863
> > Department: Support THREDDS
> > Priority: Normal
> > Status: Closed
> > ===================
> > NOTE: All email exchanges with Unidata User Support are recorded in the 
> > Unidata inquiry tracking system and then made publicly available through 
> > the web.  If you do not want to have your interactions made available in 
> > this way, you must let us know in each email you send to us.
> >
> >
> >
> 
> =Dennis Heimbigner
> Unidata
> 
> 
> Ticket Details
> ===================
> Ticket ID: ZIZ-818863
> Department: Support THREDDS
> Priority: Normal
> Status: Open
> ===================
> NOTE: All email exchanges with Unidata User Support are recorded in the 
> Unidata inquiry tracking system and then made publicly available through the 
> web.  If you do not want to have your interactions made available in this 
> way, you must let us know in each email you send to us.
> 
> 
> 

=Dennis Heimbigner
  Unidata


Ticket Details
===================
Ticket ID: ZIZ-818863
Department: Support THREDDS
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.