[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[THREDDS #ZIZ-818863]: Thredds inflation of data
- Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
- Date: Tue, 07 Nov 2017 11:48:23 -0700
Martyn- are you making any progress on this?
> Yes I had to modify the code. I have attached a zip containing
> the relevant .class and .java files.
> If you want to do it some other way, let me know.
> Basically now if you set the debug flag H5iosp.filter thru the GUI,
> it will also dump the incoming and outgoing inflation sizes.
>
> > Did you have to rebuild the JAR files to get the flag on, or did you manage
> > to do it through ToolsUI? If you had to rebuild it, any chance you could
> > share that build with me so I can retest here with the flag on?
> >
> > Thanks
> >
> > Martyn
> >
> > Martyn Hunt
> > Technical Lead, Mainframe
> > Met Office FitzRoy Road Exeter Devon EX1 3PB United Kingdom
> > Tel: +44 (0)1392 884897
> > Email: address@hidden Website: www.metoffice.gov.uk
> >
> > -----Original Message-----
> > From: Unidata THREDDS Support [mailto:address@hidden]
> > Sent: 02 November 2017 20:09
> > To: Hunt, Martyn <address@hidden>
> > Cc: address@hidden; Hunt, Martyn <address@hidden>
> > Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> >
> > Ok, got it. I turned on the debug flag in H5tiledLayoutBB and got the
> > attached output. Looks like everything is as it should be.
> > inflate bytes in= 2296850 bytes out= 7077888 inflate bytes in= 2292229
> > bytes out= 7077888 inflate bytes in= 2286027 bytes out= 7077888 inflate
> > bytes in= 2281751 bytes out= 7077888 inflate bytes in= 2278732 bytes out=
> > 7077888 inflate bytes in= 2270775 bytes out= 7077888 inflate bytes in=
> > 2265708 bytes out= 7077888 inflate bytes in= 2267066 bytes out= 7077888
> > inflate bytes in= 2280367 bytes out= 7077888 inflate bytes in= 2292362
> > bytes out= 7077888 inflate bytes in= 2300321 bytes out= 7077888 inflate
> > bytes in= 2299787 bytes out= 7077888 inflate bytes in= 2295357 bytes out=
> > 7077888 inflate bytes in= 2278990 bytes out= 7077888 inflate bytes in=
> > 2258863 bytes out= 7077888 inflate bytes in= 2236726 bytes out= 7077888
> > inflate bytes in= 2219994 bytes out= 7077888 inflate bytes in= 2197604
> > bytes out= 7077888 inflate bytes in= 2178159 bytes out= 7077888 inflate
> > bytes in= 2164433 bytes out= 7077888 inflate bytes in= 2151000 bytes out=
> > 7077888 inflate bytes in= 21
40669 bytes out= 7077888 inflate bytes in= 2136334 bytes out= 7077888
inflate bytes in= 2129073 bytes out= 7077888 inflate bytes in= 2125608 bytes
out= 7077888 inflate bytes in= 2115306 bytes out= 7077888 inflate bytes in=
2105754 bytes out= 7077888 inflate bytes in= 2091580 bytes out= 7077888
inflate bytes in= 2075651 bytes out= 7077888 inflate bytes in= 2054752 bytes
out= 7077888 inflate bytes in= 2034731 bytes out= 7077888 inflate bytes in=
2016893 bytes out= 7077888 inflate bytes in= 1998545 bytes out= 7077888
inflate bytes in= 1971600 bytes out= 7077888 inflate bytes in= 1955350 bytes
out= 7077888 inflate bytes in= 1950053 bytes out= 7077888 inflate bytes in=
1943446 bytes out= 7077888 inflate bytes in= 1935437 bytes out= 7077888
inflate bytes in= 1920268 bytes out= 7077888 inflate bytes in= 1901586 bytes
out= 7077888 inflate bytes in= 1886620 bytes out= 7077888 inflate bytes in=
1881664 bytes out= 7077888 inflate bytes in= 1884698 bytes out= 707788
8 inflate bytes in= 1895758 bytes out= 7077888 inflate bytes in= 1907741
bytes out= 7077888 inflate bytes in= 1927959 bytes out= 7077888 inflate bytes
in= 1952942 bytes out= 7077888 inflate bytes in= 1984151 bytes out= 7077888
inflate bytes in= 2003863 bytes out= 7077888 inflate bytes in= 2001965 bytes
out= 7077888 inflate bytes in= 1976532 bytes out= 7077888 inflate bytes in=
1944348 bytes out= 7077888 inflate bytes in= 1915409 bytes out= 7077888
inflate bytes in= 1887227 bytes out= 7077888 inflate bytes in= 1868649 bytes
out= 7077888 inflate bytes in= 1865444 bytes out= 7077888 inflate bytes in=
1870893 bytes out= 7077888 inflate bytes in= 1883281 bytes out= 7077888
inflate bytes in= 1899309 bytes out= 7077888 inflate bytes in= 30905 bytes
out= 7077888 inflate bytes in= 30905 bytes out= 7077888 inflate bytes in=
30905 bytes out= 7077888 inflate bytes in= 30905 bytes out= 7077888 inflate
bytes in= 30905 bytes out= 7077888 inflate bytes in= 30905 bytes
out= 7077888 inflate bytes in= 30905 bytes out= 7077888 inflate bytes in=
30905 bytes out= 7077888 inflate bytes in= 30905 bytes out= 7077888 inflate
bytes in= 30905 bytes out= 7077888 inflate bytes in= 30905 bytes out= 7077888
> >
> >
> > > You should be able to pull it from here
> > >
> > > https://drive.google.com/file/d/0B5gLZiYwJyg_YTY0QllISlZwaVU/view?usp=
> > > sharing
> > >
> > > Thanks
> > >
> > > Martyn
> > >
> > > Martyn Hunt
> > > Technical Lead, Mainframe
> > > Met Office FitzRoy Road Exeter Devon EX1 3PB United Kingdom
> > > Tel: +44 (0)1392 884897
> > > Email: address@hidden Website: www.metoffice.gov.uk
> > >
> > > -----Original Message-----
> > > From: Unidata THREDDS Support
> > > [mailto:address@hidden]
> > > Sent: 01 November 2017 18:05
> > > To: Hunt, Martyn <address@hidden>
> > > Cc: address@hidden; Hunt, Martyn
> > > <address@hidden>
> > > Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> > >
> > > Sure, go ahead and send me the file.
> > >
> > > > No, but I'd not set that up for the ToolsUI, but it definitely is
> > > > inflating the data. Could I send you an example NetCDF file, as you
> > > > are more familiar with this you may get a better result.
> > > >
> > > > Thanks
> > > >
> > > > Martyn
> > > >
> > > > Martyn Hunt
> > > > Technical Lead, Mainframe
> > > > Met Office FitzRoy Road Exeter Devon EX1 3PB United Kingdom
> > > > Tel: +44 (0)1392 884897
> > > > Email: address@hidden Website: www.metoffice.gov.uk
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Unidata THREDDS Support [mailto:address@hidden]
> > > > Sent: 01 November 2017 15:53
> > > > To: Hunt, Martyn <address@hidden>
> > > > Cc: address@hidden; Hunt, Martyn <address@hidden>
> > > > Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> > > >
> > > > Odd. Did you see any log output from java.util.zip?
> > > >
> > > > > No extra output appears in the console when following those
> > > > > instructions. I've tried turning on some other debug flags and they
> > > > > do produce data, so not sure why the H5iosp one isn't producing any
> > > > > extra output on the console. Where can we go next?
> > > > >
> > > > > Thanks
> > > > >
> > > > > Martyn
> > > > >
> > > > > Martyn Hunt
> > > > > Technical Lead, Mainframe
> > > > > Met Office FitzRoy Road Exeter Devon EX1 3PB United Kingdom
> > > > > Tel: +44 (0)1392 884897
> > > > > Email: address@hidden Website: www.metoffice.gov.uk
> > > > >
> > > > > -----Original Message-----
> > > > > From: Unidata THREDDS Support [mailto:address@hidden]
> > > > > Sent: 30 October 2017 17:47
> > > > > To: Hunt, Martyn <address@hidden>
> > > > > Cc: address@hidden
> > > > > Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> > > > >
> > > > > You can run ToolsUI from the command line locally.
> > > > > You will need to download the ToolsUI jar at
> > > > > http://artifacts.unidata.ucar.edu/content/repositories/unidata-releases/edu/ucar/toolsUI/4.6.10/toolsUI-4.6.10.jar
> > > > >
> > > > > Once you have it, you should be able to run the command:
> > > > > java -jar toolsUI-4.6.10.jar
> > > > > This will start up ToolsUI.
> > > > > You then need to do the following:
> > > > >
> > > > > 1. Follow the menu Debug->Debug Flags->H5iosp and click on the
> > > > > "filter" entry.
> > > > >
> > > > > 2. You will then need to put the path to the file in the file window
> > > > > at the top and hit return.
> > > > >
> > > > > 3. The simplest way to invoke the inflation code is to right click on
> > > > > one of the compressed variables listed in the right-hand window. Then
> > > > > click the write binary data entry; it should pick a default directory
> > > > > and default output file name. Click close.
> > > > >
> > > > > The debug output should appear in the console window in which you did
> > > > > the jar command.
> > > > >
> > > > > It does not look very useful to me, but you should also be able to
> > > > > get the java.util.zip log information and that may tell you more
> > > > > since it should only contain the inflation calls from H5tiledLayoutBB.
> > > > >
> > > > >
> > > > > > Do I need to run ToolsUI on the server that is running thredds, or
> > > > > > can I run it locally on my PC? I've had a play with ToolsUI and I
> > > > > > can't see where it is writing the debug info to?
> > > > > >
> > > > > > Thanks
> > > > > >
> > > > > > Martyn
> > > > > >
> > > > > > Martyn Hunt
> > > > > > Technical Lead, Mainframe
> > > > > > Met Office FitzRoy Road Exeter Devon EX1 3PB United Kingdom
> > > > > > Tel: +44 (0)1392 884897
> > > > > > Email: address@hidden Website: www.metoffice.gov.uk
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Unidata THREDDS Support
> > > > > > [mailto:address@hidden]
> > > > > > Sent: 27 October 2017 23:00
> > > > > > To: Hunt, Martyn <address@hidden>
> > > > > > Cc: address@hidden
> > > > > > Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> > > > > >
> > > > > > Actually, ToolsUI supports enabling/disabling debug flags from
> > > > > > within the running program. No need to create your own program that
> > > > > > wraps it.
> > > > > >
> > > > > > In the ToolsUI main menu, go to "Debug"->"Debug Flags"->"H5iosp"
> > > > > > and select the "filter" menu item.
> > > > > >
> > > > > > Ticket Details
> > > > > > ===================
> > > > > > Ticket ID: ZIZ-818863
> > > > > > Department: Support THREDDS
> > > > > > Priority: Normal
> > > > > > Status: Closed
> > > > > > ===================
> > > > > > NOTE: All email exchanges with Unidata User Support are recorded in
> > > > > > the Unidata inquiry tracking system and then made publicly
> > > > > > available through the web. If you do not want to have your
> > > > > > interactions made available in this way, you must let us know in
> > > > > > each email you send to us.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > > =Dennis Heimbigner
> > > > > Unidata
> > > > >
> > > > >
> > > > > Ticket Details
> > > > > ===================
> > > > > Ticket ID: ZIZ-818863
> > > > > Department: Support THREDDS
> > > > > Priority: Normal
> > > > > Status: Open
> > > > > ===================
> > > > > NOTE: All email exchanges with Unidata User Support are recorded in
> > > > > the Unidata inquiry tracking system and then made publicly available
> > > > > through the web. If you do not want to have your interactions made
> > > > > available in this way, you must let us know in each email you send to
> > > > > us.
> > > > >
> > > > >
> > > > >
> > > >
> > > > =Dennis Heimbigner
> > > > Unidata
> > > >
> > > >
> > > > Ticket Details
> > > > ===================
> > > > Ticket ID: ZIZ-818863
> > > > Department: Support THREDDS
> > > > Priority: Normal
> > > > Status: Open
> > > > ===================
> > > > NOTE: All email exchanges with Unidata User Support are recorded in the
> > > > Unidata inquiry tracking system and then made publicly available
> > > > through the web. If you do not want to have your interactions made
> > > > available in this way, you must let us know in each email you send to
> > > > us.
> > > >
> > > >
> > > >
> > >
> > > =Dennis Heimbigner
> > > Unidata
> > >
> > >
> > > Ticket Details
> > > ===================
> > > Ticket ID: ZIZ-818863
> > > Department: Support THREDDS
> > > Priority: Normal
> > > Status: Open
> > > ===================
> > > NOTE: All email exchanges with Unidata User Support are recorded in the
> > > Unidata inquiry tracking system and then made publicly available through
> > > the web. If you do not want to have your interactions made available in
> > > this way, you must let us know in each email you send to us.
> > >
> > >
> > >
> > > You should be able to pull it from here
> > >
> > > https://drive.google.com/file/d/0B5gLZiYwJyg_YTY0QllISlZwaVU/view?usp=sharing
> > >
> > > Thanks
> > >
> > > Martyn
> > >
> > > Martyn Hunt
> > > Technical Lead, Mainframe
> > > Met Office FitzRoy Road Exeter Devon EX1 3PB United Kingdom
> > > Tel: +44 (0)1392 884897
> > > Email: address@hidden Website: www.metoffice.gov.uk
> > >
> > > -----Original Message-----
> > > From: Unidata THREDDS Support [mailto:address@hidden]
> > > Sent: 01 November 2017 18:05
> > > To: Hunt, Martyn <address@hidden>
> > > Cc: address@hidden; Hunt, Martyn <address@hidden>
> > > Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> > >
> > > Sure, go ahead and send me the file.
> > >
> > > > No, but I'd not set that up for the ToolsUI, but it definitely is
> > > > inflating the data. Could I send you an example NetCDF file, as you
> > > > are more familiar with this you may get a better result.
> > > >
> > > > Thanks
> > > >
> > > > Martyn
> > > >
> > > > Martyn Hunt
> > > > Technical Lead, Mainframe
> > > > Met Office FitzRoy Road Exeter Devon EX1 3PB United Kingdom
> > > > Tel: +44 (0)1392 884897
> > > > Email: address@hidden Website: www.metoffice.gov.uk
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Unidata THREDDS Support [mailto:address@hidden]
> > > > Sent: 01 November 2017 15:53
> > > > To: Hunt, Martyn <address@hidden>
> > > > Cc: address@hidden; Hunt, Martyn <address@hidden>
> > > > Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> > > >
> > > > Odd. Did you see any log output from java.util.zip?
> > > >
> > > > > No extra output appears in the console when following those
> > > > > instructions. I've tried turning on some other debug flags and they
> > > > > do produce data, so not sure why the H5iosp one isn't producing any
> > > > > extra output on the console. Where can we go next?
> > > > >
> > > > > Thanks
> > > > >
> > > > > Martyn
> > > > >
> > > > > Martyn Hunt
> > > > > Technical Lead, Mainframe
> > > > > Met Office FitzRoy Road Exeter Devon EX1 3PB United Kingdom
> > > > > Tel: +44 (0)1392 884897
> > > > > Email: address@hidden Website: www.metoffice.gov.uk
> > > > >
> > > > > -----Original Message-----
> > > > > From: Unidata THREDDS Support [mailto:address@hidden]
> > > > > Sent: 30 October 2017 17:47
> > > > > To: Hunt, Martyn <address@hidden>
> > > > > Cc: address@hidden
> > > > > Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> > > > >
> > > > > You can run ToolsUI from the command line locally.
> > > > > You will need to download the ToolsUI jar at
> > > > > http://artifacts.unidata.ucar.edu/content/repositories/unidata-releases/edu/ucar/toolsUI/4.6.10/toolsUI-4.6.10.jar
> > > > >
> > > > > Once you have it, you should be able to run the command:
> > > > > java -jar toolsUI-4.6.10.jar
> > > > > This will start up ToolsUI.
> > > > > You then need to do the following:
> > > > >
> > > > > 1. Follow the menu Debug->Debug Flags->H5iosp and click on the
> > > > > "filter" entry.
> > > > >
> > > > > 2. You will then need to put the path to the file in the file window
> > > > > at the top and hit return.
> > > > >
> > > > > 3. The simplest way to invoke the inflation code is to right click on
> > > > > one of the compressed variables listed in the right-hand window. Then
> > > > > click the write binary data entry; it should pick a default directory
> > > > > and default output file name. Click close.
> > > > >
> > > > > The debug output should appear in the console window in which you did
> > > > > the jar command.
> > > > >
> > > > > It does not look very useful to me, but you should also be able to
> > > > > get the java.util.zip log information and that may tell you more
> > > > > since it should only contain the inflation calls from H5tiledLayoutBB.
> > > > >
> > > > >
> > > > > > Do I need to run ToolsUI on the server that is running thredds, or
> > > > > > can I run it locally on my PC? I've had a play with ToolsUI and I
> > > > > > can't see where it is writing the debug info to?
> > > > > >
> > > > > > Thanks
> > > > > >
> > > > > > Martyn
> > > > > >
> > > > > > Martyn Hunt
> > > > > > Technical Lead, Mainframe
> > > > > > Met Office FitzRoy Road Exeter Devon EX1 3PB United Kingdom
> > > > > > Tel: +44 (0)1392 884897
> > > > > > Email: address@hidden Website: www.metoffice.gov.uk
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Unidata THREDDS Support
> > > > > > [mailto:address@hidden]
> > > > > > Sent: 27 October 2017 23:00
> > > > > > To: Hunt, Martyn <address@hidden>
> > > > > > Cc: address@hidden
> > > > > > Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> > > > > >
> > > > > > Actually, ToolsUI supports enabling/disabling debug flags from
> > > > > > within the running program. No need to create your own program that
> > > > > > wraps it.
> > > > > >
> > > > > > In the ToolsUI main menu, go to "Debug"->"Debug Flags"->"H5iosp"
> > > > > > and select the "filter" menu item.
> > > > > >
> > > > > > Ticket Details
> > > > > > ===================
> > > > > > Ticket ID: ZIZ-818863
> > > > > > Department: Support THREDDS
> > > > > > Priority: Normal
> > > > > > Status: Closed
> > > > > > ===================
> > > > > > NOTE: All email exchanges with Unidata User Support are recorded in
> > > > > > the Unidata inquiry tracking system and then made publicly
> > > > > > available through the web. If you do not want to have your
> > > > > > interactions made available in this way, you must let us know in
> > > > > > each email you send to us.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > > =Dennis Heimbigner
> > > > > Unidata
> > > > >
> > > > >
> > > > > Ticket Details
> > > > > ===================
> > > > > Ticket ID: ZIZ-818863
> > > > > Department: Support THREDDS
> > > > > Priority: Normal
> > > > > Status: Open
> > > > > ===================
> > > > > NOTE: All email exchanges with Unidata User Support are recorded in
> > > > > the Unidata inquiry tracking system and then made publicly available
> > > > > through the web. If you do not want to have your interactions made
> > > > > available in this way, you must let us know in each email you send to
> > > > > us.
> > > > >
> > > > >
> > > > >
> > > >
> > > > =Dennis Heimbigner
> > > > Unidata
> > > >
> > > >
> > > > Ticket Details
> > > > ===================
> > > > Ticket ID: ZIZ-818863
> > > > Department: Support THREDDS
> > > > Priority: Normal
> > > > Status: Open
> > > > ===================
> > > > NOTE: All email exchanges with Unidata User Support are recorded in the
> > > > Unidata inquiry tracking system and then made publicly available
> > > > through the web. If you do not want to have your interactions made
> > > > available in this way, you must let us know in each email you send to
> > > > us.
> > > >
> > > >
> > > >
> > >
> > > =Dennis Heimbigner
> > > Unidata
> > >
> > >
> > > Ticket Details
> > > ===================
> > > Ticket ID: ZIZ-818863
> > > Department: Support THREDDS
> > > Priority: Normal
> > > Status: Open
> > > ===================
> > > NOTE: All email exchanges with Unidata User Support are recorded in the
> > > Unidata inquiry tracking system and then made publicly available through
> > > the web. If you do not want to have your interactions made available in
> > > this way, you must let us know in each email you send to us.
> > >
> > >
> > >
> >
> > =Dennis Heimbigner
> > Unidata
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: ZIZ-818863
> > Department: Support THREDDS
> > Priority: Normal
> > Status: Open
> > ===================
> > NOTE: All email exchanges with Unidata User Support are recorded in the
> > Unidata inquiry tracking system and then made publicly available through
> > the web. If you do not want to have your interactions made available in
> > this way, you must let us know in each email you send to us.
> >
> >
> >
>
> =Dennis Heimbigner
> Unidata
>
=Dennis Heimbigner
Unidata
Ticket Details
===================
Ticket ID: ZIZ-818863
Department: Support THREDDS
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata
inquiry tracking system and then made publicly available through the web. If
you do not want to have your interactions made available in this way, you must
let us know in each email you send to us.
