[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[THREDDS #ZIZ-818863]: Thredds inflation of data



Ok, got it. I turned on the debug flag in H5tiledLayoutBB
and got the attached output. Looks like everything is
as it should be.
 inflate bytes in= 2296850 bytes out= 7077888
 inflate bytes in= 2292229 bytes out= 7077888
 inflate bytes in= 2286027 bytes out= 7077888
 inflate bytes in= 2281751 bytes out= 7077888
 inflate bytes in= 2278732 bytes out= 7077888
 inflate bytes in= 2270775 bytes out= 7077888
 inflate bytes in= 2265708 bytes out= 7077888
 inflate bytes in= 2267066 bytes out= 7077888
 inflate bytes in= 2280367 bytes out= 7077888
 inflate bytes in= 2292362 bytes out= 7077888
 inflate bytes in= 2300321 bytes out= 7077888
 inflate bytes in= 2299787 bytes out= 7077888
 inflate bytes in= 2295357 bytes out= 7077888
 inflate bytes in= 2278990 bytes out= 7077888
 inflate bytes in= 2258863 bytes out= 7077888
 inflate bytes in= 2236726 bytes out= 7077888
 inflate bytes in= 2219994 bytes out= 7077888
 inflate bytes in= 2197604 bytes out= 7077888
 inflate bytes in= 2178159 bytes out= 7077888
 inflate bytes in= 2164433 bytes out= 7077888
 inflate bytes in= 2151000 bytes out= 7077888
 inflate bytes in= 2140669 bytes out= 7077888
 inflate bytes in= 2136334 bytes out= 7077888
 inflate bytes in= 2129073 bytes out= 7077888
 inflate bytes in= 2125608 bytes out= 7077888
 inflate bytes in= 2115306 bytes out= 7077888
 inflate bytes in= 2105754 bytes out= 7077888
 inflate bytes in= 2091580 bytes out= 7077888
 inflate bytes in= 2075651 bytes out= 7077888
 inflate bytes in= 2054752 bytes out= 7077888
 inflate bytes in= 2034731 bytes out= 7077888
 inflate bytes in= 2016893 bytes out= 7077888
 inflate bytes in= 1998545 bytes out= 7077888
 inflate bytes in= 1971600 bytes out= 7077888
 inflate bytes in= 1955350 bytes out= 7077888
 inflate bytes in= 1950053 bytes out= 7077888
 inflate bytes in= 1943446 bytes out= 7077888
 inflate bytes in= 1935437 bytes out= 7077888
 inflate bytes in= 1920268 bytes out= 7077888
 inflate bytes in= 1901586 bytes out= 7077888
 inflate bytes in= 1886620 bytes out= 7077888
 inflate bytes in= 1881664 bytes out= 7077888
 inflate bytes in= 1884698 bytes out= 7077888
 inflate bytes in= 1895758 bytes out= 7077888
 inflate bytes in= 1907741 bytes out= 7077888
 inflate bytes in= 1927959 bytes out= 7077888
 inflate bytes in= 1952942 bytes out= 7077888
 inflate bytes in= 1984151 bytes out= 7077888
 inflate bytes in= 2003863 bytes out= 7077888
 inflate bytes in= 2001965 bytes out= 7077888
 inflate bytes in= 1976532 bytes out= 7077888
 inflate bytes in= 1944348 bytes out= 7077888
 inflate bytes in= 1915409 bytes out= 7077888
 inflate bytes in= 1887227 bytes out= 7077888
 inflate bytes in= 1868649 bytes out= 7077888
 inflate bytes in= 1865444 bytes out= 7077888
 inflate bytes in= 1870893 bytes out= 7077888
 inflate bytes in= 1883281 bytes out= 7077888
 inflate bytes in= 1899309 bytes out= 7077888
 inflate bytes in= 30905 bytes out= 7077888
 inflate bytes in= 30905 bytes out= 7077888
 inflate bytes in= 30905 bytes out= 7077888
 inflate bytes in= 30905 bytes out= 7077888
 inflate bytes in= 30905 bytes out= 7077888
 inflate bytes in= 30905 bytes out= 7077888
 inflate bytes in= 30905 bytes out= 7077888
 inflate bytes in= 30905 bytes out= 7077888
 inflate bytes in= 30905 bytes out= 7077888
 inflate bytes in= 30905 bytes out= 7077888
 inflate bytes in= 30905 bytes out= 7077888


> You should be able to pull it from here
> 
> https://drive.google.com/file/d/0B5gLZiYwJyg_YTY0QllISlZwaVU/view?usp=sharing
> 
> Thanks
> 
> Martyn
> 
> Martyn HuntÂÂ
> Technical Lead, Mainframe
> Met OfficeÂÂFitzRoy RoadÂÂExeterÂÂDevonÂÂEX1 3PBÂÂUnited Kingdom
> Tel: +44 (0)1392 884897ÂÂ
> Email: address@hidden ÂWebsite: www.metoffice.gov.uk
> 
> -----Original Message-----
> From: Unidata THREDDS Support [mailto:address@hidden
> Sent: 01 November 2017 18:05
> To: Hunt, Martyn <address@hidden>
> Cc: address@hidden; Hunt, Martyn <address@hidden>
> Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> 
> Sure, go ahead and send me the file.
> 
> > No, but I'd not set that up for the ToolsUI, but it definitely is inflating 
> > the data.  Could I send you an example NetCDF file, as you are more 
> > familiar with this you may get a better result.
> >
> > Thanks
> >
> > Martyn
> >
> > Martyn HuntÂÂ
> > Technical Lead, Mainframe
> > Met OfficeÂÂFitzRoy RoadÂÂExeterÂÂDevonÂÂEX1 3PBÂÂUnited Kingdom
> > Tel: +44 (0)1392 884897ÂÂ
> > Email: address@hidden ÂWebsite: www.metoffice.gov.uk
> >
> >
> > -----Original Message-----
> > From: Unidata THREDDS Support [mailto:address@hidden
> > Sent: 01 November 2017 15:53
> > To: Hunt, Martyn <address@hidden>
> > Cc: address@hidden; Hunt, Martyn <address@hidden>
> > Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> >
> > Odd. Did you see any log output from java.util.zip?
> >
> > > No extra output appears in the console when following those instructions. 
> > >  I've tried turning on some other debug flags and they do produce data, 
> > > so not sure why the H5iosp one isn't producing any extra output on the 
> > > console.  Where can we go next?
> > >
> > > Thanks
> > >
> > > Martyn
> > >
> > > Martyn HuntÂÂ
> > > Technical Lead, Mainframe
> > > Met OfficeÂÂFitzRoy RoadÂÂExeterÂÂDevonÂÂEX1 3PBÂÂUnited Kingdom
> > > Tel: +44 (0)1392 884897ÂÂ
> > > Email: address@hidden ÂWebsite: www.metoffice.gov.uk
> > >
> > > -----Original Message-----
> > > From: Unidata THREDDS Support [mailto:address@hidden
> > > Sent: 30 October 2017 17:47
> > > To: Hunt, Martyn <address@hidden>
> > > Cc: address@hidden
> > > Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> > >
> > > You can run ToolsUI from the command line locally.
> > > You will need to download the ToolsUI jar at 
> > > http://artifacts.unidata.ucar.edu/content/repositories/unidata-releases/edu/ucar/toolsUI/4.6.10/toolsUI-4.6.10.jar
> > >
> > > Once you have it, you should be able to run the command:
> > > java -jar toolsUI-4.6.10.jar
> > > This will start up ToolsUI.
> > > You then need to do the following:
> > >
> > > 1. Follow the menu Debug->Debug Flags->H5iosp and click on the "filter" 
> > > entry.
> > >
> > > 2. You will then need to put the path to the file in the file window at 
> > > the top and hit return.
> > >
> > > 3. The simplest way to invoke the inflation code is to right click on one 
> > > of the compressed variables listed in the right-hand window. Then click 
> > > the write binary data entry; it should pick a default directory and 
> > > default output file name. Click close.
> > >
> > > The debug output should appear in the console window in which you did the 
> > > jar command.
> > >
> > > It does not look very useful to me, but you should also be able to get 
> > > the java.util.zip log information and that may tell you more since it 
> > > should only contain the inflation calls from H5tiledLayoutBB.
> > >
> > >
> > > > Do I need to run ToolsUI on the server that is running thredds, or can 
> > > > I run it locally on my PC?  I've had a play with ToolsUI and I can't 
> > > > see where it is writing the debug info to?
> > > >
> > > > Thanks
> > > >
> > > > Martyn
> > > >
> > > > Martyn Hunt
> > > > Technical Lead, Mainframe
> > > > Met OfficeÂÂFitzRoy RoadÂÂExeterÂÂDevonÂÂEX1 3PBÂÂUnited 
> > > > Kingdom
> > > > Tel: +44 (0)1392 884897
> > > > Email: address@hidden ÂWebsite: www.metoffice.gov.uk
> > > >
> > > > -----Original Message-----
> > > > From: Unidata THREDDS Support
> > > > [mailto:address@hidden
> > > > Sent: 27 October 2017 23:00
> > > > To: Hunt, Martyn <address@hidden>
> > > > Cc: address@hidden
> > > > Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> > > >
> > > > Actually, ToolsUI supports enabling/disabling debug flags from within 
> > > > the running program. No need to create your own program that wraps it.
> > > >
> > > > In the ToolsUI main menu, go to "Debug"->"Debug Flags"->"H5iosp" and 
> > > > select the "filter" menu item.
> > > >
> > > > Ticket Details
> > > > ===================
> > > > Ticket ID: ZIZ-818863
> > > > Department: Support THREDDS
> > > > Priority: Normal
> > > > Status: Closed
> > > > ===================
> > > > NOTE: All email exchanges with Unidata User Support are recorded in the 
> > > > Unidata inquiry tracking system and then made publicly available 
> > > > through the web.  If you do not want to have your interactions made 
> > > > available in this way, you must let us know in each email you send to 
> > > > us.
> > > >
> > > >
> > > >
> > > >
> > >
> > > =Dennis Heimbigner
> > > Unidata
> > >
> > >
> > > Ticket Details
> > > ===================
> > > Ticket ID: ZIZ-818863
> > > Department: Support THREDDS
> > > Priority: Normal
> > > Status: Open
> > > ===================
> > > NOTE: All email exchanges with Unidata User Support are recorded in the 
> > > Unidata inquiry tracking system and then made publicly available through 
> > > the web.  If you do not want to have your interactions made available in 
> > > this way, you must let us know in each email you send to us.
> > >
> > >
> > >
> >
> > =Dennis Heimbigner
> > Unidata
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: ZIZ-818863
> > Department: Support THREDDS
> > Priority: Normal
> > Status: Open
> > ===================
> > NOTE: All email exchanges with Unidata User Support are recorded in the 
> > Unidata inquiry tracking system and then made publicly available through 
> > the web.  If you do not want to have your interactions made available in 
> > this way, you must let us know in each email you send to us.
> >
> >
> >
> 
> =Dennis Heimbigner
> Unidata
> 
> 
> Ticket Details
> ===================
> Ticket ID: ZIZ-818863
> Department: Support THREDDS
> Priority: Normal
> Status: Open
> ===================
> NOTE: All email exchanges with Unidata User Support are recorded in the 
> Unidata inquiry tracking system and then made publicly available through the 
> web.  If you do not want to have your interactions made available in this 
> way, you must let us know in each email you send to us.
> 
> 
> 
> You should be able to pull it from here
> 
> https://drive.google.com/file/d/0B5gLZiYwJyg_YTY0QllISlZwaVU/view?usp=sharing
> 
> Thanks
> 
> Martyn
> 
> Martyn HuntÂÂ
> Technical Lead, Mainframe
> Met OfficeÂÂFitzRoy RoadÂÂExeterÂÂDevonÂÂEX1 3PBÂÂUnited Kingdom
> Tel: +44 (0)1392 884897ÂÂ
> Email: address@hidden ÂWebsite: www.metoffice.gov.uk
> 
> -----Original Message-----
> From: Unidata THREDDS Support [mailto:address@hidden
> Sent: 01 November 2017 18:05
> To: Hunt, Martyn <address@hidden>
> Cc: address@hidden; Hunt, Martyn <address@hidden>
> Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> 
> Sure, go ahead and send me the file.
> 
> > No, but I'd not set that up for the ToolsUI, but it definitely is inflating 
> > the data.  Could I send you an example NetCDF file, as you are more 
> > familiar with this you may get a better result.
> >
> > Thanks
> >
> > Martyn
> >
> > Martyn HuntÂÂ
> > Technical Lead, Mainframe
> > Met OfficeÂÂFitzRoy RoadÂÂExeterÂÂDevonÂÂEX1 3PBÂÂUnited Kingdom
> > Tel: +44 (0)1392 884897ÂÂ
> > Email: address@hidden ÂWebsite: www.metoffice.gov.uk
> >
> >
> > -----Original Message-----
> > From: Unidata THREDDS Support [mailto:address@hidden
> > Sent: 01 November 2017 15:53
> > To: Hunt, Martyn <address@hidden>
> > Cc: address@hidden; Hunt, Martyn <address@hidden>
> > Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> >
> > Odd. Did you see any log output from java.util.zip?
> >
> > > No extra output appears in the console when following those instructions. 
> > >  I've tried turning on some other debug flags and they do produce data, 
> > > so not sure why the H5iosp one isn't producing any extra output on the 
> > > console.  Where can we go next?
> > >
> > > Thanks
> > >
> > > Martyn
> > >
> > > Martyn HuntÂÂ
> > > Technical Lead, Mainframe
> > > Met OfficeÂÂFitzRoy RoadÂÂExeterÂÂDevonÂÂEX1 3PBÂÂUnited Kingdom
> > > Tel: +44 (0)1392 884897ÂÂ
> > > Email: address@hidden ÂWebsite: www.metoffice.gov.uk
> > >
> > > -----Original Message-----
> > > From: Unidata THREDDS Support [mailto:address@hidden
> > > Sent: 30 October 2017 17:47
> > > To: Hunt, Martyn <address@hidden>
> > > Cc: address@hidden
> > > Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> > >
> > > You can run ToolsUI from the command line locally.
> > > You will need to download the ToolsUI jar at 
> > > http://artifacts.unidata.ucar.edu/content/repositories/unidata-releases/edu/ucar/toolsUI/4.6.10/toolsUI-4.6.10.jar
> > >
> > > Once you have it, you should be able to run the command:
> > > java -jar toolsUI-4.6.10.jar
> > > This will start up ToolsUI.
> > > You then need to do the following:
> > >
> > > 1. Follow the menu Debug->Debug Flags->H5iosp and click on the "filter" 
> > > entry.
> > >
> > > 2. You will then need to put the path to the file in the file window at 
> > > the top and hit return.
> > >
> > > 3. The simplest way to invoke the inflation code is to right click on one 
> > > of the compressed variables listed in the right-hand window. Then click 
> > > the write binary data entry; it should pick a default directory and 
> > > default output file name. Click close.
> > >
> > > The debug output should appear in the console window in which you did the 
> > > jar command.
> > >
> > > It does not look very useful to me, but you should also be able to get 
> > > the java.util.zip log information and that may tell you more since it 
> > > should only contain the inflation calls from H5tiledLayoutBB.
> > >
> > >
> > > > Do I need to run ToolsUI on the server that is running thredds, or can 
> > > > I run it locally on my PC?  I've had a play with ToolsUI and I can't 
> > > > see where it is writing the debug info to?
> > > >
> > > > Thanks
> > > >
> > > > Martyn
> > > >
> > > > Martyn Hunt
> > > > Technical Lead, Mainframe
> > > > Met OfficeÂÂFitzRoy RoadÂÂExeterÂÂDevonÂÂEX1 3PBÂÂUnited 
> > > > Kingdom
> > > > Tel: +44 (0)1392 884897
> > > > Email: address@hidden ÂWebsite: www.metoffice.gov.uk
> > > >
> > > > -----Original Message-----
> > > > From: Unidata THREDDS Support
> > > > [mailto:address@hidden
> > > > Sent: 27 October 2017 23:00
> > > > To: Hunt, Martyn <address@hidden>
> > > > Cc: address@hidden
> > > > Subject: [THREDDS #ZIZ-818863]: Thredds inflation of data
> > > >
> > > > Actually, ToolsUI supports enabling/disabling debug flags from within 
> > > > the running program. No need to create your own program that wraps it.
> > > >
> > > > In the ToolsUI main menu, go to "Debug"->"Debug Flags"->"H5iosp" and 
> > > > select the "filter" menu item.
> > > >
> > > > Ticket Details
> > > > ===================
> > > > Ticket ID: ZIZ-818863
> > > > Department: Support THREDDS
> > > > Priority: Normal
> > > > Status: Closed
> > > > ===================
> > > > NOTE: All email exchanges with Unidata User Support are recorded in the 
> > > > Unidata inquiry tracking system and then made publicly available 
> > > > through the web.  If you do not want to have your interactions made 
> > > > available in this way, you must let us know in each email you send to 
> > > > us.
> > > >
> > > >
> > > >
> > > >
> > >
> > > =Dennis Heimbigner
> > > Unidata
> > >
> > >
> > > Ticket Details
> > > ===================
> > > Ticket ID: ZIZ-818863
> > > Department: Support THREDDS
> > > Priority: Normal
> > > Status: Open
> > > ===================
> > > NOTE: All email exchanges with Unidata User Support are recorded in the 
> > > Unidata inquiry tracking system and then made publicly available through 
> > > the web.  If you do not want to have your interactions made available in 
> > > this way, you must let us know in each email you send to us.
> > >
> > >
> > >
> >
> > =Dennis Heimbigner
> > Unidata
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: ZIZ-818863
> > Department: Support THREDDS
> > Priority: Normal
> > Status: Open
> > ===================
> > NOTE: All email exchanges with Unidata User Support are recorded in the 
> > Unidata inquiry tracking system and then made publicly available through 
> > the web.  If you do not want to have your interactions made available in 
> > this way, you must let us know in each email you send to us.
> >
> >
> >
> 
> =Dennis Heimbigner
> Unidata
> 
> 
> Ticket Details
> ===================
> Ticket ID: ZIZ-818863
> Department: Support THREDDS
> Priority: Normal
> Status: Open
> ===================
> NOTE: All email exchanges with Unidata User Support are recorded in the 
> Unidata inquiry tracking system and then made publicly available through the 
> web.  If you do not want to have your interactions made available in this 
> way, you must let us know in each email you send to us.
> 
> 
> 

=Dennis Heimbigner
  Unidata


Ticket Details
===================
Ticket ID: ZIZ-818863
Department: Support THREDDS
Priority: Normal
Status: Open
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.



NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.