Hi Kevin:
Aggregations should not cause "Too many open files".
There must be a file leak bug.
heres what would help us debug it:
0. ulimit -a to see how many open files you have (should be 1 or 2k)
1. restarting tomcat will close open files.
2. to monitor what files are open, use:
/usr/proc/bin/pfiles [Tomcat Process ID]
(you may need to use sudo?)
3. then do whatever (minimal) thing to trigger the problem, then run pfiles and
see what files are open. Send me the pfiles output.
a few more notes are here:
http://www.unidata.ucar.edu/software/thredds/v4.4/tds/internal/Troubleshooting.html
John
On 10/24/2013 11:50 AM, Kevin Manross wrote:
>
> Hi there,
>
> I am trying to aggregate netcdf files for some rather large datasets
> with tomcat6.0.36/TDS4.19 - 20131008.1358 and am running into a "Too
> many open files" error.
>
> My catalog scan is:
>
> <dataset name="ASR 30 km 2d surface analysis"
> ID="ds631.0_anl_2d-AGG"
> urlPath="ds631.0/9/best"
> harvest="true">
> <serviceName>all</serviceName>
> <netcdf
> xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
> <aggregation dimName="time" type="Union">
> <scan location="/glade/p/rda/data/ds631.0/asr30km.anl.2d/"
> suffix=".nc" subdirs="false"/>
> </aggregation>
> </netcdf>
> </dataset>
>
> which contains about 4020 files and totals 470G.
>
> The message that the server sends when trying to access the OPeNDAP
> service is:
>
>
> Error {
> code = 404;
> message =
> "/glade/p/rda/data/ds631.0/asr30km.anl.2d/asr30km.anl.2d.20010619.nc (Too
> many open files)";
> };
>
>
>
> When trying the CdmRemote service catalina.out repeatedly reads:
>
> Oct 24, 2013 11:20:39 AM org.apache.jk.common.ChannelSocket
> acceptConnections
> WARNING: Exception executing accept
> java.net.SocketException: Too many open files
> at java.net.PlainSocketImpl.socketAccept(Native Method)
> at
> java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:375)
> at java.net.ServerSocket.implAccept(ServerSocket.java:478)
> at java.net.ServerSocket.accept(ServerSocket.java:446)
> at org.apache.jk.common.ChannelSocket.accept(ChannelSocket.java:311)
> at
> org.apache.jk.common.ChannelSocket.acceptConnections(ChannelSocket.java:668)
> at
> org.apache.jk.common.ChannelSocket$SocketAcceptor.runIt(ChannelSocket.java:879)
> at
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
> at java.lang.Thread.run(Thread.java:679)
>
>
> The cache settings in my threddsConfig.xml are the default settings.
>
> Have I overlooked something?
>
> -kevin.
>
>
> --
> Kevin Manross
> NCAR/CISL/Data Support Section
> Phone: (303)-497-1218
> Email:manross@xxxxxxxx <mailto:manross@xxxxxxxx>
> Web:http://rda.ucar.edu
>
>
> _______________________________________________
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
>
