Recently I was working on a Java project. The program was intended to run for a long time (weeks at a time) and process incoming requests. I started the overnight test to see how it would do receiving requests over a 12 hour period.

After five hours the program crashed with this message:

java.io.IOException: Too many open files

So I had to dig in and figure out what was causing this.

First of all, the number of open files allowed is an Operating System dependent setting.  This is a limited resource that is shared by all processes on the machine.  The standard programming practice is to make sure to close all files that the program opens. On Unix systems this applies to socket connections as well.

I inspected my code and looked for all open statements.  I made sure that each open statement had the accompanying close.  After looking over my code I was certain that everything was closed.  But I was still seeing the accumulation of open files as messages were handled.

Usually in these type of situations I turn to Google to see if someone else has dealt with this problem too.  After a bit of searching I found the clue.

My program uses the Runtime.getProcess() method to get a process and then exec() an external script.  What I didn't know was that the API automatically opens three streams (stdout, stderr, stdin) each time the getProcess() is called.  It is the responsibility of the caller to close those streams when done.

I added the code to close those and everything was fine.

Of course, after doing this I read the documentation for the Runtime class.  Sure enough, it mentions the three streams...

It's amazing what a little bit of knowledge can do.

Comments

Add New Search RSS

Priya - Thanks a lot!!!! IP:129.237.121.8 | 2007-02-09 18:56:34 I have been struggling with this problem since two days. I agree with you "It's amazing what a little bit of knowledge can do."

mike Super Administrator | 2007-02-10 11:34:31 Glad the info helped.

dave IP:85.115.14.34 | 2007-02-20 23:42:42 Thanks a lot! Regards, Dave ------------------- http://dlinsin.blogspot.com/2007/02/javas-runtimeexec-openes-too-many-files.html

Tobin IP:129.33.1.37 | 2007-02-28 06:56:28 Mike, Can you provide a URL to the Javadoc explaining the need to close the 3 streams opened by exec( )? I'm not seeing it on Process or Runtime (just recommendations that the streams be buffered). Thanks, Tobin

mike Super Administrator | 2007-02-28 07:12:35 Hmm. I looked through the docs and I see that they don't explicitly state that the streams needs to be closed. They do need to be closed, though. Nothing else will close the streams. I looked at the latest Java docs (version 6) and they still don't mention the need to close the streams. Interesting.

Mike - THANK YOU IP:88.198.107.250 | 2007-03-09 05:10:29 I've been struggling with this for weeks.

mike Super Administrator | 2007-03-09 06:09:18 I'm glad to be able to help.

Btorb - closing IP:203.181.243.11 | 2007-04-09 04:37:58 Thanks, i'm running new experiments now and hopefully they work. The api doesn't tell you to close these streams, only that "The subprocess is not killed when there are no more references to the Process object, but rather the subprocess continues executing asynchronously.". So, well done, beautiful knowlegde...

raazju - How to be expert in Java Progr Registered | 2007-05-12 02:35:09 Hi! This is Rajendraa Dhakal from Nepal could you could you guide me???? sincerely Rajendra

mike Super Administrator | 2007-05-12 08:27:06 What is your question?

Only registered users can write comments!