Re: [Boost-users] asio's io_service uses too many file descriptors

From: pete_at_[hidden]
Date: 2007-08-15 13:50:55

Hey Darren, 

On Wed, Aug 15, 2007 at 06:09:43PM +0100, Darren Garvey wrote:
> On 15/08/07, pete_at_[hidden] <pete_at_[hidden]> wrote:
> IIUC, you can mix and match too (see the HTTP3 example, which uses an
> io_service pool). In your case, having a pool of io_services, each being run
> in a pool of threads might help you.

I used some of that for my original model when moving to asio from native
sockets.

However, I think I just found my problem.  It appears I'm an idiot!
Doh!  I was having problems when porting it originally, so I started
using log4cxx from the apache folks for doing some thread safe logging
to debug the issue.  I think it's the log4cxx that is actually causing
the problem now.  It's using up my FD's it appears, as I don't have any
issues when I remove the logging.  The exceptions I've been looking for
are all from boost::asio and not from log4cxx, so I think I was blinded
by that.

When I removed all the threaded logging, the problem has vanished---I
think.

Sorry for being a weenie. 

Thanks for your response!  It helped me work out the issue by trying to
answer your questions.

Have a good one,

Pete

 
> I have tried both. Both have the same issue.  If I use standard low level
> > sockets, my
> > app runs very fast and concurrency pays off, but with asio it's a
> > dog that runs our of resources.
> 
> 
> Are you using Linux 2.4? According to the docs, you are limited to
> FD_SETSIZE number of sockets. Can you check what that is defined as on your
> system? You shouldn't run into this problem a newer Linux kernel, as that
> uses epoll, which apparently doesn't have the same limitations.
> 
> <snip>
> >
> All I want to do is launch hundreds of threads to make these connections
> > and asio doesn't seem to permit that sort of abuse.
> 
> 
> Asio should allow that sort of abuse. Can you provide more details about
> your system?
> 
> As an aside, if you fork() before running your 'main loop', you can get
> around the FD_SETSIZE limit (which is a per-process one).
> 
> Regards,
> Darren

> _______________________________________________
> Boost-users mailing list
> Boost-users_at_[hidden]
> http://lists.boost.org/mailman/listinfo.cgi/boost-users