[Boost-users] How to make boost asio fork safe

Subject: [Boost-users] How to make boost asio fork safe
From: Ranadheer (p_ranadheer_at_[hidden])
Date: 2012-02-20 08:40:35

I have built a C++ library using boost ASIO. The library needs to be both
thread-safe and fork-safe.
It has service scheduler thread, which calls "_io_service.run())". To
support fork-safety, I've registered pre_fork, post_fork_parent and
post_fork_child handlers. The pre_fork() handler, calls
"_io_service.notify_fork(boost::io_service:fork_prepare();",
post_fork_parent handler calls
"_io_service.notify_fork(boost::asio::io_service::fork_parent);" and
post_fork_child calls
"_io_service.notify_fork(boost::asio::io_service::fork_child);".

The problem I'm facing in, when the fork() happens, the service scheduler
thread might be in middle of some operation and might have acquired lock on
data members of io_service object. So, the child process seems them in the
same state and in the post_fork_child() when we call
"_io_service.notify_fork(boost::asio::io_service::fork_child);" it tries to
acquire the lock on the same object and hence get blocked indefinitely (as
there is no thread in child to release the lock).

The stack trace I see in the child process, which is blocked, is -

 fffffd7ffed07577 lwp_park (0, 0, 0)
 fffffd7ffecffc18 mutex_lock_internal () + 378
 fffffd7ffecfffb2 mutex_lock_impl () + 112
 fffffd7ffed0007b mutex_lock () + b
 fffffd7fff26419d
__1cFboostEasioGdetailLscoped_lock4n0CLposix_mutex__2t5B6Mrn0D__v_ () + 1d
 fffffd7fff2866a2
__1cFboostEasioGdetailQdev_poll_reactorMfork_service6Mn0BKio_serviceKfork_event__v_
() + 32
 fffffd7fff278527
__1cFboostEasioGdetailQservice_registryLnotify_fork6Mn0BKio_serviceKfork_event__v_
() + 107
 fffffd7fff27531c __1cDdesGtunnelQServiceSchedulerPpost_fork_child6M_v_ () +
1c
 fffffd7fff29de24 post_fork_child () + 84
 fffffd7ffec92188 _postfork_child_handler () + 38
 fffffd7ffecf917d fork () + 12d
 fffffd7ffec172d5 fork () + 45
 fffffd7ffef94309 fork () + 9
 000000000043299d main () + 67d
 0000000000424b2c ???????? ()

Apparently the "dev_poll_reactor" is locked (because it seems to be
dispatching some pending events) in the service scheduler thread when the
fork has happened which is causing the problem.

I think to solve the problem, I need to ensure that service scheduler thread
is not in the middle of any processing when the fork happens and one way to
guarantee that would be to call "io_service.stop()" in pre_fork() handler
but that doesn't sound like a good solution. Could you please let me know
what is the right approach to make the library fork safe?

The code snippets looks something like this.

/**
 * Combines Boost.ASIO with a thread for scheduling.
 */
class ServiceScheduler : private boost::noncopyable
{
public :
    /// The actual thread used to perform work.
    boost::shared_ptr<boost::thread>             _service_thread;

    /// Service used to manage async I/O events
    boost::asio::io_service                      _io_service;

    /// Work object to block the ioservice thread.
    std::auto_ptr<boost::asio::io_service::work> _work;
...
};

/**
 * CTOR
 */
ServiceScheduler::ServiceScheduler()
    : _logger("ServiceScheduler"),
      _io_service(),
      _work(std::auto_ptr<boost::asio::io_service::work>(
              new boost::asio::io_service::work(_io_service))),
      _is_running(false)
{
}

/**
 * Starts a thread to run async I/O service to process the scheduled work.
 */
void ServiceScheduler::start()
{
    ScopedLock scheduler_lock(_mutex);
    if (!_is_running) {
        _is_running = true;
        _service_thread = boost::shared_ptr<boost::thread>(
                new boost::thread(boost::bind(
                        &ServiceScheduler::processServiceWork, this)));
    }
}

/**
 *  Processes work passed to the ASIO service and handles uncaught
 *  exceptions
 */
void ServiceScheduler::processServiceWork()
{
    try {
        std::cout<<"Running io_service"<<std::endl;
        _io_service.run();
    }
    catch (std::exception& e) {
        TUNNEL_LOGGER_ERROR(_logger, "Exception while running ioservice"
                << e.what());
    }
    catch (...) {
        TUNNEL_LOGGER_ERROR(_logger, "Uncaught exception while running "
                << "ioservice");
    }
}

/**
 * Pre-fork handler
 */
void ServiceScheduler::pre_fork()
{
    _io_service.notify_fork(boost::asio::io_service::fork_prepare);
}

/**
 * Post-fork parent handler
 */
void ServiceScheduler::post_fork_parent()
{
    _io_service.notify_fork(boost::asio::io_service::fork_parent);
}

/**
 * Post-fork child handler
 */
void ServiceScheduler::post_fork_child()
{
    _io_service.notify_fork(boost::asio::io_service::fork_child);
}

I'm using boost 1.47 and running the application on Solaris i386. The
library and application are built using studio-12.0.

Thanks
Ranadheer

--
View this message in context: http://boost.2283326.n4.nabble.com/How-to-make-boost-asio-fork-safe-tp4404069p4404069.html
Sent from the Boost - Users mailing list archive at Nabble.com.