Re: [boost] Using Boost.Coroutine to mock threads

From: Edd Dawson (lists_at_[hidden])
Date: 2008-01-07 14:12:36

Hi Johan,

Johan Nilsson wrote:
> [Comments below]
> 
> Edd Dawson wrote:
>> So! I had this slightly wacky thought. Would it be possible to
>> implement the boost.Thread API using Boost.Coroutine? Coroutine
>> yield()s would be performed inside blocking calls to
>> whatever_lock_type::lock(), thread::join(), condition::wait() and so
>> on, perhaps; I haven't completely thought this through, yet... :)
> 
> You are entering an extremely interesting area for those of us that are
> truly test-infected. I've got no comments on the technical feasibility, but
> for certain testing scenarios the possibility of controlling
> "thread"-scheduling would definitely be valuable.

After experimenting for a few hours yesterday, I'm near certain that this is
do-able. I've got enough done this weekend to have rough versions of
boost::thread, a couple of the mutexes and boost::barrier to play with.

It's encouraging to hear that somebody thinks that there's some value in this. I
have to admit that I'm really rather new to methodologies that place a high
degree of importance on testing (though I fully agree with the reasoning and
have resolved to learn more this year).

> As a pretty experienced TFD:er with threaded code, I should give you a
> warning though: the race conditions and deadlock sequences that you are
> foreseeing, and therefore writing tests for, are generally the ones that you
> will handle correctly. You should of course be writing tests for those as
> well as anything else, but there is no real replacement for running
> "higher-level" unit tests as well with a high amount of contention involved.

Good point, thank you.

> Also, a general recommendataion is to split the code into smaller units
> whose logic can be separately tested without the involvement of threads
> before aggregating them into a larger component (as e.g. the task_queue
> above). What about adding a non-public task_queue_container class that
> contains enough observer methods to be able to verify the desired behaviour,
> both in single-threaded and multi-threaded contexts, and letting the public
> task_queue delegate most of its job to that implementation?

I'm sure that's the correct way to do things but at this early stage in my
education I find it hard to see this helping things just yet. I suspect it's
probably because I subconsciously want an neat little solution that covers all
cases when there probably isn't one. For example, I still couldn't test the FIFO
behaviour I require (as far as I can see) without having some kind of
cooperative threads implementation.

This is my problem, of course and I hope I'll "fix" it with more reading, but I
felt I should explain why I didn't find your suggested rearrangement in the
first place :)

> All that being said though, I still think your idea is worthwhile pursuing.

That's good.

> If you don't get any positive feedback from the Boost community, feel free
> to e-mail me directly if you ever get any further in this area.

Thank you. If I get anywhere useful with this I'll post the code on the web. I
know I'll find writing tests for the coroutine-threads a little tricky, so I
might come after you for advice on that :)

Thanks,

Edd