Re: [boost] [Test][Thread] Regression since 9 December?

Subject: Re: [boost] [Test][Thread] Regression since 9 December?
From: Raffi Enficiaud (raffi.enficiaud_at_[hidden])
Date: 2015-01-26 04:27:16

Andrey Semashev <andrey.semashev <at> gmail.com> writes:

> 
> On Sunday 25 January 2015 23:04:44 Gennadiy Rozental wrote:
> > Andrey Semashev <andrey.semashev <at> gmail.com> writes:
>
> [snip]
>
> 
> 
> > > Anyway, I decided to remove Boost.Test from Boost.Atomic tests.
> > 
> > You are free to do so, but just to make record strait: the issues in that
> > case were caused by regression in Boost.Timer and minor misconfiguration in
> > Boost.Thread.
> 
> At least Boost.Atomic tests were also affected, and there was a pull request 
> fixing that. I can see a list of multiple affected libraries in this 
> discussion.

If I understand correctly, the pull request that was merged
(<use>/boost/test//boost_unit_test_framework) states only that the headers
of boost.test are there when the tests of atomic are built. So this is just
fixing the way now the modular boost works (the header having moved to
libs/xxx/include/boost). So the impact on the link part should be null.

I have a patch that fixes the issues you have with boost.test (it is just a
matter of two lines to change). However, it would be good to understand why
it worked and why this is not the case anymore.

I had a closer look to the tests of boost.atomic:

- the tests that are failing are using the test_execution_monitor in a
include only (no linking) way. However, I went back through the
documentation until boost 1.31 and I cannot see anywhere in the
documentation such an interface to boost.test. These tests were added in
2012 if I am correct, and already using a non supported interface to boost.test.
- it is hard to tell if those tests were not failing before the changes that
were made and reverted. I compiled the boost.atomic tests by reverting
boost.test back to c3ec41073f8385879721882373d39794c50791dd of *8 Nov 2012*,
and the problem still occurs (version
427a757bbfba4118acb7e4b34dfb8f91dc121c28 of boost.atomic). So my guess is
that something from the *environment* is causing these troubles, but the
environment is a complex entity.

So, if you are interested in this patch, I can send it to you. But several
issues are raised concerning the modular boost workflow:

- it is hard to tell, when we start a feature branch, what was caused by the
modifications made in this feature branch. We can only see the status of the
develop branch and the master branch. This is already good, but I think we
have to think of a better way of testing modular boost. A possible solution
is to test (on identified subset of workers) a feature branch against the
state of boost right before the branch was created. This is to be jointly
used with a "delta" of errors shown in the test report, because aiming at
having the full boost green before branching is maybe something really hard
to have, especially from the develop branch. But at least with this we can
see what we are breaking before merging back to develop.
- it would be nice I think to have for the test report the history, for at
least something like 2 or 3 months. Then we would be able to see when a test
started failing, which would ease a lot the identification of the source of
the errors. 
- a nice thing would also to have the commit message from "Automated Commit
<automated_at_[hidden]>" having the recap between the delta of revisions
that are commited to the main boost.

All these I think would ease the global workflow we have and addressing some
issues we are facing:
- feature branches are showing problems only when merged back to develop
- identifications of the sources of the problems is kind of hard

Best,
Raffi Enficiaud