Re: [Boost-build] bpl-test sources list being automatically sorted?

Subject: Re: [Boost-build] bpl-test sources list being automatically sorted?
From: Haoyu Bai (divinekid_at_[hidden])
Date: 2009-08-02 03:30:31

On Sat, Jul 25, 2009 at 5:50 PM, Vladimir Prus<ghost_at_[hidden]> wrote:
> On Monday 13 July 2009 Haoyu Bai wrote:
>
>> Hi,
>>
>> When working on Boost.Python I got a problem with bpl-test, the
>> Boost.Build rule for declaring Boost.Python test case target. In
>> summary, when I call "bjam polymorphism2_auto_ptr" to run the test
>> case, it can run correctly. But when I call "bjam" to run the entire
>> test suite, the test case "polymorphism2_auto_ptr" will be failed
>> because the order of the sources was changed.
>>
>> In detail, I have a test case entry:
>>
>> [ bpl-test polymorphism2_auto_ptr
>> Â  : polymorphism2_auto_ptr.py polymorphism2.py polymorphism2_auto_ptr.cpp
>> ]
>>
>> The first file, "polymorphism2_auto_ptr.py", is the file that will be
>> run by the Python interpreter, and the second one "polymorphism2.py"
>> is a dependency. So "polymorphism2_auto_ptr.py" should be keep as the
>> first one. But when calling with "bjam" to run the entire test sutie,
>> the order of these 3 files is changed. "polymorphism2.py" will become
>> the first file and cause this test case fail.
>>
>> It seems the sources list get sorted when calling "bjam", but is not
>> when calling "bjam polymorphism2_auto_ptr". So is this a feature or a
>> bug, or something wrong in my Boost.Build usage? How can I avoid the
>> unwanted sort?
>
> I am unable to reproduce this problem, and code inspection does not
> reveal any place where source would be sorted or otherwise rearranged.
> Can you give me instructions how to reproduce this? If you point me
> at some SVN branches, that's fine, but please make sure this reproduces
> in unmodified state of some revision -- and tell me that revision.
>
> - Volodya
>

Hi Volodya,

My BPL py3k SVN branch is located at:
https://svn.boost.org/svn/boost/sandbox-branches/bhy/py3k

Currently, the SVN HEAD revision which can reproduce this problem is
revision #55360.

Also, I have checked in some debugging code to output the dependencies
list in different stage:
https://svn.boost.org/trac/boost/changeset/55360

And steps to reproduce this:

$ svn co https://svn.boost.org/svn/boost/sandbox-branches/bhy/py3k
$ cd libs/python/test/
$ bjam polymorphism2_auto_ptr # (you don't need to build it with py3k,
any verison of Python is ok)

We will got some debugging output includes:
...
bpl-test=
polymorphism2_auto_ptr.py
enum.py
polymorphism2.py
polymorphism2_auto_ptr.cpp
...

beforerun=
polymorphism2_auto_ptr.py
enum.py
polymorphism2.py
polymorphism2_auto_ptr.cpp
libboost_python-gcc44-d-1_39.so.1.39.0
...
**passed** ...

Notice that the sequences in the two stages (calling bpl-test rule and
calling python-test-generator.run rule) are same. Now we call:
(I suggest using "bjam | less" to see the output)

$ bjam

The result will be:

...
bpl-test=
polymorphism2_auto_ptr.py
enum.py
polymorphism2.py
polymorphism2_auto_ptr.cpp
...

beforerun=
libboost_python-gcc44-d-1_39.so.1.39.0
polymorphism2.py
polymorphism2_auto_ptr.py
enum.py
polymorphism2_auto_ptr.cpp
...

...patience...
...patience...
warning: <p.>polymorphism2.py depends on itself
warning: <p.>enum.py depends on itself
...found 2637 targets...
...updating 264 targets...
...
====== BEGIN OUTPUT ======
Traceback (most recent call last):
  File "../../../bin.v2/libs/python/test/polymorphism2_auto_ptr.test/gcc-4.4.1/debug/polymorphism2.py",
line 90, in <module>
    from polymorphism2_ext import *
ImportError: No module named polymorphism2_ext
...
...
...failed capture-output
../../../bin.v2/libs/python/test/polymorphism2_auto_ptr.test/gcc-4.4.1/debug/polymorphism2_auto_ptr...

You see, the order in the python-test-generator.run stage (labeled as
"beforerun") is changed, though apparently it is not  sorted in
alphabet order because the dummy enum.py is still in the middle.
Because the "polymorphism2.py" is reordered as the first .py file and
being recognized as the main entry point, this test will be failed, as
you see in the output.

I also noticed a strange warning "warning: <p.>polymorphism2.py
depends on itself". So, I tried to commented out "[ bpl-test
polymorphism2 ]" line in Jamfile.v2, and then, the problem is not
occur anymore! I also tried to move the "[ bpl-test polymorphism2 ]
" line to a position after "polymorphism2_auto_ptr" test case, and no
more fails too. So at least I got a workaround now, but still got the
"warning: <p.>polymorphism2.py depends on itself".

You might want to review my 2to3 support code for bpl-test:
https://svn.boost.org/trac/boost/changeset/55359  But I think it is
irrelevant to this problem because I can also reproduce this problem
in Boost's trunk branch.

Any ideas? Thank you!

-- 
Haoyu Bai
School of Computing,
National University of Singapore.