Re: [Boost-build] [python-port] msvc toolset patch

Subject: Re: [Boost-build] [python-port] msvc toolset patch
From: Vladimir Prus (ghost_at_[hidden])
Date: 2011-11-20 04:06:06

On 18/11/11 22:33, Juraj Ivančić wrote:
> Hi all
>
> As I have several years experience with both BB and Python, and a very short fuse when waiting for
> bjam to start building a project, I recently started porting msvc toolset to python and I have
> reached the point where I would require some small help. The attached patch contains what I did so far.

Hi Juraj,

this is very exciting development!

> The msvc.py implementation corresponds almost 1-1 to the msvc.jam. IOW I tried to do it carefully
> and correctly, without using any fancy Python features. Currently it is lacking midl/mc support as
> this requires porting appropriate jamfiles. I plan on doing this soon.
>
> The first problem I ran into was this:
>
> rule compile.c.pch ( targets + : sources * : properties * )
> {
> C++FLAGS on $(targets[1]) = ;
> get-rspline $(targets[1]) : -TC ;
> get-rspline $(targets[2]) : -TC ;
> local pch-source = [ on $(<) return $(PCH_SOURCE) ] ;
> if $(pch-source)
> {
> DEPENDS $(<) : $(pch-source) ;
> compile-c-c++-pch-s $(targets) : $(sources) $(pch-source) ;
> }
> else
> {
> compile-c-c++-pch $(targets) : $(sources) ;
> }
> }
>
> This rule is a bit problematic because it decides the action which is to be taken depending on some
> of the properties. This is problematic because, AFAICU, python port uses opposite logic. Generators
> are given an action identifier. Only once the action is decided can the rule (or in this case
> function) be invoked. Is this correct or did I miss something? I was unsure how to tackle this
> problem, so I decided to leave it until consulting with someone more knowledgeable.

Yeah. In bjam, 'compile.c.pch' can be just a function. In Python port, at present, one has to
register 'compile.c.pch' as an action, using Engine.register_action or Engine.register_bjam_action.
Then, Action.actualize uses Engine.set_update_action passing action id.

The most expedient solution, I think, is to allow associating 'compile.c.pch' with just python
function. I think something like attached will be sufficient. Note that 'register_action' already
accepts a 'function' parameter, so we can port the above jam function into python. It would have
to use Engine.set_update_action with either 'compile-c-c++-pch-s' or 'compile-c-c++-pch' as real
action name. Note that I haven't though everything through - there might be more issues downstream.

> But this was nothing. Who would except that PCH work on the first attempt, right? But then came
> another issue is the one which was quite painful. Even after porting the entire msvc module, I
> believe correctly, to Python it still failed to compile a trivial project correctly:
>
> exe test : test.cpp ;
>
> What happens is that BB decides it needs to build test.dll in addition to test.exe. Not exactly sure
> why. I did notice that when I change msvc.link generator so that it no longer accepts IMPORT_LIB as
> input, then the DLL is not built, as expected. This led me to the conclusion that the problem is
> outside the scope of msvc module, and that there is some bug in constructing dependency tree. Thoughts?

Oh, this apparently means that we find the following transformation:

        test.cpp -> test.obj -> test.lib + test.dll -> test.exe

This is strange. In jam version, the mechanism to prevent this from happening is check for non-empty
$(name) in builtin.jam:lib-generator.run. The name is the desired final target name, and is only
set on the top-level of transformation, so the generator for dll/lib part should get empty name
and refuse to work.

The same check is present in builtin.py -- so maybe something else is broken, and it's given a
non-empty name. I think it's best to just set breakpoint in LibGenerator.run and see what's 'name',
and then work from there. I've tried to check, but things won't work for me on Linux due to import
of _winreg.

For avoidance of doubt, standard:

        import pdb
        pdb.set_trace()

works just fine inside Boost.Build code.

I've looked over your patch, and everything seems reasonable (but it's obviously hard to review 
msvc.py). I think you need to add your copyright to msvc.py, though.

Thanks,
Volodya