Re: [jamboost] RFC: Proposal to port BBv2 to Python

From: Vladimir Prus (ghost_at_[hidden])
Date: 2005-01-21 06:03:55

Hi Pedro,

> > We can't just take SCons and use it. First of all, we must decide how
> > to best
> > use it and if some of SCons design must be changed before we're able
> > to use
> > it. We must also understand all the features it provides as build
> > engine to
> > be sure we're using them all. And finally, I think that SCons needs
> > some
> > performance work, too.
>
> Right. But I hear that the SCons people are taking care of that, which
> is good news.

I hear that, too. 

> > So, switching to SCons is additional work. Maybe, we should first
> > switch to
> > Python?
>
> Agreed.

Ok.

> > One issue is that currently I plan to introduce per-project
> > id-to-project
> > mapping (so "/boost" can mean different things in different projects).
> > Does
> > this leave any functionality for the 'ProjectRegistry' class?
>
> I'm not sure what you mean here.
> Will this imply extended functionality for that class?
> Will this render it unnecessary?

Looks like the last variant. Each project will have a method to give a project 
by project id. So, global project id -> project mapping seems unnecessary.

> >>
> >> Mechanisms must be put in place that allow to clean the dynamically
> >> create
> >> artifacts between builds, thus allowing the process to be repeated
> >> without
> >> having to perform "static" re-initialization.
> >
> > You mean, adding new projects between builds?
>
> I meant rebuilding without the need to add all types, features,
> targets, etc.
>
> Actually, the use-case I'm most interested in has three phases:
>
> - Global initialisation: types, features, etc. Stuff that is rarely
> changed, namely in a "controlled" production environment.
> - Project-specific initialisation: project and targets
> - Build: virtual-targets, actual-targets
>
> I want to be able to do the third-phase repeatedly as fast as possible.

I see. Seems reasonable desire.

> > I think the next milestone can be such a basic.
>
> When do you intend to release it? 

A couple of weeks, I hope.

> Do you plan many changes before that? 

No, I need to implement new <tag>/stage logic and then only bugfixes remain.

> I'm planning on spending some time working on this now (e.g., this
> weekend), so I'll just use the version I have and hope that you don't
> change it too much :-)

I hope to commit <tag>/stage today, then.

> > It might be better to stop any
> > jam-V2 development until we figure out how Pythonisation goes on.
>
> Excellent. I was considering merging diffs but that could turn out to
> be really unwieldy.

Sure.

> > declare a new feature. It means that Python modules must be something
> > exposed
> > to bjam. Then, why don't we expose the Feature.py module to bjam as
> > soon as
> > the module is written, so that we quickly detect any problems?
>
> I meant rewrite everything and then make it work. Why?
> The rational, test-oriented, left-hand part of my brain insists that we
> should keep everything working all the time.
> However, I have a gut feeling that it will take too long to integrate
> bjam and Python in the way it is needed to do so.
> I mean:
> In order to use bjam to parse Jamfiles and directing the commands to
> the Python engine, a basic one way mechanism would-suffice (I may be
> missing something here, though).
> OTOH, to perform a piecemeal transition to Python, one would need to be
> able to invoke functions from bjam to Python and vice-versa. Worse, we
> would need to be able to pass objects between them and manage their
> lifetimes.
> I have no experience whatsoever with the internals of scripting
> languages, so I may be overwhelmed with just a minor issue!
> For instance, I tried out the patch you sent (importing Python modules
> from within bjam) and, despite being so simple, it worked very well.
> Proposal: would you be interested in trying out that integration while
> I keep on working on the Python port?

I'll try. But after second though I take back part of my original argument. I 
think we don't want to allow extending PyBB with jam. Extending code can 
generally be complex, I always suggested to put it into separate modules, and 
not all users even wrote it. Allowing extending PyBB in jam code would bring 
more complexities, and I'm not convinced it's needed.

> >> - Mailing list?
> >
> > The same? ;-)
>
> I just asked because I didn't want to annoy people who are just using
> BB, let alone to scare off newcomers :-)

Ok, I think using this list is OK. We don't have a lot of traffic yet.

> > targets. Are you interested in some specific information?
>
> I should have been clearer. My understanding is that scanners simply
> define the regex they are interested in and somehow pass it on to the
> build engine which does the actual scanning. Then, the scanner sets up
> the dependencies. Bottom line, the "hard work" is done by the build
> engine.

That's right. bjam takes the regexps and scans the files. After that, it calls 
back Boost.Build with the found matches. 

> >> I tried to sort these in such a way that we can build a testable
> >> system
> >> really soon.
> >>
> >> To port now:
> >> ------------
> >> build/feature.jam
> >> build/property.jam
> >> build/property-set.jam
> >> build/type.jam
> >> build/scanner.jam
> >
> > Maybe, move virtual-target.jam here? So that we can write simple
> > script to
> > create a.o from a.cpp?
>
> Ok.

I also think we should try to port make.jam as soon as possible. It's the 
first rule which was implemented in V2 and is very simple, but yet would 
allow to test on 'project_test3' and 'project_test4' tests.

- Volodya