Re: [jamboost] gcc libraries order

From: David Abrahams (dave_at_[hidden])
Date: 2003-03-25 13:38:09

Ali Azarbayejani <ali_at_[hidden]> writes:

> David Abrahams wrote:
>> 
>> Ali Azarbayejani <ali_at_[hidden]> writes:
>> 
>> I understand that you're decomposing orthogonal concepts here. I like
>> that in principle, and it may be the right move for our
>> implementation, but I also feel fairly well convinced that most users
>> _should_ never have to know about these distinctions.
>
> Ok, I think I agree with that sentiment. The orthogonal low-level
> functionality is there when needed, but most users employ a simpler
> expression.

I sure hope so! That was always the intent behind the design.

>> Hmm, the way <library> in requirements expands to something which
>> includes a usage-requirement worries me a little now. Currently
>> there's no provision for this sort of cross-category composite feature
>> expansion. I begin to wonder if we shouldn't have a different way to
>> deal with usage requirements that puts them all in the requirements
>> section:
>> 
>> <use>b <usage:link-to>b # **
>> 
>> or something.
>
> I just received Vladimir's response...apparently I misunderstood
> this...<library> in Req does NOT imply <link-to> (i.e. <library> in
> Usage-Req).

...but /shouldn't/ it?

Incidentally, I'm not sure what he means by "some additional
semantics".

>> > # If, e.g., b's headers are in a's public interface, you might do
>> > # the following to expose b's usage requirements. (Again, from a
>> > # purist SW architecture point of view, I always recommend against
>> > # this sort of sneaky thing...but it comes up and this is how to do
>> > # it.)
>> > lib a : a.cpp : <library>b : : <uses>b ;
>> 
>> If a and b have any templates or inline functions, it may not matter
>> whether b's headers are in a's public interface. <use>b might still
>> be imposed on whoever is using a.
>> 
>> Should there be some way for a to say that any usage requirements it
>> inherits get propagated on to its dependents?
>
> Doesn't <library> (or <link-to>) in Usage Requirements *already*
> propagate (while <include> or <define> do not? or do they?) ?

I have no clue. The definition of <library> is:

feature library : : free dependency ;

so it is not "propagated", but on the other hand "propagated" also
denotes travel in the other direction: from dependents to
dependencies.

> I don't know what's right...and I'm happy to help specify...but I'm
> trying to understand what the current specification and/or
> implementatation is doing.

I'm not sure we should be trying to analyze the current behavior too
closely. I think it's clear that it hasn't been carefully thought
out in every detail, and there's no specification. We should be
trying to figure out what *should* be happening and work from there
to implement it.

>> > # If you wanted to have a statically-linked DLL, you would have to
>> > # do something like this. This would somehow have to interact with
>> > # the <link> feature. Note the difference.
>> > lib a : a.cpp : <uses>b <link-to>b ;
>> 
>> Which is the statically linked DLL (whatever that is!) in this case?
>> a or b?

This was:
> lib a : a.cpp : <uses>b <link-to>b ;

> Library "a" is the DLL, "b" is a static library linked into "a". The
> MSVC linker allows you to create a DLL (a dynamic or "shared" library)
> by statically linking to other libraries (e.g. "b"). In this case, it
> would be natural to say
>
> lib a : a.cpp b ;

Yes.

> which is confusing if you use the same expression to specify a shared
> lib "a" that depends on a shared lib "b".

Why? It seems reasonable to me that if b is built shared, all you
can do is inherit usage requirements from it, and if b is built
static you must link it in directly.

> But using the <link-to> feature that I defined, you can make the
> distinction.
>
> lib a : a.cpp : <link-to>b ;
> lib c : c.cpp : : : <link-to>b ;
>
> The first says "I link to b", the second says "my client links to b".

Nice for BBv2 implementors to be able to distinguish these, but nice
for users? I'm not convinced.

> If "a" is built <link>shared, it will statically link to "b" (I don't
> think this is possible with GCC) 

Sure it is, as long as "b" is built with <shared-linkable>true
(i.e. -fPIC). A GCC static library is just an archive of object
files; you can link them into anything that can link with .o files.

> and no user of "a" will know or care about "b". However, if "c" is
> built <link>shared, the <link-to>b will propagate to all direct and
> indirect users of "c" so that "b" appears on the final link line.
>
> (It's not clear to me whether either MSVC or GCC/AR can build a
> static library that contains other static libraries

AR can do it, but IIRC the linker will fail when it you tries to find
all the objects in the outer static library: the inner static lib
won't be recognized as a .o file. Essentially, no it can't do it.

Not really sure about MSVC, but it's easy to test.

> so if "a" is built <link>static, I'm not sure what would happen.)

One interesting possibility is that the build system is smart enough
to link all of b's object files into a without ever building b.

> Btw, I'm not necessarily advocating this approach, but just using it
> to illustrate some of the things I don't understand about the
> current implementation.

I think we're all in the same boat, brother.

>> > # These would be unusual, but illustrate what <library> in
>> > # usage-requirements means.
>> > lib a : a.cpp : : : <uses>b <link-to>b ; # client to lib a uses and links b
>> > lib a : a.cpp : : : <library>b ; # same as above
>> >
>> > # N.B. the second line would basically would pass on the <library>
>> > # property to the client of "a", which would then expand it to
>> > # Requirement <uses>b and ((for a "lib") Usage-Requirement <link-to>b
>> > # or (for a "exe") Requirement <link-to> b). Thus, functionally,
>> > # <library> only gets expanded when it is a Requirement.
>> 
>> Sneaky. This makes me think that (**) above was really on the right
>> track.
>
> Are we questioning the distinction between Requirements and
> Usage-Requirements or just the syntax for expressing them?

More importantly it's the architecture I'm questioning: the separate
internal handling of lists of regular and usage requirements. If you
can have a composite feature that can expand into both kinds of
requirements that organization gets very difficult. Sometimes, of
course, it's better to hide architectural organization behind an
interface, but in this case I think it might be better not to resist
exposing it.

> This proposed syntax might make your redundant <include> issue even
> uglier:
>
> lib a : a.cpp : <include>./include <usage:include>./include ;
>
> You may be thinking
>
> lib a : a.cpp : <include>./include ;
>
> applies both to me and to my users, but then how do you restrict to
> one or the other? <build:include> and <usage:include> perhaps?

Or maybe <*:include>./include to specify both. I guess it really
depends which is the more common usage. Probably it's both, so your
last suggestion above is on the right track.

Let me stress, however, that I'm not wedded to this particular syntax
(it may conflict with subfeature handling anyway) nor am I even deeply
concerned with syntax at this point. One interesting idea would be to
use "private:" to specify non-usage requirements.

Hmm, it occurs to me that usage requirements are almost the opposite
of propagated features, except that it's not clear exactly how far up
the dependency chain they propagate.

>> > # Executables
>> >
>> > exe e : e.cpp : <link-to>b ; # e merely links to b
>> 
>> Explicitly and stubbornly ignoring b's usage requirements?
>> OK, but it's inadvisable, right?
>
> Yes, inadvisable, but permissible and perfectly well-defined.

Sure.

>> > exe e : e.cpp : <uses>b <link-to>b ; # e directly uses b and links to b
>> > exe e : e.cpp b ; # same as above (uses and links)
>> > exe e : e.cpp : <library>b ; # same as above (uses and links)
>> 
>> I like these, especially the middle one.
>
> That, IMO, would be the "normal" way.

As I said, I sure hope there is no real problem achieving that,
because it rings true for me.

>> As I said, I have some concerns about <library> and the cross-category
>> expansion. In any case, the orthogonal <use> and <link-to> features
>> make sense to me.
>> 
>> >> One other concern I have is the common need to repeat properties in
>> >> the requirements and usage-requirements sections, particularly some
>> >> defines and most includes. It would be nice to eliminate that
>> >> redundancy.
>> >
>> > The above proposal addresses some of that...at least you don't need to
>> > specify both <uses> and <link-to> most of the time.
>> 
>> That really doesn't help with either of the cases I mentioned, I
>> think.
>> 
>> I'm thinking of:
>> 
>> lib x : x.cpp
>> : <include>./include <define>XSPOOF
>> : : <include>./include <define>XSPOOF ;
>
> But, you know, there is a difference between "I use" and "my client
> uses". When the include paths or macros are all contained in the
> private implementation, you really do want them just in Requirements,
> and when they apply only to the client, you really do want them just
> in Usage Requirements, so I wouldn't want to get rid of the
> distinction, even at the cost of having to specify twice.

Of course I don't mean to eliminate the distinction; that would be a
grave mistake. I just want to make things, like, oh, let me see if I
can find an example... The Boost Libraries! ...clean to specify.
They and their clients all must have the boost root directory in the
#include path.

> So I'm happy that they are separate. My solution to the above is.
>
> Req += <include>./include ;
> Req += <define>XSPOOF ;
>
> lib x : x.cpp : $(Req) : : $(Req) ;

I still think there ought to be an easier/cleaner way. That
introduces a level of indirection which harms readability. People
also seem to have a hard time adjusting to the use of bjam variables
;-)

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com