Re: [Boost-build] Muktiple depoendent projects vs. boost build MkDir

From: Vladimir Prus (ghost_at_[hidden])
Date: 2008-04-08 15:37:02

On Tuesday 08 April 2008 23:00:14 Mat Marcus wrote:
> On Tue, Apr 8, 2008 at 10:24 AM, Vladimir Prus <ghost_at_[hidden]> wrote:
> >
> > On Tuesday 08 April 2008 00:09:42 Mat Marcus wrote:
> > > I am having some difficulty figuring out how to get boost build to do
> > > my bidding. Perhaps someone here can help. I have to completely
> > > separate project directory trees which I'll call tree1 and tree2. Each
> > > of these trees has at least the following files at the top level. Here
> > > is an abstracted picture of my directory structure:
> > >
> > > tree1
> > >   - project-root.jam
> > >   - boost-build.jam
> > >   - Jamfile.v2
> > >   - deep hierarchy of subdirectories some of which contain there own
> > > Jamfile.v2's
> > >
> > > tree2 (optional, depends on tree1 relative location and artifacts)
> > >   - project-root.jam
> > >   - boost-build.jam
> > >   - Jamfile.v2
> > >   - deep hierarchy of subdirectories some of which contain there own
> > > Jamfile.v2's
> > >
> > > built_artifacts:
> > >   - Build dir used by both tree1 and tree2 Jamfiles
> > >
> > > tree2, if present, expects to be located as a peer to tree1 in the
> > > directory structure. Furthermore, built artifacts (from either tree)
> > > are to be located in built_artifacts. tree2 depends on some of the
> > > artifacts in tree1. Users of tree1 can "build the world" by invoking
> > > bjam when the working directory is tree1. Jamfiles in tree2 contain
> > > relative references to projects in tree1.
> > >
> > > Now I can pose my question.  I would like for users of tree2 to be
> > > able to build both tree1 and tree2 by simply invoking bjam when the
> > > working directory is set to tree2. In order to make this possible,
> > > tree2/Jamfile.v2 contains a statement along the lines of build_project
> > > ../tree1 . This seems to work fine in case tree1 is already built. But
> > > if a user sets the working directory to tree2 and invokes bjam when
> > > the built_artifacts directory doesn't yet exist, then strange things
> > > begin to happen. As far as I can tell, what happens is that first
> > > tree1 is built successfully, but as boost build reaches the point
> > > where it tries to create tree2 artifacts I receive errors such as:
> > >
> > > MkDir1 ..\built_artifacts
> > > A subdirectory or file ..\built_artifacts already exists.
> > >
> > >     mkdir "..\built_artifacts"
> > >
> > > ...failed MkDir1 ..\built_artifacts...
> > >
> > > If I run bjam again from the same location, the build continues
> > > successfully for a while. After several invocations of bjam all
> > > targets are built successfully.
> > >
> > > In the past I've spent a bit of time debugging this (on mac, win,
> > > cygwin), and it seems the issue relates to a combination of the fact
> > > that mkdir returns an error code when it tries to create a directory
> > > that already exists, and the fact that boost build asks for the same
> > > dir to be created at two different points in the build process under
> > > the above configuration.
> > >
> > > For now I provide a thing wrapper script around bjam that will execute
> > > it multiple times. But the error output doesn't make a good impression
> > > on my library users so I'd really like to find a better solution. Can
> > > anyone here advise me?
> >
> > This can only happen if Boost.Build tries to create the same directory
> > twice with different paths. Can you provide the output of "bjam -n"
> > for the case where error happens, so that I know what those two different
> > paths are?
> >
> > - Volodya
> 
> That's helpful  to know. bjam -n output is attached for the error case.
> 
> First Mkdir appears to be on line 16.
>   MkDir1 C:\asl\staff\mmarcus\sandbox\built_artifacts
> 
> Second mkdir appears on line 8964
>   MkDir1 ..\built_artifacts
> 
> Additional details:
>   proj1/project-root.jam contains:
>     path-constant TOP : . ;
> 
>   proj1/Jamfile.v2 contains
>     project
>       # ...
>       :  build-dir
>      $(TOP)/../built_artifacts ;
> 
>   proj2/project-root.jam contains:
>     path-constant TOP : . ;
> 
>   proj1/Jamfile.v2 contains
>     project
>       # ...
>       :  build-dir
>      ../built_artifacts ;
> 
> Thanks to your observation, I find that changing the last line from
> ../build_artifacts to match proj1's structure:
> $(TOP)/../built_artifacts seems to have allowed me to workaround the
> MkDir failure in this case. Thank you very much! Out of curiosity,
> would it be difficult to make MkDir could be more forgiving in this
> area?

This is why I've asked you for the log. It seems we can either normalize
all paths into absolute ones, or we can make MkDir detect that the path
has already magically appeared and skip the mkdir call. I'll think which
one is easier.

Thanks,
Volodya