Re: [Boost-users] [bjam] Generated path approaches 255 character OS limit

Subject: Re: [Boost-users] [bjam] Generated path approaches 255 character OS limit
From: Vladimir Prus (vladimir_at_[hidden])
Date: 2008-11-06 15:43:12

Mat Marcus wrote:

> On Thu, Nov 6, 2008 at 11:31 AM, Vladimir Prus
> <vladimir_at_[hidden]> wrote:
>> Mat Marcus wrote:
>>
>>> On Thu, Nov 6, 2008 at 9:54 AM, Vladimir Prus <vladimir_at_[hidden]> wrote:
>>>> Steven Watanabe wrote:
>>>>
>>>>> AMDG
>>>>>
>>>>> Peter Klotz wrote:
>>>>>> When compiling Boost 1.37.0 using this bjam command line under Windows I obtained the error
>>>>>> message below:
>>>>>>
>>>>>> bjam -q -sNO_COMPRESSION=1 --build-type=complete --without-graph --without-math --without-mpi
>>>>>> --without-python --without-serialization --without-signals --without-wave --toolset=msvc-8.0
>>>>>> architecture=x86 address-model=64
>>>>>>
>>>>>> failed to write output file
>>>>>> 'bin.v2\libs\test\build\msvc-8.0\debug\address-model-64\architecture-x86\asynch-exceptions-on\link-static\runtime-link-static\threading-multi\libboost_prg_exec_monitor-vc80-mt-sgd-1_37.lib.rsp'!
>>>>>>
>>>>>> Obviously the path I started from (70 characters) combined with the bjam generated path (192
>>>>>> characters) exceeds the OS limit of 255 characters per filename.
>>>>>>
>>>>>> Does the bjam generated path really have to be this long? Can I shorten it somehow?
>>>>>>
>>>>>
>>>>> You can pass --abbreviate-paths to bjam
>>>>
>>>> Probably I should just go ahead and implement MD5 paths.
>>>>
>>>> - Volodya
>>>
>>> Is there a way to get boost build to generate the same paths when
>>> using either an msvc-compiled bjam or a cygwin compiled-bjam? Right
>>> now, if I understand correctly, if I invoke bjam with:
>>>
>>>     some_path/bin.cygwinx86/bjam.exe threadapi=posix target-os=cygwin gcc
>>>
>>> target-os-cygwin doesn't show up in the generated path. But if I invoke:
>>>
>>>     some_path/bin.ntx86/bjam.exe threadapi=posix target-os=cygwin gcc
>>>
>>> the generated path will include
>>>
>>>   .../target-os-cygwin/threadapi-pthread/...
>>>
>>> I would like to force the same path (or hash) to be used. As far as I
>>> know, this is not possible today. It would be nice if hashes were
>>> implemented to contain all features used in the build, since the 255
>>> character ceiling should no longer be a problem. Then it would be
>>> possible to share the directory used for built artifacts, regardless
>>> of how the build tool was compiled.
>>
>> With hashing, the paths will be the same. The target-os feature is
>> defaulted to the host-os bjam was built on -- with cygwin considered
>> a OS. When build paths are constructed, the properties which values
>> are equal to default values and hidden from the path. This behaviour
>> optimizes for the case where you're building on one OS for the same
>> OS.
>>
>> If hashing is used, then there's no need to suppress default values,
>> since it does not make paths anyway longer, so the property set
>> will be exactly the same.
> 
> This is good news.
> 
>> Oh, wait, no :-( I think the <host-os>
>> property will be different.
> 
> While I understand the rationale behind making target-os default to
> host-os, I don't see why we would want host-os in the path. Could you
> give the reason? I've encountered issues with libraries that make
> inappropriate assumptions based on host-os in the past.

Well, we certainly need to have the host-os property, so that we can
adjust build behaviour where host os matters, but on the other hand,
you are probably right -- we don't need to record that in the property
set associated with the final build products.

>> I am not really sure what is the right solution for your problem.
>> If we always include target-os in the target path, this will make
>> the common usage less nice.
> 
> How does including target-os in the path make common usage less nice,
> especially if you continue to set it to host-os by default?

If I have a linux box, and building only native applications, why do
I have to see:

        target-os=linux

in the build path?

>> It's probably possible to declare that
>> cygwin is actually windows, but I don't know if cygwin users will
>> be happy.
> 
> I'm not suggesting that. Right now I use
> 
> 
> toolset.add-requirements
>   <toolset>gcc:<target-os>cygwin
>   <toolset>gcc:<threadapi>pthread ;
> 
> toolset.add-requirements
>   <toolset>msvc:<target-os>windows
>   <toolset>msvc:<threadapi>win32 ;
> 
> in my user-config.jam. This hack gives me the desired defaults,
> regardless of how bjam.exe was compiled. I was hoping that maybe
> hashing would solve the remaining piece of my problem by allowing all
> features to be encoded.

Given your point about host-os, hashing probably will solve your problem.
Want me to hack together an experimental patch?

- Volodya