Re: [Boost-build] Tarball/archive support in bjam

Subject: Re: [Boost-build] Tarball/archive support in bjam
From: Matthew Chambers (matthew.chambers_at_[hidden])
Date: 2009-08-11 10:57:21

That's a cool approach; it's a compromise between supporting it natively 
and using simple SHELL calls to do the archive manipulation. But it 
would be tricky to get it to work on Windows. I suppose some kind of 
initialization check could be done in the tgz rule to make sure that the 
gzip command exists in the current PATH and if not, to exit with an 
informative warning; linking the user to GnuWin32 on Windows and their 
package management system on *nix would go a long way.

Thanks,
-Matt

John Bito wrote:
> Making a tarball target is similar to the (Java) jar target that I'm
> working on.  The built-in composing generator seems that it ought to
> work. For your case, I'd start with something like:
>
> import type ;
> import targets ;
> import project ;
> import feature ;
> import generators ;
>
> type.register TAR : tar ;
> type.register GZIP_TAR : tgz ;
>
> feature.feature compression :
>     # specify the list of valid compression options here
>     : free incidental
>     ;
>
> rule tarball ( target-name : sources * : requirements * : default-build * )
> {
>    targets.create-typed-target GZIP_TAR : [ project.current ] : $(target-name) :
>        $(sources:G=) : $(requirements) : $(default-build) ;
> }
>
> generators.register-composing tar : CPP HPP EXE LIB : TAR ; # not sure
> how to add generic source file types
> generators.register-standard tgz : TAR : GZIP_TAR ;
>
> # Convert requirements to variables to substitute in the command line.
> rule tgz ( targets * : sources * : properties * )
> {
>     COMPRESSION on $(targets) = [ feature.get-values compression ] ;
> }
> # actual commands to run
> actions tgz
> {
>     gzip $(COMPRESSION) < $(<) >$(>) ;
> }
>
> actions tar
> {
>     tar -cf $(<) $(>) ;
> }
>
> tarball tarballname : [ glob *.cpp ] : <compression>--best ;
>
> Disappointingly, it errors out:
>
> error: properties found in the 'sources' parameter
>
> I'd like to know how to things properly so that doesn't happen.  It
> seems that the default composing generator assumes that its
> dependencies are targets coming from other generators and rather than
> non-derived files.
>
> I'll be working on this more later in the week, so maybe I'll have
> something that actually works.
>
> Good luck!
> John
>
> On Mon, Aug 10, 2009 at 13:36, Matthew
> Chambers<matthew.chambers_at_[hidden]> wrote:
>   
>> Hi all,
>>
>> I'd like to have support for .tar.gz, .tar.bz2, .tar.lzma, and .zip in bjam.
>>
>> I work on a library project with a BBv2 build system and we have tarballs
>> that we use as input and tarballs that we want to be build artifacts. We
>> currently package the full Boost tarball in our SVN repo and in our source
>> releases. Instead of that, we'd like to have a "build release package" rule
>> that will run bcp on our project (preferably only on updated dependencies),
>> add/update any new files in a bcp tarball, and finally build several
>> tarballs to distribute our source code and probably some Windows binaries
>> too. Some of the source tarballs would be subsets of the full source to make
>> it easy for users to pick a subset of our library.
>>
>> It would be very nice to handle all this inside of the BB dependency/update
>> web. When input tarballs change it would trigger all of its dependents to
>> rebuild. When output tarballs have updated dependencies it would trigger the
>> output tarball to rebuild. The logic should quite the same as what already
>> exists with the lib target. I suppose it's actually building the archive
>> (libarchive?) support into bjam that would be the messy part! Perhaps we
>> could adopt a system like boost::iostreams where the archive support is
>> compiled conditionally.
>>
>> I'd like to see rules with semantics like the lib target where it can be
>> either an input or output (or both?).
>> Here's some example code:
>> Also pasted on codepad: http://codepad.org/iNE5kl8o
>>
>> # extract a tarball
>> tarball some-source-tarball-id
>> : # sources (empty list specifies input tarball?)
>> : # requirements
>> # specifies the filepath of an existing tarball (only applicable for input
>> tarball)
>> <file>path/to/a/tarball.tgz
>>
>> # override default location to extract files;
>> # does defaulting (and, for relative paths, anchoring) to current Jamfile's
>> directory make more sense than using the build path?
>> <location>path/to/extracted/files
>>
>> # no default-build
>> # no usage-requirements
>> ;
>>
>> # create/update a tarball from the source tree
>> tarball output-source-tarball-id
>> : # sources (can be files or targets; for targets, the target's output files
>> are archived)
>> # add all .cpp and .hpp files, preserving their relative paths
>> [ glob-tree . : *.?pp ]
>>
>> : # requirements
>> # <location> on a <source> could override the path of the file in the
>> tarball?
>> <source>some-disjoint-header.hpp/<location>override/path/in/tarball
>>
>> # specify archive compression
>> <compression>lzma # alternatives: none, gzip, bzip2, zip; the zip would not
>> actually be a tarball
>>
>> # override name of output (default is the rule name)
>> <name>source-tarball # add file extension automatically if omitted?
>>
>> # override path of output;
>> # does defaulting (and, for relative paths, anchoring) to current Jamfile's
>> directory make more sense than using the build path?
>> <location>path/to/output/tarball
>>
>> # no default-build
>> # no usage-requirements
>> ;
>>
>> # create/update a tarball from the output of some build targets
>> tarball output-binary-tarball-id
>> : # sources
>> some-exe-target
>> : # requirements
>> <library>some-library-target/<with>features
>> <dependency>some-make-target
>> <name>binary-tarball.tar.bz2
>>
>> # no default-build
>> # no usage-requirements
>> ;
>>
>>
>> So, would anyone else find this helpful? I might be able to implement it in
>> bjam if it's approved of as a new feature.
>> _______________________________________________
>> Unsubscribe & other changes:
>> http://lists.boost.org/mailman/listinfo.cgi/boost-build
>>
>>     
> _______________________________________________
> Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
>