[Boost-build] Tarball/archive support in bjam

Subject: [Boost-build] Tarball/archive support in bjam
From: Matthew Chambers (matthew.chambers_at_[hidden])
Date: 2009-08-10 16:36:06

Hi all,

I'd like to have support for .tar.gz, .tar.bz2, .tar.lzma, and .zip in bjam.

I work on a library project with a BBv2 build system and we have 
tarballs that we use as input and tarballs that we want to be build 
artifacts. We currently package the full Boost tarball in our SVN repo 
and in our source releases. Instead of that, we'd like to have a "build 
release package" rule that will run bcp on our project (preferably only 
on updated dependencies), add/update any new files in a bcp tarball, and 
finally build several tarballs to distribute our source code and 
probably some Windows binaries too. Some of the source tarballs would be 
subsets of the full source to make it easy for users to pick a subset of 
our library.

It would be very nice to handle all this inside of the BB 
dependency/update web. When input tarballs change it would trigger all 
of its dependents to rebuild. When output tarballs have updated 
dependencies it would trigger the output tarball to rebuild. The logic 
should quite the same as what already exists with the lib target. I 
suppose it's actually building the archive (libarchive?) support into 
bjam that would be the messy part! Perhaps we could adopt a system like 
boost::iostreams where the archive support is compiled conditionally.

I'd like to see rules with semantics like the lib target where it can be 
either an input or output (or both?).
Here's some example code:
Also pasted on codepad: http://codepad.org/iNE5kl8o

# extract a tarball
tarball some-source-tarball-id
: # sources (empty list specifies input tarball?)
: # requirements
 # specifies the filepath of an existing tarball (only applicable for 
input tarball)
 <file>path/to/a/tarball.tgz

 # override default location to extract files;
 # does defaulting (and, for relative paths, anchoring) to current 
Jamfile's directory make more sense than using the build path?
 <location>path/to/extracted/files

# no default-build
# no usage-requirements
;

# create/update a tarball from the source tree
tarball output-source-tarball-id
: # sources (can be files or targets; for targets, the target's output 
files are archived)
 # add all .cpp and .hpp files, preserving their relative paths
 [ glob-tree . : *.?pp ]

: # requirements
 # <location> on a <source> could override the path of the file in the 
tarball?
 <source>some-disjoint-header.hpp/<location>override/path/in/tarball

 # specify archive compression
 <compression>lzma # alternatives: none, gzip, bzip2, zip; the zip would 
not actually be a tarball

 # override name of output (default is the rule name)
 <name>source-tarball # add file extension automatically if omitted?

 # override path of output;
 # does defaulting (and, for relative paths, anchoring) to current 
Jamfile's directory make more sense than using the build path?
 <location>path/to/output/tarball

# no default-build
# no usage-requirements
;

# create/update a tarball from the output of some build targets
tarball output-binary-tarball-id
: # sources
 some-exe-target
: # requirements
 <library>some-library-target/<with>features
 <dependency>some-make-target
 <name>binary-tarball.tar.bz2

# no default-build
# no usage-requirements
;

So, would anyone else find this helpful? I might be able to implement it 
in bjam if it's approved of as a new feature.