Re: [boost] [Git] Moving beyond arm waving?

Subject: Re: [boost] [Git] Moving beyond arm waving?
From: Dave Abrahams (dave_at_[hidden])
Date: 2011-02-09 10:58:41

At Tue, 08 Feb 2011 15:10:03 -0600,
Rene Rivera wrote:
> 
> the "existing testing" is using the current scripts but with Git sources.

Thanks for explaining; it really wasn't obvious to me.

Using the current scripts but with Git sources... obviously the scripts would
need to be adjusted to fetch sources differently, and assuming we're
modularized, Jamfiles would need to be adjusted to the new directory structure.
Aside from that, I can't think of anything that would be different.  Does that
answer your question?

> >> in incremental mode with a "forwarding headers tree plus separate sources"?
> >> What would testers need to rely on? Cmake? Python? Ryppl? Ctest? SSH? Git?
> >> FTP? HTTP? HTTPS? Git protocol?
> >
> > Here's where I think we'll end up:
> >
> > - testers need to set up and register a buildbot buildslave.  I believe we can
> >    make that relatively trivial for people
> 
> Even though I like buildbot.. I'm less than thrilled about adding setup steps
> for testers. 

What I mean by "relatively trivial" is "one command," or nearly so.

> As one of the testing goals is to make it simple for someone to
> provide testing resources, and hopefully fine-grained resources. Which means...
> 
> > - that would imply a reliance on, at a minimum:
> >
> >    - Python
> >    - A net connection
> >
> > Other things they will probably need to have, or the testing process will need
> > to get for them:
> >
> >    - CMake
> >    - C and C++ compilers
> >    - BuildBot
> >    - Twisted
> >    - Git or LibGit2 or Dulwich
> >    - Git or HTTP or HTTPS protocol capability
> 
> ...that any additional required programs/scripts would ideally be automatically
> installed to a sandbox (i.e. locally to not need system administrator
> access). 

Naturally.  You know I've been thinking about these things.  virtualenv is
really good for that.

> For example; I've had a good deal of experience dealing with installing
> Twisted and it doesn't approach being nice about being installed anywhere other
> than in the system location. 

https://gist.github.com/818662

> But my experience might be dated by now, since I
> haven't looked at it in more than a year. And as you can see from my other
> posts, I'm trying to figure out what the experience is like of installing Git is
> :-) With less than pleasurable results so far :-(

Yeah, well as I said there are several alternatives to actually having Git.  I'd
rather not make testers install it either.

> >>>> 3. Or is there no way to get a complete with source integrated tree?
> >>>
> >>> Please define "complete with source integrated tree" so I can answer that
> >>> question.
> >>
> >> I mean a tree in the form we have it now, or equivalent to what we intend to
> >> *release* as the *single* Boost C++ Libraries package.
> >
> > Well, we need to decide the details, but I can imagine two possibilities for a
> > Boost distro:
> >
> > 1. It is equivalent to the result of doing a "git clone" of the Boost superproject
> >     plus "git submodule update --init" to get all the subprojects (possibly with
> >     the .git directories deleted)
> >
> > 2. It's like #1 plus a generated forwarding headers directory.
> >
> > Either of these is pretty trivial to generate.
> 
> I see. Is there a way to do the same without Git? I.e. do the tar/zip archives
> available from github contain the same files as the clone+submodules?

Yes.  The tarball won't contain the submodules, but references to them that can
be similarly downloaded and untar'd.

> > My plan is to require as little as possible.  I've had some good experiences
> > with writing BuildBot recipes that get all the pieces needed into the
> > environment before running the tests, and I think that can be done
> > straightforwardly.
> 
> OK. A comment on that at the end...
> 
> >> My fear is that we end up with a system that is conceptually hard to understand
> >> how to put together and hence hard to make work. For example, testing at the
> >> moment doesn't depend on SVN. It's possible to download the tree as a tar
> >> archive through the almost universally available HTTP protocol.
> >
> > Yes.  Well, *incremental* testing today implies either having SVN or some kind
> > of rsync-like mechanism to avoid updating unchanged files.
> 
> Currently yes. But it's possible to make it not be required. It would be
> straightforward to change the regression scripts to download a new tree, using
> the tar archive option 

Don't forget Windows; no tar there usually.

> and assuming the time stamps are correctly preserved (it
> does already IIRC), and move over the "bin.v2" directory from the old tree to
> the fresh one. And since all the generated files and timestamps in the bin.v2
> dir are appropriate one could do an incremental test run on the new tree.

So there are straightforward ways of doing these things in either
system... someone just needs to code them.

> >> This is just one example. I'd have to see what the entire testing pipeline
> >> looks like to figure out where the "locked in" points are. I.e. I need to know
> >> what's actually going on, rather than just a "run this and that's it" answer.
> >
> > Provided I can automate everything else, I think all you need to know is what
> > ports have to be open, right?
> 
> ...And the comment I promised above...
> 
> No, there's more. The assumption is that testers have the capability to open a
> port.  Also assumes that we want them open a port.

Wha?  I never meant to imply we were going to ask testers to open a port!

The current system has to communicate over the web, which means *some* port has
to be open somewhere, to talk to the outside world.  The same would apply, and
we'd make sure any of the most commonly-open ports (e.g. 80) will work.  Just as
it is now.  That's a no-brainer, isn't it?

> I'd argue that we don't want them to open a port because it's yet another
> manual setup step they have to do.

Of course we don't!

> Essentially if I want to make it possible for someone at home, with their
> cable/DSL connected PC, to runs tests, I need to make it as simple as: 1. Have
> Python >= 2.4; 2. download "test.py"; 3. run "python test.py". In my dreams it
> would be even simpler, and just be download an installer, run it, and nothing
> else. 

Exactly.

> Which brings up another item; Your description so far also assumes the
> tester will need to install/setup a variety of programs. Which is moving in the
> wrong direction, IMO.

I think you misunderstood my earlier remarks.  The point of the list above was
to enumerate both a. what you need to have to get started and b. what the system
will ultimately cause you to have on your machine either via manual or automatic
installation.  Some people care about what software will get installed, and
that's what I thought you were asking about.  As I said earlier, 

       My plan is to require as little as possible

I mean that literally.

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com