Re: [Boost-testing] Too many results?

From: David Abrahams (dave_at_[hidden])
Date: 2007-10-15 22:29:18

on Thu Sep 27 2007, Rene Rivera <grafikrobot-AT-gmail.com> wrote:

> I've been watching my Linux machine take incrementally more and more 
> time processing the regression results and uploading them to the OSL 
> server. If I'm lucky not too many new results come in, and it takes 3.5 
> hours (it used to take about half that). Still not entirely bad... But 
> the really bad part is that the generated results, which are compressed 
> into a ZIP file, takes more than 200M. It takes a long time to upload 
> that through my limited pipe. So I'm looking for suggestions on how to 
> improve the turn around time, and of course suggestions on how to reduce 
> the over all resources needed are welcome... Note, I'm looking for 
> things we can do quickly.
>
> The one though I've had so far is to use "rsync" instead of uploading a 
> ZIP, and them zipping on the OSL server. But the uncompressed sources 
> are more than 1.5Gig so it would make for close to 2Gig of disk space 
> use on the OSL server. Which is something I'd rather avoid.
>
> So, any clever ideas out there?

Use better compression?  Tokenizing the output and storing token
indices could be a huge win.

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com