On Thu, Jan 8, 2015 at 8:27 AM, Rene Rivera <grafikrobot@gmail.com> wrote:

On Thu, Jan 8, 2015 at 7:07 AM, Tom Kent <lists@teeks99.com> wrote:

I had my machine go down last week, and I'm only now getting back to this. I just tried a couple runs and had two issues:

1) It takes a *long* time to run >30min....I think the previous version was <5min on the same machine. Did it use to be an incremental process? Most of results don't change between runs, so does it need to re-process everything? Can it just check timestamp (or maybe checksum) of the result sets?

It shouldn't be longer, except for the first time. As it's running the exact same commands, just wrapped in a shell script. Can you tell which part is taking so long?

I've added some timing outputs before each step in the script (and manually gaged some with a stopwatch)...here's what I found:

After cleaning out my directory (getting everything from scratch) and running (roughly adding the numbers to the left manually) here's what I saw:

$ cat times_first_run.log 

build_setup Fri Jan 9 22:26:11 UTC 2015 --- 0s

git_update boost_root Fri Jan 9 22:26:11 UTC 2015 --- 10.5m

git_update boost_regresssion Fri Jan 9 22:36:50 UTC 2015 --- 1s

git_update boost_bb Fri Jan 9 22:36:51 UTC 2015 --- 3s

update_tools Fri Jan 9 22:36:54 UTC 2015 --- 0s

bootstrap Fri Jan 9 22:36:54 UTC 2015 --- 30s

build bb Fri Jan 9 22:37:27 UTC 2015 --- 5m

build_results develop Fri Jan 9 22:42:31 UTC 2015 --- 17m

build_results master Fri Jan 9 22:59:35 UTC 2015 --- 13m

upload_results develop Fri Jan 9 23:12:48 UTC 2015 --- 30s

upload_results master Fri Jan 9 23:13:20 UTC 2015 --- 30s

complete Fri Jan 9 23:13:56 UTC 2015

Immediately running aging at the end of the first output resulted in:

$ cat times_second_run.log 

build_setup Mon Jan 12 14:09:14 UTC 2015 --- 0s

git_update boost_root Mon Jan 12 14:09:14 UTC 2015 --- 2.5m

git_update boost_regresssion Mon Jan 12 14:11:47 UTC 2015 --- 1s

git_update boost_bb Mon Jan 12 14:11:48 UTC 2015 --- 1s

update_tools Mon Jan 12 14:11:49 UTC 2015 --- 0s

bootstrap Mon Jan 12 14:11:49 UTC 2015 --- 30s

build bb Mon Jan 12 14:12:25 UTC 2015 --- 2.5m

build_results develop Mon Jan 12 14:15:01 UTC 2015 --- 13m

 - downloading + unzipping files (2 downloaded) --- 30s

 - reading xml, merging results, generating links --- 11.5m

 - writing documents --- 1m

build_results master Mon Jan 12 14:28:11 UTC 2015 --- 9.5m

 - downloading + unzipping files (0 downloaded) --- 5s

 - reading xml, merging results, generating links --- 9m

 - writing documents --- 30s
upload_results develop Mon Jan 12 14:37:38 UTC 2015 --- 30s

upload_results master Mon Jan 12 14:38:12 UTC 2015 --- 30s

complete Mon Jan 12 14:38:54 UTC 2015

Are those times similar to what you're seeing? This was a mult-core machine with 3.5GB of RAM, so nothing too tiny.