Re: [boost] Issues with intel's compiler and newer builds of GCC

Subject: Re: [boost] Issues with intel's compiler and newer builds of GCC
From: Bryce Lelbach (admin_at_[hidden])
Date: 2010-10-20 13:29:05

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, 20 Oct 2010 11:41:09 -0500
"Hartmut Kaiser" <hartmut.kaiser_at_[hidden]> wrote:

> > > This is now a problem for the MSVC-7.1 test, too. It, too, times out.
> > > The logs from the build machine have warnings. These warnings are
> > > identical to ones that I get with MSVC-7.1 when building Serialization
> > > (rather, the warnings shown on the build logs are a subset of the
> > > warnings I get, the logs seem to cut off after a certain amount of
> > > warnings, e.g. 65k). Building Serialization with MSVC-7.1 passes all
> > > tests on my machine and builds all examples, without any trouble.
> > >
> > > I'm sure the argument could be made that five minutes (300 seconds) is
> > > more than reasonable, and the XML grammar for Serialization could
> > > definately be refactored (it's structure is nearly identical to the
> > > old Spirit.Classic grammar; I didn't want to get fancy). However, the
> > > grammar works, and I'd really like to not have false negatives from
> > > the build machines. This is, to my knowledge, the first component of
> > > Boost that uses Spirit 2.x (Hartmut/Joel please correct me if I am
> > > wrong. Wave uses Spirit Classic, I believe). The Spirit 2.x examples
> > > aren't compiled by the tests, so I am assuming that this is the first
> > > time the build bots have compiled larger Spirit 2.x parsers.
> > >
> > > The compile times for MSVC 10, GCC and clang are all far more
> > > reasonable than Intel and MSVC 7.1 on my machines.
> > 
> > I agree that the time limit can be pretty annoying - I had to refactor a
> > large number of the Boost.Math tests to avoid long run times.  However,
> > the limit is not unreasonable either - it is important that the tests
> > cycle in a reasonable time - remember that the CPU time on the build bots
> > has all been graciously donated, and isn't entirely free of cost.  We also
> > need to consider the impact on the end user of long compile times, also on
> > the occasional "casual" tester of Boost.Serialization - the time taken to
> > run all the tests is pretty long, so anything that can be done to reduce
> > that would be a big win.
> > 
> > Would it be worth getting the spirit2 developers in on this to see if
> > there is any low-hanging fruit that can be pulled?
> 
> Not that I know of :-(
> We're trying to bring down Spirit's compilations times, but that's a slow
> process...
> 
> Couldn't we customize the max time before cutoff for compilers known to be
> slow (icc) only?
> 
> Regards Hartmut
> ---------------
> http://boost-spirit.com

> 
> 
> 
> _______________________________________________
> Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

> 

There's a lot of room in the grammar for refactoring; some of the productions in
the grammar are now deprecated in the XML standard (in favor of less annoying
rules).

However, I really don't want to refactor the XML grammar half-heartedly. Currently,
the grammar works perfectly, and the tests should reflect that.

If my schedule is freed up a bit in the future (I have recently taken on and am
currently working on something of an ambitious scope), I would implement the
following changes to Serialization archives:

  * Rewrite all the existing archives using Spirit 2.x (Karma for output archives,
    Qi for input archives).
  * Create a new interface, allowing users to write new archive types using Qi
    and Karma.

I'm hesitant to refactor the XML grammar at this point, because the current
implementation of the grammar is a bit un-Spirit-like. In fact, the template
class basic_xml_grammar is not even a qi grammar (e.g. it doesn't inherit from
qi::grammar). Under the hood, the Spirit stuff for Serialization is somewhat
unorthodox. This is how Ramey originally did it; it worked and performed well
at runtime with Spirit Classic, and it works and performs well at runtime now.
I could make it work better and perform faster at runtime and compile time, but
the best way to do that would involve ripping up a lot of the existing archive
internals.

I'll contact the test runner. IMHO, timeouts should not have a fixed value;
instead, new tests should not timeout at all on their first run, but the time
they take to run should be recorded. The runtime duration of each run after the
first should also be recorded. This would allow a range of predicted durations
to be computed, and the high value in that range should be used to compute a
timeout duration. This would also be helpful because it could show Boost TMP
maintainers how changes in their libraries affect compile times.

- -- 
Bryce Lelbach aka wash
http://groups.google.com/group/ariel_devel

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAky/JuEACgkQO/fqqIuE2t6bwACgwVT+2Nn/s3Yp+DyqtxK5aaH2
WAcAoNMfmKyzzNELcBnOIgNCXzbLxlgW
=j/WR
-----END PGP SIGNATURE-----