Re: [boost] [serialization] feedback on a possible port of the XML archive grammar to Qi

Subject: Re: [boost] [serialization] feedback on a possible port of the XML archive grammar to Qi
From: Robert Ramey (ramey_at_[hidden])
Date: 2010-09-10 13:07:01

Bryce Lelbach aka wash wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 09/09/2010 02:32 AM, Robert Ramey wrote:
>> Note that there is an extensive test suite for the serialization
>> library. You should run that on your local machine.
>
> I've finished and done (preliminary) testing on the new grammar. All
> tests passed on x86_64-linux-gnu-gcc-4.5.1 (link=shared,
> variant=debug), four warnings (unsigned/signed comparisons in other
> parts of Serialization that I haven't touched). I've got my machine
> running the full test suite (link=shared,static,
> variant=debug,profile,release).

OK this looks very good.  Just out of curiousity - what method
did you use to run the tests?

> I'm also running the profile.sh script in performance (I can't find
> any existing performance data for Serialization to compare against in
> the HTML docs, though).

Very good. I've only made a few performance tests.  Perhaps
you want to make one or two for xml_?archives.  I reallize this is
sort of a pain.  But it is very helpful and the test will be a permanent
contribution.

> I can't do Windows tests: I currently only have Linux machines
> available to me.
>
> My local copy of the boost trunk is checked out with svn via https.
> How should I get the changes to you for review? Patch?

If you have update access to the trunk, the easiest for me would
be if you just checked them in.  And BTW, you'd get test results
right off the test matrix.  If you don't have trunk accesss, I guess
the best would be to send me the changed files.

>> My understanding is that the new library is much faster than the
>> currently used one so people should be happy about that.
>
> I don't think we'll see a notable speed increase (at least, not with
> the work I've done so far). The bottleneck in the xml parser is the
> input stream -> intermediary string -> spirit design pattern in
> basic_xml_grammar<CharType>::my_parse.

Hmmm - how do you know this?

> Removing the middleman string is a bit of a problem. A stream
> iterator such as std::istream_iterator can't be used with Spirit,
> because of the backtracking present in a a recursive descent parser
> such as Qi.

> Spirit provides a multi_pass iterator (fulfills forward iterator)
> that can wrap an input iterator for use with Qi. Rules that present
> the possible need to backtrack will cause buffering with the
> multi_pass iterator.

As I remember that is what I used.  I found it to be no problem
in 8 years.  In my recollection, it simplified my code.  So
you should really look into this.

> So, to (hopefully) get a respectable increase in speed, I'll have to
> refactor the new Qi grammar to minimize use of rules that will
> backtrack, and then modify the grammar and interface to use the
> multi_pass iterator. If the new parser doesn't break horribly on
> Windows/other compilers, I'll get started on the multi_pass stuff
> this weekend.

I'm quite confident that if it passes with a recent version of gcc
it will pass with recent versions of windows compilers (with a
fiew tweaks).  So before you check it in I would like to see.

a) a performance test
b) usage of the multi-pass iterator.

Robert Ramey