Re: [Boost-docs] [quickbook] Direct code import...

Subject: Re: [Boost-docs] [quickbook] Direct code import...
From: Daniel James (dnljms_at_[hidden])
Date: 2013-01-10 00:29:24

On 5 January 2013 21:27, Rene Rivera <grafikrobot_at_[hidden]> wrote:
> After having my Mac's drive get trashed I lost a set of notes I had on what
> needed to get finished for this. Does anyone have an idea on what needs to
> get done to complete this feature on boostbook-dev branch?

I'm not sure. The quickbook-dev branch was completely merged to trunk,
but I reverted the glob feature because it didn't completely work. One
problem was that it uses a bitset for all the possible character
values, which is fine when using 8-bit characters, but not when using
16-bit characters (on windows). There might have been others, I can't
remember.

At the time I wanted to support unicode, but since then I've found out
it's trickier than I thought, because different filesystems use
different normalisation forms, or more typically no normalisation at
all. Which makes consistent cross platform handling of unicode a bit
tricky. For example in HFS+ 'é' is actually two code points. So a glob
which is aware of UTF-8, but not aware of combining characters, won't
match 'café.txt' with 'caf?.txt', or even 'ca?é.txt' (that's on HFS+),
but will for other filesystems. I had hoped Boost.Locale might help
out here, but I think it requires ICU for normalisation, which is IMO
far too large a dependency.

So... for the time being, it'd probably be best if the glob only
supports ascii. If the glob contains non-ascii characters (i.e.
anything > 127), it should be an error, if a file does, it should be
ignored (actually could allow '*' to match non-ascii characters). This
won't be that much of a problem since I expect most people don't
really expect unicode filenames to work anyway (GNU find's glob has
the same issue, although interestingly, GNU bash has fixed it in
recent years).

I've attached an updated patch to restore the glob implementation.
Since the quickbook-dev branch was merged, I added support for
tracking dependencies, which the glob implementation doesn't do. I
will probably need to rework the dependency tracking a bit as it
combines checking for a file's existence with tracking (i.e. every
time quickbook looks to see if a file exists, it does it through the
dependency tracker so that it will be recorded). I think I'll need to
add glob support to the dependency tracker. Probably don't need to
implement the glob in the glob in the dependency tracker, but just
record it as a special type of checked path.

Since no new quickbook developments are going to be included in the
new boost release, and the next will use git, it probably makes sense
to switch to using git now. I'm currently running git-svn to create a
new repo, but it won't be the proper one (it's missing some branches,
and I haven't mapped the users, and probably other issues, the kde
svn2git might work better). It should be possible to copy new changes
over to the final repo, so it'll be a good place to start.