Re: [Boost-docs] [quickbook] Direct code import...

Subject: Re: [Boost-docs] [quickbook] Direct code import...
From: Rene Rivera (grafikrobot_at_[hidden])
Date: 2013-01-10 03:57:32

On Wed, Jan 9, 2013 at 6:29 PM, Daniel James <dnljms_at_[hidden]> wrote:

> On 5 January 2013 21:27, Rene Rivera <grafikrobot_at_[hidden]> wrote:
> > After having my Mac's drive get trashed I lost a set of notes I had on
> what
> > needed to get finished for this. Does anyone have an idea on what needs
> to
> > get done to complete this feature on boostbook-dev branch?
>
> I'm not sure. The quickbook-dev branch was completely merged to trunk,
> but I reverted the glob feature because it didn't completely work. One
> problem was that it uses a bitset for all the possible character
> values, which is fine when using 8-bit characters, but not when using
> 16-bit characters (on windows). There might have been others, I can't
> remember.
>

I don't see how the bitset would be a problem. As it's templated to adjust
to the bitsize of the character type. Hence it works for woth wide and
narrow character types. But obviously it has no idea about variable sized
encodings (UTF-*, etc.) and will misbehave in such cases. But there's
nothing that can be done about that other than making the path strings
return from the filesystem lib be decoded correctly to the compiler wide
characters. In other words, it relies on `fs::path::generic_wstring` doing
something sensible. Which I'm guessing from your further exposition below
that it doesn't do something sensible?

Or am I totally missing the point? And you are talking about the glob
specification string itself?

> At the time I wanted to support unicode, but since then I've found out
> it's trickier than I thought, because different filesystems use
> different normalisation forms, or more typically no normalisation at
> all. Which makes consistent cross platform handling of unicode a bit
> tricky. For example in HFS+ 'é' is actually two code points. So a glob
> which is aware of UTF-8, but not aware of combining characters, won't
> match 'café.txt' with 'caf?.txt', or even 'ca?é.txt' (that's on HFS+),
> but will for other filesystems. I had hoped Boost.Locale might help
> out here, but I think it requires ICU for normalisation, which is IMO
> far too large a dependency.
>

Right, see above.. And, yes, including an ICU dependency would be more than
we could expect (at the moment).

[...]

I've attached an updated patch to restore the glob implementation.
> Since the quickbook-dev branch was merged, I added support for
> tracking dependencies, which the glob implementation doesn't do. I
> will probably need to rework the dependency tracking a bit as it
> combines checking for a file's existence with tracking (i.e. every
> time quickbook looks to see if a file exists, it does it through the
> dependency tracker so that it will be recorded). I think I'll need to
> add glob support to the dependency tracker. Probably don't need to
> implement the glob in the glob in the dependency tracker, but just
> record it as a special type of checked path.
>

As long as the special glob dependency changes correctly based on the
overall results set of the glob just keeping such a special glob should
work for the common cases. About the only problem I can see is when a glob
result set includes a file that is also included some other way (directly
or through a glob). It might be tricked into misbehavior. But it might be
it doesn't matter depending on how the tracking is used. Can't say I know
how it works since it's news to me it exists ;-)

> Since no new quickbook developments are going to be included in the
> new boost release, and the next will use git, it probably makes sense
> to switch to using git now. I'm currently running git-svn to create a
> new repo, but it won't be the proper one (it's missing some branches,
> and I haven't mapped the users, and probably other issues, the kde
> svn2git might work better). It should be possible to copy new changes
> over to the final repo, so it'll be a good place to start.
>

I would highly recommend getting the user mapping working. It makes for far
more readable histories. Dave should have a usable mapping file at this
point (I hope). So we could use that.

-- 
-- 
-- Grafik - Don't Assume Anything
-- Redshift Software, Inc. - http://redshift-software.com
-- rrivera/acm.org - grafik/redshift-software.com
-- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo