Re: [Boost-docs] [quickbook] Direct code import...

Subject: Re: [Boost-docs] [quickbook] Direct code import...
From: Rene Rivera (grafikrobot_at_[hidden])
Date: 2011-10-30 04:14:34

First, I need to say I'm not forcing this on anyone. I put in the svn
repo as: a) I wanted to have it available someplace other than my one
machine; b) I thought it might be something to spur conversations as to
what literate programming support might look like. Hence I wasn't
expecting it to go much further in its current form.

Note, below I haven't actually looked at the dev branch code to see hwo
that works.. So my answers may be totally bogus. I.e. mostly I'm
answering from the context of the trunk quickbook version. So please
correct my ignorance as you see fit ;-)

On 10/29/2011 7:34 AM, Daniel James wrote:
> On 29 October 2011 03:48, Rene Rivera<grafikrobot_at_[hidden]> wrote:
>> [import ../../../boost/predef/version_number.h]
>> [import ../../../boost/predef/architecture/.*\.h]
> I still don't like this much, but since it looks like this will be
> forced on me, here's a review:
> It would have been better if you'd branched from quickbook-dev, as
> there are some changes to import/include there that are very relevant.

OK. I was going under the plan that after the release you would merge
that dev branch to trunk. And then I would work from that state. Mostly
because it's hard enough to keep track of two quickbooks. I'd hate to
add a third for myself. And I was in a hurry to start documenting, but
not in a hurry to see this integrated into the main-line quickbook.

> There's the conflicting include source file thing that I implemented
> there but it should be pretty easy to adapt to your purposes. If you
> try building the example I pointed you to, or even just looked at
> 'gold' test output, it should be obvious what it does.

OK. I guess I missed, or forgot about, that pointing :-) I'll look again
at the previous posts.

> I'd rather use 'include' than 'import', the idea being that 'include'
> is for including content, 'import' is for importing templates and
> macros. This should mean that 'include' wouldn't import the other
> snippets in the file. With your change it isn't clear what an 'import'
> does without looking at the imported file.

Well, there's at least one problem with using "include".. It would mean
duplicating the file parsing that "import" already does. And if we
restructure so that the grammar can be used, it would then complicate
the "import" also since it has to now also ignore the escape comment
parts that "include" is using. Which I guess means that both the file
inclusion and file import should be combined into a comprehensive file
reading and parsing mechanic. And then based on the template name doing
the parsing we either do import like things, or we do include like things.

> 'quickbook-dev' has a 'call_code_snippet' function which is a lot
> easier to use than 'do_template_action'.

Didn't seem that hard to use do_template_action. Although I did spend
longer than I expected trying to figure out the combination of values it
wanted :-\ But I guess any modularity in this context is a good thing.

> I'd rather use explicit syntax to distinguish between paths and
> patterns. This change breaks the windows path support, which I think

Technically it doesn't break it. As long as you don't use a regex for
the file it behaves as before.

> means that it can no longer be used to build some older versions of
> the maths documentation (I want to phase out support for windows
> paths, but try to keep quickbook backwards compatible, that's why I
> just added a warning for now). I also plan on implementing escapes for
> import, include, xinclude etc. eventually (1.7), which might interact
> badly with this, requiring double escapes, while an explicit syntax
> would indicate that escapes need to be treated differently.

OK. But would a more general concept of typed value
quoting/escaping/parsing be better?

> I'd also use a version switch as well. I think things like this should
> be approached with some caution.

How would a version switch actually work here. I can only see it working
with linear development. What I have is a branch.

> I'd rather use globs than regular expressions, it isn't that hard to
> implement a glob matcher (I think I might have one I wrote somewhere)
> and I find it nicer for matching paths. Would also not require adding
> an extra dependency.

I would rather not use globs. For various reasons: they are limited and
likely people will want more control; writing a bunch of custom code for
something that already exists seems foolish to me; regex are very well
understood and documented whereas globs tend to be special to whomever
implements it.

> I haven't tested it, but I don't think your code will work on windows.
> For example, the line:
> std::string f = i->path().filename().native();
> should probably be something like:
> std::string f = detail::path_to_generic(i->path().filename());

That would be true if 'f' was a full path. But it's only the file name
part which can't have the characters that path_to_generic manipulates.
Additionally this code is never executed in a backward compatibility
context since it's only when one specifies a regex as the argument to
the import.

> I think it'd be better to return 'std::set<include_search_return>'
> than turn 'include_search_return' into a stack. It took me a little
> while to work out what you were doing. Actually, I'd probably go for a
> vector and just sort it. I admit that 'include_search_return' is a
> really bad name, I wouldn't be against renaming it.

I was going for the minimal amount of code mucking about. Hence the
double duty of the structure. If I was implementing this more generally
I'd try to provide a more appropriate abstraction to file searching and

> If there are matches at different points in the search path, they're
> ordered by absolute path which may vary on different computers. I
> think my preference would be to keep them in the order they appear in
> the search path, with 'filename_relative' as a secondary sort field in
> case of collisions. Or perhaps 'filename_relative' could be the
> primary sort field, and the include path order the secondary sort
> field.

Good point. Although the ordering within different search paths is
likely to never come up. As it's more likely that if one is concerned
about some specific ordering one would put the imports/inclusions in a
specific order.. As I did in my use case.

> If you expand the '!' snippets in a separate loop after creating the
> other snippets, I think it would allow to you to expand code snippets
> from later in the file which should be useful.

Hm... Very good idea. That would allow for some very convenient document
writing. As in allowing one to write a concise explanation of code at
the top with code included from the actual program. Much like the
example you point to below. But in a much improved organization as it
doesn't reorder the code itself and hence not make the code flow less

But your idea is contradictory to your previous point about using
"include" instead of "import". The above would be, according to your
definition of the semantics, a combined import and include. Which is
what I implemented. So perhaps we want to reconsider the semantics (and
names) of "include" and "import".

> I wouldn't call this literate programming, as I think it annoys
> literate programming enthusiasts. See:

I guess it depends on whether one considers literate programming a
general concept of having documentation and code interleaved vs. about
some particular tool that implements the ideas ;-)

-- Grafik - Don't Assume Anything
-- Redshift Software, Inc. -
-- rrivera/ (msn) - grafik/
-- 102708583/icq - grafikrobot/aim,yahoo,skype,efnet,gmail

This archive was generated by hypermail 2.1.7 : 2017-11-11 08:50:41 UTC