Boost logo

Boost :

From: Matthew Wilson (mwilson_at_[hidden])
Date: 2002-06-13 00:35:19


James

Have you considered the worth of making your tokeniser deal with separator
that are not one character in length, ie strings? I imagine this would
expand its appeal.

Also, your current implementation looks to be char type only. Presumably it
is in your list of to-dos to make its char type parameterisable?

Matthew

"James Curran" <JamesCurran_at_[hidden]> wrote in message
news:ae7bph$m0o$1_at_main.gmane.org...
> Out of personal need I've written a TokenizerFunction which might be
> worth sharing.
>
> It's designed to parse a collection of name/value pairs (with the key
> feature for me being that everything after the first delimiter is the
second
> field, regardless of delimiters it migth contain). It will always return
an
> even number of tokens. For example, given the input:
>
> Name:James
> File:C:\temp\test.txt
> Flag:
>
> then the code
>
> name_value_pair nv(':', '\n'); // == to nv()
> boost::tokenizer<name_value_pair > tok(input, nv);
> boost::tokenizer<name_value_pair > ::iterator iter = tok.begin();
> while (iter != tok.end())
> {
> cout << '[' << *iter << ']' << "/";
> ++iter;
> cout << '[' << *iter << ']' << endl;
> ++iter;
> }
>
> would print
>
> [Name] \ [James]
> [File] \ [C:\temp\test.txt]
> [Flag] \ []
>
> The code is at:
> http://www.noveltheory.com/boost/namevalueiter.h and
> http://www.noveltheory.com/boost/NVtest.cpp
> (I used the source for offset_separator as a template, so it should be
> grossly off boost standards, but so far I haven't tried to meet them
either)
>
>
> --
> Truth,
> James Curran
> www.NJTheater.com (Professional)
> www.NovelTheory.com (Personal)
>
>
>
>
>
>
>
> _______________________________________________
> Unsubscribe & other changes:
http://lists.boost.org/mailman/listinfo.cgi/boost
>


Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk