It's a poorly kept secret that I've been adding new features to Xpressive on CVS HEAD for a while now. So here are the features already implemented (documentation forthcoming) for Xpressive 2.0: << Semantic Actions >> Specify code to execute when parts of a regex match, a-la Spirit's semantic actions. Eg.: if you want to parse a string of name/value pairs into a std::map, you might: std::map<std::string, int> result; std::string str("aaa=>1 bbb=>23 ccc=>456"); // Like "(\\w+)=>(\\d+)": sregex pair = ( (s1= +_w) >> "=>" >> (s2= +_d) ) [ ref(result)[s1] = as<int>(s2) ]; sregex rx = pair >> *(+_s >> pair); if(regex_match(str, rx)) { assert(result["aaa"] == 1); assert(result["bbb"] == 23); assert(result["ccc"] == 456); } The actions are placed on a queue and executed in order only when the regex match succeeds. << Custom Assertions >> Use the check() function to create a boolean predicate that can participate in the match. Here's a regex that recognizes two integers only if the first is less than the second: sregex rx = ( (s1= +_d) >> ' ' >> (s2= +_d) ) [ check( as<int>(s1) < as<int>(s2) ) ]; Unlike actions, predicates execute immediately. You can also define the predicate out-of-line as a function object. << Dynamic Regex Grammars with Named Regexes >> Using regex_compiler, you can map a name to a regex object, and then refer to that regex from another by name. In this way, you can build grammars from regexes at runtime. sregex_compiler comp; sregex rx = comp.compile("^bar(?$RE)baz$"); comp.compile("(?$RE=)\\d+ \\d+"); There's an alternate syntax for associating a name with a regex that you can use to nest a static regex in a dynamic one. Eg., the last line above could be: comp["RE"] = +_d >> ' ' >> +_d; With these changes, you can now nest static and dynamic regexes within each other freely, giving you lots of flexibility to build grammars and modify them on the fly. << Named Captures >> For dynamic regular expressions, you can create a named capture with (?P<name> ...). You can refer back to the named capture with (?P=name). In substitution strings (for use with regex_replace()), you can refer back to a named capture with \\g<name> when using the format_perl or format_all flags. And more .... I'd like to give props to Dave Jenkins who has been doing m4d things with this stuff already, and who has given me a laundry list of other features he'd like. The new features are only available in CVS HEAD, for now. Feedback is welcome. -- Eric Niebler Boost Consulting www.boost-consulting.com
"Eric Niebler" <eric@boost-consulting.com> wrote in message news:46605932.7000600@boost-consulting.com...
It's a poorly kept secret that I've been adding new features to Xpressive on CVS HEAD for a while now. So here are the features already implemented (documentation forthcoming) for Xpressive 2.0:
Having used most of these new features, I'd like to emphasize what a huge jump in expressive power (pun intended) this gives to Xpressive. Now, you can define a complete static context free grammar at compile time and replace any part or parts of it at run time. You get both the speed of a static grammar and the flexibility of a run-time grammar. Also, the custom assertions let you embed semantic checks right into your regular expression. This is much better than waiting to check semantics after a regular expression has completed. Great job, Eric. Dave Jenkins
On 6/1/07, Eric Niebler <eric@boost-consulting.com> wrote:
It's a poorly kept secret that I've been adding new features to Xpressive on CVS HEAD for a while now. So here are the features already implemented (documentation forthcoming) for Xpressive 2.0:
<< Semantic Actions >>
Specify code to execute when parts of a regex match, a-la Spirit's semantic actions. Eg.: if you want to parse a string of name/value pairs into a std::map, you might:
std::map<std::string, int> result; std::string str("aaa=>1 bbb=>23 ccc=>456");
// Like "(\\w+)=>(\\d+)": sregex pair = ( (s1= +_w) >> "=>" >> (s2= +_d) ) [ ref(result)[s1] = as<int>(s2) ]; sregex rx = pair >> *(+_s >> pair);
if(regex_match(str, rx)) { assert(result["aaa"] == 1); assert(result["bbb"] == 23); assert(result["ccc"] == 456); }
The actions are placed on a queue and executed in order only when the regex match succeeds.
Neat! Great work Eric! I want to use this with my bimaps (I will add a section Bimap and Boost.Xpressive to my Bimap and Boost.? docs) but I do not know the correct approach. Because bimaps have other constraints than maps, I can not provide operator[] for insertion. I have to use the usual insert. This works as you said: (operator[] is provided for this bimap because list_of collection type does not impose additional constraints) ---------------------------------------------------------------------------- typedef bimap< std::string, list_of< int > > bm_type; bm_type bm; std::string str("aaa=>1 bbb=>23 ccc=>456"); sregex pair = ( (s1= +_w) >> "=>" >> (s2= +_d) ) [ xp::ref(bm.left)[ s1 ] = as<int>(s2) ]; ---------------------------------------------------------------------------- But when operator[] is not present I have tried to use insert but failed. Being able to parse a file is a nice addition to bimap docs. I know I have to wait to your docs but I want to know now :) Can you help me? Regards Matias
Matias Capeletto wrote:
I want to use this with my bimaps (I will add a section Bimap and Boost.Xpressive to my Bimap and Boost.? docs) but I do not know the correct approach. Because bimaps have other constraints than maps, I can not provide operator[] for insertion. I have to use the usual insert.
This works as you said: (operator[] is provided for this bimap because list_of collection type does not impose additional constraints)
---------------------------------------------------------------------------- typedef bimap< std::string, list_of< int > > bm_type; bm_type bm;
std::string str("aaa=>1 bbb=>23 ccc=>456"); sregex pair = ( (s1= +_w) >> "=>" >> (s2= +_d) ) [ xp::ref(bm.left)[ s1 ] = as<int>(s2) ]; ----------------------------------------------------------------------------
But when operator[] is not present I have tried to use insert but failed. Being able to parse a file is a nice addition to bimap docs. I know I have to wait to your docs but I want to know now :) Can you help me?
Sure. What is the signature of bimap::insert? In xpressive/regex_action.hpp, there is a lazy insert function object that can be used to insert stuff into sequences, strings and associative containers. For instance, my example could have been written as: sregex pair = ( (s1= +_w) >> "=>" >> (s2= +_d) ) [ insert(ref(result), make_pair(s1, as<int>(s2))) ]; which is also equivalent, via some Proto magic, to: sregex pair = ( (s1= +_w) >> "=>" >> (s2= +_d) ) [ ref(result)->*insert(make_pair(s1, as<int>(s2))) ]; But be sure to sync up. I just had to add xpressive::make_pair. :-) -- Eric Niebler Boost Consulting www.boost-consulting.com
On 6/1/07, Eric Niebler <eric@boost-consulting.com> wrote:
Matias Capeletto wrote:
But when operator[] is not present I have tried to use insert but failed. Being able to parse a file is a nice addition to bimap docs. I know I have to wait to your docs but I want to know now :) Can you help me?
Sure. What is the signature of bimap::insert? In xpressive/regex_action.hpp, there is a lazy insert function object that can be used to insert stuff into sequences, strings and associative containers. For instance, my example could have been written as:
sregex pair = ( (s1= +_w) >> "=>" >> (s2= +_d) ) [ insert(ref(result), make_pair(s1, as<int>(s2))) ];
Great!
But be sure to sync up. I just had to add xpressive::make_pair. :-)
I can not get cvs to work... I think that there is something wrong with sourceforg. I wanna try it... buuu...
which is also equivalent, via some Proto magic, to:
sregex pair = ( (s1= +_w) >> "=>" >> (s2= +_d) ) [ ref(result)->*insert(make_pair(s1, as<int>(s2))) ];
Budu stuff... :) The signature of bimaps insert is the same as standard map inserts. I have a question about the performance of this. Will make_pair create a std::pair? If this is true we are there is an extra copy of each element that I would want to avoid. Is there a way to xpressivize the following code: result.insert( map_type::value_type(s1,s2) ) ? Thanks Regards Matias
-- Eric Niebler Boost Consulting www.boost-consulting.com _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Matias Capeletto wrote:
which is also equivalent, via some Proto magic, to:
sregex pair = ( (s1= +_w) >> "=>" >> (s2= +_d) ) [ ref(result)->*insert(make_pair(s1, as<int>(s2))) ];
Budu stuff... :)
Budu? Wikipedia tells me that's a fish sauce. Hardly complimentary. ;-)
The signature of bimaps insert is the same as standard map inserts. I have a question about the performance of this. Will make_pair create a std::pair?
Yes.
If this is true we are there is an extra copy of each element that I would want to avoid.
Possibly not, if your compiler does RVO.
Is there a way to xpressivize the following code:
result.insert( map_type::value_type(s1,s2) )
Yes. sregex pair = ( (s1= +_w) >> "=>" >> (s2= +_d) ) [ ref(result)->*insert( construct<map_type::value_type>(s1, as<int>(s2))) ]; Pls sync again, if you can. :-P -- Eric Niebler Boost Consulting www.boost-consulting.com
On 6/1/07, Eric Niebler <eric@boost-consulting.com> wrote:
Matias Capeletto wrote:
which is also equivalent, via some Proto magic, to:
sregex pair = ( (s1= +_w) >> "=>" >> (s2= +_d) ) [ ref(result)->*insert(make_pair(s1, as<int>(s2))) ];
Budu stuff... :)
Budu? Wikipedia tells me that's a fish sauce. Hardly complimentary. ;-)
Voodoo stuff, black magic... :P
The signature of bimaps insert is the same as standard map inserts. I have a question about the performance of this. Will make_pair create a std::pair?
Yes.
If this is true we are there is an extra copy of each element that I would want to avoid.
Possibly not, if your compiler does RVO.
The copies are: std::pair --> map_type::value_type --> stored_type if you directly pass a map_type::value_type, then: map_type::value_type --> stored_type
Is there a way to xpressivize the following code:
result.insert( map_type::value_type(s1,s2) )
Yes.
sregex pair = ( (s1= +_w) >> "=>" >> (s2= +_d) ) [ ref(result)->*insert( construct<map_type::value_type>(s1, as<int>(s2))) ];
Pls sync again, if you can. :-P
I can't :( Anyway, that was what I was looking for. Thanks! Matias
Xpressive has just reached 2.0! The new version has lots of cool new features (see below). Subversion and the File Vault both have the latest code. It works with HEAD and with Boost v1.34.1. Below is an earlier message announcing the feature set of xpressive 2.0. The latest docs are here: http://boost-sandbox.sourceforge.net/libs/xpressive/doc/html/index.html Also, xpressive.zip in the File Vault (Strings - Text Processing directory) contains the docs in pdf format. Please grab the latest code and let me know of any problems you encounter so I can get them fixed for 1.35. -- Eric Niebler Boost Consulting www.boost-consulting.com Eric Niebler wrote:
It's a poorly kept secret that I've been adding new features to Xpressive on CVS HEAD for a while now. So here are the features already implemented (documentation forthcoming) for Xpressive 2.0:
<< Semantic Actions >>
Specify code to execute when parts of a regex match, a-la Spirit's semantic actions. Eg.: if you want to parse a string of name/value pairs into a std::map, you might:
std::map<std::string, int> result; std::string str("aaa=>1 bbb=>23 ccc=>456");
// Like "(\\w+)=>(\\d+)": sregex pair = ( (s1= +_w) >> "=>" >> (s2= +_d) ) [ ref(result)[s1] = as<int>(s2) ]; sregex rx = pair >> *(+_s >> pair);
if(regex_match(str, rx)) { assert(result["aaa"] == 1); assert(result["bbb"] == 23); assert(result["ccc"] == 456); }
The actions are placed on a queue and executed in order only when the regex match succeeds.
<< Custom Assertions >>
Use the check() function to create a boolean predicate that can participate in the match. Here's a regex that recognizes two integers only if the first is less than the second:
sregex rx = ( (s1= +_d) >> ' ' >> (s2= +_d) ) [ check( as<int>(s1) < as<int>(s2) ) ];
Unlike actions, predicates execute immediately. You can also define the predicate out-of-line as a function object.
<< Dynamic Regex Grammars with Named Regexes >>
Using regex_compiler, you can map a name to a regex object, and then refer to that regex from another by name. In this way, you can build grammars from regexes at runtime.
sregex_compiler comp; sregex rx = comp.compile("^bar(?$RE)baz$"); comp.compile("(?$RE=)\\d+ \\d+");
There's an alternate syntax for associating a name with a regex that you can use to nest a static regex in a dynamic one. Eg., the last line above could be:
comp["RE"] = +_d >> ' ' >> +_d;
With these changes, you can now nest static and dynamic regexes within each other freely, giving you lots of flexibility to build grammars and modify them on the fly.
<< Named Captures >>
For dynamic regular expressions, you can create a named capture with (?P<name> ...). You can refer back to the named capture with (?P=name). In substitution strings (for use with regex_replace()), you can refer back to a named capture with \\g<name> when using the format_perl or format_all flags.
And more .... I'd like to give props to Dave Jenkins who has been doing m4d things with this stuff already, and who has given me a laundry list of other features he'd like.
participants (3)
-
Dave Jenkins -
Eric Niebler -
Matias Capeletto