- Boost - lists.boost.org

template-defined regexp proposal
by Vit Stepanek 31 Jan '11

31 Jan '11

Hi, I've implemented the basic regexp functionality using few simple template classes. Any regexp can be created by inserting the template classes one into another in the required order. Although it's in the design state, I'd like to find out if there's any interest in providing this functionality. Let me present you some short introduction to the topic. Some basic classes include - String Match (full string matching) - Is-in Match (matches any character in the list) - Or Match - Quantity and so on. These are the "building blocks" of the regexp. Each class does a very little bit but when combined in the specific order, you gain a clearly defined string matching algorithm of any kind and any complexity. In addition, it's quite simple to add any missing functionality. You just need to define you're desired algorithm and mix it with the others. I add few examples that should describe how's it all done. Regexp: ^[a-z]+ is implemented as: Quantity< Range< char_a, char_z >, 1 > re; (sorry if you don't like the indentation, but that's just to ease the reading) Regexp: ^0(x|X)[a-fA-F0-9]+ (matching the hexadecimal number) The required strings are hardcoded using macro "STRHOLDER", which generates a simple struct containing the given string (to avoid the need of pushing strings in the runtime and thus splitting the matching logic into two levels). STRHOLDER( 0, "0" ); STRHOLDER( xX, "xX" ); MultiTie< StrMatch< StringHolder_0 >, IsIn< StringHolder_xX >, Quantity< OrMatch< DigitChar, OrMatch< Range< char_a, char_f >, Range< char_A, char_F > > >, 1 > > re; The "MultiTie" ties the match parts to the consecutive sequence. Some other examples I tested include - email address regexp - quoted string matching (containing escaped quoting chars) Currently the implementation counts 21 template classes, which are enough to implement (I hope) almost any regular expression. Each "brick" is called using operator ()(). That allows to pass it as an argument to the standard algorithms (like std::replace_if etc). I've already implemented the availability to push parameters to the regexp in the runtime (but that's not the major feature). Some rationale for the defense: String matching is widely applied part of the coding. Often it's also one of the problematic domains (especially when using self-implemented matching functions without any general solution). Once it's done, it can be hard to change it, because all what can be seen is HOW is the match done, but not WHAT does it do. Even when using some higher-level methods, it still isn't self-descriptive. Template-defined regexp provides one solution for this issue by hiding the HOW under the mask of WHAT, and taking the advantage of compile-time check. Only to mention that - the performance is not touched by any kind of interpreting - everything is prepared after compilation (but I don't want to touch anybody's baby here!). Please don't look at: - the proposed name "template-defined regexp", I use it as the temp. name - the naming of the classes - implementation details (yes, they aren't here, because it's not necessary for now) - term "regexp" - I'm not sure if this can be called so... Your opinions are welcomed! Vit Stepanek ------------

3 7

[date_time] Active Maintainer? Patches on Trac, Cleaned-up, and Tested.
by Dean Michael Berris 31 Jan '11

31 Jan '11

I don't know where to look for this kind of information -- anybody maintaining Boost.Date_Time now? There are a few patches that have been languishing which might be good to look at and/or apply to trunk: https://svn.boost.org/trac/boost/ticket/1574 https://svn.boost.org/trac/boost/ticket/1674 https://svn.boost.org/trac/boost/ticket/1726 I've verified these locally to not cause any regressions with GCC 4.5 on Ubuntu 10.10 64-bit Linux locally. /me doesn't have commit powers on trunk (yet); maybe someone can apply? TIA, HTH -- Dean Michael Berris about.me/deanberris

1 0

Immutable data structure survey results
by Gregory Crosswhite 31 Jan '11

31 Jan '11

Hey everyone, Here are the results of the survey that I ran for the name of a potential immutable byte sequence data structure; each line contains the name that was entered and the number of times that it appeared: chain: 9 boost::text: 3 sequence: 2 string: 2 atomic_buffer: 1 blob: 1 buffer: 1 bytes: 1 chord (from the Latin chorda meaning string): 1 const_buf: 1 const_string: 1 data_block: 1 fixed_sequence<char>: 1 frame: 1 immutable::string: 1 immutable_sequence: 1 imstring: 1 rod: 1 rope: 1 sequence<char>: 1 strand: 1 tier: 1 whole_buffer: 1 wire: 1 yarn: 1 TOTAL: 37 To be transparent, I have also attached the script that I used to generate the above, which includes the raw results pasted inside a multi-line string. (Yes Artyom I included your vote in the tally.) I apologize for any problems you might have had with this service. I considered using another free service online which would be better suited for this purpose, but I wasn't sure of how many responses there would be and most services put a cap on 100 responses for their free services, whereas the service I have access to through my university has no such restriction. Since the actual number of responses was less than half of 100 I will know for future reference that one of the other free services should work just fine for this purpose. Thanks to everyone who responded! Cheers, Greg

3 2

[1.46.0 beta 1] Release candidate files available
by Beman Dawes 31 Jan '11

31 Jan '11

Release candidate files for the 1.44.0 beta are available at http://boost.cowic.de/rc/ As always, the release managers would appreciate it if you download the candidate of your choice and give building it a try. Please report both success and failure. This helps ensure the candidates build OK before we push them out to SourceForge. Thanks, --The release managers

6 7

[string] Realistic API proposal
by Artyom 31 Jan '11

31 Jan '11

Hello All, I'd like to provide a realistic string API proposal: 1. It is Unicode aware extension of std::string 2. It is fully compatible with std::string giving an ability to be included in C++XYZ 3. It can be implemented today! Basically it extends std::string with a) const_code_point_iterator - for iterating over string - bidirectional b) code_point_iterator - back inserter c) provides Unicode and Locale functionality: - normalization - case handling - comparison - search d) marks non const operator[],at() as deprecated Advantages: ----------- 1. It can be implemented easily (when boost.locale is given) 2. It **is** compatible replacement of std::string 3. It allows to use std::string meanwhile under the hood as storage giving high efficiency when assigning boost::string to std::string when the implementation is COW (almost all implementations with exception of MSVC) 4. It is full unicode aware 5. It pushes "UTF-8" idea to standard C++ 6. You don't pay for what you do not need. Proposed API: ------------- namespace boost { // Fully bidirectional iterator template<typename UnitsIterator> class const_code_point_iterator { public: const_code_point_iterator(UnitsIterator begin,UnitsIterator end); // begin const_code_point_iterator(UnitsIterator begin,UnitsIterator end,UnitsIterator location); // current pos const_code_point_iterator(); // end #ifdef C++0x typedef char32_t const_code_point_type; #else typedef unsigned const_code_point_type; #endif const_code_point_type operator*() const; ... }; /// Output iterator template<typename BackInserter> class code_point_iterator { public: code_point_iterator(BackInserter out); // begin code_point_iterator(); // end #ifdef C++0x typedef char32_t code_point_type; #else typedef unsigned code_point_type; #endif code_point_type operator*() const; ... }; template<typename Char,typename Traits=std::char_traits<Char>, typename Alloc=std::allocator<Char> > class basic_string { public: // { boost specific typedef std::basic_string<Char,Traits,Alloc> std_string_type; // } boost specific // All std::string standard functions based // Deprecated interfaces that exist for backward compatibility // as they not Unicode aware value_type &at(size_type indx); value_type &operator[](size_type indx); iterator begin(); iterator end(); // { boost specific compatibility functions with std::string, they would go // as std::string becode extended with boost::string new interfaces // basic_string(std_string_type const &other) : data_(other) {} basic_string(std_string_type const &other,size_type index,size_type len) : data_(other,index,len) {} ... operator std_string_type() const { return data_; } // } boost specific compatibility functions // // Unicode Support // // ------------------------ // // // UTF Codepoint iteration // #ifdef C++0x typedef char32_t code_point_type; #else typedef unsigned code_point_type; #endif typedef boost::const_code_point_iterator<const_iterator> const_code_point_iterator; const_code_point_iterator code_point_begin() const { return const_code_point_iterator(begin(),end()); } const_code_point_iterator code_point_end() const { return const_code_point_iterator(begin(),end(),end()); } typedef boost::code_point_iterator<std::back_inserter<basic_string> > code_point_iterator; code_point_iterator back_inserter() { return code_point_iterator(std::back_inserter<basic_string>(*this)); } basic_string &operator+=(code_point_type code_point); basic_string operator+(code_point_type code_point) const; void append(code_point_type code_point); // // Lexical operations on string // // Case handling basic_string upper_case(std::locale const &l=std::locale()) const; basic_string lower_case(std::locale const &l=std::locale()) const; basic_string title_case(std::locale const &l=std::locale()) const; basic_string fold_case() const; // locale independent // Unicode normalization typedef enum { nfc, nfkc, nfd, nfkd } normalization_mode; basic_string normalize(normalization_mode mode = nfc) const; // normalized string constructor basic_string(basic_string const &,normalization_mode mode); basic_string(Char const *,normalization_mode mode); basic_string(Char const *,size_t n,normalization_mode mode); template<Iterator> basic_string(Iterator begin,Iterator end,normalization_mode mode); void append_normalized(basic_string const &other,normalization_mode mode = nfc); void append_normalized(Char const *,normalization_mode mode = nfc); void append_normalized(Char const *,size_t n,normalization_mode mode = nfc); basic_string concat_normalized(basic_string const &other,normalization_mode mode = nfc) const; basic_string concat_normalized(Char const *,normalization_mode mode = nfc) const; basic_string concat_normalized(Char const *,size_t n,normalization_mode mode = nfc) const; // Unicode validation bool valid_utf() const; typedef struct validate_utf{}; // Unicode validation tag // Create string validating it basic_string(basic_string const &,validate_utf const &); basic_string(Char const *,validate_utf const &); basic_string(Char const *,size_t n,validate_utf const &); template<Iterator> basic_string(Iterator begin,Iterator end,validate_utf const &); // Create string validating and normalazing it basic_string(basic_string const &,validate_utf const &,normalization_mode mode); basic_string(Char const *,validate_utf const &,normalization_mode mode); basic_string(Char const *,size_t n,validate_utf const &,normalization_mode mode); template<Iterator> basic_string(Iterator begin,Iterator end,validate_utf const &,normalization_mode mode); // Search and comparison typedef enum { primary = 0, ///< 1st collation level: base letters secondary = 1, ///< 2nd collation level: letters and accents tertiary = 2, ///< 3rd collation level: letters, accents and case quaternary = 3, ///< 4th collation level: letters, accents, case and punctuation identical = 4 ///< identical collation level: include code-point comparison } level_type; // // search(...) return pair of index and size as string you search for may have different size // std::pair<size_type,size_type> search(basic_string const &other) const; std::pair<size_type,size_type> search(basic_string const &other,level_type level) const; std::pair<size_type,size_type> search(basic_string const &other,std::locale const &l) const ; std::pair<size_type,size_type> search(basic_string const &other,level_type level,std::locale const &l) const; std::pair<size_type,size_type> search(basic_string const &other,size_t index,size_t size) const; std::pair<size_type,size_type> search(basic_string const &other,size_t index,size_t size,level_type level) const; std::pair<size_type,size_type> search(basic_string const &other,size_t index,size_t size,std::locale const &l) const; std::pair<size_type,size_type> search(basic_string const &other,size_t index,size_t size,level_type level,std::locale const &l) const; std::pair<size_type,size_type> search(Char const *) const; std::pair<size_type,size_type> search(Char const *,level_type level) const; std::pair<size_type,size_type> search(Char const *,std::locale const &l) const ; std::pair<size_type,size_type> search(Char const *,level_type level,std::locale const &l) const; std::pair<size_type,size_type> search(Char const *,size_t size) const; std::pair<size_type,size_type> search(Char const *,size_t size,level_type level) const; std::pair<size_type,size_type> search(Char const *,size_t size,std::locale const &l) const; std::pair<size_type,size_type> search(Char const *,size_t size,level_type level,std::locale const &l) const; int compare_to(basic_string const &other) const; int compare_to(basic_string const &other,level_type level) const; int compare_to(basic_string const &other,std::locale const &l) const ; int compare_to(basic_string const &other,level_type level,std::locale const &l) const; int compare_to(basic_string const &other,size_t index,size_t size) const; int compare_to(basic_string const &other,size_t index,size_t size,level_type level) const; int compare_to(basic_string const &other,size_t index,size_t size,std::locale const &l) const; int compare_to(basic_string const &other,size_t index,size_t size,level_type level,std::locale const &l) const; int compare_to(size_t index,size_t size,basic_string const &other) const; int compare_to(size_t index,size_t size,basic_string const &other,level_type level) const; int compare_to(size_t index,size_t size,basic_string const &other,std::locale const &l) const ; int compare_to(size_t index,size_t size,basic_string const &other,level_type level,std::locale const &l) const; int compare_to(size_t index,size_t size,basic_string const &other,size_t index,size_t size) const; int compare_to(size_t index,size_t size,basic_string const &other,size_t index,size_t size,level_type level) const; int compare_to(size_t index,size_t size,basic_string const &other,size_t index,size_t size,std::locale const &l) const; int compare_to(size_t index,size_t size,basic_string const &other,size_t index,size_t size,level_type level,std::locale const &l) const; int compare_to(Char const *) const; int compare_to(Char const *,level_type level) const; int compare_to(Char const *,std::locale const &l) const ; int compare_to(Char const *,level_type level,std::locale const &l) const; int compare_to(Char const *,size_t size) const; int compare_to(Char const *,size_t size,level_type level) const; int compare_to(Char const *,size_t size,std::locale const &l) const; int compare_to(Char const *,size_t size,level_type level,std::locale const &l) const; int compare_to(size_t size,Char const *) const; int compare_to(size_t size,Char const *,level_type level) const; int compare_to(size_t size,Char const *,std::locale const &l) const ; int compare_to(size_t size,Char const *,level_type level,std::locale const &l) const; int compare_to(size_t size,Char const *,size_t size) const; int compare_to(size_t size,Char const *,size_t size,level_type level) const; int compare_to(size_t size,Char const *,size_t size,std::locale const &l) const; int compare_to(size_t size,Char const *,size_t size,level_type level,std::locale const &l) const; // UTF validation bool is_valid_utf() const; private: std_string_type data_; }; } // boost

4 8

Re: [boost] [1.46.0 beta 1] Release candidate files available
by Cromwell Enage 30 Jan '11

30 Jan '11

When running "bjam --toolset=gcc stage" on the command line, I get the following message: tools/build/v2/build/configure.jam:145: in builds-raw *** argument error * rule UPDATE_NOW ( targets * : log ? : ignore-minus-n ? ) * called with: ( <pbin.v2/libs/regex/build/gcc-4.2.1/debug>has_icu : : ignore-minus-n : ignore-minus-q ) * extra argument ignore-minus-q (builtin):see definition of rule 'UPDATE_NOW' being called It's the same error that's been preventing me from building auto-index. Cromwell D. Enage

3 4

Boost-1.46.0-beta1 fails to build in tools/regression/
by Philipp Reh 30 Jan '11

30 Jan '11

tools/regression expects boost::filesystem to be version 2 but version 3 is now the default. The attached patch fixed the build for me.

1 0

Re: [boost] RE process (prospective from a retired FreeBSD committer)...
by Brian Wood 30 Jan '11

30 Jan '11

Dave Abrahams wrote: >At Sun, 30 Jan 2011 16:43:17 +0300, > Vladimir Prus wrote: > > >> whereas most of the core (which is half-zillion times bigger than >> Boost), lives in two modules, which contain wildly different >> functionality, are updated by multiple people, and then built and >> used and tested by users all the time. So, this proves that Boost >> has a lot of grows ahead before having all code in place place will >> becomes, even in theory, a problem. > > The problem isn't where the code is located, it's: > > a. how much coordination is required to get things done > > b. the mental drag of having to deal with all the parts of Boost that > simply aren't relevant to you. > I've recommended to "chew the meat and spit the bones," when it comes to Boost since at least 2007. This approach sounds like it will make it easier to produce a tasty Boost according to one's understanding. So I'm in favor of it, though I rather expect achieving it will be difficult. This *less taxing* approach (in terms of hard drives, download times, ...) will be a breath of fresh air. -- Brian Wood Ebenezer Enterprises http://webEbenezer.net (651) 251-9384

1 0

Stack-based vector container
by Gregory Crosswhite 30 Jan '11

30 Jan '11

Hey everyone, In the course of my work as a computational scientist I have written a container that emulates std::vector, but which is backed by an array whose size is fixed at compile time. Concretely, the class looks something like the following: template<typename T,unsigned int buffer_size> class static_vector { ... protected: T data[buffer_size]; T* end_ptr; size_t current_size; } where in the [...] there are methods that emulate (at this time, partially) the interface of std::vector, save that the size of the static_vector is not allowed to grow beyond the buffer_size. The advantage of using a static_vector over vector is performance. In my code I have performed benchmarks that show up to a 30% performance boost of static_vector over vector; the benefit occurs most stongly when working with lots of arrays of small objects, so that the use of dynamic memory rather than stack memory appears to add significant overhead to the computation. (Note that I when running these benchmarks I would use std::vector::reserve() to pre-allocate the array so that std::vector would not perform multiple allocations as the container grew, so that should not have been a source of overhead.) It seems to me that it would be useful to have this functionality generally available as a library, and in particular it might be a good fit for the boost family of libraries. Is there any interest out there for having such a library in boost? If so, I would be happy to do the work of taking what I have and fleshing it out so that it can function as a drop-in replacement for vector, which means modeling the Random Access Container and Back Insertion Sequence concepts as well as adding capacity() and reserve(size_t) (no-op) methods. Thoughts? Cheers, Gregory Crosswhite

17 53

GIL io_new review
by Phil Endecott 30 Jan '11

30 Jan '11

Back in 2006 I had some code that used ImageMagick's Magick++ library to read and write images in various formats and do some basic processing. That library works well and is very feature-full, but is not what you would call "Boost like", so I was excited when GIL came along. I looked at it at the time of its review, and I thought at first that I must have missed a big chunk of the documentation where all the algorithms were described! Of course, it has no image processing algorithms; it's just a layer; if you want to scale or rotate or merge images, you have to do that yourself. I don't think I wrote a review in the end; my feeling was - in comparison with ImageMagick - "come back when you've finished" - but I didn't want to say that in public. So it's great that now, 4 years later!, we are finally getting some GIL extensions. Thank you Christian for - I hope! - getting the ball rolling. I hope we'll see more. Having said that, I suspect that even with lots of extensions GIL might not really be what I need. Here's a typical problem that I will have to address in the next few weeks: I have a 1e12 pixel image - a map - which is supplied as a few thousand 5000x5000 TIFF tiles; I have to join them together, tweak the colours, draw some lines on top, and re-chop it into a few million 256x256 PNG tiles. The issue of course is that that image can't be held in RAM. I will do this with my own wrappers around libtiff and libpng, in rows or chunks. So, do the image concepts (and concrete classes) that GIL defines accommodate that sort of problem? Even if they do, is a "virtual image" of some sort a good way of doing it? I suspect not. But even if the image concept is not useful, presumably other GIL concepts like pixel formats and algorithms in future extensions (pixel format conversion, blending) could still be useful. So, along comes Christian's IO extension. Wrapping the "legacy" image libraries is something that I have done a few times now and it can be painful, not least because of the need to read a lot of documentation; the libraries all differ and it would be great to get a unified set of wrappers that provide a sane C++ interface. But as you can see, I don't really want something that reads and writes complete GIL images, and that is what Christian's extension does. Ideally, this extension would have separated out the wrapping of the libraries from the actual GIL image integration, with the GIL image integration just being a layer on top of the image library wrappers. Since it doesn't do that, it's not much use to me. It is, however, better than what GIL currently has. So for that reason I suggest that it should be accepted. > - What is your evaluation of the design? In addition to my comments above, I note that it's necessary to jump through hoops (using Boost.IOStreams) just to read an image from memory. Surely this should be a fundamental operation; it would be much better to have a ctor that takes begin and end pointers for the encoded image data. > - What is your evaluation of the implementation? When I first looked at the library I noticed that the error handling was clearly broken. Although that seems to have been resolved, it has given me a poor impression of the quality (i.e. likely correctness) of the implementation. It seems that this error handling has received almost no testing. Thorough testing will be difficult for this library because as well as all the usual permutations of platform features, compilers etc. the variations in the image libraries must also be considered. The library version and compiler settings could both affect the correct operation of this code. I have some concerns about the likely performance of the code due to excessive copying of the data both in and out of the image library. Although I've not measured the performance impact, I feel it would be very desirable (and is certainly possible) for this library to have zero copying. > - What is your evaluation of the documentation? Satisfactory. (As a meta-comment - I think libraries being reviewed really should have their docs available online somewhere, and preferably also their source code in a source browser of some sort. In fact, I thought this was already a requirement. Can it be added to the review managers' checklist for future reviews?) > - What is your evaluation of the potential usefulness of the extensions? It's better than what was there before. > - Did you try to use the extensions? With what compiler? Did you have > any problems? No, I've not compiled anything. > - How much effort did you put into your evaluation? A glance? A quick > reading? In-depth study? Probably half a day during the review, and a similar amount of time a few months ago. > - Are you knowledgeable about the problem domain? I've written a few wrappers for libjpeg, libpng and libtiff. I've never used GIL though I've read extensively about it. > And finally, every review should answer this question: > - Do you think the extensions should be accepted as a part of Boost.GIL > library? On the grounds that this is better than what is currently there, and on the assumption that Domagoj Saric is not imminently going to post something else that would supersede this, I believe that it should be accepted. (I have not looked at the "toolbox" at all, and I've not seen it mentioned in any other reviews yet. My "yes" is just for the io_new functionality.) Regards, Phil.

7 39