September 2004 - Boost - lists.boost.org

RE: [boost] [filesystem] large file support test added
by Ben Hutchings 10 Sep '04

10 Sep '04

Beman Dawes <bdawes(a)acm.org> wrote: <snip> > Normally we would have a regression test that identifies the systems > which fail. But we don't test large file support directly because to > do so would be a burden on those who run the regression tests; such > tests would chew up gigabytes of disk space and might also be very > slow. <snip> why so? Unix file-systems support sparse files, as does NTFS.

2 1

[serialization] - xml bug?
by Jeff Flinn 10 Sep '04

10 Sep '04

Robert, The following demonstrates a problem with serialization of strings conataining '<' and/or '>'. This is on VC7.1 using serialization20 with boost_1_13_0. I'm not in a position to check this on the CVS version at this time. Is this still an issue? Thanks, Jeff ///////////////////////////////////////// #include <boost/archive/xml_oarchive.hpp> #include <boost/archive/xml_iarchive.hpp> #include <string> #include <fstream> int main(int argc, char* argv[]) { std::string lStringOut("<empty>"); { std::ofstream lOut( "C:/xmltest.xml" , std::ios_base::out | std::ios_base::trunc ); if( lOut.good() ) { boost::archive::xml_oarchive oa( lOut ); oa << boost::serialization::make_nvp( "MyString", lStringOut ); } } std::string lStringIn; { std::ifstream lIn( "C:/xmltest.xml" ); if( lIn.good() ) { boost::archive::xml_iarchive ia( lIn ); ia >> boost::serialization::make_nvp( "MyString", lStringIn ); } } assert( lStringOut == lStringIn ); return 0; }

1 0

[filesystem] large file support test added
by Beman Dawes 10 Sep '04

10 Sep '04

Something special (defining some preprocessor macro) has to be done for 32-bit POSIX-like operating systems to enable support for file sizes greater than can be represented by 31 bits. The exact preprocessor macro name that has to be defined varies from system to system. Boost Filesystem currently supplies the correct macro for Linux, but not some of the UNIX systems. Normally we would have a regression test that identifies the systems which fail. But we don't test large file support directly because to do so would be a burden on those who run the regression tests; such tests would chew up gigabytes of disk space and might also be very slow. It recently occurred to me that sizeof(stat.st_size) > 4 is a pretty good surrogate for an actual test on a large file. While in theory it could yield a false positive, in practice it should identify whether or not large file support has been correctly enabled. It should never yield a false negative. Accordingly, I've added a new regression test, large_file_support_test.cpp. Initially it should pass on Windows and Linux systems, but fail on any UNIX systems that need additional macros defined. We can then experiment with adding additional macro definitions to get the test to pass on any systems it is currently failing. --Beman

1 0

dynamically created program options
by Rodolfo Schulz de Lima 10 Sep '04

10 Sep '04

Hi, I wonder if it's possible to handle dynamically created program options with boost::program_options. For instance: I have a config file that specifies configure options for dynamic loaded modules in my project. So the cfg file would be: [module.foo1] module_file = foo.so listen_port = 564 enabled = true [module.foo2] module_file = foo.so listen_port = 565 enabled = false [module.bar1] module_file = bar.so listen_port = 145 enabled = true As can be seen, foo1, foo2 and bar1 names are defined by the user in the configure file, and the program doesn't know about them before parsing the configuration. But if we parse that (somehow), an exception saying that the option "module.foo1.module_file" is invalid will be thrown. Can I manage the options_description "on the fly"? Another less complicated situation would be something like this: modules = foo1 foo2 bar1 [module.foo1] . . . [module.foo2] . . . [module.bar1] . . . So when parsing the "modules" option, we would know the new options name and it would be a matter of appending more option descriptions to the object. Regards, rod

2 1

ublas::vector from std::vector
by Neal D. Becker 10 Sep '04

10 Sep '04

I noticed there is no constructor for ublas::vector that takes a pair of iterators. Is there no direct way to construct a ublas::vector from a std::vector? Could there either be a constructor: (STL-style) template<typename input_iterator> vector (input_iterator begin, input_iterator end); or: (range-style) template<typename cont_t> vector (cont_t const&)

1 0

mingw build failed at regex
by Jim Pritts 10 Sep '04

10 Sep '04

Hi all, I am compiling boost 1.31.0 with mingw. Specifically the packages I am using are: boost 1.31.0 with regex patch binutils-2.13.90-20030111-1.tar gcc-core-3.4.1-20040711-1.tar gcc-g++-3.4.1-20040711-1.tar mingw-runtime-3.3.tar Boost builds find until bjam reaches the regex library. There seems to be a problem with wide character streams in the regex library. I don't think the latest mingw supports wide character streams. Is this correct? If so, how do I compile the regex library? If there is no workaround, is it possible to use mingw with STLPort and boost? The following is an excerpt form the actual error message I get: g++ -c -Wall -ftemplate-depth-100 -DBOOST_REGEX_CONFIG_INFO=1 -DBOOST_REG EX_DYN_LINK=1 -g -O0 -fno-inline -mno-cygwin -I"c:\build\bin\boost\libs\regex \build" -I "C:\boost_1_31_0" -o "c:\build\bin\boost\libs\regex\build\boost_reg ex.dll\mingw\debug\c_regex_traits.obj" "C:/boost_1_31_0/libs/regex/build/../src /c_regex_traits.cpp" ...failed gcc-C++-action c:\build\bin\boost\libs\regex\build\boost_regex.dll\min gw\debug\c_regex_traits.obj... gcc-C++-action c:\build\bin\boost\libs\regex\build\boost_regex.dll\mingw\debug\c _regex_traits_common.obj gcc-C++-action c:\build\bin\boost\libs\regex\build\boost_regex.dll\mingw\debug\c pp_regex_traits.obj C:/mingw/bin/../lib/gcc/mingw32/3.4.1/../../../../include/c++/3.4.1/streambuf: I n instantiation of `std::basic_streambuf<wchar_t, std::char_traits<wchar_t> >': C:/boost_1_31_0/libs/regex/build/../src/cpp_regex_traits.cpp:92: instantiated from `<unnamed>::parser_buf<wchar_t, std::char_traits<wchar_t> >' C:/boost_1_31_0/libs/regex/build/../src/cpp_regex_traits.cpp:590: instantiated from here C:/mingw/bin/../lib/gcc/mingw32/3.4.1/../../../../include/c++/3.4.1/streambuf:13 5: error: no type named `pos_type' in `struct std::char_traits<wchar_t>' C:/mingw/bin/../lib/gcc/mingw32/3.4.1/../../../../include/c++/3.4.1/streambuf:23 4: error: no type named `pos_type' in `struct std::char_traits<wchar_t>' C:/mingw/bin/../lib/gcc/mingw32/3.4.1/../../../../include/c++/3.4.1/streambuf:23 9: error: no type named `pos_type' in `struct std::char_traits<wchar_t>' C:/mingw/bin/../lib/gcc/mingw32/3.4.1/../../../../include/c++/3.4.1/streambuf:57 4: error: no type named `pos_type' in `struct std::char_traits<wchar_t>' C:/mingw/bin/../lib/gcc/mingw32/3.4.1/../../../../include/c++/3.4.1/streambuf:58 6: error: no type named `pos_type' in `struct std::char_traits<wchar_t>' C:/boost_1_31_0/libs/regex/build/../src/cpp_regex_traits.cpp: In instantiation o f `<unnamed>::parser_buf<wchar_t, std::char_traits<wchar_t> >': C:/boost_1_31_0/libs/regex/build/../src/cpp_regex_traits.cpp:590: instantiated from here ... Thanks, Jim.

2 1

Empty member optimization
by Joao Abecasis 10 Sep '04

10 Sep '04

Hi! I have written a small utility class to provide "empty member optimization" for arbitrary types. The implementation makes use of the empty base class optimization (EBCO) but avoids injection of the class. A uniform and contained interface is exposed wether the optimization applies or not: template <typename T, size_t Index> compressed_pair { // several constructors may be provided but, for now, // default and copy constructor typedef T value_type; value_type & get(); value_type const & get() const; }; I believe this could be used as the basis for a compressed_tuple. Here's a sample use: template <typename T0, typename T1> struct A : private compressed_member<T0> , private compressed_member<T1, 1> { typedef compressed_member<T0> member0; typedef compressed_member<T1, 1> member1; T0 const & first() { return base0::get(); } T1 const & second() { return base1::get(); } }; There is a catch with the current implementation: some regular empty member (or regular empty member of some compressed_member) could end up sharing address with an empty compressed_member. This happens because compressed_member<T> does not inherit from T but hides this "relationship" from the compiler. Is there any interest in such an utility? To test compressed member I wrote another (toy) utility: empty_store. empty_store is an empty class with a static map that associates an instance's address to data. Checks are made everywhere to ensure that no two instances share the same address. The code can be found (zipped) here: *http://tinyurl.com/4e85l* Best regards, João Abecasis

4 8

ublas indirect_array, shared_array, array_adaptor doc?
by Neal D. Becker 10 Sep '04

10 Sep '04

I noticed in ublas/storage.hpp a number of interesting-looking types, such as array_adaptor, shared_array, indirect_array. I did not find any documentation on these.

1 0

class random_device from <boost/nondet_random.hpp> has no implementation
by steven.robbins＠videotron.ca 10 Sep '04

10 Sep '04

Hi, Most of the boost libraries' implementation is either in header files, or in a compiled library. Class boost::random_device is an exception: the implementation is found in libs/random/random_device.cpp, but this file is not built into any library. This issue has been raised by a Debian user (http://bugs.debian.org/261719) and earlier (http://lists.boost.org/MailArchives/boost-users/msg03506.php) but never addressed. I wanted to confirm whether this is a bug or is by design. Thanks, -Steve

1 0

Re: [boost] IOStreams Formal review -- Guide for Reviewers
by Jonathan Graehl 09 Sep '04

09 Sep '04

First, let me apologize for not being able to review the actual code (yet). The interface and correctness/performance of implementation are all I really care about for now :) >There are some special cases where the copying is wasteful, though. For >instance: > >1. If filter1 is just a passive observer, simply counting the number of >occurences of '\n', or copying the data to a logging stream, then the end user >should really be writing directly to buf2. Filter1 could process the data using >the same buffer as filter2. > >2. Same as above, but filter1 also modifies the data in-place, making >character-by-character modifications. (E.g., a toupper filter). This can be >handled the same way. > >3. If 'resource' is a stream or stream buffer, it could be assumed to do its own >buffering. In that case, filter2 should be writing directly to resource, instead >of to streambuf3: > > filter2.write(resource, buf2, buf2+ n2). > >These three cases can be handled easily by modifying the existing framework. I >didn't add special treatment because it occurred to me rather late in >development. > > > I'd certainly feel a bit more proud of the library if it handled these cases (1 and 3 seem most important). It seems well worth a few days' delay. >There is another class of cases in which the current setup is wasteful, but I >think it is rather domain-specific: > >4. Most filters in the chain modify only small parts of character sequences, >leaving big chunks unchanged. > > > Well, basic_newline_filter would do this - at least when replacing CRLF with a single '\n' character. >To optimize case 4 would require a major library extension. My feeling is that >it is not necessary at this point, but I'd like to know what others think. > > > It should certainly wait if you don't have a design already in mind. The library is good enough without this. >----------------------------------------------------- > >Part III: Interface questions: > >1. How to handle read and write requests which return fewer characters than >requested, though there has been no error, and EOF has not been reached. I think >some answer to this question is necessary to allow the library to be extended >later to handle models orther than ordinary blocking i/o. I mention three >possibilties here, http://tinyurl.com/6r8p2, but only two are realistic. I'm >interested to know how important pople think this issues is, and what is the >best way to resolve it. > > > I think #2 would be most in line with what people are used to under Posix (-1/EAGAIN). Blocking (option 1) actually doesn't make any sense at all except at the ends (source/sink), unless you put each filter in a chain into its own thread and use something like semaphores. I suppose the idea is, if you're in the middle of a filter chain, and somebody gives you some input which would overflow your buffer, you attempt to empty out your buffer to the next guy, but if he can't take enough of it to allow you to accept the whole input (or even, none of it), you have to tell the guy who sent it to you you can't take it, and he has to hold onto it in his buffer. That seems reasonable. I might in fact prefer that my source or sink resource act like #1 (block until at least one character or EOF), and I assume this would be the default behavior if I open it in the default, blocking mode, but it isn't possible to have filters act that way; they need to pass the "can't take your data" feedback all the way back through the stack to the end user, who then needs to hold onto it and select/spin/whatever on the underlying sink resource ... >2. The stack interface. Is the interface to the underlying filter chains rich >enoguh? Originally is was similar to std::list, so that you could disconnect >chains at arbitrary points, store them, and reattach them later. I decided there >wasn't much use for this, so I simplified the interface. > > I'm sure someone, somewhere, sometime will want to perform splices/appends on filter chains, but I can't imagine why, either. At least, keep the simple interface and put the more complicated, flexible one in the appendix. >3. Exceptions. James Kanze has argued repeatedly that protected stream buffer >functions should not throw exceptions (http://tinyurl.com/5o34x) I try to make >the case for exceptions here: http://tinyurl.com/6r8p2. What do people think? > > > > I sympathize with both arguments; either way seems fine to me. There is no real performance penalty for an exception that is thrown at most once per stream (EOF), but he's right that the existing interface (which end users never see) seems to specify a return value in that case. But, as you say, if you want to support async IO, it's moot - you do have to return the number of characters successfully read/written, so you need the std::streamsize return type, so ... you may as well return EOF instead of throwing it. That is, I see no reason to throw an EOF value if you support async IO (which I think would be lovely). >So the question is: Should an open() function be added to the closable >interface, to eliminate the need for first-time switches? Alternatively, should >there be a separate Openable concept? > > > Without a doubt, an Openable concept. If you add open() to Closeable you'd really want to change the concept name ;) For example, if I implement a first-time flag (no real hardship), I'll have to remember to add the first-time test not only when data is processed, but also when the stream is closed (need to handle empty input properly). I suspect people will forget this at least once. There's also a minor performance gain: the interface would be called when the stream is initialized, I assume, and not require a first-time flag check with each use. Admittedly, inconsequential. -Jonathan Graehl

2 6