- Boost-users - lists.boost.org

wrapping stdlib fstream with boost filtering streams
by Daniel Oberhoff 06 Jul '07

06 Jul '07

Hi, I have a homebrewed matlab file library for which I am considering using the boost filtering streams for zlib compression. In Matlab(tm) files, variables may be compressed individually, meaning there will be compressed blocks inside the file surrounded by umcompressed data. Thus I would like to have (de)compressors which I can plug in at the start of the block and take back out when I am done. I went and applied the boost filtering streams but I have some problems, so before I go into hardcore debugging I would like to ask if I am using the lib right (because the docs are a bit sparse at the relevant spots). This is how I start up the stream(s): File::File(std::string path, std::ios_base::openmode mode) : _fstream(path.c_str(), mode | std::ios_base::binary) { if (!_fstream.is_open()) throw_error("could not open matfile " << path); _istream.set_auto_close(false); _ostream.set_auto_close(false); _istream.push(_fstream); _ostream.push(_fstream); } and these are the methods for starting/stopping the compressed blocks for reading/writing: void File::startunzipping(std::streampos len) { stopunzipping(); _unzend = _fstream.tellg() + len; _unzcount = 0; _istream.pop(); _istream.push(boost::iostreams::zlib_decompressor()); _istream.push(_fstream); _unzipping = true; } void File::stopunzipping() { if (_unzipping==false) return; _istream.pop(); _istream.pop(); _istream.push(_fstream); _unzipping = false; _fstream.seekg(_unzend); } void File::startzipping() { stopzipping(); write_type_and_size(TypeAndSize(Types::Compressed,0),false); _zstart = _fstream.tellp(); _zcount = 0; _ostream.pop(); _ostream.push(boost::iostreams::zlib_compressor()); _ostream.push(_fstream); _zipping = true; } void File::stopzipping() { if (_zipping==false) return; _ostream.flush(); _ostream.pop(); _ostream.pop(); _ostream.push(_fstream); _zipping = false; _zend = _fstream.tellp(); } Two open questions that come to mind: 1) does the compressor flush the zlib compression buffer when I flush/ pop it? 2) is fstream a bidirectional stream (my guess due to separate seekg/ seekp methods) or a seekable stream (one read/write head)? Because I currently require the former. btw: the above code runs without exceptions, but when I read back variables I have just written they are garbled.

2 1

[asio] File I/O
by William Kemper 05 Jul '07

05 Jul '07

I know I'm not the first person to bring this up and it seems that some work has been done: http://lists.boost.org/Archives/boost/2006/03/102350.php . Does anybody know if Felipe has posted his code somewhere? Or when we might an implementation of asynchronous file I/O in asio itself? Failing that - is there some class in the library (or examples) which would be a suitable starting point or reference/guide for creating a non-socket asynchronous I/O service? Thanks in advance.

2 1

[Boost-user]multithreaded app hangs in boost libraries
by Farberov, Inga 05 Jul '07

05 Jul '07

Hello, I have written a static library that uses boost program_options, threads, date-time and serialization libraries. There is also an extensive use of shared_ptr. I am working on a multithreaded application, so when I link my library into the executable, I use multithreaded versions of the boost libraries. When I compile my library and application that links to it, I use -pthread flag in attempt to make it multi-threaded. Everything compiles and links without a problem, however when I try to run the application, it hangs. I have tried to bedug it and when I suspend a hanging thread it gives me following stack trace: 18 _dl_sysinfo_int80() 0x003317a2 17 __lll_mutex_lock_wait() 0x0062e30e 16 _L_mutex_lock_35() 0x0062af3b 15 <symbol is not available> 0x080c80e8 14 <symbol is not available> 0x00472ff4 13 <symbol is not available> 0x080c80e8 12 <symbol is not available> 0xbffff6b0 11 <symbol is not available> 0xbffff208 10 pthread_mutex_lock() 0x0042097e 9 pthread_mutex_lock() 0x0042097e 8 scoped_lock() /usr/include/boost/detail/lwm_pthreads.hpp:72 0x080598eb 7 boost::detail::sp_counted_base::add_ref_copy() /usr/include/boost/detail/shared_count.hpp:122 0x0805b679 6 shared_count() /usr/lib/boost/include/boost-1_34/boost/detail/shared_count.hpp:216 0x0805b65b 5 shared_ptr() /usr/include/c++/3.4.6/bits/stl_construct.h:81 0x0807a7c8 4 boost::program_options::options_description_easy_init::operator() libs/program_options/src/options_description.cpp:178 0xb7fd7189 3 Settings() ../Settings.cpp:9 0x08081808 If this post sounds familiar, that's because this is my second attempt to get this problem resolved. Previously I have been told that this sounds like I've built my application with single-threading, and linked it to multi-threaded build of a boost library, or vice versa. Could you please let me know what's the right way to build my app with multi-threading. I am using Boost 1.34 libraries on Linux and compile with gcc 3.4.6 Thank you in advance for your help, Inga

2 3

Problem with unit test BOOST_AUTO_TEST_MAIN
by Sakharuk, Vlad (GMI Development) 05 Jul '07

05 Jul '07

Hi all, here is my code: #define BOOST_AUTO_TEST_MAIN #include <boost/test/auto_unit_test.hpp> BOOST_AUTO_TEST_CASE( test1 ) { BOOST_CHECK(true); } I link it dynamically with boost ver 1.3.3.1 with gcc-3.4.4 on Linux 2.6. (Note I did not compile this library myself.) However there is a lot of other libraries linked to the module but never used. Bellow is output I am getting. I suspect that other libraries have embedded boost tests. BUT 1) There is no test1 in my output. 2) How I can remove all unnecessary test and stick only to those in my files only? Here is command I have used test1 --output_format=XML --log_level=success Thanks Vlad. ** Demo doc read from disk: ** ** Printing via doc.Print ** <?xml version="1.0" standalone="no" ?>  <ToDo>  <Item priority="1" distance="close">Go to the <bold>Toy store!</bold> </Item> <Item priority="2" distance="none">Do bills</Item> <Item priority="2" distance="far & back">Look for Evil Dinosaurs!</Item> </ToDo> ** Printing via TiXmlPrinter ** <?xml version="1.0" standalone="no" ?>  <ToDo>  <Item priority="1" distance="close"> Go to the <bold>Toy store!</bold> </Item> <Item priority="2" distance="none">Do bills</Item> <Item priority="2" distance="far & back">Look for Evil Dinosaurs!</Item> </ToDo> ** Printing via operator<< ** <?xml version="1.0" standalone="no" ?><ToDo><Item priority="1" distance="close">Go to the<bold>Toy store!</bold></Item><Item priority="2" distance="none">Do bills</Item><Item priority="2" distance="far & back">Look for Evil Dinosaurs!</Item></ToDo> ** Demo doc processed: ** <?xml version="1.0" standalone="no" ?>  <ToDo>  <Item priority="2" distance="close">Go to the <bold>Toy store!</bold> </Item> <Item priority="1" distance="far">Talk to: <Meeting where="School"> <Attendee name="Marple" position="teacher" /> <Attendee name="Voel" position="counselor" /> </Meeting> <Meeting where="Lunch" /> </Item> <Item priority="2" distance="here">Do bills</Item> </ToDo> ** Demo doc processed to stream: ** <?xml version="1.0" standalone="no" ?><ToDo><Item priority="2" distance="close">Go to the<bold>Toy store!</bold></Item><Item priority="1" distance="far">Talk to:<Meeting where="School"><Attendee name="Marple" position="teacher" /><Attendee name="Voel" position="counselor" /></Meeting><Meeting where="Lunch" /></Item><Item priority="2" distance="here">Do bills</Item></ToDo> ** Basic structure. ** [pass] Output stream correct. [pass] Root element exists. [1][1] [pass] Root element value is 'ToDo'. [ToDo][ToDo] [pass] First child exists & is a comment. [1][1] [pass] Sibling element exists & is an element. [1][1] [pass] Value is 'Item'. [Item][Item] [pass] First child exists. [1][1] [pass] Value is 'Go to the'. [Go to the][Go to the] ** Iterators. ** [pass] Top level nodes, using First / Next. [3][3] [pass] Top level nodes, using Last / Previous. [3][3] [pass] Top level nodes, using IterateChildren. [3][3] [pass] Children of the 'ToDo' element, using First / Next. [3][3] [pass] 'Item' children of the 'ToDo' element, using First/Next. [3][3] [pass] 'Item' children of the 'ToDo' element, using Last/Previous. [3][3] ** Parsing. ** [pass] Element parsed, value is 'Element0'. [Element0][Element0] [pass] Reads attribute 'attribute0="foo0"'. [foo0][foo0] [pass] Reads incorrectly formatted 'attribute1=noquotes'. [noquotes][noquotes] [pass] Read attribute with entity value '>'. [>][>] [pass] Error row [3][3] [pass] Error column [17][17] ** Streaming. ** [pass] Stream round trip correct. [pass] String printing correct. [pass] Query attribute: int as double [0][0] [pass] Query attribute: int as double [1][1] [pass] Query attribute: double as double [2][2] [pass] Query attribute: double as int [0][0] [pass] Query attribute: double as int [2][2] [pass] Query attribute: not a number [2][2] [pass] Query attribute: does not exist [1][1] [pass] Location tracking: Tab 8: room row [1][1] [pass] Location tracking: Tab 8: room col [49][49] [pass] Location tracking: Tab 8: doors row [1][1] [pass] Location tracking: Tab 8: doors col [55][55] [pass] Location tracking: Declaration row [1][1] [pass] Location tracking: Declaration col [5][5] [pass] Location tracking: room row [1][1] [pass] Location tracking: room col [45][45] [pass] Location t etc ... -------------------------------------------------------- This message w/attachments (message) may be privileged, confidential or proprietary, and if you are not an intended recipient, please notify the sender, do not use or share it and delete it. Unless specifically indicated, this message is not an offer to sell or a solicitation of any investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Merrill Lynch. Subject to applicable law, Merrill Lynch may monitor, review and retain e-communications (EC) traveling through its networks/systems. The laws of the country of each sender/recipient may impact the handling of EC, and EC may be archived, supervised and produced in countries other than the country in which you are located. This message cannot be guaranteed to be secure or error-free. This message is subject to terms available at the following link: http://www.ml.com/e-communications_terms/. By messaging with Merrill Lynch you consent to the foregoing. --------------------------------------------------------

2 1

[Serialization] serializing handles to pointers, and serializing as garbage collection
by Sean Cavanaugh 05 Jul '07

05 Jul '07

I'm a bit new to boost and am trying to determine if I can coax the existing boost serialization system to work for a fairly standard asset loading system. Assets in my domain of game programming are things such as bitmaps, model geometry, audio data, shaders, and various chunks of meta data attached to them. The quantity of such data should always be assumed to vastly exceed the ability to load it all at once, which means getting sneaky when dealing with persistence. From my point of view the boost::serialization code is missing a lot of important features, so I am kind of surprised that they are not there. A) Serializing for persistence: Assets can reference other assets via pointers, but serializing for persistence should only store names of other assets, and query a resource manager to help convert the pointer into a name (or vice versa). A model object might reference via a pointer a shader and a texture in addition to its geometric data. When I serialize the model for persistence I want to store the geometric data, but when it comes to the shader and textures I only want to store their names. I could manually do this in each method's serialize methods. However the serialization system should also be able to do this via type traits, so that I don't have to hack up the serialize methods. The type trait method for this is preferable, since the method bodies of serialize could then be written as the originally would want to be, so that completely different kinds of archivers can operate on them, especially ones that depend on actually being able to walk into the pointers. Its the archive's call how to walk the data really. B) Serializing for garbage collection: A full set of raw assets for an engine can easily push into the terabyte range, which means you never actually work with all of the data at once. To work you need to be able to load what you need as you go, but to keep your working set down you need to evict unused assets as much as possible. Another type trait could be used to build a list of reachable objects containing the type trait from an array of root objects which are serialized. You can then compare this list to the full list of what the resource manager has currently loaded, and either delete or save the modified but now unreachable nodes. This system has the advantage of being useful for other purposes, such as making making an asset browser, without having to also implement a full blown system for reflection and introspection. C) Serializing for counting memory: You can also make the archive simply count the memory overhead of your objects. It needs the ability to count memory without counting the memory twice, and it also needs to be able to either do this for a single object (i.e. don't follow reference to other assets), or for a hierarchy (i.e. 'how much memory does using this asset require, factoring in its dependencies'). So my real question boils down to : I can't really be the first person to want it to be able to do these things right? Am I crazy for wanting to coax the boost serialization code into working this way?

2 1

Re: [Boost-users] [Boost-user]multithreaded app hangs in boost libraries
by Ovanes Markarian 05 Jul '07

05 Jul '07

May be you should not initialize it globally??? ;) What happens if you initialize it from main? With Kind Regards, Ovanes Markarian On Thu, July 5, 2007 15:23, Farberov, Inga wrote: > My application actually hangs in the initialization phase, before I even > get a chance to start any threads - in the main thread. The stacktrace > that I have attached resulted from a call to init program_options > object. > Inga > > -----Original Message----- > From: boost-users-bounces(a)lists.boost.org > [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Ovanes > Markarian > Sent: Thursday, July 05, 2007 9:16 AM > To: boost-users(a)lists.boost.org > Subject: Re: [Boost-users] [Boost-user]multithreaded app hangs in boost > libraries > > Can you please describe a bit more? How do you start your threads > (before your app enters main or > from the main)? Is there any chance, that you create threads and > afterwards leave main? > > > Try to build in some simple logging mechanism, which will produce logs > to cout from your threads > and main as well... Do not forget to lock access to cout. Try may be > create an instance of a > global object, which will output log message from its constructor. > > > With Kind Regards, > > Ovanes Markarian > > > On Thu, July 5, 2007 15:05, Farberov, Inga wrote: >> Hello, >> >> [...] >> Inga >> >>

1 0

Re: [Boost-users] [Boost-user]multithreaded app hangs in boost libraries
by Ovanes Markarian 05 Jul '07

05 Jul '07

Can you please describe a bit more? How do you start your threads (before your app enters main or from the main)? Is there any chance, that you create threads and afterwards leave main? Try to build in some simple logging mechanism, which will produce logs to cout from your threads and main as well... Do not forget to lock access to cout. Try may be create an instance of a global object, which will output log message from its constructor. With Kind Regards, Ovanes Markarian On Thu, July 5, 2007 15:05, Farberov, Inga wrote: > Hello, > > > > I have written a static library that uses boost program_options, > threads, date-time and serialization libraries. There is also an > extensive use of shared_ptr. > > I am working on a multithreaded application, so when I link my library > into the executable, I use multithreaded versions of the boost > libraries. When I compile my library and application that links to it, > I use -pthread flag in attempt to make it multi-threaded. > > > > Everything compiles and links without a problem, however when I try to > run the application, it hangs. I have tried to bedug it and when I > suspend a hanging thread it gives me following stack trace: > > > > 18 _dl_sysinfo_int80() 0x003317a2 > > 17 __lll_mutex_lock_wait() 0x0062e30e > > 16 _L_mutex_lock_35() 0x0062af3b > > 15 <symbol is not available> 0x080c80e8 > > 14 <symbol is not available> 0x00472ff4 > > 13 <symbol is not available> 0x080c80e8 > > 12 <symbol is not available> 0xbffff6b0 > > 11 <symbol is not available> 0xbffff208 > > 10 pthread_mutex_lock() 0x0042097e > > 9 pthread_mutex_lock() 0x0042097e > > 8 scoped_lock() /usr/include/boost/detail/lwm_pthreads.hpp:72 > 0x080598eb > > 7 boost::detail::sp_counted_base::add_ref_copy() > /usr/include/boost/detail/shared_count.hpp:122 0x0805b679 > > 6 shared_count() > /usr/lib/boost/include/boost-1_34/boost/detail/shared_count.hpp:216 > 0x0805b65b > > 5 shared_ptr() /usr/include/c++/3.4.6/bits/stl_construct.h:81 > 0x0807a7c8 > > 4 > boost::program_options::options_description_easy_init::operator() > libs/program_options/src/options_description.cpp:178 0xb7fd7189 > > 3 Settings() ../Settings.cpp:9 0x08081808 > > > > If this post sounds familiar, that's because this is my second attempt > to get this problem resolved. > > Previously I have been told that this sounds like I've built my > application with single-threading, and linked it to multi-threaded build > of a boost library, or vice versa. > > Could you please let me know what's the right way to build my app with > multi-threading. > > > > I am using Boost 1.34 libraries on Linux and compile with gcc 3.4.6 > > > > Thank you in advance for your help, > > Inga > > > > _______________________________________________ > Boost-users mailing list > Boost-users(a)lists.boost.org > http://lists.boost.org/mailman/listinfo.cgi/boost-users

2 1

[tuples] How to impose external allocator onto a class?
by Zeljko Vrba 05 Jul '07

05 Jul '07

I want to define custom new and delete operators for tuples. The usual way to do it would be, I presume, to derive from a base class and define these operators in the derived class. However, doing this for tuples would require me to provide all possible templated derivations for up to 10 arguments, including constructors. (And then maybe even defining new make_tuple functions). Is there a better way to accomplish this task?

1 0

[serialization] bad alloc and deserialization
by Cyril 05 Jul '07

05 Jul '07

Hello all, I have some trouble using boost::serialization : this code ============== try { std::stringstream erroneousStream; erroneousStream << "not a message ! @@@"; boost::archive::text_iarchive input(erroneousStream); Message* deserializedMessage; input >> deserializedMessage; // failure } catch (boost::archive::archive_exception& e){ std::cerr << e.what() << std::endl; } catch (std::exception& e) { std::cerr << e.what () << std::endl; } catch(...) { failmsg("unexpected !"); } =============== fails with a bad alloc exception. I don't think this is a normal behavior. So, what can I do to make this code fail correctly (with an archive_exception or even better with a serialization_exception) ? I must precise that in a normal scenario (with a real serialized message, this works perfectly well ! Any other idea ? Tanks in advance for you answers.

1 0

[Boost-docs] [Improving Boost Docs] Help Request, Request for Testers
by Stuart Dootson 05 Jul '07

05 Jul '07

As many of you will be aware, Matias Capeletto has been working at improving the Boost documentation in various ways, one of which is adding webpage functionality. Now, hopefully all these changes are cross-browser compatible, but to make sure, we need testers using as many platforms and browsers as we can get hold of. We currently have the combinations shown below covered, but could do with coverage a) for ones we don't already have, and b) to provide confirmation of existing test results. Firefox 2, Opera 9, Konqueror on Linux Firefox 2, IE 6, IE 7 on Windows XP Firefox 2, Camino 1.5, Safari 3 on Mac OS X If you are willing and able to help with testing (see http://svn.boost.org/trac/boost/wiki/BrowserTestingChart for details of what's involved), please let me know. I will then add you to the list of testers I have and will let you know when updated docs are available and re-testing is required. I'll then collate results (manually at first, but not for long - I'm lazy). So - please get in touch! Stuart Dootson

3 2