April 2010 - Boost - lists.boost.org

Re: [boost] Crypto Proposal
by Chad Seibert 14 Apr '10

14 Apr '10

Straight from the maintainer. Hi Chad, I don't think I would be comfortable with this; past conversations on the list have suggested a sizeable population of the users is avoiding boost for one reason or another and there was some pushback in terms of even using boost in the library in a way that would require it for basic functionality. This change would leave these users high and dry. I'll admit I perhaps didn't read your proposal as carefully as I should have; I had been under the impression that you would be creating more or less a boost.crypto that provided crypto functionality that happened to use botan internally but did not expose much or any of botan's API directly. Sorry for the miscommunication/mess. :( -Jack I don't know what this means in terms of the project - whether we'll go with a different project or just borrow bits from Botan. I am truly sorry I didn't look into this earlier; it was entirely my fault and I apologize for not looking into this issue earlier. Chad Seibert _________________________________________________________________ Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox. http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:…

3 2

Re: [boost] Crypto Proposal
by Chad Seibert 14 Apr '10

14 Apr '10

Thanks for all the feedback! > On 11/04/2010 7:25 PM, Rob Riggs wrote: > On 04/11/2010 10:53 AM, Chad Seibert wrote: >> Here's an example of boostified code, from one of the examples on the >> site. It's not too different from the original code (it's just >> commented out); this is because I don't want it to be a chore to port >> from Botan to Boost.Botan. Also, it is intuitively designed and I see >> no reason to change it. Of course, if it needs redesigning, we can >> discuss that afterwards > ... >> //std::auto_ptr<PKCS8_PrivateKey> >> privkey(PKCS8::load_key(arg_ca_key, rng, arg_passphrase)); >> std::auto_ptr<boost::botan::pkcs8::private_key> >> privkey(pkcs8::load_key(arg_ca_key, rng, arg_passphrase)); > Hi Chad, > > Would it make sense to have privkey() return the private key by value? > Is it non-copyable? > I think the reason is because a PKCS#8 key can be RSA/DSA or whatever else. > I'd like to be able to hold keys by value but I'm not sure whether that > is useful. I can't really see a need to copy keys around, but then again > who knows? > A Boostified Botan should consider this problem as Botan requires the > use of lots of yucky pointers in places because of the polymorphism. It's returned by pointer, since private keys are memory locked by SecureVector (see http://files.randombit.net/botan/doxygen/html/namespaceBotan_1_1PKCS8.html) to prevent flushing to disk (and other potential bugs noted here: http://files.randombit.net/botan/doxygen/html/secmem_8h-source.html) A bigger issue I see is AlgorithmIdentifier/OID mess I see in http://files.randombit.net/botan/doxygen/html/namespaceBotan.html#378aea09f…. Definitely needs to get replaced with something else. There is some rational reasoning with this, though. For example, when we load a pksc#8 key, we get an algo id, which leads to a huge if/else if statement (http://files.randombit.net/botan/doxygen/html/namespaceBotan.html#378aea09f…) It's also dependent on which libraries get build (the BOTAN_HAS_* defines). In any case, it's still far from perfect and needs to be thoughtout. I'll think about it, and get back to you. > On 12/04/2010 7:52 AM, Domagoj Saric wrote: > IMNHO that's a failed test by any standard (worse than OpenSSL and Crypto++)... > (must I really pay for e.g. virtual inheritance, dynamic_casts, > by-std::string-runtime-algorithm-lookups etc. etc... for such a simple use > case?)... > 1) If you write such simple applications, you are very lucky > 2) I think one "issue" on Botan for Windows is the entropy gatherer. > It's slow and takes a lot of memory if.. memory serves. For applications > where entropy is not THAT important, I replace it. For applications > where it IS important, I would take out life insurance. That's because you built Botan in whole; it has the ability to build only parts of itself. If we don't like the inheritance structure, I can certainly remove it when porting. For example, despite public keys being very similar in structure, they should NOT inherit from a common base simply because they are entirely different objects. Also, I certainly agree that the runtime lookup of algos is REALLY BAD and doesn't take advantage of the IH, as noted above. I agree it needs to be replaced. As for the RNG's they are easily swapped, so maybe a less expensive one should be created in the process? >> From this I gather that the original author plans to continue to develop and > maintain the original library, in which case I wonder what would be the purpose > of having a library that is nothing more but a 'parallel' 'boostified' version > of the original that in itself offers nothing new and is updated only after the > original one is updated? > I also think it is useless if Jack does not intend to treat the Boost > version as *the* version. > However, in the transition period, I think it is only sensible that the > student manually merges changes. Will take some planning... I agree that this version should become *the* version, but I was completely absent-minded and didn't bring that up when discussing this with him. I'll ask him ASAP. Hopefully this alleviates some issues, Chad Seibert _________________________________________________________________ Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox. http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:…

3 2

FW: deadline_timer and a crash at ~io_service -> timer_base::destroy()
by Vance Grkov 14 Apr '10

14 Apr '10

Hi, I am getting a crash at the scope end (before the "catch") below in the code. This is the stack trace and the source. Does anyone know what is the problem with this code? Appreciate your help! 000801c7() > BoostThreadPoolTest.exe!boost::asio::detail::timer_queue<boost::asio::time_traits<boost::posix_time::ptime> >::timer_base::destroy() Line 211 + 0xe bytes C++ BoostThreadPoolTest.exe!boost::asio::detail::timer_queue<boost::asio::time_traits<boost::posix_time::ptime> >::destroy_timer_list(boost::asio::detail::timer_queue<boost::asio::time_traits<boost::posix_time::ptime> >::timer_base * & t=0x00338da0) Line 415 C++ BoostThreadPoolTest.exe!boost::asio::detail::timer_queue<boost::asio::time_traits<boost::posix_time::ptime> >::destroy_timers() Line 189 C++ BoostThreadPoolTest.exe!boost::asio::detail::win_iocp_io_service::shutdown_service() Line 148 + 0x2c bytes C++ BoostThreadPoolTest.exe!boost::asio::detail::service_registry::~service_registry() Line 75 + 0xf bytes C++ BoostThreadPoolTest.exe!boost::asio::detail::service_registry::`scalar deleting destructor'() + 0x2b bytes C++ BoostThreadPoolTest.exe!boost::asio::io_service::~io_service() Line 52 + 0x2e bytes C++ BoostThreadPoolTest.exe!wmain(int argc=1, wchar_t * * argv=0x003371d8) Line 152 + 0x27 bytes C++ BoostThreadPoolTest.exe!__tmainCRTStartup() Line 583 + 0x19 bytes C BoostThreadPoolTest.exe!wmainCRTStartup() Line 403 C kernel32.dll!_BaseProcessStart@4() + 0x23 bytes #include "stdafx.h" #include <iostream> #include <boost/thread.hpp> #include <boost/bind.hpp> #include <boost/shared_ptr.hpp> #include <boost/asio.hpp> #define THREAD_POOL_SIZE 10 void TimerCallback(boost::system::error_code ec, std::size_t i, std::size_t retry) { std::stringstream tmp; tmp << "Timer(" << ec.value() << ":" << ec.message() << "-" << retry << "-" << i << ")\n"; } // // MAIN // int _tmain(int argc, _TCHAR* argv[]) { try { boost::asio::io_service io(THREAD_POOL_SIZE); boost::asio::io_service::work *moreWorkToCome = new boost::asio::io_service::work(io); // Create a pool of worker threads. std::vector<boost::shared_ptr<boost::thread> > threads; for (std::size_t i = 0; i < THREAD_POOL_SIZE; ++i) { boost::shared_ptr<boost::thread> thread(new boost::thread( boost::bind(&boost::asio::io_service::run, &io) )); threads.push_back(thread); } // Lets enqueue more tasks to run after a given time std::vector<boost::asio::deadline_timer *> timers; for (std::size_t i = 1; i<50; i++) { boost::asio::deadline_timer *timer = new boost::asio::deadline_timer(io); timer->expires_from_now(boost::posix_time::seconds(3)); timer->async_wait(boost::bind(&TimerCallback, boost::asio::placeholders::error, i, 0)); timers.push_back(timer); } std::cout << "\nALL TASKS QUEUED\n"; // Now that all tasks have been queued we don't want to hold on any of the worker threads delete moreWorkToCome; std::cout << "\nRELEASE THE THREAD POOL\n"; // Wait for all tasks to be processes which also means all threads in the pool will exit. for (std::size_t i = 0; i < threads.size(); ++i) threads[i]->join(); // Cancel all timers (just in case) for (std::size_t i = 0; i < timers.size(); ++i) timers[i]->cancel(); io.stop(); std::cout << "\nPOOL DOWN\n"; } <-------------------------------------------- CRASH catch(...) { std::cout << "ERROR"; } std::cout << "\nEXIT\n"; return 0; } Hotmail: Trusted email with powerful SPAM protection. Sign up now. _________________________________________________________________ Hotmail: Free, trusted and rich email service. https://signup.live.com/signup.aspx?id=60969

2 1

[range] iterator_range::operator[] broken
by Eric Niebler 14 Apr '10

14 Apr '10

Neil, I know you patched this a mere 5 hours ago, but it's still broken. The attached patch *really* fixes the problem, I think. -- Eric Niebler BoostPro Computing http://www.boostpro.com

2 4

[GSoC] Boost.Matrix RFC
by Kornel Kisielewicz 14 Apr '10

14 Apr '10

Hello all, Last year I submitted a proposal for Boost.Matrix as a GSoC project, that provoked quite a bit discussion. Considering the fact that it wasn't unanimously rejected, and that a few things have changed since then (Boost.Geometry acceptance) as did my skills, I'd like to take the liberty of resubmitting my proposal to the list, with a request for comments. The proposal follows: == Boost.Matrix == uBLAS is Boost's general linear algebra library, and it is a fine library indeed. However, in some cases it's design decisions, while quite rational for the general use case, are restrictive. This proposal is about a new library that would address some of these issues in the specific situation of vector and matrix operations on fixed sized structures. Suggested is a new library boost::matrix. The library would introduce a single container class that allows representation of fixed size vectors and matrices, and their respective standard operations. The library would define the following template template<typename T, std::size_t Rows, std::size_t Cols, class Traits> matrix and standard operations on it. The class would make heavy use of template specialization to allow efficient vector operations (matrices with size 1xN ), and provide efficient operations for the common sizes (2-4). The class will provide the interface of STL containers with the exact concept depending on the Traits used. Care will be taken to allow transparent usage within STL and other boost libraries. In general, this library would be similary scoped as Boost.Quaternion. Recently a new library has been accepted into boost -- Geometry. As the library is deeply about matrices and vectors, and has much in common with this proposal, I'd like to take a deep focus on providing maximum interopeability between Geometry and Matrix. == Rationale == Boost.Matrix aims to be for uBLAS, what Boost.Array is for vectors - a fixed size version to remove the overhead. The need is however greater, because while uBLAS provides a template for a fixed-sized version of it's arrays, the template is just a cover for the original uBLAS matrices. While the performance is comparable, the size includes a unnecessary field size. In cases that make heavy use of fixed sized matrices and vectors ( like real-time computer graphics ), such a tradeoff is unacceptable - due to increased memory consumption as well as effective vectorized math operations, and alignment of structures passed ( a single 4D vector should effectively fit a 128 bit register, and a single 128 bit move operation ). Also, knowing the dimensions of a matrix/vector at compile time, allows for a more efficient operation implementation, both from the programmers side ( better algorithms ), as well as from the compilers side (optimization). Finally, the uBLAS introduces a lot of unneccessary bloat for users wanting only a 2,3,4 vector and 3,4 matrix classes. == Primary deliverables == Boost.Matrix library, with the following features: * a boost::matrix base container class for 2-dimensional matrices, with the proper behavious and semantics of a container, written following the boost coding guidelines, and with full portability in mind * specializations for efficient vector (1xN matrix) handling * operators for readable but efficient vector/matrix math * operator specializations for matrices and vectors of the most common size (2-4) for most efficient implementation * efficient implementation of interoperability of Boost.Matrix with other boost libraries (uBLAS, Quaternion and upcoming Boost.Geometry in particular) == Secondary deliverables == * performance test suite, for testing optimizations * unit tests for the whole library providing full operation coverage and code-coverage * documentation for the whole library in the BoostBook format, as well as doxygen markup for all the code == Bonus deliverables == ( if time allows these will be delivered during program timeline, if not, they will be implemented afterwards ) * performance tests on different compilers/architectures, further optimizations * support for further boost libraries ( Boost.Assign, sandbox libraries, etc. ) == Design issues == * compile-time concept checking to only generare functions that make sense for the given type ( no floating point operations for integral types ) * usage of template expressions to unroll the operations at compile-time to allow the compiler to optimize them ( needs to be tested case by case with the performance test suite ) and elimination temporaries * care to keep most useful memory alignment ( to provide ease of use with direct usage in architecture dependent pipelines ) -- possibly an alignment trait * design with architecture-dependent optimizations in mind ( ease of expansion ) * keeping all the standard boost practices, including workarounds for non-conformant compilers == Research == Several existing implementations were studied and evaluated before the repeated submission of this idea. I intend to encompass most of the good things I found in the following implementations into a small but very useful library: * Boost.uBLAS (obviously) * Eigen - http://eigen.tuxfamily.org/ (heavyweight) * CGAL - http://www.cgal.org/ (license issues) * Matrix Template Library - http://osl.iu.edu/research/mtl/ (similar issues to uBLAS) * VCGLib - http://vcg.sourceforge.net/ (GPL) * nVidia's vector and matrix classes (found in nVidia examples) -- regards, Kornel Kisielewicz http://chaosforge.org/

9 16

Re: [boost] [iterator] reverse_iterator<counting_iterator<T>>
by Arno Schödl 13 Apr '10

13 Apr '10

"Stewart, Robert" <Robert.Stewart(a)sig.com <mailto:Robert.Stewart@sig.com> > wrote in message news:DF2E67F3D097004694C8428C70A3FD690470B26B7F@msgbal516.ds.susq.com <news:DF2E67F3D097004694C8428C70A3FD690470B26B7F@msgbal516.ds.susq.com> ... > this->base() calls iterator_adaptor<>::base(), which returns a reference to the underlying iterator it holds. Passing that reference to boost::prior() means that prior() pre-decrements the iterator within the iterator_adaptor. It would be odd for that >decrement operator to return anything but a reference to the decremented object, so boost::prior() returns a reference to its decremented argument, a reference to the decremented iterator within the iterator_adaptor. > I see no problem. boost::prior(...) internally makes a copy of its iterator argument, decrements it and returns it by value, so far so good. counting_iterator::operator* then returns a reference into this (temporary) iterator. Returning from reverse_iterator::dereference() destroys the temporary iterator, but still returns the reference pointing into it. I overlooked that I already filed a defect a while ago for it: https://svn.boost.org/trac/boost/ticket/2640 <https://svn.boost.org/trac/boost/ticket/2640> It would be great to have it fixed, because the bug is quite subtle and may go undetected for a while. Arno -- Dr. Arno Schoedl · aschoedl(a)think-cell.com Technical Director think-cell Software GmbH · Chausseestr. 8/E · 10115 Berlin, Germany http://www.think-cell.com · phone +49 30 666473-10 · toll-free (US) +1 800 891 8091 Directors: Dr. Markus Hannebauer, Dr. Arno Schoedl · Amtsgericht Berlin-Charlottenburg, HRB 85229

3 4

[serialization] serialization lib not thread safe?
by Jean-Marie Auffray 13 Apr '10

13 Apr '10

I use boost serialization in multi threaded application. Sometimes application crashed. I manage to avoid bug if I recompile boost serialization with BOOST_SPIRIT_THREADSAFE defined in jam file : http://archive.netbsd.se/?ml=boost-build&a=2008-07&t=8023861 But I would like keep boost default installation. I read multi thread limitations here : http://www.boost.org/doc/libs/1_42_0/libs/serialization/doc/special.html But I don't find how to use singleton to avoid crash, I attach test files to reproduct crash. I use 1.36 in test files, but same bug seems to occure in next release. How can I add singleton from serialization to avoid crash? I try some solutions but failed to find a good implementation. Thanks for help.

2 1

release managers: permission to merge?
by Frank Mori Hess 13 Apr '10

13 Apr '10

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 According to the calendar, the release branch went into "closed except by permission" yesterday. Is it all right for me to merge to release the fix for this ticket: https://svn.boost.org/trac/boost/ticket/4089 The changeset in question is https://svn.boost.org/trac/boost/changeset/61243 I've tested it locally, but have just comitted it to trunk so the tests there haven't cycled yet. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAkvEbewACgkQ5vihyNWuA4XawACg1jS8IZun6NJ52GkCa2V3i4nc PAIAoJlaXBs4NqY5mfGiHMoVAYJsTSVp =NiyZ -----END PGP SIGNATURE-----

2 1

Guidelines on wiki
by Daniel James 13 Apr '10

13 Apr '10

Hello, I've copied the guidelines onto the wiki now, you can see them at: https://svn.boost.org/trac/boost/wiki/Guidelines/ Any comments? I've changed the beta site to redirect to them. I haven't linked to them from the front page of the wiki as they aren't 'official' yet. I'll be moving more pages over soon-ish. Any ideas about how the wiki should be structured? Daniel

5 10

[iterator] reverse_iterator<counting_iterator<T>>
by Arno Schödl 13 Apr '10

13 Apr '10

Hello, boost::reverse_iterator::dereference() is typename super_t::reference dereference() const { return *boost::prior(this->base()); } boost::constant_iterator<T>::reference_type is T const&. So reverse_iterator< constant_iterator<T> >::dereference returns a reference to a temporary variable. Not good. I suggest removing the reference from boost::constant_iterator::reference_type, making it T. The rule would be that reference_type must only be a reference if the reference is pointing to some underlying container, not to the iterator itself. Arno -- Dr. Arno Schoedl · aschoedl(a)think-cell.com Technical Director think-cell Software GmbH · Chausseestr. 8/E · 10115 Berlin, Germany http://www.think-cell.com · phone +49 30 666473-10 · toll-free (US) +1 800 891 8091 Directors: Dr. Markus Hannebauer, Dr. Arno Schoedl · Amtsgericht Berlin-Charlottenburg, HRB 85229

2 1