- Boost - lists.boost.org

[GIL] new IO extension review
by Domagoj Saric 08 Dec '10

08 Dec '10

Hi everybody, this is my first official Boost review so <insert the usual disclaimer> ;) I'm glad to see that at last more people have joined the discussion... In case some have not noticed, there was a very recent 'pre-discussion' about io_new titled "[gil] New IO release" and can be found here: http://lists.boost.org/Archives/boost/2010/10/index.php http://lists.boost.org/Archives/boost/2010/10/171450.php ...where I've already written (about) my thoughts and objections in detail: http://lists.boost.org/Archives/boost/2010/10/171732.php For the sake of brevity (and tight time management :) I will not repeat all the details here, rather try to reiterate through them briefly (omitting tests, examples and/or detailed argumentation) while perhaps adding some new thoughts... Mateusz, can you please confirm whether this approach (to assume that the "[gil] New IO release" notes and conclusions are part of the final review discussion) is OK or should I somehow repack that discussion into the new one..? ------ What is your evaluation of the design? ------ While the design can be considered an improvement from GIL.IO I nonetheless consider it lacking. As the new interface already constitutes a breaking change, this opportunity could have been used to provide a much more powerful and flexible interface (while still maintaining ease of use). A brief reiteration from "[gil] New IO release", the free function interface is inflexible/weak because it: - makes it difficult or practically impossible to reuse underlying backend instances across calls which, for example, has serious performance implications for ROI based access or when you want to read the image info/metadata before reading the image... - (as is done now) couples the choice of the backend with the choice of the image format (i.e. it makes it much more difficult to use/add alternate backend implementations for various image formats, for example if one wants/has to use LodePNG or GDI+ or ... as a backend) - does not allow access to the underlying backend which is necessary both for faster/easier solving of real world problems (either later found deficiencies in the interface or more complex/unconventional/corner use cases) and a more future-proof design (that can be more easily extended without breaking changes and/or introduction of additional overheads) ------ What is your evaluation of the implementation? ------ Considering that the GIL.IO implementation already has an inefficient implementation (which was even hinted by the "this will be simplified" comments in the code) I would hope io_new to provide a step forward in this regard...unfortunately it turns out it is a big step backward, both in terms of code bloat and code speed (as demonstrated by the tests and in-depth analysis in the "[gil] New IO release" thread)... In fact, the mentioned test practically demonstrates a 'theoretical' effect: on an Intel i5 CPU, with a relatively large cache, io and io2 benefit from being compiled/optimized for speed (as opposed for size) while io_new actually executes slower when compiled for speed (probably because the inherent code size increase/excessive inlining which overflows even a large CPU cache)...this effect was not demonstrated when the same test was run a much weaker Via C7-M CPU (with a much smaller cache)... In addition to efficiency concerns, the io_new implementation (or 'internal design') is also inflexible in certain areas where it causes duplication/repetitive code between different backend wrappers (thus not making it easier to extend with new backends). ------ What is your evaluation of the documentation? ------ Did not build it... ------ What is your evaluation of the potential usefulness of the extensions? ------ A makeover for GIL.IO is long over due and the general ideas that Christian put in io_new are mostly very welcome even though I have serious objections as to how those ideas were actually put in practice... ------ Did you try to use the extensions? ------ Yes, but only JPEG, PNG and TIFF IO extensions (did not try the opencv, sdl and toolbox extensions)... ------ With what compiler? ------ MSVC++ 10.0 ------ Did you have any problems? ------ Yes, for example the broken TIFF reading-with-conversion, Christian did fix it later but the fix seems more like a patch as it demonstrates an inherent inflexibility (mentioned at the end of the implementation section) in the io_new design that requires the backend-wrapper maintainer to provide the (possibly very long) conversion switch cases for each backend... ------ How much effort did you put into your evaluation? ------ As mentioned, I dug only through the core IO extension (and specifically through the LibJPEG, LibPNG and LibTIFF backend wrappers) but the areas I did investigate I examined relatively thoroughly (and over a longer period of time...in parallel, over the many months of development of io2)... ------ Are you knowledgeable about the problem domain? ------ My main use case consists of simple image loading on application startup (for the application skin) and only minor adjustments (like PC to Mac gamma fixup). However, as I was unsatisfied with existing solutions (including the direction in which io_new went) I decided to roll my own solution and for this to get accepted I had to look into and cover all reasonably possible use cases (which included going through the GIL.IO documentation as well as old/previous GIL.IO related discussions). ------ Do you think the extensions should be accepted as a part of Boost.GIL library? ------ (To make it clear) The way they are now - no. However, as mentioned earlier, a makeover of GIL.IO is undoubtedly needed and the work Christian has done so far is truly massive. For example, the properties system could prove to be quite useful (as a C++ abstraction for the various options and metadata specific to each image format) if done right (if not coupled to a particular backend, so that, for example, the PNG properties can equally be used, if supported, with LibPNG, LodePNG, GDI+, WIC ....)... Actually, IMO some of the extensions (like new GIL image/pixel formats) should maybe enter GIL itself, not merely as extensions (this would possibly even make it easier to implement the more complex IO backends/formats, like TIFF, that support those types of images)... -- "What Huxley teaches is that in the age of advanced technology, spiritual devastation is more likely to come from an enemy with a smiling face than from one whose countenance exudes suspicion and hate." Neil Postman

3 2

Re: [boost] Boost on Sun CC without STLPort
by pfee＠talk21.com 08 Dec '10

08 Dec '10

Beman Dawes wrote: > On Tue, Dec 7, 2010 at 11:12 AM, Dave Abrahams <dave(a)boostpro.com> wrote: >> On Tue, Dec 7, 2010 at 11:07 AM, Dean Michael Berris >> <mikhailberis(a)gmail.com> wrote: >>> Hi Everyone, >>> >>> Is there any specific reason why Boost isn't tested against Sun CC >>> without STLPort -- aside from the obvious reason that there's nobody >>> running the regression tests on that platform? Is it something >>> inherently wrong in Sun's standard library implementation that's >>> preventing Boost from working properly on the Sun compiler? >> >> It used to be wicked noncompliant, to the point where it was a waste >> of time to even try. I'm not sure what the status today, though. > > It is way better than it used to be. My understanding is that in the early days of Sun CC 5.x, the compiler was non-compliant. Over time, the compiler improved, but the libCstd ABI was set in stone. The compiler has become more capable, but the RogueWave derived STL is still configured to be non-compliant in order to maintain the ABI set when the compiler was less capable. I presume backwards compatability was considered more important than standards compliance. Here's an OpenSolaris proposal to change this (not sure of the status of Solaris 11). The RogueWave STL has become the Apache STDCXX project, http://stdcxx.apache.org/. If that becomes the new STL on Solaris, then Boost may be usable with the default compiler and STL configuration. http://arc.opensolaris.org/caselog/PSARC/2008/549/proposal.txt However Boost + SunStudio/libCstd is likely to be non-starter as Boost requires a high level of standards compliance. Even Boost + SunStudio/STLport didn't work for quite some time. Here's a blog I used to follow as SunStudio/STLport approached usability with Boost. The first version that worked out of the box was 1.34.1. http://blogs.sun.com/sga/category/Boost Thanks, Paul

2 1

How to make a soft copy of a temporary Python object inside C++
by Raul Durand 07 Dec '10

07 Dec '10

Hello everybody, Maybe it is obvious but I could not figure out how to solve it yet.. Consider the A and B classes: Class B { public: int func() {return 1; } }; class A { public: void setB(B & b_instance) { b = &b_instance; // if in Python setB is called as: a=A(); a.setB(*B()*) where B() creates a temporary, // the b pointer becomes invalid after setB() function gets out of scope. } B * b; }; BOOST_PYTHON_MODULE(my_module) { _class<B>("B") .def("func",&B::func); _class<A>("A") .def("setB", &A::setB); } The function A::setB sets the b data member of class A. b is a pointer to an object of class B. In Python I can use: a=A() b=B() a.setB(b) that's ok since 'b' exists as long as 'a', which could be dangerous. Things get worse when in Python I write: a=A() a.setB(B()) because the pointer b in class A gets invalid when the setB function gets out of scope. So, how can I avoid Python for destroying the temporary and keep the object alive in C++? Does it have anything to do with the object ownership? Is there a way to change the ownership? Can I make a soft copy of the b_instance object inside the setB function? so it would increase the object counter and keep it alive and consequently the b pointer will not get invalid. I want to avoid making a deep copy. Thanks in advance. Raul Durand

2 3

[assign] operators+= and -= for set/reset bits in std::bitset
by Winkler, Bernd 07 Dec '10

07 Dec '10

Dear All, I was a little bit tired of using string initialization of std::bitset like: bitset<8> flag_pattern(string("00111000")); What I found more readable was this: Bitset<8> flag_pattern; //initialisation to "00000000" flag_pattern += 1,3,4,5; //set bits at indices 1,3,4,5 to true: 01011100 flag_pattern -= 1,3; //set bits at indices 1,3 to false: 00001100 In that way I can specify an easy readable configuration of flags: enum { left, right, center } flag_pattern += left,right; If interest exists I could contribute the respective code for the assign library. Best regards, Bernd P.S. Please let me know if I missed some formal criteria for this post as I am new to this NG

4 3

Re: [boost] [assign] operators+= and -= for set/reset bits instd::bitset
by Winkler, Bernd 07 Dec '10

07 Dec '10

>Doesn't the interpretation of that depend on the endianness of your architechture? I don't know but I can imagine. If so, it would be one more reason to address and/or initialize single or multiple bits using e.g. enum values .. preferably in an easy way like the one shown. Bernd

1 0

[trac] Performance for last week?
by Rene Rivera 07 Dec '10

07 Dec '10

First after disabling the trac source browser for most of the week here's the new breakdown of the top ten sections the web server spent time on (including www.boost.org) total_time = 319654701.643986 43.88 140251178.52 /svn 43.88 140251171.39 /svn/boost 42.47 135748391.81 /svn/boost/!svn 40.86 130621926.05 /svn/boost/!svn/vcc 40.86 130621926.05 /svn/boost/!svn/vcc/default 27.50 87894354.72 /trac 26.23 83846699.34 /trac/boost 14.81 47331832.95 /doc 14.26 45569562.87 /doc/libs 10.90 34846318.37 /style-v2 As you can see trac is no longer at the top of the list. And SVN is taking up almost half the time. Which is to be expected since many SVN interactions take considerable clock time to complete as data is sent to clients. So big questions with respect to last week: * Is trac performance to an "acceptable" point? * Do you miss the trac source browser, and desperately want it back? And now regarding SPAM.. I did some updates of the spam modules that improve overall performance, the dns sub-module was know to cause global trac delays. I was hoping to introduce CAPTCHAs with the updates.. But there was a problem with the one I got yesterday. I'll try a new update tonight and hopefully we can really cut down on the spam once and for all :-) PS. No I haven't looked at what, if anything, needs to get done to reduce the svn times. -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org (msn) - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim,yahoo,skype,efnet,gmail

4 3

GIL IO extension review
by Kenny Riddile 06 Dec '10

06 Dec '10

Sorry if this review seems short, but most of my time is spoken for. Still, I think this extension is useful enough that I found time to review it. I've been using the new version of the GIL IO extension for over a year now and am glad to finally see it up for review. I've primarily used it for loading various image formats for use as OpenGL textures. I recently gained some experience with extending the library to support an additional image format (TARGA) as well. - What is your evaluation of the design? My use case for the library is conceptually simple, so I would expect the library to make it simple to accomplish. The provided read_image() and write_view() functions are about as simple an interface as one could expect, but without sacrificing flexibility and extensibility due to the usage of tag dispatching and standard iostreams. As a user, I have no complaints about the design and how it integrates with GIL. - What is your evaluation of the implementation? I hadn't looked at the implementation much prior to recently adding support for reading and writing TARGA images, and my evaluation of the implementation is from that standpoint. The fact that I (of all people) was able to easily add support for a new format without much documentation on the subject suggests that the relevant parts of the implementation are easily understandable. I mostly used the BMP implementation as a reference, and didn't notice anything egregious or suspicious. I have some concerns about the usage of setjmp/longjmp in the PNG and JPEG implementations (honestly I get suspicious any time I see those). I think there was discussion elsewhere about adding additional test cases to exercise the error cases that result in a longjmp so that all of the supported platforms can be verified. I think this is a good idea. Another nitpick that came up when adding TARGA support are the read_int8, read_int16, read_int32, write_int8, etc. member functions of the Device models in io_device.hpp. To me, these names suggest that these functions return and accept signed integers when in reality they return and accept unsigned integers. - What is your evaluation of the documentation? The documentation was sufficient enough for me to quickly figure out how to read and write images from all of the supported formats. The section on extending the library to support new formats was enough to get me started, but I ended up just relying on the existing BMP code as a reference. This suggests that maybe that section of the documentation should be fleshed out a little more. - What is your evaluation of the potential usefulness of the extensions? This new IO extension is extremely useful. It nicely fills huge gaps in the existing IO extension, such as loading from streams instead of only being able to supply a file name. The interface is simple enough to integrate into just about any design, while still providing both flexibility and extensibility. - Did you try to use the extensions? With what compiler? Did you have any problems? I've used the proposed IO extension to read and write BMP, PNG, JPEG, and TARGA files with both MSVC9 and GCC in combination with the last several Boost releases. I had no problems. - Do you think the extensions should be accepted as a part of Boost.GIL library? Yes, provided additional tests are added to exercise currently untested error cases, and they pass on all platforms officially supported by Boost. This extension is just as simple to use as the existing GIL IO extension, yet is better in every way. Therefore, I see no reason not to accept it as a replacement.

2 1

Re: [boost] Partial preprocessing for Fusion, et.al. (was: [netlib] Update: 0.9 will not be header-only)
by Hartmut Kaiser 06 Dec '10

06 Dec '10

> On Sun, Dec 5, 2010 at 2:02 AM, joel falcou <joel.falcou(a)lri.fr> wrote: > > In my own experience, problem lies in Proto/MPL/Fusion preprocessor > > time more than anything else. > > MPL became managable thansk to the preprocessed headers, maybe > > Fusion,Proto and maybe Spirit should do the same > > +1 > > Does anybody have an idea on how this can be done? I seem to recall that > GCC only had options for preprocessing when compiling an object file -- > that would mean for making a .cpp file for each of the headers that need > to be pre-processed independently, running it through the compiler's > preprocessor (or wave) to get the appropriate output. > > Maybe someone already has a script to do that which can be adapted? What everybody is looking for is partial preprocessing, i.e. expansion of only some of the macros. Wave does not directly support this (does any preprocessor?), but you can do is to use #pragmas to enable/disable output generation. This creates the effect of partially expanding only a subset of the input. Here is an example: File: example.hpp: // stuff here will be preprocessed but not be output #if defined(__WAVE__) && defined(CREATE_PREPROCESSED_FILES) #pragma wave option(preserve: 2, line: 1, output: "preprocessed/example.hpp") #endif // stuff here will be preprocessed and written to the file // preprocessed/example.hpp (here: relative to example.hpp) #if defined(__WAVE__) && defined(CREATE_PREPROCESSED_FILES) #pragma wave option(output: null) #endif // stuff here will be preprocessed but not be output File: example.cpp // stuff here will be preprocessed but not be output #include "example.hpp" The crucial trick is to invoke the wave driver with the --output=- (short option: -o-) command line option, which starts off with the output suppressed. Having to add -DCREATE_PREPROCESSED_FILES is obvious. Regards Hartmut --------------- http://boost-spirit.com

2 4

How to correct when Auto-link gets it wrong in VS
by Paul Bergen 06 Dec '10

06 Dec '10

I installed boost both with bjam and with the BoostPro installer, both times with the same option. I specified that it should install as Multi-Threaded Debug DLL. I'm using Visual Studio 2008 EE. I followed the steps here: http://www.boost.org/doc/libs/1_45_0/more/getting_started/windows.html But Visual Studio won't link, it says it can't find: libboost_system-vc90-mt-sgd-1_44.lib, which is because the file really is: boost_system-vc90-mt-gd-1_44.lib According to this page: http://www.boost.org/doc/libs/1_45_0/more/getting_started/windows.html#libr… the problem is that VS is trying to find the static library, although I installed the dynamic one. I can't find in the getting_started where you tell VS which version to use. Thanks.

6 6

[netlib] Update: 0.9 will not be header-only
by Dean Michael Berris 06 Dec '10

06 Dec '10

Hi Everyone, To those who are following the development of cpp-netlib (which shall be submitted for review /soon/ as Boost.Netlib), the current state of 0.9-devel and thus what's going to come out officially as 0.9 in a couple of weeks, will not be header-only. Aside from the obvious reason that the compile-times are starting to become unbearable (>10 minutes to build the whole test suite!!) and after noticing that the intended simplicity of the API with tag-metafunction dispatch is limiting the expressiveness of the HTTP client types, the 0.9 release will be the first release that will require an externally linked-in static library. The biggest culprit at the moment has been the implementation of the URI parser that uses Boost.Spirit Qi and >10 rules in the URI parsing grammar. The URI implementation is really declarative and nicely written (thanks go out to Jeroen Habraken for making that happen). The down-side of this has been the compile times: building the simple HTTP client takes close to one minute on the machine I'm using -- and by moving it to an externally-linked static lib, the compile of the same HTTP client is cut down to 10 seconds. Before you ask if I'm using GCC, the answer is no, I'm using Clang when developing so that I get as much speed as I can when doing the edit-rebuild-test cycle. I've written about what I've been doing and what I'll be doing in my three recent articles at C++ Soup. If you want to get more details, please follow the links below. http://cplusplus-soup.com/2010/12/03/cpp-netlib-moving-away-from-header-onl… http://cplusplus-soup.com/2010/12/02/cpp-netlib-demolition-day/ http://cplusplus-soup.com/2010/12/01/cpp-netlib-reducing-compilation-times/ Have a great weekend everyone, please send your qestions/comments/suggestions over either directly to me or through the various mailing lists (Boost-devel, Boost-users, cpp-netlib-devel). -- Dean Michael Berris deanberris.com

9 11