December 2014 - Boost - lists.boost.org

[compute] Review
by John Bytheway 31 Dec '14

31 Dec '14

Here's my review. 1. What is your evaluation of the design? The design of building on OpenCL seems sound, for the reasons Kyle has explained. The fact that it has real users is very encouraging. I don't really like the name (I was unable to guess what the library was about from the name), but it's not worth changing at this point. I am worried about the compile-time configuration with macros. I would hope that BOOST_COMPUTE_HAVE_THREAD_LOCAL could be read from Boost.Config. For the two features which require linking against other Boost libraries, my instinct would be to have them on by default, rather than off by default (that should be easier if and when Boost.Compute is part of the Boost distribution). It would be very nice to have range versions of all the algorithms. I find myself using the boost::range algorithms much more than the std:: ones these days. Similarly, it would be good to provide the safer 4-iterator versions of equal, is_permutation, mismatch (and any I've forgotten) as were added to C++14. Some minor things: - Consider renaming boost::compute::lambda to be consistent with one of the other placeholder-containing namespaces from std or Boost. (You can keep the old name as an alias for backwards-compatibility). - Why the default of .25 as the bernoulli_distribution parameter? - Did you consider using Boost.Predef for your version macro? 2. What is your evaluation of the implementation? Only looked at a couple of things while trying to understand the docs. 3. What is your evaluation of the documentation? Overall, not too bad. More tutorial would be good. In particular, something using some of the standard algorithms. Some concerns: - All the header links off <http://kylelutz.github.io/compute/boost_compute/reference.html> are broken. - All or most of the things in "See also" sections look like they ought to be links but aren't. - For all your function parameter documentation, the list of parameters seems to be sorted alphabetically by parameter name rather than in the order they are passed to the function. This is really weird... - As others have mentioned, some kind of complexity estimate or promise for the algorithms is very important. In particular it would be good to understand how much parallelism they take advantage of and how they ought to scale with device compute capability. - The warning about accumulate being slow for floats should be much more prominent. - <http://kylelutz.github.io/compute/boost/compute/copy.html> mentions copying from e.g. a std::list to a compute::vector. It would be nice if it gave some kind of indication as to how efficient we can expect such an operation to be. In particular, I guess it ought to do some kind of buffering on the host? - The way I normally copy into vectors is call v.reserve(...) and then copy to std::back_inserter(v). I tried that with a compute::vector and unsurprisingly it was very inefficient. I see you already warned about this in the push_back docs, but perhaps it should also be highlighted in the documentation of compute::copy. - Is there no version of inclusive/exclusive_scan or iota using an arbitrary associative operator? - <http://kylelutz.github.io/compute/boost/compute/includes.html> it's unclear whether the subrange has to be contiguous. - min_element and max_element should cross-reference minmax_element. - What source of randomness does random_shuffle use? - Why does reduce return its result through an OutputIterator? - <http://kylelutz.github.io/compute/boost/compute/remove.html> Does this really remove things or use std::remove semantics? You should clarify and document the return value. (Same with remove_if, unique) - I can see how sort_by_key might be useful, and I'm slightly surprised that there's no stable_sort_by_key (which seems like it would be more useful than stable_sort). Any particular reason? - The boost/compute/types headers docs seem to be broken or missing. In particular I couldn't find reference documentation for float4_ et al. - I was interested to see allocators, but the docs seem uninformative: <http://kylelutz.github.io/compute/boost/compute/pinned_allocator.html> - Given the confusion with std::future, you might want to clarify whether compute::future's destructor blocks. - Also, <http://kylelutz.github.io/compute/BOOST_COMPUTE_CLOSURE.html> doesn't explain what format the lists (arguments and capture) should be in. Preprocessor functions often take lists in other formats, such as a Boost.PP Sequence <http://www.boost.org/doc/libs/1_57_0/libs/preprocessor/doc/data/sequences.h…>, and your example only has one-element lists, so it's not enough to clarify this point. (To be clear: I'm not suggesting you should use Boost.PP sequences, merely that you should make it clear in the docs whether you are). - <http://kylelutz.github.io/compute/BOOST_COMPUTE_CL_CALLBACK.html> is empty, as is <http://kylelutz.github.io/compute/BOOST_COMPUTE_MAX_ARITY.html> - <http://kylelutz.github.io/compute/boost/compute/valarray.html> - presumably compute::valarray has overloaded arithmetic operators? Are they mentioned anywhere? - <http://kylelutz.github.io/compute/boost/compute/vector.html> the unusual behaviour of vector's size_t constructor (not initializing the elements) should be warned more prominently. - <http://kylelutz.github.io/compute/boost/compute/device.html> the documentation of the enum type in the synopsis seems to be messed up. Similarly memory_object. - <http://kylelutz.github.io/compute/boost/compute/function.html> I'm puzzled by this interface. What's the name for? Would one ever use this constructor rather than e.g. make_function_from_source? - <http://kylelutz.github.io/compute/boost/compute/field.html> My reading of this is that when you get the type wrong you are guaranteed some kind of meaningful error report, and not just undefined behaviour; is that right? Perhaps you should clarify. - I can't guess what atomic_min and atomic_max do. - The right arrow ('next') link from <http://kylelutz.github.io/compute/boost/compute/atomic_xor.html> is broken (looks like the docs for the placeholders don't work). - The iterators should state what iterator category they have. - I can't guess what svm_ptr is for. Later I discovered svm_alloc and svm_free. Is there an RAII way to manage these things? - What does default_device do if there are no devices? Do the various environment variables mentioned have any kind of precedence? (Probably linking to the corresponding OpenCL docs is sufficient for such questions). Do they have to be exact matches? Are they case-sensitive? - What does find_device do if the requested device does not exist? - Can you give examples for type_definition? My guess was that it did the thing which type_name later turned out to do... - BOOST_COMPUTE_ADAPT_STRUCT should state that it must be used at global scope (i.e. not within a function or other namespace). - Congratulations for getting compile-time detection of internal padding into BOOST_COMPUTE_ADAPT_STRUCT :). You might want to mention in the docs that this will be detected, so people don't worry too much. - It looks like the performance tests don't include the time for copying data to (and, where applicable, from) the device. You should mention this on the Performance page. - You should probably add C++AMP to your things compared with in the FAQ. 4. What is your evaluation of the potential usefulness of the library? High. I've wanted something like this in the past and not found this. 5. Did you try to use the library? With what compiler? Did you have any problems? I used it with g++ 4.8.2 in C++1y mode. Some quick tests seemed to work OK, and indeed I was able to sort some large vectors of scalars faster using boost::compute::sort than std::sort, which is nice. I was pleased to see that 64-bit integers were handled fine. 6. How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? Couple of hours reading and trying simple examples. 7. Are you knowledgeable about the problem domain? I have done a couple of small projects with CUDA, and am generally knowledgeable, but with no previous OpenCL experience. 8. Do you think the library should be accepted as a Boost library? Be sure to say this explicitly so that your other comments don't obscure your overall opinion. Yes. OpenCL seems to have fairly broad acceptance, but is rather difficult to use safely and with C++-isms. This library fills a useful niche. In an ideal world it will eventually be rendered obsolete by things like parallelism entering the standard library, but I doubt that will happen soon if at all. This provides a good solution now, and we shouldn't dismiss it simply because something sparkly is approaching, which may or may not turn out to be awesome and widely adopted. This library has clearly been a lot of work. Congratulations on making it this far, I hope the review is not too stressful :). John Bytheway

3 12

[system] Problem with boost::system::system_category() under QNX Momentics
by Piotr Orzechowski 31 Dec '14

31 Dec '14

At the beginning I must assure You that I've tried almost every solution from similar topics , still they didn't solve my issue. I'm building binaries from Boost 1.57 using **MinGW**. Boost libraries I'm using: *thread* and *system* build without any errors. I was trying: > toolset=gcc --target-os=qnxnto and toolset=gcc --target-os=qnx, both didn't work. I'm including correctly both libraries: > -lboost_system-mgw48-mt-d-1_57 -lboost_thread-mgw48-mt-d-1_57 after using: > `#define BOOST_SYSTEM_NO_DEPRECATED` I got rid of `boost::system::generic_category()` error, still I can't get rid of: > boost::system::system_category() I'm working in **QNX Momentics**. I'm totally out of ideas, I'm really looking forward for any help I can get.

2 2

Define to test for deprecated std::auto_ptr
by Fletcher, John P 31 Dec '14

31 Dec '14

I have some tests which are throwing up std::auto_ptr as deprecated with C++11. Is there a define that can be tested for in user code so that I can switch to std::unique_ptr when it is available? Something like this: #ifdef BOOST_NO_CXX11_UNIQUEPTR // use auto_ptr #else // use unique_ptr #endif Is there a general flag for C++11 or C++14 or is it all done on particular features? Thanks John

3 5

[compute] CL_QUEUE_OUT_OF_ORDER_EXEC_MODE_ENABLE
by Gruenke,Matt 31 Dec '14

31 Dec '14

I assumed there was some behind-the-scenes mechanism to maintain order of higher level operations, but I hadn't dug around to find them. I thought maybe there was an event object embedded in the device memory containers, used to track any pending writes to them, that could be added to the wait list of subsequent operations. But I don't see anything like this. So, will enabling out-of-order execution will break the higher level operations? Or am I missing something? Matt ________________________________ This e-mail contains privileged and confidential information intended for the use of the addressees named above. If you are not the intended recipient of this e-mail, you are hereby notified that you must not disseminate, copy or take any action in respect of any information contained in it. If you have received this e-mail in error, please notify the sender immediately by e-mail and immediately destroy this e-mail and its attachments.

3 6

[compute] Review period starts today December 15, 2014, ends on December 24, 2014
by Antony Polukhin 31 Dec '14

31 Dec '14

Dear All, Review of the Compute library starts today on Mon 25st of December 2014 and will last for ten days. The Compute library provides a C++ interface to multi-core GPGPU and CPU computing platforms based on OpenCL. The project is hosted on GitHub at https://github.com/kylelutz/compute. Docs are available at http://kylelutz.github.io/compute/ Sources could be downloaded as a ZIP archive via https://github.com/kylelutz/compute/archive/master.zip Some pre-official reviews could be found in Boost Library Incubator: http://rrsd.com/blincubator.com/reviews/?library_id=669 Please, answer the following questions in your review: 1. What is your evaluation of the design? 2. What is your evaluation of the implementation? 3. What is your evaluation of the documentation? 4. What is your evaluation of the potential usefulness of the library? 5. Did you try to use the library? With what compiler? Did you have any problems? 6. How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? 7. Are you knowledgeable about the problem domain? And finally, every review should answer this question: 8. Do you think the library should be accepted as a Boost library? Be sure to say this explicitly so that your other comments don't obscure your overall opinion. -- Best regards, Antony Polukhin

7 8

[compute] Review comments
by Asbjørn 30 Dec '14

30 Dec '14

Hi, not a proper review, just some comments from a library user based on studying documentation and writing a couple of simple programs using Boost.Compute on Intel's and NVIDIA's OpenCL implementations (i7 CPU and GTX970 GPU respectively). I used MSVC2013 for the tests. Some of these comments I've shared previously on this list, but I'm repeating them here for completeness. I have some background in heterogeneous development and OpenCL in particular, but I'm no expert. I found Boost.Compute easy to pick up and providing a nice C++ layer, with a nice mix of low and high level. Comments in no particular order: 1) As a library user and one who's not a C++ guru by any means, the error messages thrown up when passing incorrect parameters can be quite confusing. For example, first time I tried to use Boost.Compute I managed to not pass the ouput iterator parameter to transform(), so that the lambda expression ended up in that position instead. This resulted in several very confusing errors inside result_of.hpp, with no errors anywhere near my or Boost.Compute source code. It would be nice if the compilation could fail in more informative manners. 2) I did miss async versions of the algorithms, so it's possible to chain together multiple calls. Even though all the data sits on the compute device, the overhead of waiting for each operation to finish before queuing the next can make the compute gains completely irrelevant. 3) I think relevant calls should have a non-throwing form returning an error code, ala Boost.ASIO. 4) Threading has been mentioned in this list. At the very least the documentation should be updated to clearly state the thread safety of each call/class, ala Boost.ASIO. Beyond that I would like to be able to share a context between several threads, each which could contain their own queue and device. 5) If thread safety or similar important features requires a compiled part, I wouldn't mind Boost.Compute being non-header-only. Though if possible making it optional like other Boost libraries do would be great. 6) The tests, and preferably the performance tests as well, should have some way of testing all devices across all platforms. Currently they only test the default device on the default platform it seems. This is insufficient IMHO. 7) I'd like some way of defining user functions either as lambda expressions, raw OpenCL-C code via make_function_from_source or both, which can be used in lambda expressions for say transform(). Something like function<int (int)> sqr = _1 * _1; transform(inp.begin(), inp.end(), out.begin(), sqr(2 * _1 + 1)), queue); Overall I think Boost.Compute can be useful right now for some use-cases, and it has good potential to be significantly more useful with a bit more work. Just my 2 øre. Cheers - Asbjørn

4 5

[filesystem] imbue_utf8
by Peter Dimov 30 Dec '14

30 Dec '14

Since nowadays it's very common for programs to use UTF-8 for path encoding everywhere, I suggest that in addition to path::imbue( loc ), there should be path::imbue_utf8(), which imbues a UTF-8 codecvt facet. Since this is the default on Mac OS X, the implementation is already present, but the user has no way to get to it.

1 0

[config] Macro for detecting thread_local storage support
by Kyle Lutz 30 Dec '14

30 Dec '14

I have looked but been unable to find any macro in Boost.Config for detecting compiler support for the C++11 thread_local storage specifier. Does such functionality exist? I would like to use something like this to automatically detect compiler support for thread_local for use by the proposed Boost.Compute library. Currently I rely on the users defining BOOST_COMPUTE_HAVE_THREAD_LOCAL in order to enable support. Hopefully there is a better way? -kyle

2 1

Re: [boost] [test] boost.test owner unresponsive to persistent problems for multiple years
by Dave Gomboc 30 Dec '14

30 Dec '14

I get the digests, so I'm quoting from a number of different emails on this thread in this message. > > Both contain a fix for this. On stack overflow the proposed fix is > > literally a single line change. > > The issue is addressed in devel branch with somewhat better fix > long long time ago. It is important that tickets be updated to reflect such information. > > If there are issues fixed in develop branch, it would help a lot > > to inform users by updating the tickets accordingly. > > Here is a blanket update message: at this point all the issues with > Boost.Test are fixed on devel branch ;). This doesn't help people who filed a ticket (perhaps years ago) but don't happen to catch this email. > > This has been a problem for about 5 years and these simple fixes don't > > make it into the library. > > This is going to be released as part of the effort of merging devel into > master. Now, when this happens depends only on docs upgrade. It seems that in addition to documentation upgrades/improvements there is at least also the issue of identifying all Boost.Test-related tickets that are currently outstanding, verifying that the issue raised within them is indeed resolved within the development branch (rather than just assuming this is true), and updating the ticket to reflect that the problem has been resolved in the development branch (though it should not be closed yet until a new Boost release comes out with the fix in place). > All in all "devel to master upgrade" is one target I really want to reach > and is what I plan to concentrate my effort on. You are still welcome to > help with this if you are up to it. Excellent! If it has really been five years (or even if it has been only one full year) since this last happened, then may I suggest that this should be your sole target for Boost.Test until this task is accomplished, that your target should be prior to the next major release of Boost, and that obtaining assistance be an important part of your plan to accomplishing this task within this timeframe? To that end, could you please provide for the Boost community a detailed outline of exactly what remains to be done in order for the merge of the latest Boost.Test to master to be completed? Please be clear about exactly what others can help with, and in the cases where you think nobody else can help you, why that is so. Please do so here on the mailing list rather than ask people to contact you privately about it, so that more people will know what remains to be done and may be willing to lend a hand so it can get done. > > 1. What are the reason(s) why-list_content is not (yet) > > added to the release? Is there any schedule for its release? > > I need to document all these new features to release new Boost.Test Can you provide a list of these new features alphabetically by name with a one-sentence description? Boosters can sign up for writing documentation paragraphs, can contribute them here, and you can collect and use them. > > Yes. I fully agree with this statement. I hope to get back to this list > > with more concrete schedule soon. And if you are interested to push this > > even sooner - please feel free to help. > > That's great news. I know a vast amount of work has gone into develop > and we'll all benefit when it finally gets merged into master. Yes, I'm > happy to support this effort as best I can. Feel free to contact me > off-list. Rather than contact people off-list, I urge you to get all of the outstanding work into the open and use this mailing list for co-ordination not only so that people can contribute without overlapping their effort needlessly, but so that the current status is visible to all of us. That last point is rather important: I know of projects that have chosen alternative testing frameworks simply because of the lack of visibility re: Boost.Test. > Thanks for working hard to move this forward. Boost.Test has always been > a critical library and as a result somewhat of a thankless task to > maintain given the constant risk of breakage across Boost as a whole. > Thanks for persevering. I absolutely concur with this. Your efforts are appreciated, Gennady! You've been shouldering the entire burden for too long. I think that by publicly coordinating and leading the remaining effort to get the latest Boost.Test onto master, you'll be able to achieve this goal more quickly. Dave

3 3

[filesystem] Request for help with unresolved tickets
by Beman Dawes 29 Dec '14

29 Dec '14

Boost.Filesystem has roughly 140 unresolved tickets, and far too many are more that a year old. Some are up to four years old. That's horrible, and entirely my fault. I really dislike processing most bug reports, so leave them until I have nothing else to do, which is roughly never. But a few bug reports are actually fun to process. These are the ones that provide a little test program that unambiguously illustrates the problem and includes full information about the compiler, platform, version of boost, etc. To encourage more such dream tickets, I've added a little bug reporting framework with docs to filesystem's develop branch. It depends on a new header (boost/detail/lightweight_test_report.hpp) that's been added to the detail library's develop branch. The hope is that this will lead to more test programs that make it easy to reproduce bugs to (1) speed my processing of tickets and (2) ease handing off tickets to other developers for fixes. There are several ways boosters can help: * Review and try out the instructions at http://boostorg.github.io/filesystem/issue_reporting.html. (Don't expect all links to work.) Were you able to easily follow these instructions? What would you like to see added or removed? * Look through the bugs in https://svn.boost.org/trac/boost/report/53, pick one you think would benefit from a test program, and add a test program as described in http://boostorg.github.io/filesystem/issue_reporting.html. (Do a git pull on develop branch of detail and filesystem libraries first.) Were you able to easily follow these instructions? What would you like to see added or removed? Thanks, --Beman

3 2