- Boost - lists.boost.org

[build] msvc.write_setup-script always called for msvc-12.0
by Peter Dimov 02 Jan '15

02 Jan '15

When I do (for example) C:\Projects\boost-git\boost\tools\boostdep\build>b2 toolset=msvc-8.0 I get ...found 1 target... ...updating 1 target... msvc.write-setup-script C:\Users\PETERD~1\AppData\Local\Temp\b2_msvc_12.0_vcvarsall_x86.cmd ...updated 1 target... ...found 1 target... ...updating 1 target... msvc.write-setup-script C:\Users\PETERD~1\AppData\Local\Temp\b2_msvc_12.0_vcvarsall_x86_amd64.cmd ...updated 1 target... ...found 1 target... ...updating 1 target... msvc.write-setup-script C:\Users\PETERD~1\AppData\Local\Temp\b2_msvc_8.0_vcvarsall_x86.cmd ...updated 1 target... ...found 1 target... ...updating 1 target... msvc.write-setup-script C:\Users\PETERD~1\AppData\Local\Temp\b2_msvc_8.0_vcvarsall_x86_amd64.cmd ...updated 1 target... which correctly generates the setup scripts for the specified msvc-8.0, but also unconditionally generates the setup scripts for msvc-12.0, which are not needed at this time. This msvc-12.0 generation occurs even if I use a non-msvc toolset, or even just do b2 --show-libraries. C:\Projects\boost-git\boost>b2 --show-libraries ...found 1 target... ...updating 1 target... msvc.write-setup-script C:\Users\PETERD~1\AppData\Local\Temp\b2_msvc_12.0_vcvarsall_x86.cmd ...updated 1 target... ...found 1 target... ...updating 1 target... msvc.write-setup-script C:\Users\PETERD~1\AppData\Local\Temp\b2_msvc_12.0_vcvarsall_x86_amd64.cmd ...updated 1 target... The following libraries require building: - atomic - chrono [...]

1 0

[filesystem] Junction support on Windows?
by Peter Dimov 02 Jan '15

02 Jan '15

Filesystem provides support for hard and symbolic links, but on Windows, hard links do not work on directories and symbolic links do not work for the majority of users because they require elevation by default. That is why, for example, "b2 headers" was changed to also use junctions, if possible. But if I, for another example, want to replicate "b2 headers" from a C++ program using Filesystem, I can't, because it offers no junction support. Should create_directory_symlink be enhanced to create a junction, if CreateSymbolicLink is not available or fails with ERROR_ACCESS_DENIED?

2 2

[filesystem] resize_file_api leaks a HANDLE
by Peter Dimov 02 Jan '15

02 Jan '15

... when SetFilePointerEx or SetEndOfFile fails. BOOL resize_file_api(const wchar_t* p, boost::uintmax_t size) { HANDLE handle = CreateFileW(p, GENERIC_WRITE, 0, 0, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0); LARGE_INTEGER sz; sz.QuadPart = size; return handle != INVALID_HANDLE_VALUE && ::SetFilePointerEx(handle, sz, 0, FILE_BEGIN) && ::SetEndOfFile(handle) && ::CloseHandle(handle); }

3 2

updated Boost.Test devel->master: documentation needs
by Dave Gomboc 02 Jan '15

02 Jan '15

> What left in docs: > > * general reformating of existing pages and separation to "user's guide" and "advanced usage scenarios" (I am working on this) > * BOOST_TEST tool, including figuring out what to do with old testing tools description and reference > * data driven test cases (Raffi works on this) > * decorators support (Andrzej volunteered to help with this) > * run by name/label description. Including multitude of new syntax options > * figure out doxigen issues > * figure out new format for examples > * few minor issues with Boost.Logos > * investigate Boost.pp style left side menu (optional) > * interaction testing (optional) > * production use of Boost.Test (optional) > * anything else missing from above (hopefully nothing big) > > Any takers? > Gennadiy Thanks for posting this list, Gennadiy. I'll volunteer to write up a section on the production use of Boost.Test. I will, however, need source material in order to do so. Boosters, have you ever or do you now use either Boost.Test or an alternative testing framework in a production-like manner? - with more complicated or less common build systems - when regression testing across multiple versions of dependent libraries - when regression testing across multiple toolchains and/or platforms - as part of deployments within virtualized containers - et cetera If so, could you please describe your setup, either here on-list or privately to myself? I will collect the responses received, and from them, prepare documentation so that people new to Boost.Test can get started with setting up production-quality regression testing environments. Dave

2 1

[compute] Review
by Yiannis Papadopoulos 01 Jan '15

01 Jan '15

Hi, This is my review of Boost.Compute: 1. What is your evaluation of the design? It seems logical to me. It is effectively a wrapper around OpenCL that provides implementations of higher-level algorithms, and allows interoperability with OpenCL and OpenGL. The Boost.Compute name is a bit misleading, as Boost.Compute supports only OpenCL-enabled devices. 2. What is your evaluation of the implementation? There is some code duplication (e.g. type traits) and various other bits and pieces that can be moved to existing Boost components. I think there should be some effort spent towards that. It seems that performance is on par with Thrust. However, there are other libraries out there (e.g Bolt) and multiple devices, so there has to be a more extensive experimental evaluation to say decidedly that it is a good implementation. 3. What is your evaluation of the documentation? Overall, it is pretty good. Given the complexity of the accelerator programming model, a few more elaborate examples in the tutorial would be welcome. 4. What is your evaluation of the potential usefulness of the library? This is difficult to answer. A lot of work has been put in this library and it seems the way to go. The interfaces are clean, the code looks solid and the developer willing. However, there is limited vendor support, there are not enough benchmarks and there are other alternatives that they have both. Given that Boost.Compute is targeted to users that know a thing or two about performance, I don't know how they can be convinced to consider using Boost.Compute against Bolt or Thrust. 5. Did you try to use the library? With what compiler? Did you have any problems? I did using an AMD 7850 on Linux with gcc 4.8. The few examples I tried, compiled and ran fine. 6. How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? I went over the documentation, I glanced over the code and ran a few examples. 7. Are you knowledgeable about the problem domain? I'm in the HPC field. I have extensive experience with MPI, OpenMP, pthreads, and less with TBB, CUDA and OpenCL. 8. Do you think the library should be accepted as a Boost library? This will be a maybe. It is a well-written library with a few minor issues that can be resolved. However, why would someone use Boost.Compute against what is out there? Average users can resort to Bolt or Thrust. Power users will probably always try to hand-tune their OpenCL or CUDA algorithm. How can we test it and prove its performance? Regards, Yiannis

5 7

[compute] Review results
by Antony Polukhin 01 Jan '15

01 Jan '15

Hi all, Extended review period for Boost.Compute library has ended. Here are the review votes for inclusion (+ means include, - means do not include, +/- means conditional include or "probably include, not quite sure"): + Denis Demidov + Sebastian Schaetz + Pavan Yalamanchili - Gruenke,Matt (+ if more backends will be supported) + Asbjørn (mini-review/comments from a library user) + Thomas M (a lot of suggestions) + Paul A. Bristow + Belcourt, Kenneth + John Bytheway +/- Yiannis Papadopoulos (there's already Bolt/Thrust for beginner and pure OpenCL for pros) +/- Vicente J. Botet Escriba (better docs and more performance tests) +/- Jason Newton (there was no explicit answer to the question "shuld be this library accepted into Boost?" but the review was positive in general) To sum up: Compute library becomes part of the Boost. Great thanks to all for the reviews and comments! Huge work was done and the issues raised are essential for library success. P.S.: Happy new year everyone! P.P.S.: Hope I did not miss some review! In case of any error, please let me know! -- Best regards, Antony Polukhin

1 0

[compute] review
by Vicente J. Botet Escriba 01 Jan '15

01 Jan '15

Hi, I have no time to do a complete review. Anyway here it is my minimal review *1. What is your evaluation of the design?** *** I would prefer that the algorithms take as first parameter the queue (which could be abstracted to an asynchronous executor). The performances let think that there is no interest in using the library for containers having less than 10^4/10^5 elements. Even if this is considered a low level library, the documentation should show how a library writer could make use of the provided interface to provide a higher one. I would prefer that the library make use of std::future or boost::future class instead of adding an additional one. Do the author tried to make use of the existing futures? if yes, which problems were found? I'm missing a range interface to most of the algorithms. I suspect that as this is a C++03 library it doesn't take advantages of some interesting C++11+/C++14 features. It is not clear which operations are synchronous, which ones could block, ... In order to get the native OpenCl handle I suggest to name the function get_native_handle instead of get. *2. What is your evaluation of the implementation?* I've not see at the implementation. *3. What is your evaluation of the documentation?* The documentation lacks of some core tutorials. I was unable to have an idea of what a context is, what can be done with a command_queue, what is a Kernel, a pipe, ...? Does it mean that the user must know OpenCL? The reference documentation should follow he C++ standard way, including pre-conditions, effects, synchronization, throws clauses. I would appreciate a description of the algorithms used, how the parallelism is used? is this completely managed by OpenCL or do the library something else? *4. What is your evaluation of the potential usefulness of the library?** *** This is essentially an optimization library. It should be quite useful, but the performances results don't probe it. *5. Did you try to use the library? With what compiler? Did you have any problems?** * No.* *** *6. How much effort did you put into your evaluation? A glance? A quick reading? In-depth study?** *** A glance. *7. Are you knowledgeable about the problem domain?* Yes and not. I don't know about OpenCL, but I'm aware of the current parallel proposals. *8. Do you think the library should be accepted as a Boost library? ** * As I have not taken too much time I'm unable to say that this library should not be included in Boost. So I vote to include it conditionally, subject to: * I would need some performances that probe the library is better than using standard STL containers in most of the cases. * The documentation should be completed with more tutorials and a better reference documentation. Best, Vicente

4 4

[compute] review
by Gruenke,Matt 31 Dec '14

31 Dec '14

> 1. What is your evaluation of the design? I agree with other comments made about synchronization. The design should be more explicit about what's asynchronous, and ideally utilize only one mechanism for synchronization. One approach that would have made this clearer is if high-level operations were actually objects that got posted to the command_queue. Regarding synchronization, I'm a also bit concerned about the performance impact of synchronizing on all copies to host memory. Overuse of synchronization can easily result in performance deterioration. On this point, I think it might be worth limiting host memory usable with algorithms, to containers that perform implicit synchronization to actual use (or destruction) of results. Give users the choice between that or performing explicit copies to raw types. Also, I agree with Thomas M that it'd be useful for operations to return events. That said, if you told me it doubled the overhead involved in dispatching operations, I'd suggest to make it optional through overloads or via a mechanism in the command_queue itself. > 2. What is your evaluation of the implementation? It seems a bit heavy to be header-only. I'd agree with Andrey Semashev, regarding thread safety, except I think it shouldn't be an option at all. That said, not all operations on all objects need be thread safe. I think the Boost.ASIO documentation provides a good example of how to express different thread safety limitations. Now, I haven't examined the use of thread-local storage, but do you feel there's a strong case to be made for the thread safety it's providing (especially, given that you seem to feel it's non-essential)? I'm a bit concerned about the use of type names ending in '_', such as float4_. Is this consistent with Boost conventions? I've only seen that used in other Boost libraries to denote class member variables. I didn't notice any unit tests specifically intended to verify exception-safety. I'd want to know that had been thoroughly vetted, before I'd consider using Boost.Compute in production code. Finally, based on the docs, it seems there's a bug in boost::compute::command_queue::enqueue_copy_image_to_buffer() and boost::compute::command_queue::enqueue_copy_buffer_to_image(). The region and origin parameters should be arrays, in the 2D and 3D cases. If this discrepancy is intentional, it should be noted. I haven't exhaustively reviewed the reference docs, so there might be other issues of this sort. > 3. What is your evaluation of the documentation? It needs more and better-documented tutorials (and I'm including the "Advanced Topics", here). Some of the reference pages could use more details, as well, especially concerning synchronization and exception usage. Regarding exceptions, there should be discussion of specifically what is invalidated by which kinds of errors. If any errors are non-recoverable, this should also be called out and perhaps derived from a different or additional baseclass (not a bad use case for multiple inheritance, actually). I agree with Mathias Gaunard that it would be helpful to discuss computational complexity, device memory requirements, and the various methods used to implement some of the algorithms. You could continue to omit these details on the simple algorithms, though. > 4. What is your evaluation of the potential usefulness of the library? This is my biggest misgiving, by far. In the very near future, I expect developers will opt for either SYCL (https://www.khronos.org/opencl/sycl) or Bolt (https://github.com/HSA-Libraries/Bolt) SYCL provides a modern, standard C++11 wrapper around OpenCL, with better concurrency control and support for integrating kernels inline. Bolt provides many of the same higher-level abstractions found in Boost.Compute, but with forthcoming support for HSA. To have the kind of lasting relevance and broad applicability to which all Boost libraries should aspire, I think Boost.Compute should be architected to support multiple backends. Though OpenCL support is currently ascendant, it's far from universal and is already flagging on some platforms (Nvidia, not the least). And HSA provides a foundation on which alternatives are actively being built. Most importantly, there exist multitudes of multi-core and multiprocessor systems which lack OpenCL support. It would be eminently useful to support these with such backends as thread pool, OpenMP, etc. And backends could be added to support new technologies, as they mature. > 5. Did you try to use the library? With what compiler? Did you have any problems? I downloaded and inspected the sources and some examples. I didn't have any questions or concerns that would be addressed by experimentation. > 6. How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? Review of the documentation, some sources, some examples, and a refresher on SYCL & Bolt. I also read all of the reviews and replies sent thus far. > 7. Are you knowledgeable about the problem domain? I've been developing production code for parallel and heterogeneous computing platforms for the majority of the last 16 years. I've followed OpenCL since version 1.0 was finalized. I've also submitted bug reports and minor patches for Khronos' official C++ OpenCL interface. > 8. Do you think the library should be accepted as a Boost library? Be sure to say this explicitly so that your other comments don't obscure your overall opinion. Not in its current state. The primary issue is that it's too OpenCL-centric. It should support more backends, even if these are just compile-time options. This is crucial for both current and lasting relevance. Beyond that, better concurrency control is crucial for optimum performance. If done well, it could even help further reduce usage errors. Matt ________________________________ This e-mail contains privileged and confidential information intended for the use of the addressees named above. If you are not the intended recipient of this e-mail, you are hereby notified that you must not disseminate, copy or take any action in respect of any information contained in it. If you have received this e-mail in error, please notify the sender immediately by e-mail and immediately destroy this e-mail and its attachments.

13 63

Boost.Build files at superproject root
by Peter Dimov 31 Dec '14

31 Dec '14

Currently, the Boost root contains the following files: boost-build.jam boostcpp.jam boostcpp.py bootstrap.bat bootstrap.sh Jamroot which are part of Boost.Build and so their rightful place is in the submodule. Would it be possible to move them there, and leave only minimal forwarding shims (that would never need to change) in the superproject?

3 5

[compute] Review
by John Bytheway 31 Dec '14

31 Dec '14

Here's my review. 1. What is your evaluation of the design? The design of building on OpenCL seems sound, for the reasons Kyle has explained. The fact that it has real users is very encouraging. I don't really like the name (I was unable to guess what the library was about from the name), but it's not worth changing at this point. I am worried about the compile-time configuration with macros. I would hope that BOOST_COMPUTE_HAVE_THREAD_LOCAL could be read from Boost.Config. For the two features which require linking against other Boost libraries, my instinct would be to have them on by default, rather than off by default (that should be easier if and when Boost.Compute is part of the Boost distribution). It would be very nice to have range versions of all the algorithms. I find myself using the boost::range algorithms much more than the std:: ones these days. Similarly, it would be good to provide the safer 4-iterator versions of equal, is_permutation, mismatch (and any I've forgotten) as were added to C++14. Some minor things: - Consider renaming boost::compute::lambda to be consistent with one of the other placeholder-containing namespaces from std or Boost. (You can keep the old name as an alias for backwards-compatibility). - Why the default of .25 as the bernoulli_distribution parameter? - Did you consider using Boost.Predef for your version macro? 2. What is your evaluation of the implementation? Only looked at a couple of things while trying to understand the docs. 3. What is your evaluation of the documentation? Overall, not too bad. More tutorial would be good. In particular, something using some of the standard algorithms. Some concerns: - All the header links off <http://kylelutz.github.io/compute/boost_compute/reference.html> are broken. - All or most of the things in "See also" sections look like they ought to be links but aren't. - For all your function parameter documentation, the list of parameters seems to be sorted alphabetically by parameter name rather than in the order they are passed to the function. This is really weird... - As others have mentioned, some kind of complexity estimate or promise for the algorithms is very important. In particular it would be good to understand how much parallelism they take advantage of and how they ought to scale with device compute capability. - The warning about accumulate being slow for floats should be much more prominent. - <http://kylelutz.github.io/compute/boost/compute/copy.html> mentions copying from e.g. a std::list to a compute::vector. It would be nice if it gave some kind of indication as to how efficient we can expect such an operation to be. In particular, I guess it ought to do some kind of buffering on the host? - The way I normally copy into vectors is call v.reserve(...) and then copy to std::back_inserter(v). I tried that with a compute::vector and unsurprisingly it was very inefficient. I see you already warned about this in the push_back docs, but perhaps it should also be highlighted in the documentation of compute::copy. - Is there no version of inclusive/exclusive_scan or iota using an arbitrary associative operator? - <http://kylelutz.github.io/compute/boost/compute/includes.html> it's unclear whether the subrange has to be contiguous. - min_element and max_element should cross-reference minmax_element. - What source of randomness does random_shuffle use? - Why does reduce return its result through an OutputIterator? - <http://kylelutz.github.io/compute/boost/compute/remove.html> Does this really remove things or use std::remove semantics? You should clarify and document the return value. (Same with remove_if, unique) - I can see how sort_by_key might be useful, and I'm slightly surprised that there's no stable_sort_by_key (which seems like it would be more useful than stable_sort). Any particular reason? - The boost/compute/types headers docs seem to be broken or missing. In particular I couldn't find reference documentation for float4_ et al. - I was interested to see allocators, but the docs seem uninformative: <http://kylelutz.github.io/compute/boost/compute/pinned_allocator.html> - Given the confusion with std::future, you might want to clarify whether compute::future's destructor blocks. - Also, <http://kylelutz.github.io/compute/BOOST_COMPUTE_CLOSURE.html> doesn't explain what format the lists (arguments and capture) should be in. Preprocessor functions often take lists in other formats, such as a Boost.PP Sequence <http://www.boost.org/doc/libs/1_57_0/libs/preprocessor/doc/data/sequences.h…>, and your example only has one-element lists, so it's not enough to clarify this point. (To be clear: I'm not suggesting you should use Boost.PP sequences, merely that you should make it clear in the docs whether you are). - <http://kylelutz.github.io/compute/BOOST_COMPUTE_CL_CALLBACK.html> is empty, as is <http://kylelutz.github.io/compute/BOOST_COMPUTE_MAX_ARITY.html> - <http://kylelutz.github.io/compute/boost/compute/valarray.html> - presumably compute::valarray has overloaded arithmetic operators? Are they mentioned anywhere? - <http://kylelutz.github.io/compute/boost/compute/vector.html> the unusual behaviour of vector's size_t constructor (not initializing the elements) should be warned more prominently. - <http://kylelutz.github.io/compute/boost/compute/device.html> the documentation of the enum type in the synopsis seems to be messed up. Similarly memory_object. - <http://kylelutz.github.io/compute/boost/compute/function.html> I'm puzzled by this interface. What's the name for? Would one ever use this constructor rather than e.g. make_function_from_source? - <http://kylelutz.github.io/compute/boost/compute/field.html> My reading of this is that when you get the type wrong you are guaranteed some kind of meaningful error report, and not just undefined behaviour; is that right? Perhaps you should clarify. - I can't guess what atomic_min and atomic_max do. - The right arrow ('next') link from <http://kylelutz.github.io/compute/boost/compute/atomic_xor.html> is broken (looks like the docs for the placeholders don't work). - The iterators should state what iterator category they have. - I can't guess what svm_ptr is for. Later I discovered svm_alloc and svm_free. Is there an RAII way to manage these things? - What does default_device do if there are no devices? Do the various environment variables mentioned have any kind of precedence? (Probably linking to the corresponding OpenCL docs is sufficient for such questions). Do they have to be exact matches? Are they case-sensitive? - What does find_device do if the requested device does not exist? - Can you give examples for type_definition? My guess was that it did the thing which type_name later turned out to do... - BOOST_COMPUTE_ADAPT_STRUCT should state that it must be used at global scope (i.e. not within a function or other namespace). - Congratulations for getting compile-time detection of internal padding into BOOST_COMPUTE_ADAPT_STRUCT :). You might want to mention in the docs that this will be detected, so people don't worry too much. - It looks like the performance tests don't include the time for copying data to (and, where applicable, from) the device. You should mention this on the Performance page. - You should probably add C++AMP to your things compared with in the FAQ. 4. What is your evaluation of the potential usefulness of the library? High. I've wanted something like this in the past and not found this. 5. Did you try to use the library? With what compiler? Did you have any problems? I used it with g++ 4.8.2 in C++1y mode. Some quick tests seemed to work OK, and indeed I was able to sort some large vectors of scalars faster using boost::compute::sort than std::sort, which is nice. I was pleased to see that 64-bit integers were handled fine. 6. How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? Couple of hours reading and trying simple examples. 7. Are you knowledgeable about the problem domain? I have done a couple of small projects with CUDA, and am generally knowledgeable, but with no previous OpenCL experience. 8. Do you think the library should be accepted as a Boost library? Be sure to say this explicitly so that your other comments don't obscure your overall opinion. Yes. OpenCL seems to have fairly broad acceptance, but is rather difficult to use safely and with C++-isms. This library fills a useful niche. In an ideal world it will eventually be rendered obsolete by things like parallelism entering the standard library, but I doubt that will happen soon if at all. This provides a good solution now, and we shouldn't dismiss it simply because something sparkly is approaching, which may or may not turn out to be awesome and widely adopted. This library has clearly been a lot of work. Congratulations on making it this far, I hope the review is not too stressful :). John Bytheway

3 12