August 2015 - Boost - lists.boost.org

[AFIO] Formal review
by Antony Polukhin 02 Sep '15

02 Sep '15

> 1. Should Boost.AFIO be accepted into Boost? This library must be in Boost, but not in it's current state. Some polishing required and some improvements. Dependent libraries must be accepted in Boost/dropped/documented and reviewed as part of AFIO. After that another review must be done. So it's NO for now. > 2. What is your evaluation of the design? NONBACTH operations seem right and good. Some simplification must be applied, by possibly removing some of the overloads and reducing amount of types visible to user (`handle_ptr`, `*_req`). BATCH operations do not seem right. They must be removed or their interface must be greatly reworked. afio::path must be hidden from user or filesystem::path must be used (I'm not 100% sure, but it looks like `\\?\` paths are OK to be in filesystem::path http://www.boost.org/doc/libs/1_59_0/libs/filesystem/doc/reference.html#lon… ). More functions for everyday use may be provided (async_copy for example) > 3. What is your evaluation of the implementation? Header files must be split into multiple smaller ones. Multiple memory allocations must be removed from library internals. Implementation must be polished. Recommendation: ABI stable part of AFIO may be moved in a separate git branch to untie the AFIO library developer's hands. In that case new version of AFIO may break the ABI without affecting users and older versions could be removed from sources. > 4. What is your evaluation of the documentation? That's one of the biggest problems of the library. Documentation is not structured and sometimes overcomplicated. Some notes for simplification: * "AFIO single page cheat sheet for the very impatient" -> "Getting started". Move table to the top of the page, replace the "Related types" column with "Boost.Filesystem sync alternative", replace numbers with checks/+/-. * Move the "Example mini-programs written using AFIO" chapter right after the "Getting started", rename chapter into "Tutorial", refactor second example to not use batch operations and to use only free standing functions. Third and fourth examples must be greatly simplified and probably split into smaller examples. * Move the content of "Step by step workshop building an AFIO-based key-value store" into the "Tutorial" after the previous examples, rename it into "Key-value store walkthrough", simplify and make shorter if possible * "Compilation" section must be somewhere around "Getting started" * Other docs must be after the "Getting Started," "Tutorial" and "Compilation" * "Reference" section must be refactored. All the overloads must be probably described and showed on a single page (all the overloads, including those that explicitly accept dispatcher) * Shorten and simplify the examples wherever possible. > 5. What is your evaluation of the potential usefulness of the library? Very very useful. > 6. Did you try to use the library? No. > 7. How much effort did you put into your evaluation? About 20 hours. > 8. Are you knowledgeable about the problem domain? I have a fair amount of experience with asynchronous programming in C++ and I have pretty extensive experience with designing interfaces for async operations. Have not dealt with async fiesystem operations before. -- Best regards, Antony Polukhin

2 1

[math] lcm errors
by Damian Vicino 31 Aug '15

31 Aug '15

Hi, I found out, lcm (RT) is silent about errors. (the bad way) Is there any way to activate them? For example, the following code: #include<iostream> #include <boost/math/common_factor.hpp> int main () { int a = 1<<30; int b = 3; int c = boost::math::lcm(a, b); std::cout << "lcm of :" << a << " & " << b << " is " << c << std::endl; return 0; } Obtains the following result: lcm of :1073741824 & 3 is 1073741824 Which is not the right answer (because it is not divisible by 3), but neither it raised an exception or returned some special value. Is there any way I can make it raise an exception? I guess I could check everywhere it as: if (c%a != 0 || c%b != 1) throw std::exception(); But probably there is something implemented that I couldn’t find in the documentation. Best regards, Damian

2 1

[asio] MSVC compiler error when compiling urdl
by Christian Henning 31 Aug '15

31 Aug '15

So URDL (https://github.com/chriskohlhoff/urdl ) is not part of boost, I know. But a nasty MSVC 2013 compiler error prevents the most simple code to compile. #define _WIN32_WINNT 0x0601 #define URDL_HEADER_ONLY 1 #define URDL_DISABLE_SSL 1 #include <urdl/istream.hpp> #include <boost/progress.hpp> #include <iostream> #include <fstream> int main(int argc, char* argv[]) { return 0; } The fix is relatively simple but it would change code is asio's handler_invoke_helpers.hpp. Basically change: using boost::asio::asio_handler_invoke; asio_handler_invoke(function, boost::asio::detail::addressof(context)); to boost::asio::asio_handler_invoke(function, boost::asio::detail::addressof(context)); I'm using VC2013. I'm using master branch of boost. Regards, Christian

2 1

Re: [boost] [afio] Formal review of Boost.AFIO
by Sebastian Theophil 31 Aug '15

31 Aug '15

-- Dr. Sebastian Theophil | stheophil(a)think-cell.com<mailto:stheophil@think-cell.com> Senior Software Engineer We are looking for C++ Developers: http://www.think-cell.com/career ________________________________ think-cell Software GmbH http://www.think-cell.com<http://www.think-cell.com/> Chausseestr. 8/E phone / fax +49 30 666473-10 / -19 10115 Berlin, Germany US phone / fax +1 800 891 8091 / +1 212 504 3039 Amtsgericht Berlin-Charlottenburg, HRB 85229 | European Union VAT Id DE813474306 Directors: Dr. Markus Hannebauer, Dr. Arno Schödl On 27 Aug 2015, at 04:45, boost-request(a)lists.boost.org<mailto:boost-request@lists.boost.org> wrote: Ok, let's revisit the original pattern code I mentioned: EXAMPLE A: shared_future h=async_file("niall.txt"); // Perform these in any order the OS thinks best for(size_t n=0; n<100; n++) async_read(h, buffer[n], 1, n*4096); Niall, is parallel read (parallel writing maybe?) the only use-case where you want a shared_future? If I understand Thomas correctly, he doubts you need the shared_future semantics because one async operation hands down a single handle to the next continuation. Essentially something like: async_file(“niall.txt”) .async_read(buffer, length_to_read) .async_truncate(length_to_read) .async_close() Your counter example was an asynchronous *and* parallel read where you need to share the file handle (or rather the future<handle>) between parallel reads. Shouldn’t this be abstracted away in the API somehow? I can’t think of many file operations you want to do N times in parallel. Truncating a file in parallel several times doesn’t seem to make much sense :-) So why not make it: async_file(“niall.txt”) // Read 100 times asynchrnously and in parallel and provide lambda returning n-th buffer and offset: .async_parallel_read(100, [&](int nCount) { return std::make_pair(buffer[n], n*4096; }) .async_truncate(length_to_read) .async_close() The 100 reads are internally not ordered but they can only begin once the file has been opened, they consume this handle together, and only after all reads are complete can we truncate. Is this not what you’re trying to do? Regards Sebastian

3 7

[AFIO] Review (or lack of it)
by Roland Bock 31 Aug '15

31 Aug '15

Hi, I vote for "No, the library should not be accepted". I do not vote this way because the library is bad. I do so because I have no way of telling. a) There is so much going on about personal hardships (e.g. hundreds of hours of family time and 12-4am sessions and fried disks and no electricity and suspected vendettas and supposedly useless reviews). None of this has anything to do with a library review. Why mention it, even if you feel it? All of this is bound to make it hard to be impartial. b) AFIO depends on other libraries which are probably going to be proposed for boost, but are not in yet. Regardless of whether there is a precedent or not, this is far from being ideal. Judging by the discussion about "Monad", getting that library into boost is not just a mere formality. Thus, Monad implicitly IS part of the review, because if it is not accepted (which we don't know today), it will become part of AFIO. Thus, the scope of the review is not defined correctly. c) There are older versions in there. There is 1.3 and 1.4 at least (are there more?). Please choose one. Go for it. Of course, boost libraries evolve over time. There will be new versions of the interface. But starting with that kind of baggage? And asking for a review that necessarily has to cover everything that is currently available? d) The implementation of 1.4 will be replaced completely. Thus, the implementation (which I consider an important part of the review), cannot be reviewed in any way that is worth anyone's time. e) Documentation * Please drop the "using namespace XYZ" from all examples. Maybe use namespace aliases. * The "cheat-sheet" for the impatient is beyond me. I have no idea what it is trying to convey. * I would certainly prefer ONE page with planned features, not sprinkled info here and there (especially not on the cheat sheet). * Redundancy: For example, all the dir() functions share the same example, don't they? I haven't done a diff and I won't. But it makes me nervous to see the same example over and over again. Why not make one page explaining all the dir() functions and then use ONE example? Also the "Note that ..." text seems to be the same on all pages. This piles up to wonderfully large amounts of documentation, but I have no intention to reading all of that ever repeating stuff trying to find the needle in the haystack that actually contains different information. This has to be condensed. * There are deprecated functions? Again, I strongly recommend to start without the extra baggage. * The mix of history and rationale is hard to read, IMO. I'd be interested in the rationale, but not in the many steps it took to get there. * I do not get the world's simplest blob store in the STL form. It is most definitely not the simplest blob store. Not even the simplest persistent blob store. Why would I need a monad there? Why is there a writable flag (why isn't constness enough to prevent writing)? Why isn't the directory created at construction once and for all? BTW: The name validation fails to check for empty string. * https://boostgsoc13.github.io/boost.afio/doc/html/afio/quickstart/workshop/… mentions AFIO v1.40 (thats 36 numbers in the future). If the pictures are not loaded, the performance numbers in the table do not say what they are (the per sec is missing). * "The /final/ named blob store design" starts with "Caution: This section was not finished". * "The performance of and problems with this third generation design" consists of "This section was not completed in time[...]" * So basically the crucial parts of the blob store sequence are incomplete at best. f) Code (since the implementation is going to change anyway, this might all be useless): * /attic: Seriously? * I am not a big fan of #if 0 in release code * I doubt that std::cerr should be in this library's code. * I wonder why there really is a need to have your own path stuff on top of boost::filesystem. If boost::filesystem is insufficient, wouldn't it be better to contribute to that one? Haven't looked too closely, though. * I looked briefly into detail/. I would not want to look for a bug there. I have not done any detailed analysis to base this on, but: I would assume that with some refactoring (moving the platform specific parts into platform specific headers), the readability could be increased dramatically. Again: This is just a guess. In the current situation, I cannot do a formal review that would result in acceptance. I recommend to treat the current review period as a pre-review and * Do a review of Monad (or whatever it will be named) first, maybe API Bind, too, depending on whether that is going to be used in AFIO * Remove all old versions, everything deprecated and do the implementation change that is planned * Cleanup the documentation. Get it read by people with different knowledge of the problem domain. I am willing to read it as somebody who is interested but certainly not very knowledgeable. * Let the dust settle a bit * Start a new review * Even if you believe in vendettas, stay calm and answer in a matter-of-fact way. My experience with the boost list is, that there are strong voices here which have a very finely tuned sense of fairness. I did invest several hours in this review, reading the mail threads, reading the documentation, looking through some code, writing this mail. As stated above, I do consider myself interested but not very knowledgeable in the problem domain. I would like to see a library for asynchronous file IO in boost, but I don't think AFIO is ready yet. Regards, Roland

12 41

[afio] running tests
by Michael Caisse 30 Aug '15

30 Aug '15

Hi Niall - I'm trying to run the tests. I'm certain I am just doing something really silly... input is welcome! I have dropped the AFIO repo into a libs/afio directory for the super project. I ran ./b2 headers which linked in the boost/afio bits. I then go to libs/afio/test and execute "b2 release" which unfortunately results in: find: libs/afio: No such file or directory Performing configuration checks ... This is the same way I run other library tests and those are working as normal. Is there some other setup step that I'm forgetting? Thanks! michael -- Michael Caisse ciere consulting ciere.com

4 4

AFIO review response
by Hartmut Kaiser 30 Aug '15

30 Aug '15

This is my (sadly very short) AFIO review. I vote NO (to not accept) the library into Boost as I'm not able to tell whether the library is useful and fit for Boost. Reason: I tried to understand the design and rationale of the library and asked a number of questions on this list to clarify things I didn't understand. Sadly, the author decided not to answer a single of my questions (not even did he directly respond to any of my emails). I'm not able to make an educated decision on any of the review criteria without having the questions answered which I asked. The reason I send my review now is that I have to make this decision today. I'll not be able to spend any more time on this before the review period ends. So far, I have invested a couple of hours reading through the documentation, following the email threads, and thinking about the design I saw. Regards Hartmut --------------- http://boost-spirit.com http://stellar.cct.lsu.edu

12 20

[android][regression] bjam failed
by Dmitry Moskalchuk 30 Aug '15

30 Aug '15

Hello, Today I've experienced problems with Android regression tests on 'develop' ('master' is OK, and 'develop' was OK too for long time): ERROR: Bjam failed (see /home/cislave/.crystax/ci-boost/regression/android-21-armeabi-v7a-hard-develop/test/develop/regression/armeabi-v7a-hard/test/results/bjam.log for full details): - BOOST_OS_LINUX == 0 : yes - BOOST_COMP_GNUC >= 4.7.4 : no - BOOST_COMP_GNUC >= 4.3.0 : no - BOOST_COMP_GNUC >= 4.3 : no - BOOST_OS_QNX == 0 : yes - zlib : yes warn: Unable to construct ../libs/predef/test/info_as_objcpp warn: Unable to construct ../libs/predef/test/info_as_objc error: Name clash for '<p/home/cislave/.crystax/ci-boost/regression/android-21-armeabi-v7a-hard-develop/test/develop/regression/armeabi-v7a-hard/test/boost_root/bin.v2/libs/dll/test>libtest_library.so.1.59.0' error: error: Tried to build the target twice, with property sets having error: these incompatible properties: error: error: - <cxxflags>-Wall -Wextra -pedantic -Wno-long-long <library>object(file-target)@57008 <library>object(file-target)@57040 <toolset-gcc:version>4.9 <toolset>gcc <xdll-path>/home/cislave/.crystax/ci-boost/regression/android-21-armeabi-v7a-hard-develop/test/develop/regression/armeabi-v7a-hard/test/results/boost/bin.v2/libs/filesystem/build/gcc-4.9/release/abi-aapcs/target-os-android/threading-multi <xdll-path>/home/cislave/.crystax/ci-boost/regression/android-21-armeabi-v7a-hard-develop/test/develop/regression/armeabi-v7a-hard/test/results/boost/bin.v2/libs/system/build/gcc-4.9/release/abi-aapcs/target-os-android/threading-multi error: - <library>object(file-target)@332654 <library>object(file-target)@332686 <toolset-clang:platform>linux <toolset-clang:version>3.6 <toolset>clang <xdll-path>/home/cislave/.crystax/ci-boost/regression/android-21-armeabi-v7a-hard-develop/test/develop/regression/armeabi-v7a-hard/test/results/boost/bin.v2/libs/filesystem/build/clang-linux-3.6/release/abi-aapcs/target-os-android/threading-multi <xdll-path>/home/cislave/.crystax/ci-boost/regression/android-21-armeabi-v7a-hard-develop/test/develop/regression/armeabi-v7a-hard/test/results/boost/bin.v2/libs/system/build/clang-linux-3.6/release/abi-aapcs/target-os-android/threading-multi error: error: Please make sure to have consistent requirements for these error: properties everywhere in your project, especially for install error: targets. Does anyone know what that means and how to fix it? I suspect this is result of recent update related to Boost.Dll, but I don't know if that's error in our scripts, preparing testing environment, or error in Boost itself. -- Dmitry Moskalchuk

1 0

[Proposal] BOOST_NO_GLOBAL_FILESYSTEM_NAMESPACE
by Ed Schouten 30 Aug '15

30 Aug '15

Hi there, I'm currently in the process of porting the Boost libraries to a new platform called CloudABI. CloudABI is a POSIX-like runtime environment that is purely built on the concept of capability-based security. In a nutshell, it means that actions that have side-effects outside of a process can only be performed if the process owns a capability (a file descriptor). In other words, open() is missing, but openat() is still there. There are only local filesystem namespaces. CloudABI allows you to reduce the impact of security exploits through sandboxing, but also makes it possible to safely run untrusted third-party applications without requiring any virtualization, containers, etc. Be sure to check out the GitHub page for more details: https://github.com/NuxiNL/cloudlibc The other day I started experimenting with getting Boost working on CloudABI and I noticed that most of the Boost sources already build on CloudABI. I somewhat expected this, because Boost sources tend to be written in a loosely coupled, testable way. But still, hats off to the Boost developers! The biggest showstopper -- as expected -- is boost::filesystem, which will likely never work on CloudABI: https://groups.google.com/a/isocpp.org/forum/#!msg/std-proposals/q1F0KDEBtf… But in many cases there are also classes that can optionally access the filesystem through utility functions (boost::locale::gnu_gettext, boost::program_options::parse_config_file(), etc). The reason why I'm sending this email is because I want to propose adding a new configuration option, BOOST_NO_GLOBAL_FILESYSTEM_NAMESPACE, that I can use to conditionally disable such utility functions. I've attached a patch for Boost.Config to this email. Does this approach seem reasonable? Do we like the name of this build option? Does the patch look all right? It seems as if generate.cpp also made some unnecessary changes to these files. What's up with that? Best regards, -- Ed Schouten <ed(a)nuxi.nl> Nuxi, 's-Hertogenbosch, the Netherlands KvK-nr.: 62051717

1 1

Re: [boost] [afio] AFIO review postponed till Monday
by Glen Fernandes 30 Aug '15

30 Aug '15

Did the review start yesterday? Glen

18 84