- Boost - lists.boost.org

Recommended Boost file structure?
by Paul A. Bristow 07 Mar '16

07 Mar '16

I've been looking at the structure of Phil Bouchard's prototype block_ptr library and am wondering if the structure of the header files is optimal/Boost_Standard/wrong. http://www.boost.org/doc/libs/1_60_0/more/getting_started/windows.html describes the Boost structure, but looks well out of date to me. (Can/should anyone still update this to reflect Modular Boost?) And looking through libraries, there seems some variation. I'm asking for views on what the best structure should be, (or if it doesn't matter much?) Some libraries have a structure modular_boost/libs/algorithm/include/boost/algorithm which contains algorithm.hpp and string.hpp that one might use directly. Others like asio contain /asio and just a asio.hpp that includes (all?) other headers. Others like multiprecision contain just a single folder modular_boost/libs/algorithm/include/boost/multiprecision but no other header files, not even forwarding header(s). So should the structure be that all headers (except forwarding headers - or a single header) are within modular_boost/libs/some_library/include/boost/some_library/ so the user should include with #include <boost/some_library/some_header.hpp #include <boost/some_library/other_header.hpp ... or to get everything useful #include <boost/some_library/some_library_fwd.hpp> possibly with a /detail folder containing headers that should not be called by users. Just checking... Paul --- Paul A. Bristow Prizet Farmhouse Kendal UK LA8 8AB +44 (0) 1539 561830

2 4

Re: [boost] [Boost-users] [Fit] formal review - should we propose some parts to Boost.Config/Boost.Core
by paul Fultz 07 Mar '16

07 Mar '16

> On Saturday, March 5, 2016 10:50 PM, Steven Watanabe <watanabesj(a)gmail.com> wrote: > > AMDG > > On 03/05/2016 07:21 PM, Vicente J. Botet Escriba wrote: >> >> I want to start a new sub-thread about some of the concerns of Steven >> Watanabe about whether some of the contents of this library fits better >> in Boost.Config. In particular the file boost/fit/returns.hpp. >> > > When I mentioned Boost.Config, I was talking about > things like > > #ifndef BOOST_FIT_NO_EXPRESSION_SFINAE > #ifdef _MSC_VER > #define BOOST_FIT_NO_EXPRESSION_SFINAE 1 > #else > #define BOOST_FIT_NO_EXPRESSION_SFINAE 0 > #endif > #endif This is can be configurable, whereas Boost.Config it is not. > > or > > #ifndef BOOST_FIT_HAS_TEMPLATE_ALIAS > #if (defined(__GNUC__) && !defined (__clang__) && __GNUC__ == 4 > && > __GNUC_MINOR__ < 7) > #define BOOST_FIT_HAS_TEMPLATE_ALIAS 0 > #else > #define BOOST_FIT_HAS_TEMPLATE_ALIAS 1 > #endif > #endif > This could be replaced with Boost.Config, however, I have ran into problems before with template aliases on gcc 4.7, so I wanted to make this configurable as well, although it could fallback onto Boost.Config right now. > In Christ, > > Steven Watanabe > > _______________________________________________ > Boost-users mailing list > Boost-users(a)lists.boost.org > http://lists.boost.org/mailman/listinfo.cgi/boost-users >

3 2

[MMap/VM] RFC
by Domagoj Saric 06 Mar '16

06 Mar '16

Hello everyone, for quite some time I've been working on a portable mmap/virtual memory library and I've finally found enough time to bring it out of the 'my internal litle tool' state into something presentable: https://github.com/psiha/mmap (C++14 currently). I would now kindly ask fellow devs for opinions (on the good, the bad, the ugly and the future;) First let me answer the basic questions (i.e. on motivation and scope)... Q&A: (1) Why? Considering Boost already offers two related solutions * Interprocess: http://www.boost.org/doc/libs/release/doc/html/interprocess/sharedmemorybet… * Iostreams: http://www.boost.org/doc/libs/release/libs/iostreams/doc/classes/mapped_fil… why do we want a new one? Even if the two mentioned solutions were adequate, the problem domain would still merit a separate, dedicated library merely considering its complexity (which will become apparent in later points). (I suspect Ion would agree here considering he actually authored a related standardization proposal:) (2) Why a completely new library instead of a repackaging of existing functionality? I'm not satisfied with (a) the API semantics/design, (b) the API 'power'/library capabilities (c) and the implementation efficiency/overhead. a) Insistence on POSIX semantics: that shared memory objects have to be persistent (kernel lifetime) and resizable while not everyone wants or needs this (https://svn.boost.org/trac/boost/ticket/4827) This has implications both: - in the interface (requiring manual cleanup guards, additional platform specific shm types like windows_shared_memory) - and the overhead of the library (emulating shm with mapped files on major platforms that do not offer full POSIX compliance: Windows, OSX, iOS and Android) MMAP solves this with one class template 'named_memory' and policies: https://github.com/psiha/mmap/blob/master/include/boost/mmap/mappable_objec… so the user can choose: named_memory<scoped, resizable>, named_memory<persistent, fixed> or any other combination and the library chooses the best implementation. On POSIX, scoped semantics are achieved with a SEM_UNDO-enabled SysV semaphore acting as a kernel/system-global reference counter (which is then also used for automatic cleanup of shm zombies left by killed/crashed processes). On Windows the user can choose the Win32 backend (file emulation) or the NativeNT (not yet finished/commited) backend (with native persistence and resizability). b) - lack of a meta layer (e.g. so one can ask is_mappable<FILE *> or is_mappable<boost::filesystem::path>, is_mappable<HANDLE> etc...) - lack of related utility functions, for example: -- map_read_only_file( path ) which will open the file for reading, query its size, create a read only file mapping and map/return a read only view of the file -- guarded_operation( view, operation, error_handler ) - execute operation wrapped in SEH(Windows)/scoped signal handler(POSIX) guards that will catch access violations (e.g. when mmaping network files) and gracefully execute error_handler (with the faulting address as the parameter) https://github.com/psiha/mmap/blob/master/include/boost/mmap/mapped_view/gu… - in MMAP the mapped_view object (which should probably be renamed to just 'view' considering the enclosing mmap namespace) is just a RAII wrapper around an iterator_range therby providing the standard begin/end(), front/back(), operator[], etc. interface (as opposed to a get_data(), get_size() like interface) - general (not specific to mapping) virtual memory functionality (to be discussed in a separate point, MMAP also completely lacks this currently) - MMAP offers a comprehensive 'flags' system (e.g. https://github.com/psiha/mmap/blob/master/include/boost/mmap/flags/flags.hpp) for everything from object-level and system-level access privileges, over object parent-child inheritance to system life-time and access-pattern optimisation hints. Quite some time was invested in this area to produce a normalized interface that works for all objects (files, mappings...) while producing (near)zero adjustment codegen. Flags are 'packed'/grouped in structs (e.g. struct access_privileges with object_access, child_access and systen_access members) with public members so that, after flags are created with a factory function (a portable API) the flags can be further tweaked for a specific platform (e.g. adding some FreeBSD specific mmap hint which isn't covered by the portable/documented MMAP API) c) It is my view (if not a "self evident truth";D) that libraries that merely wrap existing low level functionality (such as OS or CRT APIs) should allow you to write code that is safe, portable and looks reasonably nice while at the same time incurs (near)zero overhead (i.e. with a reasonably intelligent compiler, produces codegen that looks nearly the same as it would had one used the underlying API directly) - and the existing solutions fail that. In the two related tickets I went into more detail on this so I'll avoid spaming this post by repeating those objections and analysis. Rather I'll present a trivial example that demonstrates what wannabeboost::mmap currently produces with a decent compiler, https://gist.github.com/psiha/c0823fefc01fa3b39662: #define BOOST_MMAP_HEADER_ONLY #include <boost/mmap/mapped_view/mapped_view.hpp> #include <boost/mmap/mappable_objects/file/utility.hpp> int main( int /*argc*/, char * /*argv*/[] ) noexcept { auto maybe_foo_view( boost::mmap::map_read_only_file( "foo" )() ); if ( !maybe_foo_view ) return static_cast<int>( maybe_foo_view.error() ); if ( maybe_foo_view->empty() ) return -1; return (*maybe_foo_view)[ 0 ]; } with Xcode 7.2.1 Clang -O3 build for x64 produces https://gist.github.com/psiha/f92a7b8a93c5ce1736ae (notice how the error handling is also correctly detected as such and placed at the end of the function, after the main return...) Besides not using shared_ptr pimpls or saving paths in std::strings as a 'nice to have' (YAGNI!:), part of the way this codegen is achieved is through the use of (also wannabe) Boost.Err (https://github.com/psiha/err) which makes it possible to avoid EH (in such simple/'localised' examples). It is also the reason for the somewhat awkward (optional<>-like) syntax: - map_read_only_file() returns a mmap::fallible_result<mmap::mapped_view> (an alias for err::fallible_result<mmap::mapped_view, mmap::error>, an "rvalue-only" type) - which is converted/'saved'/'pinned' into a result_or_error (the maybe_foo_view variable) with the additional operator() call - the !maybe_foo_view checks whether the call succeeded or the returned object contains an error - if error return the error (errno) code - else check if the view/file is empty (optional<>-like syntax for accessing the contained object through the -> and * operators) - else return the value of the first character in the file. Boost.Err was recently discussed on this list so I'll skip most of that now, let me just say that it also supports classic EH coding style (it auto adapts, no need for reconfiguration, macros or anything like that), i.e. the above code can be rewritten as: int main( int /*argc*/, char * /*argv*/[] ) noexcept { using namespace boost::mmap; try { read_only_mapped_view const foo_view( map_read_only_file( "foo" ) ); if ( foo_view.empty() ) return -1; return foo_view[ 0 ]; } catch ( std::runtime_error const & ) { return error::get(); } } ...now let me reverse the Q&A direction ;) 3. Library scope: currently the library covers the topic of memory mapping (of filesystem objects and virtual memory), however I think that 'the final' library should cover all resonably portable aspects of virtual memory (and be called something like vm with mmap as a nested namespace), tackling thingies such as: - process working set - portable low memory event handling, madvise, memfd... - prefetching, locking/unlocking virtual memory to/from physical memory (e.g. for realtime sensitive data) - allocators capable of contiguous resizing (for implementing realloc or vector.resize() that does no copying or moving, simply maps new pages at the end of the current allocation) - https://fgiesen.wordpress.com/2012/07/21/the-magic-ring-buffer ... does anyone have an objection/different approach to this? 4. Windows offers/has/uses the concept of an intermediate "mapping" object (i.e. you don't create a mapped view of a file, rather you create a 'mapping' of a file and then a view of the mapping). This of course complicates things but also gives more power, e.g. you can have a r/w file open and create a mapping (or several mappings) of it that only covers a part of its size and has stricter (e.g read only at object/process level) or wider access privileges (e.g. on the system level, only the parent process/user can access the file but the mapping is accesible by everyone). Boost.MMAP retains this distinction in its API (mapping vs mapped_view classes)...comments/thoughts on this please? 5. The above (Windows specific) 'mapping' concept makes it possible to create "named file mappings" (so that the object gets a system global name, like a shared memory object) which then another process can open by its name, erasing the difference between file mappings and vm mappings for client processes (kind of like the interface vs hidden implementation distinction). MMAP uses this on Windows for file-backed/'emulated' shared memory (e.g. that which needs persistence and/or resizabilty) - it is created named so that client processes can open and access it as 'normal'/native Windows shared memory. It might actually be possible to make this at least partially portable to POSIX systems that use virtual filesystems for implementing shared memory (e.g. Linux which uses /dev/shm) where we could symlink the file to the shm filesystem directory...(unfortunately I have no way of testing this as I have no Linux machine setup, I develop only for Windows, OSX, iOS and Android)...does this make any sense/would it be worth the hassle (i.e. ease any real world problems)? 6. The security part of the 'flags system' models the POSIX API: for named (system level) created objects you specify the permissions for the 'user', 'group' and 'world'. 'Behind the scenes', on Windows, 'user' maps to the user that created the process, 'group' maps to all groups that the 'user' belongs to and 'world' maps to the Everyone group..? When not using the predefined privilege/permission levels [e.g. process_default, unrestricted, nix_default (644)] things are currently pretty verbose here: namespace flags = boost::mmap::flags; using ap = flags::access_privileges; auto const default_privileges ( ap::system::user ( ap::all ) | ap::system::group( ap::read ) | ap::system::world( ap::read ) ); I have a prototype implementation that shortens this with user defined literals, so that one can write something like "rwxr-xr--"_perm...All in all, this thing looks like it merits a separate library in its own right... 7. Windows has another quirk up its sleeve: global vs local session objects - if you want to create a shared memory object visible across terminal sessions (this includes a server process running as a service, thereby in session 0, and client process created by a logged on user, or/also using the NativeNT backend/API to create a native persistent shared memory object which requires that it be created in the global session '0') you have to prefix its name with "Global\" (and have admin privileges)... I'm still pondering how to model this in the API or whether it should be automatically handled at all, e.g. whether to deduce if the Global\ prefix should be automatically added (based on the process privileges, type and access privileges of shm object being created etc) or leave it up to the user to know "if I'm running on Windows and want to do this and this I have to use the Global\ prefix"... There's more but the openning post is already too long so I'l stop...for now ;) -- "What Huxley teaches is that in the age of advanced technology, spiritual devastation is more likely to come from an enemy with a smiling face than from one whose countenance exudes suspicion and hate." Neil Postman --- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

4 5

code naming conventions
by Michael Marcin 06 Mar '16

06 Mar '16

I've written a lot of code in the boost/standard library style. I've become increasingly annoyed at naming my variables when coding in this style. Looking to the standard library implementations for guidance isn't much help as they often use conventions like local variables are named with a leading double underscore and member variables are named with a leading underscore and capital letter. I.E. things I'm not allowed to do as I'm not a standard library implementer. Good names are generally taken by types leading to distress when trying to name objects and functions. Here's a concrete example: // type takes the most natural name enum class format { rgba, rgb, float // enum value wants reserved name }; class image { public: // getter wants the same name format format() const noexcept { return this->format; } // setter wants the same name void format( format format ) noexcept // parameter wants the same name { this->format = format; } private: // member variable wants the same name format format; }; format bits_to_format( int bits ) { // local variable wants the same name format format = format::rgba; if ( bits == 24 ) { format = format::rgb; } return format; } // unrestricted template parameter does *not* want the same name template< typename Format > std::ostream& print_format( std::ostream&, Format format ); // Concept template parameter *does* want the same name template< Format Format > std::ostream& print_format( std::ostream&, Format format ); What I end up doing right now is something like: // type takes the most natural name enum class format { rgba, rgb, float_ // add trailing underscore to reserved name }; class image { public: // prefix getter with get_ format get_format() const noexcept { return _format; } // prefix setter with get_ void set_format( format format_ ) noexcept // trailing underscore on parameter { _format = format_; } private: // leading underscore on member format _format; }; format bits_to_format( int bits ) { // cutesy abbreviation for local variable format fmt = format::rgba; if ( bits == 24 ) { fmt = format::rgb; } return fmt; } template< typename Format > std::ostream& print_format( std::ostream&, Format format_ ); // cutesy abbreviation for constrained template type template< Format Fmt > std::ostream& print_format( std::ostream&, Fmt format_ ); Having to come up with cutesy abbreviations for every local variable that are still meaningful is especially distracting. Surely enough code has been written that this has been solved. I looked through several boost library implementations but didn't see anything that stuck out as a clear convention. Adobe's ASL library seems to append _t postfix to type names and _m postfix to member variables. They also use very terse names for template parameter type names as well as parameter names and local variables. Like in: template <typename I, // I models ForwardIterator typename N, // N models IntegralType typename T, // T == result_type(P) typename C, // C models StrictWeakOrdering(T, T) typename P> // P models UnaryFunction(value_type(I)) -> T I lower_bound_n_(I f, N n, const T& x, C c, P p) { while (n != 0) { N h = n >> 1; I m = boost::next(f, h); if (c(p(*m), x)) { f = boost::next(m); n -= h + N(1); } else { n = h; } } return f; } What's your preferred convention? If possible please answer by modifying the initial concrete example and providing annotations.

3 2

[MSM] Surprising Entry Point behaviour
by Michael Caisse 06 Mar '16

06 Mar '16

Hi Christophe - I'm seeing some surprising behaviour when I have a sub-machine that can only be entered via an entry point. I have attached a diagram for quick reference. Code is below to demonstrate the "issue". Moving into the Sub state should only generate transitions into A with a 'none' event. The only way to get to B is via a transition from A with the 'event' event. MSM is trying to generate code into B for a transition with a 'none' event. This could be valid if the transition from Top::X wasn't via the entry point and just directly to Sub. I don't think there is a path to B with a 'none' event. Thoughts? Take care - michael ---------------- #include <boost/msm/front/state_machine_def.hpp> #include <boost/msm/front/functor_row.hpp> #include <boost/msm/back/state_machine.hpp> using boost::msm::front::none; using boost::msm::front::Row; namespace msm = boost::msm; struct event { int foo; }; struct Sub_ : public msm::front::state_machine_def<Sub_> { struct Enter : public msm::front::entry_pseudo_state<0> {}; struct A : msm::front::state<> {}; struct B : msm::front::state<> { template<class FSM> void on_entry(event const & evt, FSM &) {} }; using initial_state = A; struct transition_table : boost::mpl::vector< // Start Event Next Action Guard // +----------+---------+--------+-----------+-------- Row < Enter , none , A , none , none >, Row < A , event , B , none , none >, Row < B , none , A , none , none > // +----------+---------+--------+-----------+-------- >{}; }; using Sub = msm::back::state_machine<Sub_>; struct Top_ : public msm::front::state_machine_def<Top_> { struct X : msm::front::state<> {}; using initial_state = X; struct transition_table : boost::mpl::vector< // S Event Next Action Guard // +----+-------+-----------------------------+-------+------- Row < X , none , Sub::entry_pt<Sub_::Enter> , none , none > // +----+-------+-----------------------------+-------+------- >{}; }; using Top = boost::msm::back::state_machine<Top_>; int main() { Top top; top.start(); } ------------------------- -- Michael Caisse Ciere Consulting ciere.com

2 1

Re: [boost] [Boost-users] [Fit] formal review - should we propose some parts to Boost.Config/Boost.Core
by Glen Fernandes 06 Mar '16

06 Mar '16

On Sat, Mar 5, 2016 at 9:21 PM, Vicente J. Botet Escriba wrote: > Others could be considered also as function.hpp, lambda.hpp and lift.hpp, as > the macros are there to workaround some missing language features, but those > are much more specialized (Boost.Core?) > Boost.Core is where we move code from existing Boost libraries that enables breaking cyclic dependencies between different Boost libraries. I'm not so sure new libraries will need to (or want to) put code there. Glen

2 1

[Fit] review
by Vicente J. Botet Escriba 06 Mar '16

06 Mar '16

Paul, could you send me a private mail, I don't reach to send you any private mail. I'm always getting Hallovicente.botet(a)wanadoo.fr, Sie erhalten diese E-Mail, weil die Gruppe, die Sie kontaktieren möchten (boost-devel-archive), möglicherweise nicht vorhanden ist bzw. Sie nicht die erforderlichen Berechtigungen haben, um der Gruppe Nachrichten zu posten. Nachstehend einige Detailinformationen zu den möglichen Gründen, warum Sie keine Nachricht posten konnten: ... :( Vicente

1 0

[Fit] review - github issuess
by Vicente J. Botet Escriba 06 Mar '16

06 Mar '16

Hi, we have started to create some issues in https://github.com/pfultz2/Fit/issues, and sometimes there are some interesting discussion. I will try to add post on the MLs the issues that are conflicting so that other are aware of the discussion. Anyway, if you are curious you can always take a look at and participate also. If you want to receive the updates I believe that you just need to click on watch (top right). Best, Vicente

1 0

[asio] Feature request: console IO
by Tatsuyuki Ishi 06 Mar '16

06 Mar '16

Hello everyone, here is a greeting for joining the list. I want to request console support in asio. Some reasons: 1. non-block stdin is useful for some applications. 2. Linux can do non-block IO on ttys. Implementation concept: 1. Linux: if using epoll, first check if target is a regular file. If not, fall back to threaded non-blocking IO. select(), poll() can be use on any valid fds. 2. Windows: this garbage doesn't support overlapped console IO. Create a separate thread for std{in,out,err}, then communicate with the thread. 3. Other *nix: I don't know. It could already be used through POSIX stream descriptor, but it is too limited and platform specific. A library can cover these gaps is great. I hope you like this idea.

3 2

[Block Pointer][Container] Works - Take 2
by Phil Bouchard 06 Mar '16

06 Mar '16

Sorry for all the confusion but I was very confused myself. Now the block_ptr<> works in all cases with Linux x86: - I removed the usage of an allocator. Now there is no need for any allocator anymore. - I removed the usage of the strange pointer proxy optimization. Now the memory manager might take more memory but the class is clean and correctly abstracted. - I removed the usage of weak references. This is just too error-prone. But I need to tell whether an address is on the stack or on the heap. There is no portable way to do this except from marking the heap boundaries by overloading an allocator or write assembly code for each architecture / OS. I prefer the latter and I think this should eventually be part of the standards. Once again the working examples are here: https://github.com/philippeb8/block_ptr/blob/master/example/benchmark.cpp https://github.com/philippeb8/block_ptr/blob/master/example/block_ptr_test1… https://github.com/philippeb8/block_ptr/blob/master/example/block_ptr_test2… Regards, -Phil

3 6