March 2017 - Boost - lists.boost.org

[stacktrace] Review
by Bjorn Reese 25 Mar '17

25 Mar '17

I vote for the unconditional acceptance of Boost.Stacktrace. The design is sound. The separation of stacktrace and frame is just right, and safe_dump_to is an elegant solution to handle async-safety (much better than the fork-exec alternative.) The implementation (I did not look at the Windows parts) looks solid. There are details that looks platform-specific -- e.g. "/proc/self/exe" in detail/addr2line_impls.hpp is not Posix -- but I did not notice anything that cannot be ported if needed. The documentation is well-structured. I appreciate that the reference documentation contains both complexity and async-handler-safety comments. This library is very useful to developers. I spent a couple of hours going through the documentation and source for the review. Prior to that I have followed the development of this library, and used an earlier version in a commercial product. I have in-depth knowledge of the Posix side (having once implemented an async-safe stacktrace.) Wishlist: Make throw_with_trace of the API. Put it in a separate header so there is no dependency on Boost.Exception unless this header is included. In order to be consistent with Boost.Exception I suggest that the throwing function is called stacktrace::throw_exception(). Add an example with basic_stacktrace that uses a custom allocator. Add a simpler example that uses std::terminate(). The current one is rather overwhelming. Notice that I am not asking for the current example to be removed. Minor comments: Add a note to Getting Started that the stored backtrace.dump only is valid if the program has not be re-compiled between runs, as this is what developers do all the time. I have no opinion about windows.h.

1 0

[stacktrace] library review
by Artyom Beilis 25 Mar '17

25 Mar '17

Hello, There is my review. Note: I reviewed the library in previous round and voted no. So the current review was mostly review of the problems I found. - What is your evaluation of the design? The design is solid and minimalist as it suppose to be for such a library. Good separation for different backends. I still lack the out of the box "throw exception with stacktrace"/catch and print trace as I written in previous review but I don't think it is show stopper. i.e. "throw_with_trace" as shown in examples should be part of the library as it is one of most useful features. - What is your evaluation of the implementation? The implementation is OK but there are several issues (mostly in backend) I need to point out: I used this trivial program for run time testing #include <boost/stacktrace.hpp> #include <iostream> void foo() { std::cout << boost::stacktrace::stacktrace() << std::endl; } void bar() { foo(); } int main() { bar(); } Addr2Line Backend: --------------------------- In order to get the stacktrace the addr2line is called for each frame - i.e. stack of 4 functions results in 4(!!!) forks need to be fixed This is sample of stace output for trivial example, it can be reduced to 2 fork/exec at most [pid 7985] execve("/usr/bin/addr2line", ["addr2line", "-Cpe", "./a.out", "0x000000000040AFC2"], [/* 75 vars */]) = 0 [pid 7986] execve("/usr/bin/addr2line", ["addr2line", "-Cpe", "./a.out", "0x000000000040B02D"], [/* 75 vars */]) = 0 [pid 7987] execve("/usr/bin/addr2line", ["addr2line", "-Cpe", "./a.out", "0x000000000040B039"], [/* 75 vars */]) = 0 [pid 7988] execve("/usr/bin/addr2line", ["addr2line", "-Cpe", "/lib/x86_64-linux-gnu/libc.so.6", "0x00007EFD582A6830"], [/* 75 vars */]) = 0 [pid 7989] execve("/usr/bin/addr2line", ["addr2line", "-Cpe", "./a.out", "0x000000000040AEB9"], [/* 75 vars */]) = 0 This is something easy to fix and I expect it to be fixed before library is delivered to the official release. Libbacktrace Backend: ------------------------------ Linux: using libbacktrace without debug information results core dump - seems, the dump happens in ::backtrace_pcinfo happens for both g++ 4.8 and 5.4 Please make sure you can workaround this problem. This MUST be fixed. Windows/MinGW I build libbacktrace for mingw (tested it to make sure I can get traces using its API) However when I used boost.stacktrace with libbacktrace the program crashed every time in any configuration I used (with or without debug information) it worked with default build (without) but its usability is limited. This MUST be fixed (or if impossible mark as libbacktrace unusable for MINGW32) OS Support FreeBSD ------------------------------- FreeBSD - 100% unusable at this point - also I must say for FreeBSD it may be tricky due to : I needed to do some changes in CppCMS's backtrace library recently due to shift of latest freebsd to clang Take a look: https://github.com/artyom-beilis/cppcms/commit/18c3f05bfa9e4008aeac39322e8a… https://github.com/artyom-beilis/cppcms/blob/master/booster/lib/backtrace/s… It also may say that you WILL have problems with Mac OS X. $ c++ -v FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on LLVM 3.8.0) Target: x86_64-unknown-freebsd11.0 Thread model: posix InstalledDir: /usr/bin $ uname -a FreeBSD freebsd11 11.0-RELEASE-p1 FreeBSD 11.0-RELEASE-p1 #0 r306420: Thu Sep 29 01:43:23 UTC 2016 root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 $ c++ -I ../boost_stacktrace/include/ -I ../boost_1_63_0/ -Wall -O0 -g st.cpp In file included from st.cpp:1: In file included from ../boost_stacktrace/include/boost/stacktrace.hpp:15: In file included from ../boost_stacktrace/include/boost/stacktrace/frame.hpp:205: In file included from ../boost_stacktrace/include/boost/stacktrace/detail/frame_unwind.ipp:36: ../boost_stacktrace/include/boost/stacktrace/detail/safe_dump_posix.ipp:36:63: error: use of undeclared identifier 'S_IFREG' const int fd = ::open(file, O_CREAT | O_WRONLY | O_TRUNC, S_IFREG | S_IWUSR | S_IRUSR); ^ ../boost_stacktrace/include/boost/stacktrace/detail/safe_dump_posix.ipp:36:73: error: use of undeclared identifier 'S_IWUSR' const int fd = ::open(file, O_CREAT | O_WRONLY | O_TRUNC, S_IFREG | S_IWUSR | S_IRUSR); ^ ../boost_stacktrace/include/boost/stacktrace/detail/safe_dump_posix.ipp:36:83: error: use of undeclared identifier 'S_IRUSR' const int fd = ::open(file, O_CREAT | O_WRONLY | O_TRUNC, S_IFREG | S_IWUSR | S_IRUSR); ^ In file included from st.cpp:1: In file included from ../boost_stacktrace/include/boost/stacktrace.hpp:15: In file included from ../boost_stacktrace/include/boost/stacktrace/frame.hpp:205: ../boost_stacktrace/include/boost/stacktrace/detail/frame_unwind.ipp:75:7: error: no member named '_Unwind_Backtrace' in the global namespace ::_Unwind_Backtrace(&boost::stacktrace::detail::unwind_callback, &state); ~~^ st.cpp:3:14: warning: treating Unicode character as whitespace [-Wunicode-whitespace] void foo() { std::cout << boost::stacktrace::stacktrace() << std::endl; } ^ st.cpp:3:17: warning: treating Unicode character as whitespace [-Wunicode-whitespace] void foo() { std::cout << boost::stacktrace::stacktrace() << std::endl; } ^ st.cpp:4:14: warning: treating Unicode character as whitespace [-Wunicode-whitespace] void bar() { foo(); } ^ 3 warnings and 4 errors generated. I have another suggestion regarding testing. I didn't run them but since you didn't catch crash with libbacktrace without debug mode you probably have all compiled with debug information I suggest to modify the set of tests that you run on entire matrix: {backend}x{debug mode on/off/external PDB/debug information}x{rdynamic on/off} - What is your evaluation of the documentation? Documentation is fairly good - What is your evaluation of the potential usefulness of the library? VERY useful one the bugs are fixed. - Did you try to use the library? With what compiler? Yes: I tested on Linux gcc 5.4/4.8/clang++ Windows MinGW 5.x; FreeBSD clang 3.8 > Did you have any problems? Lots - see notes above. - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? Several hours running code reading backeds code. - Are you knowledgeable about the problem domain? Yes I had written one for CppCMS that runs on many platforms. And finally, every review should attempt to answer this question: - Do you think the library should be accepted as a Boost library? Yes, Subject to: - Fix of build/run issues on all major platforms including MINGW and FreeBSD - it was too easy to find issues I really concerns me - it is show stopper for the library to be released as part of boost. - FIx crashes with libbacktrace - Fix of addr2line performance - no need to fork/exec for each frame Note: I understand that it is hard for the author to fix all stuff on all platforms - but stacktrace by definition is doing something non portable in portable way. Best Regards, Artyom Beilis

1 0

Boost MSM parallel behavior with delayed self-transitions?
by Christophe Henry 24 Mar '17

24 Mar '17

Hi Danylo, >Hi Christophe, > >Thanks for your input. I'm going to use std::thread for the parallel >functionality, I think. it will work with synchronization. For simple cases it's manageable. >How stable/complete is Asynchronous? Can I rely on >it, or would I still have a moderate chance of running into bugs? It's very stable and used intensively in production code (in the industry) since 2 years. You might find some bug of course, but no stability problem has been found since a long time. > >Best, > >Danylo. > >PS. I love your documentation! What do you use for state machine diagrams - >MagicDraw? Enterprise Architect. Regards, Christophe

2 4

[Stacktrace] - review comments
by mbartosik 24 Mar '17

24 Mar '17

There are a few major uses of a stack trace library which have differing needs. 1) Fast signal-safe gathering of the stack frames the current thread (something that can run in either a Windows VectoredExceptionHandler or a unix Signal handler). 2) Gather of the stack frames for an exception in the current thread. 3) Gathering of stack frames for the current thread even if it means remote loading of symbol files. 4) Gathering of the stack frames for a paused thread (could even be in a different process). 5) Resolving symbols which is a VERY heavy weight task, potentially resolving symbols long after the target process for which the stack trace was made has exited. e.g. deserializing a stack trace from a log file and creating an annotated version with symbols added. In the case where I what to gather a stack trace for the current thread while handing a seg-fault or first-chance exception call I want to gather a code pointer for each stack frame efficiently (meaning no more than one pointer per frame, and no unnecessary calls or remote symbol loads) I believe that the library does address this, however, I am unsure of whether it can be requested to be more or less aggressive in what it does to walk the stack of highly optimized code, for example, in the most aggressive case it could cause symbols to be remotely downloaded from a webserver (e.g. http://msdl.microsoft.com/download/symbols) to download Microsoft .PDB files which contain information necessary to walk the stack of highly optimized code. I would like to see some control over how aggressive it is in walking the stack. On x86 the least aggressive would be walking the ESP / EBP+retaddr singly linked chain, the most aggressive would be to remote download .PDB files and use one of the Microsoft APIs to do the walking. I am unsure that it can gather a stack trace from a different thread. This is admittedly a less common case, and possibly not worth holding up the library for, although it would be good to hear how the author thinks such a library interface would look. Is there / should there be, any control over what is included in the frame to_string() e.g. should all arguments be included, should full path of source file be included? If so how is this to be controlled? If not then should what going in there be documented? Now for the major issue. I asked earlier at what point does symbol resolution happen, and I was told it is on-demand and that the storage per frame is a single pointer, implying that there is no caching. Since resolving symbols is very very expensive it is desirable to have caching of function/frame names. I can certainly image a developer streaming a frame to the a log file multiple times and then being surprised when his/her program slows down orders of magnitude. Caching of frame names is contrary to the idea of having just one pointer stored per frame. This makes me wonder if the underlying OS or library calls provide sufficient caching to avoid the cost of a second call to boost::stacktrace::frame::to_string being too great? Resolving of the symbols requires not just an absolute address, but if symbols are to be resolved 'after exit'/post mortem by another process it requires the RVA (relative virtual address) and the module for code (e.g. path to shared library / dll, etc.) This can be deduced from the absolute virtual address in the process, plus the module load address (which maybe different each time the module loads). What I would like to see is the ability to have an efficient stack trace object that requires only one pointer per frame. Plus something that is able to resolve symbols,* including post-mortem after a process has exited*. These two goals I think result in multiple classes. A light weight stacktrace class, in here to_string() would yield a pointer value. A symbolic_stacktrace class, in here to_string() would yield a human readable symbol resolved string. For the case of creating a symbolic_stacktrace from a serialized stacktrace post-mortem a module_list would also be required, in the case that the same process is used a symbolic_stacktrace would be creatable only from a stacktrace. Whether boost::stacktrace::frame stores a void * or a function pointer is a little interesting -- as technically the pointers are not function pointers but usually code pointers as they do not point to the beginning on a function. Having said all this, the current design does cover a simple case. Summary: * Control of how aggressive stack walking is. * Ability to walk stack of another thread including in another process. * boost::stacktrace::frame::to_string should be efficient for 2nd and 3rd and multiple calls, but not require more than a single pointer to be stored in the boost::stacktrace::frame. This implies either a different class for symbol resolution or external caching. * Control over what boost::stacktrace::frame::to_string includes (or define it). * Ability to work with post-mortem data. e.g. serialize a boost::stacktrace plus other information (a list of modules) and then deserialize and resolve symbols after the process has exited. * Work with boost serialization - Mark -- View this message in context: http://boost.2283326.n4.nabble.com/Stacktrace-review-comments-tp4692904.html Sent from the Boost - Dev mailing list archive at Nabble.com.

1 0

Using clang 4.0.0 with b2/bjam fails missing cstddef in boost/config/select_stdlib_config.hpp
by Paul A. Bristow 24 Mar '17

24 Mar '17

I have installed the shiny new Clang at c:\llvm\ with MS Visual Studio 2015 update 3 I have added Clang to my user-config.jam thus using clang ; or in full detail: using clang : # version 4.0.0 : # location Clang compiler .exe location. # from http://llvm.org/releases/download.html dated 9Mar2017 c:\\LLVM\\bin\\clang++.exe : # options <warnings>on <cxxflags>-v ; and tried to invoke it thus I:\modular-boost\libs\hello_boost\example>b2 toolset=clang-4.0.0 -d2 --debug-configuration > clang40_24Mar17.log (.log file attached in case that helps). It starts well but comes to a terminal halt missing cstddef in boost/config/select_stdlib_config.hpp clang version 4.0.0 (tags/RELEASE_400/final) Target: x86_64-pc-windows-msvc Thread model: posix InstalledDir: c:\LLVM\bin "c:\\LLVM\\bin\\clang++.exe" -cc1 -triple x86_64-pc-windows-msvc19.0.24123 -emit-obj -mrelax-all -mincremental-linker-compatible -disable-free -main-file-name hello_boost.cpp -mrelocation-model pic -pic-level 2 -mthread-model posix -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -target-cpu x86-64 -momit-leaf-frame-pointer -v -dwarf-column-info -debug-info-kind=limited -dwarf-version=4 -debugger-tuning=gdb -coverage-notes-file "I:\\modular-boost\\libs\\hello_boost\\example\\..\\..\\..\\bin.v2\\libs\\hello_boost\\example\\hello_boost.test\\clang-linux-4.0.0\ \debug\\hello_boost.gcno" -resource-dir "c:\\LLVM\\bin\\..\\lib\\clang\\4.0.0" -D BOOST_ALL_NO_LIB=1 -I "..\\..\\.." -internal-isystem "c:\\LLVM\\bin\\..\\lib\\clang\\4.0.0\\include" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\Include\\10.0.10586.0\\ucrt" -O0 -Wno-unused-local-typedef -Wno-dll-attribute-on-redeclaration -Wall -fdeprecated-macro -fdebug-compilation-dir "I:\\modular-boost\\libs\\hello_boost\\example" -ferror-limit 19 -fmessage-length 0 -fms-extensions -fms-compatibility -fms-compatibility-version=19.0.24123 -std=c++14 -fdelayed-template-parsing -fno-inline -fobjc-runtime=gcc -fcxx-exceptions -fexceptions -fdiagnostics-show-option -o "..\\..\\..\\bin.v2\\libs\\hello_boost\\example\\hello_boost.test\\clang-linux-4.0.0\\debug\\hello_boost.obj" -x c++ hello_boost.cpp clang -cc1 version 4.0.0 based upon LLVM 4.0.0 default target x86_64-pc-windows-msvc #include "..." search starts here: #include <...> search starts here: ..\..\.. c:\LLVM\bin\..\lib\clang\4.0.0\include C:\Program Files (x86)\Windows Kits\10\Include\10.0.10586.0\ucrt End of search list. In file included from hello_boost.cpp:15: In file included from ..\..\..\boost/config.hpp:44: ..\..\..\boost/config/select_stdlib_config.hpp:18:12: fatal error: 'cstddef' file not found # include <cstddef> I have two versions of cstddef C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\include\cstddef exists and refers to other files. contains // cstddef standard header #pragma once #ifndef _CSTDDEF_ #define _CSTDDEF_ #include <yvals.h> and another at C:\programfiles\mingw-w64\x86_64-6.3.0-win32-seh-rt_v5-rev1\mingw64\lib\gcc\x86_64-w64-mingw32\6.3.0\include\c++; My PATH has (amongst other items) C:\Programfiles\mingw-w64\x86_64-6.3.0-win32-seh-rt_v5-rev1\mingw64\bin; C:\programfiles\mingw-w64\x86_64-6.3.0-win32-seh-rt_v5-rev1\mingw64\lib\gcc\x86_64-w64-mingw32\6.3.0\include; C:\programfiles\mingw-w64\x86_64-6.3.0-win32-seh-rt_v5-rev1\mingw64\lib\gcc\x86_64-w64-mingw32\6.3.0\include\c++; C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\include; Is this expected (rather than clang-msvc-14.0)? notice: will use 'clang++' for clang-linux, condition <toolset>clang-linux-4.0.0 Anyone can give a clue as to what I am doing wrong? Thanks Paul --- Paul A. Bristow Prizet Farmhouse Kendal UK LA8 8AB +44 (0) 1539 561830

1 0

[build] VC++ 2017 version number?
by Beman Dawes 24 Mar '17

24 Mar '17

Boost.build in develop is treating VC++2017 RTW (aka RTM) as msvc-14.1. Yet the Visual Studio 2017 RTW Help "About Microsoft Visual Studio" shows the version as "15.0.0-RTW+26228.4" Shouldn't boost.build follow Microsoft's numbering? Seems like calling 2017 RTW anything except 15.0 is a recipe for confusion. --Beman

12 15

Re: [boost] Constructing string_ref from rvalue string
by Mathias Gaunard 24 Mar '17

24 Mar '17

On 23 Mar 2017 7:02 am, "Jonathan Müller via Boost" <boost(a)lists.boost.org> wrote: I've just blogged about this topic: http://foonathan.net/blog/2017 /03/22/string_view-temporary.html (Planned that post before this thread) In my opinion I think `boost::string_ref` and `std::string_view` are two different things, They're not. They were always supposed to be the same thing. There is a lot of code out there that uses std::string_view and boost::string_ref interchangeably depending on which is available.

3 2

Re: [boost] [partly OT] Re: [review queue] What to do about the library review queue?
by Glen Fernandes 24 Mar '17

24 Mar '17

peterkochlarsen wrote: > Also an unmaintained library should be marked as such as should > libraries that are deprecated, replaced by standard C++ features > (shared_ptr comes to mind) shared_ptr (or Boost.SmartPointers in general) is not unmaintained, and is not deprecated. It actually has been evolving past the C++11 std::shared_ptr. For example, boost::shared_ptr supported array forms in 1.53 and this was eventually added to C++ in C++17. (Interesting aside: That functionality above was even, and still is used, in certain Microsoft projects, where boost::shared_ptr is used instead of std::shared_ptr). The same goes for other Boost libraries like Boost.Thread. They: - May have features not in the C++ standard library equivalent - May be evolving past the current C++ standard - Are in use by actual projects and shouldn't be removed Glen -- View this message in context: http://boost.2283326.n4.nabble.com/review-queue-What-to-do-about-the-librar… Sent from the Boost - Dev mailing list archive at Nabble.com.

15 51

Is there any interest in type-safe container of bool flags with noexcept guarantees?
by Роман Орлов 23 Mar '17

23 Mar '17

Hello Boost community, I'd like to determine interest in this kind of container. Motivation For managing sets of bool flags developers often use std::bitset or c-like manual bit manipulations. Each approach brings problems. Some disadvantages of using std::bitset: * unpredictable and greedy allocation of bit storage; * setters/getters can throw exceptions; * it operates in low-level terms like "a bit number"; * we can easily assign one instance of std::bitset to another with the same number of bits but semantically they may differ. Proposal Instead of managing bits via numbers let's manage them via types. So start by declaring some types class eats_meat; class eats_grass; class has_tail; Then bind these types to flag identifiers typedef typed_flags<eats_meat, eats_grass, has_tail> animal; Unlike std::bitset the 'animal' instance allocates a minimal possible storage (in this case sizeof(animal) == 1). Create flags from scratch animal wolf; wolf.set<eats_grass>(false); wolf.set<eats_meat, has_tail>(); wolf.set(flag<has_tail>{1}, flag<eats_meat>{1}); Create flags with a flexible human-readable constructor wolf = animal{flag<has_tail>{1}, flag<eats_meat>{1}, flag<eats_grass>{0}}; I have implemented a minimal working prototype in my github repo https://github.com/compmaniak/typed_flags If you're interested you can try it online here http://melpon.org/wandbox/permlink/55g1czUjwSFO8LS1 Expected benefits * delegating calculation of a bit position for a certain type to compiler gives us a strong type safety and prevents typos; * noexcept guarantee of setting and getting bit values; * we shouldn't take care of bits because we work in terms of typed properties; * great readability, taking a short glance on a type definition tells you what this type is about. Some people on reddit (https://redd.it/601upg) found this library useful. Any thoughts on the idea of the container would be much appreciated. -- Roman

8 13

[release] 1.64.0 please test master snapshot
by Rene Rivera 23 Mar '17

23 Mar '17

Looks like 14.1/141 won out the version hysterics. And I've merged in the PRs for that change. Please give the latest master snapshot archives a try. And report both failures and success: <https://dl.bintray.com/boostorg/master/> If nothing Earth shattering comes up this will become the beta on Wednesday 22 (before 24:00 CDT). -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail

5 5