Boost logo

Boost :

Subject: Re: [boost] Checking interest in stacktrace library
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2016-06-19 14:55:29

On 18 Jun 2016 at 19:42, Gennadiy Rozental wrote:

> > I think that'd be great - even more so if you could integrate this with
> > boost.test, so I could state an expectation of what my function calls.
> > I.e. something like that:
> >
> > void f(int) {g(); h(); g();};
> >
> > BOOST_CHECK_CALLS(f(42), g, h, g);
> This is specific case for interaction based testing which is usually being
> done based on mocks. We would have very hard time to present something more
> or less generic without mocks.
> That being said, I'd be willing to work closely with anyone interested in
> bringing stacktrace to boost and integrate it into Boost.Test. It would be
> very useful to report stacks at the point of unexpected exceptions or fatal
> errors.

I have some buns in the oven on that front: It's the "other half" of
automated testing, the part unit test frameworks or dependency
injection don't usually cover: the Monte Carlo proof of correctness

Longer run i.e. not in 2016 it'll use LLVM to trace every edge of
execution and flow of data between the public facing library API and
any external functions/syscalls it makes. It will then automatically
permute the inputs and effects from external functions/syscalls to
formally prove a piece of C++ behaves as specified. Trunk clang and
trunk GCC is currently required for the requisite compiler and
runtime support, hence I won't land this till 2017 at the earliest,
though I have a cool proof of concept test here locally which even
works right in the MSVC ABI (well done clang!).

KernelTest is a natural fit for other tricks too, like running
libfuzzer, the AFL, all the sanitisers, valgrind etc. It multithreads
and multiprocesses naturally, so you can fire all your CPUs at the
problem for the many hours it takes to cover every permutation.

You can see an example of usage in AFIO v2's very first unit test:
_create_close/runner.cpp with an example of a test kernel at
_create_close/kernel_async_file_handle.cpp.hpp. As you'll see, the
simplest use case is to supply a kernel, an array of outcomes, kernel
parameters, preconditions and postconditions. KernelTest will iterate
the kernel and check everything is correct, notifying your unit test
framework of choice of any failures. The new fast ringbuffer logging
system will trace paths of execution in the kernel, optionally
logging stacktraces as it goes, to aid future permutation solvers. I
have yet to implement disk persistence for the logging ringbuffer,
it's a todo.

This stuff only started working for the first time the Thursday just
past as I've wrestling with ICEs in clang or VS2015. I've been
working on it exclusively since the ACCU conference in April as it
would be embarassing to present AFIO v2 at CppCon without any unit
test suite as I had to do at ACCU.

I would stress in the strongest possible terms that nobody should
remotely consider using any of this stuff any time in 2016, wait till
2017 or probably 2018. But interested people may wish to star my
github repos and watch out for conference videos, it's highly likely
KernelTest + the monadic outcomes at its very core will make an
appearance during the 2017 conference circuit.


ned Productions Limited Consulting

Boost list run by bdawes at, gregod at, cpdaniel at, john at