Subject: Re: Streamlining benchmarking process
From: stefan (stefan_at_[hidden])
Date: 2019-05-09 21:32:53
On 2019-05-09 5:11 p.m., Mateusz Loskot wrote:
> Now, I'm wondering this:
> if we maintain benchmark structure similar to tests e.g.
> - test/algo1.cpp, benchmark/algo1.cpp
> - test/algo2.cpp, benchmark/algo2.cpp
> could we define build config that allows to build
> 1) exe per .cpp - useful to bisect, find regressions
> 2) exe from multiple algorithms, related to the same problem of course,
> Â with one is used as baseline - comparative benchmarking
I'm in favor of 1). Whether that's truly one algorithm or not, however,
depends on whether the same code is parametrized to yield multiple
you can see how a single compilation unit / executable can represent the
same algorithm for multiple (template) parameters - value-types, or in
our case, pixel types or other compile-time parameters.
Even if we combine multiple algorithms into a single executable, I think
an important requirement is to be able to perform benchmark runs on
algorithms individually, as that allows to generate comparison charts
freely. And it scales better as we add more variants.
-- ...ich hab' noch einen Koffer in Berlin...
Boost list run by Boost-Gil-Owners