[boost] What's wrong with this benchmark

Subject: [boost] What's wrong with this benchmark
From: Benedek Thaler (thalerbenedek_at_[hidden])
Date: 2016-02-16 12:56:09

Dear Boost,

In a previous mail, I introduced a container library, `double_ended`.
This library has benchmarks [0]. However, the benchmarks show
unexpected results.

This is what I do [1]:
- compile program using G++ 5.3, -O2
- remove CPU0 from kernel scheduler pool (isolcpu=0 boot flag)
- disable CPU scaling
- pin benchmark thread to CPU0
- in the test, create a vector
- reserve memory for 1M unsigned
- write a byte to each page reserved, to warm up the page table
- get TSC value (machine has constant_tsc flag)
- insert 1M unsigned value
- get TSC value again, subtract previous value
- repeat test 25 times, average results

I do this with 3 containers: std::vector, boost::container::vector and
double_ended::devector. While the latter two produces almost
equivalent results, std::vector seriously falls behind.

- No reallocation happens between the TSC sampling
- Test order doesn't matter: every order produces the same results
- Container size or sample count doesn't matter, same results.
- The deviations of individual measures of each container is small,
the performance is consistent.
- I compared the produced ASM of std::vector and
double_ended::devector, they are very similar, almost the same
instructions in different order [2].

What do I miss here?

Thanks,
Benedek

[0]: http://erenon.hu/double_ended/double_ended/benchmarks.html
[1]: https://github.com/erenon/double_ended/blob/master/benchmark/push_back.cpp
[2]: http://pastebin.com/tUL54i9f