|
|
Boost : |
From: Mat Marcus (mat-lists_at_[hidden])
Date: 2008-04-05 20:11:12
> I've found _SECURE_SCL less evil than claimed. It does decrease performance,
> but in my experience, it does so in situations where the code is already
> suboptimal (repeated calls to vector<>::operator[] and use of
> vector<T>::iterator instead of T*). On the plus side, it does catch bugs.
>
> On balance, I don't believe that we should override the default, although we
> probably should provide a way for the user to set it to 0 when building
> Boost.
I'm surprised to hear you say this, as my experience is so different.
To see whether this might be because things changed with in VC 9, I
ran one of our benchmark suites to measure the _SECURE_SCL penalty for
release builds. The benchmarks attempt, amongst other things, to
measure the abstraction penalty by comparing the performance of
different algorithms when using a vendor's vector with associated
iterator type vs. an array with pointers. When reading the results
below, double pointer means an array of doubles (pointer iterators),
where double vector means a vector of doubles (vector<T>::iterator).
Note that sometimes the penalty is as much as an order of magnitude.
This strengthens my view that _SECURE_SCL for release builds as a
misfeature, and I still am of the opinion that, at the least, boost
build should disable 'secure' STL for release variations in the
default case. (I won't go into detail with the source code for the
test or the exact build settings at this point in the discussion. If
you are interested in the source for these tests, let me know and I'll
see whether they are ready for public release.)
- Mat
###
### release build with _SECURE_SCL enabled
###
c:\boxer\performance\projects\msvc8\Release\stepanov_vector.exe
test description absolute operations ratio with
number time per second test0
0 "double pointer verify2" 3.03 sec 989.77 M 1.00
1 "DoubleClass pointer verify2" 3.05 sec 984.57 M 1.01
2 "double vector" 16.14 sec 185.86 M 5.33
3 "DoubleClass vector" 18.19 sec 164.95 M 6.00
4 "double vector reverse" 26.23 sec 114.35 M 8.66
5 "DoubleClass vector reverse" 26.20 sec 114.49 M 8.65
6 "double vector reverse reverse" 31.25 sec 96.00 M 10.31
7 "DoubleClass vector reverse reverse" 31.25 sec 96.00 M 10.31
Total absolute time for Vector accumulate: 155.34 sec
Vector accumulate Penalty: 5.92
test description absolute operations
ratio with
number time per second
test0
0 "insertion_sort double pointer verify2" 1.16 sec 2.60 M
1.00
1 "insertion_sort DoubleClass pointer verify2" 1.58 sec 1.90 M
1.37
2 "insertion_sort double vector" 19.58 sec 0.15 M
16.94
3 "insertion_sort DoubleClass vector" 18.00 sec 0.17 M
15.57
4 "insertion_sort double vector reverse" 30.56 sec 0.10 M
26.44
5 "insertion_sort DoubleClass vector reverse" 29.06 sec 0.10 M
25.14
6 "insertion_sort double vector reverse reverse" 43.03 sec 0.07 M
37.22
7 "insertion_sort DoubleClass vector reverse reverse" 43.05 sec 0.07 M
37.24
Total absolute time for Vector Insertion Sort: 186.02 sec
Vector Insertion Sort Penalty: 16.49
test description absolute operations rati
o with
number time per second test
0
0 "quicksort double pointer verify2" 1.64 sec 14.63 M 1.00
1 "quicksort DoubleClass pointer verify2" 1.95 sec 12.29 M 1.19
2 "quicksort double vector" 5.03 sec 4.77 M 3.07
3 "quicksort DoubleClass vector" 5.22 sec 4.60 M 3.18
4 "quicksort double vector reverse" 7.16 sec 3.35 M 4.36
5 "quicksort DoubleClass vector reverse" 7.34 sec 3.27 M 4.48
6 "quicksort double vector reverse reverse" 8.84 sec 2.71 M 5.39
7 "quicksort DoubleClass vector reverse reverse" 9.06 sec 2.65 M 5.52
Total absolute time for Vector Quicksort: 46.25 sec
Vector Quicksort Penalty: 3.52
test description absolute operations rati
o with
number time per second test
0
0 "heap_sort double pointer verify2" 1.81 sec 13.24 M 1.00
1 "heap_sort DoubleClass pointer verify2" 1.88 sec 12.80 M 1.03
2 "heap_sort double vector" 7.84 sec 3.06 M 4.33
3 "heap_sort DoubleClass vector" 7.89 sec 3.04 M 4.35
4 "heap_sort double vector reverse" 10.80 sec 2.22 M 5.96
5 "heap_sort DoubleClass vector reverse" 10.31 sec 2.33 M 5.69
6 "heap_sort double vector reverse reverse" 13.23 sec 1.81 M 7.30
7 "heap_sort DoubleClass vector reverse reverse" 13.22 sec 1.82 M 7.29
Total absolute time for Vector Heap Sort: 66.98 sec
Vector Heap Sort Penalty: 4.46
###
### release build with _SECURE_SCL disabled
###
c:\boxer\performance\projects\msvc8\Release\stepanov_vector.exe
test description absolute operations ratio with
number time per second test0
0 "double pointer verify2" 3.03 sec 989.45 M 1.00
1 "DoubleClass pointer verify2" 3.05 sec 984.57 M 1.00
2 "double vector" 3.06 sec 979.75 M 1.01
3 "DoubleClass vector" 3.05 sec 984.57 M 1.00
4 "double vector reverse" 3.06 sec 979.75 M 1.01
5 "DoubleClass vector reverse" 3.05 sec 984.57 M 1.00
6 "double vector reverse reverse" 3.09 sec 969.62 M 1.02
7 "DoubleClass vector reverse reverse" 3.06 sec 979.43 M 1.01
Total absolute time for Vector accumulate: 24.45 sec
Vector accumulate Penalty: 1.01
test description absolute operations
ratio with
number time per second
test0
0 "insertion_sort double pointer verify2" 1.16 sec 2.60 M
1.00
1 "insertion_sort DoubleClass pointer verify2" 1.58 sec 1.90 M
1.37
2 "insertion_sort double vector" 1.58 sec 1.90 M
1.37
3 "insertion_sort DoubleClass vector" 2.08 sec 1.44 M
1.80
4 "insertion_sort double vector reverse" 1.55 sec 1.94 M
1.34
5 "insertion_sort DoubleClass vector reverse" 2.03 sec 1.48 M
1.76
6 "insertion_sort double vector reverse reverse" 1.59 sec 1.88 M
1.38
7 "insertion_sort DoubleClass vector reverse reverse" 2.09 sec 1.43 M
1.81
Total absolute time for Vector Insertion Sort: 13.66 sec
Vector Insertion Sort Penalty: 1.53
test description absolute operations rati
o with
number time per second test
0
0 "quicksort double pointer verify2" 1.75 sec 13.71 M 1.00
1 "quicksort DoubleClass pointer verify2" 2.06 sec 11.64 M 1.18
2 "quicksort double vector" 1.80 sec 13.36 M 1.03
3 "quicksort DoubleClass vector" 2.14 sec 11.21 M 1.22
4 "quicksort double vector reverse" 1.78 sec 13.48 M 1.02
5 "quicksort DoubleClass vector reverse" 2.13 sec 11.29 M 1.21
6 "quicksort double vector reverse reverse" 1.78 sec 13.48 M 1.02
7 "quicksort DoubleClass vector reverse reverse" 2.16 sec 11.13 M 1.23
Total absolute time for Vector Quicksort: 15.59 sec
Vector Quicksort Penalty: 1.13
test description absolute operations rati
o with
number time per second test
0
0 "heap_sort double pointer verify2" 1.80 sec 13.36 M 1.00
1 "heap_sort DoubleClass pointer verify2" 1.83 sec 13.12 M 1.02
2 "heap_sort double vector" 1.88 sec 12.80 M 1.04
3 "heap_sort DoubleClass vector" 2.05 sec 11.73 M 1.14
4 "heap_sort double vector reverse" 2.14 sec 11.21 M 1.19
5 "heap_sort DoubleClass vector reverse" 2.28 sec 10.52 M 1.27
6 "heap_sort double vector reverse reverse" 1.88 sec 12.80 M 1.04
7 "heap_sort DoubleClass vector reverse reverse" 2.03 sec 11.81 M 1.13
Total absolute time for Vector Heap Sort: 15.88 sec
Vector Heap Sort Penalty: 1.12
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk