[RFC] Boost library name mangling and Microsoft's 'secure STL' feature.
Hi all. There have been requests to make Boost Build cleanly support Microsoft's 'secure STL'/'checked iterators' feature. Here's is one suggested implementation and how that will affect building Boost libraries on Windows: We add a new <msvc-checked-iterators> Boost Build feature with values 'off' and 'on' and 'off' being the default. The effect of the feature would be to define the SCL_SECURE preprocessor constant to 1 for feature value 'on' or to 0 for feature value off. This would affect only code compiled using MSVC compiler versions 8.0 & above. Libraries compiled with compilers affected by this setting would have their name mangling changed as follows: * If the setting is 'off' there would be no change. * If the setting is 'on' the name would get en extra letter 'c' added to its 'runtime tag' letter group (together with 's' for statically linked libraries, 'd' for debug, 'p' for the stlport standard library, etc.). Effects this would have on Boost libraries: * If we implement the proposed solution the default Boost library build behaviour will change. Boost libraries built by default will have Microsoft's secure STL (a.k.a. checking iterator) support disabled while currently, when we do nothing explicit about those settings, Microsoft compilers enable that support by default. * Making a complete Boost library build using the MSVC toolset will take twice as long as there may now be twice as many property combinations affecting the build. On the other hand user can always easily choose from the build command line which versions he wants in case he wants a subset. Selecting msvc-checked-iterators=on would get the current default behaviour. Some extra ideas to consider: * Add an additional 'default' feature value which would cause Boost Build to do nothing. This would on the other hand mess up library name mangling unless Boost Build were taught to know exactly what defaults each compiler or compiler version uses. Please give your comments on at least the following: * Feature name. * Any additional values that might be needed. * Any additional compiler flags/options/preprocessor constants that might be related to this. * Default value. * Library name mangling. * Any additional compilers this might affect, e.g. Comeau, Intel, etc. * Any additional runtime libraries this might affect, e.g. stlport. Many thanks for any feedback you can provide. Best regards, Jurko Gospodnetić
Jurko Gospodnetić wrote:
There have been requests to make Boost Build cleanly support Microsoft's 'secure STL'/'checked iterators' feature.
Here's is one suggested implementation and how that will affect building Boost libraries on Windows:
We add a new <msvc-checked-iterators> Boost Build feature with values 'off' and 'on' and 'off' being the default.
It sounds OK to me except 2 issues. Are there other toolsets with checked iterator options? I thought gcc and stlport had them. Perhaps the feature should be <checked-iterators>. The default value should obey The Principle of Least Astonishment. IMO this means it should be equivalent to the toolset defaults. Thanks, Michael Marcin
2008/9/3 Michael Marcin <mike.marcin@gmail.com>:
Jurko Gospodnetić wrote:
There have been requests to make Boost Build cleanly support Microsoft's 'secure STL'/'checked iterators' feature.
Here's is one suggested implementation and how that will affect building Boost libraries on Windows:
We add a new <msvc-checked-iterators> Boost Build feature with values 'off' and 'on' and 'off' being the default.
It sounds OK to me except 2 issues.
Are there other toolsets with checked iterator options? I thought gcc and stlport had them. Perhaps the feature should be <checked-iterators>.
Microsoft offers two different features: checked iterators and debugging iterators. Trying to map both to a single feature probably wouldn't make sense here.
The default value should obey The Principle of Least Astonishment. IMO this means it should be equivalent to the toolset defaults.
In this case, I believe that the principle of Don't Slow Down Inner Loops By An Order of Magnitude should prevail. See http://easyurl.net/high_cost and http://lists.boost.org/boost-build/2008/05/19098.php for the original rationale. - Mat
on Wed Sep 03 2008, Michael Marcin <mike.marcin-AT-gmail.com> wrote:
Jurko Gospodnetić wrote:
There have been requests to make Boost Build cleanly support Microsoft's secure STL'/'checked iterators' feature.
Here's is one suggested implementation and how that will affect building Boost libraries on Windows:
We add a new <msvc-checked-iterators> Boost Build feature with values 'off' and 'on' and 'off' being the default.
It sounds OK to me except 2 issues.
Are there other toolsets with checked iterator options? I thought gcc and stlport had them. Perhaps the feature should be <checked-iterators>.
I was thinking the same. GCC has them, MSVC has them, STLPort has them, probably others have them.
The default value should obey The Principle of Least Astonishment. IMO this means it should be equivalent to the toolset defaults.
I guess it depends on whether you think it's more astonishing to have any chance of high performance absolutely slaughtered even in MSVC release builds, and to have different defaults for the checked-iterator setting on different platforms, or more astonishing for the default to be different from what the toolset gives you out-of-the-box. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Jurko Gospodnetić wrote:
There have been requests to make Boost Build cleanly support Microsoft's 'secure STL'/'checked iterators' feature.
Two alternatives... Use the existing "runtime-debugging" feature to set the SCL_SECURE define. This has the advantage that the default behavior is the current behavior and there's no new feature name to fret about. Name the new feature "runtime-iterator-debugging", with values "off" and "checked". Set it to default to "off" for variant=release or variant=profile, and "checked" for variant=debug. This does change the current behavior, but in a less jarring way. It also is a generic but clear name which covers other STD implementations that have such iterators. It also allows for adding other kinds of iterator debugging if other STDs implement then now or in the future. -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org (msn) - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim,yahoo,skype,efnet,gmail
On Wed, 3 Sep 2008, Jurko Gospodneti? wrote:
* If the setting is 'on' the name would get en extra letter 'c' added to its 'runtime tag' letter group (together with 's' for statically linked libraries, 'd' for debug, 'p' for the stlport standard library, etc.).
Does the MSVC library ABI change depending on whether "secure STL" has been enabled? If not, I'd suggest adding a build option but not adding yet another runtime tag. Linking to boost libraries has already become tag soup -- try writing a portable configure script to figure out which name mangling was installed on a user's machine... Its been done, but it runs 2^n slower than simply checking whether '$CXX -l<name>...' succeeds. Later, Daniel
on Wed Sep 03 2008, dherring-AT-ll.mit.edu wrote:
On Wed, 3 Sep 2008, Jurko Gospodneti? wrote:
* If the setting is 'on' the name would get en extra letter 'c' added to its 'runtime tag' letter group (together with 's' for statically linked libraries, 'd' for debug, 'p' for the stlport standard library, etc.).
Does the MSVC library ABI change depending on whether "secure STL" has been enabled?
Yes -- Dave Abrahams BoostPro Computing http://www.boostpro.com
dherring@ll.mit.edu wrote:
On Wed, 3 Sep 2008, Jurko Gospodneti? wrote:
* If the setting is 'on' the name would get en extra letter 'c' added to its 'runtime tag' letter group (together with 's' for statically linked libraries, 'd' for debug, 'p' for the stlport standard library, etc.).
Does the MSVC library ABI change depending on whether "secure STL" has been enabled? If not, I'd suggest adding a build option but not adding yet another runtime tag.
It does, so it is impossible to link incompatible binaries. :( We had to adjust site-config.jam and recompile boost to get it to work, since we disable these iterators by default. (Their performance hit is too severe.) Cheers, /Marcus
Hi all.
There have been requests to make Boost Build cleanly support Microsoft's 'secure STL'/'checked iterators' feature.
After collecting some feedback both here and on IRC below are the summarized design suggestions. Please do keep the feedback coming in and see feedback requests at the end of this mail. First some collected background information related to Microsoft's checked & debugging iterator support to get a clearer image of what we are modeling. And could someone please collect a similar summary for other compilers/libraries?
------------------------- Checked iterator support: ------------------------- * http://msdn.microsoft.com/en-us/library/aa985965.aspx * Use _SECURE_SCL to enable. * Applicable with both debug and release run-time library variants. * Ensures that you do not overwrite the bounds of your container. * _SCL_SECURE_NO_WARNINGS may be used to disable warnings that get displayed due to using an unchecked iterator with a checked or standard algorithm versions.
----------------------- Debug iterator support: ----------------------- * http://msdn.microsoft.com/en-us/library/aa985982.aspx * Use _HAS_ITERATOR_DEBUGGING to enable. * Must use debug run-time library. * Detects invalidated iterator usage. * Detects using incompatible iterators in algorithms, e.g. those not belonging to the same container when expected to.
Now the design summaries in order of preference. Note that the list is a 'work in progress' and the ordering and the 'importance' are based on my own personal choices and current understanding of all the feedback received so far. Feel free to holler if you do not agree with them and I'll update as needed. -------- Design B -------- Improved version of the originally suggested 'Design A' allowing for debugging iterators and making the features msvc toolset specific, although the exact implementation described here is based on the msvc compiler documentation: Feature: <checked-iterators> 'on' --> _SECURE_SCL=1 'off' (default) --> _SECURE_SCL=0 Name mangling: <checked-iterators> adds 'c' to library name if enabled (assuming it's applicable). Feature: <debugging-iterators> 'on' --> _HAS_ITERATOR_DEBUGGING defined 'off' (default) --> _HAS_ITERATOR_DEBUGGING not defined <runtime-debugging>off & <debugging-iterators>on are incompatible. Name mangling: <debugging-iterators> adds 'e' to library name if enabled (assuming it's applicable). ------------------------- Design D (by Rene Rivera) -------------------------
Name the new feature "runtime-iterator-debugging", with values "off" and "checked". Set it to default to "off" for variant=release or variant=profile, and "checked" for variant=debug. This does change the current behavior, but in a less jarring way. It also is a generic but clear name which covers other STD implementations that have such iterators. It also allows for adding other kinds of iterator debugging if other STDs implement then now or in the future.
This does not allow freely combing checked-iterator and debugging-iterator concepts. On the other hand it does not seem like such a loss to add a feature value 'full' which would enable both checked & debugging iterators and not allow enabling debugging iterators without checked ones. We need more information on how other compilers & libraries implement this to decide whether this is 'good enough'. Name mangling: <checked-iterators>checked adds 'c' to library name if enabled (assuming it's applicable). Name mangling: <debugging-iterators>full adds 'f' to library name if enabled (assuming it's applicable). ------------------------- Design C (by Rene Rivera) -------------------------
Use the existing "runtime-debugging" feature to set the SCL_SECURE define. This has the advantage that the default behavior is the current behavior and there's no new feature name to fret about.
This is a simple solution requiring no library name changes and no new features. On the other hand this limits the user to not being able to select checked iterator debugging in release builds and does not allow the user to select debugging iterators independently of checked ones. Again, as before, we need more feedback about how other compilers & libraries implement these features in order to decide whether this solution is 'good enough'. ---------------------------------------------------- Design A (discarded design, overruled by 'Design B') ---------------------------------------------------- This is the same design as described in the original mail posting. Turns out to be pretty much incomplete as it misses the 'debugging iterators' concept completely. Feature: <msvc-checked-iterators> 'on' --> _SECURE_SCL=1 'off' (default) --> _SECURE_SCL=0 Name mangling: <msvc-checked-iterators> adds 'c' to library name if enabled (assuming it's applicable). Additional comments. * We really need to mangle library names to make it clear which libraries have checked/debugging iterators enabled and which do not, at for those compiled with MSVC, as these options produce binary incompatible libraries. This can be done by either adding new tag letters for these features or by smuggling them in under the guise of some other feature, e.g. <runtime-debugging>. * We can choose clean default value consistent between different toolsets or we can make each toolset define its own 'default'. Having each toolset define its own default though might be problematic to implement. Currently I prefer one clear default for all. However which gets chosen does not seem all that important. * All this will affects only code compiled using MSVC compiler versions 8.0 & above. It may also affect other non-MSVC toolsets as we get more information on them. * Intel toolset already unconditionally disables the _SECURE_SCL setting. Please give your comments on at least the following: * Feature names. * Any additional values that might be needed. * Any additional compiler flags/options/preprocessor constants that might be related to this. * Default values. * Library name mangling. * Any additional compilers & libraries this might affect., e.g. Comeau, Intel, gcc, stlport, etc. Could someone please provide more detailed information on how they support these or similar features. Thanks again for all the help on this. Best regards, Jurko Gospodnetić
Jurko Gospodnetić wrote:
First some collected background information related to Microsoft's checked & debugging iterator support to get a clearer image of what we are modeling. And could someone please collect a similar summary for other compilers/libraries? Just a quick note: GCC's scheme works thus:
1) If _GLIBCXX_DEBUG is defined, the debug containers are made available. 2) These containers lie in the std::__debug namespace. To actually use them, the programmer needs to somehow use them explicitly. This can be done through using statements or declarations, through typedefs, or namespace aliases. There is no macro or compiler switch that replaces the normal containers automatically. It is therefore an explicit choice by the programmer to use those containers, for every individual usage. No switching in the build system is necessary or even possible. Sebastian
2008/9/4 Sebastian Redl <sebastian.redl@getdesigned.at>:
Jurko Gospodnetić wrote:
First some collected background information related to Microsoft's checked & debugging iterator support to get a clearer image of what we are modeling. And could someone please collect a similar summary for other compilers/libraries?
Just a quick note: GCC's scheme works thus:
1) If _GLIBCXX_DEBUG is defined, the debug containers are made available. 2) These containers lie in the std::__debug namespace. To actually use them, the programmer needs to somehow use them explicitly. This can be done through using statements or declarations, through typedefs, or namespace aliases. There is no macro or compiler switch that replaces the normal containers automatically.
It is therefore an explicit choice by the programmer to use those containers, for every individual usage. No switching in the build system is necessary or even possible.
Eh? I'm fairly sure that if the macro is defined, standard containers are replaced by their debug counterpart. This is all you need to do to enable debugging (I have caught many bugs in my code just by defining that macro). In fact IIRC debugging containers are always available even if you do not define the macro and you can access them by explicitly using the __debug namespace (but I've never done that). HTH, -- gpd
on Thu Sep 04 2008, "Giovanni Piero Deretta" <gpderetta-AT-gmail.com> wrote:
2008/9/4 Sebastian Redl <sebastian.redl@getdesigned.at>:
Jurko Gospodnetić wrote:
First some collected background information related to Microsoft's checked & debugging iterator support to get a clearer image of what we are modeling. And could someone please collect a similar summary for other compilers/libraries?
Just a quick note: GCC's scheme works thus:
1) If _GLIBCXX_DEBUG is defined, the debug containers are made available. 2) These containers lie in the std::__debug namespace. To actually use them, the programmer needs to somehow use them explicitly. This can be done through using statements or declarations, through typedefs, or namespace aliases. There is no macro or compiler switch that replaces the normal containers automatically.
It is therefore an explicit choice by the programmer to use those containers, for every individual usage. No switching in the build system is necessary or even possible.
Eh? I'm fairly sure that if the macro is defined, standard containers are replaced by their debug counterpart. This is all you need to do to enable debugging (I have caught many bugs in my code just by defining that macro).
That's my experience too. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
David Abrahams wrote:
on Thu Sep 04 2008, "Giovanni Piero Deretta" <gpderetta-AT-gmail.com> wrote:
2008/9/4 Sebastian Redl <sebastian.redl@getdesigned.at>:
Jurko Gospodnetić wrote:
First some collected background information related to Microsoft's checked & debugging iterator support to get a clearer image of what we are modeling. And could someone please collect a similar summary for other compilers/libraries?
Just a quick note: GCC's scheme works thus:
1) If _GLIBCXX_DEBUG is defined, the debug containers are made available. 2) These containers lie in the std::__debug namespace. To actually use them, the programmer needs to somehow use them explicitly. This can be done through using statements or declarations, through typedefs, or namespace aliases. There is no macro or compiler switch that replaces the normal containers automatically.
It is therefore an explicit choice by the programmer to use those containers, for every individual usage. No switching in the build system is necessary or even possible.
Eh? I'm fairly sure that if the macro is defined, standard containers are replaced by their debug counterpart. This is all you need to do to enable debugging (I have caught many bugs in my code just by defining that macro).
That's my experience too.
You're right, sorry. There's a strong using statement importing the entire debug namespace in debug mode. Sebastian
On Thu, 4 Sep 2008, Sebastian Redl wrote:
David Abrahams wrote:
on Thu Sep 04 2008, "Giovanni Piero Deretta" <gpderetta-AT-gmail.com> wrote:
2008/9/4 Sebastian Redl <sebastian.redl@getdesigned.at>:
Jurko Gospodneti? wrote:
First some collected background information related to Microsoft's checked & debugging iterator support to get a clearer image of what we are modeling. And could someone please collect a similar summary for other compilers/libraries?
Just a quick note: GCC's scheme works thus:
1) If _GLIBCXX_DEBUG is defined, the debug containers are made available. 2) These containers lie in the std::__debug namespace. To actually use them, ... It is therefore an explicit choice by the programmer to use those containers, for every individual usage. No switching in the build system is necessary or even possible. ... You're right, sorry. There's a strong using statement importing the entire debug namespace in debug mode.
Would it be possible to do something like this in boost? For header-only libraries, the right preprocessor macros would control which iterators to use -- nothing needs to change. (?) For other libraries, bjam would compile the sources multiple times, using preprocessor macros to put checked iterators and other variants in boost::__variant_xyz namespaces and linking them all together into a single library. Then the macros selected by the user's code would control which symbols are used out of this library. Such a system would have the benefit that users could link against other libraries which use boost without having to arrange that they all use the same build variant. Properly initializing/using singletons in the presence of multiple variants could be difficult... - Daniel
Feature: <checked-iterators> 'on' --> _SECURE_SCL=1 'off' (default) --> _SECURE_SCL=0
I believe that the default should match the VC default, which is 1. This has no performance implications. Whether _SECURE_SCL is 0 or 1 in a particular project is decided by the user, not us. The autolink code will check the macro value and attempt to link to the appropriate Boost library. What is _built_ by default should match what is _used_ by default; it doesn't matter which one is faster, the choice is not ours to make.
Name mangling: <checked-iterators> adds 'c' to library name if enabled (assuming it's applicable).
Feature: <debugging-iterators> 'on' --> _HAS_ITERATOR_DEBUGGING defined 'off' (default) --> _HAS_ITERATOR_DEBUGGING not defined
Not defined means "on" in debug builds, "off" in release builds. Defined to 0 means off. Defined to 1 means "on" in debug builds, "off", warning and redefintion back to 0 in release builds. If we build no debug libraries, this default doesn't matter for MSVC. If we do, the default should again match the compiler default, either 1 or not defined.
On Thu, Sep 4, 2008 at 10:26 AM, Peter Dimov <pdimov@pdimov.com> wrote:
Feature: <checked-iterators> 'on' --> _SECURE_SCL=1 'off' (default) --> _SECURE_SCL=0
I believe that the default should match the VC default, which is 1.
This has no performance implications. [snip]
I wouldn't say that it has *no* performance implications. Default settings often have large impact since many people don't take the time to learn the meaning of the various settings or tune them. In my experience, this is especially true with a complex tool like boost build. I expect this would result in a large number of executables that are needlessly slow, as is the case today. Vendors are not infallible--I believe that Microsoft made a mistake in defaulting the flag to 'on' for release builds, and that most users just assume that they are not paying a release price for this "security". We have an opportunity to employ to follow conventional best-practices with boost build defaults, and I still lean in favor of doing so. Of course there are dangers. Mixing and matching a component built with the IDE defaults with a component built with (different) boost build defaults is a recipe for disaster. Then again, it is no great thing to ship a product with slow generic inner loops. In the end, I suppose that one could make the argument that one way or another users must be educated. Nonetheless, I think we should preserve the spirit of C++ by avoiding needless release performance tax on user-written generic algorithms. - Mat
Mat Marcus:
I wouldn't say that it has *no* performance implications. Default settings often have large impact since many people don't take the time to learn the meaning of the various settings or tune them. In my experience, this is especially true with a complex tool like boost build.
Yes, you are right. The default does have performance implications for people who use Boost.Build for their projects. (Although I wouldn't expect that particular audience to not take the time to learn the settings.) What I said applies to the "ordinary" user of Boost. He downloads Boost, builds or installs it, then makes use of it in his own project. In this case, the state of _SECURE_SCL is determined by the user. Whatever is built or installed by default should match what most of our "ordinary" users need, so that the autolink "just works". (Or we could just build both.)
on Thu Sep 04 2008, "Mat Marcus" <mat-lists-AT-emarcus.org> wrote:
On Thu, Sep 4, 2008 at 10:26 AM, Peter Dimov <pdimov@pdimov.com> wrote:
Feature: <checked-iterators> 'on' --> _SECURE_SCL=1 'off' (default) --> _SECURE_SCL=0
I believe that the default should match the VC default, which is 1.
This has no performance implications. [snip]
I wouldn't say that it has *no* performance implications. Default settings often have large impact since many people don't take the time to learn the meaning of the various settings or tune them. In my experience, this is especially true with a complex tool like boost build. I expect this would result in a large number of executables that are needlessly slow, as is the case today. Vendors are not infallible--I believe that Microsoft made a mistake in defaulting the flag to 'on' for release builds, and that most users just assume that they are not paying a release price for this "security". We have an opportunity to employ to follow conventional best-practices with boost build defaults, and I still lean in favor of doing so.
Of course there are dangers. Mixing and matching a component built with the IDE defaults with a component built with (different) boost build defaults is a recipe for disaster.
Yes, but unless you go out of your way to disable auto-link, that can't happen. IIUC the only thing that will happen if we "ship" something different from MS's default is that people using the default settings will find they can't link to the binaries we've "shipped." Maybe we need 3 builds: debug - Matches typical vendor debug settings release - Matches typical vendor release settings performance - Modifies release settings as necessary to get maximal performance Whether it makes sense to "ship" all three of these, I suppose, depends on what we mean by "ship." ;-) -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Mat Marcus wrote:
On Thu, Sep 4, 2008 at 10:26 AM, Peter Dimov <pdimov@pdimov.com> wrote:
Feature: <checked-iterators> 'on' --> _SECURE_SCL=1 'off' (default) --> _SECURE_SCL=0 I believe that the default should match the VC default, which is 1.
This has no performance implications. [snip]
I wouldn't say that it has *no* performance implications. Default settings often have large impact since many people don't take the time to learn the meaning of the various settings or tune them. In my experience, this is especially true with a complex tool like boost build. I expect this would result in a large number of executables that are needlessly slow, as is the case today. Vendors are not infallible--I believe that Microsoft made a mistake in defaulting the flag to 'on' for release builds, and that most users just assume that they are not paying a release price for this "security". We have an opportunity to employ to follow conventional best-practices with boost build defaults, and I still lean in favor of doing so.
Of course there are dangers. Mixing and matching a component built with the IDE defaults with a component built with (different) boost build defaults is a recipe for disaster. Then again, it is no great thing to ship a product with slow generic inner loops. In the end, I suppose that one could make the argument that one way or another users must be educated. Nonetheless, I think we should preserve the spirit of C++ by avoiding needless release performance tax on user-written generic algorithms.
For fun I took Steven Watanabe's recent filter iterator profiling app from another thread and ran it in VC9 SP1 in both modes. The app was posted here http://lists.boost.org/Archives/boost/2008/09/141851.php _SECURE_SCL=0 9.297 seconds 5.812 seconds 8.75 seconds 4.688 seconds 2.922 seconds 3.016 seconds Press any key to continue . . . _SECURE_SCL=1 12.64 seconds 9.281 seconds 12.531 seconds 6.907 seconds 5.578 seconds 3.015 seconds Press any key to continue . . . Thanks, Michael Marcin
Hi Peter.
Feature: <checked-iterators> 'on' --> _SECURE_SCL=1 'off' (default) --> _SECURE_SCL=0
I believe that the default should match the VC default, which is 1.
This has no performance implications. Whether _SECURE_SCL is 0 or 1 in a particular project is decided by the user, not us. The autolink code will check the macro value and attempt to link to the appropriate Boost library. What is _built_ by default should match what is _used_ by default; it doesn't matter which one is faster, the choice is not ours to make.
I fail to see how the auto-link code takes things out of our hands. Of course, auto-link code will need to be aware of the default settings (i.e. those with not explicit macro being defined) for each supported compiler in order to work correctly in any build environment. However, do not quite understand how this affects what default feature value choice we make in Boost Build. Which ever choice we make (on, off or compiler default), we will need to encode the on/off decision in the library's name. We will also need to set the corresponding preprocessor constants during compilation.
Name mangling: <checked-iterators> adds 'c' to library name if enabled (assuming it's applicable).
Feature: <debugging-iterators> 'on' --> _HAS_ITERATOR_DEBUGGING defined 'off' (default) --> _HAS_ITERATOR_DEBUGGING not defined
Not defined means "on" in debug builds, "off" in release builds. Defined to 0 means off. Defined to 1 means "on" in debug builds, "off", warning and redefintion back to 0 in release builds.
Hmm... are you sure about this? I have not checked any header files but states:
To enable iterator debugging in debug builds, set _HAS_ITERATOR_DEBUGGING to 1 To disable iterator debugging in debug builds, set _HAS_ITERATOR_DEBUGGING to 0
Thanks. Nice catch. I'll integrate this into the next design summary post. For those interested - the relevant header file distributed with MSVC is yyval.h. The official documentation on the other hand does not mention this 'default' behaviour (e.g. see http://msdn.microsoft.com/en-us/library/aa985939.aspx). Thanks for the feedback. Best regards, Jurko Gospodnetić
[Bah, the original message went out too soon... here's a finished one. I apologize or the noise.] Hi Peter.
Feature: <checked-iterators> 'on' --> _SECURE_SCL=1 'off' (default) --> _SECURE_SCL=0
I believe that the default should match the VC default, which is 1.
This has no performance implications. Whether _SECURE_SCL is 0 or 1 in a particular project is decided by the user, not us. The autolink code will check the macro value and attempt to link to the appropriate Boost library. What is _built_ by default should match what is _used_ by default; it doesn't matter which one is faster, the choice is not ours to make.
I fail to see how the auto-link code takes things out of our hands. Of course, auto-link code will need to be aware of the default settings (i.e. those with no explicit macro being defined) for each supported compiler in order to work correctly in any build environment. However, I do not quite understand how this affects what default feature value choice we can make in Boost Build. The auto-link code should then fine based on the preprocessor constants we set, no matter which choice we considered to be 'the default one' in Boost Build. Which ever choice we end up making (on, off or compiler default), we will need to encode the on/off decision in the library's name and set the corresponding preprocessor constants during compilation.
Name mangling: <checked-iterators> adds 'c' to library name if enabled (assuming it's applicable).
Feature: <debugging-iterators> 'on' --> _HAS_ITERATOR_DEBUGGING defined 'off' (default) --> _HAS_ITERATOR_DEBUGGING not defined
Not defined means "on" in debug builds, "off" in release builds. Defined to 0 means off. Defined to 1 means "on" in debug builds, "off", warning and redefintion back to 0 in release builds.
Thanks. Nice catch. I'll integrate this into the next design summary post. For those interested - the relevant header file distributed with MSVC is yyval.h. The official documentation on the other hand does not mention this 'default' behaviour (e.g. see http://msdn.microsoft.com/en-us/library/aa985939.aspx). Thanks for the feedback. Best regards, Jurko Gospodnetić
Jurko Gospodnetić wrote:
Hi all.
There have been requests to make Boost Build cleanly support Microsoft's 'secure STL'/'checked iterators' feature.
Here's is one suggested implementation and how that will affect building Boost libraries on Windows:
We add a new <msvc-checked-iterators> Boost Build feature with values 'off' and 'on' and 'off' being the default.
The effect of the feature would be to define the SCL_SECURE preprocessor constant to 1 for feature value 'on' or to 0 for feature value off. This would affect only code compiled using MSVC compiler versions 8.0 & above.
Libraries compiled with compilers affected by this setting would have their name mangling changed as follows:
* If the setting is 'off' there would be no change.
* If the setting is 'on' the name would get en extra letter 'c' added to its 'runtime tag' letter group (together with 's' for statically linked libraries, 'd' for debug, 'p' for the stlport standard library, etc.).
Effects this would have on Boost libraries:
* If we implement the proposed solution the default Boost library build behaviour will change. Boost libraries built by default will have Microsoft's secure STL (a.k.a. checking iterator) support disabled while currently, when we do nothing explicit about those settings, Microsoft compilers enable that support by default.
I would prefer to have the compiler default the Boost default as well, and yes I'm aware of the performance implications.
* Making a complete Boost library build using the MSVC toolset will take twice as long as there may now be twice as many property combinations affecting the build. On the other hand user can always easily choose from the build command line which versions he wants in case he wants a subset. Selecting msvc-checked-iterators=on would get the current default behaviour.
Some extra ideas to consider:
* Add an additional 'default' feature value which would cause Boost Build to do nothing. This would on the other hand mess up library name mangling unless Boost Build were taught to know exactly what defaults each compiler or compiler version uses.
Please give your comments on at least the following:
* Feature name.
OK, but can we generalise to things like STLPort and libstdc++ debug features as well?
* Any additional values that might be needed.
* Any additional compiler flags/options/preprocessor constants that might be related to this.
* Default value.
Same as the compiler/library default. BTW, any changes made here will have to be synchronised to a revised auto_link.hpp as well: just a heads up on that one. John.
On Wednesday 03 September 2008 22:57:20 Jurko Gospodnetić wrote:
[...] MSVC compiler versions 8.0 & above.
Sorry for the rant to follow, but please erase that term from your head. The problem with that term is that it doesn't help at all, it is rather a dangerous form of half-knowledge. Now, what's the issue? The issue is that MS versions their compilers independently from the IDE's they ship it with. That means that VC8 actually includes version 13 of their compiler. Or, rather, of their compiler_S_, because they support more than one target platform. Also, those target platforms are not just win32 and now recently the Intel and AMD 64 bit platforms, but also a handful of platforms for embedded use (MIPS, SH, ARM..). Further, they don't only ship that compiler with VC, but also with a tool called "Platform Builder", which allows you to select components for a CE OS and assemble/compile those into a bootable image. Lastly, the question what associated stdlib is used is independent of the compiler, e.g. that 'secure STL' you are writing about may or may not be available under CE - the compiler version doesn't tell you. As far as your initial RFC goes, I haven't looked too deeply at it, but if anything important comes up I'll add that. Otherwise, I'm all for activating the diagnostic mode optionally. cheers! Uli
participants (12)
-
David Abrahams -
dherring＠ll.mit.edu -
Giovanni Piero Deretta -
John Maddock -
Jurko Gospodnetić -
Marcus Lindblom -
Mat Marcus -
Michael Marcin -
Peter Dimov -
Rene Rivera -
Sebastian Redl -
Ulrich Eckhardt