Re: [boost] [iostreams][vc8]deprecetedstandardlibrarymemberfunctions; fatal linker error [OT]

From: Martyn Lovell (martynl_at_[hidden])
Date: 2005-07-06 22:49:03

Hi everyone, 

I wanted to take a few moments to address the range of points raised in this
thread, and to provide a little background.

Herb provided a great summary of our motivations here. We want to help
developers write more secure code, and we want to do that by default, so
that the out-of-the-box experience is as secure as we can make it. We've
reviewed lots of code in the last few years as part of the intensive
security focus at Microsoft. Though we've already used these lessons to
secure our own code, we wanted to help them be applied to other code too. 

To make this happen, we added "deprecation" warnings to a bunch of standard
APIs. Most warnings tell you what function to use instead. For example, the
warning for strcpy will point you at strcpy_s, which takes a buffer size as
a second parameter. As you can imagine these kinds of changes mostly impact
the C standard library.

The C++ standard Library is in much better shape. It was designed well from
the ground up and does not have extensive security problems. But there are
areas (such as use of pointers for iterators) where it's possible to write
dangerous unbounded constructs similar to a strcpy. We try to issue
deprecation warnings on each of these constructs.

While doing this, we're not making any compromise on producing a product
that conforms to the standard -- we've improved our conformance in VS8 over
our high level in VS7.1, mostly in minor ways. And while we do warn by
default on insecure usages, we always provide a way to disable these
warnings, and (in builds after the Beta 2 build most of you are using),
these warning texts include specific advice on how to disable the warning if
it is unwanted. We want to make sure it is as easy as possible to write a
portable product.

Though these changes are currently language extensions, we're also working
to make them available to the whole C and C++ community, not just users of
the Microsoft product. We've been working for a couple of years to
standardize the changes to the C library in the C standard committee - and
this process has been very productive and helpful. It should also lead to
several non-Microsoft implementations of these functions. Note that this
standards submission includes all the deprecated functions present in the C
standard itself. We deprecated other Microsoft extensions, such as
_splitpath, which has been replaced by _splitpath_s, but have not submitted
these to standardization since the original standard does not cover these
areas.

We're hoping to submit the C++ library changes for similar standardization
at a later date - once we have more experience of using them and broader
feedback from the community - and once we've completed the standardization
work on the C portions of the library.

As a general point, if anyone comes across something deprecated that they
think should not be, please do open a bug. As someone noted earlier in the
thread, a couple of these did slip through, and we have recently
un-deprecated them. We've applied this library to pretty large codebases, so
we have some experience, but your experience would be very useful to us.

I wrote a long article on these issues for MSDN magazine. It has more
details and examples of these issues.

http://msdn.microsoft.com/msdnmag/issues/05/05/SafeCandC/default.aspx

As an initial step when upgrading a project, _CRT_SECURE_NO_DEPRECATE (for C
functions) and _SCL_SECURE_NO_DEPRECATE (for C++) is a great way to ensure
your application compiles. But after that initial phase, I would encourage
you to think about how to remove these warnings without disabling them. In
many cases, you can make portable code changes to resolve the warning
without taking any dependency on our extensions. In others, you may need to
write extra helper code or make non-portable changes, but the security
benefit is worth it, especially for library code like boost. 

If someone in the boost community decides to take this on as a project, I
would be glad to work with them to help them understand the issues involved
and work through any problems.

As others have noted, both istream::read and sgetn were mistakenly
deprecated, and this has been fixed in our latest builds. I just checked the
current sources.

[[[
Aside:
Someone asked why sgetn got deprecated. Here is a short version of the
reason. Consider a much simpler C function, memcpy

memcpy(d, s, cnt);

You can make an argument that cnt is a count of the source, size of
destination or "both". A reasonable function to add is memcpy_s, which
clarifies the distinction

memcpy_s(d, dsize, s, scnt);

There has been debate and feedback on how much safer this form is. Having a
consistent pattern where a destination pointer is followed by its size is a
significant aid to secure code reviews, static analysis of code, and
consistency among buffer writing code. On the other side, many find one
integer parameter here to be "enough". 

Early in our implementation we deprecated memcpy (and other cases like
this), and added memcpy_s. Later in our implementation, we chose to
undeprecate memcpy (and its ilk), and keep memcpy_s as an alternative for
those who want it. 

sgetn got deprecated (and undeprecated) for similar reasons. Essentially, we
changed our implementation for a few functions based on customer and user
feedback, and when we did that, we missed sgetn.
]]]

When you see a warning from std::copy, it means that we believe you've given
us an "unchecked" iterator - either one that is a pointer, or one that we
cannot establish is "checked". If you have custom iterators, you can use
traits to mark them as checked. If you have code samples that warn where you
think they shouldn't please submit an issue in the product feedback centre,
and we'll investigate and fix if possible. The sooner we get these, the
better, as we are getting close to our release.

Someone noted that their b2 wouldn't install, perhaps because of beta 1
detritus. If you can't install b2, you likely won't be able to install the
final version either. I suggest you open a feedback issue and get
suggestions on how to clean out your machine. 

Wherever we have deprecated something, we have a recommended replacement.
Often, the compiler warning will tell you exactly what this replacement is.
For a complex case (passing a pointer to std::copy), it's not always
possible for us to be this precise, but you will get general advice.

I will keep an eye on this thread, but please feel free to follow up with me
directly as well if you need more information

Martyn Lovell
Development Lead
Visual C++ Libraries
aka martynl_at_[hidden]