Boost logo

Boost :

Subject: Re: [boost] [interprocess] OS version dependencies
From: Domagoj Saric (dsaritz_at_[hidden])
Date: 2010-10-23 16:35:11

"Marsh Ray" <marsh_at_[hidden]> wrote in message
> On 10/19/2010 10:28 AM, Domagoj Saric wrote:
>> I'm aware of that, however considering that: - Interprocess already
>> uses the native NT API
> The documented parts or undocumented parts?

Search for "ntdll.dll" in details/win32_api.hpp ;)

>> - interdependencies between Microsoft's/Windows' own components (and
>> the sheer number of those) make it reasonable to assume breaking
>> changes to the native NT API can in practice ever occur only between
>> major OS releases
> Or service packs, or whenever.
> :
>> The functions and structures in Winternl.h are internal to the
>> operating system and subject to change from one release of Windows to
>> the next, and possibly even between service packs for each release.
>> To maintain the compatibility of your application, you should use the
>> equivalent public functions instead. Further information is available
>> in the header file, Winternl.h, and the documentation for each
>> function.
> I.e., "fair notice is given: ignore it at your peril".

Right, I know this, that is why I wrote >in practice<...
And notice that the wording is between releases and possibly between service
packs...without the 'whenever' part...

>> - in practice breaking changes to system APIs have to be expected
>> between major OS releases even for Win32 and other 'old&public' API
>> as was demonstrated for example by the changes for interactive
>> services between XP and Vista
> That was a rare architectural change to Windows (explicit kernel support
> for multiple terminal services sessions) that broke the previous
> single-user assumption. App developers had been told that change was
> coming since post-NT4 days, IIRC. It doesn't discredit the general claim
> of compatibility across Win32.

However you look at it, it still did happen...and it is not the only such happens even on much more higher levels...for example a single
hotfix broke backward compatibility for the cryptographic API in Vista...
Also please note my point correctly, it is not to 'discredit' the Win32 API
but the (in practice) unjustified over-emphasis on the discrepancy between
the stability of Win32 API and the native NT API (and the arguments drawn
from that 'unjustified over-emphasis')...

> To the contrary, look at all the processes on your Windows system. How
> many are created as acutal native NT processes? Maybe two or three
> (winlogon, csrss), and last I checked the first thing they did was to load
> kernel32 like everybody else. Step through MS's own internally developed
> code and you'll see that even they use the documented Win32 APIs as much
> as possible.

Loading the kernel32.dll first is no proof of not using the native NT
API...e.g. Boost.Interprocess will also cause kernel32.dll to load first
(because of static linking) while at the same time using the native NT API
(through dynamic runtime loading of ntdll.dll, which is by then already
loaded by kernel32.dll anyway...)...

Similarly for 'as much as possible', while I have nothing against the
principle per se, it obviously does not imply 'always'/'only'/'solely'...

>> I'd consider this 'danger' practically
>> irrelevant (more like 'academic/by the book paranoia')
> My impression from reading the list is that a lot of Boost developers
> appreciate the "by the book" observance of documented APIs. It's a design
> principle, not just a paranoid fear based on a perceived probability of
> breakage. Interfaces are defined not just for the purposes of assigning
> "blame", but so that we can discuss program correctness rather than just
> making empirical statements about observed behavior.

Right, however, the whole point of my unholy POV on 'almost dogmatic
by-the-bookness' boils down to the fact that "in theory there is no
difference between practice and theory, in practice there is"...
IOW in practice the choice is not black and white, there is only the more
treaded path and the less treaded path(s)...If there are no objections to
the more treaded path the choice is obvious but if even the more treaded
path gives you thorns in your feet the possibility of exploring the less
treaded path(s) may become a viable cost-benefit analysis outcome (which is
where I think we basically agree)...

But, this is again 'academic' side tracking :)
To return to the subject, IMNHO, from the practical/empirical POV, the
issue/goal/ideal of 'correctness' must also include some measure of
Somehow, when using today's software on today's
trilions-of-bytes-of-memory-and-trilions-of-operations-per-second in
Joe-Sixpack-hardware that still performs far below one's expectations, this
seems self-evident...

The fact that a browser plugin is able to 'correctly' bring up a functional
dialog means little (to me) if it, for that single dialog, also brings in a
15 MB Qt-like monster of an 'auxiliary' library...and when you multiply that
by the number of plugins one can have......

> Next time you read industry reports like "IT departments are delaying
> deployment of the update (containing 43 critical security fixes) for a few
> months until they better assess app compatibility" you might take that as
> evidence that app developers as a whole are not excessively "paranoid".

I'm not sure if I understand this point...If it is something along the lines
that 'you cannot call A large if there exists B that is larger' such
arguments are a priori false...
IOW I am aware that when it comes to paranoia (as well as irrational/vanity
based policies) few things compare to the corporate world :D but this does
not necessarily excuse app developers ;)

> Sometimes, we have to do some fairly "heroic" and undocumented things to
> implement the functionality of our product. DLL injection, API hooking,
> the type of stuff antivirus programs need to do. Right now I'm working on
> the Vista implementation of a feature that works three different ways on
> three different versions of Windows. So we are willing to go there when
> necessary and, most importantly, when there is no other option. But we do
> it with our eyes open, knowing the maintenance effort it may bring.

And the practical question is, is there a significant discrepancy between
the number of these inter-version specifics/incompatibilities caused by
public/Win32 API incompatiblities and those caused by native NT API changes?

> Again, if it's necessary to implement useful functionality, that's fine.
> Yeah I agree that the Nt* APIs aren't going to change all that often.

And this is the core point, where we seem to agree fully ;)


Of course, I would also hold this requirement as self-evident...Considering
that the relative number of API functions used would probably not be large
I'd go for both implementations (if the Win32 API allows it) that one can
choose from with a configuration macro...

Finally, after all this hard talk, if we consider that the undocumented part
of the native NT API (of which, for the use in question, only a few
functions would have to be used, probably NTQuerySection and
NTExtendSection) is actually documented by the SysInternals guys that now
work for Microsoft I again must conclude that the whole issue much more
'theoretical' than 'practical'...

ps. As you have correctly deduced by splitting this into a different thread,
the issue/idea of using the native NT API is only a side issue of the
original ideas which were to make the Interprocess shared/mapped memory
interface more cross-platform like and the implementation of shared/mapped
memory for both Interprocess and IOStreams more efficient/less bloated...

"What Huxley teaches is that in the age of advanced technology, spiritual
devastation is more likely to come from an enemy with a smiling face than
from one whose countenance exudes suspicion and hate."
Neil Postman 

Boost list run by bdawes at, gregod at, cpdaniel at, john at