Re: [boost] Interest request for pointer+bit compression optimization

Subject: Re: [boost] Interest request for pointer+bit compression optimization
From: Neil Groves (neil_at_[hidden])
Date: 2009-09-08 03:47:15

On Mon, Sep 7, 2009 at 10:19 PM, Edouard A. <edouard_at_[hidden]> wrote:

> >You appear to be assuming that the OS guarantees that it uses the lowest
> >available virtual addresses. Would you please provide a link to the
> >documentation? I was under the impression that the 8TB limit is merely a
> >practical one since this was the maximum hardware specification available
> >for testing.
>
> The OS isn't floating around in virtual memory space! The layout is strict
> and doesn't change and of it's precisely and correctly documented in the
> driver developer's manual.
>
>
I would be alarmed if it was floating around. My concern was the perceived
lack of documented guarantees. I was ignorant of any documentation with
respect to the pointer ranges in the 64-bit Windows Operating Systems. My
motivation was to clarify the safe useable ranges and ensure we weren't
going to use undocumented implementation details by default.

> There is some randomness added for security, but the whole user space
> process is very strictly bounded. That's true for all OSes I've programmed
> drivers for, which is, I agree far from being exhaustive.
>
> For Windows x86 anything between 0x0 and 0x7fffffff is for user mode, with
> the first pages made inaccessible to help debug null pointers errors. I
> also
> remember that there's some clutter near the 2GB boundary (not sure about
> it,
> look it up if you're interested). Of course that goes up a bit if you
> extend
> the user space to 3GB.
>
> For Windows 64 running on IA64 anything between 0 and 0x6fbfffeffff is for
> user mode. It goes up to 0x7fffffeffff for AMD64.
>
> Again the first pages are inaccessible for the same reasons. Must something
> like a 32 kb locked range.
>
>
I was unaware of these exact address ranges, but you are correct. I have
found documentation to this effect here:
http://www.amd64.ru/download.php?uid=24504

The documentation for these ranges is not nearly as easy to find as the safe
ranges for the 32-bit platforms.

>I prefer using the least-significant bits when the alignment allows such an
> >optimisation. This seems less error-prone and better documented as a safe
> >idiom.
>
> I believe in freedom.
>
> The fact that these are safe documented user ranges alters my opinion. Does
anyone know of a way to determine the linker flag setting
/LARGEADDRESSAWARE, or is it better to implement a pessimisation? I agree
with Dave that looking at existing implementations is a good idea. Does
anyone know which files are of interest in the CLang code, or know of some
search hints?

I would like to see this generalised and optimally implemented too.

> --
>
> EA
>
>
Regards,
Neil Groves