Re: [boost] Interest request for pointer+bit compression optimization

Subject: Re: [boost] Interest request for pointer+bit compression optimization
From: Neil Groves (neil_at_[hidden])
Date: 2009-09-07 16:38:07

On Mon, Sep 7, 2009 at 9:16 PM, Edouard A. <edouard_at_[hidden]> wrote:

> >This is incorrect. The pointer limit is only limited to under 0x80000000
> if
> >you are linking without the /LARGEADDRESSAWARE flag. It is quite common to
> >use this flag to enable the best performance of a 32-bit application on a
> >64-bit version of Windows. It can also have some benefit when using the
> >boot.ini /3GB tuning.
> >
> >http://msdn.microsoft.com/en-us/library/wz223b1z%28VS.80%29.aspx
>
> Yes there is the possibility to expand the user space to 3GB at the cost of
> global system performance and system instability (hope your drivers aren't
> assuming anything about the address of the buffers in the irps they'll get
> ;)  although if I remember correctly to get the WHQL certification you need
> to run in 3GB so if you have WHQL drivers you should be fine) on Windows
> 32-bit (the infamous /3GB switch you mentioned).
>
> I forgot that Windows 64 32-bit VM automatically expanded the user space to
> 3GB when the switch was on. That's safer in this context as the drivers
> deal
> with 64-bit address and all requests go through the VM.
>
> What you say about driver stability is very valid. However linking with
/LARGEADDRESSAWARE does not force this behaviour, it only signifies to the
OS that your application is capable of using the extra address space safely.
For many applications this is useful, for a great many others it merely
leads to confusion.

> Anyway, I don't mean to argue about the ideal size of the user space in
> Windows 32, you're right the library should be able to be detect whether or
> not a 32-bit application has got 3GB or 2GB space.
>
> > On 64-bit OSes you really can use the large pointers to store data and
> > therefore have very compact structures in memory. Correct me if I'm wrong
> > but currently most OSes only allow several gigabytes of virtual memory
> for
> > any given user process (something like 100-200 GB), that means there's
> room
> > for expression! :D
> >
> >
> >The limit is as high as 8TB on some Windows 64-bit Operating Systems.
> >http://msdn.microsoft.com/en-us/library/aa366778%28VS.85%29.aspx
>
> You're right! I was mixing up virtual and physical memory limits. That's
> the
> amount of physical memory that is limited to 128 GB (mmm I see you can even
> load up 2 TB on the datacenter edition, having your whole movie collection
> in memory is some sort of major feature methinks :p).
>
> That leaves (in theory) around 21 bits to store information.
>
>
You appear to be assuming that the OS guarantees that it uses the lowest
available virtual addresses. Would you please provide a link to the
documentation? I was under the impression that the 8TB limit is merely a
practical one since this was the maximum hardware specification available
for testing.

I prefer using the least-significant bits when the alignment allows such an
optimisation. This seems less error-prone and better documented as a safe
idiom.

> --
>
> EA
>
>
Regards,
Neil Groves