[Boost-users] [phoenix] V2, V3 and the amount of memory needed by the compiler to just include them

Subject: [Boost-users] [phoenix] V2, V3 and the amount of memory needed by the compiler to just include them
From: Mathieu Champlon (mathieu.champlon_at_[hidden])
Date: 2011-02-23 18:58:28

Hello,

Phoenix is an outstanding component, I have no doubt about that.
However I have had some mixed results for my use case using V2 and now 
switching to V3.

I'm writing a (header only/template) library which uses Boost.Phoenix 
internally.
By nature this library (a mock object library) often ends up in 
precompiled headers because it's convenient to have it de facto included 
for whole (unit test) applications.
By default Microsoft Visual Studio allocates a certain amount of memory 
for compiling a precompiled header, and if it actually needs more then 
it fails to compile.

For instance with only :
   #include <boost/phoenix/phoenix.hpp>
Here is the error message with vc80 :
   ...boost/fusion/container/vector/vector10.hpp(52) : error C3859: 
virtual memory range for PCH exceeded; please recompile with a command 
line option of '-Zm160' or greater
   ...boost/fusion/container/vector/vector10.hpp(52) : fatal error 
C1076: compiler limit : internal heap limit reached; use /Zm to specify 
a higher limit

The location of the error is irrelevant, the value suggested by the 
compiler is what matters. Here it suggests to add /Zm160 to the command 
line.
This is pretty huge, the default is actually /Zm100, and I'd say "big" 
Boost libraries are around 50.
/Zm100 now means 75MB since vc90 instead of 50MB for vc80, which helps a 
bit, but still...

Here is a quick survey I did with vc80, using a /Zm1 to trigger an error 
every time.
The number on the right is the required /Zm value for including only the 
file on the left.

boost/phoenix/phoenix.hpp 160
"equivalent of content of boost/phoenix/phoenix.hpp for Phoenix 2" 107
boost/phoenix/core/nothing.hpp 61
boost/spirit/home/phoenix/core/nothing.hpp 52
boost/phoenix/statement/throw.hpp 62
boost/spirit/home/phoenix/statement/throw.hpp 54
boost/phoenix/bind.hpp 43
boost/spirit/home/phoenix/bind.hpp 56
boost/bind.hpp 17
boost/lambda/bind.hpp 27
boost/lambda/lambda.hpp 30
boost/function.hpp 36
boost/function_types/function_type.hpp 31
boost/test/auto_unit_test.hpp 41
boost/concept_check.hpp 16
boost/type_traits/remove_reference.hpp 12
boost/proto/proto.hpp 55
boost/proto/domain.hpp 20

Note that an empty pch requires /Zm11

Now extracting only the includes from Phoenix to other Boost libraries 
using :
   grep -rh "include <b" . | sort -u
And removing all boost/phoenix, the result is /Zm56 for V3 and /Zm65 for V2.
So V3 reduces the code of Phoenix itself but now uses (a lot) more of 
other Boost libraries, which incidentally is consistent with the main 
target of V3 switching to Boost.Proto

Of course the resulting amount of memory required for my library is not 
the sum of the memory required for each included library taken 
separately (mostly because they re-use each others).
And furthermore I can mostly work around the issue by using only 
phoenix::bind and rewrite some of the missing bits (for instance 
phoenix::throw_ and phoenix::nothing)
But even then and with carefully selecting the includes, it ends up 
pretty close to the default /Zm100.
This means that if the users of my library add a couple more libraries 
to their pch (Boost.Test comes to mind), they will most certainly go 
over the limit.
All right, after boggling a bit, they will add a bigger /Zm and forget 
about it for a while, until they upgrade either my library or Boost.
Suddenly a lot of their test applications fail to compile because 
instead of some /Zm112 it now requires /Zm113, and they will need to go 
over all their projects and update them...
For having this happening at my day job on a regular basis, I can attest 
first hand it is indeed annoying !

I started by saying I have had mixed results using Phoenix, this is 
because on one hand I resorted to rewriting what I needed using only 
phoenix::bind and I fear for the future usability of Phoenix, but on the 
other hand switching to V3 boost/phoenix/bind.hpp actually saves some 
memory compared to V2 boost/spirit/home/phoenix/bind.hpp !

Regards,
MAT.