Max Boost Fusion tuple parameters
Hello, What is the maximum Boost Fusion tuple param count? I was thinking of using it for about 200-300 elements. Anyone done this? I've heard of defines for this, but never actually *knew* of anyone actually doing it. Thanks, Rodrigo Madera
On 06/10/13 13:40, Rodrigo Madera wrote:
Hello,
What is the maximum Boost Fusion tuple param count?
I was thinking of using it for about 200-300 elements.
Anyone done this?
I've heard of defines for this, but never actually *knew* of anyone actually doing it.
Thanks, Rodrigo Madera
I found some relevant information on this by: 1) going to: http://www.boost.org/doc/libs/1_53_0/libs/fusion/doc/html/index.html 2) then following the _Class template tuple_ link to: http://www.boost.org/doc/libs/1_53_0/libs/fusion/doc/html/fusion/tuple/class... 3) then following the _vector_ link to: http://www.boost.org/doc/libs/1_53_0/libs/fusion/doc/html/fusion/container/v... 4) which said: You may define the preprocessor constant FUSION_MAX_VECTOR_SIZE before including any Fusion header to change the default. Example: #define FUSION_MAX_VECTOR_SIZE 20 HTH. -regards, Larry
What is the maximum Boost Fusion tuple param count?
I was thinking of using it for about 200-300 elements.
Anyone done this?
I've heard of defines for this, but never actually *knew* of anyone actually doing it.
Thanks, Rodrigo Madera
I found some relevant information on this by:
1) going to: http://www.boost.org/doc/libs/1_53_0/libs/fusion/doc/html/index.html 2) then following the _Class template tuple_ link to: http://www.boost.org/doc/libs/1_53_0/libs/fusion/doc/html/fusion/tuple/class... 3) then following the _vector_ link to: http://www.boost.org/doc/libs/1_53_0/libs/fusion/doc/html/fusion/container/v...
4) which said: You may define the preprocessor constant FUSION_MAX_VECTOR_SIZE before including any Fusion header to change the default. Example:
#define FUSION_MAX_VECTOR_SIZE 20
Template argument number in a template declaration is limited by compilers. For example, in MSVC10/11 it's limited to 64. So I doubt setting FUSION_MAX_VECTOR_SIZE to 200 would work.
On 06/11/13 09:31, Igor R wrote:
What is the maximum Boost Fusion tuple param count?
I was thinking of using it for about 200-300 elements.
Anyone done this?
I've heard of defines for this, but never actually *knew* of anyone actually doing it.
Thanks, Rodrigo Madera
I found some relevant information on this by:
1) going to: http://www.boost.org/doc/libs/1_53_0/libs/fusion/doc/html/index.html 2) then following the _Class template tuple_ link to: http://www.boost.org/doc/libs/1_53_0/libs/fusion/doc/html/fusion/tuple/class... 3) then following the _vector_ link to: http://www.boost.org/doc/libs/1_53_0/libs/fusion/doc/html/fusion/container/v...
4) which said: You may define the preprocessor constant FUSION_MAX_VECTOR_SIZE before including any Fusion header to change the default. Example:
#define FUSION_MAX_VECTOR_SIZE 20
Template argument number in a template declaration is limited by compilers. For example, in MSVC10/11 it's limited to 64. So I doubt setting FUSION_MAX_VECTOR_SIZE to 200 would work.
Thanks Igor. I had no idea about that :( In this case, I'd guess the only solution for Rodrigo is to use Christopher Schmidt's variadic tuples: http://article.gmane.org/gmane.comp.lib.boost.devel/194322 I remember (a year or more ago) looking at the Christopher's implementation and, IIRC, it used a combination of the boost preprocessor library and variadic templates to speed the compilation times. Again, IIRC, Christoper built up a N*CHUNK size tuple of T element by creating N structs each containing CHUNK elements of type T. I think the N structs were recursively inherited; so that struct tuple_chunked<N> inherited from tuple_chunked<N-1>. Thus, if CHUNK were 64 (in the case of MSVC`10/11, then creating a tuple of 200 would just require about 200/64 template instantiations. The last I heard about Christopher's effort was this post: http://article.gmane.org/gmane.comp.lib.boost.devel/234501 The code location (I don't know if it's still valid) was give in post: http://article.gmane.org/gmane.comp.lib.boost.devel/228820 HTH. -regards, Larry
On 06/11/13 10:11, Larry Evans wrote:
On 06/11/13 09:31, Igor R wrote:
What is the maximum Boost Fusion tuple param count?
I was thinking of using it for about 200-300 elements.
Anyone done this?
I've heard of defines for this, but never actually *knew* of anyone actually doing it.
Thanks, Rodrigo Madera
I found some relevant information on this by:
1) going to: http://www.boost.org/doc/libs/1_53_0/libs/fusion/doc/html/index.html 2) then following the _Class template tuple_ link to: http://www.boost.org/doc/libs/1_53_0/libs/fusion/doc/html/fusion/tuple/class... 3) then following the _vector_ link to: http://www.boost.org/doc/libs/1_53_0/libs/fusion/doc/html/fusion/container/v...
4) which said: You may define the preprocessor constant FUSION_MAX_VECTOR_SIZE before including any Fusion header to change the default. Example:
#define FUSION_MAX_VECTOR_SIZE 20
Template argument number in a template declaration is limited by compilers. For example, in MSVC10/11 it's limited to 64. So I doubt setting FUSION_MAX_VECTOR_SIZE to 200 would work.
Thanks Igor. I had no idea about that :(
In this case, I'd guess the only solution for Rodrigo is to use Christopher Schmidt's variadic tuples:
http://article.gmane.org/gmane.comp.lib.boost.devel/194322
I remember (a year or more ago) looking at the Christopher's implementation and, IIRC, it used a combination of the boost preprocessor library and variadic templates to speed the compilation times. Again, IIRC, Christoper built up a N*CHUNK size tuple of T element by creating N structs each containing CHUNK elements of type T. I think the N structs were recursively inherited; so that struct tuple_chunked<N> inherited from tuple_chunked<N-1>. Thus, if CHUNK were 64 (in the case of MSVC`10/11, then creating a tuple of 200 would just require about 200/64 template instantiations.
The last I heard about Christopher's effort was this post:
http://article.gmane.org/gmane.comp.lib.boost.devel/234501
The code location (I don't know if it's still valid) was give in post:
I probably jumped to the conclusion that "template argument number" meant "number of template instantiations". Sorry for that mistake. I guess "template argument number" means the maximum arity of any template. IOW, if "template argument number" was 5, then any template taking over 5 arguments would cause the MSVC10/11 compiler to diagnose an error. IOW: tuple<T1,T2,T3,T4,T5> would be OK, but: tuple<T1,T2,T3,T4,T5,T6> would cause MSV10/11 to issue a compiler error. Is that right? Also, I also read somewhere that MSVC had not yet implemented variadic templates: http://stackoverflow.com/questions/9589192/how-do-i-change-the-number-of-tem... Sorry for noise. -regards, Larry
I probably jumped to the conclusion that "template argument number" meant "number of template instantiations". Sorry for that mistake. I guess "template argument number" means the maximum arity of any template. IOW, if "template argument number" was 5, then any template taking over 5 arguments would cause the MSVC10/11 compiler to diagnose an error. IOW:
tuple<T1,T2,T3,T4,T5>
would be OK, but:
tuple<T1,T2,T3,T4,T5,T6>
would cause MSV10/11 to issue a compiler error. Is that right?
Yes, right.
Also, I also read somewhere that MSVC had not yet implemented variadic templates:
Variadic templates are implemented in the updated MSVC 11 toolchain ("Microsoft Visual C++ Compiler Nov 2012 CTP"). http://www.microsoft.com/en-us/download/details.aspx?id=35515
On 06/12/13 04:54, Igor R wrote:
I probably jumped to the conclusion that "template argument number" meant "number of template instantiations". Sorry for that mistake. I guess "template argument number" means the maximum arity of any template. IOW, if "template argument number" was 5, then any template taking over 5 arguments would cause the MSVC10/11 compiler to diagnose an error. IOW:
tuple<T1,T2,T3,T4,T5>
would be OK, but:
tuple<T1,T2,T3,T4,T5,T6>
would cause MSV10/11 to issue a compiler error. Is that right?
Yes, right.
Also, I also read somewhere that MSVC had not yet implemented variadic templates:
Variadic templates are implemented in the updated MSVC 11 toolchain ("Microsoft Visual C++ Compiler Nov 2012 CTP"). http://www.microsoft.com/en-us/download/details.aspx?id=35515
Thanks for the link. Now with variadic templates, can a tuple with > 64 elements be created? IOW, would the following: template<typename... T> struct tuple; tuple<T1,T2,...,T64,T65>*pt=0; compile? FWIW, I used Christopher's variadic fusion library to compile a 300 element tuple. It took a while, but it compiled. Also, timing results showing the compile time vs tuple size with gcc4.8 are: TUPLE_SIZE= 10 1.97 0.19 2.14 TUPLE_SIZE= 50 2.45 0.16 2.59 TUPLE_SIZE= 100 3.49 0.32 3.90 TUPLE_SIZE= 200 7.90 0.57 8.85 TUPLE_SIZE= 300 15.93 0.96 17.09 The timing results were produce on ubuntu with: TIME.cmd=time --format '%U %S %e' I think this gives: user-time system-time elapsed-time It also appears that when TUPLE_SIZE >=100, compile time goes up linearly. The code compiled was: typedef slim::vector < int_value< 0 > , int_value< 1 > , int_value< 2 > , int_value< 3 > . . .
tuple_t; int run(tuple_t const&a_tuple) { int sum=0; sum+=slim::at_c< 0 >(a_tuple).value; sum+=slim::at_c< 1 >(a_tuple).value; sum+=slim::at_c< 2 >(a_tuple).value; sum+=slim::at_c< 3 >(a_tuple).value; . . . sum+=slim::at_c< 49 >(a_tuple).value; return sum; } and was generated by the attached python file with TUPLE_SIZE as the argument. -regards, Larry
On Thu, 13 Jun 2013 14:27:30 +0200, Larry Evans <cppljevans@suddenlink.net> wrote:
It also appears that when TUPLE_SIZE >=100, compile time goes up linearly.
Take a look at memory usage either, as soon as it starts swapping, it's far from linear. Your peers will curse you for this code. CDT indexer (eclipse) takes a lot time and memory on such constructs. The size of the produced binary (due to debug info for all these small inlined internal helper functions) must not be forgotten. It's hard to justify half-Gb executable to the colleagues. -- Slava
On 06/14/13 04:39, Slava wrote:
On Thu, 13 Jun 2013 14:27:30 +0200, Larry Evans <cppljevans@suddenlink.net> wrote:
It also appears that when TUPLE_SIZE >=100, compile time goes up linearly.
I think that linear remark was wrong. It goes up by a factor of about 2 between 100 and 200 and then by another factor of 2 between 200 and 300. That's exponential IIRC :( Of course I could be misunderstanding again. It happens :(
Take a look at memory usage either, as soon as it starts swapping, it's far from linear. Your peers will curse you for this code. CDT indexer (eclipse) takes a lot time and memory on such constructs. The size of the produced binary (due to debug info for all these small inlined internal helper functions) must not be forgotten. It's hard to justify half-Gb executable to the colleagues.
-- Slava
Ah! Thanks for the tip. I had no idea that was the case. Would spirit's fusion do any better? Maybe a better question is would there be any reason for such large tuples. I asked Rodrigo that question in another post; however, I've not heard back from him. --Larry
On 06/14/13 09:08, Larry Evans wrote:
On 06/14/13 04:39, Slava wrote:
On Thu, 13 Jun 2013 14:27:30 +0200, Larry Evans <cppljevans@suddenlink.net> wrote:
It also appears that when TUPLE_SIZE >=100, compile time goes up linearly.
I think that linear remark was wrong. It goes up by a factor of about 2 between 100 and 200 and then by another factor of 2 between 200 and 300. That's exponential IIRC :( Of course I could be misunderstanding again. It happens :(
Take a look at memory usage either, as soon as it starts swapping, it's far from linear. Your peers will curse you for this code. CDT indexer
Yes. Looking at the performance monitor as it was compiling TUPLE_SiZE=500 showed almost all the memory was used and, IIRC, about 40% of swap.
(eclipse) takes a lot time and memory on such constructs. The size of the produced binary (due to debug info for all these small inlined internal helper functions) must not be forgotten. It's hard to justify half-Gb executable to the colleagues.
-- Slava [snip]
Using the slim::accumulate template instead of using python to generate the summation code doesn't make much difference: tuple_size compile_time 10 2.01 50 3.37 100 6.26 200 18.51 300 43.79 400 109.28 500 195.42 Attached is the driver. I did try to eyeball a fit with qtiplot which showed: compile_time = 12*2^(max(0,tuple_size-100)/100) looked quite similar at tuple_size>=100. --Larry
Sorry for not writing back at this thread. It's been very informative. So thanks! Larry, I had this issue for a hypothetical matrix computing problem, but nothing much practical. I'm sticking to smaller vectors now. Best regards, Rodrigo On Fri, Jun 14, 2013 at 3:38 PM, Larry Evans <cppljevans@suddenlink.net> wrote:
On 06/14/13 09:08, Larry Evans wrote:
On 06/14/13 04:39, Slava wrote:
On Thu, 13 Jun 2013 14:27:30 +0200, Larry Evans <cppljevans@suddenlink.net> wrote:
It also appears that when TUPLE_SIZE >=100, compile time goes up linearly.
I think that linear remark was wrong. It goes up by a factor of about 2 between 100 and 200 and then by another factor of 2 between 200 and 300. That's exponential IIRC :( Of course I could be misunderstanding again. It happens :(
Take a look at memory usage either, as soon as it starts swapping, it's far from linear. Your peers will curse you for this code. CDT indexer
Yes. Looking at the performance monitor as it was compiling TUPLE_SiZE=500 showed almost all the memory was used and, IIRC, about 40% of swap.
(eclipse) takes a lot time and memory on such constructs. The size of the produced binary (due to debug info for all these small inlined internal helper functions) must not be forgotten. It's hard to justify half-Gb executable to the colleagues.
-- Slava [snip]
Using the slim::accumulate template instead of using python to generate the summation code doesn't make much difference:
tuple_size compile_time 10 2.01 50 3.37 100 6.26 200 18.51 300 43.79 400 109.28 500 195.42
Attached is the driver.
I did try to eyeball a fit with qtiplot which showed:
compile_time = 12*2^(max(0,tuple_size-100)/100)
looked quite similar at tuple_size>=100.
--Larry
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
On 06/10/13 13:40, Rodrigo Madera wrote:
Hello,
What is the maximum Boost Fusion tuple param count?
I was thinking of using it for about 200-300 elements.
Hi Rodrigo, 200-300 elements seem like an awful lot! I'm curious about what application would require such a large tuple. Could you tell us for what you were planning to use the large tuple? TIA. -regards, Larry
participants (4)
-
Igor R -
Larry Evans -
Rodrigo Madera -
Slava