GDB-like debugger for template metaprograms
Hi, Metashell 2.0.0 has a template metaprogram debugger with a GDB-like interface (thanks to András Kucsma), which can be useful for Boost developers. It can show you how your metaprogram is executed step-by-step. You can set breakpoints, step into or over the metafunction calls and inspect the "template metaprogramming stack trace" at any point during this: the chain of recursive template instantiations that lead to that point. Here is an example:
#include <boost/mpl/transform.hpp> #include <boost/mpl/vector_c.hpp> #include <boost/mpl/plus.hpp> #include <boost/mpl/lambda.hpp> #include <boost/mpl/int.hpp> using namespace boost::mpl; #msh mdb transform<vector_c<int, 1, 2, 3, 4, 5>, plus<_1, int_<1>>>::type For help, type "help". Metaprogram started (mdb) rbreak ^boost::mpl::plus Breakpoint "^boost::mpl::plus" will stop the execution on 25 locations (mdb) continue Breakpoint "^boost::mpl::plus" reached boost::mpl::plus<mpl_::integral_c<int, 1>, mpl_::int_<1>, mpl_::na, mpl_::na, mpl_::na> (Memoization) (mdb) bt #0 boost::mpl::plus<mpl_::integral_c<int, 1>, mpl_::int_<1>, mpl_::na, mpl_::na, mpl_::na> (Memoization) #1 boost::mpl::quote_impl<boost::mpl::plus<mpl_::integral_c<int, 1>, mpl_::int_<1>, mpl_::na, mpl_::na, mpl_::na>, true> (TemplateInstantiation) #2 boost::mpl::quote5<plus, mpl_::void_>::apply<mpl_::integral_c<int, 1>, mpl_::int_<1>, mpl_::na, mpl_::na, mpl_::na> (TemplateInstantiation) #3 boost::mpl::apply_wrap5<boost::mpl::quote5<plus, mpl_::void_>, mpl_::integral_c<int, 1>, mpl_::int_<1>, mpl_::na, mpl_::na, mpl_::na> (TemplateInstantiation) ... #19 transform<vector_c<int, 1, 2, 3, 4, 5>, plus<_1, int_<1>>>::type
You can try Metashell from your browser: http://abel.web.elte.hu/shell Or you can install it on your computer: https://github.com/sabel83/metashell#version-200 Binary installers are available for a number of Linux distributions, Windows and Mac OS X. Feedback is welcome. Regards, Ábel
Hi, Great tool! Results of my second try below: (shame Firefox ignores undescore, shift+insert works)
#msh evaluate std::is_pointer<char>
<stdin>:1:26: error: use of undeclared identifier 'std' <stdin>:1:46: error: expected '(' for function-style cast or type construction <stdin>:1:48: error: expected unqualified-id // You don't need the evaluate add pragma to evaluate this metaprogram. The following command does this as // well: std::is_pointer<char>
std::is_pointer<char>
<stdin>:1:26: error: use of undeclared identifier 'std' <stdin>:1:46: error: expected '(' for function-style cast or type construction <stdin>:1:48: error: expected unqualified-id Probably I missed something, I just noted this because I would expect this as the intuitive way. Thanks, Benedek
Hi Benedek, On 2014-11-26 21:46, Benedek Thaler wrote:
std::is_pointer<char> <stdin>:1:26: error: use of undeclared identifier 'std' <stdin>:1:46: error: expected '(' for function-style cast or type construction <stdin>:1:48: error: expected unqualified-id
Probably I missed something, I just noted this because I would expect this as the intuitive way.
You seem to be missing a #include:
#include <type_traits> std::is_pointer<char>::type std::integral_constant<bool, false>
Regards, Ábel
On 26/11/2014 21:22, Abel Sinkovics wrote:
Hi,
Metashell 2.0.0 has a template metaprogram debugger with a GDB-like interface (thanks to András Kucsma), which can be useful for Boost developers. It can show you how your metaprogram is executed step-by-step. You can set breakpoints, step into or over the metafunction calls and inspect the "template metaprogramming stack trace" at any point during this: the chain of recursive template instantiations that lead to that point. Here is an example:
#include <boost/mpl/transform.hpp> #include <boost/mpl/vector_c.hpp> #include <boost/mpl/plus.hpp> #include <boost/mpl/lambda.hpp> #include <boost/mpl/int.hpp> using namespace boost::mpl; #msh mdb transform<vector_c<int, 1, 2, 3, 4, 5>, plus<_1, int_<1>>>::type For help, type "help". Metaprogram started (mdb) rbreak ^boost::mpl::plus Breakpoint "^boost::mpl::plus" will stop the execution on 25 locations (mdb) continue Breakpoint "^boost::mpl::plus" reached boost::mpl::plus<mpl_::integral_c<int, 1>, mpl_::int_<1>, mpl_::na, mpl_::na, mpl_::na> (Memoization) (mdb) bt #0 boost::mpl::plus<mpl_::integral_c<int, 1>, mpl_::int_<1>, mpl_::na, mpl_::na, mpl_::na> (Memoization) #1 boost::mpl::quote_impl<boost::mpl::plus<mpl_::integral_c<int, 1>, mpl_::int_<1>, mpl_::na, mpl_::na, mpl_::na>, true> (TemplateInstantiation) #2 boost::mpl::quote5<plus, mpl_::void_>::apply<mpl_::integral_c<int, 1>, mpl_::int_<1>, mpl_::na, mpl_::na, mpl_::na> (TemplateInstantiation) #3 boost::mpl::apply_wrap5<boost::mpl::quote5<plus, mpl_::void_>, mpl_::integral_c<int, 1>, mpl_::int_<1>, mpl_::na, mpl_::na, mpl_::na> (TemplateInstantiation) ... #19 transform<vector_c<int, 1, 2, 3, 4, 5>, plus<_1, int_<1>>>::type
From a glance, it appears to be missing important things to make it useful: - pretty printing complex types so that you can actually read them I don't think the formatter thing is a solution. Consider an arbitrarily deep expression template tree, for example. - easily get the source responsible for the instantiation full path and line information for each element in the backtrace is needed. Ideally you want to output that in a format where most smart editors can directly recognize it and jump there in a click.
Hi Mathias, Thank you for the ideas! Actually, both of those things are on our backlogs. Hopefully these will be included in the next major release. Regards, András On Sat, Nov 29, 2014 at 6:56 PM, Mathias Gaunard < mathias.gaunard@ens-lyon.org> wrote:
On 26/11/2014 21:22, Abel Sinkovics wrote:
Hi,
Metashell 2.0.0 has a template metaprogram debugger with a GDB-like interface (thanks to András Kucsma), which can be useful for Boost developers. It can show you how your metaprogram is executed step-by-step. You can set breakpoints, step into or over the metafunction calls and inspect the "template metaprogramming stack trace" at any point during this: the chain of recursive template instantiations that lead to that point. Here is an example:
#include <boost/mpl/transform.hpp> #include <boost/mpl/vector_c.hpp> #include <boost/mpl/plus.hpp> #include <boost/mpl/lambda.hpp> #include <boost/mpl/int.hpp> using namespace boost::mpl; #msh mdb transform<vector_c<int, 1, 2, 3, 4, 5>, plus<_1, int_<1>>>::type For help, type "help". Metaprogram started (mdb) rbreak ^boost::mpl::plus Breakpoint "^boost::mpl::plus" will stop the execution on 25 locations (mdb) continue Breakpoint "^boost::mpl::plus" reached boost::mpl::plus<mpl_::integral_c<int, 1>, mpl_::int_<1>, mpl_::na, mpl_::na, mpl_::na> (Memoization) (mdb) bt #0 boost::mpl::plus<mpl_::integral_c<int, 1>, mpl_::int_<1>, mpl_::na, mpl_::na, mpl_::na> (Memoization) #1 boost::mpl::quote_impl<boost::mpl::plus<mpl_::integral_c<int, 1>, mpl_::int_<1>, mpl_::na, mpl_::na, mpl_::na>, true> (TemplateInstantiation) #2 boost::mpl::quote5<plus, mpl_::void_>::apply<mpl_::integral_c<int, 1>, mpl_::int_<1>, mpl_::na, mpl_::na, mpl_::na> (TemplateInstantiation) #3 boost::mpl::apply_wrap5<boost::mpl::quote5<plus, mpl_::void_>, mpl_::integral_c<int, 1>, mpl_::int_<1>, mpl_::na, mpl_::na, mpl_::na> (TemplateInstantiation) ... #19 transform<vector_c<int, 1, 2, 3, 4, 5>, plus<_1, int_<1>>>::type
From a glance, it appears to be missing important things to make it useful:
- pretty printing complex types so that you can actually read them I don't think the formatter thing is a solution. Consider an arbitrarily deep expression template tree, for example.
- easily get the source responsible for the instantiation full path and line information for each element in the backtrace is needed. Ideally you want to output that in a format where most smart editors can directly recognize it and jump there in a click.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/ mailman/listinfo.cgi/boost
Hi Mathias, Thank you for checking it. On 2014-11-29 18:56, Mathias Gaunard wrote:
- pretty printing complex types so that you can actually read them I don't think the formatter thing is a solution. Consider an arbitrarily deep expression template tree, for example. We have some ideas on how to improve pretty printing of types and displaying complex template instances based on our own experience.
How would you change/improve pretty printing in a way you find useful? How would you display for example a deep expression template tree? Regards, Ábel
On 29/11/2014 22:15, Abel Sinkovics wrote:
Hi Mathias,
Thank you for checking it.
On 2014-11-29 18:56, Mathias Gaunard wrote:
- pretty printing complex types so that you can actually read them I don't think the formatter thing is a solution. Consider an arbitrarily deep expression template tree, for example. We have some ideas on how to improve pretty printing of types and displaying complex template instances based on our own experience.
How would you change/improve pretty printing in a way you find useful? How would you display for example a deep expression template tree?
It can be hard to match '<' and '>' together for long symbol names involving instantiations of templates where the parameters are themselves template instantiations. I think it would be great to have an option to automatically indent the type when printing so that it's easier to interpret. Instead of a<b,c<d>,e<f<g,h>>,i>, you could print it as a< b, c<d>, e< f<g, h>
, i
My example is still relatively short and uses one-letter names, but with real cases it can quickly become unreadable (just try reading Boost.Proto expression templates types without indentation, it's pretty horrible) In any case, my other suggestion is probably more important: I'd like to be able to use this tool to track how a specific instantiation happens, and since I'm working on a large project I'd need to have the reference of where the template being instantiated is and where the instantiation occurs.
On 11/29/2014 04:06 PM, Mathias Gaunard wrote:
On 29/11/2014 22:15, Abel Sinkovics wrote:
Hi Mathias,
Thank you for checking it.
On 2014-11-29 18:56, Mathias Gaunard wrote:
- pretty printing complex types so that you can actually read them I don't think the formatter thing is a solution. Consider an arbitrarily deep expression template tree, for example. We have some ideas on how to improve pretty printing of types and displaying complex template instances based on our own experience.
How would you change/improve pretty printing in a way you find useful? How would you display for example a deep expression template tree?
It can be hard to match '<' and '>' together for long symbol names involving instantiations of templates where the parameters are themselves template instantiations.
I think it would be great to have an option to automatically indent the type when printing so that it's easier to interpret.
Instead of a<b,c<d>,e<f<g,h>>,i>, you could print it as
a< b, c<d>, e< f<g, h>
, i
Or, keep *all* the delimiters(< , and >) lined up, as in the following ( where the delimiters are lined up only for the expr_prefx template):
expr_prefx < op_ator< []> , expr_prefx < op_symb<lit,lit1>
, expr_prefx < op_symb<act,action1>
This output was produced by: template < typename Operator , typename... Operands > friend std::ostream& operator<< ( std::ostream& os , expr_prefx < Operator , Operands... > ) { os <<"expr_prefx\n< " <<indent_buf_in<<Operator{}; using swallow = int[]; // guaranties left to right order (void)swallow { 0 , ( os <<"\n" <<indent_buf_out <<", " <<indent_buf_in<<Operands{} , 0 )... }; os<<indent_buf_out<<"\n>"; return os; } the indent_buf_in is an io manipulator that uses boost::iostreams and an iostreams indent filter, which I can provide. HTH. -regards, Larry
Hi Larry, On 2014-11-29 23:57, Larry Evans wrote:
Or, keep *all* the delimiters(< , and >) lined up, as in the following ( where the delimiters are lined up only for the expr_prefx template):
expr_prefx < op_ator< []> , expr_prefx < op_symb<lit,lit1>
, expr_prefx < op_symb<act,action1>
What we are planning to add is an interactive "template instance viewer" which can recursively open/close template paramter lists similar to most editor's support for showing/hiding blocks. It seems to me that this feature could give the flexibility to concentrate on the important parts of a complex template instance like this one. Regards, Ábel
Hi Mathias, On 2014-11-29 23:06, Mathias Gaunard wrote:
It can be hard to match '<' and '>' together for long symbol names involving instantiations of templates where the parameters are themselves template instantiations.
I think it would be great to have an option to automatically indent the type when printing so that it's easier to interpret.
Instead of a<b,c<d>,e<f<g,h>>,i>, you could print it as
a< b, c<d>, e< f<g, h>
, i
My example is still relatively short and uses one-letter names, but with real cases it can quickly become unreadable (just try reading Boost.Proto expression templates types without indentation, it's pretty horrible) Metashell has indentation support, but it is turned off by default since it is not complete yet. You can experiment with it: you need to start it with the --indent flag.
In any case, my other suggestion is probably more important: I'd like to be able to use this tool to track how a specific instantiation happens, and since I'm working on a large project I'd need to have the reference of where the template being instantiated is and where the instantiation occurs. Metashell uses Templight to get the template instantiation details and to get the source location, Templight need to be able to provide that. We are looking into it, so it will come.
Regards, Ábel
On 11/30/2014 2:30 AM, Abel Sinkovics wrote:
Hi Mathias,
On 2014-11-29 23:06, Mathias Gaunard wrote:
It can be hard to match '<' and '>' together for long symbol names involving instantiations of templates where the parameters are themselves template instantiations.
I think it would be great to have an option to automatically indent the type when printing so that it's easier to interpret.
Instead of a<b,c<d>,e<f<g,h>>,i>, you could print it as
a< b, c<d>, e< f<g, h>
, i
My example is still relatively short and uses one-letter names, but with real cases it can quickly become unreadable (just try reading Boost.Proto expression templates types without indentation, it's pretty horrible) Metashell has indentation support, but it is turned off by default since it is not complete yet. You can experiment with it: you need to start it with the --indent flag.
Building Metashell on Windows seems pretty flawed: 1) Metashell depends on Templight. 2) Templight supposedly depends on an older version of Clang than what is currently in the Clang svn source tree. 3) The instructions say "Download the source code from github" soon followed by "Download the source code on your Windows host". Huh ? 4) The instructions tell me to build Clang with Templight. Suppose I already have Clang regularly built from source using mingw/gcc. Am I actually building another version of clang just for Metashell and using that version instead as Clang when working with Metashell ? Ugh !!! It would be much better if whatever is needed from Clang were part of the latest clang source rather than some backdated version. 5) There is something about re-downloading the MingW header files. Is his ever necessary ? Why should I be doing this on Linux if I am building for Windows.
In any case, my other suggestion is probably more important: I'd like to be able to use this tool to track how a specific instantiation happens, and since I'm working on a large project I'd need to have the reference of where the template being instantiated is and where the instantiation occurs. Metashell uses Templight to get the template instantiation details and to get the source location, Templight need to be able to provide that. We are looking into it, so it will come.
Hi Edward, On 2014-11-30 20:08, Edward Diener wrote:
Building Metashell on Windows seems pretty flawed:
1) Metashell depends on Templight. 2) Templight supposedly depends on an older version of Clang than what is currently in the Clang svn source tree. Metashell uses the functionality Templight provides. Since Templight is not part of Clang (yet?), you need to build a Templight-enabled Clang for Metashell. The patch is not following the Clang trunk continuously - it is upgraded to the latest trunk sometimes. If you'd like to port the patch to the latest in trunk, I'm happy to merge it.
3) The instructions say "Download the source code from github" soon followed by "Download the source code on your Windows host". Huh ? I've removed this extra instruction. Thank you for pointing it out.
4) The instructions tell me to build Clang with Templight. Suppose I already have Clang regularly built from source using mingw/gcc. Am I actually building another version of clang just for Metashell and using that version instead as Clang when working with Metashell ? Ugh !!! It would be much better if whatever is needed from Clang were part of the latest clang source rather than some backdated version. Of course it would be much easier if Templight was part of Clang, but it isn't at the moment, so you need a special build. Let me know if you have a better idea than this approach.
5) There is something about re-downloading the MingW header files. Is his ever necessary ? Why should I be doing this on Linux if I am building for Windows. You probably don't need to do this, as they are part of the source tree. The script it was done with is there, so you can see how they were collected and I expect it to be useful for upgrading to newer versions of these headers. The reason why it is a Linux script is that I found it easier to write it this way - it shouldn't matter which platform you do this on.
Regards, Ábel
On 11/30/2014 4:36 PM, Abel Sinkovics wrote:
Hi Edward,
On 2014-11-30 20:08, Edward Diener wrote:
Building Metashell on Windows seems pretty flawed:
1) Metashell depends on Templight. 2) Templight supposedly depends on an older version of Clang than what is currently in the Clang svn source tree. Metashell uses the functionality Templight provides. Since Templight is not part of Clang (yet?), you need to build a Templight-enabled Clang for Metashell. The patch is not following the Clang trunk continuously - it is upgraded to the latest trunk sometimes. If you'd like to port the patch to the latest in trunk, I'm happy to merge it.
It would be more reasonable for the developers of Templight to port their patch to the latest clang trunk. I e-mailed them suggesting this, but did not receive a reply as yet.
3) The instructions say "Download the source code from github" soon followed by "Download the source code on your Windows host". Huh ? I've removed this extra instruction. Thank you for pointing it out.
4) The instructions tell me to build Clang with Templight. Suppose I already have Clang regularly built from source using mingw/gcc. Am I actually building another version of clang just for Metashell and using that version instead as Clang when working with Metashell ? Ugh !!! It would be much better if whatever is needed from Clang were part of the latest clang source rather than some backdated version. Of course it would be much easier if Templight was part of Clang, but it isn't at the moment, so you need a special build. Let me know if you have a better idea than this approach.
5) There is something about re-downloading the MingW header files. Is his ever necessary ? Why should I be doing this on Linux if I am building for Windows. You probably don't need to do this, as they are part of the source tree. The script it was done with is there, so you can see how they were collected and I expect it to be useful for upgrading to newer versions of these headers. The reason why it is a Linux script is that I found it easier to write it this way - it shouldn't matter which platform you do this on.
I was able to build Metashell successfully under Windows with your instructions. Two further suggestions for your "Building Metashell on Windows" instructions: 1) The Visual Studio build for Metashell should I believe specify that the configuration should match that of Clang build, ie. 'release'. Or do I have that wrong and Metashell is supposed to be built/run as the 'debug' version ? 2) You should specify where the final metaclass.exe will be built. Evidently it is in the bin/app/Release directory for me. I will try out your product. It is great to see you working on this as it may be a great boon to template metaprogramming.
Hi Edward, On 2014-12-02 00:25, Edward Diener wrote:
I was able to build Metashell successfully under Windows with your instructions. Two further suggestions for your "Building Metashell on Windows" instructions:
1) The Visual Studio build for Metashell should I believe specify that the configuration should match that of Clang build, ie. 'release'. Or do I have that wrong and Metashell is supposed to be built/run as the 'debug' version ? I've updated the instructions. You can do either a Debug or a Release build against the Release Clang build.
2) You should specify where the final metaclass.exe will be built. Evidently it is in the bin/app/Release directory for me. I've added this as well.
Thank you for the suggestions. Regards, Ábel
Hi all, I've been working on a re-vamped version of Templight ( https://github.com/mikael-s-persson/templight). So, I have a few things to mention in relation to the points raised in this discussion that relate to issues with the templight patch that metashell relies on. First thing's first, my version of templight also includes a GDB-style template meta-program debugger: https://github.com/mikael-s-persson/templight#using-the-templight-debugger The crucial difference that it runs on-line... i.e., breakpoints will pause the compilation to provide you with a shell that allows you to query information from the AST directly ("while it's hot"). It is not complete or thoroughly tested yet, but it's pretty powerful (e.g., allows you to, for example, evaluate the value of constants expressions in the context of the template instantiation or to evaluate the actual type of typedefs and similar dependent types). Technically, the possibilities are unlimited as far as gaining information about whatever you want during the compilation (suggestions are welcome!). The point was raised that a problem with the original templight patch is that it requires people to sync to a much older version of Clang, and support for Windows is not great. My version of templight solves those problems by having a much more up-to-date patch for Clang, that should work on the latest svn trunk of it. Also, I generate separate applications for Templight ("templight(.exe)", "templight++(.exe)" and "templight-cl.exe") which means that you don't have to deal with a templight-enabled clang executables that might interfere with your other workflows. And evidently, with templight-cl.exe, you have a MSVC-compatible version of templight too, based on the latest work on clang-cl.exe as a drop-in replacement for cl.exe. The patch on Clang is very minimal (part of my re-vamping was to pull most of the templight code out of the main clang code-base, and only leaving the minimal hooks), that patch has been submitted to clang, but it is still pending approval (I probably need more pressure from the community to get this patch through). Some of the finer points raised:
From Mathias Gaunard: pretty printing complex types so that you can actually read them ... It can be hard to match '<' and '>' together for long symbol names involving instantiations of templates where the parameters are themselves template instantiations.
The basics of this has been implemented in my version of Templight. I created a protobuf-based output format in which template names are compressed by re-using the repeating sub-strings in the names. To achieve this, I had to use the natural splits that exist in template names, based on matching "<", ",", and ">" delimiters to extract template arguments from the complex template instantiation names. I've tested this quite extensively, on many of the example programs from Boost libraries like Spirit and Proto, and it works quite well (I'm sure there are some corner-cases where it breaks down, but I haven't encountered any so far). When you output a protobuf-formatted template instantiation trace from templight, you actually get a linked-tree type of structure which you could easily use to make any kind of pretty-printing you like (or fancy collapse-able stuff too). Abel, I recommend that you check out that implementation, as it's really simple yet effective (also, the compression ratio and processing speed difference compared to the original YAML outputs is huge): https://github.com/mikael-s-persson/templight/blob/master/templight_messages... https://github.com/mikael-s-persson/templight/blob/master/lib/TemplightProto... https://github.com/mikael-s-persson/templight/blob/master/lib/TemplightProto...
From Mathias Gaunard: full path and line information for each element in the backtrace is needed. Ideally you want to output that in a format where most smart editors can directly recognize it and jump there in a click.
Of course, this is necessary! That is what I have done on my templight debugger. I don't know why Abel's MDB wouldn't have that. The whole point of a GDB-style debugger is to run it behind a smart editor in the fashion you describe.
From Mathias Gaunard: I'd like to be able to use this tool to track how a specific instantiation happens, and since I'm working on a large project I'd need to have the reference of where the template being instantiated is and where the instantiation occurs.
Currently, my templight debugger does not have a command to query the location of where the template being instantiated is, only where the instantiation occurs. But that is only for historical reason (this is what templight traces record), but I could easily add this feature. I'll work on it.
From Edward Diener: Building Metashell on Windows seems pretty flawed:
Building anything on Windows is flawed ;) For the record, my version of templight, that includes the interactive debugger builds fine under Windows, and has the MSVC-compatible templight-cl.exe program too.
From Edward Diener: 2) Templight supposedly depends on an older version of Clang than what is currently in the Clang svn source tree.
My version is up-to-date and much less intrusive on Clang, and therefore, with fewer foreseeable problems in applying it to whatever version of the Clang source tree you are sync'd with (as long as it is not too old). However, my version of templight is not currently compatible with metashell / MDB, because that's not my responsibility (but it is my hope).
From Edward Diener: Suppose I already have Clang regularly built from source using mingw/gcc. Am I actually building another version of clang just for Metashell and using that version instead as Clang when working with Metashell ? Ugh !!! It would be much better if whatever is needed from Clang were part of the latest clang source rather than some backdated version.
That is a point I have already raised with Abel some time ago. Clearly, this is a very impractical situation to be in. This was the whole reason for minimizing the patch necessary on Clang and then submitting it for permanent inclusion in Clang. I have done this, just waiting for the Clang gate-keepers to manifest any kind of interest. I just wanted to point these things out. I'm very enthusiastic about the future of this, and I believe it will have to go through my version of Templight, to have any hope of success in the near future. Cheers, Mikael Persson.
On 12/2/2014 6:25 PM, Mikael Persson wrote:
Hi all,
I've been working on a re-vamped version of Templight ( https://github.com/mikael-s-persson/templight). So, I have a few things to mention in relation to the points raised in this discussion that relate to issues with the templight patch that metashell relies on.
First thing's first, my version of templight also includes a GDB-style template meta-program debugger:
https://github.com/mikael-s-persson/templight#using-the-templight-debugger
The crucial difference that it runs on-line... i.e., breakpoints will pause the compilation to provide you with a shell that allows you to query information from the AST directly ("while it's hot"). It is not complete or thoroughly tested yet, but it's pretty powerful (e.g., allows you to, for example, evaluate the value of constants expressions in the context of the template instantiation or to evaluate the actual type of typedefs and similar dependent types). Technically, the possibilities are unlimited as far as gaining information about whatever you want during the compilation (suggestions are welcome!).
The point was raised that a problem with the original templight patch is that it requires people to sync to a much older version of Clang, and support for Windows is not great. My version of templight solves those problems by having a much more up-to-date patch for Clang, that should work on the latest svn trunk of it. Also, I generate separate applications for Templight ("templight(.exe)", "templight++(.exe)" and "templight-cl.exe") which means that you don't have to deal with a templight-enabled clang executables that might interfere with your other workflows. And evidently, with templight-cl.exe, you have a MSVC-compatible version of templight too, based on the latest work on clang-cl.exe as a drop-in replacement for cl.exe. The patch on Clang is very minimal (part of my re-vamping was to pull most of the templight code out of the main clang code-base, and only leaving the minimal hooks), that patch has been submitted to clang, but it is still pending approval (I probably need more pressure from the community to get this patch through).
Some of the finer points raised:
From Mathias Gaunard:
pretty printing complex types so that you can actually read them ... It can be hard to match '<' and '>' together for long symbol names involving instantiations of templates where the parameters are themselves template instantiations.
The basics of this has been implemented in my version of Templight. I created a protobuf-based output format in which template names are compressed by re-using the repeating sub-strings in the names. To achieve this, I had to use the natural splits that exist in template names, based on matching "<", ",", and ">" delimiters to extract template arguments from the complex template instantiation names. I've tested this quite extensively, on many of the example programs from Boost libraries like Spirit and Proto, and it works quite well (I'm sure there are some corner-cases where it breaks down, but I haven't encountered any so far). When you output a protobuf-formatted template instantiation trace from templight, you actually get a linked-tree type of structure which you could easily use to make any kind of pretty-printing you like (or fancy collapse-able stuff too).
Abel, I recommend that you check out that implementation, as it's really simple yet effective (also, the compression ratio and processing speed difference compared to the original YAML outputs is huge):
https://github.com/mikael-s-persson/templight/blob/master/templight_messages... https://github.com/mikael-s-persson/templight/blob/master/lib/TemplightProto... https://github.com/mikael-s-persson/templight/blob/master/lib/TemplightProto...
From Mathias Gaunard:
full path and line information for each element in the backtrace is needed. Ideally you want to output that in a format where most smart editors can directly recognize it and jump there in a click.
Of course, this is necessary! That is what I have done on my templight debugger. I don't know why Abel's MDB wouldn't have that. The whole point of a GDB-style debugger is to run it behind a smart editor in the fashion you describe.
From Mathias Gaunard:
I'd like to be able to use this tool to track how a specific instantiation happens, and since I'm working on a large project I'd need to have the reference of where the template being instantiated is and where the instantiation occurs.
Currently, my templight debugger does not have a command to query the location of where the template being instantiated is, only where the instantiation occurs. But that is only for historical reason (this is what templight traces record), but I could easily add this feature. I'll work on it.
From Edward Diener:
Building Metashell on Windows seems pretty flawed:
Building anything on Windows is flawed ;)
For the record, my version of templight, that includes the interactive debugger builds fine under Windows, and has the MSVC-compatible templight-cl.exe program too.
From Edward Diener:
2) Templight supposedly depends on an older version of Clang than what is currently in the Clang svn source tree.
My version is up-to-date and much less intrusive on Clang, and therefore, with fewer foreseeable problems in applying it to whatever version of the Clang source tree you are sync'd with (as long as it is not too old). However, my version of templight is not currently compatible with metashell / MDB, because that's not my responsibility (but it is my hope).
From Edward Diener:
Suppose I already have Clang regularly built from source using mingw/gcc. Am I actually building another version of clang just for Metashell and using that version instead as Clang when working with Metashell ? Ugh !!! It would be much better if whatever is needed from Clang were part of the latest clang source rather than some backdated version.
That is a point I have already raised with Abel some time ago. Clearly, this is a very impractical situation to be in. This was the whole reason for minimizing the patch necessary on Clang and then submitting it for permanent inclusion in Clang. I have done this, just waiting for the Clang gate-keepers to manifest any kind of interest.
I just wanted to point these things out. I'm very enthusiastic about the future of this, and I believe it will have to go through my version of Templight, to have any hope of success in the near future.
I think this is great. I surely hope that templight can become part of the clang build-from-source process, as well as having the clang compiler switches for templight documented. The ability to unroll template instantiations by the compiler, so that the template metaprogramming can see what is happening at each step when a template is instantiated, is a great boon for the template metaprogrammer.
participants (7)
-
Abel Sinkovics -
András Kucsma -
Benedek Thaler -
Edward Diener -
Larry Evans -
Mathias Gaunard -
Mikael Persson