Compile Time String in C++14
Hi all, I found a way to make CT strings in C++14, with the help of generic lambda. FYI, Ábel Sinkovics has presented a c++11 CT string in his mpllibs::metaparse[1] lib before, but it requires some PP trick which has a predefined limit MPLLIBS_LIMIT_STRING_SIZE. The new trick doesn't suffer from the limitation, but it assumes a null-terminated string. It's fairly simple, see below: ```c++ template<char...> struct str {}; template<char c> struct ch {}; template<int n> using idx = std::integral_constant<int, n>; template<class S, char c, char... cs> auto accum(S s, ch<c>, str<cs...>) { return accum(s, s(idx<sizeof...(cs) + 1>()), str<cs..., c>()); } template<class S, char... cs> auto accum(S, ch<0>, str<cs...> ret) { return ret; } template<class S> auto make_str(S s) { return accum(s, s(idx<0>()), str<>()); } #define LIT(s) make_str([](auto idx) { return ch<s[idx.value]>(); }) ``` Then you can use `doSomething(LIT("abc"))` in your code. What do you think? [1] http://abel.web.elte.hu/mpllibs/metaparse/
TONGARI J <tongari95 <at> gmail.com> writes:
Hi all,
I found a way to make CT strings in C++14, with the help of generic lambda.
FYI, Ábel Sinkovics has presented a c++11 CT string in his mpllibs::metaparse[1] lib before, but it requires some PP trick which has a predefined limit MPLLIBS_LIMIT_STRING_SIZE.
The new trick doesn't suffer from the limitation, but it assumes a null-terminated string. It's fairly simple, see below:
```c++ template<char...> struct str {};
template<char c> struct ch {};
template<int n> using idx = std::integral_constant<int, n>;
template<class S, char c, char... cs> auto accum(S s, ch<c>, str<cs...>) { return accum(s, s(idx<sizeof...(cs) + 1>()), str<cs..., c>()); }
template<class S, char... cs> auto accum(S, ch<0>, str<cs...> ret) { return ret; }
template<class S> auto make_str(S s) { return accum(s, s(idx<0>()), str<>()); }
#define LIT(s) make_str([](auto idx) { return ch<s[idx.value]>(); }) ```
Then you can use `doSomething(LIT("abc"))` in your code. What do you think?
This is genius, thanks a lot for posting. I've been searching for a way to do this to implement compile-time strings in Boost.Hana[1]. Louis [1]: https://github.com/ldionne/hana -- View this message in context: http://boost.2283326.n4.nabble.com/Compile-Time-String-in-C-14-tp4666747p466... Sent from the Boost - Dev mailing list archive at Nabble.com.
Hi Louis, 2014-08-26 0:01 GMT+08:00 Louis Dionne <ldionne.2@gmail.com>:
This is genius, thanks a lot for posting. I've been searching for a way to do this to implement compile-time strings in Boost.Hana[1].
Glad to hear :) Actually I've found an even simpler way: ``` template<char...> struct str {}; template<std::size_t... Ns, class S> auto make_str_impl(std::index_sequence<Ns...>, S s) { return str<s.get()[Ns]...>(); } template<class S> auto make_str(S s) { return make_str_impl(std::make_index_sequence<sizeof(s.get()) - 1>(), s); } #define LIT(s) make_str([] \ { \ struct \ { \ static constexpr decltype(auto) get() \ { \ return s; \ } \ } _; \ return _; \ }()) ``` Cheers
Tongari J wrote:
Hi Louis,
2014-08-26 0:01 GMT+08:00 Louis Dionne <ldionne.2@gmail.com>:
This is genius, thanks a lot for posting. I've been searching for a way to do this to implement compile-time strings in Boost.Hana[1].
Glad to hear :)
Actually I've found an even simpler way:
Nice! And with some small tweaks it works in C++11, assuming you have some implementation of index_sequence: ``` template<char...> struct str {}; template<std::size_t... Ns, class S> auto make_str_impl(index_sequence<Ns...>, S) -> decltype(str<S::get()[Ns]...>()) { return str<S::get()[Ns]...>(); } template<class S> auto make_str(S) -> decltype(make_str_impl(make_index_sequence<sizeof(S::get()) - 1>(), S())) { return make_str_impl(make_index_sequence<sizeof(S::get()) - 1>(), S()); } #define LIT(s) make_str([]() \ { \ struct \ { \ static constexpr decltype(s) get() \ { \ return s; \ } \ } _; \ return _; \ }()) ``` Well VS2013 complains about a static data member in locally defined class but I assume it's a VS bug. Regards, Adam
2014-08-26 19:43 GMT+08:00 Adam Wulkiewicz <adam.wulkiewicz@gmail.com>:
Tongari J wrote:
Hi Louis,
2014-08-26 0:01 GMT+08:00 Louis Dionne <ldionne.2@gmail.com>:
This is genius, thanks a lot for posting. I've been searching for a way to do this to implement compile-time strings in Boost.Hana[1].
Glad to hear :)
Actually I've found an even simpler way:
Nice! And with some small tweaks it works in C++11, assuming you have some implementation of index_sequence:
I believe this still requires C++14 for returning the local struct in lambda, which is not a one-liner.
Tongari J wrote:
2014-08-26 19:43 GMT+08:00 Adam Wulkiewicz <adam.wulkiewicz@gmail.com>:
Tongari J wrote:
Hi Louis,
2014-08-26 0:01 GMT+08:00 Louis Dionne <ldionne.2@gmail.com>:
This is genius, thanks a lot for posting. I've been searching for a way to do this to implement compile-time strings in Boost.Hana[1].
Glad to hear :) Actually I've found an even simpler way:
Nice! And with some small tweaks it works in C++11, assuming you have some implementation of index_sequence:
I believe this still requires C++14 for returning the local struct in lambda, which is not a one-liner.
I'm not that well versed in the ways of the standard but I do know that it compiles in GCC 4.8.1, 4.8.3, 4.9.1 and clang 3.4 with -std=c++11 switch. So either those compilers are buggy or C++14 isn't really required. As said before MSVC2013 complains about something else. Regards, Adam
Adam Wulkiewicz wrote:
Tongari J wrote:
2014-08-26 19:43 GMT+08:00 Adam Wulkiewicz <adam.wulkiewicz@gmail.com>:
Tongari J wrote:
Hi Louis,
2014-08-26 0:01 GMT+08:00 Louis Dionne <ldionne.2@gmail.com>:
This is genius, thanks a lot for posting. I've been searching for a way to do this to implement compile-time strings in Boost.Hana[1].
Glad to hear :) Actually I've found an even simpler way:
Nice! And with some small tweaks it works in C++11, assuming you have some implementation of index_sequence:
I believe this still requires C++14 for returning the local struct in lambda, which is not a one-liner.
I'm not that well versed in the ways of the standard but I do know that it compiles in GCC 4.8.1, 4.8.3, 4.9.1 and clang 3.4 with -std=c++11 switch. So either those compilers are buggy or C++14 isn't really required. As said before MSVC2013 complains about something else.
Actually besides the complaining about static data member in local struct (get() member function) MSVC2013 indeed complains about the returning of local struct. AFAIU all of the presented implementations should fail on this compiler. Regards, Adam
Adam Wulkiewicz wrote:
Adam Wulkiewicz wrote:
Tongari J wrote:
2014-08-26 19:43 GMT+08:00 Adam Wulkiewicz <adam.wulkiewicz@gmail.com>:
Tongari J wrote:
Hi Louis,
2014-08-26 0:01 GMT+08:00 Louis Dionne <ldionne.2@gmail.com>:
This is genius, thanks a lot for posting. I've been searching for a way to do this to implement compile-time strings in Boost.Hana[1].
Glad to hear :) Actually I've found an even simpler way:
Nice! And with some small tweaks it works in C++11, assuming you have some implementation of index_sequence:
I believe this still requires C++14 for returning the local struct in lambda, which is not a one-liner.
I'm not that well versed in the ways of the standard but I do know that it compiles in GCC 4.8.1, 4.8.3, 4.9.1 and clang 3.4 with -std=c++11 switch. So either those compilers are buggy or C++14 isn't really required. As said before MSVC2013 complains about something else.
Actually besides the complaining about static data member in local struct (get() member function) MSVC2013 indeed complains about the returning of local struct. AFAIU all of the presented implementations should fail on this compiler.
A remark, the first mentioned MSVC error is caused by the lack of the support for 'constexpr' keyword in this compiler.
Regards, Adam
Hi, On 2014-08-26 04:22, TONGARI J wrote:
Actually I've found an even simpler way:
``` template<char...> struct str {};
template<std::size_t... Ns, class S> auto make_str_impl(std::index_sequence<Ns...>, S s) { return str<s.get()[Ns]...>(); }
template<class S> auto make_str(S s) { return make_str_impl(std::make_index_sequence<sizeof(s.get()) - 1>(), s); }
#define LIT(s) make_str([] \ { \ struct \ { \ static constexpr decltype(auto) get() \ { \ return s; \ } \ } _; \ return _; \ }()) ```
Compile time strings without the length limit would be great. The way I'm using them is instantiating template classes with the string as a template argument. So the way I'd use it would be something like: ---- template <class String> struct metafunction_expecting_a_string; metafunction_expecting_a_string<decltype(LIT("Hello"))> ---- I've tried your solutions with the latest Clang in svn, but the compiler complained about having a lambda expression in an unevaluated operand. Did you manage to use these strings as template arguments somehow? Regards, Ábel
Then you can use `doSomething(LIT("abc"))` in your code. What do you think?
I think I'm missing something. Isn't the point of compile-time strings to parse them or process them at compile-time? The `make_str` is not `constexpr` and neither is the lambda. So it can't be passed to a `constexpr` function nor can it be passed to template parameter because of the lambda. Unless there is something going on that I'm unaware of. Paul Fultz II -- View this message in context: http://boost.2283326.n4.nabble.com/Compile-Time-String-in-C-14-tp4666747p466... Sent from the Boost - Dev mailing list archive at Nabble.com.
I also fail to see the advantage of this implementation. I mean, it still uses index_sequence (or, in the first version, a variadic template parameter pack of chars), which is the main problem with using compile-time strings; it's the common problem in most real constexpr string implementations, as far I as know. An implementation of a constexpr string class that would avoid forming this type of very costly (non-scalable) template instantiation is starting to become interesting from a usability point of view, but short of that, it's hard to justify using it for anything but trivial tasks. That was just a remark. And as Paul remarks, you have obviously omitted the "constexpr" specifier in many places. So, can you explain how your implementation differs or improves upon existing constexpr string implementations? Here is a short list of existing compile-time string implementations (in case some people reading this are not familiar with the options available to solve that problem): Scott Schurr's very simple demo of "str_const": https://github.com/boostcon/cppnow_presentations_2012/blob/master/wed/schurr... Which is very similar (and better explained) to Andrzej Krzemienski's "StrWrap": http://akrzemi1.wordpress.com/2011/05/11/parsing-strings-at-compile-time-par... Or Sprout's very comprehensive implementation (basically a compile-time implementation of std::string): https://github.com/bolero-MURAKAMI/Sprout/blob/master/sprout/string/string.h... Or yet again, my own more light-weight version which supports compile-time concatenation, hashing and int-to-string functions only: https://github.com/mikael-s-persson/ReaK/blob/pp_dev/src/ReaK/core/base/cnst... which I explain in more details here (warning: it's a bit more of a tutorial for beginners, not aimed at experts): https://www.daniweb.com/software-development/cpp/code/482276/c11-compile-tim... @ Abel:
So the way I'd use it would be something like: ... Did you manage to use these strings as template arguments somehow?
AFAIK, you can use any of the above-listed implementations of constexpr strings in the way that you showed, but not the one discussed here. Compile-time strings are not a new thing, they are simply not very practical because they cause so much compilation overhead (because of compile-time char-by-char operations, deep compile-time recursions, and costly template instantiations), IMHO. Cheers, Mikael Persson.
Mikael Persson wrote:
Here is a short list of existing compile-time string implementations (in case some people reading this are not familiar with the options available to solve that problem):
<snip> Thanks for the links and valuable info! I already played before with Scott Schurr's implementation but wasn't able to do with it everything I wanted (see below).
@ Abel:
So the way I'd use it would be something like: ... Did you manage to use these strings as template arguments somehow? AFAIK, you can use any of the above-listed implementations of constexpr strings in the way that you showed, but not the one discussed here.
Do you know about some example where this is actually done? I can't see a straightforward way to pass such constexpr string as a template parameter and do something reasonable with it internally. In particular to generate a C++ type from a string literal. I think this is the most important concept. Correct me if I'm wrong but it seems to me that if you're dealing with constexpr strings you must handle them in constexpr functions and AFAIU that's not what Abel wanted. The technique proposed by TONGARI allows to generate types, at least in some cases, still not in Abel's case. Or am I missing something? Regards, Adam
2014-08-27 8:24 GMT+08:00 Mikael Persson <mikael.s.persson@gmail.com>:
I also fail to see the advantage of this implementation. I mean, it still uses index_sequence (or, in the first version, a variadic template parameter pack of chars), which is the main problem with using compile-time strings; it's the common problem in most real constexpr string implementations, as far I as know. An implementation of a constexpr string class that would avoid forming this type of very costly (non-scalable) template instantiation is starting to become interesting from a usability point of view, but short of that, it's hard to justify using it for anything but trivial tasks. That was just a remark.
And as Paul remarks, you have obviously omitted the "constexpr" specifier in many places.
So, can you explain how your implementation differs or improves upon existing constexpr string implementations?
My intention was to implement a compile-time Spirit that operates on CT string and returns different types depending on the input, so my usage is a bit different. Unlike the way Abel used in his metaparse, where every thing is a template parameter, as `metafunction_expecting_a_string<decltype(LIT("Hello"))>`, I'd like use `string(LIT("Hello"))`, which I think is more like the way Boost.Hana adopts. I don't think constexpr is needed in my case, because what I need is the type (i.e. the constants encoded in the type), not the value. For example, here's how I implement the string_parser: ```c++ struct string_parser : parser<string_parser> { template<class S> struct parse_impl; template<char... cs> struct parse_impl<str<cs...>> { template<char... etc> static auto apply(str<cs..., etc...>) { return success(str<etc...>(), str<cs...>()); } template<class S> static auto apply(S s) { return fail(s); } }; template<class S, class Str> static auto parse(S s, Str) { return parse_impl<Str>::apply(s); } }; constexpr string_parser string = {}; ``` which can then be used: ``` string(LIT("123")).parse(LIT("1234")); // returns (true_, str<'4'>, str<'1', '2', '3'>), a pack of (result, rest-input, attr) ```
Here is a short list of existing compile-time string implementations (in case some people reading this are not familiar with the options available to solve that problem):
Scott Schurr's very simple demo of "str_const":
https://github.com/boostcon/cppnow_presentations_2012/blob/master/wed/schurr...
Which is very similar (and better explained) to Andrzej Krzemienski's "StrWrap":
http://akrzemi1.wordpress.com/2011/05/11/parsing-strings-at-compile-time-par...
Or Sprout's very comprehensive implementation (basically a compile-time implementation of std::string):
https://github.com/bolero-MURAKAMI/Sprout/blob/master/sprout/string/string.h...
Or yet again, my own more light-weight version which supports compile-time concatenation, hashing and int-to-string functions only:
https://github.com/mikael-s-persson/ReaK/blob/pp_dev/src/ReaK/core/base/cnst...
which I explain in more details here (warning: it's a bit more of a tutorial for beginners, not aimed at experts):
https://www.daniweb.com/software-development/cpp/code/482276/c11-compile-tim...
Thanks for the information, I know some of them, will check out later.
@ Abel:
So the way I'd use it would be something like: ... Did you manage to use these strings as template arguments somehow?
AFAIK, you can use any of the above-listed implementations of constexpr strings in the way that you showed, but not the one discussed here.
I think Abel needs the value encoded in the type, so something like Sprout's string may not help.
I think Abel needs the value encoded in the type
Ok. Now I get it. That was the critical piece of information I did not get from the previous communications. Thanks for clearing that up. I think that the term "compile-time string" led to the confusion, because there is already a pretty good tradition of compile-time string construction and manipulation. In the sense that it pre-computes or pre-parses strings during compilation. The idea of computing types out of compile-time strings is a different problem altogether. And I would not call this "compile-time strings", I would call them "meta-programming strings", because that's what they are, strings used to construct types (i.e., meta-programming). The terms used completely threw me off there.
I can't see a straightforward way to pass such constexpr string as a template parameter and do something reasonable with it internally.
Yeah, sorry, my bad. I completely missed that point somehow. I guess the "metafunction" should have been a hint, but I guess it must be getting late. You're right, the barrier to cross from constexpr to template parameters is pretty much impenetrable as far as strings are concerned (btw, you can easily use a constexpr string to generate a unique hash that can be used to construct a type that is unique to the original string, but I guess that's not quite what you are looking for). So yeah, you're right, you cannot do that with compile-time strings.
My intention was to implement a compile-time Spirit that operates on CT string and returns different types depending on the input, so my usage is a bit different.
Ok, I see, you want to remain purely TMP the whole way, and the end-goal of your expressions are types, not strings. (BTW, if your goal is compile-time string parsing, then you ought to be using constexpr strings, not TMP strings, because as much overhead as there is when using constexpr for this kind of work, I would imagine the overhead of using TMP for this would be astronomical, intractable) In that perspective, I can totally see how your MACRO might make sense, if it was legal. The trick really is to circumvent the standard rules that essentially make it impossible to map a string literal into a template parameter pack of chars. I think that with your use of the lambda expression, you tried to cheat your way out of those rules, but there is a specific rule for that loophole, $5.1.2/2: "A lambda expression shall not appear in an unevaluated operand" This means that you cannot try to smuggle a string literal out of an unevaluated lambda in order to get it into a type. And the rationale for this rule is specifically aimed at avoiding opening the can of worms that it would be if people were allowed to do meta-programming from lambda's, because you could hide all sorts of monstrous things in a lambda and then pull them out into templates. That pretty much closes off your proposed avenue, doesn't it? If you got this to work, I would have to imagine it is due to working with a non-compliant compiler. I tried your code with both clang 3.5 and GCC 4.8.2, and they both reject it on the grounds of the rule that I just stated. You'll have to find another way to do this. It is a tough problem. And when it comes to working with string literals, I'm afraid the pre-processor is pretty much all you have, as Abel evidently realized when creating the MPLLIBS_STRING macro. I personally find this problem very frustrating because we all know that all compilers could do a lot more work with strings at compile-time, either as constexpr or as template parameters. And we wouldn't have to resort to such ridiculous and inefficient solutions as macro expansions or template instantiations with packs of chars. Sorry again for my initial misunderstanding of your aim with the code you showed. Cheers, Mikael Persson
Mikael Persson wrote:
$5.1.2/2: "A lambda expression shall not appear in an unevaluated operand" ... That pretty much closes off your proposed avenue, doesn't it?
Only in the case of metaprograms. The proposed technique still may be used in a place where a lambda expression could be evaluated, so when it's passed as a function parameter and the return type is deduced in compile-time. Regards, Adam
participants (6)
-
Abel Sinkovics -
Adam Wulkiewicz -
Louis Dionne -
Mikael Persson -
pfultz2 -
TONGARI J