Re: [Boost-bugs] [Boost C++ Libraries] #12859: std::string_view as token

Subject: Re: [Boost-bugs] [Boost C++ Libraries] #12859: std::string_view as token
From: Boost C++ Libraries (noreply_at_[hidden])
Date: 2017-04-25 20:22:21

#12859: std::string_view as token
-------------------------------------------------+-------------------------
  Reporter:  Johel Ernesto Guerrero Peña         |      Owner:  jsiek
  <johelegp@…>                                   |     Status:  new
      Type:  Feature Requests                    |  Component:  tokenizer
 Milestone:  To Be Determined                    |   Severity:
   Version:  Boost 1.63.0                        |  Optimization
Resolution:                                      |   Keywords:  string_view
-------------------------------------------------+-------------------------

Comment (by Dam <damian.meden@…>):

 Few things here,

 Base on what the tokenizer is, which basically it's a non owning helper
 that relies on the fact that someone else owns the std::string, and the
 connection between the owner and the tokenizer class is a pure pair of
 iterator, this make me think that it can be a little bit ambiguous, like,
 adding one layer between the std::string and the tokenizer, so instead of
 :
 Std::string -> boost::tokenizer now it will become std::string ->
 std::string_view -> boost::tokenizer. See my point? This could be a good
 discussion, '''I would like to know your thoughts about this.'''
 Also, from top of my mind I can think that this is still in most compilers
 experimental features, so that may be an issue to deal with at boost
 level. I would check that deep.

 Anyway, I did play a little with this and I came up with some POC, which
 can’t be added to boost anyway as it wasn’t properly tested, but at least
 it helped me to see how it may look like. You can check the diff here ->
 https://github.com/boostorg/tokenizer/compare/develop...dmeden:not_safe_string_view_use_v1


 Tested with GCC 6.3.

 '''Again, this code is not boost ready yet,''' there are many
 things(iterators, etc) that needs to be tested and validated that I didn’t
 put it into the code yet.

 Thanks,
 Dam

-- 
Ticket URL: <https://svn.boost.org/trac/boost/ticket/12859#comment:3>
Boost C++ Libraries <http://www.boost.org/>
Boost provides free peer-reviewed portable C++ source libraries.