modifed boost::stringtok

From: hicks (hicks_at_[hidden])
Date: 2002-02-06 09:52:12

Hello,

I have created a modifed version of boost::stringtok() (the template
function in boost\pending)
with the aim of proving a more convient programmer interface.
(It is called stringtok_2() below, but the interface is identical to
stringtok()).

The present version expects a container of objects which can be assigned
a std::string.
The modifed version is designed to accept a container of objects which
can be converted to from std::string,
at present "int" and "float" in addition to "std::string".

E.g.

char ac[] = "1 2 3 4 5";

std::vector<int> vi;
boost::stringtok_2(vi,ac);

std::vector<float> vf;
boost::stringtok_2(vf,ac);

std::vector<std::string> vs;
boost::stringtok_2(vs,ac);

As you can see in the cases for "int" and "float", the programmer can
avoid having to create a vector of std::string
and then convert them to int (or float) after calling stringtok.

The implementation is achieved through template specialization.

Behind the interface is an overloaded stringtok_2,
template<typename Container, class Cvt>
stringtok_2(Container l, Cvt cvt, const std::string s, const std::string ws)
which takes a functor argument cvt which does the string to type conversion.

The hithertoe used interface
template<typename Container>
stringtok_2(Container l, const std::string s, const std::string ws)
calls the overloaded stringtok_2 with a converting functor
string_cvt_ftr<Container::value_type>.

string_cvt_ftr has a default declaration and two specializations, one
for "int" and one for "float".
template <class T>
struct string_cvt_ftr
{
T operator()(const std::string& s) { return s; }
};
template <>
struct string_cvt_ftr<int>
{
int operator()(const std::string& s) { return std::atoi(s.c_str()); }
};
template <>
struct string_cvt_ftr<float>
{
float operator()(const std::string& s) { return std::atof(s.c_str()); }
};
That is the partial specialization.

Note that a vector of std::string can still be passed as the container
argument.

Since the interface semantics of are a superset of boost::stringtok(), I
wonder if
this could be substituted as a version-up, and use the same name.
However, judging from the mail I see on this list, it is possible that
some compilers
might not handle it, so maybe that wouldn't be a good idea.

I am sending this to boost-users since I do not currently recieve plain
boost mail.

I include the full code (including test code) below.

Craig Hicks
Engineer, KGK Tokyo

////////////////////////////////////////////////////////////////////
// stringtok_2.h

//---------------------------------------------------------------------------

#ifndef stringtok_2H
#define stringtok_2H
//---------------------------------------------------------------------------

#include <string>
#include <cstring> // for strchr

/*****************************************************************
// modified with partial template specialization for int, float
containers 2002.2.6
// Craig Hicks

/*****************************************************************
* This is the only part of the implementation that I don't like.
* It can probably be improved upon by the reader...
*/
namespace stringtok_2_ns
{
inline bool
isws (char c, char const * const wstr)
{
using namespace std;
return (strchr(wstr,c) != NULL);
}

}

namespace boost
{

/*****************************************************************
* Simplistic and quite Standard, but a bit slow. This should be
* templatized on basic_string instead, or on a more generic StringT
* that just happens to support ::size_type, .substr(), and so on.
* I had hoped that "whitespace" would be a trait, but it isn't, so
* the user must supply it. Enh, this lets them break up strings on
* different things easier than traits would anyhow.
*/

template <typename Container, typename Cvt>
void
stringtok_2
(
Container &l, Cvt cvt, std::string const &s,
char const * const ws = " \t\n"
)
{
using namespace stringtok_2_ns;

typedef std::string::size_type size_type;
const size_type S = s.size();
size_type i = 0;

while (i < S) {
// eat leading whitespace
while ((i < S) && (isws(s[i],ws))) ++i;
if (i == S) return; // nothing left but WS

// find end of word
size_type j = i+1;
while ((j < S) && (!isws(s[j],ws))) ++j;

// add word
l.push_back(cvt(s.substr(i,j-i)));

// set up for next loop
i = j+1;
}
}

template <class T>
struct string_cvt_ftr
{
T operator()(const std::string& s) { return s; }
};
template <>
struct string_cvt_ftr<int>
{
int operator()(const std::string& s) { return std::atoi(s.c_str()); }
};
template <>
struct string_cvt_ftr<float>
{
float operator()(const std::string& s) { return std::atof(s.c_str()); }
};

template <typename Container>
void
stringtok_2
(
Container &l, std::string const &s,
char const * const ws = " \t\n"
)
{
string_cvt_ftr<Container::value_type> cvt;
stringtok_2(l, cvt, s, ws);
}

} // namespace boost

#endif

///////////////////////////////////////////////////////////////////
// test code

void test_stringtok_2(std::ostream& os)
/*
Expected output::
int
1, 2, 3, 4, 5,
float
1, 2, 3, 4, 5,
string
1, 2, 3, 4, 5,
*/
{
char ac[] = "1 2 3 4 5";

os << "int" << std::endl;
std::vector<int> vi;
boost::stringtok_2(vi,ac);

std::copy(vi.begin(), vi.end(),
std::ostream_iterator<int,char>(os,", "));
os << std::endl;

os << "float" << std::endl;
std::vector<float> vf;
boost::stringtok_2(vf,ac);

std::copy(vf.begin(), vf.end(),
std::ostream_iterator<float,char>(os,", "));
os << std::endl;

os << "string" << std::endl;
std::vector<std::string> vs;
boost::stringtok_2(vs,ac);

std::copy(vs.begin(), vs.end(),
std::ostream_iterator<std::string,char>(os,", "));
os << std::endl;
}