- Boost - lists.boost.org

typedefs within Concept or as Trait?
by Michael Fawcett 11 Jan '08

11 Jan '08

This question doesn't pertain to a particular Boost library, so I apologize if it's too off-topic for the list. According to http://www.boost.org/more/generic_programming.html , the Associated Types of a Concept are either nested in the class definition, or accessed through a traits class. With iterators, traits were used because you can't nest typedefs within pointers, which are valid iterators in many cases. With containers (such as AssociativeContainerConcept), the Concepts require nested typedefs. My questions are - when creating your own library, when do you choose nested typedefs over a traits class, and what effects does it have on a library user, both from a usability and extensibility point of view. What questions do you ask yourself to come to a decision? Thanks, --Michael Fawcett

5 9

Status 1.35.0
by Peters, Richard 11 Jan '08

11 Jan '08

Hello, Some time ago (I think it was november or december), it was mentioned somewhere that the 1.35.0 would be targeted for the end of 2007. Obviously it hasn't been released yet. What is the expected date at which boost 1.35.0 will be released? Our project is currently using 1.33.1, and I'm trying to decide if we should wait for 1.35, or go for 1.34 for now. Best regards, Richard Peters This message and attachment(s) are intended solely for use by the addressee and may contain information that is privileged, confidential or otherwise exempt from disclosure under applicable law. If you are not the intended recipient or agent thereof responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately by telephone and with a 'reply' message. Thank you for your co-operation.

2 1

[Logging] Loglite v 0.4.0 - Getting to something
by JD 11 Jan '08

11 Jan '08

Hello, Loglite, the small logging library is back with new features. Issues fixed (http://code.google.com/p/loglite/issues/list) 2 - Manage channel 3 - mask instead of level 4 - boost::call_once in logger::get_instance 5 - crashes in multi-threaded 6 - Call stack pb in multithreaded environment 7 - Memory leak and unflushed files on unix systems 8 - Eclipse project files 9 - client/server example crashes 10 - Scope logging 11 - Add function and pretty function name as log element *Scope Logging*: ex:http://loglite.googlecode.com/svn/0.4.0/test/logging_test_scope.cpp You can use a special scope object which log at creation and desctruction. Now the logger maintains a stack of scope and a scope_element has been added to display the stack status in the trace itself. int macro_bar() { BOOST_LOG_SCOPE(BOOST_LOG_LEVEL_1, bar_scope); BOOST_LOG_L1("Doing some processing in macro_bar"); } *Element Formatting*: ex:http://loglite.googlecode.com/svn/0.4.0/test/logging_test_format.cpp Now the date, time and filename element can be formatted. Date and Time can be formatted thanks to the Boost.DateTime format. The filename element can be configured to print it's path or only the filename. boost::logging::format file_format(" L" >> bl::mask >> "," >> (*new bl::filename_element(bl::filename_element::filename_only)) >> "(" >> bl::line >> ")," >> (*new bl::date_element("%d/%m/%Y")) >> " " >> (*new bl::time_element("%H:%M:%S%F")) >> "," >> bl::trace >> bl::eol); *New elements*: to print function name, function signature, qualifier, scope, etc. *CMake makefile*: Compile using CMake. And still... *Log over the network*: ex:http://loglite.googlecode.com/svn/0.4.0/test/logging_test_iostream_clien… and http://loglite.googlecode.com/svn/0.4.0/test/logging_test_iostream_server.c… To centralized the logging of several executables, running on different machines, thanks to Boost.Asio. *Log with shared memory*: ex.:http://loglite.googlecode.com/svn/0.4.0/test/logging_test_iostream_shme… Avoid slowing down your process. With the shared memory, a independent thread is actually writing the trace to the output stream. *Log directly in zip and/or encrypted format*: ex:http://loglite.googlecode.com/svn/0.4.0/test/logging_test_iostream.cpp Thanks to Boost.Iostreams you can log to a encrypted and/or compressed file very easily. Many thanks to Ilya Pokolev and others for their help. The homepage is here: http://code.google.com/p/loglite/ The SVN repository is here: http://loglite.googlecode.com/svn/trunk/ A tarball is available here: http://loglite.googlecode.com/files/loglite-0.4.0.zip Feel free to ask questions, give comments and remarks, and better, participate to the development. Jean-Daniel

2 3

Re: [boost] [Boost-users] formal review of Switch library ends tomorrow (Wednesday, January 9th) - reviews needed
by Stjepan Rajko 10 Jan '08

10 Jan '08

On Jan 9, 2008 6:55 AM, Tobias Schwinger <tschwinger(a)isonews2.com> wrote: > Stjepan Rajko wrote: > > * What is your evaluation of the design? > > A default case function object returning 'void' should be assumed (and > 'assert'ed) not to return in a context where a (non-void) result is > expected (implementation hint: the built-in comma operator allows void > arguments and an overloaded comma operator doesn't). > I can see why having a non-returning default case function object would be a common use case (e.g., throw on non-specified case), but technically, why should the default case be treated differently than the regular cases in this respect? E.g., what if I wanted to throw on case 3 but return something by default? It seems that the cleanest solutions here would be to either allow void-returning function objects (when a return type is expected) both for regular and default cases (and assert that the call does not return), or disallow the void-returning function objects all together (when a return type is expected). Just my 2 cents... Regards, Stjepan

1 0

Trac open bugs - by assignee
by Marshall Clow 10 Jan '08

10 Jan '08

[ 194 open bugs today, 195 yesterday ] If you are maintaining a library, and have not logged into the trac system at <http://svn.boost.org/trac/boost/>, please do so ASAP. Until you do that, no tickets can be assigned to you, and will remain under "None". Instructions on working with tickets can be found at: <http://svn.boost.org/trac/boost/wiki/TicketWorkflow> and a mapping of user names to "real names" can be found at: <http://svn.boost.org/trac/boost/report/16> Bug summary 18 az_sw_dude 17 dgregor 15 djowel 12 doug_gregor 11 grafik 11 dave 11 agurtovoy 11 None 8 vladimir_prus 8 jsiek 7 johnmaddock 7 anthonyw 6 rogeeff 5 witt 4 bemandawes 3 pdimov 3 nesotto 3 jmaurer 3 igaztanaga 3 ebf 3 danielw 2 urzuga 2 turkanis 2 pavol_droba 2 nasonov 2 matias 2 danieljames 2 <Blank> 1 t_schwinger 1 speedsnail 1 shammah 1 samuel_krempp 1 marshall 1 kaalus 1 hljin 1 hkaiser 1 dlwalker 1 burbelgruff 1 aaron_windsor <Blank> 1307 [lambda] invalid boostbook <Blank> 1434 Boost.Preprocessor_ small fix for the documentation None 1068 Mersenne twister disables streaming for Visual C++ None 1094 Finding the correct library to link (feature request for pkg-config support)h None 1222 lambda::ret<T> should declare result_type. None 1247 Typos in "boost/random/additive_combine.hpp"_ lines 68-69 None 1270 uniform_smallint doesn't compile with lagged_fibonacci random number generators None 1407 Boost is doesn't install on MacPorts using Leopard None 1493 SVN files making it through to install None 1515 [lambda] C4512 warnings from VC8.0 None 1544 uniform_01 w/ref template param causes ref to ref error None 1546 Method boost::random::uniform_int::operator()(engine_ value) is missing None 1548 [variant][docs]typo: formal arg_ operand_ should be v in times_two arglist aaron_windsor 778 top-level configure is broken agurtovoy 1001 gcc 4.1.2 segfaults on mpl/test/apply.cpp agurtovoy 1051 MPL header ordering triggers bug agurtovoy 1099 unable to iterate through a "non-explicitly-declared" mpl::set agurtovoy 1215 Boost 1.34.1_ mpl: Wrong workaround detection using MinGW32 (or.hpp_ line #33) agurtovoy 1317 has_xxx randomly broken in msvc agurtovoy 1469 [mpl] nested typedef of set has two different meanings agurtovoy 1476 always<> is not nullary. agurtovoy 1528 g++ 4.3.0 reports error because of redeclaration of next agurtovoy 1549 Boost.MPL: 'index_of' algorithm is not documented agurtovoy 588 mpl::remove compile error with gcc 4.1.0 agurtovoy 640 documentation mistake anthonyw 1154 tss destruction segfaults in conjunction with dlopen/dlclose anthonyw 1168 [thread] tss crashes randomly anthonyw 1268 xtime_cmp changes to deal with overflow were reverted anthonyw 1470 Boost.Thread: Compilation errors with non-__cdecl calling convention anthonyw 1547 [thread] error C2065: 'NULL' : undeclared identifier in thread_primitives.hpp anthonyw 876 xtime documentation incomplete anthonyw 881 errors when build with Apache stdcxx az_sw_dude 1028 typo in boost header: date_time/time_defs.hpp az_sw_dude 1078 from_iso_string cannot read the output from to_iso_string az_sw_dude 1098 Urgent libboost-date-time1.33.1 : bugs on ia64 ? az_sw_dude 1147 date_time library contains global #define ERROR az_sw_dude 1160 warning: month_format shadows a member of this az_sw_dude 1162 warning: ticks shadows a member of this az_sw_dude 1289 posix_time::time_facet forgets to pass a_ref argument to base class az_sw_dude 1299 change in typedef stringstream_type in format_date_parser.hpp causing compilation error az_sw_dude 1306 invalid docbook in date_period.xml az_sw_dude 1384 Linking problem with date_time library on Visual C++ 2008 Express az_sw_dude 1386 date_time documentation Jamfile breaks building of boost docs az_sw_dude 1524 format_date_parser.hpp(221) : cannot convert parameter 1 az_sw_dude 287 wrong usage of ios_base::narrow az_sw_dude 316 date_time type conversion warning az_sw_dude 604 from_ftime incorrectly processes FILETIME filled with zeros az_sw_dude 642 operator<< for gregorian::date_duration not found az_sw_dude 889 Insane Dependencies az_sw_dude 890 date_time library supports only the current dst rule bemandawes 1378 filesystem path invalid name for hidden files (LINUX) bemandawes 1389 filesystem::ostream << boost::format generates operator ambigious error (in MSVC++) bemandawes 1458 Darwin Tiger throws std::runtime_error: locale::facet::_S_create_c_locale name not valid bemandawes 752 directory_iterator doesn't work with catch burbelgruff 1503 typeof bug (big integral constant) danieljames 1554 When BOOST_HASH_NO_EXTENSIONS is defined hash doesn't support new string types. danieljames 1555 When BOOST_HASH_NO_EXTENSIONS is defined hash doesn't support boost::system::error_code danielw 1044 Argument pack inaccessable in a functions return type danielw 1054 Parameter takes an "infinite" amount of time to compile with the Intel compiler. danielw 1097 keyword.hpp generates many warnings C4180: qualifier applied to function type has no meaning; ignored dave 1010 linking boost.python on freebsd dave 1019 iterator_facade::pointer should be the return type of operator-> (DR 445) dave 1085 Python rules for compiling in Mac OS X dave 1179 [boost.python] can not export a union in VC2005sp1 dave 1181 [boost.python] can modify enum value dave 1387 Review / update docs to reflect changes dave 1460 Python classes with multiple bases are convertible only to the first class in the lists dave 1482 boost::python::throw_error_already_set should be marked noreturn dave 1516 [python] Failing VC9 regression tests dave 865 Use of C++0x keyword as identifier dave 910 gcc strict-aliasing problems with python dgregor 1003 [iostreams] copy-paste error in typedefs for wchar multichar_filters dgregor 1139 [iostreams] Regex filter segfaults when zero matches found dgregor 1140 boost::iostreams linked_streambuf bug dgregor 1149 Minor doc error in basic_null_device dgregor 1164 Floyd Warshall broken with unsigned edge weights dgregor 1185 iostreams::direct_streambuf<T_Tr>::is_open always returns false if output iterator is not null dgregor 1192 [boostbook] problem when two files have the same name dgregor 1199 configure doesn't work with path containing whitespaces (MacOS X) dgregor 1232 mapped_file_source::is_open() is broken under windows dgregor 1310 [result_of] const-qualified function pointer doesn't work dgregor 1326 Unable to check graph isomorphism using LEDA adapter dgregor 1391 dominator_tree_test on Borland/CodeGear outputs overly long line dgregor 1394 Translate libs/graph/build/python/Jamfile to BBv2 dgregor 1395 Write Tests for Python Bindings dgregor 556 Bundled graph properties dgregor 732 Johnson All-Pairs needs better "no path" information dgregor 736 Const correctness violation djowel 1170 Emit Warnings instead of # symbols in the syntax highlighter djowel 1171 Quickbook title output djowel 1172 non-utf djowel 1176 Space after inline code djowel 1206 Escaping from within code? djowel 1273 CR+LF newlines in position_iterator djowel 1276 spirit problem with _GLIBCXX_DEBUG djowel 1277 Closing section like [pre with } causes Quickbok to crash djowel 1280 sections in imported source files get put inside <para> elements in docbook output djowel 1281 Quickbook produces invalid BoostBook/DocBook XML djowel 1324 tuple should have a swap() that operates element-by-element djowel 1396 wrong result_of invocation around transform_view djowel 1436 [quickbook] inline code inside headings djowel 1484 Quickbook produces illegal docbook for some docinfo fields. djowel 1500 [Spirit/Phoenix] C4512 Compiler warnings from VC8 dlwalker 613 boost/crc.hpp uses non-standard conforming syntax doug_gregor 1021 [Graph][reverse_graph] Problem using reverse_graph adapter with an adapted user defined graph doug_gregor 1076 compile error: disconnecting struct with operator()() doesn't work in VS2005 SP1 doug_gregor 375 LEDA graph adaptors do not handle hidden nodes properly doug_gregor 402 random_vertex/random_edge are unnecessarily inefficient doug_gregor 403 Document copy_component doug_gregor 447 reverse_graph and constness of property maps doug_gregor 708 Dijkstra no_init version should not require VertexListGraph doug_gregor 733 Numbered headers don't work with "preferred" syntax doug_gregor 735 Fruchterman-Reingold grid performance can be improved doug_gregor 738 Memory leaks with signal::connect? doug_gregor 829 Implicit graph not implement doug_gregor 852 Problem with Boost and GCC 4.1 ebf 1507 [variant] C4512 warnings from VC8.0 ebf 965 [doc] boost::variant tutorial - final example uses v1_v2 should be seq1_seq2 ebf 993 Variant should use least common multiple of alignments_ rather than maximum grafik 1041 Cygwin requires PATH setting. grafik 1062 Cygwin install stage fails with combination of patches grafik 1322 trac rejects patch attachment as spam grafik 1417 Regresson tests hang in Darwin (Tiger) grafik 1430 Regression fails under Cygwin grafik 1433 Trac's spam detection rejects too many valid messages grafik 1512 Broken link to vacpp in bjam docs. grafik 1513 Broken link to bcpp in bjam docs. grafik 616 Boost Jam_ and non english directorys grafik 959 linking fails when Boost is compiled with STLport 5.1.* on Linux (names mismatch) grafik 986 Problem building Python modules on boost 1.34.0 hkaiser 1079 boost-1.34.0 fails to compile. concept checks not satisfied. hljin 1522 bit_range friend decl in bit_aligned_pixel_reference.hpp needs to be fully qualified for gcc 4.3 mainline igaztanaga 1080 boost::interprocess win32 global file mapping issue igaztanaga 1210 interprocess::barrier hangs during second use under linux (boost 1.33.1) igaztanaga 1390 Interpocess: managed shared memory failing completely under OS X jmaurer 351 Diff in state of mersenne_twister gen between GCC3.41 & CW9 jmaurer 599 uniform_01 copies engine instead of using a reference jmaurer 819 uniform_int<> with type's maximum availaible range error johnmaddock 1081 Empty string rejected as perl-style regex johnmaddock 1314 Using SSE-intrinsics in VC8 causes static assert in type_with_alignment.hpp johnmaddock 1411 Fix BOOST_PARTIAL_SPECIALIZATION_EXPLICT_ARGS johnmaddock 1440 libs/config/test/link/Jamfile.v2 confuses process_jam_log johnmaddock 1446 POSIX API doesn't use C locale johnmaddock 1451 error: use of C99 long long integer constant johnmaddock 1511 Non latin symbols in regexp jsiek 1531 segfault in dynamic_bitset::reference assign operator jsiek 1540 Poisson distribution very slow for large mean (and may also overflow). jsiek 416 Spelling of Edmonds-Karp-Algorithm jsiek 575 Calling subgraph::global_to_local on a root graph jsiek 636 strange compiling problem for transitive_closure jsiek 815 remove_edge core dumps on self-circle jsiek 875 No iterator based constructor for adjacency_matrix jsiek 900 graphml reader namespace handling is broken kaalus 1184 property_tree assignment (operator) bug marshall 1252 [pool] severe overhead with unaligned sizes. matias 1406 Understanding the Tables outdated matias 1539 Missing semi-colon in random library quitck start example nasonov 1220 lexical_cast requires RTTI nasonov 754 boost::any - typeid comparison across shared boundaries nesotto 1284 [range] sub_range assignment issue nesotto 1464 boost_range_array unused in several array_size() functions nesotto 1565 iterator_range disables msvc warning 4996 pavol_droba 1152 rle_example fails pavol_droba 1385 begin must be boost-qualified pdimov 1106 shared_ptr<T> should not convert to shared_ptr<U> when T* doesn't convert to U* pdimov 1444 scoped_ptr::operator== should be equivalent to a.get()==b.get()_ but is currently !!a==!!b pdimov 779 gcc-4.2 atomicity.h location rogeeff 1383 signal.h features not found on Mac OS rogeeff 1489 BOOST_CHECK_CLOSE fails to compile with templated value type rogeeff 1542 [test] compiler errors when building with stdcxx library rogeeff 1560 Performance testing with boost::test rogeeff 1561 Using BOOST_AUTO_TEST_CASE with boost::program_options rogeeff 1566 Missing example files in documentatio samuel_krempp 859 boost::format ignores a user defined locale shammah 386 boost::pool_allocator breaks with vector of vectors speedsnail 1138 Problems with TSS and static thread library t_schwinger 1100 Building boost.function_types fails cvs rev-1.1_ no chdir command in bash turkanis 1523 [iostreams] VC8/VC9 compiler warnings turkanis 1545 [boost/rational.hpp] 'noskipws' is not a member of 'std' urzuga 574 [boost::lambda] Compile error with libstdc++ debug mode urzuga 781 Lambda: (_1 + "y")(string("x")) Doesn't Compile vladimir_prus 1165 more quoting problems vladimir_prus 1274 Useless error message vladimir_prus 1321 invalid docbook in tool documentation vladimir_prus 1527 [program_options] Second (and subsequent) lines in tabbed text not aligned correctly vladimir_prus 1530 Program options documentation - Overview - Syntactic Information example typo vladimir_prus 1537 no value given in 'mysql_password =' vladimir_prus 469 multitoken broken vladimir_prus 689 [program_options] Endless loop with long default arguments witt 1046 Boost.Python quickstart instructions mention nonexistent target witt 583 Fixes for build on IBM pSeries for AIX and Linux witt 834 Homepage regression links witt 957 The "Getting Started" page does not mention the stdlib option witt 973 zip_iterator has value_type == reference -- -- Marshall Marshall Clow Qualcomm, Inc. <mailto:mclow@qualcomm.com> The famous British scientist, Lord Kelvin, said: "When you measure what you are speaking about and express it in numbers, you know something about it, but when you cannot (or do not) measure it, when you cannot (or do not) express it in numbers, then your knowledge is of a meagre and unsatisfactory kind."

1 0

boost::execution_monitor impl under windows
by James Talbut 10 Jan '08

10 Jan '08

From searching, this was last discussed in Sep 2003, but didn't come to an adequate conclusion - and things have changed since then anyway. Including prg_exec_monitor.hpp on VS2005 produces the following warning: execution_monitor.ipp(216) : warning C4535: calling _set_se_translator() requires /EHa It is my opinion that the use of _set_se_translator here is a mistake for the following reasons: 1. It doesn't achieve what it's trying to achieve if my test involves multiple threads. The translator is thread specific and there is no way for my thread function to apply the translator (especially not when I may not have access to the thread function). 2. It requires the use of /EHa which has an effect on the code being run. Code under test should be run using the same options as release, and /EHa is not appropriate for (most) release code. 3. If I've really triggered a system fault then I can no longer trust the state of my app, so I cannot rely on it to correctly report its problem. I may have just destroyed the stack, attempting to unwind is just going to cause more problems - system errors should be handled externally. I accept that others may prefer the simplicity of trying to handle system failures internally, so I propose changing to execution_monitor.ipp to only define BOOST_MS_STRUCTURED_EXCEPTION_HANDLING if BOOST_NO_MS_STRUCTURED_EXCEPTION_HANDLING is not defined: # ifndef BOOST_NO_MS_STRUCTURED_EXCEPTION_HANDLING # define BOOST_MS_STRUCTURED_EXCEPTION_HANDLING # endif I will then carry out my testing after running something like: "Program Files\Debugging Tools for Windows\cdb" -iaec "-c \".lines;!analyze -v;kP 4;q\"" to catch system errors. Thoughts? Jim PS. What I'd really like to see is a cross platform out of process JIT reporter for just one application. . ________________________________________________________________________ This e-mail, and any attachment, is confidential. If you have received it in error, do not use or disclose the information in any way, notify me immediately, and please delete it from your system. ________________________________________________________________________

2 5

bounded_iterator
by Chad Parry 10 Jan '08

10 Jan '08

I created a "bounded_iterator" concept that seems really useful. A bounded_iterator behaves just like a regular iterator, except that the termination condition can be checked by just casting the iterator to a bool. Here's an example: int array_range[] = { 1, 1, 2, 3, 5 }; for (bounded_iterator<int *> i(array_range); i; ++i) { std::fill_n(std::ostream_iterator<int>(std::cout), *i, *i); } I like it for the same reason that I like iterator_range: it frees the developer from having to worry about both a begin and an end iterator. It can be used as a wrapper for operations on a container or a sub-range. Just like iterator_range simplifies the invocation of functions for ranges, bounded_iterator simplifies the traversal of ranges. In my application I was storing lots of iterators in containers, and using bounded_iterator proved to be a better abstraction than storing iterator pairs or ranges. The main disadvantage to using a bounded_iterator is that it competes with the standard iterator concept, and it's usually more confusing to support many ways to do the same thing. Here is a simple implementation: template<typename base_iterator> class bounded_iterator : public boost::iterator_adaptor<bounded_iterator<base_iterator>, base_iterator> { base_iterator const end; typedef boost::iterator_adaptor<bounded_iterator, base_iterator> super_type; struct unspecified_bool_helper { int true_tag; }; typedef int unspecified_bool_helper::*unspecified_bool_type; public: template<typename source_iterator> bounded_iterator(source_iterator begin, source_iterator end) : super_type(begin), end(end) { } template<typename source_range> bounded_iterator(source_range &range) : super_type(boost::begin(range)), end(boost::end(range)) { } template<typename source_range> bounded_iterator(source_range const &range) : super_type(boost::begin(range)), end(boost::end(range)) { } operator unspecified_bool_type() const { return super_type::base() == end ? 0 : &unspecified_bool_helper::true_tag; } }; -- chad

3 4

Re: [boost] formal review of Switch library ends tomorrow (Wednesday, January 9th) - reviews needed
by Tobias Schwinger 10 Jan '08

10 Jan '08

Reposting to boost.devel. Steven Watanabe wrote: > AMDG > > Tobias Schwinger <tschwinger <at> isonews2.com> writes: > >> As pointed out in other posts I think the default default case behavior >> should be changed not to throw but to use the default constructor if no >> default case function object is specified. > > Will do. > >> A default case function object returning 'void' should be assumed (and >> 'assert'ed) not to return in a context where a (non-void) result is >> expected (implementation hint: the built-in comma operator allows void >> arguments and an overloaded comma operator doesn't). > > Personally, I'd rather just forbid a default that returns void > when the return type is non-void. The rationale was to allow non-returning default function objects that work regardless of the type (such as 'throw_'). It's probably not that important... > I don't think the comma operator > is a completely fool-proof way to detect void. It is. > It could cause an > ambiguity if the other type also defines an overloaded comma operator. No, because the operator is binary and one operand can have a reserved type which is only internally used by the test code. Regards, Tobias

1 0

Again: No mail for bug submitter if bug is closed
by Jens Seidel 10 Jan '08

10 Jan '08

Hi, yesterday bug #1550 which was opened by me (and has me in CC: too!) was closed but I still miss an associated mail. Can you please check the mail log whether the mail was rejected? It happened in the past that mails could not be delivered to me if a postmaster account is missing on the server. Can you check this? What is the problem? (Yep, I checked my SPAM folder.) Jens

2 1

Strings tagged with their character set
by Phil Endecott 10 Jan '08

10 Jan '08

Dear All, Something that I have been thinking about for a while is storing strings tagged with their character set. Since I now have a practical need for this I plan to try to implement something. Your feedback would be appreciated. The starting point is the idea that the character set of a string may be known at compile time or at run time, and so two types of tagging are possible. First compile-time tagging: template <character_set> class tagged_string { ... }; tagged_string<utf8> s1; tagged_string<latin1> s2; Some typedefs would be appropriate: typedef tagged_string<utf8> utf8string; Now run-time tagging: class rt_tagged_string { private: character_set cs; public: rt_tagged_string(character_set cs_): cs(cs_) ... ... }; rt_tagged_string(utf8) s3; (Consise-yet-clear names for any of these classes would be great.) I propose to implement conversion between the strings using icode and/or GNU recode. It would be easy to allow this conversion to happen invisibly, but it might be wiser to make conversion explicit. I'm not sure what the 'character_set' that I've used above should be. It needs to be some sort of user-extensible enum or type-tag. We need character types of 8, 16 and 32 bits. wchar is not useful here because it's not defined whether it's 16 or 32 bits. So I propose the following, modelled after cstdint: typedef char char8_t; typedef <implementation-defined> char16_t; typedef <implementation-defined> char32_t; I then propose a character_set_traits class: template <character_set> class character_set_traits; template <> class character_set_traits<utf8> { typdef char8_t char_t; const bool variable_width = true; ... }; For the fixed-width, compile-time-tagged strings I think it makes sense to inherit from std::basic_string< character_set_traits<charset>::char_t >. The only problem I can see with this is that latin1string s1 = "hello world"; s1.substr(1,5) <--- this returns a std::string, not a latin1string If latin1string has a constructor from std::string (which is its own base type) that's fine, i.e. we can still write: latin1string s2 = s1.substr(1,5); but unfortunately we can also write latin2string s3 = s1.substr(1,5); which is not so good. So a different approach is to define a set of character-set-specific character types, and build string types from them: typedef char8_t latin1char; typedef char8_t latin2char; For variable-width character sets, the methods of std::string are less useful (though far from useless). I understand that there's already a utf8 iterator somewhere in Boost, can it help? For run-time character sets, is there any way to provide e.g. run-time iterators? I imagine these strings being used as follows: - Input to the program is either run-time or compile-time tagged with any character set. - Data that is not manipulated in any way it just passed through. - Data that will be processed will first be converted to a suitable, compile-time-tagged, character set, and if appropriate converted back afterwards. So the absence of (useful) string operations on run-time-tagged or variable-width character set data is not a problem. For conversions, there is the question of partial characters in variable-width character sets. If a program is processing data in chunks it may be legitimate for a chunk boundary to fall in the middle of a UTF8 character. IIRC, icode has a method to deal with this which we could expose in a stateful converter: charset_converter utf8_to_ucs4(utf8,ucs4); while (!eof) { utf8string s = get_chunk(); ucs4string t = utf8_to_ucs4(s); send_chunk(t); } utf8_to_ucs4.flush(); - but many applications may only need a stateless converter. I will be working on this over the next couple of weeks, so any feedback would be much appreciated. Regards, Phil.

16 55