- Boost - lists.boost.org

use of strcpy, strcat, etc in Boost libraries
by Reece Dunn 01 Oct '04

01 Oct '04

Hi All, In my post on the "vc8.0 toolset (bjam)" thread, I commented that VC8 also warns when using the str[n]xxx APIs, because MS declares them deprecated. There are several issues here: [1] How does this impact Boost code that uses the <cstring> API? Especially since they deprecate the strn variations as well, which are safer, and considering that the "safe" versions are not yet a part of the standard. Would it be possible to put this in the Boost <cstring> header? I.e. using the safe versions for VC8 and using the C9x API for the other compilers/libraries. [2] Does this affect the Intel compiler using the Dinkumware libraries because Intel does not like how the library declares items deprecated. (This is already an issue using the <locale> header and <hash_xxx> extensions). [3] What are the wider security issues of using strcpy, etc in Boost code? The libraries should use the strnxxx variants for better protection against buffer overrun attacks, but I found several instances where strcpy was used (I haven't checked for strcat, strcmp and others) instead of strncpy: boost/regex/v4/regex_cstring.hpp (104): use of strcpy [via re_strcpy] libs/regex -- various files using strcpy libs/serialization/test/test_tools.hpp (69): use of strcpy [via STRCPY] libs/smart_ptr/test/smart_ptr_test.cpp (139/223): use of strcpy tools/build/jam_src -- execnt.c; regexp.c; variable.c [4] Doing this, I found that regex was using re_strcpy as a workaround for Borland using strxxx as macros when using intrinsics. Serialization uses STRCPY for a similar workaround, and my fixed_string library uses the following: # if defined(__BORLANDC__) && defined(strcpy) // replace Borland instrinsic string function # undef strcpy inline char * strcpy( char * dest, const char * src ) { return( __strcpy__( dest, src )); } # endif So: should the workarounds be placed in <cstring> and affected libraries use the Boost version. NOTE: I suggested telling Boost.Build to disable the deprecated API warning for the upcoming release and then address these issues for the next (1.33) release. Thoughts? Comments? Regards, Reece

4 6

RE: [boost] Re: [shared_ptr] Interlocked* - possible bug?
by Ben Hutchings 30 Sep '04

30 Sep '04

John Torjo <john.lists(a)torjo.com> wrote: <snip> > But now, by looking again at atomic_conditional_increment. I'm not a > threading expert, but it seems pretty costly as it is implemented > now. Could it not go multiple times (n>=2) thorughout the for? Yes. > Current implementation: > > inline long atomic_conditional_increment(long volatile & value) > { > for(;;) > { > long tmp = value; > if(tmp == 0) return 0; > if(InterlockedCompareExchange(&value, tmp + 1, tmp) == tmp) > return tmp + 1; > } > } > > > Couldn't it become: > > inline long atomic_conditional_increment(long volatile & value) > { > long tmp = value; Suppose we read the count as 2 here... > if(tmp == 0) return 0; ...and that another thread decrements the count to 1 around here... > long new_val = InterlockedCompareExchange(&value, tmp + 1, tmp); > if(new_val == tmp) return tmp + 1; ...so that InterlockedCompareExchange fails and returns 1, so this doesn't return; and suppose that another thread decrements the count to 0 around here... > else return InterlockedIncrement(&value); ...then this increments the count to 1, which is wrong. > } The increment has to be conditional on the count being non-zero, and neither the Win32 API nor the x86 processor (AFAIK) can perform that atomically. So the loop is unavoidable. There is room for a slight optimisation in that the value returned by InterlockedCompareExchange can be used the next time round: inline long atomic_conditional_increment(long volatile & value) { long old = value, older; do { if (old == 0) return 0; older = old; old = InterlockedCompareExchange(&value, old, old + 1); } while (old != older); return old + 1; }

2 1

Output formatters: different approaches
by Vladimir Prus 30 Sep '04

30 Sep '04

During the review of the output formatters library, there were suggested several alternative approaches. For example, Gennadiy thinks the library is too complicated, and Hartmut suggests creating a library dual to Spirit -- which would be extra powerfull. That's not all. Independently, I've talked with Vaclav Blazek, who's the author of the Teng text template engine (http://teng.sourceforge.net/) which is also a similiar domain -- it's too about formatting objects. It would be great if we had some common foundation, so that I could output ptr_vector<Function> using both a simple formatter, or a flexible one, or even some template engine. Such foundation can be pretty simple: 1. A fixed set of type "categories" (sequence, map, tuple, something else) and a mechanism to obtain the category for a specific type. The current library even has some mechanism, but I'm not sure if's as simple as possible and is extentable. 2. A mechanism to iterate over elements of composite type. After some though, I believe that the mechanism should be boost::serializaton. Take, for example, sequences. The reviewed version of the library contains the code to handle boost:array, vector, list, and pair. The amount of code devoted for this is significant. If serialization library were used (it provides 'save' and 'load' functions for those types), the implementation would be simpler: The basic formatter could look like: class formatter { template<class T> void write(T &t, is_primitive) { os << t; } template<class T, class Category> void write(T &t , Category { os << start_delimiter<Category>(); os << t; os << end_delimiter<Category>() } template<class T> formatter& operator&(const T& t) { write(t); } }; The idea of the library, as I see it, is to provide object serialization, but - without the overhead serialization library has - with extra layout tweaks I don't see why it's necessary to reinvent the object traversal code which is already in boost::serialization. For example, boost::serialization support scoped_ptr. It would not make sense to add separate support for scoped_ptr in outfmt. And, as I've already mentioned I'd like the 'serialize' approach even for class members. Even if serialization headers which provide support for vectors, lists and so on drag too many dependencies, I think this is fixable. On the top of this, it's easy to provide various output facilities. - A simple multi-line indenting formatter. No customization at all, no overhead - A more advanced solution which uses nested formatters (like the revieved version), and stores formatting information in iword/pword - An interface with template system: e.g. Template t(...); vector<Function> f; t["functions"] = f; would traverse 'f' creating a tree of values which can be used in template. Thoughs? - Volodya

3 2

RE: [boost] Re: Re: [shared_ptr] Interlocked* - possible bug?
by Ben Hutchings 30 Sep '04

30 Sep '04

Peter Dimov <pdimov(a)gmail.com> wrote: > Ben Hutchings wrote: <snip> > > I feel the naming of functions may give a false sense of generality > > and encapsulation when they actually only work in the way they are > > being used currently. > > I'm not sure which functions do you have in mind here. I'm perhaps overstating the case, but still: Firstly, atomic_read. I think people generally expect atomicity and ordering to go hand-in-hand - though I realise they don't have to - and the other functions whose names begin with "atomic_" do prevent reordering of their memory accesses, so this name could be misleading. Secondly, use_count's return values can only be used in very specific ways (e.g. the comparison with 0 as an optimisation in weak_ptr<>::lock). I realise that the use_count functions in shared_ptr and weak_ptr are documented as "only for debugging and testing purposes", but I was kind of thinking the name ought to suggest that. (Also the documentation fails to point out that return values can immediately become incorrect in multithreading systems, and has postconditions based on use_count() that won't necessarily be true if tested in real code, though this perhaps ought to be obvious.) > http://66.102.9.104/search?q=cache:1ZQ1uXtFUEIJ:www.pdimov.com/cpp/ > shared_count_x86_exp2.hpp Thanks for that.

1 0

Re: [boost] Re: MultiMethod/Visitor Interest
by Danil Shopyrin 30 Sep '04

30 Sep '04

_____ From: Danil Shopyrin Sent: Monday, September 27, 2004 11:13 AM To: 'boost(a)lists.boost.org' Subject: RE: Re: [boost] Re: MultiMethod/Visitor Interest Hi! http://tinyurl.com/5qxky - it's a "preliminary" version of my solution. The [maybe] complete version can be found at http://www.codeproject.com/cpp/mmcppfcs.asp. And some of mentioned drawbacks are already eliminated. Background. My solution is an "idealized" solution. It is stands as an answer to question: "can multimethods be implemented in C++ using only 'good' programming techniques?" It is fully compile time and absolutely type safe. And there is no of RTTI. This is an academic result. Moreover, this is ready to practical use (I do it). There is a big resource of flexibility. By reducing the 'type safety sugar' this solution can be extended. Look on "Compilation in separate modules" section. And so on. I hope that the approach can be extended to met 'practical solution' requirements. But this is a hard job to one person to produce consistent set of these requirements. However, I think that "clear" solution is also practical in many cases. Further, about drawbacks: 1) Compile time. Yes, this solution is strongly compile time. But I think it is an advantage :-) 2) You are right. There is no "next_method". I was never think of them. 3) Intrusive. It is eliminated. 4) "distinguishes the first argument". It is eliminated. 5) Gnarly user code. Look at up-to-date "tiny use case" at http://sdanil.narod.ru/mm/m2vd3_v5.zip -- Danil

5 9

RE: [boost] Re: Call for Volunteers: the Synopsis project
by Stefan Seefeld 30 Sep '04

30 Sep '04

> From: Doug Gregor [mailto:dgregor@cs.indiana.edu] > Sent: September 30, 2004 10:30 > > I should mention that the design of the C++ parser allows > to manipulate > > the generated parse tree in-memory and write it out into a > file again > > in > > a non-lossy way, i.e. it could indeed be used as a source-to-source > > compiler. > > That's an excellent goal, and you have me interested. It's a > huge task, > requiring a great deal of expertise (collectively, we have that) and > will require a lot of time. I think it would be wise to attempt to > isolate some of the interesting-but-disjoint problems early, > so we can > put out a "request for libraries" of some sort. Things that come to > mind quickly: a unification algorithm (for template > instantiation/partial ordering), a tree > manipulation/rewriting library > (for AST transformations), and a flexible symbol table library. in what sense do you use the term 'library' here ? While I fully agree that is is very useful to approach this big task by splitting it into modules, I doubt it is feasible or even useful to use these entities as separate deployment units. However, I think we can come up with a high level design that respects the data flow requirements (the complex interactions between the parser and the symbol lookup table notably) yet is flexible enough to let the individual modules evolve in parallel. In fact, I have good hope that I could use the same framework to analyze C code (something I hope the Free Software community around GNOME / GNU will benefit from) by using some form of polymorphism in the lexer, the parser, and the symbol lookup. Regards, Stefan

2 1

[tokenizer and token_iterator] request for policy or redesign
by Thorsten Ottosen 30 Sep '04

30 Sep '04

Hi John, I vaguely recall that the standard committee rejected the tokenizer proposal because of performance problems. A thing that would boost performance a lot would be to allow the user to specify a "token match type". So I have some stupid questions: 1. Why is the three template paramters not defined as template < class TokenizerFunc = char_delimiters_separator<char>, class Type = std::string class Iterator = typename range_const_iterator<Type>::type > class tokenizer ?? 2. what is the requirement exactly on the Type parameter? 3. would it not be possible to allow typedef boost::tokenizer< fun, boost::sub_range<string> > tokenizer; ? I believe this would give a huge speedup. br Thorsten

6 23

[Serialization] Remaining failures markup
by Aleksey Gurtovoy 30 Sep '04

30 Sep '04

Robert, Pending msvc-stlport issue aside, the serialization library reports show quite a few failures with other toolsets, in particular: cw-9.3 borland-5.6.4 cw-8.3 msvc vc7 I'm sure you've well aware of these; what I'd like to ask you is to comment on them in the "explicit-failures-markup.xml" in the similar way you commented on "*_warchive" tests, so that a) the library users are not scared away by all these yellow cells, and b) the release manager and everybody else besides the author have a simple way of telling whether the library is in the OK state or not. Thanks in advance, -- Aleksey Gurtovoy MetaCommunications Engineering

1 0

[Random] GCC 2.95.3 regressions
by Aleksey Gurtovoy 30 Sep '04

30 Sep '04

Given that the code compiles with the latests versions of the compiler, the following (from http://tinyurl.com/58lll) is most likely to be a compiler bug, but since the test used to pass some time ago, I was wondering if anyone is interested in making it work again or simply commenting on it before I mark it green: /boost/head-regression/boost/boost/random/xor_combine.hpp: In method `uint32_t boost::random::xor_combine<boost::random::linear_feedback_shift<unsigned int,32,31,13,12,0>,0,boost::random::linear_feedback_shift<unsigned int,32,29,2,4,0>,0,0>::min() const': /boost/head-regression/boost/boost/random/xor_combine.hpp:74: instantiated from `boost::random::xor_combine<boost::random::xor_combine<boost::random::linear_feedback_shift<unsigned int,32,31,13,12,0>,0,boost::random::linear_feedback_shift<unsigned int,32,29,2,4,0>,0,0>,0,boost::random::linear_feedback_shift<unsigned int,32,28,3,17,0>,0,0>::min() const' /boost/head-regression/boost/libs/random/random_test.cpp:251: instantiated from `instantiate_urng<boost::taus88, unsigned int>(const basic_string<char,string_char_traits<char>,__default_alloc_template<true,0> > &, const boost::taus88 &, const unsigned int &)' /boost/head-regression/boost/libs/random/random_test.cpp:377: instantiated from here ... Date: 2004-09-26 06:35 set -e "/usr/local/gcc-2.95.3/bin/g++wrap" --limit-memory=600 --limit-cpu=1800 -c -Wall -ftemplate-depth-255 -g -O0 -fno-inline -I"../bin/boost/libs/random/test" -I "/boost/head-regression/boost" -o "../bin/boost/libs/random/test/random_test.test/gcc-2.95.3-linux/debug/random_test.o" "/boost/head-regression/boost/libs/random/random_test.cpp" "/usr/bin/objcopy" --set-section-flags .debug_str=contents,debug "../bin/boost/libs/random/test/random_test.test/gcc-2.95.3-linux/debug/random_test.o" Anyone? -- Aleksey Gurtovoy MetaCommunications Engineering

1 1

[Lambda] GCC 3.4.1 failures
by Aleksey Gurtovoy 30 Sep '04

30 Sep '04

Jaakko, Could you please take a look/comment on the following failure in "bind_tests_simple_f_refs" (from http://tinyurl.com/52xlg) "C:\MinGW341\bin\g++" -c -Wall -ftemplate-depth-255 -g -O0 -fno-inline -mno-cygwin -I"C:\MinGW\Bronek\boost\results\bin\boost\status" -I "C:\MinGW\Bronek\boost\boost" -o "C:\MinGW\Bronek\boost\results\bin\boost\status\bind_tests_simple_f_refs.test\mingw-3.4.1\debug\bind_tests_simple_f_refs.obj" "C:\MinGW\Bronek\boost\boost\libs/lambda/test/bind_tests_simple_f_refs.cpp" C:\MinGW\Bronek\boost\boost\libs/lambda/test/bind_tests_simple_f_refs.cpp: In function `int test_main(int, char**)': C:\MinGW\Bronek\boost\boost\libs/lambda/test/bind_tests_simple_f_refs.cpp:69: error: call of overloaded `bind(int (&)())' is ambiguous C:/MinGW/Bronek/boost/boost/boost/lambda/detail/bind_functions.hpp:48: note: candidates are: const boost::lambda::lambda_functor<boost::lambda::lambda_functor_base<boost::lambda::action<1, boost::lambda::function_action<1, T> >, typename boost::lambda::detail::bind_tuple_mapper<Result (&)(), boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type>::type> > boost::lambda::bind(Result (&)()) [with Result = int] C:/MinGW/Bronek/boost/boost/boost/lambda/detail/bind_functions.hpp:70: note: const boost::lambda::lambda_functor<boost::lambda::lambda_functor_base<boost::lambda::action<1, boost::lambda::function_action<1, boost::lambda::detail::unspecified> >, typename boost::lambda::detail::bind_tuple_mapper<const Arg1, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type>::type> > boost::lambda::bind(const Arg1&) [with Arg1 = int ()()] C:\MinGW\Bronek\boost\boost\libs/lambda/test/bind_tests_simple_f_refs.cpp:70: error: call of overloaded `bind(int (&)(int), int)' is ambiguous C:/MinGW/Bronek/boost/boost/boost/lambda/detail/bind_functions.hpp:188: note: candidates are: const boost::lambda::lambda_functor<boost::lambda::lambda_functor_base<boost::lambda::action<2, boost::lambda::function_action<2, T> >, typename boost::lambda::detail::bind_tuple_mapper<Result (&)(Par1), const Arg2, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type>::type> > boost::lambda::bind(Result (&)(Par1), const Arg2&) [with Result = int, Par1 = int, Arg2 = int] C:/MinGW/Bronek/boost/boost/boost/lambda/detail/bind_functions.hpp:210: note: const boost::lambda::lambda_functor<boost::lambda::lambda_functor_base<boost::lambda::action<2, boost::lambda::function_action<2, boost::lambda::detail::unspecified> >, typename boost::lambda::detail::bind_tuple_mapper<const Arg1, const Arg2, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type>::type> > boost::lambda::bind(const Arg1&, const Arg2&) [with Arg1 = int ()(int), Arg2 = int] [...] ? -- Aleksey Gurtovoy MetaCommunications Engineering

2 2