- Boost - lists.boost.org

[release] 1.74.0 deadline for new libraries is almost here!
by Marshall Clow 26 Jun '20

26 Jun '20

Tomorrow the master branch will be closed for new libraries and breaking changes. See the calendar here for more dates: https://www.boost.org/development/ <https://www.boost.org/development/> The next deadline is 1-July, when the master branch will be closed for major changes Thanks, — The Boost Release Team

2 3

Boost.Text review
by Phil Endecott 25 Jun '20

25 Jun '20

This is my review of Zach Laine's proposed Boost.Text. Zach is offering several things here: Unicode, a "better string", a trie, and more. I'm pleased that these components are all individually exposed to users and are independent of each other, rather than some being hidden as implementation details. I've spent a couple of days looking at a few parts of the library. I'll discuss at the Unicode stuff first. Boost gained support for Unicode functionality when Boost.Locale was accepted almost a decade ago, but I was not particularly impressed by that library and have not used it in my own work. Does the current proposal do better? I think it does, and though it's not perfect I think it would be worth accepting but subject to a concern about copyright. See another thread for the copyright discussion. Basically, if I understand correctly, Zach's code includes tables that are derived from Unicode data and the licence for that data requires a copyright attribution. The Boost licence does not require attribution and I support that. So if I've understood the situation correctly - and I am not a lawyer! - this means that sadly Boost must REJECT the affected code. I'll say some more about the other aspects of the Unicode code, just in case the copyright issue turns out to be surmountable. I've tried just a couple of things. Firstly, I tried the UTF-8 to UTF-32 conversion. This seems to work, and the API is reasonable. Performance is not bad, but it could be better (see another thread); that could be fixed later. I think it would be worth having just a UTF library in Boost! Secondly I looked at the line-splitting algorithm, i.e. template<typename CPIter, typename Sentinel, typename Extent, typename CPExtentFunc> unspecified lines(CPIter first, Sentinel last, Extent max_extent, CPExtentFunc cp_extent, bool break_overlong_lines = true); Again this seems to work, and the API is reasonable. Reference documentation is somewhat lacking though; in the description of lines(), the requirements for CPExtentFunc need to be described, and the "unspecified" return type needs to document that it includes members including iter and hard_line_break. I think there are probably similar omissions in much of the reference documentation. I'm unconvinced by the use of the word "sentinel" here. A sentinel is a value with a special meaning, e.g. a terminating null; what you have here is an end-iterator. It's good that you're allowing the begin and end iterators to have different types, but I think you should name that type something like CPEndIter. The implementation of line splitting seems to call its word length callback more often than necessary, i.e. when a word overflows a line, that word is subsequently measured a second time. That should be easily fixable. Moving on to other parts of the proposal, I'm less happy with text::string. The first thing I noticed was the very strange decision to use int, rather than (signed?) size_t, for indexing; yes I do work with chunks of text larger than 2 GB! I would REJECT text::string for this alone. I also dislike the use of operator() to extract substrings; the meaning just isn't obvious: auto x = y(4); Is that the 4th character of y? Or the substring from 4 to the end? No, it's the substring from the start to 3. I'd need to add a comment to explain what it's doing. But actually there's another problem with that line: x is a non-owning view of that substring. I think that methods that return a view (or span, etc.) should have a name that makes that clear, e.g. "subview". (This wouldn't have been a problem before auto.) This came up in the static_string review, where the substr() method was originally proposed to return a view; that was renamed subview() with substr() returning a copied string. I don't mind that the string isn't templated on the char type, but I do miss the allocator support; I've used strings with arena allocators (for performance) and with shared memory allocators. I know allocator support is a lot of work but it is necessary, IMO. text::text seems to be the obvious combination of text::string with unicode functionality, and I've not looked at that. I have had a look at the trie; I have sometimes wondered to what extent a trie can be implemented in C++ that works as much as possible like a standard container so this is interesting. The docs should mention briefly why trie is here, i.e. where it is used, just in anticipation of that obvious question from users. trie::operator[] returns an "optional ref". But it's not like a std::optional; as well as operator* it has conversion operators to access the value. Zach notes that this is "wonky" when the value type is bool, but it also seems to break for me when it's int: boost::trie::trie_map<std::string, std::string> counts; auto x = counts["Hello"]; if (x) std::cout << "present\n"; else std::cout << "absent\n"; // Prints absent, OK. boost::trie::trie_map<std::string, int> int_counts; // Now int. auto y = int_counts["Hello"]; if (y) std::cout << "present\n"; else std::cout << "absent\n"; // ../text/include/boost/text/trie.hpp:78: // boost::trie::optional_ref<T, Const>::operator T&() & // [with T = int; bool Const = false]: Assertion `t_' failed. operator T&() is being called by if(y), rather than operator bool(). There aren't has_value() and value() methods that I can use to work around this. Like a few other things in this proposal, Zach has implemented trie::operator[] how he wishes the standard library worked. As another example, dereferencing a trie_map iterator doesn't return a pair<KEY,VALUE> as a normal associative container would, but rather a struct with members called key and value. At worst, this will hit problems that the standard solutions avoid (as above). At best, it will make the library more difficult to learn for users who are familiar with the standard library (including its problems), and make generic code less likely to work with it. As a test I wrote a simple program to count word frequencies: using counts_t = std::map<std::string,int>; counts_t counts; while (std::cin.good()) { std::string s; std::cin >> s; counts[s]++; } for (auto i: counts) { std::cout << i.first << " = " << i.second << "\n"; } It would be great if I could just change it to: using counts_t = boost::trie_map<std::string,int>; but I can't do that; I need to change things: struct count { int value = 0; }; // Wrap the int. boost::trie::trie_map<std::string, count> counts; while (std::cin.good()) { std::string s; std::cin >> s; // operator[] doesn't insert, so: auto r = counts[s]; if (r) r->value++; else counts.insert(s,count{1}); } for (auto i: counts) { std::cout << i.key << " = " << i.value.value << "\n"; } That works; how about performance? Trie should perform well when the stored strings frequently share common prefixes, so I've tested with a large web server log file; this has timestamps and URLs which share common prefixes. The results: std::map: run time 1.2 seconds, memory 1007 pages trie_map: run time 1.8 seconds, memory 4868 pages So that's rather disappointing. Some general comments: Some people care about whether libraries are header-only or not, and the docs should mention this, and possibly say which functionality can be expected to work header-only. I got quite a few missing field initializer and unused parameter warnings; please fix these if at all possible. (gcc 6.3, -W -Wall). I think you should re-order the docs so that the "Hello World" example is much earlier. Some functions clearly need to embed large Unicode tables; it would be useful if the docs mentioned roughly how much the executable size is increased by. Some source files are executable. Conclusions: I fear that the only part of this that Boost should accept is the UTF transcoding. The implementation is not perfect but we could work on that. The other Unicode stuff would be good to have but the copyright issue is a blocker. Regards, Phil.

10 14

Re: [boost] Boost.Random
by degski 25 Jun '20

25 Jun '20

On Thu, 25 Jun 2020 at 03:24, <pbristow(a)hetp.u-net.com> wrote: > > > > > *From:* degski <degski(a)gmail.com> > *Sent:* 24 June 2020 15:25 > *To:* boost <boost(a)lists.boost.org> > *Cc:* Paul A. Bristow <pbristow(a)hetp.u-net.com> > *Subject:* Re: [boost] Boost.Random > > > > > > On Mon, 22 Jun 2020 at 06:59, Paul A Bristow via Boost < > boost(a)lists.boost.org> wrote: > > Using > > > std::mt19937::result_type(std::chrono::system_clock::now().time_since_epoch().count()); > > as seed looks nice and 'portable', assuming C++11. > > No need to build Boost.Chrono library - the disadvantage that I was trying > to avoid. > > But using random_device looks even simpler, again assuming Standard C++11. > > using std::random_device; > random_device seeder; > static std::mt19937 generator(seeder()); > > > > This can be better, you should seed the generator with a std::seed_seq, > which you would seed with the std::random_device. > > > > If you use sax::aes_random_device non of the above is needed, the seeder > (can) serve as a CPRNG, with a speed comparable (but faster) than > std::mt19937_64 (on Windows). > > > > > Random zealots seem to have reservations about random_device - and pretty > much anything? > > > > As I said before about random zealots! > > > > I am only testing round tripping of iostream, so true randomness is > entirely unimportant. > > And since the tests take many hours, any time for a randomish seed is also > of zero concern > > > > > > ps: > > > > All seem to work at C++11 with latest MSVC, Clang and GCC. > > > > We almost 2021, C++11 is almost complete on all compilers. > > > > But not necessarily standard libraries – GCC and Clang fail > roundtrip/loopback binary > decimal > binary using std::hexfloat ☹ > C++17 has introduced <charconv>, this does what the old stuff does, but much better (faster, better (C-) API). Here your caveat applies, I'm not sure this is in every standard library. STL has done most of the heavy lifting, as IIRC the MSVC implementation will be ported to GCC/Clang, when deemed 'finished'. I have not seen any announcement anywhere, so I guess it's on-going. I think you'll get std::hexfloat when the respective C++17-STL's have <charconv>. > > But Thanks > > Paul > > > > -----Original Message----- > > From: Boost <boost-bounces(a)lists.boost.org> On Behalf Of Christopher > Kormanyos via Boost > > Sent: 20 June 2020 11:41 > > To: Paul A Bristow via Boost <boost(a)lists.boost.org> > > Cc: Christopher Kormanyos <e_float(a)yahoo.com> > > Subject: Re: [boost] Boost.Random > > > > > > > So would following the crowd and using time(0)> be simplest? > > I never liked time(0) for that particular use casebecause in the old > days it had multiple > > millisecondresolution and lacked the seed resolution sometimes. > > I do not know it the code below is best practice,but I usually use some > kind of variation of below.You > > can switch system_clock for high_resolution_clock. > > All this is straight off-the-rack C++11. > > > > > > #include <chrono> > > #include <iostream> > > #include <random> > > > > // Use time point now seed to get a different set of values each time. > > const std::mt19937::result_type seed = > > > std::mt19937::result_type(std::chrono::system_clock::now().time_since_epoch().count()); > > > > // uint32_t > > static std::mt19937 gen(seed); > > > > int main() > > { > > std::cout << gen() << std::endl; > > } > > > > > > > > On Thursday, June 18, 2020, 10:27:27 AM GMT+2, Paul A Bristow via > Boost <boost(a)lists.boost.org> > > wrote: > > > > > > > > > -----Original Message----- > > > From: Boost <boost-bounces(a)lists.boost.org> On Behalf Of John Maddock > > > via Boost > > > Sent: 17 June 2020 18:48 > > > To: Paul A Bristow via Boost <boost(a)lists.boost.org> > > > Cc: John Maddock <jz.maddock(a)googlemail.com> > > > Subject: Re: [boost] Boost.Random > > > > > > > > > On 17/06/2020 15:53, Paul A Bristow via Boost wrote: > > > > I have wanted to use boost::random::random_device; as a seeder for > my generator. > > > > > > > > #include <boost/random/random_device.hpp> // For > > > > boost::random::random_device; seeder > > > > > > > > But using this requires that I link to a library file // LINK : > > > > fatal error LNK1104: cannot open file > 'libboost_random-vc142-mt-gd-x64-1_73.lib' > > > > > > > > So I have instead used C++ std random device successfully > > > > > > > > using std::random_device; > > > > random_device seeder; > > > > // Use seeder to get a different set of values each time. > > > > static boost::random::mt19937 gen(seeder()); // uint32_t > > > > > > > > But is there any way I can stick to the Boost version (I imagine > > > > that it might prove more > > portable? > > > > Or is this a delusion?) > > > > > > What do you mean by portable? random_device is inherently > > > non-portable because it's.... random ;) > > > > By portable I mean 'works on as many platforms and C++ standard versions > as possible'. > > > > > In many ways this is something that the std:: version does best as the > > > system implementer knows > > best > > > how to implement on their OS. Or you could just link to Boost.Random > > > of course which would work nearly everywhere too I'm sure. > > > > I was just puzzled why Boost.Random needed to *link* when > std:random_device doesn't appear to. Is > > it quietly linking to a standard library? > > > > Paul > > > > PS Thanks for the even-more-random suggestions but I really, really > don't care how randomly random it > > is for my application. > > > > So would following the crowd and using time(0) be simplest? > > > > > > > > > > _______________________________________________ > > Unsubscribe & other changes: > http://lists.boost.org/mailman/listinfo.cgi/boost > > > > _______________________________________________ > > Unsubscribe & other changes: > http://lists.boost.org/mailman/listinfo.cgi/boost > > > _______________________________________________ > Unsubscribe & other changes: > http://lists.boost.org/mailman/listinfo.cgi/boost > >

1 0

Boost.Random
by pbristow＠hetp.u-net.com 24 Jun '20

24 Jun '20

I have wanted to use boost::random::random_device; as a seeder for my generator. #include <boost/random/random_device.hpp> // For boost::random::random_device; seeder But using this requires that I link to a library file // LINK : fatal error LNK1104: cannot open file 'libboost_random-vc142-mt-gd-x64-1_73.lib' So I have instead used C++ std random device successfully using std::random_device; random_device seeder; // Use seeder to get a different set of values each time. static boost::random::mt19937 gen(seeder()); // uint32_t But is there any way I can stick to the Boost version (I imagine that it might prove more portable? Or is this a delusion?) Suggestions welcome. Paul

5 8

Boost Python issue with converters
by Dominik Cirmirakis 23 Jun '20

23 Jun '20

Hi, I have a question regarding the below code. It's an example how to pass a custom class via shared_ptr to embedded python code and it works when boost is dynamically linked. Unfortunately the same code with statically linked boost doesn't work with the following error message: "No to_python (by-value) converter found for C++ type: class boost::shared_ptr<class Foo>". I don't understand why a different linking can affect type recognition of a registered converter. What am I missing? Can anybody help me out? Thanks, Dominik https://stackoverflow.com/questions/8225934/exposing-a-c-class-instance-to-… |#include<boost/shared_ptr.hpp>#include<boost/make_shared.hpp>#include<boost/python.hpp>#include<string>#include<iostream>namespacebp =boost::python;structFoo{Foo(){}Foo(std::stringconst&s):m_string(s){}voiddoSomething(){std::cout <<"Foo:"<<m_string <<std::endl;}std::stringm_string;};typedefboost::shared_ptr<Foo>foo_ptr;BOOST_PYTHON_MODULE(hello){bp::class_<Foo,foo_ptr>("Foo").def("doSomething",&Foo::doSomething);};| |#include<boost/shared_ptr.hpp>#include<boost/make_shared.hpp>#include<boost/python.hpp>#include<string>#include<iostream>namespacebp =boost::python;structFoo{Foo(){}Foo(std::stringconst&s):m_string(s){}voiddoSomething(){std::cout <<"Foo:"<<m_string <<std::endl;}std::stringm_string;};typedefboost::shared_ptr<Foo>foo_ptr;BOOST_PYTHON_MODULE(hello){bp::class_<Foo,foo_ptr>("Foo").def("doSomething",&Foo::doSomething);};intmain(intargc,char**argv){Py_Initialize();try{PyRun_SimpleString("a_foo = None\n""\n""def setup(a_foo_from_cxx):\n"" print 'setup called with', a_foo_from_cxx\n"" global a_foo\n"" a_foo = a_foo_from_cxx\n""\n""def run():\n"" a_foo.doSomething()\n""\n""print 'main module loaded'\n");foo_ptr a_cxx_foo =boost::make_shared<Foo>("c++");inithello();bp::objectmain =bp::object(bp::handle<>(bp::borrowed(PyImport_AddModule("__main__"))));// pass the reference to a_cxx_foo into python:bp::objectsetup_func =main.attr("setup");setup_func(a_cxx_foo);// now run the python 'main' functionbp::objectrun_func =main.attr("run");run_func();}catch(bp::error_already_set){PyErr_Print();}Py_Finalize();return0;}|

1 0

[review][text] Text review has ended
by Glen Fernandes 22 Jun '20

22 Jun '20

The formal review for the Text library ended on June 20th. Thank you to everyone who participated by submitting a review, or by contributing to the discussion. A result will be posted on Friday, 2020-06-26 but continued discussion on the list in the meanwhile is more than welcome. Glen

1 0

[review] [text] Text formal review
by Glen Fernandes 22 Jun '20

22 Jun '20

Dear Boost community, The formal review of Zach Laine's Unicode library, Text, begins today (June 11th) and ends on June 20th. The library provides three layers: - The string layer, a set of types that constitute "a better std::string" - The Unicode layer, consisting of the Unicode algorithms and data - The text layer, a set of types like the string layer types, but providing transparent Unicode support Code: https://github.com/tzlaine/text Docs: https://tzlaine.github.io/text The master branch of the repository will be frozen for the duration of the review. Text requires C++14 or higher, and supports GCC 5+, Clang 3.6+, and MSVC 2017. Zach initially presented the library at C++Now in 2018. He also intends to standardize a library solution for Unicode based on this work. Talk: https://www.youtube.com/watch?v=944GjKxwMBo Slides: https://tinyurl.com/boost-text-slides We encourage your participation in this review. At a minimum, kindly state: - Whether you believe the library should be accepted into Boost * Any conditions for acceptance - Your name and knowledge of the problem domain. You are strongly encouraged to also provide additional information: - What is your evaluation of the library's: * Design * Implementation * Documentation * Tests * Usefulness - Did you attempt to use the library? If so: * Which compiler(s) * What was the experience? Any problems? - How much effort did you put into your evaluation of the review? Even if you are not submitting a formal review, your feedback on the library is welcome. Glen

10 21

[text] Copyright notices
by Phil Endecott 20 Jun '20

20 Jun '20

Dear All, Looking at the proposed Boost.Text source, I find that: 1. Many files don't have a copyright header. Random example: https://github.com/tzlaine/text/blob/master/include/boost/text/unencoded_ro… I guess this is an oversight and is easily fixed. 2. Some files have a copyright header which is not the Boost licence. Example: https://github.com/tzlaine/text/blob/master/include/boost/text/transcode_al… This one has a requirement that binary distribution includes the copyright notice, which is not a requirement of the Boost licence, and will be problematic for some users. Are the only affected files the SIMD implementation, (c) Robert N Steagall? If so, can this be disabled (by default?) by the user to avoid the copyright notice requirement? 3. The documentation includes a Unicode copyright notice: https://tzlaine.github.io/text/doc/html/boost_text__proposed_/unicode_copyr… This also includes a requirement that the copyright notice is included in data files, software or associated documentation, whatever that means. It's not clear to me what this actually covers; again, can some functionality (e.g. UTF-N-to-M conversion without normalisation) work without the Unicode data? Regards, Phil.

8 15

[text] Comparison with Unicode features of Boost.Locale
by Phil Endecott 19 Jun '20

19 Jun '20

Dear Zach & others, Boost already has some Unicode functionality in Boost.Locale. Would you like to compare the pros/cons of each? Regards, Phil.

3 3

[text] SIMD UTF-8 decoding
by Phil Endecott 18 Jun '20

18 Jun '20

Dear All, I have been looking at the UTF-8 decoding code in the proposed Boost.Text, as this is a problem I've looked at myself in the past. I've mentioned an issue with the copyright in another message. Here are my other observations. 1. The SIMD code is x86-specific. It doesn't need to be; I think it could use gcc's vector builtins to do the same thing and be portable to other SIMD implementations. (Clang provides the same builtins; I'm not sure about what you need to do on MSVC/Windows.) See: https://gcc.gnu.org/onlinedocs/gcc/Vector-Extensions.html 2. The SIMD code only seems to provide a fast path for bytes < 0x80, falling back to sequential code for everything else. I guess I was expecting something more sophisticated. 3. The code used for bytes >= 0x80, and in all cases for non-x86, is here: https://github.com/tzlaine/text/blob/master/include/boost/text/transcode_it… around lines 400-560. It implements a state machine, which surprises me; it takes much less code and gives better performance if you write out the bit-testing and shifting etc. explicitly. This seems to be about 50% slower than my existing UTF-8 decoding code. 4. There aren't enough comments anywhere in the code I've looked at! Regards, Phil.

4 6