- Boost - lists.boost.org

[review] [text] Text formal review
by Glen Fernandes 22 Jun '20

22 Jun '20

Dear Boost community, The formal review of Zach Laine's Unicode library, Text, begins today (June 11th) and ends on June 20th. The library provides three layers: - The string layer, a set of types that constitute "a better std::string" - The Unicode layer, consisting of the Unicode algorithms and data - The text layer, a set of types like the string layer types, but providing transparent Unicode support Code: https://github.com/tzlaine/text Docs: https://tzlaine.github.io/text The master branch of the repository will be frozen for the duration of the review. Text requires C++14 or higher, and supports GCC 5+, Clang 3.6+, and MSVC 2017. Zach initially presented the library at C++Now in 2018. He also intends to standardize a library solution for Unicode based on this work. Talk: https://www.youtube.com/watch?v=944GjKxwMBo Slides: https://tinyurl.com/boost-text-slides We encourage your participation in this review. At a minimum, kindly state: - Whether you believe the library should be accepted into Boost * Any conditions for acceptance - Your name and knowledge of the problem domain. You are strongly encouraged to also provide additional information: - What is your evaluation of the library's: * Design * Implementation * Documentation * Tests * Usefulness - Did you attempt to use the library? If so: * Which compiler(s) * What was the experience? Any problems? - How much effort did you put into your evaluation of the review? Even if you are not submitting a formal review, your feedback on the library is welcome. Glen

10 21

[text] Copyright notices
by Phil Endecott 20 Jun '20

20 Jun '20

Dear All, Looking at the proposed Boost.Text source, I find that: 1. Many files don't have a copyright header. Random example: https://github.com/tzlaine/text/blob/master/include/boost/text/unencoded_ro… I guess this is an oversight and is easily fixed. 2. Some files have a copyright header which is not the Boost licence. Example: https://github.com/tzlaine/text/blob/master/include/boost/text/transcode_al… This one has a requirement that binary distribution includes the copyright notice, which is not a requirement of the Boost licence, and will be problematic for some users. Are the only affected files the SIMD implementation, (c) Robert N Steagall? If so, can this be disabled (by default?) by the user to avoid the copyright notice requirement? 3. The documentation includes a Unicode copyright notice: https://tzlaine.github.io/text/doc/html/boost_text__proposed_/unicode_copyr… This also includes a requirement that the copyright notice is included in data files, software or associated documentation, whatever that means. It's not clear to me what this actually covers; again, can some functionality (e.g. UTF-N-to-M conversion without normalisation) work without the Unicode data? Regards, Phil.

8 15

[text] Comparison with Unicode features of Boost.Locale
by Phil Endecott 19 Jun '20

19 Jun '20

Dear Zach & others, Boost already has some Unicode functionality in Boost.Locale. Would you like to compare the pros/cons of each? Regards, Phil.

3 3

[text] SIMD UTF-8 decoding
by Phil Endecott 18 Jun '20

18 Jun '20

Dear All, I have been looking at the UTF-8 decoding code in the proposed Boost.Text, as this is a problem I've looked at myself in the past. I've mentioned an issue with the copyright in another message. Here are my other observations. 1. The SIMD code is x86-specific. It doesn't need to be; I think it could use gcc's vector builtins to do the same thing and be portable to other SIMD implementations. (Clang provides the same builtins; I'm not sure about what you need to do on MSVC/Windows.) See: https://gcc.gnu.org/onlinedocs/gcc/Vector-Extensions.html 2. The SIMD code only seems to provide a fast path for bytes < 0x80, falling back to sequential code for everything else. I guess I was expecting something more sophisticated. 3. The code used for bytes >= 0x80, and in all cases for non-x86, is here: https://github.com/tzlaine/text/blob/master/include/boost/text/transcode_it… around lines 400-560. It implements a state machine, which surprises me; it takes much less code and gives better performance if you write out the bit-testing and shifting etc. explicitly. This seems to be about 50% slower than my existing UTF-8 decoding code. 4. There aren't enough comments anywhere in the code I've looked at! Regards, Phil.

4 6

[container] Proposal for ring queues
by Andrey Semashev 17 Jun '20

17 Jun '20

Hi, Some time ago I have proposed to include ring_queue and small_ring_queue containers to Boost.Container: https://github.com/boostorg/container/pull/121 The ring queues work similarly to std::queue with the main difference being that they use a ring buffer internally to store elements. The small_ring_queue also allows to allocate a static storage for a fixed amount of elements. The benefit of this is that dynamic memory allocations can be practically avoided when the number of enqueued elements does not exceed some limit. In his reply in that PR Ion was not very enthusiastic about including these containers into Boost.Container mostly because (a) there was presumably not much interest from users and (b) because the current implementation requires C++17 while Boost.Container supports C++03. I would like to ask if there is interest in the community for these kinds of containers. I know I would find them useful, and at least one person commented in the PR to the same effect. However, I would still like to see if more people need this. If there is interest, what would be the preferred course of action? 1. Should we work on inclusion into Boost.Container or maybe some other library? Candidates? Boost.CircularBuffer was mentioned in the PR, although I'm not sure it is suitable for these components. Creating a new submodule is also a possibility, although it seems like an overkill. 2. Somewhat related to p.1, what minimum C++ version is preferred. E.g. if we agree that Boost.Container is the right place for these components, I could work on lowering minimum C++ version, although C++03 would complicate things considerably. Thanks.

9 28

Re: [boost] [text] Copyright notices
by Andrey Semashev 16 Jun '20

16 Jun '20

On 2020-06-16 17:43, Hans Dembinski wrote: >> On 16. Jun 2020, at 16:41, Hans Dembinski <hans.dembinski(a)gmail.com> wrote: >> >>> >>> On 16. Jun 2020, at 16:32, Andrey Semashev via Boost <boost(a)lists.boost.org> wrote: >>> >>> On 2020-06-16 16:45, Hans Dembinski via Boost wrote: >>>>> On 16. Jun 2020, at 14:53, Niall Douglas via Boost <boost(a)lists.boost.org> wrote: >>>>> >>>>> On 15/06/2020 17:19, Phil Endecott via Boost wrote: >>>>> >>>>>> Are the only affected files the SIMD implementation, (c) Robert N >>>>>> Steagall? If so, can this be disabled (by default?) by the user >>>>>> to avoid the copyright notice requirement? >>>>> >>>>> If that's *the* Bob Steagall I'm almost certain he'll relicence his code >>>>> under Boost if you ask him. >>>> I think a multi-platform library like Boost should avoid using SIMD intrinsics and write the code so that the auto-vectorisers of the compilers understand it - if that is possible. I don't claim it is possible here, but I have a lot of trust in the auto-vectorisers of gcc and clang. Godbolt is very helpful in designing code that can be auto-vectorised well. >>> >>> In my experience, auto-vectorization is useless in all but trivial cases. >> >> Care to give me an example? > > I don't know what you consider trivial, but these examples here listed under "vectorizable" do not look trivial to me. > > https://gcc.gnu.org/projects/tree-ssa/vectorization.html The loops I see there are trivial. Any loop where you have two arrays and you apply an operation OP between elements of the arrays is trivial and actually not very often met in practice. In practice you often have: - Conditional expressions in loop body. - Non-trivial loop exit conditions. - Horizontal operations between elements of the same array. - Array elements more dense than int. (Auto-vectorizers tend to widen elements as per C++ promotion rules.) - Input data provided via pointers that may theoretically overlap. (Auto-vectorizers tend to be pessimistic about data overlap, unless persuaded by non-portable means.) Any of the above is typically poorly handled by auto-vectorizers, if at all. At least, that was my observation when I wrote code that I subsequently had to manually vectorize. PS: I'm sending this to the ML.

1 1

Re: [boost] linux build and version symlinks question
by Peter Dimov 16 Jun '20

16 Jun '20

> Aleksei Nikiforov wrote: > > > Besides just removing those symlinks after installation, what's the > > proper way for disabling creating these additional symlinks when > > building/installing boost using b2? > > I tried `b2 install install-no-version-symlinks=on`, but it had no effect. > So I guess the answer is "none". I suppose the right thing to do is to > reverse the change and reinstate the original behavior to not install > these symlinks on `b2 install`. I'm not quite sure about `b2 stage`, but > it should probably be consistent with `b2 install`, because many packaging > scripts use `stage` instead of `install` for some reason. See https://github.com/boostorg/boost_install/issues/43 and https://github.com/boostorg/boost_install/commit/ea28ed441fb04f2c5839e99e37…. If you see something wrong with these changes please let me know.

2 1

linux build and version symlinks question
by Aleksei Nikiforov 15 Jun '20

15 Jun '20

Hi. I've noticed that some versions ago boost started creating additional symlinks when built for linux. In addition to boost_${library}.so -> boost_${library}.so.A.B.C symlink, two more symlinks are created: boost_${library}.so.A -> boost_${library}.so.A.B.C and boost_${library}.so.A.B -> boost_${library}.so.A.B.C. Additional symlinks started appearing approximately around version 1.70.0. For example, boost 1.67.0 has library libboost_filesystem.so.1.67.0 and a symlink libboost_filesystem.so. Boost 1.73.0 has library libboost_filesystem.so.1.73.0 and 3 symlinks: libboost_filesystem.so, libboost_filesystem.so.1 and libboost_filesystem.so.1.73. My guess is that this change was introduced in following commit by removing install-no-version-symlinks feature for all libraries: https://github.com/boostorg/boost/commit/47ef674f752ad2cfeb581665cbea18ee6f… I couldn't find any mention of such change in changelog or any explanation or description of intention of this change. Did I miss it? Was it an unintentional change? In binary linux distributions, *.so files and header files are usually packed into devel packages, while *.so.* are packed into library packages. In case of boost and those symlinks it can lead to an issue. For example, during upgrade from boost 1.72.0 to boost 1.73.0, while both boost 1.72.0 libraries and 1.73.0 libraries are needed temporarily, boost_${library}.so.1 symlinks from both packages may conflict. These symlinks with same names point to different libraries. It's not an issue for devel packages and *.so symlinks with same names since you don't need two devel packages being installed at same time during upgrade. Besides just removing those symlinks after installation, what's the proper way for disabling creating these additional symlinks when building/installing boost using b2? Kind regards, Aleksei Nikiforov

4 9

[double-ended] where is it?
by degski 15 Jun '20

15 Jun '20

I remember going through this review. It was an active, lively and constructive review. Ever since I've been dying to use this new-fangled vector. The repo is dead, no dev branch being worked on. I am happy to implement this container starting from scratch, using the discussions we've had and the valuable comments as a base. I can give it a try, but it will (by the time it gets out of the sausage machine) be a C++20 lib. If that is not acceptable, then no-can-do. degski 🕔https://mega.nz/C!AlJBkaLI

4 4

Re: [boost] [review] [text] Text formal review
by Bjorn Reese 13 Jun '20

13 Jun '20

Why does Boost.Text contains trie types in their own boost::trie namespace?

2 1