- Boost-users - lists.boost.org

Interest in a Unicode library for Boost?
by Zach Laine 04 Nov '19

04 Nov '19

About 14 months ago I posted the same thing. There was significant work that needed to be done to Boost.Text (the proposed library), and I was a bit burned out. Now I've managed to make the necessary changes, and I feel the library is ready for review, if there is interest. This library, in part, is something I want to standardize. It started as a better string library for namespace "std2", with minimal Unicode support. Though "std2" will almost certainly never happen now, those string types are still in there, and the library has grown to also include all the Unicode features most users will ever need. Github: https://github.com/tzlaine/text Online docs: https://tzlaine.github.io/text If you care about portable Unicode support, or even addressing the embarrassment of being the only major production language with next to no Unicode support, please have a look and provide feedback. I gave a talk about this at C++Now in May 2018, and now it's a bit out of date, as the library was not then finished. It's three hours, so, y'know, maybe skip it. For completeness' sake: https://www.youtube.com/watch?v=944GjKxwMBo&index=7&list=PL_AKIMJc4roVSbTTf… https://www.youtube.com/watch?v=GJ2xMAqCZL8&list=PL_AKIMJc4roVSbTTfHReQTl1d… Zach

11 30

Re: [Boost-users] Interest in a Unicode library for Boost?
by Zach Laine 02 Nov '19

02 Nov '19

On Fri, Nov 1, 2019 at 3:41 PM Mathias Gaunard <mathias.gaunard(a)ens-lyon.org> wrote: > This email appears to only be addressed to me and not the list, is > that intentional? > I typed this on my phone so maybe things didn't work exactly as I intended. > It was not intentional. > On Fri, 1 Nov 2019 at 16:09, Zach Laine <whatwasthataddress(a)gmail.com> > wrote: > > > Right. Unicode encodes all natural languages that anyone has taken the > time to put into Unicode. I stand by the implication that natural > languages are crazy. > > Natural languages -- and their transliteration to bytes -- are > arbitrary conventions yes, but they're sufficiently consistent to be > used by millions of people in their personal and professional life > everyday. > There are rules, and however alien they might look to someone who only > needs the fairly straightforward ASCII system, they are not crazy, and > have their own reasons. > > > > So then maybe don't use those parts? They're independent; you don't > have to use them to use the Unicode algorithms. > > Separation of concerns, they do appear coupled in the documentation. > That could just be due to the way the documentation is written. > Could be ... could you point to something in the docs that left you with that impression? I explicitly divided up the documentation along the lines of the separation of concerns found in the code -- the string layer, the Unicode layer, and the text layer. I also tried to be explicit that the first is without dependencies, the second does not depend on the first or the third, and that the third depends on the second. > > Clearly you are more capable than I am. It took me a lot longer to do > than 2 months. Why did you never submit this for a Boost review? You were > thinking about it, ~10 years ago, but you never did.... > > The use case for Unicode is quite niche, and I wasn't particularly > interested in working in that niche further. > Any serious natural language processing engine would probably bypass > Unicode and come up with its own heuristics to parse text, and > applications like GUI toolkits or text editors are specialized enough > that they'd probably never use a library which wasn't specifically > designed for their use case in mind. > That leaves everyday programming, which very rarely needs to do > anything with text beyond treating it as a byte stream, and I believe > wanting to impose the heavyweight Unicode machinery systematically > everywhere is a mistake. > > Anyway, I was just mentioning this because there is prior work > (Boost.Unicode) that you apparently never looked at. > Of course being developed for only a short period of time it doesn't > have the same level of polish as your library. > Ok. FWIW, I did know about your library -- I was there for the BoostCon presentation -- and I was looking forward to using it. Ten years later, I did not look at it again, simply because it had been ten years. > >> and that the database itself is not even accessible, > > > > > > That's also intentional. Another goal of the library is to make Unicode > as simple as possible for naive users who just want to do the basics. If I > find requests for any new feature that has a compelling use case, I'll add > that. > > Being able to replace things like std::isalpha etc. is pretty basic I'd > say. > Having an API that tells you a number of properties about a code point > is one of the most useful parts of Unicode. > I agree I the abstract, but I think that there is a real problem that the properties for a given code point are not necessarily consistent across the various Unicode algorithms. For example, there are multiple notions of "number" that are treated differently in the bidirectional algorithm. Moreover, I don't know of a great use case for a boost::text::is_alpha(). Specifically, it seams that if you are looking for alphabetical characters, you are usually doing something like word-breaking, for which there is already an algorithm, doing a regex match, for which is_alpha() is insufficient, etc. I'm open to hearing about such use cases, of course. > > I don't consider 1.5MB for a database containing all human languages in > widespread use on computers to be a ridiculous size, but YMMV. > > I suppose that's reasonable. > I think the Boost.Unicode one was only 500k but that's an old Unicode > version, and I'm not sure it ever had full collation support. > > > > >> > >> It also doesn't provide the ability to do fast substring search, which > >> you'd typically do by searching for a substring at the character > >> encoding level and then eliminating matches that do not fall on a > >> satisfying boundary, instead suggesting to do the search at the > >> grapheme level which is much slower, and the facility to test for > >> boundary isn't provided anyway. > > > > > > I honestly don't know what you mean here. > > I'm not sure how to clarify this better, but I'll try. > > To search for the utf-8 substring "foo" in the utf-8 string "I really > like foo dogs", there is no need to iterate the string per code point > or per grapheme as you do in your examples. You can just perform the > search at the code unit level, then check that the position before and > after the match does not lie inside a grapheme cluster, i.e. they are > on a valid boundary. > What you need to be able to do that is a function that tells you > whether an arbitrary position in your sequence of utf-8 code units > lies at a grapheme cluster boundary or not (which would probably be a > composition of two separate functions, one that test whether the code > unit is on a code point boundary, and one that tests whether the code > point is on a grapheme cluster boundary). This functionality is not > provided. > > This sort of thing is briefly touched upon in Unicode TR#29 6.4. > I see. This seems like it might be really useful to add. I'll open a ticket for it on Github. Zach

2 3

Re: [Boost-users] Interest in a Unicode library for Boost?
by Zach Laine 01 Nov '19

01 Nov '19

Mathias pointed out that I sent this just to him. So I'm replying again to get this onto the list. Sorry for the noise. On Fri, Nov 1, 2019 at 11:09 AM Zach Laine <whatwasthataddress(a)gmail.com> wrote: > On Fri, Nov 1, 2019 at 6:35 AM Mathias Gaunard < > mathias.gaunard(a)ens-lyon.org> wrote: > >> On Sat, 26 Oct 2019 at 02:11, Zach Laine via Boost-users >> <boost-users(a)lists.boost.org> wrote: >> > >> > About 14 months ago I posted the same thing. There was significant >> work that needed to be done to Boost.Text (the proposed library), and I was >> a bit burned out. >> > >> > Now I've managed to make the necessary changes, and I feel the library >> is ready for review, if there is interest. >> > >> > This library, in part, is something I want to standardize. >> > >> > It started as a better string library for namespace "std2", with >> minimal Unicode support. Though "std2" will almost certainly never happen >> now, those string types are still in there, and the library has grown to >> also include all the Unicode features most users will ever need. >> > >> > Github: https://github.com/tzlaine/text >> > Online docs: https://tzlaine.github.io/text >> >> I would start by removing the superlative statements about Unicode >> being "hard" or "crazy". >> It's not that complicated compared to the actual hard problems that >> software engineers solve everyday. The only thing is that people >> misunderstand what the scope of Unicode is, it's not just an encoding, >> it's a a database and a set of algorithms (relying on said database) >> to facilitate natural text processing of arbitrary scripts, and does >> compromises to integrate with existing industry practices prior to all >> those scripts being brought together under the same umbrella. >> > > Right. Unicode encodes all natural languages that anyone has taken the > time to put into Unicode. I stand by the implication that natural > languages are crazy. > > >> Now the string/container/memory management, this is quite irrelevant. >> That sort of stuff has nothing to do with Unicode and I certainly do >> not want some Unicode library to mess with the way I am organizing how >> my data is stored in memory. >> Your rope etc. containers belong in a completely independent library. >> > > So then maybe don't use those parts? They're independent; you don't have > to use them to use the Unicode algorithms. > > >> What's important is providing an efficient Unicode character database, >> and implementing the algorithms in a way that is generic, working for >> arbitrary ranges and being able to be lazily evaluated (i.e. range >> adaptors). >> I already did all that work more than 10 years ago as a two-month GSoC >> project, though there are some limitations since at that time ranges >> and ranges adaptors were still fairly new ideas for C++. It does >> however provide a generic framework to define arbitrary algorithms >> that can be evaluated either lazily or eagerly. >> > > Clearly you are more capable than I am. It took me a lot longer to do > than 2 months. Why did you never submit this for a Boost review? You were > thinking about it, ~10 years ago, but you never did.... > > >> To be honest I can't say I find your library to be much of an >> improvement, at least in terms of usability, since the programming >> interface seems more constrained (why don't things work with arbitrary >> ranges rather than this "text" containers) > > > They do, of course. I'm not sure why it is you think otherwise. > > >> and verbose (just look at >> the code to do transcoding with iterators), > > > Are you referring to the verbosity of: > > char const * some_utf8 = /* ... */ ; > out = std::ranges::copy(boost::text::as_utf32(some_utf8), out); > > , or: > > out = boost::text::transcode_utf_8_to_32(utf8_first, utf8_last, out); > > , or something else? > > >> the set of features is >> quite small, > > > That is quite intentional. I want to standardize *basic* Unicode > support. I feel that what I have in Boost.Text is the basic set that users > will need, just to support languages or formatting conventions that are not > common in their favorite environment. For instance, today there is no > standard way of taking UTF-8 and turning it into UTF-16, or vice versa; > this library is intended to work at that level. That is, it is intended to > fill in needless gaps in Unicode support that exist in C++ -- gaps that no > other major language besides C has. It is specifically not intended to > replace all ICU functionality. Do you have specific things in mind that > you think ~90% of Unicode-aware C++ users will need? Note that I did not > say 100%. > > >> and that the database itself is not even accessible, > > > That's also intentional. Another goal of the library is to make Unicode > as simple as possible for naive users who just want to do the basics. If I > find requests for any new feature that has a compelling use case, I'll add > that. > > >> and >> last I remember your implementation was ridiculously bloated in size. >> > > I don't consider 1.5MB for a database containing all human languages in > widespread use on computers to be a ridiculous size, but YMMV. > > >> It also doesn't provide the ability to do fast substring search, which >> you'd typically do by searching for a substring at the character >> encoding level and then eliminating matches that do not fall on a >> satisfying boundary, instead suggesting to do the search at the >> grapheme level which is much slower, and the facility to test for >> boundary isn't provided anyway. >> > > I honestly don't know what you mean here. If you use the text::text or > text::string types, those are just contiguous sequences of bits, like a > std::vector or std::string. text::text exposes iterators to those bits > which can be used to get grapheme, code point, and/or UTF-8 byte views of > the underlying data. If you are using something else besides text::text or > text::string two types, you presumably have access to your own bits in your > own representation. What prevents you from doing whatever substring search > you like, via std::search(), std::ranges::includes(), or something else? > Boost.Text is not intended as a string algorithms library. > > I'm pretty sure I made similar comments in the past, but I don't feel >> like any of them has been addressed. >> > > I think you're referring to this email you sent in the Boost.Text > interest thread from 14 months ago: > > """ > The Unicode library I did as a SoC project in 2009 was significantly > smaller than that and if I recall correctly it has more data than the one > in your library. > Clearly some work can be done here to better optimize the database size. > """ > > I did make it a bit smaller. The other comments are new. > > Zach > >

1 0

Serialization of an instance of a Derived Template Class
by Pritam Gharat 01 Nov '19

01 Nov '19

Hi, I am using Boost to serialize pointers of a base class which points to instances of a derived template class. I am using boost 1.7.0 and running it on Ubuntu 16.04. The source code of the example is available at https://github.com/PritamMG/Boost-Serialization-Example My example has a base class Bar_A (definition available in include/Abstract.hpp) which has a virtual function toStr(). It has two derived classes Bar_B and Bar_C. Bar_B is a template class. I have registered the derived classes in sc/Test.cpp. Class ObjectNew (include/Object.hpp) has a member pointer of type Bar_A which points to instances of classes Bar_B and Bar_C. I get an error as "unregistered class". I am not sure for which class do I get this error message. There is also an issue of non-default. If Bar_B does not have a default constructor, save_construct_data and load_construct_data are over-ridden and there I run into an error "too many template-parameter-lists". How do I define these functions? Thanks, Pritam

2 4

Re: [Boost-users] [boost] Interest in a Unicode library for Boost?
by Zach Laine 30 Oct '19

30 Oct '19

On Wed, Oct 30, 2019 at 9:47 AM Peter Koch Larsen < peter.koch.larsen(a)gmail.com> wrote: > I saw that talk (on video) and was quite impressed but never got to > try your library out. > Would be happy to use a library like this! Can you tell me how it > relates to the proposed std? > There is no relationship yet. Boost.Text is an experiment meant to gather feedback, which will in turn drive the standardization effort. I believe all large libraries should have Boost-equivalent visibility in the C++ community before being standardized. That being said, some of my fellow SG-16 members (that's the committee's Unicode Study Group) have started proposing some actual APIs for standardization. The first of these is the most low-level: transcoding (see http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1629r0.html ). JeanHeyd's approach is more ambitious than mine -- I really only care about Unicode. His proposal covers conversions among all possible encodings, including an API that lets users add implementations for the encodings they care about that are not covered by the standard. There are no other in-flight Unicode API proposals that I'm aware of. Zach

2 2

Boost.Numeric.Interval: Please specify rounding control mechanism Inbox x UVI x
by Bastien Commelongue 29 Oct '19

29 Oct '19

Hello, I am building an app for IOS (ARM) and I had this issue while compiling: *Boost.Numeric.Interval: Please specify rounding control mechanism.* C99 version should be used in this case but __USE_ISOC99 is not defined. Modifying the condition with this code (using __cplusplus) fixed the issue: *boost/numeric/interval/hw_rounding.hpp(36)* *#if defined(BOOST_NUMERIC_INTERVAL_NO_HARDWARE) && (defined(__USE_ISOC99) || defined(__MSL__) || (defined __cplusplus && __cplusplus >= 201103L))# include <boost/numeric/interval/detail/c99_rounding_control.hpp>#endif* Let me know if I missed something or if you plan to integrate this modification in a next version. Cheers, Bastien Commelongue -- [image: UVI] <http://www.uvi.net/> Bastien Commelongue | Software Development A. 159 rue Amelot - 75011 Paris - France T. 0033671833955 • M. b.commelongue(a)uvi.net <email(a)uvi.net> [image: Facebook] <http://www.facebook.com/UVI.Official> [image: Twitter] <http://twitter.com/UVIofficial> [image: Youtube] <http://www.youtube.com/user/UVIofficial> [image: SoundCloud] <http://soundcloud.com/uvi-official> [image: Instagram] <http://instagram.com/weareuvi/>

1 0

[BGL] minimum cut separating two given vertices on undirected graph
by Ziye Tang 18 Oct '19

18 Oct '19

Hi all, This is my first time posting, and I've read the posting guideline. I might still make some mistakes in the posting style. In particular, I am not sure how to insert code snippet into email. Please let me know when I do make such mistakes. I need to find a min-cut separating a given pair of vertices on an undirected graph. Conceptually this is how I would solve it: first I bidirect the graph, treat one vertex as the source and the other as the sink. Then run any max flow algorithm from source to sink. When the algorithm finishes, in the residual graph, identify connected component which contains a given vertex. This gives the min-cut. In order to implement it, I use push_relabel_max_flow. According to the doc I need to augment each edge with a reverse edge. Since I bi-directed the graph already, this would give rise to a multi-graph where each edge, say (i,j) has two copies, with the first copy treated as the `true' edge and the second copy as the reverse edge of arc (j,i). Then I set capacity and residual capacity map accordingly. But now I am getting a segmentation error when calling push_relabel_max_flow. Could you help me understand what is going on? I've attached my code snippet below. Here g is my graph, n is the number of vertices, xval is a 2 by 2 positive matrix where each corresponding entry is the capacity of the edge. for (int i=0; i<n; i++) for (int j=i+1; j<n; j++) { graph_traits<Graph>::out_edge_iterator e_start, e_end, e_start2, e_end2;; boost::tie(e_start, e_end) = boost::edge_range(i,j,g); // check if an edge exists if (e_start == e_end) continue; boost::tie(e_start2, e_end2) = boost::edge_range(j,i,g); // set reverse edge, set its capacity to be zero g[*e_start].rev = *(--e_end2); g[*e_start2].rev = *(--e_end); g[*e_start].capacity = xval[i][j]; g[*e_start].res_capacity = xval[i][j]; g[*e_start2].capacity = xval[i][j]; g[*e_start2].res_capacity = xval[i][j]; // now end points to the second parallel edge g[*e_end].capacity = 0; g[*e_end].res_capacity = 0; g[*e_end2].capacity = 0; g[*e_end2].res_capacity = 0; //g[e].rev = boost::edge(j,i,g).first; std::cout << "(" << i << ", " << j << "): " << g[*e_start].capacity << ", " << g[*e_start2].capacity << endl; } Thanks in advance!

1 0

Re: [Boost-users] Help needed with boost::process::child and mysql cli
by Label 17 Oct '19

17 Oct '19

Hello, > Date: Tue, 15 Oct 2019 07:28:54 +0000 > From: name name <jerry-dev(a)outlook.com> > To: "boost-users(a)lists.boost.org" <boost-users(a)lists.boost.org> > Subject: [Boost-users] Help needed with boost::process::child and > mysql cli > Message-ID: > <MN2PR01MB5710C3F35D092D2FDF0B376DF0930(a)MN2PR01MB5710.prod.exchangelabs.com> > > > Content-Type: text/plain; charset="windows-1252" > > Hi, > > Im use boost::process::child to create a mysql cli child process to > import sql file. > Normally im use this command: > /usr/bin/mysql mysql -h127.0.0.1 -utestuser --password='!@#QWE123qwe' > -P3306 --default-character-set=utf8mb4 --max-allowed-packet=1GB < > /root/create_db.sql > It?s worked well in bash. > > But when use boost::process::child, std_err report ERROR 1045 (28000): > Access denied for user 'testuser'@'localhost' (using password: YES). > Is there any bugs in my code? [...] I have no experience with boost::process, only one idea: Maybe the apostrophs around the password are normally eaten by the Linux shell, and you have to leave them out when building your arguments? // connection info args.push_back("-h127.0.0.1"); args.push_back("-utestuser"); args.push_back("--password=!@#QWE123qwe"); // <= args.push_back("-P3306"); Freundliche Grüße * Best regards * Met vriendelijke groeten * Meilleures salutations Holger Gerth _________________________________________________________________________________ EB-Soft Gesellschaft fuer elektronische Beschriftungs-Software mbH Neckargartacher Str. 114, 74080 Heilbronn, Germany Tel: +49 (0)7131 / 9466-0 Email: info(a)ebsoft.de Fax: +49 (0)7131 / 1099-0 Homepage: www.ebsoft.de Amtsgericht-Registergericht-Stuttgart HRB 102882 Geschaeftsfuehrer: Michael Ehring, Helmut Sigler _________________________________________________________________________________

2 1

Help needed with boost::process::child and mysql cli
by name name 15 Oct '19

15 Oct '19

Hi, Im use boost::process::child to create a mysql cli child process to import sql file. Normally im use this command: /usr/bin/mysql mysql -h127.0.0.1 -utestuser --password='!@#QWE123qwe' -P3306 --default-character-set=utf8mb4 --max-allowed-packet=1GB < /root/create_db.sql It’s worked well in bash. But when use boost::process::child, std_err report ERROR 1045 (28000): Access denied for user 'testuser'@'localhost' (using password: YES). Is there any bugs in my code? Os: ubuntu 18.04 LTS Boost: 1.71.0 MySql:5.7.27 Create_db.sql: DROP DATABASE IF EXISTS test_db; CREATE DATABASE test_db; Thanks [code] #include <iostream> #include <boost/algorithm/string/join.hpp> #include <boost/iostreams/copy.hpp> #include <boost/asio/io_context.hpp> #include <boost/process.hpp> using namespace boost::process; using namespace boost::iostreams; using namespace boost::asio; std::string SearchExecutableInPath(std::string const& filename) { try { return search_path(filename).generic_string(); } catch (...) { return ""; } } int RunProcess( std::string const& executable, std::vector<std::string> const& args, std::string input_file_path ) { io_context ioCtx; auto work = make_work_guard(ioCtx); std::thread ioThread([&ioCtx] { ioCtx.run(); }); std::future<std::string> dataOut, dataErr; child c; if (!input_file_path.empty()) { c = child( boost::filesystem::absolute(executable), boost::process::args(args), std_out > dataOut, std_err > dataErr, std_in < boost::filesystem::absolute(input_file_path), ioCtx ); } else { c = child( boost::filesystem::absolute(executable), boost::process::args(args), std_out > dataOut, std_err > dataErr, std_in.close(), ioCtx ); } c.wait(); int result = c.exit_code(); std::string output = dataOut.get(); std::string error = dataErr.get(); if (!output.empty()) { std::cout << output << std::endl; } if (!error.empty()) { std::cout << error << std::endl; } work.reset(); ioThread.join(); return result; } int main() { // /usr/bin/mysql mysql -h127.0.0.1 -utestuser --password='!@#QWE123qwe' -P3306 --default-character-set=utf8mb4 --max-allowed-packet=1GB < /root/create_db.sql std::vector<std::string> args; args.reserve(8); // mysql cli args.push_back("mysql"); // connection info args.push_back("-h127.0.0.1"); args.push_back("-utestuser"); args.push_back("--password='!@#QWE123qwe'"); args.push_back("-P3306"); // encoding args.push_back("--default-character-set=utf8mb4"); // maybe large sql file args.push_back("--max-allowed-packet=1GB"); int const ret = RunProcess( SearchExecutableInPath("mysql"), args, "/root/create_db.sql"); std::cout << "ret code: " << ret << std::endl; return 0; } [/code]

2 1

bg::cs::geographic and nearest query
by vexakul 14 Oct '19

14 Oct '19

Hi all! I need to store geographic points in rtree and be able to answer nearest query. Can someone provide an example similar to this <https://www.boost.org/doc/libs/1_65_1/libs/geometry/doc/html/geometry/spati…> but with point type bg::model::point<float, 2, bg::cs::geographic<bg::degree>>? Is that supported? For me it fails to compile. -- Kind regards, Sergey. -- Sent from: http://boost.2283326.n4.nabble.com/Boost-Users-f2553780.html

3 7