- Boost - lists.boost.org

[gsoc-2013] proposal for approximate string matching
by Yu Jiang 03 May '13

03 May '13

Hi, I'm a student from Tsinghua University in china and interested in the idea about "approximate string matching". Below is my proposal for this. Could you please view it and give me any kinds of suggestions on it? Thanks! (1) Personal Details * Name: Yu Jiang * College/University: Tsinghua University, China * Course/Major: Computer Science and Technology * Degree Program: M.Sc. * Email: sunlightt07(a)gmail.com * Homepage: do not have one * Availability: - How much time do you plan to spend on your GSoC? Since I have a long summer holiday, I can spend 30 hours a week. - What are you intended start and end dates? I will follow the calendar of GSoC. I have plenty of time from June to September. - What other factors affect your availability (exams, courses, moving, work, etc.)? I have some courses this term which will end before June. 15th. However, these courses don't have heavy workload, I still have enough time to work on GSoC. (2) Background Information * Please summarize your educational background (degrees earned, courses taken, etc.). I majored in Computer Science and got my bachelor degree in Tsinghua University from 2008 to 2012. I learned some basic courses such as algorithm, data structure, algebra and logic. I also learned many advanced courses such as database, operating system, network, formal language, compiler, computer architecture. Currently I'm working for my master degree. My research interest is database. * Please summarize your programming background (OSS projects, internships, jobs, etc.). This is the first time I plan to take part in an OSS projects. I have worked as an intern in Hulu, an online video website, for one year, mainly focus on frontend webpages. I have used C++ to finish many course projects, such as a simple ftp server and client, completing a tiny os and so on. Recently I am learning the new features in C++11 and get to know much more about the C++ language. * Please tell us a little about your programming interests. I really love to optimize on the implantation of an algorithm by making full use of the compiler and the architecture(e.g. cache, SIMD). Besides, I like to write webpages using different kinds of excellent frameworks. * Please tell us why you are interested in contributing to the Boost C++ libraries. Boost is the most famous C++ libraries. It is powerful and can be used in many different kinds of programs. It's my honor to add some new features into it. * What is your interest in the project you are proposing? It's related to my research interest. Nowadays data are produced rapidly. To clean or integrate them so that we can get a better view, "Approximate string matching" is rather a powerful tool. I'd like to see such functions in boost library. * Have you done any previous work in this area before or on similar projects? Yes. My lab has focused on researching on such algorithms for 3-5 years. The "best" algorithm for general string similarity search and join is called "passjoin", which is invented by our lab. I have implemented it and done a lot of optimizations. Early this year we attended a string similarity search & join competition[1] and achieved champion. The code is mainly written by me. [1] http://www2.informatik.hu-berlin.de/~wandelt/searchjoincompetition2013/Resu… * What are your plans beyond this Summer of Code time frame for your proposed work? I will make efforts to make it accepted by the boost library. If it can be released someday, I will maintain it. * Please rate, from 0 to 5(0 being no experience, 5 being expert), your knowledge of the following languages, technologies, or tools: -C++: 3 -C++ Standard Library: 3.5 -Boost C++ Libraries: 2 -Subversion: 3 * What software development environments are you most familiar with? Eclipse. I use it in both windows and linux. * What software documentation tool are you most familiar with? Doxygen. (3) Project Proposal Approximate string matching is a "hotspot" in recent years. It's widely used in big data processing such as search engine, DNA matching. It also can be used in simple tasks like spell checking. However, there isn't a high performance and well-designed library in such area. I wish to see such algorithms in boost so that many people can benefit from this! My proposal contains two parts: A. To implement basic similarity metrics. Edit distance and Jaccard may be the most popular ones. Well, I'll implement more similarity metrics, such as Hamming distance, dice and cosine. B. To implement the fastest algorithm for string similarity search and join. Similarity metrics will be edit distance only since it is most commonly used. Similarity search is to find out all the strings in a given set S, which are "similar" to a query string. Similarity join is to find out all the string pairs<r,s>, where r is in set R and s is in set S and r is similar to s. They are the most useful operators in large data processing. However, the brute-force method for such problem is too slow. And it's not easy to write a correct and high performance algorithm without reading the details in many research papers and debugging them for a long time. To avoid reinventing the wheel, I strongly propose to implement it in boost library. Here is a prototype for the interface: Part A. basic similarity metrics (use edit distance as an example) (1) int edit_distance(const Range1T&, const Range2T&); // return edit distance. (2) int edit_distance(const Range1T&, const Range2T&, int threshold); // return edit distance if it is no larger than threshold, else return -1 or threshold + 1. (3) int edit_distance(const Range1T&, const Range2T&, equal_func); // use uqual_func to determine whether two elements are equal or not. (4) int edit_distance(const Range1T&, const Range2T&, insert_cost_func, delete_cost_func, substitute_cost_func); // use three cost functions to calculate the cost for one insert/delete/substitute operation. (5) some functions to return the sequence of operations? Version (1) may be the most common case. Version (2) can use the given threshold to do optimization for the speed (O(m*n) -> O(threshold * min(m, n))). It's useful since sometimes we only care about the exact edit distance if it is small. Version (3) and (4) extend the flexibility of this utility algorithm. I'm not sure whether to support (5) or not. Part B. string similarity search and join algorithms, focus on edit distance class searcher { void insert(IDType id, const string &word); // insert a word to the index, id is used as an identifier of this word. void erase(IDType id); // erase a word from the index, id should be the same one used in insertion process. vector<Result> search(const string &query, int threshold); // performing similarity search, which will return all the strings which are similar to the query string. Result may be a pair of the id and real edit distance. }; class joiner { vector<PairResult> join(const string_and_id_container &R, const string_and_id_container &S, int threshold); // performing similarity join. Usually we are doing on-line join, which means we cannot pre-build the index. So we only need this one function. It will return all the <r,s> pair where r is in R and s is in S and r is similar to s. PairResult may be a tuple of r's id, s's id and real edit distance. }; (4) Proposed Milestones and Schedule Now – June 15 To be more familiar with the Boost library. Discuss the details and interfaces with my mentor and on the develop mail list. June 15 – July 15 Develop a beta version of the approximate matching library. Most functions will be done at this time. July 15 – August 1 Review my code and improve its performance and quality. Discuss with my mentor for mid-term evaluation and future works. August 1 – September Write documentation, test code for my library. Make it "boost" like.

4 12

[GSoC 2013] Proposal Boost.Math Bernoulli Numbers and Applications to Special Functions
by Łukasz Marecik 03 May '13

03 May '13

Hi, I would like to introduce my proposal to project "Boost.Math Bernoulli Numbers and Applications to Special Functions". It is available at https://google-melange.appspot.com/gsoc/proposal/review/google/gsoc2013/luk… Please note that HTML version is not the best one: the formatting is not very nice, there is also lack of equations and source codes (there are provided in external files). So please take a look at PDF-version of my proposal (follow the link at Google Melange site). Kind regards, Lukasz Marecik

1 0

GSoC: Data structures and algorithms for integer ranges.
by Jeremy Murphy 03 May '13

03 May '13

Hello Boost community, I realized almost too late (two days ago) that GSoC 2013 is happening, and that Boost is in. Obviously I have missed the boat on "engaging with the community" early, but I still plan to submit a proposal. The one thing I would love to get some brief feedback on in these last hours is scope: how much should I expect to get done in 12 weeks? Unsurprisingly, it is one of the hardest questions for an inexperienced worker in any field to answer. The core of my proposal is to implement i) a van Emde Boas tree and ii) three counting sort algorithms: "counting", radix and bucket. Is that enough? Too much? Any advice would be helpful, thanks. Cheers. Jeremy

1 0

Google summer of code 2013
by Jawad Elomari 03 May '13

03 May '13

Hi everyone, I want to join the Google summer of code this year, for the PaGMO/PyGMO<http://www.google-melange.com/gsoc/org/google/gsoc2013/pagmo> library. I'm interested in the Racing projects, but I'm not sure which Boost library is most suitable for it. Any ideas? Regards,

1 0

[GSoC 2013] Implicit routines in boost.odeint
by Artem Serov 03 May '13

03 May '13

Hello again! I have been working on proposal since aproximately 15 of April, investigating boost odeint, some theory of numeric methods of solving ODE and implementing a draft of the design of implicit routines. I discussed some issues privately with mentors of odeint project Mario Mulansky and Karsten Ahnert and have taken into account all the points. Here is the 1st version of the proposal for Boost.odeint implicit routines. Note: The main idea i kept in mind was to describe steps how i will do the project as clear and detailed as possible. I tried to show my acquired experince of odeint structure and concepts, my understanding of the issue and concrete ways of tackling it. Note: The proposal entirely lacks introduction, list of literature and sources! i will settle it as soon as possible. Here is the link to the proposal: https://docs.google.com/file/d/0B-2QwVSmhDBIYTB0eVZxdHhsNUE/edit?usp=sharing -- Thanks, Artem Serov

2 3

GSoC: Student application deadline in less than 24 hours
by Boris Schaeling 02 May '13

02 May '13

Students interested in applying for a GSoC project should submit their application as soon as possible. The deadline is in less than 24 hours. Click here to apply: <https://google-melange.appspot.com/gsoc/homepage/google/gsoc2013> Boris

1 0

[gsoc-2013] Boost.Expected
by Pierre T. 02 May '13

02 May '13

Hello, My name is Pierre Talbot, I participated to GSoC 2011 and began the implementation of the Boost.Check library (for checking the validity of any number having a check digit — credit card number, ISBN, …). It was two years ago and the design is still evolving due to many variation points in the code. One of the function looks like : template <typename check_algo, typename range> boost::optional<typename check_algo::checkdigit_type> compute_checkdigit(const range &x); Using boost::optional tell us that: "A check digit should be computed if 'x' is a valid sequence". Since 'x' has many reasons to be incorrect, many errors could be raised. Based on a policy template class, it launches exception or returns with an empty optional . Then I though a lot about a better way to do it, allowing the user to get an exception or an error code. But it was quite complex for a so little part of my library… I decided that the policy "throw exception or nothing" should be enough. Yesterday, I watched the video of Mr. Alexandrescu on Boost.Expected and I think it would be a very useful library. It could mainly be useful in system programming (where many error codes can arise from a single call), but in any code where many exception errors can be thrown (like in Boost.Check). As you suspected, I'm interested in coding Boost.Expected during the summer as a GSoC student. Firstly, it could be very useful to list the resources on the subject (aside the talk), I have several articles that I will talk about later. Secondly, and hoping you'll debate, I would like to ask your opinion about several ideas and facts: 1) In the Boost project description, we can read: "adding a class expected-or-error_code". Obviously, the main design decision made by Alexandrescu is to consider that error code are exception. Do we need an abstraction of exception/error code ? Why do you think a "expected-or-error_code" class is interesting ? 2) Consider this code (from Alexandrescu slides) : // Caller string s = readline(); auto x = parseInt(s).get(); // throw on error auto y = parseInt(s); // won’t throw if (!y.valid()) { // handle locally if (y.hasException<std::invalid_argument>()) { // ------------------ < The flagged line. // no digits ... } y.get(); // just "re"throw } The flagged line has some tastes of "return to the past" flavor. Back to the C procedural language, the basic error code handling system was a lot criticized because: * Readability of the code decrease ; * Error handling occurs in the middle of the execution flow ; * <Add your favourite reason here>. Several links on the subjects (if you have others, I'm interested) http://www.joelonsoftware.com/articles/Wrong.html http://www.joelonsoftware.com/items/2003/10/13.html http://blogs.msdn.com/b/oldnewthing/archive/2005/01/14/352949.aspx http://nedbatchelder.com/text/exceptions-vs-status.html Basically, only the last one is clearly for exception. The main argument against the procedural approach is the readability. I would say that the Expected approach just differ by allowing to rethrow exception. But if you want to handle it, you must code multiple if-than-else statements. So I considered a complementary approach working with Expected to handle multiple error cases: string visa_number = readline(); expected<char> expected_checkdigit = compute_checkdigit<visa>(visa_number); if(expected_checkdigit.valid(visa_error_resolver)) { visa_number += expected_checkdigit.get(); std::cout << visa_number << std::endl; } With this code, there is only a if statement, and no more multiple error cases handles. But what is this error_resolver ? It may be declared as : // Somewhere in visa.hpp. A type list. typedef error_list<size_error_exception, unknown_character_exception, checkdigit_encoding_exception, …> visa_errors; // Somewhere in the user code. error_resolver<visa_errors, expected_type> visa_error_resolver; // in this case, expected_type is a char. // Initialize error handler on specific exception/errors. visa_error_resolver.on<size_error_exception>(size_error_handler) .on<unknown_character_exception>(unknown_character_exception) ... Now we are agree that visa_error_resolver can be reused everywhere we want to resolve an error on a visa number. What are the handlers ? There are [Function|Functor|Lambda] (pick up your favourite) with this form : expected<ExpectedType> size_error_handler(const size_error_exception&) expected<ExpectedType> unknown_character_exception(const unknown_character_exception&) Now you can understand for what the type list "error_list" stands for, we can store these handlers into the error_resolver and call them without any virtual cost. Why the return type of error handler is expected<ExpectedType> ? Consider this size_error_handler code : expected<ReturnType> size_error_handler(const size_error_exception& e) { std::cout << "The number you gave has a bad size." << std::endl; std::cout << "Enter it again : " << std::endl; return read_visa_checkdigit(); } read_visa_checkdigit can call recursively valid() until it's valid. Though there are some ways to make this treatment iterative. A basic treatment could be to print an warning message and just returns (in this case, valid returns false): expected<ReturnType> size_error_handler(const size_error_exception& e) { std::cout << "Warning: the VISA field is incorrect." << std::endl; return expected<ReturnType>::fromException(e); } Results: * The error code handling is delegated to specific functions ; * The readability is still excellent ; * You can easily re-use your error handler function ; * If you don't like it, you can still throw exception on failure with "get()". I can code a "proof of concept" if you think this is a good idea. Do not hesitate to comment it, point out programming pitfalls, request further clarification, or anything you judge useful. Thank you for reading it ! Pierre Talbot

10 63

Re: [boost] [gsoc-2013] Boost.Thread/ThreadPool project
by Niall Douglas 02 May '13

02 May '13

> > I have just read N3388 - Using Asio with C++11 and a think I start to > > understand the ASIO design. > > Could we say the completion handlers are continuations? > > And that the user use to master which thread executes the continuation by > > calling the io_service::run() function? > > Pretty much, except that multiple threads can call io_service::run > concurrently. Basically Boost.Asio is a work sharing queue plus a > collection of waitqueues to wait for specific events. Far clearer than my explanation. Thanks. > That's orthogonal. Boost.Asio provides the funding blocks. A nicer > interface that doesn't require an explicit continuation passing style > transformation can be built with Boost.Context or a similar library. > But it probably needs a future implementation that integrates with > ASIO, which is one of the point of Niall Douglas. Spot on. Especially as parts of ASIO are due to enter with TR2. I worry that a thread pool designed without ASIO in mind would become an orphan isolate. Niall --- Opinions expressed here are my own and do not necessarily represent those of BlackBerry Inc.

2 1

[testing] run.py and cross compilation
by Antony Polukhin 02 May '13

02 May '13

Hi, Is there some possibility to compile sources on one platform (x86) and then run compiled tests on other platform (ARM/Android) using run.py script? This is required, because Android SDK exists only for x86 so there is no way to compile tests on ARM Android device. -- Best regards, Antony Polukhin

2 3

[GSoC2013] My updated proposal to the XML library project
by Amos Ji 02 May '13

02 May '13

Hi, I have updated my proposal to the XML library project. If you're interested in it, you can review it at http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/amosji/3… And there is also a pdf version: https://www.dropbox.com/s/3zgxkpb9bx45auo/Boost_XML_library.pdf Any comments are appreciated. Thanks a lot. Regards, Mingchao

1 0