Discuss: Editorial Control Over Repositories and Libraries

From: Vinnie Falco (vinnie.falco_at_[hidden])
Date: 2021-08-21 18:21:10

So, I have a whole bunch of new libraries marching towards completion
and proposal for Boost. I have Boost.URL and Boost.HTTP-Proto in the
works, and there is some code that I want to share between the two.
Files like these:

Algorithms to assist processing RFC7230 compliant input:

<https://github.com/vinniefalco/http_proto/blob/9d3144c60b3426509b45cc63dbb3abfa95bd85e1/include/boost/http_proto/bnf/char_set.hpp>
<https://github.com/vinniefalco/http_proto/blob/9d3144c60b3426509b45cc63dbb3abfa95bd85e1/include/boost/http_proto/bnf/ctype.hpp>
<https://github.com/vinniefalco/http_proto/blob/9d3144c60b3426509b45cc63dbb3abfa95bd85e1/include/boost/http_proto/bnf/list.hpp>

These are public interfaces, and they are equally useful to both URL
and HTTP-Proto. They will also be likely useful to processing
websocket handshake field values. It would be nice if I could put them
in a separate repository with its own tests, documentation, and CI.
But I don't know that this is enough meat on the bone so to speak, to
justify as a complete new Boost library offering.

When Boost.URL returns a std:::string the user has the option of
providing a custom allocator. The library comes with this interface
which uses a local stack buffer:

<https://github.com/CPPAlliance/url/blob/111b83cc4726f3144194dc3546e096d60ace317f/include/boost/url/static_pool.hpp>

This can be used to avoid dynamic allocation when, for example - you
want to apply decoding to a URL-encoded input, which happens often.
Boost.HTTP-Proto would benefit from this class as well. Again we have
the case of a common component that belongs in a separate repository.
But this allocator isn't really related to the BNF-processing
utilities mentioned above. So where would it go?

As I use MSVC exclusively for regular development, I have put together
a small unit test facility which is tailored to my workflow:

<https://github.com/vinniefalco/http_proto/blob/9d3144c60b3426509b45cc63dbb3abfa95bd85e1/test/include/test_suite.hpp>

This has the flavor of Boost lightweight test, but also:

* Redirects output to the Visual Studio IDE if a debugger is attached
* Supports the model where one executable is built which holds all the tests
* A command-line argument facility to specify individual test suites to run

I use this in all of my Boost libraries and to-be-proposed libraries.
I wouldn't say that it is user-facing. Currently I put a copy of the
file in each library but I have made some changes and it would be
better if I could put it in a shared place in the Boost tree. But it
isn't public so it shouldn't be grouped with the BNF and the allocator
stuff above.

I am working on new libraries which do ASIO networking, and they use
this handy mocking class which is used for testing stream algorithms:

<https://github.com/boostorg/beast/blob/21cd552399aa8167ed53c21a74f3711c2c316d2f/include/boost/beast/_experimental/test/stream.hpp>

This is a public interface, which users deploy for their own tests
(for example the windows-tls project uses it). Again the case where I
want to use it in tests for my own new libraries but without making a
copy of the source. In other words in its own shared place. It does
require ASIO, so I'm not sure it should be grouped with the rest of
the stuff above.

Now going back to these new libraries, I have Boost.HTTP-Proto which
is a brand new library that speaks HTTP/1 but you have to bring your
own networking. It is purely a low-level protocol library, but with a
scope expanded from Beast to support more useful things that users
need. Since networking is obviously needed I am developing the
companion library Boost.HTTP-IO in parallel which uses ASIO to drive
the protocol library. They use separate repositories and thus the
protocol library is physically isolated, has its own tests, CI, and
documentation, and so forth.

There is a cosmetic issue where users have to write
boost::http_proto::request if they want to declare a request container
but then they have to write boost::http_io::write if they want to send
the request over a socket (note the different namespaces). This is
kind of annoying, and becomes even more annoying when you have to
remember which namespace the error constant is located in:
http_proto::error::end_of_message or http_io::error::end_of_file.

One possibility is for these libraries to use the same namespace:
boost::http. Or boost::http_proto. But this breaks the rule of
one-repository, one-unique-namespace.

Once these libraries are done I will at some point want to write
Boost.Websocket-Proto and Boost.Websocket-IO which are similar to
their HTTP counterparts (*). And then we will have twice the number of
namespaces and error constant enums:
  boost::http_proto
  boost::http_io
  boost::websocket_proto
  boost::websocket_io
and
  boost::urls

These all have their own error enums called "error". If we create a
separate repository for common code that all 5 of these libraries use,
then that's a sixth namespace also with error code.

Okay so that's the stuff that I've been pondering as I write this new
code, are any of these real problems? What can I do about it? I'm
interested in your thoughts.

* The purpose of writing these new, seemingly duplicative libraries is
to build on the experience of Beast to smooth out the discovered rough
edges and provide a superior interface and implementation.

-- 
Regards,
Vinnie
Follow me on GitHub: https://github.com/vinniefalco