Re: [boost] [BPM] Supporting an alternative to meta/libraries.json

Subject: Re: [boost] [BPM] Supporting an alternative to meta/libraries.json
From: Rene Rivera (grafikrobot_at_[hidden])
Date: 2016-01-06 13:26:17

I should have prefaced all my replies with.. I don't feel strongly opposed
to this change. Just more like "meh" ;-) So please take my comments in that
context.

On Tue, Jan 5, 2016 at 4:52 PM, Louis Dionne <ldionne.2_at_[hidden]> wrote:

> Rene Rivera <grafikrobot <at> gmail.com> writes:
>
> >
> > On Mon, Jan 4, 2016 at 12:46 PM, Louis Dionne <ldionne.2 <at> gmail.com>
> wrote:
> >
> > > Dear list,
> > >
> > > Currently, all Boost libraries are required to have a
> meta/libraries.json
> > > file
> > > at their root, which contains some meta-data describing the library.
> As I
> > > understand it, this meta-data is then used to generate the list of all
> > > libraries with their description when a release is done.
> > >
> >
> > That's one use.
>
> If there are other uses, could you please elaborate? This is not
> rhetorical;
> such uses would point out other places that need to be changed to
> accommodate
> my proposal.
>

I don't know all of them ATM. Daniel maintains some extra tools that deal
with that file. So he will need to tell us.

> > I find having such a directory at the root of every library annoying,
> since
> > > it clutters it. Furthermore, for metaprogramming libraries, the name
> can be
> > > slightly confusing since meta/ might be expected to contain something
> > > different.
> > >
> >
> > 1. I don't see how it can be confusing to have that dir.
>
> If not confusing, it can definitely be surprising. The convention for
> storing
> meta data and configuration data is to use dotfiles, not meta/ directories.
>

It's a convention on Unix style systems. Not so much on Windows. For
example, Appveyor doesn't use a dotfile (although maybe they support
alternate names?)

Also, my point is not only about confusion, but mostly about clutter. The
> meta/
> directory at the root of Hana appears in my editor, my file browser and
> everywhere else as a directory. It is given the same priviledged place as
> other directories such as include/, when it should be a mere detail.
>

Personally I consider hiding of information (aka files) by IDEs to be a UI
bug. Which is especially painful to me as I spend most of my personal
programming time on OSX :-) It also means I had to dig around IDE
documentation to disable that bug.

> 3. The contents of the meta dir are not restricted to Boost. The
> > information in it could be used more generally.
>
> I think it is very unlikely that any organization beyond Boost itself is
> using
> this data.

True :-)

> But even then, I'm not proposing to take this data away, merely to
> support a more idiomatic way of providing it.
>
> > 4. We are likely going to add more data files to that dir for other uses.
> > For example it would be convenient to add release notes data in there.
>
> The argument of "we might use it in the future" is not a very strong one to
> me, since the meta/ directory has been used for years for the same purpose
> without change, AFAICT.
>

Good point.. Although when it comes to infrastructure changes everything
works in multi-year time frames. So movement might not be apparent.

> Also, as a side note, I would probably oppose storing release notes in such
> a directory. Some libraries are also released on their own, and it would be
> messy to store Boost-specific release notes in there. Plus, some people
> might
> want to use GitHub to provide release notes (this is my case). And even if
> we
> were to always include release notes in a library, we could argue that
> using
> a CHANGELOG file or similar is more in line with existing practice.
>

You are going to need to mention what goes into here <
http://www.boost.org/users/history/version_1_60_0.html> somewhere. And I'd
prefer to move away from having authors continually submit PRs to the
website for this.

However, if having a single .boost file honestly seems too restrictive for
> your planned use cases, and if you were to cast an influential NO vote on
> this proposal for that reason, I would
>
> 1. Be suspicious about use cases so complex they can't be handled by a
>    JSON file (my 400LOC Travis script is basically a JSON file!)
>

Sure. Or in my envisioned future use case.. A quickbook file.

> 2. Suggest using a .boost _directory_ instead of a single file
>

Renaming the directory seems like a better choice to me.

> > > I propose that we allow libraries to specify their metadata in a
> `.boost`
> > > file
> > > located at their root. Using a dotfile is the most common way of
> providing
> > > information for an external service (for example .travis.yml), and
> dotfiles
> > > are usually more discrete when viewing the project in an editor.
> > > Furthermore, making `boost` part of the filename is more descriptive.
> > >
> >
> > I don't see the point. It's a Boost library. It's already has "Boost" in
> > the "name".
>
> While it is obvious that Boost.XXX is a Boost library from its name, it is
> less than obvious what purpose serves a meta/libraries.json file at the
> root
> of such a library, and even less a meta/ directory alone. Using a .boost
> file
> instead says two things:
>
> 1. It consists of metadata, configuration or other nitpicks, since it's a
>    dotfile and this convention is widely used.

> 2. This metadata or configuration is related to Boost, since it has boost
> in
>    the filename.
>
> What if Travis had decided to use a ci/ directory instead of a .travis.yml
> file? Clearly, using Appveyor and Travis at the same time would have been
> confusing, since both of them are CIs. Using .travis is just more focused,
> and it carries the intent better.
>

It's a case of facets. For CIs they are facets of a repository. Which is
not really the case for Boost. It's single a Boost library. Now I grant you
that it's becoming possible to have a library be independent and a Boost
library. So perhaps that's the issue you are trying to resolve? I.e. How to
manage a library that has both an independent, outside of Boost, existence
and is part of Boost?

> > > I have submitted a pull request to BPM [1] adding the ability to fetch
> the
> > > meta
> > > data in a .boost file. The meta-data can still be provided in
> > > meta/libraries.json,
> > > so this should not affect existing libraries.
> > >
> >
> > It would as it's more maintenance and confusion as to where to have that
> > data. And as was pointed out that's not the only consumer of that data.
>
> As Glen suggested, we could also move all the modules to the dotfile
> approach.
> Otherwise, I think you can see from my pull requests that the amount of
> added
> maintenance is quite small.
>

If we change.. I prefer to just move everything to the new way. Having
multiple supported ways hasn't worked out so well for us in the past ;-)
And it's easier to deal with one way for tools (see my other post).

Plus, unless we realize the the whole world needs to be changed to implement
> this proposal, I volunteer to do it. I have already started doing so, but
> other places might need to be changed too. I would of course need some
> cooperation from people that are more knowledgeable about the Boost
> infrastructure than I am.
>

Right. You will definitely need help. At least to go in and merge all the
PRs involved in this. Since only admins (aka the release team) has enough
privileges to do this unilaterally. And it needs to be unilaterally, since
the last time we did such a change (creating that meta dir) it was
impossible to get authors to help.

> - Perhaps we could store the meta-data for all the libraries in the same
> > >   place, and get rid of the individual meta-data files?
> > >
> >
> > No. The point of having that dir in each library is to decentralize the
> > information so that it can be easily managed by the individual authors
> > instead of needing continual release manager intervention.
>
> Ok, let's keep it decentralized.
>
>
> > > - If this suggestion is accepted, is there more to modify than BPM?
> > >
> >
> > Yes. There are other tools (website & release related) . And of course
> > documentation.
>
> I already submitted a PR to the website. I wasn't able to find any other
> related reference to meta/, libraries.json or a combination of these when
> searching through the code base. Feel free to point out to me anything that
> would have been left out.
>

I'll try and find the other uses. But I don't know all of them.

> In summary, I'm against this as the perceived benefit is minimal compared
> > to the "implementation effort".
>
> In summary,
>
> 1. Using dotfiles allows other tools to see this meta information as it is,
>    and to deal with it properly.
>
> 2. Using dotfiles sticks to a well-established convention for storing meta
>    data in a non-intrusive way:
>
>     Travis  => .travis.yml
>     Git     => .git, .gitignore, .gitmodules, ...
>     Bash    => .bashrc, .bash_profile, .bash_history, ...
>     Hg      => .hgignore
>     Rbenv   => .rbenv
>     LLDB    => .lldb
>     ...
>     Boost   => meta/libraries.json
>
> Can't we just do like everybody else?
>

We are, historically, not fond of doing it like everybody else ;-)

>
> I would like to invite you to reconsider your opinion about this proposal.
> Since we're talking about naming, it's of course not a killer change and
> the
> benefits are hence not gigantic. But the benefits are nonetheless there,
> and
> a step in the good direction is worth taking, even if it is a small one.
>

I always reconsider (although I'm still strongly against cmake.. but that's
a different ball of wax)

:-D

-- 
-- Rene Rivera
-- Grafik - Don't Assume Anything
-- Robot Dreams - http://robot-dreams.net
-- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail