Re: [boost] date_time -> serialization (Was: spirtit -> serialization)

Subject: Re: [boost] date_time -> serialization (Was: spirtit -> serialization)
From: Andrey Semashev (andrey.semashev_at_[hidden])
Date: 2014-06-16 16:23:09

On Mon, Jun 16, 2014 at 11:02 PM, Julian Gonggrijp
<j.gonggrijp_at_[hidden]> wrote:
>
> There is a very obvious value in version control: dependencies may
> change from one Boost commit to the next. The dependency handler
> should work not only for end users, but also for maintainers and
> testers who check out any point in Git history.

Given that the dependency tracking tool is able to reconstruct that
information from headers, this doesn't seem like a loss. Remember,
that cached metadata is only supposed to speed up things for the most
frequent use cases, which are checking out the most recent version
(for testers and maintainers) or installing a release (for users).
Everything that cannot be reconstructed (the "optional" annotations)
is still in git.

Thinking about it more, the requirement to build the dependency graph
before the the download may require the cache to be available for any
given git commit. This is probably a good reason to make the cache
version controlled.

> Perhaps it could be stored in a git
> note [1].

I'm not very familiar with git and don't know anything about git
notes. Maybe they fit for this purpose. But my request would be that
these notes are not required to be added by maintainers.

Do git notes affect history? If yes, it would be undesirable if
libraries history is spammed with automated commits adding notes with
dependency info.

> It is friendly to tell end users in advance what dependencies will be
> installed, but that can be solved by other means. A very simple
> solution would be to list the dependencies on the Boost website.

That doesn't really work for obvious reasons: (a) the advertised
dependencies will get out of sync with reality sooner or later and (b)
you can't realistically request users to consult the website when they
are about to install a Boost library. The tool should provide that
information.

It is possible that the tool is not able to do that, if the cache is
not available for the given commit to be checked out. The tool should
notify the user about this problem but still allow to download the
necessary components "blindly", by parsing headers for dependencies.

> A
> slightly more advanced solution would be to have the handler download
> only the dependency file associated with the release tag using git
> archive [2], before cloning the entire module (that might not work
> with git notes, though). For releases, the dependency information
> could also simply be aggregated in the superproject archive.

Ok. As I said, I specifically did not require any particular means for
delivering metadata into the tool. If this is possible with git,
provided that usability is satisfactory, I'm all for it.

Another alternative is to create a new git submodule to store the cache in.

> The advantage of just storing a plain file in the module directory is
> that it certainly works, even if you download an archive without git
> history, and without a need to set up a new FTP server or other web
> service. I would prefer to start there and investigate prettier
> solutions later.

We're discussing a mechanism that will require mass changes to the
libraries and possibly the workflow. I'd say the system should be
designed without fundamental flaws from the start. It may not
implement everything from the beginning, that I agree.

>> What I meant is that there should be some kind of "official" Boost repository
>> which will be used by the packaging tool (let's call it boost-pkg for
>> brevity). That repository will serve the metadata for boost-pkg. The metadata
>> will be updated when a certain new release is published into it, whether that
>> is a new library release through a git tag or the whole Boost release. I'm not
>> sure if updating the metadata upon a tag creation can be automated, but at
>> least for major Boost releases this should be doable.
>
> This seems to confirm that you are not interested in dependency
> handling for maintainers and testers (yet).

What makes you think so?

>> Continuing my fantasy,...
>
> This gets more and more ambitious. Don't take me wrong, I like what you
> describe, but I think it will be easier to get there if we take it one
> step at a time.
>
> I think you have now almost reinvented Ryppl, but I might be mistaken.

Yes, I let myself get carried away a little, and I'm not asking to
implement all that. I'm just assessing the possibilities.

>>> In addition, for maintainers:
>>> 3. Create/update the configuration file by running the handler tool
>>> before pushing to the public repo (this could be a git hook).
>>
>> This would be a blocker, for me at least. I'm sure I will forget doing that
>> and will be very much annoyed. A git hook that scans the headers on every push
>> doesn't sound very good.
>
> But you do agree that caching is a good idea, right? You seem to
> believe that caches should only be created for releases.

I agree that the cache is a good idea, as long as it's just a cache.
I'm just saying that its role is auxiliary and it should not be
managed by developers. I did not say that the cache should only be
created for releases (note that I mentioned checking out develop and
master with boost-pkg). But it might be more difficult to build the
cache in time for heads of branches; there will be some latency
between the commit and its metadata.

> I think a
> dependency handler would have at least as much value to maintainers
> and testers, if it works for any commit.

I'm not arguing with that. And it should work, even if there is no
cache whatsoever.