Re: [boost] [git-help] Clone / fetch specific submodule SHA?

Subject: Re: [boost] [git-help] Clone / fetch specific submodule SHA?
From: Rene Rivera (grafikrobot_at_[hidden])
Date: 2013-12-24 23:32:36

On Tue, Dec 24, 2013 at 4:37 PM, Daniel James <daniel_at_[hidden]>wrote:

> On 24 December 2013 21:28, Rene Rivera <grafikrobot_at_[hidden]> wrote:
> > On Tue, Dec 24, 2013 at 3:12 PM, Beman Dawes <bdawes_at_[hidden]> wrote:
> >
> >> On Tue, Dec 24, 2013 at 3:16 PM, Rene Rivera <grafikrobot_at_[hidden]>
> >> wrote:
> >>
> >> > Does anyone know of a way to do a submodule update and clone that only
> >> > fetches a specific SHA? In particular the SHA referenced in the
> >> > superproject?
> >> >
> >>
> >> Isn't that what git submodule update does if you give it a path to the
> >> submodule? Assuming the super-project is on branch master, of course. So
> >> for Boost.Timer, do:
> >>
> >> cd modular-boost    # if needed
> >> git checkout master   # if needed
> >> git submodule update libs/timer
> >>
> >> Or did I misunderstand your question?
> >>
> >
> > Yes, and no.. submodule update "checks out" the SHA of the commit in the
> > reference of the superproject. It doesn't fetch or clone anything. What
> I'm
> > asking is if there's a way to, give that I have the superproject (shallow
> > clone, or partial history) to first fetch/clone the SHA of referenced in
> > the superproject *only* and then update to that fetched SHA *only*.
>
> With recent versions of git:
>
> git submodule update --depth 1
>

That's one of the things I tried previously.. And it doesn't behave as one
might reasonably expect. The problem is that it fetches *only* the last
commit regardless of anything else. This most likely is not the commit that
is referenced by the super-project.

It's still slow though, and pretty pointless if you have a local
> mirror.

I don't have a local mirror.. I have nothing. Since I must assume, and
test, the state testers are going to be of start from scratch.

> For older versions I think you're going to need a bit of shell
> scripting. 'git submodule foreach' sets a variable with the hash, you
> can then do something like the commands at
> http://stackoverflow.com/a/3489576/2434 to clone it.
>
> But 'git submodule' is pretty slow anyway, it's a shell script which
> has to check lots of configuration options.

<sarcasm> Wonderful </sarcasm>

> It can be faster to
> implement things yourself that doesn't bother with configuration. For
> example you can read the submodules directly from the .gitmodules
> file, and get their details using git ls-tree:
>
> git config -f .gitmodules -l \
>     | grep '^submodule\.\w*\.path' \
>     | cut -d = -f2 \
>     | xargs git ls-tree HEAD
>

I've though about doing such things.. And it is something I do for the
version of the scripts that uses Dulwich. But, alas, Dulwich chokes on the
Boost repos.

If you just want to make a copy of a repo, and don't care about git,
> it can be quicker using git archive:
>
> git archive master | tar -x -C location
>

That doesn't work, and it was the first thing I tried more than a year ago
now. "archive" doesn't recurse submodules, hence it's useless unless I
"manually" recurse submodules. I might consider going back to that at some
point though as a fallback when git isn't around at all (as in no git
command and dulwich isn't working).

> Could put those two together to make a pretty fast shell script. Or
> are you doing this in python? I can try to write something if you
> want.
>

It's all Python, except that now I'm having to exec the git cli. And
ideally I would love it if it was pure Python, i.e. no git cli exec.
Because I prefer minimizing external tool configuration.

-- 
-- 
-- Grafik - Don't Assume Anything
-- Redshift Software, Inc. - http://redshift-software.com
-- rrivera/acm.org - grafik/redshift-software.com
-- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo