[git-help] Clone / fetch specific submodule SHA?
Does anyone know of a way to do a submodule update and clone that only fetches a specific SHA? In particular the SHA referenced in the superproject? Rene. -- -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo
On Tue, Dec 24, 2013 at 3:16 PM, Rene Rivera <grafikrobot@gmail.com> wrote:
Does anyone know of a way to do a submodule update and clone that only fetches a specific SHA? In particular the SHA referenced in the superproject?
Isn't that what git submodule update does if you give it a path to the submodule? Assuming the super-project is on branch master, of course. So for Boost.Timer, do: cd modular-boost # if needed git checkout master # if needed git submodule update libs/timer Or did I misunderstand your question? --Beman
On Tue, Dec 24, 2013 at 3:12 PM, Beman Dawes <bdawes@acm.org> wrote:
On Tue, Dec 24, 2013 at 3:16 PM, Rene Rivera <grafikrobot@gmail.com> wrote:
Does anyone know of a way to do a submodule update and clone that only fetches a specific SHA? In particular the SHA referenced in the superproject?
Isn't that what git submodule update does if you give it a path to the submodule? Assuming the super-project is on branch master, of course. So for Boost.Timer, do:
cd modular-boost # if needed git checkout master # if needed git submodule update libs/timer
Or did I misunderstand your question?
Yes, and no.. submodule update "checks out" the SHA of the commit in the reference of the superproject. It doesn't fetch or clone anything. What I'm asking is if there's a way to, give that I have the superproject (shallow clone, or partial history) to first fetch/clone the SHA of referenced in the superproject *only* and then update to that fetched SHA *only*. -- -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo
On Tue, Dec 24, 2013 at 3:28 PM, Rene Rivera <grafikrobot@gmail.com> wrote:
On Tue, Dec 24, 2013 at 3:12 PM, Beman Dawes <bdawes@acm.org> wrote:
On Tue, Dec 24, 2013 at 3:16 PM, Rene Rivera <grafikrobot@gmail.com> wrote:
Does anyone know of a way to do a submodule update and clone that only fetches a specific SHA? In particular the SHA referenced in the superproject?
Isn't that what git submodule update does if you give it a path to the submodule? Assuming the super-project is on branch master, of course. So for Boost.Timer, do:
cd modular-boost # if needed git checkout master # if needed git submodule update libs/timer
Or did I misunderstand your question?
Yes, and no.. submodule update "checks out" the SHA of the commit in the reference of the superproject. It doesn't fetch or clone anything. What I'm asking is if there's a way to, give that I have the superproject (shallow clone, or partial history) to first fetch/clone the SHA of referenced in the superproject *only* and then update to that fetched SHA *only*.
PS. The reason I ask is because waiting the 5 to 10 minutes for all the submodules to clone for the testing scripts, i.e. the time I'm waiting around to see if changes I made to the testing scripts work, is getting rather depressing and annoying. -- -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo
On 24 December 2013 21:28, Rene Rivera <grafikrobot@gmail.com> wrote:
On Tue, Dec 24, 2013 at 3:12 PM, Beman Dawes <bdawes@acm.org> wrote:
On Tue, Dec 24, 2013 at 3:16 PM, Rene Rivera <grafikrobot@gmail.com> wrote:
Does anyone know of a way to do a submodule update and clone that only fetches a specific SHA? In particular the SHA referenced in the superproject?
Isn't that what git submodule update does if you give it a path to the submodule? Assuming the super-project is on branch master, of course. So for Boost.Timer, do:
cd modular-boost # if needed git checkout master # if needed git submodule update libs/timer
Or did I misunderstand your question?
Yes, and no.. submodule update "checks out" the SHA of the commit in the reference of the superproject. It doesn't fetch or clone anything. What I'm asking is if there's a way to, give that I have the superproject (shallow clone, or partial history) to first fetch/clone the SHA of referenced in the superproject *only* and then update to that fetched SHA *only*.
With recent versions of git: git submodule update --depth 1 It's still slow though, and pretty pointless if you have a local mirror. For older versions I think you're going to need a bit of shell scripting. 'git submodule foreach' sets a variable with the hash, you can then do something like the commands at http://stackoverflow.com/a/3489576/2434 to clone it. But 'git submodule' is pretty slow anyway, it's a shell script which has to check lots of configuration options. Tt can be faster to implement things yourself that doesn't bother with configuration. For example you can read the submodules directly from the .gitmodules file, and get their details using git ls-tree: git config -f .gitmodules -l \ | grep '^submodule\.\w*\.path' \ | cut -d = -f2 \ | xargs git ls-tree HEAD If you just want to make a copy of a repo, and don't care about git, it can be quicker using git archive: git archive master | tar -x -C location Could put those two together to make a pretty fast shell script. Or are you doing this in python? I can try to write something if you want.
On Tue, Dec 24, 2013 at 4:37 PM, Daniel James <daniel@calamity.org.uk>wrote:
On 24 December 2013 21:28, Rene Rivera <grafikrobot@gmail.com> wrote:
On Tue, Dec 24, 2013 at 3:12 PM, Beman Dawes <bdawes@acm.org> wrote:
On Tue, Dec 24, 2013 at 3:16 PM, Rene Rivera <grafikrobot@gmail.com> wrote:
Does anyone know of a way to do a submodule update and clone that only fetches a specific SHA? In particular the SHA referenced in the superproject?
Isn't that what git submodule update does if you give it a path to the submodule? Assuming the super-project is on branch master, of course. So for Boost.Timer, do:
cd modular-boost # if needed git checkout master # if needed git submodule update libs/timer
Or did I misunderstand your question?
Yes, and no.. submodule update "checks out" the SHA of the commit in the reference of the superproject. It doesn't fetch or clone anything. What I'm asking is if there's a way to, give that I have the superproject (shallow clone, or partial history) to first fetch/clone the SHA of referenced in the superproject *only* and then update to that fetched SHA *only*.
With recent versions of git:
git submodule update --depth 1
That's one of the things I tried previously.. And it doesn't behave as one might reasonably expect. The problem is that it fetches *only* the last commit regardless of anything else. This most likely is not the commit that is referenced by the super-project. It's still slow though, and pretty pointless if you have a local
mirror.
I don't have a local mirror.. I have nothing. Since I must assume, and test, the state testers are going to be of start from scratch.
For older versions I think you're going to need a bit of shell scripting. 'git submodule foreach' sets a variable with the hash, you can then do something like the commands at http://stackoverflow.com/a/3489576/2434 to clone it.
But 'git submodule' is pretty slow anyway, it's a shell script which has to check lots of configuration options.
<sarcasm> Wonderful </sarcasm>
It can be faster to implement things yourself that doesn't bother with configuration. For example you can read the submodules directly from the .gitmodules file, and get their details using git ls-tree:
git config -f .gitmodules -l \ | grep '^submodule\.\w*\.path' \ | cut -d = -f2 \ | xargs git ls-tree HEAD
I've though about doing such things.. And it is something I do for the version of the scripts that uses Dulwich. But, alas, Dulwich chokes on the Boost repos. If you just want to make a copy of a repo, and don't care about git,
it can be quicker using git archive:
git archive master | tar -x -C location
That doesn't work, and it was the first thing I tried more than a year ago now. "archive" doesn't recurse submodules, hence it's useless unless I "manually" recurse submodules. I might consider going back to that at some point though as a fallback when git isn't around at all (as in no git command and dulwich isn't working).
Could put those two together to make a pretty fast shell script. Or are you doing this in python? I can try to write something if you want.
It's all Python, except that now I'm having to exec the git cli. And ideally I would love it if it was pure Python, i.e. no git cli exec. Because I prefer minimizing external tool configuration. -- -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo
On 25 December 2013 04:32, Rene Rivera <grafikrobot@gmail.com> wrote:
I don't have a local mirror.. I have nothing. Since I must assume, and test, the state testers are going to be of start from scratch.
You could use a local mirror for faster development, and then use a remote server for final testing once everything else is working.
It can be faster to implement things yourself that doesn't bother with configuration. For example you can read the submodules directly from the .gitmodules file, and get their details using git ls-tree:
git config -f .gitmodules -l \ | grep '^submodule\.\w*\.path' \ | cut -d = -f2 \ | xargs git ls-tree HEAD
I've though about doing such things.. And it is something I do for the version of the scripts that uses Dulwich. But, alas, Dulwich chokes on the Boost repos.
If you just want to make a copy of a repo, and don't care about git,
it can be quicker using git archive:
git archive master | tar -x -C location
That doesn't work, and it was the first thing I tried more than a year ago now. "archive" doesn't recurse submodules, hence it's useless unless I "manually" recurse submodules.
That's why I described how to manually get the submodules' hashes. Having a clean export is useful for the documentation build, so I implemented it last night: https://github.com/danieljames/boost-build-docs/commit/9cde80a5a2c248f0aa350... A python script could be better and faster because it can put the output of 'git config' into a dict, and would also need to only run 'git ls-tree' once to get hashes. And also because I'm not very good at shell scripting.
participants (3)
-
Beman Dawes -
Daniel James -
Rene Rivera