Hey all, Often, I find bugs in Boost that I need to patch manually before the next release. What I used to do before was keep a copy of Boost in my source tree which made this easier: just patch it directly. This made it easy to modify, but annoying to post patches and honestly, they just got lost most of the time. With Git, it should become easier to contribute said patches as well as pull in whichever particular changes I need. However when I attempted such a fork recently, I was a bit lost. As I understood it, the process should have been: 1. Create a fork of https://github.com/boostorg/boost and use this fork in my project 2. Create a fork of the library/libraries I wanted to patch 3. Update the submodules in my boostorg/boost fork to point to my forked libraries 4. Post a pull request for the respective libraries. Once the changes have been accepted, point back to the official boostorg repo I failed terribly, no doubt due to PEBKAC. Is there a chance that someone could write a tutorial or a how-to guide for the recommended process of forking Boost with Git? The main thing I need is the ability to have a patched version of Boost that I can use while the patches are being reviewed/accepted/rejected. Thanks, Sohail
On Fri, Nov 22, 2013 at 8:41 PM, Sohail Somani <sohail@taggedtype.net>wrote:
I failed terribly, no doubt due to PEBKAC. Is there a chance that someone could write a tutorial or a how-to guide for the recommended process of forking Boost with Git? The main thing I need is the ability to have a patched version of Boost that I can use while the patches are being reviewed/accepted/rejected.
Could you clarify at which point in the process do you observe failure?
On 11/22/2013, 3:38 PM, Klaim - Joël Lamotte wrote:
On Fri, Nov 22, 2013 at 8:41 PM, Sohail Somani<sohail@taggedtype.net>wrote:
I failed terribly, no doubt due to PEBKAC. Is there a chance that someone could write a tutorial or a how-to guide for the recommended process of forking Boost with Git? The main thing I need is the ability to have a patched version of Boost that I can use while the patches are being reviewed/accepted/rejected.
Could you clarify at which point in the process do you observe failure?
Good question. The first thing I noticed was EOL-related issues which prevented me from (I think) checking out different branches. For example, bootstrap.bat was "modified" even though I had never touched it after a clone. Eventually, I got past this somehow but I can't remember how. I was checking out on a Mac. In terms of how I failed, it was more along the lines of not understanding Git submodules properly. Although I'm going off memory, I believe the submodules file had some sort of relative path like ../serialization.git. I tried to change these to my own fork URL but after that, I could not figure out how to update the modules because some modules used the relative path and some used my own fork's path. I wish I had the right error for you but I don't since it was a couple of weeks back. I am no Git expert, clearly, but I know that there are a simple set of steps to do what I need done. Someone just needs to figure out the magic steps. If you want to suggest some specific steps, I can try them over the weekend if I have time. Thanks, Sohail
On Fri, Nov 22, 2013 at 10:20 PM, Sohail Somani <sohail@taggedtype.net>wrote:
Good question. The first thing I noticed was EOL-related issues which prevented me from (I think) checking out different branches. For example, bootstrap.bat was "modified" even though I had never touched it after a clone. Eventually, I got past this somehow but I can't remember how. I was checking out on a Mac.
I can't help with this specific problem but you should probably investigate git configuration files (locally) and check if your git version is appropriate.
In terms of how I failed, it was more along the lines of not understanding Git submodules properly. Although I'm going off memory, I believe the submodules file had some sort of relative path like ../serialization.git. I tried to change these to my own fork URL but after that, I could not figure out how to update the modules because some modules used the relative path and some used my own fork's path. I wish I had the right error for you but I don't since it was a couple of weeks back.
I'm not a git expert but I think you did the right approach. Once you have your own repository and your boost super repo clone now uses your github repository as submodule of the library to modify, it should work as you expected. Then don't forget to push your local changes to your library fork and then pull the changes into your boost super repo fork.
I am no Git expert, clearly, but I know that there are a simple set of steps to do what I need done. Someone just needs to figure out the magic steps. If you want to suggest some specific steps, I can try them over the weekend if I have time.
I didn't try the process yet but I expect your approach to be the right way as it is the usual way to deal with super repos and submodules. (from my experience at least)
On Fri, Nov 22, 2013 at 4:20 PM, Sohail Somani <sohail@taggedtype.net>wrote: ... The first thing I noticed was EOL-related issues which prevented me
from (I think) checking out different branches. For example, bootstrap.bat was "modified" even though I had never touched it after a clone. Eventually, I got past this somehow but I can't remember how. I was checking out on a Mac.
The EOL-related issues you describe were an irritation until recently, but seem to have cleared. Please let us know if a fresh clone still has those problems for you. --Beman
On 11/22/2013, 6:20 PM, Beman Dawes wrote:
... The first thing I noticed was EOL-related issues which prevented me
from (I think) checking out different branches. For example, bootstrap.bat was "modified" even though I had never touched it after a clone. Eventually, I got past this somehow but I can't remember how. I was checking out on a Mac.
The EOL-related issues you describe were an irritation until recently, but seem to have cleared. Please let us know if a fresh clone still has those problems for you.
I checked my ~/.gitconfig and I saw that I had autocrlf=input. This seems like it was added by some GUI tool I installed at some point. Removing this and doing another clone seems to have fixed the problem. Thanks, Sohail
On Fri, Nov 22, 2013 at 2:41 PM, Sohail Somani <sohail@taggedtype.net>wrote:
Hey all,
Often, I find bugs in Boost that I need to patch manually before the next release. What I used to do before was keep a copy of Boost in my source tree which made this easier: just patch it directly. This made it easy to modify, but annoying to post patches and honestly, they just got lost most of the time.
With Git, it should become easier to contribute said patches as well as pull in whichever particular changes I need. However when I attempted such a fork recently, I was a bit lost. As I understood it, the process should have been:
1. Create a fork of https://github.com/boostorg/boost and use this fork in my project
2. Create a fork of the library/libraries I wanted to patch
3. Update the submodules in my boostorg/boost fork to point to my forked libraries
4. Post a pull request for the respective libraries. Once the changes have been accepted, point back to the official boostorg repo
I failed terribly, no doubt due to PEBKAC. Is there a chance that someone could write a tutorial or a how-to guide for the recommended process of forking Boost with Git? The main thing I need is the ability to have a patched version of Boost that I can use while the patches are being reviewed/accepted/rejected.
Sohail, We certainly need such docs, and I'll put something together. But I'm not a Git expert and have never been through the pull request process for a sub-module. So let's ask for some help to make sure your approach is optimal. Git experts: Is the approach Sohail outlines above optimal? If not, what would you suggest? --Beman
Hi guys,
Beman Dawes wrote:
Sohail Somani wrote:
With Git, it should become easier to contribute said patches as well as pull in whichever particular changes I need. However when I attempted such a fork recently, I was a bit lost. As I understood it, the process should have been:
1. Create a fork of https://github.com/boostorg/boost and use this fork in my project
2. Create a fork of the library/libraries I wanted to patch
3. Update the submodules in my boostorg/boost fork to point to my forked libraries
4. Post a pull request for the respective libraries. Once the changes have been accepted, point back to the official boostorg repo
Git experts: Is the approach Sohail outlines above optimal? If not, what would you suggest?
I would do something like the following: After having retrieved boost, or as much of boost as I needed, I would: 1. Fork that particular repository on GitHub. So, perhaps I'd fork boost.gil. 2. Go to the particular submodule on my local machine and create a branch for my work. For example, I'd go to gil and run: `git checkout -b fix_casts` 3. Make my changes and commit on my branch. 4. Next, I would add my forked repository as one of the remotes for that submodule: `git remote add personal git@github.com:ncrookston/gil.git` 5. Finally, I would push my branch to my personal repo: `git push personal fix_casts` 6. Once on github, I can make the pull request. This means you don't have to change all your submodules, nor change things back. HTH, Nate
On 11/22/2013, 4:39 PM, Nathan Crookston wrote:
Hi guys,
Beman Dawes wrote:
Sohail Somani wrote:
With Git, it should become easier to contribute said patches as well as pull in whichever particular changes I need. However when I attempted such a fork recently, I was a bit lost. As I understood it, the process should have been:
1. Create a fork of https://github.com/boostorg/boost and use this fork in my project
2. Create a fork of the library/libraries I wanted to patch
3. Update the submodules in my boostorg/boost fork to point to my forked libraries
4. Post a pull request for the respective libraries. Once the changes have been accepted, point back to the official boostorg repo
Git experts: Is the approach Sohail outlines above optimal? If not, what would you suggest?
I would do something like the following:
After having retrieved boost, or as much of boost as I needed, I would:
1. Fork that particular repository on GitHub. So, perhaps I'd fork boost.gil. 2. Go to the particular submodule on my local machine and create a branch for my work. For example, I'd go to gil and run: `git checkout -b fix_casts` 3. Make my changes and commit on my branch. 4. Next, I would add my forked repository as one of the remotes for that submodule: `git remote add personal git@github.com:ncrookston/gil.git` 5. Finally, I would push my branch to my personal repo: `git push personal fix_casts` 6. Once on github, I can make the pull request.
This means you don't have to change all your submodules, nor change things back.
Thanks, but how would you share this repo with the rest of your team members? At the very least, build machines. They would not be very amenable to cloning a repo from dev machines. In this case, you'd need to fork the boostorg/boost repo, correct?
On Fri, Nov 22, 2013 at 10:47 PM, Sohail Somani <sohail@taggedtype.net>wrote:
Thanks, but how would you share this repo with the rest of your team members? At the very least, build machines. They would not be very amenable to cloning a repo from dev machines. In this case, you'd need to fork the boostorg/boost repo, correct?
You should fork A. the super repository. boostorg/boost, which will give you a way to modify which submodules are used; B. the specific library Fork them either on github or in a place available to your team, as long as they can access it it's fine. Then locally: 1. clone your boost super fork; 2. in this clone, change the address of the submodule you want to use your own library fork with; 3. this is a change, commit it, then push it up to your fork; 4. now your boost super repo fork is pointing to your sub library fork as a submodule, instead of the official repository of this library; 5. this one I'm not sure is necessary: now pull from your fork and make sure your local version is using the right submodule address (to your library fork); 6. each time you want to upgrade any of these forks, you'll need to pull from the original repositories and then merge with your changes; 7. if the forks are on github, indeed you can easily submit pull requests but still continue with your version if it's not accepted yet or ever; Unfortunately I can't check if all these steps works right now but will do it in a bit more than a week.
Hi, I was going to respond, but this covered most of what I was going to say. One minor addition: Klaim - Joël Lamotte wrote:
Sohail Somani wrote:
Thanks, but how would you share this repo with the rest of your team members? At the very least, build machines. They would not be very amenable to cloning a repo from dev machines. In this case, you'd need to fork the boostorg/boost repo, correct?
You should fork
A. the super repository. boostorg/boost, which will give you a way to modify which submodules are used; B. the specific library Fork them either on github or in a place available to your team, as long as they can access it it's fine.
Then locally: 1. clone your boost super fork; 2. in this clone, change the address of the submodule you want to use your own library fork with;
Just to clarify this step, since the other urls in the .gitmodules file are relative, you'll need to make them absolute and point them to the boostorg ones (unless you want to fork *all* the boostorg repos).
3. this is a change, commit it, then push it up to your fork; 4. now your boost super repo fork is pointing to your sub library fork as a submodule, instead of the official repository of this library; 5. this one I'm not sure is necessary: now pull from your fork and make sure your local version is using the right submodule address (to your library fork); 6. each time you want to upgrade any of these forks, you'll need to pull from the original repositories and then merge with your changes; 7. if the forks are on github, indeed you can easily submit pull requests but still continue with your version if it's not accepted yet or ever;
With that, I think it should work, though I'll also be unable to test for a couple days. Thanks, Nate
Nathan Crookston wrote:
Klaim - Joël Lamotte wrote:
You should fork
A. the super repository. boostorg/boost, which will give you a way to modify which submodules are used; B. the specific library Fork them either on github or in a place available to your team, as long as they can access it it's fine.
Then locally: 1. clone your boost super fork; 2. in this clone, change the address of the submodule you want to use your own library fork with;
Just to clarify this step, since the other urls in the .gitmodules file are relative, you'll need to make them absolute and point them to the boostorg ones (unless you want to fork *all* the boostorg repos).
3. this is a change, commit it, then push it up to your fork; 4. now your boost super repo fork is pointing to your sub library fork as a submodule, instead of the official repository of this library; 5. this one I'm not sure is necessary: now pull from your fork and make sure your local version is using the right submodule address (to your library fork); 6. each time you want to upgrade any of these forks, you'll need to pull from the original repositories and then merge with your changes; 7. if the forks are on github, indeed you can easily submit pull requests but still continue with your version if it's not accepted yet or ever;
With that, I think it should work, though I'll also be unable to test for a couple days.
Okay, I actually just tried this and it worked just fine. My steps were: 1. Fork boostorg/boost 2. Clone *just* boostorg/boost, not all the sub-projects (as this will fail): - `git clone git@github.com:boostorg/boost.git modular-boost` 2.5 (Optional) Create a branch (I called mine 'updated') 3. Edit .gitmodules and change all the relative URLs to absolute, SSH or HTTPS (whichever kind you or your company firewall likes). - I ran `:%s/\.\./https:\/\/github.com\/boostorg\/utility.git/g` in vim to use https. 3. Fork the particular repos you want to apply patches to: - For this example, I forked 'range'. 4. Edit the 'range' url line in boost/.gitmodules to point to your fork: - I changed the https://github.com/boostorg/range.git to git@github.com:ncrookston/range.git. 5. Commit your gitmodules change, push it to your fork: - `git commit -a -m "create local boostorg"; git push origin updated` 6. Now get all the submodules: - `git submodule update --init`. You should see it fetch most libraries from boostorg, but those you've edited will be registered with the URL you gave in step 4. If you need to make a fresh clone it's straightforward: If you created a new branch: - `git clone -b updated --recursive git@github.com:ncrookston/boost.git modular_boost` If you did not: - `git clone --recursive git@github.com:ncrookston/boost.git modular_boost` HTH, Nate
On Sat, Nov 23, 2013 at 12:55 AM, Nathan Crookston < nathan.crookston@gmail.com> wrote:
Okay, I actually just tried this and it worked just fine.
Nice!
My steps were:
1. Fork boostorg/boost 2. Clone *just* boostorg/boost, not all the sub-projects (as this will fail): - `git clone git@github.com:boostorg/boost.git modular-boost` 2.5 (Optional) Create a branch (I called mine 'updated') 3. Edit .gitmodules and change all the relative URLs to absolute, SSH or HTTPS (whichever kind you or your company firewall likes). - I ran `:%s/\.\./https:\/\/github.com\/boostorg\/utility.git/g` in vim to use https.
By the way, what was the benefit of using relative paths for submodules? Also, isn't there a git command to automatically get the full paths?
3. Fork the particular repos you want to apply patches to: - For this example, I forked 'range'. 4. Edit the 'range' url line in boost/.gitmodules to point to your fork: - I changed the https://github.com/boostorg/range.git to git@github.com:ncrookston/range.git. 5. Commit your gitmodules change, push it to your fork: - `git commit -a -m "create local boostorg"; git push origin updated` 6. Now get all the submodules: - `git submodule update --init`.
You should see it fetch most libraries from boostorg, but those you've edited will be registered with the URL you gave in step 4.
If you need to make a fresh clone it's straightforward: If you created a new branch: - `git clone -b updated --recursive git@github.com:ncrookston/boost.git modular_boost` If you did not: - `git clone --recursive git@github.com:ncrookston/boost.git modular_boost`
Hi, Klaim - Joël Lamotte wrote:
Nathan Crookston wrote:
3. Edit .gitmodules and change all the relative URLs to absolute, SSH or HTTPS (whichever kind you or your company firewall likes). - I ran `:%s/\.\./https:\/\/github.com\/boostorg\/utility.git/g` in vim to use https.
By the way, what was the benefit of using relative paths for submodules? Also, isn't there a git command to automatically get the full paths?
There was a thread a few days ago where it was decided to use relative paths to avoid needing to know whether the user wanted SSH or HTTPS. It was feared that different companies would have different restrictions, requiring everyone to either edit their .gitmodules file, or use the insteadOf submodule modifier. I was hoping we could simplify the instructions by using instead of to sub an absolute path for a relative one: git config url."https://github.com/boostorg/".insteadOf .. (and many other incantations) No luck so far :( If we could then you only have to change the URL for those you've cloned. I'm not sure I understand your last question. I assume you mean automatically get the full paths for the submodule repositories. At which point? When updating submodules? Nate
On Sat, Nov 23, 2013 at 2:15 AM, Nathan Crookston < nathan.crookston@gmail.com> wrote:
I'm not sure I understand your last question. I assume you mean automatically get the full paths for the submodule repositories. At which point? When updating submodules?
When cloning the super repo. I don't know if such an option exists but it would make one less step for people wanting to make pull requests.
On 11/23/2013 12:55 AM, Nathan Crookston wrote:
Okay, I actually just tried this and it worked just fine. My steps were:
I just went through the exercise and believe that there are some mistakes in the description.
1. Fork boostorg/boost 2. Clone *just* boostorg/boost, not all the sub-projects (as this will fail): - `git clone git@github.com:boostorg/boost.git modular-boost`
You should clone ncrookston/boost, not boostorg/boost.
2.5 (Optional) Create a branch (I called mine 'updated') 3. Edit .gitmodules and change all the relative URLs to absolute, SSH or HTTPS (whichever kind you or your company firewall likes). - I ran `:%s/\.\./https:\/\/github.com\/boostorg\/utility.git/g` in vim to use https.
The "utility.git" part should be omitted, otherwise all URLs will contain it. I used the following command to modify the file: sed --in-place=.bak -e 's/\.\./https:\/\/github.com\/boostorg\//g' .gitmodules
3. Fork the particular repos you want to apply patches to: - For this example, I forked 'range'.
Minor comment: bullet numbering is wrong. If my observations are correct, then it may be a good idea to update: https://svn.boost.org/trac/boost/wiki/StartModPatchAndPullReq
On 27 March 2014 12:42, Bjorn Reese <breese@mail1.stofanet.dk> wrote:
If my observations are correct, then it may be a good idea to update:
https://svn.boost.org/trac/boost/wiki/StartModPatchAndPullReq
If you're only changing one module at a time, forking the super project is probably more trouble than it's worth. But if you need to, I think it would be easier to keep the boostorg repo as the 'origin' remote so that the relative urls will continue to work, and add your fork's remote with a different name. I'd do something like: git clone --recursive git@github.com:boostorg/boost.git git remote add danieljames git@github.com:danieljames/boost.git git push --set-upstream danieljames master
On 03/27/2014 02:14 PM, Daniel James wrote:
If you're only changing one module at a time, forking the super project is probably more trouble than it's worth. But if you need to, I think it would
I do not know if I need to. I was just following the description. What is the procedure for making pull-request to a single module without forking the super project?
On 27 March 2014 14:01, Bjorn Reese <breese@mail1.stofanet.dk> wrote:
On 03/27/2014 02:14 PM, Daniel James wrote:
If you're only changing one module at a time, forking the super project is
probably more trouble than it's worth. But if you need to, I think it would
I do not know if I need to. I was just following the description.
What is the procedure for making pull-request to a single module without forking the super project?
I assume you already have a full check out of master. To create your branch: - Go to the repos' page and click the 'fork' button to create a fork. - Copy the 'ssh clone url' from the right hand side. - In the local copy of the module: git remote add remote-name (paste url here) git checkout origin/develop -b feature/branch-name git push --set-upstream remote-name feature/branch-name Now when you push changes it should go to your fork. You can create the pull request through the web interface, although be careful to make it against 'develop'. When you update the modules from the super project (using 'git submodule update --init'), it might put you back on the upstream branch, so you need to remember to checkout your branch when you want to work on it. That's a bit of a pain, but for a single module it's less work than managing changes to the super project.
On 03/28/2014 12:37 PM, Daniel James wrote:
- Go to the repos' page and click the 'fork' button to create a fork. - Copy the 'ssh clone url' from the right hand side. - In the local copy of the module:
git remote add remote-name (paste url here) git checkout origin/develop -b feature/branch-name git push --set-upstream remote-name feature/branch-name
Now when you push changes it should go to your fork.
This works just like a dream. Thanks.
Hi Bjorn, Bjorn Reese wrote:
On 11/23/2013 12:55 AM, Nathan Crookston wrote:
Okay, I actually just tried this and it worked just fine. My steps were:
I just went through the exercise and believe that there are some mistakes in the description.
I agree with all of your edits, thanks. I also agree with Daniel's suggestion, given that we're using relative URLs. Thanks, Nate
participants (6)
-
Beman Dawes -
Bjorn Reese -
Daniel James -
Klaim - Joël Lamotte -
Nathan Crookston -
Sohail Somani