Executable files in Boost git repostories
We have again run into the situation where files with a Linux executable permission have been committed to various Boost git repositories, with Jim King creating a PR and list of these files in Boost Admin. I have fixed these for the repositories for which I have write access, and created PRs for the other repositories. But this begs the question as to what Boost's stance should be about adding actual executable files to a Boost git repository ? As an example a Linux bash command file was added to a particular repository and I created a PR to remove the executable file permission from the file. But the maintainer of the repository feels this is wrong and the Linux bash file should retain the executable file permissions and that the file should be part of the repository. But of course I am more interested here about the general principal of the matter. Obviously operating system command/batch files are executable files, but should they be so in a repository. Finally should be not have some sort of git hook that somehow eliminates this recurring problem of files with executable permissions being periodically committed to a Boost git repository ? Most all of the files I "fixed" or created PRs for are clearly source files what can not be "executed" in any way. But this happening again is a real problem to have to clean up each time.
On 4/12/2018 13:24, Edward Diener wrote:
We have again run into the situation where files with a Linux executable permission have been committed to various Boost git repositories, with Jim King creating a PR and list of these files in Boost Admin. I have fixed these for the repositories for which I have write access, and created PRs for the other repositories. But this begs the question as to what Boost's stance should be about adding actual executable files to a Boost git repository ? As an example a Linux bash command file was added to a particular repository and I created a PR to remove the executable file permission from the file. But the maintainer of the repository feels this is wrong and the Linux bash file should retain the executable file permissions and that the file should be part of the repository. But of course I am more interested here about the general principal of the matter. Obviously operating system command/batch files are executable files, but should they be so in a repository.
Files which are not actually executable scripts should not have that bit set, of course. Files which are executable scripts generally should have the bit set, even in the repository. However then the question becomes: are these scripts only for the maintainer's use (in which case perhaps they shouldn't be in the repository?) or are they intended for user use (in which case what happens on platforms that cannot run the script?) So for portability reasons, in my opinion, it's probably better to get b2 to do things rather than writing custom scripts that don't work on all platforms.
On 2018-12-03 19:31, Gavin Lambert via Boost wrote:
On 4/12/2018 13:24, Edward Diener wrote:
We have again run into the situation where files with a Linux executable permission have been committed to various Boost git repositories, with Jim King creating a PR and list of these files in Boost Admin. I have fixed these for the repositories for which I have write access, and created PRs for the other repositories. But this begs the question as to what Boost's stance should be about adding actual executable files to a Boost git repository ? As an example a Linux bash command file was added to a particular repository and I created a PR to remove the executable file permission from the file. But the maintainer of the repository feels this is wrong and the Linux bash file should retain the executable file permissions and that the file should be part of the repository. But of course I am more interested here about the general principal of the matter. Obviously operating system command/batch files are executable files, but should they be so in a repository.
Files which are not actually executable scripts should not have that bit set, of course.
Files which are executable scripts generally should have the bit set, even in the repository.
However then the question becomes: are these scripts only for the maintainer's use (in which case perhaps they shouldn't be in the repository?) or are they intended for user use (in which case what happens on platforms that cannot run the script?)
So for portability reasons, in my opinion, it's probably better to get b2 to do things rather than writing custom scripts that don't work on all platforms.
Why do we even have to discuss this here globally, rather than trusting project maintainers to know what they are doing ? Stefan -- ...ich hab' noch einen Koffer in Berlin...
On 12/4/18 3:40 AM, Stefan Seefeld via Boost wrote:
Why do we even have to discuss this here globally, rather than trusting project maintainers to know what they are doing ?
Apparently, maintainers can't be trusted as this issue comes up on and on again. Look at this as a sort of automated testing - you shouldn't be able to push bogus or straight dangerous stuff because you're affecting all git users.
On 2018-12-04 5:06 a.m., Andrey Semashev via Boost wrote:
On 12/4/18 3:40 AM, Stefan Seefeld via Boost wrote:
Why do we even have to discuss this here globally, rather than trusting project maintainers to know what they are doing ?
Apparently, maintainers can't be trusted as this issue comes up on and on again.
Take it up with those maintainers, but don't attempt to overrule them. That's precisely the wrong direction to take.
Look at this as a sort of automated testing - you shouldn't be able to push bogus or straight dangerous stuff because you're affecting all git users.
This is exactly the kind of thinking that annoys me with Boost. You are trying to solve the wrong problem ! We need more autonomy, not less ! Don't patronize maintainers by imposing artificial (and often misguided) rules. Instead, empower boost maintainers to take full responsibility of their projects, then establish criteria for projects to be boost members, and drop projects that don't meet these requirements. Stefan -- ...ich hab' noch einen Koffer in Berlin...
On 12/4/18 6:41 PM, stefan via Boost wrote:
On 2018-12-04 5:06 a.m., Andrey Semashev via Boost wrote:
On 12/4/18 3:40 AM, Stefan Seefeld via Boost wrote:
Why do we even have to discuss this here globally, rather than trusting project maintainers to know what they are doing ?
Apparently, maintainers can't be trusted as this issue comes up on and on again.
Take it up with those maintainers, but don't attempt to overrule them. That's precisely the wrong direction to take.
Look at this as a sort of automated testing - you shouldn't be able to push bogus or straight dangerous stuff because you're affecting all git users.
This is exactly the kind of thinking that annoys me with Boost. You are trying to solve the wrong problem ! We need more autonomy, not less !
Don't patronize maintainers by imposing artificial (and often misguided) rules.
Instead, empower boost maintainers to take full responsibility of their projects, then establish criteria for projects to be boost members, and drop projects that don't meet these requirements.
I disagree. I really don't want to go into the debate about autonomy vs. moderation, I'll just say that from my point of view a certain level of moderation and uniformity is needed. If a library wants to do everything in its own way, not caring to be part of the Boost ecosystem, it might as well be maintained as a standalone project. And this particular issue is not about infringing on anyone's rights, really. Unless you want the right to shoot yourself and everyone else in the foot. Marking files as executable, when they are not actual scripts, is never the right thing to do.
On 2018-12-04 10:54 a.m., Andrey Semashev via Boost wrote:
On 12/4/18 6:41 PM, stefan via Boost wrote:
Don't patronize maintainers by imposing artificial (and often misguided) rules.
Instead, empower boost maintainers to take full responsibility of their projects, then establish criteria for projects to be boost members, and drop projects that don't meet these requirements.
I disagree. I really don't want to go into the debate about autonomy vs. moderation, I'll just say that from my point of view a certain level of moderation and uniformity is needed. If a library wants to do everything in its own way, not caring to be part of the Boost ecosystem, it might as well be maintained as a standalone project.
I don't disagree. In fact, I believe that's what I just said: establish criteria for adherence, and judge projects by those. But the discussion (or at least, my argument) isn't about the right level of uniformity. It's about patronizing (and infantilizing) project maintainers. Stefan -- ...ich hab' noch einen Koffer in Berlin...
AMDG On 12/03/2018 05:31 PM, Gavin Lambert via Boost wrote:
On 4/12/2018 13:24, Edward Diener wrote:
<snip>Obviously operating system command/batch files are executable files, but should they be so in a repository.
Files which are not actually executable scripts should not have that bit set, of course.
Files which are executable scripts generally should have the bit set, even in the repository.
However then the question becomes: are these scripts only for the maintainer's use (in which case perhaps they shouldn't be in the repository?)
Such scripts belong in the repository anyway. Else what would we do for libraries whose maintainers' disappear? Or if a library has multiple maintainers?
or are they intended for user use (in which case what happens on platforms that cannot run the script?)
Just provide multiple scripts?
So for portability reasons, in my opinion, it's probably better to get b2 to do things rather than writing custom scripts that don't work on all platforms.
I agree with Stefan. Deciding how to handle these scripts is the responsibility of each individual maintainer. In Christ, Steven Watanabe
On Mon, Dec 3, 2018 at 4:52 PM Steven Watanabe via Boost <boost@lists.boost.org> wrote:
However then the question becomes: are these scripts only for the maintainer's use (in which case perhaps they shouldn't be in the repository?)
Beast needs a couple of shell scripts with the execute bit set otherwise they won't run correctly on Travis. For example: <https://github.com/boostorg/beast/blob/develop/tools/build-and-test.sh> (Note in the header above the file it says "Executable File", because the -x bit is set). Thanks
On 4/12/2018 13:52, Steven Watanabe wrote:
However then the question becomes: are these scripts only for the maintainer's use (in which case perhaps they shouldn't be in the repository?)
Such scripts belong in the repository anyway. Else what would we do for libraries whose maintainers' disappear? Or if a library has multiple maintainers?
Depends what the scripts do, of course. I was thinking of scripts that used paths or packages local to the maintainer's machine and so wouldn't work for anyone else anyway.
Just provide multiple scripts?
For Windows and Linux, that's not too difficult. What about other platforms? Especially those that the maintainer cannot test?
I agree with Stefan. Deciding how to handle these scripts is the responsibility of each individual maintainer.
That is true, but it's not silly to provide guidelines.
On 12/4/18 3:31 AM, Gavin Lambert via Boost wrote:
However then the question becomes: are these scripts only for the maintainer's use (in which case perhaps they shouldn't be in the repository?) or are they intended for user use (in which case what happens on platforms that cannot run the script?)
So for portability reasons, in my opinion, it's probably better to get b2 to do things rather than writing custom scripts that don't work on all platforms.
I agree with you but only as long as the script is supposed to be run by users, and routinely so. There's no reason to limit yourself to Boost.Build otherwise. Remember, Boost.Build is not a general scripting language, it's a build system. It's not as flexible or convenient as a shell script in what a shell script can do, provided that you're in a POSIX environment. Let alone, Python, for example. And before anyone asks, if you don't have a POSIX environment, you should really get one. Even on Windows it shouldn't be much of a problem. It will do you good beyond Boost as well.
AMDG On 12/03/2018 05:24 PM, Edward Diener via Boost wrote:
We have again run into the situation where files with a Linux executable permission have been committed to various Boost git repositories, with Jim King creating a PR and list of these files in Boost Admin. I have fixed these for the repositories for which I have write access, and created PRs for the other repositories. But this begs the question as to what Boost's stance should be about adding actual executable files to a Boost git repository ? As an example a Linux bash command file was added to a particular repository and I created a PR to remove the executable file permission from the file. But the maintainer of the repository feels this is wrong and the Linux bash file should retain the executable file permissions and that the file should be part of the repository. But of course I am more interested here about the general principal of the matter. Obviously operating system command/batch files are executable files, but should they be so in a repository.
Yes.
Finally should be not have some sort of git hook that somehow eliminates this recurring problem of files with executable permissions being periodically committed to a Boost git repository ? Most all of the files I "fixed" or created PRs for are clearly source files what can not be "executed" in any way. But this happening again is a real problem to have to clean up each time.
That would be reasonable as long as it doesn't accidentally reject files that should be executable. In Christ, Steven Watanabe
On 12/3/2018 7:41 PM, Steven Watanabe via Boost wrote:
AMDG
On 12/03/2018 05:24 PM, Edward Diener via Boost wrote:
We have again run into the situation where files with a Linux executable permission have been committed to various Boost git repositories, with Jim King creating a PR and list of these files in Boost Admin. I have fixed these for the repositories for which I have write access, and created PRs for the other repositories. But this begs the question as to what Boost's stance should be about adding actual executable files to a Boost git repository ? As an example a Linux bash command file was added to a particular repository and I created a PR to remove the executable file permission from the file. But the maintainer of the repository feels this is wrong and the Linux bash file should retain the executable file permissions and that the file should be part of the repository. But of course I am more interested here about the general principal of the matter. Obviously operating system command/batch files are executable files, but should they be so in a repository.
Yes.
Finally should be not have some sort of git hook that somehow eliminates this recurring problem of files with executable permissions being periodically committed to a Boost git repository ? Most all of the files I "fixed" or created PRs for are clearly source files what can not be "executed" in any way. But this happening again is a real problem to have to clean up each time.
That would be reasonable as long as it doesn't accidentally reject files that should be executable.
Then probably the right thing to so is to establish which file types are not executable files and then check only those file types to see if they have the executable bit set and reject the push if any of them do. Working the opposite way it would be much too easy erroneously identifying a file as not executable when it might be.
In Christ, Steven Watanabe
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On 12/4/18 3:24 AM, Edward Diener via Boost wrote:
We have again run into the situation where files with a Linux executable permission have been committed to various Boost git repositories, with Jim King creating a PR and list of these files in Boost Admin. I have fixed these for the repositories for which I have write access, and created PRs for the other repositories. But this begs the question as to what Boost's stance should be about adding actual executable files to a Boost git repository ? As an example a Linux bash command file was added to a particular repository and I created a PR to remove the executable file permission from the file. But the maintainer of the repository feels this is wrong and the Linux bash file should retain the executable file permissions and that the file should be part of the repository. But of course I am more interested here about the general principal of the matter. Obviously operating system command/batch files are executable files, but should they be so in a repository.
Yes, script files that are supposed to be run should have the executable bit set in git as well. The problem is when this bit is set for files that are not supposed to be executable (e.g. C++ source files or docs). These instances need to be fixed and, preferably, prevented from appearing in the future.
Finally should be not have some sort of git hook that somehow eliminates this recurring problem of files with executable permissions being periodically committed to a Boost git repository ? Most all of the files I "fixed" or created PRs for are clearly source files what can not be "executed" in any way. But this happening again is a real problem to have to clean up each time.
Yes, I'm in favor of adding some sort of a server-side git hook. For that all script files need to have an appropriate extension so that they can be whitelisted to have an executable permission. Though I'm not sure how server-side git hooks work with GitHub. A separate concern of mine is binary executables. IMHO, those should be outright banned, executable or not. Not that I remember seeing ones in git, but while we're at it, we could enforce it as well.
On Tue, Dec 4, 2018 at 4:03 AM Andrey Semashev via Boost < boost@lists.boost.org> wrote:
Yes, I'm in favor of adding some sort of a server-side git hook. For that all script files need to have an appropriate extension so that they can be whitelisted to have an executable permission. Though I'm not sure how server-side git hooks work with GitHub.
I don't think there's anything like git server-side hooks. Best you can do is write a GitHub Application. As otherwise it's all client side hooks. Other option is to add these checks to the regular library conformance testing < https://github.com/boostorg/boost/blob/master/status/boost_check_library.py
.
-- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net
On 12/4/2018 8:03 AM, Rene Rivera via Boost wrote:
On Tue, Dec 4, 2018 at 4:03 AM Andrey Semashev via Boost < boost@lists.boost.org> wrote:
Yes, I'm in favor of adding some sort of a server-side git hook. For that all script files need to have an appropriate extension so that they can be whitelisted to have an executable permission. Though I'm not sure how server-side git hooks work with GitHub.
I don't think there's anything like git server-side hooks.
https://git-scm.com/docs/githooks and https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks suggest otherwise, although understanding how to write a hook from the sparse documentation seems difficult to me. Maybe someone who has written a git hook before understands how we can cycle through the files in a 'git push' and, if one of those files is a source file with its executable bit set, reject the push.
Best you can do is write a GitHub Application. As otherwise it's all client side hooks. Other option is to add these checks to the regular library conformance testing < https://github.com/boostorg/boost/blob/master/status/boost_check_library.py
.
On Tue, Dec 4, 2018 at 9:28 AM Edward Diener via Boost < boost@lists.boost.org> wrote:
On 12/4/2018 8:03 AM, Rene Rivera via Boost wrote:
On Tue, Dec 4, 2018 at 4:03 AM Andrey Semashev via Boost < boost@lists.boost.org> wrote:
Yes, I'm in favor of adding some sort of a server-side git hook. For that all script files need to have an appropriate extension so that they can be whitelisted to have an executable permission. Though I'm not sure how server-side git hooks work with GitHub.
I don't think there's anything like git server-side hooks.
https://git-scm.com/docs/githooks and https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks suggest otherwise, although understanding how to write a hook from the sparse documentation seems difficult to me. Maybe someone who has written a git hook before understands how we can cycle through the files in a 'git push' and, if one of those files is a source file with its executable bit set, reject the push.
GitHub doesn't allow those server side hooks. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net
On 12/4/2018 10:34 AM, Rene Rivera via Boost wrote:
On Tue, Dec 4, 2018 at 9:28 AM Edward Diener via Boost < boost@lists.boost.org> wrote:
On 12/4/2018 8:03 AM, Rene Rivera via Boost wrote:
On Tue, Dec 4, 2018 at 4:03 AM Andrey Semashev via Boost < boost@lists.boost.org> wrote:
Yes, I'm in favor of adding some sort of a server-side git hook. For that all script files need to have an appropriate extension so that they can be whitelisted to have an executable permission. Though I'm not sure how server-side git hooks work with GitHub.
I don't think there's anything like git server-side hooks.
https://git-scm.com/docs/githooks and https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks suggest otherwise, although understanding how to write a hook from the sparse documentation seems difficult to me. Maybe someone who has written a git hook before understands how we can cycle through the files in a 'git push' and, if one of those files is a source file with its executable bit set, reject the push.
GitHub doesn't allow those server side hooks.
Are you saying that we have no access to the .git or .git/hooks subdirectory of a Github hosted repository ?
On Wed, Dec 5, 2018 at 3:50 PM Edward Diener via Boost < boost@lists.boost.org> wrote:
On 12/4/2018 10:34 AM, Rene Rivera via Boost wrote:
GitHub doesn't allow those server side hooks.
Are you saying that we have no access to the .git or .git/hooks subdirectory of a Github hosted repository ?
GitHub doesn't allow running scripts on their servers. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net
On 6/12/2018 10:53, Rene Rivera wrote:
On Wed, Dec 5, 2018 at 3:50 PM Edward Diener wrote:
Are you saying that we have no access to the .git or .git/hooks subdirectory of a Github hosted repository ?
GitHub doesn't allow running scripts on their servers.
You can write a webhook to run a script on a different server. But at best this could yell at someone after the fact; it can't prevent accepting a push (although it can mark a pull request as invalid -- that's how the CI hooks in).
Am 05.12.18 um 23:43 schrieb Gavin Lambert via Boost:
GitHub doesn't allow running scripts on their servers.
You can write a webhook to run a script on a different server. But at best this could yell at someone after the fact; it can't prevent accepting a push (although it can mark a pull request as invalid -- that's how the CI hooks in).
Simply do it in CI. All Boost builds are run (at least) on travis, right? So all you need is a simple shell script (or a one-liner in .travis.yml) that finds executable files with a forbidden suffix and returns a message and error status code when found. This will block all PRs and also flag commits pushed directly. Client-side hooks are worthless for that as you can't force everyone to install them.
On Thu, Dec 6, 2018 at 3:40 AM Alexander Grund via Boost < boost@lists.boost.org> wrote:
Simply do it in CI.
See <https://lists.boost.org/Archives/boost/2018/12/244572.php> -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net
On 12/5/2018 4:53 PM, Rene Rivera via Boost wrote:
On Wed, Dec 5, 2018 at 3:50 PM Edward Diener via Boost < boost@lists.boost.org> wrote:
On 12/4/2018 10:34 AM, Rene Rivera via Boost wrote:
GitHub doesn't allow those server side hooks.
Are you saying that we have no access to the .git or .git/hooks subdirectory of a Github hosted repository ?
GitHub doesn't allow running scripts on their servers.
Ok. Then the best alternative is to write a client-side commit hook.
participants (9)
-
Alexander Grund -
Andrey Semashev -
Edward Diener -
Gavin Lambert -
Rene Rivera -
stefan -
Stefan Seefeld
-
Steven Watanabe -
Vinnie Falco