Fwd: Boost.Python build error in 1.58 RC1, 2 and 3 (OK in Beta 1)
---------- Forwarded message ---------- From: <mark.incley@synx.com> Date: Wed, Apr 8, 2015 at 1:29 AM Subject: Boost.Python build error in 1.58 RC1, 2 and 3 (OK in Beta 1) To: mclow.lists@gmail.com Hi Marshall, Please excuse the direct mail, but my mailing list posting wasn't picked up on by anyone. I believe I've found a problem in Boost.Python that causes build errors if the user has Python 3.0 or later on their system. This change appears to have been introduced after 1.58 Beta 1. Using: bjam address-model=32 runtime-link=shared toolset=msvc-12.0 --stagedir=./stage/x86 C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\INCLUDE\xtgmath.h(206) : warning C4273: 'round' : inconsistent dll linkage C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\INCLUDE\math.h(516) : see previous definition of 'round' libs\python\src\exec.cpp(91) : error C2664: 'FILE *_Py_fopen(PyObject *,const char *)' : cannot convert argument 1 from 'char *' to 'PyObject *' Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or function-style cast call "C:\Users\MARK~1.INC\AppData\Local\Temp\b2_msvc_12.0_vcvarsall_x86.cmd" >nul cl /Zm800 -nologo @"bin.v2\libs\python\build\msvc-12.0\debug\link-static\threading-multi\exec.obj.rsp" ...failed compile-c-c++ bin.v2\libs\python\build\msvc-12.0\debug\link-static\threading-multi\exec.obj… The problem appears to be this Boost.Python code that was changed some time after 1.58 Beta 1 and appears in 1.58 RC1 within boost\libs\python\src\exec.cpp at around line 91: 1.58 beta 1: FILE *fs = fopen(f, "r"); 1.58 RCs: FILE *fs = _Py_fopen(f, "r"); ← f is a char *, hence the error shown above. It expects a PyObject * as the first parameter. This code occurs within a test for #if PY_VERSION_HEX >= 0x03000000 which may explain why no-one else has reported it – I have Python 3.3.0 on my HDD. Cheers, Mark. _____________________________________ Sent from http://boost.2283326.n4.nabble.com
On Wed, Apr 8, 2015 at 7:35 AM, Marshall Clow <mclow.lists@gmail.com> wrote:
---------- Forwarded message ---------- From: <mark.incley@synx.com> Date: Wed, Apr 8, 2015 at 1:29 AM Subject: Boost.Python build error in 1.58 RC1, 2 and 3 (OK in Beta 1) To: mclow.lists@gmail.com
Hi Marshall,
Please excuse the direct mail, but my mailing list posting wasn't picked up on by anyone. I believe I've found a problem in Boost.Python that causes build errors if the user has Python 3.0 or later on their system. This change appears to have been introduced after 1.58 Beta 1.
It appears to be this change: https://github.com/boostorg/python/commit/a911c17fd6e6532807d3a17d597eb391bd... -- Marshall
On Wed, Apr 8, 2015 at 7:45 AM, Marshall Clow <mclow.lists@gmail.com> wrote:
On Wed, Apr 8, 2015 at 7:35 AM, Marshall Clow <mclow.lists@gmail.com> wrote:
---------- Forwarded message ---------- From: <mark.incley@synx.com> Date: Wed, Apr 8, 2015 at 1:29 AM Subject: Boost.Python build error in 1.58 RC1, 2 and 3 (OK in Beta 1) To: mclow.lists@gmail.com
Hi Marshall,
Please excuse the direct mail, but my mailing list posting wasn't picked up on by anyone. I believe I've found a problem in Boost.Python that causes build errors if the user has Python 3.0 or later on their system. This change appears to have been introduced after 1.58 Beta 1.
It appears to be this change:
https://github.com/boostorg/python/commit/a911c17fd6e6532807d3a17d597eb391bd...
Which was applied directly to the master branch - never appearing on develop. -- Marshall
On Wed, Apr 8, 2015 at 7:52 AM, Marshall Clow <mclow.lists@gmail.com> wrote:
On Wed, Apr 8, 2015 at 7:45 AM, Marshall Clow <mclow.lists@gmail.com> wrote:
On Wed, Apr 8, 2015 at 7:35 AM, Marshall Clow <mclow.lists@gmail.com> wrote:
---------- Forwarded message ---------- From: <mark.incley@synx.com> Date: Wed, Apr 8, 2015 at 1:29 AM Subject: Boost.Python build error in 1.58 RC1, 2 and 3 (OK in Beta 1) To: mclow.lists@gmail.com
Hi Marshall,
Please excuse the direct mail, but my mailing list posting wasn't picked up on by anyone. I believe I've found a problem in Boost.Python that
causes
build errors if the user has Python 3.0 or later on their system. This change appears to have been introduced after 1.58 Beta 1.
It appears to be this change:
https://github.com/boostorg/python/commit/a911c17fd6e6532807d3a17d597eb391bd...
Which was applied directly to the master branch - never appearing on develop.
That's annoying, can we roll this back and do an rc4? It sure would be nice if boost::python supports Python 3. Tom
On 08/04/15 01:58 PM, Tom Kent wrote:
On Wed, Apr 8, 2015 at 7:52 AM, Marshall Clow <mclow.lists@gmail.com> wrote:
On Wed, Apr 8, 2015 at 7:45 AM, Marshall Clow <mclow.lists@gmail.com> wrote:
On Wed, Apr 8, 2015 at 7:35 AM, Marshall Clow <mclow.lists@gmail.com> wrote:
---------- Forwarded message ---------- From: <mark.incley@synx.com> Date: Wed, Apr 8, 2015 at 1:29 AM Subject: Boost.Python build error in 1.58 RC1, 2 and 3 (OK in Beta 1) To: mclow.lists@gmail.com
Hi Marshall,
Please excuse the direct mail, but my mailing list posting wasn't picked up on by anyone. I believe I've found a problem in Boost.Python that causes build errors if the user has Python 3.0 or later on their system. This change appears to have been introduced after 1.58 Beta 1.
It appears to be this change:
https://github.com/boostorg/python/commit/a911c17fd6e6532807d3a17d597eb391bd... Which was applied directly to the master branch - never appearing on develop.
That's annoying, can we roll this back and do an rc4? It sure would be nice if boost::python supports Python 3.
Yes. I'm sorry for that. Stefan -- ...ich hab' noch einen Koffer in Berlin...
On Wed, Apr 8, 2015 at 10:58 AM, Tom Kent <lists@teeks99.com> wrote:
On Wed, Apr 8, 2015 at 7:52 AM, Marshall Clow <mclow.lists@gmail.com> wrote:
It appears to be this change:
https://github.com/boostorg/python/commit/a911c17fd6e6532807d3a17d597eb391bd...
Which was applied directly to the master branch - never appearing on develop.
That's annoying, can we roll this back and do an rc4? It sure would be nice if boost::python supports Python 3.
It would be nice, but would it work? Would that enable Boost.Python to support Python 3? I thought that this change was to make it work on Py3 (though, apparently it was never tested). [ Looking at the code, I'm pretty sure that reverting this change would not affect Py2 use ] -- Marshall
On 08/04/15 10:35 AM, Marshall Clow wrote:
---------- Forwarded message ---------- From: <mark.incley@synx.com> Date: Wed, Apr 8, 2015 at 1:29 AM Subject: Boost.Python build error in 1.58 RC1, 2 and 3 (OK in Beta 1) To: mclow.lists@gmail.com
Hi Marshall,
Please excuse the direct mail, but my mailing list posting wasn't picked up on by anyone. I believe I've found a problem in Boost.Python that causes build errors if the user has Python 3.0 or later on their system. This change appears to have been introduced after 1.58 Beta 1.
Using:
bjam address-model=32 runtime-link=shared toolset=msvc-12.0 --stagedir=./stage/x86
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\INCLUDE\xtgmath.h(206) : warning C4273: 'round' : inconsistent dll linkage C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\INCLUDE\math.h(516) : see previous definition of 'round' libs\python\src\exec.cpp(91) : error C2664: 'FILE *_Py_fopen(PyObject *,const char *)' : cannot convert argument 1 from 'char *' to 'PyObject *' Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or function-style cast
call "C:\Users\MARK~1.INC\AppData\Local\Temp\b2_msvc_12.0_vcvarsall_x86.cmd" >nul cl /Zm800 -nologo @"bin.v2\libs\python\build\msvc-12.0\debug\link-static\threading-multi\exec.obj.rsp"
...failed compile-c-c++ bin.v2\libs\python\build\msvc-12.0\debug\link-static\threading-multi\exec.obj…
The problem appears to be this Boost.Python code that was changed some time after 1.58 Beta 1 and appears in 1.58 RC1 within boost\libs\python\src\exec.cpp at around line 91:
1.58 beta 1: FILE *fs = fopen(f, "r"); 1.58 RCs: FILE *fs = _Py_fopen(f, "r"); ← f is a char *, hence the error shown above. It expects a PyObject * as the first parameter.
I'm confused. The Python C API provides a "_Py_fopen(const char *, const char *)" function (matching the above call), as well as a "_Py_fopen_obj(PyObject *, const char *)" function. Unfortunately I don't have Windows and thus can't attempt to reproduce the problem. (See https://hg.python.org/cpython/file/0ca028765488/Python/fileutils.c#l1035) What is the exact version of Python you are using ?
This code occurs within a test for #if PY_VERSION_HEX >= 0x03000000 which may explain why no-one else has reported it – I have Python 3.3.0 on my HDD.
I can't reproduce the error on Linux, FWIW, Stefan -- ...ich hab' noch einen Koffer in Berlin...
It's Python 3.3.0. In the file c:\Python33\include\fileutils.h there's only the following two fopen functions: PyAPI_FUNC(FILE *) _Py_wfopen( const wchar_t *path, const wchar_t *mode); PyAPI_FUNC(FILE*) _Py_fopen( PyObject *path, const char *mode); It appears there's only a wide char version for Windows. Mark. -- View this message in context: http://boost.2283326.n4.nabble.com/Fwd-Boost-Python-build-error-in-1-58-RC1-... Sent from the Boost - Dev mailing list archive at Nabble.com.
On 09/04/15 02:58 AM, Mark Incley wrote:
It's Python 3.3.0. In the file c:\Python33\include\fileutils.h there's only the following two fopen functions:
[...] *Sigh*. It apears the new API was introduced here: https://hg.python.org/cpython/rev/ef889c3d5dc6 That's the price to pay for using non-public APIs... I have just committed https://github.com/boostorg/python/commit/3e405b6fd5db5615bbef241763de070118..., which should preserve the new functionality for Python 3.4, and use a little workaround for Python 3 < 3.4 Thanks, Stefan -- ...ich hab' noch einen Koffer in Berlin...
On Thu, Apr 9, 2015 at 6:00 AM, Stefan Seefeld <stefan@seefeld.name> wrote:
*Sigh*. It apears the new API was introduced here: https://hg.python.org/cpython/rev/ef889c3d5dc6 That's the price to pay for using non-public APIs...
I have just committed
https://github.com/boostorg/python/commit/3e405b6fd5db5615bbef241763de070118... , which should preserve the new functionality for Python 3.4, and use a little workaround for Python 3 < 3.4
Stefan -- PLEASE STOP COMMITTING DIRECTLY TO MASTER! That's not how we do development on Boost. Commit your changes to the develop branch, give the testers a chance to run, and *then*, if everything seems OK, merge from develop to master. -- Marshall
On Thu, Apr 9, 2015 at 3:51 PM, Marshall Clow <mclow.lists@gmail.com> wrote:
PLEASE STOP COMMITTING DIRECTLY TO MASTER!
That's not how we do development on Boost.
Commit your changes to the develop branch, give the testers a chance to run, and *then*, if everything seems OK, merge from develop to master.
Can't this be handled by code / permissions? If not this is unlikely to be the last time this happens. -- Olaf
On Thu, Apr 9, 2015 at 10:01 AM, Olaf van der Spek <ml@vdspek.org> wrote:
On Thu, Apr 9, 2015 at 3:51 PM, Marshall Clow <mclow.lists@gmail.com> wrote:
PLEASE STOP COMMITTING DIRECTLY TO MASTER!
That's not how we do development on Boost.
Commit your changes to the develop branch, give the testers a chance to run, and *then*, if everything seems OK, merge from develop to master.
Can't this be handled by code / permissions?
Could you explain what you mean by "handled by code / permissions"? While I understand how a continuous integration framework might only merge to master when develop tests are passing satisfactorily, I'm not aware of any way to prevent breaking merges with our current testing approach. --Beman
Beman Dawes <bdawes <at> acm.org> writes:
On Thu, Apr 9, 2015 at 10:01 AM, Olaf van der Spek <ml <at> vdspek.org>
wrote:
On Thu, Apr 9, 2015 at 3:51 PM, Marshall Clow <mclow.lists <at>
gmail.com>
wrote:
PLEASE STOP COMMITTING DIRECTLY TO MASTER!
That's not how we do development on Boost.
Commit your changes to the develop branch, give the testers a chance to run, and *then*, if everything seems OK, merge from develop to master.
Can't this be handled by code / permissions?
Could you explain what you mean by "handled by code / permissions"?
While I understand how a continuous integration framework might only merge to master when develop tests are passing satisfactorily, I'm not aware of any way to prevent breaking merges with our current testing approach.
Perhaps it would already help if the default branch for github pull requests would be set to "develop" instead of "master" Boost-wide, as is already the case for some of the repositories?
On 09/04/15 11:40 AM, Beman Dawes wrote:
On Thu, Apr 9, 2015 at 10:01 AM, Olaf van der Spek <ml@vdspek.org> wrote:
On Thu, Apr 9, 2015 at 3:51 PM, Marshall Clow <mclow.lists@gmail.com> wrote:
PLEASE STOP COMMITTING DIRECTLY TO MASTER!
That's not how we do development on Boost.
Commit your changes to the develop branch, give the testers a chance to run, and *then*, if everything seems OK, merge from develop to master. Can't this be handled by code / permissions?
Could you explain what you mean by "handled by code / permissions"?
While I understand how a continuous integration framework might only merge to master when develop tests are passing satisfactorily, I'm not aware of any way to prevent breaking merges with our current testing approach.
The entire issue is a result of me accepting a pull request to master, followed by a fix to an issue only observed on master. In addition I also wasn't fully aware of the use of master essentially as a release branch. While I will be more careful in the future to avoid such mistakes, I also think it might be useful to prevent (forbid) pull requests to master to begin with. Thanks, Stefan -- ...ich hab' noch einen Koffer in Berlin...
On Thu, Apr 9, 2015 at 1:30 PM, Stefan Seefeld <stefan@seefeld.name> wrote:
The entire issue is a result of me accepting a pull request to master, followed by a fix to an issue only observed on master. In addition I also wasn't fully aware of the use of master essentially as a release branch. While I will be more careful in the future to avoid such mistakes, I also think it might be useful to prevent (forbid) pull requests to master to begin with.
Pull request permissions was something brought during the original transition to Github.. As in Github doesn't have a way to prevent pull requests. But I found this <https://nopullrequests.appspot.com/>. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail
On Thu, Apr 9, 2015 at 5:40 PM, Beman Dawes <bdawes@acm.org> wrote:
On Thu, Apr 9, 2015 at 10:01 AM, Olaf van der Spek <ml@vdspek.org> wrote:
On Thu, Apr 9, 2015 at 3:51 PM, Marshall Clow <mclow.lists@gmail.com> wrote:
PLEASE STOP COMMITTING DIRECTLY TO MASTER!
That's not how we do development on Boost.
Commit your changes to the develop branch, give the testers a chance to run, and *then*, if everything seems OK, merge from develop to master.
Can't this be handled by code / permissions?
Could you explain what you mean by "handled by code / permissions"?
While I understand how a continuous integration framework might only merge to master when develop tests are passing satisfactorily, I'm not aware of any way to prevent breaking merges with our current testing approach.
Is it not possible to only allow merges from develop into master? -- Olaf
On Fri, Apr 10, 2015 at 7:40 AM, Olaf van der Spek <ml@vdspek.org> wrote:
Is it not possible to only allow merges from develop into master?
NO. Github doesn't have that kind of permissions control. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail
On Friday 10 April 2015 09:19:48 Rene Rivera wrote:
On Fri, Apr 10, 2015 at 7:40 AM, Olaf van der Spek <ml@vdspek.org> wrote:
Is it not possible to only allow merges from develop into master?
NO. Github doesn't have that kind of permissions control.
FWIW, I don't think limiting maintainers from accessing their libraries is the right way to go in the first place. One of the reasons why we moved to git was to decentralize Boost and allow more freedom to developers. What needs to be changed is the release process which currently relies on the global freeze instead of branching or forking.
On 04/10/2015 05:28 PM, Andrey Semashev wrote:
On Friday 10 April 2015 09:19:48 Rene Rivera wrote:
On Fri, Apr 10, 2015 at 7:40 AM, Olaf van der Spek <ml@vdspek.org> wrote:
Is it not possible to only allow merges from develop into master?
NO. Github doesn't have that kind of permissions control.
FWIW, I don't think limiting maintainers from accessing their libraries is the right way to go in the first place. One of the reasons why we moved to git was to decentralize Boost and allow more freedom to developers. What needs to be changed is the release process which currently relies on the global freeze instead of branching or forking.
I don't think there's a primarily technical problem. No matter how release process goes, we either need to assume that 'master' branch of any component is good to be released at all times, or coordinate with maintainers of 120 components to determine which revision of 'master' is really good to be released. You can guess which approach release managers would prefer. -- Vladimir Prus CodeSourcery / Mentor Embedded http://vladimirprus.com
On Wednesday 15 April 2015 15:32:12 Vladimir Prus wrote:
On 04/10/2015 05:28 PM, Andrey Semashev wrote:
On Friday 10 April 2015 09:19:48 Rene Rivera wrote:
On Fri, Apr 10, 2015 at 7:40 AM, Olaf van der Spek <ml@vdspek.org> wrote:
Is it not possible to only allow merges from develop into master?
NO. Github doesn't have that kind of permissions control.
FWIW, I don't think limiting maintainers from accessing their libraries is the right way to go in the first place. One of the reasons why we moved to git was to decentralize Boost and allow more freedom to developers. What needs to be changed is the release process which currently relies on the global freeze instead of branching or forking.
I don't think there's a primarily technical problem. No matter how release process goes, we either need to assume that 'master' branch of any component is good to be released at all times, or coordinate with maintainers of 120 components to determine which revision of 'master' is really good to be released.
You can guess which approach release managers would prefer.
I'm not arguing against taking master as a starting point for a release; it's a reasonable compromise. What I'm saying is that once release process has started there should be no restriction on updating master in submodules (i.e. the freeze), the release should branch off the main code base and get finished regardless of the modifications developers make. It is possible that master is broken at the point of branching - and part of release process is to fix it for the release or revert to a last known good version. But the probability of breakage is greatly reduced with this approach. Note that when I say "branch" above I don't necessarily mean "git branch". In fact, it may be more convenient to use forks to isolate the code to be released from the main code base and use pull requests for patches that need to get into the release.
Andrey Semashev wrote:
What I'm saying is that once release process has started there should be no restriction on updating master in submodules (i.e. the freeze), the release should branch off the main code base and get finished regardless of the modifications developers make.
Something like this? http://thread.gmane.org/gmane.comp.lib.boost.devel/247192 Nowadays, I'd no longer advocate for the odd branch naming where develop corresponds to master - it's too late for that. But the general idea still stands.
On Wednesday 15 April 2015 16:22:33 Peter Dimov wrote:
Andrey Semashev wrote:
What I'm saying is that once release process has started there should be no restriction on updating master in submodules (i.e. the freeze), the release should branch off the main code base and get finished regardless of the modifications developers make.
Something like this?
http://thread.gmane.org/gmane.comp.lib.boost.devel/247192
Nowadays, I'd no longer advocate for the odd branch naming where develop corresponds to master - it's too late for that. But the general idea still stands.
I suspect we won't be able to get away with just the superproject branching. If instead of reverting a broken submodule we want to fix it, and the maintainer is unresponsive for some reason, we need either a branch in the submodule or a fork of it. Since the branch can be modified by someone else than a release manager, the fork seems more preferred. The release branch of superproject would have to refer to the fork in this case.
Andrey Semashev wrote:
I suspect we won't be able to get away with just the superproject branching. If instead of reverting a broken submodule we want to fix it, and the maintainer is unresponsive for some reason, we need either a branch in the submodule or a fork of it. Since the branch can be modified by someone else than a release manager, the fork seems more preferred. The release branch of superproject would have to refer to the fork in this case.
In what way is this an argument against the suggested structure? It supports this workflow. I don't think that something like that will ever prove necessary, but it's possible.
On Wednesday 15 April 2015 18:14:35 Peter Dimov wrote:
Andrey Semashev wrote:
I suspect we won't be able to get away with just the superproject branching. If instead of reverting a broken submodule we want to fix it, and the maintainer is unresponsive for some reason, we need either a branch in the submodule or a fork of it. Since the branch can be modified by someone else than a release manager, the fork seems more preferred. The release branch of superproject would have to refer to the fork in this case.
In what way is this an argument against the suggested structure? It supports this workflow. I don't think that something like that will ever prove necessary, but it's possible.
I didn't say I disagree with your superproject structure. The post you referenced only described downgrading submodules as a measure of fixing breakage, I'm just saying we might need to apply patches instead. For that it is not sufficient to only manage the superproject.
Hi Andrey, Am Mittwoch, 15. April 2015, 15:52:13 schrieb Andrey Semashev:
On Wednesday 15 April 2015 15:32:12 Vladimir Prus wrote:
I don't think there's a primarily technical problem. No matter how release process goes, we either need to assume that 'master' branch of any component is good to be released at all times, or coordinate with maintainers of 120 components to determine which revision of 'master' is really good to be released.
You can guess which approach release managers would prefer.
I'm not arguing against taking master as a starting point for a release; it's a reasonable compromise. What I'm saying is that once release process has started there should be no restriction on updating master in submodules (i.e. the freeze), the release should branch off the main code base and get finished regardless of the modifications developers make. It is possible that master is broken at the point of branching - and part of release process is to fix it for the release or revert to a last known good version. But the probability of breakage is greatly reduced with this approach.
The main problem is the necessary regression testing needed to actually verify that the regressions a) are indeed fixed b) don't introduce new failures. And our current testing system is hardcoded to "master" and "develop" branches. And can't test anything else, as it relies on the test runners admins to set up which branch to test.
Note that when I say "branch" above I don't necessarily mean "git branch". In fact, it may be more convenient to use forks to isolate the code to be released from the main code base and use pull requests for patches that need to get into the release.
As soon as we get an up-to-date regression testing and reporting system, we can use any branching scheme you can imagine. And a lot more probably. That would mean using either Jenkins or BuildBot as a centralized server and some dashboard reporting tool for visualisation. But I don't know anyone with enough spare time to set up such a system without being paid and working full time. Not to mention the hardware necessary for this... Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! * voice: ++49 4257 300 ! Fährstraße 1 * fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen * jhunold@gmx.eu ! Germany
On Wed, Apr 15, 2015 at 8:24 AM, Jürgen Hunold <jhunold@gmx.eu> wrote:
Hi Andrey,
Am Mittwoch, 15. April 2015, 15:52:13 schrieb Andrey Semashev:
On Wednesday 15 April 2015 15:32:12 Vladimir Prus wrote:
I don't think there's a primarily technical problem. No matter how release process goes, we either need to assume that 'master' branch of any component is good to be released at all times, or coordinate with maintainers of 120 components to determine which revision of 'master' is really good to be released.
You can guess which approach release managers would prefer.
I'm not arguing against taking master as a starting point for a release; it's a reasonable compromise. What I'm saying is that once release process has started there should be no restriction on updating master in submodules (i.e. the freeze), the release should branch off the main code base and get finished regardless of the modifications developers make. It is possible that master is broken at the point of branching - and part of release process is to fix it for the release or revert to a last known good version. But the probability of breakage is greatly reduced with this approach.
The main problem is the necessary regression testing needed to actually verify that the regressions a) are indeed fixed b) don't introduce new failures.
And our current testing system is hardcoded to "master" and "develop" branches. And can't test anything else, as it relies on the test runners admins to set up which branch to test.
I wouldn't call it hardcoded.. But yes it relies on testers signifying the branch to test. And yes it could be more flexible.
Note that when I say "branch" above I don't necessarily mean "git branch".
In fact, it may be more convenient to use forks to isolate the code to be released from the main code base and use pull requests for patches that need to get into the release.
As soon as we get an up-to-date regression testing and reporting system, we can use any branching scheme you can imagine. And a lot more probably.
Not sure what you mean by "up-to-date". Could you be more specific? And I don't accept "use -insert-name- tool" as an answer. I want to know structure and process answers (as tools are just tools). That would mean using either Jenkins or BuildBot as a centralized server and
some dashboard reporting tool for visualisation.
No it doesn't mean that. Although Jenkins or BuildBot (or a variety of other testing resource management) system could be used in a solution. But I'm not aware of *any* current reporting system that meets Boost needs. I do spend a minuscule amount of time writing such a system.
But I don't know anyone with enough spare time to set up such a system without being paid and working full time. Not to mention the hardware necessary for this...
But as you just said.. I volunteer my unpaid free time to maintain and improve the testing infrastructure. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail
Hi Rene, Am Mittwoch, 15. April 2015, 08:42:40 schrieb Rene Rivera:
On Wed, Apr 15, 2015 at 8:24 AM, Jürgen Hunold <jhunold@gmx.eu> wrote:
And our current testing system is hardcoded to "master" and "develop" branches. And can't test anything else, as it relies on the test runners admins to set up which branch to test.
I wouldn't call it hardcoded.. But yes it relies on testers signifying the branch to test. And yes it could be more flexible.
More flexible and more responsive are the keys I think. Turnaround times are bound to be long if there is no central distribution of tasks. Or at least branches to test, which should be sufficient for Boost.
Note that when I say "branch" above I don't necessarily mean "git branch".
In fact, it may be more convenient to use forks to isolate the code to be released from the main code base and use pull requests for patches that need to get into the release.
As soon as we get an up-to-date regression testing and reporting system, we can use any branching scheme you can imagine. And a lot more probably.
Not sure what you mean by "up-to-date". Could you be more specific? And I don't accept "use -insert-name- tool" as an answer. I want to know structure and process answers (as tools are just tools).
Well, I was referring to the tools below.
That would mean using either Jenkins or BuildBot as a centralized server and
some dashboard reporting tool for visualisation.
No it doesn't mean that. Although Jenkins or BuildBot (or a variety of other testing resource management) system could be used in a solution.
I'm using BuildBot at work myself and have first-hand reports on Jenkins.
But I'm not aware of *any* current reporting system that meets Boost needs. I do spend a minuscule amount of time writing such a system.
Unfortunately, you are right. There is no out-of-the-box system. And I think it is hard to write one for Boost due to the diversity of testing methods used. From Boost.Test over flyweight test to somethings home grown. Not to mention that we only have volunteer testers with some real handicaps dedicated by some strange policies... I dream of something like Gerrit (see https://codereview.qt-project.org/#/q/status:open,n,z for the Qt one) for Boost. But this requires a working CI system, a completely different workflow and whatever. But on the other hand I don't know if growing a boost centric regression testing/CI framework is feasible. Especially if looking at the experience with Boost.Build.
But I don't know anyone with enough spare time to set up such a system without being paid and working full time. Not to mention the hardware necessary for this...
But as you just said.. I volunteer my unpaid free time to maintain and improve the testing infrastructure.
Which I really appreciate. My main point was to highlight the fact that we can't change the release process without tackling the testing infrastructure. And even adding another long standing release branch or whatever is not easy at the moment. Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! * voice: ++49 4257 300 ! Fährstraße 1 * fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen * jhunold@gmx.eu ! Germany
On Wednesday 15 April 2015 15:24:53 Jürgen Hunold wrote:
Hi Andrey,
Am Mittwoch, 15. April 2015, 15:52:13 schrieb Andrey Semashev:
On Wednesday 15 April 2015 15:32:12 Vladimir Prus wrote:
I don't think there's a primarily technical problem. No matter how release process goes, we either need to assume that 'master' branch of any component is good to be released at all times, or coordinate with maintainers of 120 components to determine which revision of 'master' is really good to be released.
You can guess which approach release managers would prefer.
I'm not arguing against taking master as a starting point for a release; it's a reasonable compromise. What I'm saying is that once release process has started there should be no restriction on updating master in submodules (i.e. the freeze), the release should branch off the main code base and get finished regardless of the modifications developers make. It is possible that master is broken at the point of branching - and part of release process is to fix it for the release or revert to a last known good version. But the probability of breakage is greatly reduced with this approach.
The main problem is the necessary regression testing needed to actually verify that the regressions a) are indeed fixed b) don't introduce new failures.
And our current testing system is hardcoded to "master" and "develop" branches. And can't test anything else, as it relies on the test runners admins to set up which branch to test.
Well, we could switch the master branch testing to release testing for the time while release is being prepared. This could probably be done with a commit to the testing scripts in master?
Andrey Semashev wrote:
Well, we could switch the master branch testing to release testing for the time while release is being prepared. This could probably be done with a commit to the testing scripts in master?
Manual switching will not be necessary. The testers will just cycle RC, master, RC, develop, and will skip testing if there have been no commits. RC doesn't change outside of a release, so it will not be tested unless needed.
Beman Dawes wrote
While I understand how a continuous integration framework might only merge to master when develop tests are passing satisfactorily, I'm not aware of any way to prevent breaking merges with our current testing approach.
I think there is a way to "lock" a branch so that no one can push a merged branch http://stackoverflow.com/questions/2471340/is-there-a-way-to-lock-a-branch-i... This looks like it might be useful to us - but don't take my word for it as I'm a git noob Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/Fwd-Boost-Python-build-error-in-1-58-RC1-... Sent from the Boost - Dev mailing list archive at Nabble.com.
On Thu, Apr 9, 2015 at 4:00 PM, Stefan Seefeld <stefan@seefeld.name> wrote:
On 09/04/15 02:58 AM, Mark Incley wrote:
It's Python 3.3.0. In the file c:\Python33\include\fileutils.h there's only the following two fopen functions:
[...]
*Sigh*. It apears the new API was introduced here: https://hg.python.org/cpython/rev/ef889c3d5dc6 That's the price to pay for using non-public APIs...
I have just committed https://github.com/boostorg/python/commit/3e405b6fd5db5615bbef241763de070118..., which should preserve the new functionality for Python 3.4, and use a little workaround for Python 3 < 3.4
This fix doesn't work on Ubuntu 15.04 (python 3.4.3). libs/python/src/exec.cpp: In function ‘boost::python::api::object boost::python::exec_file(boost::python::str, boost::python::api::object, boost::python::api::object)’: libs/python/src/exec.cpp:93:31: error: cannot convert ‘PyObject* {aka _object*}’ to ‘const char*’ for argument ‘1’ to ‘FILE* _Py_fopen(const char*, const char*)’ FILE *fs = _Py_fopen(fo, "r"); If I'm not mistaken, the version check is incorrect, it should be: #if PY_VERSION_HEX >= 0x03040000 (note that 4 is shifted left for 16 bits, not 20).
On Wed, May 6, 2015 at 2:03 PM, Andrey Semashev <andrey.semashev@gmail.com> wrote:
On Thu, Apr 9, 2015 at 4:00 PM, Stefan Seefeld <stefan@seefeld.name> wrote:
On 09/04/15 02:58 AM, Mark Incley wrote:
It's Python 3.3.0. In the file c:\Python33\include\fileutils.h there's only the following two fopen functions:
[...]
*Sigh*. It apears the new API was introduced here: https://hg.python.org/cpython/rev/ef889c3d5dc6 That's the price to pay for using non-public APIs...
I have just committed https://github.com/boostorg/python/commit/3e405b6fd5db5615bbef241763de070118..., which should preserve the new functionality for Python 3.4, and use a little workaround for Python 3 < 3.4
This fix doesn't work on Ubuntu 15.04 (python 3.4.3).
libs/python/src/exec.cpp: In function ‘boost::python::api::object boost::python::exec_file(boost::python::str, boost::python::api::object, boost::python::api::object)’: libs/python/src/exec.cpp:93:31: error: cannot convert ‘PyObject* {aka _object*}’ to ‘const char*’ for argument ‘1’ to ‘FILE* _Py_fopen(const char*, const char*)’ FILE *fs = _Py_fopen(fo, "r");
If I'm not mistaken, the version check is incorrect, it should be:
#if PY_VERSION_HEX >= 0x03040000
(note that 4 is shifted left for 16 bits, not 20).
With the above fix it compiled for me. Created a pull request: https://github.com/boostorg/python/pull/19 It would be nice if the patch in the 1.58 release notes was also updated.
participants (13)
-
Andrey Semashev -
Beman Dawes -
Jürgen Hunold -
Marcel Raad -
Mark Incley -
Marshall Clow -
Olaf van der Spek -
Peter Dimov -
Rene Rivera -
Robert Ramey -
Stefan Seefeld -
Tom Kent -
Vladimir Prus