[Boost-bugs] [Boost C++ Libraries] #8001: stl_input_iterator generators garbage collection problems

Subject: [Boost-bugs] [Boost C++ Libraries] #8001: stl_input_iterator generators garbage collection problems
From: Boost C++ Libraries (noreply_at_[hidden])
Date: 2013-02-07 09:00:47

#8001: stl_input_iterator generators garbage collection problems
--------------------------------------------------+-------------------------
 Reporter:  Wichert Akkerman <wichert@…>          |       Owner:  rwgk   
     Type:  Bugs                                  |      Status:  new    
Milestone:  To Be Determined                      |   Component:  Python 
  Version:  Boost 1.52.0                          |    Severity:  Problem
 Keywords:                                        |  
--------------------------------------------------+-------------------------
 I am running into some unexpected errors when using stl_input_iterator. I
 have a fairly simple converter registered to convert an iterable list of
 values (custom enum) to a unordered_set. The code looks like this:

 {{{#!c++
 struct advicetype_set_from_python {
         advicetype_set_from_python() {
                 converter::registry::push_back(&convertible, &construct,
 type_id<std::unordered_set<AdviceType>>());
         }

         static void* convertible(PyObject *obj) {
                 if (obj==Py_None)
                         return obj;

                 PyObject* iter = PyObject_GetIter(obj);
                 if (iter==NULL) {
                         throw_error_already_set();
                         return 0;
                 }
                 Py_DECREF(iter);
                 return obj;
         }

         static void construct(PyObject* obj,
 converter::rvalue_from_python_stage1_data *data) {
                 std::unordered_set<AdviceType> result;
                 if (obj!=Py_None) {
                         object o = object(handle<>(obj));
                         stl_input_iterator<AdviceType> begin(o), end;
                         result.insert(begin, end);
                 }
                 void *storage =
 ((converter::rvalue_from_python_storage<std::unordered_set<AdviceType>>*)data)->storage.bytes;
                 new (storage) std::unordered_set<AdviceType>(result);
                 data->convertible = storage;
         }
 };
 }}}

 When using this code we were seeing segfaults due to what appears to be
 invalid reference counters. Since I suspected a problem in this convertor
 I wrote a simple test which triggers the converter a number of times:

 {{{#!python
 for i in range(5000):
     my_func({AdviceType.one, AdviceType.two})
 }}}

 Which had an interesting result: the tests immediately aborted with a
 "Fatal Python error: GC object already tracked" error. Commenting out the
 contents of the if-statement in construct() made the problem go away,
 which makes we think the bug is in the stl_input_iterator used there. To
 test this I rewrote my code to use a standard !PyObject_GetIter /
 !PyIter_Next loop and that indeed made all problems go away.

-- 
Ticket URL: <https://svn.boost.org/trac/boost/ticket/8001>
Boost C++ Libraries <http://www.boost.org/>
Boost provides free peer-reviewed portable C++ source libraries.