[Pyrex] Serious memory leak in a simple extension type

[Lukasz Pankowski]

Jonathan Doda <jdoda at sympatico.ca> writes:

> Hello.  I was hoping someone on this list would be able to help me
> figure out why my extension type is leaking.  I've attached the
> extension type, and a simple test case.  The test case will fairly
> quickly consume all the memory on my computer, but if I import my
> original pure python class instead of the pyrex extension type memory
> consumption remains steady at about 2.2 MB.

Hi,

This must be a bug in Pyrex.

Consider the method with object type as return (which does not have a
leak, so is a workaround)

    cdef object _add(self, Vector other):
        return Vector(self.x + other.x, self.y + other.y)

as opposed to the orginal (which exhibits the leak)

    cdef Vector _add(self, Vector other):
        return Vector(self.x + other.x, self.y + other.y)


Declaring Vector as result type here has two effects:

1. It adds a check of returned value to be of declared type.
2. Adds a buggy Py_INCREF which makes the result immortal.


Here is a fragment of diff (diff -u cvector.c-object cvector.c-Vector)
between Pyrex outputed C (with object vs with Vector _add methods
(slightly simplified)) which demonstrate this two effects:

@@ -180,11 +180,13 @@
   __pyx_2 = 0;
   __pyx_1 = PyObject_CallObject(((PyObject*)__pyx_ptype_7cvector_Vector), __pyx_3); if (!__pyx_1) {__pyx_filename = __
pyx_f[0]; __pyx_lineno = 95; goto __pyx_L1;}
   Py_DECREF(__pyx_3); __pyx_3 = 0;
-  __pyx_r = __pyx_1;
+  if (!__Pyx_TypeTest(__pyx_1, __pyx_ptype_7cvector_Vector)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 95; goto __p
yx_L1;}
+  Py_INCREF(__pyx_1);
+  __pyx_r = (struct __pyx_obj_7cvector_Vector *)__pyx_1;
   __pyx_1 = 0;
   goto __pyx_L0;
 
-  __pyx_r = Py_None; Py_INCREF(__pyx_r);
+  __pyx_r = Py_None; Py_INCREF((PyObject *) __pyx_r);
   goto __pyx_L0;
   __pyx_L1:;
   Py_XDECREF(__pyx_1);


-- 

=*= Lukasz Pankowski =*=