[Pyrex] first post, ref count problem

[Greg Ewing]

leonardo wrote:

> Before Chain used 5.42 seconds to append 1000000
> items,

Yow, if you need to deal with that many items,
you might want to consider using a different
approach altogether.

Are you sure you can't use a Python list instead
of a linked list? That would be a lot faster and
only use one word for each item. Note that you can
make direct calls to PyList_Append etc. if you
want for manipulating it, which should be fairly
fast.

> and at the end the final del doesn't free the
> memory anyway,

There must have been another reference somewhere
keeping your objects alive.

> The produced code (of the original version) is a long
> jungle, it contains LOTS of increfs or decrefs,

You only need to look at the spots surrounding
where you've used explicit casts to or from a
Python object type. The file/lineno comments in
the generated code will help you find those.

Another thing you can do to make it a bit safer
is to include a declaration like

   cdef extern from "Python.h":
     ctypedef struct PyObject

and then declare your manually-counted references
as PyObject * instead of object. Then you can pass
them directly to Py_INCREF and Py_DECREF without
having to cast. This should cut down the number of
casts you need to check by about half.

> Maybe this is a too much early speed optimization of
> mine.

Very premature, I'd say. I expect the time taken
to deallocate each item would completely swamp
the time taken to walk through the list items.

> My version needs just 1 assignament for every
> Item, inside the loop, yours 2.

But at the expense of more pointer dereferencing
and an extra loop termination test in the middle. I
can't really see it making a significant difference
either way.

As always, measure before optimising.

--
Greg