[Pyrex] cdef __init__

[Sam Rushing]

This is a performance problem we've had for quite a while at IronPort,
and I think I've finally got a handle on it.
Hopefully this is useful to other folks.

For a primer on the issue, grep through the archives for the thread
"Cannot cdef __init__?".  Basically, constructing extension types is
more expensive than it could be because it uses the normal python
calling convention.  It's hard to make an object with a constructor as
fast as PyList_New(), for example.

The attached hack supports a very specific form of fast constructor. 
It's not ideal, and hopefully Greg can find a cleaner and better way to
wedge it in than I did.

What the patch does is this: whenever Pyrex sees the following:

  <type> (arg0, arg1, ...)

It changes it to something like this:

  <type.__fast_init__> (AllocExtNode (<type>), arg0, arg1, ...)

...if a cdef __fast_init__() method is defined.  A thoroughly
uncivilized hack quietly adds a 'return self' to the end of this
method's body, since this is something that's pretty easy to forget.

AllocExtNode() expands into a call to a utility function, which simply
calls _PyObject_GC_New() and _PyObject_GC_TRACK() on the fresh object. 
This seems to be the minimum necessary initialization for a GC object.

This constructor seems to run about 7X faster.  Without GC, it runs
about 10X faster but dumps core because of the uninitialized gc slots. 8^).

In other words, if extensions types could probe for the need to add
HAVE_GC, it would run 10X faster and the objects would be 12 bytes smaller.

Here's a short example:

cdef class fnord:

    cdef public int x, y, z

    cdef __fast_init__ (self, int x, int y, int z):

        self.x = x

        self.y = y

        self.z = z

def f0():

    return fnord (3, 4, 5)


I tried to find a way to automagically transmogrify a pre-existing "def
__init__()" method, but because of the different way in which def and
cdef methods are parsed it frightened me away.

-Sam

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: pyrex_init.diff
Url: http://lists.copyleft.no/pipermail/pyrex/attachments/20070324/4e196fb0/attachment.ksh 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 186 bytes
Desc: OpenPGP digital signature
Url : http://lists.copyleft.no/pipermail/pyrex/attachments/20070324/4e196fb0/attachment.bin