[Pyrex] Opaque structures and Extension Types

[Roberto Cavada]

Hello,

     I have a hopefully interesting problem, which I would like to
share somehow to find a possible solution. Unfortunately this is a
bit long to descibe in details.

In a project I make a large use of opaque C structures. Each
structure has a public interface in the header, containing the
opaque type definition and the functions set that can be used with it.

For example:

/* ---- Person.h */
typedef struct Person_TAG* Person_ptr;
#define PERSON(x) ((Person_ptr) x)

Person_ptr Person_create(char* name);
void Person_destroy(Person_ptr self);
char* Person_get_name(const Person_ptr self);

Here Person_TAG is a structure that is defined in private interface
that are not visible to users. In a way Person is an opaque class
that can be controlled by using public methods create, destroy and
get_name.

Method 'create(char* name)' works as follows:
1. Mallocs structure Person_TAG
2. Calls the internal function person_init(Person_ptr self, char* name)

Meaning that the construction is splitted into two separated phases
resp. for the memory allocation for the structure, and for the
structure's fields initialization.

That's easy and simple. Things get slightly more complicated when
(single) inheritance is added. Here derived classes contain base
class structure as first member. For example:

/* ---- Worker_private.h */
#include "Person_private.h"
typedef struct Worker_TAG {
    Person __base__;

    /* worker members ... */
} Worker;

Construction of a worker is done by calling Worker_create, that
allocates space for a Worker structure, and calls base class init
and its own init function (resp. person_init and worker_init)

Virtual method can be defined by using class members that can be
overridden, or by exploiting klass structures like in gobject.

now I need to wrap a C class hierarchy in corresponding python
classes. And I would like to use pyrex for this.

I think the natural way should use Extension Types wrapping the
opaque structures, like in:

cdef class Person:
    cdef Person_ptr p

That would allow me to exploit also the constructor and destructor
to handle the life cycle of 'p'. Here to build a Person_ptr I have
to call Person_create, and to dispose it I can call Person_destroy.
That's fine.

Problems come with derived Extension Types, like in Worker:

cdef class Worker (Person):
    # the opaque 'p' is inherited from Person

This would allow me to use Worker wherever a Person is required,
that is perfect, and makes tricky casts I have to use in C a far far
remembrance:

/* in C it was ugly: */
Worker_ptr w = Worker_create("rob")
char* name = Person_get_name(PERSON(w));

# in python it would be a natural:
w = Worker("rob")
name = w.get_name()

However, I cannot figure out how I could make pyrex handle the C
structure creation and initialization. The problem is that derived
class contains their base classes, and the allocation part of
derived class allocates also the room for the base class structure.
   For example, Worker_create (m)allocates memory for Worker_TAG
structure, which in turn contains Person_TAG structure. After
allocation, the C constructor Worker_create calls person_init at
first, and finally calls worker_init to initialize the created instance.

In pyrex Extension Types, __cinit__ of base class is called of
course before __cinit__ of derived class, without the possibility of
controlling those calls. My way of handling allocation and
initialization of classes in C would require that __cinit__ of
derived class would be not called at all, as for example
Worker_create already handles the allocation and initialization of
Person and Worker structures.

Similarly for deallocation and destruction.

Any comment/solution? Am I missing something?

Thanks in advance,
r.