[Pyrex] C-API implementation in Pyrex 0.9.6

[Stefan Behnel]

Greg Ewing wrote:
> Stefan Behnel wrote:
>> But that's not a problem if the code uses a header file in the
>> background. I
>> understand your objections to that, but it does have the advantage of
>> making
>> it clear what the public C API is, as only those types, attributes and
>> functions would be defined in the .pxd file.
> 
> I still think it would make the dependencies too convoluted.
> It would mean that if foo.pyx imports spam.pxd, then foo.c
> depends not only on spam.pxd but spam.pyx as well.

I don't see that as a major drawback. If you change the public API of the spam
module (be it declared in the .pyx or in the .pxd), you will always have to
recompile spam.pyx anyway, which will regenerate the .h file that the public
.pxd would depend on. There is not much you gain by avoiding that dependency.

And note that this only applies to Pyrex code, C code will always depend on
the generated .h file anyway.


> I appreciate that it would be nice if Pyrex could do better,
> but even if it did, the dependency would still be there,
> i.e. if you change anything that alters the layout of an
> extension type, you *will* have to recompile all other modules
> that either cimport it or use the C API. There's no getting
> around that, and I prefer to make it obvious and explicit
> rather than having it be a potential source of hidden
> pitfalls.

If we separate the definition of an extension type into a public part and a
private part, the complete layout (or rather struct size) of extension types
is only relevant to external code when subclassing. Pyrex could always keep
the layout of the public fields at the top, so type /usage/ would be binary
compatible. When subclassing, however, Pyrex generates a new struct that
contains the parent struct at the top, so the parent struct size is required
at compile time. I think we agree that this would be very hard to make working
without going through complete definitions, e.g. in .h files.

I can understand your interest in keeping this dependency explicit, but since
it only regards subclassing, I wouldn't mind just documenting it as a
requirement of the subclassing process. I'm pretty sure no user would mind
having to recompile code that subclasses a C type when the internal
representation of that type changes (which usually implies a new release). The
docs could just present this as a drawback of the performance gain you get
from a Pyrex-subclassed type, and everyone will be happy.

Note that people who provide a public C-level API for their modules can
usually be expected to care about binary compatibility themselves (and if they
don't, their users will beat them until they become aware), so documenting
this should be sufficient IMHO.


> BTW, I have set up a Mercurial repository in my source tree
> and I've been committing the last few changes into it. Where
> should I go from here? What's the best way of making it
> available to you and others?

I'll let Robert or William respond to that, they have more experience with HG
than I do.

Stefan