[Pyrex] Re: Pyrex versus ctypes

[Thomas Heller]

Bob Ippolito <bob at redivi.com> writes:

> On Jun 10, 2004, at 6:01 PM, David McNab wrote:
>
>> After a year or so's hacking with Pyrex, I've come across the
>> 'ctypes' package (http://starship.python.net/crew/theller/ctypes/).
>>
>> ctypes allows the calling of C functions from *pure-python* code,
>> without requiring the generation/compilation of a C wrapper module.
>>
>> Pure python code using ctypes can even receive callbacks from C
>> funcs, and manipulate C data structures, pretty much most of what
>> one would need for interfacing with C.
>>
>> All this feels too good to be true.
>>
>> Can anyone offer some thoughts on the ideal demarcation lines
>> between Pyrex and ctypes, and some thoughts on pros and cons, and
>> which situations indicate Pyrex over ctypes and vice versa?
>>
>> For instance, how severe are ctypes' overheads with constructing the
>> C call frame, and rendering the return back to python?
>
> Here's a few ctypes disadvantages off the top of my head (I hope the
> advantages are obvious, since we're all Python users here, so I won't
> list them):

You might also get different answers if you ask in another forum ;-),
although Bob has some points here.

> ctypes can be one royal pain in the ass to debug.  There are still
> bugs in *it*, at least on non-win32 platforms.  I would wait a release
> or two unless you have very simple requirements (bugs that I've seen
> are pretty much limited to structure packing and python callbacks) or
> want to become a committer too :)

True, although there is progress.  I have to use sourceforge's
compilefarm, which is a pain to use, for platforms other than windows
and x84 suse running under vmware here.  Currently I have it running on:
windows, suse 9.1 x86, redhat 9.0 x86, Suse 8 ES AMD 64 (Ha! a 64-bit
platform, with minor failures in the unittests), Sparc Solaris 9 with
more failures, and MacOSx 10.2 with even more failures.  I need help for
more platforms...

> ctypes is not designed for speed, I'm not sure exactly what the
> overhead is, but I guarantee it's more than Pyrex.
>
> It takes more time to wrap a C API with ctypes than with Pyrex because
> you can't leverage anything that the C compiler and preprocessor knows
> about.

Yes, but there's a Tools/Scripts/h2py in Python which converts C header
files into python modules.
And, on the other hand, to create a Windows messagebox, for example,
it doesn't take more than to write
   ctypes.windll.user32.MessageBoxA(0, "Hello World", "ctypes", 0)


> It's not possible to poke back inside the Python C API with ctypes at
> this time (since it releases the GIL and has no GIL-holding call
> facility, though it wouldn't be too hard to make one).

I never had a need for this, but you're right, it would be easy to add.

David McNab <david at rebirthing.co.nz> writes:

> Bob Ippolito wrote:
>> It takes more time to wrap a C API with ctypes than with Pyrex
>> because you can't leverage anything that the C compiler and
>> preprocessor knows about.
>
> That one does ring my alarm bells somewhat.
>
> Pyrex's hack of allowing #define'd literals to be made available via
> 'ctypedef enum' allows the tokens to be used symbolically, where with
> ctypes one has to specify the values as well, which makes ctypes
> wrappers brittle against future api versions.
>
> It's a shame that Pyrex can't preprocess .h files and expose their
> contents into the wrapper module's namespace. There's still a
> brittleness wrt struct definitions.

Maybe h2py could also come handy here?

Thomas