[Pyrex] Cython development (was: array setting and math function annoyance)

[Stefan Behnel]

Hi,

Arc Riley wrote:
>> It's always better to improve things than to branch for trivial changes
> 
> I agree.  It's troubling to see this proliferation of project-specific Pyrex
> branches each with only minor changes.
> 
> In contrast, however, we've been at the mercy of Greg to decide when to
> accept patches, his available time to rewrite them, then later release, and
> finally for a new release to be packaged by distros.  I understand this is a
> frustration you guys had too, behind your reason to form Cython, but if we
> used Cython we would be in the same situation with you than we are with
> Pyrex.

I see the situation a bit different. I think we have learned from the problems
with Pyrex, which were the reason why so many people started their own Pyrex
branch somewhere.

Cython is a very open project, and we really try to attract people who want to
contribute, instead of creating new branches for no good reason.

Cython has a much (much!) faster release cycle than Pyrex, and it has an open
development repository (Pyrex also has one by now, BTW). So you can use
developer versions with the fixes/features you need even before a new release
comes out.

http://hg.cython.org/


> We're not interested in the language pre-processor politics, we've stayed
> out of them so far, but given all our logistics we really need greater
> control over that part of our own project.

No idea what you mean here. If you are referring to the recent discussions on
the mailing list, then I would say that patches will be reviewed and good
ideas will eventually be implemented depending on our resources, but big
changes take time, so don't "fear" them by tomorrow. In any case, we do care
about not breaking existing code without very good reason, so I would say
you're on the safe side there.


>> You can't just ignore the most obvious way to build a C extension for Python
>> (i.e. using distutils), and then come and complain that your hand-written
>> setup doesn't work.
> 
> Oh, we're using Python's distutils to build the C source.
> 
> That 100 lines in our setup.py is a workaround for Pyrex.Distutils failure
> of detecting source changes and rebuilding .c files only when necessary.
> That part would be better generalized and in Distutils rather than in our
> setup.py since any project could take advantage of it

I wrote a patch so that you can do that with Cython. You will just have to
care for the dependency tracking yourself, and provide your distutils
Extension with the complete list of sources (including .pxi and .pxd files).
Cython (or rather distutils) will then build only those extensions where any
of the source files in that list has changed.


>> improve Cython's distutils helper (build_ext.py) to handle the problem of
>> calculating dependencies better.
> 
> That's what we're looking at, yes.
> 
> I think the most pragmatic solution for us right now is a branch, given our
> release time table and Cython's level of recent commit activity

is that good or bad?


> and that these changes will undoubtedly require discussion.
> We'll of course provide patches.

Is there anything in addition to my patch that you need here?

Stefan