[Pyrex] array setting and math function annoyance

[Stefan Behnel]

Hi,

Arc Riley wrote:
> I ask because we arrived at the conclusion that it'd be much easier to
> branch Cython with the changes we need rather than inherit and override.

It's always better to improve things than to branch for trivial changes,
except when the branch will become a fork with different goals (as with Pyrex
and Cython). If your goal is to benefit from Cython and the only problem is
some build problems in your custom local setup, than a branch is the wrong
solution.


> Whether you guys plan Cython to continue insisting projects using it to have
> a similar structure as Sage directly impacts this discussion.

Sage is not the only project that uses (or drives) Cython, and I seriously
doubt that many projects have "a similar structure as Sage", except to the
extent where extensions in Sage have a similar structure as the ordinary
Python extension project.


>> Depends on how you do it. Do you use distutils or call Cython by hand?
> 
> We're not using Cython right now, until a workaround or alternative is setup
> it's not an option.
> 
> We are currently "from Pyrex.Compiler import Main" which runs through
> roughly 100 lines of code to detect whether any source files have been
> modified since the .c file for it was last generated (and regenerating
> dependencies when a .pxd is modified), along with some OS-specific code
> handling library name conversions and OSX frameworks.

You can't just ignore the most obvious way to build a C extension for Python
(i.e. using distutils), and then come and complain that your hand-written
setup doesn't work. That's like complaining to gcc that their new C99
compliant version doesn't build your K&R C code, while refusing to include the
right compatibility header file.

Look at the way the distutils part of Cython handles this case
(Cython/Distutils/build_ext.py), that should get you going.


> If
> I had my way, the Distutils would handle these source files the same way it
> handles .c source files - specifying which sources need to be linked each
> extension.  Our current method of .pyx files filled with include statements
> for .pxi files in the same directory is a workaround for Pyrex's lack of
> this functionality.

Then I would suggest you a) start using distutils instead of replicating
functionality that is already there, well maintained and widely used, and b)
improve Cython's distutils helper (build_ext.py) to handle the problem of
calculating dependencies better.

Now that we touched this problem: I'd suggest allowing to add .pxd and .pxi
files as Extension source files in distutils, and just strip them out when
building the list of .c sources in build_ext.py. I expect that to be enough
for distutils to determine the right dependencies.

Please provide a patch that relieves you from your problems.

Stefan