[Pyrex] Cython development

[Stefan Behnel]

Hi,

Arc Riley wrote:
>> I wrote a patch so that you can do that with Cython. You will just have to
>> care for the dependency tracking yourself, and provide your distutils
>> Extension with the complete list of sources (including .pxi and .pxd
>> files).
>> Cython (or rather distutils) will then build only those extensions where
>> any of the source files in that list has changed.
> 
> Thank you for this, but note that it's minor compared to the various
> functions which search for cimports and the like.  We've worked around what
> it builds a package as (I hope) using the  full_module_name argument to
> compile but this is only the beginning.

Fair enough.


> Setting the package directory '':'src' would, as I understand it, cause
> Cython to compile our extensions not as (ie) src.colors.soy.colors but as
> colors.soy.colors; closer, but still a no-go.

Ah, and I noticed that you do not seem to provide your fully qualified package
names as first parameter to Extension(). I guess, when I dig deeper, I would
find even more things like this. You really can't blame Cython if you provide
it with incomplete information all over the place.


> Honestly I think a distutils.extension.Extension parameter for psources=
> (etc), treating the Pyrex/Cython/etc sources in a style akin to compiling C
> sources, compiling each source to a separate .c file, and appending those
> generated .c files to sources= is a better direction to go

That would just unnecessarily complicate things IMHO. There's one parameter
for any kind of sources now, that should be enough.


>> Is there anything in addition to my patch that you need here?
>
> Immediately, the most direct and painless solution would be to handle
> cimport name searching by only it's file name, rather than it's directory
> path, to match Pyrex's behavior.

... or, you could provide a directory structure that actually matches your
package structure, rather than trying to push Cython into working on
inconsistent package directories (lacking the main package, for example), by
providing an incomplete view on the package structure. Fixing your package_dir
parameter and your Extension() name should get you pretty far, I assume.

Stefan