[Pyrex] Dangerous weirdness happening

[David McNab]

Hi,

I'm wanting to distribute a python app with pre-built extension modules,
to spare people (particularly windows users, and non-python programmers)
the hassle of having to install Pyrex and build these modules.

With my app, I'm stipulating 'python 2.2 or later', and packaging it
with my pyrex extensions built under python 2.2.

My code suppresses the RuntimeWarning when loading these modules into
python 2.3 or later

In most ways, the 2.2-built pyrex module loads fine into 2.3 and works.
As do other 2.2-build extensions (eg Metakit).

But I'm seeing a worrying side-effect with Pyrex modules - anything
which causes .__str__() or .__repr__() to be called on a dict or list
causes the following exception:

SystemError: ../Objects/dictobject.c:515: bad argument to internal
function

To reproduce this:
- please download my SSLCrypto module from
   http://www.freenet.org.nz/python/SSLCrypto - the source tarball.
- download openssl-dev (if you don't already have it).
- Build it under python2.2 (type 'python2.2 setup.py build')
- cd into build/lib.linux-i*86-2.2
- type 'python2.3'
- type '[]', 'dir([])' or 'repr([])', and notice normal output
- type 'import SSLCrypto'
- type '[]', 'dir([])' or 'repr([])', and notice the exception.

I can't see anything in my code which would cause this, since there is
no 'mainline' execution - just a few imports and class defs.

This exception feels a bit disturbing, to say the least.

What I'm wanting is for non-python people to be able to download and
install my app, without having the complication of Pyrex, different
python versions, building etc shoved in their faces.

One approach I could use is to rip the python version from the front of
sys.version, and use that for version-specific pyrex extension imports. 

For example, instead of doing:

   import SSLCrypto

I could do:

  import sys, re

  def my_import(name):
      mod = __import__(name)
      components = name.split('.')
      for comp in components[1:]:
          mod = getattr(mod, comp)
      return mod

  def importMyExt(name):
      versStr = sys.version.split(" ", 1)[0]
      versDigits = re.findall("([0-9]+)[.]([0-9]+).*", versStr)[0]
      versDir = "_".join(versDigits)
      mod = "myext.v" + versDir + "." + name
      return my_import(mod)

That way, I'd just need a directory 'myext', with subdirectories 'v2_2',
'v2_3' etc, each of which contain the version-specific builds of my
extensions.

BTW - the above code works.

But, it's **SUCH** a pain.

Can anyone care to comment on whether it's possible to import pyrex
modules built for earlier pythons, without the side-effects?

Or, how can I accommodate multiple Python versions in my app, without
rubbing my users' noses in intricacies they don't want to know about?

Should I bundle a copy of Pyrex and the .pyx files for my extensions,
and have my install script do:
* unpack and install pyrex
* build and install all my extensions
?

Sorry I've gone on a bit here. I do look forward to some enlightenment
on this issue.

-- 
Cheers
David