[Pyrex] Question, plus suggestions for improvement?

[Bill Mill]

Hello,

If you're only interested in helping to make a cool algorithm go
faster, skip to the next section. First, though, I have a question
about why pyrex acts the way it does:

=== Question: why does int work here (should be object?) ===

I wanted to create a pyrex script to permute python lists, and I was
pleased to find that it was fantastically easy. I wrote a little
iterator (see it at http://www.bigbold.com/snippets/posts/show/2351 ),
and it works beautifully. However, looking at the code, I thought it
would only work on integers. For an example of why, here's a short
snippet:

cdef int ... x, y, z
...
x = self.lst[-3]

I thought, on inspection, that x, y, and z would need to be declared
as objects to accept arbitrary values from a list. However, if I pass
in a list of strings or floats, everything works just fine. My
question is: why? Does x, as an int, just blindly accept a pointer to
the variable pointed to by self.lst[-3]? I've tried reading the
source, but I've come up confused.

Here's the C source for x = self.lst[-3]:

     __pyx_3 = PyInt_FromLong((-3)); if (!__pyx_3) {__pyx_filename =
__pyx_f[0]; __pyx_lineno = 69; goto __pyx_L1;}
     __pyx_2 = PyObject_GetItem(((struct
__pyx_obj_10permutagen_Permute2 *)__pyx_v_self)->lst, __pyx_3); if
(!__pyx_2) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 69; goto
__pyx_L1;}
     Py_DECREF(__pyx_3); __pyx_3 = 0;
     __pyx_1 = PyInt_AsLong(__pyx_2); if (PyErr_Occurred())
{__pyx_filename = __pyx_f[0]; __pyx_lineno = 69; goto __pyx_L1;}
     Py_DECREF(__pyx_2); __pyx_2 = 0;
     __pyx_v_x = __pyx_1;

So, as I read it, __pyx_3 is the index variable, and __pyx_2 is the
item from the list. __pyx_1, however, attemps to cast __pyx_2 into a
PyIntObject. If pyx_2 is a string, why should this work?

=== Optimization ===

The source for my permutation iterator is at
http://www.bigbold.com/snippets/posts/show/2351 . It already runs at
about the same speed as the probstat
(http://probstat.sourceforge.net/) permutation function, but I think
that's all because of algorithmic superiority. My code uses Knuth's
answer to one of the exercises in fascicle 2b
(http://www-cs-faculty.stanford.edu/~knuth/fasc2b.ps.gz), which is all
about permutations. Does anybody have any idea how I could optimize
it?

1) I see very few native C operations in the source code, just
eyeballing it. How could I restructure my code to bring more in?

2) Could I make list access faster? Optionally convert the list into
an array of (integers, characters, strings, etc) by a user's argument?

Anything other optimizations, I'm sure there are things I haven't
thought of, or didn't get from my brief reading of the manual?

Thanks,
Bill Mill
bill.mill at gmail.com