[Pyrex] Should Pyrex be in standard Python?

[Paul Prescod]

Charles Hixson wrote:

> 
> Do you have an example?  I'm thinking of, perhaps, a simple gtk version 
> of "Hello, World!", with the string stored in a python string, and the 
> calls to gtk being executed directly from pyrex.

I don't know anything about GTK+. But here's an example of wrapping an 
XML parser:

http://www.prescod.net/python/pyrexopt/pyxpat/pyxpat.pyx

Maybe you could do the GTK+ version!

> It seems to me that such a thing would be a nice addition to the web page.

I agree.

> Also, do you have any relative timings for pyrex and C executed over the 
> exact same code (well, in translation of course but literal 
> translation).  I know that Python provides increased flexibility, but 
> how much do you need to pay over the areas where that isn't needed?

It's hard to say. If you don't use Pyrex's features the difference 
should be small. If you start taking advantage of them then you will get 
a complete range of performance figures from C to pure Python. It helps 
to look at the generated C code to recognize when Pyrex features are 
costing you (e.g. dynamic method calls, exception handling, reference 
counting).

So the real question is "how low can Pyrex go." My initial test is kind 
of unbelievable. Pyrex beats C (by a lot). In my second test, Pyrex is 
30% slower which is not bad considering that it is not really an 
optimizing compiler.

Test 1:

Maybe someone out there can help me understand. Am I doing something 
bone-headed that gives Pyrex a huge advantage??? (OS X 10.2, GCC 3.1)

bash-2.05a$ cat cmodule.c
#include <stdio.h>
#include <math.h>

static int fast_fib(int n,int a,int b){
     if(n==0){
         return b;
     }else{
         return fast_fib(n-1, b, a+b);
     }
}

static int fib(int n){
     return fast_fib(n, 1, 1);
}

void test_fib(){
         int i, j, num;
         long sum;

         sum = 0;
         num = pow(10, 8);

         for(j=0; j<num; j++){
                 for(i=0;i<5;i++){
                         sum += fib(i);
                 }
         }
         printf("%d:%ld", num, sum);
}

int main(void){
     test_fib();
     return 1;
}

bash-2.05a$ gcc cmodule.c
bash-2.05a$ time ./a.out
100000000:1900000000
real    2m15.950s
user    2m6.420s
sys     0m0.680s

================================================

bash-2.05a$ cat pyrexmodule.pyx

cdef extern int cfib(int n)

cdef int fib(int n):
         return fast_fib(n, 1, 1)

cdef int fast_fib(int n, int a, int b):
         if n==0:
                 return b
         else:
                 return fast_fib(n-1, b, a+b)

cdef test_fib():
         cdef int i, j, num
         cdef long sum

         num = 10**8
         sum = 0

         for j from 0 <= j < num:
                 for i from 0 <= i < 5:
                         sum = sum +  fib(i)

         print "%d:%d" % (num, sum)

test_fib()

bash-2.05a$ time python -c "import pyrexmodule"
running site customization
running build_ext
skipping 'pyrexmodule' extension (up-to-date)
100000000:1900000000

real    0m35.738s
user    0m32.720s
sys     0m0.340s

I don't frankly know how Pyrex can do in 35 seconds what idiomatic C 
does in 2 minutes. I can only imagine it is some misconfiguration or 
mistake. Let's see where Java comes in.

bash-2.05a$ time java fib
100000000:1900000000

real    0m52.334s
user    0m48.100s
sys     0m0.430s

Slower than Pyrex. (but faster than bizarro C)


2.

I re-did the test as a single Pyrex program calling out to a C function, 
C won:

C 27.9692320824
100000000:1900000000
Pyrex 35.3512070179
100000000:1900000000

So this way, Pyrex is less than 30% slower than C and Pyrex is not an 
optimizing compiler yet. No way to include Java in this test because it 
isn't directly callable from Pyrex. (Java code below)

import java.lang.Math;

public class fib{
	static public void main(String []args){
		test_fib();
	}

	static int fast_fib(int n,int a,int b){
	    if(n==0){
		return b;
	    }else{
		return fast_fib(n-1, b, a+b);
	    }
	}

	static int fib(int n){
	    return fast_fib(n, 1, 1);
	}

	static void test_fib(){
		int i, j, num;
		long sum;

		sum = 0;
		num = (int)Math.pow(10, 8);

		for(j=0; j<num; j++){
			for(i=0;i<5;i++){
				sum += fib(i);
			}
		}
		System.out.println(""+num+":"+sum);
	}
}

  Paul Prescod