[Pyrex] Pyrex & Unicode

[Farzin Shakib]

I am really struggling with with Pyrex and unicode.

How do I pass a unicode string to a pyrex function and within pyrex
convert it to a char* so I can pass it to a C function, and vice versa?

Consider the following simple Pyrex code

cdef extern from "string.h":
int strlen( char* s )
cdef extern from "Python.h":
object PyString_FromStringAndSize( char* s, int len )
cdef extern from "util.h":
void utilPutString( char* file, char* text )
char* utilGetString( char* file )

def putStr( char* file, char* text ):
utilPutString( file, text )

def getStr( char* file ):
cdef char* text
cdef int len
text = utilGetString( file )
len = strlen( text )
retText = PyString_FromStringAndSize( text, len )
return( retText )

Now the following works

text = u'apple'
putStr( 'file.txt', text )
text2 = getStr( 'file.txt' )
if text == text2: print "PASSED"
else: print "FAILED"

as long as text is an ASCII. But if text is a real unicode, it does not;
such as:

text = u'??'
putStr( 'file.txt', text )
text2 = getStr( 'file.txt' )
if text == text2: print "PASSED"
else: print "FAILED"

I get

Traceback (most recent call last):
File "test.py", line 5, in ?
putStr( 'file.txt', text )
File "test.pyx", line 9, in test.putStr
utilPutString( file, text )
UnicodeEncodeError: 'ascii' codec can't encode characters in position 
0-1: ordinal not in range(128)


How do I modify the above pyrex to make this work?

Thanks
Farzin