[Pyrex] Can pyrex codes call python objects/methods such as String, re, datetime, os diretively

[Robert Bradshaw]

Doing a list with cdef ints is the fastest way to do loops. Iterators  
will not gain you any speed (in fact they'll probably be a bit slower).

> 	iter = x.__iter__()         # get iterator
> 	try:
> 	    while 1:
> 	        y = iter.next()         # get each item

the underlying implementation as

         for y in x:
             ....

except the former does a dictionary lookup and uses python  
conventions to call "next" which is slower. The advantages of  
iterators are primarily convenience and memory.

As for your question of indexing into lists, using the python/C api  
directly is faster by at least a factor of 7 to 8. (In SageX we  
dynamically check at runtime to determine if the element being  
indexed is a list/tuple and use the direct access functions rather  
than the generic __getitem__ that x[i] in generic Pyrex produces.  
This gives us over a 7-fold speedup with only about a 1-2%  
performance hit for generic objects).

As others have mentioned, probably the only way you'll see  
significant gains is by accessing the oracle database more directly,  
as I am almost certain most of the time is spent allocating/ 
deallocating python objects for the query results. It's always a good  
idea to profile to be sure.

- Robert


On Jul 2, 2007, at 5:43 PM, liangxing wrote:

> Thanks for your hands.
>
> I was looking for one alternative to get access to my oracle  
> database. But however, I fail to find any thing better than  
> cx_Oracle in terms of supporting my needs.
>
> By the way, do anyone think it is an abstacle of looping timing.  
> That  is,  should I replace current 'loop structure' with some  
> stuff else.
>
> Once I thought if I choose the __iter__ attributes, and using  
> xx.next() to implement the function.
> Is it a good idea or even worse?
> as:
> 	iter = x.__iter__()         # get iterator
> 	try:
> 	    while 1:
> 	        y = iter.next()         # get each item
>
> -----原始邮件-----
> 发件人: Greg Ewing [mailto:greg.ewing at canterbury.ac.nz]
> 发送时间: 2007年7月3日 6:27
> 收件人: liangxing
> 抄送: pyrex at lists.copyleft.no
> 主题: Re: [Pyrex] Can pyrex codes call python objects/methods such  
> as String, re, datetime, os diretively
>
>
> liangxing wrote:
>> My case is retrieveling all record items from a Oracle Database,
>> within a loop execution. By calling cursor.fetchmany() multi-times,
>> I found it is quite slow.
>
> Since the Oracle interface is probably returning an
> ordinary Python list of tuples, there's not much you
> can do to speed it up.
>
> To get a substantial gain using Pyrex, you would
> need to bypass the Python interface and use whatever
> C interface Oracle provides directly.
>
> -- 
> Greg Ewing, Computer Science Dept,  
> +--------------------------------------+
> University of Canterbury,	   | Carpe post meridiem!          	  |
> Christchurch, New Zealand	   | (I'm not a morning person.)          |
> greg.ewing at canterbury.ac.nz	    
> +--------------------------------------+
> _______________________________________________
> Pyrex mailing list
> Pyrex at lists.copyleft.no
> http://lists.copyleft.no/mailman/listinfo/pyrex