[Pyrex] array.array vs long ints for bit vectors

[Josiah Carlson]

Use an array + bit masking.  When using the array, use 16 bit unsigned
ints.  Why?  While 32 bit ints would be faster when running in pure
Python, you commonly end up with having to deal with the sign of your
ints, which is easy, but annoying.  On little-endian platforms, you even
get the convenience of getting consistant behavior regardless of whether
you want to deal with bytes, shorts, longs, or long longs when masking.
(use the buffer interface [1] to get the raw pointer).

Why wouldn't you want to use a long int instead of an array?  It's
immutable, and you have to deal with doing Python long-int math all the
time. With the array, all of your math can be in pure C, which will be
/far/ faster.

 - Josiah


[1]
cdef extern from "Python.h":
    int PyObject_AsCharBuffer(object obj, void **buffer, int *buffer_len)

cdef int get_buffer(obj, char **data, int *length) except -1:
    cdef int result
    cdef void *vd
    result = PyObject_AsCharBuffer(obj, &vd, length)
    data[0] = <char *>vd
    return result


Nitin Madnani <nmadnani at wam.umd.edu> wrote:
> Hi Guys
> 
> I am planning to use a bit vector as a member of an extension type. I  
> first thought that I can easily do this by just using a long int for  
> the vector but then I realized that I would like the bit vector to  
> grow only to the size needed (which would be defined by an argument  
> to the __init__ method of the type. So, then I remembered the 'array'  
> module but then this would also store each bit as a byte (according  
> to the python documentation). So, if I had a 20-bit map, 'array'  
> would take >20 bytes to store this bitmap (only the buffer would be  
> 20 bytes) ? I have also read that a long int also comes with a bit of  
> baggage.
> 
> So, is there no other memory-efficient option other than to write an  
> extension myself? However, I am guessing whatever I come up with  
> would be slower compared to the both long ints and arrays. So, should  
> I just bite the bullet and live with the wasted memory ?
> 
> 
> Thanks !
> Nitin
> 
> PS: I know this question is sort of out of place here but since I am  
> planning to use it in Pyrex, I wanted to ask it here. I also know  
> little about either long ints or the 'array' module, so please  
> correct me if I am being completely naive.
> 
> 
> _______________________________________________
> Pyrex mailing list
> Pyrex at lists.copyleft.no
> http://lists.copyleft.no/mailman/listinfo/pyrex