[Pyrex] Python Strings and PyMem_Malloc/Free

[Grant McDonald]

Hi,

I fear I may have been operating under a false assumption with regards to
creating Python strings. This assumption goes like this:

1) Whatever memory you PyMem_Malloc you should always PyMem_Free.
2) For strings you malloc and need to turn into python strings assign them
to a python variable in Pyrex then PyMem_Free the pointer to the string
before you return the python object.
3) This shouldn't present a problem because PyString_FromString copies the
string to an internal representation (this is my assumption).

I believe the last statement may be completely erroneous :). Here is some
code based upon this assumption:

cdef class Interface:
    cdef TRK_HANDLE trkHandle
    cdef TRK_ASSOC_HANDLE trkAssociationHandle
    cdef TRK_RECORD_HANDLE trkRecordHandle
    cdef TRK_NOTE_HANDLE trkNoteHandle
    cdef TRK_ATTFILE_HANDLE trkAttFileHandle
    cdef TRK_IMPORT_HANDLE trkImportHandle
    cdef TRK_EXPORT_HANDLE trkExportHandle

    .
    .
    .

    def GetNote(self):
        cdef TRK_UINT bufferSize, remaining, max, titleSize, result
        cdef LPSTR buffer, title, temp_buffer
        
        result = TrkGetNoteDataLength(self.trkNoteHandle, &bufferSize)
        if result != TRK_SUCCESS:
            raise ExtensionExceptions.TrackerError, result
            
        buffer = <LPSTR>PyMem_Malloc(sizeof(char)*bufferSize)
        if buffer == NULL:
            raise Exception, "Could not malloc %d bytes!!" % bufferSize
        temp_buffer = <LPSTR>PyMem_Malloc(sizeof(char)*bufferSize)
        if temp_buffer == NULL:
            raise Exception, "Could not malloc %d bytes!!" % bufferSize
        memset(buffer, 0x00, bufferSize)
        memset(temp_buffer, 0x00, bufferSize)
            
        if bufferSize > 10000:
            max = 10000
        else:
            max = bufferSize
            
        remaining = bufferSize
        pyNote = ''
            
        while remaining != 0:
            result = TrkGetNoteData(self.trkNoteHandle, max, temp_buffer,
&remaining)
            if result != TRK_SUCCESS and result != TRK_E_DATA_TRUNCATED:
                PyMem_Free(buffer)
                PyMem_Free(temp_buffer)
                raise ExtensionExceptions.TrackerError, result
            strcat(buffer, temp_buffer)
            memset(temp_buffer, 0x00, bufferSize)
        
        titleSize = 200
        
        title = <LPSTR>PyMem_Malloc(sizeof(char)*titleSize)
        if title == NULL:
            raise Exception, "Could not malloc %d bytes!!" % titleSize
        memset(title, 0x00, titleSize)
        
        result = TrkGetNoteTitle(self.trkNoteHandle, titleSize, title)
        if result != TRK_SUCCESS:
            PyMem_Free(buffer)
            PyMem_Free(temp_buffer)
            PyMem_Free(title)
            raise ExtensionExceptions.TrackerError, result
        
        pyTitle = title
        pyNote = buffer
        
        # it is this code that appears to be problematic
        PyMem_Free(buffer)
        PyMem_Free(temp_buffer)
        PyMem_Free(title)
        
        return (pyTitle,pyNote)

As the code stands calling this function _sometimes_ crashes the Python
interpreter (usually the 3rd or 5th call) with an access violation (mem
address 0x0000000 which would imply a null pointer reference). If I comment
out the final three calls to PyMem_Free then the problem does not arise.

If my assumption is wrong does the same hold true for structures? I have
similar malloc/free calls for structures which save data from them and then
free the structure memory before returning.

Any information is appreciated,

Grant M.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.copyleft.no/pipermail/pyrex/attachments/20050630/015bf551/attachment.html