[Pyrex] Pyrex and char *x[]

[Georg Grabler]

Hello, first of all - thank youf or this very detailed answer on my
question. It helped me out a little, but i now want to give code
examples. Since it's oss anyway (currently trying a python binding for
libalpm, the arch linux package manager, with a own API interface, not
reflecting the crappy original one .. and the original one is really
something to hate).

To the garbage collection issue: It makes me think a little about my
garbage collection at all. My current code structure works fine, don't
really want to run into troubles with this, so i wanted to ask
further, and i don't expect the garbage collection running while i'm
just running test scripts.

The current addList function defines as follows (different name ofc):
def addIgnorePkg(self, char *in_pkgname):
    cdef t_config.pyalpm_list *ptr

    if self.IgnorePkg != NULL:
      ptr = self.IgnorePkg
      while ptr.next != NULL:
        ptr = ptr.next
      ptr.next = <t_config.pyalpm_list *> malloc(sizeof(pyalpm_list))
      ptr.next.last = ptr
      ptr = ptr.next
      ptr.next = NULL
      ptr.pkgname = in_pkgname
    else:
      self.IgnorePkg = <t_config.pyalpm_list *> malloc(sizeof(pyalpm_list))
      self.IgnorePkg.pkgname = in_pkgname
      self.IgnorePkg.next = NULL
      self.IgnorePkg.last = NULL
    return

  def addIgnorePkgList(self, ignorelist):
    cdef char *cString
    cdef char *copy
    cdef int length

    for pyString in ignorelist:
        self.addIgnorePkg(pyString)
    return

  def remIgnorePkg(self, char *in_pkgname):
    cdef t_config.pyalpm_list *ptr

    if self.IgnorePkg == NULL:
      return 1

Accordingly, on destruction time the list self.IgnorePkg is iterated,
and the objects destroyed, by using
def remIgnorePkg(self, char *in_pkgname):
    cdef t_config.pyalpm_list *ptr

    if self.IgnorePkg == NULL:
      return 1

    ptr = self.IgnorePkg
    while ptr != NULL:
      if ptr.pkgname == in_pkgname:
        if (ptr.next != NULL):
          ptr.next.last = ptr.last
        if (ptr.last != NULL):
          ptr.last.next = ptr.next

        if self.IgnorePkg == ptr and self.IgnorePkg.next != NULL:
          self.IgnorePkg = self.IgnorePkg.next
          self.IgnorePkg.last = NULL

        ptr.next = NULL
        ptr.last = NULL
        free(ptr)
        if ptr == self.IgnorePkg:
          self.IgnorePkg = NULL
        ptr = NULL
        return 0
      ptr = ptr.next
    return 1


Since i'm experiencing problems with the memcpy, i didn't use it. Does
this mean, that i actually could come into troubles for python
releasing my objects in the structure? Doesn't pyrex tell python that
the object is still in use?

To be true, the malloc:
I used the following structure you wrote, which gave me the error that
the void* can't be converted into a char*
copy = memcpy(copy, cString, length)
So i casted the memcpy result
copy = <char*> memcpy(copy, cString, length)

resultin in
new1 ���
new2 ���
new3 ���
new4 ���

what's quite strange. So well, it did not work out, since the values
were new1, new2, new3, new4. After the copy, by printing the values, i
got the following (even more strange output)
oew4 ���
stdout
stdout
stdout

By using the structure above, it works flawlessly.
I've attached the current config.pyx file to the e-mail, sometimes
it's easier to see the whole thing.

Thank you,
Georg

On 5/16/07, Matt Hammond <matt.hammond at rd.bbc.co.uk> wrote:
> On Wed, 16 May 2007 06:29:41 +0100, Georg Grabler <ggrabler at gmail.com>
> wrote:
>
> > Hello everybody.
> >
> > I want an array to be passed to a function, so basically i started the
> > function as follows:
> >
> > def addToList (self, char *array[]):
> > ....
> >
> > This throws an error compiling:
> > "Cannot convert Python object argument to type 'char(*(*))'"
>
> Functions and classes defined python style (without a "cdef" prefix) are
> made available to python and therefore can only deal with python objects -
> which c-style arrays are not. Similarly c-style strings are not the same
> as python strings.
>
> Could you give a simple example of how you would use addToList from
> python? Would you be passing it something like a list or tuple, or using a
> library defined specialist array type?
>
> Assuming you'd be passing it a list/tuple, eg:
>
>      items=["hello","doctor","yesterday","tomorrow","continue"]
>      X.addToList(items)
>
> There is also the complication, since you're dealing with strings, of
> handling garbage collection issues: if you don't copy the string data out
> of the python string object, you need to inform python that you're still
> using that object, otherwise it may get garbage collected and the storage
> containing the string might be re-used!
>
> Of course, the simplest, though not necessarily most efficient, solution
> is to copy the string data out of the string into a newly allocated
> character array.
>
> If you want to declare c datatypes, or functions with c datatype
> arguments, then they need to be prefixed with cdef; and they will not be
> accessible from python (the specific variables, functions or classes
> declared with cdef)
>
>
> Here's roughly what I end up doing (in the .pyx file):
>
>
>      cdef extern from "string.h":
>          cdef void *memcpy(void *, void *, int)
>
>
>      class MyClass:
>
>          def addToList(self, listitems):
>              cdef char *cString
>              cdef char *copy
>              cdef int length
>
>              for pyString in listitems:
>                  cString = pyString        # pyrex auto converts
>                  length = len(pyString)
>
>                  # now make our own copy of the string data
>                  copy = memcpy(copy, cString, length)
>
>                  self.addList(copy)
>
>
>          cdef addList(self, char *str):
>              # your code
>
> Caveat: I've not tested this code above; but as I said, I've found this
> general approach works for me.
>
> Remember python string's don't use null termination conventions, so if
> you're retrieving them from your list for later use, you probably also
> need to store the length of the string somewhere too. Hope that's not
> teaching you to suck eggs :-)
>
>
> Hope this helps
>
>
> Matt
> --
> | Matt Hammond
> | Research Engineer, FM&T, BBC, Kingswood Warren, Tadworth, Surrey, UK
> | http://kamaelia.sf.net/
> | http://www.bbc.co.uk/rd/
>
> _______________________________________________
> Pyrex mailing list
> Pyrex at lists.copyleft.no
> http://lists.copyleft.no/mailman/listinfo/pyrex
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: config.pyx
Type: text/x-csrc
Size: 1892 bytes
Desc: not available
Url : http://lists.copyleft.no/pipermail/pyrex/attachments/20070516/14b0fcf9/attachment.bin