[Pyrex] Adding custom statements to pyrex ? (or passing through macros to C files)

[Michael Sparks]

Hi,


This is a question about which bits of pyrex I would have to touch in
order to allow Duff's device type tricks in Pyrex. I've been playing with
duff's device recently to allow translation of some python generators to C
& C++ code, but from my perspective it'd be nice if I could hack support
into Pyrex for this.

The approach I've taken for C is as follows:
   * For each generator create a struct, a struct creator/initialiser and
     a "next" function. (the struct holds function state, the next
     function implements the logic)

In C I have to pass through the structure (obviously) as a first
parameter to the next function, whereas with C++ I can just let C++ use
an implicit this pointer.

To make the C code look clearer, I define a small bunch of macros for each
bit of Duff's device. Making this a bit more concrete, the macros look
like this:
[NB, no error checking in all that follows!]

#define Resume switch (this->state) { default: this->state=__LINE__;
#define Yield(x) this->state=__LINE__ ; return (x); case __LINE__ :
#define Endresume this->state=__LINE__; case __LINE__: break; }

The actual next method (which maps directly to the generator), looks like
this:

int Duffs_next(Duffs * this) {
   Resume;
      for(this->i=0; this->i<10; this->i++) {
          Yield(0);
          printf("%s: %d\n", this->tag, this->i);
      }
   Endresume ;
}

The structure for capturing local state looks like this:
typedef struct Duffs {
   int state;
   int i;
   char * tag;
} Duffs;

The initialiser looks like this:
Duffs * Duffs_init(char *tag) {
   Duffs * d;
   d = (Duffs *)malloc(sizeof(Duffs));
   d->state=0;
   d->tag = tag; /* Nasty due to aliasing */
   return d;
}

Sample usage then looks like this:

int main(int argc, char *argv[]) {
   Duffs * G;
   Duffs * H;
   int i;

   G=Duffs_init("G");
   H=Duffs_init("H");
   for(i=0; i<14; i++) {
      Duffs_next(G);
      if(i>0) { Duffs_next(H); }
   }
   printf("CORO.C\n");
   return 0;
}

Which whilst verbose does have a number of advantages (IMO over taking a
state machine approach.

Essentially I'd like to be able to do the same thing in Pyrex. There's two
possible routes AFAICT:
   1 Simply make it so that macros like the 3 presented up top can be
     passed through by pyrex to the underlying compiler without major
     issue. (Perhaps something like 'cmacrodef NAME' to allow pass
     through.) This would involve boilerplating some of these things,
     but I can live with that.

   2 The alternative is to modify Pyrex to generate the boiler plate
     itself when it sees a yield statement. This is probably a lot more
     work, but perhaps a more satisying approach. (Macro passthrough
     whilst more flexible probably opens pyrex up to all sorts of nasty
     abuses... :-/ )

I suspect 1 is a lot simpler than 2, but lack of familiarity with the
internals of Pyrex means I'm not sure where to start with this. I'm not
expecting anyone else to do the work, but would appreciate any pointers
from anyone who has hacked pyrex's internals as to which bits I should
look at/where to start. (I'm currently going through the code
familiarising myself with the structure :)

If I'm completely missing something, that'd be nice to hear too :)

Thanks in advance for any comments :)


Michael.