[PP-main] Technical/implementation matters

[Hans Petter Jansson]

>> Can you write up a summary of the protocol you recommend, and post it,
>> with an orientation toward why it's good for this service? Perhaps point
>> us to the relevant docs as well (maybe you've already done this-- I
>> don't remember)

> Hans Petter is on the list, most likely he can give you a quick summary. The
> tools we aim to use are based on Flux, which is at
> http://projects.simplemente.net/flux/

Right.

Flux provides a hierarchical data model, which can embody any kind of
information with a tree structure. It does not enforce a specific semantic
format on top of this - it's like XML in this respect, but more minimal. For
instance, there is no separation of data and metadata (markup). Defining this
is up to the implementor. If you're familiar with LISP lists, you'll see
similarities to that.

The primary "storage format" of these trees is abstract - residing in main
memory - and defined by its API. The in-memory data can be associated with a
disk database which is used transparently without deviating from the API used
for in-memory trees. Kind of like a private swap partition, but the database
does not use its own partition and is persistent between program invocations.
[Note: The databasing is only in CVS so far, and not thoroughly tested.]

There are two exchange formats. One is that of printed trees - 7-bit ASCII
that looks like LISP lists with C braces. It's human-readable, and makes for
nice configuration files and such.

The second exchange format is for machine-to-machine streaming. It's 8-bit,
efficient and not human-readable. It works by sending one tree at a time, and
combined with the Flux proxy, lets you set function callbacks based on the
contents of the root node and an optional transaction value.

Flux also has a strong encryption layer that you can apply to this with close
to no extra work.

---

So, what you have is a layer that lets you work with a purely semantic data
representation, exchange it with users and remote programs and cache it
efficiently on disk (if you need to save main memory). I'm paying attention
to CPU and memory needs, so they remain small.

In the future we'll also see a layer similar to the disk database I
mentioned, which will provide network transparency - trees shared between
processes on different networked devices, working on them as if the whole
thing resided in local memory at any given time. This is mostly on the idea
stage still, but can benefit from the existing database code.

---

I've taken care of the low-level tasks for you. The semantics you have to
define yourselves. :)

Documentation is on the Flux pages, as Joakim pointed out. Look in the
"token trees" section. What's there is not complete, but it should give you
an idea.


--
coffeecocacolagravitywars - Hans Petter Jansson - amigadracoempirefluxstuff
toomuchsugarkingcrimsonemperorcrimzgoodbadandonedayitsallgonnaendupinflames