[PP-main] Technical/implementation matters

[Joakim Ziegler]

On Fri, Mar 03, 2000 at 04:05:01PM +0000, Andrew Cooke wrote:
> At 08:38 AM 3/3/00 -0600, you wrote:
> >On Thu, Mar 02, 2000 at 06:43:57PM -0500, Rusty Foster wrote:

>>HTTP could work, but it's really sub-optimal for this sort of thing. If I had
>>to choose an existing protocol, I'd rather consider NNTP. But I don't see
>>what's wrong with brewing our own. Of course, I have an ulterior motive:
>>We're doing some software here that's specifically made for transferring
>>pre-parsed tree data (for instance XML) and knows how to take XML as input,
>>and output it on the other end, diff trees, and things like this. It's also
>>handy because it's extremely easy to use (it's a C library, with very
>>high-level functions). It also has crypted streams built in, without the need
>>for SSL libraries or anything of the sort.

>>I'm not a big fan of the trend lately to use HTTP for just about everything.
>>It's not what HTTP was made for, and it's showing.

>>My idea is to use the comm tool we're making at the bottom level, and then
>>supply some tools that use it, which let you do stuff like poll the server
>>for new messages, etc., command line. That's the most UNIXy and flexible way
>>of doing it, I think. Opinions?

> SSL is complicated, but it's well understood, has a good free
> implementation (OpenSSL) and it works.  No offence, but I would be very
> wary of a protocol designed by someone else that hadn't had the same level
> of public scrutiny.  SSL with certificates, a trusted CA, a good
> implementation and knowledgable users *guarantees* not just privacy, but
> that you are talking to who you expect.  The "if" list is long, but I don't
> know of anything that does better.

Certificates and a trusted CA are not really an option. The main reason for
this is that the existing CAs work on a very closed model, and even though I
think this is a good project for someone, somewhere, I don't think it's in
our interest to try to establish our own CA, with all the hassle that goes
along with that.

I agree that SSL is rather commonly used. However, the system we're making
works quite well, and is based on implementations that have been publicly
scrutinized for quite some time, and are also patent free, namely Twofish and
ElGamal, with the majority of code taken directly from GPG.

Take a look at our Flux library, http://projects.simplemente.net/flux/ if
you're interested.


> HTTP is everywhere.  You don't need to worry about firewalls - if anything
> goes through, you can bet that they let http through.  The main problems
> are a lack of state and that it's limited to a single request/response on a
> single open connection.  You can do better with persistent connections (but
> have to measure content length before sending data) and cookies etc help
> with state.  But if you stick to simple http you know that things will
> work.  Perhaps you are only going to be transferring data between hosts run
> by people who know what they are doing - in that case I guess it doesn't
> matter that you stay simple.  But have you considered that indviduals at
> work may want to use software that connects to the system you are designing
> to build, for example, personalized news.  Maybe that example is way
> off-line, but can you be *sure* that you aren't excluding many many users
> who may want to access this data some time in the future and find that only
> HTTP goes through firewalls.

HTTP can be nicely used as one option for an end-user protocol. Actually, it
will be, given that most users will access web sites, using standard
browsers. However, between sites, for the actual feed, it's not a good
option. The firewall argument doesn't really work here, since sites are not
likely to be behind a firewall whose config they can't change. Anyway, it's
easy to make the raw XML available over HTTP, that's really the beauty of it,
the content and the delivery method can be independent. But I don't think I'd
recommend using HTTP as the primary or preferred delivery method for the feed
to the client sites.

HTTP is non-trivial to use for pushing data (you'll need to run an HTTP
server on each client site), and the whole protocol is really skewed towards
what it was originally designed for, which is letting browsers get web
documents from a server. A very different task from what we're trying to do
here. As I said in an earlier mail, if we were to go with an existing
protocol, NNTP would probably be much better suited.


> One final comment - more personal opinion that the above - avoid new stuff
> as much as possible.  You are doing a lot of work.  Getting something
> working on existing protocols is probably quicker.  Yes, it's interesting
> designing better more efficient protocols, but no-one wants to document
> them and no-one understands them.  Efficiency is always over-rated.
> Optimize last.

We've already done the work, really. It's going into Conglomerate and FIDEL,
the two projects we're mainly working on at the company I work. I urge you to
take a look at the work we've been doing before judging it.


-- 
Joakim Ziegler - simplemente r&d director - joakim at simplemente.net
 FIX sysop - free software coder - FIDEL & Conglomerate developer
      http://www.avmaria.com/ - http://www.simplemente.net/