LateNightHacking Louis Journal

Auth

I just got home from the airport. There's a couple things I'd like to talk about, but sometimes it seems so time consuming just getting the thoughts typed out. It's easy to skimp on writing things down.

I've been thinging recently about symbolic manipulation. I'm intrigued with some ideas I've had about what might be possible. I've had some ideas about this ever since reading Godel, Esher, Bach, if not before. It seems like with a good symbolic manipulation engine, you could do all sorts of things, from parsing to compiling to revers engineering and more. Plus, anything you could do with an expert system. Prolog is an old language I'd like to learn, just to know it's advantages and disadvantages. You could do pattern matching - like regular expressions on steroids.The obvious problem is that of combinatorial explosion and memory consumption. I'm curious if you could use iterators and mementos to keep things managable. I know there's a lot of information out there already on the topic. As I said, regular expression engines, inference engines, and compilers are all relevant. Mathematica and its ilk are examples of powerful symbolic manipulation engines put to good use. I want to know how they work, and if my ideas have any merit. Are my ideas new? Or if not, how badly to they fail and why? What tricks do you need to know to write an effective rule base for an inference engine? I think with the right set up, you could run a distributed inference engine. Would that extend the class of problems that are amenable to this approach? Again, more things that are relevant to symbolic manipulation: searching (through a solution space) and associated heuristics (AI); bin packing, travelling salesman, and associated NP-completeness issues. I want to know what makes a regex engine tick and how to write a good one. I want to know how to do regex over a graph that's more complex than a linked list. So many fascinating aspects! And yet for the most part not relevant to anything I'm directly working on.

And yet I wonder. My current challenge is trying to figure out how to write a rule base for mapping between our different instrument services so we can keep them in sync. I've seen a few patterns that say "define a mapping", but I haven't seen any patterns on how to define an effective mapping. I definitely want to separate the mapping rules from the engine that applies them, because I want to be able to add new rules (new instrument types, or new business requirements) without having to write new code (or as little as possible). I want to encode the details of the mapping in a way thats flexible enough so that it can be used in multiple ways. For example, I want to be able to take an instrument from one system and create the equivalent in another. I also want to be able to take two representations of an instrument from two different systems and compare them, producing a difference report. I also want to be able to do those two things in both a "batch process" mode and in an event based (interactive) process. Presumably, this is all feasible, but I don't know what the data structure should look like. It would be nice if the mapping was reversible (the rules for going from A to B could be used to get from B to A) but I don't think that's feasible. The solution is still pretty vague in my head at the moment. My plan is to get some simple concrete examples and get them working. Part of the problem is that I'm looking at mapping on the order of one hundred fields, so it's easy to get distracted jumping from odd case to odd case. I've got to find simple concrete examples and stick to them!

The other problem that's reared its head is the problem of what I'll call the definite negative. It's easy to say that if A has p, then B should have p. But what does it mean if A doesn't have p? Does it mean that B should not have p either? I'm not sure. Not necessarily. Not mathematically. (Maybe I should call it the problem of the inverse.) Either I need a rule that says "And B should have nothing else," which will doubtless be too restrictive, or I need lots of little rules that say "if A does not have p, then B should not have p". Ok, having written it out, the anwer is pretty obvious. However, it's not obvious at first glance that you need the inverse rules, and worse, it's hard to figure out if the inverse rules are really true or not. Oh well, at least I know what to look for.

I ran in to this same problem when implementing subscriptions to sets. It's easy to generate a message saying which items are members of a set. On the other hand, you can't exactly generate a message saying all the items which are not members of a set! I ended up going with the first solution listed above, which basically means generating a message saying "And there are no other items in the set". Actually, I treat that as a flag I send with every message. That stupid flag has to get carefully propagated through the whole system for it to be valid, and yet I don't know that anyone has ever looked at it. It's probably just an annoyance I've inflicted on my fellow developers. Sometimes I wonder if the system would be better off without it, but then I'd have to live under the shadow of knowing that the system could potentially give me the wrong answer at any time and wondering when that rare event would strike and cause a catastrophe.

I'm reading Quicksilver by Neil Stephenson, and I definitely like it. Some people didn't like it because the story jumps around, but that doesn't bug me. Reading about smart people doing strange things is fun, and reminds me of my college days. :) And reading about dumb people doing strange things, and about spys, intrigues, and machinations ain't bad either. Here's today's great quote:

...For fame's a weed, but repute is a slow-growing oak, and all we can do during our lifetimes is to hop around like squirrels and plant acorns.
-- Waterhouse to Hooke, pg 798, Quicksilver, Neil Stephenson

Whee! There's a package here! That's one advantage to ordering stuff over the internet. You don't get things immediately, but you get to savor the anticipation of whatever you ordered. And it comes wrapped - you get to open it like a Christmas present!