Emacs of the Web

Bill de hÓra contributes to the "one click subscription" discussion, or what he dubbed "clicksub"...

I don't use browsers much anymore and will be using them even less next year. Aggregators are so much better than browsers for following content. Really, if you have to read stuff on the web and are using a browser for that, you should try an aggregator. And then, what's the browser good for?...

To be honest, next year's browsers need to be aggregators, else I don't see the point in using them...

Instead this would be great: at some point weblogs flip over and the HTML website bits will become secondary fluff to the XML content, like how PDFs are secondary web fluff to HTML today. The frontpage would be the feed, the archives would be Atom entries...

Honestly, permalinking to a html file is starting to look more and more like a bug. Why not point to the XML entries?

I was going to blah blah about some analogies with Emacs. Hard-core emacs users find themselves snickering about artificial boundaries around "applications" -- there is very little concept in the world of Emacs that "this thing is a word processor" but "that thing is an email reader" and "some other thing is a directory explorer". And the list goes on for outliners, class browsers, PIMs, etc.

The underlying problem may be there is no (not yet) Emacs of the Web. But Bill what I get from Bill's post is my Emacs perspective is mostly a "front-end perspective". There is a whole back end perspective on the future of feeds, pages, formats that is in its infancy.

Emacs was born as a set of flexible front-end parts in the 1970's. Even in the world of objects and MVC, a significant number of programmers were just starting to grasp the same concepts in the late 1980's and early 1990's. By and large, looking at our software landscape, we still have not grasped the concept. In fact, the browser may be the closest thing we have to this on a widespread basis!

Not only does Bill show this is not seriously the case, by now we really should have moved on to the back end of resources, formats, and hyperlinks. The head spins. We have a lot of catching up to do.

When to create syntax in Lisp?

From the gambit email list, I respond to this question about writing macros...

In your opinion, is it appropriate to use a macro to abstract away repetitive boiler-plate code? Or is this better done in a procedure?

This is almost always a procedural abstraction rather than syntax, especially for beginners with Lisp... better to spend a lot of time with just procedural abstraction, higher-order functions, etc.

Syntactical abstraction I use for controlling the order of evaluation and sometimes to put a pretty syntax around use of lambda.

As an example an old Lisp control structure is called PROG1. This structure takes a sequence of statements, evaluates each in order, and returns the result of the first statement after the last has been evaluated. This is can be expressed as a Gambit macro, but I'll call it begin1 to be more like Scheme's begin than the old Lisp's progn.

? (define-macro (begin1 first-statement . remaining-statements)
    (let ((result-var (gensym 'first-result)))
      `(let ((,result-var ,first-statement))
         , at remaining-statements
         ,result-var)))
? (begin1 1 2 3)
1
> (begin 1 2 3)
3
? (define result-var 5)
? (begin 1 2 result-var)
5
? (begin result-var 1 2 3)
3
? (begin1 result-var 1 2 3)
5
? (begin1 0 (display result-var) (newline) 1 2 3)
5
0

The begin1 is an example of control abstraction.

An example of the latter use of pretty syntax... consider a scenario where you are using a resource and you want some "before" and "after" actions. The base level way to implement this is with higher-order procedures. But who wants to write (lambda () ...) all the time? So on top of this build some pretty syntax. [Note that a better lambda notation like Smalltalk's block syntax would reduce the need for these situations. (pdf)] For example...

(define (call-when-ready procedure)
  (wait-until-ready time-out)
  (if (not (ready?))
      (call-when-ready procedure)))

Use it like this...

(call-when-ready 
  (lambda () 
    (display "I am glad this is finally ready!")
    (newline)
    (do-something)))

This is fine when someone else is generating the code for you. Normally you might want to abstract the procedure as a sequence of statements...

(when-ready
  (display "I am glad this is finally ready!")
  (newline)
  (do-something))

And so when-ready is defined as a macro that calls call-when-ready...

(define-macro (when-ready . body)
  `(call-when-ready
     (lambda ()
       ,@body)))

Will Farr added a neat example in the same thread...

Making little sub-languages for specialized processing e.g.

(with-vectors (v1 v2 v3) (v1 <- (+ v2 v3)))

for summing up the vectors v2 and v3 and storing it in v1. (I'm not going to put this macro up because it's long---the code is buried in this post: http://wmfarr.blogspot.com/2005/06/bigloo-macros-and-repl.html .)

[This] involves changing the evaluation rules for a piece of code (the vector assignment is evaluated once for each index with the corresponding variables bound to elements of a vector).Procedures would not work for either of them; you have to have a macro.

Because

Because the world is round...

Identities, Associations, and Behaviors

Doc Searls has the lead article in the September Linux Journal on "Identity". This is yet another topic I have barely understood. My interest is growing though with the concept of "user centric identity".

The title of Searl's article is "Independent Identity" which points out an irony of sorts that's been on my mind. My impression is that this is not about "identity" at all. Rather it is about "association".

What use is an independent identity? I have to associate my identity with some other body's identity to accomplish anything. The crux of the "user centric identity" effort is how to limit the ramifications of that association.

The other thought that's been brewing is the relationship between capabilities and identity. We know our current approach to permissions is not sufficient to limit or even audit the results of association. On the other hand if I can reliably limit the available behaviors of my associates, and they mine, then we can mutually benefit from our association.

Identity without behavior does not lead to associations. Since we are after beneficial associations, I'm assuming we need to focus on how "identity management" ultimately translates into appropriate behaviors and reliably excludes inappropriate behaviors. A capability system requires identity management to communicate with the outside word. I also assume an identity management system requires capabilities or something equally secure to ultimately implement trustworthy associations.

I'm looking forward to Phil Windley's new book on Digital Identity to get a better handle on some of these ideas.

DZ

and now, daily zen...

By sitting alone all day long
I clear my mind of a thousand thoughts.
To speak of this is beyond our words;
Only by sitting under the quiet forest
Can we ever understand.

- Fa Yen (885-958)

uuid

Termite currently has a dependency on libuuid to generate universally unique identifiers. Since there is no need for the result to conform to the uuid_t data type, just that the result is unique across space and time, I wrote a uuid generator in Scheme which will make running Termite easier on more platforms. I loosely followed the pattern for generating primary keys in EJB Design Patterns which does not require a singleton or a database, etc. i.e. multiple instances of this uuid maker can exist in the same address space or in different address spaces, and on different hosts. I had fun replacing the absolutely horrible C code from libuuid with short, readable, Scheme.

> (load "my-uuid.scm")
"/home/pdlogan/dev/termite/my-uuid.scm"
> (define make-uuid (make-uuid-maker))
> (make-uuid)
"1A10E03B-4FF3486B-00000057-F7CA93A2"

The result is a 128-bit unsigned integer but the only requirement is that it is a unique Scheme object across space and time (within a reasonable amount of certainty). The code is actually following the convention for uuid's, i.e. display each 32-bit part in hex separated by dashes.

Each call to make-uuid-maker returns a new closure around an "address part" and an "object part". Every uuid this closure makes will be unique to its IP address and unique to the internal id of the object in the specific Scheme runtime. (Note: the original uuid generators use the network card MAC address but the IP address is easier to get and just as good for this purpose.)

Each call to one of these closures combines its address part and object part to a "time part" and a "random part". The time part is roughly the number of milliseconds since January 1, 1970 mod (2^32)-1. The random part is a random 32-bit unsigned integer. So within the closure's IP address and randomly allocated object, each resulting uuid is also made unique to the millisecond and a random number *within* the millisecond.

(define (make-uuid-maker)
  (let ((address-part (or (hex-encode-ip-address)
     (hex-encode-random-integer)))
 (object-id-part (hex-encode-random-object-id)))
    (lambda ()
      (let ((time-part  (hex-encode-current-milliseconds))
     (random-part (hex-encode-random-integer)))
 (string-append time-part "-" address-part "-" object-id-part "-" random-part)))))

The IP address is the host's internet address if it has one, or an intranet address, otherwise the address part becomes just another 32-bit random integer.

(define (hex-encode-ip-address)
  (let* ((address (or (get-internet-address)
        (get-intranet-address))))
    (if address
 (lengthen-with-leading-zeros
  (number->string (string->number
     (let ((result ""))
       (for-each
                                (lambda (n)
       (set! result (string-append result (number->string n))))
    (u8vector->list address))
       result))
    16))
 #f)))

The object id closed within a uuid maker is also random. If a closure is always created at a specific point, e.g. in initialization, then this object runs the risk of having the same id each time. Instead a random number of objects (not more than 100) are created before the one captured in the closure.

(define (hex-encode-random-object-id)
  (let loop ((n (random-integer 100)))
    (let ((id (object->serial-number (cons 1 2))))
      (if (= n 0)
   (lengthen-with-leading-zeros (number->string id 16))
   (loop (- n 1))))))

Well that was fun. If you see any serious flaws let me know. There are some Gambit-specific dependencies here but nothing too difficult to replace. The formatting's really bad, but I am off to the beach for the day so later.

Left Field

This will take some digesting to grok...

"If the Java Content Repository (JCR) API expert group's vision bears out, in five or ten years' time we will all program to repositories, not databases, according to David Nuescheler, CTO of Day Software [4], and JSR 170 spec lead. Repositories are an outgrowth of many years of data management research, and are best understood as fancy object stores especially suited to today's applications."

John Dougan comments...

My goodness, that data model reminds me of RDF. Is there a database/datamodel equivalent to Greenspun's 10th Law?

Interesting comparison to think about. For the sake of argument more than having made any real deep theory, I have previously wondered aloud whether there are a small number of common data persistence patterns.

AJAX lessons

Philippe Bossut's notes from the OSCON 2005 tutorial on Learning AJAX...

Most interesting stuff I learnt
Size of the data you can send and cache in a typical XMLHTTPRequest: 4K (small!). Best data type to return: JavaScript instead of XML.

(Alex Russell's PDF)

json-py update

Some fixes to json-py are on sourceforge. Also included is Jim Washington's minjson.py which is faster but not as compliant (as per the spec).

json.py is intended to be a straightforward implementation that can be easily duplicated in other languages.

The Right to (Apply for) Work?

Do American programmers have the right to work in America?

The answer may surprise you. Or not, in our new plutocracy.

Update: Someone commented...

US Constitution does not garantee employment opportunity for any specific type of job.

And of course this comment is correct. So I clarified the title of this entry. See the parenthesis.

This comment really misses the point of the referenced article. The visas are intended to bring in temporary skills from outside the US where those skills are in short supply.

If US citizens are unable to even *apply* for the more than 50,000 open positions, how can anyone tell that these positions cannot be filled by Americans? That is the point... the powers that be are circumventing the purpose of the visas (i.e. fill a temporary labor shortage) in order to gain an advantage over American workers (i.e. bring in more labor and lower the demand for salaries) that is unintended by the visa laws.

Is that more clearly stated? This is not about a right to work. It is about a law and the circumvention of that law for anti-competitive purposes at a time when many Americans are losing careers and only replacing them with lower paying jobs. That may well be a trend of the emerging global economy, but that does not excuse the practices by the US department of anti-labor in these 50,000 plus positions. There is no right to work, but neither is there a right to exclude Americans from applying for jobs in America, for crying out loud. Get real.

Things That Make Me Go Hmmm

Some things might make *you* go hmmm. However there are other things that make *me* go hmmm.

While the APP is 'just a' publishing technology and Atom is 'just a' syndication technology, there are five reasons to that microprotocols - namespaced module extensions that inform and affect application behaviour - will appear as an alternative to WS and SOAP based technologies by targeting enterprise computing concerns like reliable messaging and tracking.

Hmmm.

Crystal Ball of Mud

Via the Register...

EDC believes PHP's recent glorious past is over, as customers spend money on "more important" technologies to build mission critical systems and vendors like Microsoft and Sun Microsystems make more of a concerted marketing push around alternatives such as ASP.NET and JavaServer Pages (JSP).

Really?

Peter Yard says (yes, via Phil Windley)...

Things have -- gotten simpler.

Really?

Liberation Theology

Again Phil Windley, this time relaying DHH on Rails, but the philosophy is generally applicable...

When you insist on flexibility over everything else, you sacrifice velocity and agility. Constraints are liberating.

Arbitrarily Complex Messages

SOAP vs. HTTP... Phil Windley blogs about Randy Ray's talk at OSCON 2005. Ray suggests an example where SOAP is better than HTTP. Phil's paraphrasing...

The CalculateRoute API allows not just a start and end point, but also an arbitrary number of waypoints along the way (to calculate the route for an entire trip).

Since this is a constant resource—the route won’t change (much)—this should be a GET request with a URL. Designing such a GET request is difficult. There’s no real standard for creating a REST protocol. There’s no WSDL.

But we can get WSDL and SOAP to fall down flat as well. This route calculator takes a fairly simple list of values. A more complex calculator could take fuzzy values... for example a preference rather than a requirement to travel through a particular location, or a preference to travel through a particular location if the total trip time is not more than N time units, or...

The example can easily exceed the ease of expressing in WSDL. Then what's the value of WSDL? WSDL maps fairly well onto currently popular imperative languages, not very well onto constraint logic programming languages or deductive logic languages, or...

GET in HTTP seems to me similar to an object reference. Sometimes a query is needed to get the reference in the first place. But once I have it, I can "dereference" it over again.

I suppose POST is the means to do the query, and POST can take arbitrary data just like a parameter in a WSDL definition.

So I guess I don't get the argument. Certainly WSDL is not the end of the story except for some small set of convenient API's similar to those you'd write in Java, C#, or C++.

HPC

Matt Stansberry writes...

What is holding High Performance Computing back?

With the hardware becoming commoditized, why aren't more companies using HPC? According to a report from IDC and the Council on Competitiveness, minimal support from software vendors and a lack of training are the culprits...

Writing code to take advantage of the new clusters means scrapping the code written for the scale up models of the past, an expensive proposition.

Why I Live in Portland

Ted Leung writes about Portland...

Being at the convention center is helping me to discover more about Portland. The free ride zone for the Max train is a thing of beauty. I had thought Seattle's bus free ride zone was good, but the Max train beats that all to pieces. Combine that with Portland's short city blocks, and wonderful downtown, and it's all very appealing. I've had lots of conversations (usually on the way to meals) about how nice Portland is.

This is where we are supposed to tell you how much it rains in Portland. That's a tough sell this time of year... typically nothing but blue skies for weeks.

But there is a reason OSCON is not held in Portland in November. 8^)

Why I Live in Oregon

A photo from an OSCON 2005 attendee shows why I live in Oregon. This spot is about an hour from my house. It doesn't get any better than this for me. If you don't like the rare 90F days in the summer, just drive over the Coastal Range an hour and you are likely to get 60F because the hot valley is sucking all the moisture in from the Pacific. Camping on the beach in August often means you'll be wearing sweatshirts and jackets in the afternoon.

(And here are all the OSCON 2005 photos.)

Whither Object Databases

Jonathan Robie, in an interview with Jon Udell, captures the reason for failure of the object database market better than any explanation I've seen...

JR: OQL is a nice language. The grammar of XQuery has benefitted a lot from it, and some of the implementation has too. Of course, it wasn't commercially relevant for the object database vendors to implement OQL in any kind of complete way.

JU: Because?

JR: Every object database had a different market, and a different model, and people weren't looking for interoperability between ObjectStore and POET, because why would you use them for the same thing?

Gems: Servers, Blogging, etc.

Phil Windley reportedly paraphrasing Paul Graham at OSCON 2005...

Someone who proposes to run Windows on servers ought to be prepared to explain what they know about servers that Google and Yahoo don't know...

The NYT front page is a list of people who write for the NYTs. Del.icio.us is a list of people who are interesting. You can see them side-by-side. You can see how little overlap there is...

Start-up environments are more like home work environments. This is probably the most productive the company is ever going to be...

Meetings count for work, just like programming, but they're so much easier.

Aw, just go read it. There are too many gems to quote.