More Thoughts on the Future of Persistence

Jay Han picks up the discussion on the future of persistence and poses some questions...

What are the experiences from orthogonally persistent OSes? Did they make say IPC any easier?

I can't answer from the OS perspective, but I can answer from the Gemstone Smalltalk perspective (since Smalltalk has been accused of being an OS, and indeed has demonstrated itself to be a pretty complete one).

Did Gemstone/S make IPC any easier? Of course the answer is yes and no.

Yes, because it implements a transactional (ACID) shared memory, so all applications are coordinated in the loosely coupled manner of a database. And yes, because the persistence between one app and the database is transparent, i.e. that relationship is nearly identical to a Smalltalk app and its image file.

No, because it is specific to Smalltalk. Integration with other languages is no better, i.e. you need to use COM or CORBA or C or SOAP or... Next question...

Amoeba had more RAM then disk and it didn't get virtual memory until late. Obviously there must have been some problems making data persistent there. Was there ever another system that had larger primary memory then secondary?

All I can say here is that Gemstone/S applications perform best when most of the persistent pages of objects fit in a Gemstone shared page cache and most of the applications run on that node that has a cache.

Palm PDA and cell phones have transparent persistence. What are other examples of a set of applications utilizing transparent persistence today?

Gemstone applications in insurance, banking, transportation, manufacturing, etc. are all based on transparent persistence. This is not entirely true because by the time a large multi-user application gets into production there is a good bit of code managing transactions, conflicts, etc. so the persistence is no longer entirely transparent. But there is no O/R mapping and there is no concern about the objects not fitting in RAM. Instead of persistence per se, the emphasis is on coordination.

Now I would argue strenuously that Gemstone/S does not have the right coordination model (i.e. a shared transactional object space). In this model *everything* in RAM is transactional (at least everything strongly connected from a persistent root). And everything in an external database is also transactional, so every connection between the transactional RAM and the transactional relational database requires a two phase commit transaction, or some confidence that the two phase rules can be relaxed.

A better model is not to make everything in RAM transactional, rather the application should use specific coorindation mechanisms, in particular one of:

• tuple space
• versioned tree
• star schema dimensional facts

These mechanisms are kinds of "databases" that could be implemented even more simply in MRAM. An application that may itself be in MRAM should still coordinate its activities with these explicit coordination mechanisms that may or may not have multiple, distributed client processes.