Storage Paradigms

Stephen Williams writes in the FoRK email list on an exchange about Amazon's new storage service...

I firmly believe that both full filesystem semantics and ACID integrity constraints are red herrings and have seen a number projects reach the same conclusion...

With database-based applications, even when you have ACID capabilities, there are a number of reasons to avoid updates, avoid "accumulators", and otherwise avoid many of the situations where you needed transactions to begin with.

Having worked on and with very complicated object-oriented databases, very complicated relational databases, and other approached, I've noodled for some time on the possibilty that there are three basic persistence patterns that can meet a ton of needs.

I will be interested to learn if Amazon has now or will offer some higher-level services on their storage service. Search, matching, versioning, etc. I see they want to keep it simple, and I agree with that. Before long though people will want to do things with their storage. Some of those things will be better done very close to the data itself.

I wonder if/when we'll see the ability to put computations very near Amazon's storage (including indexes, calculations, searches, etc.) that are aware of the format of the stored data, that is secure, etc. Storage is just the most basic start of a shared grid of services.