Amazon has added a new company to their S3 and EC2 services called SimpleDB. Rather than providing a traditional relational database, they’ve created an object-oriented database that is supposed to be simpler to use. That’s probably not a bad idea, given the popularity of ORM libraries like Hibernate and ActiveRecord.
They’ve already created libraries for Java, Perl, C#/.NET, and PHP, and the documentation is thorough.
The thing I always wonder about with solutions like these is what happens when you get beyond the simplest mode of usage. It’s great to abstract away all that nasty SQL until your database gets big and your queries start getting slow, but then you need to get closer to the metal and figure out what’s going on in there. How do you do that with SimpleDB? Or is it expected that most users won’t build big enough sites with SimpleDB to have to worry about that?
For what it’s worth, SimpleDB reminds me of the database API that was provided to Ning developers when the site launched. Here’s some documentation.
Update: Here’s a blog post with more scoop on SimpleDB. The software is written in Erlang.
December 14, 2007 at 12:50 pm
My understanding of relational database theory is that although the data format may be somewhat unnatural, the optimizations that can be performed on such strictly structured data are powerful enough to make it worthwhile. In my experience, the relational data we rely on is growing at least as fast as the hardware’s ability to process it in a timely manner.
I think your point about getting closer to the metal is important, too, in SQL’s continuing success from a practical perspective. Any decent programmer can pretty easily understand how even complex structures represented in these flat relationships are going to work together inside the computer. Get into schema-less structured data, and how you tie it all together gets a lot harder to comprehend.
I think it’s like everything else in the computer world. We’ll switch away from relational databases once our systems are fast enough that we don’t need them anymore. When we can structure our data naturally and still get acceptable performance given the inevitable inefficiencies, RDBMSes will die a natural death. But I don’t think that will happen any time soon.
December 14, 2007 at 2:16 pm
Here is a different perspective on SimpleDB: http://marcelo.sampasite.com/brave-tech-world/Amazon-SimpleDB-What-nobody-is-t.htm
December 14, 2007 at 8:16 pm
The trend towards schemaless or at least loosely-typed databases is pretty darn strong. I say this as someone working on something that is related to this… there are a bunch of different directions out there. Wikis are one aspect of it.
The Amazon approach looks very good in concentrating on a small number of operations on simple key-value data. (This is what I thought Google Base would be about.)
For many many applications this is all that is needed. Well, nested structures are useful but of course you can emulate those with key-values anyway. There’s no versioning built-in but again you can do that yourself (although it’s sure nice to have it in the basic query API).
This is a pleasing development to me because it increases the legitimacy of loosely-typed or flexible or schemaless databases, but businesses in our target markets can’t be storing data in Amazon’s database.
December 15, 2007 at 11:22 am
Interesting comments on your own blog, Marcelo. I think the LDAP comparison is interesting and accurate.
December 15, 2007 at 10:15 pm
As someone who has worked in enterprise software dev for a number of years, I have a hard time wrapping my mind around schemaless data storage. I guess that’s a good reason to check this out.
December 16, 2007 at 1:49 pm
My bet – we’ll see (from Amazon or from other quarters) a layer with a full relational model conforming interface. SimpleDB will handle the tuples, S3 will handle the BLOBS, EC2 will grind the queries, and the application developer won’t have to worry about it.
December 16, 2007 at 1:55 pm
My bet – we’ll see (from Amazon or from other quarters) a layer with a full relational model conforming interface. SimpleDB will handle the tuples, S3 will handle the BLOBS, EC2 will grind the queries, and the application developer won’t have to worry about it.
August 25, 2008 at 6:39 am
“you need to get closer to the metal and figure out what’s going on in there” – that’s right and presently there is no way to do that, another very interesting schemaless database is “Poseidon” from Brainwave platform, packed along with a complete development and deployment suite.