Relational databases don’t reflect the way people really think about data. Key-value databases do.– Jacob Davies

Most people don’t think about data, period. There is a skill in breaking down a set of data into components and then arranging them in a meaningful structure.

What if you had a set of Football Scores? Would a key-value database work there? Oh, but wait, you want to also show the standings of each football team in its respective division, and tie those records back to the original team stats, and tie the team roster into individual game leaders as well as season leaders.

Then I want to graph, slice, and dice the whole mess across seasons to show most improvement by player position, most consistent player, and best ROI on the coach (how much $$$ did a good coach cost for a winning record vs. the highest paid coach)?

These are real world problems, btw.

Relational databases don’t reflect the way people really think about data. Key-value databases do.

What was funny (in the end) to me was that I took a discrete math course 1st or 2nd semester 1st year CS undergrad. I already knew a fair amount of it, learnt some more, and just could not see what use it would be to what I knew about computer programming (which I thought was a lot, because I didn’t know enough to know what I didn’t know yet). Set theory? What does this have to do with the price of tea in China?

So, 2nd year, along came a course in relational databases. And I was having some minor difficulties in getting the concepts until suddenly something went CLICK in this really rather visceral way and what I’d learnt in set theory and what I was trying to learn about relational databases suddenly all slotted in together into a perfect understanding of the subject. Love those moments of enlightenment.

Then I got my first real job, which happened to involve building an application backed by a relational database, and that rapidly showed me that I didn’t know a damn thing about designing database applications.

Not me personally, of course, but yeah, there’s a lot of COBOL out there still running against them. And SQL has a fair amount in common with COBOL – both were at least partially intended to be used by non-programmers, hence the verbose syntax, and the SQL datatypes have more in common with COBOL than with modern ideas.

Relational databases are really clever, but you have to understand set theory somewhat – either explicitly or intuitively – to make good use of them. And for a lot of people that’s clearly not a natural way to think (including me, though I can do it).

Normalization is another area where ordinary intuition about organizing information does you no good at all. (And I tend to think it’s overrated anyway (though obviously important still!) Data duplication is a fact of life to be dealt with, and you can’t make it go away by forcing an ever-less-intuitive structure on your data.

Sorry for the confusion!

It’s not that I think relational databases are all that. Actually, the biggest problem I have with them is that they never bothered to solve the data-binding problem in a standard way, leaving the door open for all these half-assed ORM frameworks.

Of course, I use Hibernate – I’m lazy, and I can’t be bothered to do all that object-relational mapping junk myself. But I gotta tell you, I still write a lot of SQL – some of it in the OO Hibernate version of SQL, but a fair amount in real, raw SQL, because you just can’t get a truly optimized query with a translation layer in the way.

Makes me want to write my own ORM that’s the barest thin layer over the database. Of course, I’m sure that’s what everyone else thought they were doing too.