rc3.org

Strong opinions, weakly held

Month: November 2009 (page 1 of 4)

ETL and MVC

Many Web developers think in MVC, the design pattern that describes Web development frameworks from Struts to Rails to CodeIgniter. What I’m finding lately, though, is that it’s not incredibly intuitive. For one thing, people know what a “view” is (the “v” in MVC), but trying to get people to understand the difference between a “model” and a “controller” can be difficult. Also, what we call controllers in most MVC implementations aren’t really controllers. The controller is a component of the MVC framework itself. Once you’ve built a couple of projects with MVC, that sort of thing doesn’t matter, as long as you get how the applications are built. But these problems present difficulties when training new developers.

The other day I was thinking about another acronym that’s relatively new to me — ETL. It stands for Extract, transform, load and is DBA jargon for the process of migrating data from one data source to another. But I think that for reports in particular, it describes how you should organize your code pretty well.

In a reporting application, extraction would refer to the process of figuring out which data the user wants using the URL and any filter parameters that are passed in, and running the query to retrieve that data. Transformation is the massaging of the data so that it can be presented in whatever format is needed. I strongly prefer having as little transformation code in my reports as possible, it’s usually much better to store the data in the format that will be needed for the reports. And then load, in this case, refers to the process of presenting the transformed data, whether it’s on a Web page or in a downloadable export file.

I think that if you can get developers to think about reports in terms of these three discrete steps, you’re more likely to wind up with code that’s more flexible and better organized that it would be otherwise. You can always yell at people and tell them not to put business logic in the view layer, but creating a mental model that explains why that’s a bad idea might help.

Revisiting “What would an economist do?”

Back in March, I asked:

I really want to buy the new Neko Case album, released last week, but I have a suspicion that Amazon.com is going to make it the deal of the day sometime soon. What would an economist tell me?

Today it’s Amazon.com’s MP3 deal of the day and you can get it for $1.99. I purchased it a few months ago when it was on their 50 albums for $5 list, but I wanted to point out that my prediction was correct for some definition of soon.

The CCD Future

Joe Gregorio writes about CCDs:

As I’ve thought about this over the years I’ve concluded that the promise of RFID was eclipsed by another technology out there that’s poised to become more and more disruptive, not only to RFID, but to a host of technologies, and that’s the CCD.

Read the whole thing.

Why you might not want to live in paradise

Notre Dame athletic director Jack Swarbick on why he didn’t stay on the west coast:

I’m really all Bay Area at this point. I’m loving it out there. In the course of this dinner, Tom tells me that he’s moving back to Detroit. I said, ‘That’s crazy, why are you doing that?’ He said: ‘If you can live anywhere in the world, you ought to live here, because it’s fantastic. It has all this natural beauty, and the weather is great. As a consequence, so many people who live here don’t have a reason to be somewhere else. They’re attracted by those things as opposed to something else.’ He said, ‘I need to be someplace where there’s a sense of community because that’s what motivates me.’

How Twitter uses Hadoop and Pig

I confess that I have never fully immersed myself into the world of Hadoop and map/reduce, but I am very interested in them. Tony Bain does a good job of explaining how Twitter is using Hadoop (and Pig, a scripting language for Hadoop) for analytics. It’s a great explanation of a practical business application for map/reduce for those of us who are still trying to get it.

The perils of public speaking

My level of empathy was higher than average when I read Danah Boyd’s post-mortem on her Web 2.0 talk today. It’s amazing how easily things can start to go sideways when just a few of the inputs shove you out of your comfort zone. For a reminder of my own public speaking misadventures, check out my posts on Ignite Raleigh. If there’s one thing I’m very glad of, it’s that there wasn’t a live stream of Twitter chatter being displayed behind my head during the talk. And also that I only had to stand there for five minutes.

Phil Agre is missing

One side effect of having been on the Internet for a long time is that there are a lot of people you’re passingly familiar with. I subscribed to Phil Agre’s Red Rock Eater mailing list well over a decade ago, and found it an incredible source of interesting stuff in the pre-blog era. Today on Twitter I learned that he has been missing for over a year. NPR has an article on his disappearance. It’s disconcerting. I hope he’s well.

Saddam Hussein and the Gervais Principle

Saddam lived in fear of a coup mounted by the Republican Guard. His solution was to create the Special Republican Guard, whose main remit was to protect him against coups particularly from the Republican Guard.    You would think that the head of this outfit would be a fearsome figure who would terrify any budding coup plotters.  Woods asked other leading figures if this was indeed the case and the answer was a resounding NO!  Why?  Saddam was well aware of the “who monitors the monitor problem” – what if the head of the Special Guard mounted a coup himself?  Saddam’s solution was not original: appoint a relative.  Make sure the appointee is a coward so he would not dream of mounting a coup.  Just in case he is tougher than you might think, choose someone stupid so he cannot mount a successful coup and is too stupid  to recognize someone else’s good ideas for a coup.

A couple of days ago, I posted a link to an essay about the Gervais Principle. Saddam Hussein had clearly internalized this principle, as he (a sociopath) promoted a member of the clueless class to a key position to mitigate personal risk.

The success of Recovery.gov

Sunlight Labs on the good things about Recovery.gov:

See– we want to see both the information government has chosen to deliver and the recipe for it, alongside its ingredients. Imagine, for instance, if Recovery.gov launched, and the Obama Administration just outright claimed success, and cited through press release, millions of jobs and economic recovery. That, after all, is the alternative. Instead, Recovery.gov’s done exactly what it’s supposed to do– make it so that people can look at the data and make up their own minds as to what is really happening.

And yeah, the data’s bad. But that’s the way its supposed to be. Real-time, online, machine readable. In a politically charged debate over a controversial program, its easy to take pot-shots at the messenger (Recovery.gov), but what Recovery.gov is doing is shining some sunlight in some really interesting places, letting your average citizen see why data quality and real-time reporting really matter.

The quotable Cormac McCarthy

I’ve been seeing snippets of Cormac McCarthy’s interview with John Jurgensen all week and I finally got around to reading the whole thing. You should too, it’s inspiring.

Older posts

© 2024 rc3.org

Theme by Anders NorenUp ↑