ETL and MVC

Many Web developers think in MVC, the design pattern that describes Web development frameworks from Struts to Rails to CodeIgniter. What I’m finding lately, though, is that it’s not incredibly intuitive. For one thing, people know what a “view” is (the “v” in MVC), but trying to get people to understand the difference between a “model” and a “controller” can be difficult. Also, what we call controllers in most MVC implementations aren’t really controllers. The controller is a component of the MVC framework itself. Once you’ve built a couple of projects with MVC, that sort of thing doesn’t matter, as long as you get how the applications are built. But these problems present difficulties when training new developers.

The other day I was thinking about another acronym that’s relatively new to me — ETL. It stands for Extract, transform, load and is DBA jargon for the process of migrating data from one data source to another. But I think that for reports in particular, it describes how you should organize your code pretty well.

In a reporting application, extraction would refer to the process of figuring out which data the user wants using the URL and any filter parameters that are passed in, and running the query to retrieve that data. Transformation is the massaging of the data so that it can be presented in whatever format is needed. I strongly prefer having as little transformation code in my reports as possible, it’s usually much better to store the data in the format that will be needed for the reports. And then load, in this case, refers to the process of presenting the transformed data, whether it’s on a Web page or in a downloadable export file.

I think that if you can get developers to think about reports in terms of these three discrete steps, you’re more likely to wind up with code that’s more flexible and better organized that it would be otherwise. You can always yell at people and tell them not to put business logic in the view layer, but creating a mental model that explains why that’s a bad idea might help.

One thought on “ETL and MVC

  1. The adherence to MVC is cargo cult programming. It never made any sense to me, maybe it makes slightly more sense in GUI apps than web apps, but I am not even sure of that. Most of what has MVC slapped on it doesn’t even resemble that original design and the division of responsibilities doesn’t map well to what kinds of things happen in web applications. In general I think when you have the amount of difficulty in explaining what you’re trying to do that MVC advocates always seem to have, you’re doing something wrong.

    I think in terms of filters, transforms, and templates (which are all kind of the same thing). Mix some data and a template. Filter something. Combine every element of X with the corresponding element of Y. That kind of thing. If you can’t explain it to someone who knows nothing about computers with simple block diagrams, you don’t really understand it.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>