As I have mentioned many times before, I’m a big adherent of unit testing. If you have a comprehensive suite of unit tests, seeing them all work before you commit a bug fix or feature to an application provides at least basic assurance that you haven’t screwed things up too badly. In the case of Web applications, the utopian goal is to be able to run your unit tests and feel assured that your application works without doing any testing through the browser at all.
There are many reasons why we’ll never see utopia, including client-side scripts and other problems related to display rather than basic input and output. However, it’s very difficult to consistently test an entire application going through it by hand using a browser, so you need to have a certain level of confidence in your unit tests. What I’ve found lately is that there’s a specific area where my unit tests usually fall short, and that’s in wiring forms up to actions in my application.
If you think of an application as an API, you can think of a URL as a method or function name, and its parameters as arguments. For example /search.cgi may take parameters like terms and language. When you write a unit test for this code, you use a test harness that pretends to post to that URL, and passes in the required parameters. If the call produces the output you expect, the test is successful. What my tests often lack, though, is a guarantee that the pages that submit to that URL use the correct parameters. There are three places where the parameters are referenced: the code itself (in search.cgi), the test, and the view that calls the URL. The view layer oftentimes goes untested.
So say someone wants to change the name of the terms parameter to query. They update the CGI program and the unit tests for it, and they update every page they can think of that calls search.cgi to update the parameter name. Most of the time, there are no unit tests that verify that the parameter names are correct in forms throughout the application, and even if you wanted to write such tests, there’s no good way to make sure that those parameter names are properly checked, since they live in the middle of freeform data. Your code coverage tool isn’t going to tell you that you missed a search form on some web page in the application.
Anyone know of a good approach for this problem? It’s been giving me fits lately.
January 4, 2007 at 2:37 pm
Take a look at: http://www.openqa.org/selenium/ https://lift.dev.java.net/ and http://jwebunit.sourceforge.net/
They all support functional testing through the browser. This would let you verify that all the web pages and forms all work. Is this what you mean?
January 4, 2007 at 2:42 pm
I’m not a huge fan of such tools, mainly because I have found them unwieldy to work with, but I think they may fill in the gap behind regular unit tests.
January 4, 2007 at 2:43 pm
More specifically, I see those as more tools for QA than for developers to write tests for their own code.
January 4, 2007 at 3:48 pm
There’s also HttpUnit, which seems a lot like those other tools but is intended to be called from your unit testing framework.
January 4, 2007 at 6:24 pm
How about something along the lines of:
It would then use assert_tag (or assert_select for those using 1.2+) to look for input fields with those names (“comment[name]”, etc.). It doesn’t really solve the duplication problem, but at least both would be specified in the functional test near to one another.
I haven’t written it, so if you get to it first, let me know 😉
January 5, 2007 at 10:29 am
Rafe – Would the integration testing in Rails do what you’re looking for?
January 5, 2007 at 9:56 pm
I work mostly in a J2EE Struts environment, and just jumped on the JUnit bandwagon in the past year. A real source of frustration for me has been testing DAO’s, since I place all the JUnit tests outside my JVM container, and we use JNDI lookups in an Oracle config file. I’ve managed to use MockStrutsTestCase with success to test the action forwards, but I have to say it seems like a rather trivial test.
I’ve been reading “JUnit Recipes”, by J.B. Bainsberger, and something he suggested really made sense to me. He says you really should not design your tests so that they test frameworks (like Spring) that have already been tested. That’s started me designing my classes so that the business logic is carried out in POJO’s, rather than classes that are heavily integrated with Spring/Struts or some other framework.
I realize this doesn’t answer your question directly — i.e. it doesn’t address Actions, except for the part about MockStrutsCase — but I think we’re dealing with the same subset of problems, i.e. how to test classes that aren’t pure POJO’s.
January 6, 2007 at 1:56 am
I couldn’t sleep, so I threw this together. It’s suitable for dropping in test_helper.rb.
# Assert that the rendered response contains input fields matching those # passed in. Using this in a functional test in close proximity to a # "get" or "post" allows keeping views and controllers in sync with # respect to the parameters they require/allow input for. # # The second argument allows the type of input being tested for to be # overridden, as in the case of textareas. # # Examples of use: # # Provide a list of input names to verify. # assert_inputs(["comment[author]", "comment[featured]", "tag_list"]) # # # Provide a hash/array combination, similar to how params is used # # within controllers and :include within AR's find method. # assert_inputs({:comment => [:author, :featured], :tag_list => true}) # # # Provide a hash/array combination, but override the necessary tag to # # "textarea" # assert_inputs({:comment => :body}, :textarea) def assert_inputs(fields = {}, type = :input) field_names = convert_to_field_names(fields) field_names.each do |name| assert_tag type.to_sym, :attributes => { :name => name }, :ancestor => { :tag => "form" } end end private # Takes a potentially nested Enumerable and converts it into an array of # input names. def convert_to_field_names(fields, prefix = "") field_names = [] if fields.respond_to? :each fields.each do |k,v| name = prefix.blank? ? k.to_s : "#{prefix}[#{k}]" if v.nil? or v == true field_names << name else field_names += convert_to_field_names(v, name) end end else field_names << (prefix.blank? ? fields.to_s : "#{prefix}[#{fields}]") end field_names end