Testing Inheritance with Inheritance

Unit testing and test driven development are things that I am very passionate about. Writing good unit tests is not always as straight forward as you would hope and it takes practice to really get good at it. Even then, there are still things that are inherently hard to test.

Inheritance is one area where unit testing becomes difficult. The difficulty does not necessarily come from actually writing the tests themselves (though obviously, some of it may be) but more in figuring out which tests to write, and how to go about doing it.

Say, for instance, I have the following class


with the following tests


And I want to extend this class for second graders to include multiplication and division. This is where I run into a dilemma. What is my first test for the second grade math implementation? I could start a number of different ways:

1. I could re-test drive addition and subtraction and refactor by adding FirstGradeMath as a parent class
2. I could copy the tests from FirstGradeMathTests and paste them into SecondGradeMathTests
3. I could assert that SecondGradeMath is a subclass of FirstGradeMath
4. I could Inherit the FirstGradeMath Tests

I'd like to review the pros and cons of each of those options.

With the first option, re-test driving addition and subtraction then making SecondGradeMath a subclass of FirstGradeMath seems like a tedious waist of time. This approach will not scale. Can you imagine writing the tests for Algebra, Trigonometry, and Calculus when each time you have to start with addition? However, this is probably the most pure test driven approach. If you prefer purity over pragmatism this may be your preferred approach.

In the second option, copying all of the tests and pasting them is going to be painful. Now you have identical tests that are exactly duplicated. Hopefully, you have some sort of static analysis tool that will let you know that you are sinning if you choose this approach. If, at some point, there are additional tests added for some edge cases the addition tests you will have to remember to duplicate them for FirstGradeMath and SecondGradeMath. This has the same scaling issue as the previous approach. How many tests are you going to have to copy, and how many times? On the positive side, your tests for SecondGradeMath do not make any assumptions about the implementation and assert that addition and subtraction both work correctly in addition to multiplication and division. 

The third option takes a very different approach than the first two. This option makes the assumption that FirstGradeMath is a solid implementation that has been thoroughly tested and is as correct as possible. The problem with this approach is that both the addition and the subtraction methods depend on data that is set in the constructor of FirstGradeMath. If, for some reason, the SecondGradeMath implementation does something differently in the constructor, it could potentially break the implementations of addition and subtraction. This may be a viable solution if you are comfortable making the necessary assumptions. If you control both the parent and the child class implementations, maybe you don't need to make any assumptions. This approach definitely has the least amount of friction, but has the potential to be risky.

The final option is somewhat of a middle ground between all of the first three approaches. By creating new tests for SecondGradeMath that inherit from the FirstGradeMath Tests you are getting a similar effect of copying the original FirstGradeMath tests. So you are covered from a regression standpoint. At the same time your test code is mirroring the implementation, so since SecondGradeMath extends FirstGradeMath, SecondGradeMathTests extends FirstGradeMathTests. 

Lets take a look at what the code would look like for the initial SecondGradeMath tests and implementation with the addition of multiplication and division:



There is one major issue with the tests at this point. They violate one of the FIRST rules of unit testing. (Fast, Isolated, Repeatable, Self-Validating, Timely). These tests aren't isolated. Each test is dependent upon some instance variable called "sut" being set up prior to the test running, and each test is dependent upon that object's state. Maybe this is alright if the only thing that we will ever need to add, subtract, multiply or divide is 2 and 2, but that is not the case here.

Inheriting unit tests requires an abstraction, an instance of a *GradeMath, however is not the correct abstraction. If we remove the "sut" variable from the setUp methods and just instantiate FirstGradeMath in the FirstGradeMathTests and SecondGradeMath in the SecondGradeMathTests then inheriting the FirstGradeMath tests are not going to really test anything about SecondGradeMath. Our abstraction level needs to be higher than an instance, so let's move up one level to the class. Here are the revised tests:


This is much better. Now each test class tests exactly one implementation but is free to instantiate that implementation however is necessary for the test at hand. This provides the isolation that the previous tests were missing.

I prefer this method of testing when inheritance is in the picture over most others. In the end, the best approach is most likely different depending on the situation.

Python Dependency Management: Part 2

If you read my post about Python Dependency Management, I'm sorry. I've changed my opinion about Buildout. While Buildout is still a great tool and does an excellent job of isolating your Python environment, it falls short in a few different areas.

The first area is documentation. It is sparse. Buildout is a tool that does a lot of things, I would even say too much, but you would never guess it by looking at the docs. While I'm sure that there are plenty of people who use Buildout, when you run into a problem and you can't find the solution on StackOverflow, IRC is really your only chance at getting help. Most likely, the only help you will find online is initial setup tutorials for using Buildout. Finally, buildout has the concept of recipes, which is an awesome idea. You can define your own custom recipe to build your project or a dependency or whatever you want. If you decide to go that route, you're completely on your own. The API for building a recipe is completely undocumented and you're better off copying an existing recipe and hacking on it until it does what you want. (that's what I did and it sort of worked).

Another thing that is an annoyance with Buildout is the bin directory. When I am working on a project that uses buildout I have to remember that $PROJECT_ROOT/bin/python is where my dependencies are and system python doesn't know anything about my project's dependencies. Some sort of shell wrapper (like RVM or Virtualenv which I will talk about later) would greatly improve the developer experience.

Lastly, I don't like having to have a bootstrap.py script that I have to run to set up Buildout, then a bin/buildout script that I have to run that requires a buildout.ini file and a setup.py. That seems like too much configuration, and too much cruft in my repo to manage a single thing. Maybe this is a bit nit picky, but the alternative is much less intrusive to my code base.

The alternative that I prefer is Virtualenv. Virtualenv has some of it's own weirdness that you need to overcome. First step is installing Virtualenv. This is easy:


Yes, I did use sudo there, but that's the only time (well almost) that you will need to install anything in system python. But something that manages isolated python environments is just fine to install system wide.

The next thing is creating a virtulaenv which looks something like this:


Once you've created it, then you have to use it. That's the first bit of weirdness. To use a virtualenv you do something like this:


Once that is done your prompt will change to look something like this:


Now you are using a virtualenv, at any time to stop using that virtualenv you can do this:


The big problem that I have with this is that now instead of Buildout's 3 required files in my project I have a directory that contains, geeze what is in there, oh yeah, ALL OF PYTHON. Sure I could just add that virtualenv name to my .gitignore, but the virtualenv's name is arbitrary, the next time I clone my project and create a new virtualenv I could call it blueberry_pie and I'd have to add that to .gitignore also. The alternative is creating the virtualenv somewhere outside of my project. So where should it go? I don't know, maybe I'll just have a bunch of virtualenvs floating all over my file system wherever I need them.




That sound bite pretty much sums up my feelings on that topic. Before I even get into what having a virtualenv means and what it can do, lets talk about how we can fix all that rigmarole. The first step is installing Virtualenvwrapper:


You may have noticed that this installation requires you to modify your ~/.bashrc. There is a very good reason for that. Virtualenvwrapper needs a single directory on your file system that it will use to house all of your virtualenvs. YAY! Also, I used sudo again. This is the last time, I promise. (sudo is ok for pip, virtualenv, and virtualenvwrapper and that's it!). Now that we are all set up with virtualenvwrapper, how do we use it? Easy. First step, like before, is create a virtualenv like so:


Well, well, well, would you look at that. Our prompt looks like:


after only a single command! Also, to stop using a virtualenv, it is the same as before:


But wait, how do I get back to it? Do I have to cd into $WORKON_HOME and do the whole:


rigmarole again? I think not! To activate a virtualenv using virtualenvwrapper simply do this:


Another extremely useful thing to do with the "workon" command is to run it with no arguments. Doing this will show you a list of every virtualenv on your system that you can activate using the "workon" command. Pretty great right?

But what does it all mean? What does an active virtualenv buy you? I'll tell you what it buys you, FREEDOM! Now when you type "python" in your shell you are actually using the python command that lives inside the bin directory of your virtualenv, same with pip, and same with any other python module that you install into your virtualenv. You don't have to do anything special, just use python and pip like you normally do (which you Do do, right?). If you get into trouble and you've installed a bad version of a package or you've manually screwed something up. Just deactivate your virtualenv and create a new one. No harm done.

What if you're done with a virtualenv? Just delete it like so:


Virtualenvwrapper provides many very useful, very succinct commands that you should check out. Now that you have all the tools you need to create a completely isolated Python environment you can use PIP to manage the dependencies required by your project. One guideline that I personally try to adhere to is that I set up a deployable project slightly differently than I set up a project that will be used as a library.

When I'm developing a deployable project, like a web application, I keep all of my dependencies listed in a requirements.txt file that pip can install using the '-r' flag. When developing a library I typically just list the dependencies in the setup.py using install_requires. I have, on occasion, made setup.py just read a requirements.txt file and populate the install_requires option dynamically, which may be a better approach. Also, in either case, keeping a separate development.txt requirements file that includes any testing dependencies and also includes a '-r requirements.txt' line is a must. The latter approach for managing dependencies for libraries using a requirements file is required when doing this.

One last thing I will mention about creating libraries is that make sure that you either document how to run your tests OR hook your test runner into "setup.py test". If everyone did the latter, the world would be a better place.

As with Buildout and virtualenv, virtualenvwrapper also has it's shortcomings. What I really want to see is something similar to RVM from the Ruby space. RVM allows you to create an isolated Ruby environment also, but it goes above and beyond what virtualenv and virtualenvwrapper provides. With RVM you can install different versions of ruby, and easily switch between which version is "active", and create different gemsets (a gemset would be similar to a virtualenv) and switch between which gemsets are active and install gems into a single gemset.


Virtualenvwrapper is excellent and if you're writing Python, you should be using it. Hopefully, someday, virtualenvwrapper will get to the point where it can manage versions of Python as well as installed Python modules. When that day comes I will be on board, until then I'm more than happy with PIP, virtualenv and virtualenvwrapper to manage my Python environments.

My Crusade for Agility: Part 13

Today was a good day, we just got a shipment of 3 brand spanking new white boards. I'm super excited to get those things hung up on the walls and see how they look. I'll share pictures as soon as they are available.

This is the 13th write-up that I've done regarding My Crusade for Agility. It has been 5 months since Part 12 and there have been quite a few really great developments. In Part 11, I mentioned that we had a new hire starting. He has been working out great and we have another new hire starting on Monday. We decided that with 4 full time developers working at our 2 tables in the pit, things are starting to get a bit cramped. So we decided to tear down some cubicle walls and make our area larger. (Pictures to come soon).

I mentioned "Another Challenge" in Part 12, which is getting the rest of the development team up to speed on all of the technologies that we are using. It's a hard problem to solve. Uncle Bob videos and Gary Bernhardt's Destroy All Software videos have been helpful. Also regular code katas where we typically pair and use TDD to solve some problem have yielded positive results. It's difficult to work a full time job and within that time teach and learn a completely new technology, or I should say a completely new category of technology that includes dozens of libraries and multiple languages.

In other news, we have taken another big step towards becoming more agile as far as our products are concerned. Up until recently we have had a single monolithic application that does everything. While it is nice as a user to be able to go to one place and have everything at your fingertips, some users don't want everything. What if we, as a company, want the ability to sell a single feature of our application as it's own product. That is difficult to do when all the code is in one place.

We decided to create an entirely stand alone application for our latest feature and built a web service api for integration with our existing product. Now our new feature is still a feature of our current application, but is also a stand alone product that can be marketed and sold separately. The development of this feature went extremely smooth. Without a doubt, we are going to be doing more features in a similar fashion.

The last thing that I will mention that I believe does contribute to our agility is Chef and Capistrano. As part of our last new web service feature we also had to tackle deployment and server setup type tasks. Being developers we automated it. We have Chef recipes that set up our web servers, database servers and even our development machines. We can get some brand new hardware with only an OS and have it up and ready to serve requests or ready to do some hard core development in a matter of minutes.

We are in a good place, a great place and more exciting things come up all the time. I will post a follow up soon with some pictures of our newly expanded development pit and the new whiteboards once they've been hung. Thanks for reading, check back soon.