Tag: unit testing

Unit Testing Matters

Unit testing is a simple practice that can be explained in one sentence: each method should have an associated test that verifies its correctness. This idea is very simple. What is amazing with unit testing is how powerful this simple practice actually is. At first, unit testing seems like a simple approach to prevent coding mistakes. Its main benefit seems obvious:

Unit testing guarantees that the code does what it should.

This is actually very good, since it’s remarkably easy to make programming mistakes: typo in SQL statements, improper boundary conditions, unreachable code, etc. Unit tests will detect these flaws. Shortly after, you will realize that it’s way easier to test methods that are short and simple. This confers to unit testing a second benefit:

Unit testing favors clean code.

This is also very good. Unit testing forces developers to name things and break down code with more care. This will increase the readability of the code base. Now, armed with a growing suite of tests, you will feel more secure to change business logic, at least when the change has local effects. This is a third benefit of unit testing:

Unit testing provides the safety net that enables changes

This is excellent. Fear is one of the prime factor that leads to code rot. With unit tests, you can ensure that you don’t break existing behavior, and can cleanly refactor or extend the code base. You might object that many changes are not always localized, and that unit tests don’t help in such case. But remember: a non-local changes is nothing more than a sequence of local changes. Changes at the local level represent maybe 80% of the work; the remaining 20% is about making sure that the local changes fit together. Unit tests help for the 80% of the work. Integration tests and careful thinking will do for the other 20%. As you become enamoured with unit testing, you will try to cover every line you write with unit tests. You will make it a personal challenge to achieve full coverage every time. This isn’t always easy. You will embrace dependency inversion to decouple objects, and become proficient with mocks to abstract dependencies. You will systematically separate infrastructure code from business logic. With time, your production code will be organized so that your unit tests can always obtain an instance of the object to test easily. Along the way, you will have noticed that the classes you write are more focused and easier to understand. This is the fourth benefit of unit testing:

    Unit testing improves software design

This is amazing! Unit testing will literally highlight design smells. If writing unit tests for a class is painful, your code is waiting to be refactored. Maybe it depends on global state (Yes, I look at you Singleton), maybe it depends on the environment (Yes, I look at you java.lang.System), maybe it does too much (Yes, I look at you Blob), maybe it relies too much on other classes (Yes, I look at you Feature Envy). Unit testing is “a microscope for object interactions.” Unit testing will force you to think very carefully about your dependencies and minimize them as much as possible. It will naturally promote the SOLID principles, and lead to better a decomposition of the software.

Honestly, I find it amazing that such a simple practice can lead to so many benefits. There are many practices out there that improve software development in some way. What makes unit testing special is the ridiculous asymmetry between its simplicity and its outcome.

More

A Simple Categorization of Unit Tests

Unit testing has become incredibly popular during the past years. As a consequence I sometimes feel like we’ve lost the focus on why we write unit test and what we expect as a return on investment.

The success of unit testing probably comes from the simplicity of the appraoch. However, we’ve learned since then that unit testing is not so easy neither. A high percentage of code coverage does not necessary mean quality software, doeasn’t mean that border cases have been covered, and can even impede software maintenance if the test suite is poorly organized.

Don’t get me wrong, I do value the benefit of unit testing.  But unit testing for the sake of unit testing has no value to me. If you think an other form of automated testing will be a most rewarding strategy, then do it. If you go for unit tests, the important questions to ask are:

  • Did the unit tests revealed bug in the production code?
  • Did you organise the unit test in a meaningful way?
  • Did you spend time to identify and test border cases?

A clear “yes” to these three questions indicates an intelligent and probably rewarding testing effort. A pertinent test suite should reveal the bugs that you introduce in the system — and don’t pretend you don’t introduce any. A well-organized test suite should give you a high-level view of the features in the module. A well-crafted test suite should cover the nasty use cases of the system before they hurt you when the system is in production.

There is an abundant literature about unit testing, but nothing that I read seemed to cover the reality of the unit tests that I write. I therefore analysed the nature of my own unit tests and came with a personal categorization which differs from existing one. Here is the 4 categories I’ve identified as well as a short description of the pro/cons of each kind.

Basic unit tests

A basic unit test is a test suite which covers one unique class and test each method in a more-or-less individual fashion. This is the core idea of unit test where each tiny functionality is tested in full isolation, possible with the help of mock objects to break the dependencies. As a result, the test should be repeatable and also independent (of the environment and of other modules).

Problem with real unit tests are:

Unit test with indirect assertions

For basic unit tests, the subject under test (SUT) is the very one for which we assert the behavior. We perform an action on the SUT and ensures it behaves correctly. This is however sometimes not possible as soon as the system become a bit more complicated. As a consequence, the actions are performed on the SUT, but we rely on a level of indirection for the assertions; we then assert the behavior of another object than the SUT. This is for instance the case when we mock the database and want to ensure that  the rows are correctly altered.

  • Coupling with the implementation is still high
  • Fake objects migth be used instead of Mocks — they contain state and aren’t purely hard-code anymore
  • Behaviour of the SUT is harder to understand due to the level of indirection

Inflection point unit test

Inflection points — a term coined by Michael Feather if I’m right — are somehow the entry points to a given software module. For utility libraries or services, the inflection points correspond to the public API. Testing these specific points is the most rewarding strategy to me, and other people think the same.

  • Inflection points are less subject to changes, and are closer to a black-box form of testing
  • Tests become the first client of the interface and give you a change to check if the API is practical
  • After having covered all the common use cases of the inflection point, the test coverage of the underlying classes should be close to 100%. If not, this indicates a potential design weakness or useless code.

Dynamic unit tests

I qualify tests as “dynamic” when their execution change from one run to the other. Primary goal of such test is to simulate the dynamicity and variability of the productive system. Such test are however quite far away from the concept of basic unit tests and could be considered to some extend as integration tests; they are however still repeatable and independent of other modules. Execution in the real system may indeed be aftected by either

  • Threading issues
  • Contextual data, e.g. cache
  • Nature of the input

Most of these tests rely on randomization, for instance to generate input data or disrupt the scheduling of threads. Though it’s more complicated,  randomization and fuzzing have been proved as effective techniques to detect issues which would never arise with fixed execution condition. Think for instance about phase of the moon bugs and date problems.