Category: Architecture & Engineering

I recently attented three keynotes: Alan Kay and Andreas Kuehlmann at TOOLS and Jeff Kramer at ICSE. Hearing these talks, the software industry seems to be stuck: we don’t know how to move beyond manual development and testing of fine-grained software artefacts. However, as software engineering matures over time, a body of best practices emerges that increases quality of development. For instance, Jeff Kramer showed that certain ideas from the architecture community had indirectly made their way to industry under another form, e.g. dependency injection.

Andreas Kuehlmann used the word hygienic to refer to teams with sound engineering practices. The wikipedia definition of hygiene is a “set of  practices employed as preventative measures to reduce the incidence and spreading of disease”. High hygiene standards favor clean code. Assertions, contracts, refactoring are examples of hygienic practices. Hygienic practices lower the risk that your software become a drity pile of code. I like this analogy with health.

Progress in health promotion didn’t happen in one day. Medical and everyday practices such as sterilization, quarantine, treatment of wastes, dental care take took time to be established. Next to these practices, the healthcare industry enjoyed several breakthrough such as the creation of vaccines, or organ transplants that enabled this remarkable progress. Progress in development is similar to progress in healthcare.

• Testing hygiene. Unit testing is similar to the daily routine of brushing ones teeth after each meal. It’s a defacto standard practice. Testing hygiene prevents bugs.
• Code quality hygiene. IDE with code styling enforcement prevent unreadable code from spreading.
• Modularity hygiene. A vast set of principles help keep software modular and prevent coupling and big balls of mud.
• I/O hygiene. Sanitize your inputs and protect yourself from a malicious external environment. Don’t disclose sensitive information to the outside.
• Monitoring hygiene. The first step to stay healthy is to identify problems and remedy before it’s too late. Yearly check-ups are becoming more and more common to detect symptoms of diseases. The same is true for applications, with monitoring, stress tests, and periodic reviews.

Hygiene refers to daily common pratices. Healthcare directly improves with improvement in hygiene, but also needs expert at time. When somebody is sick, he is sent to the hospital to follow a cure. For software, when an unreliable component is detected, it is put in quarantine, then analyzed and fixed with proper tools and techniques.  Expert troubleshooting is the equivalent of surgery.

Finally, what makes software remarkable is that similarly to organisms, software is grown, not built. Also, organisms and software are highly dynamic systems. This makes the analogy even stronger.

Hygienic programmers grow clean code and run healthy software.

MORE

• Software security is like medicine
• http://blog.8thlight.com/uncle-bob/2014/05/02/ProfessionalismAndTDD.html
• http://m.ustream.tv/recorded/61492206 (Second half)
• http://www.laputan.org/selfish/selfish.html#TheSelfishClass
• https://twitter.com/andrestaltz/status/1062976018904956928
• https://dzone.com/articles/software-development-hygiene-why-do-we-brush-our-t

I discussed yesterday with Raffael Krebs, a master student at SCG, who was disappointed of his design in Java that lacked extensibility. The main problem was that he wanted parallel class hierachies, which you can’t do easily in Java.

This problem isn’t new. Let’s illustrate the problem and explore a bit the various design in pure Java, then with existing language extension.

Below is a sample situation. The core of the model is composed of Document, Item, Text and Image. The Reader reads data from a file and creates the model. The client uses the model. To be generic, it must be possible to specialize the model with as little effort as possible, that is, provide reuse in a natural way. One would like conceptually to specialize (that is, subclass) the model for instance for HTML documents.

This conceptual view can not be achieved with Java. There are several problems: (1) multiple inheritance is not supported, (2) the Reader class still references the core model and will not create instance of the more specialized one, and (3) in most statically-type languages, parameters in a subclass must be covariant to those in its superclass.

The two first problems can be circumvented with interfaces and factories. The third one might require a downcast in add(), if specialized classes have added methods. If only existing methods have been overriden, traditional polymorphism is enough.

Generics provide a partial solution to the problem. Document can accept a parameteric type <A extends Item>, so that HtmlDocument can be defined as an extension of Document<HtmlItem> (see this answer for an example). The downcast in method add() disappears. Item is however abstract, and the system assumes there are two implementations of it, Text and Image. With generics, the class Document can not create instances of these with regular instantiations new Text() or new Image(). First it would not comply necessary with the parametric type <A extends Item>, second, these instantiations would not be rewired correctly to their corresponding class in the other family. If classes were first-class, one could at least define abstract static factory methods createText() and createImage() in Item. This is however not  possible in Java, and static method can not be overriden, because classes are not first-class.

In a dynamicaly typed language, the usage of interface vanished, as well as the third issue. However, in both case problem (1) and (2) remain.

The lack of multiple inheritance in this case leads to code duplication between the classes for which not inheritance relationship exist. To avoid such code duplication, delegation could be used and an instance of HtmlText could delegate to an instance of BaseText. As pure delegation is not supported, this require writing boilerplate forwarding methods, which is still clusmy. Another way to avoid code duplication would be to factor the duplicated code into traits or mixin that can be reused among unrelated classes.

No matter what, this design is ugly–and it is really found in practice.

Revisiting the three problems

The  problems discussed before can be rephrased into three separated questions:

– How can the reader produce the right kind of document (either basic, or html) without invasive changes?

– How can variations of a class hierarchy be created with minimal efforts and no boilderplate code ?

– How can the type system be aware of logical relationship between types, e.g. HtmlDocument always goes with HtmlItem?

The fourth problem

One fourth problem not depicted but that happens in practice, is that two class hierarchy relate to each other, but in slightly incompatible ways. This typically happens due to software evolution, when method are renamed, signature are changed, etc. An example would be to change the “data” in Item from binary byte[]  to base64 string. While certain kinds of changes can be accomodated by providing convertion functions (they act as wrapper), this bloats the design and prevent clean evolution.  Sometimes it is impossible to create an inheritance relationship between the two variants of the classes, which  means that whole graphs must be adapted back and forth from one representation to the other. We state the fourth problem as follows:

– How can graph of objects be converted from one logical representation to another easily, whithout that the convertion logic bloats the code?

The fifth problem

One fifth problem not discussed so far is actually a requirement:

– How can the new families be introduced a posteriori, without entailing modification or invalidation of exisitng code.

This problem is usually referred to as modularity.

Language constructs

These problems are clearly related to each others, as is showed by the example.  A definitive solution to all three of them is to my knowledge still not available, but there has been progress in programming language to provide means to tackle part of them.

• Design patterns (strategy, factory)
• Generics
• Static & dynamic reuse mechanisms (delegation, traits, talents)
• Family polymorphism
• Virtual classes (newspeak/gbeta, dependency injection, first-class class)
• Object Algebras
• Local rebinding (classbox)
• Open classes
• Multi-dimensional dispatch (subject-oriented programming, context-oriented programming, worlds, namespace selector)
• Non-standard type coercion (translation polymorphism in Object Team & lifted Java, expander)
• Type versioning (type hot swapping, upgradeJ, revision classes)

It seems to me there is room for something actually missing that would solve the problem depicted above–or myabe it exist but I am not aware of it?

Links

• OOP parallel class hierachies on LtU

If are bad, if are evil. If are against object-oriented programming. If defeats polymorphism — These popular remarks are easier said than enforced.

Ifs can pop up for various reasons, which would deserve a full study in order to build a decent taxonomy. Here is however a quick one:

• Algorithmic if. An algorithmic if participate in an algorithm that is inherently procedural, where branching is required. No much can be done for these ones. Though they tend to increase the cyclomatic complexity, they are not the most evil. If the algorithm is inherently complex, so be it.
• Polymorphic if. A class of object deserve some slightly different treatment each time. This is the polymorphic if. Following object oriented principles, the treatment should be pushed in the corresponding class, and voila! There are however million of reasons why we don’t want the corresponding logic to be in the class of the object to treat, defeating object oriented basics. In such case, the problem can be alleviated with visitor, decorator, or other composition techniques.
• Strategic if. A single class deals with N different situations. The logic is still essentially the same, but there are slight different in each situation. This situation can be refactored with an interface, and several implementations. The implementations can inherit from each other, or use delegation to maximize reuse.
• Dynamic if. Strategic if assumes that the situation doesn’t change for the lifetime of the object. If the behavior of the object needs to change dynamically, the situation becomes even more complicated. Chances are that attributes will be used to enable/disable certain behavior at run-time. Such if can be refactored with patterns such as decorators.
• Null if. Test for nullity is so common that it deserves a special category, even though it could be seen as a special case of another category.  Null if can happen to test the termination of an algorithm, the non-existence of data, sanity check, etc. Various techniques exist to eradict such if depending on the situation: Null Object pattern, add as many method signature as required, introducing polymorphism, usage of assertions, etc.

A step-by-step Anti-If refactoring

Here is a step-by-step refactoring of a strategic if I came accross . I plan to send it to the anti-If compaign and took then the time to document it. The code comes from the txfs project.

Let’s start with the original code:

public void writeFile (String destFileName, InputStream data, boolean overwrite)
throws TxfsException
{
FileOutputStream fos = null;
BufferedOutputStream bos = null;
boolean isNew = false;
File f = new File (infos.getPath (), destFileName);
if ( !overwrite && f.exists () )
{
throw new TxfsException ("Error writing in file (file already exist):" +
destFileName);
}

try
{
if ( !f.exists () )
{
isNew = true;
}
DirectoryUtil.mkDirs (f.getParentFile ());
try
{
Copier.copy (data, f);
}
finally
{
if ( isNew && isInTransaction () )
{
addCreatedFile (destFileName);
}
IOUtils.closeInputStream (data);
}
}
catch ( IOException e )
{
throw new TxfsException (“Error writing in file:” + destFileName, e);
}
}

Not very straightforward, isn’t it? The logic is however quite simple: if the overwrite flag is set, file can be written even if it already exists, otherwise an exception must be thrown. In addition to that, if a transaction is active, the file must be added to the list of created file, so that they can be removed in the transaction is rolled back later.  The file must be added even if an exception occurs, for instance if the file was partially copied.

What happens is that we have two concerns: (1) the overwrite rule and (2) the transaction rule.

Let’s try to refactor that with inheritance. A base class implements the logic when there is no transaction. And a subclass refines it to support transactions.

public void writeFile (String destFileName, InputStream data, boolean overwrite)
throws TxfsException
{
FileOutputStream fos = null;
BufferedOutputStream bos = null;
boolean isNew = false;
File f = new File (infos.getPath (), destFileName);
if ( !overwrite && f.exists () )
{
throw new TxfsException ("Error writing in file (file already exist):" +
destFileName);
}

try
{
// if ( !f.exists () )
// {
//    isNew = true;
// }
DirectoryUtil.mkDirs (f.getParentFile ());
try
{
Copier.copy (data, f);
}
finally
{
// if ( isNew && isInTransaction () )
// {
//     addCreatedFile (destFileName);
// }
IOUtils.closeInputStream (data);
}
}
catch ( IOException e )
{
throw new TxfsException (“Error writing in file:” + destFileName, e);
}
}

The transaction concern is removed from the base method. The overridden method looks then like:

public void writeFile (String destFileName, InputStream data, boolean overwrite)
throws TxfsException
{
try
{
super.writeFile( destFileName, data, overwrite );
}
finally
{
addCreatedFile (destFileName);
}
}

But we have then two problems: (1) we don’t know if the file is new, and it’s always added to the list of created file. (2) if the base method throw an exception because the file already exists and the flag is false, we still add it to the list of created file when we shouldn’t.

We could change the base method to have a return code (e.g. FileCreated and NoFileCreated). But return code are not generally a good solution and are quite ugly.

No, what we must do, is remove some responsibility to the method. We then split it into two methods createFile and writeFile. One expects the file to not exsits, the other the file to exists.

void writeFile (String dst, InputStream data ) throws TxfsException
void createFile (String dst, InputStream data ) throws TxfsException

(The method writeFile which takes the additional overwrite flag can be composed out of the two previous one )

So our simplified writeFile method looks like:

public void createFile(String destFileName, InputStream data)
throws TxfsException
{
try
{
super.createFile(destFileName, data);
}
finally
{
// we add the file in any case
addCreatedFile(destFileName);
}
}

Alas, there is still the problem that if super.writeFile fails because the file already exists, we add it to the list.

And here we start to realize that our exception handling scheme was a bit weak. The general TxfsException is rather useless: we need to refine the exception handling to convey sufficient information to support meaningful treatment.

void writeFile (String dst, InputStream data )
throws TxfsException, TxfsFileDoesNotExsistException

void createFile (String dst, InputStream data )
throws TxfsException, TxfsFileExistsAlreadyException

Here is then the final code:

public void createFile(String destFileName, InputStream data)
throws TxfsFileAlreadyExistsException, TxfsException
{
// I can not use a finally block because
// if failure happens because file already existed       
// I must not add it the list
try {
super.createFile(destFileName, data);
// we add the file if creation succeeds
addCreatedFile(destFileName);
}
catch (TxfsFileAlreadyExistsException e)
{
// we don't add the file if it already exists
throw e;
}
catch (Throwable e)
{
// we add the file in any other case
addCreatedFile(destFileName);
}
}
}

Conclusion

Anti-If refactoring is not so easy. There are various kind of ifs, some of which are ok, and some of which are bad. Removing ifs can imply changing the design significantly, including the inheritance hierarchy and the exception hierarchy.

A tool to analyze and propose refactoring suggestion would be cool, but for the time being, NoIF refactoring is probably in the hands of developers which can ensure the semantics equivalence of the code before and after the refactoring.

Links

http://www.technology-ebay.de/the-teams/mobile-de/blog/an-exercise-in-unconditional-programming.html