Developing Software Together

Refactoring: Primitive Obsession

2019-02-18

“How did you refactor that?”

After my video about “Primitive Obsession”, a viwer asked me on Twitter how exaclty I refactored the code towards the final result.

Which steps did I take? Which transformations did I perform in the code?

In this video, I want to show you how I would refactor this piece of code to get rid of the primitive obsession.

Intro (TL;DR)

My name is David and I am a trainer and technical agile coach with over 12 years of experience.

Today I want to show you how to refactor some code that over-uses primitive data types. I will not talk about why this is a problem - I already did this in the video “Primitive Obsession”, and I have added a link to that video in the description.

Now, I only want to get rid of the problem.

So, let’s look at the original code again.

By the way, if you like this video, please subscribe and share it with your friends and followers. That would be awesome.

Question / Problem

It is a controller from an application that helps people pay their taxes. The controller gets a tax account number, validates it, reads the tax office ID from it and then displays information about the tax office.

If the first two digits of the tax account number - which are the tax office ID - are “46”, it will display “Linz” as the tax office name and the IBAN of the tax office in Linz.

I have added some more code compared to my last video. I have added a view for this controller. It is mostly empty, but here you can see how the controller would be called from the view.

The controller registeres a TaxAccountNoChangedListener with the view, and every time the text field changes, the view will call all these listeners. Then, the method onTaxAccountNoChanged of the controller will do it’s thing, and this is the method I want to refactor.

In addition to what I showed in the last video, I have added a constructor with dependencies. And I have added some more validation: The method now outputs an error when the tax office was not found.

public void onTaxAccountNoChanged(String taxAccountNo) {
    String errorMessage = TaxAccountNoUtil.validateTaxAccountNo(taxAccountNo);
    if(errorMessage != null) {
        payTaxesView.showTaxAccountNoValidationError(errorMessage);
    } else {
        String taxOfficeIdInput = taxAccountNo.substring(0, 2);
        Integer taxOfficeId = Integer.valueOf(taxOfficeIdInput);

        TaxOffice taxOffice = taxOfficeRepository.findByNumber(taxOfficeId);
        if(taxOffice == null) {
            payTaxesView.showTaxOfficeError("The tax office \""+taxOfficeIdInput+"\" does not exist.");
        } else {
            String taxOfficeName = taxOffice.getName();
            String taxOfficeIban = taxOffice.getIban();

            payTaxesView.showTaxOfficeName(taxOfficeName);
            payTaxesView.showTaxOfficeIban(taxOfficeIban);
        }
    }
}

I needed this to be able to add some tests. And I needed some automated tests to show me that I did not break anything.

Remember: Refactoring means changing the structure of the code without changing the functionality. It means applying small, safe transformations to the code. And then, after every small step, I must check whether I did not change the functionality.

Yes, I can never proof or be 100% sure that I did not change any functionality. But when I work in small, safe steps and use the compiler and some automated tests to verify every step, I can be pretty sure. I would even argue that when I did not do that - work in small, safe steps and verify every step - then it would not really be refactoring, just some changes to the code.

So, I needed those tests before refactoring but how I wrote them is not important right now. That’s maybe a topic for a later video.

Solution

So, small, safe steps. And I want to get rid of the primitive data types.

As a first step, I introduce an object for taxAccountNo. I give it an ugly name for now, to remind me that I am not done here: I want to later remove the string from most parts of the code.

I press Alt+Enter to and let my IDE create the field automatically. I also create a getter (by pressing Alt+Enter again), but I rename it to “asString” because this is the string representation of that object. I do not use the name “toString” here because toString should be human readable and is not supposed to be used by other code.

Now I replace all other uses of the method argument taxAccountNo with my newly created object. The tests still pass. Now I rename the parameter and the variable so that their names indicate better what they represent. Now, the parameter gets the ugly name because it contains the “uglier” value.

I press Ctrl+Alt+Shift+T here to open the “Refactor This” menu. I am sure other IDEs have similar functionality.

public void onTaxAccountNoChanged(String taxAccountNoUnvalidatedUserInput) {
    TaxAccountNumber taxAccountNo = new TaxAccountNumber(taxAccountNoUnvalidatedUserInput);

    String errorMessage = TaxAccountNoUtil.validateTaxAccountNo(taxAccountNo.asString());
    if(errorMessage != null) {
        payTaxesView.showTaxAccountNoValidationError(errorMessage);
    } else {
        String taxOfficeIdInput = taxAccountNo.asString().substring(0, 2);
        Integer taxOfficeId = Integer.valueOf(taxOfficeIdInput);

        TaxOffice taxOffice = taxOfficeRepository.findByNumber(taxOfficeId);
        if(taxOffice == null) {
            payTaxesView.showTaxOfficeError("The tax office \""+taxOfficeIdInput+"\" does not exist.");
        } else {
            String taxOfficeName = taxOffice.getName();
            String taxOfficeIban = taxOffice.getIban();

            payTaxesView.showTaxOfficeName(taxOfficeName);
            payTaxesView.showTaxOfficeIban(taxOfficeIban);
        }
    }
}

Now I want to move the validation into a factory method of TaxAccountNo since it should never be possible to create a valid object from an invalid input. I extract a static factory method from the constructor call and then I move this method to the TaxAccountNo class.

I will move the validation code into this method soon, but first I want to create a second variable for the sanitized input that will be available after validation.

public void onTaxAccountNoChanged(String taxAccountNoUnvalidatedUserInput) {
    TaxAccountNumber taxAccountNo = TaxAccountNumber.fromString(taxAccountNoUnvalidatedUserInput);

    String errorMessage = TaxAccountNoUtil.validateTaxAccountNo(taxAccountNo.asString());
    if(errorMessage != null) {
        payTaxesView.showTaxAccountNoValidationError(errorMessage);
    } else {
        String taxOfficeIdInput = taxAccountNo.asString().substring(0, 2);
        Integer taxOfficeId = Integer.valueOf(taxOfficeIdInput);

        TaxOffice taxOffice = taxOfficeRepository.findByNumber(taxOfficeId);
        if(taxOffice == null) {
            payTaxesView.showTaxOfficeError("The tax office \""+taxOfficeIdInput+"\" does not exist.");
        } else {
            String taxOfficeName = taxOffice.getName();
            String taxOfficeIban = taxOffice.getIban();

            payTaxesView.showTaxOfficeName(taxOfficeName);
            payTaxesView.showTaxOfficeIban(taxOfficeIban);
        }
    }
}

I want to change the error handling to Exceptions, so I first duplicate the validation and add an exception.

public static TaxAccountNumber fromString(String taxAccountNoUnvalidatedUserInput) throws TaxAccountNumberFormatException {
    String taxAccountNoSanitized = sanitize(taxAccountNoUnvalidatedUserInput);

    String errorMessage = TaxAccountNoUtil.validateTaxAccountNo(taxAccountNoSanitized);
    if(errorMessage != null) {
        throw new TaxAccountNumberFormatException(errorMessage);
    }
    return new TaxAccountNumber(taxAccountNoSanitized);
}

I could later inline the validation method from the util class; and if this was the last method on that class, I could remove the util class completely. But that would require some changes to the validate method.

public void onTaxAccountNoChanged(String taxAccountNoUnvalidatedUserInput) {
    try {
        TaxAccountNumber taxAccountNo = TaxAccountNumber.fromString(taxAccountNoUnvalidatedUserInput);
        String taxOfficeIdInput = taxAccountNo.asString().substring(0, 2);
        Integer taxOfficeId = Integer.valueOf(taxOfficeIdInput);

        TaxOffice taxOffice = taxOfficeRepository.findByNumber(taxOfficeId);
        if(taxOffice == null) {
            payTaxesView.showTaxOfficeError("The tax office \""+taxOfficeIdInput+"\" does not exist.");
        } else {
            String taxOfficeName = taxOffice.getName();
            String taxOfficeIban = taxOffice.getIban();

            payTaxesView.showTaxOfficeName(taxOfficeName);
            payTaxesView.showTaxOfficeIban(taxOfficeIban);
        }
    } catch (TaxAccountNumberFormatException e) {
        payTaxesView.showTaxAccountNoValidationError(e.getMessage());
    }
}

I want to continue with the onTaxAccountNoChanged - method for now. The next type I want to get rid of is the taxOfficeId. I basically apply the same steps again: I create a new class, extract a factory method and move that method. But now I also extract another method, getTaxOfficeId and move it to TaxAccountNo.

public class TaxOfficeId {
    private final Integer taxOfficeId;

    private TaxOfficeId(Integer taxOfficeId) {
        this.taxOfficeId = taxOfficeId;
    }

    public static TaxOfficeId fromString(String taxOfficeIdInput) {
        Integer taxOfficeId = Integer.valueOf(taxOfficeIdInput);
        return new TaxOfficeId(taxOfficeId);
    }

    public String asString() {
        return String.format("00",taxOfficeId);
    }

    public Integer asNumber() {
        return taxOfficeId;
    }

    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;
        TaxOfficeId that = (TaxOfficeId) o;
        return Objects.equals(taxOfficeId, that.taxOfficeId);
    }

    @Override
    public int hashCode() {
        return Objects.hash(taxOfficeId);
    }
}

I can now use the new class TaxOfficeId in the query method of the repository. But now I also have to change a test, since the old method is not called anymore on the mock object.

public void onTaxAccountNoChanged(String taxAccountNoUnvalidatedUserInput) {
    try {
        TaxAccountNumber taxAccountNo = TaxAccountNumber.fromString(taxAccountNoUnvalidatedUserInput);
        TaxOfficeId taxOfficeId = taxAccountNo.getTaxOfficeId();

        TaxOffice taxOffice = taxOfficeRepository.findBy(taxOfficeId);
        if(taxOffice == null) {
            payTaxesView.showTaxOfficeError("The tax office \""+taxOfficeId.asString()+"\" does not exist.");
        } else {
            String taxOfficeName = taxOffice.getName();
            String taxOfficeIban = taxOffice.getIban();

            payTaxesView.showTaxOfficeName(taxOfficeName);
            payTaxesView.showTaxOfficeIban(taxOfficeIban);
        }
    } catch (TaxAccountNumberFormatException e) {
        payTaxesView.showTaxAccountNoValidationError(e.getMessage());
    }
}

The last step now is to create new classes for the tax office name and the IBAN. Then I can rename the methods for diplaying to only show. But I also have to change the tests again.

@Test
public void showsTaxOfficeInformationIfTaxOfficeWasFound() {
    TaxOffice taxOffice = mock(TaxOffice.class);
    when(taxOffice.getName()).thenReturn(new TaxOfficeName("Linz"));
    when(taxOffice.getIban()).thenReturn(new TaxOfficeIban("AT03 0100 0000 0552 4464"));
    when(taxOfficeRepository.findBy(TaxOfficeId.fromString("46"))).thenReturn(taxOffice);

    controller.onTaxAccountNoChanged("46-999/9460");

    verify(payTaxesView).showTaxOfficeName(new TaxOfficeName("Linz"));
    verify(payTaxesView).showTaxOfficeIban(new TaxOfficeIban("AT03 0100 0000 0552 4464"));
}

I can now, as a final step for this video, move some value classes to the class TaxOffice and give them nicer names.

Conclusion

Now look at the code. This does look nicer than before.

I could refactor even further. For example, the classes for the value objects all have very similar functionality. They all wrap a single primitive value. They all have a hashCode and equals method and they look very similar.

I am sure I could unify or simplify some things here. But for now, I am done here.

CTA

What do you think of these refactoring steps? What would you have done differently? Please tell me in the comments or on Twitter, I am @dtanzer there.

And if you liked this video, please subscribe and share it with your friends and followers using the share buttons below - That would be awesome!

Improve Retrospectives

2019-02-11

Quick show of hands: Who here likes retrospectives? Who loves them?

Many people I have met so far would have kept their hands down. They would rather work on the next feature than sit in a retrospective.

That’s because, in many teams, retrospectives are boring and ineffective.

Intro (TL;DR)

In many teams, retrospectives are boring and ineffective. But that does not mean that it has to be like that.

If this is your team, change something. Working on having better retrospectives is one of the cheapest things you can do to improve your overall situation.

Some time ago, I wrote a book about “Agile Anti-Patterns”. And most of the problems I described there could be cured by better retrospectives. At least to some extent.

By the way, there is a link to the book in the description of this video!

And “better”, or “good”, in this case means that

  1. The retrospective is not boring
  2. People talk about the “big”, “systemic” problems
  3. The retrospective results in real, positive changes

But before I’ll talk about those three points, let’s look at the problem again, in a bit more detail.

Question / Problem

Imagine a team that is doing some “agile” “process”. Like, Scrum, or whatever. Every two weeks, the team has a “Retrospective”.

But those retrospectives are boooring. Nobody likes them. People would rather work on their next task than be sitting in this meeting.

And nothing ever really changes - at least, nothing important. Yes, the team records action items at the end of each retrospective. But sometimes they don’t get done at all. Sometimes they get done, but have no visible effect. And sometimes they did solve the problem, but it does not matter because the problem was very small and unimportant.

Now imagine a team that does not do any retrospectives at all. Which team is using their time more efficiently?

The second team, of course. The first team not only wasted the time it took to have the retrospective and to work on the action items, it also had a meeting that demotivated everyone who was there. Those people will probably not go back to their desks afterwards and work on happily and fully motivated.

Does this mean you should stop doing “Retros”? NO! But if there’s one thing you could do now to improve your way of working, then look at your retrospectives and try to improve them.

Solution

To get better, your retrospectives should not be boring, they should tackle the big problems and they should result in real, lasting change.

To avoid that your retrospectives become boring,

  1. Keep them short
  2. Make sure everyone is involved and
  3. Have some variety in how you do them

Keep the meeting short - but not too short. To facilitate a good retrospective with actionable results, you will probably need 90 minutes. Longer than that will be too long for a sprint retrospective, but longer retrospectives might be necessary at the end of a quarter, year or project.

Maybe you can do a great retro in 60 minutes sometimes, but shorter than this is definitely too short. I would even consider 60 minutes too short for most teams, most of the time.

At the beginning, make sure that everyone will be involved. Try to get everyone to say something within the first few minutes, otherwise some might not say anything during the retrospective at all.

Get the book “Agile Retrospectives: Making Good Teams Great” by Esther Derby and Diana Larsen and also have a look at the “Retromat” by Corinna Baldauf. Both will teach you activities that will help you get everyone involved at the beginning.

Also, both follow a structure with 5 distinct phases and activities for each phase. Following this structure will help you to add some variety to your retrospectives, so they will be less boring.

But even when your retrospectives are not boring and when everyone is involved, the time will be wasted when you do not tackle important problems or when you spend the retrospective finding out whom to blame. So,

  1. Remind people that we believe performance is mostly determined by the system
  2. Create an environment where everyone feels safe
  3. Spend time not only looking at symptoms, but also at root causes.

The “retrospective prime directive” is one way of reminding people that we are not here to assign blame, but to improve the system. It says:

"Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand."

Norm Kerth, Project Retrospectives: A Handbook for Team Review

But do not only read it out loud, talk about it. Talk about what it means and what it means to you, as a team, for this retrospective.

You need an environment where everyone feels safe. Make sure people know that, whatever they say, there will be no finger pointing, no loughing and no shouting. Let them know that in this meeting, they can talk about everything and point out any problems - and that this will not have negative consequences for them in the future.

And make sure that nobody ever breaks these rules. At one past client, an influential team member shouted at the team during a retrospective once. Called them unprofessional. Threatened to have some fired. Every retro after that was a complete waste of time because nobody would ever talk about the interesting problems again.

Also, do not only look at symptoms and simple solutions. It will not help to treat the brain tumor only with aspirin to make the headache go away. Before brainstorming possible solutions, dig deeper and try to find the real problem.

The retrospective should also result in real, lasting change.

If your retrospectives do not yield some visible results, the other two qualities - not boring, tackles big problems - won’t rescue your retros. If there are no results, people will still consider the meeting a waste of time, and it will become boring and shallow again.

To get started,

  1. Create less action items at the end
  2. Define success criteria and measure success before the next retrospective
  3. Use your influence or find people who can help you

Create less action items. Why do I want you to try that? Because with less action items, chances are higher that they will be done within reasonable time.

Some teams identify five problems, then 5 ways to tackle the problem for each. And at the start of the next retrospective, none of the 25 action items are “done”.

So, choose one problem. Define one or two experiments - things you want to try to solve the problem or to make your life a little bit better. Then make sure they are done next time to meet.

Also, define success criteria for these experiments. And say when you want to measure success. In two or four weeks, or whenever you want to measure, is not enough that the experiment was “done”. You also want to know whether it was successful and what exactly has changed.

Sometimes, especially in large companies, all the interesting problems cannot be solved by the team alone. They are caused by the processes, the rules and the culture of the oranization.

But just because you cannot solve those problems alone does not mean you should not try to solve them. Find people who can help you. Or find people who can help you find people who can help you. Make your problems and impediments known.

Conclusion

When you have retrospectives that are not boring because

  1. They are short
  2. Everyone is involved and
  3. There is some variety in how you do them

When you have retrospectives that tackle the big, systemic problems because

  1. People know that performance is mostly determined by the system
  2. The retro happens in an environment where everyone feels safe
  3. People spend time not only looking at symptoms, but also at root causes

When you have retrospectives that result in real, positive changes because

  1. The team creates less action items at the end and makes sure they get done
  2. The team defines success criteria and measure success before the next retrospective
  3. The team uses their influence to find people who can help them

People will start to enjoy retrospectives again. They will participate more, and the whole team will benefit.

CTA

Tell me in the comments how your retrospectives work for your team, and what could be improved about them. Or shoot me a Tweet to @dtanzer, if you prefer discussing on Twitter.

And if you liked this video, please share it with your friends, followers and colleagues (e.g. with the share buttons below)!

Primitive Obsession

2019-02-04

Did you ever see some piece of code that uses primitive data types, like Strings or integers, in it’s public API? I am sure you have, because most code is written like that.

But that can cause problems. Today, I want to talk about a code smell called “Primitive Obsession”. And I want to show you how code often becomes easier to understand and “nicer” by eliminating primitive data types - especially, but not only, from it’s public API.

Intro (TL;DR)

My name is David and in this video series, I am talking about things you can do to become a better software developer.

I am a freelance consultant and coach - and in the last 12 years I have strived to constantly learn and improve my skills as a software developer, architect, and also as a tester, team coach and trainer.

I have learned that improving your software design is crucial for delivering better software faster and cheaper. Having a low internal quality in your software costs you a lot, and in many different ways.

Today I want to talk about a very specific design smell: Primitive Obsession. You can notice this smell when your code uses a lot of primitive data types, like integers or booleans but also strings and dates, especially in its public API.

Primitive Obsession is a more general version of “Stringly Typed Code”, a code smell where strings are used in places where other data types would be more appropriate.

Question / Problem

How does primitive obsession look like and what exactly is the problem? Consider the following piece of code…

This code comes from a hypothetical software that helps me pay my taxes in Austria. In a first step, users would input their tax account number.

The software would first verify that the tax account number is correct and also load some information about the tax office, which the software will display to the users. And because we are only selling to Austrian customers and only support Austrian tax account number, it can use the first two digits - the tax office number - for doing so.

46-999/9460

The tax account number looks like this: The first two digits are the tax office number. In this example, 46 means “Linz”. The next six digits, ususally written with a slash in the middle, are the account number within this tax office. And the last digit is a checksum.

The example code gets the tax account number that was entered by the user, validates it, extracts the tax office number and loads the tax office information. If everything worked out correctly, it displays the tax office information. Otherwise, it outputs an error.

For some reason, maybe a performance optimization or because it is required by some backend, the tax office number string is also converted to an integer. If you think, right now, “Nobody would ever do that”: I have seen very similar things at multiple past clients. Well, maybe a little bit less obvious stuff. And not with tax account numbers, but with other structured data.

public void onTaxAccountNoChanged(String taxAccountNo) {
    String errorMessage = TaxAccountNoUtil.validateTaxAccountNo(taxAccountNo);
    if(errorMessage != null) {
        showTaxAccountNoValidationError(errorMessage);
    } else {
        String taxOfficeIdInput = taxAccountNo.substring(0, 2);
        Integer taxOfficeId = Integer.valueOf(taxOfficeIdInput);

        String taxOfficeName = taxOfficeRepository.findByNumber(taxOfficeId).getName();
        showTaxOfficeName(taxOfficeName);
    }
}

The first version of the code

This code uses primitive data types - Strings and Integers - all over the place. validateTaxAccountNo returns the error message as a string. The taxAccountNo is a string. We get the tax office number as a string and convert it to an integer. The tax office name is returned as a string by taxOffice.getName()

Use of primitive values in this code

But the code is reasonably short and easy to understand, so where is the problem?

First, these data types are too general. They allow more operations than should be possible with an tax account number or a tax office number. Integer allows us to add, substract, … and also to compare to other numbers. All of that should never be possible for a tax office number.

On the tax account number, we can again do operations that should not be possible, like calling the string manipulation functions, searching inside, and so on.

If you think, right now, “nobody would ever do that”, think again.

Second, the use of types does not add clarity or allow you to remove duplication. It’s even the other way around: To add functionality around tax account numbers, we needed a class called TaxAccountNoUtil. Now there are two places in the code that know about tax account number. We have duplicated the knowledge about tax accounts.

And third, you are voluntarily giving up compile time safety. Suppose you see a method call like this:

A function call - Which parameter is which argument?

When the first two parameters are integers and the other three are strings, nothing is stopping a programmer from supplying them in the wrong oder. When some programmer decides to re-order the arguments or add another argument, it will be really hard to find all the effected places.

The compiler could help you detect defects like that. But in this case, it cannot because you are using primitive data types.

Solution

As a first step, I will do the most simple thing. I will introduce types for TaxAccountNumber, TaxOfficeId and TaxOfficeName - and there is already a class for TaxOffice. They only wrap the primitive value - They do not do anything in addition.

public void onTaxAccountNoChanged(String taxAccountNo) {
    try {
        TaxAccountNumber taxAccountNumber = TaxAccountNumber.fromString(taxAccountNo);

        String taxOfficeIdInput = taxAccountNumber.asFormattedString().substring(0, 2);
        TaxOfficeId taxOfficeId = TaxOfficeId.fromString(taxOfficeIdInput);

        TaxOfficeName taxOfficeName = taxOfficeRepository.findBy(taxOfficeId).getName();
        showTaxOfficeName(taxOfficeName);
    } catch (TaxAccountNumberFormatException e) {
        showTaxAccountNoValidationError(e.getMessage());
    }
}

Even though this was just an intermediate step, I already made some progress. The fromString method of TaxAccountNumber should never return an invalid object. So I had to move the validation there. That’s what J.B. Rainsberger calls “Attractive Code” - Once you have a place where you can put code for a certain concept, this place will attract other code.

This method can still use the old TaxAccountNoUtil class, but I will later even inline the methods from there and remove TaxAccountNoUtil. Another win: We should not have “Util” classes anyway.

And if you do not like that exception being thrown, there would be other solutions. But that would be too much change for this video.

I now have types for TaxAccountNumber and TaxOfficeId, and so far, they only wrap the primitive type. But they already give me two other advantages:

  • I can make the constructor private, forcing everyone to use the factory methods. There I can do very specialized validation, depending on which factory method was called. Also, the factory methods allow me to give a “name” to the constructor, which would otherwise be impossible in Java.
  • I can make the wrapped primitive final. The classes now start to represent what “Domain Driven Design” calls a value object.

Private constructor, immutable fields

Now I have my attractive code. What else could I do with that?

What I will do now is “just” cleanup. I can remove the redundant calls to “fromString” and “asFormattedString” and move this code to the specialized types. And since I now also have a type for the tax office name, I do not have to name the mehtod for showing it showTaxOfficeName. I can name it showNameOf and rely on the fact that the compiler will select the right method.

Final result

Conclusion

The code now is shorter and easier to read.

It has added compile time safety: It is now impossible to supply the wrong number to findBy or to call the wrong method for showing the name. The compiler even forces me to deal with the validation error that I could have ignored in the first version of the code.

Yes, I wrote a lot of code around this method, so the overall amount of code was increased. That took me maybe half an hour.

But this code was easy to write. And it allows me to encapsulate the data better. It allows me to make all these objects immutable - that is, read-only - which makes writing correct correct easier.

The code also added clarity to onTaxAccountNoChanged which will be the first method a programmer will see when they debug a problem with tax account numbers later. If I can save the next programmer a few minutes of debugging time, and if I maybe can prevent a simple parameter ordering mistake, saving another programmer a few minutes, this change was already worth it.

CTA

Have you ever seen code with primitive obsession? What about the software you are working on right now - What are the worst places regarding to this, and did it ever cause a (small or larger) problem?

Or, do you already create classes for value objects and entities, like in the last version of my code? How does that feel? Which advantages does it give you?

Tell me in the comments or shoot me a Tweet to @dtanzer. And do not forget to subscribe to this channel or follow me on Twitter, so you’ll never miss any updates.

Object-Oriented Design Pitfalls

2019-01-28

Do you remember how you first learned about object-oriented software design? How they teach it at University?

You probably looked at objects from the real world and their is-a or extends relationships. A circle is-a shape, and it extends shape. A cat is-an animal, and it extends animal.

Today I want to re-visit this kind of approach to learning object-oriented design and discuss it’s advantages and shortcomings.

Intro (TL;DR)

My name is David and in this video series, I am talking about things you can do to become a better software developer.

I’ve been working as a freelance developer, technical coach and trainer for some time now. I am helping my clients - among other things - to handle their legacy code bases better.

And many problems I see in those legacy code bases are caused by not putting enough emphasis on software design. By taking a simplistic approach, almost as simplistic as the one described before.

Like, find the nouns and verbs in the specification and that’s basically the design.

So, when we talk about software design in the workshops I teach, I also spend some time trying to show people that the “first-gut-feeling” approach to design does not work.

Because when you know why and how this approach does not work, you will understand the techniques you need to learn more easily.

Question / Problem

Some software designs look like someone searched for nouns and verbs in the specification (or the business domain). Then they made classes directly from the nouns and methods from the verbs.

And designs like that seem right at the first glance, because they model the objects from “the real world”.

But then you run into problems. Writing tests is hard. There are a few classes that are huge and do all the work, and other classes that barely do anything at all. There are big domain objects that contain 15, 20 or more fields.

There are services that depend on services that depend on services - many layers down the onion - even though all of them are stateless.

There are classes that have many dependencies which contributes to your testability problem.

The design is neither extensible nor portable. You cannot re-use the business logic in a slightly different situation because of technical limitations. Adding additional features requires you to change many pieces of code in many different locations.

And all of that slows you down. Every. Time. You. Want. To. Change. Something.

What happened here? Well, the “objects from the real world” approach to software design almost inevitably leads to bad design.

Let me give you a little example.

Solution

I want to design a very simple class hierarchy with you. There are Cats and Dogs, and both of them are Animals. An Animal has a position and we can tell it to move to another position.

Animals also have a method speak. When we call this method on an object of type Cat it would output “meow”, when we call it on a Dog it would output “woof”.

Class diagram showing a hierachy of animals with Cat and Dog

But now we got new requirements and we need a new class Worm. “Easy”, we think, it’s also an animal.

But how do we implement “speak” here?

Class diagram showing a hierachy of animals with Cat and Dog

All right, leave it empty. Worms don’t speak. That’s not really nice, because even calling “speak” does not make sense on a worm.

But we can live with that. There is not much harm in leaving a method empty. Right?

Wait, we got new requirements again. And we need another class to support them. We need a Coral.

Now, corals do not speak too, but they also do not move.

We have the same problem as before, but slightly worse: How can we implement moveTo here?

Class diagram showing a hierachy of animals with Cat and Dog

Think about it for a moment… I think in this class hierarchy, there are only two reasonable ways to implement moveTo: Do nothing or raise an error (like, throw an Exception).

What if we did nothing? A programmer would never call moveTo on a Coral, right?

Maybe they wouldn’t, but they might not know that what they have is a Coral. They get an Animal object, call moveTo, at some point later in time call getPosition - and the animal would still be at its old position.

That would be surprising to the programmer. I am almost certain that they would fire up the debugger to find out what’s going on here. This behaviour violates the “Principle of Least Surprise”.

So, it’s “raise an error”, right? Well, not so fast…

Suppose there is some code that iterates over all animals and moves them. This code now does not work anymore.

By adding the coral class the way we did it - By raising an error in a situation that is perfectly fine from the caller’s perspective - We broke some totally unrelated code.

How can adding new code break existing code?

It was possible because we violated the “Liskov Substitution Principle”: We wrote a class that, for some calling function, is not compatible with its base class.

Conclusion

How can we add the coral class without causing any problems? We cannot - because this is the wrong question.

We cannot do it because looking at the “static structure” of the “real world” and creating a software design from it does not work. At least it does not work in most cases.

The objects and relationships we - humans - see are often not the objects and relationships our software is interested in.

In software, we try to implement some behaviour. We want to make computers do something. In order to do that, we must understand and model behaviours from the real world.

And we must take care to only model the behaviours and objects our software needs to do its job. Whenever we model less or more, we make our design needlessly complicated.

The simplistic object oriented design we learn at school - “Cat is-an Animal” or “Car has tires” - is useful for learning to think in objects. But it is not necessarily useful for creating real software systems.

We must do more in order to model those real software systems. We must think about coupling and cohesion. Follow software design principles. Know about design patterns. Know the rules so well that we know when to follow them and when to break them.

But those are topics for upcoming videos. Today I wanted to show you how a too simplistic design can fall short pretty quickly.

CTA

How do you and your team design your software? Do you have a process? Does your design look like “nouns+verbs” or is it more sophisticated?

Did you already experience situations where a design that looked absolutely reasonable stopped working because of new requirements? What are you planning to do next?

Tell me in the comments or on Twitter - I am @dtanzer there. And do not forget to subscribe to this channel, so you do not miss any updates!

The TDD Trap

2019-01-21

Test Driven Development, or TDD, seems to be very popular among some software developers, but most of us are still skeptical and not using it.

Those who use it say that TDD helps them to write better code faster. Those who do not use it often say they have tried it and it does not work.

How can some technique be very helpful to some people and not work at all for others? I think a part of the reason is what I used to call the “mock objects trap”, but now call the “TDD trap”…

Intro (TL;DR)

My name is David and I am coaching and teaching software development teams as a freelancer. In this video series, I want to show you things you can try to become better at developing software - alone and as a team.

You can view the videos on your own. Or you could do so together with others, maybe as an introduction to a learning lunch at your company or to a meetup.

Today I want to talk about test driven development. But I do not want to talk about how exactly it works or how to get started yet - I want to leave that for a future video.

Today I want to talk about how some people love it and some do not even want to try it again.

Those who love it say that it helps them write better code and that they are faster when doing TDD. And because they are faster, they also develop their software cheaper.

Those who do not like it say that they have tried it and it clearly does not work. Writing the tests beforehand slows them down. The existing tests prevent them from making changes. Keeping the tests green over time is a major effort for them.

And I think that, at least some of those people, have fallen into the TDD trap.

Question / Problem

I have seen the end result of this trap happen at different clients, and from the stories I hear, it usually starts somewhat similar to this:

Somebody decides that the team needs more tests. Maybe it’s a manager who sets a KPI or some team member who keeps insisting that some of the bugs would not happen if they had more regression tests.

Most probably, not everyone on the team fully buys into that idea. Everyone starts to write tests, but at least some of the team members only do so because they have to.

So, the team starts writing tests, but they never have a formal or informal training - neither internal nor external - because writing tests seems easy. One can learn JUnit in a few hours.

Some people try TDD. But they also do without formal or informal training, because TDD seems easy. Red, Green, Refactor.

But people do not take extra care to write good tests. They do not put effort into designing their tests;

A) because they have never learned how good tests look like and
B) because testing is still an after-thought for most.

They try TDD, but it does not really work. People have a feeling that it slows them down; writing the tests before the production code is super-hard and refactor? They don’t see anything to refactor after most “Green” steps, so they stop thinking about it at all.

But they produce a lot of tests. A lot of bad tests.

And then, after some months, they want to change something. But the tests get in their way. After every small change, dozens of tests are red. Even after valid changes.

They conclude that “TDD does not work here”. This team has fallen into the TDD trap.

Solution

The underlying cause of the problem here is that neither writing unit tests nor TDD are easy. Those are, like the game “Othello”, “a minute to learn, a life-time to master” - kind of activities.

And, to be honest, there’s a second underlying problem here. But that one would be totally off-topic for this video. Maybe for a later one ;)

Back to the first cause: TDD is rather easy to “learn”. I can explain the basic rules to you in 10 minutes, and then you can start to test-drive some code.

And this is exactly what I do at the beginning of the first day when I teach TDD trainings. So, what do I do with my training attendees for the rest of the two days?

After the ten minutes explanation, people are able to do the red-green-refactor circle.

The TDD cycle with annotations about where to do design

But it feels unnatural for them. They get stuck a lot. They take steps that are too big and then have problems writing the next test. They write themselves into a corner, but they do not want to delete code or tests, so they do not know how to get out again. They write “bad” tests that make their lives harder down the road. And they forget the “refactor” step, so it’s actually red-green-red-green-… for them.

To practice TDD successfully, you need to know more than the basic rules.

You must learn to take really small steps, and to not write more code than what is absolutely necessary. Because whenever you write more code than necessary - even only a little more - the next step gets harder.

You must learn about triangulation and about which next test - out of the list of all possible tests - to write.

You must learn how to use the “specific / generic cycle” to drive your code an design.

Diagram explaining the specific/generic rule

You will rely on your individual judgement a lot, so you have to develop a “gut feeling” for good tests and good code.

And there is a lot more to learn.

Learning all these things takes time and a lot of practice. But with a little bit of guidance from someone who has done it before, you can learn some of them a little bit faster. And you can hopefully avoid some common pitfalls.

Conclusion

And the TDD trap is about those pitfalls.

It happens when people think that TDD is easy. They read about it and think that they can do it too.

They try it and it does not work for them. So, they conclude TDD does not work at all.

And it can be worse. Sometimes they create a mess while using TDD. They write bad tests and they keep all of them. And months later, when they want to refactor something or write a new feature, the bad tests get in their way.

Some of those people start to hate TDD. They write angry blog posts. Others just dismiss it, never use it again and joke about the people who are still using such an “obviously flawed” method.

But it does not have to be like that.

If you have had problems with TDD, or if you are just starting out, try to find people who are using it successfully. Go to user-groups or meetups. Go to conferences. Ask around. Ask on Twitter.

Try to find someone who has “been there” and tell them your story. Ask questions.

This can make learning TDD a much more “painless” experience for you.

CTA

Did you already have a bad experience with TDD? Do you think someone you know fell into the TDD trap? Please tell me in the comments!

Or, if you want a longer discussion, tell me on Twitter: I am @dtanzer there.

And do not forget to subscribe to this channel or follow me on Twitter, so you do not miss any updates.

Slow Down to Move Faster

2019-01-14

Did you ever hear phrases like “Agile is not faster” or “You need to do un-intuitive things to become agile”? Did that make sense to you?

To me, in the beginning, those phrases made no sense at all. But now I think that they are the main reason why so many companies do not get “agile” right. They “do” agile only to become faster. But they do not “get” some of the most important underlying principles.

Today, I want to talk about a very important but also un-intuitive principle: You must slow down to move fast.

Intro (TL;DR)

My name is David and in this video series, I will talk about things you can do to become a better software developer.

I am a freelance consultant and coach - and in the last 12 years I have strived to constantly learn and improve my skills as a software developer, architect, and also as a tester, team coach and trainer.

Many of the things I learned - like today’s topic - were hard to grasp for me. Because some of things we have to do to become better make no sense at all. Until you think hard about them and try them and learn more - And suddenly, those things are perfectly reasonable and the opposite does not make sense anymore.

For example, “Slow down to move faster”.

Today, I want to talk about how agile software development is not “fast”. And how it can still be worthwhile, because it enables you to deliver more value in less time.

Which means that it is faster, although it is slower. Confused? Me too. Let me start again, this time a little bit slower.

Question / Problem

Many things we do in agile software development seem to be slower than in a “traditional project”.

We are doing some planning, some design, some architecture in every iteration. We even change our design and architecture on a regular basis. This definitely adds some overhead.

We take more care when writing code. We write a lot of tests. We run them often. We re-think our design decisions after every green test and refactor often.

We spend a lot of time automating things and testing that automation. We automate user acceptance testing, performance testing, regression testing, deployments and even releases to production.

We work in pairs or even in small groups. Five people working togehter, on the same feature, even the same line of code? That cannot be faster than 5 people working in parallel, not getting in the way of each other.

And yet, often it is.

How can that possibly be true? How can slowing down make us faster?

Solution

Let’s look at two teams. Team “Niagara” works in a waterfall-style, plan-then-do-then-integrate, command-and-control way. People do exactly what they are told or what was written down in the plan.

Team “Gazelle” is more agile. The team experiments a lot, they try to get good feedback early, they release often and take care to do things right and to automate everything.

We ask both teams to deliver 5 features for us: A, B, C, D and E.

Team Niagara first plans the project, then they sketch out their software architecture. They do some project preparation work, like setting up their repositories, build systems, CI server, IDEs and so on.

Then they start to implement the features. First feature A, then B, C, D and E. They integrate and test everything. Things went more or less as planned, so they release to their users.

Chart showing the progress of Team Niagara

It took them 7 weeks.

Team Gazelle, on the other hand, start with a little bit of project planning, software architecture and preparation work. Then they implement a very basic version of Features A and B and they integrate and test all the time. After a week, they release what they have to real users.

Users cannot really use the software productively yet, but they give feedback. And from that feedback, the team learns that users, right now, really need D and E.

So, the team works on D and E next, but they also have to do some project planning, preparation work and software architecture. A week later, they release again.

The users try it, and they like D and E, but they would need a more elaborate version of those two features to use them productively. The team listens to their feedback, and they work even more on D and E.

Also, during that week, users report some defects - they seem to be really trying the software! - so the team fixes those right away.

During the next review, where the team gathers feedback from other stakeholders, some “friendly” users agree to use the software in production now, since it has enough features to provide at least some value.

Also, a user suggests that a feature G would be awesome. And, someone from operations brings some usage statistics and they suggest that the current implementation of feature E will not scale.

Now the team addresses those issues. They completely re-do E and start with G. They fix defects and make some minor corrections to D. A week later, they release again.

And this cycle of implementation, release, feedback and re-planning continues until, after 10 weeks, they are done.

Chart showing the progress of Team Gazelle

So, delivering the software took longer with the agile approach, and the team did not even deliver feature C at all! Was that really better?

Well, the story does not end here. With software, it never ends after the release.

Right after the rollout, team Niagara gets the first phone calls. The software does not scale. They have major slowdowns and production outages.

They have to work night-shifts to prepare a quick fix, and then work even more to create the real solution to the scaling problem. They have to re-architect feature E, which causes the problems. But some design that is in place for feature C makes re-doing E harder, so it takes even longer than at the first time.

While they work on that, bug reports keep rolling in, so the team starts fixing those.

And people complain that the software does not really help them to do their job, because some major parts at the beginning of all workflows are missing. Management decides that the team needs to implement feature G.

But because all that code in place - The fixes for E, all the other features and the hastily-done bugfixes, implementing feature G takes quite long.

After the next hotfix release, the team analyzes some metrics. They realize that only a tiny percentage of the users uses feature C. To make future maintenance easier, they decide to remove a part of C.

Chart showing the PROBLEMS of Team Niagara

While team Niagara did all that, team Gazelle moved on to the next piece of work, for another application. In parallel, they fixed some minor defects and collected feature requests for the next major release.

Conclusion

In the end, they delivered faster and cheaper, they are now ready to start working on the next major release and they were able to provide value in another application while they had some slack.

One could say that team Niagara made mistakes during requirements engineering and software architecture. They should have forseen those things that made them slower in the end!

But there will always be unforseen things. Even the best requirements engineer or software architect cannot anticipate everything that could ever happen.

So, by getting feedback early, being able to react quickly and working hard to have high quality code, design and architecture, the “agile” team was faster.

CTA

In a future episode, I want to talk in more detail about some aspects of this example. And I will try to answer your questions. So ask them right here in the comments or ping me on Twitter - I am @dtanzer there.

And subscribe to this channel so you don’t miss any updates!

Learning Lunch

2018-12-28

How can you, as a team, start with continuous improvement when your company does not want to give you enough time? And if they do not give you a budget to spend?

A learning lunch - where, once a week, you use your lunch time to learn something new - can be a great starting point. And it requires only a small investment of your team members’ personal time and money.

Intro (TL;DR)

My name is David and in this video series, I will talk about things you can do to become a better software developer. I will keep the videos short so you can watch them while commuting or during a break. Either alone or with your team.

I have been a freelance consultant, coach, trainer and developer for more than 12 years now. And in those 12 years, I have learned a lot - mostly in my spare time.

And I am still learning. But now I have a family and learning or working in my spare time is not feasable anymore. Also, I think nobody should be required to learn in their spare time.

So how can you learn and improve continuously - alone or as a team - without “wasting” your precious free time?

Today I want to show you one technique that could work for you. Today I want to talk about the “learning luch”. Try to spend one lunch break every week or every two weeks learning new things together with your team. And convince your manager to “pay” for half of the time.

Question / Problem

If you want to get good at developing software - as a developer, but also as a team - you have to learn a lot. I mean, a lot! When I started collecting topics for this video series, I was baffled by the number of things I want to talk about. And I am still in the process of learning most of those things myself - even after more than twelve years of working in this industry.

So, there is a lot to learn. You’d better start now.

Just… The day has only so many hours. And many of us spend most of them at work. We could learn in our spare time. But that is not an option for many of us. We have children, friends, hobbies, better things to do. And we must sleep - a lot - so we can be productive again tomorrow.

That means we must find ways to learn on the job. If you are lucky, your employer gives you all the time you need for learning. Realistically, there will be some limits.

And in many companies, there is almost no time and no budget for learning at all. By the way, let’s talk about good and bad employers in a later episode.

So, your company does not give you enough time and money to learn all the things you would like to learn. That means, your mission is from now on to

  1. Prioritize what you want to learn
  2. Find ways to learn things in short bursts
  3. Learn on the job, but in a way that does not take “working time” away from you
  4. Change the attitude and policy of your company towards learning.

I will talk more about things you can do to fulfill this mission in later episodes. Today, I want to start with something simple.

Solution

You will just do the learning during your lunch time. Once a week, together with your colleagues.

“Wait”, you say, “that means I will do the learning in my spare time! You said that was not an option!”

And you are right. You will be spending some of your own time, learning things that you need for your job. But you will do it at a time where you are at (or near) your company anyway. And you will also personally benefit from the things you learn here: You will become a better developer, and your job will become more satisfying for you.

Yes, I do believe that your employer should give you all the time and resources required to learn everything you need to do your job well. But when they don’t? What are your options?

If you are interested in this job and you want to do it well, start learning some of those things on your own. If not, you should seriously reconsider whether you are at the right place right now. But I will talk about that in a later video.

If you decide to start learning on your own time, your lunch break is a very good time for that. You usually cannot do much else anyway, you are at work. And your colleagues are already there, so you can learn from each other.

All you need to start a learning lunch is some people who are willing to join you and a place where you can do it. And your cafetaria or a restaurant is probably not the right place.

You need a room that is reasonably quiet and where you do not disturb anyone else. Everyone should be able to sit comfortably, at a table. But the room should also not be too big - If most of the seats are empty, it will not feel like you are having lunch together.

You need a big screen or projector for pesentation, and probably a good internet connection for the person who does the presentation. You also need a whiteboard or a flip chart for the discussion afterwards - even better if you have both. Also have differently colored sticky notes and painter’s tape if you can.

Everybody should bring note taking paper and a pen. But make sure that nobody except the presenter brings a computer. You want to facilitate a discussion, not an hour where everyone is surfing the internet on their own. So, phones should also stay mostly in the pockets of their owners.

And you need food. Either you order or buy food togehter, or everyone brings whatever they like.

I would not recommend cooking together, even if you have a suitable kitchen. While cooking together can be a great team-building experience and can be fun, it will just take too much time. You want to spend your lunch break mostly with learning and discussing.

One person prepares a topic. But do not spend too much time preparing. This does not have to be a sleek presentation.

Just state a problem, challenge or question, and solve it together. Like,

  • I would like to explore whether trunk-based development or feature-branching is more suitable to our way of working.
  • I want to practice outside-in TDD by writing a tic-tac-toe game.
  • I want to show you a cool thing I’ve found yesterday and then explore how we could use it.

You could even just watch tutorial videos together (wink) and then discuss them.

You can use different techniques - techniques that are useful in other situations too and that I want to present in later videos - during the learning lunch, like:

  • Mob Programming
  • Prepare presentations
  • Create a card game
  • Write a story
  • Power-point karaoke
  • Chaos Cocktail-Party
  • Learning Matrix

After the learning lunch, take some 5-10 minutes to prepare a conclusion. Write the topic on a flip chart page, and then write down what you did, a short summary of the discussions and, most importantly, what you have learned.

Add drawings, use differently colored markers, sticky notes, index cards - be creative. Make it beautiful.

Then, hang it on an office wall or in a team room, where everyone can see it. If there is already a flip-chart page from last week, take a picture of it and throw it away first.

After some time, when you have 8-12 pictures of learning results, show them to your manager. Explain to them what you have learned. Ask them to allow you to do a part of the learning lunch during working time. Ask them to pay for the food.

By this time, you will be able to show them the positive impact the learning lunch has on your team. So the manager has a good reason to accept your request.

(And if they do not accept: We will talk about good and bad employers in a later episode).

Conclusion

To recap; You should not use your precious free time to learn stuff you need on your job.

But if your employer does not give you enough time to learn and improve? What are your options? Use your free time, but a part of your free time that you cannot spend on hobbies / family / sleep anyway.

Like, your lunch break. Prepare a small - and I mean small - task for your team, and then discuss, program, research, write together. While having lunch.

And prepare a poster, like a single flip-chart page, with your results. Keep pictures of these posters to remember later how much you have already learned in those lunch sessions.

CTA

Now go ahead and organize a learning lunch at your company. And then, tell me in the comments how it went. Or tell me on Twitter - I am @dtanzer there.

And subscribe to this channel or follow me on Twitter if you want to get more videos like this.

What does an Effort Estimate Mean?

2018-05-21

When we give a size-, effort- or time estimate, like “5 Story Points”, “7 ideal engineering days”, “this will be finished before End of May”, what does that actually mean? And how useful is the number?

That depends on what question we are actually answering with that number…

But We’d have to Know the Future…

Some people argue that, in order to estimate accurately, you need a time machine: Only when you know the future, you will be able to make accurate descriptions.

But that’s a classic straw man: While the statement is true, it completely misses the point. The estimates are not supposed to predict the future accurately. They are meant to be another data point for our decisions.

Other Factors

Our estimates will always be “wrong”. No matter how much time we invest.

We can never fully anticipate all the things that might go wrong during development. Or how often we will be interrupted by more urgent stuff. Or how our software architecture and design will have changed when we start. Or how third party systems will behave.

When we get better at producing high-quality software - get better at “crafting software” - some of those factors get smaller. But they never go away.

But We’d have to Specify Exactly…

This is something I hear from teams a lot. “Let’s write down what we discussed before we estimate, so that later, we will implement exactly what we estimated”. So, the team here wants a very exact, detailed specification before giving an estimate, to make sure the number is as “accurate” as possible.

That behaviour comes from our own perfectionism and from fear of the consequences of wrong estimates. Both are very real, and both probably highlight some major cultural problems in this team and company.

Specifying, in great detail, when producing an estimate will prevent them from working in a truly agile way.

But can they even estimate, without knowing the future, without knowing everything that might go wrong, and without even knowing exactly what they’ll have to do?

Some Version of That Feature

Everything we do has an expected benefit for at least some of the stakeholders. Hopefully without annoying some others.

When there is so much uncertainty - no clear, exact specification, all the other factors - an estimate cannot mean “We will deliver exactly that feature within roughly that time frame”.

But it can mean “We are pretty confident that we can deliver some software that will bring most of the expected benefit within roughly that time frame”.

When we learn to create features iteratively, we will have a first version of the feature ready long before the time is up. And then we can focus on adding more and more of the expected benefit.

Are there Other Ways?

Our estimates will always be “wrong”. No matter how much time we invest. So, is it even worth creating them?

Also, when we use the definition “some software that will bring most of the benefit”, there is always the danger that some people “misunderstand” us (sometimes even deliberately) and turn our estimates into commitments.

And the usefulness of our estimates also depends on what kind of software we produce and where we are in the life cycle of the project.

But I will write about that later. Today, I want you to review what an estimate means in your team. And discuss whether this definition is useful within your context and what you can improve. And if you are allowed to, please tell me about it!

Planning Poker - What Could Go Wrong?

2018-05-03

A lot, it turns out. Or, maybe, not “go wrong”… But the result you get might not be the result you expected.

This article is part of the series Planning Software Development.

Suppose your team must estimate the effort of the work packages (User Stories, …) it is supposed to work on.

I know, sometimes the estimates are not needed at all and sometimes the need for estimates hints some deeper dysfunctions. Let’s put that aside for now.

Your team must or wants to estimate, and you are doing “Planning Poker”.

Planning Poker

In Planning Poker, some person (often called the Product Owner) presents a user story to the whole team. Then, every team member (developers, testers, …) secretly select a “poker card” that shows an estimate. Everyone turns their card simultaneously. Then they discuss why the numbers differ so much. After that, they play another round.

Often, those poker cards do not contain all possible numbers, but some exponential sequence (Fibonacci, modified Fibonacci, power of two), so that for larger numbers, there will be more uncertainty.

The benefits of this method are that team members do not influence each other when picking a number (some agile consultants call it a Wideband Delphi Technique for this reason). And still the numbers will be more accurate, because everyone’s opinion was heared during the discussion. And the whole team will commit to the estimates, because they created them together.

Those good things are happening… In the very best case. Which is probably not happening in your team.

Bored People

The PO discusses the user story - It’s the tenth of today. Your have small stories, as you are supposed to. The Scrum Master says “Everyone select your card. And turn your cards in 3… 2… 1… Now!”

The cards show 2, 2, 3, 3, 3, 5, 5, 5.

Someone on the team says “Let’s just make it a five and move on”. And everyone agrees “Yes, this is definitely a five!”.

Doing planning poker for lots and lots of small stories can become boring for everyone involved. People just want to get done - Assign a number - any number - to those stories, and get back to work. So, they become sloppy, like in the example above.

But, on the other hand, you should have small stories that go into development. And, as we already have established, your team must or wants to estimate the work packages they will be developing.

Commitment

Commitment to an estimate is a real problem.

I have listed it as a potential benefit above: When the estimates or deadlines come from outside the team, people will happily ignore them at best, or become very sarcastic and stubborn at worst. They will only do exactly what is instructed - even if it is stupid. So, the estimates must come from the team themselves.

But commitment to an estimate creates more problems than it solves. When people are committed to an estimate - instead of to an outcome - they will push back very hard against changes that would prove their initial assumptions wrong.

Yes, this is again a hint there might be a problem with the engineering culture of this team or organization. So, the problem is not only caused by planning poker.

But I have seen this in several teams that used planning poker, so it is a real danger. And this problem is caused, in part, by the team commitment that planning poker creates.

Power Dynamics

People always follow their leader. Even if they do not know it. So, if there is a leader, that person’s opinion will dominate the estimates.

And I am not talking about somebody with formal authority. Someone who forces their opinion on others. That would be a major dysfunction in an agile team - one that you should address.

I am also not talking about team members who think they have no authority at all, who think they should submit to the group opinion, who do not dare to speak out. This is also a dysfuntion in your team. It is harder to detect and address, and you must be careful when addressing it, but you should work on it.

I am talking about the person everyone trusts. The most senior or the most talented person on the team. The person who always helps and mentors others. Who is there for everyone else.

The cards show 2, 2, 3, 3, 3, 5, 5, 5. That person showed a 5.

There is a short discussion, where everyone is heared. Then there is a new round of estimating. Suddenly, all cards show 5.

If this happens once, it is OK. If this is happening often, examine your biases.

To Recap…

Planning Poker should, in theory, eliminate a lot of biases and make estimating quick an painless. But in reality, it often fails to achieve that.

The results that you are getting are probably not what you expected. They are often not an un-biased, thoughtful estimate where everybody’s voice was heared. And the shared commitment to the estimate - and not the outcome - creates its own problems.

I do not want to advise against Planning Poker. If you must estimate, it is still the most painless method that I know of. And it is easy to get started for unexperienced teams, so it moves one obstacle - “But we need numbers!” - out of your way quickly.

But after you did it for some time, examine the problems I have outlined above. If you have them, see if you can solve them (iteratively, over multiple sprints, in retrospectives). And also examine whether you can completely do without workpackage-level estimates.

Why We do not Want to Give an Answer

2018-04-25

This article is part of the series Planning Software Development.

When we develop software, people often will ask us questions like:

  • When will it be done?
  • How long will it take?
  • How much will it cost?
  • What will be done by 2019-02-25?
  • Can [feature set] be done by 2019-02-25?
  • What can I get for 200 000€?
  • What team size do we need to finish before 2019-02-25?

Those are related but slightly different questions. And a good answer to them would be useful to some people, often even to the people on the development team - the people who should give the answer - themselves.

And we often do not want to answer those questions. And we have reasons.

Our Answers will not be “Good”

Or at least not good enough for our own standards.

There are some inherent difficulties in estimating software development: Things about our work, things happening in our teams, that make estimating hard. And by hard I mean that our estimates will be “inaccurate”.

That is not a problem per se: All estimates (and also all forecasts, which are a type of estimate) are inaccurate. The value is still useable and useful, because it is based on the best information we have available right now.

But we do not want uncertainty. We are humans, after all. So, we do not want to produce data where we know that it is “wrong” from the start.

Our Answers May not be Useful

Sometimes, we do not want to give an estimate because we know we cannot produce it to a degree of certainty that would be useful.

We may be reluctant to answer “How long will [feature X] take?”, because there are too many factors that influence this number.

We might be tempted to answer “How much work will [feature X] be?”, because under some conditions, this question is easier to answer. But we have to know beforehand what exactly “Feature X” means. So if we answer this question far ahead of starting to work on “Feature X”, this may impede our agility, if we are not careful.

We can often produce the most meaningful answers to “How many work items can be done within the next month”, because this is a capacity-based forecast. In a stable team that already has enough historical data to run some simulations, this number will be the most accurate of the three.

But we will be most reluctant to give an answer when we think that the person asking does not need it at all.

Our Answers are often Ignored

Somebody asks “us” - the development team - a question. We answer with an estimate. When we suspect that our answer does not and cannot change the outcome of some meaningful decsisions, we feel like we have wasted our time.

“Can [Release] be done before September 22nd?” - “No, probably not.” - “But sales promised that to one important customer. Try it anyway.”

“Please estimate those 20 user stories, so we can sequence and cut scope” - “OK, here are the estimates…” - And then nothing happens…

“Based on our estimates and acutals, we are way behind schedule. We should cut scope now so we do not face crunch time in two months.” - “No, cutting scope is not an option.”

I bet every developer, tester and other development team member knows situations like those. And after hearing something like that, they probably have been thinking: “Why am I even talking to you when you ignore me anyway?”…

Our Answers Might be Used Against Us

Whenever we give an estimate (or a forecast, which really is just a special kind of estimate), we risk that somebody thinks this is a promise. At least in dysfunctional organizations.

And it does not matter how much disclaimers we add.

“In the best case scenario…”, “Our simulation said the most likely release data is…”, “Measured in perfect engineering days…”, “…but you know that a story point is really a range.”

The micro-manager will not hear those. They will come back later, screaming “But you promised…” or “Why is our velocity down this sprint?”. If this is happening in your organization, you have some major cultural problems. You are seeing Agile Anti-Patterns. And in such a dysfunctional environment, we do not want to give any answers that might be used against us.

No Answers - The Solution?

For what it’s worth, do not answer questions when you do not need the answer. But…

I am talking about not wanting to give answers today. If we could give them, they might be useful. Even if they are not usefull all the time, they may be useful in some phases of software development. At least to some people, and at least in some situations.

Even though our answers may not be “perfect” or even good enough for us, there are situations / times where we are able to give better answers than in other situations.

And when the answers are ignored or used against you, you are seeing some severe dysfunction in your organization. In such a situation, you probably should not answer those questions - But you probably are also not allowed to deny an anser. So, now might be a good idea to look for “Agile Anti-Patterns” and to start addressing them - You have bigger problems than just giving “good estimates”.