Estimating Software Work from Transitions: Don't Fill In Your Time Sheets!

Last week I was reminded of a Classic Problem in software project management, when our management team told us that they wanted us to start filling in time sheets again. This was almost universally met with derision and a very obvious intention to completely ignore the request. Not exactly polite, but this is a peek at a pretty standard problem in our industry.

Now, rather than get into why software developers don't want to fill in time sheets, I'll instead focus on what we want out of it as project managers, and how (if possible) we can make the process of gathering the information easier, less of a bind, and perhaps even completely invisible.

What we want out of time sheets may fall into different categories of information (time spent on Project X, time spent on activities we can get R&D tax credit for, time spent project managing, time spent designing etc.), but the underlying form of that information is invariant: time spent doing something. And more to the point, was that time useful, value adding, or was it a waste, could it be reduced in future? This second point is an interesting one the more I think about it, and not entirely available to us by traditional means: who enters on their time sheet "4 hours coding up feature Z and 3 hours waiting for the build, before 15 minutes of testing"?

So, I take my inspiration for potential solutions to this problem from two sources: Kanban, and an almost throwaway joke in a meeting about time-sheets made a long time ago (the last time we were asked to please fill in our time sheets):

1. Kanban teaches us to capture the dates things are started and finished on our cards, so we can ultimately know and define our class of service. These dates (and times if we want to be that precise) give us the various times it has taken us to achieve certain value adding tasks, as well as the times a bug or feature has spent waiting in a frame of reference meaningful to the bug or feature. This meaningful frame of reference is simply the time it's taken to deliver. Knowing how much of that time was spent value adding and how much was waiting allows us to aim for a better class of service and achieve it through quantitative project management.
2. The throwaway joke I remember was along the lines of "can't we just run-length encode our time sheets". Simple, and actually a very efficient way of describing how your time has been spent if you don't do many different things, but probably not in the same class as the Kanban card information.

What I realised when thinking about these things and the processes we currently use, is that most of our work items reside in databases with various states: "bug unresolved", "bug addressed", "feature in test" and so on. More to the point, the database captures the date and time at which these status changes happened.

This is when it hit me: calculate the times spent simply by querying the database for the times between transitions. Now, it is likely that there aren't enough status fields to capture all of the states work can be in: "queued", "ready" and "waiting" certainly don't feature as standard, and these are necessary to differentiate between "touch time" and "wait time".

So, do I think we all need to fill in our time sheets? The answer is a very qualified perhaps not. (I'll get off the fence when I'm the one held responsible, and not someone else!)

The reason for this is that first, developers and testers all need to use the bug and feature database as their primary way of "accessing their work". As soon as people make a little paper list of things on Monday, work through them all week and set 10 bugs all to "ready for test" at 5 pm on Friday, it breaks down, so one qualification is that good systems need to be in place for people to manage their current work in progress.

The second qualification is that all work we do has to be entered into the database, so the time between a certain feature changing to "in development" and then to "test ready" doesn't contain a massive chunk of project management overhead, or build slave rebuilding for instance. Despite the times between transitions being very meaningful to the class of service we provide, and very in touch with the real effect our work is having on the bottom line, I still think that it's useful from a project management perspective to know what categories our wasted time falls into, so we can attack the cause and get leaner.

So, in the end, where does this leave me? I think it leaves me with a clear idea of how we could capture the time it takes us to do our work in software development without the additional overhead of the traditional time sheet, but it also leaves me with a decent list of things to do before this is truly possible!