Where Did Our Knowledge Go?

I found myself pondering where a lot of the knowledge created in our processes and code goes recently. I overheard some people discussing re-factoring some code. Re-factoring it to the way it used to be, although they didn't know it: they hadn't done the original re-factor. Now I found it rather difficult to give a convincing argument other than "there were very good reasons why we moved away from that", but I couldn't remember all of them: they had come from a variety of people and it was a decision made by consensus.

So where did that knowledge go? More to the point, how can we keep hold if it when we've made the same decision again. (And also, wasting a load of time discussing it and perhaps some time coding it before someone has the eureka moment and re-spots why it was a bad idea in the first place).

As I learned from Don Norman in The Design of Everyday Things, that knowledge of how things work should be "placed in the world", as opposed to in the user's head, and I believe the same is true of our code. Not just how it works, but why it is the way it is also. This was actually a bit of a u-turn for me, as I used to dislike code comments that stated a reason for change, or worse (as I then saw it), an original line of code commented out with a commented reason for doing so. Let's look at the ways our codebase contains our knowledge and how it can perhaps encapsulate and remember the decision making process for us.

• First of all, we strive to write code that communicates design intent implicitly: this is just good practice. However, all this effectively communicates is the way the code is now, and has no history.
• Comments are a good place for a more complete presentation of a design, perhaps including the reasons why a different candidate design might not be so suitable. One of our engineers lately told me that he was focussing quite a lot of attention on commenting the implementation of classes as much as their interfaces, and I can see how this helps to retain a lot of our knowledge by explaining the design criteria for the implementation, rather than just the final decision. Comments are fairly cheap, and there are many other reasons for commenting well. All that's required here is an additional perspective on the comment, a new reason for writing it.
• Our source code control system can potentially tell us about the decision behind changes, provided we write good commit messages. Thinking back over some of the commits I have made that covered a re-factoring, I can honestly say that I didn't record enough of the knowledge generated during that change. Most importantly, there possibly wouldn't be enough there to dissuade another engineer from switching back. Source code control is cheap, practically free, and focussing just a little more effort on those commit messages could potentially give a lot of bang for buck in the context of this discussion.
• Finally, good design documentation made as part of our development process can be an excellent place to look if you are reviewing the decision over a particular aspect of a design. This is, however, perhaps the most expensive ways to keep a record of the technical and design specification decisions that we wish to live on.

What have I learned from this musing? For a start, I've learned that my memory is nowhere near good enough 2 or 3 years down the line and I need a concrete recording of the way we design our software. I've also learned that I can rely on some automated tools to keep a track of these things for me, but if I want to be sure, then any decision making logic needs to be recorded, ideally at the site where it will be needed. Since starting this blog, I've actually put some of this into practice and done the following:

1. Written a load of comments in a few new classes to explain the reasoning behind the storage choices. I could just assume everyone has read Effective STL (and that's thankfully a valid assumption with my colleagues!), but that doesn't communicate the fact that I traded insert speed for fast iteration, because the class is accessed thousands of times per seconds, and only updated once in a blue moon.
2. Written much more complete commit messages that explain the original problem, what caused it and, most importantly, how the changes made addressed it.

Perhaps there are more code-bound ways to achieve some of this. I'm thinking along the "modern C++" that the likes of Alexandrescu propose, where policies can be used that embody the decisions quite clearly and explicitly, and most importantly in code. After all, someone can edit or delete my comments and the code will still work just the same. If I come back to any of my carefully crafted files to discover a carefully explained decision and implementation has been replaced with something new and the comment:

// This seemed to work better

Then I'll be pleased that someone improved on the design, but pissed off that they didn't care to enlighten me!

But I'm sure I'll continue in the same vein as I've set out on, as I've found that I feel much better about the code I've recently committed. This is because I'm much more confident about how it will be read and received by other coders down the line, as they'll understand the design process, as well as the end result.

Personal Kaizen

I look at the diary and, oh my, seven months have passed since I last updated my personal Kaizen. This is not good. It means that I'm either busy, extremely tired, or losing interest. It had better not be the latter of the three.

So what have I tried to make my professional life better? The truth is that I haven't tried much in the way of specific practices, gadgets, software or technology, rather I've focussed on some much bigger ideas. But first, the geek stuff:

• I downloaded and tried synergy and am extremely impressed! It's allowing me to seamlessly operate my XP and Windows 7 PCs side by side, keeping a total of 8 cores pretty maxed out when things are going well. The main difference to me however is a significant reduction in shoulder and neck pain as I'm no longer twizzling between two keyboard and mouse configurations! The next thing I need is a good free intranet shared clipboard...
• ...however, I used a very basic Windows feature (XP, Vista and 7) to help me with sharing just today, by copying all of the stuff from My Documents on each of my computers to a network folder and then pointing the My Documents folders to it: simple stuff, but the end result is quite pleasing, as the network sharing is effectively hidden, making it feel quite seamless.
• A solid state hard drive was going spare after someone left our office and I was recommended to use it as my build drive, as I usually have the biggest solutions to re-build (328 projects I counted today: usually taking an hour and a half from clean!) This has worked quite well and seems to improve the more stuff I cram on there that our code includes and links to. It'll only take one full build (source, libraries and binaries), which is annoying, as I build at least four different products from time to time. It quite surprised me that the SSD out guns a RAID0 for read/write access by such a margin, but the bottleneck is still link time: can only break that one down with more, faster processors...

So, that's the very thin list of widgets and practices; on to the Big Ideas.

• The first has been Communication. Dumb or what?! I found an old book about interpersonal relationships and psychology in a charity shop recently and it made me realise how much relationships affect our professional lives. I hope my colleagues don't disagree with any of this, but I now make a concious effort to listen when people are talking to me, rather than thinking about my next answer. I'm also finding it easier to read situations, armed with a little bit of the psychology and non-verbal communication ideas I've read. A really big one, which didn't just come from the old book, but from The Art of Project Management (Theory in Practice (O'Reilly)) also, is the communication process. That is, remembering that communication isn't just sending an email, the information has to be transmitted, received, understood, and acted upon. The chapter in Scott Berkun's book about "how not to annoy people" was definitely my favourite and the one that influenced me the most.
• The second thing I've tried to focus on has been Honesty. Make that honesty and courage. What I mean by this is being open about my thoughts on all matters at work (but in a professional way), be it whether I think we're doing enough to improve our development process, right up to expressing a fear that we might be making a very bad strategic descision. I always try to pose these issues as questions to allow a discussion to ensue, but sometimes, we have to grow a pair and stand up and say "no I completely disagree". Don't we?

What's next? Well, I've been reading Agile Testing: A Practical Guide for Testers and Agile Teams, and fancy putting that and some testing principles I learned at an IMechE training course on R&D project management into practice. Also, I've started to learn C#, so want to try it out on a side project in my own time.

So, a very poor first half of the year for personal Kaizen. Where has the time gone? I've no doubt the second half of the year will whistle by in just as much of a blur, but I feel more energised now than in the winter and spring - the number of posts I'm making here is usually a good indicator - so I expect I'll be managing more of my personal mini Kaizen events.

Doing Support When You Should Be Coding? You Didn't Finish Properly

While I was on a training course recently, we were going through the lifecycle of a project and looking at the transition from implementation to support. I realised that I often see people (and find myself) dragged through this transition. Why is this?

The only answer I can come up with is that finishing the implementation stage properly (designing and writing our software well) is key to keeping developers developing and empowering customer service and support to do all of the support and servicing. To discuss what I mean by "finishing" effectively, it's probably easiest to reverse engineer it by looking at what it is us developers are asked to do by support and service people, and even the customer:

• How does something work under the hood? This is applicable if you make scientific, mathematical or engineering software, because someone's going to want to know exactly what the frequency response of that band pass filter is or whether you've included radiation terms in your transport equations. Make sure it's documented (and unit tested to prove it does what you say). Even a well written pithy tool-tip can cover this kind of thing for the simpler cases, but it's most often going to need some comprehensive documentation. Of course, if it already exists in the public domain, reference it.
• How do I achieve my goals? Basic "user guide" stuff this, but if you haven't effectively communicated implicitly in the design or via the help and documentation, then people will keep coming back to you for this. You need to put the knowledge in the world for people, and documentation is the weakest form of this: better, make the way to use your software obvious in its interaction design.
• Does it do X? If it isn't obvious what the user can achieve with your software, then you're looking at the same problems as in the previous category. If you've implemented features that aren't in the specification and documentation, then perhaps you've just been doing the things you like, rather than what the product manager asked for. If you've done something good and it hasn't made its way to the headline, then you've denied the project manager the timely opportunity to get something into the documentation/help and make a fuss about it as a headline feature.
• Why doesn't it Do Y? This is perhaps a bit more of a product management issue as a development issue, but a strong product vision that covers all the bases of a users needs in a cohesive manner, coupled with conscientious implementation will not suffer these problems. If you keep getting asked, then find a way to get it done: campaign to your product manager to get it in the next service release or the next major upgrade.
• What just went wrong? Where's my data gone? Now this is even worse: if your software's making mistakes that people have to come back to you to unpick, then it's unstable and not worthy of release status. Recall it and test it before releasing back into the wild with a clean bill of health.

Every time we don't finish off and hand-over effectively, we burden ourselves with a few more support duties. At first it's OK, in fact it's flattering when someone comes to ask you something that only you know sometimes, but after a while it gets tedious. Is a quarter of you day too much to be spent answering customer queries and supporting the customer service team? Half of your time? Ask your manager and they'll probably have something to say about it.

Usability Design of an Iron

I've just been doing some ironing and suddenly saw the root of an infuriatingly bad design.

Ever get the annoying spit back and bubble blowing when you're re-filling the iron? I certainly do, and I just noticed that the way to avoid it is to put the iron down flat (as if I'm actually ironing something). Great, it no longer spits boiling water at me. However, now I don't know when to stop, as the iron's facility for telling me when it's full only works when it's stood upright!. Plus, I have to put it heating element down, so it's heating something up, or burning the ironing board cover (or worse, my shirt!).

What's the problem? Feedback and joined-up design thinking. There is feedback designed into the design for re-filling the iron, but it hasn't been designed in conjunction with the filling mechanism. Worse still, the filling mechanism has been design assuming the iron is off. I don't know any family men or women that have a small enough ironing pile to get through it with a single, cold fill at the start.

Rant over, I'm going to try and finish the ironing now.