Tuesday, June 24, 2008

Improve Product, Not Process

Believing that process improvement will improve the product is like believing spending more time looking at the map before taking a trip will prevent you from running into a traffic jam along the way. Process improvement creates paint-by-numbers, it creates fast food. Which is fine if that's your goal, but don't expect anything other than paint-by-numbers or fast food. Otherwise, we're treading into Einstein's definition of insanity "Doing the same thing over and over and expecting different results."

If you feel you must improve your process, plan your process for change. There are too many uncontrollable variables to have any expectation of a successful delivery with a single waterfall method. So plan your process for more frequent changes, more frequent reviews, and more frequent releases. Hmm, this sounds like agile (without a capital A).

Tuesday, June 17, 2008

Just Because You Can, Doesn't Mean You Should

A note on the Agile adage "Do the Simplest Thing That Could Possibly Work. (DTSTTCPW)" DTSTTCPW does not mean "Do the Easiest Thing," or put another way "Just Because You Can, Doesn't Mean You Should." (Hey look, a title!) There's a file line between those two ideas, because the Simplest Thing to one person isn't necessarily the Simplest Thing for everyone else involved. Reducto ad absurdum example: suppose someone keeps the requirements in a Word doc on their desktop. Anyone who wants to know what they have to do has to go look at that one person's desktop. Now that's surely the Simplest Thing in this case, but that is definitely not the Simplest Thing for everyone.

So, as usual, we have a problem. How do we figure out the Simplest Thing when it comes to organization and communication? Here's some guidelines:
  • if you have to search for it, it's not in the right place
  • if you have to use different tools, you don't have the right tool
  • if you have to ask, it's not documented well enough
  • if you have to change the process to get something done, the process needs to change
The first point will stop the practice of "Put it on the data server" in a unnavigable directory structure that combines software, business development, schedule, test, birthday lists, and everything else under the sun. Do you have a number of bookmarks and shortcuts to help you navigate your data drive? So do I. This problem can be traced back to the problem that the project is not focused on the Product, but more on everything else that goes into the Product. If the data is not related to the person who has to wade through it, get rid of it (or at least put it somewhere else). Sourceforge once again comes up as the model. All the documentation related to the Product is stored there. Want to use it? Read the documentation. Want to see the bugs? Look in the tracker. Want to help? Download the source for Subversion or CVS. It's a very simple idea that's very easy to overlook: The Product is the Project (hopefully Scott McNealy will forgive me).

A documentation system based on a file system just isn't effective anymore. Sure, it works. Much like any outdated technology still works, there's just a much better way to do it.

Tuesday, June 10, 2008

The Simpsons as a Software Model


Like most things in life, The Simpsons show us the way. Or, rather in this case, the way NOT to do something. In "Oh Brother, Where Art Thou?" Homer gets to request all the features he wants in his dream car. In effect, he gets to design the car himself. You can see the result on the left. That's like what happens if our customers get all the features they ask for.

Our end product ends up looking like that. Lots of features "stuck on" with very little concern for the overall design. Sure, it will work, and come out as a finished product, but the true art of software comes from taking all those features and still producing a cohesive, solid design. But, like most designs that are "customer-driven," it is way over budget (at least it's not behind schedule, as well). Now, if you're in a market where you can continue to charge the customer to make up the budget, that's a feature, and probably a goal. However, in the real world (and the Simpson's world as well), an ugly, poorly designed product won't sell.

So how do we get from Homer's vision (which, incidentally, he was quite pleased with) to a mass-market friendly one including all the features the customer requested? One simple step that seems to be largely overlooked is "Saying No." I can't believe I've even got to put this down, but if the customer makes an outrageous request, saying no (and explaining what we should do instead) will cut the largest potential budget/schedule over-runners. For example, I'm currently supporting a custom data synchronization engine for my current project. Now, we are using a database that supports replication, but it is not available. If we'd just said "No" (and for the record I said "No" but my managers didn't care, because they didn't want to say "No" to our customer), we would be able to use the built-in replication. But we didn't. C'est la vie.

Thursday, June 05, 2008

A Software Fable

Gather 'round, kids. It's story time. There's so few fables written nowadays, and few of them would apply to software (if you read between the lines). So here goes.

Once upon a time, there were two groups of engineers out for a walk. They came upon a creek, and, being engineers, decided to dam the creek. The first group set to work immediately measuring the depth and width of the creek, calculating flow rates and how high the water would rise once dammed. The second group looked at the creek and started throwing rocks into it. The first group, having taken all their measurements, set off to carve a rock big enough to block the creek. They left the second group still throwing rocks into the creek.

Time passed, and the second group came back with their huge rock. But unfortunately for them, the rain had turned the creek into a raging river and their rock was still not big enough! Or it would have been a raging river if it was flowing. To their surprise, they saw a dam built out of the rocks that the second group had been throwing! The first group was amazed that something so big had been build by so many little rocks. They went and asked the second group how they did it. The second group said "Some of us looked for more rocks to put in a pile, and the rest of us threw the rocks from the pile onto the dam. So while you were measuring and calculating, we were already solving the problem. If anything goes wrong with the dam, we just need to throw more rocks. If it rains more, we just need to throw more rocks. You need to carve a whole new rock."

Many morals apply to this story: do the simplest thing that could possibly work, don't over-engineer the solution. But the one I like is this: if you leave engineers in the woods for too long, this kind of thing is bound to happen.

Monday, June 02, 2008

Ghost Town

No, not the blog. I know I missed a week due to The Big Release deadline on Friday. I do this as a hobby, and comment about my real job. So if I miss a week, my loyal readers (of which as far as I know there aren't any), will just have go without.

That business aside, I started wondering about the New Developer Problem, or more specifically, the worst-case version of it. "What if everyone here quit/disappeared/died/otherwise didn't work here any more?" Maybe I'm on too much of an Indiana Jones train of thought lately, but I see it like an archeology expedition. Some guys in fedoras pry open the door to our dusty office and see a bunch of dusty cubes. They sit down and fire up the boxes. Would they
  • be able to figure out what we were working on?
  • be able to continue on with what we were doing?
The first task is possible. Assuming they can get user logins (Dr. Jones would probably Post-It surf around to find user names and passwords), they can probably click enough desktop files and folders to get an idea of what was going on. But does that combination of information contained in SDFs, schedules, budget figures, requirements, etc. actually describe any of what was going on? If you found a document in Egypt that said "June 1, 2500 B.C - build ground floor," would you be able to ascertain that you were looking at requirements for the Pyramids? Because that's what our artifacts tell the outside observer: not much.

The second task is much more daunting. Even if Dr. Jones was able to figure out what we were making, the first question would be "Where is it?" followed by "How do I make it work?" and "How do I fix it if it breaks?" Can he find the answers to those questions buried in all the process-mandated documentation? Maybe it's possible, but in my experience, it's unlikely. Most of my projects have been treated as one big project, rather than a large number of small projects. So all the documentation was (is) created with respect to the whole project, and didn't reflect the individual pieces of the system. That's like trying to understand the internet by reading the HTML and HTTP specs. Yes, you'll understand what it does, but how it does it is totally ignored.

Now imagine Dr. Jones's expedition into a open-source development model software facility (adverb chain ahoy!). He walks in, clicks the browser icon on the desktop (hopefully the only icon there), and the browser goes to the project home page. Now he can see the project roadmap, open bugs, read documentation, and download code and start making changes and fixes. With that kind of organization based on the product, you don't need to be an Indiana Jones. Even Marcus Brody could find his way around that project, and he got lost in his own museum (supposedly).

Most of this problem comes from each group involved doing the easiest thing for them. For testers, they create Word docs. For systems engineers, they create Rational UML. For software engineers, they create code. But all those things add up to create documentation focused on whatever each group finds important. The documentation needs to be correlated to the end product, and should be stored in a common place that can show the correlation of the documentation to the product, i.e. a project website. Sure, software can be built in many ways, but you are at risk if you project depends on "project experts" or "domain specialists." If there are areas that aren't immediately understandable, that's an area for improvement of documentation or refactoring.