Are You Wasting Time?

No, I don't mean reading my blog or surfing the internet or watching youtube videos. I mean even when you are working, are you spending time doing repeatable tasks that could be automated, and are you doing tasks that do not contribute to the development of the product?

The first question is the easy one to ask, but can be a hard one to fix. For example, does your build require you to "do something" to go from the built baseline to a running executable to test or deploy? If not, what's holding it up? Is the build simply not setup to do that, or is some portion of your system not configured to do it? If it's the second case, that's a bigger problem, but it can still be overcome by working with your system administrator and addressing the holdup.

The second question is the bigger task, and will usually require buyoff from your program Higher-Ups. I see this problem as the corollary to the Agile Manifesto, which states (among other things) "Working software over comprehensive documentation." I like to sum that statement up simply "Does this help me do my job? Would this help someone else do my job?" If the answer is "No" don't do it. You can quickly see where that would be a problem with the Higher-Ups, and the "That's the Way We Do It" crowd.

Another corollary to the Agile Manifesto is "It's OK to Screw Up", meaning, do something, give it to the users, and they'll tell you what they like and don't like about it. But do it fast enough to have enough time to fix the problems, and add the new features to it. This can't be done in a rigid environment, where it takes weeks just to get a fix to the users, never mind the amount of time to develop a new feature. To do it, and do it fast, and get it right, actually takes less overhead than you'd think.

Building Blocks or Jigsaw Puzzle?

Is your system building blocks for a larger system, or jigsaw puzzle pieces to build the same system? A puzzle piece fits into one and only one spot. A building block will fit into one spot, but can fit into other spots as well. I also like to think of it as a "spiky vs. smooth" interface.

What make good building blocks? The obvious buzz-words "extensible, re-usable, decoupled, cohesive modules" apply, but it's easier too look at. How hard is it to add new data to the system, and how hard is it to make the system do something else? If the answer to any of those is "not easy", then you're building jigsaw puzzle pieces. Look at your method signatures. Do they perform specific functions on specific types of data? Those are puzzle pieces. Do they perform specific functions on generic data, or (better still) generic functions on generic data? Those are building blocks.

So what makes reusable, extensible data objects? Obvious answers are object inheritance and using a syntax like XML. But no matter how you do it, you need to identify similarities between data and only handle unique data in special cases. Abstraction of data into generic objects for modeling and design helps describe the mappings between similar data.

An example of what I'm talking about. How many times have you seen "The Bank Example?" But imagine that example built to handle one very specific type of currency and a couple specific use cases:
depositDollar
and
withdrawDollar.

Simple, straightforward, and almost totally non-reusable (without refactoring). The use cases would be diligently mapped to methods named
depositDollar(Dollar dollar)
and
Dollar withdrawDollar()

in class AccountAccess. Dollar, being a good data class, would contain all the useful metadata about your dollar: serial number, creation year, number of wrinkles, that sort of thing. On the server, the business logic would need to store the Dollar and withdraw the Dollar from the Account, and do all the associated checking that goes along with it. So now, how do we add a new data type or another operation? Yup, create a new use case and a new method. Repeat ad infinitum (or at least ad naseum).

Now we've got a system doing very similar things on very similar data, but it's hard to refactor because you're used to thinking about the specific types of data, that a Dollar is somehow different than a Quarter, or a Euro. The only thing that can pull us out of this mess is to re-evaluate the data, and more importantly, what the users want to do. They want to put money into their account and withdraw money from their account. Now, generically, we can use methods
deposit(Money amount)
and
Money withdraw(Money amount)

where Money is, of course, an interface that Dollar, Pound, Euro, etc. all implement.

Now, I realize that there will be have to be some "business logic" in the system, otherwise the system doesn't do anything unique. But the amount of uniqueness should be low, and decrease, not increase, as the system expands.

Branches, Refactoring, and Deprecation

One of the biggest hurdles in refactoring is answering the question "How do I merge changes into something that's no longer there?" I will attempt to lay out a useful strategy for addressing that problem.

In our example, we've got a class that has a number of methods. The number of methods continues to grow (and the amount of logic in those methods grows as well), to the point that the class is unmanageable. It's no longer clearly defining its task (if it ever did), and has wandered off into areas that are outside its scope. But, because of its importance to many areas of the system, and because of its poorly defined task, it has a lot of bugs in it. The bugs may be from improper implementation, or incomplete analysis. Either way, there's a lot going on in here.

The obvious answer is "Refactor it." However, that's not going to be easy, since there are still bugs that need to be fixed during and after refactoring. How do we deal with this? With a cunning use of deprecation and branching.

Step one is to create a branch for the refactoring. The syntax and details of doing this are different for every version control system out there, so I won't get into it. Now you've got an area set aside just for refactoring without any outside influence and are able to verify that you haven't broken anything, through automated or manual regression testing.

Step two is to deprecate the methods or classes that you've modified during refactoring. But don't remove them! That will keep a merge point for you to pick up an additional changes that happend during refactoring and testing, and inform other developers that the class is about to change and the current method is no longer supported. In addition, a beneficial practice when deprecating (I'll use Java as an example, since I'm most familiar with it) is to include the @see tag to point the other developers at the right method.

Step three is merge in any changes into your branch. This will be a two step process: merge the changes into the original spot, then apply them into the refactored locations. This may seem time consuming, but is really the only way to ensure that your branch is up to date before merging into the main line. Re-run any tests for the changes to make sure that you merged the changes correctly (at this point, the benefit of automated testing over manual testing should be more obvious). Now you can merge your branch into its parent.

Step four is only necessary if other people are working on their own branches as well. After such time has passed that all the branches have picked up the refactored class, you may delete the deprecated methods or classes. This additional wait allows the other branches to migrate to the new organization at their own pace. They will have to do the same step you did before merging: apply any changes they've made to the original method to the refactored one.

Development by Telephone Game

Does your development process have a lot in common with the Telephone Game (also called Chinese Whispers or Russian Scandal)? I mean, how many steps, and how many people, are involved in getting what the users want into their hands?

Do the users tell their management, who tell the prime contractor's systems engineers, who tell your system engineers, who tell you what to build? Then, do you give your build off to the integration team, who gives it to the maintenance team to give to the users? Let's see, that's seven steps between the users and the end product.

That's one key area where open-source does it right, because they have to. The user submits a bug or new feature to the developers. The developers fix it or build it and give it back. That's one step (OK, there's probably a couple internal steps for reviews, and testing, but I overlooked those in the first example, so I'm overlooking them in this example too). And funny enough, it creates a stable product that delivers the functionality the user wanted faster than "Doing it the right way."

Are you naive enough to think "But think of how good the product would be if they did it our way?" The reason open source works (and work fast) is that there aren't multiple steps between the user and his product. The best tools are used BY the developers (see GCC for example) during development. If there's uncertainty, the developer talks to the user. There's no "pushing issues up the chain" and waiting for resolution. There's just a need, and the fulfillment of that need.

So call it Agile, call it XP, call it whatever you want. But if you base your development around creating user functionality instead of data functionality or system functionality, you will produce more user-usable functionality faster, simply because everything you build addresses some user need.