Using design patterns effectively to solve problems

I promised myself that my blog posts would be short and pithy, but it appears I have a hard time being honest with myself.

This is part rant, part instructional based on having been through this myself a couple of times and now watching others on my project teams going through the same thing. How many developers and architects have spent hours poring through books, white papers, code snippets, etc., about software patterns because that's what you were told that smart engineers do? How many times have we used a pattern thinking it was a good fit to solve a problem but turned out to be a bad fit? We all might have done these things at some point or another, but why?

There's a skill of pattern application that's been severely neglected - the skill of recognizing patterns and the problems they're intended to solve. So how does one learn how to do that? We can all pontificate and theorize, if I see X, then I do Y, or Bill is a really smart engineer and he did this on a past project and it worked really well, but what happens when you've got a problem staring you in the face with no frame of reference in your prior experience? How do you tell IF a pattern will solve the problem? How do you avoid the trap of using patterns just for the sake of using them and overengineering your solution? A by-product of this is learning how to defend a particular design approach by highlighting the problems you're intending to solve and how they releate back to your design.

I've been involved in authoring design patterns (note: absence of shameless book plug), and it's not as easy as it looks - it's some of the most challenging communication I have ever crafted in my career. Each concept is meticulously pressure-tested for relevance and flow in the overall theme of the content, and by the time you're done, you've re-written it 3 times, your editor has changed it in ways they don't understand, and you're constantly drawing from the deepest recesses of your own knowledge and experience as a subject matter expert to summarize a complex problem in profound simplicity. Ideally, the end product is a well-crafted pattern that is deceptively simple so that the non-expert can read it as a statement of common sense. The pattern within your pattern, is to optimize the discoverability of the problem and the solution for someone who is not a subject matter expert.

As someone who uses patterns to solve design problems and works with others who do the same, I run into the challenge that pattern content is becoming more difficult to consume. Are we all getting dumber as time goes by? I don't think so. In my opinion it's because we are having technology to do more and more for us and correspondingly the problems are becoming more complex. The effect is that a person is unable to quickly and easily recognize the problem that a pattern is trying to solve and how the key points of the solution relate back to that problem. How does an engineer become better at establishing the relationship between problem and solution in patterns?

From my experience, there are a few different things that a software engineer can do to sharpen their pattern/problem recognition skills:

1) Experience - nothing beats the best teacher - you just gotta get out there and do it. Inevitably you'll make mistakes, but a key guideline to avoid problems is not to get hung up on extremes - focus on solving the problem and test your solution with realistic scenarios. Try too hard, and it can lead to over-engineering your solution or using the wrong pattern to solve a problem. If you try to make a pattern fit for a problem it's not intended to solve, you have the square peg/round hole effect that creates an unwieldy solution. Too little effort will leave you short on anticipating how your solution will be put to use, and there's a good chance that you'll overlook some key problems or scenarios that will be much harder to solve later on once the system goes into production. Make a checklist of steps/points in the solution approach that map directly back to the problem points. You can use this checklist verify that you've solved the problem effectively with your solution and to show others how you've done it. Peoples' eyes tend to gloss over when trying to describe software design without sticking to the relevant points that show them the goodness in what you're doing right away - a bullet/checklist is an effective tool to keep it to the bare essentials.

2) Case studies. Look at what others have done and find similarities between the problem they solved and the one you're trying to solve. White papers have a place in this, but sometimes run the risk of being too theoretical or too esoteric for easy consumption. Something I've found effective both as a pattern author and consumer is a precise problem statement matched with key points of a solution. If you're looking at examples or case studies, focus on drawing commonalities between them and your problem. If you take this approach, there's a good chance that you can also identify the key points that can be used to address the need in your particular situation. If you're not even sure if the material is relavent to your problem after looking at it for a short period of time, then put it down and look at something else; otherwise, you may force yourself to see similarities that don't exist and implement an ineffective solution.

3) Learn the language of patterns. Another approach is to study how patterns are communicated. If you're serious about doing this, consider a different perspective by looking at patterns related to other engineering disciplines outside of software. I gained a lot of insight about how to publish patterns so that they can be consumed effectively from a book called "A Pattern Language" by Christoper Alexander. It focuses on building architecture and urban planning, but you can really get a feel for how patterns are supposed to work and the power of their application. I know very little about the subject and I found I was able to understand design aesthetics that you instinctively feel as a human being but can't pick out why you particularly like or dislike a room, building, neighborhood, etc. Ultimately, I was able to carry on a reasonably intelligent conversation with a subject matter expert to tell them what I wanted for an office layout. That experience translated well into software engineering to understand the desired effect of a pattern.

As time permits, I'll share some more recent experiences using patterns to solve design problems in the hope that others might benefit from what I've learned.