This translates into preparation, more than anything else: Preparing by understanding your options, preparing the organization for change, and preparing to measure your success.

Be thorough during evaluation

The most common oversight in preparing to adopt a new methodology is simply not evaluating all of the available options. It’s an easy pitfall to succumb to: There are so many processes, so many methodologies, so many choices, how can someone possibly make the right choice? Surely all of these published techniques are mature and “real,” does it even matter which methodology we choose? Yes. It matters a great deal. Each methodology has its strengths and weaknesses and very few methodologies can be applied to every development project.

The wide variety of methodologies is a reflection of the complexity of the software development industry. We have many choices in executing any strategic operation, whether a military incursion, a football game or planning for building a house. Likewise, the software industry has evolved a wide variety of processes, each one suitable for different scenarios. While it is certainly true that many methodologies can be successfully applied to many different projects we can’t make the assumption that any one methodology will work equally well in every situation. Adopting a heavy process in a project involving a small team and a short-term schedule is almost always a poor idea, as it leads to extending the project timeline to support unnecessary project artifacts. But less obvious is the impact of pairing a lightweight process with a medium-sized project. How many people is “too many” for an Extreme Programming (“XP”) project? At what point does the lack of formal project controls start to make the project unpredictable? Will the business stakeholders feel the project is not adequately managed? These questions, and many more, emphasize how important it is to prepare thoroughly before choosing a methodology.

Given the plethora of potential methodologies, it’s easy to just pick one and get started. The temptation to simply choose a well-regarded methodology, buy a well-reviewed book on the subject, and forge ahead can be strong. But this “textbook approach” can prove deadly. Without studying the methodology beforehand it’s easy to choose the wrong methodology — and even if a mistake of this magnitude becomes clear over time, it’s usually too late to change course. And much like reading instructions too quickly, it’s easy to realize too late that the process is wrong: Incorrectly implemented, or not the right fit for the situation.

Another pitfall to the “textbook approach:” It leads to following a process blindly and over-adopting, particularly with more comprehensive methodologies that have more to offer. The fallout from this: Teams come to think that comprehensive methodology is a “bad thing,” heavily weighted and full of red tape, unnecessary work and overhead. Using the textbook as an instruction manual makes it impossible to have a complete view of processes and artifacts offered by the methodology and, therefore, the value and appropriateness of each.

Prepare the team and the organization

Just as evaluating and selecting new methodology can be a mine field, so can the actual adoption process. A common oversight when preparing to adopt a new methodology is not planning for the upheaval it will cause: Training and learning curves, changes in operational behavior and metrics, and impact the schedules. Changing the way a business works means everyone has to relearn what they do on a daily basis. This means considering what it will take to implement the methodology within an organization as a whole, and achieving a level of investment in the effort by all the stakeholders.

Team members need to be trained, business units need to be integrated into the process, schedules adjusted to accommodate the new methodology and in most situations a significant learning curve will translate into a slow, steady adoption — as opposed to a sudden, rapid adoption. The former approach provides an opportunity for participants to learn the usefulness of different aspects of the methodology and to gauge its success. The latter approach — attempting to make a complete, rapid transition — often leads to failure during adoption. Too many interdependent processes that are not well-understood by the team leads to poor execution. This can lead to missteps during a pilot project, a time at which such mistakes are highly visible. Not having a steady, progressive and measurable improvement against existing techniques means criticism will come easily.

Measure your success

Creating positive, measurable metrics that demonstrate the benefit of a new methodology is critical. Part of the process is making sure training costs and the cost of adoption is tied directly to business goals. By coupling the business to the methodology, all stakeholders have a vested interested in success. Good metrics demonstrate that progress is being made — both providing a positive measure of success, and avoiding the need for a “big bang” success right out the gates. And, if you aren’t already tracking metrics and measuring success, this is an ideal time to find a management methodology that will.

Related posts:

1. Making Scrum work: Common failings in adopting Scrum
2. Scrum is not an agile methodology
3. Why Agile isn’t enough (and why it doesn’t work)

Steve Muench, PhD, asserts that “Training is one of the chief methods of maintaining and improving intellection capital. Because of this, an organization’s training can affect its value.” (Tech Transfer Newsletter, 2004). In fact, the market-to-book value of companies significantly correlates with training as a percentage of payroll according to Bassi and Van Buren (1999). Furthermore, numerous studies have found that the organizational benefits of training are extensive. According to one, by Loewenstein and Spletzer (1998), “the effect of an hour of training on productivity growth is about five times as large as the effect on wage growth.” Research by Bartel (2000) also finds that employers, not employees, ”reap almost all the returns to company training,” going on to conclude that return on investment generally ranges from 100 to 200 percent ROI.

So, the evidence is out there — yet, why are companies cutting back when they should be investing?

One possible reason is a lack of good communication regarding the value of training itself. This is particularly true in small and mid-sized companies. The large ones seem to have it figured out: They do the analysis and understand how important “sharpening the saw” is. When asked if Amgen had made any cuts to training given the economy, CEO Kevin Sharer said, “We haven’t cut back on that at all. Developing people is the future of the company.” (Fortune, 2009).

But smaller organizations don’t have the maturity to understand the importance of training.

Training is investing in the future. Unfortunately, it’s not enough to simply ask the powers-that-be whether they want to make that investment or whether they’d rather remain stagnant while the competition passes them by. In a numbers-driven world, it comes down to dollars and cents more often than not. This means getting out the trusty spreadsheet and doing some math to show its value.

It’s not hard. The key is keeping your eye on the “line of sight” from training to profitability. More than anything, this means making sure that training is directly applicable to the bottom line. Training must become a resource, just like any other cost-oriented tool or expense. There’s no problem buying new software if it leads to profitability. There’s no problem hiring new staff when it means more revenue down the road. So, make the case for training: Demonstrate the return on investment (ROI) that a training purchase will generate, in terms of revenue to the business.

ROI analysis can easily be applied to training expenditures. It’s the same process as any other expense: Does the investment make sense and is it necessary to meet the organizational goals? Sometimes it’s an easy analysis to make. For example, if your company is currently losing money because of a poor process or incorrect practice, compare the current loss to the cost of correcting the problem (that is, the training that will fix the situation). Is the cost of training less than the ongoing loss (keep in mind that the loss is cumulative, continuing to accrue every year until it’s fixed)?

More complicated ROI analysis needs to focus on the financial value of your organization’s goals, and break that value down along the “line of sight” to the training investment. For example, let’s say your product launch is valued at $1 million, but half of that valuation is on the assumption you deliver a specific, key feature. That means your direct key performance indicator (KPI) for that feature is $500,000. Let’s further say that your team can only deliver two-thirds of the desired functionality without training. This means that the training program’s relevance to the KPI is about $166,000. Will the training itself cost less than $166,000? If so, you’ve got a business case to do the training. (You can round out the case with your ROI figure: If the training costs $20,000 then your ROI would be about 830%, which is a pretty darned incredible return on investment!)

If you’d like a more scientific, in-depth study of ROI methods, take a look at my article Should Training be an Integral Part of a Project Budget to Increase Project Profitability?, featured on the leading PM site My Project Management Expert.com. There’s a lot more research and some discussion of current ROI methods, as well as a couple of good examples.

Related posts:

1. Should Training be an Integral Part of a Project Budget to Increase Project Profitability?
2. It’s never a good time for training
3. Common oversights in choosing methodology

Indeed, a good question: What would we do without the hero?

Let’s flash forward a little ways. The hero has grown tired of the project and leaves. Fears abound that the project is doomed; there’s no way the team will survive without him. Concerns run deep, until the hero tells us he’ll stay on-board as a part-time consultant. Of course, he’s pretty busy these days — but, well, he’ll give us a “preferred” rate and show up at least a few days each week. He’ll keep the project afloat, no worries… in between getting his new company off the ground, of course. The one we’re funding with expensive consulting dollars. Either that, or the hero feels the project isn’t challenging enough, and leaves to greener pastures and more exciting projects, still promising to provide part-time support to keep the project afloat.

Either situation is dismal. Both demonstrate a project and a team that’s being held hostage to the whims of the “hero.”

The truth of the matter is simply this: Heroes are bad for the team, bad for the project and bad for the company.

Heroes are motivated by making themselves indispensable to the project by becoming irreplaceable. Most often this takes the form of information hoarding. The hero understands quite well that the established culture supports his role only while he and he alone can solve the project’s problems. One of the most effective ways to make sure this happens is to keep critical information away from the team. He’s the only expert in a few critical areas, and refuses to share his knowledge because it would be inefficient or too difficult to convey to someone else.

This leads to situations where the hero is subtly motivated to make sure there are instances in which only he can save the project. By ensuring the hero is the only one with the answers, the only one who can solve the problem at hand, he becomes indispensable — and, also, a tremendous project risk. Not only is everyone on the team constantly put in second-place to the hero, they also suffer as a whole when the hero is at his best. In times when the hero “shines,” the team is struggling to overcome roadblocks they haven’t been prepared to deal with. The project falls into jeopardy because the team cannot focus on a solution as a whole. Instead, the team churns in vain trying to contribute to the solution that only the indispensable hero can tackle.

Consequently, team morale is often a casualty of the hero culture. The continued failure of the team to exceed project goals, instead running into roadblock after roadblock, is tiring. Compound this with the fact that only one person, the hero, is enough of an over-achiever to solve the problems of a seemingly over-ambitious project and the team begins to become demoralized. Nobody likes to come face to face with failure repeatedly. Doing so in an environment that doesn’t provide the tools to better oneself is futile, and the hero team leader is certainly not motivated to fix the situation. In point of fact, most heroes tend to be pretty lousy team mentors, as being a hero implies putting oneself above everyone else. This means that while the hero has holed up inventing the next great solution, the team is left to its own devices. This vacuum of leadership provides a poor environment for anyone interested in advancing their skills and career, let alone seeking out a workplace that provides opportunity to learn.

Environments that support the hero become detrimental to the team. With the team leader focused on fulfilling the role of hero, collaboration suffers. Without a truly open, collaborative, information-sharing environment projects function at low efficiency (or, in some cases, fail to function completely as information is silo’d and isolated). In contrast, a healthy team elevates everyone, sharing information and skills, making the hero obsolete and making the entire team indispensable. A collaborative team rapidly turns into a formidable force when its collective attention turns to any problem — such teams turn out productive results faster and more efficiently. Each individual’s growth becomes a focal point, and the positive experience and knowledge gained from such a working environment becomes a lesson that each team member can share. In an open, collaborative, delegating environment the team lead will mentor the team; constantly challenging each person to take on more responsibility and grow into new opportunities. The best team leads are not information hoarders but information sharers, almost trying to engineer themselves out of the equation by teaching everything they know. In fact, this kind of leadership becomes truly indispensable because so few people are able to teach, motivate, mentor and unify a group of people toward a single purpose effectively. It’s not the information hoarded in the team lead’s head that makes him invaluable, it’s his ability to create a hyper-productive team and get things done.

As a project leader it’s important to confront the hero mentality head-on, making it clear that a hero is known for what it is: A detriment and risk to the project. Heroes are a liability. They are a bottleneck to progress, introduce the risk of “hostage projects,” generally make poor mentors and leaders, and always create specific situations that are harmful or dangerous to the project.

Unfortunately, putting an end to hero culture is often not an easy task. Many workplaces that embody hero culture don’t understand the problem — they think everything is “just fine,” and look to the hero as a critical resource, someone that has saved the project or the company time and again. Especially in young or inexperienced organizations, the difference between a supportive environment and a destructive one is unclear. The hero continues to operate above and apart from the team, often disregarding what little authority or direction he disagrees with. Inexperienced team members don’t know they should have a team lead that is mentor, guide and teacher. Organizations can be set up in such a way that the hero figure is empowered beyond reasonable boundaries, as often happens when clear structure and accountability is not in place. Combined, these problems can create the environment where a hero-mentality, embodied in someone who is ill-suited to create unity, lead, and mentor, ends up holding the project hostage. Most often, the evidence will point to the root problem: The team will constantly run into problems only one person can solve; team members will disagree with the hero and often not achieve consensus; much of the team will be “out of the loop,” particularly where the hero is involved; overall, the team will operate either apart from the hero or as individuals, not a cohesive group. Perhaps most indicative: The hero has sole authority over the project, yet little accountability. This is common in so-called “flat” organizations. I tend to avoid organizations that strive to be “flat,” as it’s really just a way of saying “we don’t want to deal with the fact that someone has to be in charge.” The simple truth is that leaders and managers need to have the authority to implement policy. Good leaders and managers will find ways to do it without using their authority unless necessary.

The easiest way to fight hero culture is from a position of authority, such as the project sponsor or project manager role. Given the advantage of authority, the hero can be given a choice: Either become a collaborative leader, or face what amounts to demotion as a new leader steps in. Some heroes won’t be able to make the right decision and will end up leaving the project — but others will embrace the idea of transforming themselves into a positive influence. I’ve seen this happen in a few cases and can honestly say the results were extraordinary. It could be that your hero’s spirit is willing but he doesn’t recognize there’s a problem. Likewise, it could be that your hero is struggling to move into a management role and needs guidance himself. Sometimes you’ll find out the hero was thrust into a leadership role without wanting it, and will happily step aside.

Whether tackling the problem from a position of authority, or from the inside, there are a few strategies that will help to ease the process. Developing a collaborative environment is a first step at ending the hero culture. This means putting in place tools and processes to share information, including an open information environment. Get information out of everyone’s head and into a tool that facilitates group review and commentary, such as a wiki or document repository. This can begin by emphasizing brainstorming sessions, collaborative documentation and group exercises to design and implement solutions. Often group development can be a productive tool, as well (some environments, such as Extreme Programming, push pair programming as one example — I don’t believe pair programming should always be put in practice, but this is a good example of when it makes sense). Project deliverables should also be managed in an open, collaborative environment. Use a project or task tracking system that lets everyone see what’s going on, what’s being delivered and how it’s being done, and most important: Focus on shared responsibility and avoid having unique specialties in favor of collaboration.

The team should also push for opportunities to advance skills and individual knowledge. Avoid “information silos,” or individuals who hold all the answers to a specific problem. It’s healthy for teams to have more than one expert in an area. Not only does it reduce risk, it also affords a more collaborative environment where two or more people can openly discuss a solution and work together on implementation. Any time you hear “only one of us knows how to do that,” immediately think in terms of how you can turn one into two or three people.

Demand a leader that is a good mentor. If the team hero isn’t up to the job, find someone who is — and be vocal about wanting an environment that supports growth. A healthy work environment includes ample opportunity to take on more responsibility. Any project manager that hears you want the responsibility will start casting about for a means to give it to you — and that usually means making sure the team leader can advance his team.

Ultimately, your work environment is in your hands. If you’re aware of the problem, identify it and and surface it — but do so in a constructive way. Have ideas and solutions ready to solve the problem, and emphasize the value it will bring to the team or organization. Focus on the facts as much as possible, like productivity gains the team will experience when it has greater bandwidth, and how getting to market more rapidly (and probably with a better, more robust product) will boost return on investment.

Related posts:

1. Hiring for the culture
2. Fix your boss (or, reduce risk to quality using a matrix approach)
3. Whole teams

Scrum can be remarkably beneficial in many kinds of software projects. But, as with any process, methodology or management technique, when used inappropriately it can cause more problems that it solves.

In this article I’ll discuss some of the common misconceptions and “lessons learned” as related to Scrum.

You don’t need training to use Scrum

This is the most important one in my mind: Training is just as necessary with Scrum as it is with any other process. I’ve run into many teams that eschew formal training in favor of simply “reading the book” or sending one lead engineer to a quick Scrum certification program.

Let’s look at it another way: For upwards of 30 years, the software development industry has followed a largely industrial process (sometimes called “waterfall” or “spiral,” depending on your particular experience). Across that time, many professionals have struggled with the complexity of software development, striving to create a process that is simple, reliable and repeatable. More often than not, to varying degrees, the industry has met with failure: Nobody has turned software development into a wholly reliable, repeatable procedure that always hits its goals dead-on.

Scrum is a process that has evolved out of this 30 year history. While the process itself focuses on simplicity and emphasizes a set of easy to follow practices, this does not make it inherently intuitive or trivial. In some regards Scrum is so contrary to established process that many team members just “won’t get it” unless they experience it in a learning-enabled setting. Training is critical, and I highly advocate sending your entire team to a Scrum workshop — or, better yet, bring the workshop to your team. Good programs exist that will turn your training session into a guided working session, which means your productivity will not be hurt by sending people off-site for training. Just the contrary — your productivity will leap ahead in just a week or two as training, learning absorption, experience and actual hands-on work all combine to create a Ken Schwaber’s original goal for Scrum: A hyperproductive team.

Scrum is all we need

Often sung to a Beatles inspired tune, I’m afraid it’s not true: Scrum is not all you need, and any idyllic fantasies otherwise will be met with dashed and broken hopes on the rocks of failure. Scrum is not a complete methodology, it’s a management control process — and that implies that you still need methodology. Don’t take my word for it, take Ken Schwaber’s, the creator of Scrum: “Scrum is superimposed on top of and wraps existing engineering practices, development methodologies and standards.” (Schwaber & Beedle, 2002; Agile Development with Scrum, p. 2).

Scrum is an excellent control process that helps us cut through the red tape of more traditional, bulkier control processes. It helps us to focus in on what’s important. It enables accountability to an extreme degree and, through accountability, achieves greater productivity. Its empirically driven measurement technique is well suited to the often experimental development environment of new products, such as inventing new software.

Nowhere in the Scrum process itself do we discuss development methodology. This means we must bring our own methodology to the table and wrap Scrum around it, enhancing processes that have not been hyperproductive and increasing their productivity. In other words: Software quality assurance, formal testing, configuration management, risk analysis, budget planning and project scope planning, verification, requirements management and specific methodologies such as the Rational Unified Process, Extreme Programming or Spiral programming must also be part of the equation.

The backlog replaces the Project Design Document

Here I’m essentially belaboring the point that Scrum is not all you need. Different development methodologies employ different kinds of design strategies and documentation strategies, but one thing is common across almost every single methodology: There is always a Project Design Document (PDD) in one form or another. The project backlog is not a complete design document. A well-implemented project backlog is going to provide enormous technical detail and will probably fulfill all the requirements for a technical specification — but this is only part of the PDD. Team collaboration is not a substitute for someone putting the product design specification in writing.

The Project Design Document addresses a host of project objectives and requirements that simply won’t show up in the project backlog, at least not in an easily interpreted, cohesive fashion. Some of these include: Overall project scope; market objectives; user interface models; business models; user acceptance criteria; original customer requirement statements or vision statements; a project baseline; key risks and mitigation strategies; user interface style guides; software architecture and software integration procedures; and the list goes on. Many of these artifacts will, in time, turn into very specific project backlog items, but many will not. It’s important to document not only the specific, day-to-day deliverables of the project, but also the overall goals, architecture, and design guidelines.

One of my favorite product suites for managing both the product backlog and project documentation is Atlassian’s JIRA and Confluence. Combined, the tools provide fantastic support for managing tasks and deliverables (the project backlog) and a collaborative documentation environment. The ability to seamlessly create links between the JIRA tickets and more in-depth, collaborative Confluence documentation is an excellent tool. It allows the team to expand on technical detail and cross reference extended documentation, all within the context of the product backlog.

The 15 minute daily scrum is unnecessary

I’ve run into a lot of resistance to the daily scrum, especially very early in the process of introducing Scrum to a new team. Nevertheless, within a matter of a month or two, the value of the daily scrum has been realized and wholly adopted by all of the teams I’ve led.

One of the worst situations you can get into is a “fuzzy scrum,” one that doesn’t start on time or that exceeds the boundaries of the meeting on a regular basis. If the Scrum Master shows up five minutes late, then politely waits ten minutes for everyone to gather, a focused 15 minute briefing turns into a 30 or 40 minute distraction that everyone resents. Stay focused on the daily scrum rules of conduct — specifically, start on-time every day, in the same meeting place. The Scrum Master must arrive early and be ready to kick off precisely on time. If someone is late, don’t hold up the meeting. Use a timer to casually ensure that everyone gets a turn, and nobody goes significantly over their time limit.

A well-structured 15 minute daily scrum will benefit the team enormously. It keeps the entire team apprised of progress and current goals. It helps to focus the team, avoiding situations where one person gets derailed, instead giving them support from the team. Remote workers and partner-vendors will stay in-touch with current goals.

I can’t count the number of times something has popped up in a daily scrum that resulted in days, potentially even weeks of gained productivity. I’ve seen team leaders that sit next to their developers react with surprise when they hear what an individual is actually working on. In situations like this, enormous effort has been saved when the team leader corrects the objective right then and there.

I’m not allowed to work outside of the sprint goals

Another very frequent misconception is that Scrum introduces artificial roadblocks; for example, forcing the team to work on current sprint goals and nothing else. This is only partially true: It is correct that Scrum focuses the team on the current sprint. The objective is to complete the sprint and then move ahead to the next set of objectives. This means that an individual team member must finish their sprint goals and, ideally, try to help out other team members with other goals, if possible.

However, once the team member is done with their sprint goals they are free to “look ahead.” There is nothing that says a team member can’t start working on the next sprint ahead of schedule or — if the next sprint doesn’t have enough definition yet — even draw directly from the product backlog.

Scrum is about productivity. It will never require the team to stop work because of an arbitrary set of procedures or policies. Quite the opposite: Well implemented, Scrum enables the team to become far more productive than previously possible.

We don’t have to adopt all of Scrum

This issue has come up with every single client and, I fully expect, will continue to do so. Technically, it’s true — but not initially.

As with any process or methodology, it’s critical to understand how the process works properly before customizing it. Scrum should be followed entirely and rigorously for at least the first month or two of a project. Ken Schwaber’s recommendation is to do so until “several sprints” have been successfully delivered and the goals of achieving a hyperproductive team are met. As Clara Ko writes in Jeff Sutherland Talks Shock Therapy For Scrum, “Perhaps there are good reasons for excluding some parts of Scrum, but without truly understanding the reason behind a Scrum practice or the implications of skipping it, agile teams struggle to become hyperproductive, or mistakenly, struggle with Scrum itself.”

In other words, before changing Scrum, become an expert in it. Once the Scrum process has become second-nature it becomes much easier to begin customizing it or cutting pieces of it out. It will also become clear why something stopped working smoothly. Past experience will give the team the awareness to know the process broke because it was changed, and the motivation to move back to a working process.

With proper training and guidance, teams uniformly experience incredible productivity gains with the Scrum process. They key is adopting Scrum correctly, and that means doing it with the right level of training and commitment, especially early on. Process change is never easy, and while Scrum may appear to be a simple process it is not an intuitive one.

Related posts:

1. Why Agile isn’t enough (and why it doesn’t work)
2. Common oversights in choosing methodology
3. Scrum is not an agile methodology

A ScrumButt is a sort of like Scrum implementation… but some changes that were too painful have been left out… Companies in this category tend to only experience moderate success with Scrum, i.e revenues up 0-35%. This is very different from the design goals Jeff Sutherland had for Scrum, i.e. to create hyper productive teams and hyper-profitable companies.

In 2005 Bas Vodde, agile pioneer and coach within Nokia Networks, created the original test. In 2008, Scrum co-creator Jeff Sutherland extended the test by introducing new questions and a scoring model. Today, Jeff works with openview venture partners to help investors make money by only investing in companies with state of the art software development capabilities.

The test itself is incredibly informative — and the results usually much more so. Merely answering the questions posed in the Nokia Test will reveal some obvious shortfalls in most Scrum implementations. But take care to answer honestly and objectively. It’s not a bad idea to make the exercise a team activity, reaching serious consensus on the answer to each question.

Related posts:

1. Making Scrum work: Common failings in adopting Scrum
2. Scrum is not an agile methodology
3. Rational Scrum

Just as people have specializations, so do processes. Applying one process to all situations is just as wrong as calling your dentist when you need brain surgery.

Rational itself is an excellent methodology and scales very well. Starting with only a few Rational artifacts, it can almost feel like Extreme Programming. Yet, the Rational body of work provides a framework that supports a much more comprehensive methodology… something less than full-on Spiral development, but still robust enough to handle large-scale, distributed team development.

However, for all that I like Rational, it does have some holes… and these are plugged most wonderfully by Scrum. In fact, Rational and Scrum benefit each other so well I’ve started referring to the combination as “Rational Scrum.” As with most methodologies, it does not define every possible response to every possible situation, but the combination of these two techniques is very robust and complete.

The processes complete each other by addressing mutual weaknesses. Scrum steps in to fill a gap in organization and team management. Conversely, Rational brings a structured, risk-driven approach to Scrum that is lacking.

Most recently I was able to put the principles of Rational and Scrum into effect with a client. The initial engagement was, let’s say, “unfortunately typical.” The client had about 45 technical staff working in a disorganized fashion on unclear requirements. Most of the work was outsourced to a third party, further distancing the team from key business knowledge. Product releases had been very slow or nonexistent for the past year—what releases there were had been seriously compromised in quality. Development felt isolated from the business and quality assurance was a poorly funded afterthought. The production support group of about five people were out of the loop on software deliveries and constantly scrambling to stay on top of customer driven emergencies.

My first task was to get my mind around what was going on. I took a trip to the outsource vendor (amazingly, this was the first trip anyone from the company had taken to the vendor) to get to know the team. I spent several weeks understanding the issues and bringing the team up to speed on what the business was really all about. Because of time differences (about thirteen hours) this sort of cross-pollination had never happened. Ideally, I would have set up a program where the entire team would have rotated through on-site “shifts” of several weeks duration, but budget was limited. This would have created a far more integrated team and given each person on-the-ground experience with the business, its goals, its priorities. I also delivered some training on Rational methodology in a compressed one-week course to the entire development team.

One of the most glaring problems was a complete lack of information share between the business and the development teams. We immediately purchased and implemented task and defect tracking (with Atlassian Jira) and a document repository (with Atlassian Confluence). Effective immediately, all information had to be migrated to the new system. If it wasn’t in there, it didn’t exist. I went so far as to return, unread, any technical email or document I received unless it was in Jira or Confluence. Absolutely no defect, feature request or deliverable could be discussed until it was put into Jira, given a once over to make sure the issue was stated clearly and, in most cases, given a quick first-pass effort estimate from development.

We also started having daily Scrums. This proved to be the most difficult challenge—the time difference pretty well necessitated a tight meeting window. Once back in the U.S., I ended up scheduling nightly telephone calls with the development teams from 8:00 PM until about 11:00 PM every day. This went on for months, but was necessary to drive improvement. We typically kicked off an evening with the “management Scrum,” limiting it to no more than half an hour (I doubled the time limit because it was virtual, organization was difficult, and at least initially there were a lot of issues to work through). Once the management Scrum wrapped up, splinter Scrums started up with each of the individual development, quality assurance, testing and professional services teams. Even following Scrum guidelines to keep the meetings on-track, we initially ended up with plenty of overflow. This was followed with a management Scrum on U.S. timetables, the following morning. Initially, it was a grueling meeting schedule for me, but we kept individual team member involvement to a minimum—preferably one daily Scrum.

Another major change was implementing the Rational Unified Process. This took upwards of six months to effect as we introduced light processes initially, adding to the Rational methodology over time. A few of the most important aspects of Rational that we adopted early on were risk-driven development and truly iterative releases. While development had been using iterative terminology, the fact of the matter is that monolithic development practice was still being followed (for example, creating massive specifications and trying to build to the specifications rather than focusing on short-term release goals and continuous improvement). By focusing the team on short-term releases, iterative release cycles and a product backlog (all done in Jira and Confluence) we quickly raised the level of understanding, across the board, of what was coming and what it would take to deliver. This information started to flow throughout the organization and we started to see its effects in how product features were prioritized, how releases where scheduled, and in customer service and the product life cycle itself. Soon, the company went from half a dozen silos of locked away information to a single, relatively smoothly operating organization with across-the-board visibility.

During this evolutionary phase of these changes we discovered all kinds of interesting things were going on. We found separate development teams working on the same foundation technology. Optimizations that could be made with small shifts in the business goals. Streamlining and efficiency gains by changing our release cycle. Massive effort savings by disregarding a few relatively unimportant features. More streamlining that came from the information sharing. The product backlog started to become self-prioritizing as business owners and other stakeholders participated in setting the upcoming release goals. Distractions, fire-fighting, context-switching and misdirection disappeared and the development team started producing true forward progress for the first time in a couple of years.

In fact, the team went from essentially zero forward progress to a steady, major new software release every six months.

We also trimmed operational costs by about $1.2 million dollars per year. Of the 45 outsourced development staff originally on the project, we reduced the count to fifteen in less than a year. And they still kept producing a major new software release every six months. Despite a team size of about one third, we were delivering more and getting more done in less time.

We implemented improved quality assurance measures and actually increased the size of the software testing team from one person to three (all within the 15 person head-count). Software quality started to improve at a dramatic rate.

Through Rational Scrum we had realized a productivity increase of somewhere in the vicinity of 300-400% if we account for the team size reduction. We had achieved positive forward progress where it had been lacking for well over a year, if not two, and had done so after cutting the development team down to one third of its original size.

Toward the tail end of my involvement, after about 18 months on the project, the team size had been scaled back a bit more. The daily Scrum meetings had become smooth and unintrusive, going from two or three hours down to about 15-20 minutes per team. New technology teams focusing on completely new product lines were being hired and integrated into the process very effectively. Even the professional services group had adopted the process and, likewise, had reduced its staffing requirements from five to two and was operating more smoothly. Customer satisfaction was up and the fire-fighting had pretty well come to a stop.

The two processes—Rational Unified Process and Scrum—gave the company the focus and clarity it need to get back on track. Working together, these processes had delivered improvement to every aspect of the organization:

1. All stakeholders in the project become a part of the development process. Everyone involved is genuinely interested in the project output and actively contributing torward progress.
2. Visibility is raised across the board through information sharing. This improves efficiency, mindshare and contributes to progress as well. The group contribution is far more valuable than individual “information silos.”
3. Truly iterative development delivers working, progressively improving software rapidly. Incremental change is easily accommodated by avoiding the “big bang” and monolithic development approach.
4. Rational’s focus on risk-driven development keeps the team focused on the most difficult, potentially risky aspects of the project first. This contributes on a number of fronts: Major problems stemming from unknowns become identified and are dealt with early in the project. Major change stemming from these unknowns has less impact because it happens early. The riskiest, most complex portions of the project are developed first, and thus mature first and receive the most testing.
5. Risk analysis and mitigation “bubbles to the top” through the daily Scrum meetings.
6. Priorities come in-line with business and technical reality as the whole team works on sensible, informed schedules.
7. Ownership and accountability become the mainstay as everyone develops a long-term interest in the project and pride of work product in their contribution. Commitments are taken seriously as a team.
8. Visibility into the entire process means there are no surprises and no last minute bad news. The business can plan according to reality and influence the overall program in an informed manner.
9. Progress is realized for effort and planning, not for putting “hours in the cubicle.” People are rewarded for good work and not for punching a clock.
10. Quality becomes integral and continuous. The Rational process integrates quality assurance and structured testing from the very beginning of the project. It is never an afterthought.

Rational Scrum delivers a solution for small-scale as well as large-scale project development. It supports the essential aspects of project and program management that make quality software development a possibility. It adds the dimensions of reliability and repeatability to projects and focuses the team on continuous improvement and productive work.

Related posts:

1. Making Scrum work: Common failings in adopting Scrum
2. Why Agile isn’t enough (and why it doesn’t work)
3. Scrum is not an agile methodology

Is this attitude regarding education part-and-parcel of the declining attitude toward education in the United States? More to the point, how will organizations continue to function if they curtail strategic learning initiatives? People don’t “just know” how to apply complicated concepts. They need training, particularly in areas such as software quality assurance, safety & reliability, validation & verification, and the basics of leadership, mentoring, and working as a team (including formal process education). And I haven’t even touched on morale issues, upward mobility and challenging our employees to aspire to improve their career path.

The fact of the matter is, in today’s economy we should be pressing for more education not less. This is called “recession proofing your organization.” Inevitable pressures to reduce staffing means fewer employees will need to do more, do it more effectively and take on new challenges they’ve never faced before. How about preparing them for it?

So, in an environment that doesn’t support training, how can organizations find educational funding?

Recognizing that training is not a wasted expense is probably number one. This needs to happen with management, but it doesn’t need to begin there. Training programs that are relevant to your work and beneficial to your employer are clear wins. Document the benefit of offering greater cross-functional capability within your team, of improving your efficiency, and reducing mistakes. Put together a training plan that demonstrates how the organization will benefit before you take a training request to your manager.

Consolidating training is also a huge win in most cases. For example, if you send five people to training courses around the country you’ll likely spend at least $15,000 (assuming the course costs $2,000 and then factoring in reasonable travel costs for each person). Consider bringing training on-site instead. Most programs that are available in public classes can also be delivered at your organization, and tailored to your specific needs. You get better, more relevant training and comparative costs are low. (An on-site workshop costs as little as $5,000-10,000 and becomes more cost-effective with more employees).

Use digital learning tools to reduce training costs as well. In particularly tight times, many web-based training programs exist. While not as effective as on-site, person-to-person training with an expert they can still provide a huge benefit to the student.

Focus on internal cost savings as well, and justify how the saved costs should at least in part be translated into better organization capability through training. For example, if you have more than one document repository (a problem many organizations suffer from) implement a single, consolidated document management system to improve efficiency and lower licensing costs.

One final idea that might find some traction: Audit your internal learning requirements and processes. Put together an analysis that demonstrates organization weaknesses and tie those weaknesses to actual issues the organization has experienced. Document the costs of handling those problems in retrospect and show how improved capability and efficiency would have avoided the problems—and will likely avoid future problems.

Related posts:

1. The case for certification
2. Articulating the value of training
3. It’s never a good time for training

In some industries involvement of the “whole team” is almost assumed. Would you buy a car that had never been tested in a safety lab? How about try out a new instrument configuration on an airplane, knowing it had been redesigned by engineers and not pilots? Does it seem reasonable to build a house without hiring an architect? Each of these examples will more often lead to failure than success.

Yet the software industry, particularly the commercial industry (as compared to Military, for example) has been ploughing along without whole teams for decades—a trend that seems to be getting more and more negative attention. Recently evolving standards such as Extreme Programming, Agile methods, Scrum and the Rational Unified Process have all incorporated whole team concepts into their methodologies.

The Whole Team Approach

The term “Whole Team” defined: What is it and why is it so important?

The concept behind the “whole team” is that building a product without representation from every stakeholder is fundamentally flawed. It takes constant, comprehensive involvement through every aspect of the product life cycle in order to guarantee timely delivery of a well-designed product. For example, without a quality assurance organization’s involvement, it is almost certain that customers will discover incomplete requirements after delivery. If the user perspective is not considered early on, it’s very likely the product won’t be accepted or even usable until a second generation is built. Failing to involve software testing and configuration management means failure to deliver working code. And not involving program management, marketing and even financial oversight can lead to a host of problems in execution. All of these situations arise from lack of whole team involvement and can easily lead to a product stumbling out the gate or outright failing.

Creating a cohesive whole team means building a project team with representation from all stakeholders in the project. Doing so requires both defining the team and creating involvement. It also means creating a structure for involvement that does not slow the project down. Creating this environment of constant, informed involvement does not mean throwing everyone into long (and ultimately unproductive) meetings every day.

In some ways defining the team is the easy part: It’s answering the question “who cares about the outcome of this project?” Project analysts, business stakeholders (such as the project sponsor and someone that represents the financial investment), designers, usability analysts, quality assurance, development, software testing, configuration management are a few. Quite often some of these groups can be represented by a single person, as is often the case when it comes to marketing, finance and business interest.

Once the stakeholders are known creating an effective, productive environment can be quite a challenge. This is particularly true with large efforts—perhaps 50 people or more. The goal is to achieve higher bandwidth of communication between team members. In fact, achieving this goal has been documented (see Agile Software Development With Scrum, Schwaber, Beedle) to increase efficiency by an order of magnitude: Over 100% not just 10 or 20%. To achieve this, you must:

1. Organize your projects around cross-functional teams—the core of the “whole team” concept. The team must represent the business and must contain analysts, developers, quality assurance and testers (as well as the related disciplines).
2. Provide the team with the tools needed to create an effective collaboration across all functions.
3. Foster an environment that creates an attitude of “I’ll do what it takes to exceed the goals of the project, on time, and with attention to quality,” not “I’ll do my small part in the overall life cycle.”

Small teams are almost self-organizing, but larger groups require more forethought. The project team must consist of analysts, developers, testers, quality assurance, a project manager, architects and so on. Once the project scope grows beyond what can be feasibly (and efficiently) organized into a single team we can organize into “teams of teams.” This means having an architecture team in charge of system architecture and deciding on the subsystems and how they interrelate. Then, a cross-functional team (including analysts, developers, testers, etc.) are in charge of each subsystem. The entire project is tied together with the management team, with representation from each of the subsystem teams, and through communication channels that must be free and open between all teams. (Several examples and case studies are cited in Agile Software Development With Scrum, among other sources).

An essential element of this process is open communication and information access. In order for each team to work closely within itself, and for it to work closely with other teams, they need access to information. Creating an environment that ensures access to requirements, design details, customer requests, business goals, defects, test plans and status, and so on is critical. Furthermore, communication must be unified. This means establishing an infrastructure for communication and setting it as the sole authoritative source for information. Invest in a document management system and an issue tracking system and develop the attitude that “if it isn’t in the system, it doesn’t exist.” Make sure there is a sole-source for information, and that email in particular does not become a “transient information store.”

Traditionally, at least in some cultures, functional organizations have evolved: All the analysts are in one group, all the designers are in another, all the quality assurance and testing in yet another. This silos information and creates a barrier to ownership that is almost impossible to overcome. Even more so in organizations that use matrix models—where, for example, an analyst works on several different projects at the same time. This leads to treating people like interchangeable components. It also generates a huge waste of effort and efficiency in context switching between projects.

In contrast, the whole team approach creates a vested interest and ownership in the project goals. Each team member develops a long-term investment in their contribution to the project. The ability to concentrate on a single goal creates focus. This is not only more efficient, but working as a team reinforces the sense of ownership between team members. Accountability is high, as is pride in work product. The team orientation avoids finger pointing and assertions such as “that wasn’t my job,” and “it works for me.” Everyone must share project responsibility and, as part of the small team dynamic, will work together to solve issues as they arise.

I think this paragraph, from The Rational Unified Process Made Easy (Booch, Jacobson, Rumbaugh), summarizes the goals quite well: “An iterative approach increases the need for working closely as a team. Avoid functional organizations, and instead use cross-functional teams of generalists, analysts, developers, and testers. Ensure that the tool infrastructure provides each team member with the right information and promotes synchornization and round-trip engineering of artifacts across disciplines. Finally, make sure that team members take joint ownership of the project results.”

Or, perhaps a bit more passionately (the “gang of three” books do tend a bit on the dry side), as Mike Beedle and Ken Schwaber wrote: “When we say Scrum provides higher productivity, we often mean several orders of magnitude higher i.e. several 100 percents higher. When we say higher adaptability we mean coping with radical change… we show through case studies that Scrum reduces risk and uncertainty by making everything visible early and often to all the people involved and by allowing adjustments to be made as early as possible.”

However, it is important to distinguish that Scrum in and of itself is not a complete process. As pointed out in Mike and Ken’s excellent book, Scrum wraps other processes and, in fact, we find that most processes tend toward artifacts that share a great deal in common with Scrum.

Processes such as Rational, spiral and traditional waterfall methods have a lot to offer when combined with agile methods, including Scrum.

Many of the traditional methods of monolithic software development also have a great deal to offer. Formal Software Configuration Management, Quality Assurance and Structured Software Testing are three such examples. Many agile methods focus so closely on rapid progress and lightweight process (or lack of process) that they verge dangerously close to unstructured chaos in and of themselves.

Finding the right balance between agile methods and heavy process is an ongoing decision—one that takes place on a per-project basis and sometimes changes over the life cycle of a project. The best we can hope to do is develop a solid foundation in good implementation principles, keep the best, and put the rest on the shelf for future reference. I’m sure whatever we’re doing in a couple of decades, it’s not going to look much like today’s practices. But I do know this: A whole team approach is one best practice that’s here to stay.

Related posts:

1. Fix your boss (or, reduce risk to quality using a matrix approach)
2. Why Agile isn’t enough (and why it doesn’t work)
3. Making Scrum work: Common failings in adopting Scrum

1. Employers should not require certification.
2. Non-skill-based certification testing procedures have little value.
3. The Alliance is deeply concerned that uncertified, skilled workers will get locked out.

After considerable rumination I’m still feeling like the Alliance’s stated opinion here is fuzzy at best, misguided at worst.

I tend to agree that employers should not require certification for all things, but I disagree with the blanket statement that employers should never require certification. Wouldn’t you like to know that your management team is involved in current program management developments in the industry? Wouldn’t it be good to know that your Software Configuration Management lead didn’t pick up his knowledge entirely by the seat of his pants? Certification, just like earning a degree, demonstrates that a student has studied and understood specific subject matter. Particularly for team leaders this is valuable across the organization.

As for the value of different kinds of certification programs, I think this is a problem that will market correct. The market will tend to favor better certification programs. Certainly, I agree that skill-based certifications heavy on essay testing, problem solving and hands-on examination is the best way to go. But it’s neither here nor there. Education and certification is a good thing to support; if there are a few poor programs out there, they’ll eventually get weeded out.

On the Alliance’s next point, that skilled but uncertified workers might be “locked out,” I can’t help but feel the concern is misplaced. First of all, there are a number of industries where certification is required and sponsored by the employer as part of on-the-job training. But more to the point, I’m not at all concerned that the overly chaotic technology industry is on the verge of adopting uniform, required certification prerequisites. I can’t shake the feeling that the technology industry would benefit from looking harder at certification programs, much like the medical, scientific and even accounting industries do. Would you want an Uncertified Public Accountant working on your tax return, or a Certified one?

I’d love to see the Alliance restate their opinion, possibly stressing some of the benefits we could, as an industry and at the individual level, realize from good certification programs:

1. Keeping knowledge current. I don’t care how good a programmer you are. There is simply no way to stay on top of the best practices body of knowledge without looking to external sources. Picking up as much of that knowledge as possible through excellent training curriculum makes sense. The individual improves their skills; the organization motivates and energizes their employees and realizes benefits from best practices.
2. New discoveries in the field. There are new breakthroughs almost every day. The training and certification programs available today provide a venue for tapping into that knowledge, and in many cases the alumni associations create communities to share and develop ideas.
3. Demonstrating skill in established standards. Good certification testifies to two things, in my mind: First, that the individual is capable and competent enough in a field to pass an examination. That’s actually pretty darned valuable in the loosely defined technology field. Second, it tells me the individual cares about their continuing education and trying to be the best they can be in an area. Certification is optional. I’ve found that the people that get certified tend to be people that are more passionate, more enthusiastic, and want to be better at what they do.
4. Establishing a few standards. There are a lot of evolving standards in technology. Some are well established and many are brand-spanking-new. With this amount of technological turnover it’s hard to know what someone means when they say “I know Agile.” Having a common point of reference is not always a bad thing.
5. Assuring baseline knowledge in senior team. Without some common ground, how can we all talk about the same domain? By having at least the senior team go through with certification it helps bring all of the benefits of continuing education to the team using a common vernacular.

I’ve managed a lot of project teams over the past few decades. Almost every team that I’ve been asked to take direction of has had a wide range of skills, some barely adequate and some exceptional. Most of us have had similar experiences: Quality assurance groups where none of the staff have been formally trained in structured software testing, configuration management or even what quality assurance programs are all about, for instance.

I’m going to go out on a limb here and say we need certification. We need to know that the program manager knows how to manage across projects; that the quality assurance manager understands product life cycles; that the configuration team has a solid grounding in configuration identification and audits. It’s a natural evolution of this industry—an industry that is still very young, and showing it’s growing pains every time we look at it.

Related posts:

1. Finding strategic learning funds
2. What do you mean, SQA isn’t testing?

It turns out that when you search for defects determines how much it costs to identify and fix those defects. The formal inspections process emphasizes discovering defects early in the software life cycle. This is reflected in the average industry costs associated with defect resolution:

• Without formal inspections:
• 50 defects found at testing stage
• 10 defects found after the project ships, at operations phase
• Average industry cost: $700,000
• Using logic and code inspections only:
• 35 defects found in logic and code stage
• 25 found at testing stage
• 5 found during operations
• Average industry cost: $525,000
• Using requirements, design, logic and code formal inspections:
• 16 defects found in requirements and design stage
• 30 found in logic and code stage
• 13 found in test stage
• 1 found in operations stage
• Average industry cost: $300,000

Attaining such drastic reduction in defect prevention costs did not become possible overnight. Looking back over the history of software development and quality assurance methodology there are clear evolutionary breaks that have fueled strides toward better, more efficient defect prevention. Early efforts in the software industry focused on process definition as a primary means to eliminate defects. By the 1970’s, efforts in process measurement and improvement, combined with formal defect detection techniques, made significant strides in software quality possible. It wasn’t until the 1980’s that defect prevention techniques—such as formal inspections and unit testing frameworks—began to evolve.

Formal Inspection refers to a structured process of trying to find defects in development documents, programming code, specifications, designs and others artifacts during various phases of the software development program. Sometimes called a “Fagan inspection,” after Michael Fagan who is credited with being the inventor of formal software inspections, the process has a dramatic effect in reducing defects early in the project life cycle. This of course translates into dramatic project cost reduction and more reliable schedule projections.

In fact, according to a JPL Executive Briefing given to NASA in the late 1980’s, NASA’s projected budget for the International Space Station software program without the use of formal inspections would top $6 billion—compared to a revised budget of $4 billion after applying the formal inspection process.

So, what are formal inspections and how can they improve your projects?

Formal inspection is a defect detection, removal and correction verification process carried out by a small group during the pre-test phases of the development life cycle. It involves the interaction of the following five components of the software life cycle:

• Well-defined inspection steps
• Well-defined roles for the participants of the inspection (moderator, reader, recorder, author, inspector)
• The software product subject to inspection
• An infrastructure that supports the formal inspections

The primary objective of formal inspections is to remove defects as early as possible in the development process. Formal inspections achieve this objective by:

• Identifying potential defects during individual preparation
• Verifying that identified defects are actual defects
• Recording the existence of defects
• Providing the code author with a list of defects to fix
• Ensuring that fixes are performed and are correct

Formal inspections were designed to help organizations involved in software projects develop better products. While its primary focus is on software quality there are a number of peripheral benefits—dramatic budget benefits being the most obvious. The overall software life cycle cost is lower since defects are found early and are easier and less expensive to fix. By introducing formal software review and testing steps early in the software development life cycle a more technically correct architecture is developed and tested earlier. This fuels improved component reuse and drives development efficiency up. The effectiveness of the testing process and closely-related software quality assurance processes are improved because testing begins early, as well. This generally means that less time needs to be devoted to testing processes on a recurring basis. Another benefit of formal inspections is the immediate feedback on the software from a group of peers and target users. This creates a support loop that integrates well with modern ideas of iterative development and creates an environment where agility and end user feedback become a target of the development cycle.

There is a significant emphasis on the “formal” part of the inspection process. Formal inspections, while designed to be effective without impacting team schedules heavily, are more rigorous and well-defined than other peer review techniques such as pair programming and walkthroughs. Because of this they are significantly more effective, but they do not take the place of milestone reviews, progress reports, status review, testing or walkthroughs. Also, the process clearly defines phases of the software life cycle at which the inspections should take place.

Inspections must be carried out by peers representing the areas of the life cycle affected by the material being inspected. The team should be limited to six or seven people at most, and everyone participating should have a vested interest in the work product. This is a particularly important requirement of the process. Also, inspections must not be used to judge the quality of the software work product or the authors that developed the product. For this reason managers should not be present in the formal inspection itself; findings should be presented to management statistically or as a group of work product findings. This will demonstrate the value of the inspection process without singling out any individual author. Using the inspection to judge the capability of authors may result in less than honest and thorough results: If coworkers feel their efforts could result in a poor performance review for the team they may be reluctant to identify defects.

The Formal Inspection Team

Every formal inspection is carried out by a team of inspectors. The inspection team consists of four to seven individuals, and no more. In fact, the team should be kept as small as possible and yet be able to complete the job at hand. The goal of keeping the team small is to keep decision making and process to a minimum. It is also important to minimize impact to the overall development life cycle—superfluous involvement ultimately means time taken away from productive work on the project.

The inspection team consists of a moderator, a reader, a recorder, the software author and other inspectors. Each team member has a specific, defined role to fulfill. In addition to each individual role, it is everyone’s job to find and report defects; all team members are considered inspectors in this regard. Where necessary, more than one author will be involved in the inspection, although it is best to keep the inspection focused to an area that involves a minimum number of authors.

The moderator’s primary role is to achieve a good inspection. This includes planning the inspection, assembling the team and managing the overall process from beginning to end. The reader’s responsibility is to guide the inspection team through the work product being inspected (keep in mind that “work product” can include specifications, designs, actual software code or functioning software). The recorders job is essentially to record every discovered defect. The author’s job is to answer questions throughout the process, present the work product as needed by the inspection team and, ultimately, to correct the defects. In addition, larger projects may require support from the project librarian in keeping track of status, statistics and overall progress of the formal inspections throughout the development life cycle.

The most important guidelines to keep in mind when creating the formal inspection team are:

• Use a fair and unbiased moderator
• Keep inspection team size reasonable (between 4 and 7 individuals)
• Select inspection team members who have a vested interest in the work product

This last point is a particularly important one. Only those individuals that care about the project will be invested in the inspection process. Team members should be close to the project; peers working in the same life cycle phase, authors of the relevant specifications or code, direct users of the work product, product quality assurance and testing personnel are all excellent candidates.

Supporting Roles

Formal inspections are an organization-wide activity. As such there are several supporting roles that are essential to assure success from the process.

Most visible is the need for support at the Project and Program Management level. This means that the appropriate manager needs to establish a schedule that allows adequate time for all stages of the inspection process. This includes monitoring the inspection process and making sure that there is sufficient inspection preparation time, that individuals are not over-burdened with tasks and that all team members understand the critical nature of the inspection process. Ensuring that all team members are properly trained in the formal inspection process will help in maintain the schedule and making sure that everyone is prepared for an inspection in the alloted time. After the formal inspections have taken place, managers must meet with the moderator and author to review open items and rework estimates. The project manager will likely review inspection summary reports for defect trends and perform defect trend analysis.

Organizations may also find a need for a Chief Moderator. The Chief Moderator will chair monthly meetings of all inspection moderators and guide the evolution of the inspection process, forms and checklists, and information gathering. The Chief Moderator may also analyze the effectiveness of inspections across participating projects, provide support in the form of guidance and answers to questions and keep current on recent information and developments in the formal inspection arena.

Stages of a Formal Inspection

The formal inspection process is broken into seven stages, although two of these are optional and can be excised from the inspection process on a case-by-case basis, at the moderator’s choice. These stages are:

• Planning—The period of time used to determine whether the product to be inspected meets the entry criteria, set the inspection schedule, plan the inspection itself, select a team of inspectors and assign respective roles, and prepare and distribute the inspection materials. This is when the moderator decides whether an overview will be necessary, as well.
• Overview—An optional stage in the inspection process. The overview provides the inspection team with background information for the inspection. This stage may not be necessary if the team is already intimately familiar with the work product being inspected.
• Preparation—A critical stage during which each member of the inspection team individually prepares for the inspection. It is crucial that individual inspectors be given adequate time to prepare, otherwise the inspection process will not be efficient and may well fail to identify defects that could otherwise be discovered. Each team member prepares for the inspection by reviewing and finding potential defects in the product being inspected before the inspection meeting. Potential defects are then discussed during the inspection meeting as a group.
• Inspection Meeting—Meeting in which team members, as a group, review the product to discover, categorize and record defects. Defects are not resolved during this meeting.
• Third Hour—Literally, a third hour to the inspection meeting (the formal inspection meeting is limited to two hours). This is an optional additional time that can be used to discuss, possibly solve or further investigate defects that have already been discovered during the Inspection Meeting.
• Rework—The period of time that the author uses to correct identified defects.
• Follow-up—A short meeting between the author and the inspection moderator used to determine if the defects found during the inspection have been corrected and to ensure that no additional defects have been introduced.

The following few sections discuss each of these stages in more detail.

The Planning Stage

Chiefly, the moderator uses the planning stage to prepare for the inspection process. This entails making sure the product is ready for inspection, selecting the inspection team and assigning team members appropriate roles, scheduling the inspection meeting and making sure meeting facilities will be available, and distributing inspection materials such as forms and background information. The inspection materials should also include detailed information on the work product being inspected, the scope of the inspection and any specific components or functionalities that will be excluded from the inspection process, and individual checklists for each inspection team member to follow.

Another key role the moderator will play is making sure that the work product being inspected is of appropriate size for the inspection. If not, the product is divided into manageable pieces and separate inspections are scheduled for each piece.

The moderator also must decide whether the inspection team members are adequately familiar with the material to be inspected or whether an overview must be held. The team should know the background material from which the product was derived and should know how the material fits into the overall system being developed.

The Overview

This is an optional stage that chiefly flows out of the moderator’s work in making sure that the team is adequately familiar with the work product being inspected. If any team members do not have sufficient depth of knowledge about the product, an overview is warranted. At the overview, the author will present the rationale behind the product, its relationship to other components in the system, its current state of development (e.g.: how complete it is and how well integrated into the overall system), its function and intended use and how it was developed.

An overview should be scheduled if any of the following criteria exist:

• The inspection team is not familiar with the product
• The product is new or is being inspected for the first time
• Inspections are new to the project
• Novel techniques have been used in the product

Preparation

This is the most critical stage of the formal inspection process. During the preparation process, each inspection team member individually prepares for the role in the formal inspection meeting. Each inspector will review the work product thoroughly; user interface inspections will be conducted field-by-field while code level inspections will be conducted by reviewing the work product line by line. Higher level specifications should be reviewed by reading the specifications and comparing them to system requirements and expected system behaviors. Comparison against relevant work product and best practices should be made during the preparation process, providing a basis for quality standards and acceptable behaviors. Checklists should be used in the work product review for guidance on typical types of defects to be found. All potential defects are carefully logged and will be discussed during the formal inspection meeting. Complete preparation logs are submitted to the moderator prior to the meeting.

Prior to the inspection meeting the moderator will review preparation logs and draft defect reports. This review will both ensure that the inspection team is ready for the formal inspection meeting and also highlight any areas that may require additional attention. If the logs indicate that the team is not ready, the inspection meeting should be rescheduled. Areas of common concern between inspectors can often provide valuable guidance on areas of the work product that require particular scrutiny.

The Inspection Meeting

The inspection meeting is the period during which the inspection team reviews the work product as a group. The meeting is held to a limit of two hours and is carefully moderated. The meeting is run by the moderator and usually starts with a very brief restatement of the goals of the formal inspection. If an overview has been scheduled it is held prior to the formal inspection meeting itself.

During the meeting, the reader interprets and provides a reading of the work product, the author clarifies information as needed, and the team identifies defects that are then classified and recorded by the team recorder.

The reader’s description should note the function of components being review (whether paragraphs of text, code blocks or a functioning user interface) and their relationship to the product and higher level documents. The moderator is tasked with keeping the meeting on track, but the reader can be interrupted at any point a potential defect enters into the discussion. A short discussion of the defect is permissible, but the moderator should limit time accordingly. Any issue that cannot be resolved within the time limit will be marked as an open issue and can be revisited during the third hour.

The team should reach consensus on whether a potential defect is, in fact, a real defect. Quite often what appears to be a defect could be a mistake on the part of the inspector. At the same time it could be a flaw or deficiency in the requirements or other specifications (in fact it is very common to discover edge cases that were never considered during the specification and design process). These are considered defects as well and must be logged as such, even though resolution steps outside the bounds of the author’s work. During this process the recorder will note down every defect, its reproduction criteria or location as appropriate, and classify the defect. Ideally, if defect tracking software is readily available and defect entry is efficient enough that it will not slow down the inspection meeting, direct entry into the system is favorable. This provides the team with an opportunity to review the defect as it is recorded, avoiding possible interpretation or notation mistakes.

If the work product cannot be fully inspected within the two hour limit, to avoid fatigue and missing defects, the moderator should conclude the meeting and schedule a follow-on inspection at a later time.

The moderator will conclude the meeting with a brief recap of the number of defect found and the severity of the defects. All information should be recorded to track the effectiveness of the formal inspection process and to look to defect trends at a later date. If a third hour is needed, the moderator will assign action items to individual inspectors at this time.

Third Hour

The third hour is used for discussion or to close open issues—it should not be an extension of the formal inspection meeting. The third hour does not need to take place immediately after the formal inspection meeting either, nor does it need to be limited to one hour: It should be scheduled at a time when the author is prepared to discuss corrections to discovered defects or if major issues, such as defects or omissions in the higher level specifications, have been discovered. Attendees to the third hour are free to obtain support from outside sources or invite third parties, including managers or technical experts, to the third hour purely for technical reasons.

Rework

The rework period is used by the author to correct discovered defects, or to coordinate the correction of defects that fall outside the author’s domain (for example, if the defects occur in a higher level specification document). The author is responsible for ensuring that all defects have been corrected before the follow-up period with the inspection moderator.

Reinspection

This is an optional stage; the moderator and the team will jointly decide whether a reinspection is warranted based the inspection meeting findings. Reinspection may be required when there are a large number of defects in the product or when one or more defects require extensive or complicated corrections. Reinspection allows the changes to the product to be reviewed by the entire team instead of just the moderator.

Follow-up

The goal of the follow-up meeting is to confirm closure on major defects that were discovered during the formal inspection and that no secondary defects have arisen during the process. The meeting is held between the moderator and the author. The author reviews the steps taken to correct the defects and the moderator ensures that all open issues have been addressed. Minor defects do not necessarily need to be addressed in the follow-up meeting. Additional reviewers can be present during the follow-up meeting if either the moderator or author feel it will benefit the process.

Once all open defects have been corrected and any other open issues have been resolved, and once the work product itself passes the exit criteria (this could be reaching a compile state or passing quality assurance testing, depending on the nature of the work product) the moderator marks the work product as having “passed” the formal inspection. The moderator will note the product as having passed the inspection on the formal inspection summary report.

Scheduling Guidelines

The scheduling of formal inspections throughout the software development life cycle targets specific phases to optimally eliminate defects as early as possible.

Traditional defect discovery and removal techniques target defects significantly later and less frequently than the formal inspection process. Expectations with traditional methods are correspondingly limited: Experience is that 60 defects escape pre-testing phases for every 1,000 lines of code written. Traditional testing steps are each approximately 50% efficient in the identification and removal of defects. Provided a typical, traditional quality assurance method that emphasizes testing of a coded and working product, followed by repeated late-stage testing, an average of 3.75 defects per thousand lines of code reaches the operational phase—and ends up in customer’s hands. The following figure demonstrates the effect of traditional testing steps on the development process:

While traditional methods tend to focus on reviewing and testing completed software, formal inspections begins the process much earlier at the requirements, specification and coding phases. In comparison to traditional defect discovery and removal this is much more effective at identifying defects early and correcting them before they become a problem. The formal inspection process achieves this by:

• Inserting inspections into the pre-test phase of the software life cycle
• The strategy is to find and fix defects when and where they are injected
• Inspections mean more defect detection steps: At least 9 as compared to 4 traditional steps

The frequent and repeated inspection process also keeps defects counts manageable, as shown in the following figure:

The formal inspection process introduces inspections as early as possible into the software development life cycle, and continues to repeat the inspection process at key stages of the life cycle. This means introducing formal inspections after the program functional design, requirements, architectural design, detailed design and software coding stages. Iterative projects can take advantage of additional inspection steps after operational code delivery as well. Note that source code level inspections should take place before unit testing, as the corrections made during unit testing can hide larger scale defects that should be exposed.

There are two striking differences realized between the formal inspection process and traditional detection process: First, the average defect rate in operational systems drops from 3.75 defects per thousand lines of code to 1.05 per thousand delivered source instructions, about one fourth the defect level. Second, because defects are identifying and corrected early in the process total defect counts remain much lower throughout the software development life cycle. This has a positive, cascading effect throughout development: Fewer defects means fewer negative behaviors become “coded in” to interfaces—and that means codependent defects are also kept to a minimum.

This implies that formal inspections are about 7.4 times more productive than formal testing practices alone (based on 3.25 errors per unit effort with inspections compared to 0.44 errors per unit effort with testing, given that inspections yield a lower return to effort against testing).

Planning the Inspection

Inspections should not be cumbersome. In fact, the formal inspection process is designed to be quick, efficient and require surprisingly little preparation time. Even so, it is important the the inspection team be well trained in the inspection process and its benefits—and fully realize the relative effort that can be saved through formal inspections.

The inspection moderator will invest more time into the process than other team members, chiefly because of the advanced planning and preparation that goes into each inspection. Likewise, the author is likely to invest more time because of the rework phase—however, if we discount the actual defect correction activities, the author’s investment in the inspection is largely in line with the rest of the team: Only a matter of a few hours. Following are the approximate time estimates recommended in a single formal inspection:

• Planning (moderator): 2 to 4 hours
• Overview: Between 30 minutes and 1.5 hours
• Preparation: 3 hours per inspector
• Inspection: 2 hours maximum
• Rework (author): Up to 5 hours
• Follow-up: 2 to 3 hours

Keeping the inspection process tightly focused on the work product and on defect discovery will help keep the inspection on-track and minimize time taken away from other tasks.

It is crucial that the inspection team be fully trained in the formal inspection process. The moderator and project manager must also make sure that there is adequate time available for the inspection, and that the team has properly prepared. If there is less than five hours available for preparation time, the inspection should be rescheduled.

Summary

Formal inspections have proven themselves through the test of time. We have an extensive body of work that demonstrates their value in effort, time and cost savings across many projects. Reports produced by organizations such as the IBM Federal Systems Division (1986) show that the formal inspection process can find between 75% and 90% of all defects at early phases of the software development life cycle. And, we can look forward to a commensurate reduction in maintenance costs: As much as 90% of corrective maintenance is eliminated (Ackerman, 1989).

In fact, formal inspections are the single most important quality improvement technique a development can make, according to experts at IBM, Bell Labs, the Software Engineering Institute and other sources.

As well as these very tangible benefits, a number of less tangible benefits come from the inspection process. Frequent inspections aid in project tracking accuracy and reduce the effort involved in traditional project management. The frequent deep-dives into the system also works to foster better team communication and collaboration on the work product. This, combined with the iterative cycle of inspections works against repeated mistakes, fueling better software quality and team development.

All of these benefits conspire to produce a better product, at a lower cost and within a shorter time than would otherwise be the case.

Related posts:

1. Quality assurance as a way of life
2. Don’t ship broken software
3. Exposing the enterprise to risk: Who decides what not to test?