Why Design a Methodology
From Mp
Contents |
Introduction
There are many well-known software development methodologies, from the classic Waterfall model to Spiral, RUP, the Agile methods, etc. Surprisingly, unlike technologies, most of these have not gone out of style, and can be found in abundance in the industry today. Less surprising, though, is the amount of differing opinions, the many success stories attached to each and every one of these, and the amount of books, websites, and materials that support, contradict, criticize, or otherwise posit upon the proper way to build software. Many carry with them excellent insight into developing software, into how developers work, think, and act; many contain aspects of pure business sense and acumen, providing businesses with approaches to running more than just software development projects.
To put it simply, there are many fascinating and important messages spattered across the "methodology industry" (this includes much of the "Process Improvement" industry, as well). Unfortunately, there is no single "grand unifying theory" to pull the best from all of these into a cohesive whole. In most cases, few developers and project managers are even aware of much of what exists, let alone have made an in-depth study of what's available. In fact, it's questionable whether any single person would have the time to make anything more than a cursory dive into the vastness that includes requirements gathering and analysis to the huge world of software design to whole sub-industries of software estimation, quality assurance (which includes testing), or delivery management and maintenance.
And yet, these are a significant part of delivering business value and extending the capabilities of enterprises, governments, and individuals. It would seem, then, that utilizing an existing, well-known methodology would not only be a safe approach, but would be the obvious and appropriate method of developing software. Why then even consider designing a specialty methodology? Why design something that you know you don't have the skills to do, something that someone else has obviously put a lot of thought into, that has been proven through practice? What value could you add that has not already been added? What insights are there to be added? And, to top it off, if you even bother to spend the time and energy, you have to document, train, and maintain it. That supposes that you can even convince the rest of your team, organization or company that what you've created is valuable, when it's not been proven or tried in the real world.
Some History
Dr. Winston W. Royce is credited for defining and identifying the Waterfall model (see his IEEE article), though he argued against its use. His real vision was similar to the Spiral Model, which was later developed by Barry Boehm, which provided the reasons behind the need for iterations (though these iterations are loosely related to time-boxed iterations used by the Agile methodologies). Wikipedia has more detail related to the Waterfall Model and the Spiral Model.
Dr. Royce's article was published in 1970, with Barry Boehm's Spiral introduction coming in the mid-1980s. The Rational Unified Process was formalized in the mid-1990s, with the Agile Manifesto being created in the early 2000s (though methodologies considered "Agile" were already in existence for many years).
There are many other methodologies defined throughout this period of time, with some being very formally defined, while others were only approaches to development.
Why Tailoring Doesn't Work
Some methodologies, such as the Rational Unified Process (RUP), are considered frameworks. They're designed to be tailored and modified for a particular project or organization's needs. In fact, in RUP's case, it defines the "world" of the process: all of the artifacts, the overall process model, the milestones, etc. It's supported by a tool set that's used to execute the entire framework (or, only those aspects that are selected, based upon tailoring needs).
Other methodologies are considered complete as is. They can be tailored as "process improvement" needs are identified and implemented. However, the tailoring is expected to be minimal.
Tailoring is based upon two basic events: up front changes to adjust the methodology to suite a project or organization's particular needs; and adjusting the process as "process improvements" are identified and implemented. In other words, tailoring may occur prior to a project's beginning and may be adjusted through the project's lifecycle (or, more commonly, in between multiple projects).
Fundamentally, tailoring is a necessity as it provides the flexibility to modify a running project's process (or across multiple projects). This would make it seem an appropriate approach to defining the methodology at the outset: take an existing methodology and tailor it prior to project start.
There are a number of problems with tailoring an existing, off-the-shelf methodology:
- Tailoring tends to be done within the scope or context of the contained methodology. This limits access to other practices that may be more appropriate for a particular organization or project.
- At what point do some major tailoring adjustments make the new methodology look nothing like the original? For example, let's say you pull Extreme Programming (XP) off the shelf and remove pair programming and an onsite customer? Is it still XP or is it really something else?
- What approach is used to tailor? What are the fundamental values, motivations, etc. that are driving the changes (or even the selection of the methodology in the first place)? The methodology may not provide guidance on approaches to tailoring.
- How are the tailoring changes managed, trained, documented, and maintained over time? This is a problem no matter what methodology is chosen, but the difficulty lies in identification: you identify the methodology used, but still have to train the delta, which may mean the same amount of work in training and maintaining a ground-up design of a new methodology.
Many organizations have "designed" their own methodology that spans an entire organization. It's typically inspired by an existing methodology (most common are the Waterfall model and RUP). In some cases, these methodologies are market-driven, meaning that they are marketed and sold as intellectual property of the organization. They're considered a value-add based upon "years of experience." The reality is that, in many cases, actual work within these organizations utilizes little of the custom-designed methodology. There are a couple of reasons for this: first, the project work tends to be "out-of-sync" with the organizational methodology. Second, the methodology was designed, not from the ground-up, but from pre-existing methodologies with little thought put into how it would be actually implemented on a real project. Hence, there's little buy-in by individual project-workers.
Unique Constraints and the Value System
If not tailoring, then what? If not a pre-existing system, then what? Is it feasible for each and every organization to have a methodology expert on staff to custom-design a methodology for each and every project? What really is the goal of a software development methodology?
Though a methodology expert may not be absolutely required, it's obviously more effective to have someone with experience running software development projects design a methodology than someone that hasn't. However, it's surprising what individual, experience developers can come up with when it comes to defining a process that they will follow that will be effective and efficient. In fact, in some cases, they have more experience with methodologies than more experience technical experts.
Designing a methodology, in order to be effective, would need to take into account those business needs and values that define a particular organization (or even an individual team within an organization). The goal of a software development methodology may be different depending upon the organization and team. This single statement changes everything, since it implies that it's not only necessary, but absolutely required, that an organization custom-design their own methodology to support their own business needs and goals.
Since the core of any methodology is built around Requirements, Build, and Publish (see The Core Process), what's needed to add to this core are the constraints and principles of the business, organization, and team. For example, an organization that supports its own internal IT department has a different approach to defining the financial aspects of software development (i.e. the IT department is typically not a revenue generator) than a consulting company that builds software for other companies. This is a constraint that may affect the methodology (for example, change control may not be as important, since the internal value-add of changes typically far outweighs the cost, where that may not be the case for a consultancy). It's important to identify real constraints. These constraints are typically environmentally controlled (meaning, they are part of the way the business works or chooses to work).
An organization's principles and values are those aspects that they choose to apply to the methodology. For example, transparency into the process from the business side may be very important to an organization. If so, the may define additional aspects to the methodology to support that (such as additional reporting, etc.). In other organizations, this transparency may not be appropriate or desired.
A classic example is the similarity and differences between the principles of Quality and Customer Satisfaction. In some organizations, quality may trump customer satisfaction. In others, the role may be switched. It's important to know "which way the wind blows." These differences can have a significant impact on the way a methodology is defined. It may not affect the high-level model, but it may have a significant impact on the individual practices that are employed.
