"A project selection model can enable the organization to consistently and efficiently make the best project choices."
The goal of project selection is to choose the projects for the organization's project portfolio that, collectively, will generate the greatest possible value . However, to be sure that the best projects are being selected, the organization must have the ability to assess the value of its candidate projects. As Peter Drucker is credited with saying, "You can't manage what you can't measure."
The previous Part 3 of this paper demonstrated that identifying projects that generate the most value isn't simply a matter of finding and using the right metrics. Yes, you do need a specific set of metrics to accurately and comprehensively assess project performance. But, you also need to know the equation or model that allows you to translate project performance estimates into a quantitative expression of project value. In other words, in order to identify the best project choices it is necessary to create a project value model. The inputs to the value model are the proper project performance metrics, they are the project- and organization-specific estimates associated with a project that, when input to the value model, provide as model output the value of that project. You identify the appropriate project valuation metrics by building a value model based on utility theory. The project value model tells you what project measures are needed as model inputs. Without a project value model, there is no way to obtain assurance that the best projects are being selected. Inability to measure project value is the fourth reason organizations choose the wrong projects.
Project Selection Models
The value of a project is, logically, the worth, to the organization, of the project's consequences. In order to make the right choices, therefore, organizations need to estimate the consequences of conducting candidate projects (something organizations with good accountability practices do anyway). They, then, must determine the value of those consequences (the job of the project value model). As demonstrated in the previous part of this paper, utility theory provides the foundation for creating a project value model. A model that reliably estimates poject value enables the organization to consistently make the right project choices.
Project Selection is an Ongoing Task
For many organizations, project selection occurs mainly as a component of the capital budgeting process when resource needs for the upcoming budget period are being planned . These organizations typically conduct a comprehensive review of candidate projects and choose a subset for execution in the coming budget period. However, pressing new needs and unexpected opportunities can arise at any time . Also, some projects may need to be terminated due to unforeseen events or because it becomes apparent that the resources being consumed could be better used elsewhere. Project selection is, thus, not just a once-per-year activity, it is an ongoing task.
Why Choosing the Best Projects is Difficult
I've described the reasons that project selection is difficult throughout this paper. To summarize:
In sum, choosing projects involves complex considerations. Selecting the best project from many alternatives is often tough. Choosing the most desirable combination of projects; that is, creating an optimal project portfolio, is even harder.
Project Selection Models Help Organizations Choose the Best Projects
The key to being able to consistently choose the best projects is, in my opinion, creating and using a quality, project selection decision model. The challenges that make project selection difficult for people—multiple considerations, uncertainty, interdependencies, the need to compare costs and benefits in different time periods—are exactly the characteristics that make the problem amenable to solution via a computer programmed with an analytic model. As described in Part 1 of this paper, people have limited information processing skills, can be biased, and are often inconsistent when making choices . An analytic model breaks a complex problem into pieces, allowing those pieces to be studied and characterized, and their relationships represented mathematically . People can then focus on the simpler task of providing inputs to the model. The processing of those inputs can be handled in accordance with sound mathematical reasoning using a valid solution algorithm and a computer.
Knowledge and effort are required initially to design and construct a project selection model. Lack of the relevant knowledge; that is, not knowing how to do it, is, in my opinion, the reason that relatively few organizations have developed formal, project selection models. This part of my paper explains, step-by-step, how to build a project selection model. Once built, a project selection model may be used again and again, both during an annual budgeting exercise and as a means for properly inserting newly identified projects into the existing priority order. A project selection model makes the criteria and logic for project evaluation explicit. When the organization has the ability to quickly and systematically evaluate the attractiveness of proposed projects, experience shows that the quality of project proposals increases. In other words, a project selection model not only helps the organization to select from among its candidate projects, it shows project managers how to design projects that generate greater value. As experience with the model accumulates, the model design can be modified and improved in ways that make its application easier, its predictions more accurate, and the insights it generates more useful.
A Project Selection Model Accounts for the Three Components of a Decision
As summarized in Figure 9, a model intended to aid decision making must account for the three basic components of a decision : (1) the alternatives that are available (what you can do), (2) objectives and preferences (what you want), and (3) information and beliefs about what will happen depending on the alternative to the decision that is selected (what you know, believe or suspect).
Figure 9: A decision model must faithfully represent the 3 components to a decision.
In the case of project selection, the alternatives are the various projects and combinations of projects that may be conducted, including alternative versions of projects if available. The objectives for project selection are the objectives of the organization, clearly stated and structured in such a way that they may be used as a basis for evaluating and comparing alternatives. The information and beliefs that matter are those relevant to predicting the consequences of doing versus not doing projects. The credibility of any project selection model requires that all three of these decision components be captured and accurately represented. In addition, essential to the capability of the model is the logic by which recommended choices are derived. The logic must be defensible, understandable, and capable of being implemented on a computer.
A Project Selection Model May Be Created by Linking a Consequence Model to a Value Model
Since the value of a project is logically the value of the project's consequences, it makes sense that a project selection model is composed of two sub-models, as shown in Figure 10 .
Figure 10: A project selection decision model consists of two component.
The first sub-model, labeled simulation, is a consequence model that takes as input project characteristics, including estimates of what the possible impacts of conducting a proposed project will likely be. This sub-model produces as output a forecast for how those impacts will alter the degree to which the organization will achieve its objectives. The second, sub-model, labeled valuation, is a value model that converts the estimated, project-induced changes to the organization's performance relative to its objectives into a measure of the value of that increment to performance.
The Value Model
Though it is the second of the two sub-models, the top-down approach to model construction means that the value model must be designed first. The value model identifies the specific organizational objectives that may be impacted by projects and provides performance measures (aka attributes) for quantifying the degree to which the objectives are achieved. Utility theory, as described on the two preceding pages, provides methods for constructing value models. The value model is a value function if there is no uncertainty over the impacts of projects on objectives as measured by the attributes. If there is uncertainty, the model is a utility function. In either case, the value model translates the changes that a project induces on the organization's ability to achieve its objectives into a number indicating the value of those changes.
The Consequence Model
The consequence model simulates the organization's performance relative to its objectives. The model forecasts the consequences of project decisions based on the facts, judgments, and uncertainties that determine the impact of project choices on the ability of the organization to achieve its objectives. Consequence models need not be very complex. A common function of the consequence model is to simulate the timing of project impacts: specifying when they will start, how long they will continue, and what will be left when they end. If the value model is a value function, the consequence model will necessarily be deterministic. If the value model is a utility function, the consequence model must be probabilistic . For deterministic models, a good decision is judged by the estimated project outcomes alone. For probabilistic models, the decision-maker is concerned not only with estimated project outcomes, but also with the amount of risk associated with those outcomes
Two Ways to Compute Project Value
As mentioned previously, there are two ways of configuring the attributes in a project selection model. The first is to specify as inputs to the model the "deltas" (changes) in attribute levels that will result from doing the project. The model is in this case designed to produce as output the increment in value that results from the decision to conduct the project. The second way is to design the model to be applied twice, once to estimate the value achieved by the organization with the project and once to estimate the value achieved without the project. The value of the project with this design is the difference in the two computed values. Oftentimes a model will be designed so that some attributes measure deltas while others require inputs corresponding to the "with project" and "without project" cases. As described earlier, computing value twice, with and without the project, is typically easier if failing to conduct a project results in a decline in the organization's performance, such as when maintenance projects are needed to prevent deterioration of the organization's assets.
Choosing a Methodology for Constructing a Project Selection Model
Decision analysis (DA) is the professional discipline and collection of tools and techniques for creating and analyzing decision models based on utility theory. Multi-objective decision analysis (MODA), also known as multi-attribute utility analysis (MUA) is, as the name suggests, the sub-field of decision analysis focused on decisions with multiple objectives [8, 9]. MODA encompasses all of DA's methods and tools for analyzing decisions with uncertainty and adds methods for explicitly accounting for multiple decision objectives. Other multi-criteria methods have been proposed and used for constructing project selection models. The reasons that I recommend MODA are as follows:
Steps for Creating a Project Selection Model
Like any model-building process, the process of building a MODA project selection decision model involves art as much as it does engineering science. However, in the 60 plus years since the specification of utility theory, decision analysts have evolved an efficient, step-by-step process for building decision models. The 12 steps of the MODA model-building process, as I define them, are identified in the table and associated Figure 11 below.
Figure 11: Steps for creating a project selection model.
Be aware that my description of the model-building process, like other attempts to reduce a creative exercise to a simple sequence of steps, is an oversimplification. Constructing a project selection model, or any decision-making model for that matter, cannot be accomplished by blindly following a linear sequence of tasks. Creating a successful model is often accomplished through trial and error. When difficulties are encountered at any step, in order to resolve those difficulties it is often necessary to revert back to a previous step from which an alternative course forward can be plotted. Perhaps most importantly, it is essential to keep the independence conditions in mind. Failing to define objectives and specify performance measure attributes in a way needed to achieve preferential independence can result in having to toss out previous effort and beginning again.
The remaining pages of this Part 4 of my paper describe each of the main steps of the MODA model building process with examples.