Lee Merkhofer Consulting Priority Systems
Implementing project portfolio management

"The error I see most often is a failure to define objectives in a way that allows the different types of project value to be identified and summed."

Identifying and Structuring Objectives

As described on the previous page, in order to construct a model for prioritizing and selecting projects, I use a 12-step process based on multi-objective decision analysis (MODA). On this page I describe in more detail the step aimed at identifying and structuring objectives. Figure 12 identifies this step and shows its relationship to the other steps in the model-construction process.


Steps for creating a project selection model

Figure 12:   The step aimed at creating an objectives hierarchy step.


The MODA model-construction process actually creates two models: (1) a consequence model for characterizing the impacts that conducting a project will have on the degree to which the organization achieves its objectives, and (2) a value model that quantifies what the organization believes those project-induced, incremental changes in the achievement of objectives are worth [1]. The two models are connected, with the consequence model providing the inputs needed by the value model.

Following MODA's top-down approach to model building, the value model is designed first.

Steps for Creating a Value Model

Of my one dozen steps for creating a project selection model, nearly half are devoted to framing the value model:

  1. Identifying and structuring objectives into an objectives hierarchy.
  2. Constructing influence diagrams.
  3. Specifying performance measures (also called attributes) for measuring performance relative to the lowest-level objectives.
  4. Checking independence assumptions.
  5. Determining the mathematical form of the equation for combining the measures of performance.

Figure 13 illustrates how the model components are used to compute the value of each project and its ranking metric:


How the model estimtes project value

Figure 13:   Project selection model logic.


Objectives

Establishing objectives is the most critical of all the modeling steps. The objectives determine the mathematical structure of the value model. Additional steps are required, obviously, to implement the value model, but the objectives and the attributes selected to measure the degree to which the objectives are achieved pretty much establish the key characteristics of the model. Almost every subsequent step in the model-building process depends on the way objectives are defined. Make a mistake here, and you are almost guaranteed to end up with a model that doesn't work. Ralph Keeney provides a detailed and easy-to-understand description, which I highly recommend, explaining why it is critical to take care in the identification of objectives [2]. Most of the ideas discussed below are explored in more depth in Keeney's book.

Identify Objectives as a Group Exercise

The listing of objectives is a task that is often best conducted as a facilitated group exercise [3]. Involving people that understand the interests of the different stakeholders to the project-selection process not only helps to ensure that all objectives are identified, it promotes buy-in and a sense of group ownership for this key aspect of the model design. The project portfolio management (PPM) team, if one has been established, should be the main participants in the specification of the objectives for prioritizing projects, and, once specified by the team, the objectives hierarchy should be reviewed and then approved or revised by senior executives.

In addition to the PPM team, you may want to ask other individuals to participate in the identification of objectives, particularly if they are influential actors within the organization or have special knowledge, expertise, or responsibilities related to certain types of projects. Unlike project decisions that are sometimes controversial, people typically find it fairly easy to agree on objectives. Taking advantage of this, I've many times participated in group exercises where the organization invited outsiders, including regulators and members of activist groups, to observe, and even participate in, the identification of the objectives to be used by the organization for prioritizing projects.

The Role of Objectives

Objectives define the reasons for conducting projects. As objectives are specified, the criteria for evaluating projects are defined. For example, if "increase revenue" is chosen as an objective, projects will be evaluated, in part, based on the estimated amount of incremental revenue they will produce. If "enhance the reputation of the firm" is an objective, projects will be evaluated based on their impacts on the organization's reputation. Another way to think about objectives is that they define the types of value that will be recognized when prioritizing projects. By specifying the above two objectives, for example, projects will be recognized as potentially creating revenue value and reputation value. Note that to create significant value, projects need not create all or even several types of project value, and many projects justify their costs by producing only a single type of value.

Resist Pressure to Use Previously Defined Lists of Objectives

Many organizations have existing lists of organizational objectives, developed, for example, as part of a strategic planning exercise. If lists of objectives already exist there will likely be pressure on the PPM team to use these objectives, as is, for the MODA model. Doing so, however, is nearly always a serious mistake. The problem is that in order to effectively serve as criteria for a project selection model, the objectives must conform to the technical requirements of MODA, such as no overlap or double counting.

My strong advice for those interested in developing a value model for their organization is to first learn the requirements, described below, for expressing MODA objectives. Then conduct an independent exercise aimed at defining the objectives for the MODA model. To reduce pressures to directly use previous-developed lists of objectives, I often advise the PPM team to adopt a term different from "objectives" to describe what the task seeks to find. For example, say the goal is to select a set of "MODA-compliant objectives" or "MODA criteria." Once the team has completed its task, the violations of MODA requirements that exist in other lists of organizational objectives can be explained and a "crosswalk" can be constructed to demonstrate that all of the objectives named in previously developed lists are captured by the MODA objectives.

Begin by Listing Objectives

The approach I typically use to initiate the process of defining objectives is to show the PPM team examples of objectives hierarchies that have been adopted by other, similar organizations (see below). I explain that once the group has generated a list of objectives, those objectives will be organized into a similar hierarchy for the team's review and modification. I then encourage the group to propose objectives for the hierarchy.

As participants suggest objectives, I add them to a list that is projected on a screen visible to all. However, before writing down a suggestion, I consider whether it satisfies the MODA requirements, specified below, and whether it will be possible to fit it into a hierarchy without overlapping other objectives at the same level of the hierarchy. If I see problems, I'll identify them to the team and look for a way to re-define the suggestion that meets MODA requirements and that everyone can agree to. If the process bogs down, I'll suggest objectives, but I won't write them down unless it is clear that everyone agrees (usually confirmed via nods) that the objective should be added to the list.

The Three Elements of a Statement of an Objective

Objectives

A precise, technical language is recommended for objectives. Specifically, a statement of an objective should contain three elements: an object of value, a context, and a preferred direction for change [4]. The object of value is a noun, for example, costs, revenue, quality, reliability. The direction for change is a verb, like minimize or maximize, increase of decrease, or enhance or reduce. The context clarifies the concern and defines scope. As an example, "maximize the safety of automobiles," is an objective statement that includes each of these three elements. The object of value is "safety," the context is "automobiles," and the direction for change is "maximize." As another example, "maximize customer satisfaction" likewise provides all three elements needed for a well-defined objective statement. If you are the facilitator for an exercise to identify objectives, make sure that each objective added to the list can be expressed as a statement that includes each of the three elements.

Criteria for Selecting Objectives

In addition to being properly stated, objectives should be:


  1. Fundamental. A fundamental objective expresses a desired end as opposed to a means for achieving an end. For example, "increase the use of seatbelts" is a means objective. A fundamental objective served by this means objective is "reduce the seriousness of injuries from traffic accidents." Specifying a means rather than an end can make it impossible to fairly compare other means for achieving the same ends, such as air bags and vehicle accident avoidance systems. When means objectives are proposed for the list, ask, "Why do we care about this?", and try to get participants to agree on a fundamental objective that underlies the means objective.
  2. Sensitive to choices. The purpose of objectives is discriminating the projects; that is, drawing distinctions among them and contrasting their performance. If an objective is not very important, if project choices don't impact it much, or if the impact to the objective is about the same for every project, that objective won't be useful for project prioritization. "World peace," is clearly a desirable objective, but I wouldn't include it unless project choices actually make some difference in the extent to which it is achieved.
  3. Measurable Objectives should define a future in measurable terms so that it is possible to verify, at some point in the future, whether and the degree to which objectives are being met. An objective can't be used to compare alternatives if it is difficult or impossible to estimate the degree to which it is achieved. For example, with regard to projects impacting pollution in lakes and streams, "fish happiness" might in theory be a good objective. However, we have no practical way to observe or quantify the degree to which fish are "happy." As another example, the stated objective of the New Zealand Conservation is to "maximize native biodiversity". Does maximizing biodiversity mean "maximizing the number of different species" or does it mean "maximizing the evenness of the distribution across species types" [5]? Objectives need to be precise and concrete enough not to be open to varying interpretations. They must be similarly understood by all parties. Measurability includes timing because project impacts occur over time; when impacts start, how long they last, and when they cease are all relevant considerations for determining project value. As objectives are proposed, think about how performance relative to the objective might be measured and look for ways to restate objectives if you foresee measurement problems.
  4. Decomposable, non-overlapping, and preferentially independent. As the list of proposed objectives expands, it will become apparent that some objectives are components of, or special cases of, others. The objective, "minimize customer wait times during telephone calls," for example, is a special case of the objective "maximize satisfaction of customer desires and needs." Identify the components of objectives as sub-objectives, but watch out for overlap and double counting. To obtain the desired additive or multiplicative forms for the aggregation equation, the value of obtaining a given increment in performance relative to an objective must be either independent of the performance levels of other objectives (preferential independence, which is required for an additive value function), or proportional to the increments in performance achieved for other objectives (which may justify a multiplicative form). You can't formally conduct tests for independence conditions until you've defined the attributes for measuring performance relative to the objectives, but you can conduct the tests informally as mental experiments. So, as objectives are proposed, ask yourself whether, and if so how, the value of an improvement in the objective depends on the levels of performance achieved for other objectives. The most common case for a multiplicative form is an objective based on the likelihood of some event, such as "maximize the likelihood of success." A common situation leading to additive form is where objectives are decomposed based on distinct groups who receive value generated by a project, such as customers, employees, local citizens, and the firm itself. As an example, a challenge for designing priority systems for organizations that provide essential services (e.g., gas, water, cable, electric power) is identifying objectives that justify the often large investments in infrastructure needed to serve growing populations. The group that benefits from these investments is new customers, and the value those customers obtain is the difference between what the services are worth to them and what they must pay to obtain the services (i.e., consumer surplus). As demonstrated by this example, it can be useful to identify the objectives of different groups impacted (either positively or negatively) by projects in order to uncover and capture distinct value components obtained by these groups. Because the groups are distinct (e.g., new customers vs. existing customers), it is likely that the values derived by each group can be added.
  5. Comprehensive, but concise and minimal in number. Collectively, the specified objectives must capture all of the significant considerations needed to comprehensively evaluate the pros and cons of projects. However, since each objective will require assessments to determine whether and to what degree each project impacts it, for practicality and simplicity, the number of lowest-level objectives should be kept as small as possible (typically 6-16). To avoid an unreasonably large number of objectives, consider doing a final screening of objectives based on importance. Objectives that are clearly of secondary or marginal importance can generally be omitted, especially if performance relative to these secondary objectives is highly correlated with performance relative to other, more important objectives.

Need for Tradeoffs

Inevitably, tradeoffs must be made when defining objectives. For example, the most direct and ends-oriented objectives may be difficult to estimate. The most accurate way of measuring an objective may not be understandable to non-technical decision makers. A practical value model needs to balance the theoretical soundness of the model against the time and effort needed to construct the model and provide the inputs that it requires [6]. Consider the users of the priority system and those who must understand it when specifying objectives.

Objectives Hierarchies

Once the PPM Team is satisfied with the list of objectives, I create an initial version of the objectives hierarchy. I start by making up a label to serve as the highest level objective, for example, using the organization's mission statement or something as simple as "create maximum value." On the second level I place the objectives that represent the broadest groupings of sub-objectives, oftentimes statements that represent the major different types of value or the different classes of stakeholders who benefit from the organization's projects . Then for each of the high-level objectives I add more detailed, lower-level objectives that define what needs to be achieved in order to achieve the higher-level objective. The process of building out the objectives hierarchy continues until the lowest-level sub-objectives provide a sufficiently complete and detailed specification of what needs to be addressed in order to account for all of the major types of value created by the projects included within the project portfolio.

The Objectives Hierarchy Makes it Easy to Spot Missing Objectives

When constructing an objectives hierarchy, I'll often add missing objectives that become apparent as holes in the hierarchy. When a decomposable objective, for example, "customer satisfaction," is placed in the hierarchy, it is easy to drill down to find and add lower-level objectives that specify the components of the decomposable objective. The components define what is meant by and is, therefore, contained within the higher-level objective. To build out a decomposable objective, I'll ask myself, "What are the components of this objective?", or, "What are the various things that need to be accomplished to accomplish this objective?" "Customer satisfaction," for example, could be decomposed by identifying different categories of customers, by identifying the different things that customers care about, or by identifying the different services or products the organization provides that are intended to satisfy customers needs. Adding sub-objectives below an objective provides the detail that explains or defines what is included within that higher-level objective.

Constructing an Objectives Hierarchy Identifies Points of Agreement and Disagreement

The process of creating an objectives hierarchy, particularly if it is conducted as a joint effort involving stakeholders or others with opposing views about what the organization should do, is very useful for clarifying points of agreement and disagreement. For example, if parties disagree about what objectives to include in the hierarchy, parties likely have different opinions about what the organization should seek to accomplish. Alternatively, if parties readily agree on the objectives that belong in the hierarchy, disagreements are likely due to differences of opinion about how effective the proposed actions would be for accomplishing agreed to objectives. Even if the objectives hierarchy is developed by the organization as a purely internal effort, the diagram can serve as an efficient tool for communicating to external parties the organization's view about its fundamental goals and to clarify opposing views that others may hold.

Constructing an Objectives Hierarchy May Expose Fundamental Questions

The task of constructing an objectives hierarchy will sometimes uncover fundamental questions about how the organization should allocate its resources and what it should seek to accomplish. For example, some government organizations and non-profits obtain partial funding from other sources for the purpose of conducting some types of projects. The obligations of the organization that receives such funding are often not clearly defined. For example, should the organization receiving such funds try to incorporate the objectives of the organization that contributes funding into its own objectives hierarchy? Or, should the organization simply choose to conduct some subset of projects that qualify for funding from the external source?

A related question has to do with how an organization receiving partial funding should optimize the allocation of its resources. Should it, for example, rank projects based on (a) the ratio of the project's estimated value to the portion of the costs that the organization pays, or based on (b) the ratio of project value to the project's total cost, including the costs paid by other entities? The former approach typically causes projects that receive partial funding to be ranked much higher, since partial funding effectively makes the project appear cheaper. In fact, treating partial funding in this way often results in projects being funded that would otherwise be seen by the organization as being unattractive based on insufficient value provided for the funds required. With the latter approach, in contrast, the organization is effectively treating funds contributed by other sources just as it treats its own funds. In this case, partial funding has no affect on how the organization prioritizes its projects and won't be utilized unless the projects that receive partial funding achieve the same level of productivity as projects that consume only the organization's resources.

Questions regarding the specific wording for some objectives often generates important but unfamiliar discussions. For example, participants in companies whose activities impact the natural environment quickly agree that "environment" should be included as a high-level objective. The discussion becomes more interesting, however, when I explain that some organizations choose the wording, "Protect the environment," while others choose wording similar to "Protect and enhance the natural environment." What follows demonstrates the importance of the MODA dictum to include in the statement of each objective a verb clarifying what the organization hopes to accomplish with regard the noun specifying the object of value. What might initially seem an esoteric matter (choosing a verb for the fundamental value") can have important impacts on what the organization does. In one such instance, a participant in the environment discussion stated, "We need to be careful, if we include 'enhance,' our project managers will start looking for environmental activities to include in their projects." Another participant then said, "Well, I guess we have to decide whether that would be a good or bad thing."

Another basic question, applicable to organizations in heavily regulated industries, is whether "regulatory compliance" is an objective or a requirement. In my experience, regulated organizations in the private sector consider projects needed for regulatory compliance to be mandated projects (oftentimes, though, the project labeled "mandated" is the least expensive means for achieving the requirement, and more expensive means of compliance are treated as optional enhancements to the minimum cost approach). In contrast, some government organizations, particularly those whose funding is insufficient to accomplish all requirements placed on it by regulators, consider complying with regulations to be an objective and not something that dictates how a portion of its limited budget should be spent. If the regulated organization is in the private sector, the team may want to add a box to the side of the hierarchy labeled "regulatory compliance" to convey the assumption that projects needed to avoid the certainty of being in non-compliance will be considered to be mandated. Even though mandated projects will automatically be designated to have highest priority, most organizations, in my experience, will compute the value of mandated projects along with non-mandated projects. Being able to show regulators that mandated projects sometimes prevent resources from being available to do other projects estimated to have a higher ratio of value to cost makes for interesting discussions with regulators.

Take Maximum Advantage of the Opportunity to Have Senior Executives Review the Objectives Hierarchy

Once the PPM Team is satisfied with a draft version of the objectives hierarchy, the diagram should be reviewed by the organization's senior executives. I typically advise the Team that executives should reserve 90 minutes for a briefing on the proposed design for the priority system and to review and refine the objectives hierarchy.

Make the most of the opportunity to involve senior executives in this first major step for the design of the priority system. An objectives hierarchy is easily understood and does not require reviewers to have technical understanding of MODA. In some instances, I've had executives make suggestions that conflict with the MODA technical requirements, but I've always found it easy to incorporate what the executives suggest in a way that doesn't undermine the technical rigor of the priority system design.

Even though I've helped dozens of organizations develop objectives hierarchies, I am continually surprised by the positive reactions of senior executives who review the objectives hierarchy developed by the PPM Team. In most instances, executives ask to be given a hard copy of the objectives hierarchy for their own use in explaining to others in and outside the organization how the project prioritization process will work. More than a few times executives asked to review an objectives hierarchy have expressed statements similar to, "You've managed to capture on a single page what this organization is all about." Don't miss the opportunity to engage senior executives and obtain a favorable assessment from them as to the PPM Team's progress in designing a system for formally prioritizing the organization's projects.

Templates for Constructing an Objectives Hierarchy

Experience shows that the objectives hierarchies for similar types of organizations are similar in structure. As illustrated below, many of the objectives, especially those at the second level of the hierarchy, are essentially the same from organization to organization, depending mainly on the degree to which obtaining financial returns for owners is a primary goal. Even for those organizations for which financial return is paramount, I've found that most are also concerned with protecting the health and safety of workers and the public; protecting the environment; avoiding adverse socio-economic impacts on local communities; increasing customer and business partner satisfaction; protecting and enhancing organizational image; and increasing organizational capability.

Below the second level, however, the the sub-objectives in the hierarchy differ because the common objectives have different interpretations depending on the types of products and services provided by the organization. The lower level objectives define exactly what is meant by the usual fundamental objectives in the given decision context. For example, improving organizational capability might mean advancing basic research for a pharmaceutical company and acquiring new diagnostic technologies for a hospital. As another example, the specific means available for enhancing recreational opportunities for an amusement park are quite different compared to those for a fly fishing resort [2]. The lowest-level objectives define each of the fundamental objectives in terms of the specific opportunities and interests of the decision making organization. The performance measures, scales, and sub-models for quantifying the value of the common types of project benefits tend to be similar, but the specifics are always different.

Useful templates for constructing objectives hierarchies can be developed based on the view of value adopted by the organization.

An Objectives Hierarchy for a Company Believing in Shareholder Value

As described previously, a privately held company may view its fundamental objective to be maximizing shareholder value. Shareholder value can be regarded as being the sum of (1) the organization's projectable earnings, as measured by the net present value (NPV) of forecasted cash flows, plus (2) its option value. Thus, a shareholder value objective (see below) can be split into two sub-objectives: (1) maximizing the NPV of identifiable and credible future cash flows and (2) maximizing option value (Figure 14). Relevant sub-objectives for increasing NPV include (1) increasing future revenue and (2) decreasing future costs. Sub-objectives for maximizing option value depend on the organization and the nature of its business, but generally such sub-objectives involve achievements that would increase the ability of the organization to identify and take advantage of future opportunities and avoid future risks, as well as other sub-objectives that, if achieved, would enhance investor expectations and, therefore, increase the market value of the company.


Sample objectives hierarchy based on shareholder value

Figure 14:   Generic objectives hierarchy for shareholder value.


An Objectives Hierarchy for a Company Believing in Stakeholder Value

If the fundamental objective of the organization is maximizing stakeholder value, then the objectives hierarchy may often be constructed by identifying who the stakeholders are and then identifying and structuring their objectives. A general structure for an organization committed to creating stakeholder value is shown below. In this case, key stakeholders are identified as customers, citizens, suppliers, and retailer/distributors. To complete the hierarchy, the specific objectives of each stakeholder group would be identified and placed under their respective stakeholder-identifying objectives.


Sample objectives hierarchy based on stakeholder value

Figure 15:   Generic objectives hierarchy for stakeholder value.

.

Figure 16 provides an objectives hierarchy developed as part of an effort to create a project priority system for a state system of colleges and universities. The hierarchy identifies a very large number of sub-objectives corresponding to different student groups and other stakeholders to the project selection process. The organization wanted a system for prioritizing e-learning project investments. Though the number of sub-objectives is very large, this does not create a major problem for assessments because most projects only impact performance for a single student category.


Sample objectives hierarchy for a state university system

Figure 16:   Objectives hierarchy for state university system.


In most organizations, the objectives hierarchy will contain objectives related to maintaining and enhancing the financial viability of the organization plus objectives corresponding to customers and, sometimes, the organization's staff. The objectives hierarchy below was constructed for a privately held hedge fund. As indicated, the executives who participated in the building of the hierarchy were very concerned about advancing the organization's trading capability. Employee satisfaction was included as an objective because of the importance of retaining highly skilled staff knowledgeable about the firms proprietary trading strategies.


Sample objectives hierarchy for hedge fund

Figure 17:   Objectives hierarchy for a private hedge fund.


Make Use of Mission and Vision Statements

Oftentimes, the organization will have a mission statement or vision statement expressed on its webpage that can be used as the highest level objective. Such statements typically suggest sub-objectives that indicate how the mission will be accomplished and the vision achieved. For example, the objectives hierarchy below was constructed for a division of a state's transportation agency.


Sample objectives hierarchy

Figure 18:   Objectives hierarchy for a state transportation agency.


The topmost objective is the agency's mission statement. The sub-objectives shown were identified by the agency's managers as indicating what needs to be accomplished in order for the objectives included within the mission statement to be accomplished.

The Next Steps

Once an objectives hierarchy has been created that passes the technical requirements of MODA, the next steps are to devise a means for measuring performance relative to the objectives and then to determine the form of the aggregation equation by which such measures of performance may be aggregated to compute project value. How these steps may be accomplished is described on the next several pages.

References

  1. R. L. Keeney & D. von Winterfeldt, "Practical Value Models," Chpt 13 in Advances in Decision Analysis: From Foundations to Applications, W. Edwards, R. F. Miles Jr., & D. von Winterfeldt (eds.), Cambridge University Press, 645-654, 2007
  2. R. L. Keeney, Value-Focused Thinking: A Path to Creative Decisionmaking, Harvard University Press, 2009.
  3. P. C. Fishburn, "Methods of Estimating Additive Utilities," Management Science, 13(7), 435-445, March 1967.
  4. D. E. Bell, "Multiattribute Utility Functions: Decompositions Using Interpolation." Management Science 25(8), 744-753, 1979.
  5. M. J. Conroy and James T. Peterson, Decision Making in Natural Resource Management: A Structured, Adaptive Approach, John Wiley & Sons, 2013.
  6. R. L. Keeney, "Utility Functions for Multiattributed Consequences," Management Science 18(5), 276-287, Jan 1, 1972.