Self-proclaimed experts (those with no real expertise in project valuation or decision analysis) advise you to use a variety of criteria for prioritizing projects, such as ROI, contribution to strategy, political significance, competitive need, complexity, risk, technical feasibility, stakeholder satisfaction, and so forth and so on. You choose the criteria you want, weight them if you like, and use the results to rank projects. Wrong—Doing this produces incorrect priorities!

Here's the quick explanation. Organizations undertake projects because they believe the consequences of those projects will be worth more than the project's cost. Thus, the issue is how much the (uncertain) project consequences are worth to the organization. Define the value of a project to be the maximum amount the organization's senior executives would be willing to pay to obtain the project's uncertain consequences. If we rank independent projects by the ratio of value-to-cost, selecting projects in rank order gives the value-maximizing project portfolio. Thus, the proper criterion for prioritizing projects is the ratio of project value to project cost.

"But," you should ask, "how can you determine what executives would be willing to pay to obtain a project's consequences?" The foundation for solving that problem was established 70 years ago when John von Neumann and Oskar Morgenstern developed a theory of utility functions.

Decision analysts have refined and expanded utility theory to the point that we now have a well-established, step-by-step process, called multi-objective decision analysis (MODA), for constructing models for quantifying value. The inputs to the project value model are the metrics for evaluating projects, and the model mathematics explain how those metrics need to be combined to compute project value.

Building project value models is not pie-in-the-sky stuff. It's done all the time and this website explains how to do it. For more, see Lack of the Right Metrics, The Correct Prioritization Metric and Finding Metrics for Valuing Projects.