C. Measurement Capacity

8. Developing a Project Assessment Framework

Currently viewing:  Refocus Home / Guidebook / Program Set Up / C. Measurement Capacity / 8. Developing a Project Assessment Framework

The main purpose of building triple-bottom-line measurement capacity is to ensure limited program resources are allocated toward the projects that generate the greatest overall organizational benefit. With the necessary measurement capacity in place, the next step involves formalizing a Project Assessment Framework. A Project Assessment Framework is simply a universal method or rating system that can be used to generate a clear and comparable valuation of each project being considered. Our objective is to establish the importance of sustainability by demonstrating it can actually enhance bottom-line performance; however, superficially identifying the most financially lucrative sustainability projects, as is typically done, will not suffice. 

Key Attributes of an Effective Framework

As part of ensuring a Project Assessment Framework is designed well enough to assess the impact of projects, it should embody the following key attributes:

  • Comprehensive: considerate of a complete spectrum of measurable impacts (economic, social and environmental), both positive and negative. That will ensure the most beneficial projects are effectively identified with the best interest of the entire organization at the heart of project prioritization.
  • Credible: although hundreds of factors can be used to assess projects, it is important to focus on those factors that decision makers are willing to employ, rather than on those that resonate with the Program Champion or sustainability experts in general.
  • Practical: it is important to keep the framework practical rather than trying to turn it into a science. The framework should be only as complex as it is manageable to use.
  • Universal: allows for any sustainability project to be consistently assessed, even if wildly different in nature. The capacity to create valuations that can be compared apples-to-apples will minimize confusion and conflict.
  • Supported: accepted and trusted by all stakeholders groups as “the” method for assessing projects. If individuals with unique interests buy into the framework developed, criticizing individual assessments becomes difficult, regardless of their bias. The development process is likely to require some healthy and worthwhile iteration as a result.

Strong Framework Mechanics

The sustainability projects being considered can be extremely diverse and complex. For a Project Assessment Framework to be most effective, valuations should be generated in ways that support decision making. The following are several examples:

  • Weighted: with many factors being considered, it is unlikely each is equally important to the organization. To reflect the difference in how much one factor is valued against another, each should be appropriately weighted.
  • Scenario-Based: without fail, some factors being measured (e.g. expected lifetime of a technology, projected increases in electricity costs, etc.) will hold uncertainty. To combat the effect of uncertainty, developing two or more scenarios (e.g. conservative, modest and aggressive estimates) can help decision makers to buy into the assessment developed.
  • Numerically-Oriented: the framework is most effective when it produces clear numerical outputs or scores that can be tallied and used as a basis for comparison. It is important there is a consistent means for rating the value a project contributes through each factor being considered like the one used in the example below.

EXAMPLE: consider how quantitatively-measured factors like carbon emissions can be used toward an overall rating. The value of emissions reduced is actually quite arbitrary; it’s the relative size of the reductions in comparison to the other projects being assessed that is important. Assigning a rating, like the one described in the social sustainability example outlined earlier in this section (e.g. high = 3, moderate = 2, low = 1, none = 0), can be useful for quantitative factors as well. Thresholds for separating projects can be thoughtfully predetermined or based on the results across the body of projects being assessed. For example, thresholds can be defined such that the expected results are split into thirds. The projects with results in the top third would receive a score of ‘3’, the middle third would receive a score of ‘2’ and the bottom third, a score of ‘1’. 

There is no set structure or formula for developing the perfect Project Assessment Framework. The result may look like a spreadsheet, a weighted criteria matrix, a scoring system, a rubric, a voting-policy, any combination of these possibilities, or something totally different. Creating a Project Assessment Framework is likely to require a few iterations as various stakeholders are involved; don’t shortcut the process, because it’s important that everyone buys into the final product.

Broader Framework Considerations

While strengthening performance is critical, that priority must be fairly balanced with other critical considerations that will contribute to the program’s long-term success including:

  • Program Vision: it is valuable to be genuine in attempting to fulfill on the program vision and objectives defined.  Building a Project Assessment Framework that is not well-aligned to the program vision can be detrimental to the program’s perceived sincerity and performance as well as damaging overall.  
  • Change Management: it is critical to engage and secure the support of relevant stakeholders as part of enabling the program to realize its full potential.  A Project Assessment Framework that doesn’t value getting people on board will limit the contribution of support and may result in resistance or even obstruction.
  • Program Development: a Project Assessment Framework should consider how projects can contribute to the program in ways beyond immediately measurable benefits. Some projects that don’t contribute as much in the short-term can help to build a needed capacity, create exposure to a new way of doing things or effectively appease the concerns of leadership.

Successful Framework Management

A Project Assessment Framework is not designed to be a fixed resource. It should be adjusted, refined and expanded as a program grows and evolves. For example, the sources of impact leadership initially recognized are likely to expand as the program generates positive results and demonstrates its capacity to improve how the organization performs. A truly effective framework will take time to develop, build credibility, and will need to be improved by learning what works and doesn’t year over year. 

Currently viewing:  Refocus Home / Guidebook / Program Set Up / C. Measurement Capacity / 8. Developing a Project Assessment Framework