Methodology

What does the Conference Board's report card measure?

The report card measures how well Canada is meeting its fundamental goal of creating a high and sustainable quality of life for all Canadians.

What is meant by “a high and sustainable quality of life for all Canadians”?

The Conference Board considers a high and sustainable quality of life for all Canadians as being achieved if Canada records high and sustainable performances in six categories: 

  • Economy
  • Society
  • Innovation
  • Environment
  • Health
  • Education and Skills

The word “sustainable” is a critical qualifier. It is not enough for Canada to boost economic growth if it is done at the expense of the environment or social cohesion. For example, to take advantage of high commodity prices by mining and exporting all our natural resources may make the country rich in the short term, but this wealth will not be sustainable in the long or even medium term. The Conference Board has consistently argued that economic growth and sustainability of the physical environment need to be integrated into a single concept of sustainable national prosperity—what we call here a “high and sustainable quality of life for all Canadians.”

While there are many definitions and approaches to the notion of sustainability, the most widely used is from the United Nations' Brundtland Commission: “Sustainable development is development that meets the needs of the present without compromising the ability of future generations to meet their own needs.” 1

How does the Conference Board choose indicators?

How Canada Performs: A Report Card on Canada attempts to grade only “outcome” indicators—indicators that tell us what Canada is achieving, rather than what efforts it is making. In the Innovation category, however, have used several input indicators as proxies for outputs.

How Canada Performs also focuses on indicators that can be influenced by public policy—that is, factors contributing to quality of life that can be modified by individual, organizational, or public efforts. Policy may influence outcome indicators directly (such as a law requiring everyone to graduate from high school) or indirectly—by influencing inputs, which in turn affect the output (such as a policy that attempts to change smoking rates in order to reduce mortality rates). Some indicators emphasize a gap in performance (i.e., differences in levels among countries); others emphasize progress toward closing a gap (i.e., differences in growth rates among countries).

All indicators used to measure performance meet the following criteria: 

  • The indicator provides valuable information on the performance or status of a particular category. 
  • The indicator can be affected by policy and is relevant to policy. 
  • The indicator data are reliable and have timely availability. 
  • The data are sufficiently consistent to permit benchmarking over time and across countries. 
  • There is general agreement that a movement in the indicator in one direction is better than movement in the other.

Where does the Conference Board obtain the data?

About 80 per cent of the data used for this benchmarking report is supplied by the Organisation for Economic Co-operation and Development (OECD). The rest comes from other reliable sources, such as the United Nations, the World Bank, and the Yale Center for Environmental Law and Policy.

The most recent year of data is used for each indicator. In some cases, such as literacy skills, the data is taken from surveys that were conducted several years ago.

How does the Conference Board choose the 17 peer countries?

We began with the countries deemed “high income” by the World Bank; this is the group of countries likely to have achieved a high and sustainable quality of life, and would therefore serve as a worthy peer group. We used three filters to determine which of these 38 countries would stay in our analysis:

Population: We eliminated countries with populations of less than 1 million. Nine countries drop from the list—Luxembourg and Iceland, for example. 

Geographic land mass: We eliminated countries of less than 10,000 square kilometres to restrict our analysis to countries that are more than city-states. Singapore drops from the list.

Income (gross domestic product) per capita: We ranked the remaining countries using a five-year average of real income per capita and included only countries that ranked above the mean. Thirteen countries drop off the list—Spain, New Zealand, and Israel, for example.

Using these criteria, 17 countries, including Canada, form our comparator group:

  • Australia
  • Austria
  • Belgium
  • Canada
  • Denmark
  • Finland
  • France
  • Germany
  • Ireland
  • Italy
  • Japan
  • Netherlands
  • Norway
  • Sweden
  • Switzerland
  • United Kingdom
  • United States

The comparator group remains constant across performance categories. This ensures that the best-performing country (in each category and on each indicator) is included in all analyses and allows us to compare performance over time.

Italy was recently dropped from three report cards because its current income per capita is not high enough for it to be considered a peer country under our methodology. While Italy had already been included in the most recent updates for the Health, Environment, and Society report cards, it was not included in the recent Education and Skills, Economy, and Innovation report cards.

How does the Conference Board assign a grade to each indicator?

We have adopted a report card–style ranking of A–B–C–D, to tie in with the title A Report Card on Canada. We assign a grade level to each indicator using the following method:

  • For each output indicator, we calculate the difference between the top and bottom performers and divide this figure by 4.
  • A country receives a report card rating of “A” on a given indicator if its score is in the top quartile; a “B” if its score is in the second quartile; a “C” if its score is in the third quartile; and a “D” if its score is in the bottom quartile.

For example, on the Innovation indicator “Scientific articles per million population,” the top performer (Switzerland) produced 1,176 scientific articles per million population in 2005 and the bottom performer (Japan) produced 424 articles. Using our method for ranking, the ranges for A–B–C–D are:

A: 988–1,176 scientific articles
B: 800–987 scientific articles
C: 612–799 scientific articles
D: 424–611 scientific articles

(Note: In this example, a high score indicates a high level of performance. For indicators where a low score signifies a high level of performance—such as scores on poverty in the in the Society category—the ranking levels are reversed.)

Two indicators are not graded using the standard methodology: inflation and labour productivity growth.

Inflation

We award an “A” grade to inflation that falls within the Bank of Canada’s inflation-control target range, which is between 1 and 3 per cent. Inflation outside this target range (either above or below) is awarded a lower grade. The further away from the target range, the lower the grade. Countries with inflation between 0 and 1 per cent or between 3 and 4 per cent earn a “B” grade. We grade inflation between 0 and 1 per cent to be a “danger zone” because it may signal that a country is slipping into deflation. The one exception is when inflation between 0 and 1 per cent is due to currency appreciation or strong productivity growth—these countries are awarded an “A” grade. Inflation between 0 and −2 per cent (deflation) or between 4 and 6 per cent is given a “C” grade. The lowest grade, “D,” is given if inflation is above 6 per cent or if prices are falling by more than 2 per cent, an indication of more severe deflation.

Labour Productivity Growth

Using the following formula, we award “A” or “B” grades to countries with positive labour productivity growth and “C” or “D” grades to countries with negative productivity growth:

  • We calculate the difference between the top performer and zero, and divide this number by 2. A country receives a report card rating of “A” on labour productivity growth if its score is in the top half of this number and a “B” if its score is in the bottom half.
  • We calculate the difference between zero and the bottom performer, and divide this number by 2. A country receives a report card rating of “C” on labour productivity growth if its score is in the top half of this number and a “D” if its score is in the bottom half. 

How does the Conference Board assign a grade to each category?

We first convert the individual indicator data to a common unit by normalizing each data point using the following formula:

Normalized value = (indicator value – minimum value) x100
(maximum value – minimum value)

Using this formula results in a data series where the best-performing country has a score of 100 and the worst performing country has a score of zero.

A composite index for each country is then calculated by averaging all the normalized indicator values. No attempt was made to give explicit differential weights to indicators according to importance; we are implicitly giving equal weight to each indicator. This is the standard approach used by most organizations in the absence of any compelling reason to apply different weightings.

We assign a grade level to the category performance using the following method: 

  • We calculate the difference between the category composite index of the top and bottom performer and divided this by 4. 
  • A country receives a report card rating of “A” for the category if its score is in the top quartile, “B” if its score is in the second quartile, “C” if its score is in the third quartile, and “D” if its score is in the bottom quartile.

Footnote

1 Gro Harlem Brundtland, Chairman. Our Common Future: World Commission on Environment and Development. (Oxford: Oxford University Press, 1987), p. 43.