11. Measuring the Rule of Law: The World Justice Project’s Rule of Law Index

Governments regulate markets and tax citizens and firms to provide public goods. This opens the possibility of corruption, standardly defined as the abuse of public office for private gain, and selective enforcement.

A careful examination of the nine factors shows that there is a partial overlap among some sub-factors. This is because various rule-of-law dimensions partially overlap in practice. For example, a free press is both a manifestation of a fundamental right in action and a non-governmental check on the government’s powers.

Some of the most relevant cross-country sources considered include the Bertelsmann Foundation (Transformation Index); Brown University (Center for Public Policy: Global E-Government Index); CEELI (CEDAW, Convention to Eliminate all forms of Discrimination against Women Assessment); CEELI (JRI, Judicial Reform Index); CEELI (LPRI, Legal Profession Reform); CEELI (Prosecutorial Reform Index); CEELI (ICCPR Legal Implementation Index, International Covenant on Civil and Political Rights); Center for Systemic Peace (Polity IV Project); EBRD (Sector Specific Assessment of Law and Practices); EBRD (Transition Report); the European Bank for Reconstruction and Development, EBRD (Country Law Assessments); Freedom House (Countries at the Crossroads); Freedom House (Freedom in the World); Freedom House (Nations in Transit); Global Insight (Global Risk Service); IJET (Country Security Ratings); Institute for Management Development (World Competitiveness Yearbook); International Budget Project (Open Budget Initiative); International Research and Exchange Board (Media Sustainability Index); New Tools in Comparative Political Economy: The Database of Political Institutions; Political and Economic Risk Consultancy (Asian Intelligence: Corruption Report); Political Risk Service (International Country Risk Guide); Public Financial Management; Reporters Without Borders (Press Freedom Index); Russell’s EMPulse, Investors’ Perceptions of the Pulse of Emerging Markets; The Global Integrity Report; Transparency International (Corruption Barometer); USAID (NGO Sustainability Index for Central and Eastern Europe and Eurasia); World Bank (Country Policy and Institutional Assessments); World Bank (Doing Business); World Bank (DPI, Database of Political Institutions 2006); Afro-barometer; CIMA (Barómetro Iberoamericano de Gobernabilidad); Gallup World Poll; Global Insight (Economic and Financial Data); Heritage Foundation (Index of Economic Freedom); Latinobarometro; Transparency International (Bribe Payers Index); US State Department (Trafficking in Persons Report); Vanderbilt University (LAPOP, The Americas Barometer); World Bank (Enterprise Surveys); World Economic Forum (The Global Competitiveness Report); Cingranelli-Richards (CIRI, Human Rights Dataset); Mo Ibrahim Foundation (Ibrahim Index of African Governance); Organisation for Economic Co-operation and Development (OECD, African Economic Outlook); Political Terror Scale; Transparency International (CPI, Corruption Perceptions Index); World Bank (Worldwide Governance Indicators); African Development Bank (Country Policy and Institutional Assessments); Asian Development Bank (Country Policy and Institutional Assessments); Amnesty International Report; Economist Intelligence Unit (Country Risk Service and Country Forecasts); Human Rights First (Annual Report); Human Rights Watch (Country Reports); Open Society Institute and EU Monitoring and Advocacy Program (EU Accession Reports); United Nations (UN, Universal Human Rights Index); and the United States Department of State (Human Rights Practices Annual Report).

In the case of CATI and face-to-face methodologies, the polling company makes three contact attempts before substituting a respondent.

In addition to these data sources, the Index incorporates third-party data to measure structural rule-of-law situations that may not be captured through general population polls or expert opinions. These variables include (a) the number of events and (b) the number of deaths resulting from high-casualty terrorist bombings (see Center for Systemic Peace, Major Episodes of Political Violence, 1946–2015); (c) the number of battle-related deaths; (d) the number of casualties resulting from one-sided violence [Source: Uppsala Conflict Data Program]; and (d) coup d’état events [Coded from the Center for Systemic Peace].

The 35 countries covered in the WJP Rule of Law Index 2010 report were Albania, Argentina, Australia, Austria, Bolivia, Bulgaria, Canada, Colombia, Croatia, Dominican Republic, El Salvador, France, Ghana, India, Indonesia, Japan, Jordan, Kenya, Republic of Korea, Liberia, Mexico, Morocco, Netherlands, Nigeria, Pakistan, Peru, Philippines, Poland, Singapore, South Africa, Spain, Sweden, Thailand, Turkey, and the United States. The following year, the 2011 report included, in addition to the previous countries, Bangladesh, Belgium, Brazil, Cambodia, Cameroon, Chile, China, Czech Republic, Estonia, Ethiopia, Germany, Guatemala, Hong Kong SAR, China, Iran, Italy, Jamaica, Kazakhstan, Kyrgyzstan, Lebanon, Malaysia, New Zealand, Norway, Romania, Russia, Senegal, Uganda, Ukraine, United Arab Emirates, the United Kingdom, Venezuela, and Vietnam. For the 2012 report, in addition to the 66 previous countries, the Index report provided scores for Belarus, Bosnia and Herzegovina, Botswana, Burkina Faso, Cote d’Ivoire, Denmark, Ecuador, Egypt, Finland, Georgia, Greece, Hungary, Macedonia, FYR, Madagascar, Malawi, Moldova, Mongolia, Nepal, Nicaragua, Panama, Portugal, Serbia, Sierra Leone, Slovenia, Sri Lanka, Tanzania, Tunisia, Uruguay, Uzbekistan, Zambia, and Zimbabwe. In addition to these countries, the 2014 report included scores for Afghanistan and Myanmar. Finally, the 2015 report included scores for all the countries previously mentioned plus Belize, Costa Rica, and Honduras for a total of 102 countries.

Experts may not be exposed to certain problems that the general public experience or may be biased against certain policies or forms of government.

The government officials of certain countries may censor or condition the administration of questions because they are perceived as challenges to the regime. In other cases, respondents may be unwilling to truthfully answer some questions either because of a perceived risk of sanctions or because of concerns about self-image when the documented behaviour does not conform to social norms (social desirability bias). This can lead to under-reporting of sensitive topics, thus making the data less valid.

The World Justice Project conducted a thorough review of about 30 surveys touching on concepts related to the Index. Some survey questions were directly incorporated into the GPP questionnaire; some survey questions were included in a modified form, and others were simply used to inform the design of new questions. The questionnaires also include vignettes, which allowed us to standardise the frame of reference for respondents around the world and as many experience questions as possible as perceptions of institutional performance may not be comparable or may not reflect actual experiences with the system.

The rule of law is a complex phenomenon, which may be perceived and experienced differently by different persons depending on their environment, background, positioning, professional expertise, attitudes, ideological tendencies, or beliefs concerning certain topics. Relying on different sources provides a richer picture of the rule-of-law situation in a country.

In the case of qualitative checks, we gather relevant quotes from studies such as the United States Human Rights Report, Freedom House’s Nation in Transit, Amnesty International country reports and Freedom House’s Freedom in the World. This information is gathered in an internal document and compared against our estimates. For the quantitative checks, we collect cross-country data from a large number of sources including the Global Integrity Report, WEF Global Competitiveness Report, Freedom House, WB Governance Indicators, Transparency International’s Corruption Perception Index, Gallup, the WJP Doing Business Report, and the Cingranelli-Richards Human Rights Database.

“An effective campaign against piracy and counterfeiting is also seen as further strengthening the rule of law. While the Philippine Development Plan 2011–2016 cites the rule of law as vital to the holistic development of citizens, the World Justice Project Rule of Law Index 2010 sadly ranks the Philippines last or close to the bottom among seven indexed Asian countries…”, Vice President of the Philippines Jejomar C. Binay (Binay 2011).

“In its 2011 Rule of Law Index, the World Justice Project surveys 66 countries to assess the state of the rule of law in each of these jurisdictions…On access to justice, the index ranks Canada 9th out of 12 wealthy Western European and North American countries. The most problematic areas, according to the index, are access to legal counsel and unreasonable delay in civil justice” Chief Justice of Canada Beverley McLachlin (McLachlin 2011).