When we read publication on ICT for Development, we are often presented with a ranking, an index, a single number something like “network readiness” (World Economic Forum, 2012). What does that mean? How many numbers were kneaded to distill a three digit value for “network readiness”? It is simple enough to address the surface concerns by glancing over the “Methodology” section, but the exact conditions under which the data was collected strongly impact the results (with implications in the fields of psychology and sociology, among others). These specifics of the data collection process are not always easily available to seekers.
In “Measuring ICT: the global status of ICT indicators” a publication from the UN ICT Task Force, a multitude of sources including national questionnaires and resident service records are used to compile profiles of ICT access and indicators around the world. But how comparable is this data, from so many independent organizations with varying collect procedures? Consistency in indicators, methods, and sampling is obviously crucial in legitimizing the data collected. The publication makes no effort to hide gaps in the data, but evaluating World Bank statistics and customer records from region telecommunications company requires a highly standardized process.
The International Monetary Fund’s Data Quality Assessment Framework (DQAF) for National Accounts Statistics was developed to “provide a flexible structure for the qualitative assessment of the national account statistics” (DQAF). Under this framework, the quality of statistics are graded according to specific prerequisites. Essentially, there are indicators for indicators. The organization lists “assurances of integrity, methodological soundness, accuracy and reliability, serviceability and accessibility” as dimensions of quality data and provides specific indicators for how other indicators should be collected.
Given the feasibility of conducting a multi-lateral study with sufficient detail for analysis, the current process of data amalgamation seems to be the best option. However, if we expect to gain an understand of ICT accessibility abroad, we must be vigilant in checking data sources and take into account possible (and probable) data inconsistencies. Fortunately, there are organizations dedicated to doing this tedious work.