Measuring measurements?

When we read publication on ICT for Development, we are often presented with a ranking, an index, a single number something like “network readiness” (World Economic Forum, 2012). What does that mean? How many numbers were kneaded to distill a three digit value for “network readiness”? It is simple enough to address the surface concerns by glancing over the “Methodology” section, but the exact conditions under which the data was collected strongly impact the results (with implications in the fields of psychology and sociology, among others). These specifics of the data collection process are not always easily available to seekers.

In “Measuring ICT: the global status of ICT indicators” a publication from the UN ICT Task Force, a multitude of sources including national questionnaires and resident service records are used to compile profiles of ICT access and indicators around the world. But how comparable is this data, from so many independent organizations with varying collect procedures? Consistency in indicators, methods, and sampling is obviously crucial in legitimizing the data collected. The publication makes no effort to hide gaps in the data, but  evaluating World Bank statistics and customer records from region telecommunications company requires a highly standardized process. 

The International Monetary Fund’s Data Quality Assessment Framework (DQAF) for National Accounts Statistics was developed to “provide a flexible structure for the qualitative assessment of the national account statistics” (DQAF). Under this framework, the quality of statistics are graded according to specific prerequisites. Essentially, there are indicators for indicators. The organization lists “assurances of integrity, methodological soundness, accuracy and reliability, serviceability and accessibility” as dimensions of quality data and provides specific indicators for how other indicators should be collected.

Given the feasibility of conducting a multi-lateral study with sufficient detail for analysis, the current process of data amalgamation seems to be the best option. However, if we expect to gain an understand of ICT accessibility abroad, we must be vigilant in checking data sources and take into account possible (and probable) data inconsistencies. Fortunately, there are organizations dedicated to doing this tedious work.


One response to “Measuring measurements?

  • dolson2110

    I agree with you on the issues in measuring data. Throughout our class discussion as we covered many different reports, it kept occurring to me as how are all these methodologies comparable on an international basis. It seems almost as an impossible task. However, as much as gathering data using different qualitative and quantitative methods can leave inconsistencies, I also feel that if analyzed efficiently may provide a more accurate depiction of ICT use etc., when retrieving data is more difficult (like in rural areas). I think this was the point you were getting at, and I upmost agree with you that it is pertinent to look more closely at the the various methodologies in the reports to best put the data into context.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: