Having heard Adam Papendieck discuss the value of big data, I looked into an article about how it’s being utilized in the Caribbean. Given the abstract nature of these huge sets of data, Michele Marius characterizes big data by the 3 V’s (and three corresponding challenges in big data processing): volume, velocity, and variety. She explains that the volume is growing exceedingly large, making conventional processing expensive and tedious. With velocity, she explains that the rate at which data is generated is vital, as many firms must be able to process and evaluate the information in real time. Lastly, as we heard from Adam, big data can come from a variety of sources, and the analysis of each source’s data together is becoming increasingly relevant.
As Adam described the trends in ICTs around the world, there are trends developing in how big data is utilized as well:
- With analytics becoming more effective and faster, the value of big data may be realized by groups other than just big corporations
- Data may become even more commoditized, but mainly of value to organizations as opposed to individuals
- There is a shift toward using big data to provide more personalized user experiences
Despite these exciting developments in this valuable yet expansive information, significant issues must be addressed. Although big data allows for better end user services, private information is often compromised, and Marius warns that consumers may have to concede even more privacy in the future. In addition, she explains that the U.S. alone may be lacking up to 190,000 trained analysts by 2018 to make effective decisions with big data.
In developing countries, the prevalence of data mining is presumably limited in volume, but not necessarily utility. As seen with the use of Twitter in crises, even relatively scarce big data can offer impressive insight. With big data processing still emerging in these areas, firms that emphasize it can gain a competitive advantage over similar firms.
Do you think a solution exists in the balance of privacy vs. firms’ interests? Can you think of any additional challenges associated with analyzing big data?