Sunday, February 17, 2013

Big Data's Trend in 2013

Ideas behind Big Data have been traditional data mining and analytics. But new technology enables the collection and analysis of unimaginable data volumes at high speeds. “Big Data” refers to methods and technologies that help businesses and individuals make better decisions by analyzing large data volumes and predicting probable outcomes. The term has been around for a few years, but 2013 may be a year when Big Data moves from the technical to the practical, as real consumers and citizens start seeing its impact.

Buying a product: Traditional and online retailers typically spent resources building huge datasets trying to understand their customer’s buying patterns using programs such as loyalty points. They offered big discounts on certain shopping days, such as Black Friday. Best example on which company follows this trend is Amazon. New technologies help companies provide real-time offers to customers based on the date, the time of the day and the location of their shopping. As companies use Big Data to store and analyze more and more information about customers and competition, shopping will become more personalized and marketing more targeted.

Election: Apart from business, Big data has an enormous impact on 2012 which was US presidential election. President Barack Obama’s campaign ran Big Data powered campaign that could micro-target individual voters most likely to be persuaded. The basic idea was to analyze every individual voter’s preferences instead of relying on traditional methods of taking polls with small sample sizes and extrapolating. Mounds of data from surveys, phone calls, voter lists and historic voting patterns drove real-time voter outreach and get-out-the-vote efforts. But Big Data was not limited to campaigns with huge technology infrastructure, as Nate Silver of The New York Times famously predicted the 2012 election outcome by applying statistical models to aggregate existing polling data.

Academics: A number of academic institutions are using Big Data to address challenges of high dropout rates and the ensuing decline in state funding. The basic approach is to ensure that students select the majors that are best suited for them and urging them to take courses that increase their chances of graduating. Even the course material can be personalized for the students based on their interest. This is all made possible by analyzing vast amounts of student data, such as test scores, previous grades and even real-time data points like clicks in an online class. Applying statistical models to each student’s profile and comparing results to similar students can predict the most likely outcomes (like succeeding in a class or completing a major) and offer recommendations.

Staying healthy: Healthcare has been a difficult domain for analytics due to privacy and restrictions that prevent the usage of data for research purposes. However the advancement of smartphones and other “self-tracking” devices is fast changing the landscape. It is now possible to collect data from healthy individuals by constantly monitoring their vital information 24 hours a day, creating a very large unbiased group that can be segmented by demographics such as age, sex and race. Analyzing large volumes of historical and real-time data can help individuals make healthy lifestyle choices, take preventive measures (e.g., flu vaccinations), predict their chances of being inflicted with a certain disease and possibly even provide personal analytics on their daily activities and how it impacts their health.

Privacy: Privacy has become a primary concern with Big Data with data collection. Often individuals fear Big Data watching their every move and knowing the details about their life. An increasing amount of data can be collected without the user’s knowledge or consent through online and smartphone. Collection, analysis and sale of personal data on the Web can range from your search habits to shopping preferences to personal health issues, and it is a booming business. Still consumers willingly share much of the data collected today.

Big Data is quickly becoming a goldmine for businesses, governments and even law-enforcement agencies, but it also attracts hackers and identity thieves. Shrewd consumers will understand how and where to best share their data, and what they get in return.
Throughout 2013 we are sure to see more impact of Big Data in other aspects of our daily lives, such as how we bank, watch TV and even stay safe. Consumers would do well to weigh the cost and benefits before allowing access to their data.


  1. Anton, very interesting view of big data.

    For other readers, there is a National Science Foundation push for Big Data Applications (see pages 5-8 in the following NSF call for proposals; Therefore, there are additional interesting venues for big data analytics?

    1. Also, provide a citation for Chelsea's post on the 17th in the part where you talk about academics:)