Roadblocks of Big Data Analytics
Big data analytics contains an
amazing set of tools but there are a few limiting factors that may prevent you
from being as successful as you are planning to be using big data analysis.
This blog is intended to provide you with some of the areas that may hinder
your progress.
One limiting factor in big data
analysis occurs before you get to analyze the data. Since the amount of data
being produced today is so large, just storing the data has become a real
issue. Hard drive capacity isn’t increasing at the rate that data production is
increasing and this is creating a major problem in many companies. According to
the article below, data production is projected to increase 50 fold by 2020
while hard drive space is only projected to increase 15 fold.
There are some solutions to this
problem but each of them has its own problems. The quick solution is to just
delete the older data so that the amount of data is minimized. While this may solve
the initial problem of data storage space, it limits the amount of information
that can be gained through big data analytics. Another method of storing all
this data is using some sort of cloud storage like we are using in this class.
This will add an extra expense for the company and many companies may not have
the financial means to go this route. Even with this route eventually we will
just simply run out of the resources to store the vast amounts of data that are
compounding daily.
Another roadblock for big data
analytics is the bandwidth needed to gather and possibly analyze the data
collected. This affects programs using satellites to provide their information
as well as any other company that may use an internet source to analyze data.
Bandwidth limits the amount of communication a network can handle and may not
allow for extremely detailed data to be processed. This could result in lost data or in most
cases just slow data transfer which could cost companies time and resources
that would normally be used to complete other tasks. The following website provides a little more
background on how this is monitored and some background information.
The factors above may not affect
you if you are a small company with smaller amounts of data but for companies
such as cell phone companies, cable companies, and social media companies these
are becoming everyday issues. You may think that it doesn’t affect you on a
personal level but this affects everyone in some way. Many cell phone companies
limit the amount of data you can use per month instead of letting everyone have
unlimited data. This is due to the vast growing issue of too much data and not enough
space to deal with it. Also as the amount of users increases the bandwidth
limitation of their networks are starting to slow down how fast the data arrives
which leaves many customers unhappy.
Hopefully new technology will
become available in the near future to prevent these problems from becoming
even greater issues but only time will tell.
Until a better solution exists big data analytics may not be able to
reach its true potential.
Chris,
ReplyDeleteVery interesting post. Definitely, there a lot of challenges. The question is whether the opportunities > challenges. Right now, it may be too early to tell.
Two additional things for you to think about:
1- Data security: With the ever-increasing data production, the question of data-security becomes more and more important. How do you prevent cyber attacks and how do you detect intrusions when they happen. There were a couple of posts that have looked into this. This is very much an area of active research.
2- Do we have analytical approaches that can help us exploit the "big data" phenomenon in different applications? So if you are storing all this much data, can you make sense of it. More importantly, can this lead you to making more informative business decisions?
Great post and discussion.
Thank you,
Fadel