The idea of big data and big data analytics is relatively new and has been taking the IT world by storm over the past year or two.
Big data generally refers to the vast amounts of structured and unstructured data that corporations and governments have in their system but is too large to be assembled in a manageable analytical environment and analyzed using conventional techniques. Nevertheless, this data is growing rapidly and is often considered to be potentially very useful to businesses in expanding their customer base, generating new sales and increasing profitability.
The source of big data comes from the transactions that businesses carry out. With modern technology many details about individual transactions can be captured. Walmart, for example, is known to process over one million transactions per hour and store the detail in various databases. Some people figure this amounts to more than 2.5 petabytes (one petabyte is equal to one quadrillion bytes) of data, more than 167 times the content of the all the books in the US library of Congress. The Sloan Sky Survey, using advanced telescopes and data collection techniques, gathered more data during its first few weeks of operation than had previously been gathered during the entire history of astronomy. The new version of the Sloan project is expected to repeat this performance every five days.
Social media, mobile units, which can be tracked and analyzed, add to the mix of data in vast quantities and was probably the tipping point for the new interest in advanced data analytics.
The idea of advanced data analytics is to take this vast quantity of data and search it for patterns and trends that can be useful for such activities as marketing strategy, sales analysis, and even more specifically to use it for things like location based marketing, where people can be tracked and then special offers made to them at strategic times during their shopping trips or other wanderings.
From the customer point of view, which means all of us, this means that wherever we go and whatever we do online, or even physically if we carry a smartphone, many of our activities are being captured and included in big data. Some of this data we may consider to be private, something we don’t necessarily want to be shared with the world.
Of course, there has been much attention given to the issue of privacy since the birth of the Information Age. We now have privacy legislation and an increased awareness of privacy issues. Also, we are constantly bombarded with new changes being made by such media as Facebook, which recently announced new practices to ”safeguard” the privacy of its users. Unfortunately, the default position in many of these new settings is one that provides the least protection and most people will ignore the announcements and not go in and actively manage their privacy.
And yet, they should be, if not concerned, at least aware and should take an active role. Their image is at stake. In extreme cases, their money and well-being is at stake as well.
The sources of big data are broad and they are getting broader every day. It calls for an increased vigilance for potential unwanted intrusions on peoples’ privacy and more active participation in preserving some semblance of privacy in their lives. This is the new challenge for our age.