Unlocking the Value of Big Data

Data is big  these days – and getting bigger. Here are a few random facts. 10,000 payment transactions are processed every second around the world. 340 million tweets and 247 million emails are sent every day. 30 billion items of information are added to Facebook per month. Most of the world’s data was created in the past two years. For $600, you can buy a disk that is big enough to store all the world’s music.

Companies, analysts, date purveyors are spending millions to develop tools to capture and analyze these data.

There are two kinds of data – structured and unstructured. Structured data  comes in the form of data sets that have been digitized in such a way that they can be consumed by importing into analytical tools that simply sort, categorize and compare the information and then apply advanced analytics to develop historical trends, comparisons, projections and various metrics relevant to the particular decisions to be made.

Unstructured data, on the other hand, as the name implies, is data that is not in a form that can be readily consumed by computer systems. Typically, for example, it includes data buried in text documents. To be useful, it needs to be scanned, extracted and then converted into structured data – a process that can be very time consuming and can lead to serious inconsistencies if left to the hands of the individual users.

In the modern world, the big data phenomenon is changing the way people make decisions. Instead of drawing small samples of data for a decision, they use whole or very large populations of data  The effect is to drastically speed up the decision making process, and reduce the element of judgement which brings in so may inconsistencies and errors.

The production of data in traditional text format is therefore an obsolete relic of the age of print, designed for human reading but not designed for serious analytics.

In the world of investing, the idea of reading annual reports and then making investment decisions is similarly a relic of a bygone age. People are using analytical tools, such as those on the websites of investment dealers and banks and online investing sites like Investorline.  Trading on the stock exchanges is now driven by computer generated trades, based on data analytics and executed in milliseconds, which as we know has created a couple of serious blips in the system in the past few years.

But the existence of unstructured data, such as print-based reports, is a drag on the system and adds to the cost of consuming the information.

Unstructured information can be converted into structured information through a process of tagging the data. This simply means that the data are enhanced by adding additional information to it – called metadata – and then feeding it into analytical tools that because the data are now computer readable, can take it from there. Fortunately, such data for investing purposes are available in the  US and several other countries through the use of a tagging mechanism called eXtensible Business Reporting Language (XBRL). Traders and analytics specialists are placing greater emphasis on these data, as they are totally structured, represent a population of most of the listed companies in the US, and therefore provide a sound basis for making investment decision based on all the facts.

Big data, both structured and unstructured, is available for investment purposes. But too much of it is still in the old fashioned print era form designed for reading. XBRL is a way of unlocking the value of those data – one that is fully in tune with the times.