According to a report recently featured on the Financial Times (PDF), hedge funds are expected to spend upwards of $600m on digital datasets this year, and up to $1bn by 2020. What’s going on? Why are investment firms hoarding all this data, and what types of data are piquing their interest in particular? Read on to find out.
A Brief History of Alternative Data
The first thing we need to understand is where all this new data is coming from, and how it differs from traditional data – which leads us to the concept of alternative data.
Hedge funds have always been data-driven organizations – in fact, these types of firms were doing analytics much before data-driven management became a sweeping trend in the business world. However, the sources of data which investment firms would be analyzing were typically quite limited, including quantitative information such as historical stock performance and earnings reports, alongside qualitative sources such as analyst call transcripts.
However, the digital revolution created troves of new data – mostly, albeit not solely, due to more and more spheres of human behavior moving to the always-measurable online world. As big data technology became more prevalent and accessible, hedge funds grew interested in using the “oil of the 21st century” to improve their predictions and make more accurate investment decisions.
Thus we have alternative data – an umbrella term referring to all manner of non-traditional data sources, which financial organizations can use to find new insights on investment opportunities. A few examples of this data can include online news, e-commerce, sensor and geospatial data. Let’s take a closer look at a few of these to better understand the concept.
5 Common Sources of Alternative Data
By the very nature of the definition just presented, there is not a closed list of alternative data sources – and as technology advances and more data is created, new sources are almost certain to pop up. So without any pretense of a comprehensive list, let’s look at some of the types of alternative data currently being used in the finance world:
- Online news: while financial services were always interested in monitoring the news, modern technology has given them the ability to do so at unprecedented scale. As a recent article on Quartz writes, if an employee previously needed to sit glued to the TV screen in case there were any pertinent developments, today that process can largely be automated. By using a news API service, hedge funds and asset managers can monitor thousands of online sources algorithmically, as well as run deeper analyses on the data to uncover various trends and insights.
- Consumer sentiment analysis: today’s customer is often quite vocal, taking to social media and online forums to express their opinions on a company, service or representative they had dealings with. The availability of aggregated reviews and online discussions in machine-readable format enables companies to use sentiment analysis techniques and algorithms in order to reveal shifts in public opinion regarding a company or its products, which could often be a predicting factor into its future performance
- Online retail: it’s no secret that e-commerce is gobbling up many types of traditional retail. One of the side effects is that today’s B2C transactions leave a much larger digital footprint, compared to the days of traditional over-the-counter transactions. By using crawled e-commerce product data, financial firms can notice fluctuations in product pricing or stock over time, which can offer further insight into the current state of a company they are keeping an eye on.
- IoT sensor data: current forecasts suggest that the number of devices connected to the internet is set to surpass 30 billion within the next two years. With everything from the transportation of goods and people, through vehicle traffic, the use of smartphones and GPS devices all leaving a digital footprint, many investment firms predict the internet of things (IoT) is destined to be one of the major sources of alternative data in coming years.
- Geospatial data: High-resolution satellite imagery, which was previously very difficult to come by, is now quite easily accessible. Using advanced image recognition, investors can extract insights such as changes in foot traffic to physical stores or lower density of traffic in certain neighborhoods, and incorporate these signals into their models in order to better predict future financial behavior.
How Today’s Financial Organizations Can Prepare for Tomorrow’s Data
Alternative data presents an exciting opportunity for many different players in the financial industry: hedge funds, asset managers and investment banks, to name a few. But to reap the full potential of the abundance of new data, these organizations should be prepared to tackle some challenges head-on.
More often than not, these challenges might require organizations to embrace change: we mentioned previously that finance was one of the first firms to truly embrace data analytics; however, working with Big Data requires a new set of technologies, skilled personnel and budgets that must be set aside for the tasks of acquiring, storing and wrangling datasets.
Furthermore – the techniques used to analyze traditional data may no longer apply; instead, one might need to employ machine learning, artificial intelligence and other emerging technologies to extract insights from large volumes of semi-structured or unstructured data. To remain truly competitive in this arena, the company needs to have a much broader expertise that reaches far beyond finance.
While this new reality will definitely prove challenging for many of the established firms within the space, those who will be quick to adapt and catch up with the rapid pace of big data technology are poised to reap significant rewards. It’s no wonder that demand for data scientists is at an all-time high.
