Data and The Final Frontier
There seems to be this obsession out there with data. It is everywhere and we are accumulating more of it every minute of every day. We are tracking and analysing more data, ever faster, as we deploy high-powered machines that can perform millions of calculations in Nano-seconds. There is even a prevailing fear that artificial intelligence will someday take over the world, all driven by ever more elaborate ways of processing gazillions of gigabytes of data.
But is it really all about the data, when it comes to making good investment decisions? Last I checked, past performance is not indicative of future results. It says so on every fact sheet I receive from my fund managers. As a matter of fact, it is actually a regulatory requirement that every investment manager has to declare on their marketing materials that they cannot predict the future. So, what is it then that everyone is going on about? For the most part, it is patterns and trends.
For instance, say we noticed that the price of a particular stock went down 2%, every time it had gone up by more than 2% on three consecutive days before. Then we could put on winning trades every time we see this particular sequence of prices. Scanning the entire universe of securities and throwing all sorts of other bits of information into the mix, such as economic data, can yield an infinite number of possible constellations or data combinations. And given historical performance, we can assign probabilities of success on all of them, that’s the beauty of data.
Of course, it can take a lot of processing power to analyse all the information. I once came across one model, which crunched so much data, that it took 18 hours to run on some of the fastest and most powerful computers on the planet. All just to produce one signal. It was built over the course of many decades and by the estimates of the inventor, more than 160-man years in development time had gone into building the algorithms. How good at predicting the future was the model? Not so good as you would expect, and it turns out the world is still random. But there is great comfort to be taken away from this: if even this model can’t do much better than flipping a coin, then I also needn’t bother looking for answers in analysing more data. Thank goodness for that.