Brief summary of the post for those wondering whether to bother: I trained a neural network that takes in images of stock data like this…
…and outputs whether that stock will go up or down over the next year. On the way, I make some remarks on end-to-end learning.
There's a current craze within the current craze of machine learning and AI, which focuses on end-to-end learning. Roughly speaking, this is the attempt to go from raw perceptual input to the desired output with as little knowledge engineering as possible.
To give an example, suppose you were trying to teach an AI to drive a car***. You could try to factor the problem: I need to detect lanes, other cars, people, and signs. I need to figure out people's plans, I need to parcel my actions into the following sub-goals, etc. Each of these sub-problems still needs to be solved on its own (and probably you could throw a neural network at these sub-problems), but at a high-level you're making a decision about how the problem breaks down:
Contrast this with a somewhat different school of thought:
That part in the middle of the figure above is a cartoon stand-in for a deep neural network. The network gets in the pixels of the image, outputs some driving action, and gets rewarded accordingly. The hope/promise is that the network will figure out on its own the necessary features and structures needed to solve the problem, without any 'hand-engineering' by a handy engineer.
(*** I am
stealing borrowing this example from Amnon Shashua, the head of the self-driving car company Mobileye, who has a bunch of interesting things to say on the topic)
These two approaches are the ends of a spectrum, rather than two completely distinct factions. Still, there are many people these days who veer heavily towards the 'end-to-end' part. And it is quite alluring: we don't need to know anything about anything, just plug in the input and desired output, and let the machine take care of the rest.
This is partly what got people so excited about DeepMind's recent Nature paper, in which a neural network was able to achieve super-human performance on certain Atari games (like Pong and Space Invaders) through reinforcement learning over the pixels of the input. The 'super-human' part is a bit of a distraction. I mean, if you had a program that explicitly knew about things like "this is the space invader sprite" and "you are the defense-turret and when you shoot it does X", then of course you could achieve super-human performance. Such a result would barely register as a worthwhile paper in a conference, let alone a prestigious journal. The wow-factor had to do with the notion that you didn't have hand-craft explicit stuff like "you are the paddle", or even what a paddle was.
DQN training to destroy humanity
I think it's an open question whether the network described in the DeepMind paper actually learned the representations that allow us humans to play these games successfully. In fact I'm pretty sure it didn't. And it matters, because while end-to-end is quite seductive and works surprisingly OK in certain regimes, I think it is a little insane.
But rather than spend the rest of this post arguing that (I've covered a bit of it elsewhere), let me instead morph into an end-to-end enthusiast, who also wants to play the stock market.
(clarification, everything from here on is still kinda true. Technically.)
Gee wiz guys, all this neural network stuff sounds awesome. But how can we make some actual money off of it? I mean besides getting a PhD in Deep Learning and being hired by a start-up.
I know, we'll bet on stocks!
Let's see, we can formulate the 'problem' of stock betting as a system that takes in some data about stocks in the past, and tries to make the right prediction about stocks in the future. Now, I don't know that much about stocks, and I don't want to bias my system by giving it any human-specific notions like 'prices', or 'companies', or 'the concept of time'.
We'll just feed in raw input into the machine in a format it can understand: visual pixels.
I downloaded all the stock data for the Fortune 500 companies going 5 years back, and created 32×32 pixel images that show how a stock fared over 365-day intervals, including the stock name and the min and max of the price (not that the machine knows anything about those, it'll have to discover them on its own if they matter):
I used 50-day jumps and 365-day intervals, so I had about 10,000 training images. But how should I label them? What counts as a 'good' stock?
There's no one way to answer that. I ended up doing the following: If the stock went up over the 365 days that followed the image (a positive correlation of 0.5 or higher), I labeled it '+1'. If the stock went down (a negative correlation of -0.5 or lower) I labeled it '-1'. Anything else was labeled '0'. The assumption is that we can turn these labels into useful actions like 'buy', 'sell/short', or 'do nothing'.
Since stocks generally go up (in the dataset, anyway), one can know nothing at all and still predict 70% of the labels just by guessing the maximum-likelihood "+1" all the time. So, 70% accuracy will be our baseline to beat.
- The input is an image of the stock performance over a given span of 365 days.
- The desired output is -1, 0, or +1 depending on how well the stock did on the *next* 365 days.
- The number to beat is 70% accuracy.
I took an off-the-shelf convolutional neural-network that can reasonably recognize different animals and threw the problem at it, using TensorFlow.
It didn't work.
I started to think that maybe the stock market present a unique challenge with its long-range beha-
NAH LET'S JUST ADD ANOTHER CONV-LAYER AND FUTZ WITH THE LEARNING PARAMETER LOL.
Here we go:
The network achieved 92.3% accuracy on the left-out examples (that is, stock images that it never saw before). For the math-y types out there, 92.3 > 70.
Hooray, let's make money.
Q1: "But Navisheker, what did the network actually learn?"
A1: I dunno, I guess it extracted the features it needed or something.
Q2: "Wouldn't it be saner to run machine-learning directly on the actual time-series data and company information?"
A2: Sure, if you want to hand-engineer your way to profit go ahead and try. My way doesn't require knowing anything about anything.
Q3: "Surely there's a simpler algorithm for getting a similar result. What if you just predicted that things that went up before tend to go up again?"
A3: That's not end-to-end, get out of here with your old-school ways! But since you asked, if you run logistic regression to try and predict the labels from the linear regression coefficient of the past 365 days, you can get up to 84%, which is not as good.