Recently I wrote an article on how I made 500k with machine learning and HFT (high frequency trading). I submitted the article to Hacker News and it ended up receiving 50,000 unique views. WOW! I thought I had something awesome to share but I had no idea that would happen! It also received 673 likes and 327 tweets. Amazing! By comparison, the actual business I'm launching has received just 135 likes so far - boohoo - though I'm confident that will increase soon.

As a result of the commentary on Hacker News as well as other places around the web I realized there were a lot of important things that I failed to mention in my article. I didn't really target what I wrote to a trader audience so now I want to take some time to answer the most common questions I received.

Q: Was this really a high frequency trading system?

I would describe my system as a combination of market making, very micro-scale statistical arbitrage, and HFT. It seems that the term HFT is not very clearly defined so let me just tell you a few things about what I was doing and you can decide for yourself:

  • My system was 100% automated. (I'd often wake up with the trading day half over.)
  • 90% of my executions were bids and offers that provided liquidity to the market (ie market making).
  • Probably 90% of all orders sent to the exchange were cancelled. (A mark of HFT.)
  • My system ran on a server at my clearing firm in Chicago (Crossland) which had direct access to the Chicago futures exchanges. Milliseconds were very important but by no means would I say I was competing on speed alone.

Q: Isn't this just a case of survivorship bias?

Quite a few people brought this up. I think it's great to keep something like this in mind because the fact is many people have tried algorithmic trading and surely most have failed. The difference is they are not writing up a blog post that gets viewed by thousands of people - that's true.

However, it seemed the implication was that I was simply the guy who got lucky. On this point I strongly disagree. I believe some of the doubt stems from the fact that many people didn't understand the difference between "running a strategy" versus HFT where you're picking off consistent profits while exploiting small market inefficiencies. To be fair this is partly my fault for not explaining what I was doing properly

In any case I am extremely confident I was not just getting lucky. There was no directional bias to my system so the fact I traded generally through an up market would not account for my results. My program made long and short trades all throughout the day and did just as well on a down day as an up. Position sizes were always small and as mentioned in the article I made upwards of a thousand trades a day and never lost more than $2000 in one day.

Q: What were the indicators you used?

My article glossed over the indicators my system used for predicting price moves partly because I didn't think they were the most interesting part of what I did and partly because I didn't want to get into any math. I thought what was cool was that I built a framework that allowed me to experiment with many indicators and quickly see which indicators had a meaningful ability to predict prices.

However, let me try to give a few more details. I'd say i had two types of indicators. The first type was based on the market microstructure (I think that's the correct term) of the contract I was trading. So for example - if there is more size on the bid versus the offer an up move was considered more likely. An up move would also be considered more likely if there was recent a trade executed on the offer.

The other type of indicator involved looking at markets that were correlated with what I was trading and making predictions based on what was happening in them (ie arbitrage). So essentially if the S&P moved up it's likely the Russell would move up as well. Most of these indicators were valid at very small time scale (ie milliseconds).

The indicators were indeed more sophisticated than what I describe here but hopefully you get the idea. Also, keep in mind predicting prices was only part of the story - I also relied on market making tricks such as keeping bids in the market if other orders were queued up behind mine but canceling them if not. As mentioned in the article my success required finding "safe" places to bid and offer.

The original article can be viewed here.