A new paper on high-frequency trading

The author is Jonathan Brogaard of Northwestern and here is the abstract:

This paper examines the impact of high frequency traders (HFTs) on equities markets. I analyze a unique data set to study the strategies utilized by HFTs, their profitability, and their relationship with characteristics of the overall market, including liquidity, price efficiency, and volatility. I find that in my sample HFTs participate in 77% of all trades and that they tend to engage in a price-reversal strategy. I find no evidence suggesting HFTs withdraw from markets in bad times or that they engage in abnormal front-running of large non-HFTs trades. The 26 high frequency trading (HFT) firms in the sample earn approximately $3 billion in profits annually. HFTs demand liquidity for 50.4% of all trades and supply liquidity for 51.4% of all trades. HFTs tend to demand liquidity in smaller amounts, and trades before and after a HFT demanded trade occur more quickly than other trades. HFTs provide the inside quotes approximately 50% of the time. In addition if HFTs were not part of the market, the average trade of 100 shares would result in a price movement of $.013 more than it currently does, while a trade of 1000 shares would cause the price to move an additional $.056. HFTs are an integral part of the price discovery process and price efficiency. Utilizing a variety of measures introduced by Hasbrouck (1991a, 1991b, 1995), I show that HFTs trades and quotes contribute more to price discovery than do non-HFTs activity. Finally, HFT reduces volatility. By constructing a hypothetical alternative price path that removes HFTs from the market, I show that the volatility of stocks is roughly unchanged when HFT initiated trades are eliminated and significantly higher when all types of HFT trades are removed.

The paper you can find here, and I thank a loyal MR reader for the pointer.


Would it not make sense to compare price movements in stocks not traded by HFTs and those they do? Or to examine price movements in a stock prior to when HFTs began to trade it and now?

@Sam: Unfortunately real-world control data is not available, because HFTs trade basically everything and the time before HFTs has other fundamental differences with the present situation. One could try comparing the Chinese market to the US market, for example, since high frequency strategies are pretty much impossible in China right now, but there are so many other systemic differences that the comparison would be perhaps even less meaningful than the comparison done in this paper.

@Derek: Wal-mart drives mom-and-pop stores out of business, but that doesn't mean that hardware is more expensive after the local hardware stores have closed down. Wal-mart still has better prices. One can still argue that something has been lost when the faceless corporation replaces the local stores, just as one can argue that something is lost when lightning-fast machines replace thinking humans, but I think your particular objection is absurd.

Jason: Your suggestions are reminiscent of the current proposed plan of not allowing orders to be canceled for a certain amount of time. In my opinion, if any such scheme were implemented, liquidity would disappear. HFT guys can provide really tight markets because they know they can hedge instantly and they know that they won't get run over too badly if news comes out (because they can quickly move their markets). If you can't cancel an order at will, or if your order might sit in a queue for up to a minute, or if your price depends on other orders that come in over the course of a minute, then it will be impossible to hedge with both haste and precision, and it will be much harder to avoid getting run over when news comes out. With such an increase in risk, liquidity providers will require much more per share in order to keep providing their service, and markets will widen out considerably.

I work as a quant at one of the major high frequency trading firms, this paper is definitely one of the better academic works I've seen on the subject. I'll add a little more. Generally the way that HFT works is by looking for a set of predictive signals in the market. Those signals are combined with liquidity and execution constraints to try to find the most profitable set of parameters after transactions costs are taken into account.

90% of these signals are fall into two major categories: 1) Looking at the price movements of related securities. A good example is SP500 versus Nasdaq. The correlation between the two is around 85% on a daily horizon, but over a horizon of 10 secs or so correlation is virtually zero. So when one moves a certain you bet that the other one will either follow or the first mover will fall back. 2) The other one is by looking at the state of the limit order book and it's evolution through time. As a very simple example if say you have 20,000 size quantity on the bid and it's been monotonically increasing and 5,000 size quantity on the ask and it's monotonically decreasing then it's very likely that the level on the ask will get wiped out first and the price will go up.

In general what these two add up to is trying to distinguish noisy trades versus signal trades. Speculators/investors/hedgers/etc. are the primary players in the market. Some of those trades contain high information (e.g. maybe a person with access to insider information buying up stock before some announcement), some of them contain virtually no information and are pure noise (e.g. granny liquidating some of her portfolio for monthly expenses). In a naive market with no HFT signals we have no way of assessing the informational content of individual trades, we only have an estimated aggregate or average informational content of trade. Market makers will set their spread and sizes according to this aggregated informational content.

But over any sample the estimated average informational content of trades will not be the same as the realized, for example one week might more than usual insider trading, one month it might make up a small fraction. There's also a ton of path dependency when you work out the math, that amounts to pure randomness. Because of this securities will not perfectly track their "true price." The deviation is still stationary, because the more out of line the prices get with the fundamentals the more speculators will step in and push it back. No one is smarter than the market 100% of the time so every time a fundamental speculator sees a price that's too low/high there's some chance that the market is right and his valuation is missing something and some chance he's right. Speculators that aren't very good are probably only going to be "beating the market" when the valuation on securities looks insanely out of whack or by distributing his portfolio over a wide range of perceived mis-valuations to reduce his volatility. Only the very best speculators are going to be able to get their fundamental valuations consistently right within a small margin of error. So without HFT/Stat Arb./technical trading/whatever you want to call it/etc. the thing that keeps securities from randomly drifting too far are fundamental speculators.

Basically what HFT is doing, instead of fundamentally valuing securites, determining the informational content of individual trades or small time frames, using the signals I mentioned earlier. A segment of the price evolution with high information content tend to look very different from noisy trades on the small scale, but when aggregated up lose this distinguishability. It's almost symmetrical when you think about it. Fundamental speculators estimate a price for the security and trust in the reliability of the price evolution process in brining the market price to their estimated "true price". HFT trusts in the reliability of the initial price as being the best estimate of the value of the security and tries to identify errors and miscalculations in the price evolution process.

So with this basic background I'd like to address some of the comments:

@Sam: It's not like there was never a time when people weren't doing stuff related to what HFT traders do today. Joseph de la Vega covered some stuff in 17th century Amsterdam that sounds eerily similar to some modern HFT algos. Dow Theory/modern technical trading was invented in at the turn of last century. Perhaps most importantly market makers are not and never have been "naive" like in the toy model I described. They have always looked at the pattern of trades coming in, the movement of other related securities, and the state of the limit-order auction and set their quotes accordingly (hence why NYSE specialists fought tooth and nail to prevent other people from seeing the book).

The main difference is that in the past most of this was done based on gut intuition or streets smarts. The advent of cheap processing power now makes it possible to statistically test and mine these signals and do this scientifically rather than intuitively. (All the articles about HFT that I see seem to imply the technology is all about trading speed, but in reality an equally if not more important component are the gigantic computer farms used to run simulations). 10,000 floor traders have been replaced by 300 PhDs and 100 teraflops of aggregated processing power. Therefore the margin of error has become much smaller, the signals more predictive, which means the trades have higher expected profitability and therefore more trades cross the transaction cost threshold. Hence the explosion in volume. If you look before HFT the spreads were higher, but I'd say even more importantly the volume was much lower. Before HFT market makers weren't willing to quote the size that was needed to handle today's volume, because they didn't want to hold that much risk on their book. Today the quote sizes are if anything smaller but the levels refill much quicker. HFT is indirectly responsible for the explosion a lot of the derivatives that we see today. E.g. nothing like the 3x ETFs could have existed with 1990s liquidity levels and transaction costs. Whether you think this is a good or bad thing is up to you, but HFT is providing people what they want: a way for consumers to trade super-huge volume and for market makers to hold only a tiny amount of risk on their books.

@Derek: Most of the participants that HFT is "driving out of the market" is informed traders. By identifying which trades are noise and which have high information content HFT lowers the cost for the former and raises the cost for the latter. With less adverse selection market makers will be willing to provide more liquidity. Take insurance as an analogy, if someone is able to come in and identify the high-risk participants it will drive them out of the market, yet the insurance pool will get better. It's funny that we have such stringent insider information laws and the issue is taken so seriously in this country, yet HFT which is hands down the most effective deterrent of insider/informed trading is widely despised. Huh?!

@Jason: How about an even better idea. Instead of randomizing queues, how about we just get rid of them all together? I mean the financial markets are suppose to be the center of global capitalism yet at a very fundamental level a lot of it is rationed by queue, not by price. This sounds like something that belongs more in Moscow than New York! Let's just eliminate tick size completely. The only reason we have the stupid things are because of some invention from the 1860s. Floating point numbers can represent 16 digits of precision today. Tick sizes are really just a price floor for market makers, telling them that they are not aloud to quote spreads below a certain value (plus a bunch of other weird effects). Let's get rid of them, instead of people competing on speed which doesn't produce any consumer surplus they should compete on the dimension of better pricing.

However I do think you are wrong about HFT profitability being all about speed. That is a crude media portrayal. It makes a big impact for simpler algos where there's an apparent mis-pricing that everyone can see. But for the more complex stuff it's more about the sophistication than the speed. A lot of the second tier shops out there would go out of business without their co-locations, but trust me you could force a firm like Renaissance Technologies to have a 1+ second latency and they would still rake it in hand over fist.

@ Nathan: Good points. However you do see even in very sophisticated large markets blatant gaming going on. Which is supposed to be illegal, but alas I don't think the braniacs at the SEC can figure out Google Finance, let alone a Matlab session. A very common thing is you'll see someone with a quote on one side of the market (say the bid) sitting somewhere in the middle-to-back of the queue. Then the person will send a ridiculously large limit order to the other side of the market (say the ask) that they know will never get filled even a little bit before they can cancel. This will cause a lot algos (especially from the less sophisticated players) to think there's an imbalance in favor of the ask. They'll start actively selling which will eat through the bid queue, then cancel their ask order. They basically manipulated the market to get their order filled. Another insidious practice is to send a large order at some price improvement over the best bid/offer, then cancel all but 1 share. 1 share is virtually no cost, and now they've just got a free bit of information about the demand schedule. Finally another trick is if you want to trade in a certain direction (say sell) place a large sell limit order at a significant price improvement to the best bid then cancel immediately, then send your "true" sell order at the same price. A lot of times someone will see a big level and think it has high support and try to join it, but it's not really there and now you've just essentially tricked someone to trading with you at a much improved price over the bid.

IMO getting rid of tick sizes would fix a lot of this. After that what I would do is charge a very nominal fee (say 5% of the exchange fee) for every cancelled order. For almost all market participants this is trivial, it only gets to the same order of magnitude as the exchange fees for those who are canceling 10x as many shares as they're filling. It also has the added benefit that it deters blind passive away level filling (i.e. putting large limit orders at far-away prices knowing that you can cancel if you want by the time it gets close, otherwise you'll get to be near the front of the queue). While this practice isn't "evil" it is lazy and does reduce the information quality of the book.

The liquidity rebates also drive a lot of this. The difference between "providing" and "taking" liquidity can be 0.5 bps or more, which is significant for most HFT trades. So there's a lot of manipulating/weird trading patterns trying to turn what would otherwise be a liquidity removing trade to a liquidity providing trade. The simplest adjustment would be to pay the liquidity rebates only to shares that have been resting for longer than some minimum threshold. Or just get rid of the liquidity rebates altogether. The CME charges a fee both to provider and taker, and I think because of it you see a lot less shenanigans there.

Forgive me if I'm wrong, but don't stocks still have a fundamental value? Don't we still buy them to get a share in a corporation's earnings? If that is the case, ought not the volume of trading for a share of those earnings--and the winnings or losses incurred by those who flip said shares on a daily basis--have little to do with the actual earnings of said company?

Perhaps you can say that short run, downward movements of the equity price put the company in jeopardy because it will not be able to raise the cash it needs to continue to operate. If that is the case, I think the company probably has bigger problems than the vagaries of high-frequency traders.

Simple information theory says that high frequency trading should work as an investment strategy since it lets you exploit other's trading delays. Look at Ebay "sniping" for a good example of this in another marketplace. Of course, the others don't always want to play when someone else has better access to the ball.

Thank You HFT Quant for your detailed explanations. The first bit information on the subject beyond noise.

If HFT is actually a bad thing (not clear to me that all of it is undesirable), the best solution I've heard proposed is a move to discrete market clearing. Perhaps several times a second, but still not continuous.

HFT Quant had a great comment above.

It is somewhat discouraging that competition and effective risk-control needs to be defended against charges of "extracting" profits and that central planning advocates regret that non-market forces can't coerce effort into "studies of ecosystem development or the effects of industrial automation" or "traffic engineering". If there are anti-competitive practices in HFT (e.g., quote-stuffing, flash orders, etc.), then they should be regulated away -- otherwise, let competition reign. Efforts to regulate private pools of capital as broker-dealers is merely an attempt to widen the reach of political graft.

As a pedantic point, liquid securities are more valuable than illiquid securities, ceteris paribus. The extra liquidity premium may or not be worth $3BN per year; but, I don't see the need to justify profits obtained from free and open exchange.

I stand corrected already. I read on Wiki that HFTs are rarely leveraged and close out their positions daily. So as long as they don't succumb to the dangers of leverage, I can't see why they would cause any systemic risk.

@Penny Hater

"As a pedantic point, liquid securities are more valuable than illiquid securities, ceteris paribus. The extra liquidity premium may or not be worth $3BN per year; but, I don't see the need to justify profits obtained from free and open exchange."

Sure, but look at all the trouble this extra liquidity causes. If the purpose is to provide investment opportunities, the benefit of liquidity in seconds is not that large. If you think in terms of a year or more to hold an investment, isn't it enough for your trades to go through on the same day? And then all this craziness can just go away.

I have nothing against HFTs making money however they can, but we don't owe it to them to make a market where their tricks work well.

"It is somewhat discouraging that competition and effective risk-control needs to be defended against charges of "extracting" profits and that central planning advocates regret that non-market forces can't coerce effort into "studies of ecosystem development or the effects of industrial automation" or "traffic engineering". "

I don't want non-market forces to coerce effort into other studies. I simply see no legitimate reason to design a game so that HFT wins. I hope after changing the rules of the game that the people who are so smart at this game can find other profitable uses for their skills, and I suggested some that might or might not be profitable.

"If there are anti-competitive practices in HFT (e.g., quote-stuffing, flash orders, etc.), then they should be regulated away -- otherwise, let competition reign. Efforts to regulate private pools of capital as broker-dealers is merely an attempt to widen the reach of political graft."

Markets are by nature monopolies. If two markets are in direct competition for the same trades, there are opportunities for arbitrage unless the weaker market provides wider spreads, which further reduces its competitive standing.

Over the last 80 years we have had giant problems regulating corrupt procedures on the NYSE and other markets, from specialists to brokers. Self-regulation regularly finds fraudulent practices among brokers who lack connections. Government regulation stumbles across fraudulent practices here and there, usually after customers collect data that allows no other interpretation. Why should we put up with this? There is nothing particularly controversial about the mechanics of running an honest automated stock market. Let the government do it with full transparency. We don't need a cabal of stockbrokers to do it. We don't need stockbrokers at all except that people who want investment advice can pay for it.

Anybody who wants to manipulate other people's perceptions of the market to influence their investments can do so. But you can't make a profit on stocks you haven't kept for a year. That's fair.

You got a problem with that?

@HFT Quant,

Re: your reply to Derek:

The kind of insider trading that gets the attention of the SEC usually involves large moves. Double-digit percentage points, at a minimum. If I know of a pending merger, or a promising drug that has just failed phase III trials, then maybe my conscience or fear of the law might stop me from tipping off my brother-in-law's cousin's sister, but HFT algorithms scooping up a penny here and a penny there surely won't, not if I'm expecting a 30% payoff. Ripples in a pond don't matter if you know a tsunami is about to come through. If you are seriously claiming that HFT is "the most effective deterrent" against insider trading, then you really need to go into a whole lot more detail about the mechanism by which it does so.

Re: your reply to Jason (advocating abolishing tick sizes altogether):

Sure, floating point numbers can store ridiculous levels of precision, but a sum of money changing hands is measured to only two decimal places. If my bid is 16.00000000... and yours is 15.99999999... to 16 decimal places, then even if each of us was offering to buy 100% of the entire float of a given stock, the total amount paid in dollars and cents will come to the exact same thing. So if your order is for all (really, really, all) practical purposes identical to mine, why should yours be filled earlier?

If I'm a retail investor or even if I'm Warren Buffett, there's just no practical benefit to making tick sizes lower than a penny. I don't need to read that the closing price for GOOG today was 482.2703482693864, just for the sake of you being able to simplify the source code of your algorithm. You guys (HFT) may account for most of the volume, but we (the rest of us, collectively) account for nearly all of the market capitalization, so arguably the markets exist to serve us more than you.

@Morgan Warstler

"Nothing HFTQuant said deters the basic theory that we don't need them."

That looks true to me. But they are just traders, aren't they? They use publicly available information to choose what and when to buy and sell. If we want to make them illegal, we have to decide exactly what it is they're doing that we want to forbid, and then enforce it. That looks like a lot of work, and something that might be easy to make mistakes at.

So I say, if they can make a profit by looking at public information and deciding from it when to buy and sell, like anybody else, then let them. We don't need them. It's arguable whether they do more harm than good. But it's like ants at a picnic, it just isn't worth it to try to get rid of them.

If you really don't want them then find a way to change the rules of the game so HFT doesn't work. I suggested one way but it might not actually fit investor needs. Your idea to tax the transactions would probably work. If the transactions cost enough then HFT guys can't make a profit off small changes. Of course, that adds an extra cost for everybody. Would it be enough to tax away the whole profit if that profit is small enough per share and otherwise leave it alone?

Whatever changes you want to make, try to look at the whole picture and try to get it clear that it will be an improvement for itself. If the main purpose is to get rid of HFT, there's always the chance it won't work, and there's the chance it will open up new scams etc. So you don't want to do it just to get rid of HFT, because if you make a mistake the costs could be huge and the evidence that HFT is particularly harmful is weak. Get rid of HFT as a side effect of something that you firmly believe is useful for its own sake.

And it really wouldn't hurt if you pu in a few teraflops of simulations looking for ways to scam your new system. Because whether you do that or not, somebody else will.

Here's a good story on HFT and claims that the placing of orders that are immediately cancelled/never meant to be filled are designed to gather information that other market participants don't have:

"they intentionally probed the market with tiny orders that were immediately canceled in a scheme to gain an illegal view into the other side's willingness to pay."

Comments for this post are closed