Trading systems

Live price ticks are messy: ordering, warm-up and a kill switch for a trading bot

In a backtest, ticks arrive once, in order. A live feed sends them late, twice, or with conflicting prices at the same timestamp, and sometimes stops altogether. Here is how our demo trading bot guards its strategy from all of that.

In a backtest, market data is perfectly behaved. Every tick arrives exactly once, strictly in time order, with no gaps you did not put there yourself. A strategy developed against that data quietly assumes the world works the same way.

A live feed does not. When we connected QuantForge's strategy engine to a live demo account for gold, we found that the same tick could arrive more than once, that an older tick could arrive after a newer one, and that two ticks with the same timestamp could carry different prices. Then there are outages: the connection drops, quotes stop, and some time later they resume.

None of this crashed the bot. That was the problem. It just fed slightly wrong data into a strategy that trusted it. This post explains the guards we added, which all sit in front of the strategy rather than inside it.

Where the mess comes from

Our broker connection runs through an API that delivers prices from more than one server instance for redundancy. Each instance streams its own copy of the feed, and nothing deduplicates them before they reach us. Add network jitter, and arrival order no longer matches market order.

The first version of the bot only observed this: it counted out-of-order ticks for a diagnostics panel and passed every tick on unchanged. Measuring a problem is a good first step. It is not a fix.

Guard 1: decide on arrival

Every incoming tick is now checked against the newest tick already accepted, before anything else happens:

private accept(tick: Tick): boolean {
  const prev = this.newestAccepted
  if (prev) {
    if (tick.ts < prev.ts) {
      this.rejectedLateTicks++
      return false
    }
    if (tick.ts === prev.ts) {
      if (tick.bid === prev.bid && tick.ask === prev.ask) this.duplicateTicks++
      else this.conflictingTicks++
      return false
    }
  }
  this.newestAccepted = tick
  return true
}

The rules are deliberately simple:

  • Older than the newest accepted tick: rejected. The strategy has already moved past that moment.
  • Same timestamp, same prices: a duplicate. Dropped.
  • Same timestamp, different prices: a conflict. The first arrival wins and the second is dropped.

The last rule is a judgement call. Picking the first arrival is not more "correct" than picking the second, but it is deterministic, and it never invents a price. All three counts are shown on the bot's monitor page so a human can see how noisy the feed is.

Two consequences matter. Only accepted ticks reset the stale-feed watchdog, so a stream of duplicates cannot make a dead feed look alive. And because accepted ticks are strictly newer, the bot's coalescing slot (which keeps only the latest pending tick when ticks arrive faster than the strategy processes them) always holds the newest tick by time, not merely the last one to arrive.

As a second line of defence, the engine itself refuses any tick older than the last one it processed. In a backtest that check never fires, because replayed data is already in order.

Guard 2: an outage resets what the strategy learned

The bot's direction rule is based on a Range Filter, a smoothed trend filter computed from closed candles (one-minute candles by default). When the feed drops for a few minutes, the candles for that period are missing or incomplete. When it resumes, the candles on either side of the gap get stitched together as if nothing happened, and the filter's direction is computed from a history that is no longer true.

So when the feed goes from healthy to interrupted, the strategy forgets everything it learned from it:

/** Forget everything learned from the feed. The filter must warm up again from live ticks. */
reset(): void {
  this.window.length = 0
  this.candles.reset()
  this.rfDir = 0
  this.last = null
}

While the feed is down, ticks do not reach the strategy at all. A follow-up fix also clears a tick still waiting in the coalescing slot from before the gap, so nothing from the old history leaks into the fresh start.

Exits are not affected. A position that is already open keeps its stop-loss and take-profit, which live on the broker side and do not depend on the strategy's view of the trend.

Guard 3: a warm-up that matches the maths

After a reset, or at start, the strategy needs enough closed candles before its direction means anything. The first version waited for 2 × period candles. That was based on a fast path in the calculation code, which skips computing anything until it has 2 × period bars.

But skipping the calculation is not the same as having a value. The Range Filter is built in stages: the absolute price change starts at the second candle, an inner moving average of it needs period more, and an outer moving average of length 2 × period − 1 runs on top of that. The first filter value exists only after 3 × period − 1 closed candles. Before that, the filter has no value and its direction is zero. For our period of 20, that is 59 candles, not 40.

/**
 * Closed candles the Range Filter needs before it has a value at all: |Δsrc| starts at index 1,
 * the inner EMA(p) at index p, the outer EMA(2p−1) at index 3p−2, so 3p−1 candles.
 */
export function rangeFilterWarmupBars(period: number): number {
  return 3 * period - 1
}

The strategy now gives no direction at all until that many candles have closed, and the tests assert that no order is ever placed during warm-up. The monitor shows warm-up progress and lists exactly why the bot is not yet ready: feed not healthy, candles still warming up, ticks dropped and why.

The lesson: when one part of a system defines a threshold, derive it from the actual definition, not from an optimisation elsewhere that happens to use a similar number.

Guard 4: stop and kill that cannot be overridden

Every automated trading system needs an off switch that works under stress. The bot has two:

  • Stop blocks new orders immediately and synchronously. A queued start that arrives afterwards refuses to run.
  • Kill stops the bot and closes every open position to flat. To trigger it, the operator has to type KILL. A button alone is too easy to hit by accident and too easy to hit twice.

A review pass found a nasty race in the first implementation: if Stop or Kill was pressed while the bot was still starting, the start sequence could finish afterwards and set the state back to "running". Start and stop are now serialised with tick processing, Kill is applied immediately, and neither can be overwritten by a start that was already in flight. Stop and Kill are never gated behind any check. Start is: it re-verifies that the account is a demo account and asks the operator to acknowledge the worst-case loss.

Heartbeat messages, which only confirm the connection is alive, can never trigger an order. Only a fresh, accepted tick can price one.

What carries over to any real-time system

  • Put ordering and deduplication in front of your logic, not inside it.
  • Make conflict rules deterministic and conservative, and count what you drop.
  • When the input stream breaks, throw away state derived from it.
  • Compute warm-up requirements from the real formula.
  • Build stop controls that cannot be undone by a race, and test them during outages.

Backtests are where an idea is tested. The live feed is where the plumbing gets tested. They need different guards.

Back to insights