Best Automated Trading Strategies 2022

To skip down to the best automated trading strategies click here.            

What is an automated trading system?

An automated trading system is a set of rules that can be programmed for a computer to automatically execute trades whenever the rules occur in a given financial market. An automated trading system is the same thing as an automated trading strategy, an algo strategy, a trading algorithm, a trading robot or an algorithmic trading strategy.

In short, any pre-determined set of buy and sell rules that can execute trades automatically can be considered an automated trading system. Trading rules can be any set of if then scenarios and do not need to be complicated mathematical models.

Simple Automated Trading System Example

The simplest automated trading strategy is a moving average cross strategy. This system buys when the stock price rises above the moving average and sells when the stock price falls below the moving average.

Moving Average Cross System Example

Algorithmic traders have long known this strategy to lose its luster in financial markets, but this system is a great example of how simple buy or sell decisions can be made using technical indicators or market prices. If you can think it, then it can be coded.

Later we will get into much more complex algorithmic trading strategies and trading algorithms. However, I will not go into black box trading where strategies are known to the computer but not known to the human in charge of the computer.

How Does Algorithmic Trading Work?

Algorithmic trading works over the internet allowing one computer or server to receive market data and send trading instructions to another computer or server. For instance, a high frequency trading firm may have their servers co-located in the exchange’s data center. This allows for lower latency in receiving new market prices and issuing new orders.

This server can also calculate the trading strategies and send buy and sell orders directly to the exchange. The exchange parses the message and executes the market makers’ desired actions.

Automated Trading System Architecture and Design Simplified

Most trading systems communicate with the exchange through Financial Information Exchange protocol or FIX protocol. However, many independent traders have brokers that provide this communication connection built into the platform and most traders never have to worry about how algorithmic trading works.

Most platforms, brokers, and software provide application programming interfaces or APIs that make it easy to connect custom code to the exchange or broker. However, software like Build Alpha or TradeStation make it possible to connect directly with data providers, brokers, and the exchanges so traders only have to worry about entries, exits and risk management.

What are different types of automated trading systems?

There are various different types of trading strategies but let’s cover the five most popular strategies below. These strategies can apply for longer term investors or day traders. For a more extensive look please check out my Algorithmic Trading Guide.

Momentum or Trend Following

Arguably the simplest and most widely used – especially amongst commodity traders. Trend following strategies aim to capture prolonged price movement in a single direction or a “trend”.

Trend strategies tend to have lower win percentages as these strategies have outsized winners in comparison to more frequent but smaller losers. The typical returns from a trend strategy may be a series of small losses and then one large win when the system captures the large trend move.

Trend Following Trading Strategy P&L Curve

These strategies are best implemented by traders with strong resolve and the ability to withstand frequent losses. Periods of chop and noise are much more common than big trend moves thus leading to your trading account not making much progress most of the time and then making large leaps “at once”.

Mean Reversion

Mean reversion trading strategies tend to have higher win percentages as they have many frequent small wins and the infrequent large loss. The typical returns from a mean reversion strategy may be a series of small wins and then one large loss when the trend breaks.

The simplest mean reversion strategy is one that aims to buy and sell large deviations from a moving average or volume weighted average price. As price moves away from the mean, however calculated, the strategy looks to enter a position with the intention of price returning to the average price.

Many traders refer to these extremes as overbought or oversold and attempt to quantify them with technical analysis, technical indicators, mathematical models or statistical concepts.

Mean Reversion Trading Strategy Explained

In theory, traders that employ mean reversion strategies can experience profit and loss of their trading account to inch higher most of the time with periodic large set backs. Many market makers and dealers experience this.

Mean Reversion P&L Explained

Statistical Arbitrage

Typically, statistical arbitrage is looking for short-lived opportunities between two securities. Many traders will isolate two correlated or related stocks such as Coca-Cola and Pepsi and monitor the spread or difference between the two. Whenever the difference between the two becomes large enough the trader places a trade buying the cheaper and shorting the expensive until the pairing comes back into normal ranges.

An obvious simplification, but many traders cannot execute pair trading or statistical arbitrage with retail brokerage accounts and software. There is also steep competition from professionals, market makers and hedge funds.

Market Making and High Frequency Trading

A market maker can be thought of as a liquidity provider that quotes both bids and offers regardless of market conditions. The market maker will hold inventory and aims to profit on the bid-ask spread or the difference between the highest price someone is willing to buy and the lowest price someone is willing to sell.

Market making is arguably the most complex form of algorithmic trading and automated trading as it requires tons of price data, quote information, data from multiple stock exchanges, active trading activities and trading decisions and is often completely automated with zero room for discretion.

Market Maker Pension Funds and Stock Exchange

High frequency trading (HFT) is market making or trading in fractions of a second based on powerful computer programs executing large numbers of orders almost simultaneously. The depths, pros and cons, and nuances of HFT and market making far exceed the scope of this post.

Pattern Trading

In the previously mentioned Algorithmic Trading Guide, I mention pattern trading as being one of the most common strategy types for algorithmic traders. This style was popularized by Jaffray Woodriff of Quantitative Investment Management in Jack Schwager’s Hedge Fund Market Wizards book (still one of my favorites).

This style looks for repeatable patterns that shift the odds one way or the other. For example, based on historical data, whenever this candlestick formation or these two events have occurred there has been a X% chance price is higher over the next N days. Again, a gross simplification, but a trading style most traders are familiar.

Price Action Technical Analysis

Is Automated Trading Profitable?

Yes. The leading 12 investment banks earned about $2 billion from portfolio and algorithmic trading in 2020 according to Coalition Greenwich. Additionally, the greatest hedge fund of all time – Renaissance Technologies – is strictly algorithmic. Listen to James Simons, the founder, speak from 36:48 to 38:00.

James Simons (full length interview) – Numberphile – YouTube

Furthermore, I have posted tons of algo trading success stories on the Build Alpha blog. Successful automated trading requires rigorous testing, robustness tests, stress testing, considering multiple market conditions, contextual data and more. Nothing manual trading does not require. I wrote a bit about strategy reliability here: Robustness Testing for Trading Strategies.

Can I automate my trading strategy?

Yes. We live in the golden age of algo trading as nearly 75% of US equity volume now comes from algorithms while algorithmic trading is expected to grow at 11.23% CAGR over the next five years. Brokers and data platforms have made trade automation incredibly simple.

My goal with Build Alpha has been to connect those with a desire to automate trading strategies with the tools to do so without having to learn or write any code. Now anyone can test, build and automate their trading strategies without any code.

What’s the best automated trading software?

Build Alpha is the best automated trading software. I am biased because as I set out to find the best automated trading software, I could not find one that gave me the end-to-end tools necessary. That is what prompted Build Alpha’s development oh so many years ago. That being said, I will review the three most popular software for automated trading strategies.

Build Alpha

Build Alpha enables traders to create, test, and automate trading strategies with no code. There are 5,000+ built-in entry and exit signals as well as the ability for users to add their own signals with a drag and drop builder. The signal library includes everything from seasonality, price action, candlesticks, chart patterns, technical indicators, market breadth, options flows, economic data, time of day and more.

Build Alpha works in two ways. First, the trader can select specific entry and exit signals from the point-and-click interface to test a specific strategy. On the other hand, Build Alpha’s genetic algorithm can take thousands of inputs such as potential entries, exits, filters, risk management, position sizing, account size and will build the best possible strategies.

Build Alpha No Code Trading Strategy Builder - Dark Mode

The trader can also specify certain thresholds that each strategy must pass such as the Monte Carlo simulation must be profitable or the profit factor must be above 1.7, etc. These automated workflows allow for the fast creation of strategies that fit any trader’s desired risk tolerance and performance thresholds. 

Build Alpha then connects to live market prices for real-time strategy monitoring or can produce fully automatable code for every strategy. This code can be copy and pasted into various broker platforms for live trading.

Take a look at these to see step-by-step walkthroughs 

TradeStation

TradeStation makes automating trading strategies very simple as TradeStation is a broker and has a really reliable platform which is a favorite among independent algorithmic traders. TradeStation also created Easy Language, a proprietary programming language aimed to make trading strategy development much simpler for traders.

Traders can write their strategy logic using Easy Language inside TradeStation’s Development Environment. Once the strategy’s code is complete, the trader can easily add the strategy to a chart to enable simulated or live trading.

A simple moving average crossover strategy may only require two lines of code using Easy Language. Here is an example below:

If Close crosses above average(close,10) then buy 100 shares next bar open;

If Close crosses below average(close,10) then sell 100 shares next bar open;

For those of you interested in a TradeStation account, email me to find out how TradeStation will pay for your Build Alpha license.

Python

Python is undoubtedly the fastest growing programming language due to its easy-to-read syntax and clean scripting style. Many traders have flocked to python as the vast number of public libraries that have already built trading functionality such as technical indicators and other position sizing logic coded continues to grow.

If you are interested in learning about using python to build automated trading strategies, then I highly recommend checking out a few other resources I have created.

How to create a trading algorithm in 3 Steps with Build Alpha

Build Alpha is the simplest way to create algorithmic trading strategies as it does not require any coding. Let’s take the moving average cross strategy above and build it in 3 clicks.

Step 1 – Select Signals

Search for SMA signals and select an entry and exit. Let’s enter when the close crosses above the 10-period SMA and exit when the close crosses above the 10-period SMA.

Build Alpha Simple Moving Average Signal Selection

Yes, you can optimize parameters. I am showing the lazy hardcoded settings. Traders can create any range of parameters to test and optimize.

Step 2 – Select Dates and Symbol(s)

Select your test dates, symbol(s), and any risk management. I will select SPY and test from 2006 to 2022.

Select Symbol and Test Dates - Build Alpha

Step 3 – Simulate

Hit Simulate and view your results. If the results are suitable (they should not be) then hit one of the “Generate Code” buttons to get fully automatable code for this (or any) strategy.

Sample Generated Code for TradeStation Easy Language

It is important to note that Build Alpha permits automated trading strategy monitoring in two ways. First, you can export generated code. Second, you can connect your broker’s data feed with existing connections to TradeStation, Interactive Brokers, QuoteMedia, Binance, and more.

Example Automated Trading Strategy

Later in this post I will share a table that automatically updates with the best technical analysis strategies for all financial markets. Each strategy uses one rule for entry and one rule for exit. These are meant to give insights to what is working now, building blocks for your own strategies, and general free information.

This strategy below has two rules: one for entry and one for exit. The trading strategy is applied to CSCO or Cisco Systems Inc which is a large multinational technology conglomerate.

Sample Technical Analysis Automated Trading Strategy

This strategy enters when the closing price crosses below the 5-period exponential moving average (EMA) and exits when the closing price crosses above the 5-period exponential moving average.

At the time of publishing, you can see a sample of this strategy’s performance on the far right over the past 1,809 trades earning $18,138.17 per 100 shares traded over the last 12 months. This strategy also would have made $1,883.81 over the last 30 days.

What are the best automated trading strategies right now?

First, how do we define best? The simplest way is total P&L. Second, what is a trading strategy? In this case, let’s look at all possible combinations of technical indicators and only use one rule for entry and one rule for exit.

Can we build more advanced systems? Yes. However, this results table serves to show us what parameter settings, lookback ranges and indicators have been performing well lately. Perhaps this info can serve as building blocks for new automated trading strategies or just free information for my fellow system traders.

In the table below, you can sort by asset class, timeframe, ticker, and minimum trade count. All strategies are sorted by their total profit and loss over the last 30 days and last 12 months. All strategies use the same position sizing. 

 

Key Points

  • Automated trading systems automatically execute buy and sell orders based on pre-defined rules.
  • There are at least 5 different types of algorithmic trading strategies
    1. Trend Following
    2. Mean Reversion
    3. Statistical Arbitrage
    4. Market Making / High Frequency Trading
    5. Pattern Trading
  • It has never been easier to automate trading strategies – even with no programming
  • Build Alpha, TradeStation and Python are easier routes to algo trading
  • The best trading strategies are changing all the time, but the table above is free to all

Summary

Automated trading strategies continue to grow in popularity as traders with no programming background can now turn their strategies into trading algorithms. Automated trading strategies are just pre-defined rules that instruct a computer when and how much to buy and sell in financial markets. I have spent the past decade involved in professional trading and can honestly answer there is no best automated trading strategy. However, a combination of robustness testing and portfolio construction can help any trader gain an edge in algorithmic trading over the trader that fails to grasp these algorithmic trading concepts.

Author

David Bergstrom Author Photo Trading Edge

David Bergstrom – the guy behind Build Alpha. I have spent a decade-plus in the professional trading world working as a market maker and quantitative strategy developer at a high frequency trading firm with a Chicago Mercantile Exchange (CME) seat, consulting for Hedge Funds, Commodity Trading Advisors (CTAs), Family Offices and Registered Investment Advisors (RIAs). I am a self-taught programmer utilizing C++, C# and python with a statistics background specializing in data science, machine learning and trading strategy development. I have been featured on Chatwithtraders.com, Bettersystemtrader.com, Desiretotrade.com, Quantocracy, Traderlife.com, Seeitmarket.com, Benzinga, TradeStation, NinjaTrader and more. Most of my experience has led me to a series of repeatable processes to find, create, test and implement algorithmic trading ideas in a robust manner. Build Alpha is the culmination of this process from start to finish. Please reach out to me directly at any time.

Automate Trading with No Coding | Complete Guide

Can you automate your trading?

Yes, you can automate your trading! In fact, according to BusinessWire, algorithmic trading is responsible for 60-73% of all U.S. equity trading. Most brokers support automated trading and even provide easy to learn programming languages to build your first automated trading system.

Three Ways to Automate Your Trading

Most broker platforms support automated trading with two options for automation. Build Alpha provides a third.

  1. A development environment to add your own code and will execute your trades when your trading rules are true.
  2. An application programming interface or API which allows your code to speak directly with the broker’s platform.
  3. Build Alpha – a no code algo trading platform.

Both of the first two solutions require tedious hours learning to code, intricacies of the platform’s development environment or API, and introduces fragile breakpoints that can harm your trading results if you are not an experienced programmer.

Later in this article, I will introduce how you can still automate your trading with no coding using Build Alpha and a variety of popular brokers. The benefits of automated trading are still possible without the ability to program!

Is Automated Trading Profitable?

Absolutely. Automated trading is simply having a computer execute trades instead of you manually clicking the mouse buttons. The largest banks, hedge funds, and the best trader of all time are staunch supporters and users of automated trading.

I also highlighted several Build Alpha success stories (with statements) in my Algorithmic Trading Guide.

Trading Algorithms Complete Guide

Many fear automated trading is not profitable because they fall for snake oil salesmen selling overfit trading strategies that were not developed to last but developed to sell. Automated trading requires robustness testing or stress testing to break the strategy before the market does. I wrote more about Robustness Testing here.

What is an automated trading system?

An automated trading system is a set of pre-defined entry and exit rules executed by a computer program. Many proper trading systems contain risk management and position sizing as well.

Simple Moving Average Example

The simplest example of an automated trading system, and probably the most popular technical analysis system, is the moving average crossover. In the moving average crossover system, a trading system would enter a long position when the faster moving average (shorter length) crosses above the slower moving average (longer length). The system would exit a long position when the faster moving average crosses below the slower moving average.

Moving Average Automated Trading Systems

How do I automate my trading with no coding?

Build Alpha is no code algo trading software that allows traders to create hundreds of algorithmic trading strategies on historical data at the click of a button. No algo trading experience needed.

There are thousands of built-in entry and exit signals to choose from and the ability to create your own with a drag and drop builder or using python. The built-in library covers everything from

  • seasonality
  • price action
  • candlestick patterns
  • chart patterns
  • technical analysis indicators
  • volume
  • volatility
  • pre- and post-market
  • market breadth
  • options flows
  • economic data
  • dark pools
  • and more

Simply search and select your desired entry signals, exit signals, risk management and position sizing then hit simulate and Build Alpha will generate the best results.

Build Alpha Trading Platform Strategy Builder

From the results window, select your desired trading system then click on one of the code generators in the lower right. The code generator will produce fully automatable code that can be copy and pasted into any of the supported broker platforms.

Automated Trading Platform Code Generation

Code Generators Supported

  • TradeStation
  • MultiCharts
  • NinjaTrader8
  • MetaTrader4
  • MetaTrader5
  • Pro Real Time
  • Interactive Brokers
  • Python

Best Automated Trading Platform

 

No Code Algo Trading Strategy Example

Step 1 – Determine the Strategy

Let’s build the famous 2-period RSI strategy from Larry Conners’ book Short Term Trading Strategies that Work on SPDR S&P500 ETF SPY.

This strategy enters when the 2-period RSI is below 10 and price is above the 200-period simple moving average. On the other hand, this strategy sells when price is above the 5-period simple moving average.

Step 2 – Set account size and position sizing

We will use a fixed size of $10,000 per position. We achieve this by setting the account size to $10,000 and the position sizing to Fixed. Both are found in the settings menu.

Expert Advisor Position Sizing

Step 3 – Select Symbol(s)

Next, we need to select our symbol as SPY.

Build Alpha Platform Symbol Selection

Step 4 – Select Entry and Exit rules

Finally, let’s select our trading decisions or entry and exit rules. The entry requires the following two conditions to be true

  • 2-Period RSI is below 10
  • Price is above the 200-period simple moving average

Alternatively, the exit signal closes the trade when price is above the 5-day moving average.

Automated Trading Strategy Signal Selection in Build Alpha

Here is a quick gif searching and selecting the following entry and exit signals in Build Alpha.

Conners RSI Build Alpha Gif

Click on the gif to view/watch

Now that we see our results, we can highlight the strategy, review its performance metrics and equity curve, and ultimately generate code for it.

Step 5 – Generate Code

In the results window, highlight your strategy and navigate over to the lower right where you will find the Generate Code buttons. Find your preferred broker platform and hit Generate.

Build Alpha Code Generators - No Code Algo Trading Software

The Generate Code button will create complete code for the highlighted strategy that can be copy and pasted into your broker’s platform. Voila! Automated trading with no coding necessary. Steps for each broker below.

Build Alpha Generated TradeStation Code - No Code Automated Trading Platforms

How do I set up my automated trading with Generated Code? Best Automated Trading Platforms

Great! How do we take the generated code and start auto trading buy and sell orders? Let’s walk through how you can take the generated code and set it up in the three most popular automated trading platforms.

TradeStation – Automated Trading Platform

TradeStation is arguably the consensus favorite among automated trading platforms. It is very easy to use, reliable and has competitive fees.

Step 1 – Open Development Environment

The first step to automating with TradeStation is to open the Development Environment found under View >> Development Environment.

TradeStation Development Environment

Step 2 – Create new strategy File

Next, create a new strategy File >> New >> Strategy

TradeStation New Strategy

Step 3 – Copy and Paste

Then copy and paste the Build Alpha generated code into the new Strategy and hit F3 to verify. You can hit CTRL+A to highlight all the Build Alpha generated code then CTRL+C to copy it all. Then inside the new Strategy in TradeStation’s Development Environment you can hit CTRL+V to paste it all.

Build Alpha Generated TradeStation Code - No Code Automated Trading Platforms

Step 4 – Add to Chart

Finally, open a new chart and set it with the symbol you wish to trade. Then right-click and select Insert Strategy!

Here is a 1-minute (no sound) YouTube video showing this exact process: Build Alpha code to TradeStation

Please note this is an older version of Build Alpha’s results window. There are many more features now!

As part of TradeStation’s confidence in the software, we have partnered to help traders have the best of both worlds. If you open a new TradeStation account and license Build Alpha then TradeStation offers a commission rebate plan to repay you until they have paid for your license. For more information, please check out TradeStation Build Alpha promo.

 

NinjaTrader – Automated Trading Platform

NinjaTrader8 is a close second and highly favored automated trading platform among futures traders. Below are four simple steps to take Build Alpha generated code and enable inside NinjaTrader8.

Step 1 – Open Script Editor

Open up the NinjaScript Editor

Open new NinjaScript Editor

Step 2 – Create new strategy file

In the NinjaScript Editor, right-click on Strategies to create a new one.

Step 3 – Name your strategy

Name your strategy and hit Generate.

Name your NinjaTrader Strategy

Step 4 – Copy and Paste

In the new NinjaTrader Strategy that appears, please hit CTRL+A to highlight all text and then hit ‘BACKSPACE’ to delete it all. Then inside Build Alpha, hit Generate NT Code. Please highlight and copy all the generated code using CTRL+A, CTRL+C. Then come back to the blank NinjaTrader Strategy and hit CTRL+V to paste it all.

Step 5 – Quick Edit

Go to line 47 and edit the class name to be the same as the designated strategy name. You can go to line 47 by hitting CTRL+G then entering 47. I named the strategy NinjaExample.

Changing NinjaTrader Strategy Name

After making this change you can verify or compile the strategy by hitting the above button.

Compiling NinjaTrader Strategy

Step 6 – Add to Chart

Open up your desired chart, right-click and select Strategies. Alternatively, you can hit CTRL+S.

NinjaTrader8 Strategy

Step 7 – Enable

Select your strategy and be sure to enable the strategy in the Properties pane.

NinjaTrader Enable Strategy

MetaTrader4 and MetaTrader5 – Automated Trading Platforms

MetaTrader4 and MetaTrader5 are both supported by Build Alpha’s code generators. However, most traders should migrate to MetaTrader5. In a recent Build Alpha survey, MetaTrader was the lowest ranked automated trading platform among the three listed in this article. I highly encourage you to check out the other two if you are currently struggling with MetaTrader.

Step 1 – Open MetaEditor

Open MetaEditor. This can often be done from your search menu or from within MetaTrader.

Step 2 – Create new EA

Create a New Expert Advisor. Expert Advisors are what MetaTrader calls strategies. Give your expert advisor any name you desire. I will use MetaExample.

Step 3 – Copy and Paste

In the new strategy that appears, please highlight and delete all text. You can hit CTRL+A and ‘BACKSPACE’ as we have done before.

In Build Alpha, hit the Generate MT4 Code or Generate MT5 Code button. Then highlight and copy all the text. This can be done using CTRL+A, CTRL+C. Back in the MetaEditor, paste all the generated code using CTRL+V. Then hit the Compile button in the top menu.

MetaTrader5 Compile button

Step 4 – Add to Chart

You can now add expert advisors to the tester or a live chart.

Can I automate my day trading?

Absolutely. Build Alpha has time frame selection, time of day filters, and a Force End of Day Exit option that forces any strategy to close open positions at the specified time.

The pre- and post-market signals allow traders to specify signals around volume weighted average price (VWAP), pre- and post-market volume or pre- and post-market highs and lows.

The Quick Test feature even allows you to test a strategy across thousands of symbols and view the optimal times to take profits, stop out, or when highs and lows of the day are made. The test even shows you most profitable and least profitable symbols as well as a summary across all tickers.

Build Alpha Quick Test Results for Day Trading

Please note it is best that day traders do not use excessive leverage or size as there is a high risk of losing money rapidly due to leverage.

Automated Trading Pros

What are the benefits of automated trading?

More Markets

Market data moves so quickly, and manual trading can rarely keep up. There are often missed trades during fast moving market conditions. Automated strategies can be one of many trading solutions to help.

Trading more markets, more systems can often mean getting to the law of large numbers faster. If your systems have edge, then this means a rising account.

Quantified Risk

Experienced traders know that having a set of rules with a risk you can afford to take is the key to financial longevity. New traders seeking massive upside often take the high risk and lose money because they do not have a system or have properly quantified their trading edge.

Computer Never Sleeps

Certain financial markets like futures, forex and cryptocurrencies trade 24 hours per day. There are tons of trading account opportunities in off market hours and auto trading and automated trading strategies can help.

Fewer Mistakes

Trading is hard. Losing your money sucks. Losing your money from human errors like fat fingering an order or entering the wrong symbol suck even more.

Automated systems help avoid these blunders. Consider whether you understand how big of an impact this can have on your emotional state. No more calling customer support to see if they will cancel a trade.

Less Emotions

Backtesting trading strategies and understanding how a system can make money or lose money over the next N trades is crucial to reducing emotions. If you have a plan, you can stick to it.

If you do not have a plan, then losing your money heightens stress, which heightens emotions, which raises your probabilities to harm your trading account.

Known Systems

Automated trading is much more than automatic order entry. Advanced trading work is understanding edge, setting proper expectations, and quantifying your risk. Many investors do not know their systems and cannot properly answer these above questions. How do you think these traders wind up?

Automated Trading Cons

Where to Start with Automated Trading?

There are so many markets, symbol, timeframes and potential signal combinations. New traders often do not know how to start. The simplest solution is to start quantifying what you do know. For example, does price need to be above a moving average? What moving average? Does yesterday need to be a bullish or bearish day? What about volume?

I wrote here about quantifying simple ideas:

Solution: Build Alpha does not require you to have a trading idea. You can of course test your own ideas, but if you do not have an idea you can select thousands of input signals at once and Build Alpha will find the best strategies for you. The genetic algorithm learns from your inputs and creates the best combinations quickly.

Computer needs to be on 24 hours?

Yes, if your computer is not connected to the internet or turned off then the automated trading systems will not send orders to the broker’s trading platform.

Solution: Rent a virtual private server (VPS) which is a computer that is on 24-7 which you can remotely log into. Amazon, Microsoft and other smaller companies have been reliably offering this service for years and often you can rent a VPS for only a few bucks per month. All PCs have remote monitoring software built-in.

What if my trading signal is not included in the built-in signals?

There are nearly 5,000-plus signals built-in, and I have done my very best to include those with above average e-ratios, but it is possible that your trading idea requires something unique.

Solution: Build Alpha has two ways to add custom signals that can be passed to the strategy builder engine. First, there is a drag and drop signal builder. It allows the trader to combine any technical indicator, math operator, and custom parameter setting.

Build Alpha Custom Signal Builder

Second, there is the ability to add custom signals with python for those with the abilities.

Need to Know

  • Most trading volume is from automated trading
  • Anyone can automate their trading even without programming
  • Automated trading pros outweigh the cons
  • Most brokerage platforms support automated trading now

Summary

Automated trading is growing every year with now nearly three quarters of U.S. stock market volume being attributed to automated trading. In the past, creating automated trading systems required access to historical market data, live market data, broker connections, and the ability to program it all. However, now, Build Alpha provides professional automated trading software to all traders. This enables all traders to create, test, and automate any trading strategy on a variety of automated trading platforms.

Automated trading platforms support both demo account and live account implementations. Cryptocurrencies, Forex, CFDS are complex instruments and come with a steeper learning curve and intricate nuances, but traders can focus on futures, stocks and ETFs as all markets are automatable.

Author

David Bergstrom Author Photo Trading Edge

David Bergstrom – the guy behind Build Alpha. I have spent a decade-plus in the professional trading world working as a market maker and quantitative strategy developer at a high frequency trading firm with a Chicago Mercantile Exchange (CME) seat, consulting for Hedge Funds, Commodity Trading Advisors (CTAs), Family Offices and Registered Investment Advisors (RIAs). I am a self-taught programmer utilizing C++, C# and python with a statistics background specializing in data science, machine learning and trading strategy development. I have been featured on Chatwithtraders.com, Bettersystemtrader.com, Desiretotrade.com, Quantocracy, Traderlife.com, Seeitmarket.com, Benzinga, TradeStation, NinjaTrader and more. Most of my experience has led me to a series of repeatable processes to find, create, test and implement algorithmic trading ideas in a robust manner. Build Alpha is the culmination of this process from start to finish. Please reach out to me directly at any time.

Stop Losses | Complete Guide and Test Results Reveal

What is a Stop Loss?

A stop loss is an order to close an existing position to limit losses when the market reaches a certain price. A stop loss is a form of risk management and intends to serve as protection from more severe trading or investing losses. The purpose of a stop loss is to automatically exit a trade when the trader cannot stomach more pain in the trade or the set up has been invalidated. Before we get into all the types and variations of stop losses including the trailing stop loss, let’s become masters of understanding stop losses as a money management tool.

Stop Loss Order Basics

How Does a Stop Loss work?

A simple stop loss example is Cody buys 100 shares of Apple stock at a purchase price of $104.00 to open a long position (aiming to profit from a rise in AAPL stock). Cody does not want to risk more than $200 on his trade.

If AAPL’s price falls to $102.00 then Cody’s 100 shares would result in a $200 loss. Cody can set a stop loss order at $102.00 so that if and when AAPL falls to $102.00 his shares will be automatically sold.

Stop Loss Order Colorized Example

When the market price reaches the stop price, the stop-loss order is converted to a market order and is generally executed immediately thereafter assuming during market hours. It is important to note that the market’s liquidity will determine the actual fill price of the stop loss order after conversion to a market order.

If there is a buyer bidding to buy at least 100 shares at $102.00 then Cody’s stop loss order will sell to that buyer at that share price thus closing his position for a $200 loss.

However, if the best bid for 100 shares is at a lower price of $101.98 then Cody’s stop market order to sell will be executed at a stock price $0.02 below his stop level resulting in a loss of $202. This is because the stop loss order converters to a market order and takes the available current market price.

Market Price or Limit Price? What is the difference between a Stop Loss and a Stop Limit Order?

In the example above, the best bid could have been $101.08 instead of $101.98 and when price falls below the stop and the next available market price is significantly below the trader’s desired exit level then significant losses may ensue.

What if in this scenario Cody would prefer to hold the position and wait for the share price rises or current price to recover before selling?

Stop Loss vs Stop Limit Orders

The stop limit order allows the trader to enter a stop price and a limit price which converts the stop order into a limit order (instead of a market order) after the stop price has been hit. Then the trade will be executed at the limit price (or better). This could alleviate some of the concerns around market orders.

Let’s revisit Cody’s position from earlier and assume he placed a stop limit order with a stop price of $102.0 and limit price $101.90. In the first example, Cody’s transaction price would be $101.98. In the second example, Cody’s stop would trigger but the limit order would never execute because price was already below his limit price ($101.08 vs $101.90). In this scenario, Cody would hold the shares and until the security price rebounds (hopefully).

The big difference is that a stop loss converts into a market order and a stop limit order converts into a limit order. The risk with stop limit orders are thinly traded markets or gaps beyond the stop limit. There are no stop limit orders guarantee to get filled at the specified limit price.

How to Calculate Stop Prices? Where to place them?

Method 1: beyond a recent high or low-price level

Those using technical analysis and chart patterns to trade will advise you to put a stop below (above) a recent low (high) that would invalidate the chart or pattern that prompted the entry.

However, one should not choose the minimum price below the most recent low but a decent margin below the recent price level so all the stock drops and natural volatile markets can happen without stopping you out of your position prematurely.

Build Alpha Chart Pattern and Stop Example

However, I have now spent years working with a high frequency trading, market making firm and I can assure you this is not a great strategy for existing positions especially in fast moving markets. We joke and call this the “draw a line stop loss”. This rings true for long and short positions and all trading hours.

Method 2: based on your risk tolerance

Another exit strategy to determining when to sell stocks on adverse price moves is to determine your entry, determine how much you are willing to lose and place your stop sell order at this amount. This may apply to traders that have pre-determined their position sizing based on the entry or their account size.

This strategy lacks forethought and traders must realize the market does not care where your pain threshold is. You may choose a price level that can be hit easily.

Method 3: something testable

Calculating stop loss orders is no different any other part of your trading system; that is, it should be fully testable. Often times using method one above does not result in something that can be backtested. Often times backtesting can result in isolating situations that produce more money or more favorable market conditions. This can also apply to placing sell stop or limit orders. Some easy ideas for something testable are:

  • fixed dollar amount
  • fixed Average True Range amount
  • dynamic dollar amount
  • a rolling N-day low (or high for a short position)

What is a Trailing Stop Loss?

A trailing stop loss aims to protect profits moving the stop price higher as the market price moves higher. Trailing stops are often an excellent tool to protect gains and combat market volatility that wishes to shake you out of profitable positions. Of course, one can use either a trailing stop market order or a trailing stop limit order.

Trailing Stop Limit Example

As the trading day progress and the trade becomes more profitable, the trailing stop can be updated instantaneously or at the end of each bar.

How to Calculate a Trailing Stop Loss?

Trailing stops are typically calculated from the maximum trade price or high watermark while in a long trade. Short positions just have reversed logic calculating from the minimum trade price during the trade.

Cody’s original transaction price was to enter a long trade at $104.00. Let’s assume Cody placed a $1 trailing stop which would initially start at $103.00 or $1.00 below his entry price. After the market opens, AAPL’s price shot up to $104.77 raising Cody’s trailing stop price to $103.77.

If trading fails to make a new high above $104.77 then Cody’s trailing stop would remain unchanged at $103.77. However, if the security price rises to a maximum price of $105.66 then Cody’s trailing stop would move to $104.66 which is above his entry price of $104.00. At this point, Cody has “locked” in some gains even if price drops. However, investing involves risk and anything can happen (look above to thinly traded or overnight gap risk comment).

What is the best Stop Loss strategy? Static or Dynamic?

Stop losses, stop limit orders, trailing stops, oh my. What are the best stop loss strategies to add to your trading systems? First, let’s discuss a few variations of what we have discussed so far.

Static vs Dynamic Stop Losses

A static stop loss is a stop that is calculated the same way regardless of the current market price and market volatility. For example, a trader may employ a $200 stop for every trade he places regardless if the stock price significantly moves from the time of his first trade to the time of his most recent trade.

Static Stop Loss Pros

  • Simplify what to expect
  • Always understand your risk
  • Easy to think in fixed (static) terms

Static Stop Loss Cons

  • Fails to account for changes to underlying
  • Fails to account for different volatility regimes

A dynamic stop loss is calculated based of the underlying price or market volatility. This allows the sell stop order to maintain an appropriate distance from the entry regardless of the market conditions. In periods of high volatility, the stop price will be farther from the stock price and in periods of low volatility the stop price will be closer to the stock price dynamically adjusting how “risky” the trade is.

Dynamic Stop Loss Pros

  • Adjusts to current market environment
  • Accounts for drastic changes in the underlying (stock going from $50 to $500)

Dynamic Stop Loss Cons

  • Difficult to calculate on the fly
  • Difficult to think and understand your risk quickly

Static vs Dynamic Stop Loss

Is a Stop Loss a good idea?

In some cases, a backtest may show a strategy perform better without a stop loss. This is much more common in mean reverting strategies as price tends to move farther away from the mean, the edge or advantage of entering (or holding) the trade actually may increase! A stop loss in this scenario could limit upside and increase what you pay in exchange commission and broker fees.

Here is a popular 2-period RSI strategy tested across common market ETFs SPY, DIA, QQQ, and IWM. The strategy buys when 2-RSI crosses above 20 and sells when 2-RSI crosses below 80.

SPY 2-period Relative Strength Index RSI

The equity curve on the left shows using a fixed stop loss of 1% or $100 per trade assuming $10,000 per position. The equity curve on the right shows the same strategy with no stop logic! Neither strategy is great as a standalone but the version without a stop produced 2x the net profit in backtests.

Comparison of 2-period RSI trading strategy with and without a stop loss

On the other hand, trend following, or convex strategies almost always benefit from having a short leash for when a stock falls.

Position Sizing based on Stop Price

The late Dr. Van Tharp wrote extensively about calculating position size based on the distance between your entry point and your stop loss. Here is his complete guide: Van Tharp Position Sizing

In short, the trader determines what percentage of his account is acceptable to risk. Typically, this is set to 2%. Let’s assume Cody has $100,000 account so he is willing to risk 2% of $100,000 or $2,000 per trade. Cody has determined his stop loss order to sell be placed at $102.00 or $2.00 below his $104.00 price entry or the prevailing market price.

We can then calculate how many shares Cody should purchase by dividing the $2.00 distance by the $2,000 max risk allowance. This results in Cody buying 1,000 shares of AAPL.

This sizing method can really up the position sizing and traders should understand the risks that you may not get executed at a price level equal to the stop price level or $102.00 which can result in substantial or significant losses. Reminder about thinly traded markets and overnight gaps. Trading liquid futures contracts can help alleviate some of this concern.

Build Alpha algo trading software supports this method with its “Volatility” based position sizing method. This is method is both dynamic and sizes based on the stop loss distance.

What is the Best Stop Loss Strategy? Optimizing various Stop Losses on different trading strategies

Enough with the discussion, let’s look at some data. We can test some specific stop loss strategies and see which one performs the best using Build Alpha. First a quick overview of stop types.

  • Fixed Dollar Stop – risking the same dollar amount each trade regardless of the underlying price.
  • Fixed Percentage Stop – risking the same percentage each trade or risking the same dollar amount and using the same fixed dollar amount to size your position. Both achieve the same.
  • ATR based Stop – risking a multiple of ATR units away from the entry price. This is dynamic as the stop will widen or narrow as ATR fluctuates. For example, two ATR units below the long entry. If ATR is 0.44 then the stop would be 0.88 below the entry.
  • Fixed Dollar Trailing Stop – risking the same dollar amount each trade but using a trailing stop instead of a fixed dollar stop.
  • ATR based Trailing Stop – risking a multiple of ATR units but as a dynamic trailing stop.

For this test, we will choose three popular strategies, trade $10,000 per position, and backtest each strategy from 2006 to mid 2022. I will use the following stops in the test

  • $100 Stop Loss (1% of position)
  • $200 Stop Loss (2% of position)
  • 1 ATR Stop Loss
  • 2 ATR Stop Loss
  • $100 Trailing Stop
  • $200 Trailing Stop
  • 1 ATR Trailing Stop
  • 2 ATR Trailing Stop

The three automated trading strategies we will use are the following:

  1. The RSI20-80 strategy – Buy when the 2-period RSI crosses above 20 and sell when it falls below 80. Same strategy mentioned above.
  2. Turn Around Tuesday – buys stock index Tuesday after a down Monday which I first published here: Turnaround Tuesday: Wall Street Cliche Or Trading Edge? – See It Market
  3. Moving Average Crossover – we will buy when the 10-period SMA crosses the 50-period SMA. I will leave any moving average optimization up to the reader.

Here are the individual results sorted by strategy type

Testing various stop losses across three unique trading strategies

The $100 trailing stop was the best performer for both the RSI and Turnaround Tuesday strategy; however, it was one of the worst performers for the moving average crossover strategy!

This drives the point home that you must TEST EVERYTHING! There is no best and everything is relative to the strategy and symbols traded.

There are tons of other variations we could test such as combinations of stops, larger stops, smaller stops, etc. but that is the beauty of Build Alpha which allows you to quickly build and test strategies with no code. It is the first professional no code algo trading software available to all traders.

Build Alpha Dark Mode Color Option Display

Need to Know about Stop Loss Orders

  • Stop losses aim to protect traders from significant losses
  • A Stop Loss is a sell order executed at a specified price to limit risk
  • A Stop loss converts to a market order after the stop price is reached
  • Stop limit order converts to a limit order after the stop price is reached
  • Trailing stop adjusts every trade or every bar and trails the stop price in the favorable direction as the trade moves in the favorable direction and remains unchanged with unfavorable price movement
  • Dynamic stops adjust to market volatility. Static stops are simpler to calculate.
  • There is no best stop loss strategy. Each strategy requires individualized testing

Stop Loss Summary

Stop losses are a risk management tool in every trader’s toolbox. Stop losses and stop limit orders aim to limit risk and protect against significant losses. Trailing stops protect against initial losses and then improve to protect gains as the trade proceeds. Mean reversion strategies are often harmed by stop losses where trend following strategies are often improved. Finding the right stop loss vs stop limit and trailing stop strategy depends on the market, timeframe, and strategy type they are applied to. Having useful algo trading software like Build Alpha to run countless backtests and optimize price levels can help identify optimal stop loss strategies for any automated trading strategy.

Author

David Bergstrom Author Photo Trading Edge

David Bergstrom – the guy behind Build Alpha. I have spent a decade-plus in the professional trading world working as a market maker and quantitative strategy developer at a high frequency trading firm with a Chicago Mercantile Exchange (CME) seat, consulting for Hedge Funds, Commodity Trading Advisors (CTAs), Family Offices and Registered Investment Advisors (RIAs). I am a self-taught programmer utilizing C++, C# and python with a statistics background specializing in data science, machine learning and trading strategy development. I have been featured on Chatwithtraders.com, Bettersystemtrader.com, Desiretotrade.com, Quantocracy, Traderlife.com, Seeitmarket.com, Benzinga, TradeStation, NinjaTrader and more. Most of my experience has led me to a series of repeatable processes to find, create, test and implement algorithmic trading ideas in a robust manner. Build Alpha is the culmination of this process from start to finish. Please reach out to me directly at any time.

Out of Sample Testing for Robust Algorithmic Trading Strategies

Why Do Automated Trading Strategies Fail?

In my decade-plus of professional trading experience, research effort, and working with thousands of individual traders, consulting for hedge funds, registered investment advisors, Commodity Trading Advisors, and family offices, I have discovered that algorithmic trading strategies fail for two primary reasons or errors and out of sample testing can help.

  • Algo strategies are often built for one market environment
  • Algo strategies are overfit to the historical data (overoptimized)

In this post, we will take a deep dive into how out of sample testing (OOS) can be an excellent approach to combating both of these algo trading pitfalls, remove some uncertainty, increase knowledge, and help estimate better forecasts.

What is Overfitting? What is Curve Fitting?

Curve-fitting, or more commonly referred to as overfitting, is creating a model that too “perfectly” fits your sample data and will not generalize well on new unseen data. In trading, this is a trading strategy that trades the historical data too well and will fail to adapt to new live data (large forecast error). Not the answer we need.

That is, the strategy has memorized the historical data values finding random, spurious patterns that will fail to repeat in new data. Developing your own trading idea, price forecasting, data mining or using machine learning models can all lead to overfitting.

Here are two visuals I found to help illustrate this idea of curve-fitting.

Out of Sample and Overfitting
Out of Sample and Overfitting values

A curve-fit trading system is a trader’s worst nightmare; that is, a trading system that looks great while testing but fails miserably once live. The trading system simply memorized the historical data’s noise and will struggle to predict the noise of new market data.

Curve Fit Forecast Fails in Live

It is very unlikely the next set of (live) data points will make a milk saucer for the cat in the above photo. Drawing the cat is overfitting the data and poor analysis. It is seeing something that is not really there. A trading strategy that sees price anomalies that are not there will struggle in live trading. OOS testing can help detect overfitting and discover robust trading strategies.

What is a Robust Trading Strategy?

A robust trading strategy is one that is adaptable to changing markets and data conditions. A fragile or overfit trading strategy is one that needs the exact data from the historical period to continue performing as expected which we know the market will not give us.

It is very unlikely that the fragile trading system will help us reach our trading goals. All algorithmic traders strive for robust trading strategies; that is, automated trading systems that have a high probability of performing as expected no matter what the market does.

A Robust System forecasts well through 2008, 2011, 2015, 2020, etc.

Finding robust trading strategies is my passion. I took the time to write an in-depth guide on professional stress tests and checks (including OOS) that can help aid in this pursuit here: Robustness Testing Strategy Guide. If you are interested in learning how to better test trading algorithms, then I highly recommend giving it a read next. I am sharing organizational knowledge to be frank.

What is Out of Sample Testing?

Out of sample testing method is splitting the historical time series data into two or more data partitions with the idea of withholding some of the historical data to act as a second, unseen test set. The OOS period acts as an estimation period for live trading.

In trading, the strategy developer would split the historical price data into two sections: the in-sample data and the out-of-sample data. He would then build his trading strategy model on the first section of the data, the in-sample data. This is called model fitting or the learning phase. Once pleased with the strategy after making tweaks, optimizations, adding or deleting rules and filters, doing analysis, he is ready to test the strategy on the second section of data, the out-of-sample data.

Build Alpha highlights the OOS performance to easily distinguish

The OOS data acts as unseen, untouched data that provides an unbiased view of how the strategy may perform on unseen data. This is a close simulation to new live data as an algo trader can hope.

If the strategy was overfit to the in-sample data, then we could detect such overfitting or curve-fitting on the out-of-sample data by noticing a degradation of performance on the out-of-sample test. That is, if we expect an average profit of $200 per trade from the in-sample data but the out-of-sample test shows a negative -$100 loss per trade then we have overfit to the historical data and the strategy is not robust. A large difference between in-sample and out-of-sample performance may indicate an error or bad assumptions.

This blog will cover the details and methods of OOS testing and how it can help combat against overfitting in algorithmic trading strategy development. On the other hand, if you prefer, please take a look at this video version: Out of Sample Testing for Algorithmic Trading.

In Sample Testing vs. Out of Sample Testing

The in-sample data is the portion of data used to develop the initial strategy, run backtests, optimize parameters, make tweaks, add filters, delete rules, etc. The strategy should arrive in its final form using only the in-sample data.

The out-of-sample data is withheld and unused during the strategy development process. After the strategy model is completed, the trader can test the strategy on the OOS data points. It is often said that if the trading model performs similarly in both the in and out of sample period then we can have increased confidence the trading model generalizes well enough to new data points. That is, we have a good estimation or forecast of what to expect live.

Proper Automated trading software should split the test results for you to reference the In-sample, the out-of-sample, and the combined results of a trading model like Build Alpha does.

In and Out Split Forecasting Software

How Much Out of Sample Data to use?

The default setting most algorithmic traders will use is 70% in-sample period and the last 30% of the historical data reserved for out of sample testing. That is, if the trader has 10 years of historical data, then the first seven years will be used to develop the trading strategy and the remaining three years will be used for OOS.

Other common approaches to in sample vs out of sample splits are to divide the data directly in half with a 50/50 split. The first 50% of the data used for in-sample and the second 50% used for OOS.

Various In and OOS Test Splits

However, the test period, OOS location, and the percentage of OOS chosen can be very critical to the trading strategy’s success. I have always heard that good science is often mostly attributable to good experimental design. In the trader’s case, good science would be setting up a proper test by choosing an appropriate test period, OOS location, and OOS percent.

Out of Sample Test Period Selection – How the Human Adds Value

Below is the S&P 500 from 2004 to 2017 with the last 40% of data designated to be OOS (highlighted in red).

Validation Set

We create a trading strategy on the data from 2004 to 2011 – the blue in sample period. However, 2012 to 2017 (red OOS) was largely straight up! If we build a long strategy that avoids most of 2008 via some filter then the strategy may do well on our OOS data, simply because of the bull market.

Did the strategy pass OOS testing or would any strategy have passed this period? The question shows research effort. You can see the importance of intelligently selecting your OOS test period’s location and size.

Let’s use the first 40% of data as OOS. In this case, it allows us to build our strategy on the most recent data (the last 60%) from 2009 to 2017.

Validation Set 2 answer copy link

Many prefer to build their models on the recent data as it is the most like the live data they will soon experience. They then test out of sample using older data and in our case 2004 to 2008 (the first 40% highlighted red above).

Why 40%? I selected a percentage that would capture the financial crisis. If we build a trading strategy from 2009 to 2017 and then test from 2004 to 2008 and it performs similarly in both periods, then we likely have uncovered persistent edge that generalizes over two unique sets of data.

Selecting out of sample location and percentage is mission critical to better forecasting. Design your test to be as difficult as possible to pass – try to break your system in the testing process. If you do not, then the market will surely break it once you go live!

Out of Sample Selection Improves Trading Strategy Robustness

Testing design and set up is undoubtedly where the human still adds value to the automated trading process. Build Alpha allows users to leverage computational power in system design, validation, and testing; however, the test set-up in BA is still an area where a smarter, more thoughtful trader can capture an edge over his competitors while adding robustness to the output.

Below I have some photos of some terrible experiment design to help drive the point home. Both present fairly simple OOS tests to “pass” and potentially increase the trader’s risk.

Poor OOS single location leads to poor predictions
Poor OOS single location leads to poor predictions 2

The main takeaway is the human can still add value to the automated trading process by proper backtest and experiment design. That is why BuildAlpha software allows the trader/money manager to adjust everything (or nothing) from OOS percent, OOS location, test periods, the minimum number of trades in-sample, and the minimum number of trades OOS.

Randomized Out of Sample Testing – Avoid Luck

Traders are lazy and may not put in the work to find the exact percentage of out of sample data to make their test as difficult as possible to pass. An alternative method is to avoid using a single location for OOS and rather use a randomized selection. Build Alpha software can automatically and randomly select the out of sample period to avoid this common pitfall. To read more about this approach please check out this blog: Randomized Out of Sample Testing – Robust Trading Strategies

What are the benefits and pitfalls of Out of Sample Testing?

To explain simply, the main benefits of out of sample testing for algorithmic trading strategy development is a first line of defense against curve fitting and better forecasting performance. That is, out of sample testing helps discard obvious curve fit systems that fail to perform well on unseen historical data. This can save a trader who takes backtest research and immediately goes live with hard earned money only to see the strategy fail immediately. We have all been there.

The pitfall, and an often-missed error, is using an overly optimistic period of data that is too easy to pass. For example, any long strategy should do well in an out of sample period that goes straight up. Being aware of this pitfall, intelligently designing your in and out periods, or using the Randomized Out of Sample test are all ways to combat this and improve your trading research.

Key Takeaways

  1. Overfitting first line of defense

    out of sample testing can help avoid the avoidable by estimating strategy performance on unseen data. OOS testing simulates live trading.

  2. Human can add value

    merit to automated trading process by selecting the proper location and size of out of sample data can help create more robust trading models and reliable statistics. Attempt to break your strategies before the market does by designing hard to pass backtests.

  3. OOS is rarely enough alone

    out of sample testing is a great first step but is not a be all end all. Check out the full Robustness Tests Trading Guide to see the next tests for creating robust trading systems and more accurate expectations.

Need to Know

  • Out of sample (OOS) testing splits the historical data prior to a backtest.
  • Develop trading systems on the in-sample data first then test the signal OOS
  • OOS can be any percentage of the historical data
  • OOS can be at the beginning, middle or end of the historical data
  • Randomized out of sample selection is a great test to combat strong trending out of sample data and compare model performance across various market conditions
  • Algos that pass OOS are a great sign but by no means a complete green light to start live-trading

Out of Sample Testing Summary

Out of sample testing is a research effort and first line of defense to discover overfit and curve fit trading strategies that are destined to fail in live trading. Out of sample testing does not guarantee trading success but can surely help avoid the avoidable. Splitting historical data into a training and testing period allows the trader to properly design, estimate, tweak and optimize a trading system on the in-sample data before doing a final test on the withheld and unseen OOS data. Out of sample testing is the most popular and common stress test for any algorithmic trading strategy. Put this test in your trading toolbox immediately.

Thank you for the professional user contributions “licensed” and to those that share knowledge. That is what makes the Build Alpha community great. I hope you enjoyed this Out of Sample Testing Guide.

Author

David Bergstrom Author Photo Trading Edge

David Bergstrom – the guy behind Build Alpha. I have spent a decade-plus in the professional trading world working as a market maker and quantitative strategy developer at a high frequency trading firm with a Chicago Mercantile Exchange (CME) seat, consulting for Hedge Funds, Commodity Trading Advisors (CTAs), Family Offices and Registered Investment Advisors (RIAs). I am a self-taught programmer utilizing C++, C# and python with a statistics background specializing in data science, machine learning and trading strategy development. I have been featured on Chatwithtraders.com, Bettersystemtrader.com, Desiretotrade.com, Quantocracy, Traderlife.com, Seeitmarket.com, Benzinga, TradeStation, NinjaTrader and more. Most of my experience has led me to a series of repeatable processes to find, create, test and implement algorithmic trading ideas in a robust manner. Build Alpha is the culmination of this process from start to finish. Please reach out to me directly at any time.

Algorithmic Trading
Complete Guide for Beginner Algo Traders

What is Algorithmic Trading?

Algorithmic trading is the act of placing buy and sell orders through a computer. The trading strategy rules can be defined and given to a computer to execute whenever the entry and exit conditions are true. Algorithmic trading can also be referred to as algo trading, automated trading, systematic trading, or mechanical trading.

Algorithmic trading, or algo trading, is the fastest growing trading style as reports already show

  • 60-73% of all U.S. equity trading was done via algorithmic trading in 2018
  • The algorithmic trading market is growing at a CAGR of 11.23% between 2021-2026.

Read more key algo trading stats here.

Algorithm Trading Market Growth

Algo trading can be applied to any market, timeframe or holding period. Many traders incorrectly assume algorithms only apply to day traders or high frequency traders. All high frequency traders utilize algorithms, but swing traders and longer-term investors may also take advantage of automated trading.

In this complete guide, I will use my decade plus of professional algorithmic trading experience and years of developing Build Alpha to walk you through the

  • basic principles
  • simple trading frameworks
  • sample strategy types
  • portfolio construction

This guide is for traders looking to get into algorithmic trading as well as those well versed. 

Can you make money with algorithmic trading systems?

The most common question I receive is, “can you make money with algorithmic trading”? First, the leading 12 investment banks earned about $2 billion from the portfolio and algorithmic trading in 2020, according to Coalition Greenwich. Check out the key algo trading stats link above.

Second, the world’s best hedge fund, Renaissance Technologies, is a large proponent of algorithmic trading systems and automated trading systems generating 66.1% average annual return since 1988.

James Simons (full length interview) – Numberphile – YouTube

Furthermore, here is a Registered Investment Advisor client with over $100 million in AUM who shared returns of a few of his clients.

Algorithm Trader Success Statement 1

Algorithm Trader Success Statement 2

Also, plenty of individual traders can do quite well. There are no limits or restrictions for account sizes. Here are some statements from Build Alpha traders using algorithmic trading strategies with all different account sizes.

Algorithm Trading TradeStation Account

Schwab Account Statement

Build Alpha User Account Statement 3

Build Alpha User Account Statement 4

Build Alpha User Account Statement 5

Here is a simple strategy that was given away many years ago that continues to do well. This strategy may be too simple as a standalone, but a great idea of what building blocks are possible with algo trading.

Free Friday Trading System 20

Free Friday Trading System 20 Update

It is important to note that all trading, automated or not, involves risk. Additionally, trading returns are often a function of the risk taken. Finally, markets change, and strategies should be monitored on-going. Please respect risk and compete against yourself, not against another trader with a different bankroll. There will be more on risk, monitoring, and position sizing below. 

What are the benefits of algorithmic trading?

  • More markets A trader can only watch so many charts at a time. Algorithmic traders can cover more markets. More opportunity means more edge. More uncorrelated edge should mean more profit; more on this later.

  • Smarter risk-taking having rules can help avoid taking unnecessary risk when the market data does not support risk taking. If the data supported it, then you would probably have an algo trading system already.

  • The computer never sleeps fatigue, burnout, hangovers, etc. all play into a manual trader’s profit and loss and buy and sell orders. We all have times where we could have used a couple Z’s. Computers don’t.

Tired Computer Trader

  • Fewer mistakes a wrong hot key, buy instead of sell, fat finger an extra zero, a wrong symbol. It is rare, but trading is hard enough as it is. Avoid these mistakes by using automated trading.

  • Quantified edge trading rules can be backtested and quantified. How much can I expect to make in the next N trades? How much can I expect to lose in the next N trades? Quantifying leads to smarter trades.

  • Less emotions angry at the reality of losing hard-earned money? The next trade after a difficult trade can make or break your month if you are not careful. Automated trading avoids trading on tilt and revenge moments. Human traders can benefit.

Emotional Trader

Relaxed Trader

  • Rule based [left brain] Having clear trading rules, expectations for profit and loss, and treating trading as a quantifiable endeavor are intensely satisfying for left-brains as this is their modus operandi elsewhere.

  • Creative [right brain] traders can get lost (in a creative way) in the process of developing an algo portfolio. Developing a new algo can be as, or more, gratifying than making a profitable trade due to the creativeness the development process evokes.

Can you automate stock trading? Can you automate ETF trading?

Stocks and ETFs are the most popular markets and can certainly be automated. The sheer number of symbols and unique companies presents seemingly endless opportunities for algorithmic traders to capitalize on. The 6.5-hour trading day from 9:30 ET to 16:00 ET are perfect for most traders but even better for algorithm traders. Automated stock traders can review pre and post market results and contain their workday within normal business hours.

Gap Risk Trading Stocks

Stocks and ETFs do present larger gap risks as unscheduled or surprising news can be released during hours where markets are closed. Since computers never sleep some automated traders feel no need to subject themselves to these risks and explore other markets.

Can you automate Futures Trading?

Futures markets are arguably the favorite market for automated traders. Futures such as equity indexes, gold, oil, and other commodities like corn, wheat, and lean hogs all trade much more frequently than stocks and ETFs; most futures contracts trade nearly 23 hours a day, 5 days a week. This extended trading schedule certainly lends itself to more algorithmic traders and the need to automate.

To learn more about futures read my Brief Guide to Futures I wrote a few years ago or check out futures contract specs on the exchange websites. Here are the specs for the S&P500 emini futures contract.  

SP500 e-mini futures contract specifications

Additionally, Futures follow the 60/40 tax rule which taxes the first 60% of gains as long-term capital gains and the remaining 40% at ordinary income rates.  Remember the lowest two ordinary brackets for long-term capital gains are 0% so futures provide excellent entry points for small account traders looking to compound their gains while minimizing the tax hit. To read more check out the Section 1256 contract rules.

Can you automate Forex Trading?

Foreign Exchange (Forex) is trading one country’s currency for another and trades 24 hours per day, 5 days per week. This around the clock schedule is perfect for automated trading. Forex is often quoted or displayed as six letter symbols such as EURUSD where if the trader were to buy EURUSD pair he would be buying euros (EUR) and selling dollars (USD). Forex does not have a central exchange the way that stocks and futures do. That means, the forex execution price is often left up to the forex trader’s broker dealer (more on brokers later). It is important to note that futures contract equivalents of most of the major forex pairs do exist.  

Can you automate Crypto Trading?

Cryptos trade 24 hours per day, seven days per week, and 365 days per year. Talk about the need for algorithmic trading! For this reason, algorithm trading is very popular among crypto traders.

For those that are unfamiliar, crypto Currencies are digital or virtual currency that uses cryptography to secure transactions. At the time of writing this, the largest market capitalization coins are Bitcoin and Ethereum. Most of crypto currencies are built upon blockchains or online digital ledgers that track all the transactions of a cryptocurrency in public purview. To search for the largest cryptocurrencies, I would recommend using Coin Market Cap.

Coin Market Cap Best Cryptocurrencies to automate

What kind of Data is needed?

Market data is the lifeblood of algo traders and comes in many forms. Regardless of the market, the most common form of data are time-based bars. Each bar contains a time interval’s Open, High, Low, and Closing price. These bars are often referred to as OHLC bars.

Bars come in all timeframes such as 5-minute, 15-minute or 60-minute bars. 15-minute bars simply means that one bar occurs every 15 minutes and at the end of 15 minutes another bar begins. Here is what a sample data file and OHLC chart look like:

OHLC Data File Market Data Feeds

OHLC Chart Market Data Feeds

There are many great free sources of daily financial data for all financial instruments. Here are a few free S&P500 daily data sources:

Side note: Build Alpha does come with a full database of all markets, 1 minute up to daily, weekly, monthly for all symbols.

What Type of Programming Language is best for algorithmic trading?

  • Excel is expanding by creating functionality to retrieve live stock prices in Excel.  Excel was my starting point into quantitative trading and allowed me to view price data, calculate indicators, and build some rudimentary backtests. For longer-term trading, excel is a suitable solution. However, you will quickly outgrow excel when you need faster testing.

Excel algorithmic trading system

  • Python is the fastest growing and most popular programming language. First, python code simpler to read and learn than any other language as python is known for its simple syntax and easy-to-read code. Second, many public code libraries are posted all over the internet. Odds are someone already built a python library for your idea. Don’t reinvent the wheel – just use the existing python library. Third, it is easy for data science and machine learning which many professional quantitative traders utilize daily.

Python Logo

  • Python’s drawback is speed. If you want to search large intraday datasets, then python is rarely the best choice. If you are interested in seeing a concrete example of building a trading signal in Excel and Python then please check out: How to Build an Algo in Excel, Python, Build Alpha

Python Algorithmic Trading Example

  • C++ is the fastest, most robust coding language. It is widely considered the best choice for competitive programming by 75% of programmers, according to Geeks For Geeks: Why Cpp Is Best. However, the added benefits of C++ come from the steep learning curve and complex syntax used. My market maker, high frequency trading mentor started me with C++ but there are definitely easier starting points in hindsight.

  • Most will not need the speed and reliability of C++ and it is often best reserved for professional quants, high frequency trading, and those looking to quickly test trading ideas across tons of data.

  • No Code – If you do not want to program, then keep reading to learn about Build Alpha – no code algo trading software.

What is a trading broker?

A broker is an intermediary between those who want to trade and the exchange. You need a broker because exchanges require those who execute trades on the exchange to be licensed. Brokers typically charge a small fee, referred to as a commission, for their service. However, many brokers have begun offering $0 commissions.

$0 commissions for algo brokers

What is the best trading broker for algorithmic trading?

The brokers I recommend for algorithmic trading are TradeStation and NinjaTrader. Let me breakdown the pros and cons as well as some feedback from the Build Alpha community.

TradeStation

is an award-winning broker well known for their trading platform and proprietary coding language Easy Language. Easy Language is a simplified language created to help traders code. Traders can create strategies with much less effort than a traditional coding language. TradeStation also has beautiful charts and a relatively fast platform.

If you are interested in opening a TradeStation account, please contact me and I can put you in touch with the Build Alpha rep. TradeStation is assisting new traders get started with Build Alpha. Ask me how.

TradeStation Logo

NinjaTrader

is a fast-growing broker that also has their own coding language called NinjaScript. NinjaScript is a simplified version of C# but is a bit more complex than TradeStation’s Easy Language. The added complexity does come with additional benefits and flexibility albeit with a steeper learning curve. NinjaTrader also has a great ecosystem of third-party vendors who release indicators and other trading tools.

Ninjatrader logo

Both TradeStation and NinjaTrader allow you code strategies in their proprietary coding languages and automate them directly inside their trading platforms. I know, what if you can’t code? Keep reading!

How to automate my trading system? Coding to Broker or no coding to broker.

You can code your strategy in Python or C++ (or any other language) and connect to a broker’s API or application programming interface. An API is essentially a public library that explains how your code can communicate with someone else’s code.

Both options above are for programmers and those traders with the appetite to learn programming. Coding strategies yourself can lead to errors and has a steep learning curve. Certainly, a measure twice to cut once endeavor.

But what if you do not want to write code? You need strategy testing software that builds algorithmic strategies and generates code for you. Does that exist? Yes. Automate your trading with no coding.

What is the Best Algorithmic Trading Software?

Build Alpha Strategy Builder

Build Alpha is built after my own hardships learning to code and the need to test an endless stream of trading ideas in financial markets.

Instead of coding every single trading idea, I thought I would build software that could test all my ideas at once with no coding.

Build Alpha allows the trader a point-and-click interface to select

  • symbols
  • date range
  • entries and exits
  • stops and targets
  • filters and regimes
  • add your own custom signals

and then passes all inputs to a genetic algorithm which will find the best algo strategies for you.

For traders that do not have time to learn to code or want to test their trading ideas faster than coding can enable, then Build Alpha is the answer. Check out the Build Alpha homepage or request a demo.

What are the types of Algorithmic Trading Strategies?

Different trading rules will create different trading system styles and characteristics. Let’s breakdown the two most popular trading strategy frameworks and their associated characteristics.

  • Trend Trading is a trading strategy that attempts to capture large directional moves or continuations in price trends. Trend trading will hold the trade until momentum shifts and is most popular among the swing trading and longer-term trading accounts. The simplest form of trend trading is buying when price crosses above a simple moving average and selling when price falls below a simple moving average. Trend Trading is often characterized by frequent small losses and infrequent large wins. The large infrequent wins often make up for the small infrequent losses.

Trend Trading Trading Algorithm Example

  • A quick algorithmic trading history lesson: The Turtle Traders were 23 novice traders who became literal millionaires overnight in the 1970s due to a trend following strategy. Two professional traders and industry experts shared their strategy with these novices to test the novice traders’ discipline. It is an amazing story, and the strategy is well known now after being detailed in books. The strategy is not quite as successful over the most recent decades, however.

  • Mean Reversion, often called countertrend, trading looks to capitalize on extreme moves in price action assuming price will revert to its average price. Mean reversion can be thought of as the opposite of trend following. Mean reversion can be simply described as a deviation away from a simple moving average or previous average price with the assumption price will return to its average price. As price moves away from the average, traders would look to buy (or sell) shares until price returns to the average. Countertrend trading is often characterized by small, frequent wins and large, infrequent losses.

Mean Reversion Trading System Example

Are there different types of Algorithm trading?

There are many different types of trading algorithms and many will be left out of this guide to keep this comprehensible. There are a handful of other common algo trading styles worth noting.

  • Seasonality According to Investopedia.com, seasonality is a characteristic of a time series in which the data experiences regular and predictable changes that recur every calendar year. A simple example is retail stocks catching a bid before holiday sales numbers or energy stocks selling off after a mild winter. Seasonal trading is the most basic kind of trading rules we can define because it is as simple as buy on this day of the year and sell on this day of the year.

Seasonality Trading Tool for Financial Markets

  • Breakout Trading is a subset of trend trading. A breakout occurs when price breaks above a significant past price level. Many technical analysts and chart pattern traders enter on breakouts. Traders must be sure to quantify the set up and breakout to truly understand if the breakout is a repeatable edge and works across symbols. Most breakout traders look to exit quickly instead of riding the potential ensuing trend.

Breakout Trading Trading Range Strategy

  • Price Action Patterns This style is best described in Jack Schwager’s book Hedge Fund Market Wizards,

  • “This category of systematic approaches whose signals do not seek to profit from either continuations or reversals of trends. These types of systems are designed to identify patterns that suggest a greater probability for either higher or lower prices over the near term.

  • Simple examples would be if today’s close is below yesterday’s low or if today’s high is above the previous two day’s highs. These price patterns can often be combined to gain a sense of where the market is likely to go next.

Price Action Technical Analysis

There are various other types of trading styles. Here are a few more (not an exhaustive list). However, these are not great starting places.

  • Pairs trading
  • Index fund rebalancing
  • Statistical Arbitrage
  • Market Making
  • News or Event Driven

How to get started with algorithmic trading? What signals are best for automated trading systems?

  • Price Action is comparisons of open, high, low, and close data. The close is higher than the close two bars ago or today opened above yesterday’s high. Candlestick patterns are variations of price action; however, they often have misleading names that do not match the data as I pointed out here: Trading Truths.

Bearish Engulfing Technical Analysis

  • Fundamentals Is the company growing revenue? Are margins expanding? Is the company reinvesting? Quantifying fundamentals is great for institutional investors but rarely used for short-term trading. Public companies are only required to give updates once per quarter.

  • Technical Analysis Indicators like volume weighted average price (VWAP) and Stochastics provide price summarizations. I have done many tests with technical analysis, and there is no holy grail. One technical indicator may provide value on a certain symbol or timeframe but may show completely different results on another. Here is a quick video I did on technical indicators: Build Alpha Auto Trading Software testing RSI and MACD Strategies.

  • Chart patterns are geometric shapes connecting price levels. Traders draw these lines under the assumption price consolidates before an explosive move. It is important to quantify and test chart patterns. I worked with a PhD in geometry to quantify chart patterns as signals in Build Alpha. Here are the most common chart patterns:

Chart Patterns Technical Indicators

  • Intermarket Perhaps you only want to trade stocks if gold is doing XYZ and avoid stocks if bonds are doing ABC. These inter-market relationships are extremely powerful and popularized by the late Murray Ruggiero. His book Cybernetic Trading Strategies is still one of my favorites.

  • Multi-timeframe Analyzing what a symbol is doing across timeframes may also lend key insights or confirmation to take a shorter-term trade. For example, enter a long position on the 30-minute if the daily and weekly charts are trending up.

Multi-timeframe electronic trading

Additional and Alternative Data to Give Context to Trading Algorithms

  • Economic Data How healthy is the economy? Is Gross Domestic Product expanding or contracting? Is employment strong or is a recession looming? Deflation or inflation? Any of these factors may influence if we are long or short or what markets we should trade.

  • Market Breadth can give insights into what is happening under the market’s hood. How healthy is the market? How many S&P500 stocks are trading above their moving average? Do we notice lots of buying activity on exchange or is most in dark pools? What percent of stocks are near all-time highs?

Market Breadth Example Market Conditions

  • Risk Factors watching the volatility index, VIX, can help gauge traders’ and market makers’ appetite for risk. If VIX is spiking, then perhaps trading styles should change. Maybe VIX above a certain level should activate hedging algos. Ignore the market’s risk readings at your own peril.

  • Options Flow Understanding options can provide an edge to your strategy development. Options metrics such as risk reversal pricing, gamma exposure, and the relationship between implied and realized volatility can add needed context to trading the underlying securities. How are market makers and dealers positioned? Do you want to fight the tide or ride the wave?

  • Alternative Data many professional traders and firms utilize alternative data with price data. This extra data can give their algorithms insights the price data cannot. Alternative data can include scrapping job postings to see who is expanding, tracking rail car weights and shipping cargo weights, monitoring satellite images over parking lots to gauge production levels, sentiment data scraped from twitter, etc. Creativity can often win the day. Build Alpha was designed to accept alternative data as I know how flexible traders need to be.

Backtesting Algo Trading Strategies and Automated Trading Systems

To create a strategy, we need to have historical price data and pre-determined trading rules or signals. We can apply any trading rules to the data to generate the historical trade results – this process is known as Backtesting. I wrote a full guide on backtesting here: Backtesting Trading Strategies.

Build Alpha can create any strategy type we have discussed and generate our list of trades with automatic backtesting. We can also view:

  • Equity curve – a profit and loss graph to show you how your account would have grown over time.
  • Performance metrics – such as winning percentage, total profit, drawdown, profit factor, and more. These metrics will be covered in detail later.
  • Ability to view trades on your chart – is an important aspect to make sure you understand what the strategy is doing.

What is the best strategy tester and backtesting software?

Build Alpha’s backtesting engine is extremely fast and unique as the code is written in C++ making it extremely fast and accurate. Accuracy is incredibly important when reviewing backtested results. I am extremely proud of this software as it helps me (and many other traders) quantify and view strategies in a matter of seconds.

There is no need to write any code to test your strategies as it is a completely code-free software. Simply select your entries, exits, risk management and hit Simulate. Build Alpha’s algorithm does the rest.

Build Alpha Input Interface Financial Markets

Algorithmic Trading Strategy Examples and Building Blocks

Losing money? Let’s backtest three example algorithmic trading strategies. All three focus on SPY, the S&P 500 ETF. Each strategy only contains one entry rule and is not optimized. I will leave optimization, adding filters, and testing on you the reader.

Price Action strategy example 1

  • If today’s low is below yesterday’s low then buy the next open and sell after one day. This strategy beats buy and hold from 2006 through 2022.

Trading Algorithms Example 1

Price Action strategy example 2

  • If the 2-period Relative Strength Index (RSI) is below 20 then buy the next open. Sell after the 2-period RSI closes above 80. This strategy has outpaced buy and hold while registering significantly less drawdown than the overall market.

Trading Algorithms Example 2

Price Action strategy example 3 

  • If the current bar closes in the bottom 20% of its daily range, then buy the next open. Sell after any day closes in the top 15% of its daily range. This strategy has also beat buy and hold over the testing period.

Trading Algorithms Example 3

These three strategies are not meant to be standalone strategies, but act as an example to demonstrate how price action signals can provide a tremendous starting point to trading system development. All these strategies were tested using Build Alpha’s built-in signal library. The next steps could be to add filters, indicators, alternative context, etc. but first let’s explore performance metrics.

Best Algorithmic Trading Performance Metrics

After running a backtest to see the hypothetical trades, we can view performance metrics to help us determine how good our algo is. Below are the most common metrics every quantitative and algorithm trader should know.

Computer Algorithms Trading Activities Performance Metrics

  • Profit and loss or P&L. This is simply the total profit or loss generated by the strategy
  • Drawdown – the amount the profit and loss fell from its highest amount. If the strategy was up $10,000 and then later was up only $8,000 then this is a $2,000 drawdown. Often simplified to the maximum drawdown as many drawdowns occur.
  • P&L / Drawdown is a metric comparing the total profit earned by the drawdown. The higher the ratio the better, generally speaking.
  • Win Percentage is the percentage of trades that returned more than $0 divided by the total count of all trades.
  • Ratio Win to Loss is a risk:reward metric comparing the average winning trade to the average losing trades. The higher the number the better.
  • Profit Factor compares the total gross dollars made on winning trades divided by the absolute value of total gross dollars lost on losing trades. This value is a ratio of two positive numbers so we cannot experience a profit factor less than 0. Additionally, any profit factor from 0 to 1 is not a profitable strategy with losses exceeding gains.
  • Sharpe Ratio developed by Nobel laureate William F. Sharpe is a metric showing the ratio of average return compared to the drawdown. More simply, the average annual return divided by the standard deviation of the return. There are arguments for Sortino Ratio which does not penalize “upside volatility” as Sharpe does, but this guide is not the place for a deeper dive.

The pitfalls of algorithmic trading strategies

I need to share the common pitfalls of strategy testing. Later in this guide I will link to a more exhaustive list of tests we can use to improve our probabilities of success, sidestepping these below landmines.

  • Poor data – missing data can materially impact your results. Assume you had missing data from 2007-2008. Not good.
  • Look ahead bias – mixing data sets such as intermarket data or multiple timeframes brings a possibility you incorporate unknowable future information. For example, we cannot use the day’s closing price to make a trade at 12 noon as the day’s close is not yet known.
  • Survivorship bias – many data providers only provide actively listed stocks. We need to include stocks that delisted, went bankrupt, merged, went private, etc. Without these symbols then our backtest may be more favorable than it should be.

Financial Industry and Stock Market scandals

  • Favorable conditions – maybe you tested in a bull market, and any strategy would have done well. Test on all market conditions and environments! I wrote about ways to combat this here: Randomized Out of Sample Testing
  • Real market environment – can be impossible to know the historical liquidity and some software will allow you to buy more shares than the total volume. This could not happen in real market environment without drastically affecting the share price.
  • Underfitting – an algo that is too simple will “underfit” the market’s past data. This means it will not capture any unique price anomalies in the future.
  • Overfitting –an algo that is too complex will “overfit” the data capturing noise and mistake it for unique price anomalies in the future. Over optimizing parameters or having too many rules are the most common culprits of overfitting. Overfitting is often referred to as curve fitting in finance. Curve-fit strategies fail as the market changes and trust me it changes.

Overfitting and Curvefitting quantitative analysis

Robustness Tests for Algorithmic Trading Strategies

A robust trading strategy can stand the test of time and changing market conditions. Quantitative trading firms employ a litany of statistical tests, stress tests, and robustness checks to attempt to find good fit and robust systems. In this section, I will introduce useful concepts from my professional trading experience and a decade-plus of primary research into the topic of trading strategy robustness.

Randomized Monte Carlo Robustness Test Algorithmic Trades

Robustness testing is a vast area, but if done well, can be a trading edge over other market participants because many fall short here. This is by no means an exhaustive list but can surely increase the probabilities of algorithmic success. 

To read more about robustness testing for algorithmic trading please check out my full guide: Robustness Testing for Algorithmic Trading Strategies

In sample and Out of Sample Testing for Algorithmic Trading Strategies

The first line of defense against overfitting is to split the historical data into two segments: the in-sample data and the out-of-sample data. The split can be placed anywhere but traditionally most traders use 70% of the data as in-sample and the last 30% of the data as out-of-sample.

To learn about intelligently splitting the data, please check out:

We build our algorithmic trading strategy on the in-sample data leaving the out-of-sample data untouched. Once happy with our strategy’s performance, we then test the strategy on the unseen, untouched out-of-sample data.

If the strategy performs well on the out-of-sample data, then we should have heightened confidence our strategy should do well on other unseen data.

Out of Sample Testing own trading algorithms

Additional Robustness Checks for Automated Trading System

Vs Random Is this a good strategy or was it pure luck? This test was discussed by Jaffray Woodriff of Quantitative Investment Management in the aforementioned Hedge Fund Market Wizards (Schwager, 2012).

The test creates the best possible random strategy to use as the benchmark to beat. The best possible random strategy is the best possible by chance (random). If we beat the best random strategy’s performance, then it is more likely real edge and not something lucky.

Build Alpha runs this every simulation. Below shows a real strategy vs. the best random strategies possible.

Vs Random Robustness Test Algorithmic Trading Platform

Vs Others – Does the strategy work on other markets? A SPY strategy that fails on other market ETFs such as QQQ, DIA, or IWM is likely overfit to the noise specific to the historical S&P500 data and will fail when noise changes. Strong performance across related markets is a strong sign of robustness.

Vs Others Robustness Test Algorithmic Trading Platform

Vs Shifted and Vs Noise – Markets like oil and corn have natural consumers whereas this is untrue in stocks. Testing across other markets may not make sense in these markets. The fix? Shift or add noise to the original data then re-trade the strategy on the shift or noise-adjusted data.

Vs Noise Robustenss Test Trading Algorithm

If the algo remains profitable on the adjusted data, then we get a confidence boost we did not overfit to the historical noise and the strategy can survive future noise variations.

If the noise test shows losing strategies, then we have overfit to the historical data. To read more about the noise test check these out:

These quick explanations skip nuances and certain tests were left out. Please read the comprehensive Robust Trading Strategy Guide to learn about other tests such as: variance testing, delayed testing, liquidity testing, and more.

Trading Algorithm Position Sizing. Can a computer size my trades?

There are a handful of popular position sizing methods used by algo traders. All these methods could be coded or generated from Build Alpha to apply to any strategy. In this section, let’s break down a few of the most popular sizing methods:

  • Fixed Size – taking the same size position every trade, i.e., 100 shares or maybe 1 futures contract or 1 forex lot. The algorithm would continue to buy and sell 100 shares regardless of price movement.
  • Fixed Dollar – the trader sets a dollar amount per trade and then the algorithm would buy that dollar amount of shares each trade. As the share price rises, the computer would purchase less shares. As the share price falls, the computer would purchase more shares. It is important to note that the share amount is calculated prior to each entry and unchanged while in a position.
  • Volatility Based – the computer will size the position based on the symbol’s volatility (typically Average True Range). As volatility rises, the computer will trade smaller positions. As volatility falls, the computer will trade larger positions. It is natural to reduce position sizes as risk (volatility) increases.
  • Risk Percentage – this method can only be used when the strategy uses a stop loss. The trader determines an amount he is willing to lose per trade. The computer will determine the distance between the entry and stop loss and then size the position so that the max loss is the amount the trader is willing to lose. This aggressive style does not account for when you cannot exit at your stop loss (gaps, illiquidity, missed orders, etc.). Some losses can exceed the amount a trader is willing to lose.

Mathematical Models for position sizing during falling market

Do I have enough capital for this automated trading system?

One of the largest and most common pitfalls traders make is not setting aside enough capital for their strategy algo. Many traders will look at the historical backtest’s maximum loss or the maximum drawdown and determine this as the amount of capital needed to trade the strategy.

Build Alpha offers a variety of Monte Carlo tests that can quickly aid traders in understanding how flawed the above sizing logic is. Let’s work from an example that had a worst trade maximum loss of $816.88 and a maximum drawdown of $981.38.

Risk Management Drawdown Algorithmic Trader

Running the Monte Carlo Analysis, we easily see that the average drawdown from the Monte Carlo variations was about $1,500 or 1.5x the original backtest’s drawdown. Not using the Monte Carlo Analysis tool could lead to undercapitalizing the strategy causing a premature plug pull and abandoning ship too soon.

Monte Carlo Resample Robustness Test Market Prices

However, there is a better approach to find the correct allocation to any trading strategy or portfolio, the Monte Carlo Drawdown method which was popularized in Howard Bandy’s book Quantitative Technical Analysis.

Funding an algorithmic trading system with Monte Carlo Drawdown Analysis

To run the Monte Carlo Drawdown analysis, the trader selects a percentage drawdown he could stomach and a starting trading account size. Most traders cannot withstand a 20% drawdown. 

Next, run 1,000+ Monte Carlo tests resampling from the historical trades to create 1,000+ new equity curves (can reshuffle the trade order as well). Then calculate the drawdown of each new equity curve and plot the 1,000+ drawdowns as a histogram. Finally, plot a cumulative distribution line on top of the histogram (in blue below). This line plots the total percentage of drawdowns as we move from left to right across the horizontal x-axis.

Monte Carlo Drawdown Analysis Sophisticated Algorithms

Tracking up the Y-axis to see where 95% intersects with the blue cumulative distribution line. This spot indicates 95% of all Monte Carlo drawdowns were less than this. Connecting this red X to the x-axis shows the drawdown of 30%.

We can say, “we are 95% confident drawdowns from this trading system will not exceed 30%”.

This value is unacceptable to most. We should strive for a drawdown less than 20%. Meaning, the starting trading account value is too small to trade this strategy and we could easily see a drawdown greater than our pre-determined 20%.

If we increase the account size and run the test again, we can see the 95% confidence spot (red X) is now at slightly less than a 20% drawdown on the x-axis.

Monte Carlo Drawdown Analysis Sophisticated Algorithms

Sizing strategies – and portfolios – properly is one of the easiest ways to ease the emotions of drawdowns. It also makes sticking to a system long enough to escape randomness more doable in turbulent markets.

Read more on this technique and see a strategy example please head here: Properly Funding Strategies with Monte Carlo Drawdowns

Identify Broken (failing) Algorithmic Trading Strategies

To define failing, we must first define passing behavior. Monte Carlo testing once again gives us a guideline. Important to note that other more advanced tests such as noise testing, Vs shifted, Monte Carlo permutation, and Variance testing could also apply. Monte Carlo results give us a cone of what is possible at trade number X.

Monte Carlo Broken Black Box Trading

In the graph above, we can see that after 100 trades the strategy’s P&L should be anywhere from $0 to $15,000. This is a large range but gives an idea of what our expectations should be at trade 100.

 If at trade 100 the P&L is between $0 and $3,000 then we certainly had some bad luck, but the strategy is not broken! The strategy is just in the lower range of what was possible (bad luck). On the other hand, if the strategy’s P&L at trade 100 is between $12,000 and $15,000 then we had tremendous luck. This success was possible despite not being a conservative estimate at trade 0.

 If we are at trade 100 and the P&L is less than $0 or greater than $15,000 then we are outside expectations. This is the first sign things are broken (not going as expected).

Build Alpha has a tool that generates Monte Carlo bands from the last trade and plots forward as many trades as desired. It is best practice to only trade strategies that have a flat or upward sloping lower band.

Monte Carlo Equity Bands Black Box Trading

If performance is falling outside the red Monte Carlo bands, then it is best to do at least one of the following:

  • Analyze what has changed in the market
  • Analyze what has changed in the strategy
  • Reduce your strategy’s position size
  • Take your strategy offline

To read more on Monte Carlo Simulation and applications to algorithmic trading please check out: Monte Carlo Simulation Complete Guide

Algorithmic trading success needs the portfolio approach

The perfect strategy or “holy grail” hunt is often the worst adventure one can embark upon. I lived this for many difficult years chasing chat rooms, magic indicators, and more to hopefully learn that the one missing piece. Turns out there is no one missing piece.

The holy grail is combining uncorrelated strategies to smooth returns. Ray Dalio of Bridgewater, the largest hedge fund in the world, shared a graphic in his book Principles showing the math behind the true holy grail: adding uncorrelated strategies together.

Holy Grail for Professional and Institutional Traders

On the right-hand side, the more uncorrelated strategies you combine the lower your probability of losing money. Lightbulb moment. I don’t need a perfect strategy; I need a bunch of good strategies working in concert.

Michael Jordan needed someone to do the dirty work: play aggressive defense, check defenders down low, grab rebounds, make hustle plays. He did not need someone else to score points. Your portfolio needs a team mentality. Some players are not great on their own (cough Rodman), but on a team, they fill a role making everyone exponentially better.  

That’s right, some strategies may not be profitable but still additive to your portfolio! Certain strategies do well in certain markets while other strategies do well in other market conditions (aka doing the dirty work the core strategies cannot).

Designing Systems for Multiple Market Conditions

In the graphic above, strategies A and B are the best standalone strategies as strategy C has a negative return! However, when combined (and scaled to the same volatility), A+C do much better than A+B. When A yins, C yangs. Conversely, A and B ebb and flow at the same time (two scorers). A+C provide an awesome example to the team, or portfolio, approach Dalio mentioned.

There is more to be gained understanding this than hunting the perfect strategy.

Key Takeaways

  1. Algorithmic trading volume is rapidly growing
  2. Algo trading is used by the largest banks and best hedge funds
  3. Algorithmic trading is possible for all asset classes: stocks, futures, mutual funds, forex and crypto
  4. TradeStation and NinjaTrader are the best data providers and brokers
  5. Python and C++ are popular programming languages for algorithmic trading
  6. Build Alpha software enables traders to build algo strategies without writing any code
  7. A portfolio of algos is better than hunting a single perfect “holy grail” strategy

 

Need to Know

  • Algo trading is when a computer executes pre-determined buy and sell rules for you
  • Algo trading does not require programming. Build Alpha is a no-code strategy builder
  • Hedge fund and professional software and testing is now available to all algo traders
  • Finding robust strategies requires additional testing after backtesting
  • Monte Carlo Drawdown technique can help size automated trading strategies correctly
  • Portfolio approach to algo trading is the “holy grail”

 

Summary of the Complete Algorithmic Trading Guide

Automated trading systems are the fastest growing portion of the market comprising nearly 75% of the daily volume according to the previously mentioned 2018 study. The 12 largest banks and top hedge funds, including the best hedge fund of all time Renaissance Technologies, all utilize algorithms and automated trading.

The data and tools to build your own algorithmic trading strategy have never been more accessible or even available to individual traders as they are today. Build Alpha is an all-in-one, start to finish algorithmic trading strategy development tool that enables traders to follow the steps laid out in this guide without having to write any code.

I am on a mission to develop Build Alpha as the best automated trading software. Please take the ten minutes to check out the demo and see how it can help any level of trader save time, reduce uncertainty, and hopefully gain an edge in algorithmic trading. If you have any questions, please email me anytime.  For more on Algorithmic Trading please check out FINRA the Financial Industry Regulatory Authority.

Author

David Bergstrom Author Photo Trading Edge

David Bergstrom – the guy behind Build Alpha. I have spent a decade-plus in the professional trading world working as a market maker and quantitative strategy developer at a high frequency trading firm with a Chicago Mercantile Exchange (CME) seat, consulting for Hedge Funds, Commodity Trading Advisors (CTAs), Family Offices and Registered Investment Advisors (RIAs). I am a self-taught programmer utilizing C++, C# and python with a statistics background specializing in data science, machine learning and trading strategy development. I have been featured on Chatwithtraders.com, Bettersystemtrader.com, Desiretotrade.com, Quantocracy, Traderlife.com, Seeitmarket.com, Benzinga, TradeStation, NinjaTrader and more. Most of my experience has led me to a series of repeatable processes to find, create, test and implement algorithmic trading ideas in a robust manner. Build Alpha is the culmination of this process from start to finish. Please reach out to me directly at any time.

Robustness Tests and Checks
for Algorithmic Trading Strategies

What is a Robust Trading Strategy?

A robust trading strategy can withstand the tests of time and evolving market conditions. A large key to successful algorithmic trading lies in gauging how likely a strategy is to fail in new market conditions. A trading strategy that can deal with market shocks, volatility, changes, and price swings is a robust strategy.  Robustness can be equated to reliability. Robustness Testing is key to identifying potential reliable trading systems.

What is a Robustness Test?

A stress test or statistical test to help validate or break an algorithmic trading strategy prior to risking real capital. Robustness tests help identify trading strategies that may fail in live trading prior to risking hard earned money. Avoiding bad trades and trading systems can be equally as important to reaching your trading goals as finding good strategies.

Trading losses make trading so much harder! Warren Buffet, the greatest investor to ever live, is quoted as saying, “the first rule of an investment is never lose money. And the second rule of an investment is never forget the first rule.

Robustness testing or checks are stress test methods for measuring reliability in systems. These methods have the ability to identify lying backtests, determine sufficient risk capital, and avoid catastrophic strategies before you potentially lose real capital.

Why are Robustness Checks important in Algorithmic Trading?

Too many algorithmic traders believe a backtest is enough to take a strategy live. Failing to check your strategy’s robustness can lead to substantial and quick losses when moving your strategy from the development stage to live trading.

A robustness check can help reduce the probabilities that a trader risks his hard-earned money on a lying backtest aka a strategy doomed to fail. Identifying strategies doomed to fail can help traders save capital.

Robustness checks allow the trader to make smarter decisions with his capital without being misled by lying backtests.

Additionally, the math behind avoiding losses is convincing

If you lose x% what gain is required to break even?

  • 10% loss needs to gain 11.11%
  • 20% loss needs to gain 25.00%
  • 30% loss needs to gain 42.86%
  • 40% loss needs to gain 66.67%
  • 50% loss needs to gain 100.00%
  • 75% loss needs to gain 300.00%
  • 90% loss needs to gain 900.00%

Warren Buffet is right. Losing money makes trading, and investing, much harder. If we have methods to help avoid losses, then what are they?

Do Robustness Tests and Checks guarantee Trading Success?

Nothing can guarantee trading or investing success. In this guide, I will walk through various robustness methods and checks that can help raise the probabilities we only risk capital on strategies with high chances of success. Nothing is a guarantee but shifting the odds in our favor is often good enough.

What are the best Robustness Tests for Algorithmic Trading Systems?

  • Out of Sample Testing
  • Randomized Out of Sample Testing
  • Vs. Random
  • Vs. Others
  • Vs. Shifted
  • Noise Testing
  • Monte Carlo Analysis
  • Monte Carlo Reshuffle
  • Monte Carlo Resample
  • Monte Carlo Permutation
  • Monte Carlo Randomized
  • Variance Testing
  • Delayed Teseting
  • Liquidity Testing
  • Walk Forward Analysis
  • Parameter Optimization / Parameter Stability Testing
  • Noise Testing Parameter Optimization

Each of these robustness techniques will be covered in detail in this guide. Feel free to skip ahead but I would not recommend it! First let’s cover some basics.

Why do my trading algorithm strategies fail?

Over the last ten years of professional trading experience and the last five plus years of interacting with the Build Alpha community, I have found there are two main reasons that trading strategies fail         

  1. Built for one market environment
  2. Overfit to the noise (overoptimized)

There are certainly other reasons, but let’s take a moment to break down these two as they are undoubtedly the most common.

First, most strategy developers fail to include context into their strategy. Is it a bull market? Are interest rates rising or falling? Is inflation increasing or decreasing? Is the economy doing well? The market and big money players need large signals to move large amounts of money.

Most individual algo traders design strategies around price action signals or technical indicators and fail to account for market context. When the market changes, these strategies inevitably suffer.

Examples of failing trading system when the market changes

Second, many strategy developers become obsessed with getting the best backtesting results possible. Traders will tweak strategy parameters until the backtest’s equity curve moves from the lower left to the upper right with zero turbulence. Ah, the perfect strategy!

Too good to be true equity curve

These unrealistic expectations cause traders to overfit the history data. Is there a reason the 37-period moving average should work best? What if the 36-period and 38-period result in bad results and negative net profit or profit factor?

The live data will ALWAYS be different than the history data, so any strategy that needs new data to be nearly identical to the historical will fail in real trading and cause significant risk.

If you are looking to build your first automated trading strategy, add context to an existing strategy or want to see a complete guide to building automated trading systems, please check out this: Algorithmic Trading Complete Guide

What is Overfitting and Over-optimization in strategy development?

Overfitting is creating a strategy that too perfectly fits the historical market data and will not generalize well on new unseen data. A trading strategy that memorizes the noise patterns of the historical data will fail when applied to new or unseen data.

An overfit strategy just learns the exact cadence of the historical market data but nothing useable going forward.

Overfitting example

It is very unlikely the data actually created this cat. It is even more unlikely the next set of plots will create a milk saucer. Bad trading advice.

Overfitting vs Good fit example

Many strategies have parameters that can be tweaked to improve the strategy’s historical performance. For example, a simple moving average system may buy when price moves above the 50-period moving average and sell when price falls below the 50-period moving average.

Traders will optimize this parameter setting of 50 to create the best possible backtest. Perhaps the best backtest results are generated when buying on a cross of the 63-period moving average and selling when price falls below the 12-period moving average.

Optimizing parameters in Build Alpha

Another example of overoptimizing would be to have very specific values such as a stop loss of $217.34 or using a signal such as 12-period RSI crosses above 66.19. These values may produce excellent historical winning trades but are surely meaningless as we go live, and we have unknown data.

This is particularly common in forex trading and strategy vendors.

Varying input parameters in Build Alpha optimization

Do not optimize every penny or hundredth to find the “holy grail”. It is the surest way to overfitting.

But how can we know if we have over optimized or found ideal strategy parameters? That is the need for Robustness testing or strategy robustness!

What are the best robustness tests for algo trading strategies?

There is no single best test and depending on the strategy’s characteristics other statistical tests may be more valuable to what you are trying to disprove than others. For example, a trend following strategy should be tested a different way than a mean reversion strategy. Let’s walk through some examples and circle back to identify a good strategy.

In Sample and Out of Sample Testing

The simplest test we can implement as a first line of defense against curve fitting is an in-sample and out-of-sample split. Most traders use the first 70% of their historical price data as the in-sample data and the last 30% as their out-of-sample data.

Out of sample testing by changing different starting bar

First, build a trading algorithm on the first 70% where you can tweak, optimize input parameters, add or delete rules, etc. Once pleased with the trading algo’s results, we can test on the unseen, untouched out-of-sample data.

If the strategy performs well on the out-of-sample data then it is a good sign that the strategy may hold up well on live, unseen data.

Strategies fail when live vs. strategies that do well: green line

There are many considerations with out of sample testing such as where to split the data, how much data is needed and more. To dive deeper, check out

Randomized Out of Sample Testing

What if the out-of-sample data was the beginning of a monster bull market? Is it possible our great out-of-sample results are lucky and we are at high risk? Any strategy could have done well during this out-of-sample period.

This is certainly a risk with out-of-sample testing, but we know how to combat this! If you do not wisely select your in and out of sample split, then you could utilize the Randomized Out of Sample test.

Ability to select random out of sample can improve net profit example one
Ability to select random out of sample can improve net profit example two

This test simply selects random bars and assigns them to either the in-sample or out-of-sample portion. Then we trade on our data and take any trades entered on in-sample dates to be the in-sample trades and any trades entered on randomly selected out-of-sample dates to be the out-of-sample trades.

To avoid any “luck” during this test, we can repeat 1,000+ times. Here is practical example Randomized Out of Sample Testing

Vs Random: using the best random trading strategy as a benchmark

One of my favorite tests, I even mentioned it in the Algorithmic Trading Guide linked above. This test comes from Jaffray Woodriff of Quantitative Investment Management and aims to answer the question, could we have done this well by luck?

The test generates the best possible random strategy by using random data with real signals, random signals with real data, a combination or mix of the two, etc. The goal for any trading strategy is to have better performance metrics than the best random strategy.

If our research can beat the best random strategy, then we can have confidence the strategy contains real edge and could not be produced by chance (random). 

Real strategy vs random strategy trades distribution

Vs Other: testing trading strategies on other markets

Ideally, we want to see trading systems do well across various related markets. A trading strategy on SPY should do well on other markets such as QQQ or DIA. If a strategy performs terribly on related markets, then it is likely the strategy was overfit to the noise in SPY’s historical market data.

Compare multiple markets for robustness

It is often unnecessary for a strategy to perform well on unrelated markets as market participants may vary drastically creating different data characteristics. Equities have a natural upward drift as companies exist to make money. Oil and corn have natural producers and consumers whereas stocks do not. Forex trading has no central exchange.

These differing dynamics bring different traders, hedgers, and investors to the respective markets. These different players leave different footprints thus possibly rendering a strategy that works in one area useless in another.

Of course, a strategy that does well across all markets – related or not – is ideal. However, these are often very hard to come by.

Vs Shifted: shifting the data to re-test trading strategies

Every charting and data provider uses hourly data from the top of the hour to the end of the hour. That is, 10:00 to 11:00, 11:00 to 12:00, and so on. However, what if we shift the opening and closing time by 1 minute? By 2 minutes, by 3 minutes, etc.? Maybe our shifted data now opens at 10:03 and closes at 11:03, for example.

Now we have additional data that has the same properties but isn’t identical to the historical market data we built our strategy upon. This data is good to evaluate if our strategy requires the exact historical data patterns for success or not.

Shifting the data works on any timeframe and may change bar patterns, high and lows, consecutive up or down streaks, indicator calculations, and more. A long strategy needing four consecutive red bars and a close in the bottom 20% would trigger a big win on the left but would not enter on the right.

Vs shifted data manipulation shows various close price for same day

Re-trading your strategy on shifted data can give insights on how overfit your strategy was to the exact patterns available in the historical market data. Past performance should maintain across shifted data sets. If performance degrades on shifted data, then it may be a warning sign your strategy is over optimized.

Vs shifted cumulative performance regardless of starting bar

The Noise Test for robust algorithmic strategies

Overfitting a strategy simply means fitting to the noise of the data and not to an underlying signal. The book The Signal and the Noise does a better job than I can differentiating the two. However, we want to find true signals not noise. Noise is random and unreliable.

How can check if we have overfit to the noise in historical market data? We can noise-adjust the historical data by adding or subtracting various amounts of noise (volatility) to the historical price bars. This will create a new open, high, low, close series for us to test our strategy on.

Noise Test Settings in Build Alpha
Build Alpha Noise Test Original
Build Alpha Noise Test Adjusted

A strategy waiting for four straight green closes to initiate a short entry would take a trade on the original data, but have the trade skipped in these two noise-adjusted variations at the circled bar. 

Repeating this noise test to generate 1,000 or more new noise-adjusted data sets is a good start. Then we can re-trade the same strategy 1,000 times and view how the same strategy would perform if the noise wasn’t exactly how it was in the historical data.

If the noise adjusted results are no longer profitable, then the strategy was fit to the historical noise. If the noise test remains profitable then we can have increased confidence our strategy was not overfit to the historical noise.

Build Alpha Noise Test Evaluate Results

To read more about the specifics of the noise test please check out a couple of these articles I wrote

Monte Carlo Simulation for Trading Strategy Development

The most popular robustness testing method. There are many variations and uses of Monte Carlo Simulations and Monte Carlo analysis so let’s break down a few below:

Reshuffle – reshuffle the order of historical trades and repeat 1,000 times to create 1,000 new equity curves. Since we are reshuffling the order, we end at the same spot, but the path changes, showing how our account could have fared if the trades did not play out in the sequence they did. You will notice a larger maximum drawdown than the backtest.

Monte Carlo Reshuffle in Build Alpha blue line is original

Resample – randomly select a historical trade until you reach your backtest’s total trade count. You can randomly select the same trade aka resampling with replacement. Repeat 1,000 times until you have 1,000 equity curves. Since we allow for the selection of the same trade, we do not end at the same spot. This test can provide more variation as the worst (or best) trade can be selected multiple times.  

Monte Carlo Resample in Build Alpha blue line is original

Advanced Monte Carlo Tests for Trading Algorithms

Randomized – We can randomize the entry and re-trade our strategy 1,000 times. On the other hand, we can re-trade our entries and randomize the exit. If your original strategy had no stop loss, then you should not randomly apply one. We should only randomly select and vary exit logic in the original strategy. If the Randomized Monte Carlo remains profitable, then it is likely we have true edge in our entry. 

Monte Carlo Randomized in Build Alpha

Permutation – reshuffling the log of inter and intrabar price changes then exponentiating to recreate new bars. This creates synthetic data containing most of the statistical properties of our original data while destroying most of the patterns. We can re-trade our strategy on 1,000 new synthetic data series to see if our strategy holds up. This technique was popularized by Timothy Masters in his latest book Permutation and Randomization Test for Trading System Development.

Variance Testing for Trading Strategy Validation and analysis

The variance test takes the historical trades and creates a probable distribution to resample. Next, the test selects random trades from this distribution until it reaches the backtest’s total trade count. It repeats this process 1,000 times.

The trader can vary a performance metric, so the test only accepts variations that perform below the original backtest’s metric minus the trader’s input. The trader can now view how varying performance metrics affects the strategy’s variance.

For example, assume the backtest had a win percentage of 61% and the trader set the test variation to 5%. The variance test would only display variations that had a win percentage of 56% or lower.

This test allows a trader to view how a strategy would perform if things do not go as the backtest suggests they should. News flash – things never go as well as the backtest suggests. Now, the trader can answer if he would be comfortable trading this strategy in this less rosy situation.

Variance Testing example in Build Alpha

In the above image, we have two Variance Test results. The left side shows the strategy with 0% variation accepting all strategies from the simulation. The right-side shows variance test results with 5% variance selected – meaning only accepting strategies with win percentages 5% below the original backtest’s win percentage.

If live trading were to go 5% worse than our backtest, our strategy is a mere toss-up. A 50-50 bet is never how we want to risk hard earned capital. We want strategies that can maintain performance with slightly less optimistic metrics than the backtest.

Delayed Testing

Delaying a strategy’s entry or exit can be a useful test to determine we have not overlooked more optimal entry or exit points but can also tell us how fragile our strategy may be. If the strategy cannot enter 1 or 2 bars later or exit 1 or 2 bars later, especially for intraday, then the strategy may be showing signs of overfitting.

Delayed Testing in Build Alpha

Often a strategy may do better with a delayed entry as a strategy becomes known others begin to front run the entry and then bail when it does not work immediately. Build Alpha permits the option to add delayed entries to any strategy from the settings menu.

Liquidity Testing

Large traders and institutions cannot execute their entire size on the open or the close. The liquidity test shows how a strategy would perform assuming the trader needed to enter from the close and until the next day’s close by taking an average of the two.

The liquidity test also tests the close and the next open and the close to the next midpoint. This test can also serve that if an entry or exit were to be missed does it make sense to put the trade on late.

Liquidity Testing in Build Alpha

This example strategy showed promise, but if you cannot get the exact fills and need to scale-in or out due to size then this strategy shows a significantly different drawdown in both 2008 and 2020 market shocks. This could mean substantial risk to your initial investment.

Walk forward analysis as a valid Robustness test?

Walk forward tests smaller in and out of sample periods across the entire period tested on a rolling basis. It then combines all the out of sample periods as your out of sample results.

Rolling Walk Forward Analysis

Walk forward testing, popularized by Bob Pardo, is a popular approach to robustness testing but it actually introduces a handful of new risks. The many downsides and pitfalls are discussed in the book Advances in Financial Machine Learning (Pardo, 2018).

Mainly, walk forward analysis only tests a single price path whereas other tests such as Noise testing, Vs Shifted, Variance testing or Monte Carlo Permutation test multiple price paths.

Additionally, due to test design, walk forward analysis splits the data many times creating an increase in data leakage. That is where data from an in-sample period is used during an out-of-sample period.

The simplest example is to imagine a trading strategy that uses a 200-period simple moving average. The first 199 bars after each data split will reference data from the previous in-sample period. More splits, more leakage.

There are many other pitfalls to walk forward despite its popularity. Remember, robustness testing is about finding what will provide a strong Sharpe ratio and trading performance in the probable future not what has done so on the known historical price path.

Robustness testing parameters through Sensitivity Analysis

Optimizing parameters often leads to more fragile strategies as the risks of overfitting often skyrocket. However, how can we find robust parameters? The simplest rule of thumb is to look for parameters areas that all show similar performance.

That is, if the best parameter setting is 20 then we should aim to see at least +/- 10% showing about as good of performance. In this case, we want to see values 18 to 22 produce similar backtest metrics.

If the parameter values next to the optimal parameter setting causes a large drop in performance, then the optimal parameter setting is too fragile and likely just overfit to the historical data. This trouble can be easily avoided by using 3D surface graphs.

3D Surface of Sensitivity analysis for parameter optimization in Build Alpha

Choosing a parameter value here is too risky as the performance nearby drops significantly. On the other hand, choosing a parameter value in the red section provides a greater perceived margin of safety as the neighboring values show similar performance.

Noise Test Parameter Optimization for trading strategies

Build Alpha takes parameter optimization one step farther equipping you with an additional robust check to the parameter selection process. Build Alpha enables the trader to add noise-adjusted samples to the optimization process so Build Alpha will find what parameter settings work best across all the noise-adjusted sample.

Build Alpha Noise Test Addition for Sensitivity Analysis

Combining parameter values in a stable region with good performing neighbors and the great performance across noise adjusted data sets then we are doing as much as we can to shift the probabilities in our favor.

The full article with a case study can be found here: Noise Test Parameter Optimization

Should my trading strategy pass every robustness test?

In a perfect world, yes. However, as algorithmic traders, we rarely live in a perfect world. Robustness testing is to help us avoid avoidable catastrophe but is not a crystal ball, unfortunately.

We can never prevent overfitting or truly know if we have not overfit. All we can do is attempt to lower the probabilities that we have overfit, and these stress-test methods help.

Trend following strategies tend to have lower win rates as they look for large, outsized moves. Running a trend following strategy through the variance test while reducing the win percentage does not have much of an affect. On the other hand, adjusting the noise for a trend following strategy seems applicable.

Thinking through what each test is testing and what your strategy is attempting to capture can help find the right tests for each specific strategy.

Remember, putting strategies on a simulated account or trading with reduced size can be another form of checking robustness on live data.

Robustness testing on my trading portfolio or on individual strategies?

Yes, you should do robustness testing on both individual strategies and your portfolio. Build Alpha provides the tools to run all the robustness checks and tests on both individual strategies and portfolios of strategies.

In the previously linked Algorithmic Trading Guide I shared my thoughts on how to build a proper trading portfolio. In this guide, I make the case that bad standalone strategies may have a place in your portfolio as they can “do the dirty work” to assist your star players. These role player strategies will rarely pass robustness tests; however, adding them to your portfolio should not harm the robustness tests of your portfolio!

Robustness Testing Takeaways

  1. Most algorithmic traders fail because they overfit strategies causing loss of risk capital
  2. Algorithmic traders would do better incorporating robustness tests and checks
  3. There is no best robustness test. Each test tests something different.
  4. A backtest is often not enough. Robust strategies require additional testing.
  5. Passing every test is ideal but not required for a robust trading strategy

Need to Knows

  • Backtesting is not enough to find robust trading algorithms
  • Robust Checks help identify when backtests are lying prior to the trader risking capital
  • Robustness tests aim to find trading systems that will weather changing market conditions
  • Overfitting and overoptimization can be reduced through robustness testing
  • Strategies do not need to pass every robustness test to be considered robust

Summary: Robust Trading Strategies require additional testing

Many traders struggle to make algorithmic trading work, but when looking at trading software available it is clear. Robustness testing is not offered in nearly any software. That is why I set out to build Build Alpha and include the various robustness tests and robustness check I have come across over the past decade in my professional trading career. Strategies must survive multiple and ever-changing market conditions.

Robust trading systems are rare and will require the astute trader to discard many strategies in the process. However, discarding strategies before risking hard earned capital is a win in my book. I hope this guide helps you break your strategies before the market does. In the long run, running additional tests and avoiding losses where possible can have a tremendous impact on achieving your trading goals and how a strategy performs.

If you have any questions or additional insights, please feel free to email me anytime.

Author

David Bergstrom Author Photo Trading Edge

David Bergstrom – the guy behind Build Alpha. I have spent a decade-plus in the professional trading world working as a market maker and quantitative strategy developer at a high frequency trading firm with a Chicago Mercantile Exchange (CME) seat, consulting for Hedge Funds, Commodity Trading Advisors (CTAs), Family Offices and Registered Investment Advisors (RIAs). I am a self-taught programmer utilizing C++, C# and python with a statistics background specializing in data science, machine learning and trading strategy development. I have been featured on Chatwithtraders.com, Bettersystemtrader.com, Desiretotrade.com, Quantocracy, Traderlife.com, Seeitmarket.com, Benzinga, TradeStation, NinjaTrader and more. Most of my experience has led me to a series of repeatable processes to find, create, test and implement algorithmic trading ideas in a robust manner. Build Alpha is the culmination of this process from start to finish. Please reach out to me directly at any time.

Edge in Trading | Professional Guide to Trading Edges

What does EDGE in trading mean?

In financial markets, trading edge is a temporary advantage over other market participants. Having an edge in trading can exist in many different forms but the two most common are information edge and price edge.

Information edge would be insider trading or material non-public information. For example, knowing a company signed a large deal that will greatly increase revenues before other market participants know would be an information edge.

Information edge billions day trading

On the other hand, price edge exists from research and data analysis and is a far more common way to build a trading career. Quantified technical analysis or knowing that prices that reach this level have historically been cheap over the next N days can present a price edge. Renaissance Technologies, the greatest hedge fund of all time, is quoted as saying

Renaissance essentially attempts to predict the future movement of financial instruments, within a specific time frame, using statistical models. The firm searches for something that might be producing anomalies in price movements that can be exploited. At Renaissance they’re called “signals.” The firm builds trading models that fit the data.” Source: The Secret World of Jim Simons | Institutional Investor

Jim Simons Price Edge Trader Hedge Fund

Regardless of the trading style, swing trader or day trader, a successful trader must possess some edge – it is of vital importance. Trading edges are necessary in the stock market, forex trading, futures markets, crypto, etc.

Trading results will evade you and success will always flow to other traders who possess the ability to identify an edge and manage risk.

What is an example of a trading edge?

An information trading edge can be knowing a company’s quarterly numbers before the public does or knowing merger and acquisition news ahead of the public. This is not legal activity but unfortunately has existed in the stock market since day one.

An example of price edge can simply be putting on a position at a price that has been historically cheap compared to other markets or to its own historical patterns. Trading edge based on price can often be quantified and is 100% legal. This is the most popular form of trading edge and what will be referred to in this post.

Trading Edge Formula

In short, edge is positive expectancy. What is and how do we calculate expectancy?

Expected value is the amount a trader should expect to gain if he were able to place infinite similar bets. EV, for short, is technically defined as the sum of all possible outcomes multiplied by the probability of each outcome’s occurrence. EV can be used as the most rudimentary form of trading edge.

EV = (Probability of Win * how much you win) – (probability of loss * how much you lose)

If EV is negative, we should not take the trade. We do not need a high win rate if the winning amount is large in comparison to what we could lose (think trend following).

There are two levers to move: win rate AND risk:reward. Too many traders get focused on one or the other. This graph is great to visualize how risk:reward and win rate relate.

Positive Expectancy Chart

All values based on $1 risk. 2.5 Reward indicates a profit of $2.5 and a loss of $1

Notice experienced traders can make more trading a 50% win rate strategy with 2:1 risk:reward compared to a trader who is accurate 70% of the time with a 1:1.

Now let’s dive into an example to make sure we fully understand expected value. Let’s discuss a coin-toss game where we have two scenarios:

  1. Heads happens 90% of the time and you win $1 each time.
  2. Tails happens 10% of the time and you lose $10 each time.

Would you play this game? Most beginner day traders salivate at 90%-win rates. However, the EV for this game is negative! The formula calculates 90% x $1 + 10% x -$10 = -$0.10. Edge needs positive expectancy.

This is a very important lesson that even with a 90%-win rate you can lose money! And trust me, you won’t win 90% of the time.

Do you need edge in trading?

Yes. If you do not have an edge, you are trading randomly, and no one can succeed randomly as trading has costs and randomness includes negative performance streaks. Most traders and trading strategies fail because they do not know or have an edge.

Most traders do not know or have an edge because they have never quantified their edge to see if it is truly an advantage over time, across symbols, and timeframes. New traders must understand this is necessary to becoming a profitable trader.

The first thing we need is an edge and a way to quantify that edge. Enter e-ratio or edge ratio.

What is Edge Ratio?

Edge Ratio is a quantified metric demonstrating how much favorable price movement occurs in comparison to how much adverse price movement occurs. Edge Ratio measures how much a trade goes in your favor compared to how much a trade goes against you. Edge Ratio or eratio quantifies trading edge.

I wrote extensively about e-ratio here: Edge Ratio in Automated Trading but will give a quick synopsis here.

We can plot eratio where the X-axis is the bars from the entry and the y-axis is how much more the trade is in your favor versus against you.

Edge Ratio Build Alpha

Blue edge ratio for trading signal. Red edge ratio for random trading signal for comparison

In the image above, we can clearly see that we have the most edge (blue line) six bars into this trade. The edge ratio is about 1.6 meaning we get about 0.6 more units of volatility in our favor than against us.

Trading any signal or entry that has an edge ratio below 1.0 is no better than a random entry determined by a monkey throwing darts.

I also spoke about edge ratio on the Better System Trader podcast.

How long might a trading edge last?

Can we know where to exit positions? Using edge ratio, or eratio, we can see when a trading edge begins to deteriorate. Knowing the edge ratio of an algorithmic trading strategy or any entry can help you determine the best exits and when your trade has overstayed its welcome.

In the above eratio image, you can see that after a hold time of six bars, the trading edge begins to deteriorate. In these cases, the best exit is around the six-bar mark as that is when the trading edge has shown most powerful.

More Winning Trades than Losing Trades does not mean Edge

Retail traders develop a trading strategy and wind-up optimizing parameters until the past performance shows more money than God. This trading approach often leads to overoptimized strategies that look great on historical data but have miserable live trading future results.

An individual trader should look for robust strategies that possess true edge and can survive various market conditions. A backtest that shows more winning trades than losing trades but has a negative or flat e-ratio is often an obvious sign of overfitting. Before you start trading new strategies live, always check the e-ratio to see there is actual edge to be captured and the backtest wasn’t pure luck.

Ways to Visualize Trading Edges

Edge Ratio is the best quantified measure of edge for algo traders, but there are a handful of other approaches to visualize your trading edge that are noteworthy.

Heatmap

we can view the forward returns from the entry where the x-axis shows bars since entry and each y-axis row is an individual trade. The color represents how profitable (green) the return was at each bar from entry. Analyzing two different signals on two different heat maps can clearly signal what was more profitable.

New Trader Heatmap

Equity Curve

the equity curve of an algorithmic trading system sums all trades to show the historical account value over time. We can think of the equity curve as the money earned (or lost) over time had a trader followed a specific rule or set of rules. Often strategy choices are obvious.

Equity Curve Mean Reversion strategies

Distributions

viewing all possible outcomes from a potential strategy can yield greater insights than just the heatmap or equity curve. We can run statistical tests like Monte Carlo Simulation and Analysis – more on this later – or other tests to generate a better sense of what is possible. We can think of this as how does my algo trading strategy do in alternate universes where the price data was similar but not exactly what my universe experienced.

Which P&L range would you prefer to be in after 50 trades? Some strategy paths are negative on the left where all are positive on the right. If markets change, trading systems like the one on the right can provide a buffer.

Probability Distributions for Mean Reversion Strategies

To read more about visualizing trading edges please check out my original post on See It Market here: What Does Trading Edge Look Like

Best Edge in Trading

Not every trade is a winner and there is no single best edge in trading. The best edge in trading is simply having one and executing it with discipline. We can quantify edge with edge ratio so that presents one way to rank our trading strategies. More capital should go toward stronger edges than weaker ones.

However, weaker edges can be combined via ensemble methods to potentially create stronger ones. I wrote about ensemble methods and strategies here: Ensemble Strategies – Build Alpha

Mental Edge in Trading for additional winning trades

Having a mental edge can also constitute a trading edge. The trading industry has convinced many traders that day trading and using a trading journal creates some mental edge. However, a mental edge should be thought of as additive and not a stand-alone edge!

For example, having a mental edge but no price edge cannot make for profitability. A mental edge in trading is the ability to remain disciplined and execute your price edge as planned (and as many times as possible).

With no price edge, having mental fortitude or mental discipline still means executing randomly but just in a calm fashion. Trading starts when we combine both mental edge and price edge.

Additionally, a mental edge can be aided with automation and algorithms. Utilizing automation is the simplest way to ensure your price edge is executed in a systematic fashion each and every time the edge reveals itself. To learn more about algorithmic trading please check out this full guide: Algorithmic Trading Guide.

Trading Edge, Winning Trades, and the Law of Large Numbers

In the short-term, everything is random. In the long run, things tend to converge toward their expected value (which is our trading edge). This mathematical concept of convergence is known as the law of large numbers.

Law of Large Numbers Graphic

Many traders fail due to strategy hopping. That is, chasing the new shiny object, the next best strategy.

A terrible trader’s journey that most are on: find a strategy that does well for a while; however, when it starts to struggle either a) tweak it or b) learn some new strategy.

Either option changes the initial strategy aka strategy hopping. This tweaked or new strategy does well live trading for a while but ultimately arrives at another decision point between a) tweaking it again or b) finding a new shiny strategy.

Traders fail to give any strategy time to play itself out. This keeps them in short-term randomness!

The best illustration of short-term randomness and long-term expected value is to view a coin toss over time. Fair coins should land on heads 50% of the time. However, in small amounts of coin tosses we see large variations in the percentages. In only 30 tosses, we saw heads come up 73% of the time and in two other 30 coin toss trials we saw heads come up 43%.

30 Coin Flips Stocks Market

However, after 100 coin tosses the percentages of heads converge toward the expected value of 50%. The law of large numbers is starting to work its magic! 

100 Coin Flips Stocks market

After 10,000 coin tosses the percentages are almost always 50%. Assuming all things remain the same, the law of large numbers is inevitable.

10,000 coin flips stocks market

Trading Example of the Law of Large Numbers

This algo strategy below has a remarkably smooth equity curve that most of us would love to see on our brokerage statements. This algo system averaged about $170 in profit per trade.

Build Alpha Equity Curve

However, if we track this algorithmic trading strategy’s average trade over time, we can see that in the beginning, when our trust in the trading system is the lowest, it is a rather bumpy ride and far from the $170 per trade edge we wind up with. It takes well over 100 trades for the average trade to converge to the actual average!

Escaping Randomness Graph

Most traders cannot stomach this short-term “randomness” and wind-up abandoning ship to chase a new shiny object, indicator, or “holy grail” strategy.

Traders that fall prey to strategy hopping never let the edge play itself out and sadly never make it. If this is you – stay on the demo account! Trading decisions like this are often missed by other participants and take time to learn.

Some never escape this short-term randomness. My mentor explained this short-term randomness, long-term obviousness concept to me and a lightbulb clicked. Edge plus time is the name of the game.

Escaping Randomness was the perfect title of my Chat with Traders interview where I discuss how to overcome randomness and why algorithmic trading can help traders think about edge and risk-taking in a more productive way. If you haven’t already, please check out Aaron’s wonderful work here: Escaping Randomness with David Bergstrom

Trading Edge Ideas and Trading Edge Examples

This blog post would be incomplete without mentioning some simple trading edges and where to begin your search to generate ideas for your own trading edges and limit your own risk.

Time of day trading

many stocks and commodities exhibit strong time of day patterns. For example, the S&P500 has been very strong for day traders from 2 AM ET to 3 AM ET for the last 15-plus years.

S&P 500 2AM Money

Buying one contract in e-mini S&P 500 at 2:00 AM ET and selling one hour later

Seasonality

many stocks show strong seasonal tendencies where repeatable patterns or price movement happens during the same calendar dates year after year.

Price Action

bar patterns and price comparisons can be a strong starting point to identify trading edges. However, always investigate and test thoroughly. Do not be misled by industry given names such as Bearish Engulfing. Read about trading “truths” here.

Relative Value

comparing one asset to another can provide an edge relative to another asset. This can assist non-directional traders, pair traders, and relative value traders with many trades in many markets.

Alternative Data

Using alternative data such as market breadth, economic data, interest rate spreads, Vix term structure, options flows, and more can provide additional areas to amplify or add trading edge to your portfolio.

Alternative Data Build Alpha Signals

some available alternative data signals in Build Alpha

Using Build Alpha to enhance Trading Edge

Build Alpha allows you to require entry signals forcing the software to use these required signals on every strategy the genetic algorithm creates. The trader can also select thousands of other non-required signals to combine with any required signals. This process will then find the best complimentary signals for the required trading signals. This can take a known edge and enhance it without much effort or time.

Additionally, Build Alpha provides various ways to visualize trading edge for any strategy including e-ratio, monte carlo distributions, equity curves, and heat maps.

Finally, no edge is complete until it passes stress testing. Build Alpha offers the largest suite of robustness tests and checks for algorithmic trading strategies. Robustness testing is an edge of its own as many strategies fail in live markets.

Please check out the full suite here: Robust Trading Strategy Guide.

Need to Knows About Trading Edges

  • Trading edge is necessary for trading success in any market
  • Information edge is often illegal. Price edge is used by the best
  • Expected Value and positive expectancy is often a great first step to quantify edge
  • Edge Ratio is the best way to quantify edge and risk
  • Let your market edge play out in time. Law of Large Numbers takes time to be profitable.
  • Various types of trading edge exist including time of day, price action, alternative data
  • Build Alpha algorithmic trading software can speed up the edge hunting process

Summary of Edge in Trading

Finding an edge in trading is often the difference between account growth and trading failure. There are two primary sources of edge in trading: information edge and price edge. Information edge consists of insider information or material non-public information. Price edge consists of data analysis and finding opportunities that others are unwilling to take.

The best traders quantify and rank their edges before deciding allocations. Edge Ratio is the best way to quantify and compare trading edges. In the end, finding one setup or trading edge is the key to long-term success in the markets. Creating a process or using software to quickly identify edges can be the difference between growth or trading randomly. Day traders be aware!

Author

David Bergstrom Author Photo Trading Edge

David Bergstrom – the guy behind Build Alpha. I have spent a decade-plus in the professional trading world working as a market maker and quantitative strategy developer at a high frequency trading firm with a Chicago Mercantile Exchange (CME) seat, consulting for Hedge Funds, Commodity Trading Advisors (CTAs), Family Offices and Registered Investment Advisors (RIAs). I am a self-taught programmer utilizing C++, C# and python with a statistics background specializing in data science, machine learning and trading strategy development. I have been featured on Chatwithtraders.com, Bettersystemtrader.com, Desiretotrade.com, Quantocracy, Traderlife.com, Seeitmarket.com, Benzinga, TradeStation, NinjaTrader and more. Most of my experience has led me to a series of repeatable processes to find, create, test and implement algorithmic trading ideas in a robust manner. Build Alpha is the culmination of this process from start to finish. Please reach out to me directly at any time.

Monte Carlo Simulation | Complete Guide for Algorithmic Trading with Free Simulator

What is Monte Carlo Simulation?

Monte Carlo Simulation is a statistical technique that injects randomness into a dataset to create probability distributions for better risk analysis and quantitative decision-making.

The most common Monte Carlo Simulation randomizes the order of a data set to demonstrate an alternative path a data set could have experienced – this is particularly useful for algorithmic traders and trading systems to see different outcomes of past trades.

In financial markets, quantitative traders use the most common Monte Carlo Simulation method to reshuffle the order of their historical trades to help them better understand how a trading system could have happened.

Algo traders aim to answer the question, if the order of trades was not identical to the backtest’s order, would I still be comfortable trading this strategy?

Monte Carlo Analysis for Probability Distribution

The equity curve, the cumulative profit of trade results, can provide insights into how smooth a trader’s account may grow following a particular algorithmic trading strategy. Monte Carlo provides a method of re-simulating the trading strategy to see how “bumpy” the ride could have been in an alternate reality and even acts as a peak into what the future may hold for the trading strategy.

The purpose of Monte Carlo Simulation is to detect lucky backtests and misleading performance metrics before risking real capital.

This video is a great introduction to Monte Carlo Methods: Monte Carlo Basics – Build Alpha

Why are Monte Carlo Simulations used?

Monte Carlo Simulations help better simulate the unknown and are typically applied to problems that have uncertainty such as: trading, insurance, options pricing, games of chance, etc. The goal is to gain a better understanding of all the possible outcomes and potential minimum and maximum values.

Traders use Monte Carlo simulations for a plethora of reasons from re-simulating trading systems to gaining a better understanding of the risk of a trading system.

 

Mean and Standard Deviation of Drawdown

Why re-simulate an equity curve? Algo traders use monte carlo simulations to determine how much luck was involved in a strategy’s backtest and if future performance is likely to look like past performance.

If the trading system was overly lucky, then it would be nice to know before risking real capital in live trading.

A trader may backtest a trading strategy and notice an acceptable maximum drawdown; however, after running a monte carlo test the drawdown may be much less tolerable. This Monte Carlo method could save the trader risking capital on a strategy he could not stomach in live trading.

The main uses for Monte Carlo simulations in trading simulation are:

  • Better understanding of possible trading drawdown
  • The proper funding of a trading strategy
  • Understanding possible win and loss streak potential
  • Setting better profit and loss expectations

Benefits of Monte Carlo Simulation

Better understanding of drawdown and future outcomes

reshuffling the order of your trades can lead to different profit and loss sequence which can result in a greater drawdown. A trader may believe the backtest’s drawdown is the worst it can get, but monte carlo analysis may show a much larger drawdown for most trading systems.  

Properly fund your trading strategy

noticing a larger maximum drawdown from a monte carlo simulation can help a trader better capitalize a trading strategy. This can make a live drawdown more bearable and allow the trader to stick to the original plan. On the other hand, a trader that experiences a live drawdown greater than the backtest’s drawdown may prematurely turn off a trading strategy if not familiar with monte carlo simulations.

Understanding possible win and loss streaks

the backtest may show a maximum of 5 or 6 losing trades in a row but a monte carlo test may show that 8 or 9 losing trades in a row is possible. The trader can better prepare for this adverse situation equipped with the insights from a monte carlo simulation. Some traders may not be able to withstand consecutive losing trades and a monte carlo simulation may help traders avoid systems susceptible to such strings of losses.

 

Random Outcomes normal distribution likelihood of winning and losing trade result

Set better expectations with quantitative analysis

many traders view the backtest as a guide for what to expect. For example, a trading system shows an average trade of $500 so a trader assumes in 10 trades I should expect to make $5,000. However, a monte carlo test may show a wide range of possible profit and loss scenarios after 10 trades. This data analysis can help a trader remain calm enough to stick to the strategy when luck becomes favorable or unfavorable.

Monte Carlo Testing for Drawdowns

Beginner traders find a successful backtest and think they have struck gold. However, many beginning traders are misled by overly optmistic backtests. Often the most important performance metrics such as net profit, standard deviation of historical trades, and consecutive winning trades will be inflated.

The maximum drawdown from a backtest is often the most misleading metric!

Monte Carlo Analysis helps traders better estimate a strategy’s drawdown potential which can help traders stick to a strategy during inevitable drawdowns.

This example strategy below shows a backtest drawdown of $1,663.90.

Build Alpha Drawdown View

However, after running a simple Monte Carlo Simulation on the same trading system we can see the worst resample drawdown from all simulations is $5,195.17 or 3.1 times as large as the backtest’s drawdown!

More Accurate Predictions of Drawdown using Monte Carlo Methods

A trader sized based on the backtest would prematurely cease trading and turn a potential winning strategy off or not have enough capital allocated to properly trade this strategy. A simple monte carlo analysis could prevent this.

Monte Carlo Methods: Different types and uses

Reshuffle

this monte carlo method reshuffles the historical trade order 1,000 times creating 1,000 new equity curves. Each equity curve has its own performance metrics such as maximum drawdown and consecutive losing trades. This method reshuffles the original trades, so all 1,000 equity curves end at the same total profit and loss amount.

 

Reshuffling Monte Carlo Method provides quick sensitivity analysis

Resample

This monte carlo method continuously and randomly selects a historical trade with equal chance from all the trades until reaching the backtest’s trade count. This uses random sampling with replacement meaning the same trade can be selected and not every trade must be selected. This process is done 1,000 times to create 1,000 new equity curves. Each curve has its own performance metrics. However, resampling with replacement means not all simulations end at the same amount.

 

Repeated Random Sampling produces Monte Carlo Graph

Randomized

The randomized monte carlo analysis technique re-trades the original strategy’s entries while randomizing each trade’s exit. The randomized exit only uses exit logic from the original strategy. That is, if the original strategy did not use a stop-loss then the randomized exit should not include a stop-loss. A strategy with a max hold of two bars may exit after one bar on the first trade but after three bars on the second, etc.

Randomized prevents lying backtests

If the Randomized Monte Carlo results remain profitable, then it is likely our entry contains some true edge. The above example saved us from a lying backtest.

Read more on robustness testing for algorithmic trading strategies here: Robustness Tests for Algo Trading and please view a Randomized Test case study here: Free Friday 8 – Randomized Monte Carlo

Permutation

This monte carlo method applies to the historical data. First, we reshuffle the log of inter and intrabar price changes then exponentiate to recreate new bars. Re-trade our trading system on the 1,000 newly created synthetic data series.

The new permutated data contains the statistical properties of the original data but destroyed most of the patterns. Profitable results are a good sign the strategy is robust to withstand variations in the underlying data. It is a bonus to see our original trading system in the top trading results.

 

Monte Carlo Permutation Test

If the trading system fails on the permutated data or has many permutated results below or around $0 P&L, then it is time to find a new strategy and stop trading that system.

Advanced Uses of Monte Carlo Simulations for Trading Systems

Monte Carlo Equity Curve Bands

Is my trading strategy broken? If a trader wants to know his expected profit in the next 100 trades, then he can randomly select 100 historical trades and add them to the end of the backtest’s equity curve. Repeat this 1,000 times and keep the 5th and 95th percentile equity curves to represent where the trader should expect to be 90% of the time in the next 100 trades.

 

Monte Carlo Equity Bands for Trade and Project Management

If future trading performance falls outside these monte carlo simulation equity bands, then it is an early warning sign the strategy is broken.

Monte Carlo Simulation Drawdown Technique

How confident am I in this strategy’s drawdown? This technique uses the monte carlo simulation resample method to create 1,000 new equity curves which simultaneously creates 1,000 new drawdown values.

We then plot the drawdowns as a histogram (green) along with a cumulative distribution line (blue). The blue line represents the total percentage of drawdowns as we move from left to right across the horizontal x-axis.

 

Drawdown Confidence Interval for Trading System Development

The red X indicates where 95% of all monte carlo drawdowns were less than the corresponding drawdown value on the x-axis. In this case we can say “we are 95% confident that drawdowns from this system will not exceed 30%”.

To improve the drawdown percentage, the trader should change the input variables. Either increase the starting capital allocated to the strategy or adjust the acceptable drawdown percentage. Understanding the drawdown probability distributions and sizing with statistical confidence can make drawdowns easier to manage.

To learn more please check out the Monte Carlo Drawdown Analysis section of the Complete Algorithmic Trading Guide or view this case study Properly Funding a Trading Strategy with Monte Carlo Drawdown.

Furthermore, I wrote an article for See It Market that covers the three primary uses of Monte Carlo Simulations: 3 Uses for Monte Carlo Simulations in Trading

How many Monte Carlo Simulations are needed?

The ideal number of monte carlo simulations for valid results are 1,000 or more. To truly leverage the law of large numbers and get reliable monte carlo simulation results one should strive for 1,000 or more simulations; however, results are generally acceptable with 100 or more simulations.

The purpose of monte carlo simulation is to detect lucky historical data backtests and misleading performance metrics in our trading ideas. However, running a single monte carlo simulation could create a lucky test result. Imagine a Resample test that luckily resamples only winning trades and creates an incredible equity curve.

Is this likely? No. Is this possible? Yes. Should we base trading decisions off of this?

No!Lucky random numbers or random variable can produce misleading results

 

To avoid a lucky result like the above, we want to run 100 or 1,000 tests in order to ensure we have a good estimate of the median or average result. That is why all the monte carlo simulation images contain hundreds of equity curve lines.

 

Random Numbers or Simulations must be large enough to avoid luck

There still may be lucky results, but we can get a better sense of what is reasonable to expect from a larger probability distribution than a single simulation.

Equity Curve Simulator and Probability Distribution

An equity curve simulator is a tool that accepts winning percentage, average win, and average loss amounts to simulate how an equity curve’s sequence may happen. It is very important to note that a trader can be profitable with a lower winning percentage and larger winning trades or with smaller winning trades and a higher winning percentage. 

To learn more about expected value please check out the complete guide to Algorithmic Trading.

Look at this this expected value display below to see how winning percentage and average win to loss ratio affect profitability:

 

Expected Value Chart across random variables

All values based on $1 risk. 2.5 Reward indicates a profit of $2.5 and a loss of $1

I have built a free equity curve simulator where you can input your expected winning percentage, average winning trade and average losing trade to simulate your expected equity curves.

 

Knowing how these values cooperate can help with algorithmic trading strategy design, risk management, avoid a poor strategy, and analyze trading results.

 Monte Carlo Simulation Excel

Microsoft Excel is often good enough for simple quantitative trading endeavors. I have built a simple excel monte carlo simulation that you can import your trades and simulate various equity curve paths.

Microsoft Excel Monte Carlo Analysis Simulator

 

Simply copy and paste your trades into column A and it will generate your equity curve in column B. Then drag columns D:N down to end at the same row number as total trade count. That is, column A and columns D:N should end at the same row. Then to simulate a new monte carlo test press ‘F9’.

To download this Microsoft Excel Monte Carlo spreadsheet please visit here: link and to read more about monte carlo simulation from Microsoft please head here: Introduction to Monte Carlo simulation in Excel (microsoft.com)

Algo Trading Software for Monte Carlo Analysis: Build Alpha – The Best Monte Carlo Tool

Build Alpha is powerful automated trading software that enables traders to create hundreds of algorithmic trading strategies with no programming needed. Any strategy created can be put through all the Monte Carlo methods listed in this guide. Simply highlight the desired strategy in the results window, then select the Monte Carlo Analysis option on the right-hand side.

 

Build Alpha Monte Carlo Analysis View

 

Build Alpha also returns statistics on winning and losing streaks from the simulations. This can help a trader understand what to expect as consecutive winning or losing trades occur.

You will see additional buttons on the right-hand side for the advanced monte carlo methods of Monte Carlo Equity Bands and Drawdowns.

 

Advanced Testing Options in Build Alpha

 

What is the best Monte Carlo Simulation?

There is no best method as each has different intended uses for the algo trader. For example, the reshuffle and resample are primarily used to get a better estimate of drawdown and risk analysis.

On the other hand, randomized and permutation tests are used for strategy robustness and future viability. These can be thought of as stress tests for a trading system.

Monte Carlo Drawdown technique is similar to reshuffle and resample but puts an emphasis on creating confidence intervals around the actual expected drawdown.

Finally, the monte carlo equity bands or probability cones are used to monitor live trading and identify broken strategies. There is no best monte carlo method as a professional quant trader should incorporate all into the strategy development process.

Key Takeaways

  1. Monte Carlo simulation is a statistical technique to help uncover luck in backtests
  2. Most popular Monte Carlo method is to reshuffle historical trades to view alternative account paths
  3. Can help estimate realistic drawdowns and possible outcomes
  4. Can help size trading strategy properly

Monte Carlo Method Need to Knows

  • There are many different Monte Carlo Methods
  • Inserting randomness or random values into historical trade results to better estimate uncertainty
  • Drawdown estimation is the most common use of monte carlo analysis
  • Large number of simulations needed to create proper probability distribution. 100 minimum but 1,000 or more is ideal
  • Can be used to monitor strategy performance and identify broken strategies

Monte Carlo Summary for Algo Trading

Monte Carlo Simulation work in various ways. The most popular monte carlo methods are used to help traders identify luck and more appropriate drawdown measures than a simple backtest can provide. Reshuffle and resample monte carlo methods help traders simulate various equity curves with alternative trade sequences to view potential outcomes occurring.

Monte Carlo Randomized and Permutation tests aim to test strategy robustness and act as a stress test to trading strategies prior to real risk taking. Monte Carlo equity bands aid traders in identifying early signs of a broken strategy. Finally, MC Drawdown assists traders find proper confidence intervals around drawdowns to ensure the strategy is funded with enough capital for the trader’s risk tolerances.

Monte Carlo Simulations are arguably the most popular quantitative trading tool to add to your algorithmic trading toolbox. More advanced readers can experiment with altering the probability density function or random number generator to make MC selections. I will save this and other variables for another discussion.

Author

David Bergstrom Author Photo Trading Edge

David Bergstrom – the guy behind Build Alpha. I have spent a decade-plus in the professional trading world working as a market maker and quantitative strategy developer at a high frequency trading firm with a Chicago Mercantile Exchange (CME) seat, consulting for Hedge Funds, Commodity Trading Advisors (CTAs), Family Offices and Registered Investment Advisors (RIAs). I am a self-taught programmer utilizing C++, C# and python with a statistics background specializing in data science, machine learning and trading strategy development. I have been featured on Chatwithtraders.com, Bettersystemtrader.com, Desiretotrade.com, Quantocracy, Traderlife.com, Seeitmarket.com, Benzinga, TradeStation, NinjaTrader and more. Most of my experience has led me to a series of repeatable processes to find, create, test and implement algorithmic trading ideas in a robust manner. Build Alpha is the culmination of this process from start to finish. Please reach out to me directly at any time.

Backtesting Trading Strategies
Everything You Need to Know

What is Backtesting?

Backtesting is applying a trading strategy on historical market data to view how a particular strategy would have performed in the past. A backtest will return the hypothetical total profit, a list of historical trades, and many other performance metrics to analyze a trading system’s potential viability in financial markets.

For example, we can take the past ten years of data and apply our strategy – in hindsight – to view how the strategy would have performed over these past ten years. The backtest’s results can help drive trading decisions such as:

  • discard the investment strategy
  • tweak the strategy
  • investigate the strategy further
  • move the strategy to a demo account for monitoring

What is Backtesting Used For?

Backtesting gives traders and investors a sense of a strategy’s profitability on historical data which may give insights into how the strategy can be expected to perform in the future.

However, please note, all disclaimers in financial markets will warn that historical performance is not indicative of future performance. However, seeing positive past performance can grant a trader confidence.

In short, backtesting is testing trading opportunities on historical data to assess the historical profitability of a trading strategy or market environment.

Backtesting can be used to check the viability of a single trading system, gain a better understanding of how a strategy would perform during certain market environments, or to compare competing trading signals when capital may be limited.

Why is Backtesting Important?

Backtesting trading strategies is important because it provides a way to analyze risk prior to risking real capital. The immediate insights a backtest provides can potentially save the trader money by helping him avoid a potential losing strategy.

If a trading strategy performs poorly, losing money during a backtest, then there should be less incentive to live trade the strategy with real capital as it may result in losing money rapidly.

On the other hand, a good backtest should be a requirement for risking capital. This does not mean that a good backtest is a greenlight to bet the farm, but a trader should desire to see a profitable backtest before committing real capital.

Backtesting provides important performance statistics of a potential trading strategy such as:

  • potential profit and loss
  • potential drawdowns
  • total trades
  • winning percentage
  • risk/reward
  • largest winning trade
  • largest losing trade
  • consecutive winning streaks
  • consecutive losing streaks

All these insights can better help prepare the trader for live trading the strategy, the risks involved, gain confidence, or choosing between two competing strategies which all help contribute to long-term success.

What do you need for Backtesting a Trading Strategy?

There are only two things you need to conduct a proper backtest:

  1. Historical Price Data for desired time period for the test
  2. Trading Idea (or Idea Generator)

These two points are enough to conduct manual backtesting where you can replay the charts, view where you would have entered, and track your strategy on paper or with virtual money. This is often enough for testing purposes before you start trading. A bit more on these two points below.

Historical Data Needed for Backtesting

What kind of Data is needed for backtesting?

The most important element for running a backtest is historical financial data. Financial data normally comes as a time series in OHLC or open, high, low, close format. Below is an example of historical SPY data.

Historical Data Sample OHLC

Resources for Free Financial Data

How much data is needed for backtesting

The goal is to get enough trades from the backtest to feel confident the trades were not lucky and perhaps find an effective trading strategy.

For example, if you flip a coin 10 times and get seven heads then we cannot determine if the coin is rigged. However, if we flip the coin 10,000 times and get 7,000 heads then we can be sure the coin is rigged.

This is because as the sample size increases, we should converge toward the expected value. This mathematical concept is known as the Law of Large Numbers. I give a full breakdown in my Robust Trading Strategies Guide.

We really want to see at least 100 trades or more to avoid “luck” playing a role. Note this is a rule of thumb and not a steadfast number. If our strategy trades once per month then we would need 100 months of data or roughly eight years. However, if our strategy trades more frequently then you can backtest with smaller data histories.

There are many other considerations such as all data coming from a bull market or should data be from both a bull and bear market, etc.? However, these advanced questions are answered in the above-mentioned guide.

For now, we need enough data to generate enough trades. This should be your only concern.

Backtesting different time periods and market conditions

It is wise to include different time periods or market environments in your backtest. For example, a long breakout strategy may do well in a bull market and poorly in a bear market. If your backtest only contains historical data during a bull market, we may be deceived into believing we have found a holy grail. Does it work on another time period?

Backtesting different markets

Do you need stock market data if you are testing a forex strategy? A forex trading strategy may only require currency pair data and fx traders may have no need for stock or future data. Trading cfds and futures or stocks may require different data sets or time periods but a strategy that backtests well across different markets can be a reliable indicator to future results. Quantitative strategies that perform well across financial instruments, showing similar performance on various symbols, can be a key component to success.

Finding Trading Ideas for Backtesting

If you have been around the markets long enough, you will begin to create your own trading ideas but there are various other ways to quickly stumble upon trading ideas to backtest. Below are some basic steps to sourcing new trading ideas.

Screen Time

The more time you spend watching live market charts and order books the more patterns you will identify. These patterns are great ideas to test for viable trading strategies.

Trading Books

Most naïve traders make false claims about trading books, “if this was profitable why would the author write about it”? In reality, there are many great trading books and ideas out there.

Trading Papers

Brilliant researchers publish white papers and research papers that leave hints, ideas, trading methods, and further areas to research. Arxiv.org is a great resource and even has a quantitative finance filter.

Technical Indicators

Viewing technical analysis indicators can give insights into how other traders are looking to summarize trading concepts. These can be incorporated into your own strategies and backtests.

Alternative Data

Using alternative data can help add context to price action signals or provide new trading ideas. For example, companies that have many job openings and listings may be expanding quickly!

Genetic Algorithm or Machine Learning

A genetic algorithm takes massive amounts of inputs and attempts to mix and match to find the best possible combinations (crude simplification). In theory, it is generating new ideas for you. This is the core behind Build Alpha algo trading software which I will describe later.

Machine learning works in much the same way and can also be thought of as a way to create new trading ideas or a predictive model.

Backtesting Programming Language or Software

To apply our trading idea on the historical data to complete our backtest we need some type of programming language or software. Below are the best options for both new and advanced backtesters.

Backtesting in Excel

Microsoft excel is excellent for quick and easy backtesting. I wrote a full example here: Backtesting in Excel. In short, here is a quick example on SPY that buys when price closes in the bottom 20% of the day’s range and sells the next day.

Excel for trading system and predictive model

Backtesting with Python

Python is the fastest growing programming language due to its easy to learn syntax and easy to read coding style. Python has many various libraries (publicly available code) you can incorporate into your own code to create technical indicators, run scenario analysis, backtest and more. Here are a few favorites:

Backtesting with C++

C++ is much faster than python and primarily used by high frequency traders to backtest terabytes of tick data and more. However, C++ is much more complex to learn and harder to read. If you are new to programming this language has a steep learning curve but is worth it once mastered.

No Code Backtesting Software

For those that do not want to be limited by excel or learn a programming language then I would highly recommend Build Alpha as backtesting software. Manual backtesting can be a time-consuming process. With a little computing power, Build Alpha’s automated backtesting engine can test hundreds of thousands of strategies per second.

Build Alpha fastest backtesting software

Build Alpha also has 6,000+ built-in entry and exit signals, the ability to create your own signals, and advaned robustness tests to identify lying backtests.

The best part is Build Alpha is completely code free and requires no programming! I will share more on Build Alpha below.

Backtesting Trading Strategies Example

Let’s backtest a trading strategy. Actually, let’s backtest three simple trading ideas so we can see the results, what a sample trade looks like, and more.

First, let’s backtest a moving average crossover. Let’s buy when the 5-period simple moving average (SMA) crosses above the 10-period SMA. We will sell when the 5-period SMA crosses below the 10-period SMA. Here is a simple image of the entry and exit signal.

Moving average crossover backtest example

Below is a sample equity curve applying this strategy to SPY historical data over the time period 2008 to 2022. Obviously not confidence inspiring.

5SMA and 10SMA SPY program

The second strategy will buy SP500 futures e-mini contract after a negative Monday and hold for one day. This strategy is known as “Turnaround Tuesday” which I originally wrote about in 2017. Let’s check performance since then using Build Alpha automated software to backtest a trading strategy.

Turnaround Tuesday System Performance

Finally, let’s buy SPY after a bearish engulfing candle and hold for one day. This strategy was originally published in 2017 where I exposed some market “truths” that could be debunked with backtesting. Let’s check performance to see if it has maintained its stellar win percentage. Spoiler: it has with a 62%-win rate since 2017 following a “bearish” engulfing candle.

Bearish Engulfing Candlestick Pattern Backtest

Analyzing Backtest Performance

In the above examples we viewed different metrics. In the first two, we viewed total profit via the equity curve, and in the third example we viewed winning percentage.

What are the best backtest performance metrics and how can we compare trading strategies after viewing backtesting results?

Main backtest performance metrics

  • Total Profit – total money gained or lost during the backtest
  • Equity Curve – cumulative profit graph of all historical trades, trade-by-trade
  • Maximum Drawdown – the worst peak to trough drop in total profit during the backtest
  • Average Trade – typical gain or loss per trade
  • Winning Percentage – total number of winning trades divided by total trades
  • Total Trades – total number of trades taken during the backtest
  • Profit Factor – total dollars earned divided by absolute value of total dollars lost
  • Sharpe Ratio – a risk-adjusted return metric created by Nobel laureate, William F. Sharpe

Calculating Sharpe Ratio

According to Investopedia, Sharpe ratio is a measure of risk-adjusted return and describes how much excess return you receive for the volatility of holding the riskier asset. It is calculated as the return minus the risk-free rate divided by the standard deviation to achieve that return. The higher the Sharpe ratio the better. Risk-adjust returns are a great way to compare strategies.

Sharpe Ratio Calculator

Below is a free Sharpe Ratio calculator where you can enter your backtest’s return percentage and standard deviation to calculate your Sharpe Ratio. Play around and see how return and volatility coexist.

Free Web Based Backtester! Backtesting with no code.

This simple web backtester will let you create and test your own ideas. It lacks a lot of the features and functionality that Build Alpha Algo Trading Software (mentioned later) has but will help you get started backtesting with no coding or data retrieval. For those of you serious about backtesting, strategy development, strategy validation and more please check out Build Alpha and speed up your learning curve.

But for now… test for free. Please share and be kind.

Backtesting Benefits

There are five major benefits to backtesting a given strategy. Below I cover each one.

1. Helps avoid bad strategies

Backtesting a strategy before risking capital can help avoid losing strategies that stand zero chance to make a profit and will perform poorly in live trading.

2. Paper trading  

Backtesting can help a trader visualize what following a trading strategy would be like. Finding a strategy that matches your personality can be a benefit to long-term success. Utilizing a risk-free demo account to conduct paper trading to gain confidence is an excellent idea before risking money.

3. Evaluate different market conditions

Backtesting enables traders to analyze a strategy on different market conditions from different historical data periods. How does this strategy do during the 2008 Great Financial Crisis? How does it do during a strong bull run from 2013 to 2015? How did it do during the Covid drop of 2020 or some future market conditions we do not know yet?

Additionally, intraday strategies can be evaluated on different days of the week or during different time windows.

4. Set proper expectations

Backtest results can help traders understand how profit and loss expectations should be set and engage in smarter money management. If a strategy earned $10,000 per 100 shares per year, then expecting a strategy to make $1,000,000 on 100 shares next year would be unreasonable investment research.

Moreover, noticing a steep drawdown during a backtest can help a trader properly fund a strategy prior to going live.

I spoke more about setting proper expectations here: Monte Carlo Analysis and Drawdowns

5. Compare Strategies

Not all traders have unlimited capital and must choose between strategies. Backtests can help traders understand where and when their capital is best put to use. Spread bets, algo trading, futures, stock gappers, trading cfds, etc. can all be compared objectively.

Backtesting Pitfalls and Common Mistakes

Poor backtest results often indicate a strategy unworthy of live trading. However, good backtest results are not a greenlight to live trade. There are a handful of backtesting pitfalls we must watch. Some backtests may be too good to be true!

Overfitting or Curve Fitting Backtesting Risk

Over optimization or curve fitting is finding a trading strategy that too closely fits the historical data and will fail to adapt to new data. The strategy essentially learns the nuances of the historical data during the backtest but fails on new data when the exact nuances do not play out exactly as they did in the previous time period.

 

Overfitting strategies backtesting pitfalls

 

Overfitting is the largest risk system traders face. Overoptimizing parameters to improve backtest results can lead to great test results and good emotions, but horrible live trading performance.

There are many advanced techniques to overcome this pitfall which I wrote about here:  Curve Fitting – Build Alpha 

Look Ahead Bias

Another backtesting pitfall, look ahead bias, is using data that was not known at the time of the historical trade. A tragic example of look ahead bias in a multi-timeframe strategy would be using the daily close to make a trading decision midday. At the time of the trade, midday, it is impossible to know the close of the same day.

These look ahead bias risks are often negated and removed with proper data cleaning or professional grade automated backtesting software.

Survivorship Bias

Running a backtest on only currently traded symbols introduces too much optimism in the backtest results as it can avoid buying stocks that eventually went bankrupt or delisted. This phenomenon is known as survivorship bias.

Much like look ahead bias, survivorship bias deals with underlying historical data issues. Many data providers only have symbols currently listed on exchanges. A proper backtest should consider all symbols that were available at all historical dates and times. This includes stocks that went bankrupt, merged, went private or delisted.

 

Enron OHLC candlestick chart

 

Additionally, choosing to run a backtest on only the best stocks also introduces survivorship bias. The best stocks cannot be known ahead of time and undoubtedly leads to overly optimistic results. If we could identify AAPL or GOOGL ten years ago then your personal finance situation would be significantly different.

We must be unbiased in our symbol universe selection to have truly reliable backtest results. Ask your data provider if they provide listed and delisted securities!

Ignoring Transaction Costs

Backtesting a trading strategy without transaction costs such as slippage or commissions can mislead traders into believing unrealistic backtesting results thus depleting their accounts in live trading.

Backtest results can show an average trade of $10 or $15 but in real markets where slippage and commissions are present the average trade may be negative.

Ignoring trading costs during backtesting can find unrealistic and unobtainable results. Always include transaction costs in your backtests for reliable results!

Backtesting vs. Forward Testing

Forward testing is creating a trading strategy and allowing it to trade on live data on a paper or simulated account. Some forward test in a real account with small size. Many traders view forward testing as a necessary component of moving a strategy from the research stage to the live trading phase.

Backtesting is great for quickly identifying the viability of a trading strategy. Forward testing creates the same hypothetical results but on new data. Forward testing takes more time but is done on the most recent data.

There is no need for a trader to choose one or the other as both can have their place in a proper strategy pipeline; however, forward testing is often not necessary if statistical significance can be obtained during the backtesting phase and rigorous robustness testing is applied.

Backtesting Portfolio

In financial markets, backtesting single trading strategies is a tremendous achievement and necessary for trading system developers. However, backtesting a portfolio of strategies goes one step farther.

Ray Dalio, founder of Bridgewater – the world’s largest Hedge Fund, released a compelling graphic demonstrating the math behind combining multiple strategies together.

 

Holy Grail Uncorrelated Strategies

 

The graphic shows the more uncorrelated strategies one can combine, the lower the probability of losing money in a given year. Wow! This epiphany is a true key to algo trading success. I wrote about it here: Algorithmic Trading Complete Guide.

That being said, it is important to have the ability or software to backtest portfolios of strategies not just single strategies. This was a key driver behind Build Alpha’s Portfolio Mode upgrade.

Advanced Backtesting Techniques

Due to the various pitfalls and brilliant minds in quantitative finance, various advanced techniques have been created to identify more reliable backtest results which hopefully lead to more sustainable live trading performance.

Out of Sample Testing (OOS)

Out of Sample testing is withholding some historical data for future testing – typically some percentage. First, develop and tweak your trading strategy on the In-Sample data. Once you have achieved desirable backtest results you can test your strategy on the withheld out of sample data.

 

Out of Sample Data

 

Ideally, performance results should be similar in both the in-sample and out-of-sample data periods. If trading performance fails on the out of sample data, then we likely avoided an overfit or poor strategy.

I wrote more exclusively about this test here: Out of Sample Testing

Randomized Out of Sample Testing

Selecting the proper out of sample period is a challenge to creating reliable backtest results. What if you select an out of sample period that contains only a bull market (like the image above)? The out of sample test may be easy to pass. Randomized out of sample testing can help avoid this.

Randomized Out of Sample testing randomly selects non-congruent dates from your historical data set to be part of the out of sample period.

 

Randomized Out of Sample Data

 

The trader can create 1,000 or more randomized out of sample periods then compare results on all of them. This helps overcome randomly selecting data points from a bull market only. Learn more about Randomized Out of Sample.

Lying Backtests. Are backtests enough?

Let’s discuss a simple case study that uses intermarket signals from the Japanese Yen to trade US 30-year bond futures. The strategy has only 2 rules for entry and produces fairly stable in-sample returns for being so simple (2003-2012 backtesting period).

 

Two rule intermarket US 30 year bond trading program through 2012

 

Graph shown shows results based trading 1 contract per signal for demonstration purposes

The intermarket strategy performed quite well on the unseen, out of sample data (2012-2016 OOS). The ability for a strategy to continue to perform on unseen data is paramount and mission critical for many system traders and strategy architects.

 

Two rule intermarket US 30 year bond trading program through 2016

 

Graph shown shows results based on 1 contract per signal for demonstration purposes

However, can we trust this backtest? In Build Alpha, there are many validation and robustness tests that can add confidence to your system development and backtesting process.

There is a special test called the “Noise Test” which creates 1,000 new price series by adding and subtracting varying amounts of noise (read volatility) from the historical data. The test then “re-trades” the strategy on the new 1,000 price series.

The idea is that if a strategy is overfit then it is overfit to the noise of the original data and is not trading the underlying signal we hope it is. Changing the noise will reduce performance if the strategy is overfit to the noise of the original data. Ideally, we want to see a strategy continue to perform well on the new, noise-adjusted price series.

 

Noise test results two rule intermarket US 30 year bond system

 

This “Noise Test” can help prevent a backtest from lying to us and hopefully prevent us from wasting further research time or taking a weak strategy live! You can see that our original backtest in blue is significantly better than the 1,000 “Noise Test” results. This is worrisome and most likely a sign of an overfit strategy aka a lying backtest that will surely fail on live data.

Viewing this simple strategy’s performance since 2016 confirms the value of the Noise Test. The strategy has been in a nosedive since going “live”, but a smart trader would have never taken it live – no matter how pretty the backtest – due to the failed Noise Test.

 

Live performance since noise test two rule intermarket US 30 year bond system

 

Any kind of trading – manual or discretionary – should be subjected to these rigorous tests before getting real money allocations. If not, you may be in for a surprise from a lying backtest or unreproducible hypothetical returns!

To read the full article, please check out The Noise Test and Lying Backtests

Best Backtesting Software

Build Alpha is the best backtesting software available. To manually backtest trading strategies is a time consuming and error prone process. Build Alpha enables the trader to test any and all ideas in a few clicks and requires no coding or programming. Build Alpha also offers the most advanced validation and robustness tests to help identify lying backtests as this has been the focal point of my professional trading career.

 

Build Alpha Dark Mode Color Option Display

 

Build Alpha’s genetic algorithm can take thousands of input signals, exit ideas, risk management rules, and more to find the best trading strategies for you. The robustness checks and tests give the ability to analyze any backtest result in depth before risking real capital. Portfolio mode allows for traders to combine strategies and backtest portfolios. To learn more head over to the Build Alpha homepage or sign up to see the demo.

Backtesting Need to Knows

  • Backtesting applies a trading strategy on historical data to view hypothetical results
  • Need data, a trading idea, software or coding to implement a backtest
  • Can backtest all financial instruments, any time period, any market condition
  • Use to identify poor strategies ahead of risking capital. A fast way of paper trading
  • Backtesting performance metrics can compare strategies or run scenario analysis
  • Positive backtest results are not a greenlight to trade a strategy; more testing required
  • Build Alpha makes automated backtesting and validation possible with no coding

Summary

Many traders implement trading strategies without testing them which often leads to depleted retail investor accounts. Backtesting enables traders to visualize historical performance of any trading strategy prior to risking real capital. Traders only need market data and a trading idea to conduct a proper backtest. A backtest can be used to weed out poor strategies, identify those worthy of future research, compare competing strategies, and more. Backtesting is a vital tool in any serious trader’s repertoire but is not a be all, end all to print money. Backtesting is a great first step, but other validation methods are needed before one should risk real capital.

Author

David Bergstrom Author Photo Trading Edge

David Bergstrom – the guy behind Build Alpha. I have spent a decade-plus in the professional trading world working as a market maker and quantitative strategy developer at a high frequency trading firm with a Chicago Mercantile Exchange (CME) seat, consulting for Hedge Funds, Commodity Trading Advisors (CTAs), Family Offices and Registered Investment Advisors (RIAs). I am a self-taught programmer utilizing C++, C# and python with a statistics background specializing in data science, machine learning and trading strategy development. I have been featured on Chatwithtraders.com, Bettersystemtrader.com, Desiretotrade.com, Quantocracy, Traderlife.com, Seeitmarket.com, Benzinga, TradeStation, NinjaTrader and more. Most of my experience has led me to a series of repeatable processes to find, create, test and implement algorithmic trading ideas in a robust manner. Build Alpha is the culmination of this process from start to finish. Please reach out to me directly at any time.

How to Build an Automated Trading System
In Excel, Python, and Build Alpha

It is no secret that many traders fail to achieve success and a level of consistency. The persistent traders will eventually understand they need to quantify what their edge is. This inevitably leads them down the road of systematic or quantitative trading but with no direction on how to begin. 

I am often asked how to build an automated trading system or how to create a trading algorithm or become a software trader. In this post, I will walk through testing a simple two rule system for the SP500 using Excel, Python and Build Alpha. The goal is to show how simple investigating quantifiable edges can be. 

I will use the same simple trading algo idea in all three platforms to hopefully show some algo trading 101 level steps in each platform. The strategy will buy when IBR (defined later) is less than 20 and the closing price is below the 10 period simple moving average. It will hold for 1 day and will not have any position sizing or risk management. Very simple to create.

We will also create visuals for the equity curve (P&L graph) and drawdown.

How to Create a Trading Algo in Excel

Starting with excel makes the most sense as the majority will be most familiar with it. I will share many screenshots and explain the steps one by one on how to do backtesting in excel. This is a simple example – no risk management stock trading excel example yet. So if you are an excel expert this may feel like a slow walk in the park – that is by design!

Step 1: Open the data file in Excel. Finance data is almost always going to be displayed in this format of Date, Time, Open, High, Low, Close, Volume (and Open Interest if futures or options). It is often referred to as OHLCV data. 

Excel0

Step 2: Calculate IBR or Interbar Rank. This is sometimes referred to as IBS or Interbar strength. It is a 1 period stochastics where we view where the close is in relation to the bar’s trading range. An IBR of 20% or lower would mean the close was in the lower 20% of the bar’s range.

The formula is simple and can be seen below. It is the difference between the close and the low of the bar divided by the range of the bar. Please note I added 0.0001 to the denominator to avoid any divide by 0 errors when/if the high and low are equal to each other. I have also multiplied IBR by 100 in order to express it as a percent.

After typing in the formula, drag this value down for all rows of data in order to calculate the IBR for each bar.

Step 3: Calculate the 10 period Simple Moving Average. Go down to the 10th bar and add the formula below. Drag this down for all rows of data to calculate the 10SMA for each bar.

Excel0

Step 4: Let’s add a new column and calculate our 10 period SMA signal. I have named this column Signal1. We will check if the current close is below the current SMA.

In order to check this, we can use an excel if statement. If the close is below the SMA return a 1 else return a 0. 

Excel3

Step 5: Adding another column named Signal2 in order to calculate our IBR signal. Again using an if statement to check if the current IBR is less than or equal to 20%. That is, if the current bar’s IBR is less than 20% return a 1 else return a 0.

Step 6: We need to combine our two signals into one signal. That is, whenever we close below the 10 SMA and have a weak close in the bottom 20% of the day’s range we want to return a 1 otherwise return a 0. A 1 would mean a green light to trade and a 0 would sit us on the sidelines as we have no edge.

All we need to do is check the sum of the previous two columns we created. If their sum is 2 then we return a 1 else we return a 0.

Step 7: Next let’s calculate the raw one day returns on the SP500. We have been using the closing price in all of our signals so we cannot truly know if our signal(s) are true until the close of the bar. Thus we cannot enter until the next bar’s open. 

Let’s assume we get a true signal on Monday’s close. We would then enter on Tuesday’s open and hold for one bar exiting on Wednesday’s open. We could exit on Tuesday’s close but for simplicity’s sake we won’t in this example.

Our excel function checks if we have data the next two days by checking if the opening price is non-zero and exists. If we have data, we can then subtract tomorrow’s open from from the open of two days from now. That is, take the difference of Wednesday and Tuesday’s opens per our example and store it in Monday’s row.

This gives us a data column that says, if we have a signal on this bar and buy the next open, hold for one day, what would our return be?

Step 8: Now let’s get our trade returns. If we have an actual signal in our ‘SignalFinal’ column from our two indicators what would our return be. 

I also multiplied by 50 to represent the point value of the SP500 emini futures contract. That is, if you buy at 3346 and sell at 3347 then you’ve earned $50 per contract not $1 per contract. Each futures contract has a point value and this one’s is 50.

Excel7

Step 9: Let’s copy our Date column and Trades column into a new tab. I clicked on the ‘A’ above ‘Date’ to highlight the entire column. I then hold ctrl and click on the ‘O’ above the ‘Trades’ column. This should highlight both columns.

Press ctrl + c to copy. Open a new tab and press ctrl + v to paste.

Step 10: Let’s do some house-keeping to remove the days we did not trade. We first need to sort by our Trade column and delete all days with “—” in the cell.

After deleting non-trade days, I then re-sort by date from oldest to newest.

To sort please highlight the columns, go to the data menu and select sort. 

Excel12
Excel0

Step 11: Time to create our P&L Graph. We need to add each trade value to a rolling sum of all trade values. The simple formula is below.

Drag this formula down to calculate the equity curve’s value after each trade.

Excel12

Step 12: In order to calculate the drawdown of our simple trading program we need to first calculate the high watermark or the rolling maximum amount our account would have achieved following this simple trading system.

Once we have the maximum or high watermark, we can then subtract our current P&L from the maximum to determine the current drawdown. I have added new columns for the maximum P&L (MAX) and the Drawdown (DD).

Excel0
Excel0

Step 13: Now we can plot our P&L Graph and Drawdown. Simply highlight the column, go to the Insert menu and select a line chart.

Excel0

How to Build a Trading Algo in Python

I have done a few other blogs on how to read a text file in python and using pandas but this example will show not only how to read in data, but how to complete all the steps above in one simple python script. The goal is building trading algorithms with python – or at least the first steps.

That way you have a very rudimentary framework for testing automated trading systems and creating trading algos in python. Hopefully this serves as an intro example of how to backtest a trading program or strategy.

Step 1: Create a new file, import our plotting library and pandas. Matplotlib is arguably the most popular python visualization library.

In the pandas read_csv call I have specified the file I’d like to read in, how to separate my columns, and what column should be the index of my pandas dataframe. 

Step 2: Let’s calculate our indicators. To calculate IBR we will use list comprehension which is a very cool python trick. It allows us to do a for loop over our entire dataframe all in one line of code. The calculation is the same as excel and will create a dataframe column named IBR as we had in excel.

We will then use the pandas rolling function to create our 10 period simple moving average (mean). I have saved our moving average values in a new dataframe column named ‘SMA’ similar to excel.

Step 3: Following similar steps as we did in excel, we can convert our indicators into actionable signals using ‘if’ logic. We will use python’s list comprehension feature which allows us to loop through our entire dataframe in a single line of code.

For instance, we will create a new dataframe column named Signal1 which will store either a 1 if IBR is less than 20 or a 0 if it is not less than 20.

Step 4: Similar to excel’s footsteps let’s continue creating our python trading algo by calculating the returns and then mapping our returns and signals into actual trade results.

First, we can use pandas built-in shift function to access tomorrow’s opening price and the opening price from two day’s from now. This is similar to how we calculated returns in excel.

To create the trade results, we need to know if there was a signal or not. We can multiply our SignalF column and our return column. If we have a signal we will have a trading return and if we do not have a signal (0) then the return will get zeroed out. I’ve also multiplied by 50 to account for the SP500 emini future contract’s point value to convert our Trades into actual dollar values for 1 contract.

Step 5: In excel we had to copy and paste our trade returns into a new tab, sort them, remove days we did not trade, etc. In Python we can simply create a new pandas dataframe, named Trades, and filter our original dataframe. We will simply return only the rows of our original dataframe where our signal was equal to 1. That is, only returning the values when we took a trade.

Step 6: To calculate our P&L and drawdown we can use the following two simple lines of code. Since our P&L is simply the cumulative sum of our trades we can easily just call the cumsum function on our Trades column. 

In order to calculate the drawdown we can just subtract our current P&L from the rolling cumulative maximum P&L. Python and pandas has a built-in cummax function we can use which will save us from creating an additional column to store our maximum values (high watermarks). This means one less column than excel.

Step 7: Plotting in python is simple. We already imported matplotlib – our plotting library – in the first step of this python walk through. Now we can call the pandas plot function, specify that we want to use subplots, and then display our plot with the show function.

How to Build a Trading Algo in Build Alpha

Step 1: Configure Build Alpha’s main screen. Set the symbol to ES which is the symbol for the SP500 emini futures contract.

Set the date range to start in 1997 and end near Sep 2020 to match the same data used in excel and python.

In the lower left set the max holding time to 1 bar in order to again match excel and python.

Step 2: Let’s select our signals. Type IBR into the ‘Filter’ search bar near the top. Scroll down and select the signal IBR <= 20 as an ‘Entry’.

Then type in SMA in the ‘Filter’ search bar near the top. Scroll down until you find the signal Close <= SMA(10). Select this signal as an ‘Entry’. 

Then hit Simulate!

Step 3: In the Results window you will view our two rule strategy at the top (highlighted in blue). Double click on the strategy to view the P&L graph. Toggle drawdown off and on by hitting the drawdown button.

What's Next?

Some logical next questions might be how can I add a stop? How can I add more signals? How can I trade automated strategies after I’ve tested in excel or python? I encourage you to try adding on to this simple trading program idea in both excel and python. Then there can be a greater appreciation for how simple Build Alpha can make things!

Want to add a stop? Click a button. Want to add a signal? Click a button.

I am often asked how to build automated trading systems or how to create automated trading systems in excel or what is the best automated trading software and what software do professional traders use, etc. This blog post and the rest of the Build Alpha blog can answer those questions. The information is out there, the tools are out there.

Trading is not easy, but it is simple. Hunt for edges, collect them, execute them.

If you want access to the python script or the excel sheet please send me an email at david@buildalpha.com.

Other Build Alpha Python Resources

Author

David Bergstrom Author Photo Trading Edge

David Bergstrom – the guy behind Build Alpha. I have spent a decade-plus in the professional trading world working as a market maker and quantitative strategy developer at a high frequency trading firm with a Chicago Mercantile Exchange (CME) seat, consulting for Hedge Funds, Commodity Trading Advisors (CTAs), Family Offices and Registered Investment Advisors (RIAs). I am a self-taught programmer utilizing C++, C# and python with a statistics background specializing in data science, machine learning and trading strategy development. I have been featured on Chatwithtraders.com, Bettersystemtrader.com, Desiretotrade.com, Quantocracy, Traderlife.com, Seeitmarket.com, Benzinga, TradeStation, NinjaTrader and more. Most of my experience has led me to a series of repeatable processes to find, create, test and implement algorithmic trading ideas in a robust manner. Build Alpha is the culmination of this process from start to finish. Please reach out to me directly at any time.

Intraday Edge: Find strategies backwards

The Goal of Intraday Trading

A large consideration of developing trading systems should be how efficient our capital is working for us. The quicker we can realize profits, the more trades we can make thus allowing our capital to compound more quickly. Additionally, sitting in positions for long periods increases our risk to extraneous events.

On the other hand, it is typically easier to find daily or higher timeframe edges than intraday edges due to the increased noise in intraday data.

From Daily to Intraday?

Is there a way to reduce the time in a position which would increase our trade count (via number of strategies) which would then allow us to arrive at the law of large numbers more quickly and therefore allow our capital to compound more quickly?

Yep. One of the new features in Build Alpha, called “Intraday Edge”, is a tool which allows us to do exactly that. Intraday Edge allows us to dig deeper into daily trading strategies to see if we can make them more efficient by reducing their holding times into smaller intraday time windows.

Maybe we can capture most of the daily strategy’s edge during only a small portion of the typical holding time. That’s right.. turning daily strategies into intraday strategies.

Intraday Edge Trading Example

First, let’s take an original daily trading system to examine the power of this new feature. I will use a simple one rule strategy that:

  • SP500 Futures Contract
  • Long Only
  • Buy when close in bottom 20% of day’s range (IBR or internal bar strength in Build Alpha)
  • Exit 1 day later

This assumes about a 23 hour risk (i.e., one Globex trading session).

Intraday Improvement

However, what if we could dig into this strategy and realize that most of the gains only come from 1 am EST to 4 am EST? We can then reduce our holding time by about 87% which now only ties up our capital for 3 hours as opposed to 23! This gives us an additional 20 hours to utilize other strategies to continue to grow our capital while still capturing a large portion of the original daily strategy’s edge.

Imagine we only had enough capital for one strategy. This Intraday Edge feature can now make our capital work much harder by finding intraday edge strategies for multiple markets/times of the day. Tying up capital for 23 hours in one daily strategy vs. trading 7 different intraday edge strategies with the same capital.

*Original strategy can be reduced by Intraday Edge which allows other intraday strategies to be traded with the same capital that was orignially tied up by the daily strategy*

In the end, it makes our once daily system much more efficient. Check out the performance metrics of the original daily system compared to the new “Intraday Edge” version.

*Daily Strategy*
*Intraday Edge*

Steps in Build Alpha

So how can this be accomplished in Build Alpha? It is simple.

  1. Highlight any daily strategy
  2. Click the Test Settings in the bottom right to configure the intraday timeframe you want to use
  3. Hit the Intraday Edge button

Build Alpha will then search all possible holding periods within the original strategy’s trading duration to see if there is a more efficient version with reduced holding times. You can include the original strategy’s exit criteria such as stops, etc. or choose to exclude them. Flexibility to test everything is always key in Build Alpha.

Additional Uses of Intraday Edge Feature

Intraday Edge can even be used on different markets at the same time. For example, imagine an original system built on Gold daily bars but then we search for an intraday edge version that trades oil but only during this specific 2 hour window while the original Gold System has an active signal.

This Intraday Edge feature essentially allows us to search for intraday and multi-timeframe strategies in a new way. In this above Gold and Oil example we have a multi-timeframe AND intermarket strategy created from a simple Gold daily strategy.

You can still search for multi-timeframe and intraday strategies in the original/traditional way. That is, just searching the intraday data from the start. However, it is often faster and easier to find daily strategies then work them into intraday ones. At least now with Build Alpha you have the option to search both ways. Something not possible elsewhere.

Automating Intraday Edge Trading Strategies

And of course, all of the adjustments from the Intraday Edge feature are then applied to the code generators so you can automate these Intraday Edge systems with one click as with everything.

As always, I will keep attempting to add flexibility and ways to dig deeper so we can have the best trading strategies possible. Leave no stone unturned and test everything!

Thanks for reading,

David

Noise Test Parameter Optimization

In short, this is a new feature that allows us to optimize strategies across noise adjusted data series as opposed to the traditional method of optimization which only optimizes across the single historical price series. Here is a reminder on the Noise Test.

The problem we face is the historical data is merely only one possible path of what *could* have happened. We need to prepare ourselves for the probable future not the certain past. In order to do this, we can generate synthetic price series that have altered amounts of noise/volatility than the actual historical data. This provides us with a rough sample of some alternate realities and potentially what can happen going forward. This is the exact type of data that can help us build more robust strategies that can succeed across whatever the market throws at us – which is our end goal in all of this, right?

Noise Test Trading Strategy Example

Let’s look at a Noise Test Parameter Optimization (NTO) case study to show exactly how it works…

I have built a strategy from 2004 to 2016 that does quite well. The strategy’s performance over this period is shown below…

BuildAlpha NTOoriginal

Now, if we right click on the strategy and select optimize, we can generate a sensitivity graph that shows how our strategy performs as we alter some parameters. This is done on the original historical price data with no noise adjusted data samples added (yet).  We simply retrade different variations of parameter settings on the single historical price data and plot the respective performances. This is how most platforms allow you to optimize parameters and I want to show how misleading it can be to traders. The rule I’ve optimized had original parameter values of X = 9 and Y = 4 (black arrow). The sensitivity graph is shown below. Each plot consists of three points: parameter 1, parameter 2 and the resulting profit.

BuildAlpha Original Annotated Optimization

We can see the original parameters are near a “sensitive area” on the surface where performance degrades in the surrounding areas. Performance drops pretty hard near our original strategy’s parameters which means slight alterations to the future price data’s characteristics can degrade our strategy’s performance quite a bit. Not what we want at all and, as we all know, there will be alterations to future price’s characteristics! How many times has a backtest not matched live results? Perhaps more robust parameter selection can help…

Parameter Selection

The more robust selection using the typical simple optimization method on the historical data shows we should probably pick a parameter more near X = 8 and Y = 8 (pictured arrow below). This is the traditional method taught in textbooks, trading blogs, etc. We optimize on the single historical data then find a flat/non-peaked area close to our original parameters and use those new parameters.

BuildAlpha Where We Should Have Been

NTO Parameter Selection

However, if we run Build Alpha’s Noise Test Optimization with up to 50% noise alterations and 50 data samples (green box below), we see a much different picture. What this does is, instead of optimizing on one historical path we now optimize across the one historical path AND 50 noise altered data series. The sensitivity graph shows a much different picture when optimized across the 51 data series. We are less concerned with the total profit and loss but rather the shape of the surface…

BuildAlpha NTO Improves

The Noise Test Optimization (NTO) shows we should actually pick values of X = 8 and Y = 2 (arrow above). This area actually outperforms the ‘robust’ area from the traditional simple one data set optimization that most other platforms and textbooks show. In other words, I am comfortable picking a peaked area of the surface from the NTO results because the peak is true across noise adjusted values and not just a single price series. Secondly, if we fall off this peak then we are still at (or near) the highest level relative to the rest of the surface.

Forward Results

Looking at how all three of these parameter selections performed from 2016-2020 can be quite telling…

BuildAlpha Original Equity Curve
*Original Parameters 2016-2020*
BuildAlpha Traditional Equity Curve
*Textbook Parameter Selection 2016-2020*
*NTO Parameters 2016-2020*

The Noise Test Optimization (NTO) returned about $20,000 more profit per contract than the other two methods in this hypothetical case study or about 33% more profit over the same 2016-2020 period. These small adjustments to commonly misused tests can really make a large difference when spread across a portfolio of strategies. It is time to stop using outdated tools. I am not saying each strategy will show NTO results like this, but failing to check is a compounding mistake few of us can afford to make.

Build Alpha also possesses the ability to optimize parameters across a basket of symbols (and their noise adjusted data as well). For example, build a strategy for SPY but optimize it across SPY, QQQ, IWM and DIA.

As always, thanks for reading

David