A major fear of system trading is "over-fitting" the data. That is, finding a trading system that too accurately fits the historical data and will not do well on new, live data.
A key test to find systems that generalize well on all data is to test ideas across markets. The idea is if it works well on a few markets then the strategy generalizes well enough and the trader/money manager can have confidence the trading rules are not too specific to the individual market's data but will work on new data and other markets as well.
Alpha Spotter allows the trader/money manager to compare the generated strategies vs. strategies generated with random signals. This feature allows the trader/money manager to discern whether the generated strategies were found by random chance (curve-fit) or actually possess some edge greater than could be found by random chance.
The dark blue line is the specific strategies equity curve. The blue lines are the top 100 strategies from the real signals. The red lines are the equities curves from the random strategies.
The trader/money manager has the ability to turn off and on the plotting of the real strategies AND the random strategies with the click of a button.
The photo on the right shows a clear edge over random - demonstrating this set of signals may possess real predictive power.
Edge Ratio or E-Ratio measures how much a trade goes in your favor vs. how much a trade goes against you. The x-axis is the number of bars since the trading signal. A higher y value signifies more "edge" at that step in time.
Measurements are normalized for volatility with 1 being the baseline. That is, the y axis is an expression of how many units of volatility more or against you your trade gets. A measure of 1.2 would indicate .2 units more of favorable volatility and a measure of 0.8 would indicate .2 units more of adverse movement.
The blue line is for the selected strategy's signal and the red line is for a "random" strategy for the same market. The red line is to serve as a baseline to beat. Build Alpha may find good strategies, but have an E-Ratio less than the red baseline then we can be less confident that it will withstand the test of time.
Another tool to make sure Build Alpha + Trader = Success.
For a more detailed explanation (with calculation example) check out this blog post: Build Alpha - Edge Ratio
The noise test is a proprietary test that adds or subtracts user selected amounts of "noise" to the original price data. The software then re-trades the selected strategy on the 1,000 newly generated price series that have varying amounts of noise.
The idea is to see if we change the amount of noise in the data does the strategy still perform well or does it fall off a cliff?
The trader or money manager is granted the freedom to control how much noise is added and how frequently price bars are adjusted. An example may be... let's adjust 70% of the price data by 20%. So in this example, we'd adjust 70% of the open,high,low,close prices by 20% (in either direction) by up to 20% of the average true range on that date.
The user can then reconfigure the settings and re-run this test in an instant.
A Meta-System is simply a trading system that attempts to trade or time another trading system. That is, can we improve performance by analyzing the patterns in the trading results.
For example, can we achieve smoother risk-adjusted returns by only trading this strategy after the last trade was a loser? How about if the last two trades were losers? What if we only traded this strategy when its equity curve was below the 5 trade moving average of the equity curve?
You can see above we can achieve smoother risk-adjusted returns while only trading this strategy once its equity curve falls below the 5 trade moving average of the equity curve (the pink line). This might warrant further research or increased leverage/position size when the strategy's equity curve begins to dip, for example.
Randomized Out of Sample testing is a simple check that a strong trend in our out of sample test period did not contribute too greatly to our favorable out of sample results. For example, it may be very easy for a strategy to pass out of sample tests if the underlying price data exhibits a very strong up trend.
Randomized out of sample testing runs 1,000 tests where each iteration selects random, non-continuous dates to serve as our out of sample period. This ensures we do not have a strong trend or pattern in our out of sample period that allowed a weak strategy to pass out of sample testing. It also allows us to view a distribution of outcomes vs. a single backtest's out of sample results.
For a more specific and detailed explanation of this testing please view this post: Randomized Out of Sample
Minimum Variance Portfolio or Modern Portfolio Theory is a simple test that generates optimal weights to give each strategy inside the portfolio in order to optimize the portfolio's risk-adjusted return or Sharpe Ratio.
The trader or money manager can scroll over the points on the graph and view the individual weights given to each strategy to create that specific point.
The highlighted red point is the result of using even weights on each strategy.
Build Alpha now highlights out of sample period to give a clear view of the performance differences in both in and out of sample testing.
The graph on the left goes long the S&P500 when the market enters into a bullish regime. When the market enters into the bearish regime most trading systems go flat.
However, Build Alpha allows you to test the same trading idea but invest in a second market instead of being flat. The idea being.. this can often lead to more profits, reduced drawdown, and an over smoother return on capital.
The graph on the right shows the equity curve when going long the S&P500 when the market enters into the bullish regime and going long US 10 Year Note Futures when the market enters into the bearish regime.
No other software or platform allows users to create market regime switching strategies.
View the strategy's drawdown in comparison to buy and hold's drawdown. No point in trading a strategy unless it offers better risk-adjusted returns than simply parking the money in some asset.