Noise Test Parameter Optimization
In short, this is a new feature that allows us to optimize strategies across noise adjusted data series as opposed to the traditional method of optimization which only optimizes across the single historical price series. Here is a reminder on the Noise Test.
The problem we face is the historical data is merely only one possible path of what *could* have happened. We need to prepare ourselves for the probable future not the certain past. In order to do this, we can generate synthetic price series that have altered amounts of noise/volatility than the actual historical data. This provides us with a rough sample of some alternate realities and potentially what can happen going forward. This is the exact type of data that can help us build more robust strategies that can succeed across whatever the market throws at us – which is our end goal in all of this, right?
Noise Test Trading Strategy Example
Let’s look at a Noise Test Parameter Optimization (NTO) case study to show exactly how it works…
I have built a strategy from 2004 to 2016 that does quite well. The strategy’s performance over this period is shown below…
Now, if we right click on the strategy and select optimize, we can generate a sensitivity graph that shows how our strategy performs as we alter some parameters. This is done on the original historical price data with no noise adjusted data samples added (yet). We simply retrade different variations of parameter settings on the single historical price data and plot the respective performances. This is how most platforms allow you to optimize parameters and I want to show how misleading it can be to traders. The rule I’ve optimized had original parameter values of X = 9 and Y = 4 (black arrow). The sensitivity graph is shown below. Each plot consists of three points: parameter 1, parameter 2 and the resulting profit.
We can see the original parameters are near a “sensitive area” on the surface where performance degrades in the surrounding areas. Performance drops pretty hard near our original strategy’s parameters which means slight alterations to the future price data’s characteristics can degrade our strategy’s performance quite a bit. Not what we want at all and, as we all know, there will be alterations to future price’s characteristics! How many times has a backtest not matched live results? Perhaps more robust parameter selection can help…
Parameter Selection
The more robust selection using the typical simple optimization method on the historical data shows we should probably pick a parameter more near X = 8 and Y = 8 (pictured arrow below). This is the traditional method taught in textbooks, trading blogs, etc. We optimize on the single historical data then find a flat/non-peaked area close to our original parameters and use those new parameters.
NTO Parameter Selection
However, if we run Build Alpha’s Noise Test Optimization with up to 50% noise alterations and 50 data samples (green box below), we see a much different picture. What this does is, instead of optimizing on one historical path we now optimize across the one historical path AND 50 noise altered data series. The sensitivity graph shows a much different picture when optimized across the 51 data series. We are less concerned with the total profit and loss but rather the shape of the surface…
The Noise Test Optimization (NTO) shows we should actually pick values of X = 8 and Y = 2 (arrow above). This area actually outperforms the ‘robust’ area from the traditional simple one data set optimization that most other platforms and textbooks show. In other words, I am comfortable picking a peaked area of the surface from the NTO results because the peak is true across noise adjusted values and not just a single price series. Secondly, if we fall off this peak then we are still at (or near) the highest level relative to the rest of the surface.
Forward Results
Looking at how all three of these parameter selections performed from 2016-2020 can be quite telling…
The Noise Test Optimization (NTO) returned about $20,000 more profit per contract than the other two methods in this hypothetical case study or about 33% more profit over the same 2016-2020 period. These small adjustments to commonly misused tests can really make a large difference when spread across a portfolio of strategies. It is time to stop using outdated tools. I am not saying each strategy will show NTO results like this, but failing to check is a compounding mistake few of us can afford to make.
Build Alpha also possesses the ability to optimize parameters across a basket of symbols (and their noise adjusted data as well). For example, build a strategy for SPY but optimize it across SPY, QQQ, IWM and DIA.
As always, thanks for reading
Very good study.
Thank you, sir!
Hi Dave,
Longtime fan, first time writer. Keep overcoming the randomness man!
Couple of questions.
Clarification
1-You say the first sensitivity graph is “no noise adjusted” yet the Noise area of the optimization show 20% for OHLC & Max. Are those 20% simply nullified because the Samples is set to 0?
Curious
2-In the NTO application, in an individual sample (of the 50), is the noise % (<=50%) for that run applied equally to every bar throughout or does it vary from bar to bar (ex. one bar high is “noised” 25%, while the next bar high is “noised” 49%)?
Hey,
Appreciate the support and the read!
1. Yes, exactly. Setting samples to 0 will not generate any noise adjusted data.
2. It varies from bar to bar. I’ve experimented with a few methods including varying it bar by bar, by blocks, uniform, etc. Current (and best implementation) is bar to bar.
Thanks,
David