Genetic Optimization in TradeStation

Automated back testing can be a great time saver for us traders.

Although an automated test will miss many factors that are in play during live trading, such as news event, significant support/resistance levels, session highs and lows.

It also helps us narrow down the universe of markets and time frames that will work best with our system.

Treat it as a prospecting tool to identify your best candidates and then perform manual back tests on the best of those to come up with more accurate results.

Exhaustive Optimization

When we use automated back testing one of the first decisions we are faced with is whether to optimize input variables one at a time or in combination. Testing them one at a time helps us identify ranges of profitability in the variable being tested.

For example if we test the length of a moving average between 9 and 36 we may find that the range of values between 15 and 23 all give us a good results.

This indicates a robust performance in this range and may lead us to select the midpoint of the range.

After repeating this process for all the variables under consideration we arrive at a solution that should prove stable.

Testing the input variables one at a time also greatly reduces the total time needed by the computer to complete the test. I ran a sample back test with five separate input variables and the automation only had to run about fifty tests in total.

The test was run on six months of 377 tick data in the crude oil continuous contract.

The risk with this approach is that complex interactions between the input variables may not be factored into the testing. For example if we test a moving average crossover the individual results may indicate stable results when the first moving average length is between 15 and 23 and the second moving average length is between 70 and 85.

When we test the two variables in combination however we may find that the system has best overall results when the moving average lengths range between 28 and 32 for the first and 45 and 50 for the second.

Testing the variables individually would have completely missed this convergence zone.

The drawback of testing the input variables in combination however is that the number of tests performed by the computer increases exponentially. In the five input test described earlier I found that the total number of tests increased from fifty to over 21,000 with a commensurate increase in the time it took to run the tests.

Below you’ll see the “Format Analysis Techniques & Strategies” window with its default optimization method of Exhaustive.

Default Optimization Method – Exhaustive

The next chart shows the beginning of the optimization run. Note that the time to complete all the tests is nearly five hours. This of course will vary depending on your computer.

Duration Estimate for Exhaustive Test

To address the problem of exhaustive simultaneous testing of multiple variables we can take a look at genetic optimization.

Genetic Optimization

Genetic algorithms have been around for decades now and have been used to solve complex problems in the sciences.

These algorithms are based on the concept of inheritance, mutation and evolution. An initial set of testing chromosomes are generated randomly, then they are tested over a number of generations and the result of succeeding generations is culled so that only the best set of chromosomes survive by the last generation.

Let’s see how this applies to genetic optimization of a trading system test:

• The chromosomes are the combination of input variables to be tested. In our example we are testing five input variables so each chromosome has five elements.
• We define our fitness function. This is the result that we want to optimize and could be net profit, profit factor, average net trade, return on capital, etc. To set the fitness function click on the View menu, select “Chart Analysis Preferences” and go to the Strategy tab.
• We randomly create an initial set of chromosomes by assigning random values to each of the five elements. Each of these values is taken from the optimization range we set for the input.
• Each chromosome is tested and the next generation of chromosomes is created by selecting those that best fit the fitness function and combining them to create children for the next generation to be tested. The chromosomes with the worst results are culled from the population and do not reproduce into the next generation.
• Mutations are introduced by randomly changing the values for some of the children chromosomes.
• The process is repeated for each new generation until either the last generation is reached or no further improvements are found for a predefined number of generations. The TradeStation default, if selected, is to end the optimizations if no further improvement is found after 15 generations.

The above is a bit of a simplification of course. The TradeStation literature explains that during the optimization areas of n-dimensional stability are also identified so that the final optimization result is not a single peak of profitability that is purely curve fitted.

You can read more about it in this article.

To use genetic optimization you first change the optimization method in “Format Analysis Techniques & Strategies”:

Select Genetic Optimization

Next we need to initialize the genetic optimization methods. The easiest way to do this is to click the “Advanced Settings…” button to open the corresponding window and then clicking on the Suggest button as seen below.

Press “Suggest” to Initialize Genetic Optimization Settings

You are now ready to start the genetic optimization. You’ll notice that the optimization status display is slightly different. You’re presented with two values for the test.

The first indicates which generation is currently being tested and the second shows which of the chromosomes within that generation is currently being evaluated.

The first thing that stands out after a few tests is that the initial estimate for completion time is half of what it was for the exhaustive test.

Initial Genetic Optimization Estimate

This still seems like a long time but what you’ll find as the genetic optimization continues is that the progress accelerates. It accelerates because as the generations progress the chromosomes start to converge around an optimal set of values and fewer tests are needed to progress to the next generation.

The next picture shows that half way through the generations the total time estimate for the genetic optimization run has already been halved.

Genetic Optimization Estimate at Midpoint

In the end the genetic optimization run took only 44 minutes. I’ve run other more dramatic tests. One particularly complex test with numerous input variables had an exhaustive test estimate of 3 ½ days. Using the genetic optimization it completed in less than two hours.

Is This the Holy Grail?

Genetic optimization is a very efficient method of simultaneously testing a large number of input variables. Based on shorter tests that I’ve run its results are usually very similar to the exhaustive optimization results.

Unfortunately it shares the same problem that all multi variable tests have and that is curve fitting. The results are optimized for the six months of 377 tick data covered in my test but if that data were slightly different then the optimal result would also have been different.

Given that market conditions change based on news events and seasonality this can easily result in disappointing forward testing results.

There are a few things that you can do to reduce the risk of curve fitting:

• Exclude a portion of the available test data from the optimization. A good rule of thumb is to only perform the optimization on two thirds of the data and leave the remaining third for the out of sample test. There is an easy way to do this in the Advanced Settings window for the optimization (see the picture above). You can specify a percentage of data to exclude either at the start or end of the sample or you can simply specify a date range to exclude from the optimization. The easiest way to do this is to use the Suggest button in the Out-of-Sample settings.
• Optimize over short time frames and perform rolling optimizations every month or every couple of months. This will tune your system based on the recent performance of the market. We all know that news and seasonality affect markets. It is unreasonable to expect similar performance during the end of year holiday season as during the spring or in the aftermath of major news that directly impact your market.
• Combine both of the following approaches and perform a walk forward optimization. A series of optimizations is performed over an in sample period and both in sample and out of sample results are recorded and evaluated. The test period is rolled forward and the optimization is repeated until the sample data is exhausted. This approach gives the best picture of how the strategy will perform with live data over a period of time. TradeStation has a walk forward optimizer built in and we will cover this approach in a future article.
• Perform a Monte Carlo simulation. In a Monte Carlo simulation not only are the input variables tested but the underlying test data is also randomly altered (within constraints). Data drift and randomness are added to the data to reflect the uncertainty of future market prices. Multiple tests are run using slightly altered test data so that the risk of fitting the results to a particular set of data values is greatly reduced. TradeStation has Monte Carlo analysis built into its walk-forward optimizer, but again that is a topic for a future article.

Back testing is fundamental to developing a profitable trading strategy and we need to use the tools at our disposal to facilitate that process. Automated back testing is one of those tools but it does have its limitations as we’ve seen.

Understanding these tools therefore helps us be more effective traders. We’ll conclude this two part series with a discussion of the TradeStation walk forward optimizer in a future article.