financialnoob.me

Blog about quantitative finance

Pairs trading with Markov regime switching

In this article I am going to test a pairs trading strategy based on the paper ‘A regime-switching relative value arbitrage rule’ (Bock and Mestel, 2009)It describes how to use Markov regime switching models to identify trading signals in pairs trading strategies.


The main reasoning behind this idea is the following. Sometimes historic relationship between two stocks in a pair can break (temporarily) and their spread can deviate significantly from its historical equilibrium leading to a new equilibrium state. It might happen because of different reasons. For example, one company in a pair might release its quarterly financial reports before another, which can cause temporary widening of the spread. Another reason for temporary regime changes may be financial crisis, period of economic recession or market crash (e.g. stock market crash of 2020 which happened because of pandemic).

Traditional pairs trading strategies are unable to detect such ‘regime changes’ which can lead to big losses. Using Markov regime switching models allows us to detect such changes and adapt our trading rules accordingly.

Let’s take a look at the proposed model. Authors assume that the current regime is determined by an unobserved (hidden) variable. This variable is assumed to follow Markov chain process. To model that process authors propose using two-state, first-order Markov switching process with switching mean and switching variance. Two-state means that we assume that there are only two possible regimes. First-order means that the probability of the next state depends only on the current state and is independent of all previous states.

The spread is constructed as a price ratio of the two stocks in the pair (stock A and stock B):

Constructing the spread

It is modeled with the stochastic process below:

Spread model

The mean of the spread and its variance are determined by the current state (s_t). Below you can see a plot from the paper thta illustrates this idea. The green shaded area shows the regime with higher mean (mu_1) and white non-shaded area shows the regime with lower mean (mu_2).

Spread with two different regimes

To determine the current state we use transition probabilities calculated as follows:

Transition probabilities

Parameters (p_0, q_0) are calculated when we fit the model on historical data. After fitting the model we can calculate the probability of the spread being in one of the two states (low-mean state or high-mean state). Luckily for us statsmodels library has a function for creating and fitting Markov switching models so we don’t need to implement it from scratch.

The trading rules are different depending on the current regime. In the low mean regime we have:

Trading rules (low mean regime)

In the high mean regime:

Trading rules (high mean regime)

Parameter delta above determines the standard deviation sensitivity (how far from the mean should the spread be for us to open a position). Parameter rho is the probability threshold (how sure we are in the current regime).

Now we are ready to implement and backtest this strategy.


I am going to use the same stock universe I used in the previous article: constituents of Vanguard Small-Cap Value ETF (ticker: VBR) from 2018–11–20 to 2021–11–19. First 30 months will be used as a formation period to select stock pairs and the last 6 months will be used for trading.

In the paper authors use Augmented Dickey-Fuller test to select stock pairs that are cointegrated. I am going to use similar approach, but apply it to pairs with the highest Kendall’s tau correlation coefficient. I already calculated Kendall’s tau for all possible pairs in the stock universe. Now we only need to apply ADF-test to selected cointegrated pairs. The code for pair selection is provided below.

Selected pairs

I’ve selected 25 pairs, but later I will try using different (smaller) numbers of pairs for trading.

Let’s try to experiment with Markov switching model on just one of the pairs. I will use CRF-ASB pair. First let’s construct and plot the spread. Here I am using all available data (formation and trading periods).

CFR-ASB pair spread

We can clearly see that starting approximately in March 2020 the behavior of the spread changes – its mean and standard deviation increase. Now I will create and fit a Markov switching model. It is done easily with three lines of code.

Parameter search_reps determines the number of different random perturbations in the starting parameter vector. Because the search is random, it is not always possible to find a good solution with just one repetition.

Model summary

The model summary is shown above. We can find parameters of the two regimes (const — mean, sigma2 — variance) as well as transition probabilities. Regime 0 has higher mean and regime 1 has lower mean. Now I will plot the spread price along with the probability of high-mean regime. Code for it is provided below.

Spread price and probability of high-mean regime

Dotted lines on the plot above represent spread mean values in high-mean and low-mean regimes. It seems that the model works as expected and is able to identify different regimes. Now we are ready to perform a backtest.


First thing I want to do is to fit Markov switching model for each pair and each trading day using 250 days rolling window. After fitting the model I will save regime parameters and transition probabilities in a dataframe. I will then use this dataframe to calculate positions. Code for doing it is provided below.

Last two lines are used to the results in a file. Instead of running this procedure you can just load it from the file provided here.

Note that on lines 17–22 I use a while loop to fit a model. The reason for this is that sometimes the fitting algorithm fails to converge in 100 repetitions and it needs to be run again.

Now let’s perform the actual backtest. Rules for opening positions were provided above. We need to figure out what values to use for standard deviation sensitivity (delta) and probability threshold (rho). In the paper authors suggest using delta=1.645 and rho between 0.6 and 0.7. That’s what I will use. In the code below each pair is processed separately and results are saved in a dataframe. That way we can experiment with a number of traded pairs and see how it affects the overall performance. I don’t use any stop-losses and the selected pairs are kept constant for the whole trading period (six months).

After that we can calculate and plot cumulative returns. I will calculate total returns using different number of traded pairs to see how it affects the total performance.

Plot of cumulative returns

As we can see on the plot above using 5 pairs gives the worst performance of all. All the other tested options perform approximately the same and their performance is not that good, especially considering that I didn’t account for any transaction costs. Let’s look at the same plot along with the cumulative returns of VBR ETF.

Plot of cumulative returns (algorithm and VBR ETF)

Well, there is at least one advantage of our algorithm compared to the ETF — its returns are a lot less volatile (but I think they are two small anyway). Finally let’s look at some of the performance metrics.

Performance metrics

According to Sharpe ratio our algorithm outperforms VBR ETF, but its returns are quite small and I want to remind you again that I didn’t include any transaction costs.

It seems that this strategy doesn’t perform that well on this particular dataset. However I think it can still be useful. Here are some ideas to try:

  • Use different size of the rolling window (maybe in modern dynamic markets 250 days is too long).
  • Try more complex Markov switching model (add more regimes, add additional time series, time-varying transition probabilitites).
  • Try using higher frequency data.
  • Use it as a part of another strategy to detect regime changes.
  • Try experimenting with parameters rho and delta.

Jupyter notebook with source code is available here.

If you have any questions, suggestions or corrections please post them in the comments. Thanks for reading.


UPDATE 25.04.22: There was a small mistake in calculating returns. I fixed it now and updated the article and corresponding jupyter notebook.

Leave a Reply

Your email address will not be published. Required fields are marked *