Home > R > Regime Detection

## Regime Detection

Regime Detection comes handy when you are trying to decide which strategy to deploy. For example there are periods (regimes) when Trend Following strategies work better and there are periods when Mean Reversion strategies work better. Today I want to show you one way to detect market Regimes.

To detect market Regimes, I will fit a Hidden Markov Regime Switching Model on the set of simulated data (i.e. Bull / Bear market environments) I will use the excellent example from the Markov Regime Switching Models in MATLAB post and adapt it to R.

The idea behind using the Regime Switching Models to identify market states is that market returns might have been drawn from 2 or more distinct distributions. As a base case, for example, we may suppose that market returns are samples from one normal distribution N(mu, sigma) i.e.

```Returns = mu + e, e ~ N(0, sigma)
```

Next we may suppose that market returns are samples from two normal distributions (i.e. returns during Bull market may be ~ N(mu.Bull, sigma.Bull) and returns during Bear market may be N(mu.Bear , sigma.Bear) i.e.

```Returns = mu + e, e ~ N(0, sigma)
mu = mu.Bull for Bull regime and mu.Bear for Bear regime and
sigma= sigma.Bull for Bull regime and sigma.Bear for Bear regime
```

Fortunately we do not have to fit regimes by hand, there is the RHmm package for Hidden Markov Models at CRAN that uses the Baum-Welch algorithm to fit Hidden Markov Models.

Next, let follow the steps from the Markov Regime Switching Models in MATLAB post.

```###############################################################################
# Load Systematic Investor Toolbox (SIT)
# https://systematicinvestor.wordpress.com/systematic-investor-toolbox/
###############################################################################
setInternet2(TRUE)
con = gzcon(url('http://www.systematicportfolio.com/sit.gz', 'rb'))
source(con)
close(con)

#*****************************************************************
# Generate data as in the post
#******************************************************************
bull1 = rnorm( 100, 0.10, 0.15 )
bear  = rnorm( 100, -0.01, 0.20 )
bull2 = rnorm( 100, 0.10, 0.15 )
true.states = c(rep(1,100),rep(2,100),rep(1,100))
returns = c( bull1, bear,  bull2 )

# find regimes

y=returns
ResFit = HMMFit(y, nStates=2)
VitPath = viterbi(ResFit, y)

#Forward-backward procedure, compute probabilities
fb = forwardBackward(ResFit, y)

# Plot probabilities and implied states
layout(1:2)
plot(VitPath\$states, type='s', main='Implied States', xlab='', ylab='State')

matplot(fb\$Gamma, type='l', main='Smoothed Probabilities', ylab='Probability')
legend(x='topright', c('State1','State2'),  fill=1:2, bty='n')
``` The first chart shows states (1/2) determined by the model. The second chart shows the probability of being in each state.

Next, let’s generate some additional data and see if the model is able to identify the regimes

```	#*****************************************************************
# Add some data and see if the model is able to identify the regimes
#******************************************************************
bear2  = rnorm( 100, -0.01, 0.20 )
bull3 = rnorm( 100, 0.10, 0.10 )
bear3  = rnorm( 100, -0.01, 0.25 )
y = c( bull1, bear,  bull2, bear2, bull3, bear3 )
VitPath = viterbi(ResFit, y)\$states

#*****************************************************************
# Plot regimes
#******************************************************************
data = xts(y, as.Date(1:len(y)))

layout(1:3)
plota(data, type='h', plotX=F, x.highlight=T)
plota.legend('Returns + True Regimes')
plota(cumprod(1+data/100), type='l', plotX=F, x.highlight=T)
plota.legend('Equity + True Regimes')

plota(data, type='h', x.highlight=T)
plota.legend('Returns + Detected Regimes')
``` The first 300 observations were used to calibrate this model, the next 300 observations were used to see how the model can describe the new infromation. This model does relatively well in our toy example.

To view the complete source code for this example, please have a look at the bt.regime.detection.test() function in bt.test.r at github.

Categories: R
1. November 1, 2012 at 6:41 am

Very nice example, in which you define a two-regime switching, is there a way to detect the number of regimes for given data. Thanks as usual.

2. November 2, 2012 at 7:02 am

I think you can try mixture of distributions approach to try to partition your data. Otherwise you will have to specify desired number of states apriori.

3. November 3, 2012 at 6:54 am

Hello,
Excellent how Markov process is quickly modeled in R. I hope you’ll expand on this by taking a real example, e.g. SPY or ^GSPC. We already know that returns are generally not normal but would be interesting to look at the output.

Many thanks.

4. January 28, 2013 at 3:03 pm

Is there any out sample testing about this? The more appropriate question to ask is how can you know which regime you are currently in right now and whehter a shift is coming?

1. November 14, 2012 at 11:57 pm
2. January 28, 2013 at 4:02 am
3. September 7, 2014 at 8:32 pm