Archive
Weekend Reading – Gold in October
I recently came across the “An early Halloween for gold traders” article by Mark Hulbert. I have discussed this type of seasonality analysis in my presentation at R/Finance this year.
It is very easy to run the seasonality analysis using the Systematic Investor Toolbox.
###############################################################################
# Load Systematic Investor Toolbox (SIT)
# https://systematicinvestor.wordpress.com/systematic-investor-toolbox/
###############################################################################
setInternet2(TRUE)
con = gzcon(url('http://www.systematicportfolio.com/sit.gz', 'rb'))
source(con)
close(con)
#*****************************************************************
# Load historical data
#******************************************************************
load.packages('quantmod')
ticker = 'GLD'
data = getSymbols(ticker, src = 'yahoo', from = '1970-01-01', auto.assign = F)
data = adjustOHLC(data, use.Adjusted=T)
#*****************************************************************
# Look at the Month of the Year Seasonality
#******************************************************************
month.year.seasonality(data, ticker)
This confirms that October have been historically bad for Gold, but we used only 8 years of history because GLD only started traded in 2004.
To get a more complete picture, there is a long history of Gold prices at the Bundes Bank. I found this data source used at the Wikiposit.
I created a helper function bundes.bank.data.gold() function in data.r at github to download prices from the Bundes Bank site.
#*****************************************************************
# Load long series of gold prices from Bundes Bank
#******************************************************************
data = bundes.bank.data.gold()
#*****************************************************************
# Look at the Month of the Year Seasonality
#******************************************************************
month.year.seasonality(data, 'GOLD', lookback.len = nrow(data))
The October have been historically bad for Gold using longer time series as well.
Next I would recommend looking at the daily Gold’s performance in October to get a better picture. You might want to use the Seasonality Tool for this purpose. Please read the Historical Seasonality Analysis: What company in DOW 30 is likely to do well in January? post for a case study on how to use the Seasonality Tool.
To view the complete source code for this example, please have a look at the bt.october.gold.test() function in bt.test.r at github.
Calling Minimum Correlation Algorithm from Excel using RExcel & VBA
I want to show the example of calling the Minimum Correlation Algorithm from Excel. I will use RExcel to connect R and Excel and will create a small VBA cell array function to communicate between Excel and R.
I have previously discussed the concept of connecting R and Excel in the “Calling Systematic Investor Toolbox from Excel using RExcel & VBA” post. Please read this post for the instructions to setup RExcel.
Following is a screen shot of the complete workbook:
You can download the MinimumCorrelation.xls workbook and experiment with it while you keep reading.
I created the “MinimumCorrelation” cell array function in VBA. Do not forget to use CTRL+SHIFT+ENTER to enter “MinimumCorrelation” function into your workbooks. The “MinimumCorrelation” function will send historical price information from Excel to the R environment. It will next execute the R script to construct weights using the Minimum Correlation Algorithm, and finally it will collect R calculations of portfolio weights and transfer them back to Excel.
Here is the R script that calls min.corr.portfolio() function. I created a VBA function “create_rcode” to create this file automatically for this example.
###############################################################################
# Load Systematic Investor Toolbox (SIT)
# https://systematicinvestor.wordpress.com/systematic-investor-toolbox/
###############################################################################
if(!exists('min.corr.portfolio')) {
setInternet2(TRUE)
con = gzcon(url('http://www.systematicportfolio.com/sit.gz', 'rb'))
source(con)
close(con)
}
#*****************************************************************
# Setup
#*****************************************************************
n = ncol(hist_prices)
hist = na.omit(hist_prices / mlag(hist_prices) - 1)
# create historical input assumptions
ia = list()
ia$n = n
ia$risk = apply(hist, 2, sd)
ia$correlation = cor(hist, use='complete.obs', method='pearson')
ia$cov = ia$correlation * (ia$risk %*% t(ia$risk))
# portfolio allocation
weights = min.corr.portfolio(ia, null)
dim(weights)=c(1,n)
Next, the “MinimumCorrelation” cell array function in VBA that calls min.corr.portfolio() function in R:
'Minimum Correlation Algorithm
Public Function MinimumCorrelation(ByRef r_values As Range) As Variant
' Start R connection
RInterface.StartRServer
' Write R code to file
create_rcode
' Put Historical Asset Prices into R
RInterface.PutArray "hist_prices", r_values
' Executes the commands in filename
RInterface.RunRFile r_filename
' Get Portfolio Allocation determined by the Minimum Correlation Algorithm into Excel
MinimumCorrelation = RInterface.GetArrayToVBA("weights")
End Function
The complete working copy of the MinimumCorrelation.xls workbook.
Please do not forget to use CTRL+SHIFT+ENTER to enter “MinimumCorrelation” function into your workbooks.
Minimum Correlation Algorithm Speed comparison
The Minimum Correlation Algorithm is a heuristic method discovered by David Varadi. Below I will benchmark the execution speed of 2 versions of the Minimum Correlation Algorithm versus the traditional minimum variance optimization that relies on solving a quadratic programming problem.
###############################################################################
# Load Systematic Investor Toolbox (SIT)
# https://systematicinvestor.wordpress.com/systematic-investor-toolbox/
###############################################################################
setInternet2(TRUE)
con = gzcon(url('http://www.systematicportfolio.com/sit.gz', 'rb'))
source(con)
close(con)
#*****************************************************************
# Setup test input assumptions
#*****************************************************************
load.packages('quadprog,corpcor')
n = 100
hist = matrix(rnorm(1000*n), nc=n)
# 0 <= x.i <= 1
constraints = new.constraints(n, lb = 0, ub = 1)
constraints = add.constraints(diag(n), type='>=', b=0, constraints)
constraints = add.constraints(diag(n), type='<=', b=1, constraints)
# SUM x.i = 1
constraints = add.constraints(rep(1, n), 1, type = '=', constraints)
# create historical input assumptions
ia = list()
ia$n = n
ia$risk = apply(hist, 2, sd)
ia$correlation = cor(hist, use='complete.obs', method='pearson')
ia$cov = ia$correlation * (ia$risk %*% t(ia$risk))
ia$cov = make.positive.definite(ia$cov, 0.000000001)
ia$correlation = make.positive.definite(ia$correlation, 0.000000001)
#*****************************************************************
# Time each Algorithm
#*****************************************************************
load.packages('rbenchmark')
benchmark(
min.var.portfolio(ia, constraints),
min.corr.portfolio(ia, constraints),
min.corr2.portfolio(ia, constraints),
columns=c("test", "replications", "elapsed", "relative"),
order="relative",
replications=100
)
I have run the code above for n=10 (10 assets), n=100 (100 assets), n=500 (500 assets), n=1000 (1000 assets)
[Please note that for n=1000 I have only run 5 replication]
n=10 (10 assets)
replications elapsed relative
min.var.portfolio 100 0.02 1.0
min.corr2.portfolio 100 0.02 1.0
min.corr.portfolio 100 0.03 1.5
n=100 (100 assets)
replications elapsed relative
min.corr2.portfolio 100 0.07 1.00
min.corr.portfolio 100 0.25 3.57
min.var.portfolio 100 0.31 4.42
n=500 (500 assets)
replications elapsed relative
min.corr2.portfolio 100 2.18 1.00
min.corr.portfolio 100 9.59 4.39
min.var.portfolio 100 139.13 63.82
n=1000 (1000 assets)
replications elapsed relative
min.corr2.portfolio 5 0.25 1.00
min.corr.portfolio 5 1.39 5.56
min.var.portfolio 5 113.27 453.08
For small universe (i.e. n ~ 100) all algorithms are fast. But once we attempt to solve 500 or 1000 assets portfolio allocation problem, the minimum variance algorithm is many times slower than the both versions of the minimum correlation algorithm.
So if we are considering a 500 assets weekly back-test for the 10 yrs the run-times in seconds (i.e. 52*10*single tun-time):
elapsed
min.corr2.portfolio 11.3
min.corr.portfolio 49.8
min.var.portfolio 723.4
To view the complete source code for this example, please have a look at the bt.mca.speed.test() function in bt.test.r at github.
Minimum Correlation Algorithm Example
Today I want to follow up with the Minimum Correlation Algorithm Paper post and show how to incorporate the Minimum Correlation Algorithm into your portfolio construction work flow and also explain why I like the Minimum Correlation Algorithm.
First, let’s load the ETF’s data set used in the Minimum Correlation Algorithm Paper using the Systematic Investor Toolbox.
###############################################################################
# Load Systematic Investor Toolbox (SIT)
# https://systematicinvestor.wordpress.com/systematic-investor-toolbox/
###############################################################################
setInternet2(TRUE)
con = gzcon(url('http://www.systematicportfolio.com/sit.gz', 'rb'))
source(con)
close(con)
#*****************************************************************
# Load historical data for ETFs
#******************************************************************
load.packages('quantmod,quadprog')
tickers = spl('SPY,QQQ,EEM,IWM,EFA,TLT,IYR,GLD')
data <- new.env()
getSymbols(tickers, src = 'yahoo', from = '1980-01-01', env = data, auto.assign = T)
for(i in ls(data)) data[[i]] = adjustOHLC(data[[i]], use.Adjusted=T)
bt.prep(data, align='keep.all', dates='2002:08::')
Next, I created the portfolio.allocation.helper() function to ease the back-testing of portfolio construction algorithms:
#***************************************************************** # Code Strategies #****************************************************************** obj = portfolio.allocation.helper(data$prices, periodicity = 'weeks', min.risk.fns = list(EW=equal.weight.portfolio, RP=risk.parity.portfolio, MV=min.var.portfolio, MD=max.div.portfolio, MC=min.corr.portfolio, MC2=min.corr2.portfolio), custom.stats.fn = 'portfolio.allocation.custom.stats' ) models = create.strategies(obj, data)$models
Please note that I assigned acronyms to various portfolio allocation algorithms in the code above.
For example, the Minimum Correlation Algorithm’s acronym is MC and the actual function that does all computations is min.corr.portfolio() function.
Next let’s create various summary reports to see the performance and allocations of different algorithms:
#*****************************************************************
# Create Report
#******************************************************************
# Plot perfromance
layout(1:2)
plotbt(models, plotX = T, log = 'y', LeftMargin = 3)
mtext('Cumulative Performance', side = 2, line = 1)
out = plotbt.strategy.sidebyside(models, return.table=T)
# Plot time series of components of Composite Diversification Indicator
cdi = custom.composite.diversification.indicator(obj,plot.table = F)
out = rbind(colMeans(cdi, na.rm=T), out)
rownames(out)[1] = 'Composite Diversification Indicator(CDI)'
# Portfolio Turnover for each strategy
y = 100 * sapply(models, compute.turnover, data)
out = rbind(y, out)
rownames(out)[1] = 'Portfolio Turnover'
# Plot and compare strategies across different metrics
performance.barchart.helper(out, 'Sharpe,Cagr,DVR,MaxDD', c(T,T,T,T))
performance.barchart.helper(out, 'Volatility,Portfolio Turnover,Composite Diversification Indicator(CDI)', c(F,F,T))
# Plot transition maps
layout(1:len(models))
for(m in names(models)) {
plotbt.transition.map(models[[m]]$weight, name=m)
legend('topright', legend = m, bty = 'n')
}
# Plot transition maps for Risk Contributions
dates = index(data$prices)[obj$period.ends]
for(m in names(models)) {
plotbt.transition.map(make.xts(obj$risk.contributions[[m]], dates),
name=paste('Risk Contributions',m))
legend('topright', legend = m, bty = 'n')
}
Looking at the summary statistics, you mind wonder what makes the the Minimum Correlation Algorithm different and why do I like it?
The overall characteristics of the Minimum Correlation Algorithm makes it attractive for me. Specifically it has reasonable perfromance, limited draw-down, low turnover and high composite diversification score. The combination of these attractive properties does differentiate the Minimum Correlation Algorithm from others.
In the next post, I will show the speed benchmarks for the Minimum Correlation Algorithm.
To view the complete source code for this example, please have a look at the bt.mca.test() function in bt.test.r at github.
Minimum Correlation Algorithm Paper
Over summer I was busy collaborating with David Varadi on the Minimum Correlation Algorithm paper. Today I want to share the results of our collaboration:
The Minimum Correlation Algorithm is fast, robust, and easy to implement. Please add it to you portfolio construction toolbox and share your experience.
Systematic Investor 