First of all, thank you very much for implementing the sparse CCA from Witten et al.! As a python user, I would be glad if I would not have to switch programming languages but be able to write one analysis pipeline purely in python. Besides, your code would allow me to integrate your CCA function into a scikit-learn pipeline. I have one question though: I was trying to reproduce the example from Witten et al. as described on page 7 of their documentation but couldn't get the same results. I am not a math expert so I cannot really explain what might cause these deviations. Maybe I also did not use the exact same parameters? The only obvious difference between your package and the PMA package is, that your function only demeans x and z while Witten et al. use z-standardization . But I don't think that this drives the deviation?
Here's the code from Witten et al.:
## Run example from PMA package ################################################
# first, do CCA with type="standard"
# A simple simulated example
set.seed(3189)
u <- matrix(c(rep(1,25),rep(0,75)),ncol=1)
v1 <- matrix(c(rep(1,50),rep(0,450)),ncol=1)
v2 <- matrix(c(rep(0,50),rep(1,50),rep(0,900)),ncol=1)
x <- u%*%t(v1) + matrix(rnorm(100*500),ncol=500)
z <- u%*%t(v2) + matrix(rnorm(100*1000),ncol=1000)
# Can run CCA with default settings, and can get e.g. 3 components
out <- CCA(x,z,typex="standard",typez="standard",K=3)
## Save x,z and canonical weights as .txt files
write.table(x,file="x.txt",row.names=F,col.names=F)
write.table(z,file="z.txt",row.names=F,col.names=F)
write.table(out$u,file="out_u.txt",row.names=F,col.names=F)
write.table(out$v,file="out_v.txt",row.names=F,col.names=F)
And here's me trying to reproduce the canonical weights:
import numpy as np
from sparsecca._cca_pmd import cca
import matplotlib.pyplot as plt
# load x and z from PMA example
x = np.loadtxt('./x.txt')
z = np.loadtxt('./z.txt')
# load canonical weights produced by PMA package
out_u = np.loadtxt('./out_u.txt')
out_v = np.loadtxt('./out_v.txt')
# run cca one more time with sparsecca package and the same settings as used
# in the pma package NOTE: Although documentation of PMA package says that the
# penalties are NULL as default, it does not seem to be the case.
# Instead 0.3 seems to be the default value for both x and z.
u,v,d = cca(x,z,penaltyx=0.3,penaltyz=0.3,K=3,niter=15)
# check canonical weights from PMA package against sparsecca package
print(np.array_equal(out_u,u))
print(np.array_equal(out_v,v))
# plot distribution of deviations (ignore zeros for plotting purposes)
deviations = np.abs(out_u-u).flatten()
deviations = deviations[deviations != 0]
plt.hist(deviations)
![image](https://user-images.githubusercontent.com/48254290/143079110-c0177c18-2c24-4584-ae32-ca4d49b1e23a.png)
As you can see most of the canonical weights only have small deviations, but there are a few that are quite different between the python implementation and the PMA package.