danielkorzekwa / bayes-scala-gp

Gaussian Processes in Scala

Algorithms for Gaussian Processes

List of algorithms:

• Collaborative multi-output Gaussian Process
• Sparse Gaussian Process Regression
• Gaussian Process Regression
• Hierachical Gaussian Process Regression
• Multi Task Gaussian Process Regression
• GPR: Learning cov hyper parameters with variational lower bound on marginal likelihood
• Gaussian Process Classification
• Multi Task Gaussian Process Classification
• Hierachical Gaussian Process Classification

This project uses Bayes-scala library for Bayesian inference.

How to use it from sbt and maven?

Only snapshot version for Scala 2.11.

Snapshot version

Snapshot artifact is built by a Travis CI and deployed to Sonatype OSS Snapshots repository with every commit to Bayes-scala-gp project.

With sbt build tool, add to build.sbt config file:

libraryDependencies += "com.github.danielkorzekwa" %% "bayes-scala-gp" % "0.1-SNAPSHOT"  

resolvers += Resolver.sonatypeRepo("snapshots")

With maven build tool, add to pom.xml config file:

  <repositories>
    <repository>
      <id>oss-sonatype-snapshots</id>
      <name>oss-sonatype-snapshots</name>
      <url>https://oss.sonatype.org/content/repositories/snapshots/</url>
    </repository>
  </repositories>
  
  <dependencies>
    <dependency>
      <groupId>com.github.danielkorzekwa</groupId>
      <artifactId>bayes-scala-gp_2.11</artifactId>
      <version>0.1-SNAPSHOT</version>
    </dependency>
  <dependencies>

bayes-scala-gp's Issues

Read on tensor flow and Auto-Encoding Variational Bayes, auto differentiation

http://arxiv.org/abs/1312.6114,

http://www.kdnuggets.com/2015/12/tensor-flow-terrific-deep-learning-library.html

Perf optimisation for CovSeIso

This test takes 5 sec to run, Different impl of CovSeIso from bayes-scala takes 0.7 sec to execute.

 @Test def perf_test_1d_df_dSf = {

    val x1 = Array(10d)
    val x2 = Array(2d)
    (1L to 2000L * 50 * 50).foreach(_ => covFunc.df_dSf(x1, x2))

  }

calibrateHgpcFactorGraph - check for taskPosteriors instead of checking just for uPosterior

instead of:

calibrated = CanonicalGaussian.isIdentical(beforeUMarginal, factorGraph.uVariable.get, 1e-3)

do:

taskPosterior.filter(taskPosterior => !isIdentical....)

Implement cogp

Theme: [sin(x),-sin(x)] multi output example

• Implement classical momentum: http://jmlr.org/proceedings/papers/v28/sutskever13.pdf
• learn w params
• learn beta params
• learn cov params

hgprPredict/hgpcPredict: use u cavity distribution for computing xPrior

hgpr:
hgprPredict.createTaskPosteriorByTaskId()

Use u cavity distribution for computing xPrior, similarly to calcHgprLoglik(), which uses cavity distribution for computing loglik

hgpc:

similar to hgpr

CogpGPVar - simplification?

In https://github.com/danielkorzekwa/bayes-scala-gp/blob/fa85daf09c79a3cd1ff3a2e7061613bfe6a38228/src/main/scala/dk/gp/cogp/model/CogpGPVar.scala

You have three lines in which I have impression could be simplified by taking just variance(y) instead of 1/variance(y). Because

1. vInv will have only values on diagonal and all are positive
2. cholesky of vinv will give you sqrt of vInv diagonal (as A_A.t = vInv)
3. transposing cholesky will not change anything (as (A_A.t).t = (A.t*A), but A it all the time a diagonal matrix so that equation is an identity for it)
4. then taking invchol pretty much multiplies two matrices with the same values on the diagonal, and should result in 1/vInv

So basically on diagonal it should true that
v == 1/sqrt(vInv)^2

    val m = DenseVector.zeros[Double](z.rows)
    val vInv = 0.1 * (1.0 / (variance(y))) * DenseMatrix.eye[Double](z.rows)
    val v = invchol(cholesky(vInv).t)

Moreover, taking inverse of a matrix is something that you should avoid, if possible, because of the numerical properties of the algorithm. Combined with using 1/10 on a big scale (see. http://ta.twi.tudelft.nl/users/vuik/wi211/disasters.html) it may bring unexpected results. Unless you wanted to introduce some random disturbance here.

Of course, I might have missed something and everything here is crucial for the algorithm to work. :)

Do we need breeze-natives?

• the same question for bayes-scala project,
• look here too: fommil/netlib-java#89

Deploy snapshot to sonatype maven repo

• use travis, like in the bayes-scala project