GithubHelp home page GithubHelp logo

mims-harvard / scikit-fusion Goto Github PK

View Code? Open in Web Editor NEW
142.0 14.0 42.0 9.51 MB

scikit-fusion: Data fusion via collective latent factor models

License: Other

Python 100.00%
matrix-factorization data-fusion latent-features data-integration embeddings knowledge-graphs

scikit-fusion's Introduction

scikit-fusion

build: passing BSD license

scikit-fusion is a Python module for data fusion and learning over heterogeneous datasets. The core of scikit-fusion are recent collective latent factor models and large-scale joint matrix factorization algorithms.

[News:] Fast CPU and GPU-accelerated implementatons of some of our methods.

[News:] Scikit-fusion, collective latent factor models, matrix factorization for data fusion and learning over hetnets.

[News:] fastGNMF, fast implementation of graph-regularized non-negative matrix factorization using Facebook FAISS.

Dependencies

scikit-fusion is tested to work under Python 3.

The required dependencies to build the software are Numpy >= 1.7, SciPy >= 0.12, PyGraphviz >= 1.3 (needed only for drawing data fusion graphs) and Joblib >= 0.8.4.

Install

This package uses distutils, which is the default way of installing python modules. To install in your home directory, use:

python setup.py install --user

To install for all users on Unix/Linux:

python setup.py build
sudo python setup.py install

For development mode use:

python setup.py develop

Use

Let's generate three random data matrices describing three different object types:

 >>> import numpy as np
 >>> R12 = np.random.rand(50, 100)
 >>> R13 = np.random.rand(50, 40)
 >>> R23 = np.random.rand(100, 40)

Next, we define our data fusion graph:

 >>> from skfusion import fusion
 >>> t1 = fusion.ObjectType('Type 1', 10)
 >>> t2 = fusion.ObjectType('Type 2', 20)
 >>> t3 = fusion.ObjectType('Type 3', 30)
 >>> relations = [fusion.Relation(R12, t1, t2),
                  fusion.Relation(R13, t1, t3),
                  fusion.Relation(R23, t2, t3)]
 >>> fusion_graph = fusion.FusionGraph()
 >>> fusion_graph.add_relations_from(relations)

and then collectively infer the latent data model:

 >>> fuser = fusion.Dfmf()
 >>> fuser.fuse(fusion_graph)
 >>> print(fuser.factor(t1).shape)
 (50, 10)

Afterwards new data might arrive:

 >>> new_R12 = np.random.rand(10, 100)
 >>> new_R13 = np.random.rand(10, 40)

for which we define the fusion graph:

 >>> new_relations = [fusion.Relation(new_R12, t1, t2),
                      fusion.Relation(new_R13, t1, t3)]
 >>> new_graph = fusion.FusionGraph(new_relations)

and transform new objects to the latent space induced by the fuser:

 >>> transformer = fusion.DfmfTransform()
 >>> transformer.transform(t1, new_graph, fuser)
 >>> print(transformer.factor(t1).shape)
 (10, 10)

scikit-fusion contains several applications of data fusion:

>>> from skfusion import datasets
>>> dicty = datasets.load_dicty()
>>> print(dicty)
FusionGraph(Object types: 3, Relations: 3)
>>> print(dicty.object_types)
{ObjectType(GO term), ObjectType(Experimental condition), ObjectType(Gene)}
>>> print(dicty.relations)
{Relation(ObjectType(Gene), ObjectType(GO term)),
 Relation(ObjectType(Gene), ObjectType(Gene)),
 Relation(ObjectType(Gene), ObjectType(Experimental condition))}

Selected publications (Methods)

Selected publications (Applications)

Tutorials

  • Large-scale data fusion by collective matrix factorization, Basel Computational Biology Conference, [BC]^2 [Slides] [Handouts]
  • Data fusion of everything, 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC [Slides] [Handouts]

scikit-fusion's People

Contributors

astaric avatar kernc avatar marinkaz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

scikit-fusion's Issues

Documentation

Hi there,
I'm trying to use skfusion with my data, but I feel there is so little documentation describing functions and object types. I'm doing a little of reverse engineering from example data, so far I've got that fuser.factor(factor_name) returns the matrix for a specifed factor and fuser.backbone(relation_name) returns the backbone matrix for a specified relations. Options for ObjectType, Relation, FusionGraph objects are not described, neither Dfmf options are. Also, a good hint guide about choosing matrix ranks would help.

Small dimensions problems

Two issues occur when working with small samples:

  • (rank = 1) problem.
    Running:
import numpy as np
from skfusion import fusion

R12 = np.random.rand(30, 40)
R23 = np.random.rand(40, 50)

t1 = fusion.ObjectType('Type 1', 2)
t2 = fusion.ObjectType('Type 2', 9)
t3 = fusion.ObjectType('Type 3', 1)

relations = [fusion.Relation(R12, t1, t2),
             fusion.Relation(R23, t2, t3)]
fusion_graph = fusion.FusionGraph(relations)

fuser = fusion.Dfmf()
fuser.fuse(fusion_graph)

yields: KeyError: (ObjectType("Type 1"), ObjectType("Type 3"))
the code works if t3 rank is >= 2

  • (dim < 5) problem.

Running:

import numpy as np
from skfusion import fusion

R12 = np.random.rand(30, 4)
R23 = np.random.rand(4, 50)

t1 = fusion.ObjectType('Type 1', 2)
t2 = fusion.ObjectType('Type 2', 9)
t3 = fusion.ObjectType('Type 3', 2)

relations = [fusion.Relation(R12, t1, t2),
             fusion.Relation(R23, t2, t3)]
fusion_graph = fusion.FusionGraph(relations)


fuser = fusion.Dfmf()
fuser.fuse(fusion_graph)

yield numpy warning and reconstructed matrices have NaN. This works if R12 has second dimension greater than 4.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.