turi-code / how-to

How-To code samples for working with GraphLab Create.

Want to contribute a GraphLab Create How-To? We welcome pull requests. Please read the contributor guide.

Want to request a new How-To or have feedback on one listed below? Please open a Git issue.

Data Ingress

• Import data from Pandas Series/Dataframes
• Import data from your Spark cluster
• Import data from your databases using SQL
• Load a JSON file into an SFrame
• Load a collection XML files into an SFrame
• Load an Avro file into an SFrame
• Extract main article content from HTML with Boilerpipe
• Load a Neo4j graph from a JSON file into an SGraph

Tabular Data Transformation

• Filter/Select rows from an SFrame
• Efficiently Calculating Basic Statistics in SArray/SFrame
• Find the top-k rows for each value of a group variable
• Collapse multiple columns of an SFrame into a single column of type list/dict
• Parse a datetime column into its components (year, month, etc.)
• Convert a column of datetime strings into UNIX timestamps
• Expand an SFrame column of type list/dict into multiple columns

Graph Data Transformation

• Join vertex data on SGraph
• Remove duplicate edges from SGraph
• Check if one SGraph is a subgraph of a second SGraph

Text Analytics

• Find the unique words used in an SArray of text documents
• Compute word frequencies for each word in an SArray of text documents
• Example of making predictions using topic model parameters

Image Analytics

• Convert an SArray of URL strings to an SArray of graphlab.Image
• Convert an SArray of array.array to an SArray of graphlab.Image
• Convert a PIL.Image to a graphlab.Image
• Convert a graphlab.Image to a PIL.Image

Graph Analytics

• Implement single source shortest path using triple_apply
• Implement weighted pagerank using triple_apply

Sparse matrices

• Convert an SFrame into a scipy.sparse matrix
• Compute approximate AtA sparse matrix product using the DIMSUM algorithm

Visualization

• Show SGraph with custom layout using vertex_positions and NetworkX
• Show SGraph with bipartite vertex coloring
• Show SGraph with custom vertex coloring
• Show time-series data as a line chart in Canvas

Miscellaneous

• Run a function on the crossproduct of option values
• Parallel web page crawling using Turi Distributed
• Use the logging module with GraphLab Create

Code submitted through pull requests will be made available under the CC0 1.0 Universal license.

how-to's Issues

Define SFrame columns type on new object

When creating an SFrame there is no way to pre-define the columns type. This creates a problem when defining an empty SFrame from a dictionary. All the columns are by default from type 'float' and in order to change their type, you need to define it column by column using SArray.astype(str).

I was hoping for the option to define in when defining the SFrame like the option in read_csv , column_type_hints:
SFrame.read_csv(url, delimiter=',', header=True, error_bad_lines=False, comment_char='', escape_char='', double_quote=True, quote_char='"', skip_initial_space=True, column_type_hints=None, na_values=['NA'], line_terminator='\n', usecols=[], nrows=None, skiprows=0, verbose=True, **kwargs)

Thanks!

invalide pointer

download GraphLab-Create-1.5.2.tar.gz from https://pypi.python.org/pypi/GraphLab-Create, install as usual

python dimsum.py
*** glibc detected *** python: free(): invalid pointer: 0x00007f3b355ab4c0 ***
======= Backtrace: =========

Appreciate if anyone can help with this issue.

graphlab.deploy.predictive_service deprecated?

I'm trying to launch a predictive service on AWS using this documentation but it can't find that method. Has support for predictive services been dropped? Is there any way to host a GraphLab Create model on an endpoint?

I'm using v2.1

import graphlab error

I have installed graphlab using pip install graphlab
anaconda & miniconda are also installed in my ubuntu 16.10

How to fix this error?

load_yelp_dataset how-to requires external data

The load_yelp_dataset snippet assumes external data. We should add a comment to direct users to get this dataset.

condo error with MBP installer

Hi Guys,

I downloaded and am trying to install the GLC Launcher for MBP. I have registered for the Academic License and I have a key already provided to me. When I run the installer, it almost completes and I keep getting the following error:

GraphLab Create Launcher
We encountered an error that prevented us from installing.
SAVE LOG
100% CompleteHide Details
Error Occurred: Updating conda package

Installation process initialized.

=== System State ===

Current time: 2017-08-22T04:35:58.291Z

{
"PATH": "/Users/sergioarroyo2017/anaconda/bin:/Applications/MAMP/Library/bin/:/Applications/MAMP/bin/php/php5.6.27/bin/:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Frameworks/Mono.framework/Versions/Current/Commands",
"SHELL": "/bin/bash",
"HOME": "/Users/sergioarroyo2017",
"USERPROFILE": "",
"TEMP": "",
"undefined": ""
}

Local settings:
{
"licenseEmail": "[email protected]",
"licenseKey": "FADD-CE3F-CA2F-4824-1F73-55E6-D586-B615",
"anacondaPath": "/Users/sergioarroyo2017/anaconda"
}

isCondaInEnvironment: true
isPipInEnvironment: true
Selected Anaconda path is /Users/sergioarroyo2017/anaconda
isCondaAvailable("/Users/sergioarroyo2017/anaconda"): true

conda info:
Current conda install:

           platform : osx-64
      conda version : 4.3.21
   conda is private : False
  conda-env version : 4.3.21
conda-build version : not installed
     python version : 3.6.1.final.0
   requests version : 2.14.2
   root environment : /Users/sergioarroyo2017/anaconda  (writable)
default environment : /Users/sergioarroyo2017/anaconda
   envs directories : /Users/sergioarroyo2017/anaconda/envs
                      /Users/sergioarroyo2017/.conda/envs
      package cache : /Users/sergioarroyo2017/anaconda/pkgs
                      /Users/sergioarroyo2017/.conda/pkgs
       channel URLs : https://repo.continuum.io/pkgs/free/osx-64
                      https://repo.continuum.io/pkgs/free/noarch
                      https://repo.continuum.io/pkgs/r/osx-64
                      https://repo.continuum.io/pkgs/r/noarch
                      https://repo.continuum.io/pkgs/pro/osx-64
                      https://repo.continuum.io/pkgs/pro/noarch
        config file : None
         netrc file : None
       offline mode : False
         user-agent : conda/4.3.21 requests/2.14.2 CPython/3.6.1 Darwin/16.7.0 OSX/10.12.6    
            UID:GID : 507:20

conda environments:

aind /Users/sergioarroyo2017/anaconda/envs/aind
nb35 /Users/sergioarroyo2017/anaconda/envs/nb35
sva_env /Users/sergioarroyo2017/anaconda/envs/sva_env
root * /Users/sergioarroyo2017/anaconda

sys.version: 3.6.1 |Anaconda custom (x86_64)| (defaul...
sys.prefix: /Users/sergioarroyo2017/anaconda
sys.executable: /Users/sergioarroyo2017/anaconda/bin/python
conda location: /Users/sergioarroyo2017/anaconda/lib/python3.6/site-packages/conda
conda-build: None
conda-env: /Users/sergioarroyo2017/anaconda/bin/conda-env
conda-server: /Users/sergioarroyo2017/anaconda/bin/conda-server
user site dirs:

CIO_TEST:
CONDA_DEFAULT_ENV:
CONDA_ENVS_PATH:
DYLD_LIBRARY_PATH:
PATH: /Users/sergioarroyo2017/anaconda/bin:/Applications/MAMP/Library/bin/:/Applications/MAMP/bin/php/php5.6.27/bin/:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Frameworks/Mono.framework/Versions/Current/Commands
PYTHONHOME:
PYTHONPATH:

License directories:
/Users/sergioarroyo2017/.continuum
/Users/sergioarroyo2017/Library/Application Support/Anaconda
/Users/sergioarroyo2017/anaconda/licenses
License files (license*.txt):
Package/feature end dates:

===================

Obtained dependency info: {
"glc": {
"version": "2.1",
"url": "https://static.turi.com/products/graphlab-create/2.1/python2.7/macosx/GraphLab_Create-2.1-cp27-none-macosx_10_5_x86_64.macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.macosx_10_11_intel.macosx_10_11_x86_64.whl",
"filename": "GraphLab_Create-2.1-cp27-none-macosx_10_5_x86_64.macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.macosx_10_11_intel.macosx_10_11_x86_64.whl",
"md5": "0c5b811e7f83ea6d9bcfe50d23795d1d"
},
"anaconda": {
"version": "4.0.0",
"filename": "Anaconda2-4.0.0-MacOSX-x86_64.sh",
"url": "https://static.turi.com/products/graphlab-create-launcher/resources/anaconda/Anaconda2-4.0.0-MacOSX-x86_64.sh",
"md5": "a3443b46f99bc6680c77c688af1b1f5a"
}
}

Anaconda installation detected. Skipping download.

Starting GraphLab Create 2.1 retrieval.

Anaconda installation detected. Skipping installation.

PATH environment variable updated.

Updating conda package.
Fetching package metadata ...
Solving package specifications:

UnsatisfiableError: The following specifications were found to be in conflict:

• conda 4.0.8* -> conda-env <2.5 -> python 2.7* -> openssl 1.0.1*
• python 3.6*
  Use "conda info " to see the dependencies for each package.

Capturing final system state:

=== Packages in GraphLab Create Environment ===

conda list:

CondaEnvironmentNotFoundError: Could not find environment: gl-env .
You can list all discoverable environments with conda info --envs.

pip version:
usage: conda [-h] [-V] command ...
conda: error: argument command: invalid choice: '/Users/sergioarroyo2017/anaconda/bin/conda' (choose from 'info', 'help', 'list', 'search', 'create', 'install', 'update', 'upgrade', 'remove', 'uninstall', 'config', 'clean', 'package')

pip list:
usage: conda [-h] [-V] command ...
conda: error: argument command: invalid choice: '/Users/sergioarroyo2017/anaconda/bin/conda' (choose from 'info', 'help', 'list', 'search', 'create', 'install', 'update', 'upgrade', 'remove', 'uninstall', 'config', 'clean', 'package')

====================================

=== System State ===

Current time: 2017-08-22T04:36:19.366Z

{
"PATH": "/Users/sergioarroyo2017/anaconda/bin:/Users/sergioarroyo2017/anaconda/bin:/Applications/MAMP/Library/bin/:/Applications/MAMP/bin/php/php5.6.27/bin/:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Frameworks/Mono.framework/Versions/Current/Commands",
"SHELL": "/bin/bash",
"HOME": "/Users/sergioarroyo2017",
"USERPROFILE": "",
"TEMP": "",
"undefined": ""
}

Local settings:
{
"licenseEmail": "[email protected]",
"licenseKey": "FADD-CE3F-CA2F-4824-1F73-55E6-D586-B615",
"anacondaPath": "/Users/sergioarroyo2017/anaconda"
}

isCondaInEnvironment: true
isPipInEnvironment: true
Selected Anaconda path is /Users/sergioarroyo2017/anaconda
isCondaAvailable("/Users/sergioarroyo2017/anaconda"): true

conda info:
Current conda install:

           platform : osx-64
      conda version : 4.3.21
   conda is private : False
  conda-env version : 4.3.21
conda-build version : not installed
     python version : 3.6.1.final.0
   requests version : 2.14.2
   root environment : /Users/sergioarroyo2017/anaconda  (writable)
default environment : /Users/sergioarroyo2017/anaconda
   envs directories : /Users/sergioarroyo2017/anaconda/envs
                      /Users/sergioarroyo2017/.conda/envs
      package cache : /Users/sergioarroyo2017/anaconda/pkgs
                      /Users/sergioarroyo2017/.conda/pkgs
       channel URLs : https://repo.continuum.io/pkgs/free/osx-64
                      https://repo.continuum.io/pkgs/free/noarch
                      https://repo.continuum.io/pkgs/r/osx-64
                      https://repo.continuum.io/pkgs/r/noarch
                      https://repo.continuum.io/pkgs/pro/osx-64
                      https://repo.continuum.io/pkgs/pro/noarch
        config file : None
         netrc file : None
       offline mode : False
         user-agent : conda/4.3.21 requests/2.14.2 CPython/3.6.1 Darwin/16.7.0 OSX/10.12.6    
            UID:GID : 507:20

conda environments:

aind /Users/sergioarroyo2017/anaconda/envs/aind
nb35 /Users/sergioarroyo2017/anaconda/envs/nb35
sva_env /Users/sergioarroyo2017/anaconda/envs/sva_env
root * /Users/sergioarroyo2017/anaconda

sys.version: 3.6.1 |Anaconda custom (x86_64)| (defaul...
sys.prefix: /Users/sergioarroyo2017/anaconda
sys.executable: /Users/sergioarroyo2017/anaconda/bin/python
conda location: /Users/sergioarroyo2017/anaconda/lib/python3.6/site-packages/conda
conda-build: None
conda-env: /Users/sergioarroyo2017/anaconda/bin/conda-env
conda-server: /Users/sergioarroyo2017/anaconda/bin/conda-server
user site dirs:

CIO_TEST:
CONDA_DEFAULT_ENV:
CONDA_ENVS_PATH:
DYLD_LIBRARY_PATH:
PATH: /Users/sergioarroyo2017/anaconda/bin:/Users/sergioarroyo2017/anaconda/bin:/Applications/MAMP/Library/bin/:/Applications/MAMP/bin/php/php5.6.27/bin/:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Frameworks/Mono.framework/Versions/Current/Commands
PYTHONHOME:
PYTHONPATH:

License directories:
/Users/sergioarroyo2017/.continuum
/Users/sergioarroyo2017/Library/Application Support/Anaconda
/Users/sergioarroyo2017/anaconda/licenses
License files (license*.txt):
Package/feature end dates:

===================
There was a problem updating the conda package. Restart GraphLab Create Launcher.

Unable to update conda:
Fetching package metadata .........
Solving package specifications: .

UnsatisfiableError: The following specifications were found to be in conflict:

• conda 4.0.8* -> conda-env <2.5 -> python 2.7* -> openssl 1.0.1*
• python 3.6*
  Use "conda info " to see the dependencies for each package.

Process completed with exit code -1

can you please help me out? I really want to use if for the Coursera Course. Thanks!

Whitelisting licensing endpoint

Hi All,

Ben here from Coursera - One of our clients is having trouble using the software on their system as they can't authenticate because of a firewall issue their side. Can you please let me know what they need to whitelist to get the software working?

Thanks,
Ben