chene5 / big-data-in-psychology

A Practical Guide to Big Data Research in Psychology sample code

A Practical Guide to Big Data Research in Psychology

Tutorial code and data repository

Tutorial 1

Tutorial 1 covers data acquisition from the U.S. GPO API. The code files here implement retrieving one day, four days, and ten years of data.

Tutorial 2

Tutorial 2 covers latent Dirichlet allocation topic modeling using MALLET. The data here are U.S. Senate Additional Statements. The Python file text_processor.py processes the raw data retrieved in Tutorial 1.

Tutorial 3

Tutorial 3 covers support vector machine classification. The code files here implement classification using the iris dataset, including visualization.

Extras

The extras directory contains the raw data retrieved in Tutorial 1.

---

Key resources

Python

https://www.python.org/

Python distributions:

• Continuum Analytics Anaconda: https://www.continuum.io/
• Enthought Canopy: https://www.enthought.com/products/canopy/

Python package repository: https://pypi.python.org/pypi

R

https://www.r-project.org/

R Studio: https://www.rstudio.com/

R package repository: https://cran.r-project.org/

Databases

MySQL: https://www.mysql.com/

MongoDB: https://www.mongodb.org/

Apache’s Cassandra: http://cassandra.apache.org/

Redis: http://redis.io/

Text analytics

TACIT: http://tacit.usc.edu/

Natural Language Toolkit: http://www.nltk.org/

Genism: https://radimrehurek.com/gensim/

Stanford CoreNLP: http://stanfordnlp.github.io/CoreNLP/

MALLET: http://mallet.cs.umass.edu/topics.php

Matlab Topic Modeling Toolbox: http://psiexp.ss.uci.edu/research/programs_data/toolbox.htm

Data mining and machine learning

scikit-learn: http://scikit-learn.org/stable/

WEKA: http://www.cs.waikato.ac.nz/ml/weka/

Network analysis and visualization

UCINET: https://sites.google.com/site/ucinetsoftware/home

Pajek: http://mrvar.fdv.uni-lj.si/pajek/

Siena/RSiena: https://www.stats.ox.ac.uk/~snijders/siena/

Gephi: https://gephi.org/

d3.js: https://d3js.org/

graphviz: http://www.graphviz.org/

Hadoop

http://hadoop.apache.org/