retowuest / uib-dl-2024

Deep Learning for Social Scientists

Deep Learning for Social Scientists

PhD Course, University of Bergen, April 18-19, 2024

Reto Wüest

Course Content

This two-day workshop serves as an introduction to deep learning with Python for social scientists. Deep learning is of special interest to social scientists because it allows the processing of time-series data, text, images, and more. On the first day, the course starts by providing participants with a high-level overview of machine learning, focusing on supervised learning. It then discusses shallow and deep neural networks as well as techniques we use to fit and assess these models. Subsequently, it briefly introduces the Python programming language. On the second day, the course introduces participants to convolutional neural networks (CNNs) and transformers, which are two widely used deep learning architectures.

Upon successful completion of this course, participants should be able to:

• understand some of the deep learning models that are most commonly used in academic research and industry;

• apply these models to real-world data using Python;

• explain the similarities and differences between these models as well as the advantages and disadvantages they have compared to more classical machine learning methods.

Prerequisites

Participants of this course should have a basic knowledge of linear algebra, calculus, and probability as well as an understanding of "workhorse" statistical models such as linear and logistic regression.

The course will be taught in Python. Prior experience in Python (or another programming language commonly used in machine learning like R) is an advantage but not a requirement.

Software

This course will use Jupyter notebooks hosted on Google Colab, so participants will only need a web browser and a Google account.

Course Readings

Our textbook for the course is:

• Prince, Simon J.D. 2024. Understanding Deep Learning. (Available for free here.)

Other useful books are:

• Bishop, Christopher M. and Hugh Bishop. 2024. Deep Learning: Foundations and Concepts. (Available for free here.)

• Zhang, Aston, Zachary C. Lipton, Mu Li, and Alexander J. Smola. 2023. Dive into Deep Learning. (Available for free here.)

Course Schedule

Day	Session	Content	Readings
Thursday	Morning	- ML and supervised learning	Prince (2024), chs. 1-2
		- Shallow neural networks	Prince (2024), ch. 3
		- Deep neural networks	Prince (2024), ch. 4
		- Loss functions	Prince (2024), ch. 5
	Afternoon	- Fitting models	Prince (2024), chs. 6-7
		- Measuring performance	Prince (2024), ch. 8
		- Introduction to Python
Friday	Morning	- Convolutional neural networks (CNNs)	Prince (2024), ch. 10
		- CNNs with Python
	Afternoon	- Transformers	Prince (2024), ch. 12
		- Transformers with Python