arthurzc23 / machine-learning-a-probabilistic-perspective-solutions

My solutions to Kevin Murphy Machine Learning Book

Machine-Learning-A-Probabilistic-Perspective-Solutions

Motivation

Hey there. I am recording the solutions of the exercises of the fourth printing of this book in this repository. The only exercises that I do not intend to do in this first run are those which explicit require MATLAB. Any computational exercise will be done in Python using a Jupyter notebook. I will follow a schema where I give a introduction and some insight into the problem, solve it and then make some remarks on the solution. I strongly reccomend the reading of the Intro and Conclusion section of the exercises that you're interested in. I intend to update the solutions in a reasonable pace, starting now (January 2017).

I hope this might help anyone who has an interest in the book and Machine Learning as a whole.

machine-learning-a-probabilistic-perspective-solutions's Issues

Exercise 4.2: Product distribution

You calculate p(y) as p(wx). This is incorrect. See: https://en.wikipedia.org/wiki/Product_distribution

Problem 3.11.d

You derive the likelihood, then say that the resulting distribution is

Ga(theta | N+1, lam+sum(x_i))

but plugging that into the definition given for the Gamma distribution in the textbook gives

Ga(theta | N+1, lam+sum(x_i)) = [lam+sum(x_i)]^(N+1) theta^N e^(- theta [lam+sum(x_i)])

which is not the likelihood that you derived.

Am I missing something here?

Solution to 2.6

When E1 and E2 are conditionally independent given H, we can derive p(e1,e2) as follows:

p(e1,e2) = sum(p(e1,e2|h')p(h')) = sum(p(e1|h')p(e2|h')p(h')) over all h' in support of H.

Thus, (iii) is sufficient for 2.6(b)

Your solution to problem 2.9(b)

The proposition in problem 2.9(b) is actually false. You can find a counter example here:
https://github.com/astahlman/MLaPP/blob/master/solutions/ch2.pdf

Getting a different result for problem 7.2

In the following solution https://github.com/ArthurZC23/Machine-Learning-A-Probabilistic-Perspective-Solutions/blob/master/7/2.pdf, my w₁ and w₂ values are as follows:

w₁=[-4/3 8/3]

w₂=[-4/3 8/3]

which are different from your solution.

Solution to 2.4

Missing absolute value in 3.8 b)

Hi,
IMHO in 3.8 b) there should be abs(x_n+1) instead of x_n+1.
If x_n+1 will be - 100*a its probablilty will be 0.

Thanks for this repo!

https://github.com/ArthurZC23/Machine-Learning-A-Probabilistic-Perspective-Solutions/blob/master/3/8.pdf

An neat approach to problem 2.17

Instead of using brute force to calculate the double integral, we can actually prove the distribution of min(X_1, ..., X_n) is a Beta distribution where X_1,...,X_n are i.i.d. and X_1,...,X_n ~ U(0,1). Then, we can directly apply the Beta mean we derived from 2.16.