geekwolverine / tensorflow-basic-exercises

Exercises for TensorFlow Tutorial

TensorFlow Basic Exercises

This repo contains exercises for basic TensorFlow tutorial.

Software Requirements

You need to have Python and pip installed. Then install TensorFlow via pip install tensorflow. Consult this page if you want to customize your setup.

To test your installation, clone this repo and then run python demo.py. No error is expected if installation ws successful. You should expect the following at the end of the output:

Optimization Finished!
Accuracy: 0.9256

Instruction

The Python script demo.py provides a basic example for logistic regression using TensorFlow on MNIST hand-written digit data. All the necessary steps, including data preprocessing, model building, and model evaluation, have been done for you so you can focus on small parts of the program. You are expected to make small modifications to the Python script demo.py to achieve the goals of the following exercises to get more familiar with TensorFlow's basics.

Exercises

Exercise 1: Minimize error using cross entropy as the cost function

• Hint a: Definition of cross entropy on Wikipedia.
• Hint b: You'll need to use tf.reduce_mean, tf.reduce_sum, and tf.log.

Exercise 2: Apply exponential learning rate decay

• Hint a: TensorFlow has a built-in function for exponential decay. See API Documentation for this here.
• Hint b: You can use global_step = tf.Variable(0, trainable=False) to initialize global step.

Exercise 3: Apply early stopping when a condition is met

• Hint a: Calculate the absolute value of the difference between current average cost and previous average cost.
• Hint b: When there's no much different, exit the for loop.

Exercise 4: Apply L1 regularization to weights

• Hint a: Definition of L1 regularization.
• Hint b: Some functions you'll need are: tf.multiply, tf.reduce_sum, tf.abs.
• Hint c: Apply this regularization term to when constructing the model.

Exercise 5: What else can you do to achieve higher accuracy (minimum 0.94)?

• Hint: You can tune learning_rate, change optimizers, change initializers, change cost function, more training steps, advanced early stoping, etc. Be creative!

Maintenance

Please contact Yuan Tang ([email protected]) if you have any questions or open an issue.

Credits

Some screenshots are adopted from TensorFlow Dev Summit, TensorFlow paper, and other online resources. Thanks!