sebtheiler / tutorials

All of the code for my Medium articles

tutorials

All of the code for my Medium articles.

Update (2021/11/28): Unfortunately, I am no longer writing new articles and this repository is no longer being updated. Thank you to everyone who has ever read my pieces! I'm so glad I could share some of my knowledge of AI and data science with you guys! It's been an honor and I'm sad to leave, but I've moved on to other projects now. As such, I can't make any guarantee that the code here is still up to date, and I probably won't reply to any issues/PRs. Thank you again for being here; it's really been my honor. Farewell! And until next time, happy coding!

(I might return one day, but that won't be soon)

My Medium account: https://medium.com/@sebastiankt9

• How to Connect Google Drive to Python?: pydrive/

• What Does a DQN Think?: dqn/

• Building a Powerful DQN in TensorFlow 2.0 (explanation & tutorial): dqn/

• Basics of Using TensorBoard in TensorFlow 1 & 2: using-tensorboard/

• Get More Out of Google Colab: gcolab-tricks/

• The Alpaca API Explained for People Who Want to Get Started Trading (semi-outdated): trading/

• Building a Twitter Sentiment-Analysis App Using Streamlit: twitter-sentiment/

• tf.GradientTape Explained for Keras Users: gradienttape-explained/

• Transforming the World Into Paintings with CycleGAN: cyclegan/

• Implementing a GAN in Keras: gans/

• Multiprocessing for Data Scientists in Python: multiprocessing/

• Implementing Adversarial Attacks and Defenses in Keras & Tensorflow 2.0: adversarial-attacks-defenses/

• Basics of Using Pre-trained GloVe Vectors in Python: using-pretrained-glove-vectors/

tutorials's Issues

one question regarding the replay buffer in DQN atari project

In the class "replay buffer", i found there is "states = np.transpose(np.asarray(states), axes=(0, 2, 3, 1))" in the def get_minibatch.
The problem is why transpose needed here? It looks like the original sequence is already correct so don't need transpose operation here.

Many thanks!

Agent learns but fairly well

Hello
firstly, thank you for this tutorial.
I succeeded in making the train_dqn run but the training did not lead to a good result.

Game number: 005100 Frame number: 01981609 Average reward: 22.1 Time taken: 38.0s

Is this normal ?
Do you have any suggestions ?

Thanks

Question regarding the dueling network architecture part

Hi, I found below code in the network part of train_dqn.py

###########################################################

Split into value and advantage streams

val_stream, adv_stream = Lambda(lambda w: tf.split(w, 2, 3))(x) # custom splitting layer
##############################################################################

It looks like the source from hidden network was divided into 2 different partial parts then one feed to state value, another one to adv value. I have also checked other implementations and paper. It looks like each flow should be the complete copy of the hidden layer rather than partial of it. Can i ask why you want to split it rather than feed the same whole data flow to both stat and adv?

Many thanks!
Edward

Problem with DCGANS tutorial

Hi,

I was trying flow the tutorial for the DCGAN but I have hit a problem.

When I run the following line gan_output = discriminator(fake_image)
I get the error Dimensions must be equal, but are 3 and 3072 for 'sequential_5/dense_14/MatMul' (op: 'MatMul') with input shapes: [?,3], [3072,1024].
I have checked the numbers and they line up with what is defined above in the tutorial.

Can you help?

Cheers

More informative console output DQN tutorial

Dear Sebastian Theiler,

thanks for your great DQN tutorial example. I was wondering if it is possible to suppress the output in the console that looks something like:
1/1 [==============================] - 0s 26ms/step
1/1 [==============================] - 0s 25ms/step
1/1 [==============================] - 0s 22ms/step
1/1 [==============================] - 0s 25ms/step
1/1 [==============================] - 0s 23ms/step
1/1 [==============================] - 0s 21ms/step
1/1 [==============================] - 0s 21ms/step
1/1 [==============================] - 0s 22ms/step
1/1 [==============================] - 0s 22ms/step
1/1 [==============================] - 0s 24ms/step
1/1 [==============================] - 0s 23ms/step
1/1 [==============================] - 0s 27ms/step
1/1 [==============================] - 0s 24ms/step
1/1 [==============================] - 0s 24ms/step
1/1 [==============================] - 0s 28ms/step
1/1 [==============================] - 0s 26ms/step
1/1 [==============================] - 0s 25ms/step
1/1 [==============================] - 0s 26ms/step
1/1 [==============================] - 0s 32ms/step
1/1 [==============================] - 0s 25ms/step
1/1 [==============================] - 0s 25ms/step
1/1 [==============================] - 0s 31ms/step
1/1 [==============================] - 0s 28ms/step
1/1 [==============================] - 0s 31ms/step
1/1 [==============================] - 0s 27ms/step
1/1 [==============================] - 0s 34ms/step
1/1 [==============================] - 0s 30ms/step
1/1 [==============================] - 0s 24ms/step
1/1 [==============================] - 0s 25ms/step
1/1 [==============================] - 0s 23ms/step
1/1 [==============================] - 0s 25ms/step
1/1 [==============================] - 0s 24ms/step
1/1 [==============================] - 0s 23ms/step
1/1 [==============================] - 0s 24ms/step
1/1 [==============================] - 0s 22ms/step
1/1 [==============================] - 0s 22ms/step
1/1 [==============================] -

or make it more informative for each line. Seems to come from Keras?

Every once in a while I see a line like

1/1 [==============================] - 0s 24ms/step
1/1 [==============================] - 0s 25ms/step
1/1 [==============================] - 0s 28ms/step
Game number: 000580 Frame number: 00104874 Average reward: 1.3 Time taken: 12.0s
1/1 [==============================] - 0s 29ms/step
1/1 [==============================] - 0s 27ms/step
1/1 [==============================] - 0s 25ms/step
1/1 [==============================] - 0s 30ms/

coming by.

Or is something going horribly wrong in my environment and console output during training should not be looking like this?

RGB image

looks like target-network and online-network gets updated at the same frequency UPDATE_FREQ

looks like target-network and online-network gets updated at the same frequency UPDATE_FREQ

DQN rank issue