Udacity nanodegree review — Deep Reinforcement Learning

I have written a few Udacity Nanodegree reviews for a few different courses after taking them and I have maintained my position that the Nanodegrees are not differentiating enough to justify its huge premium. However, this tone may change in this review.

Photo from author: Udacity Nanodegree certificate, what you will get if you complete this course.

Firstly, I have always appreciate the fact that online learning materials have been so different compared to my college days that nowadays, even the free materials are so so good.

This is why I have always remained critical of Udacity Nanodegrees when they charge hundreds or even thousands of dollars for the course, when their course materials and content is really only “as good”. However, among the 6 courses I have taken so far, if there is one course that can change my criticism of Udacity Nanodegree, it has to be this Deep reinforcement Learning Nanodegree.

This review will be broken into the following sections:

1. Overall course content and quality
2. Pros and cons when compared to other courses
3. Concluding remarks

Photo from Pixabay: just like the photo portrays, there are many concepts covered in this course and here is a summary of the topics

Course content and quality

Course content summary

The following are the topics covered by this course, to give you a high level idea of what is covered:

Part 1. Introduction to Deep Reinforcement Learning
Part 2. Value-Based Methods
Part 3. Policy-Based Methods
Part 4. Multi-Agent Reinforcement Learning
Part 5 Extracurricular 1. Special Topics in Deep Reinforcement Learning
Part 6 Extracurricular 2. Neural Networks in PyTorch
Part 7 Extracurricular 3. Computing Resources
Part 8 Extracurricular 4. C++ Programming

Course content in details

I have taken notes when I was taking the course and the following screenshots of my notes will give you some more details of what each section covers.

Photo from author: screenshot from course notes

Course quality

On a very high level, I will say the course content is great and well designed. As you can tell, it covers the fundamental topics in deep reinforcement learning and gradually increase in complexity.

It is not stretching you too much as long as you follow along and most of the concepts covered are essentially and useful if you want to get serious about deep reinforcement learning.

The course is a mixture of videos, exercises, text readings and all instructors are lovable and explain concepts really clearly and concisely. It covers what you need to know without going into too much details.

Instructors

The instructors are really good. This is actually one of the differentiating factors of this course, where instructors all “show there faces rather than simply sharing your screen” and I think all of them are approachable and friendly. I have followed most of them on Twitter too.

Projects

There are a total of 3 projects and I personally think this is the most exciting part of the nanodegree. These are project 1 on navigation, project 2 on continuous control and project 3 on collaboration and competition.

You will find a full list of the projects for this project from their official website (https://www.udacity.com/course/deep-reinforcement-learning-nanodegree--nd893) but here is one personal highlight, which I want to talk about in this article.

In the second project, a Deep Q-Network agent was trained to navigate a provided environment, which is a large square world, to collect bananas. The goal of your agent is to collect as many yellow bananas as possible while avoiding blue bananas. A reward of +1 is provided for collecting a yellow banana, and a reward of-1 is provided for collecting a blue banana. To solve the environment, the trained agent must get an average score of +13 over 100 consecutive episodes. While this sounds a bit tedious, but this is my first time training an agent using deep reinforcement neural network and I found it pretty exciting. I got exposure to set up my algorithm in different project files, configure deep Q network algorithm, building architecture for neural network etc.

The project is just challenging enough while guiding you to try the essential concepts.

Picture from Pixabay: next we talk about pros and cons of this nanodegree when compared to other online courses

Pros

1. Projects are reviewed: all your four submissions will be reviewed by a Udacity reviewer with feedback provided. You will only pass if you meet al project requirements. Another plus on the projects is the setup of project requirements, which guide you to think systematically, from the overall objective, to project setup, to architecture design, to parameter configuration, to conclusion and further research.
2. Graduate certificate: if you pass all project reviews, you will get a certificate (similar to the first photo in this article, which is my certificate)
3. Instructors are lovable and (look) friendly: I am following many of them on Twitter because I do miss their energetic voice
4. Learning community: I have to say that the learning community of Udacity is one of the best, partly because there are Udacity mentors answering the questions and most of them have been very good and helpful in answering your questions. The community is very active too, with both students and reviewers actively asking and answering question. It is built within Udacity course and one that most reminds me of stack overflow, which we all know and we all love.
5. Course content covers concepts which are just challenging enough while covering essential topics. Challenging is of course a relative benchmark and beginners may find this course too challenging, in which case I would recommend spending more time on Python and ML fundamentals first. But, from my personal experience, there are plenty of good and fee entry level ML courses online but I have not personally come to good intermediate level ML courses. Please correct me by leaving a comment below but to my best knowledge, this course content is relatively well designed for intermediate ML and is less likely to have free/cheaper alternatives online.

Cons

1. So expensive! There are two options: three month and pay as you go and I will highly recommend, if you are a beginner, to learn from free courses first to familiarize yourself with Python and ML, and then, you can finish this course the assignments much quicker if you choose pay as you go option.

Concluding remarks

I know that I may have been too hard on Udacity Nanodegree because of its huge premium as well as the fact that there are way too many free alternatives.

For example, how can you charge people hundreds and thousands to teach Python, when there are so many free YouTube videos or much cheaper alternatives such as Udemy, Coursera, Edx and so on.

However, to my best knowledge, I found that Deep Reinforcement Learning Intermediate courses covering similar topics are still lacking on the internet and until more free / cheaper alternatives become readily available, I am convinced that the premium of this course can be justified, provided you have the extra purchasing power and want to further up skill yourself in the area of deep reinforcement learning.

Please let me know in the comment session if you have come across good courses covering similar concepts and I will update this article as more information becomes available to me.