Machine Learning Curriculum
Machine Learning is a branch of Artificial Intelligence dedicated at making machines learn from observational data without being explicitly programmed.
Machine learning and AI are not the same. Machine learning is an instrument in the AI symphony — a component of AI. So what is Machine Learning — or ML — exactly? It’s the ability for an algorithm to learn from prior data in order to produce a behavior. ML is teaching machines to make decisions in situations they have never seen.
Machine Learning in General
Study this section to understand fundamental concepts and develop intuitions before going any deeper.
A computer program is said to learn from experience
E
with respect to some class of tasksT
and performance measureP
if its performance at tasks inT
, as measured byP
, improves with experienceE
.
 Opinionated Machine Learning Course by Fast.ai
 Machine Learning Crash Course with TensorFlow APIs Google’s fastpaced, practical introduction to machine learning
 Artificial Intelligence, Revealed a quick introduction by Yann LeCun, mostly about Machine Learning ideas, Deep Learning, and convolutional neural network
 How do I learn machine learning?  Quora
 Intro to Machine Learning  Udacity hands on scikitlearn (python) programming learning on core ML concepts
 Machine Learning: Supervised, Unsupervised & Reinforcement  Udacity the 2 instructors are hilarious
 Machine Learning Mastery carefully laid out stepbystep guide to some particular algorithms
 Andrew Ng’s Course on Coursera recommended for people who want to know the details of ML algorithms under the hood, understand enough maths to be dangerous and do coding assignments in Octave programming language (Note: Andrew said that you do not need to know Calculus beforehand but he talked about it quite often so you will regret if you don’t know Calculus :laughing:)
 ML Recipes  YouTube Playlist a really nicely designed concrete actionable content for ML introduction
 Machine Learning is Fun Part 1 simple approach to machine learning for nonmaths people
 Machine Learning with Python  YouTube Playlist
 Machine Learning Yearning by Andrew Ng
 Machine Learning Crash Course: Part 1
 https://www.kadenze.com/courses/machinelearningformusiciansandartistsiv
 Rules of Machine Learning: Best Practices for ML Engineering
 Most Shared Machine Learning Content on Twitter For The Past 7 Days
 MIT 6.S099: Artificial General Intelligence This class takes an engineering approach to exploring possible research paths toward building humanlevel intelligence.
Reinforcement Learning
Building a machine that senses the environment and then chooses the best policy (action) to do at any given state to maximize its expected longterm scalar reward is the goal of reinforcement learning.
 OpenAI Spinning Up This is an educational resource produced by OpenAI that makes it easier to learn about deep reinforcement learning (deep RL).

[Basic Reinforcement Learning GitHub](https://github.com/vmayoral/basic_reinforcement_learning) An introduction series to Reinforcement Learning (RL) with comprehensive stepbystep tutorials.  Advanced Topics: RL 2015 (COMPM050/COMPGI13) by David Silver (the guy behind AlphaGo)
 An Introduction Book by Richard S. Sutton and Andrew G. Barto
 Deep Reinforcement Learning: Pong from Pixels
 Lecture 10: Reinforcement Learning  YouTube
 A Survey Paper
 Deep Reinforcement Learning: A Tutorial  OpenAI
 CS 294: Deep Reinforcement Learning, Spring 2017
Deep Learning
Deep learning is a branch of machine learning where deep artificial neural networks (DNN) — algorithms inspired by the way neurons work in the brain — find patterns in raw data by combining multiple layers of artificial neurons. As the layers increase, so does the neural network’s ability to learn increasingly abstract concepts.
The simplest kind of DNN is a Multilayer Perceptron (MLP).
 DeepLearning.ai new 5 courses specialization taught by Andrew Ng at Coursera; It’s the sequel of Machine Learning course at Coursera.

[Advanced Machine Learning Specialization Coursera](https://www.coursera.org/specializations/aml) consists of 7 courses  A friendly introduction to Deep Learning and Neural Networks
 A Neural Network Playground Tinker with a simple neural network designed to help you visualize the learning process
 Deep Learning Demystified  Youtube explain inspiration of deep learning from real neurons to artificial neural networks
 Learn TensorFlow and deep learning, without a Ph.D. This 3hour course (video + slides) offers developers a quick introduction to deeplearning fundamentals, with some TensorFlow thrown into the bargain.
 A Guide to Deep Learning by YN^2 a curated maths guide to Deep Learning
 Practical Deep Learning For Coders Course at Fast.ai taught by Jeremy Howard (Kaggle’s #1 competitor 2 years running, and founder of Enlitic)
 Deep learning  Udacity recommended for visual learner who knows some ML, this course provides high level ideas of deep learning, dense intuitive details put in a short amount of time, you will use TensorFlow inside the course
 Deep Learning Resources (Papers, Online Courses, Books)  deeplearning4j.org
 Introduction to Deep Neural Networks  deeplearning4j.org
 NVIDIA Deep Learning Institute because GPU are efficient at training Neural Networks, NVIDIA notices this market !
 Deep Learning Book recommended for math nerds who want to understand the theoretical side, the book is crafted by our deep learning wizards (Goodfellow, Bengio and Courville)
 Unsupervised Feature Learning and Deep Learning
 DeepMind Publications
 DeepLearning.TV  YouTube broad overview of deep learning, no implementation, just pure ideas
 CS224d: Deep Learning for Natural Language Processing
 Deep Learning Summer School, Montreal 2015
 UFLDL Deep Learning Tutorial
 Neural networks class  YouTube Playlist
 http://deeplearning.net/
 https://developer.nvidia.com/deeplearning
 http://neuralnetworksanddeeplearning.com/index.html a handson online book for deep learning maths intuition, I can say that after you finish this, you will be able to explain deep learning in a fine detail.
 https://github.com/lisalab/DeepLearningTutorials
 https://www.kadenze.com/courses/creativeapplicationsofdeeplearningwithtensorflowi You will implement a lot of things inside TensorFlow such as Autoencoders, Convolutional neural net, Feedforward neural nets, Generative models (Generative Adversarial Networks, Recurrent networks), visualizing the network, etc. You will have lots of assignments to finish. The course director (Parag) is also approachable and active.
 Deep Learning Lectures by Yann LeCun Why not learn Deep Learning from the guy who invented Convolutional nets?
 6.S094: Deep Learning for SelfDriving Cars a course at MIT
 The Neural Network Zoo a bunch of neural network models that you should know about (I know about half of them so don’t worry that you don’t know many because most of them are not popular or useful in the present)
 6.S191: Introduction to Deep Learning a course for 2017
 The GAN Zoo a list of GAN papers which have their own name
 A Microsoft CNTK tutorial in Python – build a neural network a comprehensive introductory tutorial for Microsoft’s CNTK framework
 https://deeplearning.mit.edu/ MIT Deep Learning taught by Lex Fridman

[Intro to TensorFlow for Deep Learning Udacity](https://www.udacity.com/course/intrototensorflowfordeeplearning–ud187) By TensorFlow
Convolutional Neural Networks
DNNs that work with grid data like sound waveforms, images and videos better than ordinary DNNs. They are based on the assumptions that nearby input units are more related than the distant units. They also utilize translation invariance. For example, given an image, it might be useful to detect the same kind of edges everywhere on the image. They are sometimes called convnets or CNNs.
 CS231n: Convolutional Neural Networks for Visual Recognition a course taught at Stanford university
 How Convolutional Neural Networks work  Youtube technical explanation including pooling operations, ReLU, fully connected layer, optimization using gradient descent
 Neural Network that Changes Everything  Computerphile
 A Beginner’s Guide To Understanding Convolutional Neural Networks
 Deep Learning for Computer Vision (Andrej Karparthy, OpenAI) this is my most favorite video of convolutional net. Andrej explains convnet in detail answering all the curious questions that one might have. For example, most articles only talk about convolution in grayscale image, but he describe convolution in images with color channels as well. He also talks about the concerns and the assumptions that convnets make. This is a great lecture!
 Understanding Neural Networks Through Deep Visualization explains how to visualize a convnet using various techniques
 Convolutional Neural Networks Tutorial in TensorFlow gives an introduction to CNNs for beginners in TensorFlow
 Capsule Networks (CapsNets) – Tutorial CapsNets are a hot new architecture for neural networks, invented by Geoffrey Hinton, one of the godfathers of deep learning.
Recurrent Neural Networks
DNNs that have states. They also understand sequences that vary in length. They are sometimes called RNNs.
 http://karpathy.github.io/2015/05/21/rnneffectiveness/
 http://colah.github.io/posts/201508UnderstandingLSTMs/
 http://www.wildml.com/2015/09/recurrentneuralnetworkstutorialpart1introductiontornns/
Best Practices
Unsupervised Domain Adaptation
Unsupervised Domain Adaptation is a type of Transfer Learning that applies a model that was trained on source dataset to do well on a target dataset without any label on the target dataset. It’s one of the technique that is practically useful in the real world when the cost of labeling target dataset is high. One of the example is to train a model on synthetic data with label and try to use it on real data without label.
 https://paperswithcode.com/task/unsuperviseddomainadaptation
 https://github.com/zhaoxin94/awsomedomainadaptation#unsupervisedda
 https://github.com/barebell/DA
 https://github.com/bbdamodaran/deepJDOT one of the easytouse implementation on keras of a unsupervised domain adaptation technique that I tried before and work great
Open Source Trained Models
 https://modelzoo.co/ Model Zoo
 SelfDriving Car by Udacity
 deepdream inceptionism  a deep model that takes an image and hallucinates animals/buildings from it
 Magenta: Music and Art Generation with Machine Intelligence
 SyntaxNet (Parsey McParseface)
 Neural Storyteller convert image caption into a romantic one
 https://github.com/facebookresearch/deepmask sharp object segmentation on image at pixellevel
 https://github.com/facebookresearch/multipathnet convnet for classifying DeepMask+SharpMask model above
 https://github.com/facebookresearch/fastText Library for fast text representation and classification; Tutorial
 https://github.com/tensorflow/models
 https://github.com/google/seq2seq A generalpurpose encoderdecoder framework for Tensorflow (you need to provide it a sequence of vectors as input and also provide sequence of vectors as output to train this deep RNN)
 https://github.com/phillipi/pix2pix Imagetoimage translation using conditional adversarial nets; TensorFlow port of pix2pix; Watch the presentation of this work: Learning to see without a teacher
Interesting Techniques & Applications
 https://paperswithcode.com/ A list of papers with evaluation metrics, and state of the art comparison.
 http://deeplearninggallery.com/ Deep Learning Gallery  a curated list of awesome deep learning projects
 How do GANs intuitively work? this is my article explaining GANs, I try to be as intuitive as possible, GANs are so awesome that I can’t just ignore and not talk about it. You can also watch the official tutorial by Ian Goodfellow.
 https://deepart.io/ transfer image style to other image
 http://www.somatic.io/
 WaveNet: A Generative Model for Raw Audio by DeepMind
 Jukedeck Musical AI
 StackGAN StackGAN: Text to Photorealistic Image Synthesis with Stacked Generative Adversarial Networks
 Learning a Probabilistic Latent Space of Object Shapes via 3D GenerativeAdversarial Modeling 3DGAN, generating 3D models using GAN
 BEGAN: Boundary Equilibrium Generative Adversarial Networks this type of GAN generates more realistic images than ordinary GAN
 Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) an activation function that is trying to be better than its predecessor (ReLU) in a lot of way
 Google Duplex: An AI System for Accomplishing RealWorld Tasks Over the Phone
Nice Blogs & Vlogs to Follow
 http://colah.github.io/ this guy knows how to explain
 https://karpathy.github.io/ this guy has taught some courses on Deep Nets
 http://ruder.io Sebastian Ruder’s Deep Learning and NLP blog
 http://www.wildml.com/
 https://machinelearningmastery.com/ Contains a lot of content and beautiful website
 https://adeshpande3.github.io/adeshpande3.github.io/
 http://culurciello.github.io/
 Sirajology’s YouTube Playlists lots of dense short hilarious introduction to ML
 Two Minute Papers on Deep Learning Playlist
 http://www.leviathan.ai/
 Welch Labs
 Distill.pub A modern medium for presenting research in Machine Learning
 deeplearn.org Deep Learning Monitor; news about deep learning papers and tweets
Impactful People
 Geoffrey Hinton, he has been called the godfather of deep learning by introducing 2 revolutionizing techniques (ReLU and Dropout) with his students. These techniques solve the Vanishing Gradient and Generalization problem of deep neural networks. He also taught a Neural Networks course at Coursera.
 Yann LeCun, he invented CNNs (Convolutional neural networks), the kind of network that is really popular among computer vision developers today
 Yoshua Bengio another serious professor at Deep Learning, you can watch his TEDx talk here (2017)
 Andrew Ng he discovered that GPUs make deep learning faster. He taught 2 famous online courses, Machine Learning and Deep Learning specialization at Coursera.
 Juergen Schmidhuber invented LSTM (a particular type of RNN)
 Jeff Dean, a Google Brain engineer, watch his TEDx Talk
 Ian Goodfellow, he invented GANs (Generative Adversarial Networks), is an OpenAI engineer
 David Silver this is the guy behind AlphaGo and Artari reinforcement learning game agents at DeepMind
 Demis Hassabis CEO of DeepMind, has given a lot of talks about AlphaGo and Reinforcement Learning achievements they have
 Andrej Karparthy he teaches convnet classes, wrote ConvNetJS, and produces a lot of content for DL community, he also writes a blog (see Nice Blogs & Vlogs to Follow section)
 Pedro Domingos he wrote the book The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World, watch his TEDx talk here
Libraries, Frameworks and Services
Glancing at their GitHub statistics can give you an estimate for how active/popular each of them is.
 Python Deep Learning Frameworks Reviewed 2017 read this to decide which framework is appropriate for you
 scikitlearn (Python) general machine learning library, high level abstraction, geared towards beginners
 TensorFlow (Python); Learning TensorFlow; Installing on Windows; Fresh Install on Ubuntu 16.04; Serving; Awesome TensorFlow; computation graph framework built by Google, has nice visualization board, probably the most popular framework nowadays for doing Deep Learning
 Theano (Python) another popular deep learning framework; Deprecated! See this announcement from Yoshua Bengio for more info.
 Lasagne (Python) Lightweight library to build and train neural networks in Theano
 Keras: Deep Learning library for Theano and TensorFlow (Python)
 Caffe (Python) originally created to tackle computer vision problems
 Microsoft Cognitive Toolkit (CNTK) Microsoft’s framework (previously known as Computational Network Toolkit)
 Torch (LuaJIT) the most popular scientific computing framework for LuaJIT
 PyTorch (Python) PyTorch is a deep learning framework that puts Python first.
 MXNet: A Scalable Deep Learning Framework supports multiple language interfaces
 MinPy (Python) NumPy interface with mixed backend execution (MXNet, autograd)
 Chainer (Python) A flexible framework of neural networks for deep learning
 Kur (YAML, Python) Descriptive Deep Learning, get started in minutes because you don’t need to code!
 DeepLearning4j (Java) not so popular, preferable for you if you like Java
 Bonsai (Inkling) a simplification layer for machine learning
 bitfusion  Software to Manage Deep Learning & GPUs contains Amazon Machine Images for many Deep Learning libraries including TensorFlow
 FloydHub a Heroku for Deep Learning (You focus on the model, they’ll deploy)
 Lobe a draganddrop tool for machine learning
 comet.ml Comet lets you track code, experiments, and results on ML projects. It’s fast, simple, and free for open source projects.
 MLflow MLflow (currently in beta) is an open source platform to manage the ML lifecycle, including experimentation, reproducibility and deployment. It currently offers three components: MLflow Tracking, MLflow Projects, MLflow Models.
 wav2letter++ Open sourcing wav2letter++, the fastest stateoftheart speech system, and flashlight, an ML library going native
 Ludwig Ludwig is a toolbox built on top of TensorFlow that allows to train and test deep learning models without the need to write code. It’s by Uber team.
 spektral A Python framework for relational representation learning in Keras
AutoML
Make machine learns without the tedious task of feature engineer, model selection, and hyperparameter tuning that you have to do yourself.
Let the machine does machine learning for you!
 https://www.automl.org/ Find curated list of AutoML libraries and researches
 https://github.com/jhfjhfj1/autokeras As of writing (24 August 2018), this library is pretty premature as it can only does classification.
 https://github.com/maxpumperla/hyperas Keras + Hyperopt: A very simple wrapper for convenient hyperparameter optimization
 https://github.com/automl/autosklearn/ Does not run on Windows, you need to install WSL (Windows Subsystem for Linux) to use it
 https://github.com/EpistasisLab/tpot Run thousands of machine learning pipelines and output the code for you
 https://github.com/ClimbsRocks/auto_ml Read what the author think about the comparison between tpot and autosklearn
 https://github.com/dmlc/xgboost eXtreme Gradient Boosting, not actually AutoML but it is quite popular
 https://mljar.com/ Commercial solution for AutoML that integrates multiple libraries like sklearn, keras, tensorflow, etc.
 https://github.com/autonomio/talos Hyperparameter Scanning and Optimization for Keras
CuttingEdge Research
Steal the most recent techniques introduced by smart computer scientists (could be you).
 https://research.facebook.com/ai/
 http://research.google.com/pubs/MachineIntelligence.html
 https://deepmind.com/research/ Research of DeepMind company
 https://www.openai.com/
 https://www.openai.com/requestsforresearch/
 State of the art performance on each ML task
 Stateoftheart result for all Machine Learning Problems
 http://www.gitxiv.com/
 http://www.arxivsanity.com/ Arxiv Sanity Preserver built by Andrej Karparthy for listing papers in arxiv
Practitioner Community
 https://www.kaggle.com
 https://gym.openai.com
 https://universe.openai.com/
 /r/MachineLearning
 https://www.facebook.com/groups/DeepNetGroup/
 https://www.facebook.com/groups/1892696574296664/ a Facebook group talking about SelfDriving Cars but it’s usually not specific to that
Thoughtful Insights for Future Research
 The Consciousness Prior by Yoshua Bengio
 What Can’t Deep Learning Do? a list of problems that deep learning faces
 Pedro Domingos: “The Master Algorithm”  Talks at Google
 The AI Revolution: The Road to Superintelligence
 https://ai100.stanford.edu/2016report
 Why does Deep Learning work so well?  The Extraordinary Link Between Deep Neural Networks and the Nature of the Universe
 These are three of the biggest problems facing today’s AI
 Four Questions For: Geoff Hinton Geoff Hinton is referred to as “godfather of neural networks”
 What product breakthroughs will recent advances in deep learning enable?  Quora
Uncategorized
 Artificial Intelligence: A Modern Approach (Online Book)
 The Principles of Modern Game AI
 Scipy Lecture Notes
 https://www.youtube.com/user/aicourses
 The Fundamentals of Neuroscience learn how our brain works so that you can discover new deep learning breakthrough
Other Big Lists
 https://github.com/jindongwang/transferlearning
 https://github.com/kmario23/deeplearningdrizzle
 http://aireads.top AI/ML Reads  a directory of artificial intelligence/machine learning resources
 https://github.com/ZuzooVn/machinelearningforsoftwareengineers
 https://github.com/josephmisiti/awesomemachinelearning
 https://github.com/ujjwalkarn/MachineLearningTutorials
 https://github.com/terryum/awesomedeeplearningpapers
 https://github.com/ChristosChristofidis/awesomedeeplearning
 https://github.com/DeveloperY/csvideocourses#machinelearning
 Open Source Deep Learning Curriculum an open source deep learning curriculum which contains a lot of course recommendations for you to consider enrolling. (Many recommendations are the same as in this list)
 Deep Learning Resources by Jeremy D. Jackson
 https://github.com/songrotek/DeepLearningPapersReadingRoadmap
 http://aimedicines.com/2017/03/17/allairesourcesatoneplace/
 A Large set of Machine Learning Resources for Beginners to Mavens
 Skynet Today Accessible and informed coverage of the latest AI hype and panic
 https://github.com/aikorea/awesomerl Awesome Reinforcement Learning
 https://github.com/artix41/awesometransferlearning Awesome Transfer Learning
I am confused, too many links, where do I start?
If you are a beginner and want to get started with my suggestions, please read this issue: https://github.com/off99555/machinelearningcurriculum/issues/4
Disclaimer
This is a really big list because I also point to other people’s list to ensure that most of the resources are accessible from this page without you looking anywhere else.
Most of these resources are the ones I enjoy reading/watching. I wouldn’t put something that I am not interested in here.
NOTE: There is no particular rank for each link. The order in which they appear does not convey any meaning and should not be treated differently.
How to contribute to this list
 Fork this repository, then apply your change.
 Make a pull request and tag me if you want.
 That’s it. If your edition is useful, I’ll merge it.
Or you can just submit a new issue containing the resource you want me to include if you don’t have time to send a pull request.
The resource you want to include should be free to study.