In November of 2022, I began to relearn machine learning. You may misunderstand this as simply refreshing or revising my knowledge, but believe it when I say I am actually “relearning” machine learning, and in this post, I will share the motivation for this.

The motivation

When I discovered machine learning in 2020, I was fascinated by the concept of making predictions based on the patterns identified from past data; I thought anything could be predicted when you throw machine learning into enough data, so I began to learn what I thought was to be known.

The way I learn effectively is what I like to call situational learning: It is when I learn a concept to apply it to a project or a particular problem at hand. It has helped me move fast on projects while learning new things.

Back to my story: While I was getting the expected results, I didn’t really understand why they were the expected results, why they came out the way they did or why anyone thinks this was the best approach to solving such problems – perhaps except for the fact that the course tutor said so. So when something went wrong, I found myself hopefully wandering in a maze of stack overflow posts just to copy and paste the most upvoted reply, and if it works, it does. We move – until we can’t.

It later dawned on me that my knowledge of ML was inadequate and I didn’t really have the knowledge of ML, just ML libraries and frameworks; my knowledge of ML wasn’t deep enough to commit myself to a long-term ML-focused project and my approach to quickly learning things to get other things out of the way will not apply here. It became clear that the way I was learning and applying ML wasn’t sustainable on the long run. Should a project become large and delicate enough, the stakes would also become higher,and thus the cost of breaking something after blindly following a Youtube video or Stack Overflow post becomes heavier. No one wants that to happen, I don’t want that to happen.

If there was a Newton’s Fourth Law, it would be:

“Knowledge of ML Libraries ≠ Knowledge of ML”

Why is it even important?

You see, I have realised that there will be a need to build or help build more complex intelligent domain-aware systems, and me being an ML buffoon will no longer cut it, so to put myself in a good position of being able to address these future problems without looking silly, even if from a systems perspective (which is my thing), I know that I need to have a good grasp of the foundational concepts of machine learning.

The method

I don’t want this post to be long as I am not a sucker for long posts myself, but here is how I have been curing my ignorance:

1. Statistics and probability: I have refreshed my knowledge on concepts like probability distribution, probability theory as well as the principles of statistical learning and I see that everything in ML makes sense because of it. Undergraduate text books were sufficent for these but also An Introduction to Statistical Learning by James and co, is also a fantastic resource.

2. Theories of Machine learning: This helped me answer why anyone thought regression was a good idea (it was). It also helped me understand how some machine learning methods were derived mathematically. My books of choice are Artificial Intelligence With an Introduction to Machine Learning by Neapolitan and Jiang and Deep Learning by Ian Goodfellow, Yoshua Bengio (I am a huge fan of them both) and Aaron Courville. I also have some little known books on ML theory but they are just a matter of preference after reading the ones above, you will be able to spot a good book on ML. Maybe I will add a reading section to this blog🤞

3. Practical Machine Learning: While I was learning the theories, I was revisiting my sources on practical guides. Introduction to Machine Learning with Python by C. Muller and Sarah Guido is still my best-money-spent. While I can’t list them all, the best free resource by far are the Fast.AI courses on practical Deep Learning for Coders and the rich catalouge of courses on Coursera and Youtube. One strategy I have adopted is to return to theoretical sources when a concept in being made reference to in a practical material I am studying, It gives me background knowledge.

4. Papers: When you begin to enjoy your newly found feeling of enlightenment, you will naturally yearn for more, and where else can you get more than the outlet of crude excellence. I have found that the original Deep learning papers by Prof. Hilton, Prof. Bengio and Yann LeCun can be very insightful. Although, I haven’t consumed as much of any of these as I would have loved to but I plan do better.

5. Practise: A lot of it; from unsuccessfully (and perhaps unnecessarily) trying to re-write some ML algorithms from scratch (which I will still do) to working on A LOT of practical projects, except that I am now less dumb.

Some of the best engineers have also followed the path of learning machine learning from scratch; John Carmack, Greg Brockman, Suhail Dorsil and many more understand that there my be more overlooked engineering problems than there are scientific ones. It might just be a massive systems architecture and engineering problem after all.

Future

Since January of 2023, I have been doing free consulting for some graduate students building on an idea that applies ML in the healthcare sector (It is not a startup). It has been really helpful in forcing me out of my comfort zone of Kaggle data to work in the real world (maybe I should be the one paying them🤔). In 5 months we have achieved a lot with my 2% knowledge of ML (imagine when I know up to 4% 😮).

My real interest is in decision intelligence; applying ML to real life decision making cases, so I may be gravitating towards applied ML in that domain. Thanks for reading this far.