Machine Learning and its intuition

Machine learning has come a long way from being considered as rocket science to a reliable and diverse business tool. It has proven to be an instrument that amplifies multiple elements of the business operation.

Machine learning for any business is all about creating an algorithm that can learn from data to make a valid prediction. These algorithms are quite different from conventional programming approaches. In machine learning, we do not tell a machine about what exactly to do as a step-by-step guide. Instead, we use a generic algorithm on a dataset and the algorithm (machine) tries to find the logic (pattern) in the data all by itself. This process is sometimes aided with a little supervision, sometimes none at all.

“Machine Learning is identifying meaningful patterns in data.”

This blog aims at telling you about some fundamentals used in machine learning through a toy example.

What is Machine Learning?

After listening to a song on Spotify, did you ever notice how it automatically recommends a song similar to the one you were looking for? Or did you notice songs recommendation under the banner “based on your recent listening”? Did you ever wonder how do they do that? This is machine learning for providing user recommendations.

Do you get coupon notifications and messages from various travel apps? Do you believe they send coupons to everyone? No, they send it to only those who have higher chances of traveling in the coming days. This is called targetted marketing and can be efficiently done by tools and techniques of machine learning.

In short, machine learning is all about identifying patterns in the data to make a useful prediction. In diverse business applications, it is used in order to utilize and monetize the collected data. For example,

• Predicting the expected share market return at EOD
• Identifying the best combination of items to put up on sale
• Filtering spam emails

Let’s try to work out a simple toy example to build an analogy between a human process and a machine process to understand and utilize the data.

An analogy of Machine Learning

Imagine one day, you realize that lately, you have been receiving a lot of spam emails. You seem frustrated to get these emails daily now, so you decide to filter such annoying emails automatically from your inbox.

At first, you manually classified your historical emails as either Spam or Not Spam. This process is called labeling, and the responses (Spam and Not Spam) are called labels. Then you try to find patterns in the body of spam emails. You noticed that spam emails usually contain stock words like “Hurray!”, “subscribe”, “offer” etc. Also, regular emails contain polite words/phrases like “please”, “awaiting your response” etc. These different combinations of words are called features. In this case, the number of stock words and the number of polite words are two different features that we have. You plotted each of these emails against these features which looked like this:

You now noticed that you had found a way to filter spam emails from others based on these words/phrases.

Of course, in this case, you were lucky to find the relevant words that could distinguish between the two. Now imagine you had a humongous amount of data to start your exploration. So instead of you picking out the essential features (stock words and/or polite words) from the emails, you have asked a machine to do this. This is the general process of Machine Learning.

In this process, you feed the data features into an algorithm in a step called training. During training, the algorithm gradually determines the relationship between the multiple features of the data. This relationship is called the model. This model can be linear (straight line) or non-linear (complex).

Often in machine learning, this model is very complex or wavy. The following diagram explains the difference between simple and complex models.

Whenever any new input data is fed to this model, it makes a prediction basis the relationship learned. This prediction is then evaluated for its correctness and if acceptable, the corresponding model is deployed for production.

Conclusion

In this article, I have tried to provide an overview of machine learning and the various steps that go in a machine learning process. Of course, the process mentioned above is just a helicopter-view of an actual machine learning process as there are many factors involved in it. In order to keep it simple and intuitive enough, I have skipped those parts in this article.