Machine Learning: The real-world guide to how it works

Machine learning can be used when a food delivery app predicts the amount of time it will take to deliver an order, when a bank detects an unusual transaction, or when an e-commerce website suggests an item to buy.

So, what is machine learning, anyway?

In its simplest form, machine learning is the ability to create computer systems that learn from data, rather than just from rules explicit to a programmer.

That is a distinction which makes it powerful.

Rule-Based to Learning from Examples.

With traditional software, the program contains a series of instructions that are explicitly followed.

A programmer could write:

Flag the order if the order amount is greater than ₹5,000 and the billing address is not the same as the delivery address.

It's a rule-based system.

A machine-learning system is not like that.

Rather than having to manually specify each fraud rule, one can provide the model with thousands of transactions in the past, and the model is provided with information regarding whether the transactions were fraudulent.

The model searches for patterns that distinguish fraudulent from non-fraudulent actions.

Then it applies them to predictions on new transactions.

The programmer is still the one who designs the system, processes the data, and specifies the learning of the model. However, the final decision rules are not handcrafted, but are learnt from examples.

What is the meaning of “Learning”?

A machine learning model doesn't learn like a human being.

It optimizes parameters of a mathematical model to make the prediction more accurate based on a selected objective function.

Suppose we want to create a model to predict the rent of an apartment in Bengaluru.

The data could contain:

● Floor area

● Number of bedrooms

● Locality

● Proximity to a commercial area

● Building age

● Actual monthly rent

The model forecasts and compares these to the current rents.

Bad predictions lead to the adjustment of the learning process, which results in an adjusted model. It discovers relationships across many examples that lead to minimizing prediction error.

That's what training is.

Training data is different from the new data.

The only point of a model is that it should do well on data it has never seen before, or at least not memorised while learning.

This is referred to as generalisation.

Let's say a model has a very high accuracy on historical house prices but has a very low accuracy on new houses.

That model has failed to get the necessary useful pattern. Perhaps it's overfitting on the training set.

That's why we almost always partition data into training and validation/test sets for our machine-learning projects.

The data is used for training, which allows the model to learn from the data.

The unseen data is used to test if the learned data generalizes to new data.

The Primary Classes of Learning

There are various types of feedback in different machine-learning tasks.

Supervised learning is based on labelled examples. A model can be trained to predict customer churn, fraud, house prices, or monthly sales.

Unsupervised learning is a type of learning that does not require a labelled outcome and looks for structure, like customer segments or unusual behavior.

Reinforcement learning gets better as it learns by doing and rewards, and is appropriate for problems where decisions can be repeated.

Self-supervised learning is the generation of learning signals from the raw data alone and is of paramount importance in current models of language and foundation models.

The correct method will vary depending on the problem and the data supplied.

Where Machine Learning Is Used

Today, Machine Learning is already integrated into numerous systems.

It is used by banks for fraud detection and risk modelling.

Retailers leverage it for recommendations, demand forecasting, and customer segmentation.

They can use it for predictive maintenance and quality control by manufacturers.

Models can be used by healthcare systems for image analysis, risk scoring, and operational planning.

Predictive models for demand forecasting, routing, and delivery estimates are used by logistics companies.

What's crucial is that machine learning isn't just one application.

It is a broad method of building systems that enhance prediction or decision-making based on data.

Machine Learning is not Artificial Intelligence.

While there are similarities between machine learning and artificial intelligence, they are not the same.

The general concept of creating systems that carry out functions related to intelligence is artificial intelligence.

One of the primary methods of developing such systems is machine learning.

An easy spam classifier can apply machine learning.

A large language model also relies on machine-learning techniques, albeit on a much larger scale and complexity.

Thus, machine learning falls under the umbrella of AI.

What is a Good Model?

A something very complicated algorithm is quite insufficient.

A good machine-learning model should:

● Data relevant and reasonably reliable

● An unambiguous target

● Appropriate input features

● Suitable evaluation metrics

● Testing using data that were not seen during the training phase.

● Monitoring after deployment

Even the best model will be useless if the data are bad or the problem is not well defined.

In reality, many times working in machine learning is more about the problem itself, getting data ready and assessing results rather than selecting an algorithm.

The significance of the concept for learners

Machine learning is a fundamental skill for Indian students pursuing data science or AI as it bridges the gap between statistics, programming, and real-world decision-making.

While it is important to know Python and SQL, concepts like overfitting, feature selection, model evaluation, and bias also play a significant role.

Programmes such as those offered by AnalytixLabs typically build from statistics and programming into supervised learning, unsupervised learning, deep learning, and newer AI techniques.

The best way to understand machine learning is as follows:

The rules are programmed traditionally. Machine learning provides examples to the computer and allows it to learn useful rules from the data.