4.1.2Structure of a Typical Classical Machine Learning Problem

Terminology Used in Classical Machine Learning introduced the words used to describe data, models, and evaluation. This module shows how those pieces fit together.

Classical machine learning forms the base for the intuition behind the problem structures you will encounter throughout this guide. It gives you a clean way to think about data, models, training, predictions, and evaluation. These ideas continue to appear in deep learning, NLP, computer vision, and audio problems, even when the models become much larger.

It can easily be called one of the most important parts of the guide for anyone who is just starting. Once you understand this structure, a new problem stops looking like one large block. You can break it into smaller parts and decide what each part needs.

What is a model really?

Classical machine learning also builds the base understanding of what a model really is. A model family is a class of functions with a fixed general form. Training chooses the parameters that turn that general form into one particular fitted function.

That fitted function tries to capture patterns in the inputs that are useful for predicting the output. In simple notation, it takes an input $x$ and produces a prediction $\hat{y}$:

$$\hat{y} = f(x).$$

Let us take a simple example where the input is $x$ and the required output is $x^2$. If we train a linear model using only $x$ as its feature, the best it can produce is a straight line:

$$\hat{y} = wx + b.$$

No single straight line can exactly match the curve $y=x^2$ over a range of values. The model family cannot represent the relationship we need. However, if we give the same linear model $x^2$ as an additional feature, it can learn the relationship exactly. This is a small example of feature engineering: changing the input representation so that the model can use it.

Why the inputs and outputs matter

Before preprocessing or model training begins, you should understand the input, the target, and the evaluation metric. These decide what information must be preserved when you clean the data and which models can solve the task.

The input

The input tells you what kind of data the model must handle. It may be a table, an image, a piece of text, an audio clip, or a combination of them. Its shape matters too. You should know how many samples and features you have, which values are numerical or categorical, whether values are missing, and whether the features have an order or some other structure.

These properties affect the models you should try. A regularized linear model such as ridge regression can work well on high-dimensional data when the useful relationship is mostly linear. XGBoost can work better when tabular features interact in nonlinear ways. However, the fact that a dataset has 1,000 features is not enough to declare either model better. The number of samples, the feature representation, the noise, and the underlying pattern all matter.

We will understand these choices in the later model modules. For now, remember the main point: the model must suit both the kind of input and the relationship you expect it to learn.

The output

The target tells you what the model must predict. Different targets create different kinds of problems:

For example, logistic regression is built for classification, while linear regression is built for continuous outputs. Multi-output is not a separate alternative to classification and regression. It only means that each sample has several targets, and those targets may still be discrete or continuous.

The target type and the submission format are also different things. The target tells you what must be predicted. The submission format tells you how those predictions must be written, such as a CSV column, a JSON list, or a file for every sample. You should inspect both at the start, but the target type is what mainly constrains the model.

The four major parts of a problem

Now that we understand why classical machine learning matters and what a model essentially is, let us look at the components of a problem.

Most problems in this domain have four major parts, or a subset of them:

1. Input and target: Understand the samples, features, available labels, and required predictions.
2. Preprocessing and representation: Clean the data and turn every sample into a form the model can use. This may include scaling, encoding, feature engineering, or feature extraction.
3. Model training: Choose a suitable model family and fit its parameters using the training data.
4. Prediction and evaluation: Generate predictions and measure them using the metric specified by the task.

The metric is listed in the fourth part, but it affects the whole pipeline. It tells you how to compare models, which mistakes are expensive, and what kind of prediction the grader expects. You should understand it before training anything.

Problems do not emphasize every part equally

Every machine learning task eventually needs predictions or some other result. However, the part where you have freedom can change a lot.

The first two examples may look like tasks with “only preprocessing,” but training and prediction still happen. The important difference is that the model is fixed, so most of your problem-solving effort goes into the representation given to that model.

The same structure appears elsewhere

Deep learning, NLP, computer vision, and audio problems can also be framed using the same four parts. A neural network may learn part of the preprocessing and feature representation by itself, but it still receives inputs, trains on some objective, produces outputs, and is evaluated by a metric.

If you can trace these four components and identify where the real freedom lies, you can apply the pipelines and algorithms you know much more efficiently. The next module, Approaching a Typical Classical Machine Learning Problem, turns this structure into a complete step-by-step workflow.