🏠

1

2

3

4

6

7

8

Preprocessing & Feature Engineering

It’s time to execute the findings of the previous step and begin Feature Engineering by defining functions for data transformations, cleaning, adding, removing, and scaling the data. Feature engineering is the most critical step and where the most time is often spent.

Data-centric focus (vs a model-centric, steps 6 and 7):

• Write functions to transform the data and automate the process for the forthcoming batches of data
• Write functions to clean the data(imputing missing values and handling outliers)
• Write functions to select and engineer features — drop redundant features, format conversion of features, and other mathematical transformations
• Feature scaling — standardize the features
• See also:
  • Video
  • Article
  • Kaggle Course

Feature Engineering is an iterative process that you may have to come back to even after selecting a model.

*click the image to enlarge: