Data in, intelligence out: Machine learning pipelines demystified

Data plus algorithms equals machine learning, but how does that all unfold? Let’s lift the lid on the way those pieces fit together, beginning to end

It’s tempting to think of machine learning as a magic black box. In goes the data; out come predictions. But there’s no magic in there—just data and algorithms, and models created by processing the data through the algorithms.

If you’re in the business of deriving actionable insights from data through machine learning, it helps for the process not to be a black box. The more you know what’s inside the box, the better you’ll understand every step of the process for how data can be transformed into predictions, and the more powerful your predictions can be.

Devops people speak of “build pipelines” to describe how software is taken from source code to deployment. There’s also a pipeline for data as it flows through machine learning solutions. Mastering how that pipeline comes together is a powerful way to know machine learning itself from the inside out.

The machine learning pipeline

machine learning pipeline IDG

To continue reading this article register now