DataOps is the transformation of data processing from a craft with manual processes to an automated data factory. Lean principles, which have proven successful in manufacturing, are equally applicable for data factories. We will describe how lean principles can be applied in practice for successful data processing.
- Data engineering is the craft of building resilient data factories.
- DataOps is lean thinking applied to data processing.
- Identifying and eliminating waste is a key element for successful DataOps.
- The most common forms of waste that prevent DataOps success.