Data Engineering Life CycleΒΆ

Let us first understand the Data Engineering Life Cycle. We typically read the data, process it by applying business rules and write the data back to different targets

  • Read the data from different sources.

    • Files

    • Databases

    • Mainframes

    • APIs

  • Processing the data

    • Row Level Transformations

    • Aggregations

    • Sorting

    • Ranking

    • Joining multiple data sets

  • Write data to different targets.

    • Files

    • Databases

    • Mainframes

    • APIs

Quick Recap of Python Getting Started - Python and pip