Data Processing - OverviewΒΆ

As part of this section we will get an overview about Data Processing using Spark with Python.

  • Pre-requisites and Objectives

  • Starting Spark Context

  • Overview of Spark read APIs

  • Understand airlines data

  • Inferring Schema

  • Previewing airlines data

  • Overview of Data Frame APIs

  • Overview of Functions

  • Overview of Spark Write APIs

  • Reorganizing airlines data

  • Previewing reorganized data

  • Analyze and Understand Data

  • Conclusion

Exercises - Recap of Python Prerequisites and Objectives