Windowing Functions

As part of this module let us get into Windowing Functions.

Starting Spark Context

Let us start spark context for this Notebook so that we can execute the code provided.

from pyspark.sql import SparkSession

import getpass
username = getpass.getuser()

spark = SparkSession. \
    builder. \
    config('spark.ui.port', '0'). \
    enableHiveSupport. \
    appName(f'{username} | Python - Windowing Functions'). \
    master('yarn'). \
spark.conf.set('spark.sql.shuffle.partitions', '2')