Big Data Processing with Apache Spark


Contact Us

We would love to hear from you. Please complete this form to pre-book or request further information about our delivery options.


I'd like to receive emails with the latest updates and promotions from Insoft.

Data Protection & Privacy

I hereby allow Insoft Ltd. to contact me on this topic. Further, I authorise Insoft Ltd. processing, using collecting and storing my personal data for the purpose of these activities. All your data will be protected and secured as outlined in our privacy policy.

Upcoming Dates

Sep 19 - Sep 20, 2022
09:00 - 17:00

Oct 17 - Oct 18, 2022
09:00 - 17:00

Nov 14 - Nov 15, 2022
09:00 - 17:00

Dec 12 - Dec 13, 2022
09:00 - 17:00

Jan 9 - Jan 10, 2023
09:00 - 17:00

Feb 6 - Feb 7, 2023
09:00 - 17:00

Big Data Processing with Apache Spark
2 days  (Instructor Led Online)  |  Data Science

Course Details

Processing big data in real-time is challenging due to scalability, information consistency, and fault tolerance. This Big Data Processing with Apache Spark course shows you how you can use Spark to make your overall analysis workflow faster and more efficient. You’ll learn all about the core concepts and tools within the Spark ecosystem, like Spark Streaming, the Spark Streaming API, machine learning extension, and structured streaming.

You’ll begin by learning data processing fundamentals using Resilient Distributed Datasets (RDDs), SQL, Datasets, and Dataframes APIs. After grasping these fundamentals, you’ll move on to using Spark Streaming APIs to consume data in real-time from TCP sockets, and integrate Amazon Web Services (AWS) for stream consumption.

By the end of this course, you’ll not only have understood how to use machine learning extensions and structured streams but you’ll also be able to apply Spark in your own upcoming big data projects.


See other courses available


After completing this course, you will be able to:

  • Write your own Python programs that can interact with Spark
  • Implement data stream consumption using Apache Spark
  • Recognize common operations in Spark to process known data streams
  • Integrate Spark streaming with Amazon Web Services
  • Create a collaborative filtering model with Python and the movielens dataset
  • Apply processed data streams to Spark machine learning APIs


Lesson 1: Introduction to Spark Distributed Processing

  • Introduction to Spark and Resilient Distributed Datasets
  • Operations Supported by the RDD API
  • Self-Contained Python Spark Programs
  • Introduction to SQL, Datasets, and DataFrames

Lesson 2: Introduction to Spark Streaming

  • Streaming Architectures
  • Introduction to Discretized Streams
  • Windowing Operations
  • Introduction to Structured Streaming

Lesson 3: Spark Streaming Integration with AWS

  • Spark Integration with AWS Services
  • Integrating AWS Kinesis and Python
  • AWS S3 Basic Functionality

Lesson 4: Spark Streaming, ML, and Windowing Operations

  • Spark Integration with Machine Learning

Target Audience

Big Data Processing with Apache Spark is for you if you are a software engineer, architect, or IT professional who wants to explore distributed systems and big data analytics. Although you don‘t need any knowledge of Spark, prior experience of working with Python is recommended.



For an optimal experience with the hands-on labs and other practical activities, we recommend the following hardware configuration:

  • Processor: Intel Core i5 or equivalent
  • Memory: 4GB RAM
  • Storage: 35 GB available space



  • OS: Windows 7 SP1 64-bit, Windows 8.1 64-bit or Windows 10 64-bit
  • PostgreSQL 9.0 or above
  • Python 3.0 or above
  • Spark 2.3.0
  • Amazon Web Services (AWS) account