Introduction to Apache Spark | Hands-on Spark for Big Data & Machine Learning TTSK7503

Apache Spark, a significant component in the Hadoop Ecosystem, is a cluster computing engine used in Big Data. Building on top of the Hadoop YARN and HDFS ecosystem, it offers order-of-magnitude faster processing for many in-memory computing tasks compared to Map/Reduce. It can be programmed in Java, Scala, Python, and R – the favorite languages of Data Scientists – along with SQL-based front ends.  With advanced libraries like Mahout and MLib for Machine Learning, GraphX or Neo4J for rich data graph processing as well as access to other NOSQL data stores, Rule engines and other Enterprise components, Spark is a lynchpin in modern Big Data and Data Science computing.Geared for experienced developers, Introduction to Apache Spark for Big Data & Machine Learning provides students with a comprehensive, hands-on exploration of enterprise-grade Spark programming, interacting with the significant components mentioned above to craft complete data science solutions.  Students will leave this course armed with the skills they require to begin working with Spark in a practical, real world environment.

  • Price: $2,195.00
  • Duration: N/A
  • Delivery Methods: Virtual
Date Time Price Option
Please contact us at info@toptalentlearning.com or 469-721-6100 for this course schedule.
For questions call: (469) 721-6100
Find More Training

Why choose TOPTALENT?

  • Get assistance every step of the way from our Texas-based team, ensuring your training experience is hassle-free and aligned with your goals.
  • Access an expansive range of over 3,000 training courses with a strong focus on Information Technology, Business Applications, and Leadership Development.
  • Have confidence in an exceptional 95% approval rating from our students, reflecting outstanding satisfaction with our course content, program support, and overall customer service.
  • Benefit from being taught by Professionally Certified Instructors with expertise in their fields and a strong commitment to making sure you learn and succeed.

Please note that this list of topics is based on our standard course offering, evolved from typical industry uses and trends. We’ll work with you to tune this course and level of coverage to target the skills you need most.

Spark Introduction

  • Big data, Hadoop, Spark
  • Spark concepts and architecture
  • Spark components overview
  • Labs: installing and running Spark

The first look at Spark

  • Spark shell
  • Spark web UIs
  • Analyzing dataset – part 1
  • Labs: Spark shell exploration

Spark Data structures

  • Partitions
  • Distributed execution
  • Operations: transformations and actions
  • Labs: Unstructured data analytics using RDDs

Caching

  • Caching overview
  • Various caching mechanisms available in Spark
  • In memory file systems
  • Caching use cases and best practices
  • Labs: Benchmark of caching performance

DataFrames and Datasets

  • DataFrames Intro
  • Loading structured data (JSON, CSV) using DataFrames
  • Using schema
  • Specifying schema for DataFrames
  • Labs: DataFrames, Datasets, Schema

Spark SQL

  • Spark SQL concepts and overview
  • Defining tables and importing datasets
  • Querying data using SQL
  • Handling various storage formats: JSON, Parquet, ORC
  • Labs: querying structured data using SQL; evaluating data formats

Spark and Hadoop

  • Hadoop Primer: HDFS, YARN
  • Hadoop + Spark architecture
  • Running Spark on Hadoop YARN
  • Processing HDFS files using Spark
  • Spark & Hive

Spark API

  • Overview of Spark APIs in Scala / Python
  • The lifecycle of a Spark application
  • Spark APIs
  • Deploying Spark applications on YARN
  • Labs: Developing and deploying a Spark application

Spark ML Overview

  • Machine Learning primer
  • Machine Learning in Spark: MLib / ML
  • Spark ML overview (newer Spark2 version)
  • Algorithms overview: Clustering, Classifications, Recommendations
  • Labs: Writing ML applications in Spark

GraphX

  • GraphX library overview
  • GraphX APIs
  • Create a Graph and navigating it
  • Shortest distance
  • Pregel API
  • Labs: Processing graph data using Spark

Time Permitting Topics

Spark Streaming

  • Streaming concepts
  • Evaluating Streaming platforms
  • Spark streaming library overview
  • Streaming operations
  • Sliding window operations
  • Structured Streaming
  • Continuous streaming
  • Spark & Kafka streaming
  • Labs: Writing spark streaming applications

Workshop

  • Attendees will work on solving real-world data analysis problems using Spark

This “skills-centric” course is about 50% hands-on lab and 50% lecture, designed to train attendees in core big data/ Spark development and use skills, coupling the most current, effective techniques with the soundest industry practices. Throughout the course students will be led through a series of progressively advanced topics, where each topic consists of lecture, group discussion, comprehensive hands-on lab exercises, and lab review.

This course provides indoctrination in the practical use of the umbrella of technologies that are on the leading edge of data science development focused on Spark and related tools.  Working in a hands-on learning environment, students will explore:

  • Spark Ecosystem
  • Spark Shell
  • Spark Data structures (RDD, DataFrame, Dataset)
  • Spark SQL
  • Modern data formats and Spark
  • Spark API
  • Spark & Hadoop & Hive
  • Spark ML overview
  • GraphX
  • Time-permitting: Spark Streaming
  • Time-permitting: Optional Capstone Workshop (Time-Permitting)

Need different skills or topics?  If your team requires different topics or tools, additional skills or custom approach, this course may be further adjusted to accommodate.  We offer additional Spark, big data, data science, Hadoop, AI / machine learning / deep learning, programming, analytics,  and other related topics that may be blended with this course for a track that best suits your needs. Our team will collaborate with you to understand your needs and will target the course to focus on your specific learning objectives and goals.

This foundation-level course is geared for intermediate skilled, experienced Developers and Architects (with basic Python experience) who seek to be proficient in advanced, modern development skills working with Apache Spark in an enterprise data environment.

Students should have attended the course(s) below, or should have basic skills in these areas:

  • Introduction to Python Programming
  • Introduction to SQL (Basic familiarity is needed, not in-depth SQL skills)

Ten (10) business days’ notice is required to reschedule a class with no additional fees. Notify TOPTALENT LEARNING as soon as possible at 469-721-6100 or by written notification to info@toptalentlearning.com to avoid rescheduling penalties.

Please contact our team at 469-721-6100; we will gladly guide you through the online purchasing process.

You will receive a receipt and an enrollment confirmation sent to the email you submitted at purchase. Your enrollment email will have instructions on how to access the class. Any additional questions our team is here to support you. Please call us at 469-721-6100.

If a student is 15 minutes late, they risk losing their seat to a standby student. If a student is 30 minutes late or more, they will need to reschedule. A no-show fee will apply. Retakes are enrolled on a stand-by basis. The student must supply previously issued courseware. Additional fees may apply.

You will receive a ‘Certificate of Completion’ once you complete the class. If you purchased an exam voucher for the class, a team member from TOPTALENT LEARNING will reach out to discuss your readiness for the voucher and make arrangements to send it.