Big Data Analytics with Hadoop

Course Objectives

This course is approximately 50% hands-on, combining expert lecture, real-world demonstrations and group discussions with machine-based practical labs and exercises.  Our engaging instructors and mentors are highly experienced practitioners who bring years of current "on-the-job" experience into every classroom

Working in a hands-on learning environment, led by our expert instructor, students will learn to:

• Define Big Data and Identify Big Data Use Cases
• Review Big Data Management Architecture and Engineered Systems
• Describe Integrated Big Data Solution and its components.
• Examine MapReduce programs and balance MapReduce jobs
• Use NoSQL Database
• Use XQuery for Hadoop
• Install, use, and administer the Big Data Appliance
• Provide data security and enable resource management
• Examine MapReduce programs and balance MapReduce jobs
• Use the BigDataLite Virtual Machine
• Use the Hadoop Distributed File System (HDFS)to store, distribute, and replicate data across the nodes in the Hadoop cluster.
• Acquire big data using the HDFS Command Line Interface, Flume, and NoSQL Database.
• Use MapReduce and YARN for distributed processing of the data stored in the Hadoop cluster.
• Process big data using MapReduce, YARN, Hive, Pig, XQuery for Hadoop, Solr, and Spark.
• Integrate big data and warehouse data using Sqoop, Big Data Connectors, Copy to BDA, Big Data SQL, Data Integrator, and GoldenGate.
• Analyze big data using Big Data SQL, Advanced Analytics technologies, and Big Data Discovery.
• Use and manage Big Data Appliance.
• Secure your data.
• Understand Big Data Cloud Service: Key Features & Benefits

Need different skills or topics?  If your team requires different topics or tools, additional skills or custom approach, this course may be further adjusted to accommodate.  We offer additional Big Data / Data Science, Hadoop, programming, analytics, Python/R, and other related topics that may be blended with this course for a track that best suits your needs. Our team will collaborate with you to understand your needs and will target the course to focus on your specific learning objectives and goals.

Course Prerequisites

This course is geared for attendees who wish to use Integrated Big Data Solution to acquire, process, integrate and analyze big data. Attendees should possess the following incoming skills:

• Basic to Intermediate IT Skills, and Big Data knowledge
• Good foundational mathematics or logic skills
• Basic Linux skills, including familiarity with command-line options such as ls, cd, cp, and su

Please see the Related Courses tab for specific Pre-Requisite courses, Related Courses that offer similar skills or topics, and next-step Learning Path recommendations.

Course Agenda

Please note that this list of topics is based on our standard course offering, evolved from typical industry uses and trends. We’ll work with you to tune this course and level of coverage to target the skills you need most.

1. Introduction to Hadoop

• Hadoop Distributed File System
• MapReduce framework
• YARN
• Other changes
• Installing Hadoop 3

2. Overview of Big Data Analytics

• Introduction to data analytics
• Introduction to big data
• Distributed computing using Apache Hadoop
• The MapReduce framework
• Hive
• Apache Spark
• Visualization using Tableau

3. Big Data Processing with MapReduce

• The MapReduce framework
• MapReduce job types
• MapReduce patterns

4. Scientific Computing and Big Data Analysis with Python and Hadoop

• Installation
• Data analysis

5. Statistical Big Data Computing with R and Hadoop

• Introduction
• Methods of integrating R and Hadoop
• Data analytics

6. Batch Analytics with Apache Spark

• SparkSQL and DataFrames
• DataFrame APIs and the SQL API
• Schema – structure of data
• Loading datasets
• Saving datasets
• Aggregations
• Joins

7. Real-Time Analytics with Apache Spark

• Streaming
• Spark Streaming
• fileStream
• Transformations
• Checkpointing
• Driver failure recovery
• Interoperability with streaming platforms (Apache Kafka)
• Handling event time and late date
• Fault-tolerance semantics

8. Batch Analytics with Apache Flink

• Introduction to Apache Flink
• Installing Flink
• Using the Flink cluster UI
• Batch analytics

9. Stream Processing with Apache Flink

• Introduction to streaming execution model
• Data processing using the DataStream API

10. Visualizing Big Data

• Tableau
• Chart types
• Using Python to visualize data
• Using R to visualize data
• Big data visualization tools

11. Introduction to Cloud Computing

• Concepts and terminology
• Goals and benefits
• Risks and challenges
• Roles and boundaries
• Cloud characteristics
• Cloud delivery models
• Cloud deployment models

12. Using Amazon Web Services

• Amazon Elastic Compute Cloud
• Launching multiple instances of an AMI
• What is AWS Lambda?
• Introduction to Amazon S3
• Amazon DynamoDB
• Amazon Kinesis Data Streams
• AWS Glue
• Amazon AMR

Course Materials

Each student will receive a Student Guide with course notes, code samples, software tutorials, step-by-step written lab instructions, diagrams and related reference materials and links (as applicable). Students will also receive the project files (or code, if applicable) and solutions required for the hands-on work.

Lab Setup Made Simple.   All course labs and solutions, data sets, Tableau course software (limited version, for course use only), detailed courseware, lab guides and resources (as applicable) are provided for attendees in our easy access, no installation required, remote lab environment for the duration of the course. Our tech team will help set up, test and verify lab access for each attendee prior to the course start date, ensuring a smooth start to class and successful hands-on course experience for all participants.