Introduction to Kafka for Developers (TTDS6760)

Quick Start to Kafka Concepts, Architecture, Programming in Kafka, Streaming, Monitoring, Tuning & More

TTDS6760

Introductory and Beyond

2 Days

Course Overview

Kafka is a real-time data pipeline processor. Its high-scalability, fault tolerance, execution speed, and fluid integrations are some of the key hallmarks that make it an integral part of many Enterprise Data architectures. 

Discover the future of data streaming with our hands-on, 2-day Apache Kafka training course designed specifically for experienced Java developers. Become an expert in harnessing the potential of fast data and streaming systems and learn how to navigate the complexities of modern streaming architectures. With practical labs and real-world examples, you'll be immersed in the cutting-edge world of Kafka and Java development.

Throughout the course you’ll explore the ins and outs of Apache Kafka and learn how it compares to other queue systems like JMS and MQ. You’ll learn about Kafka's unique architecture and understand how to effectively produce and consume messages with Kafka & Zookeeper. Through hands-on labs, you'll gain experience in scaling Kafka, navigating multiple data centers, and implementing disaster recovery solutions, while exploring essential Kafka utilities.

You’ll also learn the powerful Kafka APIs and become proficient in configuration parameters, Producer and Consumer APIs, as well as advanced features such as message compression and offset management. Gain hands-on with Kafka, including benchmarking Producer send modes, comparing compression schemes, and managing offsets. Experience real-world applications like Clickstream processing to solidify your expertise. Then you’ll round off your Kafka journey with an in-depth look at the Kafka Streams API, monitoring, and troubleshooting techniques. Learn how to optimize your Kafka deployment with best practices for hardware selection, cluster sizing, and Zookeeper settings.

By the end of this course you'll be equipped with the core skills required to tackle your next Kafka project with confidence.

 

Learning Objectives

Working in a hands-on learning environment you’ll learn to:

  • Implement and configure Apache Kafka effectively, demonstrating a deep understanding of its unique architecture, core concepts, and the differences between Kafka and other queue systems (JMS/MQ).
  • Utilize Kafka APIs proficiently, including the Producer and Consumer APIs, and apply advanced techniques such as message compression, offset management, and Producer send modes.
  • Design and develop streaming applications using the Kafka Streams API, performing complex operations like transformations, filters, joins, and aggregations, while working with KStream, KTable, and KStore concepts.
  • Monitor and troubleshoot Kafka deployments, identifying performance bottlenecks, addressing common issues, and employing best practices for hardware selection, cluster sizing, partition sizing, and Zookeeper settings.
  • Apply the skills and knowledge acquired throughout the course to real-world scenarios, showcasing the ability to develop, deploy, and optimize Kafka-based streaming applications for a variety of use cases.

Course Objectives

Working in a hands-on learning environment you’ll learn to:

  • Implement and configure Apache Kafka effectively, demonstrating a deep understanding of its unique architecture, core concepts, and the differences between Kafka and other queue systems (JMS/MQ).
  • Utilize Kafka APIs proficiently, including the Producer and Consumer APIs, and apply advanced techniques such as message compression, offset management, and Producer send modes.
  • Design and develop streaming applications using the Kafka Streams API, performing complex operations like transformations, filters, joins, and aggregations, while working with KStream, KTable, and KStore concepts.
  • Monitor and troubleshoot Kafka deployments, identifying performance bottlenecks, addressing common issues, and employing best practices for hardware selection, cluster sizing, partition sizing, and Zookeeper settings.
  • Apply the skills and knowledge acquired throughout the course to real-world scenarios, showcasing the ability to develop, deploy, and optimize Kafka-based streaming applications for a variety of use cases.

Course Prerequisites

This course is geared for experienced Java Developers and architects with Java development background who are new to Kafka.  This course is not for non-developers.

In order to be successful in this course, and to participate in the hands-on labs, you should possess:

  • Basic Java programming skills; practical Java development background.
  • Reasonable experience working with databases
  • Basic Linux skills and the ability to work from the Linux command line
  • Basic knowledge of Linux editors (such as VI / nano) for editing code.

Take Before: Students should have practical skills equivalent to or should have attended the following course(s) as a pre-requisite:

  • TT2104: Fast Track to Core Java Programming for OO Developers (C+, C#, etc.)

Next Steps / Follow-on Courses: We offer a wide variety of follow-on courses for next-level Kafka, Apache tools, data science, machine learning, DevOps, Java development skills and more. Please see our Learning Journey & Career Experience Paths for options based on your specific role and goals. 

Course Agenda

We can collaborate with you to tune this course and level of coverage to target the skills you need most. Course agenda, topics and labs are subject to adjust during live delivery in response to student skill level, interests and participation.

Introduction to Streaming Systems

  • Understanding Fast data
  • Streaming terminologies
  • Understanding at-least-once / at-most-once / exactly-once processing patterns
  • Popular streaming architectures
  • Lambda architecture
  • Streaming platforms overview
  • Lab: Hands-on first look at Kafka

Introducing Kafka

  • Comparing Kafka with other queue systems (JMS / MQ)
  • Kafka Architecture
  • Kaka concepts: Messages, Topics, Partitions, Brokers, Producers, commit logs
  • Kafka & Zookeeper
  • Producing messages
  • Consuming messages
  • Consumers, Consumer Groups
  • Message retention
  • Scaling Kafka
  • Kafka across multiple data centers and disaster recovery
  • Lab: Getting Kafka up and running
  • Lab: Using Kafka utilities

Using Kafka APIs

  • Configuration parameters
  • Producer API - sending messages to Kafka
  • Consumer API - consuming messages from Kafka
  • Producer send modes
  • Message compression
  • Commits , Offsets, Seeking
  • Managing offsets - auto commit / manual commit
  • Lab: Writing Producer / Consumer
  • Lab: Benchmarking Producer send modes
  • Lab: Comparing compression schemes
  • Lab: Managing offsets
  • Lab: Clickstream processing

Kafka Streams API

  • Introduction to Kafka Streams library
  • Features and design
  • Streams concepts: KStream / KTable / KStore
  • Streaming operations (transformations, filters, joins, aggregations)
  • Using Streams API: foreach / filter / map / groupby
  • Lab: Kafka Streaming APIs

Monitoring & Troubleshooting Kafka

  • Monitoring tools overview
  • Monitoring Kafka
  • Cluster level and host level monitoring
  • Identifying performance bottlenecks
  • Troubleshooting common Kafka issues

Bonus Content / Time Permitting

Kafka Best Practices

  • Avoiding common mistakes
  • Hardware selection
  • Cluster sizing
  • Partition sizing
  • Zookeeper settings
  • Compression and batching
  • Message sizing
  • Monitoring and instrumenting
  • Troubleshooting

Course Materials

Student Materials: Each participant will receive a Student Guide with course notes, code samples, software tutorials, step-by-step written lab instructions, diagrams and related reference materials and resource links. Students will also receive the project files (or code, if applicable) and solutions required for the hands-on work.

Hands-On Setup Made Simple! Our dedicated tech team will work with you to ensure our ‘easy-access’ cloud-based course environment is accessible, fully-tested and verified as ready to go well in advance of the course start date, ensuring a smooth start to class and effective learning experience for all participants. Please inquire for details and options.

Raise the bar for advancing technology skills

Attend a Class!

Live scheduled classes are listed below or browse our full course catalog anytime

Special Offers

We regulary offer discounts for individuals, groups and corporate teams. Contact us

Custom Team Training

Check out custom training solutions planned around your unique needs and skills.

EveryCourse Extras

Exclusive materials, ongoing support and a free live course refresh with every class.

Attend a Course

Please see the current upcoming available open enrollment course dates posted below. Please feel free to Register Online below, or call 844-475-4559 toll free to connect with our Registrar for assistance. If you need additional date options, please contact us for scheduling.

Course Title Days Date Time Price
Introduction to Kafka for Developers (TTDS6760) 2 Days Dec 5 to Dec 6 10:00 AM to 06:00 PM EST $1,995.00 Enroll
Introduction to Kafka for Developers (TTDS6760) 2 Days Jan 30 to Jan 31 10:00 AM to 06:00 PM EST $1,995.00 Enroll
Introduction to Kafka for Developers (TTDS6760) 2 Days Apr 3 to Apr 4 10:00 AM to 06:00 PM EST $1,995.00 Enroll
Introduction to Kafka for Developers (TTDS6760) 2 Days Jul 17 to Jul 18 10:00 AM to 06:00 PM EST $1,995.00 Enroll
Introduction to Kafka for Developers (TTDS6760) 2 Days Oct 23 to Oct 24 10:00 AM to 06:00 PM EST $1,995.00 Enroll

Mix, Match & Master!
2FOR1: Two Courses, One Price!

Enroll in *any* two public courses (for 2023 *OR* 2024 dates!) by December 31, for one price!  Learn something new, or share the promo!

Click for Details & Additional Offers

Learn. Explore. Advance!

Extend your training investment! Recorded sessions, free re-sits and after course support included with Every Course
Trivera MiniCamps
Gain the skills you need with less time in the classroom with our short course, live-online hands-on events
Trivera QuickSkills: Free Courses and Webinars
Training on us! Keep your skills current with free live events, courses & webinars
Trivera AfterCourse: Coaching and Support
Expert level after-training support to help organizations put new training skills into practice on the job

The voices of our customers speak volumes

Special Offers
Limited Offer for most courses.

SAVE 50%

Learn More