21 hours (usually 3 days including breaks)
- Programming experience in any language
- An understanding of Big Data concepts (Hadoop, etc.)
Stream Processing refers to the real-time processing of "data in motion", that is, performing computations on data as it is being received. Such data is read as continuous streams from data sources such as sensor events, website user activity, financial trades, credit card swipes, click streams, etc. Stream Processing frameworks are able to read large volumes of incoming data and provide valuable insights almost instantaneously.
In this instructor-led, live training (onsite or remote), participants will learn how to set up and integrate different Stream Processing frameworks with existing big data storage systems and related software applications and microservices.
By the end of this training, participants will be able to:
- Install and configure different Stream Processing frameworks, such as Spark Streaming and Kafka Streaming.
- Understand and select the most appropriate framework for the job.
- Process of data continuously, concurrently, and in a record-by-record fashion.
- Integrate Stream Processing solutions with existing databases, data warehouses, data lakes, etc.
- Integrate the most appropriate stream processing library with enterprise applications and microservices.
- Software architects
Format of the Course
- Part lecture, part discussion, exercises and heavy hands-on practice
- To request a customized training for this course, please contact us to arrange.
- Stream processing vs batch processing
- Analytics-focused stream processing
Overview Frameworks and Programming Languages
- Spark Streaming (Scala)
- Kafka Streaming (Java)
Comparison of Features and Strengths of Each Framework
Overview of Data Sources
- Live data as a series of events over time
- Historical data sources
- In the cloud (AWS, etc.)
- On premise (private cloud, etc.)
Setting up the Development Environment
Installing and Configuring
Assessing Your Data Analysis Needs
Integrating the Streaming Framework with Big Data Tools
Transforming the Input Data
Inspecting the Output Data
Integrating the Stream Processing Framework with Existing Applications and Microservices
Summary and Conclusion
The lab exercises. Applying the theory from the first day in subsequent days.
The trainer was passionate and well-known what he said I appreciate his help and answers all our questions and suggested cases.