A queue-based, publish-subscribe message-passing system is developed. The system is configured to copy data from a storage device and to forward the data to the downstream sinks in the form of events. The copying of the data is either based on a set interval, or it may get triggered as soon as data appears in the configured source location. Using a queue provides high availability, fault-tolerance, scalability and delivery assurance features and further enables exporting results to multiple downstream systems at a time.

The Streaming Egress pattern is generally applied together with the Streaming Source, Realtime Access Storage and High Velocity Realtime Processing patterns.

Functionality is added to the Big Data platform to enable exporting analysis results as soon as they get computed. A system is developed that continuously copies the computed result to the interested clients in realtime.

1. Readings from a pressure sensor that arrive every five seconds are analyzed in realtime by a realtime processing engine.
2. The analysis results are stored in a storage device.
3. A publish-subscribe system copies the analysis results to a pressure monitoring application every ten seconds.
4. The introduced latency with regards to the availability of the analysis results is very limited.
5. Due to this, the engineer is able to make timely decisions.

This pattern is covered in BDSCP Module 11: Advanced Big Data Architecture.

For more information regarding the Big Data Science Certified Professional (BDSCP) curriculum,
visit www.arcitura.com/bdscp.

The official textbook for the BDSCP curriculum is:

Big Data Fundamentals: Concepts, Drivers & Techniques
by Paul Buhler, PhD, Thomas Erl, Wajid Khattak
(ISBN: 9780134291079, Paperback, 218 pages)

Please note that this textbook covers fundamental topics only and does not cover design patterns.
For more information about this book, visit www.arcitura.com/books.