Overview

The simplest way to describe a pattern is that it provides a proven solution to a common problem individually documented in a consistent format and usually as part of a larger collection. The notion of a pattern is already a fundamental part of everyday life. Without acknowledging it each time, we naturally use proven solutions to solve common problems each day. Patterns in the IT world that revolve around the design of automated systems are referred to as design patterns.

Design patterns are helpful because they:

• represent field-tested solutions to common design problems
• organize design intelligence into a standardized and easily referenced format
• are generally repeatable by most IT professionals involved with design
• can be used to ensure consistency in how systems are designed and built
• can become the basis for design standards
• are usually flexible and optional (and openly document the impacts of their application and even suggest alternative approaches)
• can be used as educational aids by documenting specific aspects of system design (regardless of whether they are applied)
• can sometimes be applied prior and subsequent to the implementation of a system
• can be supported via the application of other design patterns that are part of the same collection
• enrich the vocabulary of a given IT field because each pattern is given a meaningful name

Furthermore, because the solutions provided by design patterns are proven, their consistent application tends to naturally improve the quality of system designs.

Note that even though design patterns provide proven design solutions, their mere use cannot guarantee that design problems are always solved as required. Many factors weigh in to the ultimate success of using a design pattern, including constraints imposed by the implementation environment, competency of the practitioners, diverging business requirements, and so on. All of these represent aspects that affect the extent to which a pattern can be successfully applied.

A pattern language is a set of related patterns that act as building blocks in that they can be carried out in one or more pattern application sequences where each subsequent pattern builds upon the former. The notion of a pattern language originated in building architecture as did the term “pattern sequence” used in association with the order in which patterns can be carried out.

The big data design pattern catalog, in its entirety, provides an open-ended, master pattern language for big data. The extent to which different patterns are related can vary, but overall they share a common objective, and endless pattern sequences can be explored.

Pattern Profiles

Each of the design patterns covered in this catalog is documented in a pattern profile comprised of the following parts:

• Requirement – A requirement is a concise, single-sentence statement that presents the fundamental requirement addressed by the pattern in the form of a question. Every pattern description begins with this statement.
• Problem – The issue causing a problem and the effects of the problem are described in this section, which may be accompanied by a figure that further illustrates the “problem state.” It is this problem for which the pattern is expected to provide a solution. Part of the problem description includes common circumstances that can lead to the problem (also known as “forces”).
• Solution – This represents the design solution proposed by the pattern to solve the problem and fulfill the requirement. Often the solution is a short statement that may be further followed by a diagram that concisely communicates the final solution state. “How-to” details are not provided in this section but are instead located in the Application section.
• Application – This part is dedicated to describing how the pattern can be applied. In can include guidelines, implementation details, and sometimes even a suggested process.
• Mechanisms – This section lists common mechanisms that can be implemented to apply the pattern. Usually, some of the mechanisms will have already been referenced in the Application section. The application of the pattern is not limited to the use of these mechanisms.

Each pattern profile begins with the Requirement statement followed a table containing concise Problem and Solution statements, an Application summary, and a Mechanisms list. The table is then followed by corresponding sections that generally provide more detail.