¿Qué es una topología en Apache Storm?

What is a topology in Apache Storm?

A topology is a graph of stream transformations where each node is a spout or bolt. Each node in a Storm topology executes in parallel. In your topology, you can specify how much parallelism you want for each node, and then Storm will spawn that number of threads across the cluster to do the execution.

Apache Storm is a distributed, fault-tolerant, open-source computation system. You can use Storm to process streams of data in real time with Apache Hadoop. Storm solutions can also provide guaranteed processing of data, with the ability to replay data that wasn’t successfully processed the first time.

Nimbus is the central component of Apache Storm. The main job of Nimbus is to run the Storm topology. Nimbus analyzes the topology and gathers the task to be executed. Then, it will distributes the task to an available supervisor. A supervisor will have one or more worker process.

Components- Storm has three critical components: Topology, Stream, and Spout. Topology is a network made of Stream and Spout. Stream is an unbounded pipeline of tuples and Spout is the source of the data streams which converts the data into the tuple of streams and sends to the bolts to be processed.

Here’s a summary of the steps for setting up a Storm cluster:

1. Set up a Zookeeper cluster.
2. Install dependencies on Nimbus and worker machines.
3. Download and extract a Storm release to Nimbus and worker machines.
4. Fill in mandatory configurations into storm.

An Apache Storm topology consumes streams of data and processes those streams in arbitrarily complex ways, repartitioning the streams between each stage of the computation however needed. Read more in the tutorial.

A worker process executes a subset of a topology. A worker process belongs to a specific topology and may run one or more executors for one or more components (spouts or bolts) of this topology. A running topology consists of many such processes running on many machines within a Storm cluster.

A running topology consists of many such processes running on many machines within a Storm cluster. An executor is a thread that is spawned by a worker process. It may run one or more tasks for the same component (spout or bolt).

Apache Storm integrates with the queueing and database technologies you already use. An Apache Storm topology consumes streams of data and processes those streams in arbitrarily complex ways, repartitioning the streams between each stage of the computation however needed. Read more in the tutorial.