Need for Kafka- Theory for Beginners

DISTRIBUTED MESSAGE STREAMING PLATFORM

Photo by Tobias Fischer on Unsplash

Streaming means flow here.

Publish & Subscribe Mechanism to stream the records.

Publish means write and subscribe means read streams(flow) of events.

Now you will say, what does the word distribute mean here?

I am coming to this point.

Example: Let us say you have 1000 Records stored in one location. If the system gets crashes, everything is lost.

Solution: To put it in a distributed system.

Pic1 from Author Notes

Now you thought to insert your data in a distributed way in five systems(200 Records each). But, again, if anyone’s system gets crashed, data is lost.

Solution: Put complete records(data) in distributed systems instead of dividing the data. Replication is done at multiple locations. Space is used more, but data is safe now.

Pic2 Taken from Author Notes

Why does it follow the messaging system?

Let us say you have two source services and three destination services. One source needs to send data to the three destination services, and similarly, the second source service needs to send data to the other three destination services. Total six connections (n*m).

Disadvantages

  • Increased Bottleneck.
  • Highly coupled systems

Now there comes the need for a messaging system.

The message is served to the messaging system, and the destination service will consume it from there.

Total Connections(n + m).

Advantages:

  • Loosely Coupled system.
  • Notifications to all the destination services.
  • Messaging systems keep destination services keep consuming data.
Pic3 from Author Notes

For example — Please Check Pic2 and Pic3.

— Any queries, hit me up in the comment sections.

— Do let me know in the comments if any correction is required(If anyone of you know prior).

--

--

--

Indigo Starseed — Entrepreneur, Here to share life lessons.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Pascal’s Big Bang

The WSO2 Research Team are Unsung Heroes of Performance

TechTalk 2021 — August Guests

Kubernetes for Beginners — Part 06 (Kubernetes commands)

Photo by Vlada Karpovich from Pexels

3 Roadblocks to Measuring the Flow of Value in Enterprise Software Delivery

The better Maven Central

Shine bright like a ruby(-ist)

Optimize your data access by using CQRS Architecture Pattern — A theoretical and practical approach

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Rishab Batra

Rishab Batra

Indigo Starseed — Entrepreneur, Here to share life lessons.

More from Medium

The object-oriented approach in neuron nets design

GIT ^ Global Information Tracker

How save and load a Tensorflow /Keras entire model with custom Layer

Property-based testing #3: Shrinking Take One