Add more to what-is-kafka.md
A few appearance tweaks. Mostly for headings.
This commit is contained in:
parent
b2107c5d12
commit
d97433279b
|
@ -184,7 +184,6 @@ p {
|
||||||
|
|
||||||
h3 {
|
h3 {
|
||||||
font-family: 'Montserrat', sans-serif;
|
font-family: 'Montserrat', sans-serif;
|
||||||
margin-top: 0;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
p {
|
p {
|
||||||
|
|
|
@ -121,8 +121,8 @@ JSON.parse(`{
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<style>
|
<style>
|
||||||
h1 {
|
h3 {
|
||||||
padding-top: 1em;
|
margin-top: 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
.standalone-data-display pre {
|
.standalone-data-display pre {
|
||||||
|
@ -199,7 +199,7 @@ JSON.parse(`{
|
||||||
}
|
}
|
||||||
|
|
||||||
.eye-catch {
|
.eye-catch {
|
||||||
margin-bottom: 1em;
|
margin: 1em;
|
||||||
}
|
}
|
||||||
|
|
||||||
.eye-catch h2 {
|
.eye-catch h2 {
|
||||||
|
|
|
@ -19,7 +19,6 @@
|
||||||
max-width: 90vw;
|
max-width: 90vw;
|
||||||
display: flex;
|
display: flex;
|
||||||
flex-direction: column;
|
flex-direction: column;
|
||||||
align-items: center;
|
|
||||||
flex-wrap: wrap;
|
flex-wrap: wrap;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
@ -9,6 +9,66 @@ headerImageAlt: "Kafka's logo"
|
||||||
Initially built by LinkedIn, Apache Kafka is a "distributed event store" and "stream processing" platform.
|
Initially built by LinkedIn, Apache Kafka is a "distributed event store" and "stream processing" platform.
|
||||||
But what does that mean exactly?
|
But what does that mean exactly?
|
||||||
|
|
||||||
|
## Basics of Kafka
|
||||||
|
|
||||||
|
The core principles of Kafka are very simple. There are three key components: producers, consumers, and the kafka cluster.
|
||||||
|
|
||||||
|
### Producers
|
||||||
|
|
||||||
|
Producers are responsible for creating new messages for your Kafka system. Every time an event occurs that the system
|
||||||
|
should know about, a producer will generate a message describing this event.
|
||||||
|
|
||||||
|
For example, a bookstore may have an application that _produces_ a message every time someone buys a new book:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"action": "purchase",
|
||||||
|
"title": "The Metamorphosis",
|
||||||
|
"price": 9.99,
|
||||||
|
"buyerId": "84694fc7",
|
||||||
|
"storeNum": "FL-231",
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The rest of the system decides what exactly to _do_ with this new information. Those parts of the system are called
|
||||||
|
**Consumers**.
|
||||||
|
|
||||||
|
### Consumers
|
||||||
|
|
||||||
|
Whenever a message is produced in the Kafka system, a consumer can act in response to that new information.
|
||||||
|
|
||||||
|
Returning to the previous example: this particular bookstore may have a service that informs authors and publishers
|
||||||
|
every time one of their books has been purchased. This service may work as a consumer. Then, every time a book is
|
||||||
|
purchased, the service _consumes_ the new message, and alerts the creators however it sees fit.
|
||||||
|
|
||||||
|
One of the main benefits of this structure is an inherent loose coupling. Producers do not care how consumers work. If a
|
||||||
|
consumer wants to send a text, rent a billboard, or order an ice cream, the producer simply is not interested. This
|
||||||
|
allows each part of the system to focus on its own piece of the domain, each staying smaller and more self-contained.
|
||||||
|
|
||||||
|
### Kafka Cluster
|
||||||
|
|
||||||
|
The Kafka cluster, in short is what facilitates the connection between producers and consumers. An indefinite number of
|
||||||
|
producers and consumers can connect to the cluster, and the cluster itself can be powered by many servers.
|
||||||
|
|
||||||
|
The cluster decides exactly which messages need to go where, and a key part of this process is its use of **Topics**.
|
||||||
|
|
||||||
|
#### Topics
|
||||||
|
|
||||||
|
"Topics" are Kafka's way of dividing up messages into different categories. A simple topic for our bookstore might be
|
||||||
|
`book_purchases`, or `coffee_orders` (for the requisite café inside). So, when a book is sold, our producer would push a
|
||||||
|
new message into the `book_purchases` topic. And of course, when someone orders a fancy latte or something hot and
|
||||||
|
bitter, a message is instead put into the `coffee_orders` topic.
|
||||||
|
|
||||||
|
Importantly, consumers do not blindly receive every message produced in the system. Instead, they subscribe to
|
||||||
|
individual topics. Our author-notification service might subscribe just to the `book_purchases` topic, not being
|
||||||
|
very interested in messages from `coffee_orders`.
|
||||||
|
|
||||||
|
Thus, every time the Kafka cluster receives a newly-produced message with a given topic, it only forwards it along to
|
||||||
|
consumers that have intentionally subscribed.
|
||||||
|
|
||||||
|
## TODO:
|
||||||
## Distributed Event Store
|
## Distributed Event Store
|
||||||
|
|
||||||
|
Essentially
|
||||||
|
|
||||||
## Stream Processing
|
## Stream Processing
|
Loading…
Reference in New Issue