Developers and architects might incorrectly. However, there may be scenarios where a synchronous Request-Response through Kafka makes sense. So it can be the result of a synchronous or an asynchronous operation. blog-synchronous-kafka. Apache Kafka or any messaging system is typically used for asynchronous processing wherein client sends a message to Kafka that is processed by background consumers. Event sourcing and Apache Kafka are related. e. Confluent. Regarding synchronous communication, as you mentioned " librdkafka can't do transactional batch delivery - there will be an individual DR per message ". Thiết lập Spring ReplyingKafkaTemplate. The subscribers then consume events from the publishers. Choose wisely the best tool for the job. Open the file server. The limitations with such a synchronous request-response pattern is that the client dictates which service will process the request, and the client must wait for the response even if it could be doing other things. get () method it will get a reply from Kafka. Part 4: Chain Services with Exactly Once Guarantees (Read Next) Part 5: Messaging as the Single Source of Truth. Request-Reply pattern: In situations where you need a synchronous request-reply communication pattern, where a client sends a request and waits for a response, a message queue with built-in support for this pattern, such as RabbitMQ’s Direct Reply-to feature, can provide a more straightforward implementation. Apache Kafka; RabbitMQ; OrderService from the FTGO Example application publishes an Order Created event when it creates an Order. 2). Waits for the response HTTP. Part 6: Leveraging the Power of a Database Unbundled. REST is purely an HTTP transport based call and you will receive a response say 200 OK on the other side, SOAP uses two varieties, Synchronous Messaging over HTTP. id. Class này kế thừa các tính chất của KafkaTemplate để cung cấp mô hình Request-Reply. How to implement the request-response message exchange paradigm with Apache Kafka, pro and cons, and a comparative with CQRS both event sourcing Domestic HighlightsSynchronous: The client sends a request and waits for the response. Applications that need to read data from Kafka use a KafkaConsumer to subscribe to Kafka topics and receive messages from these topics. The Connection ObjectWith that said, lets define what problems REST solves best: Synchronous Request/Reply – HTTP (the network protocol on which REST is transported) itself is a request/response protocol, so REST is a great fit for request/reply interactions. Request–response; Request. The following functionality is currently exposed and available through Confluent REST APIs. In a typical request/response synchronous messaging scenario, you will find a service (server) and a consumer (client) that invokes the service. One of EIP is Request-Reply. Message processing is synchronous. Send a request message and receive a reply message. In this case, you use Kafka to pass notifications of what happens in the different services. A producer fires an event, events are organized into topics and a consumer subscribes to a topic. e. There are various techniques, each with advantages and disadvantages. Asynchronous - The client doesn’t block, and the response, if any, isn’t necessarily sent immediately Given that, it seems that moving from "synchronous" to "asynchronous" communication actually just swaps one synchronous service (e. To get around this, I suggest using predefined request and response queues, removing the overhead of creating a temporary queue. Provide broker log excerpts. Like a phone call, the client sends a request and waits for a response to come back. Hans. Synchronous request/reply with Kafka Normally an application using Apache Kafka would be built using event-driven architecture. Asynchronous tasks (fire-and-forget, or making use of a call-back mechanism) are. Make synchronous request. In this article, we will write a code using Java 1. Creating an API can entail having to make synchronous tasks available, i. But the alternative symbol makes the meaning of sending a message easier to understand for some stakeholders. Synchronous communication requires synchronous mediums and Kafka is not one. netty. Here is a fully contained example:Named it "client" and "server" Due to some restriction I must use synchronous request-reply pattern with kafka. The question is, would the benefits be worth the effort in your particular circumstances. 0. Web APIs also use the request-response messaging mechanism to exchange data, in which the originator of the communication (client) initiates the message with a request to a service provider. It will allow the logging, metrics, and tracing to be linked together for a particular request in the centralized. In this blog post, I’ll review the Kafka ecosystem and tools and discuss the different options for MuleSoft and Kafka collaboration. Reasonably choose the best tool for the job. RecordMetadata recMetadata = producer. Async vs Sync. Steps to reproduce. More specifically, it is a message exchange pattern in which a requestor sends a. Contrarily, data streaming with. Synchronous tasks are high-priority tasks that require immediate execution and user feedback. The questionBuilding synchronous APIs on an asynchronous event bus using Azure Service Bus. However, the alternative symbol makes the meaning of sending a message easier to. Client configuration. You should always use service tasks for synchronous request/response. "Kafka Streams, Apache Kafka’s stream processing library, allows developers to build sophisticated stateful stream processing applications which you can deploy in an environment of your choice. ·. Stack Overflow | The World’s Largest Online Community for DevelopersThis is only possible with fast, real-time streaming of data, leveraging microservices built based on an event-driven architecture. The issue is that multiple services can trigger user_create workflows, and they will expect for the response. Sounds a lot like a synchronous system such as a REST API and you wouldn’t be wrong for thinking that. 4) pub/sub, and NATS (0. get (); Producer. You should use send tasks for sending asynchronous messages, like. Part 3: Using Apache Kafka as a Scalable, Event-Driven Backbone for Service Architectures. What scale and volumes does a REST Proxy for Kafka support? Don’t underestimate the power of the REST Proxy as a data plane because Kafka provides batch capabilities to scale up to many parallel REST Proxy instances . Still, the need for asynchronous messaging had been recognized based on user feedback and some new use cases, such as proactive life event-based services. Thus, to respond to the same User/HTTP request is 'hassle free'. Synchronous send A simple way to send message synchronously is to use the get () method. For information about configuring AWS Identity and Access Management (IAM) for integrated services, see IAM Policies for integrated services. The Request Reply Enterprise Integration Pattern provides a proven mechanism for synchronous message exchange over asynchonous channels: The. The webservice request/response is a synchronous InOut pattern, and the sub-routes are asynchronous. 3, last published: 5 years ago. I'm looking to respond to a REST endpoint with a Success/Failure response that dynamically accepts a topic as a query param. The Grpc implementation will fail immediately after disconnecting the consumer, and grpc must be configured. Asynchronous: The client does not wait for a response and just sends the request to a message. So we know when we send the. Can I use Pact V4 Synchronous Messages to write contract test for Kafka with request-reply pattern or is there a better option? If yes, what am I doing wrong?Initialize the project. The consumer remains as it is. 3). Persistency – Data persistency is the Ability to retrieve messages after the crash. But I have to send the response back the result as response back to API gateway and back to front-end application. You have built an event-driven system leveraging Apache Kafka. The subscribers then consume events from the. connection = 1, it just means the ordering of messages is guaranteed within a partition it has nothing to do with synchronization. It is very simple. 9. 7. Some data may include in a message or event. " as necessary in configuration). Pub-sub is a way to decouple the two ends of a connection and communicate asynchronously. New search experience powered by AI. And across message broker. – Arthur. This blog post explores when (not) to use this message exchange pattern, the differences between synchronous and asynchronous communication, the pros and cons compared to CQRS and event sourcing, and how to implement request-response within the data streaming infrastructure. The exception thrown by send () is. For example when the user sends an HTTP request, I want to produce a message on a specific kafka input topic that triggers a dataflow eventually resulting in a response produced on an output topic. Respond with a response message that employs the stored UUID value from the request message as response message. Teams. Apache Kafka on Confluent Platforms. Thiết lập Spring ReplyingKafkaTemplate. To get started, make a new directory anywhere you’d like for this project: mkdir kafka-producer-application-callback && cd kafka-producer-application-callback. 0. Here’s the key things to know about asynchronous APIs: Synchronous APIs provide instant responses; asynchronous APIs use callbacks. Stack Overflow | The World’s Largest Online Community for DevelopersThe app that is handling the sync API (such as a REST API call over HTTP) would publish to a request topic, including in the request message a unique CorrelationID (that you generate in your app) and then at the other end, your Async app can processes these requests from the request topic, and reply to a response topic using the. Provide logs (with "debug" : ". HTTP/REST and Kafka are frequently combined to take advantage of the best of both worlds: decoupling with Kafka and synchronous client-server communication with. But when I tested this against following two scenarios : Topic not pre created The callbacks are not getting called. The controller subscribes to this response topic to receive the response to the query initiated. Set a function to be called to establish a unique correlation key for each request record. KafkaException: Seek to current after exception; nested exception is org. The standard Apache Kafka Producers/Consumer. The client sends a request to the server, and then the server sends an HTTP or HTTPS response back. Oct 27, 2022. The problem with a lot of benchmarks is that they end up measuring service time rather than response time,. Stack Overflow is leveraging AI to summarize the most relevant questions and answers from the community, with the option to ask follow-up questions in a conversational format. 100–200: Informational Messages; 200–300: Success Messages; 300–400:Redirect Messages; 400–500: Client Errors; 500–600: Server Errors; HTTP 1. Step 3: Configure the Event Consumer. Learn more about TeamsA synchronous client constructs an HTTP structure, sends a request, and waits for a response. We were waiting for a response from…New search experience powered by AI. (by modifying the ProductAddedNotification to publish to Kafka/Service Bus,. g. A Kafka producer has three mandatory properties: 1. This setup consists of publishers (pub) and subscribers (sub), where publishers broadcast events, instead of targeting a particular subscriber in a synchronous, or blocking, fashion. This will cause the server to insert the response information attribute in the CONNACK packet, and the requestor can use response information to construct the. Looking around it seems that using the inbound channels AttributeMap could be the place to store such request details. For broker. 2. When you invoke a function synchronously, Lambda runs the function and waits for a response. Now, we want to take the same example and change the send () method call to a synchronous blocking call. Open akadnikov opened this issue Mar 19, 2023 · 5 comments. – Arthur. So we know when we send the request but we don't know when the answer will come. Check out “ Service Mesh and Cloud-Native Microservices with Apache Kafka, Kubernetes and Envoy, Istio, Linkerd ” for more details on this topic. There are two options when using the same reply topic: Discard unexpected replies: When configuring with a single reply topic, each instance must use a different group. Intermediate Topics in Synchronous Kafka: Using Spring Request-Reply. Then route a response jms message received from a separate InOnly endpoint back to the webservice client as the response. The service task is the typical element to implement synchronous request/response calls, such as REST, gRPC or SOAP. For example, if you use Kafka along with Avro. /** * The prefix for Kafka headers. HTTP is a Request/Response Protocol. But I have to send the response back the result as response back to API gateway and back to front-end application. reply keyword. Synchronous communication means that the microservice or client is blocked while waiting for the response to a request, whereas asynchronous communication is able to continue before receiving a response (which might come later or never). Many datastores support read and write operations where a request returns one response, but much fewer provide an ability to subscribe to. When you specify a service in the "Resource" string of your task state, and you only provide the resource, Step Functions will wait for an HTTP response and then progress to the next. JS. Synchronous Request-Response over Kafka with Redis. After sending the request, the frontend will display a progress bar and will wait. Kafka is usually used at the center of scalable solutions, but it happens to be designed to function asynchronously. I am using the same replyTopic and correlationId as received in the consumer to publish the event. execution. A community-developed, free, opensource, automated testing framework for microservices API, Kafka and Load testing. One of EIP is Request-Reply. Using ReplyingKafkaTemplate across two different applications. First let’s start with our pom. The Kafka producer is conceptually much simpler than the consumer since it has no need for group coordination. Synchronous requests are sequential, leading to delays; asynchronous enables parallel processing. The CompletableFuture is a JRE class tha implements the CompletionStage. Bridging the Synchronous and Asynchronous Worlds. Kafka is widely used for the asynchronous processing of events/messages. HTTP request/response with two resources. Not quite. After receiving the request, it retrieves the data from the request and saves it to Kafka. Apache Kafka on Confluent. Inboxes or special per-client topics so again you have to use regular topics or individual topic partitions to send request and response messages. Communicating between microservices can happen through a synchronous Request/Response pattern or the asynchronous event/message pattern. Synchronous behaviour: Client constructs an HTTP structure, sends over the socket connection. A microservice can be event driven and also can support Restful APIs but both serve different prospective. You could also use the instanceIndex as the REPLY_PARTITION header and use fixed reply partitions;. In a typical request/response synchronous messaging scenario, you will find a service (server) and a consumer (client) that invokes the service. Apache Kafka is a an open-source event streaming platform that supports workloads such as data pipelines and streaming analytics. When max. All forms of gRPC communication supported: unary, client streaming, server streaming, and bidirectional streaming. Kafka and RabbitMQ is the best tools for this operations. 12-2. I wanted to wait until the API response contained particular string. Hide the complicity of Kafka client. This pattern is a little less generally useful than the previous two. In many clients, the thread that makes the request blocks while waiting for a response. cd spring-kafka-server mvn spring-boot:run. In this case, all instances receive each reply, but only the instance that sent the request finds the correlation ID. Kafka only guarantees the order of messages within one partition. For delayed responses, you need to implement asynchronous communication based on. The standard Apache Kafka. In this blog, we used Kafka as one of the inter-service communication methods in our microservices, especially for handling blog approval processes. This talk discusses multiple options on how to do a request-response over Kafka — showcasing producers and consumers using single and multiple topics, and more advanced considerations using the interactive queries of ksqlDB and Kafka Streams. clients. Provide broker log excerpts. Synchronous messaging is possible but impacts scalability. The first thing to notice is that its infeasible to create a consumer and temporary queue per client in Spring since pooling resources is required overcome the JmsTemplate gotchas. Connect and share knowledge within a single location that is structured and easy to search. Applications that need to read data from Kafka use a KafkaConsumer to subscribe to Kafka topics and receive messages from these topics. The first one is synchronous, and so blocks the caller thread until the response is received. Unfortunately, the battle is an apple-to-orange comparison that often includes misinformation and FUD from vendors. This architecture has a Gateway API that pushes the requests to a Service Bus (KAFKA). Stack Overflow | The World’s Largest Online Community for DevelopersProcess streams of records in real-time. Synchronous communication. Q&A for work. Step 4:. You have built an event-driven system leveraging Apache Kafka. bootstrap. $ npm install --save kafkajs npm-hook-receiver @slack/webhook. I'd like to route a webservice request to an InOnly endpoint of a jms queue. HTTP and Kafka complement each other in various ways. Enterprise messaging technologies, such as IBM MQ, RabbitMQ and ActiveMQ, have provided asynchronous communication within and across applications for many years. . As far as I understand, the problem is that we do not use the built-in Kafka ACL mechanism for restricting access to Kafka-topics, but we use the Rager-Kafka-Plugin. Spring Cloud Stream - Send message synchronously with wait time. Can anyone please guide me how can I achieve the synchronous request and response between API gateway and micron service via Kafka. In other words, the producer needs to get the response of the produced message from the consumer,. In Synchronous communication, the caller waits for a response before sending the next message, and it operates as a REST protocol on top of HTTP. To start this app: Start kafka with compose 'docker-compose up' Start server running '. Asynchronous Communication with Apache Kafka. Finally, we can put all these ideas together in a more comprehensive ecosystem that validates and processes orders in response to an HTTP. Example of such communication is REST (Representational State Transfer) based APIs where request and response are sent through HTTP (Hyper TextWhen connecting, the client will request the server to send response messages through setting the request-response information attribute in the CONNECT packet. You have built an event-driven system leveraging Apache Kafka. the program flow transforms from a sequence of synchronous operations, into an asynchronous stream of events. This queue is specific to the client's server and hence responses to different clients will go to different queues. Before we jump to how to use Kafka to make asynchronous inter-service communication, there is some preparation we need to do: 1. The request data received at API Gateway is forward to Micro service via Kafka. So the API response might not have the expected string until after waiting for a few seconds. Request-response communication with REST / HTTP is simple, well-understood, and supported by most technologies, products, and SaaS cloud services. send (new ProducerRecord<String, String> ("topic-name", "key", "value")). HTTP / synchronous request-response is an anti-pattern for streaming data and will not work if large scale is required for the streaming application. But still the receiver of the response throws No pending reply exception. Kafka request/response message exchange pattern is done the same way as in most other messaging systems. Request Response in Spring. Kafka Architecture : Synchronous to Asynchronous [1] Kafka is a powerful stream processing tool, but it's an asynchronous tool. So I try to use ReplyingKafkaTemplate. The leader broker will write the record to its partition and send the acknowledgment without worrying whether the followers have been able to replicate the message or not. ReplyingKafkaTemplate not getting response back. The service task is the typical element to implement synchronous request/response calls, such as REST, gRPC or SOAP. You can use the AWS managed Kafka service Amazon Managed Streaming for Apache Kafka (Amazon MSK), or a self-managed Kafka cluster. , a listening port on the message broker like. Sorted by: 66. Orchestrators. e. Stack Overflow is leveraging AI to summarize the most relevant questions and answers from the community, with the option to ask follow-up questions in a conversational format. First, Client initial a command to REST service using POST (sync), then REST service take this command and forward it to Processor ms (after doing some conversion) via Kafka (async). Nest automatically sends the reply back in a new event that ends with a . 1. The request data received at API Gateway is forward to Micro service via Kafka. This is where kafka-go comes into play. 4). Kafka client generates a random UUID and sends a single Kafka request message. Service A receives a request from a consumer for data that is stored in service B. Object implements Producer <K,V>. You should always use service tasks for synchronous request/response. I am going to use Kafka as a message broker in my application. To invoke a function synchronously with the AWS CLI, use the invoke. That thread is blocked until the last byte of the response is written on the wire. default. Send messages to a particular topic with the payload and event key ID. Metadata - Most metadata about the cluster – brokers, topics, partitions, and configs – can be read using GET requests for the corresponding URLs. Stack Overflow | The World’s Largest Online Community for DevelopersMost people are familiar with the synchronous request/response style of communication like REST, GraphQL or RPC. It has nothing to do with REST webservice, its structure, or the supporting server. It combines messaging, storage, and. We also want to capture the metadata acknowledgment and print the offset number at which the message is. But if we want to use request-reply pattern we can use communication, even though we can use Kafka in request-reply pattern with some workaround. I want to use request topic and corresponding requestreply topic dynamically generated on the basis of request sent from user (UI). Share. App Connect supports connection to the following Kafka implementations: Apache Kafka. However, synchronous request-response communication is an anti-pattern for many data streaming use cases around Apache Kafka. This challenge is however not new. Synchronous requests are sequential, leading to delays; asynchronous enables parallel processing. Hình bên dưới là là 1 service minh họa đơn giản để tính tổng của 2 số yêu cầu đồng bộ (synchronous – sử dụng mô hình Request-Reply). Teams. the service is stateless. Hence it should be used only when needed. The client-project will send a string message to the server-project over kafka, then the server-project will reverse the string and return it back to the client. Can we have a mechanism to communicate with PL/SQL with Kafka and return the response object type in the database. This is the way HTTP is behaving. You have built an event-driven system leveraging Apache Kafka. apache-kafka; synchronous; request-response; Malik Rashid Ahmad. Synchronous communication is the most straightforward solution when trying to make services communicate. The property visible below sets the time after which the caller will receive a timeout while waiting for a response: 1. Though we can have synchronous request/response calls when the requester expects immediate response, integration patterns based on events and asynchronous messaging provide maximum scalability and. We will also create an HTTP POST REST endpoint, which accepts student details and returns randomly calculated result and percentage. Kafka is a high-performance, low-latency, scalable and durable log that is used by thousands of companies worldwide and is battle-tested at scale. The Grpc implementation will fail immediately after disconnecting the consumer, and grpc must be configured. Stack Overflow | The World’s Largest Online Community for Developers2. timeout. Business microservices architecture we all in general and clients access servers, or redirect the feed. Apache Kafka is a streaming platform intended for large. See the documentation. This talk discusses multiple options on how to do a request-response over Kafka — showcasing producers and consumers using single and multiple topics, and more advanced considerations. default. DataServiceLookup case class Step 2: Server Flink application consumes the Request Kafka Topic, parses the incoming message and enriches the message with the response. 1 GB limit for trigger connections and responses from invoke connections. cd spring-kafka-server mvn spring-boot:run. The second is asynchronous, and the returned Uni gets the response when received. 1M seed round led by 468 Capital with participation from Pear VC, Alumni Ventures and an angel syndicate including former NGINX CEO Gus Robertson and DigitalOcean. JS client --> Spring RestController --> send request to Kafka topic --> read response from Kafka reply topic --> return data to client. 9 client for Node. 0, it proposes a flexible programming model bridging CDI and event-driven. We also want to capture the metadata acknowledgment and print the offset number at which the message. Synchronous invocation. As a summary for Asynchronous communication, we can consider that the client microservice sends a message or event to the broker systems without waiting for a response. I have a binding function like the following (please note that I'm using the functional style binding). get () method makes the send method from Asynchronous to synchronous so that everything runs on the same thread. The partitioners shipped with Kafka guarantee that all messages with the same non-empty. We can use the non-blocking call if application requirements permit. Get the latest news from us to your. 1. Start our producer service on the spring-kafka-server. In this context, synchronous means that all parties need to be active at the same time to be able to communicate. 4. In this example, we are going to send messages with ids. requests. The difference between asynchronous and synchronous APIs. At a high level, they all support some form of messages. Each partition is an ordered, immutable. hystrix. Asynchronous — Message Queues, Databases, Files, E-Mail, Cloud storage. Client configuration. In this case ack = all means that the leader will not respond untill it receives acknowledgement for the full set of in-sync replicas (ISR) and the maximum wait time to get this. Restful API is mostly used synchronous communication and event driven is asynchronous mode of communication. we can run it), minimal program demonstrating the problem. In the case of Message ID pattern, the client's JMSReplyTo property tells the server where the response should be sent. per. 12-2. Apache Kafka version. 2. I understand that the Callback can return a series of retriable and non-retriable exceptions. DataServiceLookup case class Step 2: Server Flink application consumes the Request Kafka Topic, parses the incoming message and enriches the message with the response. Sounds a lot like a synchronous system such as a REST API and you wouldn’t be wrong for thinking that. Kafka maintains a cache of metadata that gets updated occasionally to keep it current and in your scenario you only wait if that cache is stale or not initialized. @SpringBootApplication public class ExampleApplication { // PayloadSender uses. Quarkus provides support for Apache Kafka through SmallRye Reactive Messaging framework. The application requires high asynchronous processing power. Synchronous Send. 8. Request-reply. The general idea is that the publisher includes a destination for a consumer to publish another message with the reply/response. Synchronous and asynchronous request-response communication can be implemented with Apache Kafka. The Asynchronous Request-Response conversation involves the following participants: The Requestor initiates the conversation by sending a Request message amd waits for a Response message. When Service A makes a blocking synchronous call to Service B, it must wait to get the response (or acknowledgment) that the request is completed. I am developing a series of microservices using Spring Boot and Kafka. The API/microservice messages are validated using an OpenAPI specification that defines the API/microservice contract. This situation is a potential problem for any synchronous request-reply pattern. Synchronous Request-Response over Kafka with Redis Each message sent by a producer would include a unique correlation-id. "Kafka Streams, Apache Kafka’s stream processing library, allows developers to build sophisticated stateful stream processing applications which you can deploy in an environment of your choice. Traditionally, request-response and event streaming are two different paradigms: Request-response (HTTP) Low latency; Typically synchronous; Point to point; Pre-defined API; Event streaming. In this context, a “request” consists of publishing a message to the. Once we have configured our Producer, we can now use it to actually send messages to the Kafka broker. The consequence of this model is that we cannot serve more than one connection within a single thread. A topic can have a zero, one or many consumers who can subscribe to the data written to it. i. This is simple to implement, but if the requestor crashes, it will have difficulty re-establishing. But. Record latency t 1 – t 0. The configuration controls the maximum amount of time the client will wait for the response of a request. On this tutorial, we'll implement an async request/response exchange between two ASP. If combining Event Notification using Kafka with traditional Request-Response, it may be necessary to implement synchronous semantics on top of asynchronous Kafka topics. Depending on your domain and. One of our usecase is to receive a message from an api produce it to topic1 and and the result to. The most used architecture to ensure this is the microservice architecture. # Add our dependencies. A distributed pub/sub platform, Kafka has impressive characteristics, such as low latency, high throughput and concurrency, fault tolerance, high availability, and robust data integrity. Request/response is a commonly used message pattern where one service sends a request to another service, continuing after the response is received. Recently, event streaming technologies (such as Apache Kafka) have grown in popularity, and they also provide asynchronous communication. Communicating between microservices can happen through a synchronous Request/Response pattern or the asynchronous event/message pattern. JS. It is very simple. In the async case, the kafka producer library will put your request in an internal queue, and unless that queue is full, you’ll get control back to your main thread immediately. The software is composed of independent small services in microservice architecture that communicate over well-defined APIs. Technically, these are two. In a PULL paradigm the client has to initiate a request and poll the server for receiving new data, which is half-duplex and unidirectional . The biggest limitation of the REST Proxy data plane is that it is a synchronous request-response protocol. It was initially conceived as a message queue and open-sourced by LinkedIn in 2011. When one service needs in some data it sends a Request to the other service which is responsible of such data. HTTP is synchronous and is based on PULL paradigm. Synchronous APIs often use HTTP or HTTPS for transport, and HTTP is a unidirectional protocol. Reading data from Kafka is a bit different than reading data from other messaging systems, and there are few unique concepts and ideas involved.