Patterns for implementing Pub/Sub messaging - Google cloud Pub Sub

 

Patterns for implementing Pub/Sub messaging

There are multiple patterns for implementing Pub/Sub messaging.

• The one-to-one pattern:

• We have a publisher, a topic, and a subscriber.
• The publisher sends messages to the topic in Pub/Sub, where they are queued, ready to be read, and the subscriber receives the messages and reads them.
• The subscriber accesses messages in a topic via a subscription, and you can have more than one subscription to a topic.

• The one-to-many pattern:

• The publisher sends messages to the topic, and each subscriber receives a message via their own subscription.
• Note that each subscriber is guaranteed to receive each message at least That's because they each have their own subscription. So in this  both subscribers get copies of the same messages.
• If you wanted multiple subscribers to get only their own messages from the same subscription, effectively round-robin the messages between subscribers, you can just make them share a single subscription.

• The many-to-many pattern:

• This is really just like the one-to-one pattern, except with multiple topics.
• If these topics were only subscribed to by a single subscriber, we'd have the many-to-one pattern.
• Which pattern you choose will all depend on your workload, and the desired workflow you're trying to achieve.

Publishing messages to Pub/Sub

• You just create a message containing your data, this is going to be a JSON payload, that's base64 encoded.
• The total size of the payload needs to be 10MB or less.
• Then you send the payload as a request to the Pub/Sub API specifying the topic to which the message should be published.

Receiving messages from Pub/Sub.

• You simply create a subscription to a topic.
• Subscriptions are always associated with a single topic.
• There are 2 types of delivery method for subscriptions.
• Pull is the default method.
• When you have created a pull subscription, you can make ad hoc pull requests to the Pub/Sub API, specifying your subscription to receive messages associated to that subscription.
• When you receive a message, note that you have to acknowledge that you've received it.
• If you don't, that message will remain at the top of the queue associated with your subscription, and you won't get the next message, or any messages after that, until you've acknowledged it.
• Push method
• Alternatively, you can configure a push subscription, which will automatically send new messages to an endpoint that you define.
• The endpoint must use the HTTPS protocol with a valid SSL certificate. And of course, something has to sit on the end of that endpoint that is capable of receiving and processing a message.

Slow start algorithm

• Pub/Sub uses a slow start algorithm to moderate the speed of messages sent to push endpoints.
• It will gradually increase in speed, but back off again if it receives any connection problems.
• It expects a valid HTTP code to acknowledge a message before it'll move on to the next one.
• Pub/Sub integrates nicely with pretty much everything in GCP, and there are client libraries
• for all popular programming languages: Python, C#, Go, Java, Node, PHP, and Ruby.
• Pub/Sub is fully supported by Cloud Data Flow, allowing you to use the Apache Beam SDK to read messages, and window, or buffer them into batches.
• You can use events in Pub/Sub topics to trigger the invocation of Cloud Functions, or use Cloud Run to be the receiver of a push subscription.
• Pub/Sub is also the foundation of Cloud IoT Core, receiving the messages and events from connected devices that can then be consumed by any other GCP service for later processing.

• If you're developing applications locally that will leverage Pub/Sub, but you don't want to use live topics, or risk running up charges, there is also a local Pub/Sub emulator available.
• To run this, you just need the Google Cloud SDK, and a Java Runtime Environment, version 7 or above.
• You can then run the emulator as a local Pub/Sub service.