Cloud Pub/Sub is designed to work at very large scales.  It can process millions of messages every second.  With so many messages coming and going, it’s not uncommon for some to be delivered out of order.  A lot of times this isn’t a big deal.  If your topic contains a list of pictures you need to resize, then resizing picture #5 before #4 isn’t a big deal.  But sometimes ordering can be important, even critical.

Luckily, Google has provided a way to guarantee message ordering.  It requires two steps.  First, you need to add an ordering key to your messages.  Second, you need to enable message ordering in your subscription.  If both those steps aren’t completed, then ordering is not guaranteed.  It’s possible for the messages to have ordering keys, but the subscription ignores them.  It’s also possible to create a subscription that tries to enforce ordering, but none of the messages have ordering keys.  So, you need both.

Now, you might assume that the ordering key is going to contain an order number.  That is NOT how it works.  You do NOT send messages with an ordering key of 1, 2, 3, 4, etc.  Ordering keys are used to mark smaller groups of messages where ordering should be enforced.  Most of the time, you don’t need to enforce ordering on every single message in a topic.  You usually just care about the ordering on certain messages.

If that sounds confusing, maybe this example will help.  Let’s say you create a new subscription and enable message ordering.  Then four messages are sent in the following order: 1, 2, 3, 4.  We will also say that the first two messages (1 and 2) have an ordering key of “alpha” and the last two messages (3 and 4) have an ordering key of “beta”.  That means you are guaranteed to receive 1 before 2, and 3 before 4.  However, since the ordering keys are different, you are not guaranteed to receive 2 before 3.  

So you could receive the messages as:

• 1, 2, 3, 4
• 1, 3, 2, 4
• 1, 3, 4, 2
• 3, 1, 2, 4
• 3, 4, 1, 2
• Etc

Because 1 and 2 share the same ordering key, 1 will always be delivered before 2.  And the same is true for 3 and 4.  If you wanted to guarantee that you received the messages as 1-2-3-4, you would need to give all of them the same ordering key.  But usually, you only care about the order of certain messages in a topic (like say, those related to the same event).

Messages without an ordering key do not have any guarantees about the order in which they will be delivered.  Messages with the same ordering key (and in the same region) are guaranteed to be received in the order that the Pub/Sub service receives them.

Before you start enforcing ordering on every message, you should be aware of the trade-offs.  First, enabling message ordering can increase your latency.  Second, message ordering can increase the number of duplicate messages received.  When a message with an ordering key needs to be redelivered, Pub/Sub will redeliver every subsequent message with the same ordering key.  This includes acknowledged messages.  So you should make sure to only enforce ordering where it is actually necessary.

Now let me actually show you how to go about enforcing ordering on a subscription.

First I need to create a topic.  I’ll call it “topic-2”.  I don’t need to enable anything fancy here, just click on “Create Topic” to confirm.

Next, I need to create a subscription.  I’ll call this “subscription-2”.  Then I need to select the topic that I just created.  And finally, I need to be sure to enable “Message Ordering”.  Now if I forget to click on this, then any message order keys will simply be ignored.

Now I can click on “Create” to confirm creation.

So at this point, I should be able to send a few messages with order keys.  I’ll go ahead and send three messages with an order key of “key1”.  And I’ll also send a fourth message without an order key.

This should guarantee that message 1 comes before message 2, and message 2 comes before message 3.  Message 4 does not have any ordering key, so it can be delivered at any time.  

So you can see that when I do a pull, I see message 1 and message 4.  I won’t see message 2 or 3 because I have not yet acknowledged message 1.   In order to see 2, I first have to acknowledge 1.   So let me do that.

Alright, so now I can see message 2.  But I still won’t be able to get 3 until I acknowledge 2.  This is how Cloud Pub/Sub guarantees order delivery.  You have to acknowledge the previous message in order to receive the next.

Now I could have acknowledged message 4 at any time.  It does not have an ordering key and so no messages are held back because of it.

And that’s how you enable and enforce message ordering.