Kinoko's TIL Log

GCP Pub Sub Topic & Subscription

The Point

Topic is where messages are published to; subscription is how a consumer receives messages from that topic.

A topic is the publishing channel for messages. A subscription is how a consumer subscribes to that channel. Separating the two lets multiple subscribers independently consume from the same topic.

Explanation

Publisher → Topic → Subscription A → Service A
                 → Subscription B → Service B

Each subscription tracks its own progress – where Service A has read up to and where Service B has read up to are independent. Messages stay in a subscription until they are acknowledged or exceed the retention period.

Two subscription types

Knowledge Sugar

Why separate topic and subscription?

If there were only topics, all consumers would compete for the same messages – once read, they are gone. With subscriptions, the same message can be consumed independently by a logging service, an analytics service, and a notification service without interference. This is the fan-out pattern.

What happens if nobody consumes from a subscription?

Messages pile up until the retention period expires (default 7 days), then get auto-deleted. So if you create a subscription, make sure something is consuming from it – otherwise it is wasted resources.

Topic ownership should follow the domain

The topic should be created by the service that owns the domain. For example, if monolith ServiceA publishes events to a topic belonging to ServiceB’s domain, the topic should live in ServiceB’s GCP project, with IAM granting ServiceA publish permission:

ServiceA (monolith)
  │  roles/pubsub.publisher (IAM grant)
  ▼
Topic (owned by ServiceB)
  ▼
Subscription → ServiceC consume

After migration completes, you only need to remove ServiceA’s publish logic – everything else stays the same.

Subscription ownership design

The subscription owner does not have to match the topic owner. Two options:

Option 1: ServiceB owns subscriptionOption 2: ServiceC owns subscription
Created byServiceBServiceC (cross-project)
ControlServiceB manages the entire message flowServiceC is fully autonomous
Best forMigration transition, centralized management neededServiceC is independent, no dependency on ServiceB

During migration, Option 1 is recommended – ServiceB owns both the topic and subscription, then grants ServiceC consume permission. This gives ServiceB full visibility over the message flow.

ServiceC connects to the subscription via IAM:

1resource "google_pubsub_subscription_iam_member" "subscriber" {
2  subscription = google_pubsub_subscription.my_subscription.name
3  role         = "roles/pubsub.subscriber"
4  member       = "serviceAccount:service-c@service-c-project.iam.gserviceaccount.com"
5}

ServiceC’s code just needs to pull using its own service account. GCP verifies IAM and grants access – no extra configuration needed.

Creating Topic and Subscription with Terraform

Like buckets, these are GCP resources that can be managed directly with Terraform:

 1resource "google_pubsub_topic" "my_topic" {
 2  name    = "my-topic"
 3  project = "service-b-project"
 4}
 5
 6resource "google_pubsub_subscription" "my_subscription" {
 7  name  = "my-subscription"
 8  topic = google_pubsub_topic.my_topic.id
 9}
10
11# Grant ServiceA's service account publish permission
12resource "google_pubsub_topic_iam_member" "publisher" {
13  topic  = google_pubsub_topic.my_topic.id
14  role   = "roles/pubsub.publisher"
15  member = "serviceAccount:service-a@service-a-project.iam.gserviceaccount.com"
16}

#gcp #til

← Back to Main Page