Presigning GCS URLs for Cloud Run App with an Attached Service Account
In modern cloud-native architectures, leveraging serverless platforms like Cloud Run offers tremendous benefits – scalability, managed infrastructure, and reduced operational overhead. When it comes to securely delivering files stored in Google Cloud Storage (GCS), presigned URLs provide a powerful solution. This guide walks you through presigning URLs for a Cloud Run application that uses an attached service account, bypassing the need for a locally stored private key.
By default, generating a signed URL for GCS typically relies on a service account JSON key that includes a private key. However, storing this key within your container is not only a security risk – since the file resides on disk and complicates key rotation – but also goes against best security practices.
In a Cloud Run environment, we intentionally do not include the Google Application Credentials file. Instead, we attach a service account to Cloud Run, which supports the principle of least privilege, by granting only the required permissions. This design minimizes security risks and simplifies credential management.
An error in that scenario where there is not Google Application Credentials file will look like this:
AttributeError: you need a private key to sign credentials.
The credentials you are currently using <class 'google.auth.compute_engine.credentials.Credentials'>
just contains a token. see https://googleapis.dev/python/google-api-core/latest/auth.html#setting-up-a-service-account for more details.
This error clearly indicates that the credentials available in Cloud Run (typically an instance of google.auth.compute_engine.credentials.Credentials) do not provide the necessary private key to sign the URL in the traditional way.
The solution: Using IAM SignBlob API for URL signing
Since we’re not using the Credentials file in a Cloud Run environment, the solution is to use the IAM SignBlob API. This API allows you to sign arbitrary data (in this case, the URL) using your service account without requiring a local private key.
How it works
Google Cloud client libraries, such as the Python google-cloud-storage library, can utilize the IAM SignBlob API to sign URLs if configured correctly. Instead of reading a private key from a file, the library makes a remote call to the IAM service to perform the signing operation on your behalf.
Granting the necessary permissions
Before you can use the IAM SignBlob API, ensure that your Cloud Run service account has the appropriate permissions. Specifically, the service account must have the roles/iam.serviceAccountTokenCreator role. This role allows the account to sign files using the IAM API.
Using Google Cloud CLI
gcloud iam service-accounts add-iam-policy-binding your-signer@your-project.iam.gserviceaccount.com \
--member="serviceAccount:your-cloud-run@your-project.iam.gserviceaccount.com" \
--role="roles/iam.serviceAccountTokenCreator"
If you prefer to manage your infrastructure as code, here’s an equivalent Terraform snippet:
resource "google_service_account_iam_member" "cloud_run_signer" {
service_account_id = "your-signer@your-project.iam.gserviceaccount.com"
role = "roles/iam.serviceAccountTokenCreator"
member = "serviceAccount:your-cloud-run@your-project.iam.gserviceaccount.com"
} Implementing the signed URL Generation
Option 1: Using impersonated_credentials (preffered solution)
- What it does:
- This approach first obtains the default credentials (which, on Cloud Run, are provided by the attached service account) and then creates an impersonated credentials object. This object calls the IAM Credentials API’s SignBlob endpoint behind the scenes to sign data using the target service account’s key (without you needing to manage the key locally).
Why it’s a preffered way:
- You don’t need to manage or store any service account key files.
- The impersonated credentials implement the signer interface that
blob.generate_signed_urlrequires. - When you’re in an environment like Cloud Run, using the IAM SignBlob API via impersonation is the recommended pattern for generating signed URLs.
I’m using a FastAPI and here is a simple code snippet:
import asyncio
import datetime
import google.auth
from google.auth import impersonated_credentials
from google.cloud import storage
from app.core.config import settings
storage_client = storage.Client(project=settings.GCP_PROJECT_ID)
async def _upload_to_google_cloud_storage(file_path: str, file_name: str) -> str:
credentials, project = google.auth.default()
signing_credentials = impersonated_credentials.Credentials(
source_credentials=credentials,
target_principal=settings.SERVICE_ACCOUNT_EMAIL,
target_scopes="https://www.googleapis.com/auth/devstorage.read_only",
lifetime=2,
)
bucket = storage_client.bucket(settings.GCS_BUCKET_NAME)
blob = bucket.blob(file_name)
await asyncio.to_thread(blob.upload_from_filename, file_path, if_generation_match=0)
url = await asyncio.to_thread(
blob.generate_signed_url,
version="v4",
expiration=datetime.timedelta(hours=1),
method="GET",
credentials=signing_credentials,
)
return url
Option 2: Using compute_engine.IDTokenCredentials
- What it does:
- This approach creates an
IDTokenCredentialsinstance, which is designed to obtain an ID token (typically used to authenticate requests to services that accept ID tokens).
- This approach creates an
Why it’s not ideal for signed URLs:
IDTokenCredentialsare meant for obtaining identity tokens (for example, when calling a service that requires an OIDC token) and do not implement the necessary signing interface for signing arbitrary blobs.- Passing an
IDTokenCredentialsinstance toblob.generate_signed_urlmight work in some cases by coincidence, but it isn’t following the intended use case. The signing mechanism required for V4 signed URLs is different from just acquiring an ID token.
Please note that this is not a recommended approach and should be only done in case the previous one does not work. Either way, in the first place you should make sure the Service Account has correct permissions to create signing token, and is correctly attached to a Cloud Run service
import asyncio
import datetime
from google.cloud import storage
from google.auth import compute_engine
from google.auth.transport.requests import Request
from app.core.config import settings
storage_client = storage.Client(project=settings.GCP_PROJECT_ID)
async def _upload_to_google_cloud_storage(file_path: str, file_name: str) -> str:
auth_request = Request()
signing_credentials = compute_engine.IDTokenCredentials(
auth_request, "", service_account_email=settings.SERVICE_ACCOUNT_EMAIL
)
bucket = storage_client.bucket(settings.GCS_BUCKET_NAME)
blob = bucket.blob(file_name)
await asyncio.to_thread(blob.upload_from_filename, file_path, if_generation_match=0)
url = await asyncio.to_thread(
blob.generate_signed_url,
version="v4",
expiration=datetime.timedelta(hours=24),
method="GET",
credentials=signing_credentials,
)
return url
How to check for attached Service Account
By navigating to Cloud Run -> Your Cloud Run Service -> Edit & deploy new revision -> Security Tab you will see what Service account is attached to your Clud Run instance. Ideally you want to create your custom Service Account with minimal possible permissions.
