Skip to content

Add Cloud Armor rules for HTTP, gRPC and GCS backends #87

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Sep 10, 2025
Merged

Add Cloud Armor rules for HTTP, gRPC and GCS backends #87

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
0c93984
Add Cloud Armor rules for HTTP, gRPC and GCS backends
haydentherapper Sep 3, 2025
ef7590d
Remove read body limit, apply json parsing to specific content types
haydentherapper Sep 5, 2025
de468fe
Update gcp/modules/tiles_tlog/network.tf
haydentherapper Sep 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
121 changes: 120 additions & 1 deletion gcp/modules/tiles_tlog/network.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,70 @@ data "google_compute_network_endpoint_group" "k8s_grpc_neg" {
zone = each.key
}

resource "google_compute_security_policy" "k8s_http_grpc_security_policy" {
name = "${var.shard_name}-k8s-http-grpc-security-policy"
project = var.project_id
type = "CLOUD_ARMOR"

rule {
action = "deny(502)"
priority = "1"

match {
expr {
expression = "int(request.headers['content-length']) > 8388608"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should be sure to document this somewhere

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack, will document this

}
}
description = "Block all incoming write requests > 8MB"
}

rule {
action = "throttle"
priority = "10"
match {
versioned_expr = "SRC_IPS_V1"
config {
src_ip_ranges = ["*"]
}
}
rate_limit_options {
enforce_on_key = "IP"
conform_action = "allow"
exceed_action = "deny(429)"
rate_limit_threshold {
count = var.http_grpc_qpm_rate_limit
interval_sec = "60"
}
}
description = "Rate limit all HTTP write traffic by client IP"
}

rule {
action = "allow"
priority = "2147483647"
match {
versioned_expr = "SRC_IPS_V1"
config {
src_ip_ranges = ["*"]
}
}
description = "default rule"
}

advanced_options_config {
json_parsing = "STANDARD"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm. will this cover both HTTP and gRPC traffic? I don't think we want the firewall parsing gRPC traffic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll add separate security policies for HTTP and gRPC to remove this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah actually i want one security policy for all write traffic. i'll dig into this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can include specific content types for when to parse the body. This should prevent gRPC traffic from being parsed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this needed for?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Armor will reject traffic that isn't valid JSON. It's an early check to keep CPU load down for requests that will never be valid. I'm creating separate security policies for http and grpc, since grpc won't be valid json.

json_custom_config {
content_types = ["application/json"]
}
}

adaptive_protection_config {
layer_7_ddos_defense_config {
enable = var.enable_adaptive_protection
}
}
}

resource "google_compute_backend_service" "k8s_http_backend_service" {
count = var.freeze_shard ? 0 : 1
name = "${var.shard_name}-${var.dns_subdomain_name}-k8s-neg-backend-service"
Expand All @@ -138,6 +202,10 @@ resource "google_compute_backend_service" "k8s_http_backend_service" {
}
}

depends_on = [google_compute_security_policy.k8s_http_grpc_security_policy]

security_policy = google_compute_security_policy.k8s_http_grpc_security_policy.self_link

log_config {
enable = var.enable_backend_service_logging
}
Expand Down Expand Up @@ -166,23 +234,74 @@ resource "google_compute_backend_service" "k8s_grpc_backend_service" {
}
}

depends_on = [google_compute_security_policy.k8s_http_grpc_security_policy]

security_policy = google_compute_security_policy.k8s_http_grpc_security_policy.self_link

log_config {
enable = var.enable_backend_service_logging
}
}

resource "google_compute_security_policy" "bucket_security_policy" {
name = "${var.shard_name}-bucket-security-policy"
project = var.project_id
type = "CLOUD_ARMOR_EDGE"

rule {
action = "throttle"
priority = "10"
match {
versioned_expr = "SRC_IPS_V1"
config {
src_ip_ranges = ["*"]
}
}
rate_limit_options {
enforce_on_key = "IP"
conform_action = "allow"
exceed_action = "deny(429)"
rate_limit_threshold {
count = var.bucket_qpm_rate_limit
interval_sec = "60"
}
}
description = "Rate limit all read traffic by client IP"
}

rule {
action = "allow"
priority = "2147483647"
match {
versioned_expr = "SRC_IPS_V1"
config {
src_ip_ranges = ["*"]
}
}
description = "default rule"
}

adaptive_protection_config {
layer_7_ddos_defense_config {
enable = var.enable_adaptive_protection
}
}
}

resource "google_compute_backend_bucket" "tessera_backend_bucket" {
name = "${var.shard_name}-${var.bucket_name_suffix}"
project = var.project_id

depends_on = [google_storage_bucket.tessera_store, google_compute_security_policy.bucket_security_policy]

bucket_name = google_storage_bucket.tessera_store.name

enable_cdn = var.enable_cdn
cdn_policy {
cache_mode = "USE_ORIGIN_HEADERS"
}

depends_on = [google_storage_bucket.tessera_store]
edge_security_policy = google_compute_security_policy.bucket_security_policy.self_link

lifecycle {
prevent_destroy = true
Expand Down
18 changes: 18 additions & 0 deletions gcp/modules/tiles_tlog/variables.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,24 @@ variable "enable_backend_service_logging" {
default = true
}

variable "http_grpc_qpm_rate_limit" {
description = "count of write requests per minute allowed to HTTP and gRPC backends"
type = number
default = 600 // 10 QPS
}

variable "bucket_qpm_rate_limit" {
description = "count of read requests per minute allowed to the GCS bucket"
type = number
default = 30000 // 500 QPS
}

variable "enable_adaptive_protection" {
description = "whether to enable layer 7 DDoS adaptive protection"
type = bool
default = true
}

variable "spanner_timeseries_role_id" {
description = "name of the project role for managing timeseries entries for Spanner - role must include permissions `monitoring.timeSeries.create` and `monitoring.timeSeries.list`"
type = string
Expand Down