Add startupProbe to otel-agent container for graceful startup

The otel-agent container can take extra time to bind to port 13133, especially when the cluster is under heavy load or resources are constrained. Without a startup probe, the pod may be restarted prematurely by Kubernetes if the readiness or liveness probes fail during this slow startup period, leading to CrashLoopBackOff errors.

By introducing a startupProbe, we provide a dedicated grace period for the otel-agent to initialize and bind to its health check port before readiness and liveness checks begin. This ensures the pod is not restarted unnecessarily during startup, improving reliability and reducing the risk of CrashLoopBackOff due to transient startup delays.

Author: tiffanny29631

manifests/templates/reconciler-manager-configmap.yaml

@@ -195,6 +195,13 @@ data:
            volumeMounts:
            - name: otel-agent-config-reconciler-vol
              mountPath: /conf
+           startupProbe:
+             httpGet:
+               path: /
+               port: 13133
+             failureThreshold: 30   # Allow up to 5 minutes (30 * 10s)
+             periodSeconds: 10
+             initialDelaySeconds: 0
            readinessProbe:
              httpGet:
                path: /

manifests/templates/reconciler-manager.yaml

@@ -91,6 +91,13 @@ spec:
         volumeMounts:
         - name: otel-agent-config-vol
           mountPath: /conf
+        startupProbe:
+          httpGet:
+            path: /
+            port: 13133
+          failureThreshold: 30   # Allow up to 5 minutes (30 * 10s)
+          periodSeconds: 10
+          initialDelaySeconds: 0
         readinessProbe:
           httpGet:
             path: /

manifests/templates/resourcegroup-manifest.yaml

@@ -275,6 +275,13 @@ spec:
         - containerPort: 55678
         - containerPort: 8888
         - containerPort: 13133
+        startupProbe:
+          httpGet:
+            path: /
+            port: 13133
+          failureThreshold: 30   # Allow up to 5 minutes (30 * 10s)
+          periodSeconds: 10
+          initialDelaySeconds: 0
         readinessProbe:
           httpGet:
             path: /