infra: enable ALB target group stickiness on prod-deploy (fixes #268, companion to #514)

[ejanalysis]

Closes #268 — file uploads intermittently fail in production with a red <h1>Not Found</h1> bar.

Companion to #514. That PR makes the identical change on dev-deploy; this one makes it on prod-deploy so the two branches do not drift.

Why this second PR is needed

ejam-infra/ exists on both deploy branches, and the two main.tf files were byte-identical before these PRs. terraform apply reads main.tf from whatever checkout it is run in. If prod were applied from a prod-deploy checkout carrying the old file, the fix would not take effect — and the apply could silently undo the console toggle right after it was set. Landing the change on both branches removes that trap.

What this changes

Adds a stickiness block to aws_lb_target_group "app" in ejam-infra/main.tf. One block, no other changes.

Why

Shiny keeps session state in the memory of one R process. A file upload arrives as a separate HTTP POST from the page load. Production runs desired_count = 2, and target-group stickiness is currently off, so the load balancer round-robins that POST to the other container — which has never heard of the session and answers 404 <h1>Not Found</h1>. The app's upload bar renders that body unescaped, which is why users see raw HTML tags in the red bar.

Reproduced 2026-07-25

Drove the real browser flow programmatically (open Shiny session over websocket → uploadInit → POST the real inst/testdata/latlon/testpoints_10.xlsx bytes to the returned uploadUrl). Per shiny:::ShinySession$handleRequest, only a POST with a live jobId can return 200 OK, so a 200 proves the request reached the session's own container.

Environment	Containers	Uploads failed
Production (ejam.publicenvirodata.org)	2	6 of 12 (50%), perfectly alternating
Dev app	1	0 of 12 (0%)

Perfect alternation is round-robin across exactly two targets. Confirmed independently: prod returns no Set-Cookie: AWSALB, and prod is served directly by the ALB (no Cloudflare in front), so that is a true negative.

Why it came back

This was fixed in Feb 2026 by hand in the AWS console and closed as "fixed not in this repo but in the configuration of the deployment." It was never codified in Terraform, so it only ever existed as out-of-band console state, and it did not survive.

---

How to apply this (for @Gabe-Epic)

Merging this PR does not change anything in AWS. No workflow runs Terraform — it is entirely manual. Two steps:

1. Immediate unbreak (~1 minute, no Terraform needed)

AWS Console → EC2 → Target Groups → ejam-prod-tg → Attributes → Edit → turn Stickiness on.
Type: load balancer generated cookie. Duration: 1 day.

This fixes production right away for users.

2. Make it permanent

Run locally from the ejam-infra/ directory of a checkout of this branch. Requires terraform + awscli and the custom ejam-terraform-deploy IAM policy on your user.

State lives in S3 bucket ejam-terraform-state-<ACCOUNT_ID>, with a separate state key per environment — so prod and dev are applied independently.

Production (this is the one that fixes the bug — prod runs 2 containers):

terraform init -backend-config="key=prod/terraform.tfstate"
terraform plan  -var-file=prod.tfvars -var="aws_account_id=<ACCOUNT_ID>"
terraform apply -var-file=prod.tfvars -var="aws_account_id=<ACCOUNT_ID>"

Dev (optional — dev runs 1 container so stickiness is a no-op there, but applying keeps the two environments consistent):

terraform init -backend-config="key=dev/terraform.tfstate" -reconfigure
terraform apply -var-file=dev.tfvars -var="aws_account_id=<ACCOUNT_ID>"

Please check the plan output

terraform plan should show exactly one in-place update to aws_lb_target_group.app (~ update in-place), adding the stickiness block. It is an attribute change, so there is no downtime.

⚠️ If the plan proposes to replace / destroy the target group, stop and check with us first — that would drop traffic.

Order matters

Do step 1 and step 2. Step 1 alone is exactly what happened in Feb 2026 — the console setting was never codified, and it was silently lost. Step 2 alone leaves production broken until someone applies.

Verifying it worked

curl -sS -D - -o /dev/null https://ejam.publicenvirodata.org/ | grep -i set-cookie

Once stickiness is on, that prints an AWSALB cookie. Right now it prints nothing.