The load balancer only sends requests to healthy targets, and finds out which they are by checking them.
The check
Every interval it sends GET on the health check path to each target. A run of passes as long as the healthy threshold makes a target healthy; a run of failures as long as the unhealthy threshold makes it unhealthy. Defaults: /, every 30 seconds, 5 passes, 2 failures, code 200.
aws elbv2 describe-target-health --target-group-arn $TG --query "TargetHealthDescriptions[].[Target.Id,TargetHealth.State,TargetHealth.Reason]" --output table| State | Reason | Meaning |
|---|---|---|
initial | Elb.InitialHealthChecking | still being checked for the first time |
healthy | gets requests | |
unhealthy | Target.Timeout | the check got no answer: a security group drops it |
unhealthy | Target.FailedHealthChecks | nothing listens yet: user data still running |
unhealthy | Target.ResponseCodeMismatch | it answered, with the wrong code (a 404 path) |
unused | Target.NotInUse | no listener forwards to this group |
draining | Target.DeregistrationInProgress | leaving: no new requests |
The door the checks come through
Checks and requests both come from the load balancer's nodes. The servers' group lets the load balancer's group in, not the world, so nobody can skip it:
aws ec2 authorize-security-group-ingress --group-id $WEB_SG --protocol tcp --port 80 --source-group $ALB_SGTuning the check
A path that only answers when the app is really up, and a faster interval:
aws elbv2 modify-target-group --target-group-arn $TG --health-check-path /health --health-check-interval-seconds 10 --healthy-threshold-count 2The other side, how long each check waits (5 s; it must stay below the interval) and which codes pass: aws elbv2 modify-target-group --target-group-arn $TG --unhealthy-threshold-count 2 --health-check-timeout-seconds 4 --matcher HttpCode=200-299. aws elbv2 describe-target-groups --names web-tg --query "TargetGroups[0].[HealthCheckPath,HealthCheckIntervalSeconds,HealthyThresholdCount]" --output text reads the settings back.
When nothing is healthy
All unhealthy: the ALB fails open and tries them all. None registered: 503. A target that refuses is a 502, one that times out a 504.
aws elbv2 deregister-targets --target-group-arn $TG --targets Id=$WEB1A deregistered target drains: no new requests for the deregistration delay (300 s), then it leaves.