ASG

Auto Scaling Group

Introduction

The Auto Scaling group maintains this number of instances by performing periodic health checks on the instances in the group.

If an instance becomes unhealthy, the group terminates the unhealthy instance and launches another instance to replace it.

Handling ASG Graceful scale down

The old connections should not tear down

There is discussion thread for same.

Solution-1: Workaround

When you realize you want to scale down your instance pool; create a NEW ELB. Only add the 'surviving' set of instances to the new ELB. Adjust your DNS entries to point to the new ELB, and wait for your DNS TTL to cut in.

Once traffic goes to zero (or once the DNS TTL has elapsed), you can stop the instance(s) which were 'removed'.

I definitely would have assumed that a health check failure would not terminate active connections, but only prevents new ones from spinning up. Active connections should be allowed to die gracefully (pending some kind of stale connection timeout).

Solution-2: ELB supports connection draining

We are pleased to announce Connection Draining, a new feature for Elastic Load Balancing. When you enable Connection Draining on a load balancer, any back-end instances that you deregister will complete requests that are in progress before deregistration. Likewise, if a back-end instance fails health checks, the load balancer will not send any new requests to the unhealthy instance but will allow existing requests to complete.

This means that you can perform maintenance such as deploying software upgrades or replacing back-end instances without impacting your customers’ experience.

Connection Draining is also integrated with Auto Scaling, making it even easier to manage the capacity behind your load balancer. When Connection Draining is enabled, Auto Scaling will wait for outstanding requests to complete before terminating instances.

To be evaluated iff applicable

Connection Draining may only go 90% to a full solution. It depends on the definition of "in-flight" and "new" requests.

If there is a state-full session active on a machine using Sticky Sessions, and that needs to be quiesced, then you want that SESSION to end, not merely the current request to the session.

The whole point of Sticky Sessions (session affinity) is clearly not understood if Connection Draining ignores it.

ASG userguide

ASG update launch config behavior

ASG Quick Reference card

Updating ASG AMI to new version

Existing connections dropped rudely when instance taken out of ELB using ASG