Everything you needed to know about spot instances.

AWS recently announced some more features for Spot instances, an exciting prospect for anyone wanting to make some significant savings. In this blog, we'll take you through what Spot instances are and how they differentiate from Reserved instances. Spot instances, next to On-demand and Reserved instances are another financial model for your EC2 workloads. With On-demand, you pay a fixed hourly price per instance. With Reserved instances, you're able to reduce your EC2 cost by around 30-40% in exchange for a 1 or 3 year commitment to AWS. While Reserved instances bring you a significant discount compared to On-demand, you are tied into this commitment. If AWS reduces their prices or introduces new instance types that are cheaper, your savings could end up lower and you'll still be paying for your Reserved instances until the end of the commitment. Spot instances work on a supply and demand model, meaning AWS takes advantage of unused instance capacity, giving you the opportunity to save as much as 90% compared to the On-demand cost.

So what's the catch?

First, we need to understand how exactly the pricing model for Spots works. When you want to use Spot instances, you submit a Spot request in the EC2 console or using the CLI. Here, you have to specify all parametres of the instance (OS, type, size, availability) and your bid price. The bid price is the maximum price your willing to pay for a specified EC2 instance. When your bid price exceeds the current market price and the instance is available you can use it at the market price. The current market price is given live updates and reflects the need for specific instances. If your bid price is lower than the current market price, you will not get the possibility to use the Spot instance. If you're using a Spot instance and the market price exceeds your bid price, your instance will be terminated or stopped (you'll receive a notification two minutes before it happens). It's good to note here that if the instance is stopped you will still be paying for EBS volumes as usual, but you can terminate or stop your instance at any point, (Spots are different from Reserved instances as they can be stopped at anytime). The way pricing works also means you're not able to forecast your costs and plan your budget with 100% accuracy, however, you can still view a specific instance market price history in the AWS console. In the example below, we're looking at an m4.xlarge instance running in a us-east-1 availability zone, (each AZ has its own market price, compared to On-demand and Reserved instances where price is set on a regional level). Let's say you want to use an m4.xlarge Spot instance in a us-east-1e availability zone. If your bid price was $0.1, (shown on the graph below) the market price would exceed your bid-price, and your instance would be stopped or terminated. However, for the majority of the time you would actually pay around $0.06 per hour, (which would be 70% less that On-demand). Now let's say your bid price was $0.18 (close to the On-demand price). In this case, your instance would be stopped less frequently because the price was only exceeded a few times in a given term. You would still pay around $0.06 for the majority of the time, but because you pay the market price, your bid should only exceed the current market price. You would be saving around 65-70%, even though your bid price was much higher. It's best to bid below the On-demand price, which brings not only savings, but also reduces the risk of your instance becoming inactive. Note: T2 and HS1 instances are not available on the Spot market Let's see how the pricing history looks in an eu-west-1 region fro certain instances: For the majority of the time, the most popular instances tend to keep their market price significantly below the On-demand price. If you were to run a single Spot instance such as m3.medium in the eu-west-1a availability zone for the previous 3 months, you instance would be running without any interruptions for a whole term, (saving you around 80% compared to the On-demand price). You could still bid your price around the On-demand price and get the same savings. The problem starts however when you want to use bigger instances (eg. m4.16xlarge). These bigger, more expensive instances tend to exceed the On-demand price, making them a bit riskier. However, over three months in the example below, the m4.16xlarge instance price exceeded the On-demand price only once. If you choose a more 'unusual' instance, there will be a higher possibility of exceeding the On-demand price. Let's look at some other examples: Instances like d2 or g3 fluctuate much more than the 'standard' instances. There's a possibility of saving a considerable amount on these, but you need to be prepared for a sudden stop or termination. What if you want to run your application on different instance types and recieve optimum savings? The answer to this would be to use Spot fleet. A Spot fleet is a collection of Spot instances which attempts to maintain the capacity you've specified. For example, when you run your application on a single m4.large On-demand instance, you can use Spot fleet to specify that your application needs to be run on multiple instances. You can also specify that your desired capacity is 1, meaning you can run on either the m4.large, m4.xlarge, m4.2xlarge etc, as all of them have the same specifications as the m4.large, (you can also choose instances from another family that match your desired instances' parametres). You can then set up your bid price globally for around the same price as the m4.large On-demand price. Spot fleet then attempts to launch the specified instances that will result in the lowest cost. This results in an overall higher ability and can lead to even higher savings. For example, if the m4.large instance price rose over the On-demand price, (some of your other instances in your spot fleet might currently have a lower market price) then your workload will be moved to another instance instead of being stopped or terminated.

What else can I do with Spot fleets?

You can use instance weighting to specify differences between various instances according to what you need. For example, your application performs best with at least 8GIB of RAM and 2 vCPUs. If you wanted to run one 8GIB plus 2 VCPUs as 1 unit (running 16 of them at a time), and use m4.large as our base unit, here's how the initial list of instances (pools) would look next to a Spot fleet:

Instance Type	RAM (GIB)	vCPU
m4.large	8	2
m4.xlarge	16	4
m4.2xlarge	32	8
m4.4xlarge	64	16

How do we weight the different instances?

In the example above, your target capacity is 16, meaning that you've requested to run 16 m4.large instances. An m4.large instance would then have the weight of 1 and an m4.xlarge (which is two times bigger) would have the weight of 2 and so on. You can weight your instances any way that suits you, but you should do it by using the instance parametres.

Spot fleet allocation strategy: Lowest Price vs Diversified prices

You can choose your strategy to be a diversified price, or the lowest price. The lowest price strategy is the default and means that your Spot instances will come from a pool of the lowest priced per unit. Diversified price allows your instances to be distributed as evenly as possible across all your pools. The bid price works with weighting, so your bid price becomes a bid unit price. If we wanted to set up the bid price as $0.2 (an m4.large On-demand price), a single m4.2xlarge instance would cost $0.4 ($0.2 per unit times 2 units by weight). You can also override the global bid price for a specific instance in you Spot fleet to either maximise or reduce the risk of interruption. If you chose to stay with the lowest price allocation strategy, your Spots will be launched in a pool that has the lowest price per unit. If your Spot fleet launched 16 m4.large instances becasue they were the lowest price at the time, at any given moment, the m4.xlarge could become cheaper per unit. This is either because the m4.large Spot price has risen, or the m4.xlarge Spot price lowers. If this does happen, your Spot fleet will switch to run on 8 m4.xlarge instances. How to weight your instances and which you include in your Spot fleet request is important. Your Spot fleet, whenever a calculated instance count is not a whole number, rounds it up. So if with the given example, we wanted 10 instead of 16 to be the target capacity and m4.2xlarge to be the cheapest per unit, your Spot fleet would launch 3 m4.2xlarge instances. This may not give you the highest savings possible, (because you need 2.5 of this instance, but Spot fleet rounds it up to 3), so be aware of that when requesting your Spot fleet. If you choose the diversified allocation strategy, your Spot fleet will attempt to distribute the workload evenly across the pools. This is the best strategy when you don't want to worry about fluctuating prices and keeping a steady usage, but still want to keep you bid price in check. In the example, your Spot fleet would choose to run 1 m4.4xlarge (8 units), 1 m4.2xlarge (4 units), 1 m4.xlarge (2 units) and 2 m4.large (2 units) instances. Whenever one of the pool's Spot prices are higher than the bid price, your spot fleet will try to adapt to the new situation whilst still trying to diversify your usage across the available pools.

It doesn't end there: Autoscaling

Autoscaling scales up or down your autoscaling group based on policies and is especially useful when you experience peaks in your application's usage. You can also do the same with Spot instances. Your autoscaling group is now your requested Spot fleet and you can set up policies based on CloudWatch alarms. With our above example, let's say you want your applications to use more instances for better availability, or less instances for better savings. You could set up a scaling policy based on CPU utilisation and, whenever you utilisation reaches 90%, you can add one new instance (or unit) to your Spot fleet. If your CPU usage is below 60%, you would want to remove one instance from you Spot fleet. This can also work by percentage, (so you can add or remove 20% of your current capacity) and you can specify the action after the alert to set up where you specify a new capacity amount. Keep in mind that Spot fleet always rounds up, so your scaling policies can have escalated action, or no action at all. It depends what type of pools you have in your Spot fleet and how you describe your scaling policies. Are Spots for everyone? No, if you think your application is likely to be stopped or terminated, then you're better off using Reserved instances. Perhaps you want to run a minimum required workload on Reserved instances, whilst also handling Spot peaks. Spots are perfect for non time-sensitive batch jobs like media encoding, data analysis, or big data, meaning any application that doesn't need to be up and running all the time is a good example of a Spot use case. If you'd like to find out more about Spot or Reserved instances, please get in touch here.