Minimizing cost is the single most important factor in someones decision to move workloads to the cloud. Yes I'm starting the blog that way, and today I'm going to discuss why. I'll give reasons why organizations should consider cloud infrastructure if they're apprehensive to migrate. I'll even cover how some organizations may be controlling cost incorrectly and ways to improve. Starting with Cost Optimization from the Well Architected Framework was important to me because it's relatable to every cloud user.
Whether you're a student or weekend warrior testing with AWS services, a startup or large entity , the cloud bill must be paid and if you're "overpaying" then you're missing the point. AWS themselves state that organizations MUST invest in Cloud Financial Management if they want to realize the benefits of cloud computing. Of course bringing down cost comes at a cost, whether that be time to market or application performance, but if you invest the time and energy into right sizing and only paying for what you need you should be able to do both. It's easy to spin up the biggest and baddest instance type available in the name of speed but if the workload doesn't fit the infrastructure you're literally giving money away.
I had this thought while scrolling on Twitter, I just optimized my algorithm for pretty funny tech content and came across this tweet from @fekdaoui:
I'm sure they're joking and its hilarious to expose your API keys in this fashion but now you're left with a heading spinning bill. Made me laugh but what happens when it isn't a joke, you'll hear of similar stories all across the Internet but they're not trolling.
Optimizing costs isn't an automatic endeavor and takes work. With the following tools and services, you should be able to control costs and unleash the true power of the cloud:
- AWS Budgets
- Right sizing | Spot Instances
- Auto Scaling
- Usage Monitoring and Alerting
- AWS Cost Explorer
The list above is just a portion of all that can be done to bring down costs and protecting yourself from large cloud bills. Resources should be enough to complete the job, fast enough to serve the end user, and smart enough to remove itself when the job is complete. This post will not regergitate what AWS has already created, their documentation is pretty good. Rather I'm directing you to the source and highlighting the importance of the principle.
AWS Budgets
Along with configuring two factor authentication when a new AWS account is created, an AWS Budget needs to be the immediate next step in your provisioning. It's very important for solo and smaller teams but large organization benefit as well, the price that triggers the budget can be a small number for the former and relatively high for the latter.
They can be set quarterly or annually and integrates nicely with other services such as CloudWatch, SNS, and Lambda. These services enable automatic alerting and response for when a budget is reached. Individuals like myself that work independently in an account for research have to control and understand cost. Bills add up and you were none the wiser to a forgotten instance for some weekend experimentation. You'll be happy you took the time when configuring an AWS Budget.
Right Sizing
If you're like me, when you're deploying onto EC2 instance you think "Does I really need this much compute or memory?". It's a complicated answer but I've found in most cases, I don't. This can vary on many different levels such as traffic flow, and memory management but should also be worked on and improved continously.
The metrics and data from AWS Cost Explorer (more on this service later) will assist in analyzing cost while AWS Computer Optimizer gives recommendations depending on your workloads.
Auto Scaling
One of the most fundamental tenants of cloud computing would be compute resource on-demand, when you need them. For some days you need 2 instances for a service while others may need 5. Depending on the time of day where its low traffic, the second service only needs 2 instances to run efficiently so you manually delete them. Then you have an unexpected surplus of traffic but the service crashes because you currently have 2 instances when you needed the original 5.
This hands to keyboard method is inefficient and is costing you users and revenue. Luckily AWS' EC2 Auto Scaling can automatically scale your service for you. This powerful feature can be scheduled, based on metrics, and health checks. For example if total CPU utilization on in an EKS cluster goes over 50%, scale up to handle the processing.
Usage Monitoring and Alerting
This section applies to every other section. Budgets, Auto Scaling, right sizing, none of them matter unless asset owners or other services are notified and can respond accordingly. Knowing is half the battle when it comes to cloud cost. With services such as CloudWatch, SNS, Lambda, and CloudTrail cost can be closely monitored and neutralized automatically.
For example if an AWS Budget is reached, an SNS email can be sent to engineers with details including the services that are commanding the most costs. This same AWS Budget can work with EventBridge and a Lambda function to automatically delete unmounted EBS volumes to immediately lower cost.
AWS Cost Explorer
Cost Explorer is a CFO's best friend. It provides a visual to cloud spend for ease of use to shareholders including forecast of what you're spending might be in coming months. The service is free to use in your account unless you're getting data programmatically so why not get familiar with it.