On January 28th, 2022, AWS sent out an email announcement informing customers that GuardDuty now supported EKS findings. By all accounts, that’s great! I’m a big fan of GuardDuty and its continued expansion to other services is awesome.
However, there were some issues with this announcement.
First, it was sent after business hours on a Friday. Not great, but I can live with that.
All customers were being opted in to this new feature automatically. That’s… fine, I guess.
Oh, and this new feature would be a free month-long trial and would begin automatically billing customers after the trial period concluded.
As you can imagine, AWS Twitter justifiably lost its mind.
If you’re first hearing about this now, let me skip to the end and assuage any sense of rising panic you may feel: AWS corrected this the following week on Feb 2nd with a second announcement. To be super clear: you aren’t about to suddenly receive a surprise bill.
I’ve done some digging into this, and while I’m sympathetic to the forces driving AWS’ decision around this, the customer experience of this was pretty unfortunate.
What is GuardDuty?
For those who aren’t familiar, GuardDuty is a threat detection service that continuously monitors your AWS accounts and workloads for malicious activity and delivers detailed security findings for visibility and remediation. I run it myself in my accounts, as do many of our cost optimization clients, and while its pricing is generally reasonable, there are occasionally surprises I wasn’t expecting. The service is (and this may be surprising to the GuardDuty team itself) one I hold in high regard. It surfaces a lot of incredibly interesting and useful stuff.
Its pricing is derived from what it analyzes. It initially charges $4.00 per million CloudTrail management events, 80¢ per million CloudTrail S3 data events, and $1 per GB of VPC flow logs and DNS Query logs, with price breaks coming for all of those dimensions with additional volume.
Ease of Adoption vs Customer Transparency
The challenge that AWS has is around balancing customer concerns.
On one hand, for customers who have enabled GuardDuty already, they have an established baseline expectation of what the GuardDuty bill is going to be. If GuardDuty suddenly becomes aware of a whole new data source, the customer could wind up being unpleasantly surprised. But, on the other hand, if I’ve already configured GuardDuty in my accounts I don’t really want to have to go back and reconfigure it again every time it gets visibility into a new service. These competing concerns make for a delicate balancing act for AWS.
The EKS component to GuardDuty is reasonably priced at $1.60 per million events. However, due to the incremental feature rollout inherent to every AWS service, the bill customers sign up for as early adopters won’t resemble the bill they get later on. Automatically opting customers into new features could create very expensive surprise bills down the road.
What AWS Got Right
I think that AWS was on the right path here by opting customers into the GuardDuty for EKS trial who are already GuardDuty users. “Hey, there’s this new thing you should be aware of that likely helps you, but you won’t notice it if we don’t enable it for you” is a good position to take from a product management and customer success perspective, though not without its difficulties as I alluded to earlier.
I further think that launching the feature itself was clearly the right thing to do. Any data source that a security service can get to make it more effective is one worth pursuing!
And of course, listening to customers and reverting the change several business days later deserves significant praise; reversing course is hard to do for most companies. I was pleasantly surprised to see AWS make the change.
Lastly, I think they found the right balance moving forward; they’ve said that they’re going to create a toggle that customers can set to allow customers to opt in to future service coverage expansions within GuardDuty. I think this is a fantastic solution.
What AWS Got Wrong
Billing automatically after the free trial concluded is, in my opinion, the critical error that AWS made. Everything else flows from that decision.
There are downstream effects of billing changes to GuardDuty. Amazon Detective charges based upon the volume of (among other things) GuardDuty findings. I don’t know that the new findings are likely to be significant enough to change the trajectory of a bill that charges $2 per GB of findings ingested, but customers are a truly varied lot; some of us do REALLY weird things in our accounts!
I also think that rolling this feature out late on a Friday made it seem as if AWS were trying to get away with something. I don’t believe this to be the case, but the optics aren’t great. Most likely, the product team was simply excited to get the feature shipped and the timing of the announcement was an unfortunate oversight.
One of the downsides to being a $1.622T company is that people are less and less likely to give you the benefit of the doubt as your scale and scope increase; folks start to see ill intent behind every email.
A future with security truly baked-in
We can go round and round for weeks on the right way to communicate feature releases, handle free trials, and what-have-you, but let me propose something altogether different and far more impactful.
If I had a magic wand, GuardDuty wouldn’t cost anything additional for customers whatsoever. In fact, I argue all security services in a cloud provider should be 100% free to the customer.
If you’re a primary provider of infrastructure, you have a strong public stance that security is vitally important and is baked into your systems. And what better way to truly make that clear, and level up security for all customers at once, than by providing all security services for free?
As a customer, I should choose AWS for a variety of reasons — high on that list should be “security.” Not “security if I can pay for it,” not “security in varying levels depending upon my configuration,” but security full-stop.
The way that GuardDuty and its sibling security services work is that their billing is derived from my use of other AWS services. Let the cost of running the security services be baked into the primary service pricing model, and surface security findings early and often without me having to do anything as a customer.
If I choose to use an AWS service, neglect to configure it, and then find myself in an infosec incident, my takeaway is less “I should have configured the security services properly” and more “the baseline security offering for AWS is garbage.” That’s an unfair accusation, but it’s also very understandable.
As cloud providers do more and more things, security is increasingly going to be a competitive issue between the large platforms. The first provider to say “we believe security is a baseline, not a feature, and therefore all of our security services are now free” is going to have an absolutely massive advantage.
“We’ve always said security is Job Zero, and now we’re putting our money where our mouth is” is an incredibly powerful market position to stake out.
Let’s see who steps up first.
