Good morning!
Welcome to issue number 115 of Last Week in AWS.
Look around your office. Someone there probably cares about the AWS bill. If you enjoy this newsletter, I’d like to ask a favor: can you introduce me to them? Contrary to what you might suspect, this is less about me trying to sell them anything than it is trying to find new and different cloud billing problems about which I can dive deeply. Plus, they’ll find me refreshingly entertaining; just as this newsletter is the highlight of your week, chatting with me will be the highlight of theirs.
I’m in Boston for re:Inforce this week. If you’re here, let me know–I’d love to chat with folks for a few minutes. And of course, if you’re based in Boston and want to talk AWS architecture, bills, navel gaze about the future, or just experience the firehose of realtime in-person snark, let me know. I’ll have some time to come by and chat / deliver an entertaining brown-bag session.
The same goes for folks in New York–I’ll be back on the east coast in two weeks to attend the NYC summit, and the same offer applies.
This issue is sponsored in part by NetApp.
Yes, NetApp! Not everyone’s environment is a born-in-the-cloud startup that sprang fully formed into the world a year ago. Some of us have on-premises data centers, which give rise to hybrid cloud environments. How do you monitor those? Consider NetApp’s Cloud Insights to grant insight into all of your infrastructure—not just the parts that live in a public cloud somewhere. Thanks to NetApp for their support of this newsletter.
This week’s issue is sponsored in part by LightStep.
With distributed systems, the current state of most monitoring rounds down to “Observerless.” Meet LightStep. LightStep offers complex APM for modern applications. Designed with modern, high-scale, high-traffic architectures in mind, LightStep makes it easy to spot, diagnose, and solve performance issues.
From the Community
Getting TLS for EFS via a proxy sounds like a half-step above actually shoving a NetApp filer into us-east-1, but there you have it.
I’m always up for a round of better logging for Lambda functions. Right now my logs are… sparse.
Despite a few technical quibbles I have with the article, this refutation of multi-cloud echoes an awful lot of what I’ve been saying for a while now.
My friend and yours, Nitzan Shapira of Epsagon opines on not being surprised by your serverless bill. I’ll admit it–I learned a few things from this post. Now to find a company whose serverless spend is meaningfully large in the context of the rest of their AWS bill. If that’s you, please hit reply. I’d very much like to speak with you just to further my own understanding.
A whirlwind tour of CDK, which is another layer of abstraction piled atop CloudFormation.
The always-nuanced Paul Johnson opines on Serverless being a doctrine instead of a technology. He’s right. It’s also increasingly becoming a religion, and I’ve got issues with that. More to come on that in a future issue.
This is nifty–a top 500 supercomputer running on top of AWS on a corporate credit card. There were no grants given, and no support involvement past the standard limit increase requests you’d need to scale up sufficiently. There’s a wonderful capability story in here–and this one is aspirational.
Segment’s engineering blog returns with a deep dive into cultivating your data lake. I love their stuff…
While a story of migrating to Aurora without downtime is always good, more entertaining are tales of migrating databases with incredible amounts of downtime.
Tim Bray of internet fame talks about the adoption of Golang, and what he likes about it. While he does indeed work at Amazon, he’s also Tim Bray; I will link against anything he cares to write.
Anthony Liguori demonstrates what an AWS Outpost rack looks like–and gives us a glance into what AWS hardware looks like in various regions.
Jobs
If you’ve got an interesting job for this newsletter’s eminently employable subscribers, get in touch!
Once upon a time, I wrote a not-particularly-flattering article about Amazon CloudWatch. As a result of that article, I got to meet Bob Wilkinson, the CloudWatch GM. He demonstrated exactly what Amazon means by “customer obsession” via thanking me for my feedback instead of punching me in the face as I oh so very richly deserved–and then fixing the things I’d pointed out! The entire team is like that–and despite what you may think, I’d endorse working on Amazon CloudWatch if massively scaled time-series problems are up your alley. Interesting problems, empathetic leadership, and the best perk of all: when the cloud catches fire, your tools are how the world watches it burn.
Choice Cuts
Amazon EKS now supports Kubernetes version 1.13, ECR PrivateLink, and Kubernetes Pod Security Policies – Rejoice, Kubernetes fans! You’re saved yet again from having to do any actual work by more k8s feature releases. I swear, you’re becoming the infrastructure equivalent of the Rust community…
Amazon Lightsail Partners with GoDaddy to Simplify WordPress Management – Congratulations to Amazon on another fantastic partner pick. It’s not every day AWS manages to find a partner with as many controversies surrounding them as AWS has service offerings. Who doesn’t just adore GoDaddy, voted 2019’s Registrar of Last Resort?
Amazon QuickSight now supports fine-grained access control over Amazon S3 and Amazon Athena! – This release gets an exclamation point! Please use QuickSight! Our families will starve! We’d love to have a customer someday!
Announcing Enhanced Lambda@Edge Monitoring within the Amazon CloudFront Console – Holy crap, you can monitor Lambda@Edge functions?! I thought they just disappeared once you deployed them, didn’t show up ever again, and just silently increased your bill while you assumed they were working.
AWS Lambda Console shows recent invocations using CloudWatch Logs Insights – I get it; partnering two companies is hard, and takes a long time. After many months of effort, a new partnership has launched between AWS Lambda and the Amazon CloudWatch Logs teams. They’re delivering similar functionality to what Epsagon and IOpipe have offered for years. Good hustle, folks…
Amazon CloudFront announces seven new Edge locations in North America, Europe, and Australia – Seven new locations, not a single one a new city–so my wall map of these doesn’t get a new pin this week.
Introducing AI Powered Speech Analytics for Amazon Connect – I’m just going to throw it out there: if you need an AI powered system to tell you that a customer is ripsh*t furious, you don’t need an AI powered system. You need customer service reps who can fog mirrors.
Introducing New Instance Sizes for Amazon EC2 C5 Instances – Bigger, faster, stronger, more confusing, same specs between the c5.24xlarge and c5.metal, not available in Ohio: this is a classic EC2 instance release.
Use IAM access advisor with AWS Organizations to set permission guardrails confidently – While I love the idea, everything about this post goes against the narrative of being able to do anything in the world of IAM with confidence. The edge cases around here are perilous–you’re either granting too much access where you shouldn’t, or removing access that only gets used in infrequent yet incredibly important situations. Worst of all, I don’t really have a better answer around this. It seems to be something of an intractable problem.
Amazon RDS now supports Storage Auto Scaling – My ass it does. Sure, this enhancement lets you automatically expand storage. Yes, that’s good and important, but if it can’t scale back down, it’s not “autoscaling!” The entire point of the cloud is elasticity, and scaling down database storage when appropriate remains an intensely manual process.
Reduced Pricing for AWS P3 Reserved Instances is now available in the AWS China (Beijing) Region, operated by Sinnet and the AWS China (Ningxia) Region, operated by NWCD – I swear, if AWS uses this to increment their “X price cuts since we launched AWS” metric I will flip a table. They’re operated by other companies, so price cuts don’t count.
You can now publish Amazon Neptune Audit Logs to Cloudwatch – This smacks of “if we don’t support audit logs in CloudWatch soon, QLDB is going to launch and absolutely eat our lunch in a really embarrassing fashion,” but what do I know?
How to prompt users to reset their AWS Managed Microsoft AD passwords proactively | AWS Security Blog – Once upon a time I worked at a shop that rotated passwords every 60 days. Don’t do this; even NIST agrees with me now. What made this extra special is that if you forgot to rotate the credential on a device of yours, the old password would trigger a lockout and you’d have to call the helpdesk to resolve it. Ideally this gets around that pattern, but honestly I don’t deal with Microsoft’s AD enough to care anymore.
How to sign up for a Leadership Session at re:Inforce 2019 | AWS Security Blog – On the off chance that you haven’t made your plans for this week, here’s how to sign up for a session. I’m even mentioned again!
Working backward: From IAM policies and principal tags to standardized names and tags for your AWS resources | AWS Security Blog – First there were tags. Now there are principal tags. If they work hard enough and demonstrate a mastery of the Amazon Leadership Principles, one day they may become distinguished tags.
Amazon EKS – Managed Kubernetes Service – Not a blog announcement as such–but AWSECS4K8S(EKS) was renamed to simply “Amazon Elastic Kubernetes Service,” effectively killing my joke about its incredibly convoluted name and further validating my “ranting at AWS like a loon on Twitter every week is the best way to spark change” approach.
Tools
A GitHub to EC2 deployment pipeline for those hard to reach integration areas.
A terraform module to help you spin up a highly available Jenkins deployment. Be forewarned, the architecture here isn’t great–but it’s Jenkins, so what are you going to do? If you need such a thing, check it out. If you can get away with using a managed CI/CD service, please do that instead.
A fun jaunt through automating the setup of a static site on S3 using Terraform.
20 Patterns to Watch for in Engineering Teams
GitPrime’s new book draws together some of the most common software team dynamics, observed in working with hundreds of enterprise engineering organizations. Actionable insights to help you debug your development process with data. Get Your Copy.
… and that’s what happened Last Week in AWS.
