Welcome to the twentieth issue of Last Week in AWS.
This email is an hour early this week due to the start of the AWS Summit in New York. I’ll be live tweeting the event livestream starting at 9:30 Eastern Daylight Time.
Community Contributions
A couple of months back I made fun of Amazon’s quickly-settled lawsuit, attempting to enforce a non-compete. Now the shoe is on the other foot, as IBM’s former CIO attempts to join AWS. IBM is suing both to prevent him from working for AWS for another year, and to remind the rest of us that there are vanishingly few reasons to work for IBM in the first place. Their own lawsuit acknowledges the limitations of their own cloud, along with showcasing their view on employee mobility.
A turnkey solution that ties API Gateway and Lambda into a multi-region solution that automatically fails over when Lambda degrades in a particular region. I like this!
Percona has written a blog post explaining the non-intuitive process to configure parameter groups for Aurora clusters.
Jeff Barr tweeted out a screenshot of the new AWS Lambda console. The most remarkable thing about AWS’s UX is that the consoles for the various products are of such uneven quality. There’s no unifying theme…
This is the first time I’ve seen Terraform leveraged to build and deploy Lambda functions. Have I been living under a rock, or is this an awesome and elegant answer that nobody’s been talking about?
I love this article and the followup not just for its cautionary tale about how budgets and easy-to-make mistakes can catch up with you in AWS, but for the radical transparency. These mistakes are incredibly easy to make. What strikes me is how well the author takes ownership of the missteps, and explains his thought processes. Very, very well done.
Amazon has joined the Cloud Native Computing Foundation. Speculation grows to a fever pitch about what Amazon’s approach to Kubernetes is going to be. Watch this space…
Cloudonaut walks us through a new CloudFormation template to simplify back up DynamoDB. Well written, and solves for a problem that many of us have struggled with.
Sticking our heads above the technical for a moment, this article gives good insight into how many businesses see cloud providers like AWS. Agree or disagree with this perspective, it’s widely shared by the people who write large checks.
Cloudonaut comes at us twice in one week with a [CloudFormation vs Terraform smackdown](https://cloudonaut.io/cloudformation-vs-terraform/. At this point, Cloudonaut could post a picture of a dog sleeping and I’d still link against it.
Continuing the bike-shedding of failing S3 between regions, this week’s features a lambda function that modifies the CloudFront distribution instead of individual resources.
NetApp shows us how to mount Amazon S3 as a file system. Please never, ever do this unless you have absolutely no other choice. I promise it won’t end well for any usage that is non-trivial. See this week’s tip for more.
Choice Cuts From the AWS Blog
New – AWS SAM Local (Beta) – Build and Test Serverless Applications Locally | AWS Blog – AWS has released a new local toolset for building serverless applications. This is well worth paying attention to; they’re giving us a start-to-finish local toolchain to build Lambda functions, API gateways, and serveral other bits and pieces locally, then “push button to deploy to production.” The implications for not only development, but automated testing can’t be understated. Oh, and be sure to vote for “spaces” in Thirty-Lamba-Functions-in-a-Hoodie Randall Hunt’s poll in the linked post. I’ve got a meaningless bone to pick with the “tabs” people.
Launch your rendering fleet in AWS with Deadline 10 (Public Preview) – “Good morning! We’ve acquired this awesome rendering tool, and we built an AWS service around it. Everything is great– but we need a name by 10AM today. Let us know what we’re calling it, please.” Introducing Deadline 10, by Amazon.
Get Ready for AWS re:Invent 2017 | AWS Blog – “Re:Invent draws near, and with it the dizzying array of fun events that will be at the show. We have much to look forward to, such as the re:PLAY party, the Tatonka Challenge, Pin the Tail on the Venture Capitalist, Whose Cloud Is It Anyway, and the Lambda Powered Slot Machine. More to come as re:Invent approaches.”
AWS Snowball and AWS Snowball Edge Available in the South America (São Paulo) Region – Snowballs come to a region that doesn’t see snow more than a couple of times a century. It’s always seemed slightly odd to me that services like this are region bounded. “Yes, you can send your box full of drives to some of our datacenters, but not others; it turns out we’re running short of ethernet cables in Brazil” is just bizarre.
Tools
“Press button, receive static website hosted on S3” brought to you by the power of Javascript.
If you run a security consulting firm that makes its living by whitelabeling tools that do the heavy lifting for you, you’re going to love Zeus; it checks all of the boxes you’d expect for a best practices security audit. It’s nice to see Scout2 have some competition.
I thought sure I’d mentioned Terraforming in a previous issue– lord knows I’ve used it enough in my various projects. It’s a ruby gem that lets you work with existing environments without repaving them in Terraform.
Finally, this handy walkthrough teaches us how to set up a Slack channel as an SNS subscriber. Use with caution– you can get tens of these per second without a limit increase. Please don’t ask me how I learned that.
Tip of the Week
There are a handful of tools and approaches out there that talk about using S3 as a filesystem. I linked to one above. This is a bad plan, and is better met by the still-not-great EFS service instead. Here’s why mounting S3 is a terrible idea.
S3 is an object store, not a filesystem– just ask any condescending storage admin you may meet. What this means is that S3’s entire abstraction model thinks in terms of objects– not files. Here’s a short list of things that may break if you treat it otherwise.
- If you update an object, it may very well not update atomically– that is to say, some calls to that object will retrieve the old version of the object.
- If you just wrote something, listing objects may very well not show the thing you just wrote.
- If you have a 25MB file, and you want to change one line within it, you get to reupload that entire 25MB object again, rather than just the block you updated.
- Every time you list objects, you’re incurring a request charge. This may surprise you in horrifying ways, as most tools don’t think of enumerating files as an expensive operation.
- Most applications expect filesystem latencies to be an order of magnitude lower than they are with S3. “Talk to the local disk” and “talk to something across the internet” aren’t even in the same latency ballpark.
- You’ve got a hard limit of single-PUT uploads of 5 gigabytes, and a maximum single object size of 5 terabytes (constructed of multipart upload requests). Very few applications are set up to deal with this kind of arbitrary limit.
But other than that, it’s a great idea!
…and that’s what happened Last Week in AWS.
