An interesting thing came out of the New York AWS Summit last week. There’s been a change made to S3 that (buried in the release announcement) indicates that you’re no longer required to spread out your objects across random key prefixes. For most use cases, this is an esoteric and not particularly well-known limitation gotten to S3; you generally only learned that this exists when you run into it face first at high-speed. In plain English, if you listed Americans in a bucket with keys that resembled s3://bucket/lastname/firstname, under load you’d find that the ‘smith’ prefix was incredibly slow.
Indeed, you often could end up in a situation of not know this existed at all, and only come away with a sense of vague disquiet when S3 performance left you wanting. “I guess the service is just crappy,” you’d think, unfairly.
The engineering that went into this is absolutely no joke. I’m always thought that this was an atavistic relic from a time when implementation details unfortunately tended be presented to the customer. You shouldn’t have to know or understand how the service works under the hood in order to get acceptable performance. I’m very happy that this artifact has now been consigned to the dustbin of history, lingering now only in updated certification tests and AWS trivia contests— which often amount to the same thing.
