Episode Summary
Pete Cheslock is an advisor and consultant who helps startups with product strategy, messaging, and other go-to-market needs. Prior to going out on his own, he worked at a slew of tech companies, holding positions such as VP of Products at CHAOSSEARCH, VP of Technical Operations at Threat Stack, Inc., Director of DevTools at Dyn, and Director of Technical and Cloud Operations at Sonian. Pete holds a master's in business administration from Babson and a bachelor's in communications from Michigan State University.
Join Corey and Pete as they discuss the differences between CHAOSSEARCH and Elasticsearch, proper etiquette for the conference badge-scanning experience, how tech can be a bubble and not everyone might know the tools you do, the increasingly prominent roles artificial intelligence and machine learning play in the AWS ecosystem, why the re:Invent experience is like a marathon, what it’s like listening to a talk on a pair of headphones, which re:Invent announcement made the least waves, why diversity amongst chip manufacturers is a good thing, and more.
Episode Show Notes & Transcript
About Pete Cheslock
Pete is Professionally Unaffiliated, but spends his time consulting and advising companies such as CHAOSSEARCH and CloudTruth.
Pete is Professionally Unaffiliated, but spends his time consulting and advising companies such as CHAOSSEARCH and CloudTruth.
Previous he was the VP of Product for CHAOSSEARCH, and before that he has been running large scale infrastructure on Amazon Web Services since 2009
Links Referenced
- re:Invent Expo Nature Walk Twitter Thread
- Twitter Username: @petecheslock
- LinkedIn URL: https://www.linkedin.com/in/petecheslock/
- Personal site: https://pete.wtf
- Company site: https://pete.wtf
- CHAOSSEARCH
Transcript
Announcer: Hello, and welcome to Screaming in the Cloud with your host, cloud economist Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.
Announcer: Hello, and welcome to Screaming in the Cloud with your host, cloud economist Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.
Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. I'm joined this week by Pete Cheslock. Pete, welcome to the show.
Pete: Hello. It's awesome to be here yet again.
Corey: Oh yes. This is now a tradition annually where we do a reinvent, recap. And conveniently, this episode is sponsored by CHAOSSEARCH, a company that you until recently were the VP of product of.
Pete: Yeah, CHAOSSEARCH. I fixed my caps lock key by moving on from CHAOSSEARCH. I'm just an advisor, consultant, helping them out on a couple of projects, and very happy fan of the company.
Corey: Which is how I started, and then they started paying me to sponsor things, so now it's cool. It's always nice when you're paid to promote something you actually like. CHAOSSEARCH, for those who are not aware or are crawling out of hibernation, which you're a few months early for, is Elasticsearch except it doesn't suck, which is not what they say in their branding because they have to be professional, whereas I can just be honest and funny at the same time.
Pete: CHAOSSEARCH, I love the product. I love the world that they're working in. We talked to so many customers storing months and years of data, or wanting to, and trying to do that with Elasticsearch, unless you have unlimited money of which, of course, there's one or two companies out there with that kind of money, there's really just no other way to do it. And it's a really awesome technology. I love, love what they're building.
Corey: As do I, and we will get to what they do in more depth soon. But the short answer is, of course, that they wind up separating compute from storage with an Elasticsearch compatible API, data lives in S3. Now, that seems a good jumping off point for what we saw at re:Invent. I think the dumbest name of the show award goes to something similar in that Amazon Elasticsearch is painful, annoying, et cetera. It's also expensive because it runs on burning piles of money, either yours or someone else's. They now have a lower cost storage tier called UltraWarm, apparently named after a type of personal lubricant.
Pete: I thought it was actually called LukeWarm, but UltraWarm, I think they might've missed the boat on that.
Corey: They tried that. It got a tepid response. It was an experience. Again, I feel like you have amazing engineers at AWS, amazing product visionaries who build these things out, and then they have a naming committee at the end that picks the best name in the list, and then they go drop the bottom on that list, and get it backwards and pick the name at the bottom, on the, "For God's sake, never name something this," section.
Pete: Well, I mean, I think they roll a dice. I think there is a bunch of dice with a bunch of names on it, and they toss it out there and one dice maybe flips up, says, "Ultra," and one dice flips up says, "Warm," and they say, "Ship it."
Corey: I feel the need at this point to be very clear that you speak for yourself and not me in this. I, on the other hand, would never disparage machine learning in such a way.
Pete: That's going to come out as the new deep dice machine learning, and it's only priced by $10 per thousand rolls.
Corey: God does not play dice, quoth Albert Einstein. Yes, we freaking do, quoth Werner Vogels.
Pete: But really, I think Amazon is known for building things their customers ask for, or maybe just a random person on the streets of Seattle ask for. But this is, again from my time at CHAOSSEARCH, this is a problem. People want-
Corey: You saw that with the DeepComposer. Someone asked for a keyboard with machine learning built into it, and it turns out that person was probably Jeff Barr. But they didn't ask enough follow up questions to learn it was a keyboard you could type in, as opposed to the kind of keyboard that you could make music out of, and now we're stuck with DeepComposer because Amazon never turns anything off.
Pete: Well, now you can make some pretty cool EDM music I assume, and maybe next year, you'll be on stage at Replay.
Corey: Right now, I think they're more concerned with how to keep me from attending Replay, but that's neither here nor there. I make nuisances out of myself. It's kind of what I do. One of the things that did surprisingly well was, and I'll throw a link to this in the show notes, was my Twitter thread on my re:Invent nature walk through the Expo.
Pete: That was actually one of ... There's obviously a lot going on on Twitter, hard to keep up on everything, but as I was catching up on threads and things at the end, that was one of definitely I think the better threads that I saw, and something that ... I personally do it, where I will walk the Expo hall and honestly look at just a lot of bad marketing or mistakes people make or how aggressive the Datadog BDRs outside the booth are. But I think you really brought it to a new art form as you are wont to do.
Corey: Yeah, for those who are unaware, I acted as if I was on a safari, guiding a tour group through various booths, pointing out various companies and making up compelling backstories about these animals and their ecosystem. Remember, don't let them scan your badge or they'll grow to depend on re:Invent attendees to survive, and we don't want that. I will say that I heard rumors, though I couldn't confirm them myself, that logs.io has taken the mantle of aggressive badge scanning from Datadog.
Pete: Interesting. I actually walked by it a few times and I never got approached from them, although I did stop by the Datadog booth to say hi to a couple of friends, and of course got the requisite scan. So I feel like I should win an award for the furthest away you can stand from a booth and still get that scan.
Corey: Oh, you'd be surprised. The fun thing's going to be the people you never saw coming. Some of the better Datadog BDRs can scan a badge clean of 200 yards.
Pete: It's a real talent. I mean, I was looking out for the ... In the old days of the Pringles can WiFi thing. I'm just waiting for one of these, for booths to have the little secret Pringles can pointing at people as they walk by.
Corey: Absolutely, although they're going to wind up doing it in something far less obtrusive than that, because the Pringles can, come on. They're going to at least wrap it with their branding.
Pete: Yeah, they're going to put the dog on it or use those socks for something. They'll find a way to get that dog on it.
Corey: One of my favorite responses when approached by someone from Datadog is, "Hi, are you familiar with Datadog?" is to get this outraged look on your face and come back with, "Date a dog? That's disgusting!" And watch how quickly they backpedal.
Pete: Well, last year I talked to a friend of mine who's at Datadog, and we were chatting and I was talking to him, and Datadog, huge booth, they have multiple booth because they're so big and spend so much on re:Invent. And I just said to him, "It must be pretty amazing. People come up ..." And I've only worked at start ups, so when people come up to me, it's like, "Where do you work? Have you ever heard of my company?" And the answer is, almost often always, "No, I haven't heard of you. Tell me what you're doing." So I said to him-
Corey: Are you sure? We're AWS. They get super offended when you say that there.
Pete: Yeah. What do you guys do again? What is Amazon about?
Corey: Our product strategy is yes. Next question.
Pete: So I said, I was like, "It must be very interesting to just, people come up and know exactly who you are," and this is where I realized that we live in the bubble of technology and that, he said, he's like, "Most people that came by had no idea what we do." And I just kind of was blown away by that statement. I'm curious if that's still the case post IPO, but I'm guessing so. There's still a lot of new people into the space and really, I mean, how would you hear of Datadog if you've never been to an industry event and are just getting involved in cloud?
Corey: Hey, if I had a SaaS product, I'd almost certainly pick them to monitor it. It was easy to make fun of them when they first came out. It's, "Look at this nonsense. It's easily understood, it's like Fisher Price for monitoring." And you look through it, and yeah, that's exactly what people needed. "Ha, look at those fools making a product that's super complex accessible to people. Ha." And it was kind of amazing going through that evolution. I mean, to be honest, the best sales pitch for Datadog for a long time was trying to use one of their competitors.
Pete: Yeah, absolutely. I mean, I was a very early Datadog user a very long time ago, and I put the agent ... And this isn't even an advertisement for Datadog. I was just a happy user. What I always said is their time to value was incredible. You could deploy their agent, click a couple of boxes, and it was like, hey, here's some dashboards. And as a monitoring snob as I often am, I was one of those people that said, "Oh, it's too simplistic, it's too basic. I don't have any of the flexibility I need, compared to other solutions in the market." But again, the people coming into the space, they're so new that they actually don't know what they need, and oftentimes, I just want to see a dashboard with a line on it, and I can point at it and say, "That looks weird." Not have to go to Harvard or MIT to get a doctoral student-
Corey: Right, and of course-
Pete: To help me monitor.
Corey: They're public. They can't have fun with it, but if I were doing some of their branding, it would be, Datadog. It's monitoring for people with real work to do. You don't have to mess with things like that, and I'm a big fan of making things more accessible to people no matter where they are, which in turn gets us to some of the announcements at re:Invent. What did you think of the various keynotes?
Pete: So I loved the keynotes. I mean, so I've actually been to every single re:Invent with one exception where I worked for a DNS company that didn't actually use any Amazon services. So seeing the evolution of the different re:Invents has been amazing. The very first one honestly was quaint by comparison. I think there were only 4,000 people at that first re:Invent. But the first few re:Invents, every single one was price drops and more price drops. The rate of new services was a lot slower and now, you can definitely see the rate of services is very high. The rate of new kind of hardware things that come out and the innovations that they're doing there are very impressive. But what I think what I saw, especially in the new product releases that came out, is just how pervasive ML and AI is within Amazon and just how it is touching so many of their services. I mean, I haven't done the actual count yet, but my guess is is that if you were to bucket together kind of announcements, ML-related announcements probably was the number one spot.
Corey: I think that you're probably spot on. To me, what really wound up striking me was the tone changed. It wasn't aimed at start ups at all at this point. At this point, it seemed like it was aimed much more at big E enterprise, where people who are just starting the transformation journey, it's like oh, you have an ancient piece of crap architecture. What do you do with that? About $8 billion of revenue a year. Why do you ask? And I think there's a lot of validity to that. There was a lot more Goldman Sachs, a lot less Netflix on the keynote floor.
Pete: Well, that is an extremely good point. Goldman Sachs I think had a booth at the event. I mean, just think about that for a minute. But you're absolutely right, which is, Amazon has a lot of customers. There's no doubt about that, and they say numbers like a million customers. But mostly, I always think that's like me and my 12 Amazon accounts for various one-off testing, so I don't know really how true it is. But-
Corey: Oh yeah. I wonder how they disambiguate that. It's like, well, we claim to have 20 million customers, at least going by the number of e-mails we send everyone.
Pete: Exactly, exactly. So, but I don't disagree. Obviously, there's a lot of people using it. But there's still a huge contingent of people that operate in the data center space, and they have hundreds of millions of dollars invested in existing data center spaces. And if you think about insurance companies and airlines and these huge enterprises that are, maybe they're playing around with cloud, and in talking to some enterprises recently, a lot of them have been using things like Lambda and Serverless, which has been a little bit of a surprise. But yeah, I think there is still a lot of market to be won in cloud for those, I don't even know if you should call them, ultra large enterprises or old stodgy data center enterprises.
Corey: Yeah, and it's easy to sit here and make fun of those companies, but there's a lot of business there. That's a trillion dollar industry or two, and rounding errors, and it really throws into stark relief how the big cloud providers have been fighting over a very small slice of the pie when it comes to interesting start ups, doing fascinating and far future things. But heck, I work on AWS bills for a living, and something I see is that the majority of spend is on EC2. It's fascinating.
Pete: It's something where if you think about the large enterprises, and this was actually a conversation that I had with multiple people this year at re:Invent which is, I had asked a friend of mine who was at a large retailer, and they, data centers, the whole deal. I mean, everything super-antiquated. And they were very aggressively using Serverless on public cloud. And I had said, I was like, "Wow, that's really innovative, and you think you would hear smaller companies adopting Serverless more than larger companies." And they just said to me, "Listen, finding software engineers is a lot easier for us than finding people with operational expertise. Those are a lot harder to find. And we can also hire software engineers from a variety of sources that normally you wouldn't be able to kind of find those ops folks. We can hire people from code boot camps and things of that nature, and so, hiring people out of college."
And they're talking to me about how they're replacing a mainframe that does all of their ERP stuff with a series of functions, and they just don't have the operational expertise and I think they're now realizing, especially those big companies, they've realized that they need to move to the cloud and now they're just trying to figure out, what are the right workloads and what are the things that they have or don't have that they need to ask for and all that fun stuff. But it's pretty amazing, and I really wonder too especially with the Amazon Outposts stuff that you can actually now get, if again that's another classic, get this technology into the hands of mega enterprises.
Corey: And that's what's fascinating to me. Before we get into Outposts, you know what else was awesome?
Pete: What was awesome?
Corey: CHAOSSEARCH. That's right. This episode is sponsored continuously by CHAOSSEARCH. Fundamentally what they do is they take an Elasticsearch compatible API and slap it on your data. The data itself lives in S3, queries take place through containers, and even after their licensing charge, it's still something like 80% less expensive for most workloads than running your own Elasticsearch cluster on top of it. The additional benefit of course is that you don't have to muck around with managing Elasticsearch. And increasingly, you're not going to be threatened to be sued by Elastic the company who is apparently pivoting business models in Oracle's footsteps and becoming a law firm.
Pete: It's like Lambda except it actually works and you can search upon it.
Corey: And it isn't defined by its constraints.
Pete: The other part too is you can go try it out now. I think that was one of the greatest features that they had been working on over the last few months was go sign up right on the web site and you can get into CHAOSSEARCH, you can point it at your S3 bucket and start indexing some data and run some queries and can get equally as mind-blown as I was when I saw that product for the first time.
Corey: Yeah, it's absolutely worth paying attention to. They were generous enough to provide a space to pass out my stickers and help boost my ego still further on the re:Invent Expo floor. So I do feel a certain sense of kinship with them. And they feel a certain sense of, for God's sake, get our name out of your mouth. Do you have any idea how disruptive you can be? And I figure that's a good mutual relationship, so my thanks to CHAOSSEARCH for their continuing and baffling support of my nonsense. So we're talking about Outposts, and I think those are fascinating because it's, how do you make cloud accessible to people who are scared of the cloud and possibly their own shadow?
Pete: Yeah, I remember last year, right, they announced it, and it was ... I don't even know if they announced it as preview or what the specific announcement was, but they just kind of said, "Hey, we're doing this Outposts thing." And I talked to a friend of mine who works there, and actually works on the Outposts team, and I just said to him, "What's going on? And where's this going? Can I buy it? Can I get an S3 in my data center? Can I get a this, can I get a that?" I think that was one of the things that I heard from the keynote is how S3 is hundreds of microservices and so it's like, "Can I get an S3 in my data center?" And he gave me the best, most Amazon response of, "Yeah, you can have whatever you want. Whatever you want." Now it is generally available, right? And you can apparently go to the portal, go to your Amazon management console, and spec out in Outposts.
What I did notice, and I'm not sure what kind of you saw with it that really caught your eye, but what I did notice is that it is fully managed. It's not just, here's a rack of servers with some software on it. It is a fully managed service, so I assume that means is that Amazon engineers are managing these remotely disparate Outposts kind of wherever they're at.
Corey: And there are workloads that are not suited for running in the cloud due to a variety of different constraints, so that does become an interesting option. What I like is the pricing is not completely out of reach. The development racks start at seven grand a month on a three year commitment. I'm still not going to put one in my living room, yet, but it at least, the option is there if I need it, mostly because I want to watch the look of befuddlement on the AWS team that comes out to do the installation. And yes, it does come with one.
Pete: Would you cook them dinner when they came over?
Corey: I don't know if it works in the opposite direction, but personally, I tend not to eat at AWS events just due to the risk of poisoning. For example, in my hotel room, it was super nice of them, they left me a live wolverine as a welcome gift.
Pete: I would've assumed it would've been a snake or something quieter that could sneak up on you.
Corey: No, no, I'm not a subtle creature, so they didn't give me a subtle creature in return.
Pete: But you survived. You survived a re:Invent all, I don't even know, six days of it?
Corey: Ugh, but it felt about three times longer, at least.
Pete: Yeah, I think we both actually had the misfortune of booking our travel such that we had days on either side, just to account for complexity of travel and other things, and so I was there pretty much all day Sunday through all day Friday. And there were things going on. I mean, it's truly a, let's say Monday through Friday with hours on Sunday. I mean, it's a truly, a five plus day marathon of cloud.
Corey: I love our annual tradition where we grab breakfast on the last day, and we just sit there and barely talk, because at that point, we're socialed out.
Pete: I actually, I feel like this year we both went really all in with meetings and workshops and whatever, all the different activities we were doing, kind of left it all on the field, because when we sat down at that breakfast and you came over and just had this look where you were full of energy, and I've seen you on stage and do that thing. But you had this look, and then you looked at me and I had this look, and we just almost were like, "What if we didn't say any words, and we just sat here quietly?"
Corey: And it would be beautiful. Unfortunately, it turns out you cannot give a session track talk that way. I tried.
Pete: I saw your session talk. That is actually a great segue into that, which was about responsible disclosure. I really loved that talk.
Corey: Yeah, that was fun. I wound up giving the talk twice. It went vastly better the second time, because my co-speaker and I hadn't the chance to rehearse in person together, but, and also, as an added bonus, the second time, the cameraman didn't fall off the podium midway through the session and have to be attended to. We're going to take a five minute break before we go back to making lighthearted jokes about cloud. Can someone make sure he's still alive please? Yeah, it turns out that it's super hard to recover tonally from something like that.
Pete: Did he fall over from laughing? I mean, I assume that's what happened.
Corey: I assume he fell asleep, because we were talking about ISO standards at that point.
Pete: Well, what I thought was most interesting was a couple of things. So I had the opportunity to get over to the Aria. I had the opportunity to get to the Mirage, which was where your redo was. I never got down to the MGM, but I think this was really the first year that the scope of how many locations had conference talks. And it wasn't like a side room at the Mirage. I was deep into the bowels of the Mirage conference center, and it was, I think the first time I'd really seen this, again I've gone to conference talks at re:Invent where it's a room and there's someone on stage and everyone is in the same room for the same talk. And the room at the Mirage you were in, there were three talks going on at the same time and we all had headphones on. Now if you were in the earlier rows, or the closer rows, you could probably just hear fine. I was a few rows back.
But I actually wanted to ask you, as a speaker, did you find it challenging, because obviously you could hear the other speakers, maybe that was popping out. But also, did you find it challenging because I actually found myself not laughing like I should have because I had headphones on. It was a weird dynamic.
Corey: That's absolutely a real challenge. I've had the benefit of having given a few of those "silent disco" style talks before, and people are not generally willing to laugh when wearing headphones unless they're sitting on a city bus and not wearing headphones. So it's challenging to wind up getting the same crowd reaction. You can power through, but it has to be a little bit over the top. It also adds a certain sense of intimacy, because now instead of me speaking to a booming projecting microphone, I'm speaking and it is whispering sweet nothings directly into your ear, just like I'm doing right now on a podcast. Shh, go to sleep, go to sleep, watch out, there's a bridge. Yeah, it's fun.
Pete: Yeah, I was very surprised to see that kind of setup in there, and I think there were actually, that room that we were in, was actually six specific talks going on. And they were sizeable. I mean, they were rooms that held a couple hundred per talk. I mean, they're cavernous rooms.
Corey: And that's sort of a challenge too. When you have 200 people attend a talk in a room that seats 600, it feels sparsely attended. Whereas if you give a talk and there are 50 people in a room that comfortably seats 20, it feels like it's a way more successful talk based upon audience size. It really changes the dynamics of the room. The worst talk I ever gave was at a puppet conference, years ago. And it was the last talk of the conference, they'd opened that free bar 20 minutes before the talk. It was a room that would've seated easily 400 or 500 people. 12 people showed up, no two sat next to each other, and it was at that point, hell with it. Let's just have a fireside chat. It was not a very well thought through process.
Pete: That actually reminds me, not re:Invent related, but I love sharing this story about room sizes for conference talks. I understand how hard it is to put on a conference and, knowing the room sizes in advance and who actually wants to go to which talk, that is a challenge I can't even begin to imagine. But a friend of mine side, "I hope you'll come and speak at my conference. I loved such and such talk that you gave." I said, "Great." And at the same time, they had actually someone drop out of the conference, and just said, "Hey, is there anyone that could help?" I directed them to a friend of mine who actually had a lot of research around the DDoS that happened at Dyn, the DNS provider, a few years ago. And I said, "Oh, you should talk to this person," who ended up talking at the conference.
Of course, as fate has it, they scheduled their talk directly against my talk, which is fine. I mean, two different types of things. But also due to a snafu, the room that I was in easily held 2,000 people. And because my talk was against something that was admittedly a lot more interesting, there were maybe 12 people that showed up in the 2,000 person room. But what was great is how sparse everyone decided to sit, and there were people sitting in the back that I just was like, wave to me if any of this is making sense. And I still give my friend a hard time about that. He feels terrible, but it was still hilarious.
Corey: And that's sort of the entire point of giving conference talks is to get out there, try new things, see how it resonates. Often I ... What always disturbs people at the more, shall we say, thoroughly produced conferences like re:Invent is I often have very little on my slide. Now I improv most of what I say, which is good, because most of what I say would never pass slide review of, "You can't actually say that in front of other human beings on our stage." Yeah, but I don't know I'm going to say it until I'm up there and it pops into my head and I go for it. And it either lands super well, or it goes over like a lead balloon.
Pete: I totally agree. I think if, especially too, I like giving talks multiple times just because you think of things on the fly. Although I practice in advance with talk notes because if I think of something, it pops in my head, I start talking about it, and then I'm like, "Wait, where was I at? I need, where." And then the notes kind of bring me back a little bit. But you're right, if you need your slides to be approved by the approaching Oracle-sized legal team at Amazon ... I mean, just kidding. Nothing can approach Oracle's legal team size, but you have to have real legal people review it for things, just keep it light. No one really wants to read slides anyway. They really are there to hear you talk.
Corey: That's something I always wanted to do was try and give a talk with no slides. I think that's called improv, but I'll have to do it at some point, just to see what happens. To some extent, that's kind of what a podcast is.
Pete: Absolutely. I think everyone listening to this can just imagine the slides we would create for this talk.
Corey: So other releases of re:Invent that were neat. They had a crap ton of announcements around Redshift, which is awesome, except for the part where Redshift runs on burning piles of other people's money. So the price just to get started is not nothing. So I need to start looking at some larger level data analytic shops and start seeing what they're doing with Redshift. It does change things a bit. You can now query Redshift and hit a whole bunch of other data stores with federated queries. It winds up having a whole bunch of export options. You can, again, have data live in S3 rather than live online, CHAOSSEARCH. That tends to be the direction that the world is going in. But there really is something for everyone. What announcement do you think made the least waves when it should have made more?
Pete: So, made the least wave. That's actually a really good question.
Corey: Yeah, what do you think is the most underrated release so far?
Pete: Well, I think it's underrated just because no one knows what to do with it, is really that DeepComposer release, which is, I think there is something there that will ... This will end up powering, right, that I don't think anyone realizes yet. And so I think in many ways, people saw this as a, it's a MIDI keyboard and a very early UI where you can play eight bars of music. At least, that was, I did the workshop for it, and it maxed out at eight bars. So that's not a lot of music that you can create. But they talked about this generative, adversarial networks. They're called GANs, and I am not an ML person. I find this stuff to be super fascinating though. And they really talked about how you can train these GANs against different musical types.
Where I think this is actually super interesting is that I've actually recently, in the last couple of years, started to learn the piano. I wanted a hobby that wasn't computers, and mostly my daughter gave up the piano and I had a piano in my house. So since I had to look at it every day, I felt like-
Corey: They tend to accumulate, don't they?
Pete: They really do. And so I'm looking at a piano every day thinking, "I'll go and learn this thing. For people that want to make music, the accompaniment around it is challenging. You don't have an orchestra or whatever and maybe you don't have all the other additional equipment, but what if you could have some ML that you can train based on a certain style of music? The demo they showed, I think, was super impressive but only to classical music nerds is, they had a training of countless Bach recordings that are all public domain, and you could play some music, and because there was so much source data that they trained, you could listen to the results at all of the epoch at each time it improved it, and you could hear, let's say, Twinkle Twinkle Little Star or something at five epochs, and then at a thousand, and at two thousand.
And at the end of it, which I'm not sure how many people actually made it that far in the workshop, just from a time constraint, at the end of it, it was mindblowing. It was like you were listening to Bach play this individual song. So I think, again, I don't know what people are going to build with this, but the technology underneath it, these generative adversarial networks, that technique of which there are papers recently published on, I think could be something that we don't even know yet. It's just so far out in the future.
Corey: I think there's a lot of neat things happening there. I guess what I found interesting was a bit more prosaic on some level. You see the ... There are a lot of things that came out, but one of the notable things in the world of hardware was the Graviton2 processor. It's been a banner year for chip manufacturers. I mean, AMD has launched their whole second generation of Epyc, AWS launched their second generation custom ARM chip called Graviton2, and Intel threw an awesome party at Replay.
Pete: So I totally agree. I actually have that tab open on my laptop because I also want to talk about that. I'm not a hardware nerd. I got started in technology when it was all hardware. I worked in a data center for a hosting company that was-
Corey: The magic smoke gets out and you have to buy a new one, that gets expensive. Software, you can get just reset your git repo.
Pete: Exactly, right? And I thought I remembered reading around the size of the, I don't even know the words, but of the circuits or the size of the transistors I suppose. I don't ... I'm sounding like an idiot because I just don't know anything around hardware, but I thought there was a lower limit that we were approaching and something they called out was this seven nanometer manufacturing technology. So it appears as though Amazon, in addition to creating everything from hardcore software services and MIDI players for music, also seem to be moving into this working with chip providers to improve how they make their hardware. It's pretty crazy, the scope they're in right now.
Corey: I think that Intel's also falling massively behind. I think that their public roadmap, they weren't able to meet, has pissed off pretty much everyone. I mean, you see Apple laptops being hamstrung. I'd be shocked if we didn't seen an ARM MacBook in the next couple of years, just based upon needing to control their own destiny. At this point, it's ... Apple laptops are not cheap, and the fact that you're getting a processor that a couple years old at this point is not super awesome. And I understand this stuff is hard. You don't do agile chip development in most cases. But at the same time, it seems that this game was entirely Intel's, and now they're really alienated most of their distribution channel based upon that. And I had AMD down as all but dead, and look at them now.
Pete: I think it's amazing to see. It's a good place to be where we have diversity in chip manufacturers. You don't want it to be just Intel everywhere. But also too, something that has happened in these recent years is security issues around specifically Intel chips, in design choices that they have made in this race for faster and faster. And my mind was appropriately blown away when I watched at DevopsDays Chicago, Jess Frazelle talk about the kind of underlying bits of hardware. You think about Ring ZERO of an operating system, but she was talking about Ring -1 and Ring -2, where you're just talking about everything that happens before your OS starts up. It was fascinating, and also frightening how complex that code is that no one really understands. And so-
Corey: Yeah, how do computers work? Nobody really knows.
Pete: It's truly amazing. And so I think it's great to see AMD and these other chip creators coming out and building new and innovative technologies as they are.
Corey: One of the things that I also think was super neat that isn't getting nearly enough attention, and it certainly deserves more, is S3 access points. Part of this is due to Amazon's inability to tell an articulate story on a keynote stage. They talk about that almost entirely like it's aimed at data lakes. Here's the trouble with that approach. No one thinks of themselves as having a data lake, even if they have near exabyte scale of data living in S3. Secondarily, S3 access points give you very granular access control to the same S3 bucket. So you can provide different end points and different applications that are scoped appropriately. That doesn't require you to have hundreds of petabytes of data. You can do that with three megs or less. It's not a big data feature. It's an access control feature, and by talking about it in a data lake context, I'm worrying that's sailing past people who could really benefit from it.
Pete: Yeah, data lake is such a polarizing term that I wish we could come up with something better for. I actually gave a talk where I didn't want to use the term data lake in talking about kind of the future of monitoring, so I called it a data bagel, and mostly just because I feel like if you can create a dumb term for something that gets people to stop and be like, "Well, what does that mean?", versus a data lake where people are like, "Oh, I know what that is," right? The data lake concept is, it's one of those things I think you and I spoke about this earlier, was, no one has said, "I want a data lake." I mean, there's probably some suit-and-jacket person saying, "Oh, we need a data lake strategy in our company."
But I think like you said just a minute ago, there are companies out there pushing petabytes and exobytes of data into places like S3, they wouldn't not call that a data lake. Even thought they have a multitude of services pulling data from there and doing different things with it, they still wouldn't call it a data lake. So I think by branding it as such, I totally agree, it's going to ... People are going to think that I'm not big enough or they just don't like the term. It could be a little bit weird.
Corey: Something else that I thought resonated with me at least was the computer optimizer. I don't think that people are going to use it for much. It tells you, "Oh, with this instance family, this is the performance profile you'd see." It will help people figure out what they should be doing, but having spent three years in this space, even if you tell people exactly what they should be running instead, very often they won't move due to inertia, due to fear, due to the fact that it's not easy to move some workloads to a different size or family of instance. And I get it. I think it's a great tool. I don't for the life of me understand why it's a top level console feature rather than buried within Cost Explorer, but we'll see.
Pete: Yeah, I'm going to be intrigued to see how many people actually use that technology. As an operator for so many years, I've wanted to resize, and I had done, this was my time at Threat Stack. We ran a cloud security platform, a lot of data. Elasticsearch was one of the many databases we used there. We continually optimized the size of it, moving from I2 instances to R4 with EVS. The NVMe instances came out and we moved it to there. But moving data is expensive, and you know this probably better than anyone about moving data within Amazon is extremely expensive, just the network transfer.
So what ... You might save money by moving to a different instance type, but the ROI could take some time if you have to transfer hundreds of terabytes of data over the network, or for people who really play in the big data world, you might have a petabyte or a petabit of network transfer. That's real money in Amazon. So you can go through all this work and the people work and the testing and whatever else. You could have a 12-month ROI on an instance type change, depending on what you're building.
Corey: Yeah, and oh, the data transfer as well. How much does your database speak to your application server? Most people don't know that, but they're about to know that in a big way.
Pete: Yeah, right about the time you start line-iteming your Amazon bill and said, "Wow, our data transfer last month was $60,000. Does anyone know why?" And I don't even know if Amazon knows why, other than they might say, "Oh, you must run Cassandra," which is a common joke within-
Corey: Oh yeah, which they now have a service for. What I found was fun was that when I was asking people at Amazon internally for the pricing of Cross-AZ data transfer, no one would answer me. And I thought they were stonewalling and trying to be insulting. It turns out, no one actually knows. So once I ran some experiments with DD and Netcat, suddenly they update their documentation almost instantly to explain what happened. It wasn't that they were stonewalling me, it's that no one knew.
Pete: I remember when you started talking about on that Twitter thread and we did the math on it, that again, I don't recommend that you build an application this way, but if you wanted cross-region availability, it actually makes more sense, and again, you don't have latency requirements of Cross-AZ. It makes more sense if let's say you were building an Elasticsearch cluster or a Cassandra cluster to basically put it in one AZ and us-east-1 and one AZ and us-east-2, and that-
Corey: CHAOSSEARCH gets around that. However ...
Pete: And that appears to be half the price of Cross-AZ traffic.
Corey: It is half the price. That's why it appears that way. And I would challenge your assertion that you wouldn't recommend building an application that way. If you could build it that way from day one, you're already designed for multi-region, and that is potentially a far more durable architecture if it needs it. And not every application does. I mean, a lot of the business stuff that I do can sustain an AZ being down for a day or two because it's not in line with what customers need. I mean, worst case, some of my AWS newsletter creation stuff lives in a single region. And if that region, us-west-2, is unavailable for a few days, well, not for nothing, I have a much more interesting story to write about freeform in the newsletter that week.
Pete: You've got to know what you're optimizing for, and you're right, which is if you can say you're multi region, you're actually probably beating out a lot of companies in the space where they are still stuck in single AZ. Because let's be honest, cross-region is hard. It's not an easy challenge to get there, and it takes time.
Corey: Oh my stars, yes. What else did you see at the event that was notable for you?
Pete: Well, there wasn't as many people as I expected. And there were more than last year-
Corey: Yeah, only 60,000 or so.
Pete: Only 60,000, which I believe was still more than last year. I think last year, I don't know if you remember-
Corey: You're right though. Most of my clients did not have a presence there this year.
Pete: And I wonder, there's a lot more local events with the summits. I've been to a few. They're all kind of mini re:Invents, single day. They're honestly, I think they're really great if there's one near you. They're free too, which is amazing. You can attend these local events for free. And also, it looks like they're splitting off some other events as well with re:Inforce, their security side. I feel like they had an event that was specific to AI and ML. But I also wondered too if that there wasn't a 50% growth or anything, but they still kind of took over the city with talks at so many locations. Again, I have no data points to support it, but it does feel like talks were spread around much more. And as at least I navigated around to different locations and places, I didn't find it to be as cramped and anxiety-inducing of so many people as it had been in previous years.
Corey: One other thing I found super interesting was the higher level analyzers, the abstractions built on top of abstractions. We see this with the IAM Access Analyzer, the S3 policy analyzer. AWS, I'm sorry, Amazon Detective, which is a security roll up above things like GuardDuty, above Security Hub and the rest. I think that that is super interesting in that it's distilling it down further and further to something actionable. Because how many security breach stories have we heard about where, oh, they installed the alarms and the alarms were false positiving all the time, so people started ignoring them, and then they missed the actual important thing. It feels like chatty monitoring systems are only there to be able to blame people with once something breaks.
Pete: I think any business that gets involved in helping to separate the signal from noise is going to have a really big success. I don't want a thousand alerts per day of security stuff. Even if I'm getting attacked a thousand times, can you distill them down to let me know what's the three most actionable things I need to do? Because at the end of the day, security people, ops people, we're all so busy, we just don't have the time, and that's where mistakes happen. So if anything can just say, "Hey, that thing right there? Go for it," that's going to be-
Corey: Right. Random IP address on the internet, port scanning me, that's one tier of problem. My firewall node is now enumerating S3 buckets is something else entirely.
Pete: Exactly. And so, that ... and if you're at a level of scale where you've got trillions of these events happening, how do you find ... Again, it's the needle in the haystack, it's the unknown. I don't even actually know what I'm looking for, but if something strange is happening of a high priority, please let me know about that one, and maybe not so much let me know about the other stuff.
Corey: One other thing that was announced in the keynote, or at least not announced but pretty obvious, and I think it's unfortunate. I think Amazon struggles across the board with empathy, and it shows, because yet again, I'm sad to report that the re:Invent house band was not put to sleep. They are clearly suffering. They are not happy. Please, put them out of this world.
Pete: I unfortunately missed the midnight madness, but I only missed it-
Corey: Oh, this was Andy Jassy's keynote, and then Andy Jassy recited the lyrics. I want an Alexa skill that is just Andy Jassy reciting song lyrics to me. I think that would be phenomenal.
Pete: Yeah, he could do it onstage as part of his keynote for the version two of DeepComposer where he just says the words, and DeepComposer is composing the music behind it.
Corey: Oh yes, I think the “Andy Jassy Sings” album would be phenomenal at holiday time.
Pete: That's a double platinum if there ever was one.
Corey: Absolutely. Anything else you noticed that was fun, exciting, worth mentioning?
Pete: Honestly, this was a weird event for me. I was helping CHAOSSEARCH with the re:Invent-
Corey: CHAOSSEARCH.
Pete: With the re:Invent event, and helping them make sure ... We did a lot of planning for it, so I wanted to make sure that they got through it and were all set. And I really spent a lot of time going to workshops. I mean, I always think about, what's the best way to do re:Invent? If you're getting certified, that's a great opportunity to go, do a boot camp and get through the certification. The workshops, they don't redo those. All the conference talks are online after the fact, so spending time trekking to different talks I find to be a lesser activity for me at least, just because I can watch them later. But I really try to optimize and spend a lot of time talking with folks, talking with friends of mine who are in town, talking with even venture folks, and just try to hear and listen, what are people building and what challenges there are.
And I think the thing that I noticed, and the thing that I keep thinking about that really resonates is that kind of sys admin and dev ops expertise, this is an expertise that is going away. I mean, more and more people are abstracting away what we used to deal with earlier in our careers. But it definitely seems to be happening more and more, and the folks that are still around doing that type of work are going to become more and more valuable, but also going to become busier and busier. So always trying to chat with people and see what are the pain points and challenges.
And this is where these public clouds can really have a big impact in helping take over something that is just not part of my business, and I just don't want to deal with. I'm not going to run Postgres unless running Postgres is part of my business. I'm just going to use RDS. And that's the obviously very simplistic way of thinking of it, but as you see all these new services come out, they're really trying to abstract more and more, even abstracting the abstractions they've created from previous products.
Corey: That is fascinating in that they're now building services that in turn roll up other services in the context a human can understand without a deep subject matter expertise on something. And in large scale environments, that's huge. I am seeing a lot more of what they're releasing that is clearly built for scale. And that's a term that gets thrown around way too lightly. But take the simple stuff. You look around on GitHub. There's an awful lot of open source tooling that does all kinds of things, cost analysis, tagging of things, enumeration of various systems. And they all assume that you're running in a small scale environment. Well, when you have 15,000 nodes in a single account in a single region, and you try and do anything to all of them at once, you hit massive API rate limits. And that is something that a lot of tooling naively does not take into consideration, because why would you? If you're building something to work on your 200 instances, you're never going to hit those rate limits that you will at 10 or 20 times that scale.
Pete: Absolutely, and I think it goes back to what you said, which is, they've been messaging to the enterprises more recently, and I actually wonder if some of these enterprises that have adopted it, let's say three years ago when they really started making a push, are now reaching scale points. Or maybe it's the enterprises that are left are like, oh, they can't handle my scale, and this is them trying to show no, actually, we really can, and let us show you how that's even possible.
Corey: That's something that I think is overlooked on the AWS side. Whenever you build something in the world of AWS, there is no way around it. They can't order freaking pizza without having a massive logistics challenge just based upon the scale that they operate at. Part of me worries that my sarcasm and my snark get overlooked or taken too seriously in that they're focusing on all the things that AWS does that I find annoying or humorous, rather than the very hard work that some very intelligent and very dedicated people do. I worry that people at AWS who build these services read my snark or sarcasm and then walk away feeling crappy as a result. That's never been how I want to come across, and if you're listening to this and you're one of those people, first, I'm sorry. Secondly, please reach out. I'd love to hear more of the stories about what it takes to build these services, because I certainly don't know. All I know is what it takes to make stupid jokes that make people laugh. It's a bit of a different scale of a problem.
Pete: I'm the same way. I've worked in a lot of start ups, and scale is relative. I mean, there are some places I've worked where it's large scale, but in comparison to who, is really the question. There's a fascinating world within Amazon, just how they even construct these services, from the product management side to the engineering side and everything else. I have to imagine that some of these services that they create go from zero to 100 miles an hour once the APIs are around, and so it's really true interesting scale. But I do like to say that when it comes to scale, and I remember the talk a couple re:Invents ago, where they basically talked about their power generation facilities that they were designing. And that's where I was like, you know what? That's when you know you've made it, is when you are designing more efficient power generation facilities in order to actually power your data centers. That's truly next level scale.
Corey: That is, I think, the most interesting and useful thing to remember. Everything that AWS releases is important to someone. None of it is important to everyone.
Pete: Absolutely, absolutely. I mean, you don't have to use everything, and I think your bank account would groan if you really did. But it's the-
Corey: That's what other people's accounts are for.
Pete: It's the optionality, which is what makes it great. And obviously they're setting the tone, but these other cloud vendors are coming up behind with different ways and different technologies as well, and I think it's the golden era of computing. I mean, we've got some amazing things. You can start a company for almost nothing and get access to a level of scale and growth that was just unheard of five years ago, 10 years ago. It's really shocking. And I've talked to a lot of start ups, and I'm doing a lot of start up advisory and consulting right now.
And the thing that has been repeated most often is how many of these companies are just consuming either Amazon or other cloud providers' services as a first level citizen, right? They're not deploying an EC2 instance to run their app. They are going with Fargate. They're going with Serverless only. They're using the hosted database services. They're using the suite of tools, because they realize that that's not their business model, and they don't want to deal with it right now. And that's amazing. It's allowing these companies to go from zero to a real product and focus on their product and get it out to market a lot sooner. It's truly remarkable to watch.
Corey: It really has, and I think that what's the most impressive of all of it to me, I guess if I had to sum it up, was that Andy Jassy has to walk a razor's edge in these keynotes, because he's got to tell stories to a very diverse audience with all kinds of challenges, and in doing so, speak to everyone while putting no one off. And that is incredibly difficult as far as needle threading goes. And if the worst commentary I have on that is, "Well, I thought the re:Invent house band was a little corny," then I don't think we have much to complain about.
Pete: No, we really don't. I mean, when we think about things to complain about at re:Invent, it usually comes to some of the things they're doing, not the stuff that they're announcing. It's like the house band or the food or something, something absolutely silly like that, which I always laugh at as well because my wife works in education. She goes to conferences too. Her conferences usually cost $50 or $60 to go to, there's no sponsors, and it's a box lunch. And she's like, "Well, what about your conference?" And I'm like, "I think Foo Fighters is playing at it? I'm not sure." We're in a far different world than pretty much everyone else.
Corey: I think that's probably the best place to leave it at this point. Pete, thank you so much for taking the time to speak with me on this podcast today. If people want to learn more, where can they find you?
Pete: So I actually updated by blog for the first time in many, many years. It's pete.wtf. I feel like it's the best use of the WTF vanity domain. And I'm actually trying to blog a little bit more. There's a couple of posts on there about stock options and things if anyone has any interest. But I am on the Twitters, petecheslock on Twitter. And that's where I post pictures of various meats that I'm smoking and veggies that I'm smoking and cooking food, and hot takes on technology when the time calls for it.
Corey: Excellent. Well thank you for your time, and thank you to CHAOSSEARCH for sponsoring this ridiculous podcast.
Pete: Thank you a ton for having me back again. This is two for two, right? So next year-
Corey: And we'll do the third one next year, assuming you still go to re:Invent.
Pete: Absolutely. I'm going to put it on my calendar right now.
Corey: Excellent. Pete Cheslock, unaffiliated start up advisor. I am Corey Quinn, cloud economist, and this is the Screaming in the Cloud re:Invent recap. Thank you all for listening.
Announcer: This has been this week's episode of Screaming in the Cloud. You can also find more Corey at screaminginthecloud.com, or wherever fine snark is sold.
This has been a HumblePod production.
