Storytelling Over Feature Dumping with Jeff Geerling

Jeff is a father, author, developer, and maker. He is sometimes called "an inflammatory enigma".

Announcer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.

Corey: Welcome to Screaming in the Cloud. I’m Corey Quinn. A bit off the beaten path of the usual cloud-focused content on this show, today I’m speaking with Jeff Geerling, YouTuber, author, content creator, enigma, and oh, so much more. Jeff, thanks for joining me.

Jeff: Thanks for having me, Corey.

Corey: So, it’s hard to figure out where you start versus where you stop, but I do know that as I’ve been exploring a lot of building up my own home lab stuff, suddenly you are right at the top of every Google search that I wind up conducting. I was building my own Kubernete on top of a Turing Pi 2, and sure enough, your teardown was the first thing that I found that, to be direct, was well-documented, and made it understandable. And that’s not the first time this year that that’s happened to me. What do you do exactly?

Jeff: I mean, I do everything. And I started off doing web design and then I figured that design is very, I don’t know, once it started transitioning to everything being JavaScript, that was not my cup of tea. So, I got into back-end work, databases, and then I realized to make that stuff work well, you got to know the infrastructure. So, I got into that stuff. And then I realized, like, my home lab is a great place to experiment on this, so I got into Raspberry Pis, low-power computing efficiency, building your own home lab, all that kind of stuff.

So, all along the way, with everything I do, I always, like, document everything like crazy. That’s something my dad taught me. He’s an engineer in radio. And he actually hired me for my first job, he had me write an IT operations manual for the Radio Group in St. Louis. And from that point forward, that’s—I always start with documentation. So, I think that was probably what really triggered that whole series. It happens to me too; I search for something, I find my old articles or my own old projects on GitHub or blog posts because I just put everything out there.

Corey: I was about to ask, years ago, I was advised by Scott Hanselman to—the third time I find myself explaining something, write a blog post about it because it’s easier to refer people back to that thing than it is for me to try and reconstruct it on the fly, and I’ll drop things here and there. And the trick is, of course, making sure it doesn't sound dismissive and like, “Oh, I wrote a thing. Go read.” Instead of having a conversation with people. But as a result, I’ll be Googling how to do things from time to time and come up with my own content as a result.

It’s at least a half-step up from looking at forums and the rest, where I realized halfway through that I was the one asking the question. Like, “Oh, well, at least this is useful for someone.” And I, for better or worse, at least have a pattern of going back and answering how I solved a thing after I get there, just because otherwise, it’s someone asked the question ten years ago and never returns, like, how did you solve it? What did you do? It’s good to close that loop.

Jeff: Yeah, and I think over 50% of what I do, I’ve done before. When you’re setting up a Kubernetes cluster, there’s certain parts of it that you’re going to do every time. So, whatever’s not automated or the tricky bits, I always document those things. Anything that is not in the readme, is not in the first few steps, because that will help me and will help others. I think that sometimes that’s the best success I’ve found on YouTube is also just sharing an experience.

And I think that’s what separates some of the content that really drives growth on a YouTube channel or whatever, or for an organization doing it because you bring the experience, like, I’m a new person to this Home Assistant, for instance, which I use to automate things at my house. I had problems with it and I just shared those problems in my video, and that video has, you know, hundreds of thousands of views. Whereas these other people who know way more than I could ever know about Home Assistant, they’re pulling in fewer views because they just get into a tutorial and don’t have that perspective of a beginner or somebody that runs into an issue and how do you solve that issue.

So, like I said, I mean, I just always share that stuff. Every time that I have an issue with anything technological, I put it on GitHub somewhere. And then eventually, if it’s something that I can really formulate into an outline of what I did, I put a blog post up on my blog. I still, even though I write I don’t know how many words per week that goes into my YouTube videos or into my books or anything, I still write two or three blog posts a week that are often pretty heavy into technical detail.

Corey: One of the challenges I’ve always had is figuring out who exactly I’m storytelling for when I’m putting something out there. Because there’s a plethora, at least in cloud, of beginner content of, here’s how to think about cloud, here’s what the service does, here’s why you should use it et cetera, et cetera. And that’s all well and good, but often the things that I’m focusing on presuppose a certain baseline level of knowledge that you should have going into this. If you’re trying to figure out the best way to get some service configured, I probably shouldn’t have to spend the first half of the article talking about what AWS is, as a for instance. And I think that inherently limits the size of the potential audience that would be interested in the content, but it’s also the kind of stuff that I wish was out there.

Jeff: Yeah. There’s two sides to that, too. One is, you can make content that appeals to anybody, even if they have no clue what you’re talking about, or you can make content that appeals to the narrow audience that knows the base level of understanding you need. So, a lot of times with—especially on my YouTube channel, I’ll put things in that is just irrelevant to 99% of the population, but I get so many comments, like, “I have no clue what you said or what you’re doing, but this looks really cool.” Like, “This is fun or interesting.” Just because, again, it’s bringing that story into it.

Because really, I think on a base level, a lot of programmers especially don’t understand—and infrastructure engineers are off the deep end on this—they don’t understand the interpersonal nature of what makes something good or not, what makes something relatable. And trying to bring that into technical documentation a lot of times is what differentiates a project. So, one of the products I love and use and recommend everywhere and have a book on—a best-selling book—is Ansible. And one of the things that brought me into it and has brought so many people is the documentation started—it’s gotten a little bit more complex over the years—but it started out as, “Here’s some problems. Here’s how you solve them.”

Here’s, you know, things that we all run into, like how do you connect to 12 servers at the same time? How do you have groups of servers? Like, it showed you all these little examples. And then if you wanted to go deeper, there was more documentation linked out of that. But it was giving you real-world scenarios and doing it in a simple way. And it used some little easter eggs and fun things that made it more interesting, but I think that that’s missing from a lot of technical discussion and a lot of technical documentation out there is that playfulness, that human side, the get from Point A to Point B and here’s why and here’s how, but here’s a little interesting way to do it instead of just here’s how it’s done.

Corey: In that same era, I was one of the very early developers behind SaltStack, and I think one of the reasons that Ansible won in the market was that when you started looking into SaltStack, it got wrapped around its own axle talking about how it uses ZeroMQ for a full mesh between all of the systems there, as long—sorry [unintelligible 00:07:39] mesh network that all routes—not really a mesh network at all—it talks through a single controller that then talks to all of its subordinate nodes. Great. That’s awesome. How do I use this to install a web server, is the question that people had. And it was so in love with its own cleverness in some ways. Ansible was always much more approachable in that respect and I can’t understate just how valuable that was for someone who just wants to get the problem solved.

Jeff: Yeah. I also looked at something like NixOS. It’s kind of like the arch of distributions of—

Corey: You must be at least this smart to use it in some respects—

Jeff: Yeah, it’s—

Corey: —has been the every documentation I’ve had with that.

Jeff: [laugh]. There’s, like, this level of pride in what it does, that doesn’t get to ‘and it solves this problem.’ You can get there, but you have to work through the barrier of, like, we’re so much better, or—I don’t know what—it’s not that. Like, it’s just it doesn’t feel like, “You’re new to this and here’s how you can solve a problem today, right now.” It’s more like, “We have this golden architecture and we want you to come up to it.” And it’s like, well, but I’m not ready for that. I’m just this random developer trying to solve the problem.

Corey: Right. Like, they should have someone hanging out in their IRC channel and just watch for a week of who comes in and what questions do they have when they’re just getting started and address those. Oh, you want to wind up just building a Nix box EC2 for development? Great, here’s how you do that, and here’s how to think about your workflow as you go. Instead, I found that I had to piece it together from a bunch of different blog posts and the rest and each one supposed that I had different knowledge coming into it than the others. And I felt like I was getting tangled up very easily.

Jeff: Yeah, and I think it’s telling that a lot of people pick up new technology through blog posts and Substack and Medium and whatever [Tedium 00:09:19], all these different platforms because it’s somebody that’s solving a problem and relating that problem, and then you have the same problem. A lot of times in the documentation, they don’t take that approach. They’re more like, here’s all our features and here’s how to use each feature, but they don’t take a problem-based approach. And again, I’m harping on Ansible here with how good the documentation was, but it took that approach is you have a bunch of servers, you want to manage them, you want to install stuff on them, and all the examples flowed from that. And then you could get deeper into the direct documentation of how things worked.

As a polar opposite of that, in a community that I’m very much involved in still—well, not as much as I used to be—is Drupal. Their documentation was great for developers but not so great for beginners and that was always—it still is a difficulty in that community. And I think it’s a difficulty in many, especially open-source communities where you’re trying to build the community, get more people interested because that’s where the great stuff comes from. It doesn’t come from one corporation that controls it, it comes from the community of users who are passionate about it. And it’s also tough because for something like Drupal, it gets more complex over time and the complexity kind of kills off the initial ability to think, like, wow, this is a great little thing and I can get into it and start using it.

And a similar thing is happening with Ansible, I think. We were at when I got started, there were a couple hundred modules. Now there’s, like, 4000 modules, or I don’t know how many modules, and there’s all these collections, and there’s namespaces now, all these things that feel like Java overhead type things leaking into it. And that diminishes that ability for me to see, like, oh, this is my simple tool that solving these problems.

Corey: I think that that is a lost art in the storytelling side of even cloud marketing, where they’re so wrapped around how they do what they do that they forget, customers don’t care. Customers care very much about their problem that they’re trying to solve. If you have an answer for solving that problem, they’re very interested. Otherwise, they do not care. That seems to be a missing gap.

Jeff: I think, like, especially for AWS, Google, Azure cloud platforms, when they build their new services, sometimes you’re, like, “And that’s for who?” For some things, it’s so specialized, like, Snowmobile from Amazon, like, there’s only a couple customers on the planet in a given year that needs something like that. But it’s a cool story, so it’s great to put that into your presentation. But some other things, like, especially nowadays with AI, seems like everybody’s throwing tons of AI stuff—spaghetti—at the wall, seeing what will stick and then that’s how they’re doing it. But that really muddies up everything.

If you have a clear vision, like with Apple, they just had their presentation on the new iPhone and the new neural engine and stuff, they talk about, “We see your heart patterns and we tell you when your heart is having problems.” They don’t talk about their AI features or anything. I think that leading with that story and saying, like, here’s how we use this, here’s how customers can build off of it, those stories are the ones that are impactful and make people remember, like, oh Apple is the company that saves people’s lives by making watches that track their heart. People don’t think that about Google, even though they might have the same feature. Google says we have all these 75 sensors in our thing and we have this great platform and Android and all that. But they don’t lead with the story.

And that’s something where I think corporate Apple is better than some of the other organizations, no matter what the technology is. But I get that feeling a lot when I’m watching launches from Amazon and Google and all their big presentations. It seems like they’re tech-heavy and they’re driven by, like, “What could we do with this? What could you do with this new platform that we’re building,” but not, “And this is what we did with this other platform,” kind of building up through that route.

Corey: Something I’ve been meaning to ask someone who knows for a while, and you are very clearly one of those people, I spend a lot of time focusing on controlling cloud costs and I used to think that Managed NAT Gateways were very expensive. And then I saw the current going rates for Raspberries Pi. And that has been a whole new level of wild. I mean, you mentioned a few minutes ago that you use Home Assistant. I do too.

But I was contrasting the price between a late model, Raspberry Pi 4—late model; it’s three years old if this point of memory serves, maybe four—versus a used small form factor PC from HP, and the second was less expensive and far more capable. Yeah it drags a bit more power and it’s a little bit larger on the shelf, but it was basically no contest. What has been going on in that space?

Jeff: I think one of the big things is we’re at a generational improvement with those small form-factor little, like, tiny-size almost [nook-sized 00:13:59] PCs that were used all over the place in corporate environments. I still—like every doctor’s office you go to, every hospital, they have, like, a thousand of these things. So, every two or three or four years, however long it is on their contract, they just pop all those out the door and then you get an E-waste company that picks up a thousand of these boxes and they got to offload them. So, the nice thing is that it seems like a year or two ago, that really started accelerating to the point where the price was driven down below 100 bucks for a fully built-out little x86 Mini PC. Sure, it’s, you know, like you said, a few generations old and it pulls a little bit more power, usually six to eight watts at least, versus a Raspberry Pi at two to three watts, but especially for those of us in the US, electricity is not that expensive so adding two or three watts to your budget for a home lab computer is not that bad.

The other part of that is, for the past two-and-a-half years because of the global chip shortages and because of the decisions that Raspberry Pi made, there were so few Raspberry Pis available that their prices shot up through the roof if you wanted to get one in any timely fashion. So, that finally is clearing up, although I went to the Micro Center near me yesterday, and they said that they have not had stock of Raspberry Pi 4s for, like, two months now. So, they’re coming, but they’re not distributed evenly everywhere. And still, the best answer, especially if you’re going to run a lot of things on it, is probably to buy one of those little mini PCs if you’re starting out a home lab.

Or there’s some other content creators who build little Kubernetes clusters with multiple mini PCs. Three of those stack up pretty nicely and they’re still super quiet. I think they’re great for home labs. I have two of them over on my shelf that I’m using for testing and one of them is actually in my rack. And I have another one on my desk here that I’m trying to set up for a five gigabit home router since I finally got fiber internet after years with cable and I’m still stuck on my old gigabit router.

Corey: Yeah, I wound up switching to a Protectli, I think is what it’s called for—it’s one of those things I’ve installed pfSense on. Which, I’m an old FreeBSD hand and I haven’t kept up with it, but that’s okay. It feels like going back in time ten years, in some respects—

Jeff: [laugh].

Corey: —so all right. And I have a few others here and there for various things that I want locally. But invariably, I’ve had the WiFi controller; I’ve migrated that off. That lives on an EC2 box in Ohio now. And I do wind up embracing cloud services when I don’t want it to go down and be consistently available, but for small stuff locally, I mean, I have an antenna on the roof doing an ADS-B receiver dance that’s plugged into a Pi Zero.

I have some backlogged stuff on this, but they’ve gotten expensive as alternatives have dropped in price significantly. But what I’m finding as I’m getting more into 3D printing and a lot of hobbyist maker tools out there, everything is built with the Raspberry Pi in mind; it has the mindshare. And yeah, I can get something with similar specs that are equivalent, but then I’ve got to do a whole bunch of other stuff as soon as it gets into controlling hardware via GPIO pins or whatnot. And I have to think about it very differently.

Jeff: Yeah, and that’s the tough thing. And that’s the reason why Raspberry Pis, even though they’re three years old, even though they’re hard to get, they still are fetching—on the used market—way more than the original MSRP. It’s just crazy. But the reason for that is the Raspberry Pi organization. And there’s two: there’s the Raspberry Pi Foundation that’s goals are to increase educational computing and accessibility for computers for kids and learning and all that, then there’s the Raspberry Pi trading company that makes the Raspberry Pis.

The Trading Company has engineers who sit there 24/7 working on the software, working on the kernel drivers, working on hardware bugs, listening to people on the forums and in GitHub and everywhere, and they’re all English-speaking people there—they’re over in the UK—and they manufacture their own boards. So, there’s a lot of things on top of that, even though they’re using some silicons of Broadcom chips that are a little bit locked down and not completely open-source like some other chips might be, they’re a phone number you could call if you need the support or there’s a forum that has activity that you can get help in and their software that’s supported. And there’s a newer Linux kernel and the kernel is updated all the time. So, all those advantages mean you get a little package that will work, it’ll sip two watts of power, sitting 24/7. It’s reliable hardware.

There’s so many people that use it that it’s so well tested that almost any problem you could ever run into, someone else has and there’s a blog post or a forum post talking about it. And even though the hardware is not super powerful—it’s three years old—you can add on a Coral TPU and do face recognition and object recognition. And throw in Frigate for Home Assistant to get notifications on your phone when your mom walks up to the door. There’s so many things you can do with them and they’re so flexible that they’re still so valuable. I think that they really knocked it out of the park with that model, the Raspberry Pi 4, and the compute module 4, which is still impossible to get. I have not been able to buy one for two years now. Luckily, I bought 12 two-and-a-half years ago [laugh] otherwise I would be running out for all my projects that I do.

Corey: Yeah. I got two at the moment and two empty slots in the Turing Pi 2, which I’ll care more about if I can actually get the thing up and booted. But it presupposes you have a Windows computer or otherwise, ehh, watch this space; more coming. Great. Like, do I build a virtual machine on top of something else? It leads down the path super quickly of places I thought I’d escaped from.

Jeff: Yeah, you know, outside of the Pi realm, that’s the state of the communities. It’s a lot of, like, figuring out your own things. I did a project—I don’t know if you’ve heard of Mr. Beast—but we did a project for him that involves a hundred single-board computers. We couldn’t find Raspberry Pi’s so we had to use a different single-board computer that was available.

And so, I bought an older one thinking, oh, this is, like, three or four years old—it’s older than the Pi 4—and there must be enough support now. But still, there’s, like, little rough edges everywhere I went and we ended up making them work, but it took us probably an extra 30 to 40 hours of development work to get those things running the same way as a Raspberry Pi. And that’s just the way of things. There’s so much opportunity.

If one of these Chinese manufacturers that makes most of these things, if one of them decided, you know what? We’re going to throw tons of money into building support for these things, get some English-speaking members of these forums to build up the community, all that stuff, I think that they could have a shot at Raspberry Pi’s giant portion of the market. But so far, I haven’t really seen that happen. So far, they’re spamming hardware. And it’s like, the hardware is awesome. These chips are great if you know how to deal with them and how to get the software running and how to deal with Linux issues, but if you don’t, then they’re not great because you might not even get the thing to boot.

Corey: I want to harken back to something you said a minute ago, where there’s value in having a community around something, where you can see everyone else has already encountered a problem like this. I think that folks who weren’t around for the rise of cloud have no real insight into how difficult it used to be just getting servers into racks and everything up, and okay, they’re identical, and seven of them are working, but that eighth one isn’t for some strange reason. And you spend four hours troubleshooting what turns out to be a bad cable or something not seated properly and it’s awful. Cloud got away from a lot of that nonsense. But it’s important—at least to me—to not be Captain Edgecase, where if you pick some new cloud provider and Google for how to set up a load balancer and no one’s done it before you, that’s not great. Whereas if I’m googling now in the AWS realm and no one has done, the thing I’m trying to do, that should be something of a cautionary flag of maybe this isn’t how most people go about approaching production. Really think twice about this.

Jeff: Yep. Yeah, we ran into that on a project I was working on was using Magento—which I don’t know if anybody listening uses Magento, but it’s not fun—and we ran into some things where it’s like, “We’re doing this, and it says that they do this on their official supported platform, but I don’t know how they are because the code just doesn’t exist here.” So, we ran into some weird edge cases on AWS with some massive infrastructure for the databases, and I ran into scaling issues. But even there, there were forum posts in AWS here and there that had little nuggets that helped us to figure out a way to get around it. And like you say, that is a massive advantage for AWS.

And we ran into an issue with, we were one of the first customers trying out the new Lambda functions for RDS—or I don’t remember exactly what it was called initially—but we ended up not using that. But we ran into some of these issues and figured out we were the first customer running into this weird scaling thing when we had a certain size of database trying to use it with these Lambda calls. And eventually, they got those things solved, but with AWS, they’ve seen so many things and some other cloud providers haven’t seen these things. So, when you have certain types of applications that need to scale in certain ways, that is so valuable and the community of users, the ability to pull from that community when you need to hire somebody in an emergency, like, we need somebody to help us get this project done and we’re having this issue, you can find somebody that is, like, okay, I know how to get you from Point A to Point B and get this project out the door. You can’t do that on certain platforms.

And open-source projects, too. We’ve always had that problem in Drupal. The amount of developers who are deep into Drupal to help with the hard problems is not vast, so the ones who can do that stuff, they’re all hired off and paid a handsome sum. And if you have those kinds of problems you realize, I either going to need to pay a ton of money or we’re just going to have to not do that thing that we wanted to do. And that’s tough.

Corey: What I’ve found, sort of across the board, has been that there’s a lot of, I guess, open-source community ethos that has bled into a lot of this space and I wanted to make sure that we have time to talk about this because I was incensed a while back when Red Hat decided, “Oh, you know that whole ten-year commitment on CentOS? That project that we acquired and are now basically stabbing in the face?”—disclosure. I used to be part of the CentOS project years ago when I was on network staff for the Freenode IRC network—then it was, “Oh yeah, we’re just going to basically undermine our commitments to you and now you can pay us if you want to get that support there.” And that really set me off. Was nice to see you were right there as well in almost lockstep with me, pointing out that this is terrible, just as far as breaking promises you’ve made to customers. Has your anger cooled any? Because mine hasn’t.

Jeff: It has not. My temper has cooled. My anger has not. I don’t think that they get it. After all the backlash that they got after that, I don’t think that the VP-level folks at Red Hat understand that this is already impacting them and will impact them much more in the future because people like me and you, people who help other people build infrastructure and people who recommend operating systems and people who recommend patterns and things, we’re just going to drop off using CentOS because it doesn’t exist. It does exist and some other people are saying, “Oh, it’s actually better to use this new CentOS, you know, Stream. Stream is amazing.” It’s not. It’s not the same thing. It’s different. And—

Corey: I used to work at a bank. That was not an option. I mean, granted at the bank for the production systems it was always [REL 00:25:18], but being able to spin up a pre-production environment without having to pay license fees on every VM. Yeah.

Jeff: Yeah. And not only that, they did this announcement and framed it a certain way, and the community immediately saw. You know, I think that they’re just angry about something, and whether it was a NASA contract with Rocky Linux, or whether it was something Oracle did, who knows, but it seems petty in retrospect, especially in comparison to the amount of backlash that came out of it. And I really don’t think that they understand the thing that they had with that Red Hat Enterprise Linux is not a massive growth opportunity for Red Hat. It’s, in some ways, a dying product in terms of compared to using cloud stuff, it doesn’t matter.

You could use CoreOS, you could use NixOS, and you could use anything, it doesn’t really matter. For people like you and me, we just want to deploy our software. And if it’s containers, it really doesn’t matter. It’s just the people in government or in certain organizations that have these roles that you have to use whatever FIPS and all that kind of stuff. So, it’s not like it’s a hyper-growth opportunity for them.

CentOS was, like, the only reason why all the software, especially on the open-source side, was compatible with Red Hat because we could use CentOS and it was easy and simple. They took that—well, they tried to take that away and everybody’s like, “That’s—what are you doing?” Like, I posted my blog post and I think that sparked off quite a bit of consternation, to the point where there was a lot of personal stuff going on. I basically said, “I’m not supporting Red Hat Enterprise Linux for any of my work anymore.” Like, “From this point forward, it’s not supported.”

I’ll support OpenELA, I’ll support Rocky Linux or Oracle Linux or whatever because I can get free versions that I don’t have to sign into a portal and get a license and download the license and integrate it with my CI work. I’m an open-source developer. I’m not going to pay for stuff or use 16 free licenses. Or I was reached out to and they said, “We’ll give you more licenses. We’ll give you extra.” And it’s like, that’s not how this works. Like, I don’t have to call Debian and Ubuntu and [laugh] I don’t even have to call Oracle to get licenses. I can just download their software and run it.

So, you know, I don’t think they understood the fact that they had that. And the bigger problem for me was the two-layer approach to destroying all the trust that the community had. First was in, I think it was 2019 when they said—we’re in the middle of CentOS 8’s release cycle—they said, “We’re dropping CentOS 8. It’s going to be Stream now.” And everybody was up in arms.

And then Rocky Linux and [unintelligible 00:27:52] climbed in and gave us what we wanted: basically, CentOS. So, we’re all happy and we had a status quo, and Rocky Linux 9 and [unintelligible 00:28:00] Linux nine came out after Red Hat 9, and the world was a happy place. And then they just dumped this thing on us and it’s like, two major release cycles in a row, they did it again. Like, I don’t know what this guy’s thinking, but in one of the interviews, one of the Red Hat representatives said, “Well, we wanted to do this early in Red Hat 9’s release cycle because people haven’t started migrating.” It’s like, well, I already did all my automation upgrades for CI to get all my stuff working in Rocky Linux 9 which was compatible with Red Hat Enterprise Linux 9. Am I not one of the people that’s important to you?

Like, who’s important to you? Is it only the people who pay you money or is it also the people that empower your operating system to be a premier Enterprise Linux operating system? So, I don’t know. You can tell. My anger has not died down. The amount of temper that I have about it has definitely diminished because I realize I’m talking at a wall a lot of times, when I’m having conversations on Twitter, private conversations and email, things like that.

Corey: People come to argue; they don’t come to actually have a discussion.

Jeff: Yeah. I think that they just, they don’t see the community aspect of it. They just see the business aspect. And the business aspect, if they want to figure out ways that they can get more people to pay them for their software, then maybe they should provide more value and not just cut off value streams. It doesn’t make sense to me from a long-term business perspective.

From a short term, maybe there were some clients who said, “Oh, shoot. We need this thing stable. We’re going to pay for some more licenses.” But the engineers that those places are going to start making plans of, like, how do we make this not happen again. And the way to not make that happen, again is to use, maybe Ubuntu or maybe [unintelligible 00:29:38] or something. Who knows? But it’s not going to be increasing our spend with Red Hat.

Corey: That’s what I think a lot of companies are missing when it comes to community as well, where it’s not just a place to go to get support for whatever it is you’re doing and it’s not a place [where 00:29:57] these companies view prospective customers. There’s more to it than that. There has to be a social undercurrent on this. I look at the communities I spend time in and in some of them dating back long enough, I’ve made lifelong significant friendships out of those places, just through talking about our lives, in addition to whatever the community is built around. You have to make space for that, and companies don’t seem to fully understand that.

Jeff: Yeah, I think that there’s this thing that a community has to provide value and monetizable value, but I don’t think that you get open-source if you think that that’s what it is. I think some people in corporate open-source think that corporate open-source is a value stream opportunity. It’s a funnel, it’s something that is going to bring you more customers—like you say—but they don’t realize that it’s a community. It’s like a group of people. It’s friends, it’s people who want to make the world a better place, it’s people who want to support your company by wearing your t-shirt to conferences, people want to put on your red fedora because it’s cool. Like, it’s all of that. And when you lose some of that, you lose what makes your product differentiated from all the other ones on the market.

Corey: That’s what gets missed. I think that there’s a goodwill aspect of it. People who have used the technology and understand its pitfalls are likelier to adopt it. I mean, if you tell me to get a website up and running, I am going to build an architecture that resembles what I’ve run before on providers that I’ve run on before because I know what the failure modes look like; I know how to get things up and running. If I’m in a hurry, trying to get something out the door, I’m going to choose the devil that I know, on some level.

Don’t piss me off as a community member and incentivize me to change that estimation the next time I’ve got something to build. Well, that doesn’t show up on this quarter’s numbers. Well, we have so little visibility into how decisions get made many companies that you’ll never know that you have a detractor who’s still salty about something you did five years ago and that’s the reason the bank decided not to because that person called in their political favors to torpedo that deal and have a sweetheart offer from your competitor, et cetera and so on and so forth. It’s hard to calculate the actual cost of alienating goodwill. But—

Jeff: Yeah.

Corey: I wish companies had a longer memory for these things.

Jeff: Yeah. I mean, and thinking about that, like, there was also the HashiCorp incident where they kind of torpedoed all developer goodwill with their Terraform and other—Terraform especially, but also other products. Like, I probably, through my book and through my blog posts and my GitHub examples have brought in a lot of people into the HashiCorp ecosystem through Vagrant use, and through Packer and things like that. At this point, because of the way that they treated the open-source community with the license change, a guy like me is not going to be enthusiastic about it anymore and I’m going to—I already had started looking at alternatives for Vagrant because it doesn’t mesh with modern infrastructure practices for local development as much, but now it’s like that enthusiasm is completely gone. Like I had that goodwill, like you said earlier, and now I don’t have that goodwill and I’m not going to spread that, I’m not going to advocate for them, I’m not going to wear their t-shirt [laugh], you know when I go out and about because it just doesn’t feel as clean and cool and awesome to me as it did a month ago.

And I don’t know what the deal is. It’s partly the economy, money’s drying up, things like that, but I don’t understand how the people at the top can’t see these things. Maybe it’s just their organization isn’t set up to show the benefits from the engineers underneath, who I know some of these engineers are, like, “Yeah, I’m sorry. This was dumb. I still work here because I get a paycheck, but you know, I can’t say anything on social media, but thank you for saying what you did on Twitter.” Or X.

Corey: Yeah. It’s nice being independent where you don’t really have to fear the, well if I say this thing online, people might get mad at me and stop doing business with me or fire me. It’s well, yeah, I mean, I would have to say something pretty controversial to drive away every client and every sponsor I’ve got at this point. And I don’t generally have that type of failure mode when I get it wrong. I really want to thank you for taking the time to talk with me. If people want to learn more, where’s the best place for them to find you?

Jeff: Old school, my personal website, jeffgeerling.com. I link to everything from there, I have an About page with a link to every profile I’ve ever had, so check that out. It links to my books, my YouTube, all that kind of stuff.

Corey: There’s something to be said for picking a place to contact you that will last the rest of your career as opposed to, back in the olden days, my first email address was the one that my ISP gave me 25 years ago. I don’t use that one anymore.

Jeff: Yep.

Corey: And having to tell everyone I corresponded with that it was changing was a pain in the butt. We’ll definitely put a link to that one in the [show notes 00:34:44]. Thank you so much for taking the time to speak with me. I appreciate it.

Jeff: Yeah, thanks. Thanks so much for having me.

Corey: Jeff Geerling, YouTuber, author, content creator, and oh so very much more. I’m Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you’ve enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you’ve hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment that we will, of course, read [in action 00:35:13], just as soon as your payment of compute modules for Raspberries Pi show up in a small unmarked bag.

Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.