Mitchell Hashimoto is Founder and CTO of HashiCorp. He is the creator of Vagrant, Packer, Serf, Consul, Terraform, Vault and Nomad - a set of open source tools that each individually are downloaded and used millions of times per year. At one point Mitchell was in the top 5 most active users on GitHub. At HashiCorp, Mitchell is helping define a remote-first culture with over 500 employees spanning dozens of countries. He loves open source, automation, and working from home.

Links Referenced: 

Announcer: Hello and welcome to Screaming In The Cloud, with your host cloud economist, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world and ridiculous titles for which Corey refuses to apologize. This is Screaming In The Cloud.

Corey Quinn: This week’s episode is generously sponsored by Digital Ocean. I’d argue that every cloud platform biases for different things. Some bias for having nearly every feature you could possibly want as a managed service at varying degrees of complexity. Others bias for, “Hey! We heard there was money in the cloud and we’d like it if you would give us some of that!” Digital Ocean is neither. From my perspective, they bias for simplicity. 

Corey Quinn: I wanted to validate that so I polled a few friends of mine about why they were using Digital Ocean for a few things, and they pointed out a few things. They said it was very easy and clear to understand what you were doing and what it took to get up and running when you started something with Digital Ocean. That other offerings have a whole bunch of shenanigans with root access and IP addresses and effectively consulting the bones to make those things to work together. Digital Ocean makes it simpler. In 60 seconds they were able to get root access to a Linux box with an IP. That’s it. That was a direct quote except for the part where I took out a bunch of profanity about other cloud providers. 

Corey Quinn: The fact that the bill wasn’t a whodunnit murder mystery was compelling as well. It’s a fixed-price offering. You always know what you’re going to wind up paying in a given month. Best of all, you don’t have to spend 12 weeks going to cloud school to understand all their different offerings. They also include monitoring and alerting across the board and they’re not exactly small-time. Over 150,000 businesses and three-and-half million developers are using them. So give them a try. Visit do.co/screaming, and they’ll give you a free $50 credit to try it. That’s do.co/screaming. Thanks again to Digital Ocean for their support of Screaming in the Cloud. 

Corey: Welcome to Screaming In The Cloud. I'm Corey Quinn. Once upon a time in 2014, I was a traveling trainer for a configuration management system that need not be mentioned here. I was on the way back, and the person next to me on the plane saw my shirt. Oh, great. A plane talker. Doesn't everyone love talking to one of those? He started talking about a system he was in the process of building and about to release that would handle infrastructure management at scale programmatically. Cool, great. We've all heard those stories before. I didn't see it going too far, but wished him well. Now we're having this conversation almost five years later. Mitchell Hashimoto, founder of HashiCorp, author of Terraform and a bunch of other things, welcome to the show.

Mitchell: Hey Corey. Nice to sit next to you again, virtually.

Corey: Oh yeah. I will freely admit I was wrong on that one. It's turned out that you can have a 98 or so percent success rate by talking to people who are building something new and saying, "Oh, that'll never work." Then you're going to get it hilariously wrong for conversations like this. But by and large, it always seems like being a pessimist is a viable strategy or a path forward.

Mitchell: It's a great way to lose nothing, but also to gain and very little too.

Corey: That's the fun thing of being a futurist, too. You can always make outlandish predictions. If you happen to be right, you'll be lauded. If you're wrong, well, everyone's wrong and no one really holds it over your head anymore.

Mitchell: Yup. Yup. To be fair, I wouldn't have been optimistic about me either, from the third person.

Corey: It seems to have worked out though. You folks have now become a name brand in the space. It's been years since I had to mention what Terraform was when giving a talk around this stuff.

Mitchell: Nice.

Corey: That said, for those who are unaware ... Because not everyone knows everything, as it turns out, can you describe what Terraform is for folks who may have never heard of such a thing?

Mitchell: Of course, yeah. So Terraform is quite literally infrastructure as code. So you describe servers, switches, DNS records, anything you would imagine. I like to say anything that would be in a "data center" to run an application. You put it into a text file, you tell Terraform to make it for you, and it does that using ... By stitching together a variety of APIs from cloud providers and SaaS providers and so on.

Corey: For those of you in the AWS ecosystem, you can think of it almost like CloudFormation done properly, which is, I'm sure, a battle that we will get into later and no one would ever disagree with. Meanwhile, you just hear people starting to seethe in Seattle at this point.

Mitchell: Yes, yes. I won't say you're right, but let's see where this goes.

Corey: Absolutely. But first, let's start with something else that'll irritate people. I noticed a while back that you had a post on Reddit, which is always a great place for a conversation to start, talking about multi-cloud. Given that I've spent roughly 18 months now talking about why multi-cloud as a best practice is almost always the wrong decision to go in, I would love to get your thoughts on this.

Mitchell: Yeah, yeah, so multi-cloud has been ... Let me start by saying that the HashiCorp's sort of core mission or vision has been enabling people to adopt multi-cloud. So that has the underlying assumption that multi-cloud is real. For a number of years early on in the company, that was pretty laughable, I would say. When we founded the company in 2012, I think that it was pretty common for people to think that there was no possible future reality, other than a fully Amazoned world.

That has changed over time, at least from my biased point of view. But yeah, what I always tell people is sort of like I've learned ... I bet this early on, but I've learned in practice since then that multi-cloud is somewhat inevitable. It's kind of like entropy in my mind. It's like you could fight it, and you could fight it, and there is good reasons to fight it. I won't disagree with you there, but at a certain point it's inevitable.

So if it is an inevitability, then how do you best equip your business or organization to be multi-cloud or ready even if you're not? So I want to make it super clear that I never tell people, and I don't advocate for people being multi-cloud early. I think you should 100% focus on a single cloud, get really good at it until you're just forced away from it for a good reason. But while you're single cloud, you should just try to adopt tooling that will make that a little bit of a smoother path.

Corey: Before we dive into this and potentially start tilting at the same windmill from opposite sides, it feels like we should define terms. When you hear the term multi-cloud, what does that mean from your point of view?

Mitchell: Yeah, that's really important. So for me, it means what I like to call workflow portability. I define multi-cloud in my mind as four different things. It's workflow, portability of work load portability, data portability and traffic portability. I've experienced when most people say multi-cloud, they immediately snap to work load portability, which is sort of the idea that any application you write could run on any cloud. For me, that's absolutely not the case.

I think you should take advantage of all the best-in-class, first-class services that make applications explicitly not portable between clouds. What is more important to me is workflow portability, which is sort of the process by which you get infrastructure, deploy applications, secure them, network them. Those sorts of core things should be multi-cloud.

Corey: I would agree with you that on a per-workload basis it tends to be something of a red herring. I would also say that, "Oh, you should never use a second provider for anything", that's nonsense. I mean, easy example of this is PagerDuty. There's not a whole lot else like that in the market and certainly none of them from any one large cloud provider.

I would also strongly suggest people use ... Even if you're on AWS, I'd still recommend you use GitHub, which is a Microsoft product, or Office 365, or G Suite. There's always going to be higher-level differentiated services that make sense to consume. I think that people tend to hear multi-cloud in conversations I've been in, and where their minds tend to go is almost a VM-centric, where you're coming from an on-prem environment where you've got this thing sitting running in VMware most likely. I should be able to seamlessly have that flow to AWS, to Azure, to GCP, to Oracle Cloud in some very strange conversations, and that nothing else should have to operate around that in any way. I just haven't seen that in the real world.

Mitchell: Exactly. Yeah, and I've seen ... I think I alluded to this in that Reddit comment, but I've seen the opposite now, where it's been long enough with AWS in existence that I have now seen a company go from startup to public pure AWS, like actually AWS poster child in a way, and then be forced to adopt another cloud and the almost tragedy that fell out of that. I think that's very interesting.

The abridged form of that story is really that there is a darling startup, which I won't name, but it's real. They went 100% into AWS, blogged about how they automated the heck out of it, and really good thought leader at AWS became a multibillion dollar public company, full AWS again. They ended up acquiring another company for a couple of hundred million dollars, and this other company was pretty large themselves. I mean, if you're getting acquired for that much, they're pretty large themselves, pretty successful. 100% on GCP.

It became this thing where ... I've talked to the executives, and it was one of those things where you're not going to block an acquisition that's good for the business of a public company based on their tech stack. Like it's just not going to happen. If the strategy makes sense, the partnership makes sense, it rarely is going to happen. So they decided to acquire this company no matter what. They bought them, and then what ended up happening was they basically churned their whole infrastructure team. Pretty expensive. It was either convert everyone to AWS, which they were violently against and the people internally were against from a cost perspective, or just figure out how to do multi-cloud.

Again, this wasn't workload portability. They didn't want to move anything over to AWS, but just how do you do governance? How do you do cost reporting? How do you do like these higher level things across clouds? They ended up basically rehiring an entire infrastructure ops team to handle this, and it was messy.

Corey: It's a form of lock-in that people generally don't tend to think about. It's not just technical or architectural, but if you have an entire team of people that are great on cloud A and then, "Oh by the way, we're migrating things to cloud B as well", very often people don't want to do that. They'd rather continue to focus on their area of expertise rather than start to dilute that across multiple providers and confuse themselves. There's also any higher-level service story starts to break down.

I think that when there's a compelling business reason to go multi-cloud, I'm all for it. What I'm agitating for has not been, "Oh, pick a provider and go all-in at the cost of your business." That makes no sense to anyone. But building something Greenfield on day one with this idea that you're going to run it on AWS, or GCP, or Azure, or wherever else you want to put that, maybe you don't do that. Maybe pick a provider. I don't care which one. They probably care which one. And just hope for the best until you have a compelling reason to move.

But trying to build something that can be seamlessly deployed to everything ... It doesn't work. If you do go down that path, you are going to find that you're constrained to some very low-level primitives, because nothing beyond more or less a bunch of VMs and maybe some databases and block storage tends to be the same across providers. Even database provisioning calls tend to start breaking down.

Mitchell: Yeah. So yeah, that's an important part of the way we view multi-cloud. One of the first things I tell anyone that is adopting Terraform for the first time is ... They read CloudFormation, but for anything or they read something like that somewhere on the Internet and they think, "Oh, this is the right once run anywhere dream." But I make it really clear, and I'm going to make it clear right now that it's not that. Like the Terraform you write is AWS specific, or Azure specific, or Google specific. You cannot rerun that and target a different cloud platform. That's not what we're doing. That would be workload portability. What we're doing is workflow portability.

So any configure write for any cloud, it's always the same command to create it, the same command to see what would happen, the same structure to investigate for cost analysis, things like that. It's higher level than the actual tech itself.

Corey: I spoke to a single tenant customer that was trying to build out a deployment on AWS and GCP. They were already a Terraform shop, so they were very conversant with the tool. But getting the networking and the security groups to coexist between providers was massively painful. That is not anything I would consider having added differentiated value. For their business case, it was worth the experiment. Didn't really pan out longer term, but the challenge that you'll see is, at that point when you try an experiment and it doesn't work out, you have to sort of stop riding two horses. Which one do you pick? Invariably, I find that data gravity tends to carry that argument pretty quickly.

Mitchell: Yeah, yeah. I would agree with you completely there. I think that's why applications tend not to move. I mean it's very rare for our multi-cloud believers, so to speak, to move an application or ... Especially because of the data associated with it. What's much more realistic is a new application or a totally different team or business unit decides for really practical and good reasons that they want to use a different platform.

Corey: When I'm doing Greenfield architecture and looking at something that, "Okay, I want to go ahead and build this on a particular provider", I generally don't bake multi-cloud requirements into that. But I will say that I do try to avoid, I guess, strategic architectural lock-in. There's a whole bunch of stories around that, but an easy one might be using something like Cloud Spanner or Dynamo DB Global Tables, just because those tend to be things that don't manifest in the same way as anywhere else. So migration isn't just a painful move a thousand things over. It's also and rearchitecture entire applications data model.

Mitchell: Yeah. But again, I don't worry too much about that because I don't think that your applications necessarily need to move or be portable. It's going to depend on your use case. But if ... Cloud Spanner is quite expensive, but if you look at Cloud Spanner, and it's going to save you a ton of time in order to get a technically correct application to market, then I think you should do it. I think that if your application is successful, it's unlikely that it'll have to be across multiple cloud platforms. So it's worth it. But at the same time, it's just a question of whether you want to be locked in technically to a provider or not.

Corey: I would agree with you, but this question often comes from higher-level strategic business decision makers. The question of, "What happens if Amazon decides to compete with us?" To which my response is always, "What do you mean "if"? Their product strategy is yes."

Mitchell: Yeah, we've seen that more in the recent years. We've seen that more and more, for sure. It's been really, really interesting watching entire industries basically create plans to shift providers overnight due to major acquisitions like that.

Corey: It's always challenging watching people go through that process and, "Well, we've decided that we're not going to put anything on AWS, because some of our customers demand it." You take a step back and you look at some of those customers who are demanding that, and those customers themselves have workloads living on AWS that aren't small. So it's one of those fun explorations of, "Do what I say, not what I do." It feels, to some extent, like a little bit of tail chasing going on in our industry.

It also feels to me, to some extent, like multi-cloud on a workload basis is actively being pushed by a number of vendors in this space. Where if you take a step back and say, "We're going to go all-in on any given provider", well at that point there may not be a story for those vendors who benefit from the multi-cloud approach to have something to sell you.

I'd like to ask from your perspective, given that Terraform's entire ... One of the reasons that Terraform exists is to support infrastructure across all providers. If someone goes all in on a particular provider, do you see that you have something to sell them?

Mitchell: Yeah, so I think we're different because we sit at that workflow level, that even these passionately single cloud companies, we're seeing them use things like Terraform in many cases. Not always, but in many cases. That's because Terraform is a lot more than just the infrastructure. I think you brought this up. It could glue together so many different levels of things. It's DNS providers, it's GitHub source control, it's CDNs, it's things like that that a copywriter alone doesn't often provide all of them.

So Terraform is a good way just to do that. Even if you say all our compute networking storage, like the traditional things are all on a single vendor, but we want to pull the latest commit from GitHub and use that as a way to tag things or something. I don't know. There's all sorts of use cases sort of like that where Terraform is super useful.

So I think that it doesn't affect us as much. We explain the benefits we view of multi-cloud from a workflow perspective, but the ironic thing is we have a lot of paying customers that are both single cloud, and we have a lot of bank customers that are actually not cloud at all. They're all just private data center. And so I think we build technical tools that work with both, even though we think that ... We use multi-cloud is a way to basically explain how workflow portability is extremely important.

Corey: I think that's the narrative that makes sense. I think that's what we're starting to see resonate in the space. I also do want to special case the idea of hybrid cloud, where you have on-premises data center environment that tends to be there for a while. Having a hybrid strategy makes sense. You can also see it as a transitional state. For a lot of these companies, that transition looks like the better part of a decade through no guilt of their own. And that's okay. One thing I want to make very clear is that I'm not trying to be architectural shaming of anyone here.

The idea of, "Well, we built a thing for certain constraints, the constraints have shifted and changed. 20 years later, we have this thing that makes money." We can't just wave our hand and turn it off because something on a whiteboard looks more appealing now. I have a lot of sympathy for that and I don't think anyone has intentionally gone out to make a series of poor decisions on this.

Mitchell: Yeah, and I mentioned it in our keynote at our last conference that ... I'm sorry if this is a negative thing, but I think we've made those transitions a lot longer for a lot of people because our tooling has made the "legacy environments" a lot more productive. So Terraform is actually one of those places we see it pretty often where people that opt Terraform before their cloud adoption story and then realize like, "Oh hey, Terraform works pretty great with something like V-sphere", and "Oh wow, we can automate our entire on-prem set up end-to-end much better. Let's just actually move that fence post of decommissioning this off another couple of years."

So again, one of those ironic things, we've signed a lot of customers where they're saying, "Okay, we're completing 100% transition to the cloud in ...", they'll say three years. Now we're talking to them two or three years later, and they're like, "Maybe we could have done that, but we're happy. We're going to push it off another three years." I'm either sorry or like kind of excited that people could get more value out of what they already paid for.

Corey: Oh, I think the idea of being able to stretch an existing investment further is compelling to everyone. I don't think anyone has any particular problems with that. You see people moving from on-prem to public cloud as well, and it's just almost a comedy of strategic errors that you see getting made along the way. Where first they'll spend eight months figuring out what it's going to cost them. Spoiler, they're going to get most of it wrong, and it's effectively an educated guess. Then they're going to start the process. It's going to take longer than they thought. It's going to be more difficult, and they're not going to save money during the initial phases until they start teaching applications to dynamically scale, which many of them don't.

But it's worth doing in most cases anyway, from a time-to-market and feature velocity story, even if the cost savings won't be captured for a longer timeline than they would expect them to. It feels like there's almost an entire cottage industry that sprung up around making those migrations take longer than they need to.

Mitchell: Yeah. We're still super in the ... It's kind of sad in this way, but we're still super in the beginner cloud usage days. Or I would say the majority of large companies right now ... You brought up auto scaling. I mean auto scaling is still so rarely used. Actually, Microsoft research published a really interesting paper about doing an analysis across, I believe it was Azure, I'm sure. But it's sort of an academic paper. I mean you could read it publicly, but they did this study on how many people are doing auto scaling and what the benefit would be if they did it theoretically. The number was laughably small. If you exclude people that use auto scaling as simple way to just make sure a single VM stays up and running, or a fixed count of VM stay up and running, it's basically zero.

So I think as we get closer and closer to actually getting to more advanced resource usage like that, then it'll get better. But really, it's more or less like the ... The conclusion of that paper and the conclusion I'm getting to is cloud adoption is more or less lift and shift still at the moment.

Corey: One of the problems I have with it is the idea that that's somehow a terrible pattern. Because the alternative is you transform as you go. "Okay, we're going to take this application and the data center, rewrite it to take advantage of cloud-offered primitives and then deploy it." Then it doesn't work, or heaven forbid, it's intermittently bad. Then you get to start going through this entire analytics process. I'm a fan of lift and shift, but you have to follow through on the second phase. Expect that to take longer than you think it will.

Mitchell: Yes, it's going to take a very long time. But I'm saying that optimistically, because if it weren't for ... And I always am super clear about if it weren't for cloud existing, then companies like ours wouldn't have even had a chance. Because the existence of cloud is forcing companies to open their minds to the idea of maybe adopting a new way to do infrastructure, or a new way to think about security or networking or things like that. So if cloud didn't exist, then I don't think something like Terraform would be nearly as popular or Vault. Like the way we do security would be different. I think software is getting better, and the infrastructure around it is getting better, even if we're still in this beginner phase of using what the platform actually gives you.

Corey: I think that you're onto something there. The idea that you can take wherever you are and have tooling that winds up meeting you there and helping you get to a better place or getting more out of what you've already bought is compelling. Now, not to get too annoying for people on this, but let's go with the exact opposite direction of making everything better, and talk a bit about CloudFormation.

So when Terraform came out, CloudFormation already existed. Was it just the multi-cloud story that inspired you to start building something that stood in its place?

Mitchell: No. So, let's talk about the history of Terraform, because there's a truth in the history of Terraform ... Which I'm sure I've said it somewhere publicly, but I'm not sure if I have, which is that when we first made Terraform, we actually didn't see the multi-cloud use case. We built Terraform as what we felt was just a better way to do infrastructure as code on single providers at a time.

It wasn't AWS only or anything. It was either you wrote Terraform code that worked just with AWS or just with Google and couldn't overlap them, basically. The multi-cloud, like multiple providers and a single configuration sharing data and stuff, that was a coincidence prior to the release of Terraform, which is fun. I think that coincidence was the thing that made me actually realize like, "Oh, I think we created something really cool here", but that wasn't actually the original motivation. The original motivation was that I felt we could do a better job.

Corey: Taking a look at it now, I would argue that by any objective standard, there's a strong argument to be made that you had. We still see scenarios where Terraform providers for new services come out before CloudFormation does, which is just ridiculous to me. I don't pretend to understand what goes on behind that. But very often, people I've seen go directly into a Terraform world for pure AWS environments, just due to a wide variety of factors, it all distilled down into it offers things that CloudFormation simply doesn't.

Mitchell: Yeah, I mean Terraform definitely offers things that CloudFormation doesn't. I'm providing resources. I think the open source community is a big help in making us go faster, but at the same time, someone did an objective analysis of CloudFormation versus Terraform and found that it was more or less a wash, in terms of resource speed. There was a bunch of CloudFormation was first at, there was a bunch of stuff Terraform was first at. It was kind of a wash.

But yeah, I mean, there's a bunch of other features. Like the ability to run it as a CLI on your own machine and not use a dedicated service. For a long time, Terraform Plan was completely unique. I think CloudFormation has something like that now. But I think the biggest thing overall is, I think, the ergonomics of the language are better in a lot of ways. I think that the fact that you could glue together different providers and enhance it by writing your own providers is extremely attractive. But you know, I'm speaking from the most biased point-of-view.

Corey: Well, absolutely. I mean you even got support for a CloudFormation provider to my understanding, where you can take any arbitrary CloudFormation template you want and do that within Terraform.

Mitchell: Yeah, and there's practical pragmatic reasons for that, which is you're transitioning to Terraform and you've converted a bunch of it, but you still have that one CloudFormation template that you have. We'll run it for you.

Corey: Oh, it also absolutely empowers my preferred developer model, which is copying and pasting directly from stack overflow, which is why I consider myself a full stack overflow developer. It winds up making a fair bit of sense for me.

So one decision you made early on that I would love to ask you about is you picked writing your own domain specific language for Terraform-

Mitchell: I knew it was going in this direction.

Corey: Right. At the time, CloudFormation was only JSON. Now it supports JSON or YAML, but I'm holding out for XML, just to ensure that I'm irritating everyone with this question. Why a DSL and would you do it again?

Mitchell: Yeah. So this always hits different people with different cords. So one thing to, I think, put into the context about the history of our company and our tools is that we've tried both ends of that spectrum. So Vagrant was the first thing we made. It's almost 10 years now, at the time that we're recording this, and it was all Ruby DSL, like Ruby config.

So you can do anything if you're proficient with Ruby in a Vagrant configuration. Some people love that, and some people hate it. Then, before we go deeper into that, then we did Packer after that. Because of what I learned from Vagrant, and sort of the sharp edges around Vagrant, I decided that the obvious solution with Packer was to use a pure data structure language. So I use JSON. JSON, a lot of people love that. The idea behind JSON was any modern language can generate JSON.

So if you wanted to use Ruby, you could actually use Ruby and just generate JSON. There you go, you have your DSL. That's not strictly true, but it solved a lot of problems. JSON was good, but then I realized that there's a lot of people that hated it. The joke I tell people is that there's ... About once or twice a year, there is a single week where I'll get two emails of someone saying, "I love HCL", which is the name of our config language in Terraform. "I love HCL." Then there'll be someone who says, "I hate HCL", or someone that'll say like, "I love Ruby", or "I love Vagrant's Ruby config", and then someone else that says, "I hate Vagrant's Ruby config."

I think when you get these totally split spectrum points-of-view, what it represents to me is that it's an opinion. There's something here that you can't really win. When we took a look at HCL, Terraform's language, we were trying to balance these two things as best as we could. So what we did with HCL was the actual native HCL syntaxes as human-friendly as possible. It's meant to be human-readable, human-writeable. It's not meant to be machine-friendly in a big way, but what we did was make it a one-to-one mapping with JSON.

So there is an exact JSON syntax that every HCL could make maps to and vice versa, and that is super useful for machines. We did this thing. I think you can't really win. We get some people, like I said, that still love it. We actually ran some like independent studies across users who had never heard of us, users who had used some parts of our tools, users who have used, all of them. HCL performed the best in terms of what people liked the most. But yeah it's a hard thing to say like right or wrong.

Corey: If nothing else, you've given people something to focus on that isn't core and central to the ethos of building the tool. It's, oh no matter what you pick, someone is going to have a problem with it somewhere on the internet, loudly and usually not particularly coherently.

Mitchell: Yeah. Yeah, and the problem is like even if they direct their hate, or whatever it is, towards me or whoever for doing this abominable thing, I understand the frustration because you know I prefer some things to other tools. It's there but it's hard to satisfy everyone.

Corey: Well, last question around the, I guess, glorious dissatisfaction of various people. Originally when Terraform came out, it was coupled fairly tightly with Packer console, Nomad at the time. This was all wrapped together in an enterprise offering called atlas that you folks offered, which was fantastic, having played with it a couple of times. The challenge that I saw with adoption, when I was talking with various clients back then, was always that it required an all-in investment, where suddenly you're using all of the HashiCorp Stack or else you're sort of on your own.

Given that you no longer sell or reference Atlas, I have to assume that that wasn't just something that came out of my own experience. Can you talk a little bit about what that tight coupling looked like, and I guess why it isn't around anymore?

Mitchell: Yeah, it's gone. The idea was that ... What's the best way to build a business out of a product? It's like ask the customer. What do they want? If you give them what they want, the theory is that they'll pay you for it. That's the dot, dot, dot, and then you're on step three profit and you're happy.

So we did that. We asked the customer what they wanted. The problem was that the "customers" we're asking were HashiCorp fanatics. When you asked that segment of user what you want, they are going to say, "We have all these HashiCorp tools. We love them. We would love if you just tied them together for us to make it work." So we did that, and we had a small group of pretty happy users of what we built.

The issues really came around with the not-fanatics, with the people who either didn't know who we were or adopted like one or two things. Because exactly like you said, there was no incremental adoption, and even worse there was not really incremental pricing. I mean, that's a fixable thing but it just made it hard to reason about. Things were expensive because there was a lot more value that they didn't want out of it, things like that.

So what we realized is what we built as an enterprise product as our business was sort of counter philosophical to how we viewed the world in open source. That was a problem, both philosophically and in practice, because our open source point-of-view has always been, "We have six open source tools that do dramatically different things", whereas one open source vendor might have put them all in a single mega tool, we've always been really clear that we split them up because it makes them easier to adopt, easier to learn, easier to interface with other software, things like that.

We completely threw away that philosophy originally for our enterprise products. Saying it now in hindsight, it's so obvious. But what we ended up doing was just breaking it down in that same way, so we could let people adopt an enterprise version of both, separately from Console, separately from Terraform, etc.

Corey: No, I think that it's one of those lessons that only really makes sense after you've either gone through it once or seen someone else go through it. I don't know if it would've been possible to predict that, going in. I mean sure, with the benefit of hindsight.

Mitchell: It seemed like it.

Corey: Everything feels so easy to predict in hindsight, but in the thick of it, you never know quite how the market's going to react, how the market's going to shift. I mean, at the time we met, I was speaking largely around configuration management. It seemed like that was a "full steam ahead" business unit. Now, it seems like the entire industry has taken a turn toward immutable infrastructure. I can either sit here, and shake my fist at this, and refuse to change, or I can find something else to be good at that isn't imperative or declarative configuration management. Instead start looking at something else. Infrastructure as code has clearly carried the day.

Mitchell: Yeah, and I was actually, coincidentally prior to recording this, I was just talking to another founder who has pretty cool, very hip, very edgy technology. One of the things I was telling that founder was, "Don't try to be revolutionary in too many things, because if you attach yourself to something that's already like rocket shipping up, their marketing dollars lift you up too. But if you fight it and you say, "No, our way is the true way", then you're just fighting something that you don't have the resources to fight.

I think we didn't try to do that, but that's what we were happening to do, was saying, "Oh yeah, you adopted these three tools but really the one true way is our mega platform." Cast aside the fact that it worked, and it was pretty cool, and things like that ... Of course, a lot of people didn't work but for the right type of user, it did work. But even if you throw that aside, it didn't matter because we were fighting what the rest and sort of the industry was telling people to do.

You kind of have to be pragmatic about that and say, "Okay, they're going to adopt this one thing we have because they see the value in that, even though we want them to use something like Terraform, let's say they're going to use something like Azure Resource Manager or something. We just have to be okay with that." So we switched to that point-of-view.

Corey: The one thing that continues to stick in my mind a year and a half after I did it was after the first year of running my newsletter, I wound up conducting a reader survey. I got almost a thousand responses and asked people, "What are you using to manage your infrastructure?" CloudFormation and Terraform were within the margin of error of being equal to each other.

Now in hindsight, I could have constructed the survey a bit better, because let's not kid ourselves. An awful lot of shops are using both in different ways. But it did turn into very clear this is as widely deployed among an AWS user base. It is large enough to be statistically significant, that this is something that's real. This is not just some random project on GitHub that someone stumbled over. You've tapped into something. You're seeing adoption across the enterprise and in startups as well, and everything in between.

Mitchell: Yeah, and I already shared this publicly, but I won't name the exact cloud, but there's a major cloud provider where they've shared with us that Terraform drives a double digit percentage of all infrastructure that they spin out. That's not just Compute, that's like the databases, the load balancers, things like that. It's not just statistically significant, that's financially significant, that's industry significant and that's really cool. But I think that goes to show that there could be an independent infrastructure as code thing that can rival the first-class, first-party solutions that the cloud providers provide.

Corey: That's just a fantastic story. It's nice to see that even in this world where it feels like the cloud providers are eating everyone's lunch, that it's provably not true. You just have to offer something that the market needs.

Mitchell: Yep. Yep, and I think we did that, and I hope so.

Corey: It would seem that the entire industry has agreed with you at this point. Mitchell, thank you so much for taking the time to speak with me today.

Mitchell: Yeah, nice talking to you.

Corey: If people want to hear more, where can they find you?

Mitchell: I don't hide. Just hashicorp.com for our company or @MitchellH on Twitter. I tend to respond to as many people as I can. I don't hide my email, so you'll figure it out through those avenues.

Corey: Sounds good to me. Mitchell Hashimoto, founder and CTO of HashiCorp. I'm Cory Quinn. This is Screaming In The Cloud.

Announcer: This has been this week's episode of Screaming In The Cloud. You can also find more Corey at screaminginthecloud.com, or wherever fine snark is sold.

This has been a HumblePod Production. Stay humble.