Episode Summary
Talia Nassi is a developer advocate at Split, a platform that combines feature flags and data to accelerate dev workflows. Prior to this role, she worked as a software engineer at WeWork, a QA engineer at Forbes, and a QA engineer at Visa. Talia graduated from UC San Diego in 2016, earning a bachelor of science degree in cognitive science with a specialization in human-computer interaction.
Join Corey and Talia on how Split Software helps dev teams create better software faster, the two characteristics successful developer advocates need, how developer advocates close the feedback loop and enable teams to continuously improve products, how dev advocates can help small startups and large organizations, how it’s hard to quantify success as a developer advocate and why that might not really matter, the journey Talia took that led to speaking at conferences and meetups around the world, the important role humor plays in a successful conference talk, and more.
Episode Show Notes & Transcript
About Talia
Talia Nassi is an international keynote speaker who delivers content on all things testing and quality. She is a developer advocate at Split.io where she works closely with engineering teams globally to ship software more efficiently. She is passionate about feature flagging, canary launches, CI/CD, testing in production, and A/B testing. She has spoken at countless conferences internationally, ranging from audiences of 100 to 4000!
Links:
- Split Software: https://www.split.io/
- Flagship Conference: https://flagshipconf.com/
- Twitter: https://twitter.com/talia_nassi
Transcript
Announcer: Hello, and welcome to Screaming in the Cloud with your host, Cloud Economist Corey Quinn. This weekly show features conversations with people doing interesting work in the world of Cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.
Corey: If your mean time to WTF for a security alert is more than a minute, it's time to look at Lacework. Lacework will help you get your security act together for everything from compliance service configurations to container app relationships, all without the need for PhDs in AWS to write the rules. If you're building a secure business on AWS with compliance requirements, you don't really have time to choose between antivirus or firewall companies to help you secure your stack. That's why Lacework is built from the ground up for the Cloud: low effort, high visibility and detection. To learn more, visit lacework.com.
Corey: The apps on cloud summit is a new action packed, not a conference, happening May 11th through 13th online. Its for everyone who makes applications in the cloud run screaming. From IT leaders to DevOps pros to you folks, whoever you might be. Take a break from screaming into the cloudy void with me to learn from some of the best of people who actually know what they’re doing. Like Kelsey Hightower, AWS blogger John Meyer, and also me, because apparently they didn’t listen to me saying I had no idea what I was doing. Register now at turbonomic.com/screaming. Theres a “swag box” ready to ship for the first two thousand registrants, so you don’t want to miss this. Thanks for Turbonomic for sponsoring this ridiculous podcast.
Corey: If your mean time to WTF for a security alert is more than a minute, it's time to look at Lacework. Lacework will help you get your security act together for everything from compliance service configurations to container app relationships, all without the need for PhDs in AWS to write the rules. If you're building a secure business on AWS with compliance requirements, you don't really have time to choose between antivirus or firewall companies to help you secure your stack. That's why Lacework is built from the ground up for the Cloud: low effort, high visibility and detection. To learn more, visit lacework.com.
Corey: The apps on cloud summit is a new action packed, not a conference, happening May 11th through 13th online. Its for everyone who makes applications in the cloud run screaming. From IT leaders to DevOps pros to you folks, whoever you might be. Take a break from screaming into the cloudy void with me to learn from some of the best of people who actually know what they’re doing. Like Kelsey Hightower, AWS blogger John Meyer, and also me, because apparently they didn’t listen to me saying I had no idea what I was doing. Register now at turbonomic.com/screaming. Theres a “swag box” ready to ship for the first two thousand registrants, so you don’t want to miss this. Thanks for Turbonomic for sponsoring this ridiculous podcast.
Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. I'm joined this week by Talia Nassi, who's a developer advocate at Split Software. Talia, thanks for joining me.
Talia: Thanks for having me. I'm excited to be here.
Corey: So, let's start at the beginning. What is a Split Software? And please don't tell me the answer is that there's two of them.
Talia: Split Software is a feature flagging and experimentation platform that allows you to create better software for your teams so you can gain insight into what your customers are doing with the use of feature flags. And Split provides all of that inside of their platform.
Corey: So, in other words, it's more or less the ability to enable certain things on the fly, disable them on the fly without having to, for example, do a whole new code deployment?
Talia: Right, exactly. And it makes the release process just a lot faster and more seamless because you don't have to have a big release, now, you can turn on a button or flip a switch and that feature will be on.
Corey: One of the earliest tech presentations I saw—and it feels like it was probably almost a decade ago—-, was talking about feature flags. It was from one of the big well-known tech companies. I don't even recall which one, which tells you exactly how well that branding thing worked. But it felt like, “Oh, this feels like something from the future that the big tech companies will do, and maybe someday a few other folks will be using it.” And that's kind of the really the last time I went deep dive, checking into that sort of thing. But looking at Split's website, for example, I wind up seeing a whole bunch of logos that I recognize, which tells me it has sort of come down from the ivory tower of the big well-known tech places and into something that, you know, human beings can use.
Talia: Yeah. Yeah, absolutely. So, we do have a ton of integrations with Google Analytics, and mParticle, and Amplitude. And then we have a bunch of supported SDKs, like, all of the main ones. I do a lot of tutorials in Python, and React, and JavaScript, but we have so many different places to start if you're interested in using our tools.
Corey: So, developer advocacy is one of those areas that means a lot of different things to a lot of different people. I mean the running joke is, what, get three DevRel folks in the room and ask what DevRel is, and you'll wind up getting eight answers at least. And again, my expression of it is usually aggressively shitposting on Twitter aimed at trillion-dollar companies. But everyone finds that they tend to embody different aspects of it. So, it sounds like a weird college entrance essay question, but what does developer advocacy mean to you?
Talia: It means someone who's an—I don't want to say an advocate for the developers, but it's someone who the developers can trust. And there's a few parts to that. So, the first part is just having some sort of engineering credibility. So, I started as an engineer, and I was a software engineer in test for five years before becoming a dev advocate. So, I do have a foundation of software engineering.
And so the second thing, I think, is dev advocates are developers at heart, so they're not going to try to sell you on a tool. Like, I don't sell Split; we have a whole sales team that deals with that. But my role is to make it easier for the developers to use Split. So, I work on things like documentation, and I work on video tutorials, and code tutorials, and create sample apps, and just make it so that if anyone wants to use Split, it's easy for them and they have a really great experience.
Corey: At some level, you would think from the naive objective perspective that making sure developers have a great experience is the entire role of product and then, in theory, the engineers that build it. One thing that I've learned through fronting a small company of my own, is that every aspect of business is way more complicated than it looks from the outside, even at tiny scale, and you folks are larger than that. Where does the breakdown fall? Because it's easy to say that, “Oh, product should wind up making sure that the developer experience is a delight.” But if that were true and completely inalienable, then developer advocacy wouldn't exist because product would already handle it? Am I right on that? Am I missing something? Or is there some nuance that's, that’s—
Talia: You’re right. So, basically, what dev advocacy does is it finishes the feedback loop. So, you have product and engineers who create the product, and then you have developers who use the product. But then if there's issues, or problems, or things that can be improved, that feedback needs to somehow get back to the product team. So, I think what dev advocacy does is it fills in this cycle of when something goes wrong, or if something should be improved, or if there is a typo in the documentation or something isn't clear, that feedback all needs to get back to product to continuously improve, and that’s, I think, where dev advocacy comes into that cycle.
Corey: It's one of the I guess, big debates of DevRel across the industry has been, well, what organization does it belong in? At some level, it seems that some folks spend more time talking about what DevRel isn't than what it is: “Oh, we're absolutely not sales.” I agree with that. “We're absolutely not marketing.” Well, I have challenges with that approach. “We're not product.” Well, okay. Yes, but no. “We're not engineering.” Well, really? You write an awful lot of code for someone who's not engineering. And so on and so forth. It feels almost like it's an intersection, and it feels like it's a very nebulous thing to define.
Talia: Yeah, it is. And I think with every company, it's different. I think, in an ideal world, DevRel would just be its own division within a company, not with marketing, not with engineering, it would just be freestanding on its own. And it does combine a lot of engineering and a lot of marketing, but what you’re marketing is not the product, you’re marketing code, and tutorials, and things like that. You're not marketing, like, the actual product.
So, I think there is an intersection, obviously, between product and engineering and DevRel, but in terms of where it should belong in a company, ideally, I feel like it should be its own division.
Corey: The challenge, of course, is from a business strategy perspective. When it's nebulous, it's difficult to assign value and determine appropriate level of investment in it. Back in the before times, for example, it was always tricky to articulate the value. “So, let me get this straight. You cost as much as an engineer, and you spend about that much again in travel to go to conferences that look like giant parties from where we sit. And I talked to the VP of DevRel, whatever that position happens to be, and they say that no, you can't tie a sales quota to you folks. And okay. And I can't tie you to all the metrics I suggest, like, inbound leads are bad. So, without anything I can use to evaluate performance of whether someone is outperforming or underperforming in the group, what happens if I just let the entire group go?”
And you see, in some cases when the pandemic hit, that's exactly what some companies did. I want to disclaim my own biases here. I believe that is a mistake, but I'm curious to get your perspective.
Talia: Yeah, there's companies that really benefit from having dev advocates, and I think those are the startups as well as the big companies. So, Split, for example, I think our team benefits from me because I'm so wonderful. I think we benefit because when developers have questions and when they need tutorials, that's where I come in. And they know that, “Okay, she's not trying to sell us anything, she really just wants us to have such a great experience.” And then there's cloud advocates like in AWS and in Microsoft, and they provide kind of the same tooling and support that we do at smaller companies. So, I think it is beneficial, but you just have to know how to do it the right way.
Corey: Back when I was on the speaking circuit—again in the before times—again, I was an independent consultant for a few years and then I wound up taking on a business partner and expanding, and one of the things I went through with taking on a business partner, who himself is engaged publicly. He's an O'Reilly author. Good for him. But there was a bit of where, how do we quantify the value of me going around and speaking at these events? And the answer that we ultimately came to—and I'm not suggesting this is perfect, ideal, or even good, if I'm being honest, was that we don’t. I know that on some level, if I go out, and I talk about the things that I do in front of an audience, good things happen. But, first, it's impossible to quantify and, two, attribution will drive you out of your mind if you let it.
Talia: Yeah, I agree with you. And I don't think that there's any way to quantify this role because you're not measuring the amount of leads because you're not doing sales, and you can't really quantify the traffic that comes to your site because it might not lead to a quantifiable number of sales. So, I really don't think there's a way to quantify success in the role, it's more of just the quality of what you're doing and how you're engaging with developers, and yeah, things like that.
Corey: And it's challenging in the context of larger organizations because as you start expanding your developer advocacy groups and your developer relations functions, they get—I don't mean to be unkind—pretty expensive at some point. And when you're investing vast seas of money and figuring out where to allocate that, and it's, “Well, this group makes good things happen, but we can't really define it,” isn't good enough from a executive level, the only reason I'm able to get away with it here is because I own half the company and cool, I can basically say, “We're going to do this,” and there isn't a whole lot of recourse. It's the first time in my career where I don't actually have to worry about getting fired. Most people don't have that particular luxury.
Talia: [laugh]. Yeah, the way that would be ideal is if the numbers didn't matter. But if for example, I put up a tutorial on setting up feature flags with Node and React, and then the next day, we see there's a lot of activity on the Node and React documentation pages, and we know that people are building these apps, I mean, it's something that you could use to provide value, but in terms of the quantifiable number, there's not a lot that you can correlate.
Corey: I don't have any answers for this. It's one of those areas that's just difficult to look at because a lot of my friends and associates are in the DevRel space and this is a common discussion. And I'm increasingly of the opinion that no one has really solved this, but I know that taking a step back, companies that have invested heavily in developer advocacy do well in this market when it's executed appropriately, and others who have cut back find themselves floundering, and there's enough data points to make it pretty clear what the right direction is. So, let's talk less than the abstract and more about you personally. There are an awful lot of things that DevRel can encompass. What parts of it do you do? What parts of it do you not do? Where do you start? Where do you stop?
Talia: One of the main things I do is I speak at conferences and meetups, and right now everything's virtual, but I speak at events, and I write blog posts and code tutorials, and I create code samples and sample apps. I also host a monthly roundtable with our developers, so anyone can join and talk about any roadblocks they had or implementation things that went wrong that we can improve. And then we also have community Slack channel where developers can talk and I can also answer questions there. And then—I think at the core of all of this, is I teach. And I think developer advocates, they should be, I think, really great teachers.
So, you're teaching different things to do with the tools that you're advocating for and the products that you're advocating for. So, that's kind of what I do. And I got started with it at my previous company. I was a software engineer at WeWork. And I was doing this really cool thing, and someone suggested, “Hey, you should do a tech talk because this is something really cool.”
And so I did a tech talk internally for the company. And then someone suggested, “Hey this is really great. You should submit it to a conference.” So, I submitted it to speak at a conference, and from there, I just kept getting invitations to more conferences and meetups, and it kind of just skyrocketed from there. And I think I realized that the part of my role that I was missing from being an engineer was this external PR teaching side of dev advocacy. So, now I'm a dev advocate, and I love what I do.
Corey: I adore the way that you started that with the idea of being a teacher. And you're right. One of the most transformative contracting projects I ever did was—must have been six, seven years ago now—where I spent a summer as a traveling trainer teaching people how to use Puppet. And that was fascinating to me from a perspective of… first you have a roomful of twenty people who are within punching-you distance, and you have three days to teach them on this thing. Secondly, they spent not small money to be there, so they're expecting an outcome.
And in many cases, some of them are kind of angry, where there's a perception that, “Oh, this software is coming to take my job away.” So, they're already coming at this from a weird place, then, of course, it's software. And computers are notorious for doing what they do best, and that's breaking. So, at some point, you'll do a demo and it fails, and good luck. Have fun.
Oh, by the way, if they wind up yelling at the company and demand their money back, you don't have a job anymore. So, okay, no pressure. I'm not saying that's the way to learn to speak publicly, but it's one of those drown or swim moments. And that really informed a lot. It was a rapid evolution.
The first class I gave, I was terrified; the second one is I got this; the third one is I don't got this; and by seven, it started to wind up getting rote, and I started experimenting more and being more genuine, and it worked super well as I went down that process.
Talia: I think it takes a specific skill to be a great teacher. And it's one that is really important in being a dev advocate is that you need to be able to teach without belittling and be able to teach people with different backgrounds. Some people who go through our tutorials have twenty years of coding experience and some people have two weeks of experience. So, I think creating content and connecting with people with all different backgrounds is really important.
Corey: There's another thing I learned by doing this was I gave a talk once the early version of it was a talk on Git. And I wanted to go super deep—honestly, I wouldn't recommend this strategy to anyone—but I wanted to learn Git better, so it's all right, how do I force myself to learn Git? I know, I'll sign up to give a talk about it in three months. [laugh]. That's what we call a forcing function because they won't reschedule the conference if you run out of time. I know this; I checked.
And where, time to learn Git. And the first iteration of that talk would have been hilarious to the five people who maintain Git, but it was super deep in the weeds. And I wound up taking a look at this and figured, you know, let's make this more broadly accessible. And it started off by even explaining what Git was. And yeah, there were jokes for all experience levels scattered throughout it, but the lesson I took from that is, it has never been a bad idea to make content more accessible to people.
At some point, you do have to draw a line somewhere. I mean, when I wind up doing content about AWS, I don't start by explaining what AWS is every time; that would get very tiring. But the idea of reducing the prerequisites that are required in order to understand the context of what's happening is very important. And there's no perfect answer. Sometimes there’s—if people don't understand an environment, you aren’t going to be able to teach them effectively because they don't have the fundamentals. So, dialing that in is very important, but given the choice, I will always trend towards being more accessible to more people.
Talia: Yeah, yeah. And I think it also depends on your audience. If you're speaking at a meetup that is college students, and people who don't have twenty years of coding experience, you might want to add those introductory steps. But if you're speaking to all the architects at Google, like, maybe don't tell them how to create a React app.
Corey: No, no, when you architects at Google, they like to get on stage and tell everyone else how they're doing it wrong. There’s a little thing called context, it's, “Yeah, your company has invested tens of billions of dollars in your developer infrastructure. We have about, ehh, eight bucks.” So, yeah, Google scale solutions do not necessarily map to others. And in fact, that's one of the things that started to irritate me and I started getting louder and louder about is when you give a talk about how you do things in the right way to proceed with things and people leave the room feeling bad because what they've built looks nowhere near that good, you've kind of failed.
The theme that I always like to go with is what you're doing is great. Now, here's how you make it even better. And that's an uplifting next step, brighter path, brighter future story. I used to think that there was something wrong with me when I would leave a talk feeling bad. And having done this enough, I'm firmly of the opinion that no, no. That was a bad talk.
Talia: Yeah. And I think you're absolutely right. I think when you leave a talk, you should feel empowered. Like, “Wow, I want to go do this thing that I just learned about and I didn't know that I could do it, and that it was so easy, and this person just enlightened me, and I just want to go do it now.” I think that's how people should be leaving talks that they hear.
Corey: This episode is sponsored by ExtraHop. ExtraHop provides threat detection and response for the Enterprise (not the starship). On-prem security doesn’t translate well to cloud or multi-cloud environments, and that’s not even counting IoT. ExtraHop automatically discovers everything inside the perimeter, including your cloud workloads and IoT devices, detects these threats up to 35 percent faster, and helps you act immediately. Ask for a free trial of detection and response for AWS today at extrahop.com/trial.
Corey: The whole point is to be uplifting and inspirational and fun. And not everyone is going to have the same approach. Other people's material that's about as well as other people's shoes. But what I find is that my way of grabbing people's attention is humor. If I can make people laugh, I have their attention and then I can teach them something.
Now, some people don't have that particular approach. Their humor doesn't work in that way, or that's not how they contextualize things, they don't know how to tell a joke, or—worst case failure mode—they confuse being funny with being actively insulting. And that doesn't fly. But that's always been my approach. I've turned them into sort of borderline stand up comedy, and everyone thinks, “Oh, that's what you have to do to give a talk?” No, that's my personality defect that I just managed to turn into something of a strength.
Talia: Yeah, I totally agree. Being humorous onstage. And honestly, mostly, what I do that works for me is I just make fun of myself. And that works really well. The other thing is, I'm like a very sarcastic person, and it comes out in my talks.
And so yeah, I think humor is a great place to start. It's been rough during COVID because when I give talks, everyone's muted and I can't tell if they're laughing at my jokes. So, sometimes I'll tell people, if you laugh at my jokes, just unmute yourself, [laugh] so I can hear you.
Corey: Oh, and the delay, too. It turns out that there are a couple of things that are baked into human behavior. I discovered this once giving a talk to the silent disco type of talk. For those who are not familiar because it’s a weird term, there are—usually a big open room, and there are three stages next to each other, and all of the audience for each talk is wearing a different colored set of headphones. So, you're basically whispering into people's ears, and you speak in a normal voice onstage in front of everyone.
But when you're sitting in a room wearing headphones, there's a societal conditioning thing of you don't laugh when someone says something funny. I mean, otherwise you turn into the person who just starts bursting out laughing at nothing while riding the train, and suddenly, oh, you're that person. So, it's challenging when you tell a joke that you know is good and you can see people smiling and laughing, but there's no visible feedback. And video—or recorded video—makes that even worse.
Talia: Yeah. I mean, it's hard for a first talk that you don't know what the audience reaction is going to be. But there's a couple of talks that I've done so many times, and I know that I'm being funny, in certain points.
Corey: So, how have you found that your approach to DevRel has shifted since the before times where now it's, “Hey, do you want to come and hang out in a big room with other people?” “Hell, no.” It's definitely forcing rethinking of a lot of this. What have you been doing?
Talia: Yeah, I feel like right now, there's just more of an emphasis on digital experiences because everything is online from COVID. And I feel like the tech world is just booming right now, so I think, yeah, there's just an emphasis on everything being digital. And I don't know that it necessarily has an effect on my role, specifically. And the only thing that I can think of is that travel is obviously limited right now, and all the events are virtual.
Corey: There's an awful lot of things that a virtual event lets you do, and just as many things it doesn't let you do. And in some ways, it feels like people are still stumbling through figuring this out. It turns out that when people are in Zoom meetings all day long, they don't want to open up Zoom to attend the conference. It's, “Oh, cool. It's just another meeting.”
Anything that doesn't have a strong social channel where you can talk to peers—which frankly, from my perspective has always been the best part of any conference I've gone to—is a problem. There are a lot of weird and broken approaches, and the technology isn't quite there in some respects, and everyone's trying to do the same thing, and, “Oh, we can do digital events. Okay, we'll make it three weeks long.” Why? How much attention do you think that people have?
People don't want to attend online conferences three days a week, as it turns out. So, it's a matter of almost separating signal from noise. I don't have any answers here. It's just a painful problem I keep seeing.
Talia: Yeah, I think you just have to choose your conferences wisely and choose what you go to wisely because there are a lot of virtual events right now, and it can be a little bit draining if you decide to go to as many as you want. I feel like you should just choose wisely, go to the ones that you really feel like will be beneficial for you, and don't go just so that you can say, “Oh, I'm not working right now.” Split has a user conference coming up March 16th and 17th. It's called Flagship, and it's just a two-day interactive virtual conference where we'll have success stories about progressive delivery, and best practices of experimentation, and just, like industry trends. And so we have speakers from Google, and LinkedIn, and ServiceNow. So, that's happening in March.
Corey: Excellent. We will of course put a link to that in the [show notes 00:24:27] because honestly, attending conferences we get actual value out of them is super important. The painful part I've always found is how do you figure out which one it is? And I hate to be judgy but I'm going to do it: the more enterprise-y it looks and the more it looks like you're reading an airport ad billboard, invariably the crappier the conference is going to turn out to be. I'm sorry, but that's how I always feel when I look at this stuff. And credit where due, I don't see that Split is falling into that trap. Hazzah.
Talia: [laugh]. Yeah, it should be a good conference. So, we'll see what happens. I'm doing a workshop at the conference on setting up feature flags. So, yeah, we'll see how it goes.
Corey: It's one of those things I keep wanting to get into. Let's actually talk a little bit about that. Feature flags are something I keep meaning to look into, which makes me feel better about not having really looked into them in any meaningful way because it feels like it needs things that I don't actually do. For example, you know, testing my code. What are the prerequisites for making feature flags something someone should care about?
Talia: There aren't any prerequisites for using feature flags. You just have to have a stable app. But once you have the app, there's so many things that feature flags allow you to do, so things like testing in production, and A/B testing, and canary releases, and percentage rollouts, and things that without feature flags would be so much harder to implement. So, I think it just makes your development much easier, and Split takes care of all of that for you.
Corey: So, testing in production is one of those things that is very frequently talked about, but it feels almost like it's chaos engineering-focused where, it sounds, for the first time you hear about it, like, an absolutely terrible idea. And then it once you look into it, no, that actually makes an awful lot of sense, but it feels like you have to educate the customer before they see the value of it. And that always felt scary to me.
Talia: Yeah. So, one of the things that I talk about—and this was actually the first talk I ever did—was testing in production because it was something that I implemented end-to-end at WeWork. And so testing in production can be really scary because you're running tests in production, you could affect real end-users, you can break things in production. But the great thing is that when you use feature flags for testing in production, you reduce the risk of anything going wrong. So, what I mean by that is you target your internal teammates inside of the feature flag.
And what that means is that you basically create a list of people, and you say only these people can see this new feature. And then you use that list of people and say, “I'm going to test my feature with just these people.” So, now your developers, and your QA, and your product people can go in and see the feature in production, test it out manually, and then turn the feature flag on once it's ready to be turned on. So, basically, you're not using a staging environment or a dummy environment; you're using production because that's where your feature is going to live. Your users aren't going to log into staging to see your feature, they're going to log into production. So, I think people do get scared, but using feature flags is a great way to mitigate that risk because if something does go wrong, your end users won't be affected. It'll just be your internal users, which are your devs, and your product, and your QA.
Corey: Yeah, it's something you really need to get the entire team on board with in some respects. The other piece of it though, is by testing in production, on some level, you at least have the potential to inherently degrade the experience for your customers or your users. At least I have a hard time seeing how that wouldn't be the case.
Talia: So, what you're actually doing is you're making the customer's experience much better. You're making sure that the features work before your customers can see it. So, if I'm releasing a feature, I want to make sure that it works in production, not in staging. So, you just want to make sure that you're doing it safely with feature flags and you have it implemented correctly, but at the end of the day, you're providing a really great user experience.
Corey: Yeah, there is something to be said for accepting a bit of short term pain in favor of making it better for everyone longer term. And I suppose that’s part of the reason why chaos engineering came out of Netflix as opposed to other places, where the failure mode of degrading a particular user's Netflix stream and having them have to restart it or whatnot, isn't super high. As long as you're not continually testing on that same one user. And if you are, honestly, the idea of having a canary list populated with people you don't like is amazing, but at that point, the stakes aren't super high. It works when you're talking about streaming movies. It feels a lot more challenging when you're doing feature enhancements on someone's bank account.
Talia: Yeah. So, we definitely don't recommend using real users to test in production, what I recommend is using test users that are in production. So, the same way that you would log in to Netflix and create an account, so you kind of do the same thing for test account. So, you just create an account in production that's used for testing that acts as a real user, but is not a real user. We had, like, automation bot after automation bot in production, and we would know that whenever any data comes in from these bots that they're actually testing in production and that they're not real users.
And so we don't use actual users when we're doing testing. We're using test users that act like real users and look like real users, but they have this identification of being a test user. We use things like a back end flagging system to identify all test users and test entities in production. So, just some sort of Boolean that's ‘is test equals true?’ Or ‘is test user equals true?’ And that's how we would differentiate test data and real data in production.
Corey: One line that I liked about this was that everyone has a test environment; some people also have an environment where production lives separately. And on one level or another, you're always going to be testing your code, just can you do it in a way that doesn't cause serious damage or harm to users? And getting people onto that path is challenging.
Talia: It is. And I think that everyone has experiences of testing in a staging environment, or a QA environment where they tested their features and it worked so great in staging and they signed off, and then as soon as they pushed to production, there's an issue or something goes wrong. And I think using those experiences to get people on board is a really great approach.
Corey: So, thank you so much for taking the time to speak with me today. If people want to learn more about who you are, or what you do, what you're up to, where can they find you?
Talia: Oh, you can find me on Twitter. I'm @talia_nassi. And yeah, and I hope that you guys come to Flagship in March.
Corey: Excellent. We'll of course put links to those things in the [show notes 00:24:27]. Thanks so much for joining me. I really appreciate it.
Talia: Thanks for having me. This was great.
Corey: It really was. Talia Nassi, developer advocate at Split Software. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an insulting comment telling me that you hope someone tests in my production.
Announcer: This has been this week’s episode of Screaming in the Cloud. You can also find more Corey at screaminginthecloud.com, or wherever fine snark is sold.
This has been a HumblePod production. Stay humble.