Kicking Off a Tech Career as a Preteen with Alex Zenla

• Alex on Bluesky: https://bsky.app/profile/alex.zenla.io

Transcript

Alex Zenla: Long story short, I got involved in open source with lots of great projects, including the Dart Project and Chromium and Chromium OS and worked on particularly Chromium OS where I was building OS images and stuff for fun. But what was interesting was I was very young. I was 12 years old. And, you could not have a Google account then legally.

I think it's still the case that if you're under 13, you're not legally supposed to have a Google account.

Corey Quinn: Yeah, that's always stopped, kids. You must be 18 to view this website. Oh, okay, never mind.

Welcome to Screaming in the Cloud. I'm Corey Quinn. I'm joined today by Alex Zenla, who is the founder and CTO at Edera. Alex, thank you for joining me. Yeah, thank you for having me. I'm very excited to chat today.

Sponsor: This episode is sponsored in part by my day job, the Duckbill Group. Do you have a horrifying AWS bill? That can mean a lot of things.

Predicting what it's going to be. Determining what it should be. Negotiating your next long-term contract with AWS. Or just figuring out why it increasingly resembles a phone number, but nobody seems to quite know why that is. To learn more, visit duckbillgroup.com. Remember, you can't duck the duck bill, Bill.

And my CEO informs me that is absolutely not our slogan.

Corey Quinn: Let's start with the, with the basics. What is an "Edera" and what might it do?

Alex Zenla: Great question. So Edera is Italian for "Ivy." the, the, that's where the name comes from, and at Edera, we build, products that help you secure your infrastructure. And it's secured by design fashion, but also give you great efficiency and, allow you to kind of utilize your, your infrastructure more efficiently. So we, we started with a, product called Protect Kubernetes that provides virtual machines, even without hardware isolation.

So that you can run virtual mach-containers inside virtual machines, even with software isolation might have heard something like gVisor. We basically do what gVisor does, but we do it more like Kata does than what gVisor does.

Corey Quinn: Yeah. So when I take a look at the space, I, I am by no means a security expert.

I'm just noisy. And while those things can look alike, in reality, I tend to bias for trusting people who are better at these things than I am. And when all the majors come out and have been saying for years that the container level is not a sufficient level of security isolation as far as boundaries go, I've tended to take that relatively seriously.

So in the workloads that I'm used to, Okay, yeah, you're going to be running a bunch of Docker containers within a host that you control. The multi tenancy happens at the hypervisor level in most cases, or if you're using it somewhere else in AWS's universe, Firecracker tends to be their expression of this.

Who is the target customer for this?

Alex Zenla: Yeah, so the target customer is someone who wants to get that isolation that Firecracker or something like that could provide, Kata containers being the most famous one, but does not want to pay for the expensive metal instances on AWS that are required to get nested virtualization.

So the biggest challenge with deploying virtual machine based container isolation is that hardware requirement of virtualization. It turns out that interestingly, as you may know, in AWS history, they utilize this technology called Zen historically. One of the things that Zen did really well is that it, did not require hardware virtualization in order to run the Linux kernel in an isolated environment.

So we took that technology and basically adapted it. to be able to run containers, securely, without using hardware isolate or hardware virtualization itself.

Corey Quinn: So you're running this on hypothetically an EC2 instance running KVM, then you wind up rolling out Xen on top of that on some level where you have the DOM0, which is the guest tenant with special hardware privileges.

Hardware, of course, in this context, being a little hand wavy. Then you have various DOMUs that wind up running on top of that. You're a wrapper around that, that type of approach.

Alex Zenla: Yes. So the way that this came to be, was actually from my experience in IoT and running containers at the edge, and we had some problems with the multiple vendors basically needed, needing to run their application on a single hardware device.

We only had one per place where we were installing these, and so I was looking for technologies that could do that really well. And Zen happened to do that extremely well without the need for hardware virtualization. The story of why we don't require hardware virtualization is that we would acquire these industrial IoT devices.

Some of them would have hardware virtualization enabled in the BIOS. Some would not because they were being shipped directly from the factory. We could not intercept them to edit the BIOS, they were just being shipped with our OS image.

Corey Quinn: And talking end users through modifying BIOS settings is, for those who have not done it, it's a good way to kill an afternoon.

Alex Zenla: Yes, exactly. And the people who install these things are electrical engineers who do know things like serial. They know how to run serial lines. They roughly know what a COM port is and all of these things. They know a little bit about networking, but BIOS, you can't rely on them to have a monitor to be able to plug in to these things.

Corey Quinn: You can't necessarily rely on BIOS to be consistent from week to week, as these things tend to get upgraded in various batches that get shipped out too. So it always seems to turn into a bit of a guessing game. And this does not scale. It just doesn't.

Alex Zenla: It does not scale at all. So what we found is that Zen is perfect for solving this problem.

So you're right. What we do is, we allow you to basically take your existing, EC2, machine images, AMIs, and you can basically run our installer. And it'll boot Xen, it'll boot your entire operating system into Xen, and then we provide a different CRI implementation for Kubernetes that allows you to then run your containers inside Xen as DOMUs.

But we have an easy on RAM for that, so you apply just a single runtime class, and you're able to deploy your containers that way.

Corey Quinn: Where's the DOMU boundary at that point? Is that per namespace? Is it per cluster? Is it per container?

Alex Zenla: Good question. It's, you get to choose. We call these Domuse Zones, which are, are, are a term for basically an isolation boundary.

We run a minimal Linux kernel in there, and then a very small Rust-based init. And, which by the way, the entire platform is written in Rust. One of the cool things that we did is, reimplemented a lot of Zen and Rust. So everything that runs in user space is Rust based. So because we run that piece inside that zone, we can actually implement multiple container support and provide the standard kind of namespace based isolation, Linux namespace.

Isolation, which allows you to then choose how you want to compose your containers on your system. So, for example, you could discriminate what containers to run where by saying a tag or an annotation in Kubernetes of a particular customer, like a customer ID, you could specify it by namespace or by default, we give you the most secure option, which is running every Kubernetes pod in a separate zone.

Corey Quinn: What is the overhead look like on this? Because things like this have been done before back in the ancient times before Docker was a thing. I know such a thing did happen for those of my younger listeners. The challenge with doing that was twofold. First, the instantiation time was generally non trivial.

And two, it wound up having an awful lot of overhead to run a bunch of those on a single system. You would lose some amount of resource to the hypervisor layer.

Alex Zenla: Yeah, great question. So, First, I'll plug our research paper into this that is fantastic. We have a wonderful researcher named Marina Moore who worked on TUF and other things like that.

She runs Edera Research and we co authored a really great paper about the performance and the the overhead. So the, the startup time is roughly 500 milliseconds on, on normal hardware. So we're able to go from nothing to container in 500 milliseconds. We think that we can even reduce that further, but this is just kind of where we're starting as a base, and we're working directly with Zen.

Corey Quinn: Half-a-second, but I want it now.

Alex Zenla: Yes, exactly. It's pretty fast. In fact, if you look at like how fast Kubernetes can even schedule a pod, it's kind of minimal in that aspect. Importantly, you can run our zones and workloads, which is what we call containers without Kubernetes as well. So, you can also kind of run it as some, somewhat of a Docker stand in. In terms of the overhead and, on IO and, CPU and, memory performance and those kinds of things, we have some really good benchmarks that show that we are roughly equal to Docker on bare metal. This is very interesting, and when you look at how the hypervisor works, it's pretty impressive. The best way that I like to think about it is that because we run Xen at that layer, we get ultimate kind of visibility into CPU scheduling, into memory management, and that kind of thing.

We, I believe our current overhead is like 10 megabytes per zone, and we think we can do even better there. We're actively working on projects that, focus on reducing memory overhead so that you can pack a lot of these things. On my personal desktop, as an example, which I think has, 64 gigs of RAM, I've run, 1500 of these independent zones running a very simple Go app.

So it can be very powerful.

Corey Quinn: That's impressive. I mean, 64 gigs is almost enough to run, two instances of Slack at the same time, but that's a separate argument there.

One of the challenges historically with a lot of this has been that dealing with container isolation has felt like it's a constant game of whack a mole, where, okay, we fix this issue, and then 20 minutes later, another one seems to come out, and for enterprise scale, that just doesn't work.

That. You're always basically on the trailing edge, regardless of how quickly you can iterate forward. I like that it sort of sidesteps that entire problem.

Alex Zenla: Yes, exactly. I watched one of your podcasts before, and you had the great comment about the CNCF landscape. You know, it's very complicated, that, that thing.

I want to get a poster on it of it on my wall.

Corey Quinn: I have one that was turned into a thousand piece jigsaw puzzle that my business partner gave me. We put that together for three days. My wife and I, it hangs, now glued, behind a frame, in my home office just because that thing is a freaking nightmare. One of the best tweets I ever saw, and that was a picture of it simply captioned "Jesus Christ."

It was, it was, it's horrifying.

Alex Zenla: It is horrifying, and you know, one of the things that I really struggled with when, when starting Edera was, "Am I just going to be another one of those tiles?" And I struggled with that. I was like, "Should I really be doing this? Should I be adding to the tools and the things in the CNCF landscape?"

Or, you know, "What am I doing here, really, if I'm just adding another layer?" And what I determined very quickly is, you know, someone's got to build a better foundation. We just got to do it. I really struggle with our current set of tools and how many layers there are. There's like four-to-five independent, roughly independent projects involved with running a container on Kubernetes.

And I think that's just in terms of executables. We're not even talking about dependencies. And I, I think that's like kind of scary. I want there to be a better way to run a container today, and I think we have a very interesting way of doing that, where you can run, basically everything from Kubernetes to the container in our platform is, is one single cohesive thing.

I think that's super cool.

Corey Quinn: Yeah, I just realized this is where Ariadne works. Yeah, as your distinguished engineer, if, if she says it's good, it's good. That I, I have no further questions on the security posture of this system. My God.

Alex Zenla: Yes. Ariadne is a great friend. We started this company together initially.

And then, the former CEO of Chain Guard, Emily, our CEO now joined us. And, we, we have an amazing team. I would put our team up against anyone. It's a tough world out there, and not just for startups, but for people. And I think, you know, we have such a great supportive environment and Ariadne just adds to that.

Absolutely.

Corey Quinn: I do have a, I guess a question I'd like to ask folks who have been. I guess doing this in a different era than I came up through, what is your history with technology? Where did you come from?

Alex Zenla: Great question. It's always hard to not go too in depth because one day I want to write a biography about it.

Yeah. So I grew up in Alabama. I was in a small town in Alabama. Auburn, Alabama. It is actually Phenix City, Alabama, but that's even more obscure than, than Auburn, Alabama. Both of my parents were teachers. My dad worked in, universities and my mom was a guidance counselor at an Alabama school. If you don't know the Alabama public school system, I did not have a lot of money.

My parents did not make a lot of money. So I did not have computers really growing up. I had a family computer, but I, you know, it was a tool. It wasn't something that I got to spend time with. So when I was like 11-years-old, well, I guess I should say my father working at university really got me interested in computers because I got to go play with all the teacher technology and everything there, including a VMware cluster that they had running at the university, which was kind of my first, like, entry into hypervisor stuff and all of that. Long story short, I got into open source very young with the Ubuntu 2.0 Project, and I was a packager. I did not have the money for Windows, so I was using Linux. I had a hand me down computer that I got from someone from my church. Like, literally, just like, it had a virus on it, they didn't know what to do. I got a computer from that. Long story short, I got involved in open source with, lots of great projects, including the Dart project and Chromium and Chromium OS and worked on particularly Chromium OS, where I was building OS images and stuff for fun.

But what was interesting was I was very young. I was 12-years-old. And you could not have a Google account then legally. I think it's still the case that if you're under 13, you're not legally supposed to have a Google account.

Corey Quinn: Yeah, that's always stopped, kids. "You must be 18 to view this website." Oh, okay, never mind.

Sponsor: This episode is sponsored by my own company, The Duckbill Group. Having trouble with your AWS bill? Perhaps it's time to renegotiate a contract with them. Maybe you're just wondering how to predict what's going on in the wide world of AWS. Well, that's where The Duckbill Group comes in to help.

Remember, you can't duck the Duckbill bill, which I am reliably informed by my business partner, is absolutely not our motto.

Alex Zenla: Yeah, so it did not stop me, that's for sure. So somehow on a Google group, on Chrome OS related stuff, I posted my age and a community manager at Google found out this resulted in a call from a Google lawyer, as well as a community manager to my home phone. I was freaking out, didn't know what was happening.

Corey Quinn: For you or like, do you like a legal guardian that we can get on the phone too?

Because if we talk to kids, there's a problem here and a different direction. Yeah, yeah.

Alex Zenla: It was my parents, but I think my parents were just like, "What did you do? Like, what did you do?" and, anyway, the Google lawyer comes on and says, "You're not allowed to have a Google account. We have to remove your Google account.

However, we love, you know, your involvement in the community. We would like to invite you to Google I/O." So in 2012, I got to go to Google I/O, all expenses paid with my father, and I got a Chromebox, which ultimately ended up being my dev machine for the next six years.

Corey Quinn: But how could you use it without a Google account?

Alex Zenla: Well, so this is a great, a funny question because one of the people I got to meet at Google I.O. was Bill Richardson, which is the person who made the bios of, all the Chromebooks in the early days. He now works on Pixel security, and he, I had some email exchanges, and he taught me how to get into developer mode.

I was not running Chrome OS on that thing. I was definitely running just Ubuntu, I think, at the time. What's interesting is I got involved in, in Dart after that, after seeing a conference, or conference talk at Google I/O about Dart itself. If you're not familiar, Dart is way more popular than it was then.

And 2013, it was like nothing. It was, it was this weird thing that Google, everyone knew that Google was going to kill off eventually. They just did not get involved with it. I got very heavily involved in the community. And I was on Google Plus at the time. I had kept so many stories about Google Plus, and I had a Google Plus circle with every Googler that I knew, and I would regularly follow and like chat up random Googlers cause I was just a kid. I didn't know what I was doing.

Corey Quinn: It's surprising how far that can carry you just by showing up and talking to people, speaking for a me here.

Alex Zenla: It is. It is. So, now we finally get to where my career started. A random company in Oakland contacted me via email and was like, "Hey, we use one of your Dart libraries. Will you come work for us?" And my immediate reply was, "I'm a kid, but thank you. Sorry. I can't." And they replied with, and I, his name was Dennis, the CTO of DGLogik at the time, an IoT company. And, He goes, "We know how old you are. We, we already did the math and figured out that you can work here legally."

So I was hired overnight, literally at like 1 a.m. my time in Alabama to work on IoT technology and Dart, and from there I started working on Google related stuff. So they had a project for, various different things. Google Fiber was one of them, where we were monitoring all of their edge data centers with IoT technology, and ultimately, I got involved in Google. And I was at a startup, and then I went to another startup that was a vendor for them as well, and then ultimately, I ended up just kind of working on their IoT platforms internally. And that was where I spent the last 10 years of my career prior to Edera, and I knew Ariadne from IRCv3 because I worked on IRCv3 stuff in my IRCbots.

I ran, I had owned irc_dart, and there was a Java library called pircbotx, if I recall, that I was contributing to as well. So, that was how I met Ariadne was just on the internet and then her community treehouse as well, and that's how we kind of reconnected after just knowing each other on IRC.

Corey Quinn: I met my business partner, Mike Julian, when we were both, network staff on freenode years ago.

It's fascinating watching even generationally, "Oh, we met on IRC," continues to be alive and well as far as origin stories go.

Alex Zenla: Yes. I mean, so I, I think I got to see the death of IRC even just as a kid, and by death, I mean, you know, the downfall of freenode, et cetera.

Corey Quinn: I was in my late thirties, shut up.

Alex Zenla: Exactly. So I think, I don't know how old I was when that happened, but I was on EsperNet because of Minecraft. All my friends are on EsperNet. Til this day, my best friend is someone I met on IRC on the Minecraft Forge Channel on EsperNet, and I think it's just, IRC is a powerful thing. Discord, it took it, I think it took it over. I think it'd be fair to say it took it over. I migrated there in 2015 from IRC and then basically didn't look back. There's something different about IRC, just the vibe of IRC is so different, and I think part of it is that, because it's just text, it feels so, like, slightly impersonal but personal at the same time, you know?

Corey Quinn: It forces you to, at least in my experience, it forces you to articulate your ideas a little bit more clearly if you want to have a good time. It's, it's sort of a, I guess a breeding ground for writers.

Alex Zenla: Yes, I would agree with that. I, so I, one of the people who used to lurk in one of our IRC channels, I contacted them a few years ago, and they happened to have logs of everything I said in IRC and my channels over, like, seven years or six years or something. God, I was a cringy kid, but everyone's cringy as a kid, I think. But, I did some analysis, and I was like, wow, I wrote a lot of stuff back then I wrote a lot of words and I think that it really taught me communication style, particularly because it's in text, you have to kind of really work hard to get an idea through. And it's also weird because I was in the Minecraft community there where you kind of have this weird blend of, like, adults who are into Minecraft programming and then kids who are, like, just like 12-year-olds who just want to play a Minecraft mod or whatever. The modding scene in Minecraft is very interesting.

It has a lot of history that is just far too much to go into, but it was so cool to be a part of it.

Corey Quinn: Now it's my turn to make you feel old. My daughters are very much into Minecraft these days, playing with each other. They are seven and four, and they love playing it on their iPads, and I'm just waiting until one of the penny drops for at least one of them to realize, "Hey, we maybe maybe there's more to it than just consuming as we find. Maybe we could make it do something different." I'm not forcing it on them because that's child abuse on some level. "Hey, have you used IRC?" It's like, guys, they'll take my kids away. No, it's a, but it's very much a, let, meet people where they are. And I've wondered about that for a long time, because as someone who started kicking computers around in the '80s and never really stopped, the road that I walked is very much closed.

If I give career advice to people, it's very boomer-coded. It's, "Oh, just have a strong handshake," and can't put it on your resume or do what I did and basically bluff your way through a technical interview at Chapman University the day that their technical reviewer was out sick and get the job offer 20 minutes later and cram like absolute hell for the next three months.

And suddenly you're there and doing it. And then just, yeah, great. Take that energy and never stop. You're always going to be learning and cramming and we're thinking you're not quite far enough. That advice doesn't scale and it doesn't necessarily map to the way the world hires technologists today. So I don't give that advice.

Alex Zenla: Yeah. You know, it's fascinating. I had a time in my life when I had to interview normally like a, like, you know, like I normally would for a job. And because I had basically been handed a job by just like happenstance my whole life, I really struggled with it. And, you know, what I have always done is focused really hard on my, on my background and doing things in my personal life. You know, this doesn't work for everyone, but I am, I love programming outside of my, outside of work.

I do it for fun. I have weird, stupid projects. I have a programming language. that I wrote in Kotlin called Pork, for example. A very silly name whose icon is literally a pig from Minecraft. You know, I just do weird, silly things like that. And I think it gives me the confidence to say that I have a portfolio other than my, and other than my, you know, career achievements.

What I say is talking to people really works. Networking is real. And when I talk to people about networking. I am, I am an introvert. Naturally. I like to say that I'm an extroverted introvert. Our CEO, Emily is like my hype woman. She's a introverted extrovert and I'm an extroverted introvert. And so we, we work very well together.

She's, she's very good at making me get out there, like doing this podcast and things like that.

Corey Quinn: Teach you to swim by throwing you off the end of the pier. They weren't trying to teach you to swim.

Alex Zenla: It kind of is like that. And, you know, it's funny, you know, we work so well together in that regard because, you know, she, she's the one that tells me my background is interesting.

Meanwhile, I'm like, Oh, it's boring. Like I'm just some kid from Alabama, you know? But the reason why I mentioned that is networking is actually. Very important, but when people think of networking, I feel like they, they think about like what I would describe as corporate networking. Like I'm, I'm going to go to this conference and talk to some corporate people and that's great, but I, I like to approach things from a more personal perspective. I like to get to know people on a personal level and, and, you know, really, truly be friends with them, not just from a corporate perspective. And I think that has done wonders for me because, you know, I'm, I'm dependable in that regard. People respond well to that. It is challenging, I think, for Gen Z, my generation, to kind of get used to the, the more boomer kind of, way of, of job seeking and things like that.

You know, I consider myself lucky in some regards to be plucked from Alabama out of obscurity. But, you know, at the same time, I worked hard for it. So I, It's a balance to kind of figure out what the best way to, what the best way is to think about it, but yeah, it's, it's a tough topic.

Corey Quinn: I hear you on this.

I have one last question I want to dive into a bit to sort of bring this full circle.

As of this recording, it has not been announced. It will be by the time this airs unless something horrifying happens to our production embargo system, but you have just raised a Series A. Tell me about that.

Alex Zenla: Yeah. So, our Series A, was, led by, M12 at Microsoft, Microsoft's Venture Fund. It's very exciting for me.

Microsoft is a very, interesting expert in hypervisors. And so, you know, it, it's been so cool to see that they understand it. Deeply and, you know, all of our previous investors invested as well. And we have some absolutely incredible investors. It's exciting to be able to do what I'm doing, particularly as an all female founded company where that typically is harder to, navigate and, and the funding space.

And I'm so excited to just be doing what I'm doing. I dreamed of like building a business, not because I wanted to be powerful or anything, but because I just wanted to do something interesting and I like that we are in a deeply technical space with people who really understand the space. And, I'm excited at our future and the cool things that we get to do with this next phase of our company.

It's, we have such an incredible team. I, I couldn't, I couldn't even list all of the absolutely fantastic people we have working here. We're very, very lucky to have, so many great people. And, I'm so happy to look forward into the future.

Corey Quinn: I really want to thank you for taking the time to speak with me today.

If people want to learn more, where's the best place for them to find you?

Alex Zenla: Yeah, probably Bluesky. I'm, I want to get a personal website that actually looks good in the future. Maybe a blog, but, on Bluesky, I'm alex.zenla.io. That's the best place to interact with me. And then from there I'm on Discord in various places as well, but Bluesky is the best.

Corey Quinn: And I will, of course, include links to that in the show notes. Thank you so much for taking the time to speak with me today. I appreciate it.

Alex Zenla: Yep. Thank you so much. It's been great.

Corey Quinn: Alex Zenla, founder and CTO at Edera. I'm cloud economist, Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five star review on your podcast platform of choice.

Whereas if you hated this podcast, please leave a five star review on your podcast platform of choice, along with an angry comment that I will then edit because that podcast platform does not understand that containers are not a security boundary.