Show logo
Explore all episodes

How Bad Is Betting Wrong On The Future?

  |  Compiler Team  
DevOps

Compiler • • How Bad Is Betting Wrong On The Future? | Compiler

How Bad Is Betting Wrong On The Future? | Compiler

About the episode

Technologists are often asked to make decisions based on future industry advancements—or basically, things that haven’t happened yet. It’s already difficult to choose the right path for a project without the pressure to be clairvoyant. But everyone wants to feel like they are leading the pack on the next big thing.

What do we need to know to make a good prediction for where technology is headed? Alternatively, what do we need to know to avoid the wrong choice? We speak to experts in the DevOps space about betting wrong on the future, how development projects go awry, and what teams can do to get things back on track.

Compiler team Red Hat original show

Subscribe

Subscribe here:

Listen on Apple Podcasts Listen on Spotify Subscribe via RSS Feed

Transcript

Okay. So Kim, you sent us this link. I'm opening it right now. This is a Wikipedia page that says... At the very top, it says "list of failed and over budget custom software projects." What is this? This is a list of some big swings on the part of so many different government organizations, large software companies. And if you look at the problems and the reasons why they failed, you'll see a lot of the same themes pop up over and over again. The usual suspects: I see scope creep, cost overrun, low user adoption, just delays, delays, delays. Nothing on this list surprises me. Oh, this is kind of fascinating. Yeah. Look at the money. The money! That's what's striking me about this list is there is a column that says expected cost. Yes. And we're talking three to six billion US dollars- Mind blown. And then there's another column right next to it that says outcome. And it's just canceled, scrapped. It doesn't look good. No. Canceled, scrapped, abandoned, discontinued, canceled, canceled, scrapped, scrapped, canceled. I shouldn't laugh because it's so painful but- It makes me wonder; technology's an environment where things tend to go wrong more than they go right. And when you're trying to build something so ambitious and so large in its scale and its scope, how do you avoid making the wrong bet on what's going to happen down the road? This is Compiler, an original podcast from Red Hat. We're your hosts. I'm Brent Simoneaux. And I'm Angela Andrews. We're here to break down questions from the tech industry, big, small, and well, sometimes strange. Each episode, we share stories from the people behind the code. Today's question, how bad is betting wrong on the future? Producer Kim Huang is here with our story. So that list I talked about earlier, it was referenced in a book. The book is called DevOps Culture and Practice with OpenShift. A lot of the text of the book talks about how to get things off the ground. And DevOps is obviously... that's what it's for is to get teams from zero to cruising altitude as soon as possible. So to that point, I wanted to talk to some of the people who are behind the book. One of those people is Noel O'Connor. I'm a senior principal architect working in the EMEA Solutions Practice. What I do at Red Hat, I do a lot of things. I'm primarily in the consulting area. Basically helping customers around the world, basically adopt–perhaps–technology and approaches. I asked Noel, who's based in Ireland, about leaders making the wrong bet on the future. Planning for what's there, what's present and not really what there could be down the road. He talks a lot about what he calls marketecture. I'm interested in finding out what marketecture is. Marketing architecture. It comes off as a... not a slang, but like a dirty word, the way that he said it, because he talked about how sometimes you have people who talk about technology and project this air of, there's a product that they're making and the product is going to do this amazing thing, but it actually doesn't work. Really good marketing has a fairly significant impact. And we've seen that down through stuff that's gone on in the last number of years on technology projects that failed due to "how hard can it be?" And it can actually be very, very hard. I mean, a lot of good marketing makes things seem really simple and really flexible. And its job is to position technology in the right place and the right light. Whereas actually a lot of times they don't want to show how this could potentially go wrong or things like the cases where it doesn't fit. Well, let's talk about the future a little bit here. Sure. Right. Because this is something that we have to make decisions here and now, and a lot of the decisions that we have to make about the products that we're building or the technologies that we're using, we have to be able to look out into the future and make a really good guess about what the future is going to be. And this is really difficult. Yes. I know it is because there are companies who… you think about it, they have this product that tends to become ubiquitous and they do just one thing, I don't want to say wrong, but going away from where people think technology should go. You're heading in the right direction, but you make one false move and you've definitely fallen off a cliff. So it's nice to be able to foreshadow, but there's really no way to know because we all want to be innovative, we all want to be on the cusp of the next big thing. We don't know what that is. So someone's going to take a bet on something and they're going to lose that bet. And that has to happen I think, in order for technology to move in the right direction. I know it doesn't sound like it makes sense, but someone has to lose in this scenario because every good idea can't be the right idea. So that's just my opinion about it. There has to be some winners and losers when we're talking about the future. It's not just about the products that we're building either. It's about the technologies that we're using. Yes. Very much so. To build and deploy those projects as well, because those are really important. And I'll say often they can be really expensive as well. Right? Yes. Yes. 100%. Yeah. But those are other decisions. It's not what you're building, it's how you're building it as well. Absolutely. And what teams have to come across and work with each other, not necessarily from a development standpoint, but just on a logistical standpoint. They have to work together to allocate budget for hires. If you need to get somebody on board that has a specific set of skills, or if you need to make a team bigger, or if you have to buy a certain type of testing environment in order to test your app. All of that needs to be factored in. And those decisions don't necessarily come from the people who are actually writing code. Right? Yeah. That's the rub. For me, I feel like guessing about the future is fine, as long as you admit to yourself and to everyone around you that you're guessing. As long as you're honest about that. Noel had something great to say about "the horizon that you can see" and how important it is to plan for the horizon that you can see within your line of sight, while still being innovative. If somebody knows what the future's going to bring, I'd like them to tell me how I'm going to win at casinos. Because it's like a... Event horizon sounds like the wrong word, but basically you can only see so far. I mean, we're in the realms of containers now and clouds. And basically if you'd said 15 years ago, was this going to happen? Nobody would've known. And also Red Hat, we constantly pop our heads above the parapet, take a look around and see... Because we work so much in open source, it's basically, there's a lot of community stuff going on and you can see trends that are building and stuff that's falling away. So Noel started out talking about how situations where you have "marketecture" people trying to have that crystal ball kind of mentality. That's a situation where it's really ripe for people to make the wrong bets in technology. I wanted to talk about the real practices and methodologies that we as technologists could employ to prevent teams from going down a bad path. And for that, I spoke to Tim Beattie, also in Ireland. He's a former Red Hatter and he wrote the book on DevOps, along with Noel. And I'm the co-founder of Stellafai, which is a solutions company that focuses heavily on visualizing and measuring business outcomes and sharing that for others to learn from. Here's the thing. When we think of DevOps as Red Hatters, we think of practices and collaboration and innovation and trying to iterate on discovery very quickly. But when other people think of DevOps, it has come to mean something else in the industry. It might not come as a surprise- Angela laughs. Oh, I know. She's already anticipating what I'm going to say. It may not come as a surprise, but sometimes DevOps is looked upon as something you hire another person to come into your company and do, instead of what teams and organizations do as far as changing their culture. I wanted to know how that happened. It became a buzzword and we were hiring for the buzzword and no one... They thought it was a product, or it was... What is it, the phrase that we used? Marketecture. It was one of those things that is like, "I'm going to buy me some DevOps and we're going to get some DevOps in here." And it's like, "No, that's not how it works. That's not how any of this works!" No, you really have to change your culture internally. It will not work or make sense unless you do that. So yes, it's more of a mindset and a methodology and it's not something you can buy online. I'm trying not to laugh. This is a very serious subject. I'm trying to be serious. I'm sorry. But I keep thinking of like, "I'm going to come in here and I'm going to DevOps everything and DevOps that chair and DevOps this desk." You're getting DevOps, and you're getting some of that DevOps. Seriously. It's funny now, but it was all the rage. For a long time "lean" was a very in demand buzzword and then agile replaced that and then DevOps replaced that. What's next? I don't know, make up a word, I guess. I mean, I already know the next hot word. It's called zero trust, but we're going to be buying some of that zero trust real soon. Oh, you get a zero trust and I get a zero trust. We're all getting some zero trust. Yes. I was curious, I wanted to posit an example for Tim to work on. In Tim and Noel's relationship, they identify each other as Tim being the practices guy and Noel being the tech guy. I wanted to–I know this sounds bad… I wanted to test Tim to see what his response would be if I were a hypothetical tech founder. Okay. So I asked him, imagine a tech founder who's trying to build an app. They have the concept, maybe. They may even have done some practices like impact mapping to figure out business goals and deliverables, but they're experiencing a lot of scope creep and a lot of increasing costs. Those things that we were seeing on that list that we looked at at the top of the episode. Yeah. For real. What you don't want to see inside of your project. Yeah. Exactly. So I asked Tim, what are some things that would be very useful for a person in that situation to think about? Yeah. Where we encourage learning, where we are prepared to adapt and throw things away and try completely different directions to run lots of experiments, sometimes very cheap experiments. And there's a load of practices we talk about in our book around building the foundation of culture to help put that in. There's something that takes time and also you have to continuously nurture and improve, because the moment, and there's some big examples in the news where there hasn't been safety and that's where companies end up in all sorts of problems. My go-to practices are around in designing of experiments, figuring out what are the hypotheses that we want to test and how can I go and prototype some of my ideas and test them in as lean and cheap a way as possible. And that doesn't necessarily mean building the tech. It means figuring out what we think might improve or provide the real gains and seeing if I can go and learn from some real end users about that. And that maybe means executing things manually or even putting a very low fidelity prototype in front of some users and getting their feedback. Bring this to life for us Kim, a little bit. So he's talking about doing these, what he is calling cheap experiments. Bring that to life for us. It's like earlier when I was talking about buying a testing environment for your applications. And I'm definitely not the expert in this, but I think what he's trying to get at is that a lot of founders get into a situation where they have, not a plan, but a list of desirable things that they want to do, and a list of things that they need in order to do them. And they go from there. So you want to do something as cheap as possible, and as fast as possible. You don't want to blow your budget on something that only gets you from point A to not point B. You want to get to the next step, to development and eventual release as soon and as fast as possible. So some of the examples here, he talks about prototyping and testing. It's something that's really important because you also need to know what your user wants. Just trying to find out the deliverables and the functional requirements you have listed, you have to find out what users want and match those up to what you want to deliver. Exactly. So that point really hits home because to find out what users want, you have to talk to them. Yeah. You really do. It sounds so simple, right? Yes. Yeah. But I'm sure if you looked at that list, some folks weren't hitting the mark because they weren't doing that early and often. Yeah. Early and often. And that is something that I think a lot of the practices within DevOps culture, really stress. Don't just do it one time and then say, "Welp." Dust yourself off, dust your hands off and go, "That's it. And I don't need to go back and find anymore information." No, you have to constantly be looking and discovering things. And as we're thinking about the future, we talked about how it's really impossible to really predict the future or know exactly the future. But you can know some things about what's happening now. In terms of your users and in terms of different things. And you can know those things by doing some of these cheap experiments. You don't have to guess at some of this stuff. You can know some of this stuff within a reasonable doubt. Right. Right. I think this is, Brent, very interesting and a really good segue from what you were saying. Yeah. Which brings me to the next point that Tim and Noel left me with. From an IT perspective, planning is dead. One thing that its made me think of, if I think back to the planning I used to do 15 years ago, 20 years ago, it wasn't planning at all, it was guesses. We were making wild guesses on everything. Moving away from linear thinking where it's a big long path from A to B, it's actually, you're going round in loops. You're going round in circles. And what you end up doing could be incredibly different to what you started out… what you thought at first. Because you've learned so much along the way. What his point is, is that planning is dead because in order to really think about the future, you have to accept that things go wrong just as often as they go right. That's true. I mean, you learn in those spots where... You learn from your mistakes, as they say. When you find out that something goes wrong: "Well, okay. All right. That's one way not to do something. Let's figure this out again." So, that's constant. And you think of what you've amassed along this process. So I get what he means when he says planning is dead. But he's not saying that all planning is dead though. Right? Of course not. I think that they're saying down the road, that planning is going to be not as important, not as imperative as it has been in the past, just because there's so many things changing all the time happening in sync with what development teams are doing. It makes your plans irrelevant within two or three years, especially if you're working on projects as big as the ones we talk about in the beginning of the episode. Some of those projects went on for so long that there were developments in the field that made the project obsolete. Is he saying that long term planning is dead and following that rigidly? Is he just saying we need to move our sight line up? It's more about short term planning in a way that has a lot of nimbleness. Is that it? Maybe. That sounds... Yeah. Yeah. To me, it sounds like that's what he's saying, because technology 30 years ago; it didn't iterate as quickly. So you could really plan for the next 10, 15 years. Technology didn't move that fast. But how can you do that now? It's almost impossible. So your sight line has to be so much shorter and you have to be okay with that, but be ready to iterate when the time comes if something changes. You said it perfectly, that resilience that we need now, because things pivot so much more quickly than they used to. So these two have written an entire book about DevOps. What does DevOps give us? What are some of those practices that help us be more nimble? Oh, definitely. Can I take this one? Yes, please. I have a couple of ideas about this. Go. So the whole purpose of DevOps is everything is in code. So you're not wedded to anything. As long as you have your repos where your code exists and your pipeline, you can really... I want to say you can be resilient as long as your code is there and it can be deployed somewhere. What the cloud brings us, it brings us that elasticity. It brings us that flexibility. I think this is where you are going with this, but correct me if I'm wrong. But what DevOps buys you is flexibility. It buys you repeatability. That's the whole thing. To be able to repeat your processes without putting your hands on things. To me, that's what it's buying folks right now. It buys them a lot of agility. Yes. It's not the long jump, it's hopscotch. There you go. You're getting to the point where a lot of people… the long jump, it sounds very appealing. But in actuality, you leave yourself vulnerable and open to a lot of challenges. In the approach with DevOps, you have this continuous feedback loop with your teams, as far as what you're doing and why you're doing it. Then you get to test that early and often. You get feedback and go right back into a discovery phase. And now you have this never ending loop of iteration. It makes development cycles easier and more efficient. Here's Tim again. I mean, I think it allows us to try things out. It allows us to experiment at relative ease, compared to maybe 10, 20 years ago where it would've taken a big investment and a big leap of faith to try something new out. All of the technology and the practices and the mindset around DevOps takes us down the path of, we should embrace the idea of experimentation. We run some experiment and it didn't work out because we learned from it. And we've done that relatively cheaply because we're able to build things and deploy them, and put them in the hands of users and learn from them much, much faster, at a much, much lower effort and lower resource cost than what we used to have to do. So when we talk about the future, does DevOps allow us to future proof? No, because the future is still an unknown. But what it does allow us to do, it allows us to just step lots of different directions into the future. And with the paths that we do then take down, we can apply with a bit more confidence, because we're actually getting some empirical outcomes as we go along. Oh, that is so interesting. So he's saying instead of making one big bet on something, maybe you can make a lot of smaller bets. Yes. And then follow where those bets lead you. Mm-hmm (affirmative). That's what discovery is. Huh. No longer do we have to go for that long game, because we're being more nimble. What was the word you used? The smaller bets. Mm-hmm (affirmative). Yeah. Why not? Follow them where they take you as opposed to going out there on a limb. That's right. Those small bets, just doing a little at a time can allow people to discover new things that they probably never thought of in the first place. And most of all, practices like those included in DevOps reduce the likelihood of you ending up on a Wikipedia list. That's a list you don't want to be on. No. All right. So Kim, Angela, let's come back to our question. If you remember it was how bad is betting wrong on the future? From the Wikipedia list, it sounds really bad. But what you've brought us today Kim is a set of practices and a way of thinking about the future to help us be more future resilient. So I'm hoping we could be very practical right now. What are just a couple of the things that we can change about how we think about the future or a couple of things that we can do to make us more resilient? So the first hurdle is to change the way we think about the future, and accept the fact that while we are building things and we are in our, by their very nature, these silos. We're in a competitor state with other organizations and other teams and other creators trying to do the same things that we're doing, trying to serve the same customers that we're trying to serve. It seems like designing things and developing things, inherently you're going to run the risk of someone coming up with some kind of alternative that makes whatever you're building obsolete or makes it undesirable in comparison. There's always a chance of that happening. I think it's important to understand that there are practices in place to get from zero to a cruising altitude, in a metaphorical way, very quickly. There are practices that you can employ to find out what the people that are going to use your app, what do they want? The people that are going to use whatever product you're trying to build, what do they really need? What is it that's going to make them choose your product amongst a host of other ones that are being offered to them? To me, that's part of trying to break down the future and trying to get away from this crystal ball mentality and going towards a, what I call hopscotch mentality, where you're getting to the next small kind of milestone, yes, at a time. Exactly. But also what I love about that metaphor is: hopscotch is fun. Yes, it is. Yes, it is. It makes it so much less stressful when you think about moving toward the future. It makes it a lot less stressful. Just a square at a time. Yeah. And it takes a lot of the weight off of the decisions that you're making. Definitely. Can you imagine having to have the forethought of what something needs to be for the next 10 years, 15 years? A lot of pressure. A lot of pressure. How about we just do one square at a time and see where it takes us. And that does it for this episode of Compiler. Today's episode was produced by Kim Huang and Caroline Creaghead. Victoria Lawton always brings us back to the future. Our audio engineer is Elisabeth Hart. Special thanks to Shawn Cole. Our theme song was composed by Mary Ancheta. A big thank you to our guests Noel O'Connor and Tim Beattie. Our audio team includes Leigh Day, Stephanie Wonderlick, Mike Esser, Laura Barnes, Claire Allison, Nick Burns, Aaron Williamson, Karen King, Boo Boo Howse, Rachel Ertel, Mike Compton, Ocean Matthews, and Laura Walters. If you liked today's episode, please follow us. Rate the show. You can also leave us a review. It really does help out the show. It sure does. And we appreciate it. We'll see you soon. All right. See you next time.

More about DevOps

Deliver with more reliability, integrate feedback, and ensure greater business value. DevOps centers on the cultural shifts and practices to make all of this happen.

Blog Post

Leveraging the GitOps ZTP pipeline to upgrade Red Hat OpenShift clusters

November 2, 2023  |  Zeid B Alshammery

...

DevOps

Blog Post

Platform engineering for Telecommunications, Media, and Entertainment (TME)

October 18, 2023  |  Fatih E. Nar, Volker Tegtmeyer

...

DevOps
Compiler

Original Shows

Building A Common Language | Compiler: Stack/Unstuck

DevOps
Technically Speaking, an original podcast from Red Hat

Original Shows

Composable infrastructure & the CPU’s new groove | Technically Speaking

DevOps
Partners
Technically Speaking, an original podcast from Red Hat

Original Shows

GitOps with Argo CD | Technically Speaking

Communities
DevOps
Open source

About the show

Compiler

Do you want to stay on top of tech, but find you’re short on time? Compiler presents perspectives, topics, and insights from the industry—free from jargon and judgment. We want to discover where technology is headed beyond the headlines, and create a place for new IT professionals to learn, grow, and thrive. If you are enjoying the show, let us know, and use #CompilerPodcast to share our episodes.