Show logo
Explore all episodes

Stack/Unstuck: Testing, PDFs, And Donkeys

  |  Compiler Team  
Application development and delivery

Compiler • • Testing, PDFs, And Donkeys | Compiler: Stack/Unstuck

Testing, PDFs, And Donkeys | Compiler: Stack/Unstuck

About the episode

We reach our penultimate episode for Stack/Unstuck, and arrive on the topic of testing. Testing isn’t necessarily part of any technology stack, but it is a vital part of building software. Sometimes, it can feel like testing is an afterthought, or just a box for busy coders to tick once completed.

We hear from our guests about how testing doesn’t need to be saved for a curtain call. It can have a starring role when identifying problems within different components of a software stack. And as we include it more in discussions and planning, and as we start thinking about it earlier in development cycles, testing can further an application’s potential, and help teams build software better.

Compiler team Red Hat original show

Subscribe

Subscribe here:

Listen on Apple Podcasts Listen on Spotify Subscribe via RSS Feed

Transcript

Recently I met with Schaffer Stewart. He's a senior software engineer at LifeOmic, but when he is not doing that, he's spending his time contributing to civic coding projects. Right now, Schaffer is working with a group called Open Raleigh Brigade on a particular project for campaign finance. The goal of the project is to make the data more accessible. Voters often research candidates before they head to the polls, right? But finding information about donations, who donated to who, what organization, and how much money can be tricky. A person can go to the Board of Elections website, but in this case, it's confusing. There's lots of things that only an expert in campaign finance would be able to understand. And let me tell you, the user experience does not get much better from there. Even if you do manage to get results, then you're given this page that's a list of 10 different campaign finance filing documents, and you have to know which out of that list you want, which document set is actually the contributions. So the problem was well established. Schaffer and his team wanted to build an MVP—a minimum viable product that would be then tested against the original version for usability and accuracy. So now we have these user stories and this idea, and so it's trying to figure out how we can start building out an MVP and testing the user stories, and testing the idea, and see what's possible. User stories are a type of documentation. They're done to capture functional requirements based on things a user would want to do with whatever they're trying to build. They were very simple things along the lines of, I should be able to search for a committee or a candidate by name and just see all the contributions associated with them. But what does one find when testing software and what effect can testing have on the software stack? This is Compiler, an original podcast from Red Hat. I'm Brent Simoneaux. And I'm Angela Andrews. We're taking you on a journey through the software stack. We call the series Stack/Unstuck. Today's episode, we are talking about testing. This is one episode of a series. So if you want to listen from the beginning, you can start from our episode, The Great Stack Debate. Let's go to producer Kim Huang for our story. We're talking about testing today. And Kim, this might sound like a silly question because I think I know what testing is, but in this context, what is testing? Simply said, software testing is something that is done ideally before something goes to market or it's released for public use, right? You want to make sure that there are little to no bugs. I know it's sometimes unavoidable that bugs come up after a product release, but you want to make sure that the software that you design is doing what it's supposed to do, and it's not fundamentally, what they say, broken on release. I think testing is super important. I mean, I know we're talking about stacks here, and this really isn't part of a stack specifically, but it's so integral to every single part of the stack. I mean, I think it shows we want to do testing because we want to make sure that we're getting it right. We want to make sure that we're giving our users the experience that we expect them to have. So it's one of those things that you really can't or shouldn't live without it. I've got a couple of questions here. Sure. One, who is doing the testing? That depends. I was going to say my favorite answer, it depends. It depends. Okay. Yeah, it does. Yeah. Well, say more about that. Well, some teams actually have testers who do testing because they're not the ones that are writing the code, they're just testing it to see that it behaves the way that the programmer said it was going to behave. So they do quality control. They're like, "Does it do X, Y, and Z?" Sometimes developers write their own tests to make sure that their code behaves the way that they expect it to behave. So I guess it depends, is really the best answer for who does the testing. And at what part of the development process do we test? All Of them. All of them. Okay. So it's sort of like you continuously do this throughout the process. I think you do to some degree. Okay. Now, if we're talking about pipelines, there is a section inside of a CICD pipeline, which is continuous integration, continuous development, where the testing happens. But I think back to when I was in bootcamp where I would write code and I would do little tests to make sure that what I wrote would produce the result that I expected. So it was just me hacking away on my keyboard, making sure that this is really run. Does it really account for all of the variables that could quite possibly be entered here? So like I said, testing happens all over the place. So testing isn't technically part of this software stack? No. Kim, I want to ask you, why are we talking about it in this series then? Why is testing important to this particular conversation that we're having? Well, there can be a lot of friction inside of teams about what to test and when to test it. Just because you're trying to get something out there and you're trying to get an iteration of your product, kind of like what Schaffer was talking in the first part of the episode. You're trying to get something out there as soon as possible, and you feel this kind of pressure even, especially, if you're a product owner, for example. You feel a pressure to get things out there very quickly. So it's easy to forego the one or the lowest rung on the ladder because you know you can just skip that. But it's not really advantageous to anyone to skip something like testing. It should be the opposite effect where testing should not only be a part of every aspect of software development, but it also kind of should inform the development itself. All right. We've got our definitions. We have level set. Yes. We've level set. Now, let's dig in. Okay. When I first started looking into making a testing episode for Stack/Unstuck, I went to the place where I normally go when I am trying to find out anything. I went to my search engines. I went to Wikipedia. I went to different websites, different articles, and I kept seeing a name over and over again. My name is Lisa Crispin, and I'm a testing consultant, an author, a trainer, a speaker at conferences, and a lover of donkeys. Donkeys? Okay. Yes. Like Shrek donkeys. Correct. Lisa Crispin is a person who raises donkeys. And knows a lot about software testing. Yes, a lot. Okay. Well, we have a little farm and we have four rescue donkeys that we take care of, and they pull carts and wagons that do work around the place, as well as just generally being cute and entertaining us and being nice companions. That's awesome. I know. I love it. Lisa says there are a number of reasons why testing is crucial to software development, and not just in a practical sense. I've actually worked with developers who were like, "Well, why do we need to test this?" "Really? Are you sincerely asking me that question?" And they were. And a lot of times they just see it as, "Oh, that's a less valuable activity, and people who are lower paid than I am should do that," which is not a great attitude. No way. These days we really have to be concerned with ethics and we have to be concerned with regulations and laws. Wow. That's an interesting take because I could see someone saying that because they think that what they're doing is just so self-important, But these are all really important jobs and really important tasks. And without good tests, you're shipping crappy code. Seriously. Yes. And crappy code carries a lot of what risks, right? Indeed. Yup. And remember what Lisa just said about all of those regulations and laws, and ethics. Well, the world is only getting more and more robust with those types of laws as it pertains to IP, as it pertains to security. And not being in compliance with those laws can obviously be very bad. A company could easily go out of business because let's say their code is not secure and somebody breaks into the database. There's just so many different ways. Or maybe their user interface is not accessible to people with disabilities, and there are regulations around that, that they're not meeting. And so they get fined. So I think we answered the question. That's why we test. Yes. Releasing untested software and applications is a pretty risky proposition. But we talk about tests typically in the context of QA and end user testing, what the person using the software sees. What does this have to do with the stack that a development team can use? Maybe you're testing at a unit level, but a lot of the problems don't come out because they occur between the different levels of the stack. It's when you go from the user interface all the way down to the database that a problem happens. Okay. Let's unpack this. So she just said something really telling, We're so focused on the front end and what users are experiencing and things like that. But testing has to go a little bit deeper than what you see because she said it needs to happen all the way to the database. So think about forms that we see on websites and we're entering our information or whatever. That information has to be passed securely. We shouldn't be able to put any old character into a field and probably do some sort of sequel injection query on a database and take it down. We need to test to make sure that what this application is doing is not going to cause harm. And we have to test for that. It seems like she's also saying that if you're just testing in one part of the stack, you're not going to discover all the problems. Exactly. Some of these problems occur between the different components of the stack. Let's talk about that concept of testing at the unit level. Maybe in this scenario, a team can't test the API level for whatever reason, but they can test at the UI level. Those tests can make development easier. Maybe it lessens the amount of errors and bugs that come back when they actually try to compile everything to make sure that everything can work. So if these types of tests can be written with each layer of the stack in mind, just like Angela was talking about at the beginning of the episode, how she was in bootcamp, a little bit of writing here, a little bit of code there, a little bit at a time makes for a more effective and more efficient development process. So this makes a lot of sense to me, and that makes me wonder why everyone doesn't do this. Maybe because it slows down your work. You know what I mean? Mm-hmm. Because if you're working on something and you are head down and you have to really think about how to error proof and find your bugs, well that takes away from putting your head down and writing your code. But I think we should give it just as much weight as we do anything else because quality is key. Security is key. Being compliant is key. And if we kind of coupled that into our code writing process, I think it would feel less burdensome. I agree. They do say that time is money, and I understand that sometimes testing code means you're not writing it. But at the same time, you only get one chance to make a first impression. And when you want to launch a product, if you want to launch an app, you want the best experience possible for whatever user or whatever customer you're trying to serve. Lisa does go a bit further about testing. She says that testing isn't just beneficial for development, it can even be beneficial for teams before work even starts. A great example is frameworks like React come with testing frameworks like Jest, which give a lot of advantages over others. And so when a team is thinking about, what coding framework to use. They need to look at what testing framework it's using too for the unit level test. And for things like API level testing or workflow testing at the user interface or in workflow level. Testing is important, but not just when something is ready to be released. It should be something that, to Angela's point before, is at every stage of the development process. So unit level tests can help catch problems at different layers of stack. And the availability of testing support and frameworks just can help teams choose the technologies that they use. I think that's interesting because you can decide, "Well, I want to do X and I want to use this language." You're now doing your homework and saying, "Well, does it come with the testing framework?" That kind of makes it easier because you are working within language that you're already really familiar with, hopefully. Yes. Right? I think that's interesting that now these frameworks have their own testing framework. I would guess it would make your job a little bit easier. Yes. So coming up, we go back to Schaffer. He and his team are starting to delve into the build, and they are going to discover that testing can change a project midstream. So at this point of the story, Schaffer and his team are putting together that campaign finance dashboard. Specifically, they're working on user stories. The team asked each other important questions. Who was going to be the primary user for this tool? What do they want to see in what they're using? Schaffer did talk about voters earlier, but even at the beginning, things started to change. If there's lay people using the tool and things are off by a little bit, or there's something that's one or two records aren't quite right. That's not the end of the world. But one of our target users is researchers and journalists. So if these people are using the tool and stuff isn't 100% accurate, then the tool is not really useful for them. If they have to look at the data and the tool and then go back and double check everything against the board of elections, which is sort of a time consuming process. So the first thing that the Open Raleigh Brigade team decided was to clean the data that they already had available. That was so development could start fresh. Having good data is critical. We have to make sure that during our cleaning process, we haven't done anything to misrepresent the data or introduced any inaccuracies into the data. They started that kind of what they call de-duplication process. Okay. And those processes can require scripts to be written, at least to shorten time so that people are not doing this too manually. Manually, yeah. And it also cuts down on errors, right? Because human error is always a factor. Schaffer stepped forward with his experience and expertise to help with this early on, and he wanted to use the technology that he knew to test and see if this could work and address some of the accuracy problems that were happening with the early iterations of the dashboard. And so in this case, I was helping out with it. So I know Node.js and Postgres, and so that was the tools I picked to try and get it done. So Node.js, which is a runtime environment for JavaScript and Postgres, which is for the database layer. So he started working with those tools and then he ran some early tests, like little incremental tests to see if things were working at the time. The MVP gave us enough data that we could take it and start building a dashboard around it, but it didn't really scale well enough as we tried to add additional data. Node and Postgres weren't really the right tools for that, so we did a second iteration, which was mostly Python. So that took us a little bit further. We were able to get more data. It worked better, it worked faster. It was just kind of nicer to use. So they had to find the right tool for the job. Yes. And in order to do that, they had to stop and test the work that they'd already done thus far. And they found out what they were doing wasn't working how they expected. Right. Well, that's the good part about starting early and not getting so deep into the process and having to change tools. So Kim, how else did testing inform their work? Schaffer talks more about testing here and a lot of what he says is very indicative of the nature of civic coding projects. So I think mostly it's just as we're building, we're testing these things to see if they work as we expect. During our regular meetings, we often would demo the project to show where we currently are, and that was also a good opportunity for people to play around with it and test it. You're on a Zoom call with 10 or 15 people and you kind of show them the tool and then most of them will pull it up on their own computer and start plugging things in. And then they'll hit an error and they'll let you know right there in the call testing and seeing what happens if I misspell someone's name, I throw a typo in there, that sort of thing. And just based on that sort of refining, fixing bugs or small behaviors that don't really make sense, adjusting those. Well, that's pretty smart because having your team all plugging away, trying to pick something apart and someone finds something, one, you have a bug and you can file a bug on Bugzilla. I'm using that term, but you can file a bug and this is something that you have to program out of your code because it's behaving in such a way you don't want this to happen. So it's cool that they have this ongoing process. And because testing is so important, you have to do it like this. You have to pick it apart, move one piece at a time, and at some point you're going to find out you're going to have an application with a lot less bugs in it. Exactly. You can even use testing to identify themes that are necessary to protect people's information. Things like addresses, phone numbers, items that people may not want everyone to know. We don't include that information in our dashboard because we don't want to make it any easier for people to abuse that sort of information. So yeah, it's a lot of just testing it internally and figuring out what we do and don't like and what we do and don't want to see. They want their dashboard to produce X and anything outside of X is not what it was meant to do. So they're trying to keep the scope super small. It is a tight line to walk. Yes. People's personal information, you do have to be super careful with exposing that. Yeah. So now that there's a MVP in place, they've been working on this at this point for many, many months. I think since 2020. So it's been actually a few years. And since then, they've built in even more complex user testing, specifically based on some of those power users that Brent was talking about before. One of the main leaders behind the project is a journalist and independent researcher. And so he's reached out to some contacts and colleagues in the journalism community, and we've gotten on some calls with them. We've given them the tool and had them test and search for people that they're familiar with to start looking at the data and making sure that the data is correct. So it's important that we get these subject matter experts and people who are in the domain to come in and start using it before we release it to a broader audience. Very smart. That is very smart, because I think you're not going to catch everything in the lab. You need people, experts, power users to kick the tires in the wild in some ways. And you also need a discerning eye, an expert eye to figure out what's missing if there's anything that's missing. I feel like especially with journalists and people who have that expertise in campaign finance who are working a specific beat, or maybe they are working as a political analyst or something, they know exactly what to, in this case, what to look for, what it should look like, what it should not look like, and what is most useful to a user in the end. Yeah. They have that discerning eye. Schaffer says the most important thing is for people that are working on development teams, especially if you're working on projects like this that are so complex. It's important for them not to feel down about having to scrap work when testing gives you a different outcome that you don't expect. It can be easy to feel like you did something wrong or you didn't write good enough code. It can at times feel like an attack on you. Don't take it personally. You built what was needed at the time and now something else is needed and that's fine. It might be in a different stack, and that's fine too. I don't think it was a waste of time. When you realize that you have the wrong tool for the job, you're doing your job seriously. You don't want to fit a square peg into a round hole. You want to make sure you have exactly what you need because your end user, they depend on that. They really do. So I get where he's saying, it could feel like, "Oh, I didn't do a really good job with this. What did I do wrong?" But sometimes this is not the right tool for the job. So good for them figuring that out before they got way too far along. Yeah. So we've been talking this whole episode about the importance of testing, especially at various stages of the development process. And to me that seems like the way to go. So from everything we've heard, that seems like what we should all be doing. So I'm curious if the tools are there, how do we build this into our culture? Well, my opinion is make it okay to change your mind. Look at Schaffer's example. When you realize you're not making the best choice, make it okay to switch and pivot. Being agile, not only in your development cycle, but also in how you test sounds like the happy medium. Testing is super important. We can't bolt it on in the end, just like we can't do security bolting it on at the end. They need to be first class citizens in the development process. Agreed. Testing is not a part of the software stack. We all know that, but it is a crucial part of the software development process. Testing can catch problems in the software stack before they become very big problems down the road. But my question is how do we change the culture around this? I think I understand the importance of it. I hear what you're saying, but to me this feels like a cultural issue. Right? This is all culture. It is. Yeah, it's all about culture. Well, how do you change any culture? You keep talking about it. You make it commonplace. It can't be taboo. I mean, think about how our language has changed over the years because we realize that this doesn't work. This does not work. We have to figure out a better way and how do we do it? We have to make it more approachable. We have to make it more accessible. We have to make it okay and... Normalize it. We have to normalize it. Exactly. So that's how any culture change happens. These weren't the ways that we did things. And what we had to do was we had to keep talking about newer and better ways. That's literally how you change the culture. It really does have to happen from up top too because that's where you get your buy-in. And then it all trickles down. Okay. So that's the stack. Did we do it? I think we did. We went through the entire stack from frontend to backend. Yeah, I think we did it. Well, up next, we're going to wrap this up. Right, Kim? We do have one more episode though, and it's kind of like our wrap-up to go over some of the things we've learned while making Stack Unstuck. And we have a few special guests who are going to come in and leave us with their thoughts as well. Remember, you can go back and listen to any of the previous episodes, from front end, to frameworks, to databases, to the operating system. Yeah. Check out any of the older episodes and we want to hear what you think about this series. You can tweet us at Red Hat and always use the #compilerpodcast. We would love to hear from you. And that does it for this episode of Compiler. Today's episode was produced by Kim Huang and Caroline Craighead. Victoria Lawton tests our patience at every step of the way. Just kidding. We love you. Our audio engineer is Kristie Chan. Special thanks to Shawn Cole. Our theme song was composed by Mary Ancheta. Thank you to our guests, Schaffer Stewart and Lisa Crispin and her donkeys for their insights. Our audio team includes Leigh Day, Laura Barnes, Stephanie Wonderlake, Mike Esser, Nick Burns, Aaron Williamson, Karen King, Boo Boo Howse, Rachel Ertel, Mike Compton, Ocean Matthews, Alex Traboulsi, and Laura Walters. If you'd like today's episode, please follow the show. Rate us, leave us a review, share it with someone you know. It really does help us out. We enjoy you listening. We enjoy putting on this podcast for you. Until next time. All right. See you next time.

More about application development and delivery

Development and delivery involves creating, testing, and deploying software applications, speaking to the entire product lifecycle. 

Blog Post

What is Red Hat OpenShift? A cooking story.

June 13, 2024  |  Dan Russo

...

Application development and delivery

Blog Post

Top articles for developers from 2023

January 19, 2024  |  Colleen Lobner, Managing Editor, Red Hat Developer

...

Application development and delivery
Compiler, an original podcast from Red Hat

Original Shows

The Truth About Netcode | Compiler

Application development and delivery
Open source
Compiler, an original podcast from Red Hat

Original Shows

The CTO And The Vision | Compiler: Re:Role

Application development and delivery
Compiler, an original podcast from Red Hat

Original Shows

Frameworks And Fundamentals | Compiler: Stack/Unstuck

Application development and delivery

About the show

Compiler

Do you want to stay on top of tech, but find you’re short on time? Compiler presents perspectives, topics, and insights from the industry—free from jargon and judgment. We want to discover where technology is headed beyond the headlines, and create a place for new IT professionals to learn, grow, and thrive. If you are enjoying the show, let us know, and use #CompilerPodcast to share our episodes.