In my last post, I highlighted common QA testing flaws. With these flaws come bug-ridden code and continual stop-and-go of the flow of work, not to mention time-consuming context switching.
All this begs two questions: In the age of lean agile continuous delivery, is there also a leaner, more agile approach to QA? And if so, what does a lean approach to QA look like?
To answer these questions, I’ll ask another: Why aren’t developers writing code correctly from the start? As I’ll discuss here, questioning assumptions about developer skills and capabilities may hold the key to quality engineering transformation enterprise wide.
Agile Quality Engineering Requires Engineers to Code and Test
The conventional wisdom is that developers can’t test. If developers could test, they would build software correctly right out of the gate. How do we know they can’t build it right the first time? Because they don’t.
This circular reasoning (developers can’t test because they don’t; they don’t test because they can’t) puts developers and testers in binary roles. Traditionally, engineers code, while testers identify and report defects. Developers have been conditioned to understand that they don’t have to work out bugs on their own. Quality is handled by the test team or by test automation – the two cures for all coding ills. Quality is neither their problem nor their responsibility.
(For point of reference, I spent more than a decade working as an enterprise developer. This topic is close to my heart).
As a result, developers have stopped learning from their own mistakes. Over time, organizational and self-imposed limitations (“I’m good at this but not good at that”) have built inefficiency into the software development and delivery process.
As it turns out, one engineer acting as both developer and Quality Engineer will be more effective than multiple individuals fulfilling separate development and testing roles. Let’s examine why.
Moving from Traditional QA to Quality Engineering
Can developers build code correctly the first time? Can agile QE help organizations eliminate scope, speed, and quality tradeoffs? I believe the answer to both questions is yes.
It’s possible through quality engineering (QE), which involves development, testing, and bug fixes made in parallel. Today, non-functional requirement testing happens so late in the process that developers often don’t have adequate time to address performance or security findings. However, a team of developers that has the knowledge and skills to test and is equipped with the proper DevOps pipeline can shorten the bug fix/rework cycle, eliminating the need for scope vs. speed vs. quality tradeoffs. However, that delivery team must be given both the responsibility and accountability for quality.
In agile QE, the merging of development and testing roles is rooted in the fact that it’s easier to teach a developer how to test for quality than it is to teach manual testers to become developers. The goal is to build in quality so that every time you commit code, the build pipeline conducts unit testing, functional testing, and baseline non-functional requirement (NRF) testing against key performance indicators to gain fast, accurate feedback. When combined with chatops, the feedback loop provides real-time response to the team if quality issues exist. The faster the feedback, the lower the Mean-Time-To-Repair (MTTR), as fast feedback both protects quality and conditions developers to build better coding practices. From my experience the faster the feedback loop the quicker I learned, which kept me from repeating the same mistakes.
How Does Quality Engineering Work?
Developers gain QE testing skills through a process of education. Behavior-driven development(BDD) and test-driven development (TDD), for example, can speed up feedback cycles, promoting continuous communication and software usability and introducing automated tests before coding begins.
For starters, the team participates in storytelling sessions from the point of view of the user. A user story consists of feature flows, or scenarios, that use a BDD language like Gherkin to describe the behavior and key details of a particular feature. Gherkin uses a special set of keywords to give structure and meaning to executable specifications. Each keyword is translated into an English language equivalent.
Using Gherkin, team members walk hypothetical users through the feature flow. Individual scenarios can be used as test cases. These stories essentially provide requirements for feature implementations and acceptance criteria. Those acceptance criteria function as three things: the exit gate for your user story, a test scenario, and an automated test script. This approach is called Acceptance Test Driven Development (ATDD).
The typical non-technical role of Product Owners and Scrum Masters can be coached to write automated tests in Gherkin (English language) and trigger those tests with the push of a button. The point here is to have the whole team actively participating in story generation in order to achieve common understanding of each feature being developed. In essence, the organization builds in quality while simultaneously enabling all members on the team to read, write, and run automation. This approach lowers the barrier to entry to test automation.
Quality Engineering Transformation Best Practices
With QE, the focus shifts from traditional QA defect discovery to built-in quality through defect prevention. Built-in quality means quality is integrated into every phase of the software development lifecycle (SDLC). The result is improved quality, faster development speed, and greater innovation – especially when you take into account the quality engineering transformation steps described below.
Step 1: Make Everyone on the Team Responsible for Quality
Transitioning from traditional QA to QE isn’t easy, and building a QE practice won’t happen overnight. Organizations will need expert guidance. You can’t achieve continuous QE testing until every member of the team takes personal responsibility for quality. It also helps to have a solid DevOps pipeline in place to help transform siloed ways of working.
Quality engineering transformation practices call for team-wide alignment. A delivery team might have a QE coach, a general practitioner who teaches developers quality testing methodologies. QE coaches need to be both highly technical and highly collaborative. They should work closely with a team of developers to review testing notes and teach them automation tools and techniques.
Step 2: Establish Clear Acceptance Criteria Up Front
A QE coach might teach developers and testers to write Gherkin. The scrum master or product owner can provide English language acceptance criteria with engineers to give them an automated tests before they write code for their story and lower story rejection rates substantially.
Hands-on exploratory testing catches issues that automation can’t. QE coaches can write up testing notes to provide exploratory testing advice to developers, as well as introduce pairing sessions, where the developer and a peer (a developer/tester) discuss features, implementation, and risks. Based on these discussions, the developer should have a good idea of what testing to do and what to look out for.
And because scripted tests limit experimentation, quality engineering transformations call for testing notes to include tips vs. prescriptive instructions to increase the quality of developer testing dramatically.
Step 3: Take Peer Reviews Seriously
With sufficient training and practice, developers can get good at testing each other’s code. However, not taking peer reviews seriously can lead to the traditional QA safety net dilemma, where developers are less thorough in reviewing their own code because they know someone else will catch their mistakes. It is critical to point out error tendencies and patterns to the developer at the outset (first line of defense).
Quality engineering transformations involve teaching developers to ask themselves the hard questions and do most of the critical thinking themselves. The developers become responsible for testing their features from story to production with team ownership and QE coach oversight and support.
Step#4: Implement Proven Accelerators
Enterprises should also bring open source / vendor-agnostic accelerators to the table by working with a DevOps partner that has instilled QE and continuous testing principles into their own organization.
(Liatrio is in the process of building just such an accelerator. You can learn more here.)
Once the delivery team has adopted a QE mindset, an organization can implement proven accelerators to expedite the delivery of high-quality products and features. Subject matter experts (SMEs) can help delivery teams design and develop continuous baseline performance tests and other non-functional requirements. These baseline NFR tests should be implemented as quality gates on all code commits.
Traditional QA vs. Quality Engineering
Traditional QA vs. Quality Engineering
Quality Engineering Transformation Fosters Continuous Innovation
Pop Quiz: You have a team 50 developers. Like most successful organizations, you need to ship high-quality releases to customers every couple weeks in order to meet demand. The stakes are high. The pressure’s on. How many QA testers do you need on your team?
Answer: 50 — the same 50 people who are doing the development work.
If you were expecting me to say you’d need 50 QA testers to match the 50 developers, my answer may have surprised you. But by now you probably know I’m not talking about traditional QA practices – I’m talking about quality engineering transformation. Quality engineering transformation has the potential to enable your team to deliver higher quality products and features at much faster speed.
I have participated in quality engineering transformations on multiple occasions, removing manual, repetitive tasks and freeing people up to solve more challenging, higher-order problems. The result is increased speed, improved quality, and a coding engine that runs more smoothly because there’s more developer/tester horsepower on the team even though the team itself may have fewer people.
In my experience, organizations undergoing a quality engineering transformation often use static code analysis tools such as SonarQube, which enables you to set and measure quality thresholds and quality checks. Teams can build quality gates into the automation so the majority of the testing once done manually can happen automatically with every code commit.SonarQube offers enterprise-wide dashboard visibility, a code audit trail, and fast feedback on quality performance. Quality is built in.
DevOps/QE transformations also lend themselves well to innersourcing, which involves open source collaboration inside your organization’s firewall. Innersourcing applies lessons learned from open source code development to internal team software development and operations, which can facilitate DevOps onboarding and greatly accelerate the production of high-quality code.
QE Revolutionizes Development, Operations, and Testing
Let’s say you currently have 120 manual testers. If your organization undergoes a quality engineering transformation, you might be worried that you’ll be down to 60 within 6 months and down to 30 within 12 months. When the dust settles, will traditional QA testers still have a seat at the table? Can QA testers evolve into quality engineering coaches whose job it is to educate and uplift the rest of the team while expediting high-quality releases? Right now, you’re commanding a field full of troops. But what happens when the troops dwindle?
Quality engineering transformations necessarily involve a drastic shift from the siloed way technical teams have long been structured. When all’s said and done you’ll be left with a smaller, more effective team of elite special forces (think Navy Seals). Quality engineering transformations usually happen as part of a larger DevOps cultural transformation. Instead of independent departments handing code back and forth, ownership of development, operations, and quality is transferred to the delivery teams and to the individual developers themselves.
The quality engineering best practices I’ve shared here can encourage developer ownership of feature development, eliminate time-consuming testing steps and unnecessary processes, and replace manual work with strategic use of automation. They can help organizations achieve consistently and quantifiably higher quality releases and foster continuous innovation throughout the enterprise.
Organizations that implement these best practices are returning testing responsibility to the developers, preventing rather than detecting bugs, promoting continuous innovation, and revolutionizing the way development, operations, and testing are done.
Interested in participating in or learning about Liatrio’s new accelerator? We’re at the early stages of developing an open, lightweight, extensible agile quality engineering testing framework, and we’re planning to share the project with the community. Reach out if you’d like to learn more.
