Organizations often struggle to build a culture that enables them to create and sustain high-performing product teams. Teams that don’t produce a high level of quality output often can’t respond to market changes—they don’t focus on innovation and can’t introduce new features and products to their customers quickly enough.
Fostering a culture of continuous improvement and using metrics to evaluate performance can help organizations build product teams capable of performing at a high level. Here, I’ll describe what high-performing product teams look like and introduce the metrics that can be used to gauge team performance. I’ll also discuss how to use the information derived from metrics to gain feedback on individual and team performance and begin to foster a culture of continuous improvement. To show how high-performing product teams can work in practice, I’ll talk specifically about a Liatrio internal engineering program called Flywheel.
Flywheel — An Example of a High-Performing Product Team
Liatrio created an engineering program called Flywheel to help our company build tools and services for clients and our internal business, as well as provide an opportunity for our engineers to work on and learn new technologies. A dedicated set of engineers in Chico, CA, works on Flywheel full time, and every 8 weeks each client-side engineer is rotated into Flywheel in order to upskill on new technologies and capabilities.
Each quarter and each month, the Flywheel team works on a set initiative. At the beginning of each month, the Flywheel team leadership group divides work into one-week sprints. Each Monday, the entire team reviews and discusses the set of stories to be completed in that sprint and aligns on the scope of work.
Stories will already have a set of Subtasks created by the Product Owner. Typically an owner is not assigned to the Story, but each Subtask in the Story will be assigned to an engineer as they start work on that Subtask. The team is responsible for breaking down the Story into more consumable pieces of work as necessary. No sub-task should take more than two days to complete. If the owner believes more time is needed, the task is broken into multiple tasks. Any engineer can step in to work on these sub-tasks and if the sub-tasks are not completed by the end of the sprint fault lies with the team not the individual. This is very important because the team can focus on bettering the overall workflow instead of searching for one individual to blame.
In terms of communication and collaboration, the Flywheel team heavily relies on messaging in public Slack channels and scheduled, open, Zoom video calls for collaboration. Public communication is strongly encouraged over private communication because it allows other team members to be aware of the conversations taking place and to deliver additional insights that would otherwise be lost. Like many high-performing product teams, Flywheel also follows a remote-first principal to ensure all team members, co-located and remote, are engaged in the development process. This is where daily, multi-hour Zoom calls come into play. Engineers are encouraged to work while in these calls with others to allow for the asking of quick questions. Exactly as they would if they were working a desk away from their team.
Below, I’ll highlight Flywheel team tools, makeup, characteristics and values, and regularly scheduled ceremonies.
Flywheel Team Tools
- Work management tool: Jira (see the screenshots below)
- Shared workspace: Confluence
- Development platform: Github
- Collaboration tools: Slack (see the screenshot of a sample Slack interaction below)
- Video conferencing: Google Hangouts/Zoom
Flywheel Team Makeup
- Product Owner
- Lead Engineer
- Scrum Master
- Full-time Co-Located Engineers
- Full-time Remote Engineers
- Rotational Engineers
Flywheel Team Culture: Characteristics & Values
An organization’s culture is transformed as it empowers people to begin a journey of evolutionary change. In a transformed organization, each team member strengthens individual competencies, and teams behave and work in a way that fosters open communication, rapid feedback loops, transparency, and continuous learning. All the while, teams continually improve their way of working and create more tangible value.
With these thoughts in mind, key characteristics and values of Liatrio’s Flywheel team include autonomy, empowerment, safety, self-governance, clear goals and deliverables, and continual learning.
Flywheel Team Ceremonies
Ceremonies are the regular meetings and gatherings needed to enable the team to carry out day-to-day work and maximize their productivity. Each ceremony clearly defined purpose, cadence, and timeframe.
Quarterly Ceremonies
- Full team retro (multiple teams come together to share developments and learning)
- Offsite team celebration
- Architecture review/future work planning (including architecture diagrams -- see the screenshot below)
- Design sessions for new features/work
Monthly Ceremonies
- Four one-week sprints
- One monthly sprint overview and planning session
- Leadership planning and work review for the upcoming month
- Leadership prioritization of work for the upcoming month
Weekly Ceremonies
- Monday morning sprint planning session to review the scope of work for the week
- ~~Engineers break down tickets into consumable half-day work blocks
- ~~Engineers create new stories if work is too complex or ask to create research/spike stories if the original story has a lot of unknown
- Friday demo/retro in the later part of the week to showcase work done and discuss improvement opportunities
- Friday team leadership ensures the backlog is groomed, prioritized, and ready for sprint planning on Monday morning
Daily Activities
- Standup first thing in the morning every day but Monday (discuss what was completed yesterday, what to focus on today, and any roadblocks)
- Scheduled collaboration sessions with other engineers (either in person or via video)
- Daily commits of code and pull request submissions
- Ticket updates on progress
- Posting of progress and challenges into a public Slack channel
Flywheel Team Metrics
The metrics described below are leading indicators of how Flywheel is performing. These metrics can be used to set targets, but the most important thing is for our organization to use the information to identify current team progress and help the team improve.
Overall, Liatrio sets targets for teams to trend towards but doesn’t set incentives for hitting specific targets or goals. Rather, we review data point trends and provide incentives to teams that are on a solid improvement trajectory and maintain a high level of output. If a team is found to be underperforming, then it’s time to review the data to see if the team is struggling with specific areas and work with that team to help them improve.
Here are the five initial metrics that Liatrio tracks:
- Number of Pull Requests Merged to Master – This data is derived from a pull request report for each team. The trend should move upward over time before eventually leveling out based on team size and maturity. The team should also merge into Master daily.
- Deployment Frequency to Dev Env and Prod – This data is derived from Jenkins or the deployment tool being used. Teams should deploy to a Dev Env at least daily and to Production at least every two weeks—ideally whenever they are ready.
- Lead Time for Changes – This data would also be coming from your deployment tool. This is how long it takes for your team to go from a code commit to running in production. For high-performing product teams, this should be less than one day.
- Time to Restore Service – How long does it take to restore service when an unexpected incident causes an outage? Ideally teams should be able to restore service in less than one day.
- Change Failure Rate – What percentage of changes to release need to be rolled back or cause a service outage? High-performing product teams should strive for less than 15%.
Additional (more advanced) key metrics are as follows:
- Knowledge Sharing (Help Others) – This data, derived from a knowledge sharing report, highlights how many people are reviewing each other’s code. A high degree of knowledge sharing is ideal. Watch out for scenarios where the same people review and approve each other’s changes, which can lead to silos within the group.
- Ratio of New Work to Legacy Refactoring – This ratio should reflect roughly 75% New Work and 25% Legacy Refactor. The trend should show an increase in New Work, but the Legacy Refactor percentage may increase when the team is focused on reducing tech-debt.
- Percentage of Churn – This percentage is the amount of rework that has to be done on commits that are less than three weeks old. Rework is work that wasn’t defined/scoped correctly or that has issues discovered later. Ideally, the data point should trend downward over time.
- Percentage of Manual vs. Automated Tests by Repo – This data is tracked via the test case management solution being used. Teams should trend toward increasing automated tests and decreasing manual tests.
- Cycle Time – The total time from start to finish of your delivery process. This data is a combination of data from Jira, Jenkins, and Source Control. It is by far the most difficult metric to track since it requires multiple data points (e.g., Ticket Moved to Progress, Branch Created, First Commit to Repo, Branch Merged to Master, Ticket Closed, and Code Deployed to Production). Teams should trend toward shorter timespans.
Considerations for High-Performing Product Teams
Here are some additional considerations for how high-performing product teams work in practice:
- A newly formed engineering team needs roughly three months to get into a good workflow cadence with predictable outcomes.
- Teams need to remain together for long periods of time (a year or more). Rotating team members out frequently disrupts teams and leads to team reforming and norming time, resulting in reduced periods of productivity. (This is something we are working on addressing in our Flywheel rotations, by including them in the demo before joining and select an engineer to pair with them for onboarding to the team)
- Product ownership and appropriate staffing are key. Teams need full ownership of the product they are developing including inception, delivery, and ongoing support.
- Leadership must explain acceptable business value priorities to an engineering team. If leadership doesn’t provide these options and tradeoffs, then the engineering team will make decisions on their own, which may lead to missed high-priority deliveries or poor outcomes.
- Leaders must ensure that engineering teams have large blocks of contiguous time available each day for work. This means scheduling standups in the morning, grouping meetings together in either the morning or late afternoon, and protecting a 3-4-hour focused time block daily to enable the team to function at a high level
Reach Out!
In an upcoming blog, we'll have a lot more to say about the ways in which Flywheel fosters a culture of continuous improvement and uses metrics to gauge individual and team performance and build high-performing product teams. In the meantime, contact us for more information or reach out on Twitter @Liatrio.
