Staff and Principal Engineers: why do we need them now? (Part 1)

More than an answer, we have a story. More than a story, we have an evolution log of how we at commercetools built our own Tech Leadership track, from the early stages of a ‘Tech Lead’ to the latest iteration within our organization structure: introducing Staff and Principal Engineer roles.

Our product is an API-first, cloud-native SaaS with API performance, uptime, data security and developer experience at the core of our value proposition. Whenever we’re describing what it takes to build our product, two essential characteristics come up: technical excellence and talent. We believe both are so important that we have created dedicated roles to cater to their specific needs, so we never have to compromise one or another. That said, like many other companies out there, we now have a People Management track (the Engineering Manager) focused on investing in our talented people and a Tech Leadership track committed to driving technical excellence.

The Origin Story of Tech Leadership

I’d say we’ve been practicing tech leadership from the get-go, and it is something that can (and should!) be practiced by engineers, independent of the title. When it comes to People Management, I won’t linger too much here as it’s not the focus of this article. Still, I will say that the need to formalize the People Management track and add Engineer Managers will always be more apparent and feel like a natural development. The shift to a formalized tech leadership structure is less obvious. However, when we noticed that tech leadership had become the person’s main task, we sat down, set clear expectations and responsibilities, and created the Tech Lead role.

We appointed the first Tech Lead in the second half of 2018 (when we were around 30 engineers) and the first three Tech Leads were all based in a team. But that changed a year later. The infrastructure team split up into three teams, with the fourth Tech Lead taking care of the Merchant Center, our back-office UI (which is written by multiple domain teams). Another Tech Lead role evolved into helping multiple teams.

We hadn’t planned to switch from team-based Tech Leads to Tech Leads helping multiple teams. We just responded to the changing needs of a growing organization. Some Tech Leads still had a “home team” where they would attend the standup and retro, but they would be working outside their teams most of the time. The advantage of having had the experience of working from within a single team (e.g. the infrastructure team before being split) gave Tech Leads a good overview of the technology each team owned while also building strong working relationships with the members of each team. Likewise, the small teams after the split allowed Tech Leads to have a quick glance at most non-trivial pull requests and follow discussions in code reviews.

The Growth Challenge

As teams grew, our roles shifted from helping a single team and coordinating with a few other teams to assisting multiple teams directly and coordinating the efforts with various other teams. However, the individual teams still needed Tech leadership. In some teams, individuals stepped up and took on TL responsibilities. In other teams, the tech leadership was missing entirely, creating problems along the way. We didn’t like where this trend was going. Additionally, even if we had ignored the missing tech leadership in individual teams, what became apparent last year was that Tech Leads could no longer handle the work to help all topics that needed cross-team work. Our roles were too broad (e.g., “Infrastructure” or “Backend”) and needed to be narrowed down.

It’s worth noting that other roles, such as Engineering Managers or Product Managers, automatically get opened up by creating new teams. I’ve seen this referred to as “gearing ratios”, e.g. a ration of 1:7 for Engineering Manager to Engineer. For tech leadership roles, this automaticity was missing. We had more than doubled the number of engineers but only added a single Tech Lead in the last two years.

With all of this written down, we can conclude that our tech leadership has evolved from being team-based to being across teams but we actually need it to be at both levels. Moreover, we see this as a fantastic opportunity for our internal talent to grow into tech leadership roles.

The Learned Lesson

Looking back, we understand what happened. We’ve learned our lessons and reflected deeply on these critical moments in our history as a team. From this new vantage point, we are thrilled to be introducing two very challenging and rewarding roles, true stepping stones for achieving a high level of mastery. In the second part, we’ll be diving deeper into the Staff and Principal Engineer roles at commercetools.