Did You Consider These 6 Organizational & Structural Aspects For AI Teams?
How to design and lead an effective ML/AI organization
It’s not a myth, you absolutely can go and sit down to build a great model all on your own. However, that may not be easy and may result in a not particularly interesting outcome. Whether you are an executive with AI on the agenda or an engineer wondering how to organize your individual initiative, an appropriate setup is critical to success. What does that really mean in practice?
I will unpack how to establish a successful AI initiative based on personal experience of more than 10 years of repeatedly building AI and data-driven teams from scratch. That said, this is my humble attempt to cover a selection of topics I have found to be important. Your milage may vary, and any feedback is highly appreciated.
Machine learning initiatives just don’t work great in isolation. However, there is more to an organization than culture alone. My research on organizational design for knowledge transfer has indicated that people and trust are at the core, supported by good structure. Here I will focus on these observations, starting will cross-functional collaboration as a key structural element.
United federation of analytics
Let’s get started with demystifying cross-functional collaboration. Such a complicated word for something rather simple. Talk to each other.
Truth be told, that happens far less in organizations and often, the reason is that there are conflicts between departments or people. It's a leadership challenge but executives and board members can make it easier: avoid creating an ivory tower data science department. It didn’t work for software architecture either. Nimble, empowered teams all across the organization with a common role model, shared culture and infrastructure work better. Imagine a bridge would be built from a single pillar; would that work well?
A federated structure, in my experience, has a far better chance to establish direct communication between model designer and the user and decision makers than large central teams. Being embedded with functional teams facilitates cross-functional collaboration by design. While some form of data, analytics and perhaps simple coding training may still be required to enable the organization working alongside each other helps with tacit transfer of knowledge. This in turn, provides a potential buffer against data and statistical fraud.
Role inflation
The second observation concerns role inflation. Most organizations need more connected and fewer roles to make AI teams work well, not more. Role inflation is bad for two reasons. Primarily because the transition is causing initial ambiguity. This may lead to either rejection or crafting of individual interpretations, no matter how hard the organization tries. This results in imagined boundaries of work and the classical silo-thinking of “I thought you were doing that” or “That’s not my job.” Not very agile and entirely frustrating when the moment comes just before go-live. Generally - and within reason - everyone shuold be motivated and empowered to contribute in an agile setup. Leading by example is a powerful motivator. Even if it would be great in the context of an AI initiative, it doesn’t always have to be writing transforms, specifications or documentation. Co-creating user interface design or ordering pizza demonstrates engagement. This should allow leaders at any level to get personally involved in the initiative. Imagine a mountain guide, what is likely more reassuring, the one leading in the front or the one sitting at the hut providing guidance on the radio?
The second issue with role inflation is that it may deter talent from joining or talent decay in the organization. Every job is the job before the next so people want to gain useful experience. If the experience results in a lock-in, that is not useful. To appreciate the impact, it may be helpful for leaders at all levels of machine learning organizations to have and, if possible, to retain some technical hands-on activities once in a while.
Role layering
That said, there probably is a case for a few roles apart from the classical agile scrum roles in a machine learning focused organization. Before jumping into these special roles, let’s establish what roles are.
To appreciate the difference of roles vs individuals, note that roles do not equate people. First, one person can fill multiple roles. In a way, this can be described as “role-layering” — such as a senior engineer holding a tester role. Second, since it seems that about 30% of questions regarding scrum masters concert the technical vs non-technical nature of that role. I will share my observation here, i.e., a totally unscientific and judgmental heavily biased point of view: non-technical scrum masters seem to struggle more than technical. I noticed that technical backgrounds help and non-technical backgrounds often lead to reporting style meetings — easily mitigated by some form of technical training (see below).
If additional roles can be limited through “role-layering”, this still leaves a few tasks in machine learning that genuinely are different.
Some person needs to be in charge of the model design — Imagine an auditor wants to understand where all that money went, who will competently explain? Whether that is a machine learning engineer or data scientists or software engineer or visual analyst or UX response engineer depends more on style than substance. Similarly, possibly a different person than the designer is taking care of the model evaluation, and again, that could be someone on the team already, such as a tester.
Additionally, someone is bound to operate the model after it is deployed. Making the model designer responsible may help to align incentives appropriately and to keep role inflation in check.
Finally, addressing data engineering is the hardest part. Conceptually data is part of the model design, realistically few people designing a machine learning model are great at designing data structures well, let alone entire data models. That role probably is very different. However, many organizations tend to make data engineering somehow an inferior role. Often asking data engineering roles to prepare data for the machine learning engineers. I have observed this kind of role design leading to unfavorable team dynamics and all engineering roles are best kept equal in true spirit of agile principles.
Let’s take stock: There is really no need for any new roles, except perhaps for the data engineer. Role inflation can be limited by mapping model design, testing and model operator to existing scrum roles. Calling the roles whatever works is fine, as long as hierarchical responsibilities between roles are omitted. Chief Vector Wrangler would be OK if it doesn’t make any other role inferior and (at the time of writing) it would still be quite a unique title...
Serving leaders wanted
Authoritative leadership is generally not liked by anyone, except perhaps the people in power. An empowering leadership culture best fits with a serving leadership style. While agile in nature serving leaders are far from idle. In fact, coaching and training people in direct and indirect reporting lines all across the organization on anything from statistical reasoning to Python may quickly lead to a bit of a challenge. This type of enablement through training is often neglected but vital to success.
Setting out to establish an organization remotely associated with machine learning will inevitably have to deal with people having a diverse level of skill and seniority. Sure, there is hiring and many just do that. Suppose there are only two or three experts missing on the team, that is the thing to do. However, the lead time to recruit, costs and most importantly, the negative impact on existing employee moral easily becomes an issue at scale. Do organizations really want the very people needed to support the AI initiative to be afraid because of job security concerns or alienate them because they are considered unfit to do this new work?
The federated machine learning organization needs not only to be empowered to act independently but also needs the skills to do so. Whether people sit down together and pair-program or setting up a whole company-wide training program is a matter of scale, but knowledge transfer is absolutely necessary. To get the training started infrastructure does not have to be very sophisticated. Even if specialist machine learning tooling is being deployed, most core skills are tool agnostic. Writing a SparkML pipeline, learning about data splits in cross-validation or creating useful statistical plots can all be done at Kaggle or Google CoLab, at zero cost.
A 55-year-old sales specialist with loads of theories from a lifetime at the front will become a sales super hero once he has mastered a little bit of data science skills. To establish such direct expert-to-modeling engagement, a culture of direct communications is critical. Clearly, the training will take time, possibly a little longer than recruiting experts. To bridge the gap, you can always get a few consultants and avoid disengagement of employees and foster support for the AI initiative.
Tolerance considered harmful
In addition, to enable and engage the organization trust in the input data and model is essential to gain the support of users. Support for an AI initiative is vital because just tolerance won’t work. A model may get deployed and even processes backed by shiny new AI models may be initiated, but if the initiative fails to get the support of the organization the desired outcome would be elusive, potentially harmful.
To see why this is important, just imagine how good a model would likely perform if users would never report issues, say, false positives, and just ignore them?
Perspective of future work
The cultural aspects discussed earlier and organizational and people measures outlined above are, in my experience, helping to establish trust and hence support. I like to believe the mechanisms at play are primarily removing the information asymmetry often associated with AI projects. Through the direct involvement in cross-functional squads, people directly observe how good the data is. Additionally, through the training, everyone can learn how their role can adapt from performing a task manually now to tuning and optimizing a model that will do the task on their behalf later. Removing fear through transparency and giving a perspective to future work is a powerful change agent to any initiative and even more important for AI initiatives.
The bottom line
The organizational dimension of AI initiatives extends from classical project management to specialized statistical evaluation and there is not a snowballs chance in hell to cover it all here. My own humble observations boil down to:
- Decentralized cross-functional teams
- A limited number of different role types
- Supervision by agile-aware, hands-on serving leaders
- Training of people similarly important as training of model
I hope to have inspired a few useful ideas that should make you reconsider sitting down alone to build your model. Given how broad the topic is, I clearly haven’t been able to covered it all in this short article, and I like to invite feedback to continue the debate in this rapidly growing field.
Further reading
- The Serving Leader (2016, Ken Jennings & John Stahl-Wert)
- Uncertainty during organizational change (2003, Bordia, Hunt, Paulsen, Tourish & DiFonzo)
- Role Transitions in Organizational Life (2001, Blake E. Ashforth)
