“The Best AI Tooling Is Almost Invisible” – Peter Boersma on Agencies, Organizational Will and How to Start Design Ops

Most companies responding to AI are doing it backwards, according to Peter Boersma: they write mandates requiring everyone to spend eight hours a week in an AI tool, when they would be better off waiting for Atlassian, Figma and Miro to build AI into the software their people already use. Boersma has spent 25 years designing design teams – as Design Operations Manager at Miro, as Director of Product Design at ServiceNow, as a consultant to IKEA's design and research operations team, and now leading the UX team at the pan-European medical supplies group Mediq. His discipline, design ops, is the work of creating the circumstances in which good design can happen: tooling, processes, planning, measurement, culture. He talks about what is actually reshaping design organizations – COVID, AI and cost pressure – why design ops cannot rescue a company that doesn’t want good design, when internal teams should call in an agency and why they usually call too late, and how to start doing design ops before anyone holds the title.

 

Let's start with scope. What does design ops actually cover, and what determines how much of it an organization needs?

Peter Boersma: It depends on the needs of the larger organization. If a company is, let's say, a 100% SaaS company, then design should play quite a big role. Currently I work for a company that is mostly a distributor of medical materials. There, design is a side function, a fringe function that is supposed to make it possible for people to order the products, while delivery, buying and selling, and the finance flows are much more core to the organization. So every organization should have the design organization that it deserves – and design ops goes with that flow.

That also means design ops isn't necessarily in charge of what the shape of a design organization is, or how it uses its tools, but it should support the selection of tooling. That could be design tooling, overall project management tooling, culture tooling, metrics tooling, or quality assurance tooling for design. Tooling can also mean processes and procedures, or the alignment of design work with the work of the people and stakeholders around them. For me, design ops is also culture building: making sure people are onboarded in the right way, that they understand how the company is organized and how designers play a role in projects and in the company as a whole. And then things like measuring design, measuring tempo, resourcing, planning and quality also fall under the responsibility of design ops.

 

Before we go further, it's worth establishing what kind of organizations you are speaking from.

Peter Boersma: One thing that is good to know is that in recent years I have mostly been working for enterprise-scale companies. Like Miro and ServiceNow, and before that SDL – Miro was already 1,200 people when I joined, and 2,000 when I left. I have supported IKEA in their design ops team, and they have something like 300 designers and researchers, a ten-person design ops team, a ten-person design system team. Larger companies. So some of my thoughts will be coloured by that. If anyone reading this in a smaller organization, they need to take my opinions with a grain of salt, because they may not be applicable to their context.

That's definitely my way of saying "it depends" – which I'm only allowed to say a couple of times a day, and every designer is only allowed to say a couple of times a day. But my advice should be seen in the light of enterprise-scale organizations.

 

So what has actually happened in this field in recent years?

Peter Boersma: Well, design ops happened. AI showing up everywhere happened. And COVID happened – so a lot of people had to learn how to work together through digital means, instead of just sitting in the same room and being able to point at the same whiteboard.

 

When I look at the changes brought about by COVID, they seem to have had a varied impact. Smaller organizations transformed quite a lot – many of them became genuinely digital – while bigger enterprises are switching back to older ways of working, to a mix of physical and digital. When we work with bigger enterprises, we see them starting to operate in a pre-COVID way again. What are you observing, and what is your experience with this?

Peter Boersma: Well, there is the typical saying that it's really hard to steer a big container ship. Enterprises are big container ships. They are slow to change. They sort of managed to adapt to COVID by sending everyone home, getting smaller offices and buying a lot of Miro licenses.

For smaller companies this became an immediate digital transformation. For larger enterprises it was more like: ah, we could change some things – let's build a digital transformation program, a function, whatever; let's hire the big five consultancies to help us with this digital transformation. Which means it is still ongoing, and in some places still starting. Sometimes it has failed miserably, and then they said: let's just get everyone back into our offices, we'll buy bigger offices again and go back to the way we used to do things.

For smaller companies, this complete digital transformation made it possible to stay in COVID mode – to stay remote, to stay connected asynchronously – because they were still changing a lot anyway. They were small companies, everything was still in flux, so this was just another thing thrown into the mix.

 

And do you think AI has a bigger impact on design operations than COVID had, or not?

Peter Boersma: This is where my opinion about AI comes in. There are many ways to attack this. I think a lot of companies are experimenting with AI too much, and writing things like AI mandates that say everyone should spend at least eight hours a week in an AI tool, or building something for themselves, or whatever.

In my opinion, if companies have a little bit of patience, then the companies that make the tools their employees are already using will build AI into those tools. If you were using Jira for project management, Atlassian has built AI tooling into Jira. You don't need to do that yourself. If you were using Miro, Miro has built all kinds of AI tools into their collaboration software. If you're using Figma, Figma is building AI tooling into their tools.

So with a little bit of patience, just keep on doing what you were doing – this is the enterprise mode. Wait a little bit, and your employees will be using AI tools that are built into the tools they were already using.

Yes, there are opportunities in creating a little bit of AI tooling yourself: some RPA automation tooling, or connecting one tool with another to optimize things for your specific situation. But I don't think that is the best way for employees to spend eight hours of their week. I think they would be better off following a training on how to use the newest AI features in the tools they are already using, than learning how to vibe code prototypes that work for them in some cases and then crash as soon as you hit an edge case.

 

You mean that the skills they develop through this experimentation won't be that useful, because the tools will change anyway – and because they shouldn't be doing these things once they are working with upgraded tools.

Peter Boersma: Yes. And it also gives your employees a chance to work with AI in its best shape. Because not everything needs to be a ChatGPT-like specialized tool. Not everything needs to be a chat. Not everything needs to be a vibe coding tool where you build a prototype. Sometimes the best AI tooling is almost invisible in the tools you're already using.

I've been using large language models in text editors, where it shows up as those little red wrinkly lines under the words – the language improvement suggestion for better phrasing. That is large language models, that is AI and machine learning, and it has been available for a while. And it is a much better way for AI to show up in that particular tool than a chat box that says: "Hey, help me write a better sentence for this."

So the shape of AI can, in my opinion, best be determined by the makers of the tools, because they know their domain. They can have entire teams working on this for three quarters of a year, instead of an employee starting to vibe code with a limited Copilot license that has just been rolled out in the enterprise.

 

Can you give a good example of a tool that has integrated AI in a way that really supports and helps work in a noticeable way?

Peter Boersma: I was an employee at ServiceNow. ServiceNow is best known for their ticketing solutions – if you have a problem with your laptop, you send a message to IT and they will solve it for you. That is one of the fifty products they make.

In this IT Solution Management product, and based on machine learning, while you are specifying your issue to the IT helpdesk, it can already provide potential solutions from the past. As I said, based on machine learning from the past, some smart copywriting and just-in-time editing by combining sentences and fragments, you can get a coherent reply, a solution proposal, for something you have only just specified.

It is not necessarily a chatbot. It is not a new Clippy. It is a response. And this can be made available to the end users who have an issue and specify that issue. It can also be made available to people in call centers who hear your story: the story can be transcribed, and the tool can start suggesting solutions.

Other examples are things like image tagging, for asset management or campaign management. If you are working with a lot of images and you want them automatically tagged, there is AI for that: machine learning, image recognition, based on large collections of previous images. You can automatically tag images with the right metadata, potentially with a human in the loop to approve them – but a lot can be prepared by the AI. Those are applications that are not necessarily a chatbot, that use machine learning to analyze and combine information, and come up with solutions for the people who need to use the software.

 

You mentioned AI and COVID as events with a potentially big impact on design operations. Can you mention other things you see changing this space, or do you think these are the big things and that's it?

Peter Boersma: Well, in general, the global economic situation, where there is a lot of pressure on margins and costs and everything, means that a lot of companies are trying to do more with less, or the same with fewer people.

 

You mean that optimization currently has a big role?

Peter Boersma: Yes. And this is not always based on proven data – proof that something can work, or that deleting a layer of management is the best solution. Companies like Facebook have tried that: just deleting a layer of managers, in the hope that the organization will sort of adjust and be just as effective, but with fewer people.

That's another influence on the shape of organizations and on the way individual members of, for example, design teams need to be supported. If suddenly there is a layer of responsibilities missing, or a layer of people missing who had certain responsibilities, then those responsibilities need to be redistributed. That might mean individual designers need additional training to take on some of those responsibilities, or that more intense communication is necessary between the designers and what used to be two layers of people above them.

So you may need to introduce new types of meetings, all-hands that didn't exist before, or Slack channels for specific topics that deal with this communication between people on different layers, or more PowerPoint decks that need to be sent around instead of one-on-one meetings.

 

Let's talk about how teams are forming. You have shared some scepticism about the use of AI. Many companies imagine the following: they have, say, thirty designers, they fire half of them, they keep the strongest people, and those will be empowered by AI and will happily produce three hundred times more than before. Based on our conversation so far, that's probably not your vision. How do you see this playing out? How do you see teams changing in their use of the tool set or the processes, and how will this senior–junior–AI triangle shift or change?

Peter Boersma: I think the shape of design organizations can stay largely the same. I don't know whether they need to shrink or grow, but the shape – a few super-seniors or principals, staff, mid-level people, juniors, that sort of triangular shape – can stay the same, and all of them can be supported by AI.

Maybe the juniors are supported more by production and generative tooling, but maybe also by quality assurance. The mid-level people can maybe be supported better by AI in existing tooling – so AI in project management tooling, or AI in performance review tooling. And strategists can be helped more by machine learning comparisons with other companies, so learning from the wider community of design and business. All of them can become somewhat more effective with AI.

We are still realizing that we need a lot of humans in the loop. They need to do quality assurance. AI needs an extra layer of AI to do evals, which is crazy – you want that to be built in, and I don't understand why it isn't built in. So learning to work with AI, and teaching AI to be a smart and meaningful contributor to the work, is going to take a lot of time and effort as well. Which means most designers at different levels will not get ten times more productive, in my opinion.

And I have to add: this is about enterprise settings. Sure, an individual designer can suddenly also code, and an individual coder can also design – as long as what they need to produce is pretty similar to what others have implemented elsewhere, and is available to the public and can be scraped by the AI models to be included in their statistics-based recommendations.

If you're working for a very specific market in a very specific team setting, then general AI tooling is at the moment still more of a hassle – or building your own AI tooling is at the moment still a hassle. Using very generic stuff doesn't do that much. And if a very generic AI can do the work that you can do, then the work wasn't too smart to begin with, and you have been replaceable from the start anyway. In my opinion.

And then, as I said, only the AI tooling that gets built into the tools we are already using will make you more effective and efficient – or mostly effective, not necessarily efficient. Because design is still very much about collaboration. It's about people with different skills coming together to compare priorities, requirements, opinions and proposals, and to give each other feedback on the work, so that the collective knowledge can decide what the best next iteration of the work is.

 

If you are moderately optimistic, what do you see in the future? What are the biggest changes you expect in this field in the next five to ten years? Of course it's very hard to predict that far ahead, so: what do you think is a realistic guess?

Peter Boersma: One thing I do see happening is that software generation can be supported by AI pretty well. But this is also a continuation of developments that have been happening since, I don't know, the 1950s or something, where you first had very basic compilers, and software generation was really slow, with punch cards and everything. This speeding up of software development has been going on for a long time, and AI is just another tool in that process.

So I don't expect magic there. I don't expect this concept where everything comes together and there is suddenly super intelligence. I don't think that's going to happen anytime soon. What we are seeing currently is a continuation of the speeding up of the creation of software, and designers contribute to this. Sometimes I joke that we make pictures of software and then software developers make the real software. Let's put it this way: we contribute to the development of software. Designers can contribute to this, and we can be helped with AI tools as well. But it is still a gradual progression. I don't think anything radical is happening in the near future just because there is now AI that supports us – just like I don't think we'll be ten times more effective or efficient or whatever. It's still very much about communication between people with different skills, opinions and decision-making power.

But because software is getting cheaper – because the creation of software is getting cheaper, because there are more tools that help more people do it – it is important that we as designers sprinkle in enough of our knowledge in that software creation process. Which means we need to, at the very least, document our design systems in a way that can be read by AI. So that AI generates things we already agree on: design principles, design tokens, design patterns, all of that. So that the knowledge we have is being reused in the slightly faster creation of software.

 

You mean that if it is easier and much cheaper to create software, there will be a lot of software solutions that have never seen a designer?

Peter Boersma: Yes – a lot of software could potentially soon be created without designers, because the AI tools just use designs from somebody else and repurpose them. But it is our job to make sure the right design knowledge is built into this.

 

So it matters what they copy and what they use. You can copy bad design and you can copy good design as well.

Peter Boersma: Yes. I think this is one way of keeping the human in the loop: making sure that the knowledge designers have already built in is injected at the right moment into this process.

 

How much can design – and design ops – actually influence the maturity and the success of a company? It's obviously an important factor, but what kind of impact does it have, and how big is that impact in your eyes? Sometimes we like to overestimate our own importance, of course.

Peter Boersma: I think there are several things at play. First, design ops is a support function, and it can only support a successful design organization if the rest of the organization wants a successful design organization. So it helps if there are design representatives at a higher level in the organization. If you have a chief experience officer or a chief design officer, these things help in establishing a culture where design is respected. And then design ops can contribute to this, can optimize, can help the design team grow into the optimal place in the organization, where it can make a meaningful contribution to business success. If nobody respects design, then you can promote the hell out of it, but ultimately it's going to fail.

That's one thing. Then, design ops can do local optimizations. We can make designers super successful, really great, wonderfully skilled, with a great design team culture. But if that isn't also connected to the umbrella of product operations – so also product managers, developers, researchers, strategists, data and analytics, marketing – if they have their own ops people and design isn't working with them, then something is wrong as well. Then whatever design tries to do is limited to the scope of design, where it should see its own responsibility within the scope of product ops.

And that means, for instance, that designers shouldn't have their own tool for project management. You sometimes see that the whole organization uses Jira and the designers use Asana or something – just because the screens are easier to read, or because it doesn't look that technical. Worst reason ever. If the rest of the organization uses Jira, designers should use Jira as well. And if necessary you can import and export and use your own tool, but it should end up in the general, product-wide project management tooling: for consistency reasons, for comparison reasons, for business reasons.

So design ops should not work in a vacuum, just like design shouldn't work in a vacuum. It should work under the umbrella of product operations. The same goes for developers. If designers say "here's a lovely design spec, it's in Figma", and it's not connected to any components in code, then you have only delivered half – well, probably less than half of the work. The design needs to be translated into code and reusable components so it can be used by developers, so they can become more efficient as well. Design can contribute to the efficiency of developers by working together on a design system that works both in design and in code.

Those are ways that design ops contributes to the success of design organizations – if the organization wants it to succeed.

 

You said earlier that companies are trying to do more with less. We see those optimization initiatives in multiple places. For example, we are working with a mid-size bank that has a fragmented team setup: they have their internal team, multiple agencies, other departments dealing with research, and so on. We are working with them on optimization as well – they want to cut some costs and at the same time improve quality, because if you consolidate the resources and departments you might end up with both. We used to work with BNP Paribas some years ago, and they did the same. When we started working with them they had one designer, then they scaled up to forty or fifty people, and then they scaled back down to thirty, because they saw that doubling the number of designers every year doesn't necessarily mean more output or higher quality. There are huge banks in the UK with hundreds of designers, and you are not necessarily impressed by their output. And if you talk to the designers individually, they are not stupid – so it's not that they hired the wrong people. But this consolidation is necessary in some cases. How should internal teams and outside agencies divide the work?

Peter Boersma: You probably know the book Org Design for Design Orgs, by Merholz and Skinner. They propose a model where some functions are centralized, and then teams of designers are assigned to clusters of business areas. Ideally not one designer for a scrum team, or one designer for an area, but a small cluster of designers, and then a sort of centralized function for things like strategy and operations, and maybe tone of voice and writing, which could be centralized in some organizations. So: a centralized model plus small teams assigned to business lines.

I like that model. It's not necessarily a full internal agency covering all skills – it's an internal agency for a couple of the centralized skills, plus decentralized, dedicated designers assigned to business lines. Over time they can rotate, and over time they can grow, depending on the needs of the different business lines. That's the model I like for day-to-day work.

Then, if there is suddenly something drastically new, you can choose to hire an agency, either for additional resources – so just extra hands, or hands and brains – or a dedicated agency that has certain skills you don't have in house at the right capacity. The outside agency does something the internal model cannot fulfil: for example bringing in AI knowledge, or bringing in branding knowledge. A lot of product teams working on existing digital portals or digital products don't always have branding and hardcore visual design skills in house, because there is an existing design system and they just keep working on that. But for a refresh, a rebrand, a new tone of voice, or a new strategic direction where you address a different set of customers, you bring in an outside agency that temporarily brings in skills you don't have.

 

The tricky thing we observe is this. It's relatively easy to understand when you need extra hands, because work is obviously piling up, so you recognize that you need some more people temporarily. But missing skills are much harder to recognize. And in some cases we see that internal teams consider it proof of their own inefficiency or incapability if they need external people. So it isn't read as "hey guys, we need extra brand experts, because we are UX people, not branding designers" – it is read as "we should know branding, but we don't, so this shows we are not strong enough." We have often seen internal teams struggling in these cases and inviting external experts a little bit too late, when it is already obvious that they have failed. What is your experience in this field?

Peter Boersma: In the past I have been a process design consultant, or a design process consultant, and more recently a design ops consultant – and there I basically have the same problem. My clients need to come to me when they realize they have a problem they cannot fix internally. That means they have to be willing to search for an external consultant to help them. And then they need to find me. That last step is the only one I have influence on: I can do marketing for myself, I can speak at conferences and be known as an expert. But those first three steps need to happen internally. The company needs to recognize they have a problem; they need to realize they cannot fix it and not feel ashamed about it; and then go looking for someone on the outside.

And yes, in some cultures this brings on feelings of shame, feelings of inadequacy. It brings on people trying it anyway and overestimating their own skills. In other cultures it is accepted that sometimes you need outside help. Everyone has their own responsibilities; other people have other responsibilities. Dutch culture is very much non-hierarchical: my manager doesn't have more responsibilities, he has different responsibilities, and that doesn't put him above me – it's just a different set. And the same for the CEO. So we will attack the CEO for, I don't know, providing bad coffee, even though it isn't ultimately his responsibility – but it is something he has some influence over.

Realizing that you cannot fix this as an internal team and may need outside help is influenced by culture, I think. Both company culture and national or regional culture.

 

During our conversation you have emphasized that design ops has a supportive role, helping organizations work more effectively. What is the relationship between design ops and design strategy?

Peter Boersma: When I say design ops is supporting, that doesn't necessarily mean it is lagging. Design ops can lead the way and can lead design organizations to a better position. They don't always have to wait for requests from design managers – "hey, can you improve the planning process for me please". No: design ops needs to listen to the organization, put their ear to the ground, have conversations, and figure out where the pain points and the opportunities for improvement are.

Having said that, in my opinion design strategy is something that happens more top-down, and design ops more bottom-up. Design strategy is a translation of business strategy into the work designers need to do, and the direction in which design activities are going. In that sense, design ops can contribute by disseminating information, and – I'm temporarily moving from design ops to research ops, which is often grouped together with it – by making sure we have insight engines: collections of research that can be queried to produce just-in-time insights for problems that are arising. And those problems are arising because a strategy is shifting. A strategy shifting means we're forgetting about some old things and moving into new territory. New territory means new questions, which can be asked of the research insights we already have – but maybe we need to do new research, and then research ops can also help.

But for me, design strategy and defining design strategy is something that happens more top-down, and design ops is more bottom-up. Design ops can support the definition of strategy, the translation, the dissemination, finding supporting information for strategies, and any of the work that follows from a certain design strategy – because that's just design work, and that's being supported by design ops anyway.

 

I have seen that many big organizations have a relatively big design team and some kind of processes – so they operate somehow – but they completely or almost completely lack a design strategy. How can design operations help in this situation? First of all, how can you run design operations without a strategy and still provide value? And second, can design operations contribute to the birth of a design strategy in a company – can it inspire people at a higher level, so that they don't just copy the competition or fall back on familiar solutions, but so that solid processes lay the ground for something new to emerge?

Peter Boersma: In a sense, as I said, one of design ops' responsibilities is to have their ear to the ground and listen to the issues designers have. And if there is no design strategy, then designers will have issues. They will have issues with prioritizing work, with evaluating their work – whether it is good enough, whether it is going in the right direction – and with determining whether design is contributing to the larger business. Because, as I said, UX strategy should be a translation of the business strategy.

So if design ops realizes through their listening sessions that designers have these issues, they can surface them, and at least the need for a design strategy will become clearer. They can help make the need for a design strategy more visible.

Whether they can help with creating the strategy, or rolling it out, or supporting it, depends on who becomes responsible for the design strategy. If you're lucky, design leaders can take on that responsibility. If they are purely people managers, then they will probably suck at it. If they are design leaders in the sense that they can define direction based on requirements from the outside, and are creative enough to come up with something that is a design strategy, then design ops can see how those people can be supported. As I said, that can be in terms of research, providing the right insights that support the strategy. It can also mean identifying which designers, based on their skill set, would like to contribute to these kinds of projects – vision projects, or illustrations of the future once the strategy is rolled out. Those kinds of skills are necessary. Some people may be more motivated and more skilled to do that work, and design ops should have a good feeling for the skill sets available in the different teams or individuals, so they might be able to provide ideas for resourcing the team that develops the strategy, or the illustrations, or things like that.

Those are ways I can see design ops contributing to developing strategy.

 

What if there is a company that doesn't have a design culture, doesn't have design ops as a role, and isn't working on it? What is the first necessary step to introduce this attitude, or introduce this role? Do they really need a person called design ops manager, or expert, or whatever – and how do they get started? Where there are established methods, improving them isn’t easy, but at least there is something to work with. What I see is that many organizations, even big ones, are completely lost about where to start. They say: "Hey guys, we have some processes, but they are in chaos. We have thirty people and we don't know how to put them into the same system – different departments, different subsidiaries, and so on – and we need this person, but okay, maybe next year." So what is the next step, or what are the next steps, when you want to have a design ops approach?

Peter Boersma: In my workshops about design ops and design ops roadmaps, I make the distinction – and I'm not the first one to do this – between the design ops role and the design ops mindset. You don't necessarily need anyone with the role of design ops manager or specialist or whatever. If enough people have the mindset, you can already get a lot of work done.

At some point it helps to have someone with the role, as a point of contact, as a logical place where coordination happens, and as a point of contact for other teams as well. But if you have enough people with the mindset, you can rotate that responsibility: hey, this quarter you're the point of contact, next quarter you're the point of contact. And then you have people not just executing potential design ops initiatives in a guerrilla style, but also someone coordinating those initiatives – doing an inventory, prioritizing, writing briefings for the design ops initiatives, distributing the work and measuring results. Those are the five steps in my workshop. If someone does that, if that coordination is done, then design ops work can still be pretty successful without anyone holding the role.

You asked where you start, where you kickstart. I would say you can kickstart by doing the first couple of those five steps and creating a draft roadmap that says: this quarter we're going to address these things, next quarter we're going to address those things, and later we're going to address more. You don't have to look much further than two quarters, especially when you start.

You may also want to identify some themes in which you want to make progress in improving the circumstances for designers. Design ops people love their P's, so it could be people, process, products, procurement – different areas in which you want to make progress. People could be onboarding, or career ladders, or skill inventories. Process could be the definition of process, or focusing on intakes at the start, or quality – that's more craft, the quality of the work – or speed of the work, things like that. Procurement could be how we deal with freelancers, or how we deal with, for example, the AI budget: how much AI budget do we need? It is a new form of procurement.

After the first steps you can get the first initiatives spread out over your first stab at themes on a roadmap, for the team to have the feeling that we're going to accomplish things, but also as an initial communication tool towards design leaders or other ops teams, saying: these are our plans to improve the circumstances for designers, so we can be more effective, happier, more productive employees, and so the design team as a whole can contribute to the business in a better way.

 

A last question: when you are working, what inspires you? Some people get inspired in the shower, on a hike, over a game of backgammon. What gives you energy and inspiration in your work?

Peter Boersma: When I joined Miro, my manager at the time – the manager of the product excellence team, the product operations team – said: "I expect adults in the room." You can translate that as: people need to bring their own professional experience, knowledge and motivation into meeting rooms, into the work. And I agree with that.

I like it when people with different opinions, experiences and knowledge come together not to fight it out, let’s say, but to make progress. Seeing that happen is inspiring to me. It could be just designers talking to other designers, or a designer talking to a business analyst, or a designer talking with a domain expert who is super knowledgeable in – in my case now – diabetes care. Seeing two or more people come together in a room is inspiring to me. It makes me want to support them more, and this is where the design ops person in me, but also the design manager in me, comes in. I want to make sure that more of that happens, where people can bring their experience, their motivation and their knowledge into the room to make good things happen.