Barry O’Reilly

Criticality is the ability of a system to reconfigure itself in order to cope with unexpected circumstances. Criticality can actually be measured in terms of the number of components and the number of connections that those components have between each other in software. And in organizations, then, we’re starting to see the same thing. And what we do in software is that we use the concept of stress, to talk about the different states that the environment can arrive at, depending on if a competitor drops their price, or if there’s a pandemic or a war or something like that. And that will allow us to investigate the system without getting hung up on things like probability, or whether something’s actually likely to happen.

At the end of the process of residual analysis in software, the actual stressors that we come up with the actual things that we propose are going to happen in the environment, they don’t matter. It’s the criticality, the flexibility that’s built into the system when we do this. That’s important.

Simone Cicero 

Hi everybody. Welcome back to the Boundaryless Conversations podcast, where we meet with pioneers, thinkers, doers, and entrepreneurs. And we speak about the future of business models, organizations, markets, and society in this rapidly changing world we live. I’m Simone Cicero, and today I’m joined by my usual cohost. Stina Heikkila, hello, hello. We’re also joined today by Barry O’Reilly. Barry is a software architect with more than 25 years of experience in the It industry. He has been in leading roles in global software companies and also spent many years educating architects. He is currently pushing a Ph.D. in Complexity science and software engineering, and he’s on the podcast today to talk about the residuality theory, of which he is the creator. This theory is a new approach to building software in a way that we believe it’s informative also for how we build organizations and more. First of all, thank you for being here, Barry. And hello, welcome to the podcast.

Barry O’Reilly

Hi Simone, thanks for having me.

Simone Cicero 

Barry, I think it’s helpful if you give our listeners a very short but comprehensive overview of what residuality theory is. And especially I think why is different from the way we normally design an architecture, software artifacts more specifically. Let’s start from there and then maybe we move into generalizing a little bit.

Barry O’Reilly 

The problem that I started out trying to solve was how do we create software architects? How do we teach people how to do this thing that we do, which is to eventually define the structure of a software application? And that turned out to be something that lots and lots of people, lots of very good architects know how to do. But we’re absolutely terrible as an industry at articulating. What exactly is it that we do and what exactly is it that we believe and what is it that is the underpinning of the kind of structural decisions that we make? And so several years ago, I started digging into this what is software architecture? What is it that’s actually happening? And what I boiled it down to was that the biggest problem we have in software engineering is that we’re constantly surprised. And we’re constantly surprised by a lot of the time by the environment that we work in. And so we spend some time trying to understand the business and the social and economic and the political environment that we’re working in, and we get a picture of that and then we define a structure, a software structure, and we put that structure into production. Something happens and it doesn’t work anymore, and the structure doesn’t hold. We have to tear the structure apart and put it back together. And this happens often enough that we become quite stressed by it. And we try to quickly change our software structure as the environment changes around us. And that leads to a brittleness or a fragility in the software that we build. And the mark of quality in a software application is how brittle or fragile is it in the face of the environment that it’s expected to execute in? And so I started looking at, well, why do we have this problem? And essentially we have this problem because in a business environment, we don’t know what’s going to happen in the future. We don’t know how things are going to play out. And I started to look at, well, how do we think about this and realize that what we do in software architecture is that we represent the environment, the business and the social and the economic environment in a very static way. And from that static representation of the environment, we then produce a static software structure. So essentially our architecture, our software structure is a picture of a picture. And we’re using that picture of a picture to describe something that is essentially a movie. So we’re trying to describe an entire movie through a still picture of a still picture. And obviously that’s an incredibly brittle and fragile way of describing what we’re working with. And we’re going to almost by definition, when we phrase it like that, we’re going to constantly be surprised by the environment. And so I started to think about, is there a different way? Is there a different way to talk about architecture, to get people to think about architecture? And it turns out that this static representation, this treating architecture as a static thing and treating the business environment as a static thing is a problem that doesn’t just belong to software engineering. It belongs to the entire history of western philosophy. And so I started to look at approaches from the complexity sciences and people who’ve done a great deal of work with uncertainty and started to think about how can we reformulate? How can we think about these things in a different way? And I looked at my own practice as an architect. What is it that I’m doing that is different than other architects, that’s allowing me to succeed in these highly complex environments, that’s allowing me to deliver software that seems to magically survive futures that it isn’t aware of? And it turns out that what I was doing and what a lot of other senior architects were doing was I was being incredibly negative. I was saying, well, okay, I understand that this is our understanding of the environment right now, but I don’t believe in it. I don’t believe that I understand this environment. I believe that something is going to happen that’s going to reveal my ignorance of this environment. I started to focus on the use of stress. In that case, instead of gathering requirements and trying to predict risks and trying to define all the possible scenarios that would possibly occur in a business or environment, I started to think about what will stress my representation, what will stress my picture, what will stress my understanding of this organization. And I started to look for ways to do that. I eventually boiled that down to this thing called Residuality Theory. Residuality theory looks at the world not as a bunch of static things or still pictures, but as a constantly shifting, constantly moving set of processes that we can’t really see and we can’t really grasp and we can’t really get a clear picture of. We experience these constantly shifting processes through these static things that are constantly breaking down. So there’s a static idea of a system, of a software system and it exists momentarily and then something in the system shifts, it changes and the structure breaks down. And that broken structure I call the residue, it’s the leftovers of the system. And those leftovers will be the thing that defines the future of the software system. And so Residuality theory then is about moving away from the idea that we can statically represent the environments that we work in and understanding that the future of the system will, our interaction with the future of the system will be a function of the residues. And this residue then is just a snapshot of a particular part of the system that’s affected by a particular kind of change. And instead of looking at a software system as we have traditionally as a set of components or a set of processes which are very static things, we look at a software application as a stack of residues moving over time a set of small differences as the business environment shifts between its different phases and its different states. And this gives us a completely different way of thinking about and describing and talking about software. So instead of trying to boil everything quickly down to a set of processes and components which are very static pictures of the environment, we instead work with the concept of residue and we try to identify the residues in a business environment. And because we’re software engineers, because we’re building structure, we actually contribute to the residue. So when something unexpected happens and the entire system is affected in a particular way, then the way that our software falls apart, the way that it is affected by stress is going to contribute to the future evolution of the business and software system together. This involves a very different way of thinking. As I said, we stop gathering requirements, we stop trying to predict risk, and we stop trying to have control over the environment. And we expect that this business environment is an unknowable set of processes that are going to lurch from position to position to position, from state to state to state. And that our job as architects is not to produce a static representation, but to produce something that can survive as the entire business system lurches from fear state to fear state. That is done by stressing the environment in a way that’s very, very different than identifying risks and probabilities and impacts and scenarios. It’s done by simply coming up with random stressors things that are going to affect the environment in some way and then asking what way will this stressor affect the environment and how will my software structure react or survive in this environment. And what changes would I make to my software structure? What would the differences be if I was to redesign my software to survive in this particular fear state? Or a tractor that the business has been pushed into by some kind of externality or internal happening? When we do this, then what we’re left with is a large number of these small differences, these residues that define what our architecture is going to be. And what we find is that when we compress these residues into a coherent solution using some tricks that we borrow from the complexity sciences, what we find is that our architectures are more likely to survive and we can. Show this empirically are more likely to survive unknown forms of stress. This is exactly the problem that we set out to try and solve, which is that we are constantly surprised. Our software structures, and indeed our business structures, are constantly affected by stress that we aren’t able to predict or see or control. We don’t even know what’s there. And that is an implicit marker of quality in software architecture. How well does this thing survive in an environment that it hasn’t been built to survive? This requires a huge shift in how we think about software architecture because we’re no longer working toward correctness. So if you’ve been educated and if you’ve got a Stem education in mathematics or computer science, you’ve been brought into your understanding of the world in a way that makes you look for correctness. Is this the correct program, is this the correct algorithm, is this the correct architecture? But what we’re instead with residuality, what we’re pursuing is something called criticality, which is, is this system able to function across a wide range of attractors which we may or may not even know about or be aware of in our environment? For most people in software, that’s a big change, a big step. But for the senior architects who’ve been delivering large platforms over many decades, it’s not really a surprise. They’ll look at this and I’ve talked to some senior architects who’ve built some very big stuff and they say, yeah, this is more or less how we think. It’s just very difficult for the senior architect in our industry to be able to articulate this in a particular way.

Simone Cicero 

Besides the technical application of your approach to architecture. Right, I understand why it’s difficult for them because it’s a completely different epistemic, I would say. Right. Because I find a lot of resonance in what you say here. It recalled me Nassim Taleb speaking about being paranoid. And when you’re paranoid you said you can be wrong 1000 times and you still will survive. If you’re not a paranoid, you’re wrong once and you’re gone. Right. And residuality in general, it’s like approaching software architecture first, maybe with an epistemic of long tails asymmetric risk multiplicative risks. This kind of approach to thinking about systems in this antifragile way overcomes the traditional mechanistic approach to design. Am I right to connect your work with this philosophical or epistemic approach? And maybe you can also, responding to this question, give us a little bit of more philosophical grounding behind your theory.

Barry O’Reilly 

Yeah, absolutely. The thinking is definitely Talebian in many ways. And one of Taleb’s key points is that it’s not what we don’t know that’s going to get us into trouble because we know barely anything, it’s what we think we know that gets us into trouble. And that’s definitely true in a software industry where most architects and developers are trained to, if not be correct, to at least give the impression of correctness. And there’s even papers going back to the 80s where I said it’s better to give an impression of knowing what we’re doing than it is to admit that we have no idea what’s going on. And that came from senior researchers in the software engineering field. And so this idea that we can be correct and that we just have to sketch a model of some processes and then some components and borrow some patterns from a completely different context on the other side of the world in a completely different industry and put these things together and somehow it’s going to work. And what you tend to get then is that you tend to hide a great deal of risk in those simplistic models. It allows you to get to an answer faster, which is appreciated in our industry because it’s all go go. But what we do is that we hide risk and we make things fragile in the way that we behave. And Taleb points this out in antifragile that you can’t trust us techies because we boil risk into the stuff that we’re doing and we have a simplistic, epistemic view of the world, which is very, very dangerous. So Taleb was, I guess you could say Taleb was my gateway drug into this world because as I realized what I was doing, which was I was stressing my environment in a way that people find to be extreme and incredibly negative. But it was getting results, I find in Taleb then, a parallel to the way I thought and the way I saw the world. And that led me to digging even deeper into the world of uncertainty and complexity. The next big influence on me, I think, after that would be the British researcher Ralph Stacey, who I’m sure you’ve met on your travels, who talks about organizations as complex, responsive processes of relating, that organizations are actually a product of many, many small interactions between people. And Stacey’s philosophical view affected me a great deal, especially his work on the dominant discourse of management, this idea that we believe that we’re in control and that we abstract away from what’s actually happening in an organization and that we believe that we can steer things to a degree which we actually can’t. And that brought me into the complexity sciences. And so if you look at the more technical stuff that I’ve published, there’s a great deal of reliance on the work of Stuart Kauffman and his NK model of complexity and how complex systems emerge and how components interact with each other and the relationships that exist there. After that, I’ve been forced to dig beyond the complexity sciences. And so I have a paper that’s just gone to the printers and it’s going to be published in the next few months. And where I’m digging into ideas from German idealism things like Kant and post structural philosophy and people like Deleuze and Derrida to find parallels for how we should think about these things and to try and find different ways of describing what this residuality theory actually is. So there’s a huge stack of influences which are fairly well described in the work that’s being published from Taleb to Stacey to Kauffman, writers and thinkers like Alicia Juarrero who’ve affected the way that I see the world. But the basic Talebian ideas are that we can’t trust a great deal of the supposed structure that exists around how we do work today and that can get us into trouble. That’s still very much there in my work.

Stina Heikkila 

In a way, software architects, they are quite inclined already to think in systems in some way. Now you have sort of you’re moving into another layer of systems thinking with sort of complexity theory and trying to look at those uncertain environments instead of having multiple different possible scenarios to explore. But even that is quite systems oriented if we look at the relationship then between software developers and software architecture and organizational development. So in the organization you have a range of people working together and doing different things. And then maybe on the backbone you have your systems and your data systems and your software programs that are supporting the organization to provide its services and products or whatever it is dealing with. I know that you had said that it’s sort of impossible to separate the design of software and the design of organizations. So could you go into that a bit more and how you see the future of those streams evolve, like with your theoretical work and how to make it practicable for organizations?

Barry O’Reilly 

I guess I’ll start by talking a little bit about the relationship between software and the organization. My view would be that it’s absolutely impossible to sit down and design an organization and design all the structures and all the elements that you have to do in order to get the organization to start moving from a business perspective and then to sit down in a different room with different people and design the software system completely independently of that, because the two things are going to feed back into each other. The way that the software works or doesn’t work will influence how the organization moves and how the organization is able to interact with its environment. And the way that the organization moves and the way that it changes and the way that things shift over time will influence the software. And the two things are influencing each other all the time. And so if you try to separate these two things, which the vast majority of organizations do, what you’ll end up with is a great deal of flux as the two things infect each other impact each other. And it will take a long time for them to settle down into a state that’s relatively stable and if they ever even do that, because things will consistently be changing in both environments at the same time. I describe the systems that we as software architects have to build as hyperliminal. What that means is that we are responsible for the structure of a system, a software system that is essentially quite rigid. And mechanistic software isn’t some great big mystery. It’s a mathematical thing, it’s a very logical thing. Software is an ordered system. It’s complicated, like a car. It’s not something mystical or magical. It does what we tell it to do, it can be tested. It does the same thing repeatedly. It doesn’t change its mind about the world that it lives in. And we force these software structures then, which are ordered and rigid, to live in a business environment which is anything but ordered and rigid, which is fluid and constantly changing and constantly moving. That fluidity, that constant change and constant movement in the business environment then impacts our software structures. That’s the thing that makes us realize that that surprises us and we have to change that structure constantly. If you’re a software architect, that means that you have to work in both of these worlds. You have to apply one way of thinking to your software structures and how you test that and how you validate that and verify it and show that it actually works. And then you have to use a completely different way of thinking to engage with the world of the business environment, with stakeholders, with customers, with people who are moving around in the system, with ideas that are moving around in this system, and with the financial and political and social aspects of what’s happening. And that’s two completely different skill sets. We don’t teach those two completely different skill sets to our architects. We teach them one and that’s the software one. And then they have a tendency to go into this business environment and treat it like it’s an engineering environment. And that’s why we get lines and boxes that describe the environment that look like circuit boards, that have absolutely no relationship to the human system that exists there. And in those circuit boards, then we’re hiding all that risk. We’re hiding our ignorance, pretending that we actually know what’s going on. If you take the approach that I’m advocating for building software. That means spending a lot more time thinking about how much you don’t know in the business environment. What is it that we don’t know here? What is it that might surprise us? What things can happen here that will stress, that will cause stress on our structure and that requires then that a software architect is able to fluidly converse in this world, which is again, a different skill set. And a lot of us who work in software architecture manage to build that skill set almost accidentally through exposure to certain projects over the courses of our careers. What tends to happen then, which is really interesting, is that you’ll go in as this skeptical architect and someone will say, well, we have this thing and we’re going to sell this product and we’re going to sell it to this market at this price and we’re going to manufacture it in this way. And you come in as a software architect with this skeptical view and you say, but where are the stressors in this? Where are the things that we haven’t understood? What are we hiding from? What are we ignorant of what might happen in this environment and what implications will that have for the software? So one of the first questions that I teach architects to ask of their business stakeholders is what happens if a competitor drops their price? I use that question as a measuring stick to see if someone’s really doing architecture or if they’re just putting software functions, grouping software functions into components and calling it architecture. So if a competitor drops their price, it has an impact on your architecture, it will turn up in your architecture somewhere because there’s going to have to be a business response to that price drop. And that response might be a change in your offering a drop in your own price, which impacts the financials, which impacts the architecture. It might be that you exit a market or you refine whatever your offering is, but it’s going to eventually impact the software architecture. That question is a very different question than what software architects are used to asking. It’s not about requirements, it’s about what’s actually happening here. What don’t we know? And very often when you ask that kind of question to business stakeholders, you’ll find out that they themselves don’t know what they would do or what this organization’s response to a price drop will be, which leads to a different set of questions percolating around in the business environment. And this actually can, in many cases, impact how we design our organization. Because we’re going to realize that, oh, if this happens, then we’ll need to be able to do this or we need to have this capacity or this ability in order to be responsive in the market. And what you find is that by taking this approach of stress in the software environment as an architect is that you trigger a whole bunch of questions about the organizational design because a lot of times when we’re designing an organization we are relying on the same impulses that I’ve described here. We make it simple for ourselves. We use patterns, we lift ideas and structures from organizations in different markets, in different countries and different cultures and we try to make them work for our particular organization. And there’s a lot of brittleness and fragility in those things as well. And I think you see that in constant reorgs rather than the software failure that we see as architects. And so whenever we do this work as architects, we’re actually crossing a boundary and we’re moving into a different kind of work and it can appear to be it’s very, very blurry. Am I doing software architecture now or am I doing business development or am I changing the design of this organization? What is actually happening here? But at the core of it you’re asking the question what should the structure of this software be? And by asking that in the right way, by viewing the world as a set of interacting processes rather than a bunch of stuff that you have to define, you can actually impact the structure of the organization and how people within the organization actually think about what is it that we do here and how do things actually work?

Simone Cicero 

I’m afraid I’m going to have one of these questions that some of our listeners sometimes complain about that I’m too convoluted in my question sometimes and my accent doesn’t make it easy, but let me try. Okay, but then how do we embed these principles into how we design the organization then? But as you were talking, I also realized, Stina in the background chatting that you’re also pointing, I think I feel decisively towards some kind of overlap of software and organization. And having said that, we are seeing a lot of trends, some of which are represented, for example, in our 3EO framework of organizations that are effectively moving into small units with clear interfaces, some kind of product centricity and unit centricity in organizations. That reflects quite a lot what’s happening with software. So, for example, there is this infamous note by Jeff Bezos when he basically enforced this idea of the two pizza rule on the side of the team and the fact that all teams had to expose interfaces between them. So that they don’t have to spend money talking and compensating and they can just expose their interfaces so that other teams can use their services accordingly. So what comes out of this evolution in organizing points to these echoes what’s happening with software, with microservices and so on. So first of all, if this sounds reasonable on your side and if we kind of embrace this idea of software and organizations overlapping units becoming more products, how do you think something bigger, how do you get to think about the system then? If there is so much focus on staying small, clearly articulating your interfaces so that all these cells can interact in ways that we don’t predict. What should we design for then at the systemic level, at the organizational level?

Barry O’Reilly 

It’s great that you mentioned Bezos’s memo because that’s for me a very important event in the history of software engineering where Bezos in 2001 sends a memo out to all the developers at Amazon and says look, this is the way we’re going to do things from Nyon. We’re going to have clear interfaces, there’s going to be no interprocess communication, no access to business logic or to data without it being done in a call over the network. And then he ends the memo with if you don’t do this you’re going to be fired. Have a nice day. And for me that memo represents a huge step in the maturity of the software industry. And so the question that you’re asking nice by the way, how do we design an organization, how do we make decisions about these kind of things? There has been a theme running through software engineering since the 1960, or I guess the NATO conferences in 1968 and a long time before that around modularity and software. And these things that I’m talking about, the quality of a piece of software is not how well it does its job and not only how well it’s done its job, but also how it responds to things that it hasn’t been designed for and that’s in a fluid environment that’s natural. And there has been an idea, we know since the dawn of software. We know that software that is split into modules and software that is loosely where those modules are loosely coupled from each other and where there’s some degree of redundancy and diversity. We know that these things make better software and we’ve known that for a very long time. But we don’t know exactly which modules and which pieces should be coupled and which pieces should be loosely coupled and where should we have redundancy and where should we have diversity. And given that we don’t know the future, it’s very, very hard to give correct answers to those questions. There’s a piece of research done in 2016 by researchers called Paul Ralph and Tempero who interviewed anonymously a whole bunch of software engineers and said how do you make decisions about the modularity in your software programs under conditions of a non anonymity? Most software engineers will say we don’t know. It’s just gut feeling. And that’s a huge problem in software. We know that modularity is important and we know that software applications that perform well in their environments are modular but we don’t actually know how to get to that modularity, how to make structured decisions. And Bezos’s memo helped us by saying that this separation is necessary. We need to do this and we need to stop optimizing for local things and we need to produce systems that are capable of interacting with other. Systems in ways that we don’t yet know in the future. And so what Bezos was describing where the principles of service orientation and the principles of service orientation are residual, they allow a system to survive more stress that it doesn’t know about in the future through the concept of interfaces and policy and things like that. And I think that organizations are moving into the same thing as we see that the world that we live in has changed dramatically just in the last five years. And we started to realize that things change incredibly quickly and they’re changing even faster as our society becomes more interconnected. And organizations are starting to face the same problems that software engineers had. We know that we need to be more fluid. And we know that smaller units, modules in our organizations will make us more fluid. And we know that clear interfaces will make that fluidity easier and there’ll be less friction. And we know that having these small units coupled to the organization in different ways so that their dependencies don’t stop them from acting in the market is important. We’re not entirely sure yet. And so when I take Bezos’s memo and SOA and take that further into software engineering, what I come to is this idea that criticality is more important than correctness. And I do that through Residuality theory. So criticality is the ability of a system to reconfigure itself in order to cope with unexpected circumstances. And that can, criticality can actually be measured in terms of the number of components and the number of connections that those components have between each other in software. And that’s an important part of actually applying residuality theory practically to software, is measuring the criticality of your system in terms of its size and its interrelatedness. And in organizations, then we’re starting to see the same thing. And what we do in software is that we use the concept of stress to talk about the different fears states, the different states that the environment can arrive at depending on if a competitor drops their price or if there’s a pandemic or a war or something like that. And that will allow us to investigate the system from different perspectives, from different realities, without getting hung up on things like probability or whether something’s actually likely to happen. And when we do, that how our software system will behave in different situations. And it allows us to adjust the structure in order to make it more flexible. And at the end of the process of residual analysis and software, the actual stressors that we come up with, the actual things that we propose are going to happen in the environment, they don’t matter. It’s the criticality, the flexibility that’s built into the system when we do this, that’s important. When we apply this to software, like I said, it bleeds out into the organization, and we start to ask awkward questions. And it causes the organization to go through the same process, and it causes the organization to think, well, we can’t be correct. We can’t exactly predict the future, but we can make small changes in terms of which organizational units do we have? How do they relate to each other? Where are the dependencies? How do we fund them? How do they relate to each other? How do they collaborate? Those kind of things tend to be tested by this kind of thinking and we tend to build up a much better picture of what kind of structure is necessary in this organization in order to be able to survive in this market. And what this does a lot of times is it uncovers problems and things that have been swept under the rug by simplistic approaches, realizing that, look, we have a dependency on this particular unit at the center of the company, they’re a bottleneck and we can’t really get anything done. And maybe we need to think about how do we modularize this, how do we spread this kind of knowledge to different areas of the organization, for example. And that kind of thing is increasing the criticality of the system rather than its correctness or its efficiency. Not sure if that answers your question.

Simone Cicero 

No, I mean, it does and it sparks more reflections. I will try to make a little reflection and then see if you have something to add on this. In the past months I’ve been also fascinated by Promise Theory, by Mark Burgess. I’m sure you are familiar with that.

Barry O’Reilly 

I’ve met it on my travels, yes. But okay, incredibly familiar with it

Simone Cicero 

In a few words I can synthesize that. The impression that I get from that is if you think to a system as a set of nodes, these nodes should be more responsible of their own outcomes and should rely a bit less on chains of promises because essentially, these chains of promises can break at some point, and then you are left there with nothing. Unless you don’t have any other paths to reach your resources that you need to produce your outcomes. Which is for me very resonant as well with the work you are putting out. In general, I think most of this work that we are seeing coming up from people like you that trying to mix architecture and design with complexity, lead us to concepts such as small is better, interface is better than complex conversations. And all these things kind of point out a very critical, or I should say maybe a critique of sides, critique of scale somehow a red alert on something like don’t excess with coupling because this is going to make things more fragile. This is a massive critique in the industrial age which has always been about kind of planning and coupling and long chains and scale and so on. So is this approach to software architecture and organizational architecture fit for the market and society we are living, which is definitely still cultural industrial? Or is it something that is coming up to prepare us for the next generation, the next society, the society that comes after the nexus that we are just kind of smelling. We squint, we can see that, but it’s not there yet.

Barry O’Reilly 

Yes, when you’re working with the complexity sciences, when you find yourself reading that kind of material and you’ve moved into that kind of world and yes, you’ve definitely said we are at the edge of what we previously have done and how we previously have thought. We are at the edge of what we can do and we need to look around for something else to be able to work with these kind of systems. One of the things I think that’s important, and we’ve seen this in the software industry a lot, is that people will tend to look for simple solutions to problem, no matter how complex the problem is. And you’ll have seen in the software industry in the last few years a tendency towards modularization that’s very extreme, something we call Microservices, build very, very small units of software and force them to interact with each other. One of the things that Residuality theory tries to correct for is the tendency to just apply a pattern blindly and assume that modularity and loose coupling are good and therefore we’ll have lots of modules and they’ll be loosely coupled from each other and we’ll solve all possible problems. But as you said, the industrial age is all about coupling things and making things efficient. And we still work in industries that hasn’t changed. We still have to produce things and we still have to produce things in an efficient way. And one of the things that Residuality does is that it investigates the attractors in a business environment and says, well, what can happen here? How can things move around? And it allows us to make decisions and say actually in this particular kiosk, it’s perfectly okay to couple these things, it’s perfectly okay to introduce efficiency here and to have these two functions, or these two components live in the same computer in the same place in memory and talk to each other directly. So it isn’t a sort of copy paste. Everything should be modular, everything should be a service, everything should be separated over the network. And what I actually find is that when you apply Residuality theory, sometimes you’re breaking rules that have been guiding lights for the software industry. Things like don’t repeat yourself, things like microservice approaches to things. And you can actually use Residuality to show software developers, look, there’s absolutely no need for a microservice approach here, or this particular bunch of components can all live very close together and be very much coupled to each other. Because we haven’t found anything in the attractor space for this business that says that they need to live separate lives, that they have different requirements in different fees states of this particular business. I think that that’s very important because the software industry is absolutely brilliant at grabbing an idea and saying let’s apply this everywhere. And you’ll find patterns deployed everywhere, whether or not they’re needed. And sometimes a monolithic, tightly coupled application is the right solution to a particular problem. And sometimes if you’re building a business, a factory to produce a particular product, then a tightly coupled efficient system is exactly what you want to have. Sometimes when you explore the business environment and explore how the system that you’ve designed responds to stress or unknowns in the environment, you can find that this actually holds, this is okay. And so we don’t have to apply these things with broad brushstrokes to everything. And one of the things that I try to teach software architects is that everything is context dependent. There’s no golden rule that says everything needs to be this and everything needs to be constructed in this particular way and everything has to follow this principle or this pattern, but that you have to approach the organizational and the social and the economic context of what you’re doing uniquely each time and figure out what is the right response in this environment. And since we can’t see the future of that environment, the only thing we can do is to explore the attractors that exist in that environment and design for an attractor network, which is what essentially what residuality theory is doing. We’re designing for an attractor network rather than a particular snapshot of how we think things are going to work. And that can actually lead to a position where systems, where software structures actually become much less complicated than if we just applied the patterns of the day, which is what software engineers have been doing for a generation or two.

Stina Heikkila 

We have been talking quite a lot about how software architecture kind of goes into the boundaries of organizational design. And you’re talking very much from the perspective of sort of teaching software architects about those other perspectives and ways of viewing things. So for instance, business operations and how things happening in that more fluid space influence how you need to design the software architecture. From the other hand, I sense this kind of urgent need for the other side. So business developers and operational managers and so on to also learn more about how software works. So I don’t know if you are witnessing that gap in the work that you do because it feels to me like sometimes there can be really difficulty to have those two worlds talk to each other. Sometimes also software is outsourced, I know what we already talked about, that it should be much more closely intertwined in the organization. But what do you think about the skill sets that people need to have in terms of software just to have a minimum of sort of literacy of understanding how those things work? Because to me it seems like that would be something really to focus on also.

Barry O’Reilly 

Yeah, and that’s definitely a problem if I go into a very traditional organization and start using my particular method for designing software. It can be quite a shock for business analysts, business developers, requirements, engineers, people who work more on the business side of things, who have been led to believe that the way to make software is to describe what you want and then tell the technical people and then they’ll build it. And it’s quite a shock for them when I come back and say, well, how is this actually going to work? And have you really thought this through? And is the business model going to cope with this particular situation and what will actually happen here? And that can be quite challenging for them because it feels like this techy guy has come in and he’s starting to jump all over our territory. One of the things that I find incredibly useful in this scenario is the tool, the framework. Cynefin it’s an incredibly simple idea. It takes 15 minutes to explain when I’m interfacing with business people. A quick lap through this is Cynefin. And then there are things in the world that we don’t understand and there are different approaches to how we solve problems depending on where we are in terms of how much we can know about the future. And this makes it much, much easier for them to get on board to work with this kind of approach. Cynefin the sort of first step. And then eventually, as we work through the architecture of the system, they start to get a better idea of what it is that we’re doing and how software is deeply related to the structure of the organization and the structure of the business and the operating models, and that these two things have to happen in sync. And it’s not something that can be firewalled. You can’t just throw stuff over a wall to developers and architects and expect that it’s all going to link up seamlessly without doing the hard work and so simple tools. I don’t try not to overwhelm business people with the philosophy or the software engineering theory, but more or less get them to understand that let’s think everything is moving here. Slowly introduce them to the idea that this is a different way of working. And most of the time, and especially with senior executives, this works incredibly well. They grasp it very quickly and they understand what it is that we’re trying to do. Because I think the concept of criticality that we work to a position where we have a structure that survives well as well as it can in a multitude of environments is much more familiar to executive thinking than it is to software thinking, where there tends to be a much greater focus on correctness. And I guess sometimes working with other business stakeholders, there can be a focus on correctness depending on how they see the world as well. But working more on a philosophical level with them tends to help a great deal.

Simone Cicero

Thank you so much. This was a fascinating conversation that didn’t diluted my expectations and I’m sure that this is going to trigger lots of interest in your work in our audience. I hope so, because this is relevant work. One thing that I really bring home. I mean, many things. But if there is one thing that I want to stress for our listeners is this idea of looking at your organization from a perspective of the stress or step the organization can face and try to not be reductive. And rely too much on what you said in correctedness because that’s not how the world that we have been always living but I would say that we are now much more aware being living within. It works. It doesn’t work according to the rules and doesn’t work according to the plans. It’s much more unpredictable than you normally expect as an average interested person. So, a couple of things I want to add for our listeners. Again, you mentioned Cynefin and we have been lucky to have Dave Snowden on our podcast as well, episode five of season two. So besides Alicia Juarrero episode four of season four, you can go back also and listen to Dave Snowden talking about how organizations can deal with uncertainty. To finish this conversation, I would like to ask you, Barry, to share your breadcrumbs with our listeners.

Barry O’Reilly 

Okay, so references and things that will lead them to a better level of understanding around this, I guess. Well, I think that if you’re going to dig deeper into the ideas of residuality, there is some stuff on YouTube that’s been recorded at conferences over the last year which give a kind of high level overview of how we get from uncertainty to Kauffman to actual practical use of things like matrices and networks to make decisions in software. Apart from that, I would absolutely recommend the work of Ralph Stacey. The idea of complex, responsive processes of relating and how organizations actually come to be and the causalities that are involved there, which was a big step for me in understanding what’s actually happening inside an organization. And why do we see so much complexity and why are we inclined to use these terms? Apart from that, I think that there’s a huge amount of material and residuality out there from an academic standpoint as well, which you can read through if you find that interesting. But I would say that the thing that’s most important that I say to software engineers is to start questioning not just the methods and the frameworks and the tools that you’ve been taught that lead us into this picture of a picture static representation of the world around us. But to start asking yourself, what is the underlying philosophy behind what I do and these frameworks that I use? How did we arrived at this particular set of ideas? Why do we use this particular set of practices, and that involves a deep reflection around what is it that we do. And one book that I recommend to all architects is a book called The Reflective Practitioner by Donald A. Shon. And so it’s a book from 1983, and it talks about how professionals are able to develop a sense of how they do their work in environments that are inherently unknowable. And it talks about the concept of technical rationality, which is this idea that there should be a framework, there should be a plan, there should be a simple number of steps that allow us to execute on our work, and it should be inherently repeatable and says, no, actually, that’s not how the world works. And this book is incredibly important and opened a lot of doors in my thinking around how do we develop as professionals environments where it’s not possible to always know what’s going on. So, Shon’s, The Reflective Practitioner, Ralph Stacey’s Complexity and Organizational Reality are two books that will take you very far away from traditional understanding of how organizations actually work and how complexity actually works. Unfortunately, there isn’t a great deal of material relating these ideas between the world of software and the world of organizational design. We haven’t got there yet, but it’s coming.

Stina Heikkila

Yeah, and a lot of things to you as well. So we look forward to keep following your work from my side as well. Thank you so much for joining us. I think that everything that you have mentioned about not looking at the world as static, and that really applies to everyone, I think, living today. We are living in such an uncertain environment and we need to get away from this reductionist idea that we can control our environment. I think that applies to all disciplines. So thank you very much for bringing those insights. Hope you also enjoyed spending time with us today.

Barry O’Reilly 

Yeah, absolutely.

Stina Heikkila 

Great. And for our listeners, as always, you can find all the references on our website, so boundaryless.io/resources/podcast, Barry O’Reilly’s episode will appear there, and there you can find all the links and we’ll make sure that everything is noted down for your easy reference until we catch up again. Remember to think boundaryless.