The Harvard Business Review Blog Network published a rather interesting article on the reasons behind the failure of the Boeing 787 Dreamliner. One of the main causes seems to be related to how Boeing outsourced work.
Boeing undertook one of the most extensive outsourcing campaigns that it has ever attempted in its history. That decision has received a lot of press coverage, and the common wisdom is coalescing around this as a cause of the problems.
Outsourcing as such is not wrong or risky. Many success stories heavily depend on outsourcing: Amazon outsources delivery, Apple outsources all manufacturing … The key is what you outsource and what you keep internal.
Rather, the issues the plane has been facing have much more to do with Boeing’s decision to treat the design and production of such a radically new and different aircraft as a modular system so early in its development.
In other words, Boeing outsourced some of the architecture of the new plane. That was new for them. So far they only outsourced manufacturing of parts after Boeing designed them internally.
if you’re trying to modularize something — particularly if you’re trying to do it across organizational boundaries — you want to be absolutely sure that you know how all the pieces optimally work together, so everyone can just focus on their piece of the puzzle. If you’ve done it too soon and tried to modularize parts of an unsolved puzzle across suppliers, then each time one of those unanticipated problems or interdependencies arises, you have to cross corporate boundaries to make the necessary changes — changes which could dramatically impact the P&L of a supplier.
This equally applies to IT solutions. If you outsource parts of the solution before you have designed the whole, you’ll end up with problems whose solutions cross supplier boundaries, impact P&L of those suppliers and require contract negotiations. To avoid this, the entire solution has to be designed before suppliers are chosen and contracts signed. That includes the design of new components, changes to existing components and, often forgotten, integration with existing (“AS-IS”) components.
Either you outsource this entirely (all of it, you can’t be cheap and only outsource 95% of it) or first design the whole internally first.
In the creation of any truly new product or product category, it is almost invariably a big advantage to start out as integrated as possible. Why? Well, put simply, the more elements of the design that are under your control, the more effectively you’re able to radically change the design of a product — you give your engineers more degrees of freedom. Similarly, being integrated means you don’t have to understand what all the interdependencies are going to be between the components in a product that you haven’t created yet (which, obviously, is pretty hard to do). And, as a result of that, you don’t need to ask suppliers to contract over interconnects that haven’t been created yet, either. Instead, you can put employees together of different disciplines and tell them to solve the problems together. Many of the problems they will encounter would not have been possible to anticipate; but that’s ok, because they’re not under contract to build a component — they’ve been employed to solve a problem. Their primary focus is on what the optimal solution is, and if that means changing multiple elements of the design, then they’re not fighting a whole set of organizational incentives that discourage them from doing it.
For Boeing it is going to be a very costly lesson. But at least they’ll have a chance to learn. Large IT projects often fail for exactly this reason: modularize a complicated problem too soon.
One last element from the article … why did Boeing do this?
They didn’t want to pay full price for the Dreamliner’s development, so, they didn’t — or at least, that’s what they thought. But as Henry Ford warned almost a century earlier: if you need a machine and don’t buy it, then you will ultimately find that you have paid for it and don’t have it.
They wanted to safe on the development of the aircraft and thought that by outsourcing the design (the ‘tough problems’) they would keep costs low.
Morale of this: don’t outsource pieces of a puzzle before the entire puzzle is known (designed or architected).
How many times did you encounter this in IT projects?
Various resources on the Internet you might find useful. At least recommended reading for a spare moment.
- Which Comes First: Technical skills, Process or Relationship. Article that highlights the mismatch between business people and more technically inclined people. A good read for any architect who wants to get closer to the business side of the organization.
ArchiMate and TOGAF. Four excellent papers comparing ArchiMate and TOGAF:
- Paper 1: Using ArchiMate with an Architecture Method
- Paper 2: Using ArchiMate with TOGAF (part one)
- Paper 3: Structural and Behavioral Views
- Paper 4: Using ArchiMate with TOGAf (part two)
Some attempts at defining the concept of “business function:
- Recent Discussions about Function vs. Process vs. Services
- Strategic Architecture – On Business Functions.
- Abstraction in all its variety: the ideas that underpin architecture frameworks. Article on the various dimensions of “abstraction”.
Last week I went to a presentation on the Building Security In Maturity Model by Gary McGraw. They interviewed about ten organisations who did have a software security team and asked them what they actually do today to make software more secure. They specifically went for a data-driven apporach, no “what could we do” but a 100% focus on “what do they do”. Piling all that information together, some magic processing (read: spreadsheet magic) and you have the Building Security In Maturity Model.
Microsoft was one of the organizatons participating in this effort. Steve Lipner himself wrote about how he experienced this effort and what he thinks from the outcome. One part of his article I would like to emphasise:
I’ve historically not been a fan of “maturity models” because many of them are so abstract and paper-oriented that you can rate “high” on the maturity model and still fail at whatever attribute of your products and processes (quality, timeliness, security) the model purports to measure. In contrast, I like the BSIMM because
· It’s specific. The measures in the BSIMM are things that an development organization actually does to produce secure software.
· It’s real-world. Gary, Sammy, and Brian made a rule that no activity would be included in the BSIMM unless at least one of the organizations they interviewed actually performed that activity.
I have about the same idea on maturity models as Steve has. But then, I am mostly sceptical about any model and framework that abstracts away reality so vigorously that I wonder how any organization can successfully use them to achieve improvement.
I have a couple of Tweet searches I follow. One of them tracks tweets with the keyword “infosec”. This morning I woke up with this tweet in the list:
now an excel 0day. woohoo. it’s always exciting in infosec.
This tweet is very typical: it’s always about hacks, attacks in the wild … I personally find that very disappointing, the above tweet even has something morbid.
Although talking about specific vulnerabilities is important, it is a lot more important to talk about avoiding those vulnerabilities in the first place. I see extended articles explaining in great detail how they hacked Adobe PDF documents, web applications or something commonly used. They do this with such pride and amusement that I get this feeling they are sorry they can’t use them in the wild. It almost looks like as if the only thing that differentiates the real authors of malware from these infosec people, is the sense of ethics the second group has. Ethics that stand in the way of making money with the vulnerability found.
As I said, a detailed knowledge of vulnerabilities is very important. But talking about how to do better and avoiding them in the first place, that gives a lot more return on investment in the long term. What could authors of (faulty) software have done to make a better product? What specific design patterns, code patterns … would have avoided the vulnerability? Wich steps in their quality control methods are missing that could have prevented the vulnerability? Every article on a vulnerability is useless for me if it doesn’t mention advice to avoid the vulnerability tomorrow. Luckily there are many authors that do, but sadly also many that don’t.
I don’t think we got this far in constructing buildings by detailing every single collapse of a building without doing any lessons learned. We also try to find out how we can avoid disasters for any future building: tools, methods, procedures and guidelines are updated as a consequence. That is what makes us move forward. That is what allows us to do bigger while at the same time become better.
Acting on today’s vulnerabilities will not protect us tomorrow. Today we need to work so we can prevent tomorrow’s vulnerabilities and help us control the overall risks.
All people in ICT have come across projects that will replace a current situation (the AS-IS) with a desired future situation (the TO-BE). At first sight it looks great: you analyze the current situation, including shortcomings and issues, and you document it in an AS-IS document. Then based on the results of this AS-IS and various gathered requirements you design and document the new state: the TO-BE. Looks good right? Not to me …
The drivers for these projects are often the same:
- an increasing perception that the current system is unable to fulfill the needs the system was created for in the first place.
- a general feeling that extending the current system, for instance to solve some of the issues, is becoming too expensive or too complex.
Before I would start designing a future TO-BE, I would like to now why the system as it is today, the AS-IS, is not fulfilling it’s goals anymore. Is it because technology has changed significantly during it’s life time? Is it because the people who designed, developed and maintained the system haven’t done a decent enough job? Is it because the system, once a perfect fit for the problem, started to loose alignment with the environment, slowly being rendered obsolete and in need of replacement?
Without proper answers to these questions and without a proper response in the TO-BE, that TO-BE is surely destined to become your next AS-IS. In a few years we will no doubt witness a presentation that explains us how the AS-IS (that TO-BE we are building today) is not good enough anymore and needs to be replaced with something new and more modern.
The world is constantly changing, so is your company and the environment it lives in. Any ICT system that operates as part of your company must realise it needs to change to keep in lign with that changing environment. If you only focus on building a static architecture that is unable to adapt to changes, you are doomed to recreate the system, in the form of a desired TO-BE, every couple of years.
Only during a smaller part of the existence of such a system, it is properly aligned to actual requirements. Most of the time in the lifespan of the system, is spend in either complaining about lack of alignment or promising improvement with the upcoming TO-BE.
I therefore don’t really believe in this AS-IS and TO-BE methodology. When you realize you are lagging behind while the world around you is changing, you won’t solve the problem by desperately catching up to the present. Because when you finally caught up (the TO-BE is delivered) you are already lagging behind again. Even if you went to great lengths to make that TO-BE as flexible as possible, you can never predict the future. If you can, give me a call.
What you want is a process that:
- periodically measures how well the system is aligned to the environment,
- identifies those elements of the system that are in danger of losing alignment,
- proposes gradual changes to the system to improve alignment.
Note how nowhere in this process we propose to redesign and reimplement the system. At a smaller scale this technique is well know in software development: it’s called refactoring. This is exactly what you also want to do at a larger scale with your architecture: refactor mercilessly. Refactoring should not be limited to the development phase but should be an integral part of the entire life cycle of a system.
Given a proper refactoring process and the obvious current, AS-IS, state of a system, I can gradually improve and align that system to an ever changing environment until the need of the very existance of the system itself is disappearing. I should avoid a big bang approach that proposes and develops a brand new TO-BE system.
Building for change is not a new slogan, yet it is not well understood nor implemented. Every day projects are born that are meant to create a new TO-BE and, sadly enough, at the same time the AS-IS of tomorrow.
The penetration of BlackBerries and alike in enterprises is continuously rising. First an exclusive gadget for the higher management, it is now being distributed to anyone who is supposedly to be non-stop reachable. At the same time, I haven’t seen any improvement in generated business value, efficiency or quality. In fact, if there is anything these devices have accomplished, it is a decrease in all of those: business value, efficiency and quality.
Before I continue, I have to give credit where credit is due. Seasoned blog readers might recognize the title “BlackBerry as a Substitute for Process”. I was inspired by the blog article “Process as a Substitute for Competence” from the Kill The Meeting blog. An excellent article and I strongly encourage everyone to hop over and read it. Secondly, I refer almost solely to “BlackBerry” when in fact I mean any hand held device that allows you to read your email any time in any place.
BlackBerries are sold with the promise you can read and reply to your email whenever you want and wherever you are. I state that this very feature is what makes BlackBerries a liability in your company’s strategy for quality, efficiency and overall improvement. BlackBerries make sure that whatever you have in place that might look like a process, is rapidly turned into a relic of the past.
Let’s consider two opposites. On one side we have a company that is focusing on (business) process improvement on the other side we have a company that doesn’t really care. History has shown us that you want to be working for the first company. Sure, no doubt that second company talks a lot about processes, they probably even have working groups, steering committees and overall projects to get better processes in place. It is however easy to burst that bubble by observing these simple facts:
- Spreadsheet after spreadsheet is introduced to manage those processes.
- Managers are more focused on and spending most of their time in short-term tactical issues instead of long-term process improvement.
- If quality is measured at all, it is done by periodic sampling.
- Your backlog is growing, or at least not reducing.
- Cost reduction is the phrase in meetings.
- Whenever process improvement is mentioned, it is actually about automation, not improvement.
Now, these symptoms would show me that you are not managing the process but you are more into incident management. Managing by incident is a dead end: costs will continue to increase, quality is a distant dream and different departments or business units will slowly start to distrust each other. With managing by incident, you are fixing the past and not creating the future.
The key to sustaining and fueling management by incident is to make it easy to act on incidents. The easier it is to “fix them”, the less pain those incidents will cause. BlackBerries are one key element in this, by giving you immediate access to email and the ability to reply to it almost instantly. You have made fixing incidents easier. Victory? Not at all, you just sank a little deeper in the quick sand incident management really is.
But wait, BlackBerries, a marvelous invention, how could we manage our company without it? How could we survive without email! Those who think email is a key element to success are fools. Don’t forget, we managed to find America, build the Suez canal and put men on the moon without the help of email, let alone BlackBerries. Those people knew the value of proper communication, processes and procedures. They made sure that when they left a meeting everyone knew what they had to do. Simply because each would go their own way and it was time consuming and expensive to communicate afterwards.
Today, people come and go at meetings, not caring about agendas, action lists or meeting minutes. Because they all know that it is so easy to email each other to get the blanks filled in.
My advice: think twice before introducing BlackBerries. They are a fine piece of technology and can be a real value to your company. But they can also fuel management by incident, a dangerous thing to happen.