PDF version of this report
You must have Adobe Acrobat reader to view, save, or print PDF files. The
reader is available for free
download.

Keys to Managing Big Projects

by Faulkner Staff

Copyright 2020, Faulkner Information Services. All
Rights Reserved.

Docid: 00021377

Publication Date: 2012

Report Type: TUTORIAL

Preview

The real world is permanently unpredictable and chaotic. This is why “too
big to fail” is often a guarantee of not just eventual failure, but of
devastating, catastrophic failure. On the one hand, this is sobering, but
it is also a sign clearly marking the path to a better construct: Anti
Fragility. The future belongs to large scale technology solutions,
systems, and teams that find ways to reap benefits from uncertainty and
chaos. This is a point worth considering, as we see events unfold that
well may initiate the shattering of the Internet.

Report Contents:

• Executive Summary
• Description
• Current View
• Outlook
• Recommendations
• Web Links

Executive Summary

[return to top of this
report]

It has long been recognized that very large IT projects can pose risks
that reach far outside the domain of the original initiative. A classic
Gartner survey on IT project management found that among initiatives with
a budget greater than $1 million USD, 28 percent fail. Another analysis
from the Standish Group looked at IT initiatives with budgets of
greater than $10 million. These had just over a six percent chance
of being on time, within budget, and delivering expected business
advantages.

A great body of research demonstrates that as the scale and budget of a
technology project grow, the likelihood of failure climbs. The best
defense against allowing IT projects to crater from mere failure into full
blown catastrophe is the exercise of executive prudence:

• Be willing to terminate unsuccessful projects – small or large – if
  they begin a consistent trend of cost overruns, schedule slips, and
  under-delivery of business advantages.
• Institutionalize requirements and solution design processes that
  ensure IT projects are consistent with corporate goals, strategy,
  standards, and culture.
• Once a solution design is defined, don’t permit expansion of its scope
  or drift from its objectives.
• Strive to minimize turnover in the project development teams, and make
  explicit use of financial incentives to create and reinforce team
  cohesion and collaboration.
• Require understandable, scheduled feedback from all project team
  leaders.
• Cultivate relationships with technical leads, be sufficiently
  knowledgeable to engage them in substantive conversations about the
  project, and do so frequently.
• Incrementally validate progress toward defined goals by building user
  experience, Q/A, system integration, and security testing into the
  solution development plan.
• Don’t deploy solutions without a green light from all testing
  organizations.
• Use an incremental, targeted deployment plan to prevent disruption of
  the enterprise business process should issues arise during roll out.

Description

[return to top of this
report]

The Black Swan Phenomenon

The term Black Swan was initially coined by Nassim Taleb,
Oxford University Professor of Risk Engineering and author of a book of
the same name. In the book, Taleb looked at factors leading up to the
financial collapse that prefigured the Great Recession. His term “Black
Swan” is a metaphor for events that are so rare that people tend to think
of them as being impossible; his core observation is that impacts of rare
events can be catastrophic, and it won’t matter how rare they
are if they happen to you.

Large scale IT projects have a nearly fantastic propensity for becoming
Black Swans. Levi Strauss was involved in an IT initiative originally
budgeted at $5 million; it ultimately cost the company $20 million. Kmart
was forced into bankruptcy by a Black Swan, eventually shedding 67,000
employees and being subsumed into Sears Holdings. And public institutions
aren’t immune. Hong Kong Airport and a public/private EU consortium for
collecting highway freight tolls each cost taxpayers hundreds of millions
of dollars more than budgeted as a result of off-the-rails IT projects.

Given this track record, it is somewhat ironic that the term “Black Swan”
has been appropriated by technologists to mean “IT projects that go
terribly wrong” because, among projects initially budgeted at $1million or
more, Black Swans are not rare. In fact, extensive global
research gives a $1 million IT project slightly more than a one in four
chance of being 150 percent over budget. As projects scale into the $10
million range, chances of realizing expected project benefits on schedule
and within budget plummet to about six percent. Why?

• Big, complex things are fragile, and unpredictability and chaos
  are their enemies.

We know that chaos and unpredictability will always be with us to a
greater or lesser degree; and we now also know a lot about IT
Black Swans because they have been intensively studied. They tend to
have these things in common:

• No one involved in designing the solution insisted on an open
  requirements definition process that included technologists,
  stakeholders, partners and users. Thus, the project began without
  rigorous, consensus-based specifications for delivery of a final product
  that was aligned with business goals.
• No one in a position of oversight mandated that continued funding be
  contingent on incrementally documented progress that was validated by
  usability, performance, and Q/A testing.
• No one took responsibility for modifying the product launch timeline
  when crucial flaws were exposed.

Another thing that we know is that almost any massive technology
initiative that culminates in a “Big Bang” rollout – the launch of an
entire megascale, critical system in a single go – should raise Black
Swan concerns. For such an approach to be successful, every
single element must function as expected and every single administrator,
partner, support person, and user must understand how to accomplish their
tasks using the system on day zero. That’s a big ask.

• Failed Big Bang deployments are notorious for longitudinal disruptions
  of the business process, sometimes imperiling the entire enterprise.

Current View

[return to top of this
report]

The real world is permanently unpredictable and chaotic. This is why “too
big to fail” is often a guarantee of not just eventual failure but of
devastating, catastrophic failure. In a nutshell, it is a blueprint
for producing Black Swans. On the one hand, this is sobering, but it is
also a sign clearly marking the path to a better construct: Anti
Fragility.

Real World Anti Fragile- Google’s Android
Like “Black Swan,” Anti Fragile is a term coined by
Nassim Taleb. After Taleb’s rigorous risk engineering analysis of
weaknesses and perverse incentives in the financial industry that
prefigured the Great Recession, he went on to explore how things – all
kinds of things – could be made Anti Fragile.

• In a nutshell, Anti Fragility is the property of things and systems
  that benefit from chaos. They are made stronger and better by
  experiencing unpredictable events and by breaking early and often, but
  in a low cost way that has more long term good consequences than short
  term bad consequences.

Perhaps the most powerful and far reaching Anti Fragile undertaking of
the last decade involves the evolution of Google’s Android. Android began
life as the solution to one of the most intractable technology barriers of
the previous decade by taking advantage of something that had long been an
insurmountable barrier to the development of a trillion dollar market:
Software on mobile devices.

Early on, Android was simply a free API that developers could use to create
apps. API commands issued from apps went to Android processing engines
living on Google’s own infrastructure. The Android engine was
sophisticated enough to effectively shortstop most malicious code. This
brought carriers on board. The API also allowed developers to write
powerful, sophisticated apps fairly easily and to perform functions that
would never have passed carriers’ validation processes for native code apps.
The combination rapidly aggregated a huge Android audience. Because Google
was processing all the input and output of the Android APIs, they had access
to the entire stream of data going to and from apps. This generated a
windfall of data acquisition – one that had a self-modifying focus so that
accumulated information gained nuance, relevance, and breadth over
time.

• Android transformed hosting anonymously sourced apps from a big
  risk into an asset generator that grew in value each time it was
  used.

• Android’s value proposition redefined inherent limitations of
  carrier network access as an opportunity.

It was a nearly textbook application of the principles of Anti
Fragility: Massive, rapid development of the Android niche did spawn some
early failures but by and large these were low cost, low consequence
failures that served to educate and redirect the engineering and design
communities. Android’s breakthrough literally enabled the evolution of
computing on mobile devices because it pried hegemony away from Apple;
however, its big-picture history establishes something even more
important:

• Open source technology has a tendency to be Anti Fragile.

The Role of Open Source Technologies In Big Projects
Today, the open source community is a vast, dynamic, global culture. By
focusing on collaborative research and innovation, it has successfully
delivered some of the most important foundation technologies of all time.
Very literally, the best ideas end up in play in the open source arena,
and the best minds in the world build on them and then share back to the
community. One such example: In 2014, Tesla placed all of its battery
technology patents in the open source domain to catalyze the development
of the aftermarket and promote the broader electric vehicle industry.

In contrast to the Tesla example, the term “open source” almost always
means open source software. There is a lot of it. In fact, for
right-now-real-world IT, there is so much open source technology powering
day to day enterprise life that it would be virtually impossible to
disentangle it from proprietary solutions and leave anything in a workable
state. So for purposes of IT initiatives, this is just a fact: Any
technology project of scale is reliant to some degree on open source
technology. For this reason, an open source strategy and philosophy
must be part of the earliest phases of planning. Here are two key
issues that should be addressed:

• Even though it is public, open source technology is explicitly
  copyrighted. The exact nature of the provisions of an open source
  copyright can vary dramatically in how they treat ownership rights to
  innovations built on top of the original work. Be aware of these
  limitations and develop open source strategy accordingly.
• Caveat adopter. “Open source” does not necessarily mean
  “interoperable” or “cross platform.” Prior to making a decsion to adopt
  an open source tool or technology, research the developer community and
  understand what the key drivers of evolution are likely to be. Make
  certain the trends are compatible with the business goals of your
  projects.

Outlook

[return to top of this
report]

Moving in the Anti Fragile Direction: Massive IT
projects aren’t going to disappear anytime soon, if for no other reason
than there is a large and powerful constituency advocates for them, and
why wouldn’t they? The perverse incentives here include the fact that
consultancy is probably more profitable cleaning up after a Black Swan
than preventing one from taking place. Given that reality, a best practice
for limiting the exposure to Black Swan enterprise technology initiatives
is creating a culture of cradle-to-grave stakeholder
involvement throughout the life cycle of a technology project. This
sort of openness and inclusivity tends to expose flaws while they are
still cheap to fix.

By far the most Anti Fragile tactic any business strategist could embrace
is sort of a 21st century update of Alan Kay’s maxim that the best way to
predict the future is to invent it:

 The best way not to get crushed by
the future is to very carefully watch what other people are inventing.
Thus, don’t ignore or underestimate the destructive potential of current
events.

Will You Be Okay If the Internet Shatters? In December
2017, long time Verizon lawyer and current FCC chairman Ajit Pai announced a
roll back of regulations that safeguard free, uncensored,
level-playing-field access to the Internet. Pai’s party-line, single vote
majority awarded a handful of ISPs (notably Verizon, AT&T, and Comcast)
monopolistic control of Internet traffic that will allow them to promote
content and search results of their own choosing while submerging the rest.
While supporters of the roll back call this “Fast Lane” service, it is more
accurately described as creating a triumvirate of monopolies that can
exploit consumers, businesses, and media outlets.

There are, however, implications to this rollback that are far more
concerning than the target Pai’s FCC has pasted on the wallet of every
consumer and business. This also amounts to an attack on the broader open
source and open standards movements.

All these companies have a decades long track record of sequestering
customers and partners with arcane, proprietary hardware and software. It
is a lead pipe cinch that developers who want to exploit “Fast Lanes” will
be forced to use proprietary APIs and will also be subject to strict
nondisclosure and non-competition agreements. The carriers can demand that
innovators populate the ISP’s app and content catalog with non-portable
intellectual property.

This is where it gets more complicated, though, because it is not very
likely that the “losers” in this particular scenario (almost everyone
except Verizon, AT&T, Comcast and the like) will stick around long
enough to have their “innovation stifled”. So. While those in Washington
DC may feel they have the internet by throat and the internet is the only
game in town, they have overlooked one important factor: Washington DC
isn’t the only town.

Expect to see, in the near to mid term, a diaspora from the web into a
myriad of special purpose and ad hoc networks that can dynamically coalesce
and disperse. And don’t expect to see global dominance in this new
world arise inside the US, because the FCC has put out a huge “Not Welcome”
mat for the global technology innovator community. Here are a few
technologies that could play an early role in a shattered internet:

• Femto cells
• Wireless mesh networks and storage systems
• Quantum cryptography

Recommendations

[return to top of this
report]

IT projects are significantly more likely to become Black Swans than
almost any other corporate initiative. And perversely, hindsight typically
confirms that heeding obvious early warning signs could have forestalled
most of the bad effects. Communication is key, and the people most
likely to observe the first signs of critical problems must be empowered
to make their observations known. They must also be abundantly
confident that there is an attentive, receptive audience for their
insights.

• The use of Agile Programming Methodologies is
  one
  way
  to
  promote timely communication among solution designers, builders, and
  testers that can help to identify and correct perverse outcomes.

How Agile Methodologies Can Forestall the Hatch of a Black Swan
The origins of Agile Software Development Methodologies go back a
long way, to a meeting of leading edge technology thinkers who sought to
define methods for “lightweight” software development. In this particular
context, lightweight refers to a contrast being drawn against heavyweight
development. Heavyweight development is characterized by rigid
bureaucracy, unproductive adherence to formal plans and inability to
respond effectively to changing circumstances. At the heart of the Agile
development approach are four key concepts:

• Effective interaction between people engaged in a software solution
  development process will yield a better product than blind adherence to
  formalized protocols.
• Creating working software is a better use of time and resources than
  creating elaborate, detailed documentation and then developing software
  that implements the documentation.
• The customer should take part in iterative collaboration throughout
  the development process as opposed to dropping out of the conversation
  after a definition of specific deliverables is in place.
• All stakeholders should be able to respond to changing conditions,
  deeper understanding, or new opportunities.

Web Links

[return to top of this
report]

Apache Foundation: http://www.apache.org/
Black Swans and Anti Fragility: http://nassimtaleb.org/
Mongo DB: https://www.mongodb.com/
Mozilla.org: https://www.mozilla.org/
Principles of Agile Methods for Software Development: http://agilemanifesto.org/principles.html
Redhat Linux: https://www.redhat.com/en

[return to top of this
report]