Agile Software Development

by Geoff Keston

Copyright 2023, Faulkner Information Services. All
Rights Reserved.

Docid: 00011494

Publication Date: 2301

Report Type: TUTORIAL

Preview

Agile development upends many commonly held notions about creating
software. Instead of relying on extensive documentation, pre-developed
requirements, and rigid methods – all of which were once considered
mandatory – Agile development emphasizes flexibility and giving developers
latitude in how they work. This approach has demonstrated important
benefits in a variety of situations, but it should be adopted only after
careful consideration of the needs and circumstances of each particular
development project.

Report Contents:

• Executive Summary
• Description
• Current View
• Outlook
• Recommendations
• Web Links
• Related Reports

Executive Summary

[return to top of this
report]

Agile development is a philosophy that favors moving away from the rigid
methods traditionally used to create software and toward approaches that
are more flexible and adaptable.

Efforts to establish adaptive methods of creating software have been
underway in one form or another for many years – since the 1980s, by some
accounts. In one of its earliest incarnations, this approach was called
Extreme Programming, or sometimes XP. But Agile development emerged more
distinctly in 2001 when a group of developers formed the nonprofit Agile
Alliance and released the Agile Manifesto. This short document presented a
list of principles
such as “Our highest priority is to satisfy the customer through early and
continuous delivery of valuable software” and “Deliver working software
frequently, from a couple of weeks to a couple of months, with a
preference to the shorter timescale.”

Agile initiatives measure progress based almost entirely on the working
software that is created. Conventional measuring sticks such as document
creation or milestones on a project plan are considered to be less
important or are not considered at all. Rather than enumerating a detailed
inventory of requirements at the beginning of the process, development
teams are encouraged to start with a basic set of requirements and then to
refine and add to this list as the project takes shape. Customers and
other stakeholders are brought into the process early on and remain
closely involved at every phase. The input these stakeholders provide
steers the direction of the project and can change key factors such as the
scope and budget. But Agile development is not undisciplined.
Projects are divided into many small steps, called “iterations,” at the
end of which the software is tested. This testing structures the
development process.

The transition from heavyweight to lightweight methods is a significant
change. For instance, members of a development team will need to not only
create code but also to effectively communicate with various people and to
work independently.

To ease this transition, it may be helpful to adopt one of the specific
forms of Agile development that have been defined. These include:

• Scrum, the most popular form
• Lean Software Development
• Extreme Programming
• Crystal
• Kanban
• Dynamic Systems Development Method
• Feature-Driven Development¹

Each of these frameworks offers specific guidance – such as the number of
people who should be on a development team or the length of an iteration –
in a host of books and papers written about them. This guidance can help
an organization translate Agile principles into specific actions.

Description

[return to top of this
report]

For much of the history of contemporary computing, software development
has been systematic and formal. Projects have been tightly controlled with
rigid requirements specified at the beginning of the process. Development
teams followed carefully detailed procedures, and the need to create
thorough documentation overshadowed many other concerns.

Picking up steam in the past decade or so has been a movement away from
these formalized software development approaches and toward approaches
that provide developers with greater independence and flexibility. The
thinking behind these Agile approaches is that heavyweight methodologies
are difficult to follow, that they slow the development cycle, and that
they ultimately become an obstacle to the goal of creating effective
applications.

Agile development comes in many forms, which are united by shared
principles more than by shared practices. These key principles are:

Iterative Development – Agile
development relies on – and demands – frequent feedback from everyone
involved in a project. This feedback is used to iteratively guide the
development process.

Small, Incremental Steps – At
the end of each iteration, goals, specifications, budgets, and other key
parameters are reassessed and – critically – they are permitted to be
changed. If necessary, even the product’s release date can be pushed back;
this is particularly likely after a project’s scope changes significantly.

Customer Involvement – At the
end of each iteration, the results – which should consist of functioning
software – are demonstrated to the customer. Usually, the term
“customer” is replaced with the broader concept of “stakeholder.” A
stakeholder could be an external customer or an employee at the same
company.

Emphasis on Communication –
Team members meet often, in some cases daily, to inform each other about
their progress and activities. These meetings enable developers to adjust
their activities and techniques to aspects of the project being worked on
by other members of the team.

Small, Self-Organizing Teams
– Agile approaches give development teams a fair amount of latitude, often
enabling them to choose which requirements of the software to focus on in
a given iteration. Teams tend to be small. The small size of Agile
development teams enable members to communicate closely and work flexibly.
Within each iteration, teams are encouraged to use creative approaches.
They are expected to be “self-organizing” rather than follow top-down
directions at each step or follow pre-developed processes. Tasks are
assigned based on the demands of a particular iteration rather than on job
titles. And roles within the team are often flexible – if any specific
roles are assigned.

Pragmatic Development – Agile
projects limit up-front planning. Requirements, budgets, scope, and other
key factors may change throughout development. Typically, the architecture
of the software is not fully determined before development begins.
Instead, the architecture and conceptual design are built gradually as an
organic part of the process. By developing “just enough code” to satisfy a
requirement, Agile development aims to create leaner software. Rather than
measuring progress based on reaching milestones along a project plan
formed at the beginning of development, Agile initiatives measure their
success based on the amount of working software created. The creation of
documentation is not disregarded, but it is given less priority than the
creation of software.

Testing – Agile methods are
not as loose as they might seem. In place of project plan milestones,
Agile development substitutes regular testing. Frequent testing (at
the end of each iteration) structures the Agile process and provides a
basis for discipline in carrying out development tasks.

Refactoring – After code is
tested and proves usable, the Agile philosophy requires that code be
“refactored,” which means that it is cleaned up and streamlined.
Refactoring is not a changing of the software’s features – the code works
before refactoring. The process helps only to clarify the code’s logic,
enabling the software to be more easily understood and changed in the
future.

Organizations that choose to adopt Agile methods do so to seek benefits
such as:

Increased Efficiency –
Heavyweight projects tend to delay the demonstration of software to
customers and other stakeholders outside the development team until near
the end of the process. As a result, a significant amount of effort may be
exerted only to find that the software does not meet the end user’s needs.
Agile methods seek frequent stakeholder input at several points throughout
the development process, enabling stakeholders to give approval or to
recommend changes before the project goes too far down the wrong path.
This minimizes wasted effort.

Greater Adaptability – In
conventional heavyweight development processes, change management is a
complex, often burdensome process. Multiple levels of approval may be
required to make adjustments during development. This can make software
developed by heavyweight processes unresponsive to changing conditions
such as new developments in the business landscape or refinements to a
company’s strategy. In some cases, the final software can even turn out to
be irrelevant to the business problem it was first intended to
address. Lightweight processes, on the other hand, have many stages at
which changes can be made without penalty. This enables development
projects to be responsive to rapidly changing environments and markets.

Reduced Risk – Agile
development’s frequent testing can reduce risk by spotting problems early
on in the process.

Current View

[return to top of this
report]

As analyst Jack Flynn reports, “Agile has become a key component of
software management for many companies.” Consider the following facts and
figures:

“At least 71 percent of US companies are now
using Agile.”

“Agile projects have a 64 percent success
rate, whereas projects under the competing methodology known as Waterfall
only have a 49 percent success rate.” (For context, the “Waterfall”
methodology, as described by analysts Ben Lutkevich and Sarah Lewis,
favors “a logical progression of [software development lifecycle (SDLC)]
steps for a project, similar to the direction water flows over the edge of
a cliff. It sets distinct endpoints or goals for each phase of
development. Those endpoints or goals can’t be revisited after their
completion.”²)

“Agile projects are nearly 1.5 times more
successful than Waterfall projects.”

“After adopting Agile, companies have
experienced an average 60 percent growth in revenue and profit.”³

Outlook

[return to top of this
report]

Having replaced Waterfall as the preferred SDLC methodology, Agile,
according to analysts Kate Brush and Valerie Silverthorne, is “at risk of
being eclipsed or consumed by the growing popularity of DevOps.”⁴

DevOps

The term “DevOps” refers to a philosophy of software development that is
practiced widely today. This philosophy has developed organically rather
than being rigidly defined by a single organization, so it lacks a
universal definition. The most basic definition is that it is the tighter
integration of software DEVelopment and IT OPerationS (see Figure 1). But
the term has taken on a much broader meaning, even encompassing ideas
about management styles and corporate cultures.

DevOps is commonly thought to entail the following:

Increased Collaboration –
Tighter, more routine collaboration is encouraged not only between
development and operations but also among other business functions.
Software development is no longer seen as simply a technical issue.
Instead, it is also a strategic concern.

Continuous Deployment –
DevOps is designed particularly for services which update software almost
daily.

An Emphasis on Ongoing Feedback and
Testing – Providers of cloud services get constant data about
how their software is being used and how it is performing. This “Big Data”
is then used to make regular updates of the software. The size of some of
the leading cloud services makes this data highly valuable in addition to
highly complex to manage and use.

A Culture That Accepts Failure
– DevOps considers problems in the development process to be inevitable,
so it encourages managers and teams to accept problems and not to blame
individual team members.

Extensive Use of Automation –
DevOps aims to automate many parts of development, testing, and
deployment.

Figure 1. A Conceptual View of Software Development and IT
Operations Integration

Source: Wikimedia Commons

DevSecOps

More recently, the software industry is promoting an extended version of
DevOps called “DevSecOps”, in which security testing, like QA-administered
functionality and performance testing, is introduced early in the SDLC.
According to analyst Paul Farrington, “The rapid emergence of the DevOps
movement [was] driven by the exponential growth of the application
economy, which has put a huge amount of pressure on development teams to
hit shorter delivery deadlines. However, the risk of moving too fast
without the proper security process in place leaves an organization open
to applications with poor functionality, buggy software or security flaws
that are all key things that impact heavily on customer loyalty and
retention.”⁵

Distilling the benefits of DevSecOps, analyst Scott Fitzpatrick observes
that “If you continuously vet [an] application for security issues from
the outset of development, it is highly unlikely that a major security
concern would arise toward the end of the delivery pipeline. It’s more
likely that only minor security concerns would exist as the application
development process comes to a close. This is a direct result of providing
visibility into the application’s quality by continuously verifying that
security standards are being implemented properly – a huge time-saver (and
potentially huge money-saver) down the line.”⁶

Recommendations

[return to top of this
report]

While the Agile form of software development was a significant
improvement over the Waterfall method, there are disadvantages.

As catalogued by analysts Kate Brush and Valerie Silverthorne, Agile’s
unfavorables include:

“The fact [that Agile] has been modified –
some would say diluted – by many organizations. This phenomenon is so
widespread that the ‘Agile my way’ practitioners are known as ‘ScrumButs’,
as in ‘We do Scrum in our organization, but ….’

“Although Agile opens the lines of
communication between developers and the business side, it’s been less
successful bringing testing and operations into that mix – an omission
that may have helped the idea of DevOps gain traction.

“Agile [has a] lack of emphasis on technology,
which can make it difficult to sell the concept to upper managers who
don’t understand the role that culture plays in software development.”⁷

Given these difficulties – plus the recent ascendance of DevOps and its
DevSecOps variant, which is aimed at building cybersecurity into new
applications – enterprise software development shops should consider
implementing an orderly – and secure – transition from Agile to DevOps or
DevSecOps software development methodologies.

They should, of course, affect this transition with an eye toward
preserving those elements of Agile which have best served their
organization.

References

¹ Kate Brush and Valerie Silverthorne. “Agile Software
Development.” TechTarget. November 2022.

² Ben Lutkevich and Sarah Lewis. “Waterfall Model.”
TechTarget. November 2022.

³ Jack Flynn. “16 Amazing Agile Statistics [2022]: What
Companies Use Agile Methodology.” Zippia, Inc. November 27, 2022.

⁴ Kate Brush and Valerie Silverthorne. “Agile Software
Development.” TechTarget. November 2022.

⁵ Paul Farrington. “DevOps and the Need for Security to
Shift Left.” 31 Media Ltd. April 9, 2018.

⁶ Scott Fitzpatrick. “From Shift Left Testing to Shift Left
Security: The Pursuit of Quality.” Sauce Labs. October 17, 2017.

⁷ Kate Brush and Valerie Silverthorne. “Agile Software
Development.” TechTarget. November 2022.

Web Links

[return to top of this
report]

• Agile Alliance: https://www.agilealliance.org/
• Agile Manifesto: https://agilemanifesto.org/
• Project Management Institute: https://www.pmi.org/
• Scrum Alliance: https://www.scrumalliance.org/

About the Author

[return to top of this
report]

Geoff Keston is the author of more than 250 articles
that help organizations find opportunities in business trends and
technology. He also works directly with clients to develop communications
strategies that improve processes and customer relationships. Mr. Keston
has worked as a project manager for a major technology consulting and
services company and is a Microsoft Certified Systems Engineer and a
Certified Novell Administrator.

[return to top of this
report]