Software development and delivery process is the backbone of
business for a professional organization in software industry. Software
development is a team activity involving more than one professional with
defined set of roles and responsibilities. The involvement of multiple people
in the process requires a well defined work flow to be maintained. Apart from
human resources the software development process (SDP) typically requires
development software tools, platforms and hardware infrastructure and most
importantly a strong collaboration mechanism within the team involving the
artifacts of the delivery. Ideally a
software development and delivery process (as per the water falls model)
involves certain technical steps or phases as briefly described below. Phases
including financial, business and management activities such as scoping,
estimation, planning are not included here.
Requirement Study
& Analysis: Where a person (Requirement Analyst) interacts with the user or
customer to understand the process and requirement that the software (to be
developed) should implement. The information captured in this phase is
generally captured in Software Requirement Specification (SRS). Test cases to
validate the system (after development) are also defined following this phase.
Architecture &
Design: Specifying and documenting the architecture and design for the
software system to visualize the basic structure or model which should be
followed to construct the software system. This phase can have two primary
steps such as -
·
Defining the business architecture or functional
design by covering the business processes and logic
·
Defining the technical architecture or system
design explaining the structure of system components required to implement the
business processes.
Design and Architecture specifications are documented in
this phase.
Development &
Coding: In this phase, different software components are built or developed
according to the design specified in the previous phase and tested. The outcome
of this phase is source code and executable code.
Deployment & System
Testing: All components developed are put together and tested to evaluate
the ability of the system to provide the functionalities specified during the requirement
capturing phase. Test cases defined during the requirement capturing phase are
executed in this phase and test results are captured and defects found during
testing are fixed.
User Acceptance:
Once system testing is completed the software is submitted to the customer for
final acceptance. This phase ideally have customer testing involvement. Defect
fixing activities (if any) are also involved in this phase.
Operation &
Maintenance: Once the software is accepted by the customer, it is finally
delivered and installed for final use. In case any change in requirement takes
place or any problems are identified by the customer after the delivery is
done, necessary changes to implement new requirement or to fix problems are
implemented in the system. In this case a new waterfall (starting with
Requirement Analysis phase) can be followed for each new requirement or defect.
Please refer to the waterfall model at
http://en.wikipedia.org/wiki/Waterfall_model.
Monitoring &
Tracking: Overall monitoring and
tracking activities for the entire project are performed by a project manager
(PM) throughout the phases of a software development lifecycle (SDLC).
The steps mentioned above are depicted in figure 1 below.
Figure 1: Software development
lifecycle phases
Æ
Majority of employees in a software development
organization (SDO) are involved in the process of software development for
customers. The development activities (projects) are performed by forming
project teams consisting of different people. Typically a lead or manager is
involved to monitor and track a project. Apart from the project team members (PTM)
other stakeholders such as reviewers, users and customer also get involved
throughout the lifecycle of a project. Nowadays most of the stakeholders come
from different geographic locations even with different time zones. This is the
reason why a project requires a strong collaboration mechanism to be in place.
Æ
Usage of standard practices, software tools and
frameworks increases the efficiency of the software under development by
reducing defects and improving performance, flexibility, extensibility etc.
Enforcing standards and best practices over a team of people can be a challenge
without a centralized control and strong collaboration.
Æ
The process of requirement study & analysis does
not only involve human interactions, it can also involve several documents and
pre-existing software systems as well. Sometimes existing source code from the
previously existing system can also be taken as the input for requirement.
Recovering the design and logic from an existing system or source code might
need a reverse engineering mechanism to be in place.
Æ
Proper event tracking technique always helps in
getting everyone in a team notified about the latest updates and help them to
react promptly.
Æ
The next problem commonly faced is increasing
gap in communication which often results in a big difference between the
requirement and the development software system. Figure 2 illustrates the same
with a popular cartoon. Usually this problem leads to wastage of investment and
following rework.
Figure 2: Gap between requirement and
implemented system
Æ
Misunderstandings in certain areas of
requirement between a person with and without software background create major
problems in communication as illustrated by the famous Dilbert Cartoon in
figure3. Ideally a predefined flow of communication and precisely designated
requirement capturing interface with relevant questions and clarifications can
help in reducing such gaps. Educating the stakeholder (without software
background) a little also helps here.
Figure 3: Misunderstanding in
requirement capturing
Æ
Communication Gaps can exist between the
requirement analyst (who understands the requirement from the customer or user)
and the architect or designer which can be minimized by recording requirement
with all necessary information which can help the architect or designer. Similar
gaps can exist between designer and developer which lead to defects introduced
during development phase.
Æ
Sometimes manual effort spent in build,
deployment and installation procedures leads to human dependency. This process
can be made automatic and time-driven with the help of a Continuous Integration
(
http://www.thoughtworks.com/continuous-integration)
technique.
Æ
Rather than putting the test cases in document
only and executing manually during the testing phase, Test Driven Development
(TDD) Tools can be used to record the use cases and execute them automatically
by integrating the same with Continuous Integration tools. Martin Fowler’s
article (
http://www.martinfowler.com/articles/continuousIntegration.html)
on Continuous Integration explains this more clearly.
Æ
Overall a central repository with version
control mechanism is a very important component to keep all artifacts which are
referred (e.g. customer supplied documents) and produced (e.g. SRS, Design
Document, Source Code etc.) during a project secured.
The primary goal of software development process is to
produce deliverables according to requirement with expected quality. The
artifacts produced by a project which can be treated as deliverables are
usually the SRS, architecture document, design document or source code etc. The
final goal of a software development project should ideally be a working piece
of software meeting customer’s requirement.
During SDLC, injection of defects is inevitable. Defects are
introduced in different phases of a software development project such as
Requirement Analysis, Design or Development. The impact of defects on delivery
process and cost to fix them is directly proportional with time.
The bugs discovered at the end
of the SDLC would cost more to get fixed. So it is better, to find problems in
the initial state of SDLC and fix them at the earliest. Based on a study (
https://buildsecurityin.us-cert.gov/articles/best-practices/code-analysis/business-case),
it has been found that, the savings potential from early defect detection is
huge; nearly 60 percent of all defects usually exist by design phase. Figure 4 shows how the cost to fix a defect
increases with time.
There are certain techniques such as Pair Programming (PP),
Continuous Integration (CI), and Test Driven Development (TDD) etc. which are
found to be highly effective in catching the bugs early. While we think of an
integrated software development environment (ISDE) as a solution, these
techniques need to be considered as well. So we are assuming that, our solution
will be equipped to facilitate such techniques. For this article, let’s assume
a name for the solution as Integrated Software Development Factory (iSDF).
Figure 4: Increasing Cost to Fix a
Defect with time
A discussion on so many pain areas together is enough to get
a strong headache for anyone. So if we quickly look for a pain killer, a
solution to address all primary pain areas in developing a software solution or
system, we need to map the resolutions of the pain areas as use cases to the
pain killer. Let’s name this pain killer as Integrated Software Development
Factory (iSDF). Typical use cases for iSDF deduced from the pain areas
described above are the following.
·
Submit Requirement: The software end-user or customer should be
able to enter his / her requirement in our iSDF using a graphical user
interface. The customer should also be able to submit associated documents
supporting the requirement. The requirement will be stored in system and later
referred by other involved in the project team. Once a requirement is submitted,
all relevant stakeholders such as requirement analyst, PM and architect should
be notified to take necessary actions to process the same.
·
Study & Analyze Requirement: The requirement analyst should have an
interface to view the requirement statements and supporting documents entered
by the customer for analysis. The analyst should also be able to interact with
the end-user / customer for clarifications on the requirement. The analyst
should be able to produce and submit an SRS document according to pre-defined
format or template. Once the SRS is submitted, required stakeholders such as PM
and architect should be notified to take necessary actions to process the same.
iSDF should capture requirement in predefined format such as use case or user
story so that other PTMs can refer to the same following a specific standard
and paradigm (e.g. Object-Oriented Paradigm). Specially, the designer should be
able to refer to the use cases or user stories by using a design tool
integrated with iSDF. This will help in reducing the interpretation gap between
requirement and design.
·
Define Architecture: iSDF should provide an interface for
the Architects to view SRS and define the architecture for the software
application based on the SRS submitted by the analyst. Relevant project members
such as PM and architecture reviewer should be notified while architecture is
defined and submitted in iSDF.
·
Review Architecture: Once the
architecture is produced, a technical reviewer should be able to review the
architecture and provide comments on the same. The architect will incorporate
changes suggested by the reviewer (if any) and update the architecture
document. Proper notification mechanism (to PM and Architect) should be in
place on review comments submission.
·
Define Test Cases: iSDF should provide an interface for the Testers to view requirement
and define required test cases for software application based on the SRS submitted
by the analyst. Relevant project members such as PM, analyst and customer (if
required) should be notified while test cases are identified and submitted in iSDF.
·
Review Test Cases: Once test cases are produced,
a reviewer (ideally analyst or business users) should be able to review the test
cases and provide comments on the same. The test case creator will incorporate
changes suggested by the reviewer (if any) and update the test cases. Proper
notification mechanism (to PM, analyst and other concerned members) should be
in place on review comments submission.
·
Define Design: Once the architecture is finalized after review, the designer
should be notified to start producing the software design. iSDF should provide
an interface for the designer to view the requirement (in form of use case or
user story), architecture and supporting documents such as SRS and architecture
document. The designer should be able produce the software application design
by using some design tools integration with iSDF and submit. Once submitted iSDF
will store the design and relevant documents and notify the PM and design
reviewer.
·
Review Design: Once
the design is created, a technical reviewer should be able to review the design
and provide comments on the same. The designer will incorporate changes
suggested by the reviewer (if any) and update the design document. Proper
notification mechanism (to PM and designer) should be in place on review
comments submission.
·
Develop & Build: The next phase to design is development
and build, which primary includes the generation of source code in desired
programming languages, customization and configuration. The developers should
be provided with proper interface (in iSDF) to be able to build or develop software
applications to work in this phase.
Integrated development environments (IDE) like Eclipse and NetBeans are
few examples of development tools which can be integrated with iSDF so that,
design models (e.g. UML) can be viewed and referred using the IDE. Code
generation options can also be integrated to generate code from the design
models whenever required. The source code, configuration files and other
associated artifacts produced during this phase should be stored in a
centralized repository secured by a version control system such as Git, SVN and
CVS etc. so that everyone in a development team should be able to share and
update the source code as required.
An automatic build tool can be integrated with iSDF so that, the build process
can be scheduled in a periodic manner or based on an event (such as on any
update in source code repository). Usually this feature is available in typical
Continuous Integration (CI) tools like Hudson, Apache Gump and CruiseControl
etc. Once triggered by a time-driven or a event-driven approach, the scope of
activities for a CI tool should be to check out the source code and
configuration files from version controlled repository, compile and build the
same, deploy the application on an execution platform e.g. an application
server for a test tool to start testing.
Technical
reviewers should be notified on any code change or repository update so that,
the new code change can be reviewed.
Figure 5: iSDF use cases
·
Review Code: Once
a new code or modified piece of code or other artifacts are submitted in
repository, a notification should be send out to assigned reviewer to review
the new or modified artifact and provide comments or approve the change. Once
the comments or approval is posted the concerned developer or changer should be
notified by iSDF notification mechanism. For any clarification, an integrated
collaboration mechanism can be used by the reviewer and changer if required. To
support required activities during the review, a code review interface, a
notification, a collaboration mechanism and workflow should be in place.
·
Test: This
is an important phase in a SDLC. iSDF should have an interface for the testers
to produce test cases associated with the requirement of a project during the
requirement analysis phase. Once the development is completed, testers should
be able to configure the test cases in automated testing tools integrated with
iSDF. For example, Fitness is a very useful wiki based tool to implement Test
Driven Development (TDD). Such tools can be integrated with the CI layer
(within iSDF) so that, build, deployment and testing can be automated removing
human dependencies. Once test results are produced, a notification mechanism
can circulate the results to concerned parties (e.g. developer, project manager
and tester etc.) for further actions.
·
Defect Tracking: Defects can be
recorded and monitored using a Defect Tracking tool such as Bugzilla, Jira,
Trac, Traxis and Mantis etc. In order to do this, iSDF should be integrated
with a Defect Tracking Tool. While defects are introduced concerned parties can
be notified by the system for further actions.
·
Deploy & Install: Different versions of application
execution platforms or environments for development, testing, staging and
production can be set up and integration with iSDF so that, once the
application is developed and tested is can be easily deployed or installed on a
designated execution environment. Developers can use a development environment
to deploy and perform unit testing on their code. On completion of development
and code review, individual code changes can be integrated and deployed on a
testing environment for the testers to start testing. Once the testing is over,
the tested application can be deployed on staging environment for customer’s
acceptance testing or review.
·
Acceptance by Customer: Our iSDF
solution should also provide an interface for the customers or business users
to test and review the completed application and provide final acceptance or
sign off. Typically, a staging environment and an interface to report defects
are provided for customer acceptance testing. The defect reporting interface
should have notification capabilities to keep the concerned parties informed.
·
Monitor & Track SDLC: Apart from an event based notification
mechanism, iSDF should also provide a facility so that the entire SDLC
followed in a project should be monitored and tracked by a project manager and
other concerned stakeholders on a regular basis. Usually the progress of a
project, issues and defects are the primary items of interest for monitoring.
·
Administration: Capabilities like User
management, user authentication, authorization, project creation, project
member allocation can be implemented as administrative activities required for
iSDF.
In order to implement the abovementioned use cases certain
technical capabilities will be necessary for our iSDF. These capabilities
(listed below) together can be considered as a good recipe for iSDF.
iSDF should have a
Web based Graphical User Interface or portal to be able to provide
all user interfaces required for requirement submission, requirement analysis,
collaboration and document management.
A
Workflow
Management Framework will be required to manage workflows and human
interactions in terms of requirement analysis and reviews etc.
A
Software
Design Tool or interface should be integrated with iSDF so that, the
requirement documents or SRS can be referred by the designer and design the
software accordingly. Ideally a use case model prepared by the requirement
analyst should be captured and shared among relevant users within iSDF in a
predefined format so that the design tool can interpret the same as the
starting point of design. Apart from preparing design models (e.g. class
diagram, sequence diagrams and activity diagrams etc.), the designer can also
prepare a design document and store the same in centralized repository of iSDF.
The centralized repository can be controlled by version control systems to
secure the documents.
A
Code
Generation feature can be added to iSDF to generate initial code structure from
design. For example, once the designer completes class diagrams for an
application and executes the code generator on the same, the code generator
should be able to read the class diagrams and generate initial source codes in
desired programming languages (e.g. Java, C++, C# etc.). Once the initial
source code structure is generated, they can be stored (checked) in a version
controlled repository for reference later by the developers.
Development
Tools or environments can be integrated with iSDF, where developers can check
out the existing source codes from version controlled repository, modify the
code, compile and check in the modified code back in the repository. Once there
is an update in source code repository, an automatic event or notification
should be generated in iSDF that should trigger the build process (through a CI
tool).
A
Reverse
Engineering facility of iSDF can help in generating design models from code
(in case existing source codes are referred as requirement artifacts). For
example, class diagrams can be generated from Java source code.
A
Reusable
Component Repository (containing standards, frameworks, reusable code etc.)
should be available with iSDF for the designer, architects and developers to
refer to and reuse during architecture definition, design and development.
An
Email Notification
mechanism to notify members involved in a project or administrator whenever
there is a change in repository. For example, the PM and requirement analyst
should be notified whenever the customer or business user submits a
requirement. Similarly architects, designers should be notified whenever the
SRS is finalized and code reviewers should be notified once there is an update
on source code.
A
Collaboration
Framework for the geographically distributed members to collaborate within
each other during a review, analysis and general discussions. The mode of
communications can be implemented through chat, email, desktop sharing and
video chatting etc.
A
Continuous
Integration platform or tool to automate build, testing and deployment
processes should be implemented in iSDF. This is make the process of compiling,
building source codes and deploying application packages on desired execution
platforms e.g. application or web server etc. Once an update is made on source
codes by the developers and automatic event will be generated which will
trigger the CI tool to start the build and deployment process. Figure 6
depicts a typical continuous integration
process.
A
Build
Tool should be provided as an integrated feature of iSDF. This tool should
ideally be integrated with the CI component so that, whenever there is an
update in code repository this tool can be automatically invoked based on an
event. On invocation, the tool should be able to check out the latest source
code changes from the code repository, compile, build and deploy the same on an
execution platform (e.g. application server) for execution. Some popular build
tools used nowadays are Apache Ant, Apache Buildr, Apache Maven, Cabal, Gradle,
Bamboo etc.
Integrated
Testing
Tools for software application testing should be provided as an important
feature of iSDF. The testing tools which can implement a test driven
development (TDD) process such as Fitnesse. It should provide the feature to
configure the test cases and test data in it. The testing tool should be
integrated with the continuous integration tool or platform so that, on each
update in source code or application configuration, testing can be executed
automatically and the results can be shared (using email) with appropriate
stakeholders. Ideally when the application is built and deployed on the
execution platform (e.g. Application Server), the testing tool should start
executing the test cases which were already pre-configured in it.
Figure 6: Continuous Integration
Process
A
Defect
or Bug Tracker tool can be integrated with iSDF to manage and track defects.
These tools can be integrated with iSDF to register and track bugs or defects
found during testing. Monitoring and tracking of defect life cycle and
maintaining a defect repository for further analysis are important aspects of
this.
A
User
Authentication & Authorization mechanism to authenticate and authorize
users (preferably directory service integration will be a modern technique for
this) should be integrated with iSDF. This will enable security and restrict
user access based on authority.
Reporting
Tool is a significant component for any application software. For iSDF, the
customer, project manager and other designated stakeholders can monitor and
track the software development process by using desired web based reporting
interfaces or dashboards.
Figure 7 shows the entire iSDF solution as conceptualized in
this article.
Figure 7: iSDF Solution
