Agile Dev West conference – June 4–9, 2017- Las Vegas, NV USA

Date : June 4–9, 2017

Location : Las Vegas, NV

Discover the latest in agile methods, technologies, tools, and leadership rinciples.

Whether you’re new to the agile process and need to get up to speed quickly, or you’re experienced and ready to take your team or organization to the next level, our hands-on, in-depth workshops have you covered. Plus, all Agile Dev West is held in conjunction with Better Software West and DevOps West conferences, allowing you to choose from three distinct programs.

To register for the event, click here.

Measuring Agile Success

Agile methodology is an iterative approach typically used in software development as an alternative to the traditional project management. The approach emphasises on building the software incrementally from the beginning instead of delivering everything at the end, and solutions and requirements evolve sequentially. Product is broken down into incremental builds which are provided in iterations. Each iteration lasts typically almost one to three weeks, and requires cross functional teams to work together in different areas such as requirement analysis, planning, coding and testing. After each iteration, a working software build is delivered and working product is shown to all the important stakeholders, including customers. The builds are incremental and the final build consists of all the required features.

In agile model, each project is handled differently, and methods are tailored so that they best suit the requirements of the project. Some of the most popular agile methods include

• Scrum
• Rational Unified Process
• Extreme Programming
• Crystal Clear
• Feature Driven Development
• Adaptive Software Development
• Dynamic Systems Development Method (DSDM)

Pros of Agile

In agile methodology, a common assumption is that the needs and requirements of the end users are dynamic, and hence, based on the continuous feedback, changes are discussed and features are added, removed or modified accordingly. There is frequent delivery of software, and a lot of emphasis is placed on the face-to-face interactions across teams. There is daily coordination between the developers and business personnel to achieve good design and technical excellence. This makes it much faster to adapt easily and regularly to the changing circumstances, even if these changes are much later in the software lifecycle. As the frequency of new increments is high, the changes can be implemented with minimal cost and time.
Compared with the traditional waterfall approach, stakeholders and developers in agile get more options and time, reducing the risk of reaching a sudden standstill. This also ensures that the end product matches with the customer requirements in a best possible way.
Agile is a very realistic approach which is suitable for the dynamic requirements, promotes cross-training and teamwork, easy to manage, requires little planning and provides flexibility to the developers.


This model is not suitable for handling complex dependencies. As there is minimum document generation, there is high individual dependency. Lack of documentation also makes the transfer of technology quite challenging. There is also additional risk of sustainability, extensibility and maintainability.

For larger projects, it may be difficult to assess the requirements at the beginning of the SDLC (software development life cycle). Additionally, agile relies heavily on customer interaction and if they are not sure about their exact requirements, the entire team can head in a wrong direction and project can get completely off track. It can also be challenging for the new employees and programmers to take the required decision during the development process.

Measuring agile success

In agile methodology, it is essential to track business metrics, as well as agile metrics. Business metrics help analyse if the solution is meeting the requirements, and agile metrics help in the analysis of the development process. Sound agile metrics help reduce confusion, and can motive team’s progress through the development cycle.

State of Agile Survey provides reports that give the software professionals a deep insight into latest agile trends, lessons learned and best practices. This helps organisations plan for successful agile transformations. In this Survey, 58% of the participating respondents confirmed that they used on-time delivery as a measure of the success of their agile initiative. The measure of on-time is based on the delivery expectations, and the measure of delivery can be assessed from burndown or burnup.

Product quality is another metric to measure agile initiative. There are multiple ways to measure quality, including revenue growth, customer satisfaction, and different technical aspects .The focus is on the speed and built-in quality of the working software. Testing and inspection are integral part of the complete development lifecycle, and there is constant monitoring of build and code health along with the testing trends.

Another metric for measuring agile success is the user or customer satisfaction. The outcome can be measured in terms of sales figures, net promoter score, usage statistics etc. Agile success can also be measured in terms of business value. If there is a contract for work completion, measuring business value becomes quite explicit. However, if market inputs drive the decisions, measuring business value become speculative or prospective.

Some companies also measure their agile success in terms of project visibility. To have transparency requires that the progress at multiple dimensions should be made visible to all the stakeholders. Metrics showing the targeted plan with the overall progress help stakeholders get a better insight.
Velocity trend is the metric used to assess predictability, and it shows the average pace of the work. Wildly fluctuating velocity reflects changing teams, or unpredictable work.

It is evident that a single metric cannot be used to assess the agile success. Each organisation is different, and every software development project is different. Teams need to figure out the best ways to pilot, extend and finally excel at Agile. This methodology requires full participation from all those involved as it is not a quick fix or a fleeting fad.

Adopting Agile Testing

The term agile is sometimes used to represent anything that is dynamic or an unstructured way of working with others. However, as per the document “Agile Manifesto” conceived by a group of software engineers, it is a focused and yet rapidly iterative software process adhering to a set of principles, and aims to promote a more efficient, humane and collaborative way of developing computer programs and IT systems.  Agile testing is the practice of software testing for bugs or performance issues that are within the context of the agile workflow. It focuses on the concept of  Whole Team, getting as much feedback as possible on code, and building the quality as early as possible. Agile testing has shorter feedback loop between product owner and team members. Agile testers are an integral part of the cross-functional team, and have their say in all the phases of SDLC (Software Development Life Cycle). The flow of information is uninterrupted due to collocated teams. Let us take a closer look at some of the key features of agile testing methodology, and why more and more companies are adopting it.

Why Become Agile?

Agile and traditional waterfall methodologies differ in terms of mentality of teams, role of testers in the team, and at what stage the testing begins.

  • In conventional testing, test execution does not start until after the complete functional development.
  • In traditional testing methods, there is usually significant delay between when the software is written and developers get the feedback. Defects and bugs are found quite late in the process which can cause serious issues if any major changes are required.
  • If the business requirement changes, it affects the already developed test cases.
  • As the communication and testing is siloed, the chances are that different groups will have different final product expectations.
  • Also, as testing is the last activity before the decided release date, the quality of testing suffers.

Traditional QA engineers rely heavily on documents, however, generating test cases may not be as cut-and-dried in agile, as agile testing QA engineers do not get big requirements documents as a base for test cases, and do not get months to write the test cases. They directly become the part of the communication streams which can be written, verbal or virtual, and collect the information they require. In agile, it is accepted that the change is healthy.  Some of the key features of agile testing are:

  1. In the Agile approach, testers and developers are seen as two sides of the same production coin that meet regularly and compare notes daily. Testers, developers and quality-assurance personnel work closely together replacing siloed functions. The split between software testers and software developers in the traditional waterfall process positions them as separate entities at different points along the production cycle, and this is the most fundamental problem that agile helps resolve.
  2. Agile testing is testing when possible, even as early as requirements gathering phase and there is Test Driven Development (TDD). Agile testing drives development by questioning stories and refining acceptance criteria during the iteration planning. It starts by working on stories, and continues with the TDD.
  3. Agile requires continuous and elaborate collaboration between stakeholders throughout the company, throughout the production process. In each and every phase of the development, testing is an essential component, and quality is built in at every stage through constant feedback from all who are responsible for the final product. Hence, the working versions with essential features of the final system are frequently produced. Each of these working systems are fully integrated and carefully tested, just like final delivery.
  4. It is a team effort where anyone can pick-up and execute a testing task. Developers and design experts also think about the testability of the product and testers can give their inputs in the early stages of software architecture or designing interface.
  5. Testing starts when the Sprint starts and entire team is responsible for the result. Testing includes test planning, build acceptance testing, functional testing, regression testing, demo testing and test automation.
  6. Testers are an integral part of the team, and their experience is fully utilized where they participate in planning and requirement analysis.
  7. The feedback loop is short and testers actively participate in the Retrospective and Planning meetings.
  8. Pair testing can be used together with the developer or other tester. Testers can also help developers create automated tests or design tests which are beneficial for both.
  9. There is continuous integration for every change committed to the source code repository.
  10. Teams switch to agile due to simplicity of the principles, transparent communication with customers and accurate planning.
  11. Agile testing may seem complex, however if implemented successfully, it leads to greater simplicity as it is more risk and priority focused, faster, adaptable, flexible and efficient.

Critics of Agile

The experience of Agile can vary, and for some, it just results in lower quality, chaotic processes, miscommunication, and a bundle of several other issues. A research was conducted by Voke Media with 200 software companies on their attempt to embrace Agile, and their report stated:

Out of over 200 participants, 64 percent said that switching to Agile Development was harder than it initially seemed.  Forty percent of respondents did not identify an obvious benefit to the practice.  Out of those who did, 14 percent thought it resulted in faster releases, and 13 percent—that it created more feedback.  Seven percent of participants noted that Agile developers were happier due to reduced future planning and documentation.

There is no doubt that there are certain challenges involved in implementing agile testing methodology. As testing happens on the fly, there is minimum documentation, which can cause problem if the team member is unfamiliar with the project. In such case, handing over the project to someone completely new can be disastrous as only senior executives are capable of taking the decisions required in agile testing. If the customer is not clear about the final outcome they want, the project can easily get taken off the track.

Additionally, agile testing principles can be quite demanding on the developers’ time, and require their commitment for collaboration throughout the project. Agile testing also requires more effort from the entire team as testing continuously gets modified or interrupted to fit the need of the situation. Agile can be quite time consuming as developers and testers spend a lot of time through the iterations to ensure the best quality throughout the project.

In the agile environment, there are frequent changes in the system, and to test each change, regression testing is required to ensure that there are no new defects in the previously developed features. Planning regression testing is difficult as agile testers are busy testing new stories for current sprint. Hence, it is necessary to have automated tests for previous sprints checked into the source repository.

Inadaptability to Agile could be due to difficulty in leaving the traditional understanding of the roles, and resistance to change. Agile is not a panacea that will solve all testing problems, however, its principles are great tools to reveal several problems, and it is up to people how effectively they are able to resolve them. Agile development and agile testing go hand in hand, and together they complete the idea of agile methodology. Frequent testing along the lines of iterative guidelines are the benchmarks of the agile testing. For the best chances of success, the testing engineers must become embedded agile team members, and embrace agile team dynamics.