Agile + Waterfall = Hybrid Method; when does it have the right application?

Recently, I was in doing feasibility and due diligence of a Technology Refresh project. This project would involve rewriting a suite of applications, that are running both in command line batch mode and desktop mode. Driven by the idea of long-term technology risk aversion, better maintenance and support, availability of skills; these applications would involve rewrite from legacy technology to latest Microsoft Framework. This technology refresh, or precisely upgrade would involve no new feature development or existing feature enhancements.

One obvious question that came during this feasibility was – ‘Should this project be executed in Agile Scrum methodology?’. In this post I am going to share my observations and rationale to decision made in answering this question. In my view, the selection of project execution method should not be based on emotion, trend and influences from project stakeholders. In this case also there were a section of people, both from customer and management; who were strongly in favor of following Agile Scrum method of project execution. At the same time there were some people who were in favor of following Waterfall approach.

Let me share some more project insights that would help in making appropriate decision to select right execution methodology –

1. Customer Adapting to Agile: Customer organization have IT and software development divisions, who are responsible to maintain existing and develop new applications required for business continuity. Organization started moving to adapt Agile methods. There are some projects already running following Scrum method, and slowly more teams are getting engaged and trained to Agile scrums. There is shift and demand to follow Agile from management, as they see benefits to agility in terms of visibility and transparency.
2. Fixed and Known Requirements: As mentioned above, it is a technology refresh, with no new feature development or enhancements; simply to upgrade retaining the applications feature as-is. However, there are some obsolete and redundant functionality that would need to be trimmed down from the to-be system. Which concludes that the project’s requirements are known and fixed.
3. Fixed Time and Fixed Budget: The overall efforts required to upgrade the applications could be estimated precisely with greater confidence. The project time and cost would be fixed and accordingly the decision on project budget and time required to involve business users could be predicted with greater accuracy.
4. Business User Engagement To Inspection: Being existing applications and no new features, business team who are the actual end-users of the applications; will get involved in acceptance testing only when the application is developed completely to test and provide signoffs for production deployment.
5. Technology & Architecture Complexity: The legacy applications are client-server based architecture. To-be applications are expected to operate in same capacity with moderate degree of scalability. The to-be technology, application architecture and design are not expected to be complex.

Considering Factors To Decide Execution Methodology

Knowing the increasing demand for Agile Scrum method, inherent natures of both execution approach and given project scenario; we can take decision appropriately. I took the listed project insights to next level to validate them in terms of their positive and negative influences on both approaches. Below are some of my findings –

Key factors that favors to adapting Agile Scrum –

1. Swim with wave, customer has at least basic level of understanding and plan to move to Agile Scrum method of project execution. Further agility would feed-in with more practice and participation.
2. Following Agile Scrum approach would help is better project visibility, transparency and control for customer.
3. There would be more opportunities to frequent inspection, reviews and course correction.
4. It will help to control cost, risk aversion, and prioritized delivery of applications.
5. By prioritizing the features and removing obsolete features; would help to mange the scope better, and include new feature if needed.

Key factors that favors adapting Waterfall Approach –

1. Fixed scope and known requirements – so better predictability and scope management. Both parties know what is being developed and will be delivered when project completes.
2. With no new features and enhancements, it as is technology refresh. Just lift and shift to latest technology with no changes to functionalities.
3. Waterfall methods need less or not continuous inputs from business team, which means low business involvement during the project life. Having known requirements, business would involve only during acceptance testing and signoffs.
4. End deliverables remain same on project completion – same business process achieved through same set of applications. Business process is averse to technology, process value-stream and end results are going to remain same in both as-is and to-be applications.
5. Induct Quality Analysts only when project enters to testing phase, instead of from the beginning of the project. This would help in cost reduction, but the quality still can be maintained with appropriate test executions.

Some key factors that strongly negates the Agile Scrum method:

1. There is likelihood to falling trap to scrum-falls & practicing non-value adding Agile practices – which are avoided if waterfall approach selected.
2. The mandate of iterative and incremental development and testing required in all iterations does not add much value here. The iteration-end working software can be only tested but can not be deployed in production; because business can not function until the complete working software with all required features are is deployed in production.
3. More of less these applications are required for business continuity, are stable and less prone to evolve or change. Which means adding money and time to implement automation for unit test, integration & build; would not have significant return on investment; specially when rolled-out to production.

Then, which execution approach and how to select?

It’s obvious now, with all these explanation we are clear – why we should not use the Hybrid approach – taking best and suitable out of both methods. So we have Hybrid Methodology, combining practices that works best in given scenario; specific to the project requirements and given context. Some strictness and rigidity adapted in waterfall need to be renounced and replaced to augment the flexibility and agility. It’s not necessary that whatever the Hybrid Method and practices that has been recommended in this case; would be necessarily applicable to all scenario – be selective in deciding your Hybrid Methodology.

Some factors that would bring agility in this scenario –

1. Improve the project governance model and project structure team appropriately. Team structure and size can vary in different project phases.
2. Increase the project visibility through frequent checkpoints and progress reporting (weekly), joint reviews (bi-weekly/monthly) and senior management reviews (monthly/quarterly) etc.
3. Adapting Agile practices like daily standup, retrospective, frequent inspection, course correction and some degree of automation (like integration / build) as required.
4. More & frequent customer IT (development and/or operations) teams involvement in review, testing & feedbacks.
5. Prioritize and group applications to plan releases as per business groups preference, availability etc.
6. Observe diluted phase-gates by having opportunity to correction (even after phase completion/signoffs). Multiple testing phases for customer IT and Business users team as planned for applications.

Conclusion

With the increased adaption to Agile methods, it does not mean that Waterfall method do not work good and not necessarily every projects should be executed in Agile Methods. By being innovative and selective to execution approach before taking the final call as per project under consideration; still Waterfall and Hybrid Approach makes a wise option in terms of cost effectiveness and team efficiency; resulting to overall benefit to both customer and service organizations.