Technological Thoughts by Jerome Kehrli

Test Driven Development is a development practice from eXtreme Programming which combines test-first development where you write a test before you write just enough production code to fulfill that test and refactoring.
TDD aims to improve the productivity and quality of software development. It consists in jointly building the software and its suite of non-regression tests.

The principle of TDD is as follows:

1. write a failing test,
2. write code for the test to work,
3. refactor the written code,

and start all over again.

Instead of writing functional code first and then the testing code afterwards (if one writes it at all), one instead writes the test code before the functional code.
In addition, one does so in tiny small steps - write one single test and a small bit of corresponding functional code at a time. A programmer taking a TDD approach shall refuse to write a new function until there is first a test that fails - or even doesn't compile - because that function isn't present. In fact, one shall refuse to add even a single line of code until a test exists for it. Once the test is in place one then does the work required to ensure that the test suite now passes (the new code may break several existing tests as well as the new one).
This sounds simple in principle, but when one is first learning to take a TDD approach, it does definitely require great discipline because it's easy to "slip" and write functional code without first writing or extending a new test.

In theory, the method requires the involvement of two different developers, one writing the tests, then other one writing the code. This avoids subjectivity issues. Kent Beck has more than a lot of examples of why and how TDD and pair programming fit eXtremely well together.
Now in practice, most of the time one single developer tends to write tests and the corresponding code all alone by himself which enforces the integrity of a new functionalities in a largely collaborative project.

There are multiple perspective in considering what is actually TDD.
For some it's about specification and not validation. In other words, it's one way to think through the requirements or design before one writes the functional code (implying that TDD is both an important agile requirements and an agile design technique). These considers that TDD is first and foremost a design technique.
Another view is that TDD is a programming technique streamlining the development process.
TDD is sometimes perceived as a way to improve quality of software deliverables, sometimes as a way to achieve better design and sometimes many other things.

I myself believe that TDD is all of this but most importantly a way to significantly reduce the "Total Cost of Ownership (TCO)" of software development projects, especially when long-term maintenance and evolution is to be considered.
The Total Cost of Ownership (TCO) of enterprise software is the sum of all direct and indirect costs incurred by that software, where the development, for in-house developped software, is obviously the biggest contributor. Understanding and forecasting the TCO and is a critical part of the Return on Investment (ROI) calculation.

This article is an in depth presentation of my views on TDD and an attempt to illustrate my perspective on why TDD is first and foremost a way to get control back on large Software Development Projects and significantly reduce their TCO.

All the work on Agility in the Software Engineering Business in the past 20 years, initiated by Kent Beck, Ward Cunningham and Ron Jeffries, comes from the finding that traditional engineering methodologies apply only poorly to the Software Engineering business.

If you think about it, we are building bridges from the early stages of the Roman Empire, three thousand years ago. We are building heavy mechanical machinery for almost three hundred years. But we are really writing software for only fifty years.
In addition, designing a bridge or a mechanical machine is a lot more concrete than designing a Software. When an engineering team has to work on the very initial stage of the design of a bridge or mechanical machine, everyone in the team can picture the result in his mind in a few minutes and breaking it down to a set of single Components can be done almost visually in one's mind.

A software, on the other hand, is a lot more abstract. This has the consequence that a software is much harder to describe than any other engineering product which leads to many levels of misunderstanding.

The waterfall model of Project Management in Software Engineering really originates in the manufacturing and construction industries.
Unfortunately, for the reasons mentionned above, despite being so widely used in the industry, it applies only pretty poorly to the Software Engineering business. Most important problems it suffers from are as follows:

• Incomplete or moving specification: due to the abstract nature of software, it's impossible for business experts and business analysts to get it right the first time.
• The tunnel effect: we live in a very fast evolving world and businesses need to adapt all the time. The software delivered after 2 years of heavy development will fulfill (hardly, but let's admit it) the requirements that were true two years ago, not anymore today.
• Drop of Quality to meet deadlines: An engineering project is always late, always. Things are just a lot worst with software.
• Heightened tensions between teams: The misunderstanding between teams leads to tensions, and it most of the time turns pretty ugly pretty quick.

So again, some 20 years ago, Beck, Cunningham and Jeffries started to formalize some of the practices they were successfully using to address the uncertainties, the overwhelming abstraction and the misunderstandings inherent to software development. They formalized it as the eXtreme Programming methodology.

A few years later, the same guys, with some other pretty well known Software Engineers, such as Alistair Cockburn and Martin Fowler, gathered together in a resort in Utah and wrote the Manifesto for Agile Software Development in which they shared the essential principles and practices they were successfully using to address problems with more traditional and heavyweight software development methodologies.

Today, Agility is a lot of things and the set of principles of practices in the whole Agile family is very large. Unfortunately, most of them require a lot of experience to be understood and then applied successfully within an organization.

Unfortunately, the complexity of embracing a sound Agile Software Development Methodology and the required level of maturity a team has to have to benefit from its advantages is really completely underestimated.
I cannot remember the number of times I heard a team pretending it was an Agile team because it was doing a Stand up in the morning and deployed Jenkins to run the unit tests at every commit. But yeah, honestly I cannot blame them. It is actually difficult to understand Agile Principles and Practices when one never suffered from the very drawbacks and problems they are addressing.

I myself am not an agilist. Agility is not a passion, neither something that thrills me nor something that I love studying in my free time. Agility is to me simply a necessity. I discovered and applied Agile Principles and practices out of necessity and urgency, to address specific issues and problems I was facing with the way my teams were developing software.

The latest problem I focused on was Planning. Waterfall and RUP focus a lot on planning and are often mentioned to be superior to Agile methods when it comes to forecasting and planning.
I believe that this is true when Agility is embraced only incompletely. As a matter of fact, I believe that Agility leads to much better and much more reliable forecasts than traditional methods mostly because:

• With Agility, it becomes easy to update and adapt Planning and forecasts to always match the evolving reality and the changes in direction and priority.
• When embracing agility as a whole, the tools put in the hands of Managers and Executive are first much simpler and second more accurate than traditional planning tools.

In this article, I intend to present the fundamentals, the roles, the processes, the rituals and the values that I believe a team would need to embrace to achieve success down the line in Agile Software Development Management - Product Management, Team Management and Project Management - with the ultimate goal of making planning and forecasting as simple and efficient as it can be.
All of this is a reflection of the tools, principles and practices we have embraced or are introducing in my current company.

After almost two years as Head of R&D in my current company, I believe I succeeded in bringing Agility to Software Development here by mixing what I think makes most sense out of eXtreme Programing, Scrum, Kanban, DevOps practices, Lean Startup practices, etc.
I am strong advocate of Agility at every level and all the related practices as a whole, with a clear understanding of what can be their benefits. Leveraging on the initial practices already in place to transform the development team here into a state of the art Agile team has been - and still is - one of my most important initial objectives.
I gave myself two years initially to bring this transformation to the Software Development here. After 18 months, I believe we're almost at the end of the road and its a good time to take a step back and analyze the situation, trying to clarify what we do, how we do it, and more importantly why we do it.

As a matter of fact, we are working in a full Agile way in the Software Development Team here and we are having not only quite a great success with it but also a lot of pleasure.
I want to share here our development methodology, the philosophy and concepts behind it, the practices we have put in place as well as the tools we are using in a detailed and precise way.
I hope and believe our lessons learned can benefit others.
As a sidenote, and to be perfectly honest, while we may not be 100% already there in regards to some of the things I am presenting in this article, at least we have identified the gap and we're moving forward. At the end of the day, this is what matters the most to me.

This article presents all the concepts and practices regarding Agile Software Development that we have put (or are putting) in place in my current company and gives our secrete recipe which makes us successful, with both a great productivity / short lead time on one side and great pleasure and efficiency in our every day activities on the other side.