Warning and introduction

This post is a tad more technical than my usual ones. In the end, it's still about processes and even if you read this as a business person, understanding how development and developers work is still very important - and I'll try to keep this easy to follow.

If you know what git, pull requests and trunk based development are, you can jump directly to the next section.

Trunk Based What?

I've been reading about Trunk Based Development for a while (hi Thierry!), but never really applied it until recently. For those (developers or not) unfamiliar with this practice, a small explanation:

Development teams work using "Version Control" tools (usually "git") - those allow for all changes in the code to stay accessible for all eternity. In other words, any member of the team can at any time check any change done since the very first version of the system. This allow to "rollback" a change if needed among other features.

Developer use a local version of the software that works on their own machine. Once satisfied with their work, they can "push" their code to a central server (something like GitHub or Gitlab).

The open source world has initiated a practice called "Pull Request (or "branch") based development". In this model, every developer can work on a specific version of the software on their own, without impacting the work of the others. This is called a "branch". Branches can be pushed without impacting anyone else (the central server can have several branches open at the same time), so it allows this "parallel" work without requiring the code to stay on the developer's machine.

When the developer is satisfied with their work, they ask a colleague to check their code and "merge" it with the current version - this is called a Pull Request.

It brings a good slew of benefits, especially in "low trust" environments (if you maintain an open source project used by million - think WordPress - you don't want Joe from the basement that you never heard about before to make a huge change without a proper review) - generally a specific timing for review before going to production - the idea there is that "more eyes" tend to help fixing errors before they can even impact a user.

It also brings issues - the main being that it's "one more step" in the process - Sarah may be done with her new feature, but until Tom has time to review her code and approve it (possibly with several back and forth), it won't go to production - and code that is not in production does not deliver value to anyone - it's "work in progress" - something we try to minimize. It's the same problem in the end as the one I outlined in Why I hire Full Stack developers and why you should too - the more steps you have between a need and a feature in production, the more complicated it is to produce value on a regular pace.

Enter Trunk Based Development

Trunk Based Development moves away from that branch/request model. Every developer works on the "trunk" (the "main" branch). Anytime a developer "push" code, it is deployed to production. In other words, anytime any developer push an update, a script take the code and execute all required step to get it on the production machine - which can be several time a day per developer.

Trunk Based Development at OpenCompass

I've been working for OpenCompass for a couple of years - basically we compute a set of indicators about agroecological practices.

Team technical context

• Two senior developers, both part time (40%)
• Django + React web application
• A decent test suite on the backend (70% coverage, around one minute to run) - nothing on the front end
• Automated Deployment via GitHub Actions
• We get a broad product direction but have a lot of autonomy in how we deliver it (that includes UX and others)

So a rather classical situation, but on a very small, autonomous and senior team.

Results: two months in

We started working "the usual way" doing regular pull requests, reviews and merge.

Around two months ago, we decided to try to move to Trunk Based Development or to be more precise to no more require branches and pull request - so if one of us start something complex or experimental, we can still use those, we just don't mandate them anymore.

The only update we did to our setup is to run the test on push, and only deploy if the tests are successful. In other words, if the tests fail, the push still work, but it's not deployed (and the agreement when that happen is to fix it immediately)

What I can see:

• 400% more updates: We're clearly pushing more/faster updates. Going from around 15 deploy per month (4 per week) to around 15 per week - something like a 4x increase
• Shit happen: We're clearly breaking the production - but I'm not sure we do this more than before (code review in my experience is not that good at catching actual bugs).
• Fast recovery time: When we break, we generally get it back within 15 minutes.

We're not a telecom company nor a bank - while having anything broken is obviously bad, it's not killing our business as long as it recover fast enough.

Feelings: two months in

As a "subjective" benefit, I feel the general developer experience to be just smoother - anything that's on my laptop will be in production during the day, or the next morning at worse. I was always for small PRs/tasks, but knowing that I don't have to wait for a review to do the next part make those little changes much easier to do.

I realize now that even if I was able to split the 2 day task into 8 2-hours one, I would not have dare to inflict 8 PRs on my team in just two days - so I was limiting myself in that way, without really realizing it.

What's next?

We agreed that the practice is there to stay - I don't want to go back.

• I'd like to know "sooner" if something is broken: We have a sentry, so actual exception are going to be caught - the problem as usual is the front end.
• Better tests would be nice, but I'm not sure what this would mean: I'm no fan of UI testing (cypress and others are costly and brittle - a bad combination) - but I'll start a basic sanity suite on the frontend anyway.
• A "1 minute" test suite feels already quite long to me: Not sure I can get this much faster, especially as the application grow.
• Delayed reviews: In order to keep the "collective ownership", we decided to setup "package code reviews" every two weeks - one of us come with a feature/package area they worked on and give a tour to the other. We'll see how that work

Generally speaking, this is a nice demonstration of the "short cycles" are better - including in ways I did not expect.

Do try this at home - but mind the context

This works really well for us - but it depends on a good amount of context:

• Small team
• High seniority
• High autonomy
• Tolerance for mistakes (including in production)
• High trust environment

The point here is not that it can only work under this context, but that the practice (and it's success or failure) is at least heavily dependent on it.

Opinions? Let me know on LinkedIn!