| Scrum smells, pt. 3: Panic-driven bug management | | https://www.mobileit.cz/Blog/Pages/scrum-smells-3.aspx | Scrum smells, pt. 3: Panic-driven bug management | <p>Bugs create a special atmosphere. They often cause a lot of unrest or outright panic. But does it
have to be that way?</p><p>Nearly every developer out there has come across the following scenario: The development team is
working on the sprint backlog when suddenly the users report an incident. The marketing manager
comes in and puts pressure on the development team or their product owner to urgently fix the
bug. The team feels guilty so some of the developers stop working on whatever they've been doing
and focus on fixing the bug. They eventually succeed, and now the testers shift their focus as
well to verify the fix as soon as possible, so the developers can release a hotfix. The hotfix
is deployed, sprint passes by, and the originally planned sprint backlog is only half-done.
Everyone is stressed out.</p><p>A similar situation is often created by a product owner: He finds a defect in functionality,
created two sprints ago, but demands an immediate repair.</p><p>Is this all really necessary? Sure, some issues have a great impact on the product or service,
and then this approach might be justifiable, but rather often this kind of urgent defect
whacking is a process that is more emotional than rational. So how to treat bugs
systematically?</p><h2>What are bugs and bug fixes?</h2><p>A defect, incident, or simply a “bug” is effectively any deviation of the existing product from
its backlog. Any behavior that is different from the one agreed upon between the dev team and a
product owner can be called a bug. Bugs aren’t only defects in the conventional meaning (e.g.,
crashes or computational errors); a technically correct behavior in conflict with a boundary set
by a user story can also be considered a defect.</p><p>Some bugs are related to the product increment being implemented in the current sprint. Other
bugs are found retrospectively: They are related to the user stories developed in past sprints.
These fall into two categories:</p><ol><li>Regressions: When a subsequent development broke a formerly functional part of the code.
</li><li>Overlooked bugs: They were always there, but no one had noticed.</li></ol><p>Conversely, a bug fix is something that adds value to the current product by lowering the
above-mentioned deviation. It requires a certain amount of effort and it raises the value of the
present product. At the end of the day, a bug is just another unit of work, and we can evaluate
its cost/benefit ratio. It is the same as any other backlog item.</p><h2>A bit of psychology</h2><p>Scrum teams and stakeholders tend to approach both defect categories differently. They also treat
them differently than the “regular” backlog items.</p><p>In my experience, there are two important psychological factors influencing the irrational
treatment of defects.</p><p>First of all, there's often a feeling of guilt when a developer is confronted with a bug. The
natural response of most people is to try to fix the error as soon as possible so that they feel
they are doing a good job. Developers naturally want to get rid of such debts.</p><p>Another factor is how people perceive gains and losses. People are evolutionarily averse to
losses because the ability to obtain and preserve resources has always been key to survival.
There have been studies concluding that on average, people perceive a loss four times as
intensely compared to a gain of the same objective value: If you lose 5 dollars, it is four
times as painful compared to the gratification of finding 5 dollars lying on the ground. You
need to find 20 dollars to have a comparable intensity of feeling as when you lose the mentioned
5. The bug/defect/incident is perceived as a loss for the team's product, especially if it's a
regression. A small bug can therefore be perceived as much more important than a newly delivered
valuable feature.</p><p>Don't get me wrong—I am not saying that bugs are not worth fixing or that they don't require any
attention. That is obviously not true. One of the key principles of scrum is to deliver a
functional, <em>potentially releasable</em> product increment in every sprint. That means that a
high
development quality is fundamental and teams should always aim at developing a debt-free
product. Nonetheless, bugs will always have to be dealt with.</p><h2>Bugs caused by newly added code</h2><p>When working on a sprint backlog, the team needs to set up a system to validate the increment
they’ve just developed. The goal is to make sure that at the end of the sprint, a feature is
free of debt, and can be potentially released. Our experience shows that during a sprint backlog
development, the team should focus on removing any bugs related to the newly developed features
as quickly as possible in order to keep the feedback/verification loop as short as possible.
This approach maximizes the probability that a newly developed user story is done by the end of
the sprint and that it is potentially releasable.</p><p>Sometimes there are just too many bugs and it becomes clear that not everything planned in the
sprint backlog can be realistically achieved. The daily scrum is the opportunity to point this
out. The development team and the product owner together can then concentrate their efforts on a
smaller amount of in-progress user stories (and related bugs). It is always better to make one
user story done by the end of the sprint than to have ten stories halfway finished. Of course
all bugs should be recorded transparently in the backlog.</p><p>Remember, a user story is an explanation of the user's need that the product tackles, together
with a general boundary within which the developed solution must lie. A common pitfall is that
the product owner decides on the exact way for developing a (e.g., defines the exact UI or
technical workflow) and insists on it, even though it is just her personal preference. This
approach not only reduces the development team's options to come up with the most effective
solution but also inevitably increases the probability of a deviation, thus increasing the
number of bugs as well.</p><h2>Regressions and bugs related to past development</h2><p>I think it's important to treat bugs (or rather their fixes) introduced before the current sprint
as regular backlog items and prioritize them accordingly. Whenever an incident or regression is
discovered, it must go into the backlog and decisions need to be made: What will be the benefit
of that particular bug fix compared to other backlog items we can work on? Has the bug been
introduced just now or have the users already lived with it for some time and we just did not
know it? Do we know the root cause and are we able to estimate the cost needed to fix it? If
not, how much effort is worth putting into that particular bug fix, so that the cost/benefit
ratio is still on par with other items on the top of the backlog?</p><p>By following this approach, other backlog items will often be prioritized over the bug fix, which
is perfectly fine. Or the impact of the bug might be so negligible that it's not worth keeping
it in the backlog at all. One of the main scrum principles is to always invest the team's
capacity in stuff that has the best return on invested time/costs. When the complexity of a fix
is unknown, we have good experience with putting a limit on the invested capacity. For instance,
we said that at the present moment, this particular bug fix is worth investing 5 story points
for us. If the developers managed to fix the issue, great. If not, it was abandoned and
re-prioritized with this new knowledge. By doing this, we mitigated the situations when
developers dwell on a single bug for weeks, not being able to fix it.</p><p>I think keeping a bug-log greatly hinders transparency, and it’s a sign that a product owner
gives up on making decisions that really matter and refuses to admit the reality.</p><h2>Final words</h2><p>I believe all backlog items should be approached equally. A bug fix brings value in a similar way
as a new functionality does. By keeping bug fixes and new features in one common backlog and
constantly questioning their cost/benefit ratio, we can keep the team going forward, and ensure
that critical bugs don't fall through.</p> | | #scrum;#agile;#project-management;#release-management | | |
| Scrum smells, pt. 1: Irregular releases—overview | | https://www.mobileit.cz/Blog/Pages/scrum-smells-1.aspx | Scrum smells, pt. 1: Irregular releases—overview | <p>There's been a lot written about what agile development and scrum should do for a team. After a team has been maturing for some time, it's easy to become blind and
insensitive to phenomena that hinder its effectiveness and restrict its potential. Let's
call them “scrum smells”.</p><p>Some teams are just starting with scrum adoption while others are moderately matured or even
experienced in it. Each level brings with it its own smells and scents. This series will focus
on both basic and advanced challenges that scrum teams commonly encounter. Today, we’ll talk
about the problem of irregular releases.</p><h2>Inability to regularly provide releasable versions</h2><p>One of the basic scrum principles is being able to provide a potentially releasable product
increment as a result of each sprint's effort. I personally believe this is one of the most
valuable and underrated benefits of the whole scrum world. Scrum says that at the end of each
sprint, the development team should produce a piece of software which the product owner can then immediately release and put to productive us, if he so chooses. That means that the software needs
to be working, regression-free and without any work-in-progress stuff. Everyone on the team
needs to have the feeling that there is no debt—be it technical or functional.</p><p>In real life, however, the production builds are provided quite randomly. Every scrum team has gone through this. At the end of the sprint the team would like to provide a final version, but there
are incomplete features or regressions severe enough that this version can not be put out for
productive use. So in the subsequent sprint the team attempts to fix the bugs, but continues
developing new features in parallel, which introduces a new source of potential problems and
uncertainty. Days and sprints go by and there is always something to be fixed in the current
version.</p><p>This vicious cycle pressures the product owner to finally release
everything that's been implemented so far. There's the always present “almost there” syndrome,
the eluding moment of problem-free release. The product owner gets nervous and it's tempting to
try to sneak one more “important” thing to the sprint, because god knows when the next release will take place. Long time-to-market is a reality. So-called hardening or
stabilisation sprints occur, where teams just try to fix the product into a usable state.</p><p>Aside from the inevitable demotivation and pressure that arise, this also causes problems with planning
and transparency. You never know where you truly are until you have no work left on the already
developed backlog items.</p><h2>Preparing the ground</h2><p>So how to increase the chance of regular end-of-the-sprint potentially releasable versions
actually happening? This is partially about a shift in mindset. Being able to provide a working,
debt-free, done software increment must be a top priority for the team during the sprint and all
activities need to focus on this one goal.</p><p>It all begins with the backlog refinement. Backlog items must be broken down to pieces as small
as possible in order to give the team a high degree of maneuverability during planning. Oftentimes
creating really atomic user stories is necessary—that means stripping the user story to the very
core of the user's needs and solving it with the most straightforward approach. All the rest
just becomes another backlog item, prioritised independently. Keeping the items too big or just
being too optimistic about keeping some “easy to do” nice-to-have things attached to the core
user story's essence is just naivety, frequently combined with a degree of convenience.</p><p>Then, at sprint planning, the team creates a strategy to steer the risk of discovering a severe
problem shortly before the end of sprint with too little time to actually solve it. It helps to
start the sprint with the riskiest features first and strive to start testing even separate
parts of them as early as possible. This way there's a greater understanding of how much work
there is really left to be done. Low risk items (like simple text changes or UI adjustments) can
be left for later into the sprint.</p><p>The development team must not plan too much work, hoping that this time “we will manage”. The
team must, based on past experience, realistically anticipate extra unexpected work even on
“safe” backlog items.</p><p>And of course there is the well-known Definition of Done. Each team member must understand what
it takes for an item to be considered done and everyone must understand it in the same way. What
is on the DoD checklist? Well, that depends on the particular team, product and technologies
being used. But if a team agrees, that DoD of each item consists of code to be written, unit tests or
automated tests, documentation, code review, end-to-end test and anything else needed. Nobody
can claim an item done until all this work has been done. This helps to create a common standard
for quality and for complexity estimates. Strictly adhering to it reduces the risk of debt
accumulation. Missing or unused DoD creates a fertile ground for debts and makes planning almost
impossible.</p><h2>Day-to-day activities and decisions</h2><p>Frequent end-to-end testing during a sprint is absolutely vital. It is a dangerous illusion to create a
single version one day before the sprint's end, test it and expect that all is going to be fine.
That's not planning, that's gambling.</p><p>To enable this, new builds need to be created as often as possible, even several times a day.
CI/CD pipelines are a must. TDD helps a lot. Automated regression tests are a must. Basically
automating as much of the manual hassle as possible removes the reasons why teams usually avoid
making builds regularly. This investment into automation is usually well worth it in the long
run.</p><p>Adding feature switches (or flags) help. If it's evident that the team is not
going to be able finish a certain backlog item (i.e. fulfill the DoD), it is “switched off” and
it doesn’t interfere with the rest of the software.</p><p>The team must also understand that one done and delivered backlog item is worth far more than ten
items in progress. The daily scrum is an ideal time for the team to evaluate the sprint
backlog's progress and mutually collaborate on pushing in-progress items closer to a “done” state
as quickly as possible. Team needs to learn to constantly re-evaluate where everyone's present effort lies and decide if there is something more valuable to concentrate on.
All sprint backlog items are the whole team's job, not individual assignments. It is all about
constant re-evaluation as to where to invest the day's efforts in order to maximise the chance of
achieving debt-free software at the sprint's end.</p><p>When a sprint backlog item gets risky and it seems there's not enough time left in the sprint,
the team needs to decide whether it wants to invest more energy in it (to increase the chance of
getting it done, e.g. putting more developers onto it) or to stop working on it altogether and
focus on another item that has a real chance of getting done. Decisions to drop a
sprint backlog item must be consulted with the product owner.</p><p>For more about strategies to achieve regular releases, please check out the follow up “Scrum
Smells pt. 2” post for more details.</p> | | #scrum;#agile;#project-management;#release-management | | |
| Scrum smells, pt. 2: Irregular releases—strategies | | https://www.mobileit.cz/Blog/Pages/scrum-smells-2.aspx | Scrum smells, pt. 2: Irregular releases—strategies | <p><a href="/Blog/Pages/scrum-smells-1.aspx">Last time</a> we talked about irregular releases and
why they happen. So without
further ado, let's pick up where we left off.</p><h2>Sprint strategy continued</h2><p>It's sometimes tempting to introduce code freezes of some sort, meaning a date after which it
isn't allowed to introduce any new code to the sprint's version (only defect fixes are allowed).
While in general this idea is not bad, setting a fixed date reduces a team's maneuverability and
often hinders effectiveness. Fixed code freezes are a waterfall-ish attempt to create a
feeling of control. Needless to say, this is often not an idea of the development team itself,
but rather some company process. Code freezes rob the team of one of the biggest values of
agile—adaptability and room to find creative ways to become more effective.</p><p>Risk in items must be evaluated individually for each sprint backlog (based on size, complexity, technical sophistication etc.). The team must then decide on the latest reasonable time frame for starting final testing. That means, for instance, that for a big item
that time frame might be around the middle of the sprint, whereas for a tiny change it is half a day
before the end of the sprint will suffice. After that moment it is risky to
include this item in the sprint's product increment. Again, the team must evaluate individually the amount of risk involved in additional work is after they finish the implementation of a story and start
final testing—and plan for this accordingly. Attempting to set a one-size-fits-all code freeze
date will usually lead to the date being too early for items of small complexity and way too
late for more complex items.</p><h2>The beauty of frequent releases</h2><p>At the end of the sprint, the product owner decides if he wants to actually release the sprint's
version or not. It may not be necessary for many reasons, but he always needs to have the option
to do so. It is generally advantageous to release as often as possible. Releasing frequent small
batches of features induces far less risk of something major going wrong and makes a potential
roll-back not that painful. And it's great for planning too, because there is usually no
“invisible” debt that would otherwise need to be done before a major release.</p><p>Typical symptoms of accumulating technical or business debt are so-called hardening sprints,
release sprints, bug fix sprints or whatever you wish to call them. Their goal is to remove any
debt before actually being able to produce a version that is potentially releasable. This is a
strong smell, as it indicates the inability to produce working versions regularly. It cures
symptoms, but the underlying habitual problem is still there.
It basically means that until the hardening sprint happens, the team has never truly had a
working, potentially releasable version of the product. If it had, the bugs (or debt of any
sort) would already be removed and there would exist no need for an extra time-out to fix stuff.
</p><p>In my opinion this often happens due to an unconstructive pressure put on the development team to
deliver new stuff quicker, created by some external factor. I've seen product owners and even
scrum masters push teams to plan more stuff for a sprint without any underlying justification
stemming from the team's past velocity. It creates a false sense of progress, but with a great
amount of uncertainty. It leads to overcommitments and naive planning, ignoring the fact that
there is “invisible” work that always emerges: defects found, teeny tiny tweaks, refactoring
here and there, small changes, analyses, backlog refinement activity. Ignoring this leads to the
hamster-in-a-wheel effect.</p><p>There is no reason to have so called hardening sprint(s) before release if you knowingly don't
push any debt ahead of you. How can you do any sort of release planning if you don't know how
many problems will appear when you actually attempt to create a releasable version after
many months of development? It is always more valuable (and, needless to say, stress-free) to
have a working, debt-free version after each sprint regularly (with seemingly less new
features), but knowing there are no skeletons hiding in the closet. Finding out problems as
early as possible to know how far you've come and how far you need to go, that's what agile is
about. That's the transparency everyone talks about, right?</p><h2>What about bugs?</h2><p>The Scrum team should understand that Done means that the team is reasonably convinced there is
no more work to be done on the new delivery. No to-dos. No bugs. No technical debt. No wording
that needs to be tweaked, no graphics that is just provisional. But realistically, software is
never bug-free. It can happen that a defect is discovered right before a sprint release. It's
tempting to block the whole release because of it.</p><p>There's an article coming up on managing defects in agile product development, but in short it is
always a good idea to ask yourself if the discovered bug is a regression compared to the last
productively used version. If not (meaning the bug already exists in the current live version),
there is rarely a reason why not to release an update even though it does not fix the defect.
And this discovered defect is then prioritised in the backlog in the same way as any other
item.</p><h2>It's a long way</h2><p>I'm not saying it's easy to achieve regular releases. My experience shows that it's not, but that does not mean the scrum team should give up on it.</p><p>Adopting this habit and values gives the product owner a great amount of predictability for
future planning because he knows that he will get something to release on a regular basis. It
also relieves the pressure of something getting postponed for a later date than planned, because
everyone knows that in a short time it's going to be released anyway. Things just suddenly get
much more tidy.</p> | | #scrum;#agile;#project-management;#release-management | | |
