At one of our projects, we're working in a scrum team counting around 10 people. The product has gradually been rolled out to more users and is getting traction. This is good because it provides us with a tangible metric of our product's reach and it gives us more opportunities to validate our ideas in the real world as we see how users react to them.
Some of the users are also proactively providing feedback about things that are cumbersome, hard to use, or outright buggy. So far we (with a major contribution from our product owner) have managed to gather and process the feedback in an organic way.
Over time this activity has grown to a considerable effort and has started to consume precious time and we've started to feel like things are slipping through. We need to find a more robust way to handle the user requests.
The goals we want to achieve by having a link to the users are far from unique. To name a few:
- Collect defect/incident reports
- Collect feature requests
- Help users with troubleshooting
- Have a feeling for the general user's sentiment
- Respond to and inform users
We don't want to have just a
passive way of collaboration with users (meaning only to collect input) but we'd like to be able to respond in a relevant way to their reports or questions. We want them to know what they can expect from us and likewise let them know they're heard and that their contribution is welcome.
Unsurprisingly, we discovered the majority of users' requests were repeats of the same thing. Users either did not understand a certain feature or a general expectation was misaligned with the product itself.
Many of the user issues could be resolved quite easily by answering directly and potentially providing further explanation if needed. It became obvious that we need to focus on improving UX here and there. Or bring a new feature because users are repeatedly struggling with certain actions.
But in order to draw useful conclusions from the numerous inputs and to convert them into prioritized backlog items, we needed more
human processing power and a way to somehow standardize the inputs.
This led us to introduce a combination of a few new concepts.
Introduction of bug reports
We implemented a bug report function. We wanted to keep it as simple for the user as possible. So after clicking the
report bug icon the only thing the user needs to write is an answer to the question
What did you expect to happen? and to click a confirmation button.
We attach a lot of metadata to this and the bug report is then visible in our product's back-office web. This alleviates the need to read through emails and keeps the form uniform to some degree. It also automatically collects crash logs.
So far we have decided not to integrate this to JIRA for a reason I will explain in a minute.
Introduction of support levels
The customer for whom we are building the product is a large corporation and one of their business activities is software services. That means they have a department full of people experienced in 1st level support activities. So it appeared as a logical option to collaborate with them on this as it was preferred to use in-house personnel.
This idea posed a few challenges nonetheless:
- How to define competencies?
- How to bring them to a level of knowledge necessary for stand-alone work?
- How to help them with unexpected things?
- Find a way to report problems they can't resolve on their own.
- How to keep them up-to-date with the latest fixes and features?
We decided to follow the 3-level support model. In our development process it meant the following:
Roughly 4 people from the help desk department were selected to join us part-time. Their competence within our development process is:
- Answering general users' questions.
- Resolving user issues that don't require a change of system data or the system itself.
- Passing unresolvable issues to 2nd level support.
- Collecting data about frequent bugs.
- Bringing ideas to mitigate the most frequent user complaints/questions.
For the time being, we have decided to bring one extra person who will tackle this alongside our product owner.
- Resolving incidents stemming from bad system data via our system's back-office web.
- Resolving issues that 1st level support lacks knowledge of.
- Adding new articles to the knowledge base for the 1st level support
- Passing issues to 3rd level support
This is the actual scrum team. So the job is to prioritize and implement what comes through the support pipeline.
Setting up new flows
In order to keep the new 1st and 2nd level support colleagues in the loop we invited them to our asynchronous communication channels. This way we could quite flexibly answer their questions without the need for yet another regular meeting and it seems it helped their learning process. As of now, they are already quite knowledgeable and their questions are getting more and more
advanced. We feared the number of questions could overwhelm us, but it seems we managed it and the ad-hoc communication is effective.
We also invited all of them to the sprint review meetings, so that they are aware of the current situation and know when the bugs reported by them are done. This is also a perfect place for them to express any general observations and opinions about how they perceive priorities of items reported by them compared to other backlog items.
For the most frequently occurring questions or problems, we created a simple knowledge base that the 1st level colleagues can use while troubleshooting user issues. 2nd or 3rd level support people occasionally add new articles if a question gets repeated.
We've been using JIRA as a tool for keeping our backlog since the beginning of the project. Our fear was that introduction of formalized support levels could bring large overhead, so we decided to keep the tools as simple as possible. We don't need any sophisticated helpdesk system.
Therefore we decided to create two new issue types in JIRA and right in our project space. We called them (no surprise here) a
1st level ticket and a
2nd level ticket. Those tickets got a very basic state flow:
Open → In progress → Closed + 2 more: Waiting for a reply and Reply obtained.
We decided to use the original JIRA project space for the 3 support levels. The main reason was to let everyone inspect the backlog if necessary and to check how things are progressing. The second reason was to avoid complexity in our flows.
By doing this, 1st and 2nd level support people can also comment on any existing issue in the backlog in case they have relevant additional information. In order to keep things organized we simply created a few new quick filters to view
only the 1st level tickets, the
2nd level tickets, or to
hide them and view the backlog just as it was before the 1st and 2nd level people joined in.
Both of the new issue types also received a button to conveniently convert the issue to a
higher one. Meaning that from
1st level ticket we create a
2nd level ticket with a click of a button. And from the
2nd level ticket, we create a
bug or a
user story in the same way. In addition, we created two new boards to display just the respective issues type and their current states.
Time will tell
So far this model has worked well for us. We get a feeling of safety knowing there is no negative sentiment accumulating among the users we wouldn't know about and could explode later. The test of time will tell if we need to modify the process. Many more users are expected to start using the product. As with anything in such an environment, we will certainly need to adapt eventually. We just don't yet know when. I will come back with an update after we're wiser again.