mobile it



Scrum smells, pt. 6: Unknowns and estimates smells, pt. 6: Unknowns and estimates<p>Today, I'd like to share some of the ideas and estimation approaches that helped us in past projects. The tricky part in long and short-term planning is how to predict the unknowns that will influence us in the future. As I wrote earlier, there are several things that usually come up and may not be visible in the product backlog when you are planning something.</p><h2>The unknowns</h2><p>In projects related to mobile app development, we usually encounter the following unplanned activities:</p><ul><li>Defect fixing</li><li>Backlog refinement activities</li><li>Collaboration on UI/UX design</li><li>Refactoring</li><li>New user stories</li></ul><p>Defect fixing is quite obvious and we have spoken about it already. You can't usually foresee what bugs will appear.</p><p>Backlog refinement activities include understanding the backlog items, analyzing the underlying technical and usability aspects, and making the backlog items meet the definition of ready. </p><p>The UI/UX design process is not just a simple decision about colors and shapes. The controls used and the screen layouts and flows usually have a large impact on how the application needs to be built, and we witness over and over again that a seemingly small aspect of the design idea can have a vast impact on the complexity of the actual implementation. So in order to keep the cost/benefit ratio reasonable, we have learned that it is necessary that the developers collaborate closely with the designers in order to prevent any unpleasant surprises. You can read more about this topic in <a href="/Blog/Pages/design-system-1.aspx">this blog series</a>. </p><p>Refactoring existing code and infrastructure setup is a must if we want to develop a product that will be sustainable for longer than a few weeks. It can also have the potential of making the dev team more effective.</p><p>New user stories are interesting. You invest a lot of time into the backlog refinement and it just looks perfect, everything is thought through and sorted. Fast forward two months into the future and you discover (with new knowledge from that past two months) that you need to simplify some stories while others have become obsolete, but more importantly, you realize that you need to introduce completely new features that are vital for app's meaningfulness. You couldn’t see this before you had the actual chance to play around with the features from the past couple of months and gather feedback from users, analyze the usage stats or see the economical results.</p><h2>Estimates</h2><p>Having most of the stuff in the backlog estimated for its complexity (size) is vital for any planning. But as we have all probably learned the hard way, estimates are almost always anything but precise. We, therefore, did not find any value in trying to produce exact estimate values (like 13.5 man-days of work), but we rather use the approach of relative estimation while using pseudo-Fibonacci numbers: 0, 1, 2, 3, 5, 8, 13, 20, 40, 100.</p><p>It is important to understand that these are dimensionless numbers. They are not hours, man-days, or anything similar. It is an abstract number used solely to set a benchmark and compare other items against each other.</p><p>So what does that mean? At the beginning of the project we pick an item in the backlog that seems to be of a common size and appears neither small nor big, a number between the 5-8 range. That will be our benchmark and all other stories are then compared to it. How much more difficult (or easy) is this or that item compared to our benchmark?</p><p>Over time, we usually found out that the initial benchmarks and estimates were completely off. But that is OK, it's a learning process. It is important to review the estimates after the actual development and from them. Was that user story really an 8? Were these two items as similar as we initially thought? If not, how would we estimate them now and why? That also means that from time to time it's necessary to revisit all the already estimated items in the product backlog. </p><p>It usually is not necessary to go into deep details with stuff that is several sprints ahead. As the team gains experience with the product domain, the developer's gut feelings get more relevant and precise. That means useful estimates can be done quite swiftly after the team grasps the particular feature's idea. Sure, some stuff in the backlog will be somewhat underestimated, some overestimated. But with long-term planning and predictions it usually suffices because statistically, the average usually gets quite reliable.</p><p>The outcome of all this is a backlog where every item is labelled with its size. It becomes clear what items are meaningfully defined. The development team has an idea about the technical solution (meaning that the size is reasonable) and what items are completely vague or for which the team members lack key business or technical information. Those are usually the items with estimates labels of “40”, “100”, or even “??”.</p><p>If such inestimable stories are buried in the lower parts of the backlog and the product owner does not even plan to bring them to the market for a long time from now, that's fine. But do any of these items have a high value for the product and do we want to bring it to the market soon? If that's the case, it sends a clear message to the product owner: back to the drawing board, let's completely re-think and simplify such user stories and expect that some team capacity may be needed for technical research. </p><p>So after all this hassle, the upper parts of the backlog will have numbers that you can do math with.</p><h2>Quantifying unexpected work</h2><p>The last piece of the puzzle requiring predictions and plans is to quantify how much of the unexpected stuff usually happens. Now, this might seem like a catch-22 situation - how can we predict the amount of something that we can't predict by its definition? At the beginning of the development, this is indeed impossible to solve. But as always, agile development is empirically oriented - over time we can find ways to get an idea about what is ahead based on past experience. As always, I am not preaching any universal truth. I am just sharing an experience that my colleagues and I have gathered over time and we find useful. So do we do it? </p><p>It's vital to visualize any team's work in the product and sprint backlog as transparently as possible. So it's also good to include all the stuff that are not user stories, but the team knowingly needs to put some effort into them (like the known regressions, researches, refactorings, etc.) into the backlog too. If it's possible to estimate the size upfront, let's do it. If it's not, either cap the maximum capacity to be invested or re-visit and size the item after it's been done. This is necessary in order to gather statistics. </p><p>Just to be clear - let's not mistake such unexpected work with a scope creep. I assume that we don't suffer from excessive scope creep, the unexpected work is indeed solely highly valuable and necessary work that was just not discovered upfront.</p><p>So now we have a reasonably transparent backlog, containing the originally planned stories and also the on-the-go incoming items. We have most of it labelled with sizes. In the next part of this series, we'll try to make some statistics and conclusions on top of all this. </p>#scrum;#agile;#project-management;#release-management
So you want to create a design system, pt. 1: Can designers and developers ever be friends? you want to create a design system, pt. 1: Can designers and developers ever be friends?<p>Design systems are all the rage these days. Done right, they make both UI design and implementation much easier and, more importantly, cheaper, while providing users with a consistent and pleasant UX.</p><p>Let's look at them from a mobile development perspective - how do you create a design system that is usable, efficient, and even removes the pain points of interaction between designers and developers?</p><h2>What exactly is a design system?</h2><p>A design system is a comprehensive framework for the design and implementation of digital products. It consists of reusable styles, components, standards, and guidelines that are used together to build any number of applications that are both coherent and allow for evolution.</p><p>A design system consists of two parts, described by this highly scientific equation:</p><p style="text-align:center;"><em>Design system = Design × System</em></p><p>The <em>design</em> part is all about how your product looks, behaves, and speaks to the user - in other words, the actual look'n'feel, perhaps developed in more detail and with more depth than usual. This is probably the easier part because historically our industry had quite a lot of experience with this kind of thing.</p><p>The <em>system</em> part is not to be underestimated because it is important for consistency, quick and cheap design and implementation, and the ability to freely shape the UX in the future. Achieving that level of flexibility is difficult, but it is also where design systems bring the most value.</p><p>Since the system part must be robust enough to allow for different designs to be created and then evolved in time while supporting the implementation, we arrive at the second fundamental equation:</p><p style="text-align:center;"><em>Design < System</em></p><p>That's not to say that design is less important or less valuable, of course not! The point is that a situation where you have a great design but a bad system is much worse than the other way around, especially in terms of implementation effort and overall flexibility. </p><p>Designers are better at, well, designing, while developers are probably a little bit better at creating systems. Let's look at how both sides can help each other and make their jobs easier.</p><p>I was lucky enough to recently implement a large-scale design system used in dozens of applications. Based on this experience, before we get to the nitty-gritty details of design systems specs and implementations, let me offer some initial tips for both designers and developers.</p><h2>Dear designers</h2><ul><li>You absolutely need to have first-hand experience with <em>all</em> the platforms you will be designing for. You need to immerse yourself in the platform, soak up the feeling of using it on a daily basis, and get to know its iconic apps.<br> If your main phone is an iPhone, you need to get an Android device too (and vice versa) - yes, I know, the culture shock is considerable in both cases, but this is your job after all. It doesn’t matter if you're going to design for cars or smart fridges or fighter jets or IoT-enabled dog houses, you need to physically spend some time with those things. </li><li>Be aware that different platforms have wildly different UI idioms, widget sets, features, and development costs, not forgetting different and sometimes very interesting bugs! UI programming is actually <em>very</em> hard (that’s an actual fact, not me complaining, I swear!) and what is free on one platform can be very expensive or almost impossible to do on another.<br>Even worse, there is usually no obvious, predictable relationship between what is simple and what is complex to implement. Even a small change in design from your perspective can cause a massive increase in the implementation effort. Be prepared to kill your darlings or at least postpone them until the next major version. </li></ul> <img alt="Two different buttons" src="/Blog/PublishingImages/Articles/design-system-1-01.png" data-themekey="#" /> <p style="text-align:center;"><em>On Android, one of these buttons is actually much more expensive to implement than the other.</em> </p><ul><li>If you don’t design it, someone else will - and that someone is likely to be someone <em>different</em> each time, so not only will some things in your app be out of your control, but they'll also look a little different every time.<br> There is no other option but to simply design, in sufficient detail, <em>everything</em> your design system will need. </li><li>Sooner or later your design will be transformed into code, in the best case 1:1. There is a lot of UI code in most applications and it is complex, expensive, and also very hard to test. How consistent and systematic your specs and designs are <em>directly and significantly</em> affects the cost of development. Consistency doesn't just help developers, it also helps you as well when designing real applications. </li><li>It is often prohibitively expensive to achieve a pixel-perfect implementation. I know it's hard, but sometimes achieving a <em>good enough</em> result is the sensible thing to do. Because the exact reasons for this are usually highly complicated, technical, and thus boring and uninteresting, you might need to trust your developers more or less blindly in this regard. </li></ul> <img alt="Text rendered twice" src="/Blog/PublishingImages/Articles/design-system-1-02.png" data-themekey="#" /><br> <p style="text-align:center;"><em>Same font size, same letter spacing - but the design tool and OS render it slightly differently anyway. And let's not start with inconsistencies across OS versions...</em></p><ul><li>Unless you are designing a special kind of application such as games or VR, please respect the platform and therefore its users. The goal is to strike the right balance between the platform itself, your customer's brand, and your creativity.<br>For example, people don't compare the iOS and Android versions of your app to each other - what they do is compare your app to other apps on their device. </li><li>Modern design tools are great, but they still have a lot of limitations that may not be obvious from your point of view - things like layout composition, exporting assets to different formats, or code generation look good at first glance, but in reality are often not completely usable. Please don't fall blindly for the marketing of the manufacturers of these tools and discuss this with your developers. </li></ul> <img alt="Bad UI source code" src="/Blog/PublishingImages/Articles/design-system-1-03.png" data-themekey="#" /><br> <p style="text-align:center;"><em>I won't name a specific tool here, but someone can commit this garbage to source control only over my dead cold body.</em></p><ul><li>Since you are the guardians of the source of truth of your design system, you must have a mechanism in place to communicate changes and updates to your design system. </li><li>Please talk to your developers often and be prepared to meet them halfway. Create great things and don’t forget to have fun in the process! </li></ul><h2>Dear developers</h2><ul><li>Because no one sees or understands the technical details as you do, it is your responsibility to communicate potential problems and difficulties in a simple, clear, and timely manner. Not only that, you need to be able to offer alternative solutions or trade-offs based on different criteria. </li><li>In a sense, you're also responsible for the delivery of the whole thing, so you have to be able to schedule your work on the whole application from a technical perspective so you don't end up with two perfect screens and 30 broken ones. </li><li>If you think something is missing from the design system, please let the designers know! Don't jump straight into implementation - let the designers do their job. Let them design the thing so it's consistent with the rest of the design system, and let them specify it so it's reusable. </li><li>You need to have a perfect overview of the UI of your platform, both from a development and user perspective. These APIs are huge in width and depth and contain many bugs and incompatibilities, but it's your responsibility to know most of them. </li></ul> <img alt="Android View class reference" src="/Blog/PublishingImages/Articles/design-system-1-04.png" data-themekey="#" /><br> <p style="text-align:center;"><em>54 subclasses is overly optimistic. And the whole thing is over 30.000 LOC long!</em></p><ul><li>As I said, there is a lot of UI code in a normal application and it is complex, expensive, and very hard to test, but also highly reusable. For these very reasons, you must <em>not</em> treat it as a second-class code! The usual principles of clean code apply here too - buzzwords like DRY, KISS, YAGNI, or separation of concerns are actually pretty good ideas. Hardcoding and duplicating things is a road to maintenance hell. </li><li>Since your implementation will ideally follow the designers' specifications and designs very closely in the code, the quality of input from the designers has a high chance of directly affecting the quality of your code.<br>If necessary, politely but firmly demand cleanliness and consistency in design system specifications, which, after all, must always be the only source of truth in the design system.<br>And since the implementation follows the design system specifications, it is also your responsibility to keep the implementation in sync with the design system over time. </li><li>It helps if you have a basic understanding of design principles, typography, color theory, etc. You don't have to do landscape painting in your spare time, but being able to talk to designers at least partially in the language of their domain is important. A light form of OCD that allows you to quickly spot misaligned elements or inconsistent colors comes in handy too. </li></ul> <img alt="OCD test" src="/Blog/PublishingImages/Articles/design-system-1-05.png" data-themekey="#" /><br> <div style="text-align:center;"><em>One of these things is not like the other.<br><br></em></div><ul><li>Please talk to your designers often and be prepared to meet them halfway. Create great things and don’t forget to have fun in the process! </li></ul><h2>A successful cooperation</h2><p>The above points help designers and developers to be on the same page when working together, which is key for a successful outcome. Next time, we'll get our hands dirty with specific elements of design systems such as colours, typography, layouts, and motion, bringing the <em>designs</em> and <em>systems</em> together in a scalable and maintainable way.</p>#design-system;#ui;#ux;#development;#android;#iOS
Product development: Perfection vs. minimalism development: Perfection vs. minimalism<p>Today I'd like to address one of the common phenomena which lead to cost and timeline inflation of projects. Maybe you have witnessed something similar: At the beginning of a project, the team makes a (rough) analysis of what will need to be done, how complex those individual parts are and how long (roughly) the development will take.</p><p>Say we're creating a simple email client application. The key parts we know of are the message inbox, message viewing, email writing, sending, and logging into the email account. Just a few key areas, nothing complicated. At the beginning of this project, the team reckons that spending a month on each of these areas should more than suffice.</p><p>They set off developing. After the login functionality, the product owner prioritises the development of the message inbox. The devs put the inbox together and after 3 weeks they claim it is done.</p><p>The testers and the product owner inspect it. They fill in the username and password, push the button and see a list of e-mail messages - it works! But the product owner and the key users think there is just too little information visible in the inbox's message cells and want to improve it. So what if we added two lines of text preview instead of one? Or is one better, so the user can view more messages at once? Or maybe let the user decide for himself and let's add a setting so that the user can set how many lines of message preview he wants to see.</p><p>It's also not visible which people were CC’d. So let's introduce another setting, where the user can toggle the display of CC'd people in the inbox. There is also no indication if an email has an attachment or not. So of course, while we're at it, let's put an icon there to indicate attachments. Also, some mail clients group message threads together in the inbox - that's a cool thing we'd definitely like to have in our app too! And just like that, there comes message grouping.</p><p>Let's do all of this now so that we have the inbox completely done and we don't need to get back to it later.</p><h2>The perfection pitfall</h2><p>I think you see where I'm heading to. Weeks pass by and we're still doing the inbox. After 3 months, we have a pimped-up inbox that everyone is proud of. We've taken half the time originally estimated for the whole project just for one feature area. But what value is the whole application to the actual users? Still very little. I can just view my messages. In a very fancy and configurable way, true. But I still can't open the message and reply back. There will be some sleepless nights later in the project.</p><p>Everyone wants to have things <em>perfect</em> - meaning nothing else can be added. It's a natural tendency of people and I've seen that on projects over and over again. Project managers, product owners, developers, key users, stakeholders, all of them want to have that perfect thing.</p><p>It seems logical to develop features that contextually belong together at once and make that application part perfect, like doing all inbox-related stuff at once. In my opinion, this is one of the biggest project traps - it leads to scope creep and the actual project goal drifts off.</p><h2>Minimum viable product</h2><p>Is there a better approach? I believe there is, but it requires discipline. Firstly, we need to cut the application by user stories (end-to-end use cases), which bring actual value to users. Is a stand-alone log-in form worth something to the user, if the app then does nothing else? No. Is a Reply button worth something, if you can't send the message afterward? No.</p><p>We need to build something that is useful and usable, not a bunch of independent technical parts. Only then can we also verify the product's function. We are building apps for the end-users, so we need to assess the application's quality from their point of view. In other words, we need to cut the items into the product backlog vertically - across all technical layers, not horizontally. In such a way, it incorporates all the necessary technical aspects to finish a use case.</p><p>Secondly, we need to know why we're building the product. Maybe you've heard of the <em>minimum viable product</em>> approach. An MVP is a product with the least functionality possible, but fully usable, useful, and enabling gathering feedback from users as soon as possible. Put another way, you can't remove anything more from the product while still keeping the product useful. We're doing the bare minimum to solve the user's main and most basic need.</p><h2>Key questions to ask</h2><p>The team (not just the product owner) must understand from the very beginning the answers to the following two questions:</p><ol><li>What is the problem we're trying to solve? What is the most basic nature of the problem we're sorting out for the user? </li><li>Who are the people using the application?</li></ol><p>With this in mind, every single backlog item can then be critically evaluated:</p><ol><li>Is this item/function solving the core user's problem?</li><li>How can we simplify it and still solve the core problem?</li><li>Isn't there any backlog item more important, without which the app does not solve the core problem and is therefore meaningless? </li></ol><p>This approach is basically about figuring out what else is there that you <em>do not need to do now</em> and can leave for later. And doing that in a brutally honest way.</p><h2>Mailing example</h2><p>So what could this look like with our e-mail client application? Let's say the target user is a person who would use the app for his personal email account, not a power-user or a corporate user. The core need of such a user would be basically to be able to view incoming emails, read them, and write new ones.</p><p>Our aim then should be to develop a trivial application doing just the necessary things to finish these use cases and to do it as quickly as possible. Does the user need rich text formatting to send a mail? Definitely not. Does he need marking messages as read? It's a useful feature, but he still can perform vital use cases without it. Does he need to delete a message? Sure it's good to keep the inbox organised, but still - even though not comfortably, he can live without it.</p><p>I am not saying you won't ever add features that are not vital. But the initial aim should be to develop a <em>bare minimum product</em> that does key functions, see how much time we needed, how much did it cost, how many resources do we have left, and then decide on what to do next. With this approach, we increase the chance that before our budget or money runs out, we have at least something useful that we potentially can put to market. This is how we can approach the project risks by managing scope very actively.</p><p>This is contrary to the common situation, where we have certain areas of the product working, including nice-to-have features, but we're still lacking coverage of other key use cases even in a very minimal way. That's bad, because now the product can't be used at all, and resources spent are sunk unless we invest more to finish the critical stuff. It is worthless at this stage.</p><p>Imagine you need to present both variants to the project sponsor after running out of resources. Will it be easier to say that the application is essentially working, but there are not enough resources to make it more user-friendly? Or that you have a cool inbox, but the user still can't send an email?</p><p>So once we have the MVP, a good idea is to give it to a group of users to gather feedback from them as soon as possible. If we still have some project resources left, let's use them to make the product better, like developing marking messages as read.</p><h2>On the right way to good quality product</h2><p>I hope you did not get the impression that I am suggesting building a product with a poor user experience. That is not true. The basic idea I am presenting is to get the most out of the resources we have. Let's act as if everything could go wrong and our resources were extremely scarce.</p><p>And if we build our MVP and see we still have enough time and money, of course, let's add more and more cool features. But you'll be adding these features in peace knowing that you have no debt; that if you stopped developing at any moment from now, you made the product most relevant for the users as you potentially could, with the resources you had; that everything in the backlog is less important than what the product already contains.</p><p>Does this guarantee you won't ever miss a project deadline or budget? Not at all. But it gives you a bigger chance it won't happen. You achieved the best possible return on investment given the circumstances.</p>#project-management;#agile;#development
Building a chatbot, pt. 2: Conversations a chatbot, pt. 2: Conversations<p>​​​​Everybody has met or talked to a person that had a script to follow. Insurance salesmen, dealers, receptionists… All these people were given a script to follow, a checklist to go through, or just a simple answer to give when asked. Not unlike a chatbot.</p><p>In the previous article, I described a chatbot that we built for making reservations. Not much was said about the inner works or about its potential. There are indeed multiple topics to cover. In this article, I would like to talk a bit about conversations and conversation flow.</p><h2>AI?</h2><p>Our application is designed to fulfill users’ requests. A user usually has a general idea about what he or she wants to achieve. But even the most knowledgeable user can’t possibly know every item and variations that the application needs to achieve a goal. Even if the user is not sure what the right way to ask is, the chatbot should be able to guide the user and in the end, find exactly what the user desires, and then fulfill his request. For each request, the bot should have a list of instructions to follow - information that the user has to provide so that the application is able to execute the user's command.</p><h2>Illusion of choice</h2><p>To provide a good user experience, we decided to leave the first step to the user. The user’s first sentence gives us a direction to follow and an idea of the user’s intent, but actually, this is the last time a user is in command of the conversation flow. From this point on, every step of the conversation is directed by the application. The bot knows what checklist it has to follow to fulfill the request. Even if the user is not specific or not sure what to say, the bot should be able to get this information from the user with properly constructed questions.</p><p>Every conversation in our application is directed by a set of checklists. These checklists can be simple, even just one item, or they can spawn other checklists, and even create cyclic conversations. What exactly we are able to create with these types of checklists will be described in a separate blog. For now, let’s focus on the following example.</p><h2>From a conversation to a checklist</h2><p>Let’s meet our example user: Joe. Joe is working an office job and has multiple meetings per day. For those, he needs to schedule reservations in a call booth or a meeting room with his colleagues. At the start of the week, he knows about two meetings that will take place. One today (for the sake of this example let’s assume it is Monday) and one tomorrow. Interaction with the bot will then look like this:</p><p style="margin-left:40px;margin-top:20px;margin-bottom:30px;"> <strong style="color:#381457;">User</strong> : <em>Book me a room for today from 11:30 to 12:30</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>Alright! Do you want to book a specific meeting room or should I just book you one that is free?</em><br><strong style="color:#ffed00;">Options</strong> : <em>Any meeting room, Specific meeting room</em><br> </p><p>For today's meeting Joe has a remote call with colleagues from out of town, therefore any available room will do the trick. He will select “any meeting room” from the options given. If there is an available room in the system, Joe will get one. Like this:</p><p style="margin-left:40px;margin-top:20px;margin-bottom:30px;"> <strong style="color:#381457;">User</strong> : <em>Book me a room for today from 11:30 to 12:30</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>Alright! Do you want to book a specific meeting room or should I just book you one that is free?</em><br><strong style="color:#381457;">User</strong> : <em>Any meeting room</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>I have booked room R.332 for today from 11:30 to 12:30 for you</em><br> </p><p>And that's it. Joe has booked a room for a given time. From a user's point of view this is quite easy. Using our mobile applications, Joe can even use voice input!. This is all great, but how does it work?</p><p>To make a reservation, the chatbot needs to pass a checklist. For the example above, where the user is making a room reservation, the checklist looks like this:</p><ul><li>Intent - make a booking</li><li>What (seat, room, call booth)</li><li>Where (any available room, or does the user want a specific room?)</li><li>When (date and time, or duration of the meeting)</li></ul><p>Joe knew most of these and intuitively created a sentence that matched his wishes best. What was not explicit in the first sentence, the bot asked for in the follow-up question. When asking follow-up questions, chatbot usually gives users multiple options to select from. These suggestions give the user an idea of how to respond and also what kind of information the bot is expecting. In the same answer, the user can also write additional information.</p><p>Now let's say that Joe needs a specific room for a meeting that takes place on Wednesday. He needs a big interactive board, a projector and a room for 10 people where he will introduce new products to customers. This room is quite busy so he needs to find a spot for his meeting. The conversation could look like this one:</p><p style="margin-left:40px;margin-top:20px;margin-bottom:30px;"> <strong style="color:#381457;">User</strong> : <em>When is room R.121 available?</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>For which date(s) do you want to see the room's availability?</em><br><strong style="color:#381457;">User</strong> : <em>Wednesday</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>How long will you need the room for?</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>I will find you times, where there is enough space for your meeting. For example: 1 hour 30 minutes or just like 15 minutes</em><br><strong style="color:#381457;">User</strong> : <em>2 hours</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>Alright.</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>The room R.121 is available on wednesday from 9:00 to 13:59.</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>Do you want to book it?</em><br><strong style="color:#381457;">User</strong> : <em>Yes</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>Ok, can you specify the time for me please?</em><br><strong style="color:#381457;">User</strong> : <em>10:00</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>I have booked room R.121 on wednesday from 10:00 to 12:00 for you.</em><br> </p><p>That's a slightly different conversation, right? But in the end, Joe will have his reservation ready. So what does the checklist on the bot’s side look like for this example?</p><ul><li>Intent - check availability</li><li>What - specific place to check</li><li>When - time-frame to check</li><li>How long - meeting duration</li><li>To book it or not</li></ul><p>Up to this point Joe has only checked for available time slots. If he wants to make a booking now, he can. For booking, the chatbot has a different checklist. That one was previously listed, but here it is again for completion:</p><ul><li>Intent - make a booking</li><li>What (seat, room, call booth)</li><li>Where (any available room, or does the user want a specific room?)</li><li>When (date and time, or duration of the meeting)</li></ul><p>Most of these parameters are already checked for the booking. Joe has already told the bot to look for a specific room. He’s even given it a date so the only thing missing is the time. Joe gave the bot a meeting duration when he was checking for room availability. Now if Joe selects only the start of the meeting, the application already knows the duration and so it will make the booking last 2 hours.</p><p>But wait! That’s not all. Up to this point, all user’s requests had no problems. Let's say that our Joe has had a long vacation, he is back in the office again, but lost track of dates:</p><p style="margin-left:40px;margin-top:20px;margin-bottom:30px;"> ​ <strong style="color:#381457;">User</strong> : <em>Book</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>Do you want me to book you a seat or a meeting room?</em><br><strong style="color:#381457;">User</strong> : <em>Meeting room</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>Alright do you want to book a specific meeting room or should I just book you one that is free?</em><br><strong style="color:#381457;">User</strong> : <em>What is available?</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>For which time frame(s) do you want to see available rooms?</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>You can tell me any time you want or you can use one of shortcuts bellow.</em><br><strong style="color:#381457;">User</strong> : <em>20.3.2008</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>Unfortunately you cannot book or check seat availability in the past.</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>Please tell me a time or a time range you want to see available rooms for.</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>For example: Today, Tomorrow, Thursday, 24.10. or something like next week or from 10:30 to 11:15</em><br><strong style="color:#381457;">User</strong> : <em>Ok what about tomorrow?</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>Here is a list of available meeting rooms in your preferred area for tomorrow:</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>1. R.121</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>1. R.122</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>1. R.123</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>1. R.124</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>1. R.125</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>These are the first ones I found. But there are some more.</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>Do you want to have some of these? You can also tell me, which specific one do you want to have</em><br><strong style="color:#381457;">User</strong> : <em>Yes</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>Alright</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>What room would that be?</em><br><strong style="color:#381457;">User</strong> : <em>R.124</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>Can you specify the time for me please?</em><br><strong style="color:#381457;">User</strong> : <em>From 10:00 just for a quick call</em><br><strong style="color:#9d9d9d;">Bot</strong> : <em>I have book room R.124 for tomorrow from 10:00 to 10:30 for you.</em><br> </p><p>Even in case of a longer conversation and with an invalid date, the bot was able to provide Joe with a reservation.</p><p>Again, I will show you the checklist that the chatbot had to go through:</p><ul><li>Intent - make a booking</li><li>What - meeting room</li><li>Specification:<br> <ul><li>Check available meeting rooms</li><li>When - with validation</li><li>Want to book one?</li></ul></li><li>Specific room selection</li><li>When - with specific time selection</li></ul><h2>Think in checklists</h2><p>The main point of these examples is to introduce the checklist philosophy. Every conversation that we have in the system is designed as a checklist with multiple options. It's not a simple checklist where you just confirm an item and move on to the next one. Some of the items can be optional, however at least one has to be mandatory. Each item on the checklist can have its own checklists or policies. There can be policies that end the conversation on the spot or point to another checklist where the conversation continues. Some of the policies and transitions can be seen in the previous examples.</p><p>For example, for time specification we want only the current day or a day in the future, but not in the very distant future. For room bookings, only single-day reservations are allowed. Furthermore, when specifying a room, we have to select from a place that exists in the system database.</p><h2>Conclusion</h2><p>Conversations with our chatbot are based on checklists with predefined policies to follow. The user is given an illusion of choice with a first request that can be created any way the user desires, and afterward the control of conversation flow is entirely coded in the chatbot. From a developer’s point of view, that gives us an easy way to test these conversations, and develop each branch of conversation independently.</p><p>From the user's point of view, the bot asks for missing pieces of information. Thanks to the firm grip on the conversation, the user should not end up in a situation where he doesn't know what to answer or how to reach a certain goal.</p><p>So is it artificial intelligence? No. It's a state machine that follows pre-set lines of conversation. However, the conversation can be very complex, and when it is complex enough, users might come to the conclusion that this is what AI looks like.</p> ​<br>#chatbot;#ai;#neural-network
Scrum smells, pt. 5: Planning fallacies smells, pt. 5: Planning fallacies<p>As the scrum godfathers said, scrum is a lightweight framework used to deal with complex problems in a changing environment. Whether you use it for continuous product development or in a project-oriented mode, stakeholders always demand timelines, cost predictions, roadmaps, and other prophecies of this sort. It is perfectly understandable and justifiable - in the end, the project or product development is there to bring value to them. And financial profit is certainly one of these values.</p><p>Many of us know how painful the inevitable questions about delivery forecasts can be. When will this feature be released? How long will it take you to develop this bunch of items? Will this be ready by Christmas? We would, of course, like to answer them in the most honest way: "I don't have a clue". But that rarely helps, because even though it is perfectly true, it is not very useful and does not help the management very much. For them, approving a project development based on such information would be writing a blank check.</p><p>I've seen several ways in which people approach such situations. Some just give blind promises and hope for the best, while feeling a bit nervous in the back of their minds. Others go into all the nitty-gritty details of all the required backlog items, trying to analyze them perfectly and then give a very definitive and exact answer, while feeling quite optimistic and confident that they have taken everything into account. Some people also add a bottom line "...if things go as planned".</p><h2>If things go as planned</h2><p>Well, our experience shows that all these approaches usually generate more problems than benefits because the impact of that innocent appendix "...if things go as planned" proves to be massive and makes the original plan fall far from reality. It actually stems from the very definition of the words project and process. A process is a set of actions, which are taken to achieve an expected result, and this set is meant to be repeated on demand. On the other hand, a project is a temporary undertaking that aims to deliver a unique outcome or product. While the process is meant to be triggered as a routine and its variables are well known and defined, a project is always unique.</p><p>So, a project is something that people do for the first time, to achieve something new. And when we do something for the first time, there are two kinds of unknowns involved: the known unknowns (knowledge we consciously know we are lacking) and the unknown unknowns (stuff we don't know and we don't even realize it). Based on the nature and environment of the project and our experience in this field, we can identify some of the unknowns and risks to a certain degree. But I don't believe that there will really be a project where all the potential pitfalls could be identified unless you actually implement the project - only then you will know for sure. If we'd like to identify all risks and analyze future problems and their potential impact, we need to try it out in real life. Only then could we be certain about the outcomes, approving or disapproving our initial expectations.</p><p>I am trying to express that uncertainty is part of every project. That means that when planning a project, we need to take that into account. So when setting up a project and trying to get a grasp of the costs, timeline, and scope, we must understand we're always dealing with estimates and planning errors. So instead of trying to pretend it doesn't exist and requiring (or providing) a seemingly "exact and final" project number, I think a more constructive discussion would be about the actual scale of the error. </p><h2>Cognitive biases</h2><p>While the above is generally logically acceptable to rational and experienced people, why do we tend to ignore or underestimate the risks at the beginning? I believe it's got something to do with how our minds work.</p><p>There is a phenomenon called the <strong>planning fallacy</strong>, first described by psychologists in the 1970s. In essence, they found that people tend to (vastly) underestimate time, costs, and risks of actions while (vastly) overestimating the benefits. The researchers measured how probable were various subjects to finish various tasks within the timeframes the subjects have estimated. Interestingly, over half of the subjects often needed more time to finish the task than was their catastrophic-case estimate.</p><p>The actual thinking processes are even more interesting. Even with past experience of solving a similar problem and a good recollection of it, people tend to think they will be able to solve it quicker this time. And that people genuinely believe their past predictions (which went wrong in the end) were too optimistic, but this time they believe they are making a realistic estimate.</p><p>There is also something called an <strong>optimism bias</strong>. Optimism bias makes people believe that they are less likely to experience problems (compared to others). So even though we can have a broad range of experience with something, we tend to think things will evolve in an optimistic way. We tend to put less weight on the problems we may have already encountered in similar situations, believing this was "back then" and now we are of course more clever, and we won't run into any problems this time. People tend to think stuff is going to go well just because they wish for it.</p><p>Another interesting factor is our tendency to take credit for whatever went well in the past, overestimating our influence, while naturally shifting the reasons for negative events to the outside world - effectively blaming others for what went wrong or blaming bad luck. This might not be expressed out loud, but it influences our views regardless. This stems from a phenomenon called <strong>egocentric bias</strong>.</p><h2>Combining psychology with projects</h2><p>So it becomes quite obvious that if we combine the lack of relevant experience (a project is always a unique undertaking up to a certain degree, remember?) with the natural tendency to wish for the best, we get a pretty explosive mixture.</p><p>We need to understand that not just the project team itself, but also the stakeholders fall victim to the above-mentioned factors. They also wish for a project to go as they planned and managers rarely like sorting out any problems that stem from a project in trouble that doesn't evolve as expected.</p><p>Yes, I have met managers who naturally expect considerable risks and don't take positive outcomes for granted. Managers who understand the uncertainties and will constructively attempt to help a project which slowly deviates from the initial expectations. When we have a manager who addresses risks and issues factually and rationally, it is bliss.</p><p>But what if that's not the case? Many managers try to transfer the responsibility for possible problems to the project teams or project managers while insisting that the project manager must ensure "project goes as estimated". Usually, their way of supporting a project is by stressing how important it is to deliver stuff in time and that the team must ensure it no matter what. And that all the features need to be included, of course.</p><p>Now when you combine the fuse in the form of pressure from stakeholders with this explosive mix, that's when the fireworks start.</p><p>So how to increase the chance of creating a sane plan, keep the stakeholders realistically informed, while maintaining a reasonably peaceful atmosphere in the development team? I think we can help it by gathering certain statistics and knowing we are constantly under the effect of cognitive biases. We'll look at this in the next part of this series.</p>#scrum;#agile;#project-management;#release-management
How to run Fastlane and GitLab with AppStore Connect API to run Fastlane and GitLab with AppStore Connect API<p>​​​​In this brief tutorial, I am going to walk you through a relatively painless process of using <span class="pre-inline">.p8</span> certificates from App Store Connect API to authenticate during Fastlane builds. As we also use GitLab as our CI, I will also show you how to pass the key from GitLab to Fastlane script.</p><h2>Step 1: Allow keys to be created</h2><p>For this step, you need to be an account holder. If you are not an account holder, you need to ask him to do it for you.</p><p>Note: Keys are generated per organization. For example, if you are a developer on project A (First organization), project B (First organization), and project C (Second organization), you will have to ask account holders of both the First organization and Second organization to allow access.</p><p>The first step is to go to Users and Access (tab Keys), you will see something like in <strong>Image 1</strong>.</p> <img alt="Image 1 - Starting point for the account holder" src="/Blog/PublishingImages/Articles/fastlane-appstore-01.png" data-themekey="#" /> <p></p><center><strong>Image 1</strong> - Starting point for the account holder</center><p></p><p>Following this, you need to click on the “Request Access” button. This will allow anyone with an Admin role to create new keys. You will see some kind of pact with the devil, or disclaimer if you prefer. You need to check the box and click submit. Visual reference in <strong>Image 2</strong>. If everything goes well, you are rewarded with the screen captured in <strong>Image 3</strong>.</p> <img alt="Image 2 - Disclaimer" src="/Blog/PublishingImages/Articles/fastlane-appstore-02.png" data-themekey="#" /> <p></p><center><strong>Image 2</strong> - Disclaimer</center><p></p> <img alt="Image 3 - The result screen" src="/Blog/PublishingImages/Articles/fastlane-appstore-03.png" data-themekey="#" /> <p></p><center><strong>Image 3</strong> - The result screen</center><p></p><p>For the account holder, the journey ends here. Now it’s your turn.</p><h2>Step 2: Obtaining the key</h2><p>For this part, you will have to have Admin rights, which is referred to in <a href="">AppStore Connect documentation</a>.</p><p>You will start as <strong>Image 3</strong> shows, Users and Access, tab Keys. Now the process is still pretty straightforward. Just tap the magic button and you will see the form for key creation. Fill in the details (<strong>Image 4</strong>), the name is not substantial, but access has to be at least <strong>App Manager</strong>. Otherwise, your Fastlane pilot will fail when updating info as stated in <a href="">Fastlane documentation</a>. If you do not need to modify info, <strong>Developer</strong> access will suffice.</p> <img alt="Image 4 - Access key form" src="/Blog/PublishingImages/Articles/fastlane-appstore-04.png" data-themekey="#" /> <p></p><center><strong>Image 4</strong> - Access key form</center><p></p><p>Next, you will see your key has been created (<strong>Image 5</strong>). There are 2 values to take into account, Issuer ID (blue rectangle) and Key ID (red rectangle). You don’t need to write them down as they are always accessible to you. What is not always accessible, however, is the “Download API Key” button. You will be prompted about this when trying to download the key.</p> <img alt="Image 5 - Newly created key" src="/Blog/PublishingImages/Articles/fastlane-appstore-05.png" data-themekey="#" /> <p></p><center><strong>Image 5</strong> - Newly created key</center><p></p><h2>Step 3: Using the key</h2><p>Now to the main part. Download the key. You can only download it only once, but it is not really a problem, just a mild inconvenience as you can always generate a new key.</p><p>There are two ways to use keys within the Fastlane script. One is by using the base64 encoded key directly and the other by using the json file described <a href="">in the documentation</a>. </p><p>I use the base64 encoded key. Open the terminal window and go to the folder where you have your key. Run the command <span class="pre-inline">cat {KEYNAME} | base64</span> and copy the result for later usage.</p> <img alt="Image 6 - Writing out base64 key" src="/Blog/PublishingImages/Articles/fastlane-appstore-06.png" data-themekey="#" /> <p></p><center><strong>Image 6</strong> - Writing out base64 key</center><p></p><p>Now that you have your key, you have to adjust your Fastlane and GitLab script. If you are not using GitLab, skip that part.</p><p>For the Fastlane script, you need to start using options from your lane. I have my base64 key as an argument with the name <span class="pre-inline">api_key</span> supplied to a script. Now you may notice the colorful rectangles. The red one should be issuer id from <strong>Image 5</strong>, and of course, the blue one is the key id from the very same image. For reference, see my lane in <span class="pre-inline">fastfile</span> in <strong>Image 7</strong>.</p> <img alt="Image 7 - Fastfile sample" src="/Blog/PublishingImages/Articles/fastlane-appstore-07.png" data-themekey="#" /> <p></p><center><strong>Image 7</strong> - Fastfile sample</center><p></p><p>If you don’t use any other tool which manages CI, you are pretty much all set. You may call your script as you did before, just add the parameter followed by “:” and value. For example, <span class="pre-inline">fastlane {MY LANE} api_key:”{MY KEY}”</span>. If you do not want to manually input it every time, you might consider some kind of key storage, for instance, keychain. Then you may skip passing the argument to the lane, but that is a topic for another day.</p><h2>GitLab adjustments</h2><p>For this to work, you first need to go to your GitLab interface and set a new variable. <strong>Image 8</strong> will help you find it. Add a name that will be later used in the script, for me it is <span class="pre-inline">APP_STORE_CONNECT_API_KEY</span> and the base64 version of your key. Don’t forget to tick the box to mask it from logs (<strong>Image 9</strong>).</p> <img alt="Image 8 - GitLab settings" src="/Blog/PublishingImages/Articles/fastlane-appstore-08.png" data-themekey="#" /> <p></p><center><strong>Image 8</strong> - GitLab settings</center><p></p> <img alt="Image 9 - Adding a new variable" src="/Blog/PublishingImages/Articles/fastlane-appstore-09.png" data-themekey="#" /> <p></p><center><strong>Image 9</strong> - Adding a new variable</center><p></p><p>Now you have to adjust your Gitlab script. Open your <span class="pre-inline">.gitlab-ci.yml</span> file and for every lane where you want to use the key add it as a named parameter as in <strong>Image 10</strong>.</p> <img alt="Image 10 - Script adjustments" src="/Blog/PublishingImages/Articles/fastlane-appstore-10.png" data-themekey="#" /> <p></p><center><strong>Image 10</strong> - Script adjustments</center><p></p><p>Now you are all set to continue doing automated submissions with GitLab and Fastlane.</p>#appstore;#fastlane;#gitlab;#iOS
Scrum smells, pt. 4: Dreadful planning smells, pt. 4: Dreadful planning<p>In a few of our past projects, I encountered a situation that might sound familiar to you: Developers are getting towards the end of a sprint. The product owner seems to have sorted the product backlog a bit for the sprint planning meeting - he changed the backlog order somewhat and pulled some items towards the top because he currently believes they should be added to the product rather soon. He added some new things as well because the stakeholders demand them. In the meantime, the team works on the development of the sprint backlog. The sprint ends, the team does the end-of-sprint ceremonies and planning we go.</p><p>At the planning meeting, the team sits down to what seems to be a groomed backlog. They go through the top backlog items with the product owner, who explains what he has prioritized. The team members try to grasp the idea and technical implication of the backlog items and try their best to plan them for development. But they find out that one particular story is very complex and can't be fitted within a sprint, so they negotiate with the product owner about how to meaningfully break it down into several smaller pieces. Another item has a technical dependency on something that has not been done yet. The third item has a functional dependency - meaning it won't work meaningfully unless a different story gets developed. The fourth item requires a technology that the developers haven’t had enough experience with. Therefore, they are unable to even remotely tell how complex it is. And so on it goes - the team members dig through the “prepared” backlog, try to wrap their heads around it, and finally find out that they can't work on every other story for some reason.</p><p>One possible outcome is that such items are skipped, and only the items that the team feels comfortable with are planned into the sprint backlog. Another outcome is that they will want to please the product owner and “try” to do the stuff somehow. In any case, the planning meeting will take hours and will be a very painful experience.</p><p>In both cases, the reason is poor planning. If there ever was a planned approach by the product owner towards the backlog prior to the planning meeting, it was naive, and now it either gets changed vastly, or it gets worked on with many unknowns - making the outcome of the sprint a gamble.</p><h2>What went wrong?</h2><p>One might think all the planning occurs exclusively at the planning meeting. Why else would it be called a planning meeting? Well, that is only half true. The planning meeting serves the purpose for the team to agree on a realistic sprint goal, and discuss with the product owner what can or cannot be achieved within the upcoming sprint, and create a plan of attack. Team members pull the items from the top of the backlog into the sprint backlog in a way that gets to that goal in the best possible way. It is a ceremony that actually starts the sprint, so the team sets off developing the stuff right away.</p><p>In order to create a realistic sprint plan that delivers a potentially releasable product increment with a reasonable amount of certainty, there has to be enough knowledge and/or experience with what you are planning. The opposite approach is called gambling.</p><h2>Definition of ready</h2><p>It is clear that the backlog items need to fulfill some criteria before the planning meeting occurs. These criteria are commonly referred to as a “definition of ready” (DoR). Basically, it is a set of requirements set by the development team, which each backlog item needs to meet if the product owner expects it to be developed in upcoming sprints. In other words, the goal of DoR is to make sure a backlog item is immediately actionable, the developers can start developing it, and they can be realistically finished within a sprint.</p><p>We had a good experience with creating DoR with our teams. However, we also found that this looks much easier at a first glance than it is in practice. But I believe it is definitely worth the effort, as it will make predictions and overall workflow so much smoother.</p><p>DoR is a simple set of rules which must be met before anyone from the scrum team can say “we put this one into the sprint backlog”. They may be dependent on the particular product or project, and they can be both technical and business-sided in nature, but I believe there are several universal aspects to them as well. Here are some of our typical criteria for determining if backlog item satisfies the DoR:</p><ul><li>Item has no technical or business dependencies.</li><li>Everyone from the team understands the item's meaning and purpose completely.</li><li>We have some idea about its complexity.</li><li>It has a very good cost/benefit ratio.</li><li>It is doable within one sprint.</li></ul><p>There are usually more factors (such as a well-written story definition, etc.), but I picked the ones that made us sweat the most to get them right.</p><h2>Putting backlog refinement into practice</h2><p>This is a continuous and never-ending activity, which in my opinion has the mere goal of getting the DoR fulfilled. As usual, the goal is simple to explain, but in practice not easy to achieve. Immature teams usually see refinement activities as a waste of time and a distraction from the “real work”. Nonetheless, our experience has proven many times that if we don't invest sufficient time into the refinement upfront, it will cost us dearly in time (not so much) later in the development.</p><p>So, during a sprint, preparing the ground for future sprints is a must. The development team must take this t into account when planning the sprint backlog. Refinement activities will usually occupy a non-negligible portion of the team's capacity.</p><p>The product owner and the team should aim at having at least a sprint or two worth of stuff in the backlog, which meets the DoR. That means there needs to be a continuous discussion about the top of the backlog. The rest of the scrum team should challenge the product owner to make sure nothing gets left there just “because”. Why is it there? What is its purpose and value in the long term?</p><p>Once everyone sees the value, it is necessary to evaluate the cost/benefit ratio. The devs need to think about how roughly complex it will be to develop such a user story. In order to do that, they will need to work out a general approach for the actual technical implementation and identify its prerequisites. If they are able to figure out what the size roughly is, even better.</p><p>However, from time to time, the devs won't be able to estimate the complexity, because the nature of the problem will be new to them. In such cases, our devs usually assigned someone who did research on the topic to roughly map the uncharted area. The knowledge gained was then used to size the item (and also later on, in the actual development). This research work is also tracked as a backlog item with it's intended complexity, to roughly cap the amount of effort worth investing into it.</p><p>Now with the approximate complexity established, the team can determine whether the item is not too large for a sprint. If it is, then back to the drawing board. How can we reduce or split it into more items? In our experience, in most cases, a user story could be further simplified and made more atomic to solve the root of the user's problem. Maybe in a less comfortable way for him, but it is still a valuable solution - remember the Pareto principle. The product owner needs the support of the devs to know how “small” a story needs to be, but he must be willing to reduce it, and not resist the splitting process. All of the pieces of the “broken down” stories are then treated as separate items with their own value and cost. But remember, there always needs to be a user value, so do vertical slicing only!</p><p>Then follows the question: “Can't we do something with a better ratio between value and cost instead?” In a similar fashion, the team then checks the rest of the DoR. How are we going to test it? Do we need to figure something out in advance? Is there anything about the UI that we need to think about before we get to planning? Have we forgotten anything in dependencies?</p><p>Have we taken all dependencies into account? <strong>Are we able to start developing it and get it done right away?</strong></p><h2>Let the planning begin!</h2><p>Once all the questions are answered, and both the devs and the product owner feel comfortable and familiar with the top of the backlog, the team can consider itself ready for the planning meeting.</p><p>It is not necessary (and in our case was also not common) for all devs to participate in the refinement process during a sprint. They usually agreed on who is going to be helping with the refinement to give the product owner enough support, but also to keep enough devs working on the sprint backlog. At the planning meeting, the devs just reassure themselves that they have understood all the top stories in the same way, recap the approach to the development, distribute the workload and outline a time plan for the sprint.</p><p>The sprint retrospective is also a good time to review the DoR from time to time, in case the team encounters problematic patterns in the refinement process itself.</p><p>Proper and timely backlog refinement will prevent most last-minute backlog changes from happening. In the long run, it will save money and nerves. It is also one of the major contributors to the team's morale by making backlog stuff easier to plan and achieve.</p>#scrum;#agile;#project-management;#release-management
Apple developer centre – organized and automated developer centre – organized and automated<p> Code signing goes hand in hand with iOS development, whether you wish to build and upload your app to your device, or you just want to upload it to the App Store. If you're new to iOS development and don't want to deal with it right from the start, you can enable automatically managed code signing, which is fine for the time being, but in a team of 50, it becomes rather ineffective. When someone removes their device and invalidates a wildcard development provisioning profile, or accidentally invalidates a distribution certificate, your pipeline will fail out of nowhere, and the robustness of continuous integration and/or deployment suffers as a consequence. </p><p> The right approach for getting rid of human-error in any process is to remove humans from the equation. Don't worry, in this case, it just means to remove their access to the developer centre. But how do you keep people able to develop their apps on real devices and distribute apps to the AppStore? </p><h2> It's a Match! Fastlane Match </h2><p> Fastlane and its match don’t need much introduction in the iOS community. It's a handy tool that ensures everyone has access to all development and distribution certificates, as well as profiles, without having access to the dev centre, as match uses git as storage for encrypted files. It offers a <span class="pre-inline">read-only</span> switch that makes sure nothing gets generated and invalidated accidentally. There are two roles in this approach - one for the admin and the developer. The developer uses match to install whatever is needed at the time of development and sets up CI/CD. He only needs access to the match git repository, not the developer centre. That's where the admin comes in - he is the one responsible for setting up all the devices, provisioning profiles, certificates, and git repository, where all the match magic happens. It's good to have at least two admins in case something goes awry while one of them is out of office. </p><h2> Match setup (admin perspective) </h2><p> The idea behind match is pretty simple, you don't have to deal with the developer centre as much, and you can instead focus on having a private git repository set up with all your certificates and provisioning profiles, all properly encrypted, of course. It supports developer and distribution certificates, a single repository can even handle multiple accounts. Match expects a specific folder structure in order to automatically find the matching type of certificates and profiles, but it's pretty straightforward: </p><pre><code class="hljs">|-certs |--development |--distribution |-profiles |--appstore |--development </code></pre><p> The certs folder contains a private key and a public certificate, both are encrypted. Profiles contain encrypted provisioning profiles. Match works with <span class="pre-inline">AES-256-CBC</span>, so to encrypt the provisioning profile you can use <span class="pre-inline">openssl</span>, which comes pre-installed on macOS. </p><h2> Certificate encryption </h2><p> First, you create a certificate in the dev centre. The certificate’s key is then exported from the keychain to the p12 container, and the certificate itself is exported to the cert file. Match expects the key and the certificate to be in separate files, so don't export them both from the keychain to a single p12 container. You need to pick a passphrase that is used to encrypt and later decrypt certificates and profiles. It's recommended to distribute the passphrase to others in some independent way, storing it in the repository (even though private) would make the encryption useless. </p><p> To encrypt key, run: </p><pre><code class="hljs">openssl aes-256-cbc -k "my_secret_password" -in private_key.p12 -out encrypted_key.p12 -a </code></pre><p> To encrypt the certificate: </p><pre><code class="hljs">openssl aes-256-cbc -k "my_secret_password" -in public_cert.cer -out encrypted_cert.cer -a </code></pre><p> You can have multiple certificates of the same kind (developer or distribution) under one account. To assign a provisioning profile to its certificate you need to use a unique identifier generated and linked to the certificate in the developer centre. The following Ruby script lists all the certificates with their generated identifiers. The identifier is used as a name for the key and for the certificate: </p><pre><code class="ruby hljs">require 'spaceship' Spaceship.login('') Spaceship.select_team Spaceship.certificate.all.each do |cert| cert_type = Spaceship::Portal::Certificate::CERTIFICATE_TYPE_IDS[cert.type_display_id].to_s.split("::")[-1] puts "Cert id: #{}, name: #{}, expires: #{cert.expires.strftime("%Y-%m-%d")}, type: #{cert_type}" end </code></pre><h2> Provisioning profiles encryption </h2><p> Provisioning profiles are encrypted in the same way as the certificates: </p><pre><code class="hljs">openssl aes-256-cbc -k "my_secret_password" -in profile.mobileprovision -out encrypted_profile.mobileprovision -a </code></pre><p> Naming is a bit easier: Bundle identifier is prefixed with the type for the provisioning profile like this: </p><pre><code class="hljs"> </code></pre><h2> Good orphans </h2><p> The typical git branching model doesn't make much sense in this scenario. Git repository is used as storage for provisioning profiles and certificates, rather than for its ability for merging one branch into another. It's no exception to having access to multiple dev centres, for instance, one for the company account, one for the enterprise account, and multiple accounts for the companies you develop and deploy apps for. You can use branches for each of those accounts. As those branches have no ambition of merging into each other, you can create orphan branches to keep them clearly separated. Then just use the <span class="pre-inline">git_branch</span> parameter to address them (for both development and distribution): </p><pre><code class="hljs">fastlane match --readonly --git_branch "company" ... fastlane match --readonly --git_branch "enterprise" ... fastlane match --readonly --git_branch "banking_company" ... </code></pre><h2> With great power... </h2><p> As the admin of a team without access to the dev centre, you're going to get a lot of questions on how to install certificates and profiles. It's helpful to set up a README in your codesigning repository that describes which apps are stored under which branches, and even includes <a href="">match documentation</a> and fastlane's <a href="">code signing guides</a>. It's also super cool of you to set up an installation script for each project, and put it under version control of the said project. Then when a new member joins the team and asks how to set stuff up, you just point them to run <span class="pre-inline">./</span>. </p><h2> Match usage (developer perspective) </h2><p> As a developer, you don't have access to the dev centre. You only need access to the git repository and a few commands to download profiles and install them on your machine. You also need to have your device registered in the account and assigned to the provisioning profile you'd like to use. But since you don't have access, you need to ask admins to set it up for you, which is a price paid by the admins for the sake of order and clarity. After that, you're all set and can run the commands to install whatever is necessary. The developer gets asked the passphrase the first time the command is run. You can choose to store it in the keychain if you'd like to skip entering it next time. </p><h2> Development profiles </h2><p> There are but a few inputs to match command: <span class="pre-inline">git_branch</span> reflects which account the app is registered in, <span class="pre-inline">app_identifier</span> is a bundle identifier of the app, and the others are also quite self-explanatory. If you're not sure which branch to use, you can go one by one and browse the profiles folder to see if the bundle identifier is listed there; it is unique across all accounts, so it should only be in one branch. </p><p> For instance, to install a development profile with certificate for the bundle id <span class="pre-inline"></span> you'd run: </p><pre><code class="hljs">fastlane match --readonly --git_branch "company" --git_url "" --app_identifier "" --type development </code></pre><p> You can also store a wildcard profile in the match repository, even if it does not have any real bundle identifier. In such a case you can just choose any identifier and use that, for instance <span class="pre-inline">*</span>: </p><pre><code class="hljs">fastlane match --readonly --git_branch "company" --git_url "" --app_identifier "*" --type development </code></pre><h2> Distribution profiles </h2><p> Distribution of the app to the App Store is basically the same as installing developer profiles, just change the <span class="pre-inline">type</span> from <span class="pre-inline">development</span> to <span class="pre-inline">appstore</span>: </p><pre><code class="hljs">fastlane match --readonly --git_branch "company" --git_url "" --app_identifier "" --type appstore </code></pre><p> Distribution to the App Store is usually scripted in a Fastfile script, which consists of many different actions in addition to match. That is outside the scope of this post and is well explained in other posts on the Internet. </p><h2> Conclusion </h2><p> You can clean up your dev centre and avoid certificates/profiles being revoked accidentally by switching the responsibility to git versioned repository using match. You can trick match to think that the wildcard provisioning profile is just some made-up bundle id in order to store it in git. You can have multiple branches for multiple types of dev centre accounts for an extra level of tidiness. On top of all that, you save your development team a lot of time by distributing the scripts to install whatever they need, and you can make life a bit easier for newcomers as well. </p> #iOS;#code-signing
Building a chatbot, pt. 1: Let's chat a chatbot, pt. 1: Let's chat<p>A few years ago, a client asked us to create an application that allows its users to create bookings for conference rooms and workspaces. That looks quite easy, right? A few database tables, a thin server, and thick web and mobile applications for a smooth user experience. Almost every company has a solution like that, so it should be fairly easy. But wait, there is a catch! The user interface has to be a chatbot!<br></p><p>That’s a completely different situation. How do we build something like that from scratch? We need to adjust our strategy a bit; we are going to need a thick server, thin web, and mobile applications. To limit the scope of this article, we will focus on the server-side.</p><h2>So it begins</h2><p>After a few searches and a fair amount of experiments we stumbled across <a href=""><em>NLP - Natural language processing</em></a>. These three words describe a key component of every modern chatbot platform. The chatbot takes ordinary sentences and transforms them into a data structure that can be easily processed further, without all the noise that surrounds core information.</p><p>Let's look at this example:</p> <img alt="Example of analysis" src="/Blog/PublishingImages/Articles/chatbot-1-01.png" data-themekey="#" /> <p>A simple sentence like this is split into multiple items that can be named and searched for. In this case, the phrase <em>“I need place”</em> is identified as a general intent that can be interpreted as a request for booking. Other items add information to this request. These <em>attributes</em> can carry either simple or complex information. In this example, the word <em>“some”</em> gives us the freedom to select any room from a list of available rooms, and the word <em>“meeting”</em> is interpreted as a request for a meeting room. Those parts were the easiest to classify. Time recognition attributes are more complex.</p><p>This is great for identifying atomic attributes in the sentence, but it's still a text. It took us almost a year to put together a comprehensive training data set for our target languages (English and German), but our bot finally understands the vast majority of user's requests. But how do you connect a room number to a specific room entity, username to the user, or date description to an actual date?</p><p>For that, we had to build an additional layer. Some of the post-processors need a whole blog post to describe it, but in the end, we managed to get a nice set of domain objects that are used in the bot’s decision-making process. In general, it looks like this:</p> <img alt="Cognitive processor overview" src="/Blog/PublishingImages/Articles/chatbot-1-02.png" data-themekey="#" /> <p>Input sentences are processed by the NLP and each <em>intent</em> or <em>attribute</em> is then passed to an <em>interpreter</em> that creates one or more objects that are used in conversation flow.</p><p>The most difficult part - the recognition - was solved (or so we thought). NLP gave us a nice structure with multiple items that can be <em>interpreted</em> as simple data objects.</p><h2>Neurons or no neurons, that’s the question</h2><p>The logic for conversion of recognized data to actions on the database was quite simple at the beginning. We had a few separated, well-defined use cases that were easy to implement. But complexity grew quite rapidly. A few <em>'if'</em>s were not sufficient anymore, so we had to look for a more robust solution.</p><p>After a little bit of research, we found that most of the solutions depend heavily on neural networks. That gives these solutions an edge with multiple short sentences, and general conversations about weather, sport, local natural wonders, etc. This is a robust solution for general use, when the conversations flow naturally from beginning to end. Decision-making is hidden in the neural network, which is trained with a sample data set. Neural networks are easy to start with, and adding new features is simple. Let's use it!</p><p>Well, not so fast... In testing, it works wonders, but as soon as we put it into the hands of test-users, we were bombarded with bugs. There was something we forgot: Real people. Users were giving us only partial information, and we didn't cover every possible angle. We quickly lost control over the conversation flow, with multiple use cases and various responses from the database.</p><p>This was not ideal. If we were aiming for a small-talk bot, a neural network would be ideal, but we were building a single-purpose bot. Users know exactly why they open a conversation with our bot - they want a reservation. We had to regain control of the conversation flow in the code and get all the information which the app needed from the user. The solution had to be simple, maintainable, testable, and scalable.</p><p>And so we rebuilt the application into a state machine where the bot is in control of the conversation flow. Simply put, it gets a <em>state</em>, in our case a point in the conversation with the user, and a list of information that the user already gave to the bot. Based on these data, it transitions into the next state - the next point in the conversation. </p><p>Users seek to achieve a goal, the bot guides them through the conversation and asks for the information needed. We didn't forget to add a few simple small talk phrases as well, but not many users come to our bot for small talk. The core of the state machine deserves its own blog post, but here is a small example:</p><h2>What does the bot say?</h2><p>Let's say we have a user who needs to book a room for an important meeting for next Monday from 9:30 to 10:30. In general, to make such a reservation, you will need a time and room specification. This is how the conversation could unfold:</p><p> <strong style="color:#381457;">User</strong> : <em>Book me room E.3.30 for next Friday from 9:30 to 10:30.</em><br> <strong style="color:#9d9d9d;">Bot</strong> : <em>I have booked room E.3.30 on 26.02. from 09:30 to 10:30 for you.</em> </p><p>Quite simple, right? But the same conversation could look also like this:</p><p> <strong style="color:#381457;">User</strong> : <em>I need a reservation.</em><br> <strong style="color:#9d9d9d;">Bot</strong> : <em>Do you want me to book you a seat or a meeting room?</em><br> <strong style="color:#381457;">User</strong> : <em>Meeting room.</em><br> <strong style="color:#9d9d9d;">Bot</strong> : <em>Alright! Would you like to book a specific meeting room or Shall I just book one that is free?</em><br> <strong style="color:#381457;">User</strong> : <em>Specific meeting room.</em><br> <strong style="color:#9d9d9d;">Bot</strong> : <em>What room would that be?</em><br> <strong style="color:#381457;">User</strong> : <em>R.23.</em><br> <strong style="color:#9d9d9d;">Bot</strong> : <em>For which time shall I reserve the room?</em><br> <strong style="color:#381457;">User</strong> : <em>Next Monday.</em><br> <strong style="color:#9d9d9d;">Bot</strong> : <em>Can you specify the time for me, please?</em><br> <strong style="color:#381457;">User</strong> : <em>9:30 to 10:30.</em><br> <strong style="color:#9d9d9d;">Bot</strong> : <em>I have booked room R.23 for 1st March from 09:30 to 10:30 for you.</em> </p><p>In the first example, the user knew exactly what he wanted. In the second conversation, the bot guides the user. These examples are on the opposite sides of the conversation spectrum, but we also cover everything in the middle. When the user states the date and time earlier in the conversation, the bot should not ask for it again. The main point is that all of these conversations are processed with the same conversation flow (same code, same tests).</p><p>What is neat about this approach is that we can take a part of the conversation and re-use it for multiple intents. For example, time validation can be reused in any conversation where a time specification is needed.</p><p>There is one part of the example that I've excluded, and that's the access to the reservation system itself. Here we simply save the request and call it a day, but in everyday use, there are some limitations - the reservation may very well be refused. All of these possibilities have to be covered, and users have to be properly informed. Again, how to do that is a topic for a whole new blog post.</p><h2>Conclusion</h2><p>As you can see, there are a number of topics to consider when building a chatbot from scratch: from NLP to decision making, to actions in the reservation system, and finally to the answers.</p><p>Thanks to rigorous testing and a clear framework, we are not blocked by bloated training data sets, and multiple devs can develop independently of each other.</p><p>Currently, our application can process multiple base intents like <em>show</em>, <em>cancel</em>, <em>check</em> or <em>book</em> in English and German. Based on these intents, the bot can give the user up to 300 different conversations with multiple responses. More conversations and variations are still in development and we hope to reach 500 in the near future. Our system is currently used by more than 1400 users and on average 2000 interactions happen every week.</p> #chatbot;#ai;#neural-network