Intro & Feedback Overview
Feedback Loops & App Center Analytics
Baselines & Noise
The course is part of these learning paths
The application lifecycle doesn’t finish with app deployment. Feedback is an important element of refining an application, whether that’s exception detection and diagnoses or improving the user experience. In this course, we will look at a suite of services that capture a vast array of feedback data, ranging from exceptions to client and server telemetry. This data can be turned into easily digestible information that can be used to trigger alerts and feedback into the development lifecycle as work items.
This course begins by describing what feedback is, and the types of feedback for improving application performance and usability. It then moves on to how we can integrate feedback into the software development lifecycle and what tools we can use to simplify that task. Finally, we will look at optimizing feedback mechanisms to get meaningful data from feedback noise.
For any feedback and questions relating to this course, please contact us at firstname.lastname@example.org.
- Designing application and user feedback loops
- Setting up crash and event notifications for App Center
- Setting up work item integration from App Center
- Making sense of App Center’s analytic and diagnostic information
- Adding Application Insights Telemetry to an application
- Setting up Application Insights alerts
- Work item integration from Application Insights
- Designing feedback dashboards
- Viewing Application Insights Telemetry data
- Discussing types of user feedback and how they can be captured
- Ways to baseline and filter feedback data
This course is intended for:
- People preparing for Microsoft’s AZ-400 exam
- App developers
- Project managers
To get the most from this course, you should have some experience with Microsoft Azure and application development, as well as knowledge of software project management concepts.
So far, we have looked at purely technical and system-oriented feedback. While system feedback is objective and relatively easy to collect, it by no means tells the whole story from a product development perspective. If your users are also your customers then making sure they are at least satisfied, if not happy, is important. This is especially true in the context of the widely adopted subscription model. Even if your users aren’t paying for your software you want to know if it is meeting their needs as best it can, enabling them to do their jobs efficiently. Ideally you would want this feedback to be integrated into your system and usage feedback loop. This serves to have one repository of issues that can feed into the development workflow system and allows you to easily relate user feedback with automatically collected system issues.
Let’s look at the types of user feedback we might want to analyze. Issues that come through the help desk fall into 2 basic categories, bugs or non-conforming behavior and incorrect usage. In a help desk situation, you have the ability to perform manual triage and create work items. The user issues can be grouped with related metrics or exception data.
You can build user feed back systems within the app like send a smile, or even put feature specific feedback at critical points like star ratings or binary answer questions. This is a double-edged sword. It allows you to target specific aspects of the app’s behavior, but you don’t want to annoy or inconvenience the user. An obvious candidate for a single question is the staple of the Net Promoter Score. On a scale of 0 to 10 how likely are you to recommend this product, service or company. Middle to low scores can be followed up with a more in-depth questionnaire to drill down to specific issues.
Surveys are another way to garner customer feedback. These could be through the app or delivered separately via email. Once again you are treading a fine line between gathering information and annoying the user. Survey design is a science in its own right, but generally speaking, questions that are multi-choice are easy to analyze, whereas free text answers, are where all the nuggets of knowledge, or the “unknown unknowns” reside. Another thing to keep in mind is that surveys where participants self-select tend to get responders who have a specific problem, so while good for finding issues may not reflect overall customer satisfaction.
Forums or user groups are another source of good customer feedback where a thread is dedicated to a particular topic. This is one place where the issues are already categorized, but still require you to pull the information from text or dialogue.
Finally, social media. In terms of how your app is truly perceived this where the gold is, but it is gold that is difficult to mine. You don’t know exactly where it is, and when you find it, it isn’t usually in an easily digestible form. This is definitely a situation where you don’t want to reinvent the wheel. There are plenty of services that monitor social media that you can subscribe to. When collecting the mentions it is usual procedure is to code the verbatim, that is categorize the data so it can be more easily analyzed.
There are several ways you can record your user feedback ranging from spreadsheets to dedicated project management tools such as JIRA, Basecamp or DevOps. The central goal is to have indicative examples of feedback in one place that can be directly related to work items. I say indicative examples because you don’t need every instance, just the ones that best illustrate the issue the work item ticket addresses. As for survey, questionnaire and rating data, this needs to be distilled down to essential and actionable facts that are correlated with user attributes such experience or time using the app.
Hallam is a software architect with over 20 years experience across a wide range of industries. He began his software career as a Delphi/Interbase disciple but changed his allegiance to Microsoft with its deep and broad ecosystem. While Hallam has designed and crafted custom software utilizing web, mobile and desktop technologies, good quality reliable data is the key to a successful solution. The challenge of quickly turning data into useful information for digestion by humans and machines has led Hallam to specialize in database design and process automation. Showing customers how leverage new technology to change and improve their business processes is one of the key drivers keeping Hallam coming back to the keyboard.