Agile Business Solutions

Category: Testing

Testing covers user aceptance testing (UAT), system integration testing (SIT), system testing, unit testing and test driven development.

  • 3 Tribes Experiment UX Camp

    3 Tribes Experiment UX Camp

    After some persuasion by colleague Patrick Sansom I presented the 3 Tribes Experiment at UX Camp Brighton 2017, an annual event full of shared learning and fun. This year there were five rooms that offered a choice of presenters and subjects covering diverse topics as user experience, user research, lean coffee discussion on design sprints, to Chris How’s memorable Machiavellian masterclass in manipulation and Richard Vahrman’s wearable musical interfaces. Something for everyone.

    It’s well worth spending a day there immersed in current thinking and practice and opportunity to meet some impressive people. Sessions are all time boxed so each slot is focused but still offers plenty of audience participation and interaction as you would expect for such an event.

    My 3 Tribes experiment examines the link between personality and empathy that can influence the way we think about solving problems. It’s been rattling around in my mind for a while and UX Camp was an opportunity to test the theory. The slides I used together with the experiment results are available to download here UXCamp-3Tribes.

    The experiment began by asking all participants to choose one of three questions they most identified with and take a red, green or blue sticky to remember their choice. Each question is based on a Myres Briggs personality test and selected to distinguish a participant’s empathy. As you will see from the slides the questions were never intended to provide an accurate determination of personality but would at least give some segregation. Next the participants were set a challenge of improving a well known end user experience problem, that of the ‘Unexpected item in the bagging area’. I asked everyone to think about their ideas individually at first so they would not be biased by others. The silence was deafening and after a long minute I let them group into their tribes to discuss ideas.

    Think around 20 attended and 15 participants returned results for the experiment. To my surprise, the results show a significant correlation between expected empathy and improvement approach taken for design and business process. All participants that chose a blue tribe sticky responded with a business needs empathy. And participants that chose a green tribe sticky responded with a design needs empathy with one also voting ‘half’ for user needs. Two thirds of participants that chose a red tribe sticky responded with a user needs empathy. Caveat – participants agreed their user empathy was biased by their past experience as customers. Each participant determined their empathy based on the examples I demonstrated after we’d discussed a few improvements from each tribe.

    I’m sure psychologists would find many mistakes in my amateur experiment and point out bias was inevitable but I do find the results interesting and hope you do too.

    UX Camp offered several speakers with similar themes including Leo Barnes and Mark McElhaw’s Mindstates which I’ll be finding out more about.

    Shows combination of user, design and business needs
    3 Tribes Combined
  • How to capture non functional requirements with stories

    How to capture non functional requirements with stories

    Non Functional Requirements
    Non Functional Requirements

    I’ve used several ways to capture or address non functional requirements (NFRs) with user stories. Agile practitioners such a Mike Cohn have written about the challenge. Most agree there are two main approaches: write each NFR as a user story or include NFRs in the definition of done (DoD). Let’s see how each approach can work in practice.

    First what do we mean by non functional requirement and is it an oxymoron? NFRs or quality goals for applications and services include: the service must be available 24x7x365 with 99.99% uptime. Or all user requests must be returned within 3 seconds. Or maybe DevOps must be notified to resolve critical outages, etc.

    Many of the quality goals are really cross functional requirements that determine choice of service architecture. Other NFRs help shape the company’s brand by promoting a consistent style and behaviour.

    A good way to decide how to approach an NFR is to ask how can I test it. User stories with clear behavioural driven acceptance criteria make an obvious place to start.

    User Stories

    As a bank customer 
I want to access my account at any time
So that I am not restricted to normal banking hours
    SCENARIO Registered customer accesses their account at any time of day
GIVEN customer has active account
WHEN customer logs in with valid credentials
THEN all account services are available for use

    Testing this story needs some thought. Any time literally means always so how many times should the automated tests run? And all account services suggested a complete set of the online account services are checked for each invocation. Clearly an onerous test. Suppose we consider a related NFR for number of concurrent users:

    As bank service provider
I want to support up to 10,000 users accessing their online account
details simultaneously
So that peak demand by customers at busy periods is satisfied

    Now we have 1000 user requests at any time of day wanting to access the service. We need one more NFR to close out acceptance criteria:

    As a bank customer
I want to see my account details within a few seconds of logging in
So that I can access my account quickly
    SCENARIO 10,000 registered customers simultaneously access their account at any time of day
GIVEN customers have active accounts
WHEN 10,000 customers log in with valid credentials within the same second
THEN all customer's account services are available for use within 5 seconds

    A combination of NFRs framed in a set of user stories help build test scenarios that ensure quality goals are achievable and measured. But if every user story needs to include the response time and common error handling it makes for a lot of repetition.

    Definition of Done

    Another approach to cross cutting NFRs is to build them into the DoD. For example a team might commit to a DoD for each story such as:

    • All acceptance criteria met
    • No critical or major defects found
    • Minor and cosmetic defects logged
    • Audit log entries captured for all transactions
    • Maximum response time achieved
    • Minimum concurrent users supported
    • Branding guidelines adhered
    • WAI accessibility requirements met
    • etc.

    The DoD becomes a wish list for all the quality goals and can quickly grow. With a tightly integrated test team that can define BDD criteria for each requirement the DoD can be used.

    Back to the question of what an NFR means. Each quality goal needs quantitive values to perform tests and determine if the outcome passes or fails. Failure of a test is logged as a defect and development effort put in to correct it. But what if the test fails when the service is operational? It’s the user who has to log the defect and that can be an expensive or costly for reputation.

    That’s why I suggest an NFR is often an oxymoron as it’s really a functional requirement or rather it needs a user story to ensure the service always behaves in a desired way. For example if we return to the availability NFR and the business and user’s needs:

    As bank service provider
I want to support up to 10,000 users accessing their online account details simultaneously
So that peak demand by customers at busy periods is satisfied
    As a bank service provider
I want to inform customers when my online service has reached maximum capacity
So that users know why they can't access their account

    Or from the user’s perspective

    As a bank customer
I want the online service to tell me when it's too busy for me to login
So that I can come back later (or go find another bank!)

    A good quality service is one that works with minimum friction but customers know things can go wrong. What customers hate is not being told when things go wrong. It’s like being on a train that just stops for a while and there’s no announcement to say why. So a good service needs to have behaviour built in to respond to operational issues and that needs a user story to define, build and test the behaviour to ensure it’s done.