fac-17 / autsera

A game to help children living with high-functioning autism to learn social skills.

A progressive web app to help children with high-functioning autism learn social skills.

Live Version

Deployed on netlify at autsera.netlify.com

Note: The current version is optimised for tablet use only. If you view it in your browser, please set the display to ipad landscape.

Local Installation Instructions (for developers)

For detailed installation instructions view our wiki on installation

Short Installation instructions

1. Clone this repo onto your local machine ``` git clone https://github.com/fac-17/Autsera.git ```

2. Install dependencies

npm i

3. Start local server

npm start

4. Open your browser to localhost:3000
5. Optional: run tests

npm test

About

The Problem

Children with high-functioning autism struggle with social communication including inference or "reading between the lines". There are books and board games that address this issue, but there is a lack of digital and interactive social communication training tools for children with ASD.

The Solution

An interactive game that makes social communication training for children with Autism effective, accessible, affordable and fun.

The Client

Founded by Inas Ismail and Omar Massoud, Autsera is a social-impact startup that uses technology to support children with autism develop their social skills. They are supported by WeinSocialTech, Google and Innovate UK. Autsera

The Team

• DevOps Lead: Jan Skowronski
• QA Lead: Victor Masson
• Scrum Master: Reuben George Thomas
• UX/UI Lead: Leonie Zorzi

Tech Stack

• Built with React (create-react-app)
• React Router
• Styled with Sass
• Tested with Jest and react-testing-library
• Installable as a Progressive Web App (PWA) using workbox
• Deployed on Netlify
• Continuous Integration with Travis
• Code coverage report by CodeCov

The App

Currently the app is a Minimum Viable Product (MVP) for tablet use only, focussing on three places with up to five interactions each. The codebase includes a data.js file which allows the product owner to add places and interactions easily.

The game requires the user to navigate through different places, collecting stars along the way by decoding social interactions.

User Journey

As a user, I want to be able to read facial expressions, understand how other people feel & respond to them so that I can connect socially.

User Stories

• As a user I want to see the progress of the game.
• As a user I want to play in different scenes that I am familiar with (playground, school, birthday party).
• As a user I want structure, repetition and consistency, with a clear beginning and a clear end.
• As a user I want to have options, but not too many. (show one interaction first, then unlock more as they finish).
• As a user I want a helper avatar to give me hints when I'm stuck, explain the game to me and keep me motivated.
• As a user, I want access to an explainer page at the start (featuring an explainer video and/or bullet points).
• As a user I want to decide my own path within the lower level scene (playground etc).
• As a user, I want to play the game on a tablet.

Consequence Scanning

Positives, to focus on:

• Improve social skills and ability to understand other people's feelings. • Learn social cues and facial expressions.

Negatives, to mitigate:

• Failure to progress in the game, feeling "like a loser". • Unable to generalise the skills learnt, bring them to the real world.

Prototype

Prototype was built with Figma

Prototype is based on these original user stories

As a user, I want to ....

• be able to access instructions from the start screen

• be able to immediately start the game from the start screen

• go back to the onboarding screen after I've read the instructions

• be able to start the game from the instructions screen

• be able to go back to the onboarding screen

• see and interact with all the places I have access to

• see the progress on all places (stars)

• see new places automatically unlock when I have enough stars

• see a linear route between places that have been unlocked

• be able to go back to place I have already completed

• clearly see when a new place is available (for instance through glow, movement, sound)

• be able to navigate back to the map

• see how many stars are available to pick in this location

• see the progress so far (filled-in stars)

• be able to navigate to any of the interactions that are unlocked

• clearly see when a new interaction is available (for instance through glow, movement, sound)

• be presented with all the options for the interaction, and see them highlighted when correct

• be notified when all correct options are selected in the interaction and I completed the interaction

• be able to navigate back to the place

• start from scratch whenever I re-enter an interaction

• see the answers in a different order whenever I re-enter an interaction

• delete my progress on the interaction whenever I hit the Go Back button

• be awarded a star permanently when interaction is completed

• see a helper-avatar on the bottom of the screen

• get a reaction from the avatar (hint, motivation, etc) whenever I click it

• get instructions from the helper-avatar when I get to the map

• get a motivational phrase from the helper-avatar in regular intervals (i.e. how many stars do I need to unlock)

• get a new reaction from the helper-avatar (hint, motivation, etc) whenever I click on it.

• see indications from the helper-avatar on what to do next (interact with x or y)

• be shown feedback from the helper-avatar on every answer selection

• be able to click the helper-avatar for a hint.

• be able to see a hint when I select the wrong option.

Prototype was refined based on these user testing results

• Users were very intuitive and quick in playing the game
• Users didn’t pause to read avatar text - trial and error approach rather than reading instructions Us
• Interaction screen: the fact that you could/should choose more then one answer wasn’t clear
• Users requested more complexity and features (particularly on map)
• Users suggested having animations
• Users were very keen to collect stars
• Suggestion of a video tutorial
• Avatar text was seen as unnecessary by some users
• When on the map, users click on the circle and not on the name

Design

Our design was inspired by Otsimo, Autsera's branding and digital assets provided by Autsera.

Design Inspiration

Moodboard

Sample assets

Assets were partyly sourced online and partly created by UX/UI lead.

Progressive Web Application

We wanted to have our app fully available offline once installed, and took the required steps to bring the app to PWA standards: configured the built-in Service Worker and JSON manifest, added various icon sizes and made it possible to cache the whole app.

One of the requirements is to have all the static assets cached, but assets filenames are coming from a data object therefore unknown at build time. We solved it by creating an index of all the assets linking their reference in the scenario object file to the relevant components.

Testing

• Using Jest, react-testing-library and Codecov.
• Focused on interaction testing rather than functional testing.
• For each component, we aimed to test each possibility offered to the user, so that each of our user journeys get tested.
• Achieved an overall 78% code coverage.
• Wrapping routed components in <Router />.

Check the full list of tests:

- **App** App renders without crashing User journey from initial App render **404** The 404 Page renders - **RouterLink** RouterLink renders - **HelpPage** The Help Page renders The Pause Button renders The Play Button renders

• HomePage The HomePage should render The HomePage should include the Buttons (both) The HomePage should not include Continue buttons

• StartButton startButton is rendered startButton calls the passed in onClick function

• MapPage The Map Page renders MapPage includes a Go Back Button MapPage includes a Playground Button Places rendered on the map match places available on data Locked places render with locked class Click on locked place has no effect PlaceCircle if placeCircle renders when given a place if placeCircle renders the text of the place correctly if placeCircle renders the right link

• Stars if it renders stars if it renders stars correct number of filled stars if it renders stars correct number of empty stars if it renders stars correct number of stars

• PlacePage The Place Page renders PlacePage has links to all interactions Locked interactions render with locked class InteractionCircle The InteractionCircle renders InteractionCircle includes the name of the interaction InteractionCircle includes the link to the interaction

• InteractionPage The Interaction Page renders Game starts from scratch when user re-enters page Selecting an answer changes the class to 'selected' Test if there is a back button with correct link' Test if setCompleted gets called with the right argument if clicking on answer gives the right response from avatar if clicking on avatar gives the right hint

• Avatar Helper renders with the right text Helper image appears when there is no speech provided Helper component shows the helpText when clicked if speechText and helpText both passed in -> helpText replaces speechText when helper is clicked if speechText and helpText both passed in -> helpText stays visible after multiple clicks speechText disappears after timeOut helpText disappears after timeOut expires

Stretch Goals

If we had more time, we would ...

• Add user authentication.
• Build a 'choose your avatar' feature.
• Add an Autsera Land leaderboard.
• Add parental control functionalities: playing time limit, progress review.
• Add audio content.

autsera's Issues

see a helper-avatar on the bottom of the screen

Add state for all stars to App.js (array)

be notified when all correct options are selected in the interaction and I completed the interaction

be able to access instructions from the start screen

Create circleLinks to all interactions

• Create Interaction Circle Component

• Use map to add Interaction Circle Components to PlacePage

Uploading images

Hello everyone. I seem you are busy coding.

I am uploading some images. working on adjectives for them.

see the progress so far (filled-in stars)

get a motivational phrase from the helper-avatar in regular intervals

examples :

• how many stars do I need to unlock
• congratulation messages when correct option is selected

be shown feedback from the helper-avatar on every selection

• positive, helpful hints for wrong answers
• positive affirmation for correct answers

Setup Travis CI

Add design to prototype

• Add Design to Figma

• Link main css file to App.js

• Add overall CSS classes

• Interaction: Add image to grid

• Get rid of cursors on signposts

• Fix Avatar on interaction and place page

• Link interaction image to data

• Add placeholder video to information page

• Fix Speech Bubble

see the progress on all places (stars)

Router has one endpoint for multiple instances of a component, figure out how to pass props to router endpoint.

• PlacePage component is rendered on the /page endpoint, but it needs to know which place to render.
• it will then read values for that place from the global data array

be presented with all the options for the interaction, and see them highlighted when correct

get a reaction from the avatar (hint, motivation, etc) whenever I click it

be able to navigate back to the place page

see a linear route between places that have been unlocked

clearly see when a new place is available (for instance through glow, movement, sound)

go back to the onboarding screen after I've read the instructions

be able to start the game from the instructions screen

Create Sample Data to be used across the game

Final User Stories

User Stories

As a user, I want to ....

Pre game

• be able to access instructions from the start screen #10
• be able to immediately start the game from the start screen #11
• go back to the onboarding screen after I've read the instructions #12
• be able to start the game from the instructions screen #13
• helper-avatar: refer to helper section at the end.

Map

• be able to go back to the onboarding screen
• see and interact with all the places I have access to #14
• see the progress on all places (stars) #15
• see new places automatically unlock when I have enough stars #16
• see a linear route between places that have been unlocked #17
• be able to go back to place I have already completed #18
• clearly see when a new place is available (for instance through glow, movement, sound) #20
• helper-avatar: refer to helper section at the end.

Place

• be able to navigate back to the map #21
• see how many stars are available to pick in this location #22
• see the progress so far (filled-in stars) #23
• be able to navigate to any of the interactions that are unlocked #24
• clearly see when a new interaction is available (for instance through glow, movement, sound) #25
• helper-avatar: refer to helper section at the end.

Interaction

• be presented with all the options for the interaction, and see them highlighted when correct #26
• be notified when all correct options are selected in the interaction and I completed the interaction #27
• be able to navigate back to the place #28
• start from scratch whenever I re-enter an interaction #29
• see the answers in a different order whenever I re-enter an interaction #30
• delete my progress on the interaction whenever I hit the Go Back button #31
• be awarded a star permanently when interaction is completed #32
• helper-avatar: refer to helper section at the end.

The Helper Avatar

The Helper Avatar is visible on every screen and as a user I want to

On the pre-game page:

• see a helper-avatar on the bottom of the screen #33
• get a reaction from the avatar (hint, motivation, etc) whenever I click it #34

On the map page:

• get instructions from the helper-avatar when I get to the map #35
• get a motivational phrase from the helper-avatar in regular intervals (i.e. how many stars do I need to unlock) #36
• get a new reaction from the helper-avatar (hint, motivation, etc) whenever I click on it. #34

On the place page:

• see indications from the helper-avatar on what to do next (interact with x or y) #35

On the Interaction page:

• be shown feedback from the helper-avatar on every selection #37
• be able to click the helper-avatar for a hint. #34
• be able to see a hint when I select the wrong option. #37

Progressive Web App check-list

Full details here.

• Site is served over HTTPS.

• Pages are responsive on tablets & mobile devices.

• All app URLs load while offline -> Use a Service Worker.

• Metadata provided for Add to Home screen -> Add a Web App Manifest file to your project.

• First load fast even on 3G -> Ensure app achieves decent performance.

• Site works cross-browser.

• Page transitions don't feel like they block on the network -> Display skeleton screens.

• Each page has a URL.

Format for Interactions - For POs to refer to as templates

Format for Interactions

1. Interaction Title

Image for the interaction

2. Options - Multiple emotions to select from

• Emotion:
  • If wrong: Text asking player to look closer at a specific facial feature
  • If right: Confirmation message

---

Example

Lisa and Suzi

Image of a Situation of two girls playing

• Image asset

Options

• Arguing - Look at ther eyes, are they smiling.
• (+) Happy - Correct, they are smiling and happy.
• Sad - Look at ther eyes, are they smiling.
• Bored - Do they look like they have a lot of energy
• (+) Focused - Correct!
• (+) Interested - Correct!
• Tired - Do they look like they have a lot of energy
• Angry - Look at ther eyes, are they smiling.

Setup code-coverage

User Journey

As a user, I want to understand how other people feel and I want to be able to read facial expressions.

User needs
The user needs a way to understand others & respond to them so that he/she can connect socially.

Consequence Scanning

Key Consequences

Scroll down for full overview

Positives, to focus on:
• Improve social skills and ability to understand other people's feelings.
• Learn social cues and facial expressions.

Negatives, to mitigate:
• Failure to progress in the game, feeling like a loser.
• Unable to generalise the skills learnt, bring them to the real world.

Overview

Intended Positives

Unintended Positives

Unintended Negatives

start from scratch whenever I re-enter an interaction

get instructions from the helper-avatar on what to do next

Setup React Router

• Install React Router
• Reusable Link Component to navigate between pages

Create circleLinks to all places

• Create Place Circle Component

• Use .map to return as many card components as required

Component Setup

see how many stars are available to pick in this place

be able to navigate back to the map

User Stories

• As a user, I want to see the progress of the game
• As a user, I want to choose my character
• As a user, I want to collect rewards/trophies
• As a user, I want to play in different scenes that I am familiar with (playground, school, birthday party)
• As a user, I want structure, repetition and consistency
• As a user, I want a clear beginning and a clear end
• As a user, I need to be given a limited time per day to play the game (chosen by the parents)
• As a user, I want to get a notification before my time runs out
• As a user, I want to review my progress
• As a user, I want a helper avatar to give me hints when I'm stuck, explain the game to me and keep me motivated
• As a user, I want an explainer video at the start that shows me how this game is played
• As a user, I want visual and audio stimulation, but I want the game to work without audio
• As a user, I want to acquire friends as I progress in the game
• As a user, I want to decide my own path within the lower level scene (playground etc)
• As a user, I want the scenes to become more difficult as I progress through the game
• As a user, I want to be able to repeat the same challenges multiple times
• As a user, I want to play the game on a tablet

Optional/to be discussed:

• As a user, I want to be given a predefined route on the top level (city)

see the answers in a different order whenever I re-enter an interaction

delete my progress on the interaction whenever I go back to the place

Create netlify project and deploy

be awarded a star permanently when interaction is completed

• Test it

Explanation of app hierarchy/structure

The game has three main parts

1. A Map, showing different places to visit
2. Places (like playground) have different Interactions to complete
3. Interactions are situations to analyze and react to (in a playground these could be different groups of children to click on)

---

Map Level

• Only shows one place first, more will unlock as you gain stars.
• linear progression, but you can always go back and play a place again.
• Each place is shown with the acquired and empty stars(when it unlocks)

Place Level

• Collect stars with progress
• Each completed interaction inside a place adds a star
• The next place will unlock once you collect a certain no. of stars( we're thinking 2 or 3)
• This amount is less than the total amount of stars available from the place
• Progress reflected by colored/non-colored areas
• There is always a button to go back to map

Interaction Level

• Get one star when you finish an interaction
• Interactions can be repeated several times (answers will be in a different order each time you play)

Create Interaction Component

• Add render to App.js

• Add a state for star

Create Stars Component

be able to immediately start the game from the start screen

be able to navigate to any of the interactions that are unlocked

Discuss Component Structure and App Data system

• Since it is a game, we have a few states passing between components, and stored in localStorage as well.
• HUD and helper avatar are shared between pages, and are big tasks by themselves.
• the app is more of an architecture challenge than a UI challenge so this needs to be discussed at an early stage

clearly see when a new interaction is available (for instance through glow, movement, sound)

Core Goals and Stretch Goals

Core Goals

• As a user I want to see the progress of the game
• As a user I want to play in different scenes that I am familiar with (playground, school, birthday party)
• As a user I want structure, repetition and consistency, with a clear beginning and a clear end.
• As a user I want to have options, but not too many. (show one interaction first, then unlock more as they finish)
• As a user I want a helper avatar to give me hints when I'm stuck, explain the game to me and keep me motivated.
• As a user, I want access to an explainer page at the start (featuring an explainer video and/or bullet points)
• As a user I want to decide my own path within the lower level scene (playground etc)
• As a user, I want to play the game on a tablet

Stretch Goals

• As a user I want to choose my character.
• As a user I need to be given a limited time per day to play the game (chosen by the parents).
• As a user I want to get a notification before my time runs out.
• As a user I want to collect rewards/trophees.
• As a user I want to acquire friends as I progress in the game
• As a carer I want to review my progress.
• As a user I want visual and audio stimulation, but I want the game to work without audio
• As a user I want the scenes to become more difficult as I progress through the game (dependent on content)

be able to go back to a place I have already completed

see new places automatically unlock when I have enough stars

see and interact with all the places I have access to