robotutorllc / robotutor_2019 Goto Github PK
View Code? Open in Web Editor NEWMain code for RoboTutor. Uploaded 11/20/2018 to XPRIZE from RoboTutorLLC/RoboTutor.
License: Other
Main code for RoboTutor. Uploaded 11/20/2018 to XPRIZE from RoboTutorLLC/RoboTutor.
License: Other
From @JackMostow on March 2, 2017 19:43
Addition prompts at e.g. 15m38s in 2017_03_02_18_51_21.mp4 say 'gusa kwenye' without shape name.
Copied from original issue: synaptek/RoboTutor#308
From @JackMostow on April 26, 2017 22:21
Got Download > RoboTutor > math > button_data.json
from download to jack tablet > RoboTutor > math https://drive.google.com/drive/folders/0B2NLAaJNgy1CSzMzZ3ZYUDJHS1k?usp=sharing
Got Download > RoboTutor > assets > add_subtract >
asm_1.json .. asm_26.json (except asm_6.json and asm_25.json that don't exist) from download to jack tablet > RoboTutor > assets > add_subtract https://drive.google.com/drive/folders/0B2NLAaJNgy1CeVJpNmZtc3JybHM?usp=sharing
CUSTOM menu shows the asm_1.. buttons -- see screenshot at https://drive.google.com/file/d/0B2NLAaJNgy1CVGZDU0Y0Zkd2SDA/view?usp=sharing
asm_1 0+1:
Tapping on asm_1 doesn't work
Assess menu hung after tap Repeat.
Kill job, relaunch RoboTutor, switch to English, pick math
Debugger menu is gone!
LAUNCH --> read Counting Cats
RoboTutor/app/src/main/assets/tutors/add_subtract/assets/asm_1.json
button_data.json at https://drive.google.com/drive/folders/0B2NLAaJNgy1CSzMzZ3ZYUDJHS1k
Other issues: eliminate empty bag over 1's column -- still appears with other bags
"Tap on the triangles 1 by 1" when only 1 triangle: liveable
Copied from original issue: synaptek/RoboTutor#342
From @JackMostow on March 8, 2017 15:28
Kid exits from Akira, picks repeat (I think), then crash at 36:06.
Copied from original issue: synaptek/RoboTutor#312
From @JackMostow on May 6, 2017 20:58
Kid writes reasonable 'j' but WRITE flashes weird misrecognition -- happens more than once (see below)
RoboTutor writes ‘saba’
13:27 “Vizuri sana!”
13:32 “jua”
Several reasonable but rejected attempts to write ‘j’
13:37
13:41 flash target ‘j’
13:45 reject (try to write it more like this)
13:50 “j”
14:00 “j”
14:04 weird artifact flashes in writing box
14:09 traces ‘j’
14:11 weird artifact
Copied from original issue: synaptek/RoboTutor#352
From @JackMostow on March 22, 2017 3:50
See:
Tapping Tutorial needs Swahili narration, not English!
Hello screen needs to elicit tapping, e.g. using RoboFinger.
Copied from original issue: synaptek/RoboTutor#320
From @JackMostow on March 2, 2017 19:4
Multiple usability issues:
Copied from original issue: synaptek/RoboTutor#306
From @JackMostow on April 25, 2017 15:27
Also:
+0, 0+
-0
x
Use prompts from Swahili translations
a. As stories?
+: easy to implement (may need to create screenshots)
-: less effective than interactive
b. In animation graphs?
+: more effective
-: harder to implement?
Copied from original issue: synaptek/RoboTutor#338
From @JackMostow on April 28, 2017 17:23
For 0+1=1, asm_1 plays the right scaffold in English:
Remember that 0 plus any number is just the number.
This prompt is not in the instruction table, which instead says:
0 plus any number is just the number. 0 kuongeza nambari nyingine ni nambari hiyo tu.
The Swahili scaffold for 0+1=1 is the wrong one – it’s the scaffold for 1+0=0:
Any number plus 0 is just the number. Nambari yoyote kuongeza 0 ni nambari hiyo tu. Indicate operand1, then +, then operand2, then operand1.
Instead it should play:
In fact 0 plus any number is just the number. 0 kuongeza nambari nyingine ni nambari hiyo tu. Indicate operand1, then +, then operand2, then result.
Copied from original issue: synaptek/RoboTutor#346
From @JackMostow on June 1, 2017 22:59
AT 19:59 in https://drive.google.com/drive/folders/0B2NLAaJNgy1CaFZjeF9mMGRITVU as shown at
.
Copied from original issue: synaptek/RoboTutor#379
From @JackMostow on March 2, 2017 19:50
at 19m42s in 2017_03_02_18_51_21.mp4 just after says 'sita' and deletes dots in balloon.
Copied from original issue: synaptek/RoboTutor#309
From @JackMostow on July 30, 2017 22:21
Can dictate 0 to 20, in order or randomly, but recalling them will require extending WRITE to hide stimulus.
Copied from original issue: synaptek/RoboTutor#402
From @JackMostow on March 12, 2017 15:21
a component to implement all counting activities including NumberScale, Tap Each, Tap N, and others
Copied from original issue: synaptek/RoboTutor#314
From @JackMostow on July 30, 2017 22:24
CCSS.MATH.CONTENT.K.CC.A.1 Learn "count list" 0..10 (like A..Z).
Add HIDE mode to READ (along with :HEAR, :ECHO, and :READ) to hide stimulus text until heard or tapped.
Could also add :LEAD ("Repeat after me") mode for RoboTutor to speak, kid to follow.
Then add activities to exploit new mode(s).
Copied from original issue: synaptek/RoboTutor#403
From @JackMostow on March 2, 2017 18:37
v.26 crashes just after 4m58s of 2017_03_02_18_24_53.mp4 after 2nd time saying to try to write more like this (letter i in a word).
Also at 13m10s 'i' in 'sifuri'.
Also at 11m51s in 2017_03_08_18_13_05.mp4 after writing sa of saba. a is in black with dot over it.
3rd attempt
Also in 2017_03_09_17_20_36 at 00m52s CRASH in WRITE.PNG on third attempt to write 's' in 'saba', while tracing 's'.
Copied from original issue: synaptek/RoboTutor#305
From @JackMostow on February 23, 2017 15:28
Sometimes RoboTutor turns DON green but not T in DON'T. This behavior is ok for hyphenated words but not contractions. Swahili matters more, and ' occurs only in ng', e.g. ng'ombe. Check how it's treated.
Copied from original issue: synaptek/RoboTutor#297
From @JackMostow on April 28, 2017 17:29
0.31.0.1 teal ghost square outlines in asm are virtually invisible against the almost-same-colored background, e.g. in asm_2 2-1 after tapping on square.
Make sure shapes in arithmetic are colors where even thin lines are visible against the background
Copied from original issue: synaptek/RoboTutor#347
From @JackMostow on March 29, 2017 21:10
BubblePop should but doesn't stop after 10 letters or shapes in random mode.
Copied from original issue: synaptek/RoboTutor#324
From @JackMostow on May 24, 2017 14:26
Import RoboTutor illustrated ready-to-import Swahili Shapes Stories at https://drive.google.com/drive/folders/0B-wWfDO90IJ4NHRSMmotdUlpejg?usp=sharing -- especially once we have picture-matching questions.
Copied from original issue: synaptek/RoboTutor#366
From @JackMostow on July 19, 2017 14:45
Clarify menu (we hope) by putting :) and :( side by side to resemble Yes-No, and :| below them.
Copied from original issue: synaptek/RoboTutor#400
From @JackMostow on April 2, 2017 2:8
This one is weird. The debug launcher flickered on and off and I think eventually turned to a gray screen.
Copied from original issue: synaptek/RoboTutor#327
From @JackMostow on April 24, 2017 1:54
Selected write 7-letter word karajibu (?).
Accepted after rewriting two characters.
Tapped Repeat icon.
Hung.
Eventually powered off on own despite being plugged in.
Copied from original issue: synaptek/RoboTutor#337
From @JackMostow on April 2, 2017 20:37
See 3 4/2/2017 screenshots in BUG VIDEOS AND SCREENSHOTS.
After finishing various activities, activity picker hung as shown.
But after selecting Close App, when I swiped up, saw the grayed RoboTutor window, and re-expanded it, it worked. I think that was true of the other reported hangs as well, which suggests that they were waiting for some event, and that Close App either provided the event or otherwise ended the waiting.
Copied from original issue: synaptek/RoboTutor#328
From @JackMostow on May 25, 2017 17:59
As shown in 2017_05_25_13_45_30 0.36.0.1 STUCK in letter dictation.mp4 at https://drive.google.com/drive/folders/0B2NLAaJNgy1CcW5FN2tyUGdIWVE, English letter dictation in write.ltr.uc.dic:level1:
Swahili write.ltr.uc.dic:level1 behaves similarly except no pink circle.
Copied from original issue: synaptek/RoboTutor#372
From @JackMostow on April 20, 2017 6:45
2017_04_12_18_12_37.mp4 at at 1m04s: crashes when kid exits at start of word_copy "saba" after using debug menu to start and quickly exit various activities
Copied from original issue: synaptek/RoboTutor#333
From @JackMostow on February 2, 2017 23:57
Enlarge digits and answer boxes, and move writing box closer to digit destination to make context easier to see while writing (or make cells big enough to write in)
Copied from original issue: synaptek/RoboTutor#273
From @JackMostow on April 28, 2017 19:14
(How) can RoboTutor assets be installed for testing on an emulator? The push.bat files via adb look for a USB device but don't find it.
Copied from original issue: synaptek/RoboTutor#348
From @JackMostow on February 19, 2017 20:10
I.e. each bubble speaks its prompt while doing something visual to associate itself with the audio, e.g. expand, vibrate, and/or change saturation or intensity.
Will enable multiple choice activities that map print to sound.
Copied from original issue: synaptek/RoboTutor#290
From @JackMostow on April 21, 2017 19:42
After starting and exiting ~10 custom math akira activities for number sense, "Let's do the same thing again" got stuck -- i.e. hung without the OS message. The screen was still gray ~1 minute later. The square button brought up the gray screen with 0.28.0.1. Only then did the OS message finally appear https://drive.google.com/file/d/0B2NLAaJNgy1CNEpuM3BFQy1FdEk/view?usp=sharing.
Copied from original issue: synaptek/RoboTutor#334
From @JackMostow on May 6, 2017 21:22
Just before accepting 'a', WRITE flashes distortion of it as 'd'
Copied from original issue: synaptek/RoboTutor#354
From @JackMostow on February 3, 2017 0:3
Use RoboFinger for clarity instead of yellow highlighting to elicit writing or tapping
Copied from original issue: synaptek/RoboTutor#276
From @JackMostow on March 13, 2017 19:6
Override initial prompt in BubblePop e.g. for phonemic awareness activities
Copied from original issue: synaptek/RoboTutor#315
From @JackMostow on March 3, 2017 15:58
Extend READ and WRITE to play back talking-mouth video clips of phonemes and syllables recorded as task #310.
Copied from original issue: synaptek/RoboTutor#311
From @JackMostow on March 15, 2017 22:9
2017_03_08_10_21_07
Skill: Reading & Writing
0:11 tap Custom; no menu
0:20 launch current tutor (1_33:a.READ) -> banner + gray screen
Same at 0:00 of 2017_03_02_09_43_17
Tap "Too hard" (:<)
0:39 tap Current: UC 4R_BP
Audio inaudible
0:47 "This is cutting off the bottom" of visual stimulus R at bottom of tablet
Then played HearBuilder
pick which of 3 words rhymes
or T/F whether rhymes
Copied from original issue: synaptek/RoboTutor#318
From @JackMostow on February 20, 2017 18:59
Art Graesser suggested a way to deal with kids resting their palm on the tablet: ignore any screen touches that aren't on target objects. We'd want to detect and log them, but filter them out of further processing -- except perhaps to ask the user to keep hands off.
Copied from original issue: synaptek/RoboTutor#292
From @JackMostow on March 17, 2017 21:54
Use StartsLike and EndsLike; highest frequency word with same first or last half as target word.
Rather than a single audio case, distinguish:
Translate and record "starts like" and "ends like"
add PROMPT_3_STARTS_LIKE
add PROMPT_3_ENDS_LIKE
In app/src/main/assets/tutors/akira/animator_graph.json add analogues of:
149 "constraint": "FTR_PROMPT3V",
150 "edge": "PROMPT_3V"
…
290 "edge": "POST_QUESTIONBOARD"
291 }
"PROMPT_3V": {
"type": "NODE",
"COMMENT": "When module is complete",
"maptype": "moduleMap",
"mapname": "INSTRUCT_3V",
"preenter": ["PROMPTPAUSE"],
"preexit": ["PROMPTRESUME", "NEXT_DATASET"],
"edges": [
{
"constraint": "FTR_EOD",
"edge": "POSTFINISHLINE"
},
{
"constraint": "",
"edge": "POST_QUESTIONBOARD"
}
]
},
"INSTRUCT_3V": {
"type": "MODULE",
"reuse":true,
"COMMENT": "????",
"tracks": [
{"type": "AUDIO", "command": "PLAY", "soundsource": "This says.mp3","soundpackage":"default" ,"repeat":false, "volume": "0.7", "mode":"flow", "features": ""},
{"type": "COMMAND", "id": "SAkira", "method": "indicate3Sign", "parms": "", "features":""},
{"type": "AUDIO", "command": "PLAY", "soundsource": "{{TestAudio}}.mp3","soundpackage":"xprize_words" ,"repeat":false, "volume": "0.7", "mode":"flow", "features": ""},
{"type": "COMMAND", "id": "SAkira", "method": "hideFinger", "parms": "", "features":""},
{"type": "COMMAND", "id": "SAkira", "method": "instruct_finger", "parms": "", "features":""},
{"type": "AUDIO", "command": "PLAY", "soundsource": "so tap here.mp3","soundpackage":"default" ,"repeat":false, "volume": "20.0", "mode":"flow", "features": ""}
]
},
"FTR_PROMPT3V": {
"type": "CONDITION",
"test": "PROMPT_3V"
},
In app/src/main/java/cmu/xprize/robotutor/tutorengine/widgets/core/TAkComponent.java add analogues of:
mTutor.setDelFeature(TCONST.PROMPT_3V);
In RoboTutor/app/src/main/java/cmu/xprize/robotutor/tutorengine/widgets/core/TAkComponent.java add analogues of:
case 3:
if(datasource[_dataIndex - 1].belowString.equals("audio")){//if it is an audio question
//TScope scope = mTutor.getScope();
//scope.addUpdateVar("TestAudio", new TString(getAboveString(datasource[_dataIndex - 1])));
mTutor.setAddFeature(TCONST.PROMPT_3V);
}else{
mTutor.setAddFeature(TCONST.PROMPT_3);
}
break;
In RoboTutor/util/src/main/java/cmu/xprize/util/TCONST.java add analogues of:
public static final String PROMPT_3V = "PROMPT_3V";
Copied from original issue: synaptek/RoboTutor#319
From @JackMostow on May 2, 2017 18:2
Tapping on every word in a sentence leaves them green but doesn't advance to the next sentence.
I think tapping once rereads the last word.
However, each additional tap causes RoboTutor to read an additional word from the text that follows.
**** Counting to 20 stuck after tapping on Counting, to, 20; Tap again “let”, “s”, “count”, … bizarre words from who knows where!
Asm_16: Counting by 10’s to 100
Tap on 10’s says 10; tap on ’s says ess
Tapping on 100 doesn’t advance, and tapping again says 100, 20, 30 without showing them – then displays 40, 50, 60
Tapping 60 says 60,60,80,90, then displays 100!
Tapping 100! Finally returns to menu
Counting by 5’s to 50
… tapping on 50 says 50, 10, 15, 20, 25,
Counting by 100’s to 1000 has same problems as other counting stories
Copied from original issue: synaptek/RoboTutor#351
From @JackMostow on July 31, 2017 17:49
Kevin – How feasible is it to make the Debugger menu scroll horizontally, not just vertically?
The English Reading & Writing menu has too many columns to fit on the screen.
Fortunately it’s the only over-wide menu I found in 0.46.2.1.
In Swahili it could be a problem if we need to add a dozen or so more columns in Math.
Copied from original issue: synaptek/RoboTutor#404
From @JackMostow on May 24, 2017 15:20
Omit English title "Swahili counting song"
Replace number words with numeral
Omit middle verse
Illustrate last page with picture of 10 fingers
Copied from original issue: synaptek/RoboTutor#368
From @JackMostow on March 26, 2017 16:26
...
2m23s tapped [ ], reducing window size
2m29s taps in TV screen area, restoring full window (unrelated?)
prompted ... if you don't like it, tap No
2m38s taps on No
2m39s CRASH "Unfortunately, login1 has stopped."
See in
FROM TABLETS > Camera > From SM-T337AZWA 1 (not screen video, only external) > 20170326_144756.mp4 AND
FROM TABLETS > Screen record > From ASR TREE 2016-06-10 (screen video is silent) > 2017_03_26_14_47_56.mp4
KID TESTING > 2017-03-21.. kid-test FaceLogin Bagamoyo > 20170326_144756.mp4 FaceLogin comments
Copied from original issue: synaptek/RoboTutor#321
From @JackMostow on February 2, 2017 23:59
Wait longer before displaying dot bags
Copied from original issue: synaptek/RoboTutor#274
From @JackMostow on April 28, 2017 22:23
[Moved READ bugs to issue #351.]
Change “let’s tap…” to “please tap…”
Asm_12 taught only -0 but scaffolds +0 with “Remember…”
Asm_13 scaffolds to “tap on the circles and count them 1 by 1”, but doesn’t display any – I think this bug comes after instruction.
Asm_15 18 – 6 is fine for 8-6, but then prompts to “tap on the circles to take them away” when the only circle is for the 1 in the 10’s place, which tapping doesn’t affect; need scaffold like for -0
Asm_9:
37-0 gives the -0 scaffold for 7 and 3 in 37-0, which is good
29-6 prompts to “tap on the circles to take them away”, which is fine for 9-6 but not for empty bag in 2 – implicit 0.
Asm_16:
10-10 prompts 0-0=0, but doesn’t do anything for 1-1, such as explain why it doesn’t have to because the resulting 0 is implicit.
13+4 for the 10 + implicit 0 prompts to “tap on the squares and count them 1 by 1” but doesn’t display any
60-40 scaffolds 0-0 ok but then prompts “tap on the stars to take them away” for 6-4 but doesn’t display any
20-0 scaffolds 0-0=0 ok but then repeats 0-0=0 for 2 minus implicit 0, where it is not appropriate
0+80 prompts to tap on non-existent stars
75+6 prompts to tap on non-existent circles for 5+6, then carries 1 automagically and prompts to tap on actual circles for 1+7 in 10’s column
76+15 prompts to tap on non-existent stars for 6+5 and for 1+7+1
84+3 correctly prompts to tap on squares for both columns
68+28 correctly prompts to tap on stars for both columns
89+5 correctly prompts to tap on triangles for both columns
9+16 prompts to tap on non-existent hexagons for 9+6 and 1+1
From: Jack Mostow
Sent: Friday, April 28, 2017 5:29 PM
To: 'Kevin Willows'
Subject: bugs in 32 math
** = serious
** Asm5 prompts to tap on the shapes (e.g. 6+1 triangles, 5+2 squares) but doesn’t show any.
Result dot bag is missing.
-----Original Message-----
From: Kevin Willows [mailto:[email protected]]
Sent: Friday, April 28, 2017 3:27 PM
To: Jack Mostow
Subject: link to ASM datasource
https://github.com/synaptek/RoboTutor/tree/master/app/src/main/assets/tutors/add_subtract/assets
Copied from original issue: synaptek/RoboTutor#349
From @JackMostow on April 2, 2017 2:3
See screenshots in FROM TABLETS > BUG VIDEOS AND SCREENSHOTS:
Finish problem, then hangs. But when recover from hang by terminating, asm screen is still there looking the same but (if I recall correctly) responds to <-.
It might happen when you answer without waiting for dot bag scaffolding.
Copied from original issue: synaptek/RoboTutor#326
From @JackMostow on April 27, 2017 0:15
The Assessment menu hung after asm_1 via CUSTOM menu, followed by a ~2-minute wait (to read email etc.). It didn't go gray, and continued to respond graphically (but not audio) to taps.
When I tried to take a screenshot of the Repeat icon in this state, I slipped and RoboTutor revived, but eventually returned to the same unresponsive state.
The only icon it responds to is the language button, which revived RoboTutor and gave the subject menu.
Now the regular Math menu is blank again.
LAUNCH brought up the initial story, which I tapped Back out of.
The Assessment menu is stuck again.
Accidentally knocking the tablet against something woke up Assessment enough to cycle through prompts and echo them when tapped, but it doesn't advance to the next screen.
BTW, I can highlight multiple -- even all -- the Assessment menu items by touching them simultaneously.
After exiting and restarting RoboTutor, its math menu is empty. Launch brings up Counting Cats.
Tapping out brings up the assessment menu, which at first speaks the tapped choice but doesn't advance.
Then it highlights but doesn't speak.
To recover I uninstalled RoboTutor and am reinstalling, which reinstalls all the assets (:<).
Copied from original issue: synaptek/RoboTutor#343
From @JackMostow on March 15, 2017 16:11
tap asm_1 just before CRASH at 12m44s in 2017_03_14_17_38_23
CRASH at 12m45s in 2017_03_14_17_38_23 after launch asm_1
tap asm_7 just before CRASH at 4m24s in 2017_03_14_18_39_51
Copied from original issue: synaptek/RoboTutor#317
From @JackMostow on March 3, 2017 15:57
Record talking-mouth video clips of Leonora speaking each Swahili phoneme and C*V syllable.
Copied from original issue: synaptek/RoboTutor#310
From @JackMostow on June 21, 2017 17:39
Allowing video stimuli and if possible targets in BubblePop and Akira would enable items with visual speech (talking-mouth video clips of phonemes or syllables) as stimuli and (in BubblePop) responses for items like "What sound does C make in CAT?" or "Tap on the letter that makes the sound /K/". Extension of audio stimuli and responses -- same in terms of treatment of time (e.g. sequential presentation), but adds visual display. Would want optional still frame when video is not playing.
Copied from original issue: synaptek/RoboTutor#397
From @JackMostow on March 9, 2017 11:28
The regular LaunchDebugger for SHAPES became empty after I installed a buggy custom data source with the non-existent "tutorDesc":"bubble_pop" instead of "bubble_pop_SHAPES". BTW, thanks for including the list of "tutor variants available" in button_data.json -- it enabled me to spot my error much sooner.
How does RoboTutor store standard (non-custom) data sources -- just in the .apk, or in files? If so, where?
How does it make persistent changes to the LaunchDebugger?
Copied from original issue: synaptek/RoboTutor#313
From @JackMostow on July 30, 2017 21:42
Add number sequence "stories" to use READ to practice counting for CCSS.MATH.CONTENT.K.CC.A.1.
Copied from original issue: synaptek/RoboTutor#401
From @JackMostow on May 24, 2017 15:26
The alphabet and counting songs and 1-word pages would look better in a larger font. How?
a. Modify READ to use large font whenever page contains only one word e.g. "mad" in m2m or a short string, e.g. "A B C D" or "1, 2, 3".
+: easy to implement
-: heuristic liable to be inappropriate now and then
b. Allow rich text.
+: more powerful
-: more work to implement
Copied from original issue: synaptek/RoboTutor#369
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.