GithubHelp home page GithubHelp logo

uwccdl / pss_rapm_model Goto Github PK

View Code? Open in Web Editor NEW
2.0 2.0 0.0 466.91 MB

A model of how reinforcement learning parameters affect performance on Raven's matrices

Common Lisp 30.28% NewLisp 0.01% Shell 0.05% Jupyter Notebook 2.24% Python 1.45% R 1.53% HTML 64.44%
act-r lisp sbcl cognitive-science neuroscience reinforcement-learning intelligence

pss_rapm_model's Introduction

A model of the role of reinforcement learning in RAPM problems

This is an ACT-R model that shows how reinforcement learning parameters (and, in particular, sensititivity to negative feedback) shapes the solving of Raven's Advanced Progressive Matrices (RAPM), a common test of fluid intelligence.

Experimental Data

The model was tested against behavioral and fMRI data collected by Stocco, Prat & Graham. The raw behavioral data in experiments 1-3 is saved in three folders, experiment1, experiment2, and experiment3. Comparisong accuracy and response time data across problem difficulty comes a different experiment from our lab, and is saved in the firestorm.txt text file.

Model Code

The model is developed in ACT-R 7.5, with the "old-style" devices written in Common Lisp. All of the model code is contained in four different Lisp files:

  1. rapm-model.lisp.
  2. rapm-device.lisp. This is the model's device, which encodes the RAPM problems and interacts with the model.
  3. rapm-problems.lisp. Contains a Lisp-like definition of a RAPM problem, together with functions to analyze them.
  4. rapm-simulations. Contains a set of functions to run large model simulations across parameter space

Loading and Running the Model

To run the model, follow these steps:

  1. Load ACT-R 7.5.x
  2. Load the rapm-device.lisp file. This will automatically load the rapm-problems.lisp file as well.
  3. Load the rapm-model.lisp file. This will load the ACT-R model code.
  4. Before running the model, initialze both model and device by calling the (rapm-reload) function. The function will properly initialize the device, reset the model, and connect the model's visual system to the device's interface.

Running Simulations

The rapm-simulations.lisp file contains many handy functions for running simulations and saving the results on a file. When saving results, each run on an simulated experiment (by default, 16 4-feature RAPM problems) will be saved as a single line.

Large-Scale Simulations

Large-scale simulations across parameter space are handled by generating multiple Lisp files that run simulations on different portions of the parameter space. The Python script generate-test.py will generate such files across a modifiable list of parameters (you might need to change the script's specific paths to fit your own system). The Python script generates a number of scripts corresponding to the different regions in which the parameter space is partitioned (by default, 64 different scripts). Each script is named test-<N>.lisp file, with N being a counter from 1 to the max number of partitions.

A series of four shell scripts manages the various Lisp files:

  1. run-sims.sh will launcha new instance of SBCL (by default; modify the Python script to use a different Lisp interpreter) on each Lisp test file. Each process' PID will be saved to a pids.txt file.
  2. kill-sims.sh will abort all the SBCL processes spawned by run-sims.sh. The script will kill, in series, all the processes with a PID listed in pids.txt (you should run this as sudo).
  3. merge.sh will merge all the generated files into a single text file, and the zip it.
  4. partial.sh will produce a file named partial.txt, which is like the file produced by merge.sh but carefully handles partially completed simulations. This is useful to inspect results before all the simulations are complete.

Unit Testing

No unit testing yet. But testing functions and testing problems are sparsed here and there.

Publications

The following publications are based on the model:

  • Stocco, A., Prat, C. S., & Graham, L. K. (2021). Individual Differences in Reward‐Based Learning Predict Fluid Reasoning Abilities. Cognitive Science, 45(2), e12941, https://doi.org/10.1111/cogs.12941.

pss_rapm_model's People

Contributors

therealdrdre avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar

pss_rapm_model's Issues

Diagonal rules

The model actually has problems in solving diagonal rules (distributions 3 and 4 in Matzen's taxonomy). The "distribution" rule is actually not included in the list used in the paper (the rule "same" is used instead).

Imperfect competitions in feature selection

Because we have multiple feature, the simple Pick/DontPick mechanisms outlined in the PSS paper would not work. When multiple features are in play, a "Dont" production is more likely to pick a bad (already explored) feature because it excludes only 1 out of N.
The best way to handle this would be through serial vetting, where the features are scanned in sequence until a "pick" is found.

Model fails after rule verification on second line

Sometimes the model fails and gets in a situation where no production fires. This seems to happen when the 'verify+not-successful' production fires on the second line. The model correctly returns to the phase where a new feature is picked, but, after the feature is attended (feature+encode-feature) and no solution has been found (check+solution-not-found-row or check+solution-not-found-row), the model stops.
The correct path would be to proceed with 'collect+by-row' (or +by-column), but the imaginal buffer does not match the requirements (specifically, that 'value' matches the cell the model is looking at, which should be 'zero; value is, instead, 'value two').

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.