PSY318 : Computational Modeling

Location: WH519

Wed 2:10-4:30

Spring 2011


Thomas Palmeri

507 Wilson Hall

343-7900 (office)

office hours: Wed 1:10-2:00 or by appointment


“To have one’s hunches about how a simple combination of processes will behave repeated dashed by one’s own computer program is a humbling experience that no experimental psychologist should miss. Surprises are likely when the model has properties that are inherently difficult to understand, such as variability, parallelism, and nonlinearity - all, undoubtedly, properties of the brain.” – Hintzman (1990)


Course Overview

This course provides an overview of the how-tos and the whys of computational modeling. This course is not intended to be a general survey of computational models of human cognition. Instead, we will talk about what models are, why we use models, how to recognize good modeling versus bad modeling, how to implement a model, how to fit a model to data, how to evaluate the fit of a model, how to compare and contrast competing models, how to evaluate special cases of a model, and how to develop and test new models. We will talk about a number of real-world practical issues involved in implementing models. We will primarily talk about models that account for response probabilities, response times, and neurophysiology in a few selected domains. We will talk about why we develop and test models, when it is appropriate and inappropriate to test models, what kinds of choices are made when developing a model, what are the best ways to use modeling most effectively, and what we can learn from models. The techniques and issues we talk about apply to all kinds of modeling, from very abstract cognitive models to micro-level neural models.


By taking this course, you will be able to implement models, simulate models, make model predictions, fit models to data, and contrast competing models. This course will also give you the tools and background to take a more critical eye to modeling work you might read in the literature. We’ll cover a variety of practical issues like using MATLAB, random number generators, Monte Carlo simulations, using the high-performance computing facility at ACCRE, speeding up simulations, using bootstrapping techniques, and simulating differential equations.



Course Web Site

I have created a course web site that will archive copies of the syllabus, homework assignments, homework solutions, and course readings. This web site will be the definitive outline and schedule for the course.

On the web site, there are links to a number of readings. The readings are password protected so that they can only be accessed by people in the course. Use the login and password supplied in class.




All the examples and assignments will be in MATLAB. If you have never programmed in MATLAB before, you will need to learn at least the basics of MATLAB programming before taking the course. If you have some proficiency in C, Visual BASIC, or another high-level programming language, you should find it fairly easy to learn MATLAB. If you have never programmed at all before, you will probably need to do some work before taking this course. We will all try to learn the nuances and tricks of MATLAB programming from each other, but this will not be a course in how to program in MATLAB. Many people will have access to MATLAB in your laboratory. If you don’t have access to MATLAB, let me know, and I’ll see how students can buy into the Vanderbilt MATLAB site license.


Course Requirements

Class Participation (20%) : This is a graduate course. I want people to discuss, disagree, and ask questions. If you’re going to miss class, please let me know beforehand. Please do the readings for the week before class.

Homework Assignments (50%) : There will be a homework assignment most weeks. The assignments will usually involve implementing something in MATLAB. I will usually give a bunch of the skeleton code ahead of time, so the programming should not be too excessive. This is like a statistics course – there’s just no substitute for doing. I have no problem whatsoever with people talking about how to do the assignments and helping each other out, just so long as everyone does their own assignment.

Final Project (30%) : The final project gives you the opportunity to do something related to the course that’s also relevant to your own research. One obvious thing to do would be to implement a model in MATLAB (or R or C or Python) and test its predictions or fit it to some data. You could also develop and test a new model. Or you can systematically compare different ways of simulating a model or fitting a model to data. We’ll talk more about the project a bit later in the semester. You might start to think about what modeling you might want to do for the project. Feel free to talk with me. Talk to your advisor. I want you to do something that will be useful for your own research. We’ll set aside some time at the end of the semester for some short presentations of what you did. I want you to turn in the modeling results (e.g., equations, figures, graphs, and tables) along with a brief description of what you did; something around a few pages would be fine, but you can write more if it’s easier to write more. The primary goal is the implement/test a model using the techniques we’ve discussed in class. The goal is not to write a long paper about the model. The goal is to do something with a model.

People Sitting In : I encourage graduate students finished with their coursework as well as postdoctoral fellows to sit in on the course. I encourage you to do most of the readings and the homework assignments. I do hope that the level of in-class questions and discussion by folks sitting in will be commensurate with their level of outside-class effort. Any graduate students sitting in should officially audit the course (see Vay Welch for information on how to do that).



Course Readings

The primary text for this course is:

Busemeyer, J.R., & Diederich, A. (2010). Cognitive Modeling. Sage Publishing.


In addition, there will be other readings that will be distributed as PDF files available on the web site.


I also recommend the following (brand new) book as a supplemental text (a couple chapter are assigned as reading):

Lewandowski, S., & Farrell, S. (2011). Computational Modeling in Cognition: Principles and Practices. Sage Publishing.



Schedule of Topics

Note that the schedule of topics and the list of readings will change as we go through the course. An up-to-date list of topics and readings will be available on the course web site. Check the web site regularly!


Everything on this schedule is subject to change. Check the web site regularly.




Week 1  (Wed Jan 12)

Introduction to Computational (Cognitive) Modeling

What is a model? Why do we model? How do we model? What do we want to learn from models?



Busemeyer & Diederich, Chapter 1


Further Readings:

Lewandowsky & Farrell, Chapter 1

Learn MATLAB by Mathworks (online)

Palmeri, T.J., & Cottrell, G. (2009). Modeling perceptual expertise. In I. Gauthier, D. Bub, & M. Tarr (Eds.), Perceptual Expertise: Bridging Brain and Behavior. Oxford University Press.


Powerpoint Slides:

Click Here for Week 1



If you have had little MATLAB experience, you might read through the MATLAB Tutorial:


MATLAB Example:

GraphExamples.m shows a simple line plot and bar graph


Some books on how to program in MATLAB:

MATLAB Programming for Engineers (Paperback) by Stephen J. Chapman

An Introduction to Programming and Numerical Methods in MATLAB (Paperback) by Stephen Otto, James P. Denier

Basics of MATLAB and Beyond by Andrew Knight


Some critiques and cautions about modeling:

Luce, R.D. (1995). Four tensions concerning mathematical modeling in psychology. Annual Review of Psychology, 46, 1-26.

Roberts, S. & Pashler, H. (2000) How persuasive is a good fit? A comment on theory testing. Psychological Review, 107, 358-367.

Uttal, W.R. (1990). On some two-way barriers between models and mechanisms. Perception & Psychophysics, 48, 188-203.


Recommended readings:

Platt, J.R. (1964). Strong inference. Science, 146, 347-353.

Marr, D. (1983). Vision. W.H. Freeman.





Week 2 (Wed Jan 19)

Implementing a Computational Model

Using the similarity choice family of models as an example. Comparing observed data with model predictions. Measures of fit. Tricks and tips for effective modeling. Generating predictions of a model.



Busemeyer & Diederich, Chapter 2

Farrell, S., & Lewandowsky, S. (2010). Computational models as aids to better reasoning in psychology. Current Directions in Psychological Science, 19, 329-335.


Powerpoint Slides:

Click Here for Week 2



Click here for Homework 2


Files for Homework 2:


Click here for Homework 2 Solution


Further Readings:

Lewandowsky & Farrell, Chapter 2

Lamberts, K. (1997). Process models of categorization. In K. Lamberts and D. Shanks (Eds.), Knowledge, Concepts, and Categories, chapter 10, pages 371-403. MIT Press.

Logan, G.D. (2004). Cumulative progress in formal theories of attention. Annual Review of Psychology, 55, 207-234.

Luce, R. D. (1963). Detection and recognition. In R. D. Luce, R. R. Bush, & E. Galanter (Eds.), Handbook of mathematical psychology (pp. 103-189). New York: Wiley.

Nosofsky, R. (1985). Overall similarity and the identification of separable-dimension stimuli: A choice model analysis. Perception & Psychophysics, 38(5), 415-432.

Nosofsky, R.M. (1986). Attention, similarity, and the identification-categorization relationship. Journal of Experimental Psychology: General, 115, 39-57.

Nosofsky, R.M. (1988). Exemplar-based accounts of relations between classification, recognition, and typicality. Journal of Experimental Psychology: Learning, Memory, and Cognition, 14, 700-708.

Nosofsky, R.M. (1992). Exemplar-based approach to relating categorization, identification, and recognition. In F.G. Ashby (Ed.), Multidimensional Models of Perception and Cognition (pp. 363-393), Hillsdale, NJ: Erlbaum.

Shepard, R.N. (1957). Stimulus and response generalization: A stochastic model relating generalization to distance in psychological space. Psychometrika, 22, 325-345.

Shepard, R.N. (1980). Multidimensional scaling, tree-fitting, and clustering. Science, 210, 390=398.

Shepard, R.N. (1987). Toward a universal law of generalization for psychological science. Science, 237, 1317.

Viken, R.J., Treat, T.A., Nosofsky, R.M., McFall, R.M., & Palmeri, T.J. (2002). Modeling individual differences in perceptual and attentional processes related to bulimic symptoms. Journal of Abnormal Psychology, 111, 598-609.





Week 3 (Wed Jan 26)

Fitting Models to Data

Maximizing the fit of model predictions to observed data. Calculus-based methods, grid search, hill-climbing, Hook and Jeeves, simplex and subplex, simulated annealing, genetic algorithms. Effective (and ineffective) model fitting techniques. Interpreting parameters. Using computational models as data analysis techniques.



Busemeyer & Diederich, Chapter 3


In-class Matlab Code: (unzip for Matlab code)


Powerpoint Slides:

Click Here for Week 3



Click here for Homework 3


Files for Homework 3:


Click here for Homework 3 Solution


Further Readings:

Busemeyer, Chapter 3 Appendix

Estes, W.K. (2002). Psychonomic Society Keynote Address: Traps in the route to models of memory and decision, Psychonomic Bulletin & Review, 9, 3-25.

Kolda, T.G., Lewis, R.M., & Torczon, V. (2003). Optimization by direct search: New perspectives on some classical and modern methods. SIAM REVIEW, 45, 385-482.

Bogacz, R., & Cohen, J.D. Parameterization of connectionist models. Overview of subplex routine.

Kelley, C.T. (1999). Iterative Methods for Optimization. SIAM. Chapters 6-8.

Kirkpatrick, S., Gelatt, C.D., & Vecchi, M.P. Optimization by simulated annealing. Science, 220, 671-680.





Week 4 (Wed Feb 2)

Comparing Different Models

Continued discussion of techniques for fitting models to data. Nested versus nonnested models. Introduce techniques for hierarchical model testing to compare nested models. Maximum likelihood methods.



Busemeyer & Diederich, Chapter 5


In-class Matlab Code: (unzip for Matlab code)


Powerpoint Slides:

Click Here for Week 4



Click here for Homework 4


Files for Homework 4:

Click here for Homework 4 Solution


Further Readings:

Myung, I.J. (2003). Tutorial on maximum likelihood estimation. Journal of Mathematical Psychology, 47, 90–100.





Week 5 (Wed Feb 9)

Continued Discussion of Model Comparison Techniques

Maximum likelihood methods. Continue discussion of techniques for comparing both nested and nonnested models.



Lewandowsky & Farrell, Chapters 4


In-class Matlab Code: (unzip for Matlab code)


Powerpoint Slides:

Click Here for Week 5


Further Readings:

Bozdogan, H. (2000). Akaike's information criterion and recent developments in information complexity. Journal of Mathematical Psychology, 44, 62-91.

Myung, J.I., Navarro, D.J., Pitt, M.A. (2006). Model selection by normalized maximum likelihood. Journal of Mathematical Psychology, 50, 167–179.




Week 6 (Wed Feb 16)

More Model Comparison Techniques and Introduction to Monte Carlo Techniques

Techniques for comparing nonnested models. AIC and BIC. Introduction to Monte Carlo methods. Random number generators.



Lewandowsky & Farrell, Chapters 5

Zhang, W., & Luck, S.J. (2008). Discrete fixed-resolution representations in visual working memory. Nature, 453, 233-235. (Click here for supplemental information for Zhang and Luck)

Moler, C. Random Thoughts. (link)


In-class Matlab Code: (unzip for Matlab code) (Zhang and Luck simulation code) (psychometric function examples from class)


Powerpoint Slides:

Click Here for Week 6



Click here for Homework 6


Files for Homework 6:

Click here for Homework 6 Solution


Papers and MATLAB libraries for fitting Psychophysical Functions using mle and Bayesian techniques:

Wichman, F.A., & Hill, N.J. (2001). The psychometric function: I. Fitting, sampling, and goodness of fit. Perception & Psychophysics, 63, 1293-1313.

Wichman, F.A., & Hill, N.J. (2001). The psychometric function: II. Bootstrap-based confidence intervals and sampling. Perception & Psychophysics, 63, 1314-1329.

Kuss, M., Jakel, F., Wichmann, F.A. (2005). Bayesian inference for psychometric functions. Journal of Vision, 5, 478-492.

Click Here for their MATLAB libraries:

Code for the MATLAB random number generator:


More Readings:

Random Numbers (see Chapter 7) from Numeric Recipes





Week 7 (Wed Feb 23)

Monte Carlo Techniques

Nonparametric and parametric bootstrapping. Confidence intervals on parameters. An introduction to random walks, diffusion, and other stochastic accumulation models.



Busemeyer & Diederich, Chapter 4


In-class Matlab Code: (unzip for in-class Matlab code)


Powerpoint Slides:

Click Here for Week 7



No homework this week


More Readings:

Ratcliff, R., & Rouder, J.N. (1998). Modeling response times for two-choice decisions. Psychological Science, 9, 347-356.





Week 8 (Wed Mar 2)

Response Time Modeling

How to fit response time data. Parameter variability in cognitive models. Parameter tradeoffs.



Van Zandt, T. (2000). How to fit a response time distribution. Psychonomic Bulletin & Review, 7, 424-465.

See also Section 5 of Chapter 3 in Busemeyer & Diederich


In-class Matlab Code:


Powerpoint Slides:

Click Here for Week 8



Click here for Homework 8

Click here for Homework 8 solution





Week 9 (Wed Mar 16)

Fitting RT models to data

Fitting the diffusion model to behavioral data (with pdf or cdf). Using the diffusion model as a data analysis tool (FAST-DM, EZ-Diff). Trade-offs of simulation time and precision. Implementing and fitting simulation models. Equations versus simulations.



Rouder, J.N., & Speckman, P.L. (2004). An evaluation of the Vincentizing method of forming group-level response time distributions. Psychonomic Bulletin & Review, 11, 419-427.

Moler, C. Random Thoughts. (link)


In-class Matlab Code:


Powerpoint Slides:

Click Here for Week 9


Homework: No homework is due next week. But I would like you to take a look at the first problem of the homework assignment (which will be due the following week). It asks you to modify the diffusion model from one that relies on simulation to one that relies on an equation (implemented in CDFDif.m). To do this assignment, you will need to see how the code in mymodel_diff.m implements the fitting techniques we discussed in class and understand how to change the code to use CDFDif instead (we will discuss CDFDif next week at the beginning of class).

Click Here for Homework for next week


Files for Homework 9:


Tools for fitting diffusion models :

van Ravenzwaaij, D., & Oberauer, K. (2009). How to use the diffusion model: Parameter recovery of three methods: EZ, fast-dm, and DMAT. Journal of Mathematical Psychology, 53, 463-473.


Wagenmakers, E.-J., van der Maas, H.L.J., & Grasman, R.P.P.P. (2007). An EZ-diffusion model for response time and accuracy. Psychonomic Bulletin & Review, 14, 3-22.
The EZ-diffusion model has been implemented in JavaScript (click here), R (click here), and Excel (click here). A Matlab implementation is here (courtesy of Alex Petrov, The detailed review of the diffusion model parameters (i.e., which studies, which tables, which values) is available here.

Wagenmakers, E.-J., van der Maas, H.L.J., Dolan, C., & Grasman, R.P.P.P. (2008). EZ does it! Extensions of the EZ-diffusion model. Psychonomic Bulletin & Review, 15, 1229-1235.
The Robust EZ software can be found here, and Robust EZ software adjusted for batch processing can be found here.


Ratcliff, R., & Tuerlinckx, F. (2002). Estimating parameters of the diffusion model: Approaches to dealing with contaminant reaction times and parameter variability. Psychonomic Bulletin & Review, 9, 438-481. 

Tuerlinckx, F., Maris, E., Ratcliff, R., & De Boeck, P. (2001). A comparison of four methods for simulating the diffusion process. Behavior Research Methods, Instruments, & Computers, 33, 443-456.

Vandekerckhove, J., & Tuerlinckx, F. (2007). Fitting the Ratcliff diffusion model to experimental data. Psychonomic Bulletin & Review, 14, 1011-1026.

Vandekerckhove, J., & Tuerlinckx, F. (2008). Diffusion model analysis with MATLAB: A DMAT primer. Behavior Research Methods, 40, 61-72.

Click here for a link to the DMAT toolbox. (It can be downloaded here.)

Voss, J., & Voss, A. (2008). A fast numerical algorithm for the estimation of diffusion-model parameters. Journal of Mathematical Psychology, 52, 1–9. [Fast-DM web page]


Further Readings:

Ashby, F. G., Maddox, W. T., & Lee, W. W. (1994). On the dangers of averaging across subjects when using multidimensional scaling or the similarity-choice model.  Psychological Science, 5, 144-151.

Grasman, R.P.P.P., Wagenmakers, E.-J., van der Maas, H.L.J. (2009). On the mean and variance of response times under the diffusion model with an application to parameter estimation. Journal of Mathematical Psychology, 53, 55-68.

Mack, M.L., & Palmeri, T.J. (2010). Modeling categorization of scenes containing consistent versus inconsistent objects. Journal of Vision, 10(3):11, 1–11.

Myung, I.J., Kim, C., & Pitt, M.A. (2000). Toward an explanation of the power law artifact: Insights from response surface analysis. Memory & Cognition, 28, 832-840.

Nosofsky, R.M., Palmeri, T.J., & McKinley, S.C. (1994).  Rule-plus-exception model of classification learning.  Psychological Review, 101, 53-79.





Week 10 (Wed Mar 23)

Simulating Stochastic and Deterministic Differential Equations

Vincentizing and other approaches to averaging response time data. Possibly a more general discussion of averaging artifacts in modeling and how to avoid them. Deterministic and stochastic dynamical systems models. How to be brave in the face of some scary looking math. A brief introduction to differential equations. Simulating models defined in terms of differential equations. Applications to modeling behavior and neurophysiology.



Usher, M., & McClelland, J.L. (2001). The time course of perceptual choice: The leaky, competing accumulator model. Psychological Review, 108, 550-592. (esp. pp. 550-564)

Smith, P.L., & Ratcliff, R. (2004). Psychology and neurobiology of simple decisions. Trends in Neurosciences, 27, 161-168.


In-class Matlab Code:


Powerpoint Slides:

Click Here for Week 10



Click here for Homework 10


Files for Homework 10:

Click here for Homework 10 Solution


Further Readings:

Boucher, L., Palmeri, T.J., Logan, G.D., Schall, J.D. (2007). Inhibitory control in mind and brain: An interactive race model of countermanding saccades. Psychological Review.

Dayan, P., & Abbott, L.F. Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems. MIT Press.

Higham, D.J., (2001). An algorithmic introduction to numerical simulation of stochastic differential equations. SIAM Review, 43, 525-546.

Purcell, B.A., Heitz, R.P., Cohen, J.Y., Schall, J.D., Logan, G.D., & Palmeri, T.J. (2010). Neurally-constrained modeling of perceptual decision making. Psychological Review.

Smith, P.L. (2000). Stochastic dynamic models of response time and accuracy: A foundational primer. Journal of Mathematical Psychology, 44, 408-463.

Wilson, H.R. (1999). Spikes, Decision, and Actions: The Dynamical Foundations of Neuroscience. Oxford University Press.



MATLAB routines from Higham paper


More Readings:

Runge-Kutta from Numeric Recipes





Week 11 (Wed Mar 30)

Continued Discussion of simulating models defined by differential equations. When a good fit can be bad

Issues of overfitting, model complexity, model mimicry. Falsifiability. Penalizing models based on flexibility. Models need to predict and generalize, not just fit. Issues of model mimicry.


Pitt, M.A., & Myung, I.J. (2002). When a good fit can be bad. Trends in Cognitive Science, 6, 421-425.


In-class Matlab Code:


Powerpoint Slides:

Click Here for Week 11


Stochastic Runge-Kutta:

Honeycutt, R.L. (1992). Stochastic Runge-Kutta algorithms. I. White noise. Physical Review A, 45, 600-603.

Honeycutt, R.L. (1992). Stochastic Runge-Kutta algorithms. II. Colored noise. Physical Review A, 45, 604-610. (Errata)


Further Readings:

Lewandowsky & Farrell, Chapter 6

Efron, B., & Gong, G. (1983). A leisurely look at the bootstrap, the jackknife, and cross-validation. The American Statistician, 37, 36-48.

Myung, I.J., Pitt, M.A., & Kim, K. (2005). Model evaluation, testing and selection. In K. Lambert and R. Goldstone (Eds.), Handbook of Cognition. Sage Publication.

Zucchini, W. (2000). An introduction to model selection. Journal of Mathematical Psychology, 44, 41-61.





Week 12 (Wed Apr 6)

When a good fit can be bad. Fitting, Predicting, and Generalizing

Issues of overfitting, model complexity, model mimicry. Falsifiability. Penalizing models based on flexibility. Models need to predict and generalize, not just fit. Issues of model mimicry. Predicting versus fitting. Introduction to Bayesian



Busemeyer & Diederich, Chapter 5 Appendix

Myung, J.I., & Pitt, M.A. (2009). Optimal experimental design for model discrimination. Psychological Review, 116, 499-518.


Powerpoint Slides:

Click Here for Week 12


Further Readings:

Navarro, D.J., Pitt, M.A., & Myung, I.J. (2004). Assessing the distinguishability of models and the informativeness of data. Cognitive Psychology, 49, 47-84.

Pitt, M. A., Myung, I. J., & Zhang, S. (2002).  Toward a method of selecting among computational models of cognition. Psychological Review, 109(3), 472-491.

Pitt, M. A., Kim, W., Navarro, D. J. & Myung, J. I. (2006). Global model analysis by parameter space partitioning. Psychological Review, 113, 57-83.

Wagenmakers, E.-J., Ratcliff, R., Gomez, P., & Iverson, G.J. (2004). Assessing model mimicry using the parametric bootstrap. Journal of Mathematical Psychology, 48, 28-50.


PSP Method:

Parameter Space Partitioning Home Page





Week 13 (Wed Apr 13)

Introduction to Bayesian Analysis.

Brief introduction to Bayesian analyses and Monte Carlo Markov Chain methods (MCMC).



Busemeyer & Diederich, Chapter 6


Powerpoint Slides:

Click Here for Week 13


Further Readings:

Kruschke, J.K. (2011). Doing Bayesian Data Analysis: A Tutorial with R and Bugs. Academic Press.

Lee, M.D. (2008). Three case studies in the Bayesian analysis of cognitive models. Psychonomic Bulletin & Review, 15(1), 1-15.

Rouder J.N., & Lu J. (2005). An introduction to Bayesian hierarchical models with an application in the theory of signal detection. Psychonomic Bulletin & Review. 12, 573-604.





Week 14 (Wed Apr 20)

Bayesian Hierarchical Modeling.

Example of Bayesian hierarchical modeling.



Shiffrin, R.M., Lee, M.D., Wagenmakers, E.-J., & Kim, W.J. (2008). A survey of model evaluation approaches with a focus on hierarchical Bayesian methods. Cognitive Science, 32(8), 1248-1284.


In-class Matlab Code:


Powerpoint Slides:

Click Here for Week 14


Bayesian Hierarchical Modeling:

See Michael Lee’s web site:

Michael Lee and E.J. Wagenmakers have written A Practical Course in Bayesian Graphical Modeling


Further Readings:

Lee, M.D., & Vanpaemel, W. (2008). Exemplars, prototypes, similarities and rules in category representation: An example of hierarchical Bayesian analysis. Cognitive Science, 32(8), 1403-1424.

Andrieu, C., De Freitas, N., Doucet, A., Jordan, M. (2003). An Introduction to MCMC for Machine Learning. Machine Learning, 50, 5-43.