This web page provides selections of code used for experiments in my
This code is not well documented or elegant, but some may prefer it to
reading the psudo-code in text. Feel free to use or build upon
anything here, but please do cite the dissertation (or the original
paper) if you find it useful.
All domains are detailed in Chapter 3.
3D Mountain Car
These two files describe the environment (domain) which can be used in
Keepaway can be downloaded from here,
along with tutorials for instaliation and programming Keepaway agents.
Inaccurate 3 vs. 2 and Inaccurate 4 vs. 3
To change the default 3 vs. 2 or 4 vs. 3 games so that Keepers have
inaccurate passing actuators, search for the line
ACT->putCommandInQueue( soc = directPass( tmPos, PASS_NORMAL ) );
in KeepawayPlayer.cc, and replace it with the line
ACT->putCommandInQueue( soc = directPass( tmPos, PASS_FAST ) );
3 vs. 2 XOR Keepaway
To change the default 3 vs. 2 game to XOR Keepaway, KeepawayPlayer.cc
should be replaced with this file and
the changes described here in should be made to
WorldModelUpdate.cc and WorldModel.h.
A Java implementation of the environment can be found here.
The Knight Joust Java implementation is here.
Value function transfer: CMAC transfer functional for 3 vs. 2 Keepaway
Relies on Version 2.0
of the Tile Coding Software as the CMAC implementation. These two functions should be added to CMAC.cc
(and CMAC.h) and then called after CMAC/RBF weights are loaded into
a target task player from a source task player, but before learing
in the target task has started.
Policy Transfer for NEAT (neural network action selectors)
After training a source population of neural networks, the network
structure needs to be augmented for the target task, and weights
need to be set. This program was used to change a neural network
(called a genome by NEAT) that was trained in the 2-job-type
Server Job Scheduling task so that it is appropriate for the
4-job-type Server Job Scheduling task. The program is called once
and processes the population of neural networks, as output by NEAT.