Multi Agent Simulator

Overview
This study investigates learning on a hyper-redundant robotic manipulator and was done under the guidance of Dr. Kagan Tumer.

The simulator was created to be a general purpose multi-agent simulation framework and can be applied to a variety of interesting multi-agent problems. The simulator source can be downloaded here.

Robotic manipulator simulation

The goal of this study is to train the manipulator to track a point of interest with high precision using learned information. The robotic manipulator control scheme employs a unique approach where each segments acts as an agent capable of making independent, asynchronous decisions. An informed local search algorithm is used to generate training data, and neural networks are then trained to create a generalized input-output relationship for each segment. This study concluded that the neural networks are only able to reproduce a rough solution to the goal coordinate, hence a local search algorithm can be used to sufficiently minimize target error.

Media

• Generation of training data
• Trained neural network
• Trained neural network with informed search
• Trained neural network with informed search on all test data

Data
This simulator implements a multi-layer perceptron neural network that is trained using back propagation. Several tests were run to determine the number of hidden units which are required to produce an accurate model with minimal training time. Tests were run using a fixed number of training iterations to determine the best neural network structure.

It is clear from empirical data that a single hidden layer is not able to capture all of the complexities of the input-output relationship. In a single hidden layer network additional units have no effect on the accuracy of the model.

Adding a second hidden layer improves model accuracy up to a point. Accuracy starts to decline with increased complexity because more training iterations are required to reproduce the training data.

To minimize training time without sacrificing accuracy 2 hidden units were used in the first hidden layer and 6 units were used in the second hidden layer.

Training Results

This figure demonstrates the RMS error (on a series of test points) and training time with respect to the number of segments in the manipulator. As expected, training time is a linear function of the number of segments (as segments increase the number of neural networks increases). The error trend is less apparent and tends to gradually increase with the number of segments. It can also be noted that overall the error is relatively high and the raw output from the neural network is insufficient to accurately track an object.

To reduce error an informed search algorithm is applied in conjunction with the neural network output. A 20x improvement can be observed with this approach without additional gains in training time. This approach is not unlike biological learning in that a generalized input-output relationship is learned, and a local search is applied for new tasks.