Former Fellows

My principal focus over the last six months has been on the classification problem; my goal has been to attempt to improve on the approximately 90% accuracy of the neural network classification algorithm, when trained on the lab's (FPL) canonical classification data set. To this end, I have implemented several classification algorithms, and, in many cases, modified these algorithms in a manner so that their classification accuracy is maximized while execution time is minimal. Additionally, I have implemented several solutions in the simulation software the lab uses to train and test subjects; as a result of these extensions, the software is far more flexible.

The focus of much of my work with the classification problem has been on two related families of algorithms; one family is generally referred to as ensemble learning, while the other consists of processes known, broadly speaking, as kernel machines. Accordingly, I have written both a core adaptive boosting algorithm (AdaBoost) and several variants on the core AdaBoost algorithm, variously emphasizing one of speed, memory usage, or accuracy. AdaBoost, in fact, is less of a learning algorithm than a learning meta-algorithm; rather than being a complete definition of a learning algorithm, it allows for the choice of many different weak learners, choosing the weak learners most highly correlated with the training data set. I implemented AdaBoost with a set of decision stumps as weak learners, which has the benefit of reducing the number of pre-specified training variables to one. Though the initial results fall short of my goal of 90% classification or better, the potential for AdaBoost is still significant – perhaps a better choice for the weak learners would improve the classification accuracy.

Accordingly, I have read into the current literature on machine learning and classification; as current work with ensemble learning methods has included work on support vector machines (SVMs), a classification method that achieved high accuracy on tests conducted in the lab several years ago, I decided to implement an SVM in C++. My own SVM implementation, which is close to completion, is in fact significantly more powerful than many other SVM implementations: I have written it to allow for the study of current research in combining kernels and embedding in AdaBoost, as well as to allow quick extensibility. Further, it will likely run faster as well, being written in a compiled language.

Finally, I have used my experience with programming to write solutions to problems my coworkers have faced in their research, in particular, with the simulation software previously mentioned. In general, much of the software is hardcoded; that is, the program is hardwired to perform specific behaviors regardless of user preferences. As a result of both my work, and the work of a former mentor, end users – in this case, my coworkers – can now specify significantly more of the simulation’s actions.

Thus, I have found working in FPL to be enjoyable; my work has both challenged me and encouraged me to learn more to meet those challenges. My background in mathematics and computer science has been essential, as all of the classification algorithms I have worked with are written in mathematical notation. I have found that understanding them requires a strong degree of fluency in math, while implementing them requires a strong degree of fluency at least one programming language. Conversely, my work in FPL provides me with an applied perspective, which adds context to my studies at school.
Dates: Start: 07/2007 End: 07/2009

Intelligent Tutoring Systems Review Paper. This project is a collaboration between Dr. Heather Priest, Dr. Randy Spain, and myself. We are performing a literature review of intelligent tutoring systems (ITSs). Based on the literature, we hope to determine basic guidelines for the best way to design an ITS. For instance, what can you do pre-training, during-training, and post-training to better adapt an ITS to the individual learner? We are looking at the characteristics of an ITS, the different levels of adaptability existing systems possess, and several different learning theories and instructional design theories that provide the pedagogical foundation for ITSs. Specifically, we are focusing on the different components that can be made to be adaptive to each individual from four approaches—looking at adaptive elements in (1) a student model—characteristics of students themselves, (2) an expert model-SME representation of performance level/knowledge level that a student should achieve, (3) an assessment model, and (4) a pedagogical model.

Publications:

Billings, D.R., & Durlach, P.J. (2009, October). Mission completion time is sensitive to teleoperation performance during simulated reconnaissance missions with a micro-unmanned aerial vehicle. In Proceedings of the Human Factors and Ergonomics Society 53rd Annual Conference, San Antonio, TX, 19 - 23 October 2009.

Billings, D.R., & Durlach, P.J. (2008, July). Effects of input device and latency on performance while training to pilot a simulated micro-unmanned aerial vehicle. (ARI Technical Report 1234). Arlington, VA: U.S. Army Research Institute for the Behavioral & Social Sciences.

Billings, D.R., & Durlach, P.J. (2008). The effects of input device and latency on ability to effectively pilot a simulated micro-UAV. Proceedings of the Human Factors and Ergonomics Society 52nd Annual Conference, New York, NY, 22-26 September 2008.

Durlach, P.J., Neumann, J.L., & Billings, D.R. (2008, May). Training to operate a simulated micro-unmanned vehicle with continuous or discrete control. (ARI Technical Report 1229). Arlington, VA: U.S. Army Research Institute for the Behavioral & Social Sciences.

Jerome, C.J., Howey, A., & Billings, D.R. (2007, September). Heuristic evaluation of a user interface for a game-based simulation. (ARI Research Note 2007-08). Arlington, VA: U.S. Army Research Institute for the Behavioral & Social Sciences.
Dates: Start: 01/2007 End: 08/2010