Research Proposal - Errorless Shaping of the p300 Event Related Potential for use with a Brain-Computer Interface
Jules A. Ochoa
University of North Texas
More Writings
Since the advent of computers, many technologies have been developed to aid humans with disabilities. Among the most extreme of these disabilities are amyotrophic lateral sclerosis (ALS) and other neurodegenerative disorders or events (i.e. stroke, injury, etc.) that leave the human in a partially or totally locked-in state. This presents multiple problems, one of which is the fact that, in this state, the individuals can no longer communicate their needs or desires. To remedy this problem, Farwell and Donchin developed a spelling system using a Brain-Computer Interface (BCI) that detects p300 Event-Related Potentials (ERP)(Farwell & Donchin, 1988). This system works by having the participant look at a matrix of characters in which the computer will flash rows and columns randomly. Whenever the row or column the participant is looking at flashes, a p300 ERP is seen. After many trials of this, the computer is able to narrow down which letter the participant is most likely looking at (Wolpaw, Birbaumer, McFarland, Pfurtscheller, & Vaughan, 2002). Since the first iteration of a BCI, there have been two major variations that use alternative brain potentials. These include BCIs that use Slow Cortical Potentials (SCPs) (Birbaumer et al., 1999; Kamiya, 2011; Kübler et al., 1999; Schneider et al., 1993; Sterman, 2000) and Auditory Evoked Potentials (AEPs) (Furdea et al., 2009; Hill & Schölkopf, 2012; Nijboer et al., 2008; Plourde, 2006).
Around the same time that Farwell and Donchin were developing their BCI, Milter, et al. and Sommer & Schweinberger were operantly conditioning the p300 to increase and decrease in amplitude (Miltner, Larbig, & Braun, 1986; Sommer & Schweinberger, 1992). The application of this to Farwell and Donchin’s work with BCIs, though, was never done. A potential explanation of this is that the use of the p300 in BCIs was overshadowed temporarily by Birbaumer et al. successfully creating a BCI using SCPs for two patients with late-stage ALS (Birbaumer et al., 1999). While these BCIs were successful at giving the patient a form of communication, the use of SCPs meant that each trial took between 4-6 seconds, compared to the 0.3 seconds for each trial using the p300 (Birbaumer et al., 1999; Kübler et al., 1999; Wolpaw et al., 2002). BCI researchers have now started using the p300 again because of its high reliability and low signal-to-noise ratio after a stimulus presentation, however, there has not been any work to explicitly operantly condition the p300 for BCI use even though there has been recognition of its importance (Wolpaw et al., 2002).
Though there has been much success with BCI technology so far, there is still the issue of decreasing the latency for the computer to identify an accurate command. In light of the p300 operant conditioning work Milter et al. and Sommer & Schweinberger did, and the fact that there has not been an application of this to BCI technology, the purpose of the current study is two-fold. The first purpose is to replicated Milter et al. and Sommer & Schweinberger using an errorless shaping procedure with increasing amplitude criterion to produce higher p300 amplitudes than could otherwise be obtained with typical trial and error learning. The second purpose is to use this high p300 amplitude with a BCI p300 speller to increase bit rate (typing speed) and maintain or decrease the baseline error rate.
Methods:
Participants & Settings: The participants for both groups will be healthy, typically developing adults aged 18 and older with no diagnosis of ALS or any other neuropathy. All participants will be naïve to BCI and EEG use and will undergo a basic eye exam before testing to ensure 20/20 vision (naturally or corrected). The experiments will all take place in the same quiet, low lit room with the participants sitting in a chair approximately 0.5 meters from a computer monitor that will display a matrix of all the letters in the alphabet, the numbers 0-9, and a backspace arrow. All participants will be wearing an EEG cap with 32 electrodes running off of the cap. The experimenter will be in the room to oversee the experiment and stop the experiment should the participant request it.
Measurement: The amplitude of the participant’s p300 brain response will be measured in real-time using a standard EEG cap with 32 active electrodes. All channels will be referenced to the FP electrode. The OpenVibe2.2 software will be used to capture the data and apply a low-pass and high-pass filter of 1 Hz and 40 Hz respectively. This same software will also control the presentation of the p300 speller matrix and feedback during the shaping process.
Experimental Design:
Groups – Participants will be divided into two groups – a shaping group and a control group. Both groups will go through the same three phases – baseline, training, and online – however, in the training session, only the shaping group will have feedback delivered contingent on higher p300 amplitude. There will not be any programmed feedback in baseline other than the natural contingencies of the computer correctly typing the character desired. The groups will also be evaluated in a multiple baseline design such that the control group will stay in the baseline phase longer than the shaping group. To minimize confounds in regards to the reinforcement rate during the training session, every participant in the control group will have their rate of reinforcement per minute yoked to a participant in the shaping group. These pairings will be determined by the order in which the participants are acquired for the experiment (i.e. participants 1 and 2, 3 and 4, 5 and 6, etc. will be paired).
Baseline – In the baseline phase, the participant will first be presented with a screen saying, “During this session, you will be shown a matrix of characters. There will be a combination of letters/numbers displayed in the text box at the top as a model for what you need to spell. Your job is to spell the sequence modeled by looking at the character you want the computer to type until the computer types a character in the top text box. Delete letters, if needed, by looking at the backspace character (left-facing arrow) until the computer deletes the character. Once the desired character has been typed, move to the next until you have successfully copied the model presented.” The next screen will present a matrix with all 26 letters, 0-9, a spacebar character, a backspace character, and the text box at the top, all of which have a black background. The characters making up the matrix will all be white. The computer will randomly flash different columns and rows, detecting which ones a p300 shows up on until a letter has been identified. Once a letter is typed there will be a 3 second time out before the next sequence is initiated. The participants in the shaping group will be required to type 3 sequences consisting of 15 characters in total. The participants in the control group will be required to type 6 sequences consisting of 30 characters in total.
Training – In the training phase, the screen will display a black background with a random character (letter, number, space, or backspace arrow) in the middle of the screen. At the top of the screen will be a blank, black text box that is outlined in white. The color of the characters (red or white) will be presented in an oddball paradigm defined by a difference in time as seen in Table 1. After each white stimuli is presented, if the participant’s p300 matched or exceeded the amplitude criterion, the whole screen will turn green, and the word “Good!” in black text will be displayed in the middle of the screen until the participant clicks the mouse to continue. If the p300 amplitude did not match or exceed the amplitude criterion, no change in the background or stimuli presentation will occur, no feedback will be given, and the trial will be repeated. The training sessions for both groups will consist of 8 blocks that will each last for 5 minutes. At the beginning of each block, the screen will display the instructions, “During this session, you will see a character presented in the [area of the screen respective to the block]. Most of these will be red, however, some will be white. Your job is to count, in your head, the number of white characters that appear. Whenever the screen turns green, that means you’re doing well!” At the end of the block, the participants will be asked to type the number of stimuli they covertly counted into a text box and what they think they are doing to make the screen turn green.
All of the characters on the screen will flash for 50 milliseconds with a 50-millisecond interstimulus interval (10 flashes per second). The average time between flashes of a white stimulus will vary depending on the block as outlined in Table 1. While the positions of all characters will remain constant within each block, the characters will change after every white character is presented. The position of the character the participant will be instructed to look at will be changed across the blocks as outlined in the column “Main Character Position” in Table 1. Other stimuli will be systematically incorporated into the screen as outlined in the column “Other Character Position” in Table 1. To ensure the participant is looking at the correct character, feedback will only be given if the amplitude criterion is met and what the computer identifies matches the predefined letter for that trial.
Online – In the online phase, the participant will first be presented with a screen saying, “During this session, you will be shown a matrix much like the one you saw in the first session. Like before, there will be a combination of letters/numbers displayed in the text box at the top as a model for what you need to spell. Look at the letter or number in the matrix that you need to type next until the computer types it for you. Once the desired character has been typed, move to the next until you have successfully copied the model presented.” The next screen will then present a full matrix consisting of all the characters in the Baseline phase matrix, however, now all the characters will be red, except for the rows/columns that are flashing white. The computer will randomly flash the columns and rows until a character choice has met some confidence criterion. The training phase is meant to increase the amplitude of the p300 every time a white character they are looking at is presented, thus decreasing the time needed to meet the desired confidence criterion, and increasing the typing speed. Once a letter is typed there will be a 3 second time out before the next sequence is initiated. All participants will be required to type the same 8 words consisting of a total of 40 characters.
References
Birbaumer, N., Ghanayim, N., Hinterberger, T., Iversen, I., Kotchoubey, B., Kübler, A., … Flor, H. (1999). A spelling device for the paralysed. Nature, 398(6725), 297–298. https://doi.org/10.1038/18581
Farwell, L. A., & Donchin, E. (1988). Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalography and Clinical Neurophysiology, 70(6), 510–523. https://doi.org/10.1016/0013-4694(88)90149-6
Furdea, A., Halder, S., Krusienski, D. J., Bross, D., Nijboer, F., Birbaumer, N., & Kübler, A. (2009). An auditory oddball (P300) spelling system for brain-computer interfaces. Psychophysiology, 46(3), 617–625. https://doi.org/10.1111/j.1469-8986.2008.00783.x
Hill, N. J., & Schölkopf, B. (2012). An online brain-computer interface based on shifting attention to concurrent streams of auditory stimuli. Journal of Neural Engineering, 9(2). https://doi.org/10.1088/1741-2560/9/2/026011
Kamiya, J. (2011). The first communications about operant conditioning of the EEG. Journal of Neurotherapy, 15(1), 65–73. https://doi.org/10.1080/10874208.2011.545764
Kübler, A., Kotchoubey, B., Hinterberger, T., Ghanayim, N., Perelmouter, J., Schauer, M., … Birbaumer, N. (1999). The thought translation device: A neurophysiological approach to communication in total motor paralysis. Experimental Brain Research, 124(2), 223–232. https://doi.org/10.1007/s002210050617
Miltner, W., Larbig, W., & Braun, C. (1986). Biofeedback of visual evoked potentials. International Journal of Neuroscience, 29(3–4), 291–303. https://doi.org/10.3109/00207458608986158
Nijboer, F., Furdea, A., Gunst, I., Mellinger, J., McFarland, D. J., Birbaumer, N., & Kübler, A. (2008). An auditory brain-computer interface (BCI). Journal of Neuroscience Methods, 167(1), 43–50. https://doi.org/10.1016/j.jneumeth.2007.02.009
Plourde, G. (2006). Auditory Evoked Potentials. Best Practice & Research Clinical Anaesthesiology, 129–139.
Schneider, F., Elbert, T., Heimann, H., Welker, A., Stetter, F., Mattes, R., … Mann, K. (1993). Self-regulation of slow cortical potentials in psychiatric patients: Alcohol dependency. Biofeedback and Self-Regulation, 18(1), 23–32. https://doi.org/10.1007/BF00999511
Sommer, W., & Schweinberger, S. (1992). Operant conditioning of P300. Biological Psychology, 33(1), 37–49. https://doi.org/10.1016/0301-0511(92)90004-E
Sterman, M. B. (2000). Basic Concepts and Clinical Findings in the Treatment of Seizure Disorders with EEG Operant Conditioning. Clinical EEG and Neuroscience, 31(1), 45–55. https://doi.org/10.1177/155005940003100111
Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., & Vaughan, T. M. (2002). “Brain-computer interfaces for communication and control, ” {Clinical} {Neurophysiology}. 113(6), 767–791.