Information about people’s current activity (their user state) and their mental task demand can be used for multiple purposes in meeting, lecture or office scenarios. Depending on the current user state and the level of task demand mobile communication devices such as cell-phones can configure themselves in a way that they notify their owner of an incoming event (e.g. a phone call) only, if this does not disturb him for instance. Furthermore information about user state and task demand of an audience can be used to provide feedback to a speaker about his talk. In this thesis a system is proposed which determines user state and task demand using electroencephalographic data (EEG data). EEG is recorded using either 16 scalp electrodes from a standard recording device which is usually used for clinical purposes, or a headband with only four electrodes over the pre-frontal and frontal cortex, which is much more comfortable to wear. The recorded data is then passed to a computer where features are extracted which represent the frequency content of the signals, features are preprocessed and finally passed to an artificial neural network or to a Support Vector Machine which predict user state and task demand. For the discrimination of the user states resting, listening, perceiving a presentation, reading an article in a magazine, summarizing the read article and performing arithmetic operations classification accuracies of 94.9% in session and subject dependent experiements, 58.9% in subject independent experiments and 62.7% in subject dependent but session independent experiments could be obtained. For the prediction of low and high task demand during the perception of a presentation accuracies of 92.2% in session and subject dependent experiements, 80.0% in subject independent experiments and 87.1% in subject dependent but session independent experiments were achieved. While all these experiments were obtained in offline scenarios, where data had been collected long before the system was trained and tested, also a prototype system has been developed which demonstrates the feasibility of user state identification and task demand assessment in real time.
35 Figures and Tables
Figure 2.1: Different anatomical parts of the human brain, with modifications from [Scientific Learning Cooperation, 1999]
Figure 2.11: Anatomical reference points which represent the starting points for finding the electrode positions defined by the 10-20 system (with modifications from [Zschocke, 1995]).
Figure 2.12: Electrode positions of the 10-20 system (with modifications from [Zschocke, 1995])
Figure 2.13: The Helmholtz double layer. Since the potential of the metal ϕm is smaller than the electrolyte potential ϕe, the hydrogen molecules of the water dipoles point the metal side.
Figure 2.17: Principle of non-polarized Ag/AgCl electrodes with KCl electrolyte. Left: excess of Cl− ions in the electrolyte, right: excess of K+ ions in the electrolyte.
Figure 2.18: Principal of differential EEG measurement. See text for the explanation.
Figure 2.20: Main components of an EEG amplifier.
Table 2.2: Different EEG frequency bands according to [Schmidt and Thews, 1997]
Figure 2.23: Functional cortex divisions according to [Schmidt and Thews, 1997] and [Dudel and Backhaus, 1996]. The cortex image is taken from [Scientific Learning Cooperation, 1999].
Figure 2.24: Oscillation mode (left) and transfer mode (right) of the thalamus.
Figure 2.3: Main components of a neuron. At the synapses information between neighboring neurons is exchanged. In this figure the axon is myelinated as in most cases which is a mean to speed up the signal transfer.
Figure 2.5: Dipole structure of cortical field potentials for a single neuron (a pyramid cell). With modifications from [Zschocke, 1995]
Figure 2.7: A closed dipole field generated by a star cell. Aff.: Afferences causing a negative pole at the outer end of the dendrites.
Figure 2.8: Volume conduction in the brain. Bold lines indicate a stronger impact on the measured signal, since the distance from the corresponding potential generator to the scalp electrode and thus the total resistance is smaller.
Figure 4.12: Topology of a multi-layer neural network for regression estimation. The hidden layer units use tanh activation functions while the output unit uses the identity as activation function as depicted by the symbols.
Figure 4.15: Two outlier examples (highlighted in gray) which make a linear separation of the classes impossible (left) and the solution of this problem using soft margins (right).
Figure 4.7: Update of prototypes during SOM training. While a prototype corresponding to a neuron which is far away from the BMU is altered little (mi), another prototype which has a neuron close to the BMU is altered much more (m j).
Figure 4.8: A single artificial neuron with a squashing function as activation function (e.g. g(h) = tanh(h)). See text for explanation.
Figure 5.3: The VarioportTM EEG amplifier. Left: the actual amplifier, right: the recorder which controls the amplifier, stores recorded data and established the connection to a computer.
Table 5.3: Mean, minimum and maximum amount of data in seconds over all recording sessions for
Table 5.4: Comparison of the amount of data in seconds available for each task demand level for data partitionings Eval1, Eval2 and EvalCombined for recording sessions (T2), (T3b), (T4) and (T6). The
Table 6.11: Mean normalized expected losses and mean accuracies (in braces) for the different sets of user states over all recording sessions of setups UD, UD4 and HB.
Table 6.14: Confusion matrices of the baseline system, the best system and the difference between both for setup UI.
Figure 6.15: Validation set accuracies for setups UD and UI for different values of k.
Figure 6.16: Validation set accuracies for setups UD and UI when different bin sizes for the FreqAvg method are used. The optimal system configuration for averaging and normalization (as determined
Figure 6.19: Comparison of the accuracies for the different recording sessions between the baseline system and the best system for all experimental setups.
Table 6.2: Confusion matrices for the different recording setups. The displayed matrices represent the
Table 6.20: Results for different normalization methods for the different experimental setups. The accuracies in the first column correspond to those shown in table 6.19 for an optimal value of k for averaging.
Table 6.22: Number of features selected per electrode for the different recording sessions of setup UD, when correlation based reduction to 80 features is performed.
Table 6.26: Parameters of the best system configuration for setups UD, SI and UI.
Table 6.4: Accuracies for different normalization methods for the different experimental setups.
Table 6.5: Comparison of results achieved without and with ICA-based eye activity removal from the data for setups UD and SI. All results are obtained with the optimal system configuration for averaging and normalization.
Figure 6.6: Validation set accuracies for setups UD (solid line) and UI (dashed line) for feature reduction with the FreqAvg method using different bin sizes. Note the non-equidistant scale of the x-axis.
Figure 6.7: Validation set accuracies for setup UI for LDA-based feature reduction to different feature vector sizes. Note the non-equidistant scale of the x-axis.
Table 6.9: Accuracies for setup UD setup UD4 and setup HB for the discrimination of all six user states.
Download Full PDF Version (Non-Commercial Use)