Hyperscanning

Hyperscanning is a neuroimaging technique that involves simultaneously recording brain activity from multiple individuals engaged in social interactions or joint tasks. Unlike traditional neuroimaging methods that focus on individual brains, hyperscanning allows researchers to investigate the neural dynamics of social interactions, cooperation, competition, and communication. This is a relatively novel neuroimaging technique that opens new possibilities for researchers outside of the typical neuroscience field and paradigm-shifting applications in BCI, neuromarketing, and others.

Imaging Techniques

Common neuroimaging techniques used in hyperscanning include functional magnetic resonance imaging (fMRI), electroencephalography (EEG), and functional near-infrared spectroscopy (fNIRS). You can find a comparison of various imaging techniques in the Methods of Brain Activity Measurements tutorial (link). While fMRI was probably the first one to be used in hyperscanning measurements, EEG and fNIRS are more applicable in most measurement settings because they are more portable and affordable. This is especially important when measuring and studying interaction between people as these techniques allow more freedom of movement or limit the interaction in other ways.

Data synchronization. It is important to note that the signals coming from neuroimaging devices or other devices that are used to monitor the activity of the people are synchronized. This allows observing how people react to some common stimulus, be it some sound or video, how brain activity of some person is affected by the actions of another person, etc. Hence, having data synchronized in time is crucial for hyperscanning experiments and measurement systems. BrainAccess software uses a Lab Streaming Layer protocol that ensures the synchronization of EEG data coming from different devices.

Applications

Social Neuroscience

Hyperscanning is a relatively new measurement tool for social neuroscience. It provides insights into how the brain processes social information and coordinates with others in real-time. It has been used to study a wide variety of cognitive and social processes such as:

Joint attention: Joint attention is the ability to focus on the same object or event as another person. Hyperscanning studies have shown that joint attention is associated with increased synchronization of brain activity between the two individuals. Brain synchronization measure is one of the most widely used parameters to estimate the person’s engagement with another person or within a group and will be discussed in more detail later.

Social cognition: Social cognition is the ability to understand the thoughts, feelings, and intentions of others. Hyperscanning studies have shown that social cognition is associated with increased activation of mirror neurons in the brain.

Communication: Communication involves the exchange of information between individuals. Hyperscanning studies have shown that communication is associated with increased synchronization of brain activity between two individuals.

Cooperation: Hyperscanning studies have shown that working together towards a common goal is associated with increased synchronization of brain activity between the two individuals.

Clinical Research

Hyperscanning is applied in clinical research to study social impairments in conditions such as autism spectrum disorder (ASD) and schizophrenia. While in the classic diagnostic approaches, the brain activity measurements are made either passively or when a person is performing certain cognitive tasks, hyperscanning allows observing people in a social setting interacting with others and hence expanding diagnostic capabilities. Potentially, these techniques could be used as part of the treatment in the future as well by providing neurofeedback for the patients.

BCI, Education, Neuromarketing, and Other

While hyperscanning is mostly employed in social neuroscience, it is expected that the hyperscanning paradigm will extend to other fields such as BCI and neuromarketing. A couple of specific examples are given below just to get a glimpse of the breadth of hyperscanning applications.

A schematic diagram of a hyperscanning setup where multiple BrainAccess HALO EEG headbands are used to monitor a group of people interacting with a person in a teacher/lecturer role and/or video material.

Performing arts. The brain activity of musicians in a band or orchestra can be measured simultaneously when they are performing. [1] Their brain synchronization level could be measured and this can be used, for example, to adjust some parameters of their instruments. 

Education process monitoring. Brain synchronization level could be measured between the teacher and the students in order to evaluate the engagement level of the students to certain teaching styles, materials, and others. [2] This can be extended to other teaching settings like meditation studios, schools, etc.

Music/video preference evaluation. Brain synchronization can be measured not only between the people but with the stimulus as well. A study was made that measured people’s brain synchronization with various songs. A correlation was found between synchronization level and how well the songs performed in terms of sales later on. [3]

Brain Synchronization

Brain synchronization measures are typically used to study the response and interaction of the brains in hyperscanning setups. These measurements essentially evaluate how related one’s brain activity is to another person or stimulus. Commonly used EEG brain synchronization measures are coherence, phase locking value, cross-correlation, mutual information, and others. An appropriate measure should be chosen for synchronization estimation depending on what social or cognitive processes are being studied and the details of the experimental setup. Some measures can be more sensitive to signal power synchronization while others are to phase locking. More advanced measures involve more general dependence and causality estimations.

Challenges

As hyperscanning allows simultaneous measurements of multiple people while they are interacting it also brings more complexity when compared to single-person observations. Some of these challenges are outlined below.

Technical aspects. As mentioned previously data synchronization coming from different sensors is essential in hyperscanning. The Lab Streaming Layer protocol can be used to address this but not all devices support it.

Motion and other artifacts. As hyperscanning is typically performed in a less constrained environment and interacts with each other, people may talk or move otherwise and this for example produces many artifacts in EEG recordings. Hence, well-designed algorithms have to be used to clean these artifacts or at least to detect them and exclude them from further calculations.

Experimental Design. Designing experiments that capture valid social interactions while maintaining experimental control is a challenge. Striking a balance between experimental rigor and the naturalistic aspects of social behavior is crucial for the meaningful interpretation of hyperscanning results. In the future, hyperscanning experiments will probably involve many sensors that would allow us to observe and quantify interactions such as 3D cameras, localizing microphones, and others.

[1] U. Lindenberger et. al.  “Brains swinging in concert: cortical phase synchronization while playing guitar”, 2009.

[2] D.Bevilacqua et. al.  “Brain-to-Brain Synchrony and Learning Outcomes Vary by Student–Teacher Dynamics: Evidence from a Real-world Classroom Electroencephalography Study”, 2018.

[3] N. Leeuwis et. al. “A Sound Prediction: EEG-Based Neural Synchrony Predicts Online Music Streams”, 2021.