Joy Hirsch details exciting new insights into the fundamental brain functions that send and receive dynamic information between two communicating brains – the new frontier in neuroscience.
Humans are profoundly social and are naturally drawn to seek connecting communications with each other. However, little is known about the most fundamental brain functions that send and receive implicit and explicit dynamic information between communicating brains.
This question is the new question in neuroscience. Investigation of dynamic social interactions between two individuals extends the fundamental unit of behaviour from a single brain to a two-brain unit, the dyad.
The human dyad as a functional unit
Of all the challenges in the life sciences, understanding the single brain has been at the top of the list. The brain is considered the most complex organ of the body and it is estimated that there are over one billion neurons in the brain along with other cell types that make more than one trillion connections. These connections are modulated by a multiplicity of neurochemical factors that provide both excitatory and inhibitory inputs and outputs, and these cells are arranged in functional layers that form both local processing nodes and neural circuits that connect distributed nodes into specialised functional units. These units finally lead to behaviour, including cognitive processes, emotions, perceptions, memories, and goal-directed actions.
The complex link between brain and behaviour remains a theoretical frontier that extends beyond the physiology. The worldwide prevalence of brain and behaviour disorders at all stages of human development, from birth to end-of-life, constitutes significant medical, political, economic, legal, and quality of life issues that are currently not preventable or treatable due to lack of knowledge about the brain.
This knowledge gap deepens when we consider questions of how two communicating brains work together to achieve ‘wireless’ communications from one brain to another. Human brain processes and organisation are conventionally studied by functional magnetic resonance imaging (fMRI) where participants are imaged one at a time in the non-interactive and solo conditions of a scanner bore. However, neural events associated with natural interpersonal interactions between two individuals are not possible in a scanner environment.
Nonetheless, the human brain is exquisitely designed to detect and send social signals. These signals include reciprocal face-to-face information in real time, including eye-to-eye contact, dynamic facial expressions, and responsive gestures which do not occur in a scanning environment including only one individual.
However, interactive social behaviour involving dynamic communications between two individuals is a fundamental aspect of human socialisation. Largely due to technological limitations that limit conventional neuroimaging to single brains, little is known about the underlying neural circuits that regulate and modulate natural interpersonal interactions and communication between two communicating brains. Consequently, the neurophysiological mechanisms of psychiatric conditions with potentially profound deficits related to social interactions (e.g. autism spectrum disorders, schizophrenia, anxiety, and depression) remain absent of an understanding based on the underlying brain mechanisms.
The development of new technologies for brain imaging during communication between two individuals in ecologically valid conditions presents a particularly impactful opportunity to address these conditions for which the information gleaned through traditional neuroimaging is insufficient.
A foundational new technology for dynamic hyperscanning: functional near-infrared spectroscopy
An emerging neuroimaging technology, functional near-infrared spectroscopy (fNIRS), uses optodes secured in a cap worn on the head and is suitable for simultaneous use on two or more subjects in natural situations with limited tolerance to head movement. Like MRI, NIRS enables the observation of working neural systems in the intact human brain. This technology takes advantage of the physiological principle that active neural tissue recruits oxygenated blood in greater proportions than non-active neural tissue. The paramagnetic effects of deoxyhaemoglobin (deOxyHb) are reduced within the local microvasculature during this recruitment process. The signal amplification in MRI, referred to as the blood oxygen level-dependent (BOLD) signal1 is due to the reduced proportion of deOxyHb and the resultant decrease in paramagnetic effects. This haemodynamic signal is also detected by NIRS using spectral absorption2 which differentiates oxyhaemoglobin, OxyHb, and deOxyHb signals. Continuous lasers (and sometimes LEDs) emit specific wavelengths of light (around 780 and 830nm) that are diffused within the tissues under the skull with sufficient power to reach the underlying cortical areas. Head-mounted detectors measure the light that is reflected back to the surface. Calculations of the difference between the emitted light for each wavelength that is diffused into the tissues and the detected light that emerges at the surface, i.e. not absorbed, represents the absorption of light by OxyHb and deOxyHb, which is converted to concentration measures.3 Both fMRI and fNIRS signals reflect changes in brain blood flow and oxygenation which are coupled to underlying neuronal activity.
Fig. 2 shows a Shimadzu LABNIRS system with full-head configuration specialised for dual-brain imaging in natural conditions in which signals are acquired simultaneously for two individuals who are engaged in an interactive task. This system enables the acquisition of real-time fNIRS signals and eye-tracking acquisitions using glasses with scene and pupil cameras synchronised to the neural signals.
In this example, each cap includes 64 channels divided into two hemispheres for each subject. Cap configurations are flexible and can be modified according to experimental aims. Acquisition rates for NIRS signals range from 10 to 33ms with spatial resolution of approximately 3cm. This temporal resolution is well-suited for measures of connectivity between active brain regions within and across brains but, compared to fMRI, relatively compromised with respect to spatial resolution. Recent investigations of interpersonal interactions between two or more persons have demonstrated the efficacy of NIRS technology, which now leads the way toward a new neuroscience of natural cross-person communication.
The new neuroscience of dynamic coupling between two communicating brains
Breakthroughs in technology, computational algorithms, and experimental paradigms promise a quantum leap in future advances for developing a theoretical framework of the social brain and for treating the many psychiatric and neurological conditions in which social functioning is often compromised.
Models of wireless communication systems between two devices are well established in engineering and networking disciplines but rarely applied within the domain of biological communication systems. For example, the Open Systems Interconnection (OSI) model4 characterises functions of a wireless communication system, as in the case of two cell phones, that send and receive information within the paired unit. Two interacting signalling devices such as a pair of cell phones form a dyadic unit, and these devices provide a basic engineering framework for cross-brain signalling such as talking and listening between two individuals. As in the model for paired devices, we expect cross-brain interactions for human dyads to reflect the exchange of information.
Neural coupling occurs when the neural patterns of a sender match the neural patterns of a receiver. It has been proposed that these matched patterns represent shared neural processes that send and receive dynamic exchanges of information.5 Recent fNIRS hyperscanning studies report findings consistent with the hypothesis. For example, coherence between frontal cortical signals during co-operation on a computer task showed greater coherence between subjects who were competing on the same task, suggesting a neurophysiological substrate sensitive to interpersonal cues that are specific for co-operation.6
The emergence of leaders and followers has been studied in groups using simultaneous NIRS recordings of left frontal and parietal brain areas. Findings revealed that the emergence of a group leader was associated with increased neural synchronisation between the leader and the follower relative to synchronisation between followers.7
Neural coupling between dyads during direct eye-to-eye contact confirmed the distributed effects of real eye contact relative to direct gaze at a comparable picture.8
Synchrony between premotor areas of two brains participating in an imitation task of finger tapping was greater than self-paced finger tapping.9
Synchrony across brains has also been shown to represent an index of interpersonal interaction,6,9,10 and sychrony between dyads playing poker was greater when the opponent was a real person than when the opponent was really a computer even though the two players were face-to-face in both conditions.11
All contribute to the advancing field of two-person neuroscience12,13 and to the emerging proposition that neural synchronisation between partners underlies reciprocal interactions and the transfer of information. These and other similar seminal findings pioneer a new opportunity to investigate dynamic and interactive neural processes between dyad units.
Dynamic neural coupling when two brains talk to each other Neural mechanisms that mediate the diversity and depth of dynamic communications such as talking and listening remain understudied despite their evolutionary significance. The Interactive Brain Hypothesis proposes a general framework of neural mechanisms that serve interpersonal interactions.
We test the specific case of the Interactive Brain Hypothesis that canonical language areas, Broca’s and Wernicke’s, are dynamically coupled across brains during interaction. Functional near-infrared spectroscopy was employed to acquire simultaneous haemodynamic signals on partners who alternated between speaking and listening to each other while doing an object naming and description task with and without interaction. Cross-brain neural coupling determined by regional wavelet analyses was taken as the neurological marker of interpersonal interaction.
In partial support of the hypothesis, cross-brain coherence was greater during the interaction than the non-interaction condition for signals originating between the superior temporal gyrus (part of Wernicke’s Area) and the subcentral area. Within-brain activity also increased in Wernicke’s Area during interaction. However, the hypothesis was not supported for Broca’s Area.
These findings, including the coherence between the superior temporal gyrus and the subcentral area during talking and listening, suggest a dynamic mechanism for human-to-human communication that is only partially associated with canonical language functions. This result is consistent with a previously undescribed dedicated neural circuit that operates between communicating brains during real-time talking and listening interactions.
Dynamic neural coupling when two brains look at each other
The shared information hypothesis predicts that coupling between a specific neural system (such as face processing) will increase with shared information about faces. This prediction was confirmed in a two-person neuroimaging investigation using fNIRS during real face-to-face contact compared with viewing of a similar dynamic video face (also wearing fNIRS optodes).
Although both conditions included gaze at a naturally moving face, only the real face-to-face condition included reciprocal and socially informative face and eye movements. Comparison of neural coupling for the two conditions provides a test of the ‘shared information for face processing’ hypothesis. Standard general linear model (GLM) comparisons were performed prior to the neural coupling analyses. Consistent with the known functional sensitivity of the right temporal-parietal junction for social processing,14 the right temporal-parietal junction including the angular gyrus was more active during the real face viewing than the video face condition. As predicted, neural coupling for the real face-to-face condition compared to the video face condition was increased between angular gyrus and fusiform gyrus, a recognised component of the dynamic face processing system.15
These findings are consistent with a systems-specific dynamic neural coupling model for live face processing in which signals from canonical face and social processing systems are synchronised across two brains during face-related interactions. This result is consistent with a previously undescribed dedicated neural circuit that operates between communicating brains during real-time face-to-face interactions.
Together, these findings suggest a new class of dyadic neurocircuitry that functions between individuals who are wirelessly sending and receiving verbal and/or non-verbal information. It is important to determine if these systems are domain specific – i.e. the interaction functions are embedded within the specialised system such as language or face processing – or if they are domain general – i.e. specialised for interaction regardless of the processing system. In the two cases above, the specific synchronous pairs of brain regions that are modulated by interaction are found to be domain specific rather than domain general. However, more research is needed to test these alternative hypotheses.
Where do we go from here?
These pioneering studies illustrate potential future directions to investigate the dynamic relationships between interacting human brains using fNIRS. Hyperscanning studies of two interacting individuals document that well-known functional neural anatomy such as the components of the language system and the face processing system are observable using fNIRS. The additional features of cross-brain coherence and synchrony between two individuals can be investigated as novel probes to characterise uncharted questions that underlie the neural events of social interaction.
Potential benefits include a landmark breakthrough in methodology and technology leading to principles of neural organisation engaged during interpersonal and reciprocal interactions. Future studies may apply these new techniques to further understand the neural underpinnings of disorders of communication as well as how the neural underpinnings of social disability in developmental disorders deviate from typical development.
The co-occurrence of the BRAIN Initiative and the emergence of fNIRS as a mainstream neurotechnology catalyses the impactful potential to probe untapped neural systems specialised for interpersonal interactions between two or more individuals.
The primary advantages of fNIRS are related to signal acquisitions in natural environments not constrained by the limitations of a high magnetic field and uncomfortable imaging conditions that restrict head motion and communication. These advantages position fNIRS as a potential leading technology for a new frontier in neuroscience that aims to understand the neural correlates of social behaviour and cross-brain interpersonal interactions.16,17
Most of the pieces are in place for the realisation of this major advance. The key priorities toward this specific end goal include:
- Computational algorithms focused on signal components that represent the neural contributions of the signal separate from systemic and other non-neural components;18,19
- Full head coverage of optodes to acquire the dynamic activity of underlying long-range neural circuits; and
- Multimodal systems that synchronise combined acquisitions of electroencephalography, fNIRS, eye-tracking measurements, and facial classifications (for example).
Together, these new dual-brain comprehensive systems are set for a quantum leap in brain science and for understanding processes that underlie fundamental social and interactive behaviours.
This is work is partially supported by the NIH: R01 MH-107513 (PI JH) and R01 MH-111629 (PI JH). The content is solely the responsibility of the author and does not necessarily represent the official views of the National Institute of Health, NIH.
- 1 Ogawa S et al. (1990) Brain magnetic resonance imaging with contrast dependent on blood oxygenation. Proceedings of the National Academy of Sciences, 87(24), 9868-9872
- Jöbsis FF (1977) Noninvasive, infrared monitoring of cerebral and myocardial oxygen sufficiency and circulatory parameters. Science, 198(4323), 1264-1267
- Matcher SJ & Cooper CE (1994) Absolute quantification of deoxyhaemoglobin concentration in tissue near infrared spectroscopy. Physics in Medicine and Biology, 39(8), 1295
- Day JD & Zimmermann H (1983) The OSI reference model. Proceedings of the IEEE, 71(12), 1334-1340
- Hasson U et al. (2004) Intersubject synchronization of cortical activity during natural vision. Science, 303(5664), 1634-1640
- Cui X et al. (2012) NIRS-based hyperscanning reveals increased interpersonal coherence in superior frontal cortex during cooperation. Neuroimage, 59(3), 2430-2437
- Jiang J et al. (2015) Leader emergence through interpersonal neural synchronization. Proceedings of the National Academy of Sciences, 112(14), 4274-4279
- Hirsch J et al. (2017) Frontal temporal and parietal systems synchronize within and across brains during live eye-to-eye contact. Neuroimage, 157, 314-330
- Holper L et al. (2012) Between-brain connectivity during imitation measured by fNIRS. Neuroimage, 15;63(1), 212-22
- Cui X et al. (2011) A quantitative comparison of nirs and fmri across multiple cognitive tasks. Neuroimage, 54(4), 2808-2821
- Piva M et al. (2017) Distributed neural activity patterns during human-to-human competition. Frontiers in Human Neuroscience, 11, 571
- Konvalinka I & Roepstorff A (2012) The two-brain approach: How can mutually interacting brains teach us something about social interaction? Frontiers in Human Neuroscience, 6, 215
- Schilbach L et al. (2013) Toward a second-person neuroscience. Behavioral and Brain Sciences, 36(04), 393-414
- Carter RM & Huettel SA (2013) A nexus model of the temporal–parietal junction. Trends in Cognitive Sciences, 17(7), 328-336
- Pitcher D et al. (2011) The role of the occipital face area in the cortical face perception network. Experimental Brain Research, 209(4), 481-493
- Pinti P et al. (2015) Using Fiberless, Wearable fNIRS to Monitor Brain Activity in Real-world Cognitive Tasks. Journal of Visualized Experiments: JoVE, (106)
- Noah JA et al. (2015) fMRI validation of fNIRS measurements during a naturalistic task. Journal of Visualized Experiments: JoVE, (100)
- Kirilina E et al. (2012) The physiological origin of task-evoked systemic artefacts in functional near infrared spectroscopy. Neuroimage, 61(1), 70-81
- Zhang X et al. (2016) Separation of the global and local components in functional near-infrared spectroscopy signals using principal component spatial filtering. Neurophotonics, 3(1), 015004-01500
This is a commercial profile. An article by Joy Hirsch will appear in our upcoming issue of SciTech Europa Quarterly, due to be published in June 2018.