2022 – 2026
Li AY, Mur M (in press). Neural networks need real-world behavior. Behavioral and Brain Sciences [pdf]
Jozwik KM, Kietzmann TC, Cichy RM, Kriegeskorte N, Mur M (2023). Deep neural networks and visuo-semantic models explain complementary components of human ventral-stream representational dynamics. J Neurosci 43: 1731-1741 [pdf]
t Hart B, et al. (2022). Neuromatch Academy: a 3-week, online summer school in computational neuroscience. J Open Source Educ 5: 118 [pdf]
2017-2021
Xiang J, Chen Y, Roussy M (2021). Behavioral inflexibility from a neuronal population perspective. J Neurosci 41: 5350-5352 [pdf]
Rafeh R, Gupta G (2020). Information-limiting correlations in neural populations: The devil is in the details. J Neurosci 40: 7782-7784 [pdf]
Diedrichsen J, Berlot E, Mur M, Schutt HH, Kriegeskorte N (2020). Comparing representational geometries using whitened unbiased-distance-matrix similarity. arXiv: 2007.02789 [pdf] +abstract
Representational similarity analysis (RSA) tests models of brain computation by investigating how neural activity patterns reflect experimental conditions. Instead of predicting activity patterns directly, the models predict the geometry of the representation, as defined by the representational dissimilarity matrix (RDM), which captures to what extent experimental conditions are associated with similar or dissimilar activity patterns. RSA therefore first quantifies the representational geometry by calculating a dissimilarity measure for each pair of conditions, and then compares the estimated representational dissimilarities to those predicted by each model. Here we address two central challenges of RSA: First, dissimilarity measures such as the Euclidean, Mahalanobis, and correlation distance, are biased by measurement noise, which can lead to incorrect inferences. Unbiased dissimilarity estimates can be obtained by crossvalidation, at the price of increased variance. Second, the pairwise dissimilarity estimates are not statistically independent, and ignoring this dependency makes model comparison statistically suboptimal. We present an analytical expression for the mean and (co)variance of both biased and unbiased estimators of the squared Euclidean and Mahalanobis distance, allowing us to quantify the bias-variance trade-off. We also use the analytical expression of the covariance of the dissimilarity estimates to whiten the RDM estimation errors. This results in a new criterion for RDM similarity, the whitened unbiased RDM cosine similarity (WUC), which allows for near-optimal model selection combined with robustness to correlated measurement noise.
Basti A, Mur M, Kriegeskorte N, Pizzella V, Marzetti L, Hauk O (2019). Analysing linear multivariate pattern transformations in neuroimaging data. PLoS one 14: e0223660 [pdf] + abstract
Most connectivity metrics in neuroimaging research reduce multivariate activity patterns in regions-of-interests (ROIs) to one dimension, which leads to a loss of information. Importantly, it prevents us from investigating the transformations between patterns in different ROIs. Here, we applied linear estimation theory in order to robustly estimate the linear transformations between multivariate fMRI patterns with a cross-validated ridge regression approach. We used three functional connectivity metrics that describe different features of these voxel-by-voxel mappings: goodness-of-fit, sparsity and pattern deformation. The goodness-of-fit describes the degree to which the patterns in an input region can be described as a linear transformation of patterns in an output region. The sparsity metric, which relies on a Monte Carlo procedure, was introduced in order to test whether the transformation mostly consists of one-to-one mappings between voxels in different regions. Furthermore, we defined a metric for pattern deformation, i.e. the degree to which the transformation rotates or rescales the input patterns. As a proof of concept, we applied these metrics to an event-related fMRI data set consisting of four subjects that has been used in previous studies. We focused on the transformations from early visual cortex (EVC) to inferior temporal cortex (ITC), fusiform face area (FFA) and parahippocampal place area (PPA). Our results suggest that the estimated linear mappings explain a significant amount of response variance in the three output ROIs. The transformation from EVC to ITC shows the highest goodness-of-fit, and those from EVC to FFA and PPA show the expected reference for faces and places as well as animate and inanimate objects, respectively. The pattern transformations are sparse, but sparsity is lower than would have been expected for one-to-one mappings, thus suggesting the presence of one-to-few voxel mappings. The mappings are also characterised by different levels of pattern deformations, thus indicating that the transformations differentially amplify or dampen certain dimensions of the input patterns. While our results are only based on a small number of subjects, they show that our pattern transformation metrics can describe novel aspects of multivariate functional connectivity in neuroimaging data.
Henriksson L, Mur M, Kriegeskorte N (2019). Rapid invariant encoding of scene layout in human OPA. Neuron 103: 161-171 [pdf] + abstract
Successful visual navigation requires a sense of the geometry of the local environment. How do our brains extract this information from retinal images? Here we visually presented scenes with all possible combinations of five scene-bounding elements (left, right, and back walls; ceiling; floor) to human subjects during functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG). The fMRI response patterns in the scene-responsive occipital place area (OPA) reflected scene layout with invariance to changes in surface texture. This result contrasted sharply with the primary visual cortex (V1), which reflected low-level image features of the stimuli, and the parahippocampal place area (PPA), which showed better texture than layout decoding. MEG indicated that the texture-invariant scene layout representation is computed from visual input within 100 ms, suggesting a rapid computational mechanism. Taken together, these results suggest that the cortical representation underlying our instant sense of the environmental geometry is located in the OPA.
Schmitz TW, Mur M, Aghourian M, Bédard M-A, Spreng RN (2018). Longitudinal Alzheimer’s degeneration reflects the spatial topography of cholinergic basal forebrain projections. Cell Rep 24: 38-46 [pdf] + abstract
The cholinergic neurons of the basal forebrain (BF) provide virtually all of the brain’s cortical and amygdalar cholinergic input. They are particularly vulnerable to neuropathology in early Alzheimer’s disease (AD) and may trigger the emergence of neuropathology in their cortico-amygdalar projection system through cholinergic denervation and trans-synaptic spreading of misfolded proteins. We examined whether longitudinal degeneration within the BF can explain longitudinal cortico-amygdalar degeneration in older human adults with abnormal cerebrospinal fluid biomarkers of AD neuropathology. We focused on two BF subregions, which are known to innervate cortico-amygdalar regions via two distinct macroscopic cholinergic projections. To further assess whether structural degeneration of these regions in AD reflects cholinergic denervation, we used the [18F] FEOBV radiotracer, which binds to cortico-amygdalar cholinergic terminals. We found that the two BF subregions explain spatially distinct patterns of cortico-amygdalar degeneration, which closely reflect their cholinergic projections, and overlap with [18F] FEOBV indices of cholinergic denervation.
Guo Y, Schmitz TW, Mur M, Ferreira CS, Anderson MC (2018). A supramodal role of the basal ganglia in memory and motor inhibition: meta-analytic evidence. Neuropsychol 108: 117-134 [pdf] + abstract
The ability to stop actions and thoughts is essential for goal-directed behaviour. Neuroimaging research has revealed that topping actions and thoughts engage similar cortical mechanisms, including the ventro- and dorso-lateral prefrontal cortex. However, whether and how these abilities require similar subcortical mechanisms remains unexplored. Specifically of interest are the basal ganglia, subcortical structures long-known for their motor functions, but less so for their role in cognition. To investigate the potential common mechanisms in the basal ganglia underlying action and thought stopping, we conducted meta-analyses using fMRI data from the Go/No-Go, Stop-signal, and Think/No-Think tasks. All three tasks require active stopping of prepotent actions or thoughts. To localise basal ganglia activations, we performed high-resolution manual segmentations of striatal subregions. We found that all three tasks recovered clusters in the basal ganglia, although the specific localisation of these clusters differed. Although the Go/No-Go and Stop-signal tasks are often interchangeably used for measuring action stopping, their cluster locations in the basal ganglia did not significantly overlap. These different localised clusters suggest that the Go/No-Go and Stop-signal tasks may recruit distinct basal ganglia stopping processes, and therefore should not be treated equivalently. More importantly, the basal ganglia cluster recovered from the Think/No-Think task largely co-localised with that from the Stop-signal task, but not the Go/No-Go task, possibly indicating that the Think/No-Think and Stop-signal tasks share a common striatal circuitry involved in the cancellation of unwanted thoughts and actions. The greater similarity of the Think/No-Think task to the Stop-Signal rather than Go/No-Go task also was echoed at the cortical level, which revealed highly overlapping and largely right lateralized set of regions including the anterior DLPFC, LPFC, pre-SMA and ACC. Overall, we provide novel evidence suggesting not only that the basal ganglia are critical for thought stopping, but also that they are involved in specific stopping subprocesses that can be engaged by tasks in different domains. These findings raise the possibility that the basal ganglia may be part of a supramodal network responsible for stopping unwanted processes more broadly.
Jozwik K, Kriegeskorte N, Storrs K, Mur M (2017). Deep convolutional neural networks outperform feature-based but not categorical models in explaining object similarity judgments. Front Psychol 8: 1726 [pdf] + abstract
Recent advances in Deep convolutional Neural Networks (DNNs) have enabled unprecedentedly accurate computational models of brain representations, and present an exciting opportunity to model diverse cognitive functions. State-of-the-art DNNs achieve human-level performance on object categorisation, but it is unclear how well they capture human behavior on complex cognitive tasks. Recent reports suggest that DNNs can explain significant variance in one such task, judging object similarity. Here, we extend these findings by replicating them for a rich set of object images, comparing performance across layers within two DNNs of different depths, and examining how the DNNs’ performance compares to that of non-computational “conceptual” models. Human observers performed similarity judgments for a set of 92 images of real-world objects. Representations of the same images were obtained in each of the layers of two DNNs of different depths (8-layer AlexNet and 16-layer VGG-16). To create conceptual models, other human observers generated visual-feature labels (e.g., “eye”) and category labels (e.g., “animal”) for the same image set. Feature labels were divided into parts, colors, textures and contours, while category labels were divided into subordinate, basic, and superordinate categories. We fitted models derived from the features, categories, and from each layer of each DNN to the similarity judgments, using representational similarity analysis to evaluate model performance. In both DNNs, similarity within the last layer explains most of the explainable variance in human similarity judgments. The last layer outperforms almost all feature-based models. Late and mid-level layers outperform some but not all feature-based models. Importantly, categorical models predict similarity judgments significantly better than any DNN layer. Our results provide further evidence for commonalities between DNNs and brain representations. Models derived from visual features other than object parts perform relatively poorly, perhaps because DNNs more comprehensively capture the colors, textures and contours which matter to human object perception. However, categorical models outperform DNNs, suggesting that further work may be needed to bring high-level semantic representations in DNNs closer to those extracted by humans. Modern DNNs explain similarity judgments remarkably well considering they were not trained on this task, and are promising models for many aspects of human cognition.
2012-2016
Pelekanos V, Mur M, Storrs KR (2016). Extracting object identity: Ventral or dorsal visual stream? J Neurosci 36: 6368-6370 [pdf]
Jozwik K, Kriegeskorte N, Mur M (2016). Visual features as stepping stones toward semantics: Explaining object similarity in IT and perception with non-negative least squares. Neuropsychol 15: 30199-30208 [pdf] + abstract
Object similarity, in brain representations and conscious perception, must reflect a combination of the visual appearance of the objects on the one hand and the categories the objects belong to on the other. Indeed, visual object features and category membership have each been shown to contribute to the object representation in human inferior temporal (IT) cortex, as well as to object-similarity judgments. However, the explanatory power of features and categories has not been directly compared. Here, we investigate whether the IT object representation and similarity judgments are best explained by a categorical or a feature-based model. We use rich models (4100 dimensions) generated by human observers for a set of 96 real-world object images. The
categorical model consists of a hierarchically nested set of category labels (such as “human”, “mammal”, and “animal”). The feature-based model includes both object parts (such as “eye”, “tail”, and “handle”) and other descriptive features (such as “circular”, “green”, and “stubbly”). We used non-negative least squares to fit the models to the brain representations (estimated from functional magnetic resonance imaging data) and to similarity judgments. Model performance was estimated on held-out images not used in fitting. Both models explained significant variance in IT and the amounts explained were not significantly different. The combined model did not explain significant additional IT variance, suggesting that it is the shared model variance (features correlated with categories, categories correlated with features) that best explains IT. The similarity judgments were almost fully explained by the categorical model, which explained significantly more variance than the feature-based model. The combined model did not explain significant additional variance in the similarity judgments. Our findings suggest that IT uses features that help to distinguish categories as stepping stones toward a semantic representation. Similarity judgments contain additional categorical variance that is not explained by visual features, reflecting a higher-level more purely semantic representation.
categorical model consists of a hierarchically nested set of category labels (such as “human”, “mammal”, and “animal”). The feature-based model includes both object parts (such as “eye”, “tail”, and “handle”) and other descriptive features (such as “circular”, “green”, and “stubbly”). We used non-negative least squares to fit the models to the brain representations (estimated from functional magnetic resonance imaging data) and to similarity judgments. Model performance was estimated on held-out images not used in fitting. Both models explained significant variance in IT and the amounts explained were not significantly different. The combined model did not explain significant additional IT variance, suggesting that it is the shared model variance (features correlated with categories, categories correlated with features) that best explains IT. The similarity judgments were almost fully explained by the categorical model, which explained significantly more variance than the feature-based model. The combined model did not explain significant additional variance in the similarity judgments. Our findings suggest that IT uses features that help to distinguish categories as stepping stones toward a semantic representation. Similarity judgments contain additional categorical variance that is not explained by visual features, reflecting a higher-level more purely semantic representation.
Henriksson L, Mur M, Kriegeskorte N (2015). Faciotopy – a face-feature map with face-like topology in the human occipital face area. Cortex 72: 156-167 [pdf] + abstract
The occipital face area (OFA) and fusiform face area (FFA) are brain regions thought to be specialized for face perception. However, their intrinsic functional organization and status as cortical areas with well-defined boundaries remains unclear. Here we test these regions for “faciotopy”, a particular hypothesis about their intrinsic functional organisation. A faciotopic area would contain a face-feature map on the cortical surface, where cortical patches represent face features and neighbouring patches represent features that are physically neighbouring in a face. The faciotopy hypothesis is motivated by the idea that face regions might develop from a retinotopic protomap and acquire their selectivity for face features through natural visual experience. Faces have a prototypical configuration of features, are usually perceived in a canonical upright orientation, and are frequently fixated in particular locations. To test the faciotopy hypothesis, we presented images of isolated face features at fixation to subjects during functional magnetic resonance imaging. The responses in V1 were best explained by low-level image properties of the stimuli. OFA, and to a lesser degree FFA, showed evidence for faciotopic organization. When a single patch of cortex was estimated for each face feature, the cortical distances between the feature patches reflected the physical distance between the features in a face. Faciotopy would be the first example, to our knowledge, of a cortical map reflecting the topology, not of a part of the organism itself (its retina in retinotopy, its body in somatotopy), but of an external object of particular perceptual significance.
Mur M (2014). What’s the difference between a tiger and a cat? From visual object to semantic concept via the perirhinal cortex. J Neurosci 34: 10462-10464 [pdf]
Mur M, Kriegeskorte N (2014). What’s there, distinctly, when and where? Nature Neurosci 17: 332-333 [pdf]
Liu N, Kriegeskorte N, Mur M, Hadj-Bouziane F, Luh WM, Tootell RBH, Ungerleider L (2013). Intrinsic structure of visual exemplar and category representations in the macaque brain. J Neurosci 33: 11346-11360 [pdf] + abstract
One of the most remarkable properties of the visual system is the ability to identify and categorize a wide variety of objects effortlessly. However, the underlying neural mechanisms remain elusive. Specifically, the question of how individual object information is represented and intrinsically organized is still poorly understood. To addressthis question, we presented images of isolated real-world objects spanning a wide range of categories to awake monkeys using a rapid event-related functional magnetic resonance imaging (fMRI) design
and analyzed the responses of multiple areas involved in object processing. We found that the multivoxel response patterns to individual exemplars in the inferior temporal (IT) cortex, especially area TE, encoded the animate-inanimate categorical division, with a subordinate cluster offaces withinthe animate category. In contrast,the individual exemplar representations in V4,the amygdala, and prefrontal cortex showed either no categorical structure, or a categorical structure different fromthat in IT cortex. Moreover, inthe IT face-selective regions (“face patches”), especially the anterior face patches, (1) the multivoxel response patterns to individual exemplars showed a categorical distinction between faces and nonface objects (i.e., body parts and inanimate objects), and (2) the regionally averaged
activations to individual exemplars showed face-selectivity and within-face exemplar-selectivity. Our findings demonstrate that, at both the single-exemplar and the population level, intrinsic object representation and categorization are organized hierarchically as one
moves anteriorly along the ventral pathway, reflecting both modular and distributed processing.
and analyzed the responses of multiple areas involved in object processing. We found that the multivoxel response patterns to individual exemplars in the inferior temporal (IT) cortex, especially area TE, encoded the animate-inanimate categorical division, with a subordinate cluster offaces withinthe animate category. In contrast,the individual exemplar representations in V4,the amygdala, and prefrontal cortex showed either no categorical structure, or a categorical structure different fromthat in IT cortex. Moreover, inthe IT face-selective regions (“face patches”), especially the anterior face patches, (1) the multivoxel response patterns to individual exemplars showed a categorical distinction between faces and nonface objects (i.e., body parts and inanimate objects), and (2) the regionally averaged
activations to individual exemplars showed face-selectivity and within-face exemplar-selectivity. Our findings demonstrate that, at both the single-exemplar and the population level, intrinsic object representation and categorization are organized hierarchically as one
moves anteriorly along the ventral pathway, reflecting both modular and distributed processing.
Mur M, Meys M, Bodurka J, Goebel R, Bandettini PA, Kriegeskorte N (2013). Human object-similarity judgments reflect and transcend the primate-IT object representation. Front Psychology 4: 128 [pdf] + abstract
Primate inferior temporal (IT) cortex is thought to contain a high-level representation of objects at the interface between vision and semantics. This suggests that the perceived similarity of real-world objects might be predicted from the IT representation. Here we show that objects that elicit similar activity patterns in human IT (hIT) tend to be judged as similar by humans. The IT representation explained the human judgments better than early visual cortex, other ventral-stream regions, and a range of computational models. Human similarity judgments exhibited category clusters that reflected several categorical divisions that are prevalent in the IT representation of both human and monkey, including the animate/inanimate and the face/body division. Human judgments also reflected the withincategory representation of IT. However, the judgments transcended the IT representation in that they introduced additional categorical divisions. In particular, human judgments emphasized human-related additional divisions between human and non-human animals and between man-made and natural objects. hIT was more similar to monkey IT than to human judgments. One interpretation is that IT has evolved visual-feature detectors that distinguish between animates and inanimates and between faces and bodies because these divisions are fundamental to survival and reproduction for all primate species, and that other brain systems serve to more flexibly introduce species-dependent and evolutionarily more recent divisions.
Goffaux V, Schiltz C, Mur M, Goebel R (2013). Local discriminability determines the strength of holistic processing for faces in the fusiform face area. Front Psychology 3: 604 [pdf] + abstract
Recent evidence suggests that the Fusiform Face Area (FFA) is not exclusively dedicated to the interactive processing of face features, but also contains neurons sensitive to local features. This suggests the existence of both interactive and local processing modes, consistent with recent behavioral findings that the strength of interactive feature processing (IFP) engages most strongly when similar features need to be disambiguated. Here we address whether the engagement of the FFA into interactive versus featural representational modes is governed by local feature discriminability. We scanned human participants while they matched target features within face pairs, independently of the context of distracter features. IFP was operationalized as the failure to match the target without being distracted by distracter features. Picture-plane inversion was used to disrupt IFP while preserving input properties.We found that FFA activation was comparably strong, irrespective of whether similar target features were embedded in dissimilar contexts(i.e., inducing robust IFP) or dissimilar target features were embedded in the same context (i.e., engaging local processing). Second, inversion decreased FFA activation to faces most robustly when similar target features were embedded in dissimilar contexts, indicating that FFA engages into IFP mainly when features cannot be disambiguated at a local level. Third, by means of Spearman rank correlation tests, we show that the local processing of feature differences in the FFA is supported to a large extent by the Occipital Face Area, the Lateral Occipital Complex, and early visual cortex, suggesting that these regions encode the local aspects of face information. The present findings confirm the co-existence of holistic and featural
representations in the FFA. Furthermore, they establish FFA as the main contributor to the
featural/holistic representational mode switches determined by local discriminability.
representations in the FFA. Furthermore, they establish FFA as the main contributor to the
featural/holistic representational mode switches determined by local discriminability.
Kriegeskorte N, Mur M (2012). Inverse MDS: Inferring dissimilarity structure from multiple item arrangements. Front Psychology 3: 245 [pdf] + abstract
The pairwise dissimilarities of a set of items can be intuitively visualized by a 2D arrangement of the items, in which the distances reflect the dissimilarities. Such an arrangement can be obtained by multidimensional scaling (MDS). We propose a method for the inverse process: inferring the pairwise dissimilarities from multiple 2D arrangements of items. Perceptual dissimilarities are classically measured using pairwise dissimilarity judgments. However, alternative methods including free sorting and 2D arrangements have previously been proposed. The present proposal is novel (a) in that the dissimilarity matrix is estimated by “inverse MDS” based on multiple arrangements of item subsets, and (b) in that the subsets are designed by an adaptive algorithm that aims to provide optimal evidence for the dissimilarity estimates. The subject arranges the items (represented as icons on a computer screen) by means of mouse drag-and-drop operations. The multi-arrangement method can be construed as a generalization of simpler methods: It reduces to pairwise dissimilarity judgments if each arrangement contains only two items, and to free sorting if the items are categorically arranged into discrete piles. Multi-arrangement combines the advantages of these methods. It is efficient (because the subject communicates many dissimilarity judgments with each mouse drag), psychologically attractive (because dissimilarities are judged in context), and can characterize continuous high-dimensional dissimilarity structures. We present two procedures for estimating the dissimilarity matrix: a simple weighted-aligned-average of the partial dissimilarity matrices and a computationally intensive algorithm, which estimates the dissimilarity matrix by iteratively minimizing the error of MDS-predictions of the subject’s arrangements. The Matlab code for interactive arrangement and dissimilarity estimation is available from the authors upon request.
Mur M, Ruff DA, Bodurka J, De Weerd P, Bandettini PA, Kriegeskorte N (2012). Categorical, yet graded – single-image activation profiles of human category-selective cortical regions. J Neurosci 32: 8649-8662 [pdf] + abstract
Human inferior temporal cortex contains category-selective visual regions, including the fusiform face area (FFA) and the parahippocampal place area (PPA). These regions redefined by their greater category-average activation to the preferred category (faces and places, espectively) relative to nonpreferred categories. The approach of investigating category-average activation has left unclear to what extent category selectivity holds for individual object images. Here we investigate single-image activation profiles to address (1) whether each image from the preferred category elicits greater activation than any image outside the preferred category (categorical ranking), (2) whether there are activation differences within and outside the preferred category (gradedness), and (3) whether the activation profile falls off continuously across the category boundary or exhibits a discontinuity at the boundary (category step).We used functional magnetic resonance imaging to measure the activation elicited in the FFA and PPA by each of 96 object images from a wide range of categories, including faces and places,but also humans and animals, and natural and manmade objects. Results suggest that responses in FFA and PPA exhibit almost perfect categorical ranking, are graded within and outside the preferred category, and exhibit a category step. The gradedness within the preferred category was more pronounced in FFA; the category step was more pronounced in PPA. These findings support the idea that these regions have category-specific functions, but are also consistent with a distributed object representation emphasizing categories while still distinguishing individual images.
Mur M, Kriegeskorte N (2012).Tutorial on pattern classification in functional imaging. In: Kriegeskorte N, Kreiman G (eds.) Visual population codes. Toward a common multivariate framework for cell recording and functional imaging. Cambridge, MA: The MIT Press
Kriegeskorte N, Mur M (2012). Representational similarity analysis of object population codes in humans, monkeys, and models. In: Kriegeskorte N, Kreiman G (eds.) Visual population codes. Toward a common multivariate framework for cell recording and functional imaging. Cambridge, MA: The MIT Press
pre 2012
Mur M (2011). High-level visual object representations in inferior temporal cortex. Doctoral Dissertation. Maastricht, The Netherlands: Universitaire Pers Maastricht [pdf]
Mur M, Ruff DA, Bodurka J, Bandettini PA, Kriegeskorte N (2010). Face-identity change activation outside the face system: “Release from adaptation” may not always indicate neuronal selectivity. Cereb Cortex 20: 2027-2042 [pdf] + abstract
Face recognition is a complex cognitive process that requires
distinguishable neuronal representations of individual faces. Previous functional magnetic resonance imaging (fMRI) studies using the ‘‘fMRI-adaptation’’ technique have suggested the existence of face-identity representations in face-selective regions, including the fusiform face area (FFA). Here, we present face-identity adaptation findings that are not well explained in terms of face-identity representations. We performed blood-oxygen level– dependent (BOLD) fMRI measurements, while participants viewed familiar faces that were shown repeatedly throughout the experiment. We found decreased activation for repeated faces in face-selective regions, as expected based on previous studies. However, we found similar effects in regions that are not faceselective, including the place area (PPA) and early visual cortex (EVC). These effects were present for exactimage (same view and lighting) as well as different-image (different view and/or lighting) repetition, but more widespread for exactimage repetition. Given the known functional properties of PPA and EVC, it appears unlikely that they contain domain-specific faceidentity representations. Alternative interpretations include general attentional effects and carryover of activation from connected regions. These results remind us that fMRI stimulus-change effects can have a range of causes and do not provide conclusive evidence for a neuronal representation of the changed stimulus property.
distinguishable neuronal representations of individual faces. Previous functional magnetic resonance imaging (fMRI) studies using the ‘‘fMRI-adaptation’’ technique have suggested the existence of face-identity representations in face-selective regions, including the fusiform face area (FFA). Here, we present face-identity adaptation findings that are not well explained in terms of face-identity representations. We performed blood-oxygen level– dependent (BOLD) fMRI measurements, while participants viewed familiar faces that were shown repeatedly throughout the experiment. We found decreased activation for repeated faces in face-selective regions, as expected based on previous studies. However, we found similar effects in regions that are not faceselective, including the place area (PPA) and early visual cortex (EVC). These effects were present for exactimage (same view and lighting) as well as different-image (different view and/or lighting) repetition, but more widespread for exactimage repetition. Given the known functional properties of PPA and EVC, it appears unlikely that they contain domain-specific faceidentity representations. Alternative interpretations include general attentional effects and carryover of activation from connected regions. These results remind us that fMRI stimulus-change effects can have a range of causes and do not provide conclusive evidence for a neuronal representation of the changed stimulus property.
Mur M, Bandettini PA, Kriegeskorte N (2009). Revealing representational content with pattern-information fMRI – an introductory guide. Soc Cogn Affect Neurosci 4: 101-109 [pdf] + abstract
Conventional statistical analysis methods for functional magnetic resonance imaging (fMRI) data are very successful at detecting brain regions that are activated as a whole during specific mental activities. The overall activation of a region is usually taken to indicate involvement of the region in the task. However, such activation analysis does not consider the multivoxel patterns of activity within a brain region. These patterns of activity, which are thought to reflect neuronal population codes, can be investigated by pattern-information analysis. In this framework, a region’s multivariate pattern information is taken to indicate representational content. This tutorial introduction motivates pattern-information analysis, explains its underlying assumptions, introduces the most widespread methods in an intuitive way, and outlines the basic sequence of analysis steps.
Kriegeskorte N, Mur M, Ruff DA, Kiani R, Bodurka J, Esteky H, Tanaka K, Bandettini PA (2008). Matching categorical object representations in inferior temporal cortex of man and monkey. Neuron 60: 1126-1141 [pdf] + abstract
Inferior temporal (IT) object representations have been intensively studied in monkeys and humans, but representations of the same particular objects have never been compared between the species. Moreover, IT’s role in categorization is not well understood. Here, we presented monkeys and humans with the same images of real-world objects and measured the IT response pattern elicited by each image. In order to relate the representations
between the species and to computational models, we compare response-pattern dissimilarity matrices. IT response patterns form category clusters, which match between man and monkey. The clusters correspond to animate and inanimate objects; within the animate objects, faces and bodies form subclusters. Within each category, IT distinguishes individual exemplars, and the within-category exemplar similarities also match between the species. Our findings suggest that primate IT across species may host a common code, which combines a categorical and a continuous representation of objects.
between the species and to computational models, we compare response-pattern dissimilarity matrices. IT response patterns form category clusters, which match between man and monkey. The clusters correspond to animate and inanimate objects; within the animate objects, faces and bodies form subclusters. Within each category, IT distinguishes individual exemplars, and the within-category exemplar similarities also match between the species. Our findings suggest that primate IT across species may host a common code, which combines a categorical and a continuous representation of objects.
Kriegeskorte N, Mur M, Bandettini P (2008). Representational similarity analysis – a general framework for relating computational theory and modalities of brain-activity measurement. Front Syst Neurosci 2: 4 [pdf] + abstract
A fundamental challenge for systems neuroscience is to quantitatively relate its three major
branches of research: brain-activity measurement, behavioral measurement, and computational modeling. Using measured brain-activity patterns to evaluate computational network models is complicated by the need to define the correspondency between the units of the model and the channels of the brain-activity data, e.g., single-cell recordings or voxels from functional magnetic resonance imaging (fMRI). Similar correspondency problems complicate relating activity patterns between different modalities of brain-activity measurement (e.g., fMRI and invasive or scalp electrophysiology), and between subjects and species. In order to bridge these divides, we suggest abstracting from the activity patterns themselves and computing representational dissimilarity matrices (RDMs), which characterize the information carried by a given representation in a brain or model. Building on a rich psychological and mathematical literature on similarity analysis, we propose a new experimental and data-analytical framework called representational similarity analysis (RSA), in which multi-channel measures of neural activity are quantitatively related to each other and to computational theory and behavior by comparing RDMs. We demonstrate RSA by relating representations of visual objects as measured with fMRI in early visual cortex and the fusiform face area to computational models spanning a wide range of complexities. The RDMs are simultaneously related via second-level application of multidimensional scaling and tested using randomization and bootstrap techniques. We discuss the broad potential of RSA, including novel approaches to experimental design, and argue that these ideas, which have deep roots in psychology and neuroscience, will allow the integrated quantitative analysis of data from all three branches, thus contributing to a more unified systems neuroscience.
branches of research: brain-activity measurement, behavioral measurement, and computational modeling. Using measured brain-activity patterns to evaluate computational network models is complicated by the need to define the correspondency between the units of the model and the channels of the brain-activity data, e.g., single-cell recordings or voxels from functional magnetic resonance imaging (fMRI). Similar correspondency problems complicate relating activity patterns between different modalities of brain-activity measurement (e.g., fMRI and invasive or scalp electrophysiology), and between subjects and species. In order to bridge these divides, we suggest abstracting from the activity patterns themselves and computing representational dissimilarity matrices (RDMs), which characterize the information carried by a given representation in a brain or model. Building on a rich psychological and mathematical literature on similarity analysis, we propose a new experimental and data-analytical framework called representational similarity analysis (RSA), in which multi-channel measures of neural activity are quantitatively related to each other and to computational theory and behavior by comparing RDMs. We demonstrate RSA by relating representations of visual objects as measured with fMRI in early visual cortex and the fusiform face area to computational models spanning a wide range of complexities. The RDMs are simultaneously related via second-level application of multidimensional scaling and tested using randomization and bootstrap techniques. We discuss the broad potential of RSA, including novel approaches to experimental design, and argue that these ideas, which have deep roots in psychology and neuroscience, will allow the integrated quantitative analysis of data from all three branches, thus contributing to a more unified systems neuroscience.
Broers NJ, Mur MC, Bude L (2005). Directed self explanation in the study of statistics. In: Burrill G, Camden M (eds.) Curricular development in statistics education. Voorburg, The Netherlands: International Statistical Institute [pdf] + abstract
Constructivist learning theory has suggested that students can only obtain conceptual understanding of a knowledge domain by actively trying to integrate new concepts and ideas into their existing knowledge framework. In practice, this means that students will have to explain novel ideas, concepts, and principles to themselves. Various methods have been developed that aim to stimulate the student to self-explain. In this study, two such methods were contrasted in a randomized experiment. In one condition students were stimulated to self-explain in an undirected way. In the other the stimulus to
self-explain was directed. We examined whether the directive method leads to a greater level of conceptual understanding. To assess conceptual understanding we asked students to construct a concept map and to take a 10-item multiple-choice test. The results are somewhat contradictory but do suggest that the directive method may be of value. We discuss the possibility of integrating that method in the statistics curriculum.
self-explain was directed. We examined whether the directive method leads to a greater level of conceptual understanding. To assess conceptual understanding we asked students to construct a concept map and to take a 10-item multiple-choice test. The results are somewhat contradictory but do suggest that the directive method may be of value. We discuss the possibility of integrating that method in the statistics curriculum.