The predicted check details category probabilities indicate that the scene is most likely a mixture of the categories “Urban” and “Boatway,” which is an accurate description of the scene. Inspection of the other examples in the figure suggests that the predicted scene category probabilities accurately describe many different types of natural scenes. To quantify the accuracy of each decoder, we calculated the correlation (Pearson’s r) between the scene category probabilities predicted by the decoder and the probabilities inferred using the LDA algorithm (conditioned on the labeled objects in each scene). Figure 4B shows

the distribution of decoding accuracies across all decoded scenes, for each subject. The median accuracies and 95% confidence interval (CI) on median estimates are indicated by the black cross-hairs. Most of the novel scenes

are decoded significantly for all subjects. Prediction accuracy across all scenes exhibited systematically greater-than-chance performance for all subjects (p < 0.02 for all subjects, Wilcox rank-sum test; subject S1: W(126) = 18,585; subject S2: W(126) = 17,274; subject S3: W(126) = 17,018; subject S4: W(126) = 19,214. The voxels selected for the decoding analysis summarized in Figure 4 were located throughout buy Afatinib the visual cortex. However, we also find that accurate decoding can be obtained using the responses of subsets of voxels located within specific ROIs (see Figures S16–S19). PD184352 (CI-1040) Our results suggest that the visual system represents scene categories that capture the co-occurrence statistics of objects in the natural world. This suggests that we should be able to predict accurately the likely objects in a scene based on the scene category probabilities

decoded from evoked brain activity. To investigate this issue, we estimated the probability that each of the 850 objects in the vocabulary for the single best set of scene categories identified across subjects occurred in each of the 126 decoded validation set scenes. The probabilities were estimated by combining the decoded category probabilities with the probabilistic relationship between categories and objects established by the LDA learning algorithm during category learning (see Experimental Procedures for details). The resulting probabilities give an estimate of the likelihood that each of the 850 objects occurs in each of the 126 decoded scenes. In Figure 4A, labels in the black boxes indicate the most likely objects estimated for the corresponding decoded scene. For the harbor and skyline scene at upper right, the most probable objects predicted for the scene are “building,” “sky,” “tree,” “water,” “car,” “road,” and “boat.” All of these objects either occur in the scene or are consistent with the scene context. Inspection of the other examples in the figure suggests that the most probable objects are generally consistent with the scene category.