Perceptual decision-making performance depends on several cognitive and neural processes. Here, we fit Ratcliff's diffusion model to accuracy data and reaction-time distributions from one numerical and one verbal two-choice perceptual-decision task to deconstruct these performance measures into the rate of evidence accumulation (i.e., drift rate), response criterion setting (i.e., boundary separation), and peripheral aspects of performance (i.e., nondecision time). These theoretical processes are then related to individual differences in brain activation by means of multiple regression. The sample consisted of 24 younger and 15 older adults performing the task in fMRI before and after 100 daily 1-hr behavioral training sessions in a multitude of cognitive tasks. Results showed that individual differences in boundary separation were related to striatal activity, whereas differences in drift rate were related to activity in the inferior parietal lobe. These associations were not significantly modified by adult age or perceptual expertise. We conclude that the striatum is involved in regulating response thresholds, whereas the inferior parietal lobe might represent decision-making evidence related to letters and numbers.