Voxel based morphometry (VBM) is a useful tool to assess differences in brain morphology between groups of patients and healthy controls. In addition, VBM enables the performance of regression analyses to determine potential correlations between performance on cognitive tests and variations in local brain morphology. Prior VBM studies investigating patients with mild cognitive impairment (MCI) have revealed different patterns of local brain atrophy. In order to extend previous findings, we investigated 18 patients with MCI and 18 age- and gender-matched healthy controls. All participants underwent extensive neuropsychological testing in addition to undergoing anatomical scanning with magnetic resonance imaging (MRI). Cohort analysis revealed bilateral decreases in gray matter density in the medial temporal lobes (MTLs) and neocortical regions of the temporal lobes in patients with MCI. Moreover, regression analyses demonstrated a correlation between immediate verbal recall and gray matter density in the left perirhinal/entorhinal cortex, while delayed free recall correlated with gray matter density in the left hippocampus. It has been proposed that performance in the immediate recall is supported by the so-called “episodic buffer”, a component of working memory that contributes to the maintenance of integrated memory traces. Accordingly, our results suggest that anatomical regions associated with verbal long-term and verbal working memory are structurally segregated within the left MTL.