The design and psychometric methodology of the National Assessment of Educational Progress (NAEP) is constantly evolving to meet the changing interests and demands stemming from a rapidly shifting educational landscape. NAEP has been built on strong research foundations that include conducting extensive evaluations and comparisons before new approaches are adopted. During those evaluations, many lessons are learned and discoveries surface that do not often find their way into widely accessible outlets. This article discusses a number of those insights with the goal to provide an integrated and accessible perspective on the strengths and limitations of NAEP's psychometric methodology and statistical reporting practices. Drawing from a range of technical reports and memoranda, presentations, and published literature, the following topics are covered: calibration, estimation of proficiency, data reduction, standard error estimation, statistical inference, and standard setting.