Because digital tutoring is rapidly expanding, more rigorous, independent evaluations of their effectiveness is critical to inform federal, state, and local policy decisions that influence their role and application of technology in educating underserved students. The in-depth observations and vignettes in this paper illustrate the challenges in documenting and measuring technology use and its impact in out-of-school time (OST) tutoring interventions on student learning. Currently, the limited, self-generated information that is disseminated by providers to parents and students does not usefully guide parent and student choices of OST tutoring providers. This paper generates important and generalizable insights on the nature of digital OST instruction, including what constitutes and contributes to high-quality OST tutoring in digital contexts. The longitudinal mixed-method design used in this study integrates rigorous, quasi-experimental analysis of OST tutoring program impacts on student achievement with an in-depth, comprehensive examination of the intervention--provider instructional practice in different program models and settings, the nature and quality of tutoring provided, and district-level program administration--in and across six large, urban school districts. Findings show that digital tutoring is, on average, negatively correlated with student performance in mathematics and reading (i.e., when comparing students served by digital vs. non-digital providers). Coupled with in-depth qualitative research into the nature of the instructional setting in digital OST, the factors impacting the design of the instructional setting, and its impact on student learning, researchers identify several characteristics or conditions that they hypothesize are likely to determine digital tutoring effectiveness: access (including hardware and software), the role of the tutor, and the nature of curriculum and assessment.