The importance of the critical interpretive synthesis (CIS) to review quantitative and qualitative research, and to critically develop new theory, is increasingly recognized and evidenced by the increase in published CIS reviews. However, the flexibility embedded in the method hampers its implementation and exacerbates concerns about trustworthiness. This paper seeks to determine the extent of transparent reporting and soundness of execution in published CIS reviews by developing assessment criteria based on CIS key features. We analyzed 77 CIS reviews published between 2006 and 2018 for their reporting practices. Findings indicate that reporting practices of CIS key features are suboptimal. We recommend that authors better document their CIS to increase the transparency of their study and suggest authors to rely on described guidelines to select and conduct their CIS. To this end, our reported evaluation criteria could assist authors, reviewers, and journal editors in their evaluation of the quality of CIS studies.