Severe fiscal tensions threaten U.S. public higher education. Many policy solutions have been suggested, but it is difficult to subject these qualitative ideas to rigorous empirical evaluation. In this work, we employ an agent-based model of a representative state-funded public university system (including a flagship campus, an urban campus, and several regional-comprehensive campuses) to examine approaches aimed at optimizing institutional performance in an era of declining state funding. These options include imposing fiscal austerity in staff compensation, incentivizing faculty productivity, increasing faculty teaching loads, improving operating efficiency, raising admission standards, and increasing the delivery of online instruction. No solution is found that stabilizes tuition costs relative to present conditions, but scenarios that facilitate faculty productivity and incorporate online instruction also lead to improvements in funded research activity and student outcomes compared to current trends. Further productivity enhancements would be possible if a means to streamline academic decision-making processes in the conduct of regular university business (in shared-governance context) could be found. These factors combine in the best cases so that graduation rates, research funding, and system quality improve. In contrast, fiscal austerity in the form of direct cost cutting saves money for the state but results in significant increases (70% to 180%) in real (inflation-adjusted) tuition and fees at the several campuses over the course of 40 simulation iterations relative to 2009-10 levels. Further, this reliance on fiscal austerity alone is highly damaging to institutional performance. Declines in faculty productivity, resulting from worsening compression of faculty compensation, lead to reductions in external-grant funding, six-year graduation rates, and system quality. Growing enrollments in response to demographic pressure offset the most deleterious effects of reduced support, without which, an unsustainable spiral of declining enrollment and rapid tuition rises can occur. Overall, the simulation results suggest that efforts to maintain system affordability and quality in the face of long-term funding constraints are best directed towards incentivizing productivity improvements. (Contains 6 tables, 1 figure, and 17 footnotes.)