There has been the growing concern about excessive numbers of false-positive results published in the scientific literature. Cognitive bias plays a considerable role in triggering false findings that involve indirect and unwitting self-deception by scientists. This study considers the sociocultural differences in cognitive bias between Korean and Indonesian scientists. A cognitive bias assessment (CBA) was developed and administered to 184 professors, lecturer assistants, doctoral students, and master's students in South Korea and Indonesia. The CBA results revealed some similar response patterns between Korean and Indonesian scientists. Additionally, the detection of 19 potential differential item functionings (DIFs) demonstrates the influence of sociocultural factors on how scientists interpret to each item. Finally, the Indonesian scientists scored significantly higher in optimism and belief bias. This study discusses the importance of awareness of cognitive biases, particularly the role of science education to reduce biases through systematic thinking, reasoning, and judgment by understanding scientific methods.