The use of the algebraic method for solving word problems is a challenging topic for secondary school students. Students' difficulties are usually associated with extracting the problem's network of relationships between quantities and with formalizing these relationships into algebraic language in a problem model. Both sources can coexist and interact; thus, it is usually not possible to determine which source of difficulty is more relevant. In addition, there are specific errors, such as the error by multiple referents for the unknown, which are directly linked to the wording of the problem text, and in which the same two sources of error coexist. In this work, we present the results of an experiment conducted with 255 secondary school students assessing the effect of two common difficulties on the accuracy of problem models and on the rate of multiple referents for the unknown. The first difficulty is the use of algebraic language in the construction of the problem model; the second is the use of the same expression to designate different quantities within the problem text. We used a 2 × 2 between-between design, with one factor related to the symbolic language (algebraic or arithmetic) in which the problem model is constructed, and the other factor related to the actual language features of the text problem. Our results indicate that overall, the main source of difficulty for students is the use of algebraic language to formalize a problem model, representing a large effect size.