In order to effectively grade persuasive writing we must be able to reliably identify and extract extract argument structures. In order to do this we must classify arguments by their structural roles (e.g., major claim, claim, and premise). Current approaches to classification typically rely on statistical models with heavy feature-engineering or on deep neural-networks that do not consider prior knowledge or other secondary features. Little research has been carried out to investigate if we can incorporate features into deep models to address AM tasks. In this work, we propose to incorporate lightweight features into deep models to classify argument components. We experimented with two state of the art (SOTA) approaches: 1) linear-Long-Short-Term Memory (LSTM) models with concatenated feature vectors; or 2) Directed Acyclic Graph (DAG) structured LSTMs. In our models we incorporated the features of argument position (e.g., if the argument is in the first paragraph) and prior knowledge of discourse indicators (e.g., in conclusion, for example). We use two baselines in our work: 1) prior work using SVM models with heavy feature engineering; 2) traditional linear-Bi-LSTMs with no task-specific features. Our results show that with a comparatively small number of lightweight features, both linear-Bi-LSTMs and DAG-Bi-LSTMs outperform SVM models that depend on more heavy feature engineering, and outperform linear-Bi-LSTMs with only general word embedding features. These results suggest that incorporating task-specific elements into deep models may potentially benefit argument mining tasks. [For the full proceedings, see ED607784.]