Thesis (M.Sc.) - Cairo University - Faculty of Computers and Information - Department of Computer Science

Improving the quality of education is always a desired goal in educational institutions, and involving technology in the educational process can improve education by supporting teachers with variety of tools to facilitate their job. One supporting solution is automatic scoring systems for students{u2019} short answers. This system can eliminate from teachers the burden of grading a large number of test questions and facilitate performing even more assessments either during lectures or via quizzes especially when the number of students is large. This research presents a learning model for short answers clustering and automatic scoring without using reference answers. The model is divided into three phases: (1) an embedding model for short answers representation, (2) a clustering model to cluster answers into groups based on their similarities, and (3) a regression model for predicting students{u2019} scores. For the embedding phase, a comprehensive evaluation of multiple state-of-the-art embedding models was applied to choose the best technique for text representation. Seven models were tested and evaluated separately by training a regression model to predict students{u2019} scores based on the cosine similarity between embeddings of students{u2019} answers and reference answers as a training feature. The study shows that using pre-trained models achieved comparable results for the task of automatic short answer scoring. The model that achieved the best results is doc2vec model which was trained on answers - students{u2019} answers and reference answers - from the benchmark dataset in order to learn vector representations of answers. The model achieved 0.569 correlation coefficient value measured based on the correlation between actual grades and predicted scores. The model achieved 0.797 root mean square error (RMSE) value for correctness of predictions

Issued also as CD