Influential Factors on Mathematical Literacy of Turkish Students: An Educational Data Mining Study Using PISA 2015 Data




Educational data mining, J48, Multilayer perceptron, Naïve bayes, Support vector machine


This study aims to classify students as successful and unsuccessful regarding mathematical literacy on Programme for International Student Assessment (PISA) 2015 database through data mining methods. The sample consists of all Turkish students who participated in PISA 2015. While data mining methods such as Support Vector Machine, Multi-Layer Perceptron, and J48 were used in data analysis, the data set was evaluated with 10-fold Cross-validation. The evaluation criteria included F-measure, Precision, Recall, Matthews Correlation Coefficient, and Receiver Operating Characteristic (ROC). In the classification of successful and unsuccessful students, analyses were conducted with 13 statistically significant variables according to Chi-SquareAttributedEval, GainRatioAttributeEval, and InfoGainAttributeEval methods. The results showed that the most important variables for classifying successful and unsuccessful students were learning time per week in total, and father’s education level. The highest ROC value was 0.720. When comparing the precision values, the lowest classification value for the Multilayer Perceptron method was 0.645. There was no single method that performed best for all criteria. Researchers should use at least two methods to obtain more accurate results.


Aksu, G. (2018). PISA başarısını tahmin etmede kullanılan veri madenciliği yöntemlerinin incelenmesi. [Investigation of data mining methods used for estimating PISA success]. (Publication No. 515513) [Doctoral dissertation, Hacettepe University]. Council of Higher Education Thesis Center, Türkiye.

Aksu, G., & Güzeller, C. O. (2016). Classification of PISA 2012 Mathematical Literacy Scores Using Decision-Tree Method: Turkey Sampling. Education and Science, 41(185), 101-122.

Aldowah, H., Al-Samarraie, H., & Fauzy, W. M. (2019). Educational data mining and learning analytics for 21st century higher education: A Review and Synthesis. Telematics and Informatics, 37, 13-49. doi:10.1016/j.tele.2019.01.007

Ammermüller, A. (2004). PISA: What makes the difference? Explaining the gap in PISA test scores between Finland and Germany. Explaining the Gap in PISA Test Scores Between Finland and Germany, ZEW Discussion Papers, No. 04-04, Zentrum für Europäische Wirtschaftsforschung (ZEW), Mannheim.

Anıl, D. (2009). Factors effecting science achievement of science students in programme for international students’ achievement (PISA) in Turkey. Education and Science, 34 (152), 87-100.

Baker, R. S., & Yacef, K. (2009). The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining, 1(1), 3-17.

Baker, R. S. (2010). Data mining for education. International Encyclopedia of Education, 7(3), 112-118.

Bakhshinategh, B., Zaiane, O. R., El Atia, S., & Ipperciel, D. (2018). Educational data mining applications and tasks: A survey of the last 10 years. Education and Information Technologies, 23(1), 537-553.

Bezek Güre, Ö. B., Kayri, M., & Erdoğan, F. (2020). Analysis of factors effecting PISA 2015 mathematics literacy via educational data mining. Education and Science, 45(202), 393-415.

Bramer, M. (2013). Principles of data mining (2nd ed.). Springer-Verlag.

Bratti, M., Checchi, D., & Filippin, A. (2011). Should you compete or cooperate with your schoolmates?. Education Economics, 19(3), 275-289.

Bresfelean, V. P., Bresfelean, M., Ghisoiu, N., & Comes, C. A. (2008, June). Determining students’ academic failure profile founded on data mining methods. In ITI 2008-30th International Conference on Information Technology Interfaces (pp. 317-322).

Bousbia, N., & Belamri, I. (2014). Which contribution does EDM provide to computer-based learning environments?. In Educational data mining (pp. 3-28). Springer.

Bunkar, K., Singh, U. K., Pandya, B., & Bunkar, R. (2012, September). Data mining: Prediction for performance improvement of graduate students using classification. In 2012 Ninth International Conference on Wireless and Optical Communications Networks (WOCN) (pp. 1-5).

Büyükkıdık, S., Bakırarar, B., & Bulut, O. (2018, September). Comparing the Performance of Data Mining Methods in Classifying Successful Students with Scientific Literacy in PISA 2015. 6th Internatıonal Congress on Measurement and Evaluatıon in Educatıon and Psychology, Prizren, Kosova.

Cassady, J. C., & Johnson, R. E. (2002). Cognitive Test Anxiety and Academic Performance. Contemporary Educational Psychology, 27(2), 270–295.

Culler, R. E., & Holahan, C. J. (1980). Test anxiety and academic performance: The effects of study-related behaviors. Journal of Educational Psychology, 72(1), 16–20.

Costa, E. B., Fonseca, B., Santana, M. A., de Araújo, F. F., & Rego, J. (2017). Evaluating the effectiveness of educational data mining techniques for early prediction of students' academic failure in introductory programming courses. Computers in Human Behavior, 73, 247-256.

Chevalier, A., & Lanot, G. (2002). The relative effect of family characteristics and financial situation on educational achievement. Education Economics, 10(2), 165-181.

D’Agostino, A., Schirripa Spagnolo, F., & Salvati, N. (2022). Studying the relationship between anxiety and school achievement: evidence from PISA data. Statistical Methods & Applications, 31(1), 1-20.

Else-Quest, N. M., Hyde, J. S., & Linn, M. C. (2010). Cross-national patterns of gender differences in mathematics: a meta-analysis. Psychological Bulletin, 136(1), 103-127.

Feingold, A. (1988). Cognitive gender differences are disappearing. American Psychologist, 43, 181–191. doi: 10.1037/0003-066X.43.2.95

Firdausi, I., Erwin, A., & Nugroho, A. S. (2010, December). Analysis of machine learning techniques used in behavior-based malware detection. In 2010 second international conference on advances in computing, control, and telecommunication technologies (pp. 201-203). IEEE.

Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How to design and evaluate research in education (8th ed.). Mc Graw Hill Higher Education.

Fuchs, T., & Wößmann, L. (2008). What accounts for international differences in student performance?: A re-examination using PISA data. In The economics of education and training (pp. 209-240). Physica-Verlag HD.

Gamazo, A., & Martínez-Abad, F. (2020). An exploration of factors linked to academic performance in PISA 2018 through data mining techniques. Frontiers in Psychology, 11, 575167.

Gilleece, L., Cosgrove, J., & Sofroniou, N. (2010). Equity in mathematics and science outcomes: Characteristics associated with high and low achievement on PISA 2006 in Ireland. International Journal of Science and Mathematics Education, 8(3), 475-496.

Gunderson, E. A., Park, D., Maloney, E. A., Beilock, S. L., & Levine, S. C. (2018). Reciprocal relations among motivational frameworks, math anxiety, and math achievement in early elementary school. J. Cogn. Dev. 19, 21–46.

Han, J., & Gao, J. (2008). Research challenges for data mining in science and engineering. In Kargupta, H., Han, J., Philip, S. Y., Motwani, R., & Kumar, V. (Eds.), Next generation of data mining (pp. 27-52). Chapman and Hall/CRC Press.

Hämäläinen, W., & Vinni, M. (2010). Classifiers for educational technology. In Romero, C., Ventura, S., Pechenizkiy, M., & Baker, R. S. (Eds.) Handbook on educational data mining (pp. 57-71.). CRC Press.

Hand, D., Mannila, H., & Smyth, P. (2001). Principles of data mining (Adaptive computation and machine learning). MIT Press.

Hembree, R. (1990). The nature, effects, and relief of mathematics anxiety. Journal for Research in Mathematics Education, 21 (1), 33–46.

Hertel, S., & Jude, N. (2016). Parental support and involvement in school. In Assessing Contexts of Learning (pp. 209-225). Springer.

Hyde, J. S., Fennema, E., & Lamon, S. J. (1990). Gender differences in mathematics performance: A meta-analysis. Psychological Bulletin, 107(2), 139-155.

Kaur, P., Singh, M., & Josan, G. S. (2015). Classification and prediction based data mining algorithms to predict slow learners in education sector. Procedia Computer Science, 57, 500-508.

Kılıç Depren, S., Aşkın, Ö. E., & Öz, E. (2017). Identifying the classification performances of educational data mining methods: a case study for TIMSS. Educational Sciences: Theory & Practice, 17(5), 1605–1623.

Keller, L., Preckel, F., Eccles, J. S., & Brunner, M. (2022). Top-performing math students in 82 countries: An integrative data analysis of gender differences in achievement, achievement profiles, and achievement motivation. Journal of Educational Psychology, 114(5), 966-991.

Koyuncu, İ., & Gelbal, S. (2020). Comparison of data mining classification algorithms on educational data under different conditions. Journal of Measurement and Evaluation in Education and Psychology, 11(4), 325-345.

Lee, J., Kao, H. A., & Yang, S. (2014). Service innovation and smart analytics for industry 4.0 and big data environment. Procedia Cirp, 16, 3-8.

Lee, J., & Stankov, L. (2018). Non-cognitive predictors of academic achievement: Evidence from TIMSS and PISA. Learning and Individual Differences, 65, 50-64.

Linnakylä, P. & Malin, A. (2008). Finnish students' school engagement profiles in the light of PISA 2003, Scandinavian Journal of Educational Research, 52(6), 583-602.

Liu, O. L., & Wilson, M. (2009). Gender differences in large-scale math assessments: PISA trend 2000 and 2003. Applied Measurement in Education, 22(2), 164–184.

Liu, X., & Whitford, M. (2011). Opportunities-to-learn at home: profiles of students with and without reaching science proficiency. Journal of Science Education and Technology, 20(4), 375–387.

Ma, X. (1999). A meta-analysis of the relationship between anxiety toward mathematics and achievement in mathematics. Journal for Research in Mathematics Education, 30 (5), 520–540.

Martínez-Abad, F. (2019). Identification of factors associated with school effectiveness with data mining techniques: Testing a new approach. Frontiers in Psychology, 10, 1-13.

Martínez-Abad, F., Gamazo, A., & Rodríguez-Conde, M.-J. (2020). Educational data mining: identification of factors associated with school effectiveness in PISA assessment. Studies in Educational Evaluation, 66, 2-10.

Masci, C., Johnes, G., & Agasisti, T. (2018). Student and school performance across countries: A machine learning approach. European Journal of Operational Research, 269(3), 1072-1085.

OECD (2016). PISA 2015 results in focus. OECD Publishing. Retrieved

OECD (2017a). PISA 2015 technical report. Organisation for Economic Co-Operation and Development, PISA, OECD Publishing.

OECD (2017b). PISA 2015 assessment and analytical framework: science, reading, mathematic, financial literacy and collaborative problem solving, PISA, OECD Publishing.

Põder, K., Lauri, T., Ivaniushina, V., & Alexandrov, D. (2016). Family background and school choice in cities of Russia and Estonia: Selective agenda of the Soviet past and present. Studies of Transition States and Societies, 8(3), 5-28.

Ramesh, V., Parkavi, P., & Ramar, K. (2013). Predicting student performance: a statistical and data mining approach. International Journal of Computer Applications, 63(8), 35-39.

Reilly, D., Neumann, D. L., & Andrews, G. (2019). Investigating gender differences in mathematics and science: Results from the 2011 Trends in Mathematics and Science Survey. Research in Science Education, 49(1), 25-50. 017-9630-6

Romero, C., & Ventura, S. (2007). Educational data mining: A survey from 1995 to 2005. Expert Systems with Applications, 33(1), 135-146.

Romero, C., & Ventura, S. (2010). Educational data mining: A review of the state of the art. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 40(6), 601-618.

Romero, C., & Ventura, S. (2013). Data mining in education. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 3(1), 12-27.

Romero, C., Ventura, S., Pechenizkiy, M., & Baker, R. S. (Eds.). (2010). Handbook of educational data mining. Data mining and knowledge discovery series. Chapman and Hall/CRC Press.

Saarela, M., Yener, B., Zaki, M. J., & Kärkkäinen, T. (2016). Predicting math performance from raw large-scale educational assessments data: a machine learning approach. In JMLR Workshop and Conference Proceedings; JMLR: W&CP, 48.

Sherman, B. F., & Wither, D. P. (2003). Mathematics anxiety and mathematics achievement. Mathematics Education Research Journal, 15(2), 138-150.

Siemens, G., & d Baker, R. S. (2012, April). Learning analytics and educational data mining: towards communication and collaboration. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 252-254). ACM.

Singh, K., Granville, M., & Dika, S. (2002). Mathematics and science achievement: Effects of motivation, interest, and academic engagement. The Journal of Educational Research, 95(6), 323-332.

Slavin, E. R. (1983). When does cooperative learning increase student achievement?. Psychological Bulletin, 94(3), 429-445.

Taş, U. E., Arıcı, Ö., Ozarkan, H. B., & Özgürlük, B. (2016). PISA 2015 national report. Ministry of Education Publishing.

Tocci, C. M., & Engelhard Jr, G. (1991). Achievement, parental support and gender differences in attitudes toward mathematics. The Journal of Educational Research, 84(5), 280-287.

Wine, J. (1971). Test anxiety and direction of attention. Psychological Bulletin, 76(2), 92–104.

Witten, I. H., & Frank, E., (2005). Data mining: Practical machine learning tools and techniques (Second Edition). Morgan Kaufmann Publishers, San Francisco, CA.

Wu, S. S., Willcutt, E. G., Escovar, E., and Menon, V. (2014). Mathematics achievement and anxiety and their relation to internalizing and externalizing behaviours. Journal of Learning Disabilities, 47, 503–514. 473154

Yayan, B., & Berberoglu, G. (2004). A re-analysis of the TIMSS 1999 mathematics assessment data of the Turkish students. Studies in Educational Evaluation, 30(1), 87-104.

Yukselturk, E., Ozekes, S., & Kılıç Türel, Y. (2014). Predicting dropout student: an application of data mining methods in an online education program. European Journal of Open, Distance and e-learning, 17(1), 118-133.

Zhang, J., Zhao, N., & Kong, Q. P. (2019). The relationship between math anxiety and math performance: A meta-analytic investigation. Frontiers in Psychology, 10, 1-17.

Additional Files



How to Cite

Büyükkıdık, S. (2023). Influential Factors on Mathematical Literacy of Turkish Students: An Educational Data Mining Study Using PISA 2015 Data. Psycho-Educational Research Reviews, 12(2), 505–521.