Influential Factors on Mathematical Literacy of Turkish Students: An Educational Data Mining Study Using PISA 2015 Data
DOI:
https://doi.org/10.52963/PERR_Biruni_V12.N2.10Keywords:
Educational data mining, J48, Multilayer perceptron, Naïve bayes, Support vector machineAbstract
This study aims to classify students as successful and unsuccessful regarding mathematical literacy on Programme for International Student Assessment (PISA) 2015 database through data mining methods. The sample consists of all Turkish students who participated in PISA 2015. While data mining methods such as Support Vector Machine, Multi-Layer Perceptron, and J48 were used in data analysis, the data set was evaluated with 10-fold Cross-validation. The evaluation criteria included F-measure, Precision, Recall, Matthews Correlation Coefficient, and Receiver Operating Characteristic (ROC). In the classification of successful and unsuccessful students, analyses were conducted with 13 statistically significant variables according to Chi-SquareAttributedEval, GainRatioAttributeEval, and InfoGainAttributeEval methods. The results showed that the most important variables for classifying successful and unsuccessful students were learning time per week in total, and father’s education level. The highest ROC value was 0.720. When comparing the precision values, the lowest classification value for the Multilayer Perceptron method was 0.645. There was no single method that performed best for all criteria. Researchers should use at least two methods to obtain more accurate results.
Downloads
References
Aksu, G. (2018). PISA başarısını tahmin etmede kullanılan veri madenciliği yöntemlerinin incelenmesi. [Investigation of data mining methods used for estimating PISA success]. (Publication No. 515513) [Doctoral dissertation, Hacettepe University]. Council of Higher Education Thesis Center, Türkiye.
Aksu, G., & Güzeller, C. O. (2016). Classification of PISA 2012 Mathematical Literacy Scores Using Decision-Tree Method: Turkey Sampling. Education and Science, 41(185), 101-122. https://doi.org/10.15390/EB.2016.4766
Aldowah, H., Al-Samarraie, H., & Fauzy, W. M. (2019). Educational data mining and learning analytics for 21st century higher education: A Review and Synthesis. Telematics and Informatics, 37, 13-49. doi:10.1016/j.tele.2019.01.007
Ammermüller, A. (2004). PISA: What makes the difference? Explaining the gap in PISA test scores between Finland and Germany. Explaining the Gap in PISA Test Scores Between Finland and Germany, ZEW Discussion Papers, No. 04-04, Zentrum für Europäische Wirtschaftsforschung (ZEW), Mannheim.
Anıl, D. (2009). Factors effecting science achievement of science students in programme for international students’ achievement (PISA) in Turkey. Education and Science, 34 (152), 87-100.
Baker, R. S., & Yacef, K. (2009). The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining, 1(1), 3-17. https://doi.org/10.5281/zenodo.3554657
Baker, R. S. (2010). Data mining for education. International Encyclopedia of Education, 7(3), 112-118.
Bakhshinategh, B., Zaiane, O. R., El Atia, S., & Ipperciel, D. (2018). Educational data mining applications and tasks: A survey of the last 10 years. Education and Information Technologies, 23(1), 537-553. https://doi.org/10.1007/s10639-017-9616-z
Bezek Güre, Ö. B., Kayri, M., & Erdoğan, F. (2020). Analysis of factors effecting PISA 2015 mathematics literacy via educational data mining. Education and Science, 45(202), 393-415. https://doi.org/10.15390/EB.2020.8477
Bramer, M. (2013). Principles of data mining (2nd ed.). Springer-Verlag.
Bratti, M., Checchi, D., & Filippin, A. (2011). Should you compete or cooperate with your schoolmates?. Education Economics, 19(3), 275-289. https://doi.org/10.1080/09645292.2011.585021
Bresfelean, V. P., Bresfelean, M., Ghisoiu, N., & Comes, C. A. (2008, June). Determining students’ academic failure profile founded on data mining methods. In ITI 2008-30th International Conference on Information Technology Interfaces (pp. 317-322).
Bousbia, N., & Belamri, I. (2014). Which contribution does EDM provide to computer-based learning environments?. In Educational data mining (pp. 3-28). Springer.
Bunkar, K., Singh, U. K., Pandya, B., & Bunkar, R. (2012, September). Data mining: Prediction for performance improvement of graduate students using classification. In 2012 Ninth International Conference on Wireless and Optical Communications Networks (WOCN) (pp. 1-5).
Büyükkıdık, S., Bakırarar, B., & Bulut, O. (2018, September). Comparing the Performance of Data Mining Methods in Classifying Successful Students with Scientific Literacy in PISA 2015. 6th Internatıonal Congress on Measurement and Evaluatıon in Educatıon and Psychology, Prizren, Kosova.
Cassady, J. C., & Johnson, R. E. (2002). Cognitive Test Anxiety and Academic Performance. Contemporary Educational Psychology, 27(2), 270–295. https://doi.org/10.1006/ceps.2001.1094
Culler, R. E., & Holahan, C. J. (1980). Test anxiety and academic performance: The effects of study-related behaviors. Journal of Educational Psychology, 72(1), 16–20. https://doi.org/10.1037/0022-0663.72.1.16
Costa, E. B., Fonseca, B., Santana, M. A., de Araújo, F. F., & Rego, J. (2017). Evaluating the effectiveness of educational data mining techniques for early prediction of students' academic failure in introductory programming courses. Computers in Human Behavior, 73, 247-256. https://doi.org/10.1016/j.chb.2017.01.047
Chevalier, A., & Lanot, G. (2002). The relative effect of family characteristics and financial situation on educational achievement. Education Economics, 10(2), 165-181. https://doi.org/10.1080/09645290210126904
D’Agostino, A., Schirripa Spagnolo, F., & Salvati, N. (2022). Studying the relationship between anxiety and school achievement: evidence from PISA data. Statistical Methods & Applications, 31(1), 1-20. https://doi.org/10.1007/s10260-021-00563-9
Else-Quest, N. M., Hyde, J. S., & Linn, M. C. (2010). Cross-national patterns of gender differences in mathematics: a meta-analysis. Psychological Bulletin, 136(1), 103-127. https://doi.org/10.1037/a0018053
Feingold, A. (1988). Cognitive gender differences are disappearing. American Psychologist, 43, 181–191. doi: 10.1037/0003-066X.43.2.95
Firdausi, I., Erwin, A., & Nugroho, A. S. (2010, December). Analysis of machine learning techniques used in behavior-based malware detection. In 2010 second international conference on advances in computing, control, and telecommunication technologies (pp. 201-203). IEEE. https://doi.org/10.1109/ACT.2010.33
Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How to design and evaluate research in education (8th ed.). Mc Graw Hill Higher Education.
Fuchs, T., & Wößmann, L. (2008). What accounts for international differences in student performance?: A re-examination using PISA data. In The economics of education and training (pp. 209-240). Physica-Verlag HD.
Gamazo, A., & Martínez-Abad, F. (2020). An exploration of factors linked to academic performance in PISA 2018 through data mining techniques. Frontiers in Psychology, 11, 575167. https://doi.org/10.3389/fpsyg.2020.575167
Gilleece, L., Cosgrove, J., & Sofroniou, N. (2010). Equity in mathematics and science outcomes: Characteristics associated with high and low achievement on PISA 2006 in Ireland. International Journal of Science and Mathematics Education, 8(3), 475-496. https://doi.org/10.1007/s10763-010-9199-2
Gunderson, E. A., Park, D., Maloney, E. A., Beilock, S. L., & Levine, S. C. (2018). Reciprocal relations among motivational frameworks, math anxiety, and math achievement in early elementary school. J. Cogn. Dev. 19, 21–46. https://doi.org/10.1080/15248372.2017.1421538
Han, J., & Gao, J. (2008). Research challenges for data mining in science and engineering. In Kargupta, H., Han, J., Philip, S. Y., Motwani, R., & Kumar, V. (Eds.), Next generation of data mining (pp. 27-52). Chapman and Hall/CRC Press.
Hämäläinen, W., & Vinni, M. (2010). Classifiers for educational technology. In Romero, C., Ventura, S., Pechenizkiy, M., & Baker, R. S. (Eds.) Handbook on educational data mining (pp. 57-71.). CRC Press.
Hand, D., Mannila, H., & Smyth, P. (2001). Principles of data mining (Adaptive computation and machine learning). MIT Press.
Hembree, R. (1990). The nature, effects, and relief of mathematics anxiety. Journal for Research in Mathematics Education, 21 (1), 33–46. https://doi.org/10.5951/jresematheduc.21.1.0033
Hertel, S., & Jude, N. (2016). Parental support and involvement in school. In Assessing Contexts of Learning (pp. 209-225). Springer.
Hyde, J. S., Fennema, E., & Lamon, S. J. (1990). Gender differences in mathematics performance: A meta-analysis. Psychological Bulletin, 107(2), 139-155. https://doi.org/10.1037/0033-2909.107.2.139
Kaur, P., Singh, M., & Josan, G. S. (2015). Classification and prediction based data mining algorithms to predict slow learners in education sector. Procedia Computer Science, 57, 500-508. https://doi.org/10.1016/j.procs.2015.07.372
Kılıç Depren, S., Aşkın, Ö. E., & Öz, E. (2017). Identifying the classification performances of educational data mining methods: a case study for TIMSS. Educational Sciences: Theory & Practice, 17(5), 1605–1623. https://doi.org/10.12738/estp.2017.5.0634
Keller, L., Preckel, F., Eccles, J. S., & Brunner, M. (2022). Top-performing math students in 82 countries: An integrative data analysis of gender differences in achievement, achievement profiles, and achievement motivation. Journal of Educational Psychology, 114(5), 966-991. https://doi.org/10.1037/edu0000685
Koyuncu, İ., & Gelbal, S. (2020). Comparison of data mining classification algorithms on educational data under different conditions. Journal of Measurement and Evaluation in Education and Psychology, 11(4), 325-345. https://doi.org/10.21031/epod.696664
Lee, J., Kao, H. A., & Yang, S. (2014). Service innovation and smart analytics for industry 4.0 and big data environment. Procedia Cirp, 16, 3-8. https://doi.org/10.1016/j.procir.2014.02.001
Lee, J., & Stankov, L. (2018). Non-cognitive predictors of academic achievement: Evidence from TIMSS and PISA. Learning and Individual Differences, 65, 50-64. https://doi.org/10.1016/j.lindif.2018.05.009
Linnakylä, P. & Malin, A. (2008). Finnish students' school engagement profiles in the light of PISA 2003, Scandinavian Journal of Educational Research, 52(6), 583-602. https://doi.org/10.1080/00313830802497174
Liu, O. L., & Wilson, M. (2009). Gender differences in large-scale math assessments: PISA trend 2000 and 2003. Applied Measurement in Education, 22(2), 164–184. https://doi.org/10.1080/08957340902754635
Liu, X., & Whitford, M. (2011). Opportunities-to-learn at home: profiles of students with and without reaching science proficiency. Journal of Science Education and Technology, 20(4), 375–387. https://doi.org/10.1007/s10956-010-9259-y
Ma, X. (1999). A meta-analysis of the relationship between anxiety toward mathematics and achievement in mathematics. Journal for Research in Mathematics Education, 30 (5), 520–540. https://doi.org/10.2307/749772
Martínez-Abad, F. (2019). Identification of factors associated with school effectiveness with data mining techniques: Testing a new approach. Frontiers in Psychology, 10, 1-13. https://doi.org/10.3389/fpsyg.2019.02583
Martínez-Abad, F., Gamazo, A., & Rodríguez-Conde, M.-J. (2020). Educational data mining: identification of factors associated with school effectiveness in PISA assessment. Studies in Educational Evaluation, 66, 2-10. https://doi.org/10.1016/j.stueduc.2020.100875
Masci, C., Johnes, G., & Agasisti, T. (2018). Student and school performance across countries: A machine learning approach. European Journal of Operational Research, 269(3), 1072-1085. https://doi.org/10.1016/j.ejor.2018.02.031
OECD (2016). PISA 2015 results in focus. OECD Publishing. Retrieved https://www.oecd.org/pisa/pisa-2015-results-in-focus.pdf
OECD (2017a). PISA 2015 technical report. Organisation for Economic Co-Operation and Development, PISA, OECD Publishing.
OECD (2017b). PISA 2015 assessment and analytical framework: science, reading, mathematic, financial literacy and collaborative problem solving, PISA, OECD Publishing. https://doi.org/10.1787/9789264281820-en
Põder, K., Lauri, T., Ivaniushina, V., & Alexandrov, D. (2016). Family background and school choice in cities of Russia and Estonia: Selective agenda of the Soviet past and present. Studies of Transition States and Societies, 8(3), 5-28.
Ramesh, V., Parkavi, P., & Ramar, K. (2013). Predicting student performance: a statistical and data mining approach. International Journal of Computer Applications, 63(8), 35-39.
Reilly, D., Neumann, D. L., & Andrews, G. (2019). Investigating gender differences in mathematics and science: Results from the 2011 Trends in Mathematics and Science Survey. Research in Science Education, 49(1), 25-50. https://doi.org/10.1007/s11165- 017-9630-6
Romero, C., & Ventura, S. (2007). Educational data mining: A survey from 1995 to 2005. Expert Systems with Applications, 33(1), 135-146. https://doi.org/10.1016/j.eswa.2006.04.005
Romero, C., & Ventura, S. (2010). Educational data mining: A review of the state of the art. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 40(6), 601-618. https://doi.org/10.1109/TSMCC.2010.2053532
Romero, C., & Ventura, S. (2013). Data mining in education. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 3(1), 12-27. https://doi.org/10.1002/widm.1075
Romero, C., Ventura, S., Pechenizkiy, M., & Baker, R. S. (Eds.). (2010). Handbook of educational data mining. Data mining and knowledge discovery series. Chapman and Hall/CRC Press.
Saarela, M., Yener, B., Zaki, M. J., & Kärkkäinen, T. (2016). Predicting math performance from raw large-scale educational assessments data: a machine learning approach. In JMLR Workshop and Conference Proceedings; JMLR: W&CP, 48.
Sherman, B. F., & Wither, D. P. (2003). Mathematics anxiety and mathematics achievement. Mathematics Education Research Journal, 15(2), 138-150. https://doi.org/10.1007/bf03217375
Siemens, G., & d Baker, R. S. (2012, April). Learning analytics and educational data mining: towards communication and collaboration. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 252-254). ACM.
Singh, K., Granville, M., & Dika, S. (2002). Mathematics and science achievement: Effects of motivation, interest, and academic engagement. The Journal of Educational Research, 95(6), 323-332. https://doi.org/10.1080/00220670209596607
Slavin, E. R. (1983). When does cooperative learning increase student achievement?. Psychological Bulletin, 94(3), 429-445.
Taş, U. E., Arıcı, Ö., Ozarkan, H. B., & Özgürlük, B. (2016). PISA 2015 national report. Ministry of Education Publishing.
Tocci, C. M., & Engelhard Jr, G. (1991). Achievement, parental support and gender differences in attitudes toward mathematics. The Journal of Educational Research, 84(5), 280-287. https://doi.org/10.1080/00220671.1991.10886028
Wine, J. (1971). Test anxiety and direction of attention. Psychological Bulletin, 76(2), 92–104. https://doi.org/10.1037/h0031332
Witten, I. H., & Frank, E., (2005). Data mining: Practical machine learning tools and techniques (Second Edition). Morgan Kaufmann Publishers, San Francisco, CA.
Wu, S. S., Willcutt, E. G., Escovar, E., and Menon, V. (2014). Mathematics achievement and anxiety and their relation to internalizing and externalizing behaviours. Journal of Learning Disabilities, 47, 503–514. https://doi.org/10.1177/0022219412 473154
Yayan, B., & Berberoglu, G. (2004). A re-analysis of the TIMSS 1999 mathematics assessment data of the Turkish students. Studies in Educational Evaluation, 30(1), 87-104. https://doi.org/10.1016/j.stueduc.2004.03.005
Yukselturk, E., Ozekes, S., & Kılıç Türel, Y. (2014). Predicting dropout student: an application of data mining methods in an online education program. European Journal of Open, Distance and e-learning, 17(1), 118-133. https://doi.org/10.2478/eurodl-2014-0008
Zhang, J., Zhao, N., & Kong, Q. P. (2019). The relationship between math anxiety and math performance: A meta-analytic investigation. Frontiers in Psychology, 10, 1-17. https://doi.org/10.3389/fpsyg.2019.01613
Additional Files
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Psycho-Educational Research Reviews
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
