Investigation of Mathematics-Specific Trend Variables in PISA Studies with Neural Networks and Linear Regression

This study aimed to examine the importance levels of mathematics-specific trend variables in PISA (Programme for International Student Assessment) 2003 and 2012 in predicting mathematics performance across years with a two-step analysis method. The sample of the study was 9703 Turkish students (N2003=4855 and N2012=4848) selected by clustered and systematic sampling methods. As data analysis methods, multilayer perceptron and radial basis functions techniques of artificial neural networks and multiple linear regression were used. In the two-step analysis, first, the least erroneous model was selected as the analysis method. Then, variable importance analysis was performed with this method. The results with the lowest relative errors were obtained by the multilayer perceptron when compared to radial basis functions. The results of neural network analysis had similar or lower error rates when compared to multiple linear regression. In both PISA cycles, significant predictors were mathematics self-efficacy, mathematics interest, student-teacher relations in school, attitudes towards the school, mathematics self-concept, mathematics instrumental motivation, and teacher support in mathematics classes, respectively. The results were discussed in the light of relevant literature.


Introduction
Large-scale assessments are gaining more and more importance in terms of comparing the education systems of countries in the globalizing world. To this end, many studies are organized by the Organisation for Economic Co-operation and Development (OECD), which are applied worldwide and affect education policies in many countries. Some of these applications are Trends in International Mathematics and Science Study (TIMSS), Progress in International Reading Literacy Study (PIRLS), The Teaching and Learning International Survey (TALIS), and Programme for International Student Assessment (PISA) in which thousands of students are involved from many countries around the world. TIMSS which is conducted in every 4 year-cycles by International Association for the Evaluation of Educational Assessment (IEA) has been designed to compare different education systems by creating a rich data source (at student, teacher, home and school level) of 4 th and 8 th grade students (Foy, Arora, & Stanco, 2017). As the focus country of the present study, The Republic of Turkey participated in this assessment from different grade levels except for years 1999 and 2003. Although Turkish students showed a growing trend in mathematics and science performance over the years, 4 th and 8 th grade students showed a performance below average by ranking 36 th and 24 th places among participating countries respectively in mathematics according to results of 2015 study (see Mullis, Martin, Foyer, & Hooper, 2016). The common purpose of these studies is to compare the potential of each country by considering its own conditions, to analyze education policies, and to make some inferences in this context.
Another study conducted for similar purposes and having valuable impacts on the educational policies of many countries is the PISA application. The PISA study in which it is aimed to compare the performance of students in 15 years-old in mathematics, science and reading literacy, has been conducted in 3 year-periods since year 2000 (OECD, 2019b). In this study, a wide variety of data including background information, questionnaires and performance tests "Mathematical literacy is an individual's capacity to formulate, employ and interpret mathematics in a variety of contexts. It includes reasoning mathematically and using mathematical concepts, procedures, facts and tools to describe, explain and predict phenomena. It assists individuals to recognise the role that mathematics plays in the world and to make the well-founded judgements and decisions needed by constructive, engaged and reflective citizens." (p.75).
This definition expresses mathematics as a discipline that requires the interpretation and use of mathematical knowledge in new and different contexts, rather than as a field where routine operations are performed by following certain procedures. Therefore, although the questions in PISA studies are mostly at the analysis, synthesis or evaluation levels of cognitive domain in Bloom taxonomy (Bloom et al., 1956), students' performance levels were evaluated by defining six proficiency levels which are over 669 and above, 607-668, 545-606, 482-544, 420-481, 358-419 scores (see OECD, 2019a). When PISA mathematics performances of Turkish students participating in PISA study are examined, while its average was 423 points in 2003, it reached its highest level in 2018 with 454 points. In mathematics literacy, the Republic of Turkey ranks 42 nd place out of the 78 participating countries in 2018 and was ranked 33 rd place out of 37 OECD countries. From 2012 to 2018, Turkey's general trend in mathematics literacy seems to be 'steadily positive' (OECD, 2019b).
Variables used in PISA studies are classified as general variables (for all cycles), domain-specific trend variables (for major domain only, included every 9 years), thematic extension variables (extensions within individual cycles), and system-level data, mainly gathered outside of PISA. General variables, which are used for each thematic area in each cycle, consist of knowledge about students, schools, teachers, and some non-cognitive variables. Thematic extension variables are used in individual thematic cycles to collect data related to the extensions in the study such as the use of ICT and career preference, and system-level data, on the other hand, are variables including data collected independent of the study. However, domain-specific trend variables are field-specific non-cognitive and process variables (OECD, 2014b). PISA practitioners consider it as important to use both a set of fixed and constant variables specific to the relevant field applied at all levels of proficiencies as both the reported main variables and using them in the calculation of proficiency estimates to obtain valid and reliable trends at country level. Scores of these trend variables were obtained from scales consisting of varying number of items. The scales for mathematics field and the items are given in Table 1. According to Table 1, general and mathematics processes variables are student-teacher relations at school, disciplinary climate in the mathematics classroom, and teacher support in the mathematics classroom. Non-cognitive outcomes (self and mathematics related cognitions) variables are mathematics anxiety, attitudes towards school (learning outcomes), instrumental motivation to learn mathematics, interest in and enjoyment of mathematics, mathematics self-efficacy, and mathematics self-concept. Scale scores for each variable were estimated by using Item Response Theory (IRT) scaling (OECD, 2005(OECD, , 2014b. As a school climate index used in PISA studies, sense of belonging to school variable was also determined as a domain-specific trend variable. However, since this variable did not used in PISA 2012 as the same as in the PISA 2003 study (three more items were added), it was not used as a trend variable in the present study. In this study, the extent to which trend variables specific to mathematics fields predicted mathematics performance in 2003 and 2012 were examined. Descriptive statistics (mean, standard deviation, and results of reliability analysis) concerning those variables are provided in method and findings parts of the present study. a) Students don't listen to what the teacher says. b) There is noise and disorder. c) The teacher has to wait a long time for students to <quieten down>. d) Students cannot work well. e) Students don't start working for a long time after the lesson begins. Teacher support in the mathematics classroom (TEACHSUP) a) The teacher shows an interest in every student's learning. b) The teacher gives extra help when students need it. c) The teacher helps students with their learning. d) The teacher continues teaching until the students understand. e) The teacher gives students an opportunity to express opinions. Non-cognitive outcomes -Self and mathematics related cognitions Mathematics anxiety (ANXMAT) a) I often worry that it will be difficult for me in mathematics classes. b) I get very tense when I have to do mathematics homework. c) I get very nervous doing mathematics problems. d) I feel helpless when doing a mathematics problem. e) I worry that I will get poor <marks> in mathematics. Attitudes towards school: Learning outcomes (ATSCHL) a) School has done little to prepare me for adult life when I leave school. b) School has been a waste of time. c) School helped give me confidence to make decisions. d) School has taught me things which could be useful in a job. Instrumental motivation to learn mathematics (INSTMOT) a) Making an effort in mathematics is worth it because it will help me in the work that I want to do later on. b) Learning mathematics is worthwhile for me because it will improve my career <prospects, chances>. c) Mathematics is an important subject for me because I need it for what I want to study later on. d) I will learn many things in mathematics that will help me get a job. Interest in and enjoyment of mathematics (INTMAT) a) I enjoy reading about mathematics. b) I look forward to my mathematics lessons. c) I do mathematics because I enjoy it. d) I am interested in the things I learn in mathematics.
Mathematics self-efficacy (MATHEFF) a) Using a <train timetable>, how long it would take to get from Zedville to Zedtown b) Calculating how much cheaper a TV would be after a 30 percent discount c) Calculating how many square metres of tiles you need to cover a floor d) Understanding graphs presented in newspapers e) Solving an equation like 3x + 5 = 17 f) Finding the actual distance between two places on a map with a 1:10,000 scale g) Solving an equation like 2(x+3) = (x + 3)(x -3) h) Calculating the petrol consumption rate of a car. Mathematics self-concept (SCMAT) a) I am just not good at mathematics. b) I get good <marks> in mathematics. c) I learn mathematics quickly. d) I have always believed that mathematics is one of my best subjects. e) In my mathematics class, I understand even the most difficult work.
Although the factors affecting students' PISA mathematics performance are very diverse, the studies examining the variables such as self-efficacy, anxiety, attitude, motivation, etc. in the affective domain come to the forefront. For example, it has been observed that some of the significant predictors of student achievement in PISA studies are self-efficacy (Cheema & Kitsantas, 2014;Kitsantas, Cheema, & Ware, 2011;Pitsia, Biggart, & Karakolidis, 2017), disciplinary climate in maths lessons (Bodovski, Nahum-Shani, & Walsh, 2013;Cheema & Kitsantas, 2014;Huang & Zhu, 2017;Ma & Wilmms, 2004), school disciplinary climate, interest in mathematics, instrumental motivation and student-teacher relationship (Pitsia, Biggart, & Karakolidis, 2017;Sheen, Lee, & Kim, 2009). In Turkey, Demir, Kılıç, and the Depri (2009) analyzed PISA 2003 data and they found that improving school climate and student learning strategies has significant effect on students' mathematics performance. In another study, while school location, gender, and interest in and enjoyment of mathematics variables were positively related to mathematics achievement, elaboration learning strategies were found to be strongly negative correlated (Demir & Kılıç, 2010). Arikan (2014) examined PISA 2012 mathematics achievements of Turkish students in terms of various factors and the results revealed that high belief, motivation, self-efficacy, self-concept, and low anxiety affect the success positively.
On the other hand, Uysal (2015) showed that the mathematics achievement of Turkish students is not related to teacher-student relations, classroom management and sense of belonging to the school but it is related to mathematics interest, mathematics self-concept and mathematics anxiety on a structural equation model. In their study Özberk, Kabasakal, and Öztürk (2017), mathematics related non-cognitive variables significantly affects mathematics achievement. They also found that the most predictive variable at student-level is self-efficacy and at the school-level is ratio of the number of mathematics teachers to the number of all teachers at school. In the present study, on the other hand, the predictive power of domain-specific trend variables in estimating PISA 2003 and 2012 mathematics performance were examined comparatively by using a two-step analysis method. The reason for using the data in 2003 and 2012 is due to the fact that mathematics-specific trend variables were used only in these years in which mathematics field was the main theme and no trend analysis has been done so far. In the studies on what factors affect Turkish students' PISA mathematics performance, the methods such as multiple linear regression (e.g. Arikan, 2014;Demir, Kılıç & Depren, 2009), hierarchical and hierarchical genaralized linear modeling (e.g. Özberk, Kabasakal, & Öztürk, 2017;Demir & Kılıç, 2010), structural equation modeling (e.g. Uysal, 2015), decision trees (e.g. Aksu & Güzeller, 2016), probit regression (e.g. Güzeller, Eser, & Aksu, 2016), logistic regression and neural networks (e.g. Tepehan, 2011) were used. However, the two-step analysis method used in the present study, consist of selecting the least erroneous method among neural network methods (multilayer perceptron and radial basis functions) and multiple linear regression models in predicting mathematics performance and then performing variable importance analysis by using selected method/s. Thus, it is aimed to prevent possible biased or erroneous results that can be obtained by using only one method. In the related literature, it is also suggested that different methods can be effective for different data types (Romero et al, 2013) and similar procedures were followed in some other studies (e.g. Göker, 2012;Yurdakul & Topal, 2015). Information on the structure and mechanism of artificial neural networks is given in the method section of the present study.
The aim of this study is to examine the importance levels of mathematics-specific trend domain variables in predicting PISA 2003 and 2012 mathematics performances of the students with artificial neural networks and multiple linear regression. For this purpose, it was sought to find answers to the following questions: (1) How do mean values of mathematics-specific trend variables vary across years?
(2) Which model or models has the least erroneous results in predicting students' mathematics performance?
(3) What are the importance levels of domain-specific trend variables in predicting students' mathematics performance in different PISA studies?

Research Design
This study is a survey research in terms of describing existing characteristics, ideas, and approaches of the students. Besides, it has a correlational quantitative structure since the relationships between variables in PISA studies are examined in the light of findings obtained with different analysis techniques.

Participant (Subject) Characteristics
The sample of the research was 9703 students of whom 4855 participated in PISA 2003 and 4848 in PISA 2012. The participants were selected by clustered and systematic sampling by the PISA practitioners. Detailed information about sampling process is provided by OECD (see OECD, 2005;2014b).

Data Collection Tools
The data collection tools used in the present study are student questionnaires and mathematics cognitive test of PISA 2003 and 2012 studies. Student questionnaires consist of questions about students' background, home, school information, and learning experiences, while the mathematics test consists of open-ended and multiple-choice question types based on real life situations. The students are given 30 minutes for questionnaires and two hours for cognitive tests during the applications. However, these durations may differ depending on whether countries take an additional scale or not apply some scales. Detailed information and procedures regarding the data collection and implementation process are shared as open access by OECD (see OECD, 2005;2014a;2014b).
PISA practitioners consider it as important to use both a set of fixed and constant variables specific to the relevant field applied at all levels of proficiencies as both the reported main variables and using them in the calculation of proficiency estimates to obtain valid and reliable trends at country level. Some detailed information about these trend variables were provided at the introduction part of the present study (see also Table 1). Mathematics self-efficacy, math anxiety, mathematics interest, mathematics instrumental motivation, attitude towards the school, disciplinary climate in mathematics courses, mathematics self-concept, student-teacher relations in school and teacher support in mathematics courses were mathematics-specific trend variables in PISA 2003 and 2012 studies. Although the sense of belonging to the school is also a trend variable, this variable has not been included in the scope of the present study since this variable was not used in the PISA 2012 study exactly as it was in PISA 2003. The reliability of the scales is provided in Table 2. According to Table 2, except for attitudes towards school (learning outcomes) and mathematics self-concept scales, the reliability of the other scales is above the medium level (α > .70) in both PISA studies. While the reliability of the mathematics self-concept scale is at medium level (.30 < α < .70) in both studies, the reliability of the attitudes towards school (learning outcomes) scale is close to zero. The acceptable threshold of the Cronbach's alpha internal consistency coefficient in the literature is .70 (Özgüven, 2019). In PISA reports, items parameters (reliability, mean, and standard deviations) for these variables were calculated and the results for them were compared along the years as well as across countries (see OECD, 2005OECD, , 2014b. Although the reliability of these scales is below acceptable values, they were used in the present study as they were used and reported in both PISA studies. In addition, it is seen that the reliability levels of all mathematics-specific trend variables scales are close to each other over the years. Item Response Theory (IRT) scaling was used in order to estimate scale scores for each variable (OECD, 2005(OECD, , 2014b.

Data Analysis
In this study, the differentiations of mathematics-specific trend variables used in PISA 2003 and 2012 studies over the years were examined by independent samples t-test. Multilayer perceptron (Minsky & Papert, 1969) and radial basis functions (Broomhead, & Lowe, 1988) techniques of artificial neural networks and multiple linear regression were used to determine the least erroneous method in predicting the extent to which these variables predict mathematics achievement. Artificial neural networks, inspired by real neural cells, emerged with a study by McCulloch and Pitts (1943) as a computer science. While the first training rule was developed by Hebb in 1949, the perceptron model introduced by Rossenbalt (1961) proposed an iterative process for the first time. Later, Minsky and Papert (1969) demonstrated that this model was not effective in solving complex problems and proposed multi-layered models. In the following years, the models developed by Kohonen (1984), Rummelhart, Hinton and Williams (1986), Broomhead and Lowe (1988) and Chua and Yang (1988) made important contributions to the development and use of new neural network models. Radial basis functions and multilayer perceptron are feed-forward networks. Multilayer perceptron are complex networks that consist of multiple perceptrons and contain hidden layers and hidden nodes (Tan et al., 2014). Radial basis functions use the Gauss function as the threshold function. The Sigmoid function in the output layer is the linear combination of hidden layer values corresponding to the posterior probability (Akpınar, 2014).
Compared to decision trees, artificial neural networks are more robust to missing and noisy data by means of inter-node weightings (Dunham, 2003). When compared with multiple linear regression and logistic regression, it does not require any assumptions about the model, and it can perform analysis effectively in cases where the dependent variable is continuous or categorical (Chattamvelli, 2009). It is recommended to overview Han, Kamber, and Pei (2011) for more information (properties, structure, mechanism, classification of neural networks and comparison of them with other methods) about neural networks and see the study of Koyuncu (2018) for an example application. Although PISA datasets have a nested structure, since we used specific variables at student level, the data was handled as single level.
In the present study, two-thirds of the data were used to train the neural networks models. Analyses were repeated 30 times and average relative errors were reported. Dependent sample t-test was used to compare the results of multilayer perceptron and radial basis functions techniques. Neural network results with the least errors including sum of squares errors were also provided in Appendix A. The average relative error of artificial neural network models was subtracted from 1 to obtain the accuracy rate. The accuracy rate and variable significance values obtained by the artificial neural networks model with the least errors were compared with the total explained variance (R 2 ) and standardized regression coefficients (β) of the multiple linear regression analysis. Multiple linear regression was performed with 80 replications by using survey weighs and all plausible values via IBM SPSS software. SPSS syntaxes were generated via IDB Analyzer software (Available online at http://www.iea.nl/data.html).

Differentiation in Mean Scale Scores across Years
In order to examine how Turkish students' scores of mathematics-specific trend variables differ across years, an independent t-test was performed, and the findings were presented in Table 3. According to Table 3, except for disciplinary climate in maths lessons and student-teacher relations at school variables, significant changes occurred in other variables from 2003 to 2012. Accordingly, Turkish students' math self-efficacy levels increased significantly, and instrumental motivation in mathematics, interest in mathematics, mathematics anxiety, attitudes towards school, mathematics self-concept, and teacher support in maths lessons

The Most Predictive Model in Estimating Mathematics Performance
In order to select the most predictive model in estimating mathematics performance, the error rates of the analysis methods were compared. The aim of this process is to select the least erroneous model/s. The accuracy and explained variance rates of the models that were obtained by subtracting the error rates from 1 are given in Table 4. According to Table 4, the results with the lowest relative errors were obtained by the multilayer perceptron and multiple linear regression methods. In order to compare artificial neural network models statistically, a series of paired-sample t-tests was performed. The results were given in Table 5. According to paired t-test results in Table 5, average relative error of multilayer perceptron models is significantly lower than that of radial basis functions models (p < .001). However, multilayer perceptron and multiple linear regression models have similar accuracy and explained variance rates in PISA 2003 data (Table 4). Therefore, the results of multilayer perceptron and multiple linear regression analysis were used together to evaluate the importance of the trend variables in for this data. For PISA 2012 data, on the other hand, multilayer perceptron results were used to compare trend variables due to its small relative error. In addition, total explained variance obtained by multiple linear regression shows that 9 trend variables explained 36% of variance in mathematics performance for PISA 2003 data and 27% of variance for PISA 2012 data. These results show that there are other variables (such as gender, escs, income, etc.) that explain PISA mathematics performance more than trend variables.

The Importance Levels of Trend Variables in Predicting Students' Mathematics Performance
After selecting multilayer perceptron and multiple linear regression as the least erroneous models, variable importance analysis of two models were performed. The results are given in Table 6. According to Table 6, mathematics self-efficacy was found to be as the most predictive variable in estimating Turkish students' mathematics performance when compared to other trend variables. However, instrumental motivation in mathematics and teacher support in maths lessons are found to be as the least important variables. Besides, these variables are not even statistically significant according to multiple linear regression results. The significance order of the variables in the analysis performed with PISA 2003 data was the same for neural network and regression models. However, except mathematics self-efficacy, instrumental motivation in mathematics and teacher support in maths lessons variables, the order of other variables has changed from 2003 to 2012. Graphical representation of variable importance results was given in Figure 1.

Figure 1. Graphical Representation of Variable Importance Results
In Figure 1, since mathematics anxiety, mathematics self-concept, interest in mathematics, and attitudes towards school variables were negatively correlated with mathematics performance, they were presented below x axis. The importance level of disciplinary climate in maths lessons, interest in mathematics, and attitudes towards school variables in predicting mathematics performance increased while the importance of the other variables declined over years.

Discussion
The purpose of this study is to reveal the importance levels of mathematics-specific trend variables used exactly as in the same formats in PISA 2003 and 2012 studies in predicting Turkish students' mathematics performances. According to the findings of this study, mathematics self-efficacy is the most important variable that predicts students' mathematics performances. Similarly, in many studies, mathematics self-efficacy has been found to be an important predictor of mathematics achievement (Cheema & Kitsantas, 2014;Chiu, 2017;Garcí a et al., 2016;Kitsantas, Cheema, & Ware, 2011;Koyuncu, 2018;Peters, 2013;Pitsia, Biggart, & Karakolidis, 2017;Schöber et al., 2018). For example, according to PISA 2003 results, it was seen that high self-efficacy levels of American students positively affect their success (Cheema & Kitsantas, 2014;Kitsantas, Cheema, & Ware, 2011). There is a similar situation for Greek students (Pitsia, Biggart, & Karakolidis, 2017). However, in a longitudinal study, the self-efficacy levels of 11 th grade students were not found to affect the National Assessment of Educational Progress (NAEP) mathematics achievement in the 12 th grade. Although this result also shows that the importance level of self-efficacy in affecting success may change over the years, an opposite result was found in a similar longitudinal study (i.e. Schöber et al., 2018) with German students. Schöber et al. (2018) found that math self-efficacy is associated with mathematics success in the following years.
The second most important variable predicting mathematics performance of Turkish students was disciplinary climate in maths lessons in both PISA cycles. Similar results were obtained in some other studies (Bodovski, Nahum-Shani, & Walsh, 2013;Cheema & Kitsantas, 2014;Huang & Zhu, 2017;Ma & Wilmms, 2004). According to PISA 2003 results, disciplinary climate in maths lessons reduce the gap in American students' academic success (Bodovski, Nahum-Shani, & Walsh, 2013;Cheema & Kitsantas, 2014;Huang & Zhu, 2017). In another study with American students, the disciplinary climate in mathematics lessons variable was found to be closely related to mathematics achievement (Ma & Wilmms, 2004). In another study, on the other hand, it was found that classroom climate did not significantly predict success (Peters, 2013). This result shows that the situation may change for different students with different characteristics. In another study comparing the factors affecting the mathematics success of American, Korean and Japanese students, school disciplinary climate was found to be a significant predictor for all three countries, while student-teacher relationship was found to be a significant predictor only in Japan (Sheen, Lee, & Kim, 2009). In Turkey, while student-teacher relationship variable is a significant predictor of mathematics success, its significance level is lower than most of the other variables. While the significance level of interest in mathematics variable is low, it is a more significant predictor than instrumental motivation. Compared to other countries, while it is similar for Korean and Japanese students, there is a reverse situation for American students (Sheen, Lee, & Kim, 2009). Chiu (2017) compared Asian and U.S. regions and found that mathematics interest is associated with mathematics achievement in 65 regions. These results show that the significance levels of variables in predicting mathematics success are affected by the differences between countries.
According to the results of the analyses, instrumental motivation in mathematics and teacher support in mathematics lessons variables do not affect Turkish students' mathematics performance in both PISA cycles. In some other studies, on the other hand, it is found that teacher support is a significant predictor of mathematics achievement (Sheen, Lee, & Kim, 2009;Yu & Singh, 2018). For example, in the study conducted with the High School Longitudinal Study of 2009 data, it was found that teacher support affects mathematics achievement and students with high previous achievements received more teacher support (Yu & Singh, 2018). In terms of instrumental motivation, this variable was found to be one of the most important predictors of PISA mathematics achievement of Greek students (Pitsia, Biggart, & Karakolidis, 2017). However, while the situation is the same for American students participated in PISA studies, there is a reverse situation for Korean and Japanese students (Sheen, Lee, & Kim, 2009). Other variables which are important predictors of Turkish students' mathematics achievements are attitudes towards school, math anxiety and mathematics self-concept. Similarly, in other studies, it is found that attitudes towards school (Ajisuksmo & Saputri, 2017;Geesa et al., 2019;Lipnevich, Preckel, & Krumm, 2016), mathematics anxiety (Gunderson et al., 2018;Passolunghi et al., 2016) and mathematics self-concept (Chiu, 2017;Lee & Kung, 2018;Susperreguy et al., 2018) are significant predictors of mathematics achievement.
The results revealed that the scale scores for disciplinary climate in mathematics lessons and student-teacher relations at school variables that predict significantly Turkish students' mathematics performances have remained stable over the years. Although PISA mathematics achievement increased from 2003 to 2018 (MEB, 2019), the fact that these variables remain constant indicates that changes in education policies do not affect teacher-student relationships in mathematics classrooms. The significant increase in students' mathematics self-efficacy and decrease in mathematics anxiety can be explained by radical changes made in 2004 in the mathematics curriculum with the adoption of the constructivist approach. Similarly, the decrease in the level of teacher support in mathematics may have resulted from the establishment of a learning environment where students are more active instead of the teacher and shape their own learning as a result of this constructivist approach. The reasons for decline in the level of variables related to Turkish students' attitude towards mathematics can be listed as the implementation of the 4 + 4 + 4 (12-years compulsory education) system, the rapid change in examination systems, and the adoption of radical changes in school types. However, which factors can be fully effective in the occurrence of this situation can be examined with some empirical studies.
The results of the present study showed that lower erroneous results were obtained with multilayer perceptron when compared to the radial basis functions, and similar or lower erroneous results are obtained with multiple linear regression. These results showed that artificial neural networks are at least as effective as regression models in predicting students' success. Moreover, according to the results of multilayer perceptron and regression analysis, the significance order of variables in prediction was the same for the PISA 2003 study. This result showed that these two methods can be used effectively in similar types of data. In some other studies, while it was found that artificial neural network methods can perform at least as much performance as regression methods (Koyuncu, 2018;Tepehan, 2011), in others, neural network methods were found to be more effective (Bahadır;Çırak, 2012). Shahiri, Husain and Rashid (2015) found that the most effective method was artificial neural networks with 98% predictive accuracy when looking at the average accuracy rates calculated in studies published in international databases between 2002 and 2015. Romero et. al (2013), on the other hand, showed that different methods can be effective for different data types. Therefore, at this point, two-step data analysis method used in the present study, which consists of choosing the least erroneous analysis technique and performs variable importance analyses with this technique, accordingly, becomes more important.
The purpose of this two-stage process is to prevent possible biased and erroneous results that can be obtained by using only one analysis method. When related literature was reviewed, it is possible to find some studies in which similar processes are followed (Göker, 2012;Yurdakul & Topal, 2015). For example, Göker (2012) aimed to develop a model that can predict possible success of students before taking the university entrance exam by using the variables that affect the success of this exam. For this purpose, the analyses were performed with Naive Bayes, k-nearest neighborhood, decision trees, Bayes networks and neural networks and Naive Bayes method (87% performance), which gave the least erroneous results, was chosen. Then, using Naive Bayes, the possible success of other students entering the system was estimated. A similar procedure was followed by Yurdakul and Topal (2015) in order to determine the factors that affect students' success, For this purpose, first of all, they performed analyses using artificial neural networks, k-nearest neighborhood and decision trees models, and multilayer perceptron model (89% performance) of neural networks which yielded the least erroneous results, was chosen as the analysis method for further analyses. Then, the most important variables (education level and economic status of the family, student's participation in extracurricular activities, availability of a suitable working environment at home, and parents-teachers communication) were determined by using this method.

Conclusions, Limitations, Implications and Suggestions
The results of the present study showed that artificial neural networks yielded as much as or less erroneous results than regression models in predicting students' success. Hence it is advised to use such kind of techniques used in data mining and machine learning as an alternative to the traditionally used regression models in predicting students' performances. The two-step analysis method used in the present study provides unbiased and results with high accuracy. While using this model to solve educational problems, it is suggested to use as much analysis techniques as possible since different methods may perform best for different data types.
It was found that Turkish students' mathematics self-efficacy levels have a high importance in predicting their PISA 2003 and 2012 mathematics performances. Similarly, it was observed that the importance levels of mathematics anxiety, disciplinary climate in mathematics courses and mathematics interest variables are high, respectively. However, instrumental motivation in mathematics and teacher support in mathematics lessons are not significant predictors of mathematics achievement. By conducting some empirical studies, the accuracy and possible causes of these results can be investigated. In addition, mathematics-specific trend variables used in PISA 2003 and 2012 explained approximately one third of the variance in mathematics achievement. In further studies, the researchers may investigate to determine the levels of the other variables in explaining the variation in PISA mathematics scores. In addition, examining these factors together with some other demographic, environmental and socio-cultural factors will provide important ideas for understanding the mathematical behavior of the students.
Educational outcomes of the present study are limited to PISA (2003/2012) Turkey data. The analysis method proposed in this study can be used in future studies with data obtained from different assessments/studies and students from different countries/regions. In addition, the results are also limited to two neural networks models and multiple linear regression models. Therefore, similar studies can be carried out with a variety of models including decision trees, Bayes algorithms, self-organizing maps, support vector machines, different model of neural networks and regression analyses. The results are also limited to mathematics-specific trend variables used in PISA studies. In future studies, the significance levels of other non-cognitive variables might be assessed and compared with those variables. Moreover, the reliability of some scales are below acceptable values, but they were used in the present study as they were used and reported in both PISA studies. The results of this study can be compared with the results of the analysis with mathematics-specific trend variables that will be used again in the 2021 PISA study to see the trend over the years. ISSN 1927-2677E-ISSN 1927-2685  Note. MLP: Multilayer perceptron, RBF: Radial basis functions.

Copyrights
Copyright for this article is retained by the author(s), with first publication rights granted to the journal.
This is an open-access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/