Sustainability Pedagogy: Effects and Impacts on High School Students’ Knowledge, Behaviour Intention and Actual Behaviour about Sustainability

The sustainability pedagogy has been considered a central framework for projects in Education for Sustainability due to its importance of equipping people with knowledge, values, skills, attitudes, and behaviors compatible with sustainability. Despite its theoretical-practical importance, there is still a gap in works that demonstrate its actual effectiveness. This work presents the outcomes of the Education for Sustainability Project (EfS) (2010-2016) developed by high school teachers and students in Sao Paulo, Brazil. The EfS considered a particular sustainability pedagogy as a guide for its activities and interventions in the school through active learning methodologies and multi- stakeholder social learning. Its effects and impacts on participant students’ knowledge, actual behavior, and behavior intention were assessed quali-quantitatively by open-ended questions pre and post-tests. The participant students presented marked changes in knowledge and behaviors (compared to a control group) considering the adopted sustainability pedagogy, evidencing its effectiveness as a framework for different projects aiming at the formation of more sustainable individuals and societies.


Introduction
Several different official documents related to Environmental Education (EE) and Education for Sustainable Development (DESD) acknowledge the importance of equipping people with knowledge, values, skills, understanding, attitudes, and behaviors compatible with a more sustainable way of life (UNESCO, 2005;UNECE, 2005;UNECE, 2011;UNESCO, 2014;UNESCO, 2017).
This need to form more co-responsible citizens has inspired educators to create and run a huge variety of educational experiences based (or not) on different theoretical-practical frameworks. In that direction, some authors have proposed different 'sustainability pedagogies' in order to work as guides for the planning, executing, and evaluating programs and projects on education for sustainability (Redman, 2013;Burns, 2015;O'Byrne, Dripps, Nicholas, 2015;Burns, Kelley, Spalding, 2019).
From a pedagogical point of view, Redman (2013) cites the importance of considering a particular and transformative pedagogy capable of competently guide a hands-on teaching-learning process by building new sustainability life skills. Inspired by his work project's tutors built a specific 'sustainability pedagogy' for the EfS, which set important pedagogical guidelines followed by all student groups (2013)(2014)(2015)(2016) considering theoretical and practical activities. The sustainability pedagogies should allow the development of different competencies, empowering students, and the community to identify, understand, and collectively search for solutions to complex challenges (Frisk & Larson, 2011). experiences remains a big challenge for educators and practitioners (Monroe, 2010), and because of that, one relevant question remains not properly addressed: Are the proposed sustainability pedagogies really effective to change people's knowledge, values, skills, understanding, attitudes, and behaviors?
The objective of this paper is to present the main effects and impacts of activities and interventions based on the sustainability pedagogy on knowledge, actual behavior, and behavior intention of high school students' who participated in the Education for Sustainability Project (hereafter EfS) from 2010 to 2016.

Methods
The EfS was conducted for seven years (2010)(2011)(2012)(2013)(2014)(2015)(2016) in a private high school in Sã o Paulo, Brazil. Most of its students' families belong to the Sã o Paulo socioeconomic elite. The school had historically no concerns about sustainability and also no tradition of creating and developing programs and projects related to environment conservation and sustainability more generally. The EfS project can be considered the very first in the school that engaged the students to rethink the space and behaviors and propose transformation towards a more sustainable school. We consider this educational experience highly significant, considering most of the students will become community leaders and decision-makers.
The groups of students developed the activities weekly (during 100 minutes) as an extracurricular course intended for the 1st and 2nd grades high school students (between 14 to 17 years old), which voluntarily took part in the EfS. The overarching goal of the project was 'to turn high school students into investigators and protagonists in the building of a more sustainable school", and for that considering structural and socioecological aspects such as water and energy use, mobility, garbage disposal, and waste management, among others.
During the classes students, teachers, and different invited school's stakeholders (for instance, inspectors, school management, maintenance personnel, cleaning staff) worked in partnership to create and execute practical interventions to build a more sustainable school, always accordingly the outcomes of different diagnostic tools (created by teachers and students) applied to the internal community.
In this paper, we focus on the main events related to the planning and implementation of an end-to-end waste management system (between 2013 and 2016(Note 1)) and if and how the 'EfS sustainability pedagogy', discussed below, was able to change participant students' knowledge, behavior intention and actual behavior about sustainability. equipment and infrastructure were the main limiting factors for effective selective disposal. Those results supported the decision to create an end-to-end waste management system. Overview of the activities related to the implementation of an end-to-end waste management system.
It is worth noticing that new groups (with different students) were formed every year. So, four separate groups of voluntary high school students continuously carried out all the interventions (described in Table 1), evidencing the concern with the permanence and continuity of previous actions.
Despite that, all four voluntary groups of students followed similar educational paths based on four general guidelines and a conceptual framework called 'sustainability pedagogy' composed of four key components (Table 2). Teachers and students drew up collaboratively both frameworks during the first three years of the EfS (2010EfS ( -2012 and guided all activities carried out by the students from 2013 to 2016.

The General Guidelines and Sustainability Pedagogy.
Based on the experiences of the first three years of the EfS, teachers, and students defined four general guidelines for the project, as follows: 1-Promote students as investigators, creators, implementers, and evaluators of creative solutions for school challenges related to sustainability; 2-Turn teachers into facilitators of the teaching-learning process linked to the development of interventions; 3-Establish a cooperative internal and external partnership network; 4-Form students able to collectively and responsibly intervene in the reality of their community.
Those four general guidelines are directly related to a new educational practice, once they point to the importance of building a new type of student and teacher who works in partnership with each other and with their community to create new forms of contextualized knowledge.
In addition to the general guidelines, the sustainability pedagogy would be capable of guiding a hands-on teaching-learning process by building new sustainability life skills and competencies (Redman, 2013). And besides that, empower students to identify, understand, and collectively search for solutions to complex challenges (Frisk & Larson, 2011). The EfS sustainability pedagogy is composed of four pedagogical key-components based on five different authors (Table 2). Based on the understanding of the complexity inherent of environmental and social systems. The comprehension of patterns and interactions allows changes in different components and processes in a more efficient way.

Foresighted strategizing b
A process that engages individuals in the design and implementation of an ideal vision about the future, making it possible to understand that actions and choices we make today have consequences in the future.
Critical and reflexive thinking c Challenges each individual to analyze different world views and how knowledge and opinions can be shaped by those around us. It promotes a deep understanding of community interests, media influences, and propaganda in our lives.
Stakeholder engagement and group collaboration d Sustainability demands collaboration to guarantee the presence of different values and perspectives on determining how things should be. It must go beyond research and consultation processes and requires multiple stakeholders in a joint analysis of reality. The creation of content and action networks promotes the transformation of social structures, building a way to share responsibilities and learning.
The combination of different aspects presented in the literature allowed the construction of a particular sustainability pedagogy applied to the EfS Project. The EfS pedagogy, besides being a guide to minds-on and hands-on activities, has the global objective to promote transformative and transgressive learning once its global objective is forming more participative, collaborative, and solidary citizens though reflexive and contextualized social learning (Burns, 2015;Lotz-Sisitka et al., 2015).
In this way, the sustainability pedagogy is also core from a procedural and analytical framework, allowing that effects and impacts analysis consider the whole participating students cohort (2013-2016).

Evaluating the effects and impacts of the sustainability pedagogy on participating students.
The evaluation of effects and impacts of programs and projects is considered an essential principle in documents in Environmental Education (EE) and Education for Sustainable Development (EDS), and it remains a big challenge for educators and practitioners (Monroe, 2010).
Considering the evaluation of the sustainability pedagogy: effects are considered "intended or unintended change due directly or indirectly to an intervention" and impacts are "higher-level effects of a program that occur in the medium or long term, and can be intended or unintended and positive or negative" (USAID, 2009).
We assessed the effects by using pre-and post-tests composed of three open-ended questions applied for both participant students (experimental group; pre n=72; post n=42) and a control group formed by randomly selected students (control group; pre n=78; post n=66) considering a 10-month time interval between tests. Students that leave the project or school are responsible for the difference in pre-and post-test sample sizes. The three questions applied for both groups were: Question 1: What do you understand by 'sustainability'?
Question 2: How do you act sustainably in your everyday life?
Question 3: How could you act more sustainably in your everyday life?
Question 1, assesses knowledge about sustainability; question 2, evaluates actual behaviors related to sustainability; and question 3, evaluates a specific component called behavior intention (BI), which is "assumed to be the immediate antecedent of behavior" accordingly to the theory of planned behavior (Ajzen 1985(Ajzen , 2012. The answers for each question were read and coded by using the online tool Qualitatively Content Analyses map (QCAmap®) (Mayring, 2014).
In December of 2017, to evaluate possible impacts of the sustainability pedagogy, ten former participating students (between 2013 and 2015; ~36% of the total cohort) were randomly selected and invited to write an essay considering the following question: How the EfS might have influenced your personal, academic and professional life?
We qualitatively and quantitatively coded the essays by using the online tool Qualitatively Content Analyses map (QCAmap®) (Mayring, 2014). The effects and impacts are below.

Results
The results considering effects and impacts of the 'sustainability pedagogy' are presented below.
We read the answers and deductively (item 3.1) or inductively (items 3.2. and 3.3) analyzed the answers of both experimental and control groups to verify possible effects on knowledge, actual and BI; and impacts (item 3.4) related to EfS participation.

Question 1: Effects on Knowledge
The answers of experimental and control groups to question 1 (in pre-and post-tests) were read and classified according to the criteria (or dimensions) of sustainability proposed by Sachs (2002;adapted) (Table 3), thus configuring a deductive analysis. It was possible to classify some answers in more than one criterion (Figure 1). The x-axis represents the criteria of sustainability; the y-axis the percentage difference of each criterion considering post and pre-tests (% criterion in post-test -%criterion in pre-test). Therefore, positive percentages represent an increase of criterion's relative importance, while negative percentages represent the opposite. Income distribution, quality of life, social justice and equality and fair natural resources access. Economical Balance between economic growth and development, continuous modernization capability of production, scientific and technological research and insertion in the international economy. Cultural Balance between tradition and innovation, autonomy in the elaboration of national projects, combination of trust and openness to the world. Territorial Balance between urban and rural, improvement of urban environment and the development regional strategies. Political Democracy, human rights, peace and international cooperation regarding scientific and technological transference.
* Sum of cultural, territorial and political criteria variations. There was a reduction in the number of associations between 'sustainability' and the ecological criterion for the experimental group and, at the same time, an increase of associations considering the social dimension. The same, but a less conspicuous pattern, is observed for the control group. All other criteria presented variations below 5% between pre and post-tests.

Question 2: Effects on Actual Behaviors.
The answers of experimental and control groups to question 2 (pre and post-tests) were qualitatively analyzed, generating a set of codes configuring an inductive analysis (Figure 2).
There was significant increase in codes present in the "Others" group considering the experimental group. Once more, the frequency variations for the control group are less conspicuous than those for the experimental group.

Question 3: Effects on Behavior Intentions (BI)
In the same way as to question 2, different BI was coded by inductive analyzes considering the responses given by both experimental and control groups (in pre and post-tests). The results are summarized in Figure 3.  The frequency variations for different BI were less significant for all three parameters analyzed. However, we noticed an increase in the relevance of 'influence on peers' (for both groups) and 'reduce consumption' (for the experimental group).

Impacts of EfS in Former Participants
In 2017, we randomly selected ten students from 2013 to 2016 were (4 from 2013 and 2014 and 2 from 2015) to identify possible impacts related to EfS participation. They wrote an essay regarding their memories related to the experiences lived in the EfS and some recognizable influences over their personal and professional choices. We read all the issues and coded them using qualitative content analysis (Mayring, 2014), and they were posteriorly quantified (Figure 4).

Figure 4. Absolute Frequencies of Citations Regarding Impacts on Former Students (Students 2, 3 and 4 Years after Intervention; n= 92 Citations)
The four most-cited categories (~70% of total citations) are related to different items of the proposed sustainability pedagogy. Furthermore, it is meaningful (from an educational/personal point of view) that former participants can acknowledge that the activities developed in the EfS were able to awaken the perception that they have become agents of change in their reality.

Discussion
Figures 1 to 3 show that all parameters analyzed (knowledge, actual behavior, and behavior intention) present more variation for the experimental group in comparison to the control group. Those results can be considered very significant because demonstrate the importance and effectiveness of the sustainability pedagogy over the participant students considering a more longitudinal experience. Besides that, Figure 4 shows that former students were able to recognize different and significant impacts related to the lived experiences in the EfS. The discussion about each indicator analyzed follows below.

Knowledge and Actual Behavior: from Environmental to Socio-Environmental
Even though 'sustainability' encompasses several different dimensions (or criteria, as defined by Sachs, 2002), the most common people's associations are related to environmental (or ecological) issues such as nature and resources preservation/conservation, pollution, waste disposal, water, and energy economy, etc. The connections with social, political, cultural, and economic dimensions are rarer .
The increase in knowledge alone is not enough to change behaviors (Fabrigar et al., 2006;Corace & Garber, 2014), and the opportunities to critically create solutions form collective challenges can be considered key to attitudes and behavior changes (Resnick, 2020).
The analyses of knowledge and actual behaviors pre-tests (Figures 1 and 2) show this trend for both experimental and control groups. Nevertheless, the experimental group post-test shows a migration from ecological to social criteria; a result not so detectable for the control group.
The experimental group's actual behaviors post-test shows significant increase of categories such as 'influence on peers', 'reflect about challenges' and 'improve relationship quality', connecting once more sustainability challenges to more social (personal and collective) related aspects.
Those results evidence (1) a desirable (and sought) increase in the complexity of the connections between human and nature/ecological relationships connected to; (2) the importance of the involvement and collaboration of different stakeholders in the proposed interventions (school direction and management, NGOs, waste disposal experts, college teachers, students, teachers, and maintenance staff, for example) broadening the understand of the complexity of social-environmental processes and; (3) the importance of the sustainability pedagogy framework, once collaboration and participation play a central role on building students' systemic, critical and reflexive thinking related to the construction of a future ideal worldview.
Despite this analyzes, economic, cultural, territorial, and political criteria (Figure 1) did not show significant change for both groups, indicating a distinguished issue for future planning and execution of further activities.

Behavior Intention: Room for Improvement
The evaluation of behavior intentions (question 3) intentionally came after actual behaviors. This arrangement arose from the assumption that previously thinking about actual behaviors would favor the recognition of different behaviors still poorly or not put into practice.
However, the most named categories of behavior intentions are also cited as actual behaviors (~57% and 64% for experimental and control groups, respectively). One possible way of explaining these results is to assume that both groups of students are really trying to have the actual behaviors declared, and its repetition as BI indicates that most of the actual behaviors can be improved somehow contributing even more to sustainability. In other words, there is a perception that actual behaviors can still be better performed. This assumption needs further investigation in order to clarify the real constraints for more sustainable behavior.
It is worth noting that 'reduce consumption' became particularly relevant for the experimental group; and 'influence on peers' for both experimental and control groups. The theme of consumption can be considered a very sensitive issue for teenagers and was frequently related to social and peer pressure during distinct EfS activities. The second one, in turn, was often associated with both groups with a complaint about people's indifference and small engagement regarding sustainability issues in the school and community. Both themes can be considered central for further investigations in order to contribute to the creation of more effective activities and results.

Impacts: Acknowledging Deeper Transformations
The analyzes of the effects (via pre-post-tests) are more common in the literature (Armstrong & Impara, 1991;Ballantyne, Fien & Packer, 2001;Bergmann & Pedrozo, 2008) than the evaluation of medium and long term deeper impacts.
One of the most desirable results of the educational process is to generate positive and significant long-lasting (personal and group) transformation. The analysis of impacts seems to be vital to understand the actual results of an intervention, allowing them to understand, adapt, readapt, direct, and redirect the educational experience as a whole.
The four most-cited categories (~78%) (Figure 4) represents the significative impacts of the 'sustainability pedagogy' worked through the different activities and interventions developed during the EfS. The appreciation and recognition of its role as a 'change agent' are connected to the core goals of the EfS and reflect the development of new values, competencies, and skills (Caravita & Clé ment, 2012;Hoffmann & Siege 2017) related to the building of a more sustainable society (Hofman, 2015).

Final Considerations
The proposed sustainability pedagogy -worked through active learning methodologies and multi-stakeholder social learning -showed itself effective in promoting measurable and desirable effects and impacts on participating students regarding their knowledge, actual behaviors, and behavior intentions about sustainability.
Social learning, in this context, is considered a collective and dialogical communication process of acquiring and building knowledge, making sense or abstracting new meaning, and disseminate knowledge (Muro and Jeffrey, 2008;Heikkila and Gerlak, 2013;Jacobi, Toledo and Grandisoli, 2016;Grandisoli, 2018;Grandisoli and Jacobi, 2020).
Thus, we consider this theoretical-practical framework as a good starting point in the planning, structuring, and execution of programs and projects in education for sustainability in different contexts, grades, and realities.
classes -have not responded to the uncertainties and complexity related to the various socio-environmental challenges. It is urgent to engage people in meaningful and contextualized activities envisioning the transition towards a more sustainable school and territory (Wals, 2017).
Maybe one of the most relevant changes in education, particularly during the 20th century, was the gradual change related to teachers' and students' roles. Students must assume a more proactive and decision-making position, and teachers must act as mediators/facilitators of the learning process, helping students to build new and more meaningful knowledge, never disregarding students' prior knowledge and worldviews (Scarpa & Campos, 2018).
Therefore, students and teachers have been working more and more as partners, establishing a collaborative and participatory two-way street. Throughout the EfS, this partnership was nurtured and valued, meeting one of the premises of active learning methodologies that states that everybody learns actively considering what is truly relevant within its context, culture, and close to its competency and skill levels (Moran, 2018).
The EfS can be considered a successful educational experience regarding the chance to develop a more longitudinal evaluation related to the combined effects and impacts of the sustainability pedagogy through active learning methodologies and multi-stakeholder social learning.
Those combined strategies of education emphasize the following: (Peter & Wals, 2013) • consider learning as more than merely knowledge-based; • provide quality of interaction with others and the environment in which learning takes place; • focus on real issues essential for engaging learners; • regard indeterminacy as a central feature of the learning process; • consider learning as cross-boundary in nature that cannot be confined to the dominant structures and spaces that have shaped education.
The evaluations of participant students (considering effects and impacts) revealed an increase in the capacity of analyses related to the complexity and uncertainty of the creation of more sustainable schools and societies. This fact emerges from the amplified comprehension of the human role regarding the current state of the societies and the planet (social dimension) as well as its responsibility in searching for alternatives to reduce its impacts (change agent).
Those outcomes demonstrate that alternative forms of education and learning have a central role to play in the development of people's cognitive, analytical and transformative skills (Wals, 2017) that directly dialogues with the new Brazilian curriculum for the high school ratified in 2018 (Brasil, 2018).
More important than personal and collective transformations is the possibility to develop new paths in education, especially for high school students, which naturally have to face different forms and intensities of pressure related to their university choice and admission.
Finally, it is due to (1) the combination of different collective constructed frameworks; (2) the continuity of EfS (2010-2016); (3) the support of the participant students and internal and external stakeholders, and (4) the well-structured evaluation instruments, it was possible to demonstrate some profound transformations in the space, students and its associated community making the learning more vivid and visible (Hattie, 2017). Therefore, it is crucial that teachers and studentswith the full support of school managementtest and apply new formats and methodologies in EE and EDS, to form more engaged and co-responsible citizens of the present and future.