Syntax Literate:
Jurnal Ilmiah Indonesia p�ISSN: 2541-0849 e-ISSN:
2548-1398
Vol. 7, No. 11, November 2022
THE COMPLEXITY OF WASHBACK EFFECT ON STUDENTS�
LEARNING IN TOEFL PREPARATION COURSE
Tetty Novitasari
Simbolon
Sekolah Tinggi Agama Katolik Negeri Pontianak, Indonesia
E-mail: [email protected]
Abstract
English language tests in Indonesia have been long
established to measure
English language proficiency, especially in higher education. Unlike the huge demand for English competency and the high
number of language test takers, research on the washback effect or the effect
of tests on language teaching and learning is still limited. This research aims
to explore the washback effect found in student learning. This research uses a
qualitative research design involving seven final year students from one
university in Pontianak as participants. Researcher collected data through classroom
observation and interview. This research focused on one type of English test,
namely the TOEFL (Test Of English as a Foreign Language). The data collected
was used to answer the nature of the TOEFL preparation course and a real
picture of all the activities that occured during the learning process. The
findings showed that there was a significant effect of the TOEFL and TOEFL
preparation course on student learning experience specifically on the depth of learning.
Despite
the finding that TOEFL
preparation course brought positive washback effect on students� learning, this
research suggests higher education institution and teachers to keep providing
better teaching and learning experience in test-oriented English class.
Keywords: English Language Test,
Washback-effect;
Students� Learning; TOEFL Course; Test-oriented English Class.
Introduction
In many
parts of the world, �English language has
been used as a medium of communication (Sadeghpour &
Sharifian, 2019). Not only used in
academic field, in a work field, the capacity to use English is also one of the
requirement to see the qualities of employees. This phenomenon might be one of
the effects of globalisation
found in community (Stromquist &
Monkman, 2014). It is becoming a common believe
that the mastery of English could help anyone to gain access not only to successful careers but also
to international friendships, further study abroad, new forms of technology and
entertainment and to social and geographic mobility.
In countries where English is not the
first language, getting a certificate or passing language test is mandatory for
many occasions such as entering and graduating from school or for the job
requirement (Menken et al., 2012). There are many types of English
language test available in Indonesia namely
IELTS, TOEFL, TOEIC, Cambridge English, PLT, etc. These different types of
English test are available to match the needs of different test takers (Chaipuapae, 2019). In Indonesia
especially in higher education, TOEFL has been used in majority of univeristies (higher
education) as a gate keeper and exit test (graduation
requirement). Unfortunately, even with
positive intention and careful planning, this policy and the test might result
far from what it is intended and might show some "unpredictable"
effects and results (Moher et al., 2012).
Testing has been a major topic that catches many researchers� attention
simply because the use of test has been a significant aspect that affecting the
curriculum, teaching and learning process even the society (Docktor & Mestre,
2014). McNamara (2000) defined the concept of an impact as "tests' effects
on the whole educational system, while washback is the effects of test on
teaching and learning." In Spratt (2005) study, it was concluded that besides the test itself,
context, teachers and learners' characteristics including their perception and
beliefs towards testing are considered variables that may contribute to the
complexity of washback. In accordance to that idea, Alderson and Wall (1993) also stated that �washback is a complex and
multidimensional phenomenon� that to identify the occurrence of washback on
certain types of text, a specific study is needed to be done.
Test-oriented
class or more specifically English test preparation class has been offered from
variety of institution in order to help test takers to pass the English test.
But again, although this class is carried out to bring a positive influence to
the learners to gain knowledge on the format of the test as well as building
their motivation in learning English, �Richards & Renandya (1993) mentioned that this type of language learning would
remove the "mixed-focus product and process model of language teaching and
learning and focuses only on learners' test score".
Burrows (2004) set in the theory of washback
that there is a �stimulus-response� relationship between teaching and testing.
What Burrow meant by �stimulus-response� was that �washback is a definite
by-product of a test and the quality of the test determines how teachers will
all have similar changes in their teaching under the test influence, be it
positive or negative. Hughes (1993) concluded the complexity of washback effect in relation
to the teaching and learning environment as follow:
Table 1
The
Trichotomy Backwash Model
|
a. Participants�students, classroom teachers,
administrators, materials developers and publishers, whose perceptions and
attitudes toward their work may be affected by a test b. Processes�any actions taken by the participants
which may contribute to the process of learning c. Products-what is learned (facts, skills, etc) and the quality of the learning |
Another study by Alderson and Wall (1993) concluded another theory that the concept of washback
hold 15 types of hypotheses describe as follow:
Table 2
Washback Hypotheses
|
(Wall & Alderson,
1993)
Studies on
washback have been long investigated and were used to focus on the teachers'
stand point (Green,
2006). Later, some researchers
were interested in investigating the possibility of washback on students'
learning too. Some studies of the washback effect on learners' learning (Green, 2006; Shih
& Lindley, 2006; Tsangari et al., 2016) found that testing does
not only affect teachers' teaching but also students' learning. It was found
that the effects found in learners were dynamic and varied on each individual.
From Watanabe (2001), it was found that �the test of the appropriate
difficulty can have a positive effect on students� motivation to prepare for
the test. However, it is the students� perceived difficulty of the test that
counts, not the objective difficulty. For this study, the researcher tries to
limit the area of the study which was inspired by Alderson and Wall (1993) evaluation of "increased specification". The
main focus of washback on students' learning experience mainly on the degree
and depth of learning. �
Combining ideas from the theory of
trichotomy (Hughes, 1993), Bailey (1996) concluded "the notion of washback to the
learners as the test-driven information provided to the test-takers and having
a direct impact on them." As mentioned in the introduction, compared to
the studies of washback on teachers, the study of washback on learner are still
limited. Green (2006) who has been consistent
in analyzing washback effect on learners concluded that "the differences
between the course contents the learners expected to receive and the course
outcomes showed that the differences were not driven by learner expectations,
but by teachers' adoptions of distinctive aims on different course types".
Green also mentioned that �teachers and the course may shape washback to the
learners and their perceptions of tests and test preparations may differ from
those of teachers.�
Based on the previous studies presented
above, it is clear that the focus on some studies was different from the
others. Some researchers focused on how test change learner's perception and
some other researchers focused on the learning product which is the performance
of learner in learning.
For this study, the researcher tries to
limit the area of the study which was inspired by Alderson and Wall (1993) evaluation of "increased specification".
The main focus of washback on learners' learning contains questions on
"how", "rate and sequence" as well as "degree and
depth". This investigation will lead to the answer for the last research
questions of this study which is to define the dimension of washback effect on
learners' learning. ��������
Research Method
Morrow�s (1986) once stated that �qualitative methods in language
testing research are useful for ��researching into language testing...to glean
insights into the ways in which multi-faceted individuals prepare for tests.� For this reason, �the researcher decided to use a qualitative
research methodology in order �to gain an emic perspective on participants�
immediate and local meaning for washback� (Davis, 1993). Although the researcher decided to use qualitative
strategy only, quantitative data from students� pre-test and post-test were
used to help the researcher investigate the effect of washback on the
participating students� learning process.
This
research was conducted in a language centre in one of the public universities
in Indonesia. The participants of this
research were 7 final year students, consisting of 3 female and 4 male
students. The participating students came from different types of major such as
economy, engineering, medicine and sociology.�
Their English level of proficiency was considered low although they have
been introduced to English since junior high. In selecting the participating students,
the researcher used random sampling. The reseracher also divided the
participants into two groups considering their main purpose in taking TOEFL. For
this research, there are two instruments used to collect data; classroom
observation and semi-structured interview. After collecting the data,
researchers conducted a transcript, displayed the data and made conclusion.
The data collected from both parties
provide information on the dimension of washback and factors/variables that
affecting it. Walking from that analysis, this research is expected to help all
the parties involved in TOEFL preparation class to create a better testing
system which can lead to better teaching and learning experience. Although the
result is restricted to specific condition, this research is expected to offer
some insights especially to teachers on existence of washback in test preparation
class so they could think of ways to help students to learn from being tested.
As Bailey (2005) said, �teachers may have limited power to influence
high-stake national and international examination, but teachers do have
tremendous power to lead students to learn, to teach and how to work with test
and test result.� This study is also intended to welcome and trigger other
researchers to investigate the washback phenomena in Indonesia as there are
still many areas needed to be explored.
Results and Discussion
In the university where this study was conducted, one of the
requirements for graduation is to pass certain TOEFL score. The university
started to apply this policy since 2002 as one of the action to motivate
students to learn English. They understand that in the future, the ability to
use English will be beneficial in every major part of student�s life and that
was one of the reasons why they compel an English test called TOEFL prediction.
The students need to pass the lowest score set by the university that is 425 or above. If student fails to meet the requirement,
student needs to retake TOEFL until one pass the test.
The nature
of TOEFL preparation course
TOEFL preparation course is conducted due to the fact that there are many students who fail several times to achieve the TOEFL minimum score. �The specific purpose of this course is to help
students to understand the format of TOEFL and the strategies to answer the
questions.
Based on the interview with the
participating students, the initial reason on why they take this course was
because they wanted to be able to answer the questions on TOEFL eventhough they
knew they had very limited understanding on the context. The TOEFL preparation
course was conducted for 30 meetings, where students had the chance to study 3
skills such as; Listening, Structure and Written Expression and Reading.
Students in this course had to take pre-test and post test, on which the result was
used to show the progress each
student made
during the course. Here is the list of
activities on the first meeting of the course:
1.
Teacher explaining to
students on
how to calculate the expected correct questions that students need to do in
order to pass certain score on each skill (section).
2.
Teacher explaining the
expected goal, then starting to explain the
format of the test, continued by categorizing each section into
topics.
3.
Teacher start giving comments
on strategies that students can apply to answer the questions as quickly as
possible.
4.
Teacher giving time to students
to analyse the questions and the explanation given by
themselves.
5.
Teacher asking students to do
the exercise from the textbook.
6.
Teacher making sure that all
students understand the questions and the strategies explained before.
7. Teaching collecting students� answer
and giving the answer keys.
8.
Teacher and all student then start
discussing and analyzing each question and then teacher pointing
out where most students make mistake.
As the goal of this course is to help students to pass
the expected TOEFL score (425 or above), almost all of the time in the course was used by teachers to introduce the format of TOEFL and
well proven strategies to answer the questions. �
Dimension of washback on students� depth of learning
This
research found that TOEFL did influence
students' learning methods and their rate and sequence of learning. From the interview with
the teacher, it was found that this TOEFL preparation course was not meant to upgrade student's language
competency,
but only to give students understanding on the big picture of TOEFL. Students were still expected to have basic English
competency and willingness to study outside of this course.
In terms of the depth of learning, the
researcher found that the intensity of washback was quite varied depending on
students' purpose of taking the TOEFL. In first group where students' goal to
take TOEFL as a graduation requirement, they stated that they had to put more
effort to study and pass the TOEFL because if they
could not pass the test, they should delay their graduation until next year.
Almost like putting students' life to the edge, this TOEFL test did influence students' depth of learning. For
students who already knew the most difficult part of the test, they even put
more effort to study on that part alone.
Student 1: �If I fail this TOEFL test, I would
definitely ruin my timeline and my dream to gradute in 4 years. I need to pass
this test so usually I spend one or two hours more outside this class to study
the format of the test and also adding some new vocabularies to my memories.�
Student 5: �Knowing that I need to take TOEFL, I become
more diligent than I used to I think. I also try to focus on specific area
where I am lacking for example in structure.�
Student 7: �Maybe because I just want to pass this TOEFL
to graduate, I think in almost 4 week I spend really great amount of time to
learn English especially the TOEFL prediction. I still need to take TOEFL to
get a job. If I don�t have high score on my next TOEFL I might end up getting
on the low position in the company.�
On the
other hand, in the second group where the reason for taking the test was for
the scholarship requirement, this students seemed to be more relaxed. They
mentioned that they were not in a hurry and still had time to develop their
language skills after this preparation course ended.
Student 2: �I took this class because I need to
understand the format of the test so I would not get panic when I need to take
TOEFL. I am here to learn the strategies and to get some experience in taking
the real TOEFL test.�
Student 3: �I feel like this class is helping me to
understand what I need to expect when I take TOEFL. But as the schedule of this
class is quite packed, I am already quite tired to spend sometime more to study
outside of this class.�
Student 4: �I do feel more motivated when I learn
English for testing purpose. I don�t know why but it feels like I have certain
goal to achieve. And now I need TOEFL score to
pass the requirement for scholarship, I think the time I spend more time to
study English than before. But I still have sometime to take TOEFL so I can
slow down a little bit to study.�
Comparing
these two group, it was clear that the depth of learning were also affected by the
purpose on why students take TOEFL test; the importance and time wise. The
intensity found on students� learning method is considered strong for students
who need TOEFL score as requirement for graduation but the intensity was weak on students who need TOEFL
prediction as requirement for scholarship.
Based on the model of washback
intensity by Green (2019), this research concluded the dimension of
washback effect on the depth of students� learning as follow:�
Dimension:
a.
The specificity
found on students� learning was considered
specific.
b.
The intensity
found on students� learning was considered
strong for students who needed TOEFL score as
requirement for graduation but the intensity was a lower on students who needed TOEFL as requirement for scholarship.
c.
The length of
washback effect found on students� learning was considered for short period for students in group 1. But in group 2
the effect could lasted for a longer period.
d.
Intentionality: The
influence of the test on students� learning was intended
When participant students were asked about their perception on the TOEFL preparation course, they all agreed that they needed this class to help them to understand
the "tips and tricks" in TOEFL. After four weeks attending the
course, participants gave comment that the overall class was helping and
motivating
especially because learning English was intimidating and they need to pass the minimum TOEFL score in a very short time.
Table 1
Minimum TOEFL
|
Participant |
Gender |
Purpose Taking TOEFL |
Test Taki-ng Experience |
Pre-Test Score |
Post-Test Score |
|
S 1 |
Male |
Exit test |
3 |
400 |
450 |
|
S 2 |
Female |
Scholarship |
3 |
410 |
467 |
|
S 3 |
Female |
Scholarship |
2 |
370 |
387 |
|
S 4 |
Female |
Scholarship |
2 |
410 |
463 |
|
S 5 |
Male |
Exit test |
3 |
400 |
410 |
|
S 6 |
Male |
Exit test |
3 |
410 |
500 |
|
S 7 |
Male |
Exit test |
3 |
383 |
433 |
CONCLUSION
This research was
conducted to investigate the complexity of washback effect on students� learning by focusing on a small scale of the group
(consisted of seven students). The data collected from classroom observation
was used to answer the nature of TOEFL prediction preparation course and a more detail data was collected through semi-structure
interview, letting students to speak their arguments on TOEFL preparation
course.
Based on the data collected, it
was found that the participants still considered taking TOEFL preparation
course to be important. Due to the limited time to study, the intensity of
washback effect on each student were varied. Although in some part the
dimensions of washback differ on each participants, in some parts such as their
perception of the learning method/new strategies introduced in TOEFL
preparation course were helpful and bring positive washback to their learning
experience especially the depth of their learning. The progress that all the
participating student made also showed that the course created more good
learning experience than bad. Despite this fact, �there are still many aspects that university,
teachers or any education practitioners in language testing need to do in order
to provide better teaching and learning experience in test-oriented English
class.
BIBLIOGRAPHY
Bailey, J. E.,
Sburlati, A., Hatzimanikatis, V., Lee, K., Renner, W. A., & Tsai, P. S.
(1996). Inverse metabolic engineering: a strategy for directed genetic
engineering of useful phenotypes. Biotechnology and Bioengineering, 52(1),
109�121.
Bailey, K. M., & Nunan, D.
(2005). Practical English language teaching: speaking.
Chaipuapae, P. (2019). Workplace
listening comprehension of Thai undergraduates in an Asian English as a Lingua
Franca context. Northern Arizona University.
Crisp, D., Atlas, R. M., Breon,
F.-M., Brown, L. R., Burrows, J. P., Ciais, P., Connor, B. J., Doney, S. C.,
Fung, I. Y., & Jacob, D. J. (2004). The orbiting carbon observatory (OCO)
mission. Advances in Space Research, 34(4), 700�709.
Docktor, J. L., & Mestre, J. P.
(2014). Synthesis of discipline-based education research in physics. Physical
Review Special Topics-Physics Education Research, 10(2), 20119.
Green, A. (2006). Models of lifelong
learning and the �knowledge society.� Compare, 36(3), 307�325.
Holt, D. H. (1993). Management:
concept and practices. New Jersey: Prentice Hall, Englewood Cliffs.
Ivancevich, John M. William F. Glueck.(1989). Foundations of Personnel/Human
Management, 21, 397409.
Hughes, S. A. (1993). Physical
models and laboratory techniques in coastal engineering (Vol. 7). World
Scientific.
McNamara, T. (2000). Language
testing. Oxford University Press.
Menken, K., Kleyn, T., & Chae, N.
(2012). Spotlight on �long-term English language learners�: Characteristics and
prior schooling experiences of an invisible population. International
Multilingual Research Journal, 6(2), 121�142.
Moher, D., Hopewell, S., Schulz, K.
F., Montori, V., G�tzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M.,
& Altman, D. G. (2012). CONSORT 2010 explanation and elaboration: updated
guidelines for reporting parallel group randomised trials. International
Journal of Surgery, 10(1), 28�55.
Morrow, S. (1986). � No Girl Leaves
the School Unmarried�: Mabel Shaw and the Education of Girls at Mbereshi,
Northern Rhodesia, 1915-1940. The International Journal of African
Historical Studies, 19(4), 601�635.
Sadeghpour, M., & Sharifian, F.
(2019). World Englishes in English language teaching. World Englishes, 38(1�2),
245�258.
Septiyanti, M., Liana, L.,
Sutriningsih, Kumayanjati, B., & Meliana, Y. (2019). Formulation and
evaluation of serum from red, brown and green algae extract for anti-aging base
material. AIP Conference Proceedings, 2175(1), 20078.
Shih, T., & Lindley, C. (2006).
Bevacizumab: an angiogenesis inhibitor for the treatment of solid malignancies.
Clinical Therapeutics, 28(11), 1779�1802.
Spratt, M. (2005). Washback and the
classroom: The implications for teaching and learning of studies of washback
from exams. Language Teaching Research, 9(1), 5�29.
Stromquist, N. P., & Monkman, K.
(2014). Defining globalization and assessing its implications for knowledge and
education, revisited. Globalization and Education: Integration and
Contestation across Cultures, 1, 1�21.
Tsangari, H., Paschalidou, A. K.,
Kassomenos, A. P., Vardoulakis, S., Heaviside, C., Georgiou, K. E., &
Yamasaki, E. N. (2016). Extreme weather and air pollution effects on
cardiovascular and respiratory hospital admissions in Cyprus. Science of the
Total Environment, 542, 247�253.
Wall, D., & Alderson, J. C.
(1993). Examining washback: the Sri Lankan impact study. Language Testing,
10(1), 41�69.
Watanabe, Y., Bednorz, Jg., Bietsch,
A., Gerber, C., Widmer, D., Beck, A., & Wind, S. J. (2001). Current-driven
insulator�conductor transition and nonvolatile memory in chromium-doped SrTiO 3
single crystals. Applied Physics Letters, 78(23), 3738�3740.
|
Copyright holder: Tetty Novitasari Simbolon (2022) |
|
First publication right: Syntax Literate: Jurnal Ilmiah Indonesia |
|
This article is licensed under: |