Showing posts with label standardized exams. Show all posts
Showing posts with label standardized exams. Show all posts

Thursday, April 21, 2011

Declining top marks in English exams puzzles educators

In October 2009, the Edmonton Journal posted the following article:

Declining top marks in English exams puzzles educators

In the article it states:

A shrinking proportion of Alberta students writing their Grade 12 English diploma exams are scoring top marks on the important test…

The key problem with the English diploma exams, according to Alberta Education officials, is the written portion, which requires students to write both a personal response and a critical/analytic response to texts…

…"It matters because how your best students do affects how your average students and how your poorer performing students do," said John Rymer, Alberta Education's executive director of Learner Assessment…

Still, McCabe [Edmonton public’s principal of student assessment] said, the results might say something larger about how teens communicate today
"It's very different from 10 years ago, but we haven't changed the format of the exams," she said. "I think it's time to examine, are there other ways for kids to show what they know other than a formal essay?"...

… "The new curriculum allows so many different ways to engage with text and so many different ways to express what you've found and learn," Rymer said. "So if there's a subtle shift in the classroom from written work to oral presentations, have I given up on other things?"

This article sums up, what I believe, are some of the issues with standardized testing. 

First, Rymer states that it is the best students who determine how the rest of the students perform.  If this is true, then in reality the test is ONLY informing the school how smart their “best” students are.  I wonder how parents would react if they realized that their child’s mark (on this test which makes up 50% of their mark) depends on how other students perform?

Second, I 100% agree with McCabe when she said it is time to realize there are other ways for students to illustrate their learning.  Recently, one of my math students, showed me how talented he is at Photoshop.  Below are two demonstrations of his talent and creativity.


He is one amazing student, and has demonstrated how creative he can be multiple times. 

I fear that on diplomas, the only way students can demonstrate creativity would be to doodle on the blank pages. I believe it is time that our standardized test “makers” realize that traditional testing is not the best way to assess knowledge, learning, or passions and need to start allowing students to be creative.

Tuesday, March 29, 2011

Who is to blame: The teacher or the test?

Here are some comments about high stakes exams from a reader of my blog:

"You seem to think that you can absolve them of their responsibility by simply blaming the exams for their lack of humanity or professionalism.

About the alleged example of the teachers only feeding the kids properly on test days, I agree that it's disgusting (if it really is true); but it still isn't the fault of the test. It's the fault of the teachers who are displaying this appaling lack of humanity.

Blaming their unethical behaviour on the exam, and not on them, is what's disgusting.


As for your comment about "this implies that only these outcomes should be emphasized in the class? This sounds a little corrupt to me." - I never intimated that only those outcomes should be emphasized. In fact, I consider anybody who ignores the curriculum in favour of 'teaching to the test' to be incompetent and think they should be fired.

And yes, that incompetence is their fault, not the test's fault. "


Full conversation at "Mandated Exams..." and "Don't teach that, its not on the test"

I know open the floor, to any and all readers of my blog.  Am I out to lunch, implying it is the test that is forcing our hand here, or is it really the imcompetence of teachers?

Please comment below.

Thursday, March 24, 2011

Don't teach that, "It is not on the test"


I have heard the following statements from different educators:

Why are you teaching that…it isn’t on the test!
Why do you need to know this?  Because it is on the exam!
If I don’t test, then I won’t know what a student knows!
These statements disgust me! 
If you believe tests hold teachers accountable, here is my comment on that.
If you believe that tests cause students to learn material, here is my comment on that.
If you believe tests inform us what students know, here is my comment on that.
I have heard educators, who are required to test their students a month before the end of the course, say the following
Now that the exam is over, I am going to let you explore!”
It saddens my heart to know that we are required to teach the “boring” stuff to get through all the material for the exam, and after the exam we can then do the “fun” stuff.  Tests are robbing students from open-ended, deep meaning, and creative tasks.  The statement above is not an indictment to the teacher saying it, but actually the test that forces the teacher to teach as such. 
Too many times teachers must tell their students that they are not able to discuss certain passions, due to the timeline of the course.  If these teachers don’t follow the timeline as such, there won’t be enough time to prepare them for the useless bubble sheet they will be provided with at the end of the course.
If you are still not convinced, in our province Gr. 3 teachers are required to teach 1352 outcomes, as all these outcomes will be tested on the provincial achievement exam.  Is this reasonable?
Don't think about thinking....its not on the test.

Friday, March 18, 2011

Mandated outcomes and truth behind high stakes exams

I listened to David Berliner’s keynote address at MACTE (Michigan Association of Colleges for Teacher education).  I found myself nodding in agreement throughout his speech.
 “The narrowing of the US curriculum, especially among the working class and the poor, even as the American people say they do not want this and it works against the best interest of American Industry
I feel the same way about the curriculum in my province.  Teachers are given scripts to follow so that the bureaucrats, many of whom do not teach anymore, can say “We know students are learning something”.  What they feel to realize is that the scripts are more limiting and constraining the education system instead of liberating it.   Instead of having lessons focused on deep understanding and true critical thinking, teachers are forced to skim the surface of many topics.  Here is the sad truth behind the Biology 30 curriculum in my province.  I fear that many don’t want us to teach students how to critically think….because these students might actually do it!
“An educated person has the ability and inclination to use judgment and imagination in solving the problems that confront them at work and at home, and to participate in the maintenance of democracy”
Education should not be about teaching students skills, but actually showing them how to use such skills.   When we focus on procedural and conceptual ideas, but never talk about problem solving, students will only learn the WHAT part of outcomes and never the WHY.  There is a difference between students learning in a class, and wanting to learn in a class.
A student, who could correctly calculate the area of a rectangle, was asked “How much carpet is needed in a room 8 feet by 10 feet?” He replied with “How would I know?”.  This student answered 10 out of 10 questions on an exam no problem.  I would hope most would agree there is a problem! 
Students who can remember facts, algorithms, and procedures, but never know when to apply them will be successful at Trivial Pursuit, but not at life.
A quote from Charles Dickens Hard Times London
"Now, what I want is, Facts. Teach these boys and girls nothing but Facts. Facts alone are wanted in life. Plant nothing else, and root out everything else. You can only form the minds of reasoning animals upon Facts: nothing else will ever be of any service to them. This is the principle on which I bring up my own children, and this is the principle on which I bring up these children. Stick to Facts, sir!"

"In this life, we want nothing but Facts, sir; nothing but Facts!" The speaker, and the schoolmaster, and the third grown person present, all backed a little, and swept with their eyes the inclined plane of little vessels then and there arranged in order, ready to have imperial gallons of facts poured into them until they were full to the brim
The sad reality is that some educators believe it is as easy as the opening heads up and we can just pour facts in.  This quote is 120 years old, and still some classes and tests are designed this way.
We can also go back further to:
"I must study politics and war that my sons may have liberty to study mathematics and philosophy." -- John Adams


Schools need to start giving students liberty and stop the perputation of mandating students to take certain courses! 
In 1975 Campbell stated the following law:
"The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor."
And follows his law with..
"achievement tests may well be valuable indicators of general school achievement under conditions of normal teaching aimed at general competence. But when test scores become the goal of the teaching process, they both lose their value as indicators of educational status and distort the educational process in undesirable ways. (Similar biases of course surround the use of objective tests in courses or as entrance examinations.)"


When any indicator, such as high stakes exams, take on too much value it corrupts the people who deal with these indicators.  I believe this is what is currently happening in our education system.  I fear that, in some schools, the quality of instruction from a teacher is judged, not by classroom experiences, anecdotal comments, students’ experiences, but solely by their high stake exam marks.  As by Campbell’s law, such scrutiny around these scores is causing the teachers, and administrators, to become corrupt.  The corruption can be seen by “test prep”, or “teaching to the test”.  Imagine how much learning could occur if every minute spent on preparing students for an exam was spent on teaching a deeper understanding of a concept.

This now brings the validity of high stakes exams into question; the higher the stakes the higher the corruption but the lower the validity.  Due to the high stakes on some exams, teachers are forced to start seeing their students as “Test successors” and “Test suppressors”, not as students.  Schools have actually been caught giving certain classes certain teachers, based on their potential to pass a high stakes test.  Now a scary example,


“Kevin, who is a high achiever and suffers greatly from Asthma, was writing a high stakes government exam.  During the test, Kevin is having troubles breathing and asks to leave.  The teacher asked Kevin to first complete his exam then take care of his asthma”


There is a school in Virginia that realized if they fed their students breakfast, through a free breakfast program, they achieved better during the day.  This school, on high stakes testing days, gave the students an extra 100-200 calories at breakfast and the scores increased by 8 points.  Sadly, after test week, the school went back to the old breakfast program. 
This school understands the merit behind eating healthy in the morning, but only provides a sufficient amount of food during test week.  This is similar to farmers fattening pigs before taking them to market….DISGUSTING!
This is what happens when we see students as scores and data and not as people.

Thursday, March 17, 2011

Teaching understanding or algorithms?

I recently read Schoenfeld’s “When Good Teaching Leads to Bad Results: The Disasters of “Well Taught” Mathematic Courses”. 

This paper reports the results of a year-long intensive study of the teaching and learning that took place in a tenth-grade geometry class, which will be called the target class. The class took place during the 1983-84 academic year in a highly regarded suburban school district in upstate New York. The study included periodic observations of the target class and of eleven other mathematics classes, interviews with students and teachers, and questionnaire analyses of students' perspectives regarding the nature of mathematics. The target class was observed at least once a week, and was videotaped periodically for subsequent detailed analysis. Two weeks of instruction near the end of the course, dealing with locus and construction problems in geometry, were videotaped in their entirety. The analyses focused both on the mathematics that was learned, and on what the students learned about the mathematics -- including how and when they would use, or fail to use, the mathematics that they had studied.

This paper explored the difference between performing mathematical algorithms and truly understanding the underlying fundamental ideas of mathematics.  Due to the structure of the study, they were able to quantify certain classroom behaviors; time spent in questioning, active learning time, amount of praise, and amount of feedback.  The study also included the type of grouping, the size of grouping and so on.  Originally, they defined “learning” as how well the students performed on achievement tests.  I say originally, because through the article it illustrates how these tests fail in significant ways to measure subject matter understanding.

The article also suggests that providing students with repetitive routine exercises that can be solved out of context and no significance provided, actually causes the subject matter to seem frivolous to students.  This monotonous work actually deprives the students of the opportunity to apply their learning in a context that is meaningful to them.

Brown and Burton (1978) developed a diagnostic test that could predict, about 50% of the time, the incorrect answers that a particular student would obtain to a subtraction problem -- before the student worked the problem!  Teachers are still currently doing this, when we provide students with “distractors” on a multiple choice exam.  If we already know what mistakes students are going to make, before they make them, we need to start changing our instructional models.

The predominant model of current instruction is based on what Romberg and Carpenter (1985) calls the “absorption theory of learning”: “The traditional classroom focuses on competition, management, and group aptitudes; the mathematics taught is assumed to be a fixed body of knowledge, and it is taught under the assumption that learners absorb what has been covered” (p. 26) This view is essentially implying that the “good” math teacher has multiple methods of covering the same outcome.  Through these multiple methods the students will eventually “get it”.  Unfortunately, what most math teachers fail to realize, is that through these multiple methods we are forcing the students to “get” something entirely different; resistance to change.

Math teachers need to stop testing students on algorithms and more on the understanding of math concepts.  Below is an example of two problems Werthemier (1959) gave to various elementary school students.


Many of the students, who were deemed as high achievers, added the terms in the numerator and then performed the indicated division.  They followed the conventional order of operations of BEDMAS, or to some PEDMAS.  An idea which should not be taught to students out of context, which I have wrote about here.  Even though these students calculated the correct answer, I would argue that these students do not demonstrate any depth of understanding of mathematics.  To truly understand the underlying substance students should recognize that repeated addition is equivalent to multiplication and division is the inverse of multiplication. 

This example truly illustrates that being able to perform the appropriate algorithmic procedures, does not necessarily indicate any depth of understanding.  Also, the sad truth; virtually all standardized exams for arithmetic competency focuses primarily on algorithmic mastery, and not deep understanding of the math concepts. 

This is the first part of the article summarized, more to come….

Wednesday, March 16, 2011

Student revolts about Standardized Exam

Kathryn Coffey is passionate about public education, and she is especially passionate about teaching reading and writing. She also interested in Cognitive Coaching, education leadership and education policy.
Here blog can be found here.

Here is her story:
 
In the spring of 2005, when my youngest son A.J. took our state’s high-stakes standardized test, the MEAP, (now the MME) he had an ax to grind. Unfortunately, no one really knew how angry he was until after he took the exam. I received a call from the principal requesting a meeting. It seems that my son had chosen to draw a very large and detailed fist with an extended middle finger where his science graph should have been. And, he chose to bubble in his multiple-choice answer sheet with “AC/DC” and “ABBA”, as well. The principal was quite distressed since my son’s unconventional test responses would most definitely effect the school’s test results and would reflect poorly on the district. So, why would he do this? It turned out that A.J. was upset about one of his teacher’s policies that he had tried unsuccessfully to address with the teacher and administration. Feeling he had not been heard, he took his revenge. I had assumed A.J.’s stunt was an isolated incident—until yesterday.

Fast-forward to March 9, 2011. A colleague who supervises pre-service teachers for a nearby university shared a disturbing conversation he overheard while he was visiting a high school classroom that morning.

At the beginning of the class, he was sitting near a group of four young ladies. As Juniors, they had just taken the Michigan Merit Exam (MME) and ACT last week. One of the other adults in the room asked these girls how the testing had gone. One said, “they’re so stupid, I paid attention for the first couple of pages, then I just started bubbling in randomly.” A second girl said, “Yeah, I couldn’t take it seriously, I don’t want (our school) to do well.” One of the other girls responded, “Yeah, if this was for us and they were still giving out the scholarships, I would have taken it seriously.”

What’s particularly interesting to me is that while A.J. decided to take his frustration out on the science exam, it was a foreign language teacher’s policy he was protesting. In addition, the comment above “I don’t want (our school) to do well,” didn’t target a particular teacher or subject area, but apparently the school in general.  

The standardized test movement is based on the assumption that students are actually going to take the test seriously, that the test will measure achievement and will measure teacher effectiveness. Neither A.J.’s performance on the science MEAP, nor that of the girls mentioned above meets those assumptions. This has me wondering, and it raises questions that I believe need to be addressed.

·      Is there any research about whether or not students actually take high-stakes standardized tests seriously?

·      How many teens out there have an ax to grind with the adults in their lives, particularly with their teachers and their schools?

·      How many angry teens would it take for testing results to be corrupted for a given school district?

·      How often does this happen?

·      Do students understand their teachers and school are being held accountable for how well they do on these exams?

Perhaps some understand it only too well. It’s entirely within a teen’s nature to subvert the adults and authorities in their lives. Why would anyone think it’s a good idea to put teacher’s careers and the viability of a school district in their hands?

Sunday, February 13, 2011

Info about Standardized exams, focused on PAT in Alberta

Here is some information on the standardized exams in Alberta for grades 3, 6 and 9. 
First off, the definition of a standardized exam:
A standardized test is a test that is administered and scored in a consistent, or "standard", manner. Standardized tests are designed in such a way that the questions, conditions for administering, scoring procedures, and interpretations are consistent and are administered and scored in a predetermined, standard manner.
I urge all parents to read and make an informed decision on whether or not their child participates in the exams in grades 3, 6, or 9.  As a parent you actually have the choice to whether or not your child participates in these exams.  For dates and times of these exams, you are to ask your child’s teacher or principal.  If you choose to not have your child participate you can withdraw your child, by sending a letter to your child’s principal.  No consequence, through marks or other tactics, can be taken against your child.  Taken off the Alberta Education’s website:
If a parent withdraws a student from participation, the school is obligated to mark the student “absent” not “excused” on the List of Students. A copy of the parent’s letter indicating that the
child will not be participating should be attached to the Principal’s Statement.

From the Alberta government’s website, the reasons of the exam are:
·         determine if students are learning what they are expected to learn.
·         report to Albertans how well students have achieved provincial standards at given points in their schooling.
·         assist schools, authorities, and the province in monitoring and improving student learning.

Also, further in the document states,
Careful examination and interpretation of the Achievement Testing Program results can help reveal areas of relative strength and weakness in student achievement. Teachers and administrators can use this information in planning and delivering relevant and effective instruction in relation to learning outcomes in the Programs of Study.”

Now, in the document, it also states:
Achievement tests can assess only part of what is to be learned. In addition, many factors contribute to student achievement. Personnel at the authority and school levels are in the best position to appropriately interpret, use, and communicate school authority and school results in the local context.”

Reading these two statements states that the PATs can help find areas of strength and weakness in a child’s achievement, but the people who are best to know what a child actually has learned are the teachers of that child.  Click here for more on Achievement VS Learning

I have done some research around these exams and also wanted to include what I read elsewhere.

First we should be aware of the costs.  In 2003, the PAT and diplomas cost the government $12 million, while they only spent $4 million on curriculum.

An informal "count me out" movement against excessive testing is gathering momentum around the globe. Hundreds of teachers in Britain have recently voted to boycott the tests. Hundreds of parents in Alberta have requested that their children be exempted from the tests. A U.S. group is even suggesting that all politicians take all the tests.  

Currently, the Alberta Teachers Association is trying to abolish the grade 3 PAT entirely, due to the fact of the age of the students.  Virtually all specialists condemn the practice of giving standardized tests to children younger than 8 or 9 years old. I say "virtually" to cover myself here, but, in fact, I have yet to find a single reputable scholar in the field of early-childhood education who endorses such testing for young children.

Also, I have read research stating that standardized exams test more on socioeconomic status then actual learning.  For decades, critics have complained that many standardized tests are unfair because the questions require a set of knowledge and skills more likely to be possessed by children from a privileged background. The discriminatory effect is particularly pronounced with norm-referenced tests, where the imperative to spread out the scores often produces questions that tap knowledge gained outside of school. This, as W. James Popham argues, provides a powerful advantage to students whose parents are affluent and well-educated. It's more than a little ironic to rely on biased tests to "close the gap" between rich and poor.

Data from the USA standardized exam, the SAT:
Family Income
Average SAT Score
$30 - $40K
885
$50 - $60K
929
$70K +
1000

The test makers call their multiple-choice tests 'objective' and would have us regard objectivity as a virtue. But the term 'objective', when applied to the tests, is really a misnomer. The objectivity resides not in the tests as a whole but merely in the fact that no subjective element enters the grading process once the key has been decided upon. Yet the choice of questions to ask, topics to cover, and the choice of format, that is, multiple-choice as opposed to essay-answer, are all subjective decisions. All 'objective' means, in the narrow technical sense, is that the same mark will be received no matter who grades the test. The chosen answer is simply judged as 'correct' or 'incorrect' in accordance with the key, no argument or rationale is permitted, and the grading can be done by computer. In this sense, all multiple-choice tests are "objective."

But it is important to realise that saying a test is "objective" does not mean that the questions are relevant or unambiguous; nor does it mean that the required answers are correct or even "the best." Even more important, calling the tests "objective" does not mean that the tests are not biased. As discussed above, standardized tests may discriminate against many of the best candidates. It is more generally accepted that these tests are biased against women, minorities, and the poor.

Bias can take many different forms. With women, test scores underpredict grades. Although women tend to score lower on standardized tests, they tend to earn higher grades in college.  At least one study has found the scores also under predict grades for Hispanic students.  Bias against black students takes a different form. Although there is no clear evidence that test scores consistently under predict the grades of black students, it seems that test scores are far less reliable predictors for black students. Or in other words, even more errors in prediction will be made for black than for black students. This form of bias is known as differential validity.

Finally, tests cause stress and depression. Teachers our on edge all year with regards to how to prepare children for the tests. Children become nervous and depressed worrying about how well they are going to do on the test day. I'll bet that doesn't help them to do better. To think that a child being tired, hungry, or nervous during a test can totally effect their results in and of itself says a lot about the fallacy of any test, least of all a government test.

Children, human beings that is, are turned into numbers. A high number or a low number. Instead of making changes in thinking of the thoughts, feelings, emotions, curiosity, of real people, our children, the powers that be see only numbers. Numbers that represent living breathing children.
If you require more info, you can refer to the websites below.
References:
"Large scale educational assessment: the new face of testing" in Passing the Test: The False Promises of Standardized Testing.
Hampton, Wayne. "Challenging the testing regime in Alberta."

Monday, January 10, 2011

Actual learning first, class average second

Recently I have had a conversation about class averages on a standardized exam.  The conversation made me remember a story I once read:

John was interviewed for job at a factory were the management consisted of Dave, his brother, and 6 relatives. The workforce consisted of 5 foremen, and 10 workers. Dave informed John that the pay was well here, with an average salary of $600 per week. After one week of work, John was upset as he only was paid $200. John stormed into Daves office, and accused Dave of lying. Dave, the magical mathematician, explained "Every week I get $4800, my brother gets $2000, my six relatives make $500, each foreman gets $400, and the ten workers get $200. Averaging to a salary of $600 per week."

Unless, we talk about mean, median, and mode, the AVERAGE, can be meaningless.

Also, what should be more important, the learning in the class or the class average?  When we start worrying about the marks students receive on an exam or diploma we lose sight of our actual goal.  Teaching to the test, inflating grades, or manipulating assessment practices is what teachers start to do, while changing pedagogy should be the answer.  The bottom line, however, class average should not be a concern to a teacher, where actual learning should be the first priority. 

Wednesday, December 8, 2010

Differentiated instruction first, differentiated assessment second.

I teach my students in many different ways, but I grade all using the same process.  There is a major problem occurring here.  Every teacher, I would hope, in Alberta knows what differentiated instruction is, but do they know what differentiated assessment is?  Educators know that every student learns in a different way, but do we know that every student can demonstrate their learning in a different way?

In our province, students write standardized exams in grades 3, 6, 9, and 12.  When our school is evaluated for “students’ achievement”, we are assessed on standardized exam participation, acceptable standard (over 50%), and standard of excellence (over 80%).  I find this very contradictory!  Our province is forcing every student to demonstrate learning in the same way, on the same exam and on the same questions.  Where is the differentiated assessment?

Currently, due to a mandate of my department, I am administering common exams to all my students.   Every test day I shake my head as I use differentiated instruction in all my courses, but then grade all my students the same. 

Next semester, I will be changing my grading process.  I will keep differentiated instruction, but I will be implementing differentiated assessment.  Students will inform me when, during the term, they want their outcomes assessed.  No longer will I grade based on my progress through the course, but actually grade the students on their own progress through the outcomes.  Students will also be allowed to demonstrate any outcome as often as they would like.

To truly be teaching for the students, we need to realize that differentiated instruction is no longer enough, we need to start implementing differentiated assessment as well.