Monday, January 27, 2014

Problems with PISA argument

First I don't believe we should put our faith in PISA scores due to the following:

Why my Child won't write optional Standardized Testing
'
As well as my other posts around why standardized testing is flawed.

However, lets for a moment assume that PISA scores are accurate.  We will also negate the fact that Alberta Class sizes have increased, ignore the fact that AISI funding is at 0, and lastly forget that our ESL population is at an all time high.

We shall enter the "Wonderland".

Well, first the argument is that we need to go "BACK TO BASICS" with elementary math classes.  Students should be "Drilled and killed" with mathematical facts and flashcards should be the norm.  Since elementary classrooms have removed these activities the PISA scores have dropped.

Well this is great, however this argument is FLAWED!

See the 2012 PISA test is completed by 15 year olds or Grade 10.  Which means in 2011 they were in Grade 9, 2010 they were in Grade 8..., 2003 in Grade 1.  The Alberta curriculum changed , and had the following implementation years...


Which means...drumroll...these students experienced elementary classrooms teaching the "old" math. Which is, of course, the exact math critics want to bring back.  These students are not the ones we should be looking at, but instead the next data set to come, as these students will be full time "new" math. (Keep in mind I don't think we should look at any PISA scores, but we are in Wonderland)

Next, someone might say that the PISA drop is due to poor retention levels.  As the new curriculum is not allowing students to retain mathematical information in the long term.  This poor retention level is causing the frustration when students try to complete higher level math.

Well..again this is flawed.

Most people have heard of PISA, but have you heard of PIAAC testing? See now this is an exam, also done on the international level, but tests 16-65 year olds.  Keep in mind, all these people experienced 100% "old" math.

Since we are assuming that Standardized test scores are accurate, lets take a look at how our country scored on this...

As you can clearly see Canada was under the OECD average in every age group in Numeracy levels.  Since all these participants took the "old" math, I guess retention was an issue then as well.

What does this mean?

I believe people are following the wonderful quote of Henry Ford
When I asked people what they wanted, they replied with faster horses

What we forget is that sometimes the "Better" system is one which we have never seen or done before, and this is another reason why I support the new math curriculum.

3 comments:

  1. Very clever and subtle ( \sarc ) strawman argument you use, "Students should be "Drilled and killed" with mathematical facts and flashcards should be the norm." Get back to us when you find parents who actually articulate their concerns in any way remotely resembling this ... you've got 3700 or so to choose from over at Dr. Tran-Davie's petition -- happy hunting!
    https://www.change.org/petitions/back-to-basics-mastering-the-fundamentals-of-mathematics

    With regard to your mockery in that line and the "we experts know so much better than you what your children need" use of the Ford story: it never ceases to astonish me the contempt with which fuzzy math proponents view concerned parents and seek to marginalize them with facile, paternalistic rhetoric. But you do us all a favour by displaying your superiority so unabashedly. But it's not only parents you seem to despise; I estimate the number of your FELLOW TEACHERS who signed that petition at close to 1/3 of the respondents, something like 1000. Guess you think they're all a bunch of troglodytes too? Air must be a bit thin up there, Dave.

    While you're pasting in schedules don't forget this item:
    http://www.education.gov.ab.ca/BacktoSchool/
    "New Math Curriculum - Leading Math Success!
    A new mathematics curriculum for Kindergarten and Grades 1, 4 and 7 is being rolled out on an optional basis starting this fall. The new curriculum is designed to further student achievement by providing a good grounding in basic mathematics. Students will receive a good grounding in basic mathematics, with an increased focus on number facts and problem solving. Changes to the K-9 Mathematics Program of Studies reflect current research in math education. Although many of the topics studied remain the same, the grade in which they appear may differ. The new Alberta Mathematics Kindergarten to Grade 9 Program of Studies will be mandatory in these grades in 2008 and taught in all grades up to 9 by 2010. A staggered curriculum implementation will ease the transition in schools and support publishers who provide resources."

    Yep, implementation a year PRIOR to the official mandatory deadline as made clear in the Fall 2007 parent information kit. Now, counting backwards, the 2012 PISA cohort had ... 1, 2, 3, 4 ... FOUR years of "new" WNCP instruction.

    This is your child's Alberta Math Education (2003): 549 on PISA normalized scale
    This " " " " on WNCP Math (2012): 517
    -- a drop of 32 points, second largest decline among Canadian provinces, from well out in first place to ... part of the pack.

    More from PISA data:
    Proportion of functionally innumerate Albertans
    2003: 7.4%
    2012: 15.1%
    (More than doubled)
    Students attaining top two levels:
    2003: 26.8%
    2012: 16.9%
    (Drop by over 1/3)

    Now the WNCP curriculum was designed, starting around 2000, by the same group (whom I call the consultant class) of (largely) Albertans responsible for resourcing teachers and instructing them in pedagogical methodology in math class. More than the paper curriculum it is this group's control of teacher training and resources that dictates how math is to be taught in Alberta classes. Are we to believe that, although they held sway over the WNCP process for the last 13 years, they had no influence over classroom methodology prior to the official adoption of WNCP? I'd be interested to hear from teachers when these approaches began to be strongly promoted in their inservice training.

    Like the results of 4 years of fuzzy WNCP, Alberta? You'll LOVE the results in 2015 after 7 years of it!

    ReplyDelete
  2. In fact, PISA does NOT test early-years outcomes. As you know from our Twitter discussion Dave, PISA questions base assessment upon the assumptions of RME, "Real Mathematics Education" -- an educational protocol premised upon the use of DISCOVERY learning. It is a soft-constructivist paradigm, like Alberta's WNCP Math.
    http://www.mei.org.uk/?page=rme
    http://www.sdpi.ie/inspectorate/insp_pisa_maths_teach_guide.pdf (See page 4)

    As such one would think students (as in AB 2008-2012) receiving this form of instruction had a distinct advantage in this assessment over backwards jurisdictions like Shangai, Hong Kong, Singapore and Korea that still emphasize the fundamentals.

    One would think this. But one would be wrong. No surprise to anyone familiar with the findings of large-scale studies like Project Follow Through, showing that students learning under systems focussing on "understanding" over "skills" tend to drastically under-perform the latter on tests of both understanding and skills.
    http://psych.athabascau.ca/html/387/OpenModules/Engelmann/evidence.shtml
    Not to mention on measures of self-esteem ("affective domain").

    ReplyDelete
  3. Oh Robert, I love how you make up statistics to make your point more prevalent.

    " I estimate the number of your FELLOW TEACHERS who signed that petition at close to 1/3 of the respondents, something like 1000. Guess you think they're all a bunch of troglodytes too? Air must be a bit thin up there, Dave."

    Where is this coming from? Unlike you, I believe that our teaching in elementary schools are top notch. As well, I would argue that MAYBE 1/3 of the TOTAL people signed are from Alberta, the province in which is argument is about. Since we are estimating, I shall too. Unless you have actual data to support your frivolous claims?

    Some school went early to the change? Did you know that the curriculum does not dictate to the teachers on how to teach. I bet not every teacher teaches through the techniques you criticize. I guess, once again, until you have real data we could argue all day creating fake numbers.

    ReplyDelete