Mathematics assessment in Queensland Schools needs improvement urgently.

1)  A central authority could outline the general requirements for a series of exam papers, such as having a suitable range of difficulty level, a range of theoretical and practical questions, familiar to unfamiliar, and a range from short response to extended.  There would be no need for the massive amount of criteria and verbal descriptors of standards that we have now.  After that, it should be up to the schools to set exams and use marks to assess the students and grade them.  Many schools believe that they must avoid using marks and use criteria sheets instead, which are matrices of cells containing verbal statements of standards.

QSA’s position is that marks can be used, but decisions on ratings cannot be made by simply using cutoff marks.  The standards matrices must be applied.  They insist that a mark of say 90% doesn’t guarantee an “A” standard of work because the student may not have fulfilled all of the “A”-descriptors in the syllabus.  However, the following extract from p32 of the Qld. MAB syllabus, produced by QSA, states that a standard can be obtained without necessarily ticking every descriptor.  I can’t see the difference:-
 
“When teachers are determining a standard for each criterion, it is not always necessary for the student to have met each descriptor for a particular standard; the standard awarded should be informed by how the qualities of the work match the descriptors overall”.

If we were simply able to award marks when assessing a well-set exam paper, it would be the job of review panels to check that suitable standards and balances have been maintained.  This is what happened before the QSA became such a powerful influence and advocate for non-marks assessment.  It worked well.  The best students got the best ratings.  We don’t need a cumbersome system in order to achieve that.  Mathematics teachers are good at setting suitably balanced assessment and also at awarding part marks for imperfect solutions and giving full credit for correct alternative solution methods.

The advantage of this would be greater simplicity in devising assessment tasks and in marking them and grading students.  There would be no loss of validity of the grades produced.  In classrooms, the change would produce better teaching and learning.  At the moment, assessment is driving the agenda, because of the many requirements to be satisfied by an “assessment package”.  That’s wrong – our emphasis should be on teaching and learning.  Young teachers including pre-service teachers, and also experienced teachers, have to spend large amounts of professional development time trying to understand QSA’s onerous assessment requirements.  But what we all need to be doing instead, is learning more about our subject and its applications and about the best ways of introducing topics to students and inspiring them in the subject.  We are prevented from using our time to the best pedagogical advantage, because of the time taken to embed a host of little detailed requirements into an “assessment package”.

The marks system is working well in NSW, Victoria, England, and other places, and in most Universities.  Some of QSA’s own QCS tests are assessed with a marks system.  It’s crazy that students suffer under QSA’s cumbersome system for school assessment, and then enter University to find that they are assessed simply and validly by using marks only.

Using marks allows teachers to differentiate between students more readily than by placing ticks in the cells of criteria sheets.  Advocates of the latter system place ticks toward the right or left edges of the cells in the criterion sheet, or in the centre, depending on how well they judge the student to have met the particular descriptor.  That’s all very subjective and unreliable.  Awarding marks according to a marking scheme, and giving credit for alternative solutions which are different from the adopted marking scheme, produces fair and defensible judgments and allows teachers to rank students in order of merit when required to do so.

2) At present there is a plethora of criteria and descriptors of standards in the syllabuses. There are dot-points such as “comment on the strengths and limitations of models, both given and developed” or “identification of assumptions and their associated effects, parameters and/or variables” which require understandings that are sometimes more appropriate to tertiary level studies.  It’s a more urgent priority for high school students to gain and apply mathematical skills to make predictions or solve problems, but these other idealistic dot-point requirements draw away from the time available in school for students to develop basic appreciations of their subject.  The tragedy of the current situation is that if a school should happen to omit even one of these micro-specifications from their assessment “package”, their proposed ratings will be reduced by review panels, even though the level of challenge in the school’s questions may be higher than that of schools who have diligently ticked all the boxes by attending to the many little dot-points.  When exam questions are designed with lots of tick-boxes in mind, the assessment can become somewhat stilted and artificial.  Questions need only be graded in level of challenge, and to be broadly divided into a basic level, suitable for students to obtain a basic “Pass”, and a higher level that would examine problem-solving skills in a range of situations and allow people to achieve the highest ratings.

3) At the website below this paragraph, you will see the newest incarnation of the standards to be applied in Years 1-10, in this case Year 1.  They are called “Standards Elaborations” and they are the means by which teachers are expected to assess their Year 1 students.  By changing the “1″ where it says “yr1″ in the URL, you can get to the other year levels:

http://www.qsa.qld.edu.au/downloads/p_10/ac_math_yr1_se.pdf

If you compare the descriptors in say columns 1 and 2 for Year 1, you might agree with me that it would be very hard for a Year 1 teacher, or anyone else, to decide which column is most appropriate for a particular student’s responses.  Who wants their children or grandchildren to bring home a results sheet like this, or a single letter rating derived from this matrix?  It would be far superior and more useful for the child to bring home a spelling or other test or project marked “18/20″ with the mistakes highlighted, etc.  Marking would be much easier and more reliable, and families could easily see how the child is progressing.  Yet this criterion sheet system is now being used throughout Years 1-10 in our State.

4) External exams would be better than the patchwork quilt system that Queensland has now, where schools all set different exams from each other, and no-one is really sure whether questions marked “unfamiliar” are really so.  The NSW system combining school assessment with the HSC works very well, and it should be easy for Qld to move to that kind of system.  It ensures more reliable comparability of students from different schools.  As the National Curriculum is implemented throughout Qld schools, it makes sense for us to adopt a national assessment approach.  QSA have been setting good external Senior exams in Qld, but have announced that they will discontinue them.  It seems easily possible that they could continue to set them for full-scale use in schools.  Money saved on review panels could be used to pay for teams of markers.

5) QSA claims that Queensland’s assessment system is “world’s best practice”.  There is no objective evidence to support this.  It is the belief of academic theorists.  No other education system has conferred this compliment on QSA – it is merely a wistful statement by insiders, and therefore of little value.  If it were world’s best practice, why haven’t NSW, Victoria, the UK, etc, converted to it?  Why don’t our University Maths departments adopt it?

Let’s improve mathematics teaching in the schools by freeing up teachers’ time so they can concentrate on improving pedagogy and inspiring students, instead of suffocating in a straitjacket of unnecessary assessment requirements.  Let parents once again receive clear simple statements of marks earned by their children.

Appalling QLD results

Comment in the Australian on appaling Queensland results in international TIMSS & PIRLS. Take Note: Queensland dragged the national average down by a massive number of points.

“ Col of Brisbane of Queensland Posted at 11:30 AM Today Take Note: Queensland dragged the national average down by a massive number of points. It is the lowest performing state (second last to NT). Start by NOT doing what Qld does. Queensland academics in the QSA and Ed Dpt have forced Queensland teachers to check-box waffly higher-order thinking in time-consuming rubrics instead of testing the real content of disciplines. These ‘criteria-standards’ sheets have been used for years in high school and have had a wash-down effect on all schools. Now they are even forced on little kids in primary school so that if you get 2 + 2 = 4 correct you only get a D. In order to get an A, a Grade One child has to give a “considered explanation of their strategies” etc). Queensland forces teachers to mark so-called ‘standards’ A,B,C,D,E instead of marks and percentages. Queensland has no comparible tests or end of high school exams, so there is no statewide accountability. In other words, to fix the Australian problem, do the opposite of Qld: by getting back to basics. Also, tests are not the problem but the solution. They are actually beneficial if the kids get feedback on how much they got right or wrong and encouraged to compete for higher marks.

Comment 72 of 72 “

Worn-out and Worried Student

Currently I am completing semester three of year twelve and have been overworked, tired and stressed for the past eight weeks of it.

The purpose of this is to ask a question about how a student is suppose to handle the workload assigned to them.

The subjects which I study are

·        MathsC
·        MathsB
·        Chemistry
·        Physics
·        English
·        IPT
·        Religion and ethics

Throughout this term I have received in the form of assessment two EEI’s in both Chemistry and Physics, a Math B and C assignment, a major programming project within IPT and an oral for English.

I have also not seen my family for more than an hour a day, been awake till the early mornings – generally 1 or 2 am, had a nonexistent social life, and spent almost every waking moment studying. I cannot believe that the life of a student should be based solely upon school work and study. This is both ridicules and insane.

However I am not alone, several of my grade have been ill through stress related sickness, several missing important assessment only making their cases worse, our teachers are doing the best they possibly can putting in enormous hours trying to get us through (one even giving up a birthday to help us study for an exam) however we still cannot cope with the work load.

What is more depressing is after all the effort, horrible hours and sleep deprivation the end grade is not and A nor even a B, generally 9/10 times its a low B to mid C. Although our assessment is easier than one standardised exam at the end of senior, it has gone from one extreme to another.

I merely wanted to express my concern not for myself as I have already completed the majority of assessment but for the up and coming grades who will also experience this.

Thank you

Worn out and worried student

Article in the Australian. April 16, 2012

NEWS  Extra money won’t fix schools

HENRY ERGAS   From:  The Australian April 16, 2012
16 comments

THE Gonski Review of Funding for
Schooling is right: our schools should provide better and more equitable
outcomes. Unfortunately, the funding model it recommends is unlikely
to achieve those goals.

That is because the review’s preferred funding model
constrains choice rather than promoting it. But the key to addressing
our schools’ problems lies in empowering competition to lift all ships,
including those of the truly disadvantaged…. more at link (if subscriber) 
http://www.theaustralian.com.au/news/opinion/extra-money-wont-fix-schools/story-fn7078da-1226327171478

 

COMMENTS ON THIS STORY

            Alec
Baker of Queensland 
Posted at 2:46 PM Today Yes to all. But why does money thrown not work?
Why can’t low-SES students achieve highly? The answer: wishy-washy assessment
in this country, which is getting worse. The Curriculum is a good guide.
But the states still manage assessment. Where holistic criteria are
used the states do poorly in interstate & overseas comparisons.
* Qld has dropped Maths outcomes by ‘2 years of learning’ by end
Yr 10 over the last 3 decades (ACER, 2009). Qld used to be a leader,
now trails in science and maths over the last decades. * Qld Yr 11&12
maths & sciences are no longer attractive due to open-ended rich-tasks
that require uni level research & writing skills; only rich kids
with tutors can cope. * Tertiary bridging courses are needed to prepare
students for engineering. * No statewide benchmark used in Qld for decades.
*Qld teachers are banned from using number marks and instead give ‘quality’
letters. Now, workloads are huge so good teachers are quitting – a vicious
cycle. NB: The Aust Curriculum (ACARA) has set good ‘achievement stds’
but has adopted Qld’s idea to set ‘sample work’ as a way of assessing
students’ core-content abilities. This ad hoc approach will further
dumb-down Aussie kids.





External exams – comments from a school principal

Hubbard’s has been involved with preparing students to sit External exams in Queensland since 1952. Hubbard’s used to be a private company. Since 2001, it has been an accredited non-state school  just like a Grammar School or your local independent school.  I have been associated with Hubbard’s since 1988  have been Principal of Hubbard’s since 1997. Hubbard’s is a non-denominational school that caters for students seeking an alternative option to finish their secondary education. Most students turn 16 or 17 while at Hubbard’s. The next most populous group would be 18 year olds. Older students are welcome too and invariably there are some who turn 19, 20, 22, 22 amongst the student body and often there are a couple of older students too. The majority of the students have very firm beliefs about their incompatibility with the internal-assessment system used throughout Queensland and/or with attending school in the normal pattern of five days per week from 9am  to 3.30pm. Some arrive from overseas early in their Year 12 year and want to finish school that year.  Some just do not like the complexity of the internal system and embrace the transparency of the External arrangements.  Whatever their reasons, they seek an alternative option. Students who join Hubbard’s at 16 i.e  in their Year 11, usually study just three subjects. They sit exams in those subjects on October/ November of that year and receive a Statement of Results for that subject. So they start and finish the two years of assessment in the one calendar year. The exams are tough and test the whole Year 11 and 12 syllabus. University lecturers can be assured that someone who passes these exams, knows their work well and their knowledge is current. Then students  usually do two other subjects in the following year i.e their Year 12

For more details see www.hubbards.qld.edu.au

And  http://www.qsa.qld.edu.au/588.html   There are three years of past exam papers for most non-language subjects in http://www.qsa.qld.edu.au/2427.html . The standard is high. They satisfy the demands of tertiary entry well.

In the letter from Peter Luxton, Acting Director QSA, sent to me earlier this year informing me of the discontinuance of the External senior system, except for languages, he mentioned ‘extensive research’ and that current providers of Senior external classes agreed with its  abolition  and were keen to take up internal assessment.  Attached are the minutes of ‘the extensive research’ meeting. (Just one!!). There were a few other questions asked, with a hidden agenda, over a few emails.  It is impossible to understand how these minutes can confirm what Mr Luxton wrote.

For a well-informed chronology of the development / deterioration of science syllabus/assessment in schools, please read attachment 3: Statement from an experienced science teacher and lecturer.  Makes great reading!!

Letter to Tanya Chilcott, Education Reporter for the Courier-Mail

I teach at a Christian Brothers boys school in Brisbane & have done so for 32 years.

For the last 4 to 5 years I have spoken out about the very serious flaws in our current Physics syllabus.

The first was (& still is) the constructing of criteria sheets to accompany every assessment piece. I’m a reasonably intelligent person but no matter how many inservices I went to (I’ve given up now) on what they were to be achieving and how to construct them, either my Science coordinator or a review panel would always have some problem with them. I’ve lost count as to how many times interpretations have changed. These things take hours to formulate & with ordinary typing skills & no secretarial help any more (all teachers have to do All of their word processing versus more efficient typing staff doing it for us in the past) this is a very onerous task which turns out to be a complete waste of time as very few students read them, let alone follow them to the letter!

Secondly, the syllabus dictates that we have alternative assessment pieces called ERTs & EEIs which stand for Extended Response Task & Extended Experimental Investigation respectively. Four of these must be set over the 2 senior years, their drafts marked and obviously the final submissions marked.  The last of these that I set took days to design (content & criteria sheet), the draft commenting took 1½  weeks of most of MY free time & then each of the final copies took ~ 1¼ hours. WITH OVER 50 STUDENTS TO MARK THIS TOOK NEARLY 3 WEEKS!! (I have four other subjects that have to be addressed professionally during this time also!). Lets not forget that the students (OR THEIR TUTORS?!) are also putting in excessive amounts of time to complete these tasks.

Thirdly, and probably the most important issue for the students, is that all of this should have some fantastic benefit for the students’ learning. BUT THIS ISN’T SO. I can categorically say (remember, 32 yrs teaching) that this system is flawed in another  major area. When the EEIs & ERTs are being done by the students they are the only assessable item in that term. Other topics still have to be taught BUT because the students know that they won’t be assessed THEY IGNORE THEM! This leaves our graduates with large holes in their preparation for tertiary studies – this never happened prior to this syllabus.

If you, via your paper, can bring the huge problems this ridiculous QSA syllabus creates for teachers, students & their parents, to the attention of the public & our politicians I will be forever grateful (my wife says I’m a fool for continuing to be part of such an impost on our lives & that I should get out of the profession!).

Teachers name noted but withheld

Regarding Summary of Meeting with Hon. JP Langbroek

On the 4 points, a few thoughts.

1.     A review – fine – I am a bit concerned at the OP – each school is pulling kids that are not doing well so the OP cohort is getting smaller and smaller. In addition, many/most schools are spending a lot of time and $$ in QCS Test practice.

2.      Long assessment/assignment pieces – YES reduce the length – particularly the EEI’s in Science.

3.     Marks – I am happy to use marks  and still do. The standards grids are OK in some areas/subjects but a number of us fought for years to be allowed to use marks in Maths. The example of a poor criteria sheet – I think you will always find poor examples of same just like you will find poor examples of tests, even these with marks.

4.     External exams – I am afraid I have to disagree with you on this one. I did external exams for both Junior and Senior and have taught external exams. I think students from smaller schools and in particular those “west of the Great Divide” would be disadvantaged with same through lack of experienced teachers.  The current system might have its faults but External exams will not be the solution. If same were a percentage of the final mark, there is no prize for guessing what would soon become the defacto “true grade”.

One of the advantages of the current system (with Maths and Science students) is that students can use all sorts of technology in the teaching and learning situation AND also the assessment domain. External Exams preclude this eg I am currently doing projectile motion in Yr 12 Maths C and using Logger Pro as a means of collecting live data and same is used in the assessment process. I tend to keep assignments at about 25% of total marks which seems to work fine.

Vagueness and Subjectivity

QSA have prepared an excellent rich task for Year 9 Mathematics.  It’s this year’s QCAT task.  But the assessment method that QSA proposes for it seems onerous and subjective.  I don’t know how they can think it’s “world’s best practice”.  It’s called a “GTMJ” (Guide to making judgments).  A traditional marks system would be very suitable for the task, being easy to follow, allowing good differentiation between students with similar performance levels, and providing perfect feedback to students about their work.  The descriptor paragraphs in the GTMJ leave me wondering: how can anyone be sure that this is the best descriptor to assess this particular standard?  It’s all someone’s opinion, and is often hard to agree with.  The process of applying the standards, even if you agree that they’re all appropriate, is very subjective, vague, and approximate.  This is the style of assessment that QSA want us to use in all types of tasks and in all year levels including Primary, and I do not want my own children (or anyone else’s) to go through this.  It’s just imprecise and unnecessary, and it does the standard of Mathematics education in our state a disservice.

See for yourself the inbuilt vagueness and subjectivity in this standards marking process, by viewing these short videoclips on QSA’s website:- http://www.qsa.qld.edu.au/14735.html
http://www.qsa.qld.edu.au/14736.html
http://www.qsa.qld.edu.au/14746.html