Chester Clinic

 

Mathematics: traditions and [new] practices

MEASURING ACADEMIC NUMERACY:
BEYOND COMPETENCE TESTING
University of Southern Queensland Academic numeracy consists of three critical elements: competence, confidence, and critical awareness of students' own mathematical knowledge and the mathematics used in students' future professions. This definition is used to frame pre-test assessment in a first year nursing program. Competence and confidence were measured using a paper and pencil test. Critical awareness was measured via students' reflections on their own performance, their relationship to mathematics, and their understanding of how mathematics relates to nursing. Results show issues related to professional numeracy practices including relatively low understanding of the connection between mathematics and nursing. Introduction
The word "numeracy" was first coined in the 1959 in the UK from the Crowther report
(1959) and redefined in the Cockcroft Report (1982) to reflect literacy. More recently it
has been hijacked by the school lexicon, largely in Australia, to the extent it is often
seen as a replacement for "mathematics", particularly in primary schools. The term is
also used in adult education, where it has taken on a context of basic skills in the
workforce or everyday life. Academic numeracy, however, has attributes of both school
numeracy and particular professional numeracies. Students need to situate mathematics
learned in school to their imagined future context. This shift is hopefully aided by the
courses students enrol in at university where they are exposed to contextualised
mathematics. Students also need to situate the mathematics in the particular academic
context. While the mathematics strongly reflects the professional context, it is not the
same. The academic context has particular extra attributes, such as reading and
critiquing journal articles, or investigating particular topics in depths students may
never experience in their professional careers. These contexts may also need
mathematically based skills. Academic numeracy is not the same as professional or
school numeracy.
The term "academic numeracy", modified from Yatsukawa and Johnston (1994), was
used by Galligan and Taylor (2005) to clarify the skills necessary for success in the
university context and defined as:
… a critical awareness which allows the student to situate, interpret, critique, use and perhaps even create mathematics in context, in this case the academic context. It is more than being able to manipulate numbers or being able to succeed at mathematics. (p. 87) MATHEMATICS: TRADITIONS AND [NEW] PRACTICES • AAMT & MERGA 2011 However, academic numeracy needs to highlight two further attributes: confidence and competence (Coben, 2000). Thus, I propose academic numeracy has three elements:  mathematical competence in the particular context of the profession and the academic reflection of the profession at the time;  critical awareness of the mathematics in the context and in students' own mathematical knowledge and involves both cognitive and metacognitive skills; and  confidence, highlighting its deeply affective nature. While mathematical competence testing is common, both at university and school, few incorporate a broader definition as proposed here. These three elements are needed to assess students' numeracy in a particular professional path—whether it is in nursing, engineering, business, or education. This paper will outline an approach taken to test nursing students' academic numeracy at the beginning of their degree. This research is part of a larger study that investigated the development of nursing students' academic numeracy over their first semester of study. Method
In 2008, 206 first year nursing students at USQ undertook a first semester course to
build nursing attributes of mathematics and computing. Students attended one two-hour
tutorial per week. One of these tutorials was taken by the researcher. Three instruments
were used to assess academic numeracy in the first two weeks of the course. The first
was an online discussion forum where students wrote about their mathematics
experiences; second was a mathematics competence and confidence test, and the third
was students' reflections on the results of the questions in the test.
Online discussion forum: In the very first tutorial, students discussed their past
experience with mathematics and were encouraged to reflect critically and honestly.
The informal discussion in class and the time given to actually writing, prompted well
thought out responses. In this first tutorial, a discussion was also held on the advantages
and disadvantages of different scales in nursing, using a pain intensity scale (Figure 1a).
Students were then asked to rate their relationship with mathematics on a five-point
scale (Figure 1b).
Figure 1. (a) Pain intensity scale (left) and (b) mathematics relationship scale (right). MATHEMATICS: TRADITIONS AND [NEW] PRACTICES They were also directed to articles on the relationship between mathematics and
nursing. Using these two exercises as a basis they were asked to reply to the questions,
"Describe your previous experiences with mathematics in a couple of sentences" and
"How do you think mathematics…will be important for you as a nursing student, and
later as a professional?" in an online forum, as part of a larger suite of questions about
introductory academic studies.
Mathematics competence and confidence pre-test consisted of 32 (Table 1) items that
was equivalent to tests undertaken previously by first year nursing students at USQ.
Students were given a paper-based version of the test in tutorial classes. They then
started the test in class and could finish it in their own time. Once they completed the
test on paper they submitted their answers online via a Computer Managed Assessment
system (CMA). The test also included a 5-point Likert style section on confidence
levels.
Student reflections: The pre-test was then used as a stimulus for student reflection.
When the test had been completed online, students were sent their results, question by
question, in a table via automatic reply email. Students then copied and pasted this
result into a word document and they then added two columns: a reflection on each
question, and a strategic response about what to do next. Examples of the type of
reflections and strategies were provided online, in the study book, and in class. Students
then emailed these results and reflections to their tutor. While this was an assessment
piece, marks were allocated on completion of the test and reflective comments and did
not depend on how correctly they answered the questions.
Results
Replies on the discussion forum were classified into one of five categories. The
categories: Hate, Dislike, Neutral, Like, Love were used as they reflected the sentiments
in Figure 1b. The results are shown in Figure 2.
Figure 2. Relationship of nursing students to mathematics (n = 206). MATHEMATICS: TRADITIONS AND [NEW] PRACTICES A sample of students comments coded from "Hate" to "Love" is provided:  I ended up changing from Maths B to A but not before developing a loathing for  Maths I do not like it. For me it's like great mystery (Dislike)  Sometimes numbers just don't make sense to me…I do blame the teachers and their inability to find why maths perplexed me so much, but in high school I do blame myself … now I find maths much better, I am no longer afraid of numbers and can grasp the theory and at least try to put it in practice—even if the answer is wrong. (Neutral)  Maths has always been an enjoyable experience for me. (Like)  I learned mathematics at school and really love mathematics … (Love) Students also described other relations with their mathematics. For example, 14 students explicitly commented on teachers' influence on their mathematics, mainly negative, but there were some positive comments, for example: I dreaded mathematics at school. It was my worst subject but that was due to never
having a constant teacher. It wasn't till year 11 and 12 when I had the one teacher
throughout that helped me a lot that I began to enjoy it a little.
I dreaded to do maths because I found it boring and the teachers were not always that
helpful.
Pre-test results were generated from the CMA system and included both an overall
percentage correct for each question and an individual student-by-student response. The
pre-test results were analysed in four ways: competence, comparison with previous
semesters, relationship between confidence and competence, and error.
Overall 192 students completed a pre-test. The pre-test had 32 questions. Table 1
shows the questions, the percentage correct, and mean confidence levels per question.
There were seven questions where fewer than 52% of the students were correct:
question 6 on estimation (23% correct); question 10 on average (31% correct); question
20 on conversion from hours to minutes (50% correct); question 22 on conversion from
grams to milligrams (51% correct); question 25 on substitution into a formula (44%
correct); question 26 on solving an equation with the unknown on the denominator
(48% correct) and question 27 on reading a scale on a syringe (48% correct). The
median mark of students was of 25 out of 32 and the middle 50% of students were in a
range from 21 to 28. While the test showed the majority of students doing relatively
well, and were comparable to previous years, nursing students are expected to be fully
competent in various aspects of nursing that require numeracy and in particular drug
calculations. During their nursing degree they will undertake a specific medical
calculations course, and have multiple instances for testing their numeracy skills.
Students ranked their confidence with each of the 32 questions (Table 1, next page)
from 1 = no confidence, to 5 = very confident. Students particularly lacked confidence
with three questions.
Question 26 (3.39 out of 5). The poor result is not surprising as students could not
use an intuitive approach (e.g., doubling or dividing by a whole number) and in general
research suggests students show an inability to scale by non integer (Steinthorsdottir &
Sriraman, 2009). If using an algebraic approach, the unknown on the denominator is a
more difficult question than one where the unknown is in the numerator (related to poor
manipulation skills, Poon & Leung, 2009).
MATHEMATICS: TRADITIONS AND [NEW] PRACTICES Table 1. Results of pre-test and confidence levels (1 = none; 5 = complete) S1 2008 (n = 192). 1. Write … in numerals: Twenty thousand two hundred and six 83 6. Estimate 512  174 7. Round 495 to the nearest 10 10. Find the average (mean)…: 21.3, 22, 24.7, 20.4, and 19 12. Express 3/4 as a decimal 13. Express 80/480 as a fraction in simplest form 14. 7.42  100 15. Find 30% of 25 16. Express 0.5 as a fraction in simplest form 17. Calculate: 2 mL -1.34 mL 18. Calculate: √81 95 19. Express 7 hours 20 minutes in minutes 20. Express 1.2 hours in minutes 22. 1.23 g = ?mg 23. The chart … When was his temperature the highest? 24 What was his temperature the last time it was taken? find b if w = 2, and h = 4. Answer as a fraction. 27. …. How much fluid is in the syringe? 28. Energy is measured in Kilojoules (kJ). Margarine contains 32.2 kJ/gram. How much energy is in 500g tub of margarine? 29. Unit in qn 27? 30 A clock gains 15 secs in a day. How long does it take to gain 2 mins? 85 31 A Paediatric patient weighing 25 kg is ordered Augmentin 10mg/kg. If Augmentin is supplied as a syrup containing 125mg/mL, how much syrup is to be measured out? 32 Unit in question 31? Question 31, an in-context proportion word problem, had an average confidence level of 3.72. It too had a relatively low pass rate (57%), reflecting school students' difficulty in MATHEMATICS: TRADITIONS AND [NEW] PRACTICES this problem solving area (Stacey & MacGregor, 1993). Nursing students have expressed anxiety in undertaking word problems (Galligan & Pigozzo, 2002). This, combined with the unfamiliarity of the terms, as the test was undertaken at the beginning of the first semester of their degree, would account for the low confidence level. Question 25 (3.81). While only 44% of students were correct with this question, 70% had either 1/8 or 0.125 as an answer. This suggests that the algebraic symbols or the squaring on the denominator may have created the uncertainty. Figure 3 shows the relationship between confidence levels and pre-test results. Overall, there was a positive correlation with about 38% of the variation of the pre-test results being accounted for by confidence level. While all the questions below the regression line could be identified as ‘overconfident', question 6 (on estimation), question 10 (on average) and perhaps question 27 (on reading a syringe), in particular appear to be overconfident (circled points in Figure 3). Figure 3. Scatterplot of confidence levels and pre-test results. An aim in the nursing numeracy course is to improve confidence and competence in numeracy ensuring students are neither confident and wrong, nor underconfident and right. The confident and wrong category is of particular concern, as they may not know they are wrong and may be less likely to check their calculations. There were 183 students that had at least one question confident (4 or 5) and wrong. On average, these students got 5.01 question incorrect (sd 2.94), and 17 students had more than 10 questions wrong in this category. Questions that were ‘overconfident' in at least 50 cases are shown in Table 2. These seven questions match the data points below the line of best fit in Figure 3. The other two questions below the line were question 1 on numerals (83% correct) and question 8 on order convention (76% correct). Table 2. Questions where there was significant overconfidence. Question Details Number confident & wrong (4 or 5 in the Likert scale) MATHEMATICS: TRADITIONS AND [NEW] PRACTICES grams to milligrams reading graph—what temperature reading the syringe The test results and comments from the researcher's class were collated into one spreadsheet that detailed students' responses and reflections for each question. Sixteen of the students submitted comments and results. These were analysed for themes, question by question. One preliminary scan and two complete scans of the document produced seven themes. The themes were also coded positively or negatively and whether the student was correct or not, thus creating four alternatives for each question. From the 512 possible comments (16 students × 32 questions), 697 identifiable points were extracted. Themes and some samples can be seen in Table 4. Table 3. Themes from student reflections. Theme Example Students' ability or understanding I never understood fractions (negative/ wrong); I found this easy however I got it wrong (positive/wrong). Confidence (20.6%) I have never been that confident with multiplication. Especially when it is with large numbers (negative/right); I feel confident with this type of question (positive/right) Complacency/checking (12%) Care needs to be taken to ensure answer received make sense (positive/right); That again is just terrible adding up and not taking the time to check. Check and re-check my answers (positive/wrong) Calculator (13.3%) Fairly confident No calculator (did not use/wrong) Knowledge/Remembering/ Once I remembered what the symbol that was it was easy. try to Thinking/Method (19.4%) unlock my suppressed maths from my brain (Positive/right);. Don't know what the sign over the 81 meant (negative/wrong). Affect (happiness/ I didn't read the syringe properly … dangerous I could of killed enjoyment/relieved); someone!! (positive/wrong). Percentages don't seem to agree with Importance/life experience (3.9%) me, I didn't think I would get it right (negative/wrong) Silly error (1.2%) I didn't read the question properly, silly mistake From the data of errors made in one class and an analysis of errors of the whole cohort, a profile of error was starting to emerge with some points to note:  Students sometimes had a variety of solutions that, in the context of nursing would probably be acceptable, so care needs to be taken to ensure the mathematics is marked appropriately or the question is worded realistically.  Students need to be aware of reading questions correctly. This can often be vital in nursing where prescriptions and directions from doctors are exact. While entering units in an answer when the question asked for no units may appear trivial, it does reflect a hidden issue of reading instructions in general.  There appears to be an underlying issue of understanding of some mathematics concepts mainly decimals and fractions and perhaps an awareness of estimation. MATHEMATICS: TRADITIONS AND [NEW] PRACTICES While students appear to be aware of their lack of understanding of fractions, there may be a limited awareness of the issue of decimals. This is particularly true when students are using their calculator.  The pre-test appeared to focus on mathematical skills with few of the questions having an explicit nursing context. However, there were very few reflective comments from students suggesting a problem with their mathematics related to a clinical setting. This is despite explicit articles students were directed to read in the first two weeks of semester. Conclusion
The instruments used in this study aimed to identify students' numeracy when they first
started the course. Numeracy was framed in terms of competence, confidence, and a
critical awareness of the mathematics in context and in students' own mathematical
knowledge.
Confidence and critical awareness included an understanding of students feeling towards mathematics. Only about one quarter of students disliked or hated mathematics. Many students described changes in feelings towards mathematics, sometimes from liking in primary to disliking in high school; others were the other way around. When directed to reading specific articles on mathematics in nursing, most students could see the relationship between mathematics and nursing in general but were unable to articulate that in specific mathematical skills. In analysing students' results from a test designed to investigate nursing students mathematics knowledge needed in nursing, students did fairly well. The percentage of students correct ranged from 23% correct for a question on estimation, to 97% for a question on subtraction. Students' confidence in their answers varied from 3.38 (out of 5) for solving an algebraic problem, to 4.84 for writing a number in numerals. However, a successful nurse needs to be confident and competent with this level of mathematics, so something close to 100% would be the aim for these students in competence and 5 for confidence. In this first test there was a significant issue with both under-confidence (right but not confident) and over-confidence (wrong and confident) with their answers. Since 2008 the implementation of tests have been refined to further engage students in being more critically aware of their own mathematical skills and the mathematics needed for their degree and to make the test easier to administer using a variation on Self-Test (Taylor, 1998). It is also planned to extend this approach to the final year of students' degrees where they can again appraise their competence, confidence, and awareness of the mathematics needed in the career in which they are about to embark. Variations of this approach have also been used in other degree programs, in Economics, Engineering and Education where the focus is not just on competence at a particular moment in time, but with an aim to assess their academic numeracy. References
Coben, D. (2000). Section 1: Perspectives on research on adults learning mathematics (introduction). In
D. Coben, J. O'Donoghue & G. E. FitzSimons (Eds.), Perspectives on adults learning mathematics: Research and practice (pp. 47–51). Dordrecht, NL: Kluwer Academic Publishers. Cockcroft, W. H. (1982). Mathematics counts: Report of the committee of inquiry into the teaching of mathematics in schools under the chairmanship of Dr W. H. Cockcroft. London: Her Majesty's Stationery Office. MATHEMATICS: TRADITIONS AND [NEW] PRACTICES Crowther, G. (1959). 15 to 18: A report of the central advisory council for education. London: Her Majesty's Stationery Office. Galligan, L., & Pigozzo, R. (2002). Assisting nursing students solve drug calculation problems using metacognition and error analysis. Literacy & Numeracy Studies, 12(1), 45–62. Galligan, L., & Taylor, J. (2005). Investigating academic numeracy in non-mathematics courses at university. In "Bildning" and/or Training: Proceedings of the 11th International Conference on Adults Learning Mathematics (pp 86–95). Kungalv, Sweden: Goteborgs Universitet. Poon, K-K., & Leung, C-K. (2009). Pilot study on algebra learning among junior secondary students. International Journal of Mathematical Education in Science and Technology, 41(1), 49–62. Stacey, K., & MacGregor, M. (1993). Origins of students' errors in writing equations. In A. Baturo & T. Cooper (Eds.), New directions in algebra education (pp. 205–212). Brisbane: Queensland University of Technology. Steinthorsdottir, O. B., & Sriraman, B. (2009). Icelandic 5th-grade girls' developmental trajectories in proportional reasoning. Mathematics Education Research Journal, 21(1), 6–30. Taylor, J. A. (1998). Self test: A flexible self assessment package for distance and other learners. Computers and Education, 31, 319–328. Yasukawa, K., & Johnston, B. (1994). A numeracy manifesto for engineers, primary teachers, historians.a civil society: Can we call it theory? Proceedings of the Australian Bridging Mathematics Network Conference (pp. 191–199). Sydney: Sydney University. MATHEMATICS: TRADITIONS AND [NEW] PRACTICES

Source: https://www.merga.net.au/documents/RP_GALLIGAN_MERGA34-AAMT.pdf