Can We Actually Assess Learner Autonomy? The Problematic Nature of Assessing Student Autonomy

This paper explores, from a theoretical basis, the difficulty in defining and assessing learner autonomy in higher education. Although the development of learner autonomy as a key aim of higher education, it is a vague and ill-defined term. As such, the assessment of learner autonomy within university programs of study is highly problematic. The author argues that the authentic assessment of genuine learner autonomy may not be possible within formal credit-bearing programs of higher education. The aim of the paper is to stimulate reflection and discussion so that university teaching staff may reflect and consider whether they can assess autonomy in the programs they are responsible for.


Introduction
Globally, higher education programs frequently claim that they develop learner autonomy. Learner autonomy has been a central aim of higher education for many years, arguably, one of the 'ultimate goals of higher education (Bajrami, 2015). Over 40 years ago, Boud (1981) argued that in terms of the goals of higher education, it was: "not just one goal among many but rather a characteristic of all of the others: it is how all skills should be displayed and all beliefs held." This is still true today, with autonomy has become increasingly important (Lamb & Reinders, 2008;Viera, 2009, Blin, 2004Cao, 2012;Kormos & Csizer, 2014;Moore, 2016). Numerous university program and module specifications, along with pre-specified learning outcomes, identify autonomy as an aim, an output and, a 'graduate attribute ' (Channock et al., 2004), along with a claim that it is being developed and assessed. Yet, the concept of learner autonomy is not a simple one (Boud, 1988;Benson, 2011). Autonomy is a problematic concept; it is multifaceted, its development being both a product of and a process in education. From a practitioner's perspective, it is frequently conflated with 'independent learning.' From an assessment perspective, it is difficult to assess with authenticity (Murase, 2015). This paper builds on previous work by providing a critical overview of problematic nature of learner autonomy and its assessment.

Background
Autonomy in learning is not a simple concept; it can mean many different things to different people. While there is common understanding amongst educators about what learner autonomy in general is, there is a lack of consensus about precisely what it means. Within programs of higher education, it would seem to be an "essentially contested concept" (Gallie, 1956), i.e., something that is impossible to conclusively define, but perfectly possible and rational for people to discuss and justify their holding of one interpretation rather than a competing one. Yet, if it is acknowledged that autonomy may not be possible to conclusively define and that there are different interpretations, it implies that its assessment is problematic.
The terms: 'autonomy,' 'independent learning,' 'autonomous learning,' and 'independent study' are often used interchangeably by practitioners and within some of the literature to describe what is essentially the same thing. Definitions and explanations of autonomy often include independence (e.g., Little, 1988), and vice-versa, definitions of independent learning often include autonomy (e.g., Moore, 1973). Autonomy is, unfortunately, neither a simple nor easily described behavior (Dam, 2003;Little, 1991). 'Autonomous learning' is rarely clearly defined (Macaskill & Denovan, 2013), and there is an overall lack of consensus as to precisely what 'independent learning' itself means (Broad, 2006). Scott et al., (2015) suggest a "myriad of inter-related definitions of autonomous/independent learning" in existence. In a similar vein, Mayer (2010)  Whilst Thanasoulas (2000) suggests that the "literature is riddled with innumerable definitions of autonomy and other synonyms for it." One of the difficulties with the term autonomy is that it refers to different things (Benson & Voller, 1997, Thanasoulas, 2000. For example: to describe for situations in which learners study entirely on their own; to refer to skills which can be learned and applied in self-directed learning; to an inborn capacity; to learners having ownership of and taking responsibility for their learning; and for the right of learners to determine the direction of their learning. Because it is difficult to precisely articulate what 'learner autonomy' is, other than in a generalized way, educators/assessors hold different understandings and interpretations. Many frequently refer to 'independent learning' instead. It follows that it is therefore worth exploring some of these terms in detail to gain a deeper understanding, to conclude as to whether or not learner autonomy can be assessed.

Independent Learning
Independent learning is a process, a method and a philosophy of education. A person acquires knowledge by their efforts and develops inquiry and critical evaluation (Candy, 1991, Meyer, et al., 2008, Meyer, 2010. Responsibility for learning is placed on and lies with the learner, not a tutor/lecturer. This responsibility includes the learner's freedom of choice in determining the aim(s), objective(s), and goal(s) of and the purpose(s) for their learning. The learner does not have to be a student registered/ enrolled in a formal program of academic study. As such, the individual learner's objectives may not necessarily, and typically, are unlikely to, be the same as those specified as learning outcomes/ objectives within a program of higher education study. Independent learning and independent study are typically described as involving 'self-regulated learning. This refers to the self-directed process by which people become masters of their learning processes (Zimmerman, 2002(Zimmerman, , 2015. Self-regulated learning involves the metacognitive, motivational, and behavioral processes and sub-processes that are personally initiated to acquire knowledge and skills. These include, among other things, goal setting, planning, learning strategies, self-reinforcement, self-recording, and self-instruction (Zimmerman, 2015). From a pedagogical standpoint, the learners are regarded as having an understanding of their learning, their situation and of being motivated to take responsibility for it. Despite the term 'independent learning,' the consensus in the literature is that the learner does not work alone. The teacher/ tutor/lecturer normally provides a structure and the learning environment itself is structured (Gorman, 1998;Perry, et al., 2006). In some versions of selfregulated learning, the student is also expected to control their learning environment in some way (Zimmerman and Schunk, 2001). From a definitional standpoint, though, a teacher-provided structure could only apply to 'directed independent learning. Independent learning is usually categorized into two distinct types. Firstly, 'directed independent learning', where students are guided by curriculum content, pedagogy and assessment, and supported by tutors/lecturers and the learning environment, and in which students play an active role in their learning experience (Thomas, et al., 2014). Secondly, 'selfdirected independent learning'. This primarily relates to Adult Education learning and processes of self-instruction. Self-directed independent learning is seen as a pedagogy in which adult students have the primary responsibility for the planning, conduct and evaluation of their learning (Caffarella, 2000, Hiemstra, 2000, Merriam & Caffarella, 1999. Self-directed independent learning does not have to occur within an educational institution or accredited program and does not have to be a part of any formal qualification. However, a formal qualification could form a part of self-directed independent learning. For example, an adult learner may choose to study philosophy outside of a university environment. They would set their own learning goals, such as to learn, within a certain period, about Hume's fork or the problem of induction and Kant's notion of ethics. They would then test their knowledge, perhaps through writing or in discussion with others. As part of their learning process, they could decide to engage in some university-level study, for example, through a short Adult Education class. Boud (1998) suggests that the main characteristic of autonomous learning is that students take "significant responsibility for their learning over and above responding to instruction." The student exercises their personal agency. Autonomous learning is variously depicted as learners taking charge of their learning (Holec, 1981;Dickinson, 1993;Benson, 2011): learning the process(es) of 'how' they, as an individual, learn and about their attitudes and attitudinal values towards learning (Dickinson, 1993): and as being about learners' modes of learning (Benson, 2011). The most frequently cited definition of autonomous learning is Holec's (1982), which is that it is "The ability to take charge of one's learning." This ability to take charge of one's learning is the "single common thread" that runs through the literature (Little, 2007). Holec's definition is a very broad one and applies equally to independent, self-directed learning. We may therefore assume that independent, self-directed learning, outside of the constraints of a student registered on a program of study and following a university-prescribed curriculum, is the same as autonomous learning.

Autonomous Learning
A more comprehensive definition of autonomous learning is provided by Candy (1991), who describes six aspects of autonomy. These are that the learner: (i) has freedom of choice; (ii) can develop goals and plans independently of pressure from others; (iii) has a capacity for reflection; (iv) has the will and the capacity to "fearlessly and resolutely to carry into practice, and through to completion, plans of action…without having to depend on others for encouragement and reassurance"; (v) can exercise self-mastery; and (vi) has a personal concept of their self as being autonomous. All of these apply equally to self-directed independent learning. Although Candy's work helps identify the 'nature' of autonomy, in respect of assessment, it does not tell us. No aspects of the definitions lend themselves easily to use for assessment purposes. Using them as the basis for clear, assessable, pre-specified learning outcomes within a university credit-bearing framework would be extremely difficult, if not impossible. How, for example, could it be determined with any level of authenticity whether a person had acted fearlessly and resolutely or not? Or that a 'power of learning' had been exercised? How, when students always have the pressure of assessment deadlines, could it be determined that a learner had developed goals and planned independently of pressure from others? How can they have freedom of choice within a prescribed program of higher education study? What exactly is 'self-mastery'?

The Nature of Autonomy
There are various questions that may be asked about the nature of learner autonomy. Is it a specific skill or a set of skills? Is it competence, an attitude, or an understanding? Is it a disposition, a behavior, or a set of behaviors? Or, is it some combination of many or all of these? Learner autonomy does not comprise a single behavior or set of behaviors that can be easily described (Little, 1981). It is a multi-dimensional construct (Benson, 2011) with many different and often unclear meanings. There are multiple and frequently somewhat vague interpretations of autonomous learning. Macaskill and Taylor (2010) argue that "there does not seem to be a single consensual definition" and that many academic journal articles "appear to discuss autonomous learning without defining exactly what they mean by it." Ecclestone (2000) argues that it is genuinely difficult to articulate with any level of precision what the criteria and outcomes are for autonomy. Defining and clearly describing autonomy is also problematic because it may manifest in many different forms, at different ages, and in different situations (O'Leary, 2007). It is clear that the nature of learner autonomy is, as Ecclestone (2007) suggests, a "slippery concept."

Distinguishing Autonomy and Directed Independent Learning
Little (2000) conceptualizes autonomy as a psychological capacity, which a learner may, or may not, decide to exercise.
Autonomy… depends on the development and exercise of a capacity for detachment, critical reflection, decision making and independent action: autonomous learners assume responsibility for determining the purpose, content, rhythm and method of their learning, monitoring its progress and evaluating its outcomes.
Little's perspective provides us with a very useful distinction between autonomy and directed independent learning. From it, we can see that whilst directed independent learners may utilize reflection, decision making and act independently, they would not, within a formal program of university study, determine the content, nor purpose, of their learning. Their learning is structured and facilitated by a tutor/lecturer, working within a defined and prescribed structure of a curriculum with specified assessment tasks, with the purpose of achieving a pass to be awarded a qualification or credit towards one. The students' learning resides within the specified parameters of their program of study, as determined by disciplinary boundaries, the educational institution and its structures, and those of external quality regulators and professional body accreditation requirements. Conversely, a self-directed independent learner, an autonomous learner, would be able to decide the purpose and content of their learning because it is not structured nor constrained by the requirements of a program of study. It follows, that directed independent learning cannot be regarded as being autonomous learning.
Almost 50 years ago, Moore (1973)  Based on Moore's definition of the nonautonomous learner, a definition of autonomous learning was developed (Holmes, 2018) -the "autonomous learner has freedom of choice in determining what to learn, how they should learn it, and by when they should have learned it." Using this definition, it may be argued that autonomous learning cannot be part of a formal program of academic study. Therefore, the author's assertion that universities should not claim that they assess their students as being autonomous learners because, by the very nature of studying a prescribed program of university study, the student is, by definition, not an autonomous learner. They may be able to exercise responsibility for their learning over and above responding to tutor instruction and may have some flexibility or choice in deciding how they learn. Yet, they cannot decide what to learn, nor when they may learn it by. Students have very little, if any, flexibility in deciding when they may submit their assessed work (Holmes, 2019); there are fixed assessment points, fixed examination boards, fixed graduation points. Students have limited freedom of choice in determining what and how; they may learn something and almost no choice when they learn something. Consequently, they cannot be autonomous learners.

Developing Autonomy
Universities may, however, be able to facilitate the development of student autonomy. Within the literature, there are opposing views. One regards autonomy as something that students need to be taught and to learn. The other regards it as preexisting/innate, something that all learners have and can exercise to some extent. Holec's (1981) work, for example, frequently cited as a key text, supported the former view, arguing that learner autonomy was not inborn but must be acquired through learning (whether formal or informal). Contrastingly, Moore (1973) argues that it is innate and is related to a person's state of development, and at some point, a person "acquires" autonomy. Holec's (1982) definition of autonomy, "The ability to take charge of one's learning," has subsequently been modified by others, frequently depending on whether they see learner autonomy as a means to an end or an end product itself of a learning process. Some (e.g., Candy, 1991;Thanasoulas, 2000) argue that it is a process, not a product and that a learner does not become autonomous, only that they work towards autonomy. Others, however, regard it as being both a process and an outcome. It is advocated as both a means to an end and as an end in itself. This is problematic for educators, assessors and learners, as it may be unclear which of these is the aim. Is it what Boud (1988) labels as being a product-orientation; that is, producing an autonomous person? Or, is it a process orientation, i.e., introducing activities to the teaching and learning process which require students to act autonomously (e.g., Dam, 1998)? Or is it, as Holec's definition would suggest, both? For assessment purposes, it would be essential to ensure which was being referred to. It is also worth noting that a process-oriented approach may not lead to a satisfactory product outcome. A capacity for behavior is not the same as demonstrating that behavior. A student may believe that they are and that they are acting autonomously, yet in practice, their behavior may exhibit few signs of this (Holmes, 2018).

Challenges in Assessing Learner Autonomy
As has been shown, using the existing definitions of learner autonomy means that a student studying a prescribed university program of study cannot be considered to be an autonomous learner. Yet, from the perspective of autonomy being a process, not a product, university educators can facilitate students 'becoming autonomous', and aspects of this may be assessable. The rationale for assessing it is that, if universities do not, then, as the literature on assessment suggests (e.g., Gibbs, 2006;Race, 2014;Torrance, 2012;Baird, et al., 2017), some students may not regard it as being important and therefore not make any effort to become autonomous. Yet, to identify if a student is in the process of actually 'becoming autonomous,' this would need to be demonstrable and assessable. As Benson argues, "If we aim to help learners to become more autonomous, we should at least have some ways of judging whether we have been successful or not" (Benson, 2001). In respect of pre-university education, Black et al. (2006) have suggested that the difficulties associated with assessing autonomy are so great that the focus should be on promoting educational practices that can increase learner autonomy without attempting to assess it. Yet compulsory education does not typically claim that it develops autonomous learners. Therefore there is no real need for its assessment in schools.

How Many Higher Education Institutions Legitimately Claim that their Students are Becoming Autonomous Learners?
From the position that autonomy is a process, not an outcome, Boud (1981) suggests that it is not an absolute standard to be met. Still, a goal to be pursued and that what is important is the direction taken by the student towards responsibility for their learning, not the magnitude of change. Building on previous work by Lewis (1978), he argues that the "only realistic goal for higher education is that students should be more autonomous when they leave a course than when they enter" (Boud, 1981), not that they have reached a specific point that may be measured or judged for assessment purposes. There is a very clear argument that all assessment involves making a judgment (Taras, 2010) and that, without a judgment, there can be no assessment. Yet Boud argues that universities should not attempt to make a judgment and therefore not assess whether a student has become, or the extent to which they have become, an autonomous learner, but, instead, to focus on whether they are in the process of becoming one. If the focus is on whether a learner is in the process of becoming autonomous, then there would be no need for them to have to demonstrate that they had become autonomous, only that they were becoming so during a program of study and were, in some way, 'more autonomous' at the end, compared with when they commenced it. Although this would necessarily involve some form of academic judgment in comparing their behavior at the start and end of their studies and there are in-practice tensions that would arise from this (Holmes, 2018), though they are not insurmountable. Whether or not a learner demonstrated they were becoming autonomous would not easily be graded for assessment purposes. Therefore any process of determining whether they were may have to lie outside of university creditbearing frameworks used for the award of degrees. However, assessment tasks that required a student to increasingly act autonomously in some way to produce their assessed work could legitimately be used to demonstrate the development of autonomy.
A simple example is that at the start of a program of study, a student needs to be provided with a prescribed reading list of specific books and journal articles that they must-read. Towards the end of the program, they can identify the reading they need to do themselves for a final-year research project or dissertation. Although this would be only one aspect of autonomy, it would demonstrate that a student was in the process of 'becoming autonomous.' Similar scenarios may be envisaged for students engaged in, for example, lab work, group work, and group research projects as they moved from the first to second to the final year of undergraduate study. Adopting such an approach might allow universities to legitimately claim that they were fulfilling one of higher education's central aims; the development of learner autonomy.
Unfortunately, one further serious challenge may be anticipated. If educators explain to their students that their developing autonomy will be assessed in some way as they progress through their degree, this creates further problems. Benson (2001) argues that measuring autonomy is problematic because autonomous behavior should be initiated by the student rather than in response to an assessment or learning task. This creates a serious challenge. If a tutor/assessor explains to learners that they are, or will be, assessing their autonomy in any way, some students will demonstrate behavior that they believe the tutor/assessor will perceive as being autonomous, although it will not be genuinely autonomous behavior. Almost by default, as soon as it is indicated to learners that their autonomy may be assessed, it may be impossible to do so authentically.
This raises a further question about universities' espoused aims of developing autonomous learners. If lecturers cannot assess genuine autonomy, then should universities claim that it is being developed in their students?

Conclusion
It is evident that whilst the development of learner autonomy is seen to be a key aspiration of higher education, the term is not at all clearly defined. Teaching staff holds different positions and understandings of what autonomy is. Whilst this does not cause any problems for the individual learner engaged in independent, self-directed learning, it does lead to serious issues in creditbearing programs of higher education. Its assessment is highly problematic. It is the author's belief that, from a theoretical perspective, for accredited programs of higher education, until more detailed and 'workable-in-practice' definitions are available, the development of learner autonomy in some shape and form is possible, yet its assessment is not.