World Bank Externally Funded Output on Evidence-Based Tertiary Education Policies for Better Employment in Belarus (P168224) A Guide to Writing and Using Learning Outcomes for Higher Education Staff in Belarus Minsk, 2019 Table of Contents EXECUTIVE SUMMARY 8 1 THE BOLOGNA PROCESS FRAMEWORK FOR LEARNING OUTCOMES 10 1.1 THE BOLOGNA PROCESS 10 1.2 THE CONTRIBUTION OF LEARNING OUTCOMES TO THE BOLOGNA ACTION LINES 11 1.3 DEVELOPMENTS IN BELARUS 13 2 WHAT ARE LEARNING OUTCOMES? 17 2.1 INTRODUCTION 17 2.2 DEFINING THE TERM ‘LEARNING OUTCOMES’ 18 2.3 WHAT IS THE DIFFERENCE BETWEEN AIMS, OBJECTIVES, LEARNING OUTCOMES, AND GENERIC DESCRIPTORS? 19 3 HOW DO I WRITE LEARNING OUTCOMES? 23 3.1 INTRODUCTION 23 3.2 WRITING LEARNING OUTCOMES IN THE COGNITIVE DOMAIN 24 3.2.1 KNOWLEDGE 25 3.2.2 COMPREHENSION 26 3.2.3 APPLICATION 27 3.2.4 ANALYSIS 28 3.2.5 SYNTHESIS 30 3.2.6 EVALUATION 31 3.3 WRITING LEARNING OUTCOMES IN THE AFFECTIVE DOMAIN 32 3.4 WRITING LEARNING OUTCOMES IN THE PSYCHOMOTOR DOMAIN 33 3.5 GENERAL GUIDELINES FOR WRITING LEARNING OUTCOMES 37 3.6 PROGRAMME LEARNING OUTCOMES 43 3.7 HOW LEARNING OUTCOMES ARE FORMULATED TODAY IN BELARUS 48 2 4 COMPETENCES AND LEARNING OUTCOMES 50 4.1 COMPETENCE – ATTEMPTS TO DEFINE IT 50 4.2 COMPETENCE – INCLUSIVE OF THE BROAD AND NARROW VIEW 53 4.3 COMPETENCE AND COMPETENCY 55 4.4 THE RELATIONSHIP AMONG COMPETENCES, OBJECTIVES, AND LEARNING OUTCOMES 56 4.5 COMPETENCE WITHIN A SPECIFIC PROFESSION 57 4.6 RECOMMENDATIONS ON THE USE OF COMPETENCES 58 5 LINKING LEARNING OUTCOMES TO TEACHING AND LEARNING ACTIVITIES AND TO ASSESSMENT 60 5.1 INTRODUCTION 60 5.2 LINKING LEARNING OUTCOMES, TEACHING, AND ASSESSMENT 63 5.3 ASSESSMENT CRITERIA AND LEARNING OUTCOMES 69 6 LOOKING TOWARDS THE FUTURE WITH LEARNING OUTCOMES 71 6.1 INTRODUCTION 71 6.2 ADVANTAGES OF LEARNING OUTCOMES 71 6.3 CHALLENGES FOR BELARUS 74 6.4 POTENTIAL PROBLEMS WITH LEARNING OUTCOMES 74 6.5 CONCLUDING POINTS 75 APPENDIX 1. GLOSSARY 76 APPENDIX 2. EXAMPLES OF LEARNING OUTCOMES FROM SELECTED MODULES IN UNIVERSITY COLLEGE CORK 79 REFERENCES 88 WEBSITES 93 3 FIGURES FIGURE 1. TEACHER-CENTRED APPROACHES PLACE THE EMPHASIS ON THE TEACHER; OUTCOME-BASED APPROACHES PLACE THE EMPHASIS ON THE STUDENT ........................................................................................................................................ 18 FIGURE 2. BENJAMIN BLOOM (1913–99) ..................................................................................................................... 23 FIGURE 3. BLOOM’S LEVELS OF THINKING ...................................................................................................................... 24 FIGURE 4. SOME ACTION VERBS USED TO TEST KNOWLEDGE ............................................................................................ 25 FIGURE 5. SOME ACTION VERBS USED TO ASSESS COMPREHENSION .................................................................................. 26 FIGURE 6. SOME ACTION VERBS USED TO ASSESS APPLICATION ........................................................................................ 27 FIGURE 7. SOME ACTION VERBS USED TO ASSESS ANALYSIS.............................................................................................. 29 FIGURE 8. SOME ACTION VERBS USED TO ASSESS SYNTHESIS ............................................................................................ 30 FIGURE 9. SOME ACTION VERBS USED TO ASSESS EVALUATION ......................................................................................... 31 FIGURE 10. THE AFFECTIVE DOMAIN AND SOME ACTION VERBS USED IN WRITING LEARNING OUTCOMES IN THE AFFECTIVE DOMAIN ................................................................................................................................................................... 33 FIGURE 11. TAXONOMY DEVELOPED FOR THE PSYCHOMOTOR DOMAIN AND SOME ACTION VERBS USED IN WRITING LEARNING OUTCOMES IN THE PSYCHOMOTOR DOMAIN .................................................................................................................. 35 FIGURE 12. THE OVERLAPPING DOMAINS OF BLOOM’S TAXONOMY ................................................................................... 37 FIGURE 13. HOW LEARNING OUTCOMES ARE DEVELOPED IN BELARUS ............................................................................... 48 FIGURE 14. AN EXAMPLE SHOWING THE RELATIONSHIP AMONG COMPETENCE, OBJECTIVES, AND LEARNING ........................... 57 FIGURE 15. DIFFERENT REPRESENTATIONS OF TEACHER AND STUDENT PERSPECTIVES ........................................................... 61 FIGURE 16. FLOWCHART SUMMARISING THE STEPS INVOLVED IN THE DEVELOPMENT AND REFINING OF .................................. 62 FIGURE 17. CONSTRUCTIVE ALIGNMENT ........................................................................................................................ 68 FIGURE 18. CONSTRUCTIVE ALIGNMENT ........................................................................................................................ 69 FIGURE 19. A THREE-CIRCLE MODEL FOR OUTCOME-BASED EDUCATION IN MEDICINE ......................................................... 72 TABLES TABLE 1. EXAMPLES OF VERBS USED IN WRITING AIMS AND LEARNING OUTCOMES ............................................................. 39 TABLE 2. MATRIX INDICATING WHERE THE PROGRAMME LEARNING OUTCOMES ARE COVERED IN THE VARIOUS MODULES AND WHERE THE MODULE LEARNING OUTCOMES MAP ONTO THE PROGRAMME LEARNING OUTCOMES ........................................ 47 TABLE 3. ASSESSMENT TASKS AND THE DIFFERENT KINDS OF LEARNING ASSESSED ................................................................ 64 TABLE 4. LINKING LEARNING OUTCOMES, TEACHING AND LEARNING ACTIVITIES, AND ASSESSMENT ........................................ 66 TABLE 5. LINKING LEARNING OUTCOMES, TEACHING AND LEARNING ACTIVITIES, AND........................................................... 67 TABLE 6. RUBRIC FOR EVALUATION OF STUDENT PRESENTATION FOR SCIENCE PROJECT......................................................... 70 BOXES BOX 1. LEARNING OUTCOMES FOR A MODULE IN RESTORATIVE DENTISTRY ......................................................................... 40 BOX 2. LEARNING OUTCOMES FOR A MODULE IN ECONOMICS .......................................................................................... 40 BOX 3. CHECKLIST FOR WRITING LEARNING OUTCOMES ................................................................................................... 41 BOX 4. EXAMPLE OF DEVELOPMENT OF KEY LEARNING OUTCOMES .................................................................................... 41 BOX 5. EXAMPLE OF PROGRAMME LEARNING OUTCOMES FOR A POSTGRADUATE COMPUTER SCIENCE DEGREE ........................ 44 BOX 6. EXAMPLE OF PROGRAMME LEARNING OUTCOMES FOR AN UNDERGRADUATE ENGINEERING DEGREE ............................ 44 BOX 7. EXAMPLE OF PROGRAMME LEARNING OUTCOMES FOR AN UNDERGRADUATE SCIENCE EDUCATION DEGREE .................. 45 BOX 8. GENERIC COMPETENCES (TUNING PROJECT) ........................................................................................................ 52 BOX 9. EXAMPLES OF COMPETENCES EXPECTED OF NEWLY QUALIFIED TEACHERS IN ENGLAND AND WALES ............................. 54 4 Introduction This handbook is an outcome of a World Bank Externally Funded Output project financed by the British Embassy Minsk supporting evidence-based tertiary education policies for better employment in Belarus. The work under the Externally Funded Output addresses (a) approaches towards tracking graduates as a tool to better understand the relevance of higher education for graduates and their performance in the labour market, and (b) the implementation of learning outcomes in higher education as a means for improving the quality of provision and ensuring clarity in the description of courses. This document forms part of the second component and provides practical guidance on how to write and use learning outcomes in the form of a handbook for academic and administration staff of Belarusian universities. The contents of this handbook are based on previous work developed for the staff working in the higher education system of Ireland.1 Whilst the fundamental guidelines for assisting universities to adopt a learning outcomes framework in teaching, learning, and assessment are the same throughout the world, this handbook has been specifically designed for the higher education system of Belarus. It therefore addresses Belarusian national challenges and takes into account national particularities. The specific areas important for the Belarusian higher education system are: • A section explaining the relationship between competences and learning outcomes, as currently, the Belarusian higher education system uses a competence approach in the development of higher education programmes and their outcomes. This section has been informed by the feedback received from the meetings and workshops on learning outcomes that took place in Minsk in 2018–19.2 • More extensive examples of learning outcomes have been included throughout the handbook that are in keeping with the needs analysis, as the staff of Belarusian universities are not as familiar with the learning outcomes model as their European colleagues, who have been implementing it for more than a decade. • An extended and more detailed section on the writing of programme learning outcomes has been included in response to the requests of programme directors working at Belarusian universities for assistance in this particular area. • Care has been taken throughout the handbook to explain key educational terms that are commonly used when bringing about innovations in higher education systems, such as aims, objectives, learning outcomes, and generic descriptors. This handbook is aimed at strengthening the professional competences of teachers and those working in higher education institutions for designing and diagnosing learning outcomes through the learning process in the 1 Kennedy, Declan. 2007. Writing and using learning outcomes: a practical guide . Cork: University College Cork. https://cora.ucc.ie/handle/10468/1613. 2 The World Bank team – including Dr. Declan Kennedy, who is the main author of this handbook – thanks all interviewees and workshop participants for their contributions, as well as the representatives of the Ministry of Education of the Republic of Belarus, the Republican Institute for Higher Education (RIVSh), and Belarusian universities, for their generous support. 5 course of the implementation of higher education programs. It is also intended to assist teachers and administrative staff working in the higher education system of the Republic of Belarus in addressing challenges when writing modules and programs in terms of learning outcomes and adopting learning outcomes in Belarus’s higher education sector. The handbook can be used for upgrading the qualifications of teachers and those working in higher education institutions, other participants in the education process in the higher education system, and additional education for adults. 6 Abbreviations ECTS European Credit Transfer and Accumulation System EHEA European Higher Education Area EU European Union 7 Executive Summary The overall aim of the Bologna Declaration (1999) is to improve the efficiency and effectiveness of higher education in Europe. One of the main features of this process is to improve the traditional ways of describing qualifications and qualification structures. As a step towards achieving greater clarity in the description of qualifications, all modules and programmes in higher education institutions throughout the European Higher Education Area (EHEA) are now written in terms of learning outcomes. International trends in education show a shift from the traditional teacher-centred approach to a student- centred approach, that is, the focus is not only on teaching but also on what the students are expected to be able to do at the end of the module or programme. Statements called learning outcomes are used to express what the students are expected to achieve and how they are expected to demonstrate that achievement. Learning outcomes are defined as statements of what a student is expected to know, understand, and be able to demonstrate after completion of a process of learning. The Republic of Belarus joined the Bologna process in 2015 and is now transforming its higher education system in order to (a) raise the quality and competitiveness of its national higher education system, and (b) implement formal recommendations for the European Higher Education Area countries. On June 1, 2018, the Minister of Education of the Republic of Belarus approved the strategic plan of activities for the implementation of the main objectives for the Belarusian education system development aligned with the principles and mechanisms of the European Higher Education Area, which is expected to facilitate this work. In parallel with the introduction of the European Credit Transfer System (ECTS), the National Qualification Framework, and the two-cycle degree structure (Bachelor and Master), the approach of teaching and learning in Belarusian higher education institutions should also be changed from a teacher-centred to a student-centred approach. The use of learning outcomes for this purpose is the best way to achieve quick and sustainable results. When writing learning outcomes, it is helpful to make use of Bloom’s Taxonomy of Educational Objectives (Bloom 1975). This classification or categorisation of levels of thinking behaviour provides a ready-made structure and list of action verbs to assist in writing learning outcomes. Most learning outcomes describe evidence of learning in areas such as knowledge, comprehension, application, analysis, synthesis, and evaluation. This area is known as the cognitive domain. The other two main domains are the affective domain (attitudes, feelings, values) and the psychomotor domain (physical skills). In general, when writing learning outcomes, begin with an action verb followed by the object of that verb. This handbook contains a list of action verbs for each area of Bloom’s Taxonomy. Sentences should be kept short to ensure clarity. Learning outcomes must be capable of being assessed. When deciding on the number of learning outcomes to write, the general recommendation in the literature is about six learning outcomes per module. The most common mistake in writing learning outcomes is to use vague terms such as know, understand, learn, be familiar with, be exposed to, be acquainted with, and be aware of. It is important to link learning outcomes to teaching and learning activities and assessment. This may be done with the aid of a grid to assist in checking that the learning outcomes map onto the teaching and learning activities as well as to the mode of assessment. 8 The advantages of learning outcomes for teachers and students are well documented in the literature. In addition, learning outcomes assist greatly in the more systematic design of programmes and modules, and represent the next development stage of the methodology of designing higher education programs relative to a competence-based approach. 9 1 The Bologna Process Framework for Learning Outcomes ‘Learning outcomes are important for recognition…. The principal question asked of the student or the graduate will therefore no longer be “what did you do to obtain your degree?” but rather “what can you do now that you have obtained your degree?”. This approach is of relevance to the labour market and is certainly more flexible when taking into account issues of lifelong learning, non-traditional learning, and other forms of non-formal educational experiences.’ – Purser, Council of Europe 2002, p. 5 1.1 The Bologna Process In June 1999, representatives of the Ministers of Education of the European Union (EU) member states convened in Bologna, Italy, to formulate the Bologna Declaration, which led to the establishment of a common European Higher Education Area (EHEA). The overall aim of the Bologna Process is to improve the efficiency and effectiveness of higher education in Europe. The agreement is designed so that the independence and autonomy of the universities and other third-level institutions would ensure that higher education and research in Europe adapt to the changing needs of society and the advances in scientific knowledge3. Some of the key points arising from the Bologna Declaration and subsequent meetings to ensure the success of the various components of the entire Bologna Process may be summarised as follows: • The EHEA will ensure the increased international competitiveness of the European system of higher education. • The traditional ways of describing qualifications and qualification structures need to be improved and made more transparent. A system of easily readable and comparable degrees is being adopted. • Every student graduating will receive a Diploma Supplement automatically and free of charge in a widely spoken European language. This supplement, the purpose of which is to improve transparency and facilitate recognition, describes the qualification the student has received in a standard format that is easy to understand and compare. It also describes the content of the qualification and the structure of the higher education system within which it was issued. • The system of degrees will comprise two main pre-doctoral cycles – the first cycle lasting a minimum of three years (a minimum of 180 ECTS credits), and the second cycle leading to the master’s degree (90 to 120 ECTS credits). The third cycle refers to the doctoral level in the Bologna Process, without specification of the number of ECTS credits. 3 http://www.bologna.ie. 10 • The introduction of a transferable system of academic credits will assist in the promotion of mobility within the EHEA by overcoming legal recognitions and administrative obstacles. • The transferable system of academic credits assists in promoting European cooperation in quality assurance. • The position of higher education institutions and students as essential partners in the Bologna Process is confirmed. • The European dimension in higher education will be promoted through interinstitutional cooperation, curricula, and mobility schemes for students, teachers, and researchers. Several follow-up meetings of Ministers of Education were held after the meeting in Bologna to move the process forward. For example, at the follow-up meeting that took place in London in 2007, the following points were made in the communiqué issued: • ‘We underline the importance of curricula reform leading to qualifications better suited both to the needs of the labour market and to further study. Efforts should concentrate in future on removing barriers to access and progression between cycles and on proper implementation of ECTS based on learning outcomes and student workload’. • ‘Qualifications frameworks are important instruments in achieving comparability and transparency within the EHEA and facilitating the movement of learners within, as well as between, higher education systems. They should also help HEIs to develop modules and study programmes based on learning outcomes and credits, and improve the recognition of qualifications as well as all forms of prior learning’. • ‘We urge institutions to further develop partnerships and cooperation with employers in the ongoing process of curriculum innovation based on learning outcomes’. • ‘With a view to the development of more student-centred, outcome-based learning, the next [Stocktaking] exercise should also address in an integrated way national qualifications frameworks, learning outcomes and credits, lifelong learning, and the recognition of prior learning’ (London Communiqué 2007, p. 7). The Republic of Belarus joined the Bologna process in 2015 and is now transforming its higher education system in order to (a) raise the quality and competitiveness of its national higher education system, and (b) implement formal recommendations for the European Higher Education Area countries. 1.2 The Contribution of Learning Outcomes to the Bologna Action Lines The Bologna Process specifies a number of ‘action lines’ in which learning outcomes play an important role (Adam 2004). The main action lines may be summarised as follows: • Adoption of a system of easily readable and comparable degrees. The use of learning outcomes as a type of common language for describing qualifications helps make these qualifications clearer to other institutions, employers, and those involved in evaluating qualifications. 11 • Promotion of mobility. Since learning outcomes help make qualifications more transparent, this facilitates student exchanges as the process of recognition of study carried out in other institutions will be made straightforward. • Establishment of a system of credits. The European Credit Transfer System (ECTS) had developed from simply being a system for recognising study at foreign institutions into a Credit Transfer and Accumulation System that takes all learning into account – not just study in other countries. The ECTS system is based on the principle that 60 credits measure the workload of a full-time student during one academic year. The position of learning outcomes in the credit system is clearly stated in the ECTS Users’ Guide (2005, p. 4): ‘Credits in ECTS can only be obtained after successful completion of the work required and appropriate assessment of the learning outcomes achieved’. Adam (2004, p. 19) summarises the situation well when he says: ‘Credits expressed in terms of learning outcomes are a powerful way to recognise and quantify learning achievement from different contexts; they also provide an effective structure for relating qualifications. The addition of the learning outcomes dimension has the potential to improve dramatically the effectiveness of ECTS as a true pan-European system’. • Promotion of cooperation in quality assurance. The use of learning outcomes as a common method for describing programmes and modules has the potential to assist in the establishment of common standards and common methods of quality assurance among institutions. It is hoped that the increased confidence in the area of quality assurance among institutions will assist in the creation of the European Higher Education Area. • Promotion of the European dimension in higher education. Since programmes will be expressed using the common terminology of learning outcomes, this greatly simplifies the development of joint degree programmes and integrated study programmes. • Lifelong learning. The use of a credit-based system linked to learning outcomes has the potential to create a flexible and integrated system to assist people of all ages to gain educational qualifications. Without the introduction of learning outcomes, the system of lifelong learning in many countries will remain complicated and disjointed. • Higher education and students. The use of learning outcomes when describing programmes and modules makes it clear to students what they are expected to achieve by the end of the programme or module. This also assists students in the choice of programmes and in actively participating in student- centred learning. Considerable progress is being made in the Bologna Process as highlighted by the European Centre for Vocational Training (CEDEFOP 2009): ‘The potential and widespread significance of learning outcomes is only just beginning to be realised. Their introduction is designed to facilitate the fundamental reform of existing qualifications and the creation of new ones fit for the 21st century. It is arguable that the main end product of the Bologna reforms is better qualifications based on learning outcomes and not just new educational structures’. (p. 82) 12 However, the Bucharest Communiqué (2012, p. 1) has summarised the challenges that lay ahead in implementing a learning outcomes framework: ‘However, as the report on the implementation of the Bologna Process shows, we must make further efforts to consolidate and build on progress. We will strive for more coherence between our policies, especially in completing the transition to the three cycle system, the use of ECTS credits, the issuing of Diploma Supplements, the enhancement of quality assurance and the implementation of qualifications frameworks, including the definition and evaluation of learning outcomes’. 1.3 Developments in Belarus In the Republic of Belarus, education standards establish mandatory minimum requirements for the organization of the education process and requirements for the content of higher education and shape a model for training a specialist with higher education in each specialty of higher education (Stage I and master’s degree). The first generation of Belarusian education standards was introduced in 1998 (previously, the learning process was based solely on education plans and programs). They had been developed based on the experience of the Union of Soviet Socialist Republics (USSR) higher education system and the experience of education standards development from other countries. The first generation of education standards provided only general requirements with regard to the knowledge and skills of a specialist. The second generation of higher education standards was introduced in 2007–2008, and built on experience gained with the first generation of education standards and respective achievements in other countries. As a result of the transition to a multistage system of higher education starting from 2007 (within the framework of the program of transition to differentiated periods of training specialists with higher education in the Republic of Belarus for 2005–2010), this generation of education standards defined the content of education at Stage 1 of higher education. In developing the second generation of education standards, it was expected to implement a competence-based approach, introduce a credit transfer system, and shorten the duration of higher education programs for most specialties. The second generation of education standards introduced requirements for the content of competences (based on the ‘Tuning Educational Structures’ project and the Dublin descriptors [see p. 21]): academic, sociopersonal, and professional. Thus, starting from the second generation of education standards, the competence-based approach was reflected only at the level of the competence and qualification characteristics of a specialist through identification and definition of three interrelated groups of competences. The third generation of education standards for Stage I of higher education was introduced in 2013. Considerable progress was made in introducing a credit transfer system and shortening the duration of education programs at Stage I of higher education. Codification of academic, sociopersonal, and professional competences was introduced. Considerable attention was given to competence identification general requirements for identification were formulated and identification forms were designed (variable requirements for the forms and methods of competence identification), and it was expected to set up and develop evaluation tools base. 13 The updated education standards did not give employers the additional opportunity for a comprehensive evaluation of the competences of a graduate because the standards reflected only the competencies gained through mastering the disciplines of the state component of the education plan. At the same time, a considerable part of professional competencies was built by the disciplines of the variable component of a higher education institution. Therefore, one of the key goals expected to be addressed through the third generation of education standards was to enhance the practical training of graduates. Belarus is now developing a new generation of education standards for bachelor’s and master’s degrees (generation 3+). Education under the new standards will be organized within the new model of higher education presented in the updated draft of the Education Code. In the system of higher education of the Republic of Belarus, learning outcomes are used in describing the content of higher education programs since 2007. The referred period (2007–2018) can be tentatively divided into 3 stages: • Stage 1: 2007–2012. Learning outcomes across the entire content of the education program expressed in the form of competencies were articulated only in education standards. • Stage 2: 2013–2017. Learning outcomes across the entire content of the education program expressed in the form of competencies were articulated in education standards and programs for academic disciplines. However, learning outcomes, as a rule, were not linked to a specific discipline or a group of disciplines (module). • Stage 3: 2018 onwards. Learning outcomes are articulated in education standards, education plans and programs for academic disciplines. Learning outcomes, as a rule, are linked to a specific discipline or a group of disciplines (modules). The concept of new approaches to the organization of the education process was developed during the enlarged session of the Republican Council of Rectors on October 9–10, 2014, and reflected in the draft of the new Education Code. New approaches to the organization of the education process provide for: • Introduction of a module approach in the organization of the education process based on the grouping of subjects and disciplines into modules ensuring that students gain one or several similar competences • Introduction of a profile specialty as a variable part (up to 50 percent of the total education period) of the education content, which takes account of the specifics of future professional work of a specialist, and is designed and approved independently by each higher education institution • Broader choice for a student of academic disciplines (academic disciplines at the choice of an undergraduate or a master student of up to 50 percent of total academic hours dedicated for a component of a higher education institution) • Use of ECTS-compatible credits as the basic measurement of learning outcomes of a student • Use of credits in organizing resumption of studies by a student/transfer to another education institution, another specialty or form of education, and in organizing academic mobility • Offering an opportunity for credit accumulation based on the outcomes of completion of certain courses in various higher education institutions under the network learning 14 • Establishment of the requirements for the outcomes of mastering the content of the education program in higher education standards for each specialty (requirements for the outcomes of mastering the content of the respective education program of higher education include the competences gained by students) • Introduction of distance learning as an independent form of learning • Introduction of network learning allowing higher education institutions together with other education institutions (including foreign institutions), research organizations, and enterprises to arrange joint training of specialists with higher education both within the joint degree programs and formation of education clusters • Review of the approach to the formation of the state component in the content of education: the education standard in the structure of the education plan lists the mandatory disciplines (state component disciplines) and total credits allocated to the state component. An education institution has the discretion to determine the number of academic hours for a concrete discipline (module) of the state component • Transition from ‘standard’ to ‘model’ education documents serving as a template for education institutions to develop the content of the education process, the requirements for which are set forth in the education standard. Prior to the entry into force of the new draft Education Code, higher education institutions worked on the design of new education content for bachelor’s and master’s programs (the education standards of 3+ generation update the competence-based approach through a clarified set of competencies and through the design of learning outcomes). The outputs in the form of the draft education standards of new generation ‘3+’ are published for public review at http://www.edustandart.by/. The developed layouts of the education standards for bachelor’s and master’s programs highlight the need for the formulation of competences presented in the following groups: universal, basic (advanced) professional, and specialized. All universal, basic (advanced) professional competencies are included in the set of the required outcomes of mastering the content of the bachelor’s (master’s) program. The list of specialized competences of a bachelor (or master) is compiled by the education institution itself, taking into account the profiling of the content of the education program. Specialized competences are established based on labor market needs, overview of foreign experience, consultations with leading employers, associations of employers in the respective sector, and other sources. The developed layouts of the education standards for bachelor’s and master’s programs also require education institutions to independently design the learning outcomes for academic disciplines (modules) of the education institution’s component, practical training, and master’s thesis, and allow them to clarify and extend learning outcomes for academic disciplines (modules) of the state component set forth in the education standard. At the same time, learning outcomes should be correlated with the required outcomes of mastering the content of the education program (competences). A set of the expected learning outcomes should enable a graduate to gain all universal and basic (advanced) professional competencies set forth in the education standard, as well as all specialized competencies defined by the education institution. 15 Module and discipline learning outcomes for the state component (to know, to be able to, to grasp) will be determined in the standardized education programs for academic disciplines (modules); for the variable component disciplines - in the education programs for academic disciplines by HEIs directly. Therefore, an approach has evolved in Belarus’s higher education system under which learning outcomes are designed from ‘the bottom’. This process requires correlation with concrete competences set forth in the education standards. 16 2 What are Learning Outcomes? ‘Learning outcomes represent one of the essential building blocks for transparent higher education systems and qualifications’. – Adam 2004, p. 3 2.1 Introduction The traditional way of designing modules4 and programmes5 was to start from the content of the course. Teachers decided on the content that they intended to teach in the programme, planned how to teach this content, and then assessed the content. This type of approach focussed on the teacher’s input and on the assessment in terms of how well the students absorbed the material. Course descriptions referred mainly to the content of the course that would be covered in lectures. This approach to teaching is commonly referred to as a teacher-centred approach (Figure 1). Among the criticisms of this type of approach in the literature (Gosling and Moon 2001) is that it can be difficult to state precisely what the student must be able to do to pass the module or programme. International trends in education show a shift from the traditional ‘teacher-centred’ approach to a ‘student- centred’ approach. This alternative model focuses on what the students are expected to be able to do at the end of the module or programme. Hence, this approach is commonly referred to as an outcome-based approach (Figure 1). Statements called intended learning outcomes, commonly shortened to learning outcomes, are used to express what students are expected to be able to do at the end of the learning period. The term learning outcome will be defined more precisely in section 2.2. 4 ‘Module’ refers to a one-semester course within one subject. 5 ‘Programme’ refers to Bachelor or Master programmes for their entire duration. 17 Figure 1. Teacher-Centred Approaches Place the Emphasis on the Teacher; Outcome-Based Approaches Place the Emphasis on the Student The outcome-based approach can be traced to the work of the behavioural objectives movement of the 1960s and 1970s in the United States. Among the best-known advocates of this type of teaching was Robert Mager, who proposed the idea of writing precise statements, which he called instructional objectives, about observable outcomes (Mager 1975). Using these instructional objectives and performance outcomes, he attempted to define the type of learning that would occur at the conclusion of instruction and how that learning would be assessed. These instructional objectives later developed into more precisely defined learning outcomes (section 2.2). Gosling and Moon (2001) state that the outcomes-based approach to teaching is becoming increasingly popular at an international level: ‘The outcome-based approach has been increasingly adopted within credit frameworks and by national quality and qualifications authorities such as the QAA (Quality Assurance Agency for Higher Education) in the UK, the Australia, New Zealand and South African Qualification Authorities’. (p. 7) With the implementation of the Bologna Process by 2010, all modules and programmes throughout most participating countries have started to be expressed using the outcomes-based approach, that is, in terms of learning outcomes. Some countries still use the competence-based approach (including Belarus), but the transition to the 3+ generation of higher education standards in Belarus already prepared universities for the introduction of learning outcomes. Some particularities of this approach need specific attention, however, which this handbook describes in detail. 2.2 Defining the Term ‘Learning Outcomes’ A survey of selected learning outcomes literature reveals several similar definitions: • Learning Outcomes are specific statements of what students should know and be able to do as a result of learning (Morss and Murray 2005, p. 8). 18 • Learning outcomes are statements of what is expected that a student will be able to DO as a result of a learning activity (Jenkins and Unwin 2001, p. 1). • A learning outcome is ‘a statement of what a learner knows, understands and is able to do on completion of a learning process’ (European Qualifications Framework, p. 3). • Learning outcomes are explicit statements of what we want our students to know, understand or to be able to do as a result of completing our courses (University New South Wales, Australia, p. 7). • ‘Learning outcomes are statements that specify what learners will know or be able to do as a result of a learning activity. Outcomes are usually expressed as knowledge, skills or attitudes’ (American Association of Law Libraries, p. 1). • Learning outcomes are an explicit description of what a learner should know, understand and be able to do as a result of learning (Learning and Teaching Institute, Sheffield Hallam University, p. 2). • ‘A learning outcome is a written statement of what the successful student/learner is expected to be able to do at the end of the module/course unit or qualification’ (Adam 2004, p. 5). Thus, we can see that the various definitions of learning outcomes do not differ significantly from each other. From the various definitions it is clear that: • Learning outcomes focus on what the student has achieved rather than merely focussing on the content of what has been taught. • Learning outcomes focus on what the student can demonstrate at the end of a learning activity. A good working definition of a learning outcome (ECTS Users’ Guide 2015, p. 22) is: Learning outcomes are statements of what a student is expected to know, understand, and be able to demonstrate after completion of a process of learning. The learning activity could be, for example, a lecture, a module (short course), a workshop, or an entire programme. Whilst it is common for teachers to plan learning outcomes for individual lessons or lectures, the emphasis in this handbook will be on writing learning outcomes for modules and programmes, as this is what is required by the Bologna Process. Learning outcomes must be simply and clearly described and must be capable of being validly assessed. 2.3 What is the Difference Between Aims, Objectives, Learning Outcomes, and Generic Descriptors? In writing the programme or module description, a common mistake is to confuse aims, objectives, and learning outcomes. To avoid this, it is important to remember the simple principles and examples provided below. 19 The aim of a module or programme is a broad general statement of teaching intention, that is, it indicates what the teacher intends to cover in a block of learning. Aims are usually written from the teacher’s point of view to indicate the general content and direction of the module. Examples of aims include: • To introduce students to the basic principles of atomic structure • To provide a general introduction to the history of Ireland in the 20th century • To give students an introduction to current theory and practice in the area of science education. • To give students an understanding of what constitutes good science teaching • To give students an appreciation of the contribution that science education can make to the overall education of young people • To help students develop the knowledge and professional skills to teach science in the secondary school • To give students a critical understanding of current debates and issues relating to science education • To provide students with the opportunity to develop their critical thinking skills to enable them to engage in highly effective science teaching in schools • To assist students to develop as reflective practitioners with an understanding of research methods in education and how these can inform practice in the classroom. The objective of a module or programme is usually a specific statement of teaching intention, that is, it indicates one of the specific areas that the teacher intends to cover in a block of learning. Examples of objectives include: • To give students an appreciation of the unique nature of carbon and its ability to bond to other carbon atoms • To give students an understanding of the concept of hybridisation • To ensure that students know some characteristic properties of alkanes and alcohols • To familiarize students with a range of families of organic compounds: alkanes, alcohols, carboxylic acids, and esters. Aims are general and long term and refer to a series of lectures or a unit of work (module). Objectives are more specific and short term. Thus, the aim of a module gives the broad purpose or general teaching intention of the module whilst the objective gives more specific information about what the teaching of the module hopes to achieve. The following phrases are commonly used in writing aims and objectives: • To give students an understanding of… • To give students an appreciation of… 20 • To familiarize students with… • To ensure that students know… • To enable students to experience… • To encourage students to… • To provide students with the opportunity to…. One problem caused by the use of objectives is that sometimes they are written in terms of teaching intention and other times they are written in terms of expected learning; that is, there is confusion in the literature in terms of whether objectives belong to the teacher-centred approach or the outcome-based approach. The situation is summarised by Moon (2002) as follows: ‘Basically the term “objective” tends to complicate the situation, because objectives may be written in the terms of teaching intention or expected learning…. This means that some descriptions are of the teaching in the module and some are of the learning…. This general lack of agreement as to the format of objectives is a complication, and justifies the abandonment of the use of the term “objective” in the description of modules or programmes’. (p. 62) Most teachers who have worked on the development of objectives for modules or programmes would have encountered the above problem. One of the great advantages of learning outcomes is that they are clear statements of what the student is expected to achieve and how he or she is expected to demonstrate that achievement. Thus, learning outcomes are more precise, easier to compose, and far clearer than objectives. From one perspective, learning outcomes can be considered as a sort of ‘common currency’ that assists modules and programmes to be more transparent at both the local and international level. The many advantages of learning outcomes will be discussed in more detail in Chapter 5. The Framework of Qualifications for the European Higher Education Area (EHEA) is commonly called the ‘Bologna Framework’. The Conference of European Ministers Responsible for Higher Education in Bergen, Norway (2005) adopted the overarching framework for qualifications in the EHEA. This framework contains three cycles: bachelor, master, and PhD (including within national contexts, the possibility of intermediate qualifications). Statements called generic descriptors for each cycle, drawn up at a meeting of Education Ministers in Dublin and thus commonly called the Dublin descriptors, are listed for each cycle. Generic descriptors are broad general statements that can be applied to every subject area, in any particular cycle in any higher education institution within the EHEA. Examples of generic descriptors include: • Demonstrate knowledge and understanding that builds upon their general secondary education • Can apply their knowledge and understanding in a manner that indicates a professional approach to their work • Have the ability to gather and interpret relevant data to inform judgements that include reflection on relevant social scientific or ethical issues. 21 The Belarusian National Qualifications Framework is currently under development, but a draft, prepared by the Ministry of Labour and Social Protection of Belarus, conforms with European requirements. The importance of learning outcomes was further highlighted in the European Qualifications Framework for Lifelong Learning by the European Union (EU) Commission, which recommended that member sta tes ‘Use an approach based on learning outcomes when defining and describing qualifications, and promote the validation of non-formal and informal learning … paying particular attention to those citizens most likely to be subject to unemployment or insecure forms of employment, for whom such an approach could help increase participation in lifelong learning and access to the labour market’ (EU Commission 2008, p. 10). Thus, learning outcomes written for programmes must map onto the generic descriptors of the Bologna Framework. The mechanism for doing this will be discussed in the next chapter. 22 3 How Do I Write Learning Outcomes? ‘In outcome-based education the educational outcomes are clearly and unambiguously specified. These determine the curriculum content and its organisation, the teaching methods and strategies, the courses offered, the assessment process, the educational environment and the curriculum timetable. They also provide a framework for curriculum evaluation’. – Harden, Crosby, and Davis 1999a, p. 8 3.1 Introduction The task of writing learning outcomes has been made considerably easier due to the work of Benjamin Bloom (1913–99) (see Figure 2). Bloom obtained bachelor’s and master’s degrees from Pennsylvania State University in the United States. He then worked with the famous educationalist Ralph Tyler at the University of Chicago, from which Bloom earned a PhD in Education in 1942. Figure 2. Benjamin Bloom (1913–99) Bloom, a gifted teacher, was particularly interested in the thought processes of students when they were interacting with what was being taught. He carried out research in the development of the classification of levels of thinking during the learning process. He believed that learning was a process and that it was the job of teachers to design lessons and tasks to help students meet the established objectives. Bloom’s most famous contribution to education was categorising the levels of these thinking behaviours into six increasingly complex levels, from the simple recall of facts at the lowest level up to the process of evaluation at the highest level (Figure 3). His publication, Taxonomy of Educational Objectives: Handbook 1, the Cognitive Domain (Bloom et al. 1956) is widely used throughout the world to assist in the preparation of evaluation materials. (The term taxonomy implies a classification, categorisation, or arrangement). The taxonomy describes how we build upon our former learning to develop more complex levels of understanding. Many teachers have used Bloom’s Taxonomy because of the structure it provides in areas like learning assessment. 23 Figure 3. Bloom’s Levels of Thinking 6. Evaluation 5. Synthesis 4. Analysis 3. Application 2. Comprehension 1. Knowledge Source: Author based on Bloom et al. (1956). In recent years, attempts have been made to revise Bloom’s Taxonomy (Anderson and Krathwohl 2001; Krathwohl 2002), but the original work of Bloom and his co-workers is still the most widely quoted in the literature. Therefore, in this handbook we use Bloom’s Taxonomy rather than Anderson and Krathwohl’s Taxonomy (2001). Bloom’s taxonomy was not simply a classification scheme; it was also an effort to arrange the various thinking processes in a hierarchy. In this hierarchy, each level depends on the student’s ability to perform at the level or levels that are below it. For example, for a student to apply knowledge (stage 3), he or she would need to have both the necessary information (stage 1) and understanding of this information (stage 2). When talking about teaching, Bloom always advocated that when teaching and assessing students, teachers should bear in mind that learning is a process and that the teacher should try to get the thought processes of the students to move up into the higher-order stages of synthesis and evaluation. This ‘thinking’ area is commonly called the cognitive (‘knowing’) domain since it involves thought processes. 3.2 Writing Learning Outcomes in the Cognitive Domain Bloom’s taxonomy is frequently used for writing learning outcomes as it provides a ready-made structure and list of verbs. These verbs are the key to writing learning outcomes. Bloom’s original list of verbs was limited and 24 has been extended by various authors over the years. In this handbook, the list of verbs has been compiled from Bloom’s original publication and a study of the more modern literature in this area. The list of verbs for each stage is not exhaustive, but it is comprehensive. A glossary of terms commonly used in this handbook is presented in Appendix 1. We now consider each stage of Bloom’s taxonomy and the verbs corresponding to each stage. Because learning outcomes are concerned with what the students can do at the end of the learning activity, all of the verbs are action (active) verbs. 3.2.1 Knowledge Knowledge may be defined as the ability to recall or remember facts without necessarily understanding them. Some of the action verbs used to assess knowledge are presented in Figure 4. Figure 4. Some Action Verbs Used to Test Knowledge 6. Evaluation Arrange, collect, define, describe, duplicate, 5. Synthesis enumerate, examine, find, identify, label, list, locate, memorise, 4. Analysis name, order, outline, present, quote, recall, recognise, recollect, 3. Application record, recount, relate, repeat, reproduce, 2. Comprehension show, state, tabulate, tell. . 1. Knowledge Source: Author. Following are examples of learning outcomes that demonstrate evidence of knowledge. Note that each learning outcome begins with an action verb: • Recall genetics terminology: homozygous, heterozygous, phenotype, genotype, homologous chromosome pair, etc. • Identify and consider ethical implications of scientific investigations. • Describe how and why laws change and the consequences of such changes on society. 25 • List the criteria to be considered when caring for a patient with tuberculosis. • Define what behaviours constitute unprofessional practice in the solicitor-client relationship. • Outline the history of the Celtic peoples from the earliest evidence to the insular migrations. • Describe the processes used in engineering when preparing a design brief for a client. • Recall the axioms and laws of Boolean algebra. 3.2.2 Comprehension Comprehension may be defined as the ability to understand and interpret learned information. Some of the action verbs used to assess comprehension are shown in Figure 5. Figure 5. Some Action Verbs Used to Assess Comprehension Associate, change, clarify, classify, construct, contrast, 6. Evaluation convert, decode, defend, describe, differentiate, discriminate, discuss, 5. Synthesis distinguish, estimate, explain, express, extend, generalise, identify, 4. Analysis illustrate, indicate, infer, interpret, locate, predict, recognise, report, restate, 3. Application review, select, solve, translate. 2. Comprehension 1. Knowledge Source: Author. Some examples of learning outcomes that demonstrate evidence of comprehension are the ability to: • Differentiate between civil and criminal law. • Identify participants and goals in the development of electronic commerce. • Critique German literary texts and films in English. 26 • Predict the genotype of cells that undergo meiosis and mitosis. • Translate short passages of contemporary Italian. • Convert number systems from hexadecimal to binary and vice versa. • Explain the social, economic, and political effects of World War I on the postwar world. • Classify reactions as exothermic and endothermic. • Recognise the forces discouraging the growth of the educational system in Ireland in the 19th century. • Explain the impact of Greek and Roman culture on Western civilisation. 3.2.3 Application Application may be defined as the ability to use learned material in new situations, e.g. putting ideas and concepts to work in solving problems. Some of the action verbs used to assess application are presented in Figure 6. Figure 6. Some Action Verbs Used to Assess Application Apply, assess, calculate, change, choose, complete, compute, construct, 6. Evaluation demonstrate, design, develop, discover, dramatise, employ, 5. Synthesis examine, experiment, find, illustrate, interpret, manipulate, modify, 4. Analysis operate, organise, practice, predict, prepare, produce, relate, schedule, select, 3. Application show, sketch, solve, transfer, use. 2. Comprehension 1. Knowledge Source: Author. Examples of learning outcomes that demonstrate evidence of application are the ability to: 27 • Construct a timeline of significant events in the history of Australia in the 19th century. • Apply knowledge of infection control in the maintenance of patient care facilities. • Select and employ sophisticated techniques for analysing the efficiencies of energy use in complex industrial processes. • Show proficiency in the use of vocabulary and grammar, as well as the sounds of the language in different styles. • Relate energy changes to bond breaking and formation. • Modify guidelines in a case study of a small manufacturing firm to enable tighter quality control of production. • Show how changes in criminal law affected levels of incarceration in Scotland in the 19th century. • Apply principles of evidence-based medicine to determine clinical diagnoses. 3.2.4 Analysis Analysis may be defined as the ability to break down information into its components, for example, looking for interrelationships and ideas (understanding of organisational structure). Action verbs used to assess analysis are shown in Figure 7. 28 Figure 7. Some Action Verbs Used to Assess Analysis Analyse, appraise, arrange, break down, 6. Evaluation calculate, categorise, classify, compare, connect, contrast, 5. Synthesis criticise, debate, deduce, determine, differentiate, 4. Analysis discriminate, distinguish, divide, examine, experiment, 3. Application identify, illustrate, infer, inspect, investigate, order, 2. Comprehension outline, point out, question, recognise, relate, separate, solve, 1. Knowledge subdivide, test. Source: Author. Examples of learning outcomes that demonstrate evidence of analysis are the ability to: • Analyse why society criminalises certain behaviours. • Compare and contrast the different electronic business models. • Categorise the different areas of specialised interest within dentistry. • Debate the economic and environmental effects of energy conversion processes. • Identify and quantify sources of errors in measurements. • Calculate gradient from maps in m, km, %, and ratio. • Critically analyse a broad range of texts of different genres and from different time periods. • Compare the classroom practice of a newly qualified teacher with that of a teacher with 20 years’ teaching experience. • Calculate logical functions for coders, decoders, and multiplexers. • Recognise trends in atomic radii in the Periodic Table of the Elements. 29 3.2.5 Synthesis Synthesis may be defined as the ability to put parts together. Action verbs used to assess synthesis are presented in Figure 8. Figure 8. Some Action Verbs Used to Assess Synthesis Argue, arrange, 6. Evaluation assemble, categorise, collect, combine, compile, compose, 5. Synthesis construct, create, design, develop, devise, establish, 4. Analysis explain, formulate, generalise, generate, infer, integrate, 3. Application invent, make, manage, modify, organise, originate, 2. Comprehension plan, prepare, propose, rearrange, reconstruct, relate, reorganise, revise, 1. Knowledge rewrite, set up, summarise. Source: Author. Examples of learning outcomes that demonstrate evidence of synthesis are the ability to: • Recognise and formulate problems that are amenable to energy management solutions. • Propose solutions to complex energy management problems both verbally and in writing. • Assemble sequences of high-level evaluations in the form of a programme. • Integrate concepts of genetic processes in plants and animals. • Summarise the causes and effects of the 1917 Russian Revolution. • Relate the sign of enthalpy changes to exothermic and endothermic reactions. • Organise a patient education programme. 30 3.2.6 Evaluation Evaluation may be defined as the ability to judge the value of material for a given purpose. Action verbs used to assess evaluation are shown in Figure 9. Figure 9. Some Action Verbs Used to Assess Evaluation Appraise, argue, 6. Evaluation ascertain, assess, attach, choose, compare, conclude, 5. Synthesis contrast, convince, criticise, decide, defend, discriminate, 4. Analysis explain, evaluate, interpret, judge, justify, measure, 3. Application predict, rate, recommend, relate, resolve, revise, score, summarise, support, 2. Comprehension validate, value. 1. Knowledge Source: Author. Examples of learning outcomes that demonstrate evidence of evaluation are the ability to: • Assess the importance of key participants in bringing about change in Irish history. • Evaluate marketing strategies for different electronic business models. • Appraise the role of sport and physical education in health promotion for young people. • Predict the effect of change in temperature on the position of equilibrium in the given reaction. • Summarise the main contributions of Michael Faraday to the field of electromagnetic induction. • Assess the Arrhenius acid-base theory in the light of the Bronsted-Lowry theory of acids and bases. Note that the verbs used in the above six categories are not exclusive to any one particular category. Some verbs appear in more than one category. For example, a mathematical calculation may involve merely applying a given formula (application – stage 3) or it may involve analysis (stage 4) as well as application. A revision of some aspects of Bloom’s Taxonomy was suggested by Anderson and Krathwohl in 2001 in which they placed Creation at the top of their taxonomy with Evaluation and Analysis lower down. There is often 31 confusion between Bloom’s Taxonomy and Anderson and Krathwohl’s Taxonomy since internet searches often show Anderson and Krathwohl’s Taxonomy instead of Bloom’s. However, in the research literature Bloom’s Taxonomy is the taxonomy most frequently quoted, and hence is the one used in this handbook. 3.3 Writing Learning Outcomes in the Affective Domain Whilst the cognitive domain is the most widely used of Bloom’s Taxonomy, Bloom and his co -workers also carried out research on the affective (‘attitudes’, ‘feelings’, ‘values’) domain (Bloom, Masia, and Krathwohl 1964). This domain is concerned with issues relating to the emotional component of learning and ranges from basic willingness to receive information to the integration of beliefs, ideas, and attitudes. To describe the way in which we deal with things emotionally, Bloom and his colleagues developed five major categories: 1. Receiving. This refers to a willingness to receive information, for example, the individual accepts the need for a commitment to service, listens to others with respect, shows sensitivity to social problems, etc. 2. Responding. This refers to the individual actively participating in his or her own learning, for example, shows interest in the subject, is willing to give a presentation, participates in class discussions, enjoys helping others, etc. 3. Valuing. This ranges from simple acceptance of a value to one of commitment, for example, the individual demonstrates belief in democratic processes, appreciates the role of science in our everyday lives, shows concern for the welfare of others, shows sensitivity towards individual and cultural differences, etc. 4. Organisation. This refers to the process that individuals go through as they bring together different values, resolve conflicts among them, and start to internalise the values, for example, recognises the need for balance between freedom and responsibility in a democracy, accepts responsibility for his or her own behaviour, accepts professional ethical standards, adapts behaviour to a value system, etc. 5. Characterisation. At this level the individual has a value system in terms of their beliefs, ideas, and attitudes that control their behaviour in a consistent and predictable manner, for example, displays self-reliance in working independently, displays a professional commitment to ethical practice, shows good personal, social, and emotional adjustment, maintains good health habits, etc. The major categories of the affective domain and some active verbs commonly used when writing learning outcomes for this domain are presented in Figure 10. 32 Figure 10. The Affective Domain and Some Action Verbs Used in Writing Learning Outcomes in the Affective Domain 5. Characterisation Accept, appreciate, assist, attempt, challenge, 4. Organisation combine, complete, defend, demonstrate (a belief in), discuss, dispute, embrace, follow, 3. Valuing hold, integrate, join, judge, order, organise, praise, question, relate, 2. Responding share, support, synthesise, value. 1. Receiving Source: Author based on Bloom, Masia, and Krathwohl (1964). Bloom and his colleagues and subsequent authors have linked the various levels in the affective domain to specific verbs. However, this level of detail is not required in the present context. Some examples of learning outcomes in the affective domain are: • Accept the need for professional ethical standards. • Appreciate the need for confidentiality in the professional-client relationship. • Display a willingness to communicate well with patients. • Relate to participants in an ethical and humane manner. • Resolve conflicting issues between personal beliefs and ethical considerations. • Embrace responsibility for the welfare of children taken into care. • Participate in class discussions with colleagues and teachers. 3.4 Writing Learning Outcomes in the Psychomotor Domain The psychomotor domain mainly emphasises physical skills involving coordination of the brain and muscular activity. A review of the literature reveals that this domain has been less well discussed in the field of education 33 than either the cognitive or affective domain. The psychomotor domain is commonly used in areas like laboratory science subjects, health sciences, art, music, engineering, drama, and physical education. Bloom and his research team did not complete detailed work on the psychomotor domain as they claimed lack of experience in teaching psychomotor skills. However, several authors have suggested various versions of taxonomies to describe the development of skills and coordination. For example, Dave (1970) proposed a hierarchy consisting of five levels: 1. Imitation: Observing the behaviour of another person and copying this behaviour. This is the first stage in learning a complex skill. 2. Manipulation: Ability to perform certain actions by following instructions and practicing skills. 3. Precision: At this level, the student has the ability to carry out a task with few errors and become more precise without the presence of the original source. The skill has been attained and proficiency is indicated by smooth and accurate performance. 4. Articulation: Ability to coordinate a series of actions by combining two or more skills. Patterns can be modified to fit special requirements or solve a problem. 5. Naturalisation: Displays a high level of performance naturally (‘without thinking’). Skills are combined, sequenced, and performed consistently with ease. This hierarchy and examples of action verbs for writing learning outcomes in the psychomotor domain are presented in Figure 11. 34 Figure 11. Taxonomy Developed for the Psychomotor Domain and Some Action Verbs Used in Writing Learning Outcomes in the Psychomotor Domain 5. Naturalisation Adapt, adjust, administer, alter, arrange, assemble, balance, bend, build, 4. Articulation calibrate, choreograph, combine, construct, copy, design, deliver, detect, demonstrate, differentiate 3. Precision (by touch), dismantle, display, dissect, drive, estimate, examine, execute, fix, grasp, grind, handle, 2. Manipulation heat, manipulate, identify, measure, mend, mime, mimic, mix, operate, organise, perform (skilfully), 1. Imitation present, react, record, refine, sketch, use. Source: Author based on Dave (1970). Subsequently, Simpson (1972) developed a more detailed hierarchy consisting of seven levels: 1. Perception: The ability to use observed cues to guide physical activity. 2. Set (mindset): The readiness to take a particular course of action. This can involve mental, physical, and emotional disposition. 3. Guided response: The trial-and-error attempts at acquiring a physical skill. With practice, this leads to better performance. 4. Mechanism: The intermediate stage in learning a physical skill. Learned responses become more habitual and movements can be performed with some confidence and level of proficiency. 5. Complex Overt Responses: Physical activities involving complex movement patterns are possible. Responses are automatic and proficiency is indicated by accurate and highly coordinated performance with a minimum of wasted effort. 6. Adaptation: At this level, skills are well developed and the individual can modify movements to deal with problem situations or to fit special requirements. 7. Origination: The skills are so highly developed that creativity for special situations is possible. 35 Other taxonomies in the psychomotor domain have been developed by Harrow (1972) and Dawson (1998). Ferris and Aziz (2005) developed a taxonomy in the psychomotor domain specifically for engineering students. In general, all the various taxonomies in the psychomotor domain describe a progression from simple observation to mastery of physical skills. Some authors have linked specific words to particular levels in the hierarchy. However, that level of detail is outside the scope of this handbook. Some examples of learning outcomes in the psychomotor domain are: • Deliver effective local anaesthesia in the mandible and maxilla and identify the appropriate agents that may be used. • Design a pump-pipeline system which deals with laminar or turbulent, single or multiphase flow with Newtonian or non-Newtonian fluid through straight, branched, or networked pipe systems. • Perform at least 10 local anaesthetic administrations and evaluate your performance with your instructor. • Prescribe and process at least 10 radiographs and evaluate them with your instructor. • Demonstrate proficiency in Cardiopulmonary Resuscitation. • Use a range of physiology equipment to measure physiological function. • Operate the range of instrumentation specified in the module safely and efficiently in the chemistry laboratory. • Administer successfully and in a safe manner with minimal risk to patient and operator, infiltration and regional nerve block anaesthesia. • Present the methodology and findings of the research project in an oral report. • Design a well-illustrated poster presentation to summarise the research project. • Examine a patient extra-orally and intra-orally. • Use the following software effectively and skilfully: MS Word, Excel, and Powerpoint. • Perform a surgical dressing using an aseptic technique. • Sketch the pump characteristic curve, the pipeline curve, and the pump-pipeline operating point, and show how each of these can be altered in a practical manner. • Record an accurate impression of the mouth and identify all anatomical features of importance. The links between the three domains in Bloom’s Taxonomy are illustrated in Figure 12. The attention of the reader is drawn to the areas of overlap among the three areas, that is, each domain should not simply be considered in isolation from the other two domains. 36 Figure 12. The Overlapping Domains of Bloom’s Taxonomy 3.5 General Guidelines for Writing Learning Outcomes There is a great deal of literature on what is considered best practice when writing learning outcomes (Bingham 1999; Fry, Ketteridge, and Marshall 2000; Jenkins and Unwin 2001; Moon 2002). In general, when writing learning outcomes, it is helpful to focus on what you expect students to be able to do or demonstrate at the end of the module or programme. It is important that learning outcomes are expressed in simple and unambiguous terms so that they are clearly understood by students, teachers, colleagues, employers, and external examiners. In general, learning outcomes specify the essential learning for a module. Therefore, when writing learning outcomes for a module, the minimum acceptable standard to enable a student to pass the module should be specified. Therefore, it is recommended that there are a small number of important learning outcomes rather than a large number of superficial outcomes. There is good advice in the literature regarding the number of learning outcomes that should be written for a module. For example, Moon (2002, p. 56) suggests that ‘it is unlikely that there will be more than eight learning outcomes per module. If there are more than ten, they are probably specifying too much curricular detail and may then be unmanageable in the process of assessment’. The Educational and Staff Development Unit of the University of Central England, UK, states that ‘we recommend that you aim for between four and eight learning outcomes for each of your modules…’6. Bingham (1999, p. 8) recommends that ‘most units have between 5 and 9 learning outcomes’. Clearly, the number of learning outcomes is also dependant on the size of the module. McLean and Looker (2006, p. 22) recommend 6 http://lmu.uce.ac.uk/OUTCOMES/UCE%20Guide%20to%20Learning%20Outcomes%202006.pdf / http://lmu.uce.ac.uk/outcomes/#4.%20What%20are%20the%20benefits%20of%20Learning%20Outcomes. 37 that ‘learning outcomes should be few enough and significant enough to be memorable and meaningful – most courses might aim for five to ten outcomes’. In short, a module with about 6 well-written learning outcomes is ideal. One of the most important points stressed in the literature is that learning outcomes must not simply be a ‘wish list’ of what a student is capable of doing on completion of the learning activity. Learning outcomes must be simply and clearly described and must be capable of being validly assessed. (The link between learning outcomes and teaching and assessment is discussed in Chapter 5.) As already discussed, Bloom’s Taxonomy (Bloom et al. 1956) is one of the most useful aids to writing good learning outcomes. The taxonomy provides a ready-made list of verbs and hence is a useful ‘toolkit’ that provides the vocabulary for writing learning outcomes. There is general agreement in the literature regarding the recommendation that when writing learning outcomes, the emphasis must be on active verbs and that certain terms should be avoided: ‘The key word is DO and the key need in drafting learning outcomes is to use active verbs’ (Jenkins and Unwin 2001, p. 1; Fry, Ketteridge and Marshall 2000, p. 51). ‘Try to avoid ambiguous verbs such as “understand’, “know”, “be aware”, and “appreciate”’ (Bingham 1999, p. 16). ‘Concrete verbs such as “define”, “apply” or “analyse” are more helpful for assessment than verbs such as “be exposed to”, “understand”, “know” “be familiar with”’ (Osters and Tiu 2003, p. 23). ‘Vague verbs such as “know” or “understand” are not easily measurable. Substitute “identify”, “define”, “describe” or “demonstrate”’ (British Columbia Institute of Technology 1996, p. 35). ‘Care should be taken in using words such as “understand” and “know” if you cannot be sure that students will understand what it means to know or understand in a given context’ (McLean and Looker 2006, p. 32). ‘Verbs relating to knowledge outcomes – “know”, “understand”, “appreciate” – tend to be rather vague, or to focus on the process students have gone through rather than the final outcome of that process, so use action verbs – “solve”, “evaluate”, “analyse” – to indicate how students can demonstrate acquisition of that knowledge’7. ‘Certain verbs are unclear and subject to different interpretations in terms of what action they are specifying. Such verbs call for covert behaviour which cannot be observed or measured. These types of verbs should be avoided: know, become aware of, appreciate, learn, understand, become familiar with8. Moon (2002) summarises the problems caused by the use of vague terms in writing learning outcomes as follows: 7 University of Central England Educational and Staff Development Unit; http://lmu.uce.ac.uk/OUTCOMES/UCE%20Guide%20to%20Learning%20Outcomes%202006.pdf / http://lmu.uce.ac.uk/outcomes/#4.%20What%20are%20the%20benefits%20of%20Learning%20Outcomes. 8 American Association of Law Libraries; http://www.aallnet.org/prodev/outcomes.asp. 38 ‘Another common fault in the writing of learning outcomes is that they refer to learning and not the representation of learning. A poorly written learning outcome might say, for example: “At the end of the module, the learner will be expected to know the health and safety practices of laboratory work. (Level 1 chemistry)”. We can only tell if the student knows these practices if she is caused to demonstrate her knowledge. She might be asked to write a report, to answer questions, to explain the practices orally and so on’ (p. 66). Fry, Ketteridge, and Marshall (2000), when giving practical advice for writing learning outcomes, recommend the use of ‘unambiguous action verbs’ and list many examples of verbs from Bloom’s Taxonomy. To demonstrate the differences between the vocabulary used in writing aims and learning outcomes, the authors list some examples of verbs, as shown in Table 1. Table 1. Examples of Verbs Used in Writing Aims and Learning Outcomes Aims Outcomes Know Distinguish between Understand Choose Determine Assemble Appreciate Adjust Grasp Identify Become familiar Solve, apply, list Source: Fry, Ketteridge, and Marshall 2000, p. 51. Gosling and Moon (2001) give succinct advice to the reader on writing learning outcomes: ‘Keep learning outcomes simple, normally use only one sentence with one verb in each outcome and avoid unnecessary jargon. Occasionally more than one sentence may be used for clarity’ (p. 20). The following guidelines may be of assistance when writing Learning Outcomes: • Begin each learning outcome with an action verb, followed by the object of the verb followed by a phrase that gives the context. • Use only one verb per learning outcome. • Avoid vague terms like know, understand, learn, be familiar with, be exposed to, be acquainted with, and be aware of. As discussed in Chapter 2, these terms are associated with teaching objectives rather than learning outcomes. • Avoid complicated sentences. If necessary, use more than one sentence to ensure clarity. • Ensure that the learning outcomes of the module relate to the overall outcomes of the programme. • The learning outcomes must be observable and measurable. • Ensure that the learning outcomes are capable of being assessed. 39 • When writing learning outcomes, bear in mind the timescale within which the outcomes are to be achieved. There is always the danger of being overambitious when writing learning outcomes. Ask yourself if it is realistic to achieve the learning outcomes within the time and resources available. • As you work on writing the learning outcomes, bear in mind how these outcomes will be assessed, that is, how will you know if the student has achieved these learning outcomes? If the learning outcomes are very broad, they may be difficult to assess effectively. If the learning outcomes are very narrow, the list of learning outcomes may be too long and detailed. • Before finalising the learning outcomes, ask your colleagues and possibly former students if the learning outcomes make sense to them. • When writing learning outcomes, try to avoid overloading the list with learning outcomes which are drawn from the bottom of Bloom’s Taxonomy (for example, Knowledge and Comprehension in the cognitive domain). Try to challenge the students to use what they have learned by including some learning outcomes drawn from the higher categories (for example, Application, Analysis, Synthesis, and Evaluation) of Bloom’s Taxonomy. It is standard practice when writing learning outcomes for a module, that the list of learning outcomes is usually preceded by a phrase like, ‘On successful completion of this module, students should be able to:’. An example of learning outcomes for a module on Restorative Dentistry is presented in Box 1. Box 1. Learning Outcomes for a Module in Restorative Dentistry On successful completion of this module, students should be able to: • Examine a patient extra-orally and intra-orally. • Formulate an appropriate treatment plan based on an understanding of the disease process present and a prediction of the likely success. • Identify dental caries and restore a tooth to functional form following caries removal. • Record an accurate impression of the mouth and identify all anatomical features of importance. • Design a partial denture with appropriate support and retention. • Administer successfully and in a safe manner with minimal risk to patient and operator, infiltration and regional nerve block anaesthesia. • Communicate with patients and colleagues in an appropriate manner. An example of learning outcomes for a module in Economics is presented in Box 2. Box 2. Learning Outcomes for a Module in Economics On successful completion of this module, students should be able to: • Recognise the main indicators of stock market timing. 40 • Describe and distinguish among the main economic indicators. • Interpret Irish National Income and Expenditure Accounts. • Differentiate between monetary and fiscal policy. • Perform economic calculations, which enable the learner to appreciate economic concepts with greater clarity. • Criticise budgetary decisions using economic criteria. • Construct and interpret company accounts and accounting ratios. • Formulate appropriate budgetary policy in response to changes in the business cycle. • Assess the stance of government fiscal policy. Further examples of learning outcomes written for various modules are presented in Appendix 2. The checklist shown in Box 3 may be of help to double-check that you have written the learning outcomes according to the standard guidelines. Box 3. Checklist for Writing Learning Outcomes ❑ Have I focussed on outcomes not processes, that is, have I focussed on what the students are able to demonstrate rather than on what I have done in my teaching? ❑ Have I begun each outcome with an active verb? ❑ Have I avoided terms like know, understand, learn, be familiar with, be exposed to, be acquainted with, and be aware of? ❑ Are my outcomes observable and measurable? ❑ Are my outcomes capable of being assessed? ❑ Have I included learning outcomes across the range of levels of Bloom’s Taxonomy? ❑ Do all the outcomes fit within the aims and content of the module? ❑ Have I the recommended number of outcomes (maximum of 9 per module)? ❑ Is it realistic to achieve the learning outcomes within the time and resources available? The example presented in Box 4 may help you see how the key learning outcomes were developed for one particular module. Box 4. Example of Development of Key Learning Outcomes Module Title: Dental Surgery – 5th Year Dental Students Draft of learning outcomes: On successful completion of this module students should be able to: Cognitive 41 • Recall anatomy and basic physiology of the head and neck • Outline aetiology, symptoms, pathology, diagnosis, and treatment of oro-facial diseases • List the steps involved in patient assessment, including procedures for specific tests • Apply this format to record a thorough case history of an unseen patient • Summarise relevant information regarding the patient’s current complaint/status • Arrange appropriate tests • Demonstrate the ability to interpret tests and reports • Evaluate all available information and knowledge to generate a differential diagnosis • Formulate an appropriate treatment plan and justify the proposal giving due consideration to patient expectations and limitations. Affective • Manage patients with facial pain and oro-facial disease • Differentiate between patients that can/cannot be safely treated by a general dentist • Develop good communication skills (verbal and nonverbal) • Master the skills required to obtain informed consent, deal with medico-legal issues and dental phobia; deliver health promotion advice • Demonstrate professional behaviour and good clinical governance • Display the capacity to value and participate in projects which require teamwork • Manage competing demands on time, including self-directed learning and critical appraisal. Psychomotor • Prescribe and process dental radiographs • Administer local anaesthetics safely • Perform basic dento-alveolar surgical procedures • Prescribe appropriate drugs • Master skills required to manage intraoperative and postoperative complications • Recognise, evaluate, and manage dental emergencies appropriately. From the above draft, key learning outcomes were identified as shown in the following list. On successful completion of this module, student should be able to: • Master the skills required to record a thorough case history, deliver health promotion advice, and obtain informed consent dealing with medico-legal issues • Summarise relevant information regarding the patient’s current condition to generate a differe ntial diagnosis • Formulate an appropriate treatment plan and justify the proposal giving due consideration to patient 42 expectations and limitations • Arrange appropriate tests and demonstrate the ability to interpret tests and reports • Administer local anaesthetics safely and perform basic dento-alveolar surgical procedures in a professional manner showing good clinical governance • Recognise, evaluate, and manage medical and dental emergencies appropriately • Differentiate between patients that can or cannot be safely treated by a General Dental Practitioner • Manage competing demands on time, including self-directed learning and critical appraisal • Master the therapeutic and pharmacological management of patients with facial pain and oro-facial disease. 3.6 Programme Learning Outcomes As discussed, learning outcomes must be capable of being assessed, that is, they should be written in a way that allows testing of whether the student has achieved the outcome. The rules for writing learning outcomes for programmes are the same as those for writing learning outcomes for modules. The general guidance in the literature is that there should be 5 to 10 learning outcomes for a programme and that only the minimum number of outcomes considered to be essential be included. In short, programme learning outcomes describe the essential knowledge, skills, and attitudes that it is intended that graduates of the programme will be able to demonstrate. When formulating learning outcomes for programmes, it is suggested that there could be value in writing two types of learning outcomes (Moon 2002). The first type of learning outcome refers to those learning outcomes that can be assessed during the programme, that is, within the various modules. The second type of learning outcome may not be assessed at all but give an indication to employers and other agencies the type of standard of practical performance that graduates of the programme will display at the end of the programme. These ‘aspirational’ or ‘desirable; learning outcomes indicate what a good-quality student would be expected to achieve by the end of the programme. The situation may be summarised as follows: ‘It is important to note that there are clear differences in the nature of programme outcomes and learning outcomes written for modules. Programme outcomes are written for a typical or average student and they may be aspirational. They are not, therefore, directly testable. For example, programme outcomes may evidence areas of learning that are the outcomes of the student’s experience of engagement in the programme, on the basis that the whole may be greater than the sum of its parts’ (Moon 2002, p. 78). When writing the programme learning outcomes, it is recommended not to simply compile all the learning outcomes from all the modules in a programme. A programme may be more than simply the sum of the various component modules. For example, there may be some overarching programme learning outcomes, such as formulate hypotheses, analyse data, and draw conclusions. In addition, you may wish to include some aspirational learning outcomes as discussed above. In addition, where there is a choice of modules within a programme, there may be skills common to all students despite the fact that the students may have taken various combinations of modules (Dillon and Hodgkinson 2000). 43 When writing programme learning outcomes, it is common practice to use an initial statement such as, ‘On completion of this programme, students will be able to….’ This statement is followed by the list of learning outcomes written according to the guidelines already discussed for module learning outcomes. As an example, some of the Programme Learning Outcomes for a Master’s Degree in Computer Science9 are listed in Box 5. Box 5. Example of Programme Learning Outcomes for a Postgraduate Computer Science Degree On completion of this programme, it is expected that the students will be able to: • Perform problem solving in academic and industrial environments • Use, manipulate, and create large computational systems • Work effectively as a team member • Organise and pursue a scientific or industrial research project • Write theses and reports to a professional standard, equivalent in presentational qualities to that of publishable papers • Prepare and present seminars to a professional standard • Perform independent and efficient time management • Use a full range of IT skills and display a mature computer literacy. A further example of some of the programme learning outcomes10 listed for an engineering degree are shown in Box 6. Box 6. Example of Programme Learning Outcomes for an Undergraduate Engineering Degree On completion of this programme, it is expected that students will be able to: • Derive and apply solutions from knowledge of sciences, engineering sciences, technology, and mathematics • Identify, formulate, analyse, and solve engineering problems • Design a system, component, or process to meet specified needs and to design and conduct experiments to analyse and interpret data • Work effectively as an individual, in teams, and in multidisciplinary settings together with the capacity to undertake lifelong learning • Communicate effectively with the engineering community and with society at large. 9 http://www.cs.manchester.ac.uk/Study_subweb/Postgrad/ACS-CS/webpages/syllabus/acs/ACS_AIMS.php. 10 http://www.ucc.ie/en/SupportandAdministration/ServiceandAdministrativeOffices/QualityPromotionUnit/LearningOutco mesConference/Presentations/DocumentFile,15075,en.pdf. 44 An example of programme learning outcomes for an undergraduate degree in science education is presented in Box 7. Box 7. Example of Programme Learning Outcomes for an Undergraduate Science Education Degree On successful completion of this programme, students should be able to: • Recognise and apply the basic principles of classroom management and discipline • Identify the key characteristics of excellent teaching in science • Develop a comprehensive portfolio of lesson plans that are relevant to the science curricula in schools • Evaluate the various theories of teaching and learning and apply these theories to assist in the creation of effective and inspiring science lessons • Critically evaluate the effectiveness of their teaching of science in the secondary school system • Display a willingness to cooperate with members of the teaching staff in their assigned school • Foster an interest in science and a sense of enthusiasm for science subjects in their pupils • Synthesise the key components of laboratory organisation and management and perform laboratory work in a safe and efficient manner • Communicate effectively with the school community and with society at large in the area of science education. Examples of the terminology used when writing Programme Learning Outcomes may be summarised as follows: Knowledge • Discuss a wide variety of … • Outline a broad range of fundamental concepts … • Describe the theories and concepts in the field of … • Identify a range of processes used in … • Discuss relationships among the various areas of … • Examine current theory in the area of … • Critique modern theories in the area of … • Examine and evaluate current problems in the area of …, etc. Skills (in a broad sense – as defined in the European Qualifications Framework) • Apply a range of techniques to solve … • Modify techniques in the area of … to solve … • Link theory with practice in order to … 45 • Analyse data to facilitate decision making in the area of … • Utilise appropriate methods, skills, and techniques to solve … • Exhibit proficiency in using a broad range of routine laboratory techniques in the field of … • Recognise limitations in the areas of … • Link relevant theories to the development of a design to … • Utilise appropriate models and techniques in the area of … to … • Select and apply the most suitable techniques to solve problems in the areas of … • Apply appropriate decision making to achieve high standards of performance in the area of … • Identify appropriate solutions to plan future developments in the area of … • Select appropriate instrumental methods to … • Utilise existing strategies to design … • Evaluate existing problems in the area of … in order to … • Initiate research ideas and evaluate research-related publications in the area of … • Implement work objectives and exercise leadership in … • Combine technical skills to define a problem in the area of … and implement suggested solutions to … • Apply technical knowledge in the area of … to solve problems related to … • Recognise existing strategies to facilitate solutions in the area of … • Formulate options and solutions to … • Diagnose problems and suggest solutions in the area of … • Transfer methodologies to new applications in the area of … • Integrate a range of acquired transferable skills such as … • Develop your personal capabilities in order to … • Engage with new developments and practices in order to … • Recognise the need for lifelong learning and professional development in the area of … • Identify and address continuing requirements for professional development in the area of • Contribute to the future development of the field of … • Interpret relevant regulations in the area of … • Recognise the relationship among science, technology, and society in the area of … • Critically appraise research in the area of …. and evaluate the work of peers. 46 Attitudes • Display an appropriate standard of professional practice in the area of … • Embrace responsibility for the welfare of others … • Display personal ethical standards in the area of … • Articulate and defend the need for personal responsibility and ethical considerations in the workplace for … • Work ethically and professionally as part of a team … • Act appropriately in unfamiliar situations in the area of … • Apply appropriate ethical considerations when … • Work as a member of a team to manage … • Accept accountability for achieving … • Work autonomously or as a member of a team in order to …, etc. Some authors recommend the use of a course mapping tool to help obtain an overview of how the programme learning outcomes are covered within the various modules (short courses) offered in the programme11. The coverage of each programme learning outcome within the courses may be shown in the form of a matrix (Table 2). Table 2. Matrix Indicating Where the Programme Learning Outcomes are Covered in the Various Modules and Where the Module Learning Outcomes Map onto the Programme Learning Outcomes Programme Programme Programme Programme Programme Programme Programme Learning Learning Learning Learning Learning Learning Learning Outcome 1 Outcome 2 Outcome 3 Outcome 4 Outcome 5 Outcome 6 Outcome 7 Module 1 Module 2 Module 3 Module 4 Module 5 Module 6 Module 7 Module 8 Module 9 Module 10 Module 11 Module 12…. etc. 11 http://www.bridgew.edu/AssessmentGuidebook/chapter4.cfm#course_mapping. 47 When writing learning outcomes for programmes, it is important to ensure that, where applicable, the learning outcomes for professional bodies are incorporated into the programme outcomes. 3.7 How Learning Outcomes are Formulated Today in Belarus The development of learning outcomes for programmes and modules today in Belarus bears some similarities to the approach proposed in this handbook (see Figure 13). However, there is an important difference: verbs to be used are not always active. As mentioned, in defining learning outcomes, authors should avoid using words such as know, understand, learn, be familiar with, be aware of, etc. (see Chapter 3.5). Figure 13. How Learning Outcomes are Developed in Belarus Source: Titovich 2018. Moreover, currently, all the education results are still defined in terms of competences and not learning outcomes. For example, the following are general descriptors for Master level programmes (International Standard Classification of Education [ISCED 7]12) in the draft National Qualification Framework: Knowledge: Highly specialized applied and theoretical knowledge related to a specific area of professional activity or research that is necessary for the development of new concepts and principles in the professional field, as well as for the collection of data and information on key knowledge in the professional and scientific fields or interdisciplinary fields. Ability to expand knowledge in related areas of professional activity. 12 https://ec.europa.eu/eurostat/statistics- explained/index.php/International_Standard_Classification_of_Education_(ISCED)#Implementation_of_ISCED_2011_.28le vels_of_education.29 48 Skills: Professional skills necessary for critical analysis, evaluation, and interpretation of complex and comprehensive information, integration of knowledge gained from related fields of professional activity, further research and solution of complex problems in the process, development of innovations, new knowledge and procedures in this area, selection of necessary methods and approaches, and evaluation of long-term and short-term results of activities carried out in the professional field. Personal competences: Responsibility for the development and planning of processes leading to significant changes and development, a high level of independence, initiation of complex processes and tasks and monitoring their implementation, responsibility for the introduction and development of new methods in the professional sphere, monitoring the work, as well as assessment of the progress of other employees, planning and organization of their professional training, participation in the planning and allocation of resources. Ability to prepare an accurate, well-structured text on a professional topic, describe your experience, provide justification for your opinions and goals in the language of study and foreign language. 49 4 Competences and Learning Outcomes ‘The relationship between learning outcomes and competences is a complex area – the subject of some debate and no little confusion.’ – Adam 2004, p. 7 The Belarusian higher education system currently uses a competence-based approach in defining education results, which warrants a closer look at the relation between competences and learning outcomes, as provided in this chapter. There is considerable confusion in the literature regarding the meaning of the term competence and the relationship between competences (also written as competencies) and learning outcomes. This chapter attempts to bring some clarity to this area by surveying the literature on this topic and by offering several recommendations on the use of the concept of competence. 4.1 Competence – Attempts to Define It It is difficult to find a precise definition of the term competence, a situation succinctly summarised by Winterton, Delamare-Le Deist, and Stringfellow (2005, p. 7) as follows: ‘There is such confusion and debate concerning the concept of “competence” that it is impossible to identify or impute a coherent theory or to arrive at a definition capable of accommodating and reconciling all the different ways that the term is used.’ Adam (2004, p. 7) comments that ‘some take a narrow view and associate competence just with skills acquired by training’. This point is echoed by Brown and Knight (1995, p. 38), who state that ‘competence probably replaces, albeit at a more sophisticated level, the concept of skills. That doesn’t necessarily make it easier to understand what competencies are, let alone how they are to be recognised’. The UK Training Agency (1989) defines competence as: ‘Standards development should be based on the notion of competence which is defined as the ability to perform the activities within an occupation. Competence is a wide concept which embodies the ability to transfer skills and knowledge to new situations within the occupational area. It encompasses organisation and planning of work, innovation and coping with non-routine activities. It includes those qualities of personal effectiveness that are required in the workplace to deal with co-workers, managers and customers.’ (p. 1) The ECTS Users’ Guide (2005, p. 11) takes a broad view of the term competence and describes it as ‘a dynamic combination of attributes, abilities, and attitudes. Fostering these competences is the object of educational programmes. Competences are formed in various course units and assessed at different stages. They may be divided in subject-area-related competences (specific to a field of study) and generic competences (common to any degree course)’. The problem with this definition of competences is that it is so general that it is difficult to describe the concept of competence with precision. 50 An updated version of the ECTS Users’ Guide (2015, p. 22) describes competences as ‘the proven ability to use knowledge, skills and personal, social and/or methodological abilities, in work or study situations and in professional and personal development’. Once again, we encounter only a vague and general definition of competence. Miller et al. (1988, p. 26) when discussing competence in nursing suggest that there are two senses in which competence can be defined. Firstly, they take a narrow view and equate competence with performance, that is, the ability to perform nursing tasks. Secondly, the authors take a broader view of competence in terms of a ‘psychological construct’, requiring evaluation of the ability of the nurse to integrate cognitive, affective, and psychomotor skills when delivering nursing care. Miller et al. (1988, p. 32) also point out that ‘competence is over defined since there are almost as many definitions of the concept as there are users of the term’. Wolf (1989) expresses the view that competence refers to the ‘ability to perform at the standards expected of employees’ and that statements of competence ‘specify the nature of the particular performable occupational role or roles’ which can be very narrow or encompass a wide range of different roles. She argues that competence is not something that can be observed directly but is a construct of a number of different areas. Neary (2002) points out that the various definitions of competence are not very specific but felt that they indicated what was expected in terms of sources of evidence of being competent. He argues that the term competence embraces several areas: ‘It would be pointless to suggest that there is a single definition. Competence includes a broad range of knowledge, attitudes and observable patterns of behaviour which together account for the ability to deliver a specified professional service. The competent individual can correctly perform numerous (but not necessarily all) tasks, many of which require knowledge, theories, principles of social sciences or comprehension of the social and cultural factors that influence the climate. Competence in this sense also involves adoption of a professional role that values human life.’ (p. 50) A project entitled Tuning Educational Structures in Europe, initiated in 200013, defines the term competence as follows: ‘Competences represent a dynamic combination of knowledge, understanding, skills and abilities. Fostering competences is the object of educational programmes’14. The Tuning Project describes three types of generic competences: • Instrumental competences: cognitive abilities, methodological abilities, technological abilities, and linguistic abilities • Interpersonal competences: individual abilities like social skills (social interaction and cooperation) • Systemic competences: abilities and skills concerning whole systems (a combination of understanding, sensibility and knowledge; prior acquisition of instrumental and interpersonal competences required). Examples of generic competences described in the Tuning project are the capacity for analysis and synthesis, the capacity to learn and problem solve, the capacity for applying knowledge in practice, the capacity to adopt 13 Tuning Educational Structures in Europe; http://tuning.unideusto.org/tuningeu/. 14 Tuning Educational Structures in Europe; http://tuning.unideusto.org/tuningeu/. 51 to new situations, concern for quality, information management skills, ability to work autonomously, capacity for organising and planning, oral and written communication in native language, and interpersonal skills. In a questionnaire developed by the Tuning Project for academics, respondents were asked to rank in order of importance the 17 generic competences presented in Box 8.15 Box 8. Generic Competences (Tuning Project) 1. Ability to work in an interdisciplinary team 2. Appreciation of diversity and multiculturality 3. Basic knowledge of the field of study 4. Basic knowledge of the field of the profession 5. Capacity for analysis and synthesis 6. Capacity for applying knowledge in practice 7. Capacity for generating new ideas (creativity) 8. Capacity to adapt to new situations 9. Capacity to learn 10. Critical and self-critical abilities 11. Decision making 12. Elementary computing skills (word processing, database, other utilities) 13. Ethical commitment 14. Interpersonal skills 15. Knowledge of a second language 16. Oral and written communication in your native language 17. Research skills. Many of the competences listed in Box 8 are so general that it is difficult to know what they mean. Without clarity, assessment of these competences would be extremely difficult if not impossible. In addition, there appear to be no rules or guidelines for the writing of competences – some are written in terms of ‘ability’, some in terms of ‘capacity’, and others in terms of skills and commitment, whilst others are written in terms of ‘knowledge’. Since the Tuning Project has been built on competences, it has been widely criticised ( Gibbs, Kennedy, and Vickers 2012; Kennedy and McCarthy 2016). 15 Tuning Educational Structures in Europe; http://tuning.unideusto.org/tuningeu/. 52 4.2 Competence – Inclusive of the Broad and Narrow View In discussing a conceptual framework of professional competency, Jarvis (1985, p. 52) suggests that analysis of the concept of competency could help to provide a basis on which a curriculum could be constructed and suggests that this would have three components: 1. Knowledge and understanding of relevant academic disciplines, psychomotor elements, interpersonal skills, and moral values 2. Skills to perform the psychomotor techniques, interact with members in the role 3. Attitudes that result in a knowledge of and commitment to professionalism, a willingness to play the role in a professional manner. Neary (2002, p. 37) agrees with the above description and points out that ‘practitioners and teachers argue that competence is more than knowledge and skills. Values, critical thinking, professional judgement, formulation of attitudes, [and] the integration of theory from the humanities and the sciences are also competencies.’ The competences expected of newly qualified teachers in England and Wales are discussed by Capel, Leask, and Turner (1997), and the list of competences published by the Department of Education and the Welsh Office (1992) takes both a broad and narrow view of the term competence (see Box 9).16 16 http://www.educationengland.org.uk/documents/dfe/circular9-92.html 53 Box 9. Examples of Competences Expected of Newly Qualified Teachers in England and Wales Subject Knowledge Newly qualified teachers should be able to demonstrate: • An understanding of the knowledge, concepts, and skills of their specialist subjects and of the place of these subjects in the school curriculum • Knowledge and understanding of the National Curriculum and attainment targets and the programmes of study in the subjects they are preparing to teach, together with an understanding of the framework of the statutory requirements • A breadth and depth of subject knowledge beyond programmes of study and examination syllabuses in school. Subject Application Newly qualified teachers should be able to: • Produce coherent lesson plans which take account of National Curriculum and attainment targets and of the school’s curriculum policies • Ensure continuity and progression within and between classes and in subjects • Set appropriately demanding expectations for pupils • Employ a range of teaching strategies appropriate to the age, ability, and attainment levels of pupils • Present subject content in clear language and in a stimulating manner • Contribute to the development of pupils’ language and communications skills • Demonstrate ability to select and use appropriate resources, including information technology. Class Management Newly qualified teachers should be able to: • Decide when teaching the whole class groups, pairs, or individuals what is appropriate for particular learning purposes • Create and maintain a purposeful and orderly environment for the pupils • Devise and use appropriate rewards and sanctions to maintain an effective learning environment • Maintain pupils’ interest and motivation. Some of the competences listed above are statements of a general nature (for example, ‘demonstrate understanding of the knowledge, concepts, and skills’) and others are learning outcomes (for example, ‘produce lesson plans…’, ‘present subject content…’). Thus, competences with a narrow focus can be written as learning outcomes. The confusion caused by using the term competence both broadly and narrowly is apparent in Dooley et al.’s (2004, p. 27) description of competence: ‘Competency-based behavioural anchors are defined as performance capabilities needed to demonstrate knowledge, skill and ability (competency) acquisition’. As Winterton, 54 Delamare-Le Deist, and Stringfellow (2005, p. 13) point out, this definition means that competency is a subset of itself! While van der Klink and Boon (2003, p. 125) describe competence as a ‘fuzzy concept’, they nevertheless suggest that it is a ‘useful term, bridging the gap between education and job requirements’. In their view, in the UK the term competence appears to refer to the performance standards for functions and professions such as those developed for National Vocational Qualifications. Van der Klink and Boon (2003, p. 126) also point out that in the United States, the term competence refer to the ‘skills, knowledge and characteristics of persons, that is traits, motives and self-concept which contribute to performance excellence’. Thus, the interpretation of the term competence appears to depend not only on the views of the authors discussing it but also varies from country to country. Many of the careers in which the concept of competence appears to be used are in the vocational area, such as nursing, teaching, and social work. The concept of competence is often embedded into requirements for recognition of qualifications or membership of a licensing organisation. Frequently, competence is measured by assessing some type of ‘performance’ in the workplace. However, van der Klink and Boon (2003) warn against associating competences exclusively with skills: ‘Staff members are not always too willing to adopt the notion of competency-based education. Teachers are concerned that competencies will result in an exclusive focus on vocational skills and this might lead to an underestimation of reflective and/or academic competencies in the curricula’. (p. 133) 4.3 Competence and Competency Some authors (Boam and Sparrow 1992; Hendry, Arthur, and Jones 1995; Mitrani, Dalziel, and Fitt 1992; Smith 1993) use the term competency (plural competencies) when referring to occupational competences. For examples, Mitrani, Dalziel, and Fitt (1992, p. 107) refer to competency as ‘an underlying characteristic of an individual which is causally related to effective or superior performance in a job’. However, other authors treat the terms competence and competency as synonymous (Brown 1993, 1994; McBeath 1990). In fact, McBeath (1990, p. 14) refers to the term competencies as ‘relatively new jargon’ and states that the word did not feature in the indexes of most management development books published up to 1986. Hartle (1995, p. 63) describes competency as ‘a characteristic of an individual that has been shown to drive superior job performance’ and refers to visible competencies of knowledge and skills as well as underlying elements of competencies such as characteristics and motives. He summarises his view of the term competencies by stating that ‘competencies describe what make people effective in a given role’. Elkin (1990) associates competences with micro-level job performance and competencies with higher management attributes. Cockerill (1989) describes output competences such as effective presentation skills, with input competencies such as self- confidence (Winterton, Delamare-Le Deist, and Stringfellow 2005). Van der Klink and Boon (2003) attempt to trace the different interpretations of the concept of competence within the educational systems of various countries: ‘There is considerable confusion about what competency actually means…. First, differences can be observed between nations along the lines of different national educational policies and different types of 55 relations between education and the labour market, many of which have an historic origin. In the British approach it refers to the ability to meet the performance standards for functions and professions such as those developed for National Vocational Qualifications (NVQs) in the UK. In the USA, competencies refer to the skills, knowledge and characteristics of persons, that is traits, motives and self-concept, which contribute to performance excellence. These differences are evident even in the words themselves: competences (UK) and competencies (USA). To put it simply: competences refer to work and its achievement; competencies concern the people who do the work…. More than in the UK or the USA, the German perspective stresses a holistic view of competency. It is not just a random collection of skills and knowledge. Competencies are defined as integrated action programmes that enable individuals to perform adequately in various job contexts within a specific profession.’ (p. 126–127) This theme is also discussed by several other authors. Burgoyne (1988) distinguishes ‘being competent’ (meeting the demands of the job) from ‘having competencies’ (possessing the necessary attributes to perform competently). Woodruffe (1991, p. 31) describes competency as ‘an umbrella term to cover almost anything that might directly or indirectly affect job performance’. He attempts to distinguish between competence and competency by describing competence as aspects of the job which an individual can perform with competency, referring to a person’s behaviour that underpins competent performance. He writes: ‘Having arrived at a definition of competency it is important to differentiate it from competence. An essential distinction is between aspects of the job at which the person is competent and aspects of the person that enable him or her to be competent…. Competencies are different from aspects of the job. They are what the person brings to the job…. Competencies are dimensions of behaviour which are related to superior job performance. They are ways of behaving that some people carry out better than others.’ (p. 32) Tate (1995, p. 27) agrees with Woodruffe’s general description of competency and warns against confusing ‘input competencies with output competences’. Even the Belarusian and Russian languages have no problem separating the meanings of the terms ‘competence’ and ‘competency’, for which reason such confusion does not exist. However, it does not help much with the application of this approach or making the results of education clearer. 4.4 The Relationship among Competences, Objectives, and Learning Outcomes The relationship among competences, objectives, and learning outcomes is discussed by Hartel and Foegeding (2004). In that paper the authors define competence as ‘a general statement detailing the desired knowledge and skills of students graduating from our course or program’ (p. 69). They provide this example of a competence in the area of food engineering and processing: ‘The student should be able to use the mass and energy balances for a given food process’. Arising from this competence Hartel and Foegeding derive two objectives and four learning outcomes, which are summarised in Figure 14. 56 Figure 14. An Example Showing the Relationship among Competence, Objectives, and Learning Outcomes in the Area of Food Engineering Competence: The student should be able to use the mass and energy balances for a given food process Objectives: • Understand scope of mass balances in food processing systems • Understand appropriate use of mole fractions and mass fractions in mass balances Learning outcomes: • Describe the general principles of mass balances in steady state systems • Draw and use process flow diagrams with labels on flow streams for mass balance problems • Solve mass balance problems associated with food processing operations • Design and solve mass balances for complex process flow systems, including batch mixing problems, multiple stage flow problems, problems with multiple inflows and outflows, recycle streams and multiple components, and processes where chemical reactions occur. Source: Author based on Hartel and Foegeding (2004). Hartel and Foegeding’s (2004) learning outcomes specify precisely how students will be able to demonstrate that they have acquired a particular competence. This point is echoed by Neary (2002) when discussing teaching for competence. Neary points out that the challenge for the teacher is to ‘elect appropriate learning outcomes which will lead to achieving the competencies, specify evaluation indicators and develop a functional delivery system’. 4.5 Competence within a Specific Profession It is helpful to understand the concept of competence by considering an example of competence used in a particular profession, for example, dentistry, something discussed in detail by Oliver et al. (2008). Oliver et al. (2008) quote the definition put forward by Chambers (1994). As Olivier et al. write: 57 ‘Chambers has provided a useful definition of competence: The behaviour expected of beginning independent practitioners. This behaviour incorporates understanding, skills, and values in an integrated response to the full range of circumstances encountered in general professional practice. This level of performance requires some degree of speed and accuracy consistent with patient well being but not performance at the highest level possible. It also requires an awareness of what constitutes acceptable performance under the circumstances and desire for self-improvement’. Oliver et al. (2008) do not appear to distinguish between competence and competency as the definition of competence that they quote is from a paper discussing competency. Oliver et al. (2008) go on to describe competences as broad statements that outline the knowledge, skills, and attitudes of the new graduate. They also state that competences may be considered similar to aims and may be supported by learning outcomes. They point out that assessment of competence relates not only to skill but also requires appropriate knowledge and attitudes, including self-awareness, that is, an ability to recognise personal strengths and weaknesses. Oliver et al. (2008) describe the link between competences and learning outcomes as follows: ‘Learning outcomes support the competences, are at a greater level of detail and form the basis of both learning and assessment. Properly constructed, competences and learning outcomes are precisely formulated to indicate what the students should know about, what the students should understand, and what the students should be able to do and how well, using language and context that indicates the level at which they will be assessed.’ When summing up, Oliver et al. (2008) recommend that the curriculum should have several features, among which are that it should be competency and outcome based. Chambers (1994) takes up this point when discussing the concept of competencies within the profession of dentistry. He describes competencies in terms of what dentists do on a regular basis to meet patients’ needs. He discusses competencies in terms of psychomotor skill performance and understanding of what is being done and supported by professional values. He highlights the problem of inferring general capacity to perform based on specific samples of behaviour and explains how the profession of dentistry has approached this problem. Chambers (1994) writes: ‘Dentistry has tended to solve this problem by emphasising the mechanical and the detailed while avoiding those things that are difficult to measure…. Competencies is a comfortable term that finds its way into conversation when a general word is needed referring to good dentistry. I have never met anyone who is against competent dentists. But it is also difficult to be precise about what exactly that means.’ In short, whilst the term competence or competency is used within certain professions, even within these professions it can be difficult to state precisely what is meant by these terms. 4.6 Recommendations on the Use of Competences Clearly, there is no single definition of the term competence. Descriptions of the term competence range from that of a broad overarching attribute to that of a very specific task. The situation is nicely summarised by Brown (1994) when discussing the use of the term in the context of managerial competence: 58 ‘One of the reasons for the debate about the usefulness of managerial competence may be the soft focus and blurred edges of the term “competence”. Social science has the habit of taking a word from our common vocabulary and altering the meaning by its adoption as a technical or academic term. This process is still happening to “competence” and a common consensus has yet to be established as to what the word should mean when used in management applications.’ (p. 289–290) The above conclusion is echoed by van der Klink and Boon (2003) when discussing the fuzzy concept of competences: ‘The fact that the concept of competencies serves as a remedy for solving rather different problems probably has to do with its diffuse nature. It is actually an ill-defined concept with no clear content, thus allowing ample interpretations. This major vagueness is partly caused by the application of the concept in various countries, different settings and for different purposes. Its vagueness is probably at the same time the explanation for its prominent status today but it makes it difficult to use the concept as a sound cornerstone for designing HRD [Human Resource Development] and educational practices.’ (p. 134) The confusion in the use of the term competence is in contrast with the clear definition of the concept of a learning outcome found in the literature (ECTS Users’ Guide 2015; Kennedy, Hyland, and Ryan 2006; Kennedy 2008). In concluding their article on competencies, van der Klink and Boon (2003, p. 136) address the question of research in this area and they ‘encourage researchers to unravel the fuzzy concept of competencies’! This is a formidable task due to the lack of clarity associated with the term and the lack of clear guidelines on how they should be written. In contrast to this, the guidelines for writing learning outcomes are clearly articulated in the literature. Given the considerable confusion in the literature, if the term competence must be used, then its meaning needs to be clearly defined for the context in which it is being used (Kennedy, Hyland, and Ryan 2009, p. 15). The literature within certain professions indicates that the term competence has a shared meaning. Hence, there is no problem with using the concept of competence since there is a common understanding of its meaning among members of that profession. The problem arises when the term competence is used in a general context without defining what it means. Therefore, to avoid confusion, it is recommended that when using the term competence, the following guidelines should be followed: 1. State the definition of competence that is being used in the particular context. 2. To ensure clarity of meaning, write competences using the vocabulary of learning outcomes, that is, express the required competence in terms of the students achieving specific programme learning outcomes or module learning outcomes. Since there is no common understanding of the term competence, learning outcomes have become more commonly used than competences when describing what students are expected to know, understand, and be able to demonstrate at the end of a module or programme. The ‘fuzziness’ of competences disappears in the clarity of learning outcomes. 59 5 Linking Learning Outcomes to Teaching and Learning Activities and to Assessment ‘If students are to learn desired outcomes in a reasonably effective manner, then the teacher’s fundamental task is to get students to engage in learning activities that are likely to result in their achieving those outcomes… what the student does in determining what is learned is more important than what the teacher does.’ – Shuell 1986, p. 429 5.1 Introduction As explained in section 3.5, it is important to write learning outcomes in such a way that they are capable of being assessed. Moon (2002) emphasises this point: ‘Certainly, all learning outcomes should be assessable; in other words, they should be written in terms that enable testing of whether or not the student has achieved the outcome’ (p. 75). Clearly, it is necessary to have some form of assessment tool or technique to determine the extent to which learning outcomes have been achieved. Examples of direct assessment techniques include the use of written examinations, project work, portfolios, a grading system with rubrics, theses, reflective journals, and performance assessment. Examples of indirect assessment methods include surveys of employers, comparison with peer institutions, surveys of past graduates, retention rates, and analysis of curriculum. The challenge for teachers is to ensure that there is alignment among the learning outcomes, the teaching and learning activities, and the assessment. This connection among teaching, learning, and assessment of learning outcomes helps to make the overall learning experience more transparent and meaningful for students. Ramsden (2003) points out that evidence collected from student course evaluations shows that clear expectations on the part of students of what is required of them are a vitally important part of students’ effective learning. Lack of clarity in this area is almost always associated with negative evaluations, learning difficulties, and poor student performance. Toohey (1999) recommends that the best way to help students understand how they must achieve learning outcomes is by clearly setting out the assessment techniques and the assessment criteria. In terms of teaching and learning, there is a dynamic equilibrium between teaching strategies on one side and learning outcomes and assessment on the other. It is important that the assessment tasks mirror the Learning Outcomes since, as far as the students are concerned, the assessment is the curriculum: ‘From our students’ point of view, assessment always defines the actual curriculum’ (Ramsden 2003, p. 178). This situation is represented graphically by Biggs (2003b, p. 3) as shown in Figure 15. 60 Figure 15. Different Representations of Teacher and Student Perspectives Teacher Perspectives: Objectives DLOs* Teaching Activities Assessment Student Perspectives: Assessment Learning Activities Learning Outcomes Source: Author based on Biggs (2003b). Note: DLOs = Desired Learning Outcomes. In stressing this point, Biggs (2003b) emphasises the strong link between the curriculum and assessment as follows: ‘To the teacher, assessment is at the end of the teaching-learning sequence of events, but to the student it is at the beginning. If the curriculum is reflected in the assessment, as indicated by the downward arrow, the teaching activities of the teacher and the learner activities of the learner are both directed towards the same goal. In preparing for the assessment, students will be learning the curriculum.’ (p. 3– 4) The importance of assessment in the teaching and learning process cannot be overemphasised. As mentioned (Ramsden 2003), as far as the students are concerned, the assessment is the curriculum. They will learn what they think they will be assessed on, not what may be in the curriculum or even what has been covered in lectures. The old adage that ‘assessment is the tail that wags the dog’ is true. Since assessment is a driving force for learning, we must have a clear idea about the type of learning we want from our students so that the assessment tasks we give them will help achieve the desired learning. A flowchart similar to the one in Figure 16 may help clarify the steps involved in the development, refining, and assessment of learning outcomes. 61 Figure 16. Flowchart Summarising the Steps Involved in the Development and Refining of Learning Outcomes and their Assessment Identify objectives of module Write learning outcomes using standard guidelines Develop a teaching and learning strategy to enable students to achieve learning outcomes Design assessment method to test whether learning outcomes have been achieved If necessary, modify module content and assessment in light of feedback Source: Author. 62 5.2 Linking Learning Outcomes, Teaching, and Assessment Assessment is often described in terms of formative assessment or summative assessment. Formative assessment has been described as being assessment FOR learning. It has been described as assessment that ‘refers to all those activities undertaken by teachers, and by the students in assessing themselves, which provide information to be used as feedback to modify the teaching and learning activities in which they are engaged’ (Black and Williams 1998). In other words, formative assessment helps inform the teacher and the students as to how the students are progressing. Formative assessment is usually carried out at the beginning of or during a programme. The students’ performance on the assessment tasks can help the teacher make decisions about the direction of the teaching to help the learning process. It has been clearly shown (Black and Williams 1998) that by giving feedback to students, formative assessment can help improve the learning and performance of students. The feedback the students receive from the teacher helps improve their learning. For example, formative assessment can include course work where the student receives feedback which will be of help in improving the next performance. It can also include discussions between a mentor and a student where areas for attention are identified (Brown and Knight 1994). The main characteristics of formative assessment include: • Identification by teachers and students of the learning outcomes and the criteria for achieving these outcomes • The provision of clear and rich feedback in an effective and timely fashion • The active involvement of students in their own learning • Good communication between teacher and students • The response by the teacher to the needs of the students. An example of formative assessment could be asking the students to give a presentation to the rest of the class. This can help enhance the student’s knowledge as well as his or her research, communication, and organisational skills. Whilst formative assessment may be used as part of continuous assessment, it is not always included in the final grade awarded to students. Indeed, many educationalists believe it should not be included. For example, Donnelly and Fitzmaurice argue that ‘in order for students to have the maximum opportunity to learn in a module, then there must be some option for a formative assessment which does not contribute to the final grade. Students can then obtain feedback which will allow them to address any gaps in their knowledge or skills’ (Donnelly and Fitzmaurice 2005, p. 14). In short, formative assessment is often seen as part of the teaching process rather than the grading process. Summative assessment is assessment that tries to summarise student learning at some point in time – usually at the end of a module or programme. Summative assessment has been described as ‘end-of-course assessment and essentially means that this is assessment which produces a measure which sums up someone’s achievement and which has no other real use except as a description of what has been achieved’ (Brown and Knight 1994). 63 Thus, the use of summative assessment enables a grade to be generated that reflects the student’s performance. Unfortunately, summative assessment is often restricted to just the traditional examination paper and does not involve other areas like project work, portfolios, or essays. Because of the nature of summative assessment, not all learning outcomes can be assessed at any one time. Assessment of just a sample of learning outcomes is common. In theory, continuous assessment is a combination of summative and formative assessment. In practice, continuous assessment often amounts to repeated summative assessments with marks being recorded but little or no specific feedback being given to students. Biggs (2003a, p. 170) points out that different assessment tasks are associated with different kinds of learning. These are summarised in Table 3. Table 3. Assessment Tasks and the Different Kinds of Learning Assessed Assessment Mode Most Likely Kind of Learning Assessed Extended prose, essay type Essay exam Rote, question spotting, speed structuring Open book As for exam, but less memory, less curriculum coverage Assignment, take-home Read widely, interrelate, organise, apply Objective test Multiple choice Recognition, strategy, comprehension Ordered outcome Hierarchies of understanding Performance assessment Practicum Skills needed in real life Seminar, presentation Communication skills Posters Concentrating on relevance, application Interviewing Responding interactively Critical incidents Reflection, application, sense of relevance Project Application, research skills Reflective journal Reflection, application, sense of relevance Case study, problems Application, professional skills Portfolio Reflection, creativity, unintended outcomes Rapid assessments (large group) Concept maps Coverage, relationships 64 Venn diagrams Relationships One-minute/three-minute paper Level of understanding, sense of relevance Short answer Recall units of information, coverage Letter to a friend Holistic understanding, application, reflection Cloze Comprehension of main ideas Source: Author adapted from Biggs (2003a). It is important that the method of assessment used should attempt to test whether the learning outcomes have been achieved. However, the evidence suggests that the range of assessment of students is limited, with approximately 80 percent of assessment being in the form of exams, essays, and reports of some kind (Brown 1999). For example, a study of University College Dublin (UCD) assessment practices (O’Neill 2002) found that a random sample of 83 teaching staff used 256 assessments when asked to describe one of their courses, that is, approximately three assessments per course. Of these assessments, the majority were summative (84 percent) and the minority formative (16 percent). Developing links among learning outcomes, teaching strategies, student activities, and assessment tasks is challenging for the teacher. Table 4 may help in developing these links. 65 Table 4. Linking Learning Outcomes, Teaching and Learning Activities, and Assessment Learning Outcomes Teaching and Learning Assessment Activities Cognitive Lectures End of module exam Knowledge Tutorials Comprehension Multiple choice tests Discussions Application Laboratory work Essays Analysis Clinical work Practical assessment Synthesis Group work Fieldwork Evaluation Affective Seminar Clinical practice Integration Peer group Presentation of beliefs, presentation ideas, and Project work attitudes Acquisition of physical Psychomotor skills Source: Author. It is important to ensure that the method of assessment adequately assesses the learning outcomes. There may not be just one method of assessment to satisfy all learning outcomes and it may be necessary to choose several assessment methods. An example using Table 4 and applying it to a module of the BSc(Ed) programme is presented in Table 5. 66 Table 5. Linking Learning Outcomes, Teaching and Learning Activities, and Assessment for a Module in BSc(Ed) Programme Learning Outcomes Teaching and Learning Assessment Activities 10-credit module Total Mark = 200 Cognitive Lectures (12) End of module exam • Recognise and apply the Tutorials (6) basic principles of Observation of classes Portfolio of lesson plans classroom management (6) of experienced and discipline science teacher • Identify the key (mentor) characteristics of high- quality science teaching • Develop a comprehensive (100 marks) portfolio of lesson plans Affective • Display a willingness to Participation in Report from school cooperate with members mentoring feedback mentor of teaching staff in their sessions in school (4) assigned school Participation in 3 End of project report. • Participate successfully in a sessions of University Peer Assisted Learning College Cork Peer project Assisted Learning (PAL) Programme (50 marks) Peer group presentation Psychomotor Teaching practice Supervision of teaching practice • Demonstrate good 6 weeks @ 2 hours per classroom presentation week. skills • Perform laboratory Laboratory work Assessment of teaching practical work in a safe and skills efficient manner (50 marks) Source: Author. The above level of detail is not, of course, required in the description of modules, but it can be helpful to create a table like the above when designing or revising modules. 67 Biggs (2003a) refers to this type of process as involving constructive alignment. This means that the curriculum is designed so that the teaching activities, learning activities, and assessment tasks are coordinated with the learning outcomes (see Figure 17). The constructive part refers to the type of learning and what the student does. The alignment part refers to what the teacher does. Figure 17. Constructive Alignment Source: Author. Biggs (2003a) points out that in a good teaching system, the method of teaching, learning activities, and method of assessment are all coordinated to support student learning: ‘When there is alignment between what we want, how we teach and how we assess, teaching is likely to be much more effective than when it is not (aligned)…. Traditional transmission theories of teaching ignore alignment.’ It is clear from the above that there are three basic areas involved in the constructive alignment of any module: 1. Clearly defining the learning outcomes 2. Selecting teaching and learning methods that are likely to ensure that the learning outcomes are achieved 3. Assessing the student learning outcomes and checking to see how well they match with what was intended. These ideas are summarised in Figure 18. 68 Figure 18. Constructive Alignment Source: Author. 5.3 Assessment Criteria and Learning Outcomes As discussed in section 3.5, learning outcomes specify the minimum acceptable standard to enable a student to pass a module. Student performances above this basic threshold level are differentiated by applying grading criteria. Grading criteria are statements that indicate what a student must demonstrate to achieve a higher grade. These statements help differentiate the levels of performance of a student. By making these criteria clear to students, it is hoped that students will aim for the highest levels of performance. Giving only a grade to a student does not provide adequate feedback on their performance since the grade simply indicates an overall level of competency. This overall grade does not identify strengths and weaknesses on specific learning outcomes. However, if the grading system is tied to some form of scoring guide, it can be a useful way of identifying areas for improvement that need to be addressed. A scoring guide that is used in assessment is often referred to as a rubric. A rubric is a grading tool used to describe the criteria used in grading the performance of students. In general, each rubric consists of a set of criteria and marks or grades associated with these criteria. Thus, rubrics help to define the criteria of the system of assessment by describing performance at different points on a rating scale. An example of a rubric for assessing the presentation of a project carried out by science education students is presented in Table 6. This rubric is adapted from one found at https://www.ncsu.edu/midlink/rub.pres.html. 69 Table 6. Rubric for Evaluation of Student Presentation for Science Project Poor (0%–39%) Average (40%–59%) Good (60%–69%) Very Good (70 %–100%) Total Audience cannot Student presents Organisa- Audience has difficulty following Student presents information in understand presentation information in logical tion presentation because student logical, interesting sequence which because there is no sequence which audience (10) jumps around. audience can follow. sequence of information. can follow. Student does not have grasp Student is at ease with Student demonstrates full Subject Student is uncomfortable with of information; student expected answers to all knowledge (more than required) by Knowledge information and is able to answer cannot answer questions questions but fails to answering all class questions with (50) only rudimentary questions. about subject. elaborate. explanations and elaboration. Student occasionally uses Student’s graphics explain and Graphics Student uses superfluous Student’s graphics relate to graphics that rarely support text reinforce screen text and (10) graphics or no graphics. text and presentation. and presentation. presentation. Student’s presentation has Presentation has three Presentation has no more Literacy Presentation has no misspellings or four or more spelling errors misspellings and/or grammatical than two misspellings and/or (10) grammatical errors. and/or grammatical errors. errors. grammatical errors. Student occasionally uses eye Student maintains eye Student maintains eye contact with Eye Contact Student reads all of report contact, but still reads most of contact most of the time but audience, seldom returning to (10) with no eye contact. report. frequently returns to notes. notes. Student mumbles, Student’s voice is clear. Student’s voice is low. Student Student uses a clear voice and incorrectly pronounces Student pronounces most Elocution incorrectly pronounces terms. correct, precise pronunciation of terms, and speaks too words correctly. Most (10) Audience members have terms so that all audience members quietly for students in the audience members can hear difficulty hearing presentation. can hear presentation. back of class to hear. presentation. Total Points: Source: Author; adapted from https://www.ncsu.edu/midlink/rub.pres.html. 70 6 Looking Towards the Future with Learning Outcomes ‘Much depends on how they (learning outcomes) are constructed and whether (and how) they include knowledge, skills, abilities/attitudes and understanding. Badly constructed, narrow and limiting learning outcomes are not appropriate for higher education where creativity and imaginative leaps are highly valued.’ – Adam 2004, p. 15 6.1 Introduction We saw in Chapter 2 that international trends in education show a shift from the traditional ‘teacher -centred’ approach to a more ‘student-centred’ approach. While traditionally the focus was on what the teacher did, in recent years the focus has been on what students have learned and can demonstrate at the end of a module or programme. Among the key characteristics of outcome-based education listed by Harden (2002a) are: • The development of clearly defined and published learning outcomes that must be achieved before the end of the programme • The design of a curriculum, learning strategies, and learning opportunities to ensure achievement of the learning outcome • An assessment process matched to the learning outcomes and the assessment of individual students to ensure that they have achieved the outcomes. (p. 118) 6.2 Advantages of Learning Outcomes Whilst there has been some criticism of outcome-based education in the literature, in general, this type of education has received strong support at an international level. For example, Jenkins and Unwin (2001) assert that learning outcomes: • Help teachers tell students more precisely what is expected of them • Help students learn more effectively: students know where they stand and the curriculum is made more open to them • Help teachers design their materials more effectively by serving as a template for them • Make it clear what students can hope to gain from following a particular course or lecture • Help teachers select the appropriate teaching strategy matched to the intended learning outcome, for example, lecture, seminar, group work, tutorial, discussion, peer group presentation, or laboratory class • Help teachers tell their colleagues more precisely what a particular activity is designed to achieve • Assist in creating examinations based on the materials delivered 71 • Ensure that appropriate teaching and assessment strategies are employed. In writing about embracing learning outcomes in medical education, Harden (2002a, p. 119) comments that ‘where it has been implemented, outcome-based education has had a significant and beneficial impact. Clarification of the learning outcomes in medical education helps teachers, wherever they are, to decide what they should teach and assess, and students what they are expected to learn’. In another paper, Harden (2002b) describes how learning outcomes have been used to develop a model for use in medical training: ‘Learning outcomes can be specified in a way that covers the range of necessary competences and emphasises the integration of different competences in the practice of medicine. An important feature of the three-circle model of learning outcomes is that it does just that. In the inner circle are the seven learning outcomes relating to what a doctor is able to do, i.e. the technical competences expected of a doctor (“doing the right thing”); in the middle circle the learning outcomes relating to how the doctor approaches his or her task with knowledge and understanding and appropriate attitude and decision- making strategies (“doing the thing right”); and in the outer circle the ongoing development of the doctor as an individual and as a professional (“the right person doing it”)’ (p. 153). The model described by Harden is shown in Figure 19. Figure 19. A Three-Circle Model for Outcome-Based Education in Medicine Source: Harden et al. 1999b. Toohey (1999) argues that learning outcomes are valuable because: • They clarify the educational purpose and can help in the design of other aspects of the programme. In a constructively aligned course, learning outcomes provide a guide to learning and assessment activities. • They express the educational purpose of the teacher to students so that students know what the course offers them and what is expected of them. In other words, learning outcomes help students to focus on what is important in the course. 72 • They help the teacher to reconceptualise his or her educational purpose from the student’s point of view, that is, in terms of what the student can be expected to be able to do as a result of completing the module or programme. Adam (2004) summarises the advantages of learning outcomes under four main headings: (a) Course and module design Learning outcomes can: • Help to ensure consistency of delivery across modules and programmes • Aid curriculum design by clarifying areas of overlap between modules and programmes • Help course designers to determine precisely the key purposes of a course and to see how components of the syllabus fit and how learning progression is incorporated • Highlight the relationship among teaching, learning, and assessment and help improve course design and the student experience • Promote reflection on assessment and the development of assessment criteria and more effective and varied assessment. (b) Quality assurance Learning outcomes: • Increase transparency and the comparability of standards between and within qualifications • Possess greater credibility and utility than traditional qualifications • Play a key role by acting as points of reference for establishing and assessing standards. (c) Students Learning outcomes provide: • Comprehensive sets of statements of exactly what the students will be able to achieve after successful study • Clear information to help students with their choice of module and programme, which can lead to more effective learning • Clear information to employers and higher education institutions on the achievements and characteristics associated with particular qualifications. (iv) Mobility Learning outcomes: • Contribute to the mobility of students by facilitating the recognition of their qualifications • Improve the transparency of qualifications • Simplify credit transfer 73 • Provide a common format that helps promote lifelong learning and that can assist in creating multiple routes through and between different educational systems. Finally, the University of Central England Educational and Staff Development Unit points out that learning outcomes can help to avoid plagiarism and provides the following two examples: • Learning outcome: Students will be able to demonstrate the origins of their ideas by referencing sources used in their work. • Assessment criterion: Accurate use of the standard referencing styles within the text for all sources used. 6.3 Challenges for Belarus One of the greatest challenges for Belarus, and for many other Bologna Process countries, is to ensure that learning outcomes do not simply involve an exercise on paper but rather that the introduction of a Learning Outcomes Framework becomes a reality and is embedded in teaching, learning, and assessment in higher education. The main indicators for that may be summarised as follows: • Showing that the National Qualifications Framework is compatible with the Framework of Qualifications of the European Higher Education Area and the European Qualifications Framework • Introducing Learning Outcomes and describing modules and programmes in terms of module Learning Outcomes and Programme Learning outcomes, that is, getting the paperwork in order as evidence of this Learning Outcomes Framework • Expressing workload in terms of ECTS credits and credit accumulation • Providing evidence that the Learning Outcomes have been achieved • Providing evidence that the Learning Outcomes, Teaching and Learning Activities, and Assessment are aligned (constructive alignment). 6.4 Potential Problems with Learning Outcomes One of the main concerns about the adoption of learning outcomes is the philosophical one that academic study should be open-ended and that learning outcomes do not fit in with this liberal view of learning (Adam 2004). This need not be the case if learning outcomes are written according to the standard guidelines and implemented within a learning outcomes framework in which the learning outcomes, the teaching and earnings activities, and the assessment are aligned with each other. However, if learning outcomes are written within a narrow framework, this could limit learning and result in a lack of intellectual challenge to students. Other potential problems are: • There is a danger of an assessment-driven curriculum if learning outcomes are too confined. • Learning outcomes could create confusion among students and staff if guidelines are not adhered to when drawing up these learning outcomes. 74 6.5 Concluding Points We have seen that international trends in education show a movement away from the emphasis on a ‘teacher- centred’ approach to a more ‘outcome-based’ approach to education. This movement has gained increased momentum from the Bologna Process, with its emphasis on student-centred learning and the need to have more precision and clarity in the design and content of curricula. Clearly, learning outcomes play a key role in ensuring transparency of qualifications and qualification frameworks, and also in contributing to the implementation of the various action lines of the Bologna Process throughout the European Higher Education Area. The requirement to make the teaching and learning process more transparent and more explicit presents a challenge to all of us involved in education. In the short term, we must prepare for the immediate challenge of expressing our modules and programmes in terms of learning outcomes. In the longer term, the adoption of the learning outcomes approach has the potential to help us embrace a more systematic approach to the design of programmes and modules. It is hoped that this handbook will help us all rise to the challenges posed by the Bologna Process and help us to integrate learning outcomes into our teaching activities, learning activities, and assessment tasks. 75 Appendix 1. Glossary Account for: Clarify, explain, give reasons for a statement. Aim: The aim of a module or programme is a broad general statement of teaching intention, that is, it indicates what the teacher intends to cover in a block of learning. Analysis: The ability to break down information into its components, for example, to look for interrelationships and ideas (Bloom’s Taxonomy). Application: The ability to use learned material in new situations, for example, put ideas and concepts to work in solving problems (Bloom’s Taxonomy). Assess: Evaluate, weigh up. Assessment: The total range of written, oral, and practical tests, examinations, projects, and portfolios that are used to evaluate the student’s progress in the unit or module of the programme. Assessment criteria: Descriptions of what the student is expected to do to demonstrate that a learning outcome has been achieved. Bologna Process: The attempt to create the European Higher Education Area by harmonising academic degree standards and quality assurance standards throughout Europe. Compare: Look for similarities between two areas. Compare and contrast: Look for some points in common between two areas and show where or how they differ. Competences: ‘Competences represent a dynamic combination of attributes, abilities, and attitudes. Fostering these competences is the object of educational programmes. Competences are formed in various course units and assessed at different stages. They may be divided in subject-area-related competences (specific to a field of study) and generic competences (common to any degree course)’ (ECTS Users’ Guide 2005). Comprehension: The ability to understand and interpret learned information (Bloom’s Taxonomy). Constructive alignment: The designing of the curriculum so that the learning outcomes are coordinated with the teaching and learning activities and with assessment. Continuous assessment: Assessment that takes place within the normal teaching period and contributes to the final assessment. Contrast: Look for differences between two areas. Coursework: The tasks that are required by a module or unit of a programme. Credit: The unit used in the European Credit Transfer System (ECTS). ECTS credits are used to measure student workload in terms of time. Credit accumulation: The system in which a specified number of credits must be obtained to successfully complete a programme. 76 Critique: Using evidence or reasoning, make a judgment about the merits of an argument, theory, or opinion. Define: State the exact meaning of a word or phrase. Describe: Give a detailed account of a topic. Diploma Supplement: An annex to the official qualification, issued in a standard international format. It provides a description of the nature, level, context, content, and status of the studies that were pursued and successfully completed by the holder of the qualification. Discuss: Explain and present various sides of an issue and any implications. Distinguish (differentiate): Determine the difference between. European Qualifications Framework (EQF): A common European reference framework the purpose of which is to make qualifications more readable and understandable across different countries and systems. Covering qualifications at all levels and in all subsystems of education and training, the EQF provides a comprehensive overview of qualifications in all European countries currently involved in its implementation. Evaluate: The ability to judge the value of material for a given purpose (Bloom’s Taxonomy). Examine: Look in detail at a particular area. Explain: Provide details about a point or area of discussion. Formative assessment: A type of assessment that helps inform the teacher and students as to how the students are progressing. Formative assessment is usually carried out at the beginning of or during a programme. The intention is that the feedback the students receive from the teacher helps improve their learning Grading criteria: Indications of what a student must demonstrate to achieve a higher grade than the minimum required to pass a module. Identify: Recognise and state the existence of a point of argument or concept. Illustrate: With the aid of examples, clarify a particular area of discussion. Integrated (master) programs: Four-year (or five-year) master programs, mostly in STEM (Science, Technology, Engineering and Mathematics) subjects, often with a research project or dissertation in the last year. An integrated master’s degree usually includes a year of study at the master’s level and three or four years at the bachelor’s level. Integrated master’s degrees are regarded as the master’s-level qualifications and constitute postgraduate qualifications within the European Higher Education Area. Joint programs: Double-degree programs, sometimes called dual-degree programs, combined degree programs, joint or simultaneous degree programs, double graduation degree programs, or other names, which imply student learning to obtain two university degrees (diplomas, programs) simultaneously – in the same university or in different universities (sometimes in different countries) – and to complete them in a shorter period of time than it would have taken to obtain them separately. Two degrees can be in one subject area (especially when the course is divided between countries) or in two different subject areas. Justify: Explain the grounds for making decisions or drawing conclusions and formulate answers to the main objections likely to be made. 77 Knowledge: The ability to recall or remember facts without necessarily understanding them (Bloom’s Taxonomy). Learning outcomes: Statements of what a learner is expected to know, understand, and/or be able to demonstrate after completion of a process of learning. Module: A self-contained fraction of a student’s programme workload for the year (or semester) with a unique examination and a clear set of learning outcomes and appropriate assessment criteria. Objective: A specific statement of teaching intention of a module or programme which indicates one of the specific areas the teacher intends to cover in a block of learning. Outline: Provide the main features or general principles of a topic, leaving out minor details and emphasising the structure and arrangement of the topic. Process: A series of actions, changes, or functions that bring about a result. Qualification: Any degree, diploma, or other certificate issued by the competent authority to verify that concrete learning outcomes were achieved after the successful completion of a reputable higher education program. Qualification framework: A formalized structure in which learning-level descriptors and qualifications are used in order to understand learning outcomes. This structure allows for the ability to develop, assess, and improve quality of education in a number of contexts. Qualifications frameworks are typically found at the regional, national, and international level. Rubric: A grading tool used to describe the criteria for grading the performance of students. In general, a rubric consists of a set of criteria and marks or grades associated with these criteria, thus providing a clear guide as to how students’ work will be assessed. State: Present in a brief, clear form without any explanation being needed. Summarise: Present a concise, clear explanation or account of the area. Give the main points and leave out minor details. (Similar to Outline, above.) Summative assessment: An assessment that tries to summarise student learning at some point in time – usually the end of a module or course – which enables a grade to be generated that reflects the student’s performance. Synthesis: The ability to put separate parts or ideas together (Bloom’s Taxonomy). Taxonomy: A classification based on certain criteria. 78 Appendix 2. Examples of Learning Outcomes from Selected Modules in University College Cork (Modules are arranged in alphabetical order according to module code) Module Title: Systems Biology Module Code: BL6005 Learning outcomes written by Dr Siobhán O’Sullivan On successful completion of this module students should be able to: • Define the characteristics of cancer cells • Distinguish between a proto-oncogene and an oncogene using an example of each, and illustrate the role they both play in the development of cancer • Illustrate the cell cycle and differentiate the different stages which occur throughout • Compare the lac and trp operons in terms of function and regulation • Discuss the importance of gene regulation in bacterial systems using appropriate examples. Module Title: Team Software Project Module Code: CS3305 Learning outcomes written by Professor John Morrison On successful completion of this module students should be able to: • Define a Project Management Plan • Tabulate Testing Plans • Contrast alternative implementation procedures • Discuss contingency plans • Construct working code and test implemented code • Schedule tasks to achieve goals • Design computer-human interfaces • Assess project outcomes with respect to initial stated requirements • Differentiate between good and bad computer-human interfaces in terms of the physical interactions and ergonomics required by the program user to achieve the desired result. 79 Module Title: DS4004 Module Code: Introduction to Dental Surgery Learning outcomes written by Dr Christine McCreary On successful completion of this module, students should be able to: • Obtain and record accurate and comprehensive medical/dental and social histories from patients exercising judgement in relation to the questions and demonstrating empathy and communication skills with the patient • Interpret the significance of the history and develop appropriate treatment plans and differential diagnoses, being cognisant of patients’ needs and wishes • Deliver effective local anaesthesia in the mandible and maxilla and identify the appropriate agents that may be used. Perform at least 10 local anaesthetic administrations • Identify the appropriate armamentarium and techniques for simple extractions in the maxilla and mandible. Perform at least 10 extractions • Summarise the different intra-oral radiographic techniques employed in general dental practice. Take at least 10 radiographs and evaluate them with your instructor • Demonstrate proficiency in CPR (cardiopulmonary resuscitation). Module Title: Dental Surgery – 5th Year Dental Students Module Code: DS5001 Learning outcomes written by Dr Eleanor O’Sullivan On successful completion of this module, students should be able to: • Master the skills required to record a thorough case history, deliver health promotion advice, and obtain informed consent dealing with medico-legal issues • Summarise relevant information regarding the patient’s current condition to generate a differential diagnosis • Formulate an appropriate treatment plan and justify the proposal giving due consideration to patient expectations and limitations • Arrange appropriate tests and demonstrate the ability to interpret tests and reports • Administer local anaesthetics safely and perform basic dento-alveolar surgical procedures in a professional manner showing good clinical governance • Recognise, evaluate, and manage medical and dental emergencies appropriately • Differentiate between patients that can and cannot be safely treated by a General Dental Practitioner • Manage competing demands on time, including self-directed learning and critical appraisal 80 • Master the therapeutic and pharmacological management of patients with facial pain and oro-facial disease. Module Title: Introducing Economics Module Code: EC1102 Learning outcomes written by Dr Noel Woods On successful completion of this module, students should be able to: • Recognise the main indicators of stock market timing • Describe and distinguish between the main economic indicators • Interpret Irish National Income and Expenditure Accounts • Differentiate between monetary and fiscal policy • Perform economic calculations, which enable the learner to appreciate economic concepts with greater clarity • Critique budgetary decisions using economic criteria • Construct and interpret company accounts and accounting ratios • Formulate appropriate budgetary policy in response to changes in the business cycle • Assess the stance of government fiscal policy. Module Title: History of Irish Education Module Code: ED401 Learning outcomes written by Dr Tracey Connolly On successful completion of this module students should be able to: • Describe the Irish education system and its evolution • Discuss enduring features of education in the period studied • Apply the skills of a historian in analysing the past • Analyse how different administrations approached education • Propose approaches that could have been made to the development of education in the context of the time • Recognise the problems that confronted the evolution of Irish education • Summarise the causes and effects of specific developments in the history of Irish education. 81 Module Title: General Practice Module Code: GP4001 Learning outcomes written by Professor Colin Bradley On successful completion of this module students should be able to: • Develop a rapport with patients such that patients are at ease in discussing their health problem(s) • Gather appropriate information on the patient’s health problem(s) including information on the patient’s own perspective on the problem(s) • Generate a reasonable range of diagnostic possibilities for undifferentiated medical problems presented by patients • Investigate these diagnostic possibilities using appropriately focused history taking and selective physical examination • Construct a general model for the safe and effective management of patients with multiple and long- term health problems • Adapt this model to the long-term health problems commonly encountered by doctors • Construct an appropriate and feasible management plan to deal with the physical, psychological, and social aspects of patient’s problem(s) • Negotiate this plan with the patient. Module Title: Politics of the European Union Module Code: GV1202 Learning outcomes written by Dr Mary C. Murphy On successful completion of this module students should be able to: • Explain the logic for the creation of the European Union • Describe the difference between supranationalism and intergovernmentalism • Critique the structure and operation of key EU institutions • Evaluate the political and economic impact of selected EU policies • Predict the future shape and nature of the EU • List the advantages and disadvantages of Irish membership of the EU • Research and present information effectively and comprehensively • Question the meaning of and motivation for political developments generally 82 • Analyse information creatively and imaginatively in seeking solutions to political problems and challenges. Module Title: Penology Module Code: LW545 Learning outcomes written by Dr Shane Kilcommins On successful completion of this module students should be able to: • Differentiate between criminal law as paper rules and criminal law in action • Outline and trace changes in punishment over time • Identify the determinants which shape punishment in late modern society • Employ different theoretical approaches to criminal law phenomena • Examine the extent to which such theories can explain occurrences in late modern Irish society • Interpret Irish criminal law cases, statutes, and policy recommendations in socio-legal terms • Connect changing values and sentiments in punishment with a changing emphasis on criminal law and procedure • Assess current criminal justice policies in terms of direction and impact (as it relates to accused, victims, agencies, and politicians) • Question the extent to which criminal law really is objective and value-free in orientation. Module Title: Environmental Microbial Genomics: the role and ecology of microbes in the environment Module Code: MB3005 Learning outcomes written by Dr John Morrissey On successful completion of this module students should be able to: • Outline the major classes of microbiota present in natural ecosystems • Explain how the physical, chemical, and biological environment influences microbial activity • Describe, using examples, different types of ecological interactions involving microbes • Explain the general importance of microbial communities for ecosystem function • Describe, compare, and contrast the methods that can be applied to study microbes and microbial communities in the environment • Explain, using examples, how diverse methods can be applied to understand microbial function in natural ecosystems. 83 Module Title: Methods in Microbiology Module Code: MB3016 Learning outcomes written by Carmel Shortiss On successful completion of this module students should be able to: • Identify the steps required to complete each experiment individually and in a group • Define the individual steps required to complete the experiment • Arrange the steps in sequence so that the experiment can be completed • Organise the reagents, cultures, media, etc. that are required for the experiment • Maintain a clear scientific record of each experiment and the data generated from the experiment in a laboratory notebook • Evaluate the data received individually and in a group discussion • Identify, individually and in a group discussion, the conclusions that can be drawn from the data • Present the completed experiment in a written report • Present the completed experiment in an oral report, identifying the outcomes of each of the steps above and paying particular attention to the conclusions. Module Title: The Child Health Research Project Module Code: PC2007 Learning outcomes written by Dr Aileen Malone On successful completion of this module students should be able to: • Define research, differentiate between good- and poor-quality research using appropriate criteria • Formulate an original research question • Perform a comprehensive literature review • Identify and access bibliographical resources, databases, and other sources of relevant information. Appreciate the strengths and limitations of different methods of data collection • Gather original data using the most appropriate research methodology • Interpret research findings • Communicate their research results appropriately in both written and verbal presentation. 84 Module Title: Applied Thermodynamics and Fluid Mechanics Module Code: PE3001 Learning outcomes written by Dr Edmond Byrne On successful completion of this module students should be able to: • Assess any pipeline system with respect to pressure differentials and fluid flow rates and design a pump-pipeline system for laminar or turbulent, single or multiphase flow of Newtonian or non- Newtonian fluid through straight, branched, or networked pipe systems • Select pumps appropriate for the range of process types encountered in the process industries • Categorise different rudimentary flow systems so as to employ Navier-Stokes equations which in turn describe these systems. Also demonstrate how these equations can be applied to more complex systems using Computational Fluid Dynamics software • Outline the nature of high-velocity compressible flow and design a choked flow. Module Title: Unit Operations and Particle Technology Module Code: PE3002 Learning outcomes written by Dr John Fitzpatrick On successful completion of this module students should be able to: • Measure particle and powder properties, in particular, particle size, moisture content, particle density, bulk density, flowability/wall friction • For each operation covered: − Explain how each operation functions − Identify the major variables that determine capital and operating costs − Perform design calculations, where possible, to evaluate these variables. Where not possible, describe how to go about estimating these variables − Perform experiments / small-scale trials to evaluate values of parameters used in design calculations (drying, filtration, fluidisation, silo design) • Discuss how particle properties and bulk powder properties influence production, storage, handling, separation, and processing of particulate systems. Module Title: Bioactive Natural Products – Pharmacognosy Module Code: PS3005 Learning outcomes written by Dr Barbara Doyle-Prestwich 85 On successful completion of this module students should be able to: • Identify the different classes of plant secondary metabolites • Explain the significance of the occurrence of secondary metabolites in different taxonomic groups, in different plant parts, and in different locations around the world (the influence of the environment) • Discuss the ethics involved in harvesting drugs from endangered plant species and the obligation on the harvester to local communities in that area • Design protocols for the up-regulation of secondary metabolites in plants using a biotechnological approach • Conduct laboratory experiments on phytonutrients (using titrations and chromatography) (and also using a genetic engineering approach for the modification of medicinally important plants) • Write a 3,000-word review article on a chosen relevant topic using published literature • Distinguish between the different available strategies for the ethical and sustainable use of natural resources • Use the information gleaned from the industrial visits to more fully evaluate the implementation of quality control systems for the extraction of natural plant products on an industrial scale • Defend the use of rare plant species for human therapeutic purposes. Module Title: Restorative Dentistry Module Code: RD3003 Learning outcomes written by Dr Edith Allen On successful completion of this module, students should be able to: • Examine a patient extra-orally and intra-orally • Formulate an appropriate treatment plan based on an understanding of the disease process present and a prediction of the likely success • Identify dental caries and restore a tooth to functional form following caries removal • Record an accurate impression of the mouth and identify all anatomical features of importance • Design a partial denture with appropriate support and retention • Administer successfully and in a safe manner with minimal risk to patient and operator, infiltration and regional nerve block anaesthesia • Communicate with patients and colleagues in an appropriate manner. 86 Module Title: Comparative and Animal Physiology Module Code: ZY3011 Learning outcomes written by Dr Sarah Culloty On successful completion of this module students should be able to: • Describe the main components of the immune system • Compare the immune system of vertebrates and invertebrates • Contrast the immune system of vertebrates and invertebrates • Evaluate the effectiveness of the different mechanisms used by parasites to evade the host’s response • Demonstrate the practical skills required to detect and measure components of the immune system • Defend the findings obtained in the laboratory practical in a scientific paper. 87 References Adam, S. 2004. Using Learning Outcomes: A consideration of the nature, role, application and implications for European education of employing learning outcomes at the local, national and international levels. Report on United Kingdom Bologna Seminar, July 2004, Herriot-Watt University, Edinburgh. Allan, J. 1996. ‘Learning outcomes in higher education.’ Studies in Higher Education 21 (10): 93–108. Anderson, L. W., and D. Krathwohl, eds. 2001. A Taxonomy for Learning, Teaching and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. New York: Longman. Biggs, J. 2003a. Teaching for Quality Learning at University. Buckingham: Open University Press. Biggs, J. 2003b. ‘Aligning teaching and assessing to course objectives.’ Teaching and Learning in Higher Education: New Trends and Innovations, University of Aveiro, Aveiro, Portugal, April 13–17, 2003. Bingham, J. 1999. Guide to Developing Learning Outcomes. The Learning and Teaching Institute, Sheffield Hallam University. Sheffield: Sheffield Hallam University. Black, P., and D. William. 1998. ‘Inside the Black Box: Raising Standards through Classroom Assessment.’ Kings College, London. Bloom, B. S. 1975. Taxonomy of Educational Objectives, Book 1 Cognitive Domain. New York: Longman Publishing. Bloom, B. S., M. D. Engelhart, E. J. Furst, W. Hill, and D. Krathwohl, D. 1956. Taxonomy of educational objectives. Volume I: The cognitive domain. New York: McKay. Bloom, B. S., B. B. Masia, and D. R. Krathwohl. 1964. Taxonomy of Educational Objectives Volume II: The Affective Domain. New York: McKay. Boam, R., and P. Sparrow, eds. 1992. Designing and achieving competency. London: McGraw-Hill. Boon, J., and M. van der Klink. 2002. Competencies: The triumph of a fuzzy concept. Academy of Human Resource Development Annual Conference, Honolulu, Hawaii, Feb 27–March 3. Proceedings Vol. 1: 327–334. British Columbia Institute of Technology. 1996. Writing Learning Outcomes. British Colombia, Canada: Learning Resources Unit. Brown, R. B. 1993. ‘Meta-competence: a recipe for reframing the competence debate’. Personnel Review 22 (6): 25–36. Brown, R. B. 1994. ‘Reframing the competency debate: management knowledge and meta-competence in graduate education’. Management Learning 25 (2): 289–99. Brown, S. 1999. ‘Institutional Strategies for Assessment’. In Assessment Matters in Higher Education, edited by S. Brown and A. Glasner. Buckingham: SRHE and OU Press. Brown, S., and P. Knight. 1995. Assessing Learners in Higher Education. London: Kogan. 88 Burgoyne, J. 1988a. Competency Based Approaches to Management Development. Lancaster: Centre for the Study of Management Learning. Capel, S., M. Leask, and T. Turner. 1997. Learning to Teach in the Secondary School. London: Routledge. CEDEFOP (European Centre for Vocational Training). 2009. The Shift to Learning Outcomes: Policies and Practices in Europe. Luxembourg: Office for Official Publications of the European Communities. Chambers, D. W. 1994. ‘Competencies: a new view of becoming a dentist.’ J Dent Education (58): 342–345. Cockerill, T. 1989. ‘The kind of competence for rapid change’. Personnel Management September: 52–56. Dave, R. H. 1970. Developing and Writing Behavioural Objectives, edited by R. J. Armstrong. Tucson, Arizona: Educational Innovators Press. Dawson, W. R. 1998. Extensions to Bloom’s Taxonomy of Educational Objectives. Sydney, Australia: Putney Publishing. Department for Education and the Welsh Office. 1992. Circulars 9/92 and 35/92: Initial Teacher Training (Secondary phase). Department for Education and the Welsh Office, London and Cardiff. Dillon, C., and L. Hodgkinson. 2000. ‘Programme specifications in a flexible, multidisciplinary environment’. Quality Assurance in Education 8 (4): 203–10. Donnelly, R., and M. Fitzmaurice. 2005. ‘Designing Modules for Learning.’ In Emerging Issues in the Practice of University Learning and Teaching, edited by G. O’Neill, S. Moore, and B. McMullin. Dublin: All Ireland Society for Higher Education (AISHE). Dooley, K. E., J. R. Lindner, L. M. Dooley, and M. Alagaraja. 2004. ‘Behaviorally anchored competencies: evaluation tool for training via distance.’ Human Resource Development International 7 (3): 315–332. ECTS Users’ Guide. 2005. Brussels: Directorate-General for Education and Culture. https://portal.mytum.de/archiv/kompendium_rechtsangelegenheiten/bologna- prozess/ects_user_guide_brussels_14_02_2005/file_view?. ECTS Users’ Guide. 2015. Brussels: Directorate-General for Education and Culture. https://ec.europa.eu/education/ects/users-guide/docs/ects-users-guide_en.pdf. Elkin, G. 1990. ‘Competency-based human resource development’. Industrial and Commercial Training 22 (4): 20–25. EU (European Union) Commission. 2008. European Qualifications Framework for Lifelong Learning. Luxembourg: Office for Official Publications of the European Communities. Ferris, T., and S. Aziz. 2005. ‘A psychomotor skills extension to Bloom’s Taxonomy of Education Objectives for engineering education.’ Exploring Innovation in Education and Research Tainan, Taiwan 1–5 March 2005. Fry, H., S. Ketteridge, and S. Marshall. 2000. A Handbook for Teaching and Learning in Higher Education. London: Kogan Page. 89 Gibbs, A., D. Kennedy, and A. Vickers. 2012. ‘Learning Outcomes, Degree Profiles, Tuning Project and Competences.’ Journal of the European Higher Education Area 15 (5): 71–87. Gosling, D., and J. Moon. 2001. How to use Learning Outcomes and Assessment Criteria. London: Southern England Consortium for Credit Accumulation and Transfer (SEEC) Office. Harden, R. M. 2002a. ‘Developments in outcome-based education’. Medical Teacher 24 (2): 117–120. Harden, R. M. 2002b. ‘Learning outcomes and instructional objectives: is there a difference?’. Medical Teacher 24 (2): 151. Harden, R. M., J. R. Crosby, and M. H. Davis. 1999a. ‘Outcome-based education: Part 1 – An Introduction to outcome-based education’. Medical Teacher 21 (1): 7–14. Harden, R. M., J. R. Crosby, and M. H. Davis. 1999b. ‘Outcome-based education: Part 5 – From competency to meta competency: a model for the specification of learning outcomes.’ Medical Teacher 21 (6): 546– 552. Harrow, A. 1972. A taxonomy of the psychomotor domain – a guide for developing behavioral objectives. New York: David McKay. Hartel, R. W., and E. A. Foegeding. 2004. ‘Learning: Objectives, Competencies, or Outcomes’. Journal of Food Science Education (3): 69–70. Hartle, F. 1995. How to re-engineer your Performance Management Process. London: Kogan Page. Hendry, C., M. B. Arthur, and A. M. Jones. 1995. Strategy through People: Adaptation and Learning in the Small- Medium Enterprise. London: Routledge. HETAC (Higher Education and Training Awards Council). 2006. Explanatory Guidelines on the Direct Application to HETAC for a Named Award. Dublin: Higher Education and Training Awards Council. Jarvis, P. 1985. The sociology of adult and continuing education. London: Croom Helm. Jenkins, A., and D. Unwin. 2001. How to write learning outcomes. https://www.ubalt.edu/cas/faculty/faculty- matters/How%20to%20write%20student%20learning%20outcomes.pdf. Kennedy, Declan. 2007. Writing and using learning outcomes: a practical guide. Cork: University College Cork. https://cora.ucc.ie/handle/10468/1613. Kennedy, D. 2008. Linking Learning Outcomes and Assessment of Learning of Student Science Teachers. Science Education International 19 (4): 375–385. Kennedy, D., and M. McCarthy. 2016. ‘Learning Outcomes in the ECTS Users’ Guide 2015 – Some Areas of Concern’. Journal of the European Higher Education Area 16 (3): 93–106. Kennedy, D., A. Hyland, and N. Ryan. 2006. ‘Writing and using Learning Outcomes’. Bologna Handbook, Implementing Bologna in your Institution C3.4–1: 1–30. Luxembourg: Office for Official Publications of the European Communities. Kennedy, D. 2006. Writing and Using Learning Outcomes: A Practical Guide. Cork: Quality Promotion Unit, University College Cork. https://cora.ucc.ie/handle/10468/1613. 90 Kennedy, D., A. Hyland, and N. Ryan. 2009. ‘Learning Outcomes and Competences’. Bologna Handbook, Introducing Bologna Objectives and Tools B2.3–3: 1–18. Brussels: European University Association. Krathwohl, David, R. 2002. ‘A Revision of Bloom’s Taxonomy: An Overview’. Theory into Practice 41 (4). Mager, R. F. 1975. Preparing Instructional Objectives. Belmont, California: Fearon Publishers. Mager, R. F. 1984. Preparing instructional objectives. 2nd ed. Belmont, California: Pitman Learning. McBeath, G. 1990. Practical Management Development: Strategies for Management Resourcing and Development in the 1990s. Oxford: Blackwell. McLean, J., and P. Looker. 2006. University of New South Wales Learning and Teaching Unit. Messick, S. 1975. ‘The standard problem: meaning and values in measurement and evaluation’. American Psychologist October: 955–966. Messick, S. 1982. Abilities and Knowledge in Educational Achievement Testing: The Assessment of Dynamic Cognitive Structures. Princeton, New Jersey: Education Testing Service. Miller, C., J. Hoggan, S. Pringle, and C. West. 1988. Credit Where Credit’s Due. Report of the Accreditation of Work-based Learning Project. Glasgow: Scottish Vocational Education Council (SCOTVEC). Mitrani, A., M. Dalziel, and D. Fitt. 1992. Competency Based Human Resource Management. London: Kogan Page. Moon, J. 2002. The Module and Programme Development Handbook. London: Kogan Page Limited. Morss, K., and M. Murray. 2005. Teaching at University. London: Sage Publications. Neary, M. 2002. Curriculum studies in post-compulsory and adult education. Cheltenham: Nelson Thornes. Oliver, R., H. Kersten, H. Vinkka-Puhakka, G. Alpasa, D. Bearn, I. Cema, E. Delap, P. Dummer, J. P. Goulet, T. Gugushe, E. Jeniati, V. Jerolimov, N. Kotsanos, S. Krifka, and G. Levy. 2008. ‘Curriculum structure: principles and strategy’. European Journal of Dental Education (12): 74–84. O’Neill, G. 2002. Variables that influence a teacher versus student-focused approach to teaching. University College Dublin, Centre for Teaching and Learning report. Sydney: University of New South Wales Press. Osters, S., and F. Tiu. n.d. ‘Writing Measurable Learning outcomes’. https://www.gavilan.edu/research/spd/Writing-Measurable-Learning-Outcomes.pdf. Purser, L., Council of Europe. 2003. Report on Council of Europe Seminar on Recognition Issues in the Bologna Process, Lisbon, 11–13 April 2002. Ramsden, P. 2003. Learning to Teach in Higher Education. London: Routledge. Shuell, T. J. 1986. ‘Cognitive conceptions of learning’. Review of Educational Research 56: 411–436. Simpson, E. 1972. The classification of educational objectives in the psychomotor domain: The psychomotor domain. Vol. 3. Washington, DC: Gryphon House. Smith, B. 1993. ‘Building managers from the inside out: competency based action learning’. Journal of Management Development 12 (1): 43–48. 91 Tate, W. 1995. Developing Managerial Competence: A Critical Guide to Methods and Materials. London: Gower. Titovich, I. 2018. Presentation at the World Bank workshop “Enabling evidence-based higher education policies for better employment: focus on learning outcomes and graduate tracking”, Minsk, October 8. Toohey, S. 1999. Designing Courses for Higher Education. Buckingham: SRHE and OU Press. UK Training Agency. 1989. ‘Development of Accessible Standards for National Certification Guidance: Note No. 1’. Sheffield Employment Department/Training Agency, Sheffield, England. Van der Klink, M. R., and J. Boon. 2003. ‘Competencies: The triumph of a fuzzy concept.’ International Journal Human Resources Development and Management, 3 (2): 125–137. Winterton, J., F. Delamare-Le Deist, and E. Stringfellow. 2005. Typology of knowledge, skills and competences: clarification of the concept and prototype. CEDEFORP: Tolouse. http://www.cedefop.europa.eu/files/3048_en.pdf. Wolf, A. 1989. ‘Can competence and knowledge mix?’ In Competency-based Education and Training, edited by J. W. Burke. Lewes: Falmer Press. Woodruffe, C. 1991. ‘Competent by any other name’. Personnel Management September: 30–33. 92 Websites 1. Details on Bologna Process: http://www.bologna.ie 2. Berlin Communiqué 2003: http://www.bologna.ie/_fileupload/publications/BerlinCommunique.pdf Bergen Communiqué 2005: http://www.bologna-bergen2005.no/Docs/00- Main_doc/050520_Bergen_Communique.pdf London Communiqué 2007. http://www.ehea.info/media.ehea.info/file/2007_London/69/7/2007_London_Communique_English_588 697.pdf Paris Communiqué. 2018. http://www.ehea.info/media.ehea.info/file/2018_Paris/77/1/EHEAParis2018_Communique_final_952771. pdf 3. American Association of Law Libraries: http://www.aallnet.org/prodev/outcomes.asp 4. University of New South Wales Learning and Teaching Unit: http://www.ltu.unsw.edu.au/content/course_prog_support/outcomes.cfm?ss=0 5. Quality Enhancement Committee, Texas A&M University, College Station, Texas, United States: http://www.tamu.edu/qep/documents/writing_outcomes.pdf 6. Tuning Educational Structures in Europe: http://tuning.unideusto.org/tuningeu/ 7. University of Central England Educational and Staff Development Unit: http://lmu.uce.ac.uk/OUTCOMES/UCE%20Guide%20to%20Learning%20Outcomes%202006.pdf / http://lmu.uce.ac.uk/outcomes/#4.%20What%20are%20the%20benefits%20of%20Learning%20Outcomes 8. The University of Manchester: http://www.cs.manchester.ac.uk/Study_subweb/Postgrad/ACS- CS/webpages/syllabus/acs/ACS_AIMS.php 9. Dr Ann Ledwith, University of Limerick: http://www.ucc.ie/en/SupportandAdministration/ServiceandAdministrativeOffices/QualityPromotionUnit/ LearningOutcomesConference/Presentations/DocumentFile,15075,en.pdf 10. Bridgewater State College: http://www.bridgew.edu/AssessmentGuidebook/chapter4.cfm#course_mapping 11. University of Wyoming: http://uwadmnweb.uwyo.edu/acadaffairs/assessment/Docs/Cap_2.doc 12. Rubrics – Monmouth University: http://its.monmouth.edu/FacultyResourceCenter/rubrics.htm 13. Bologna Declaration 1999: https://www.eurashe.eu/library/bologna_1999_bologna-declaration-pdf/ 93