Setting the level of assessment

The level of the qualification will be related to the European Qualification Framework (EQF) levels and will be determined by reference to the target population and the demands of progression. Currently the Open University approve an MAEL course at level 4 and this will be the basis for the application to Ofqual. Given the appropriate level selected, the criteria will be downloaded and used by the MD to inform the processes of qualification specification. The level of competing qualifications will also be a relevant consideration. Again this will be subject to on-going review, but any change will need to be ratified by all relevant regulatory bodies.

The CE will use the level descriptors provided in Ofqual (2015). Qualification and Component Levels: Requirements and guidance for all awarding bodies and all qualifications. Section 3.6 in the setting of competencies – dividing these into the categories of ‘knowledge and understanding’ and ‘skills’. He/she will survey the range of expected competencies and match these with the descriptors given. The competencies will then be phrased in language which is appropriate for learners at this level, and other informed readers, to be able to assess the expected knowledge, understanding and skills which will be developed before the qualification is awarded.

Other evidence may be taken into account when assigning a level, including agreed industry standards such as accreditation specifications, benchmark and content statements from within the industry and through comparisons with existing similar qualifications. Not all the components of a qualification have to meet all of the level descriptors and where a qualification includes a progression, the levels of the earlier parts will of necessity be lower. The descriptors set out what the typical holder of a qualification will know and be able to do, based on a best-fit approach, at the end of a course and in order to be awarded the qualification.

As part of the development process, the CE will complete the a table to ensure a reasonable spread of competencies against the descriptors. The table of level descriptors only shows levels 3 and 4, the former included to cover formative aspects of the qualification, the latter for summative assessment. At this point, MAEL is only applying for level 4 recognition from Ofqual. If other levels are applied for then the relevant parts of the descriptors specification will be applied.

No qualification from MAEL shall be assigned to more than one level. All separable components of a qualification will be set at the same level as the full qualification and distinct and different levels will not be assigned within a unit (albeit some formative tasks will be at a lower level to aid progression. All reasonable steps will be taken to ensure that the level of any component is consistent with the level of the whole qualification.

Should a qualification be seriously proposed which is different in type or content to any which have previously been made available, then Ofqual must be notified promptly by the nominated registered person.

 

Initial marking of assessments

 

For each qualification which MAEL makes available, it will ensure that it has in place effective arrangements to ensure that, as far as possible, the criteria against which learners’ performance will be differentiated are:

(a) understood by assessors and accurately applied, and

(b) applied consistently by assessors, regardless of the identity of the assessor, learner or Centre, and

(c) made available to learners in good time before an assessment is due to be submitted.

 

If MAEL offers an option as to assessment tasks which may be completed by a learner in an assessment, or as to which assessments which may be completed by the learner (including units), it will:

(a) undertake thorough investigation in order to reasonably conclude that there is no material inconsistency between the level of demand of optional tasks or assessments, and/or

(b) ensure that it is unlikely that any perceived inconsistency will prejudice a group of learners.

If these criteria are not met, or it appears that there is a reasonable prospect of that occurring, then MAEL will make a reasonable alteration to the criteria against which learners’ performance will be differentiated for the optional task(s) or assessment(s) so as to prevent that prejudice from occurring. Where such a reasonable alteration is MAEL will ensure that the alteration is applied uniformly in the marking of every task or assessment in relation to which a learner has taken that option.

 

Marking will solely reflect the extent to which learning outcomes (LOs) have been met.  Learners will receive written feedback on all formative assessments.  This will normally follow a 3-part format:

(a)  General comments focusing of points which are meritorious

(b)  Specific comments relating to identified features of the assessment task

(c)  Pointers for improvement in subsequent assessment.

Written feedback is not normally given for summative assessments.  In cases of failure, Centres may request (and be given) specific feedback to assist a learner who elects to re-take a component of the summative assessment.

 

Plagiarism will be checked for as part of the marking process.  If detected, it would normally mean a student is dismissed from their course (however, Centres may, at their own discretion, issue one final written warning.)   In order to avoid this, learners are advised to use the Harvard system of referencing in their work, and details must be provided about this by Centres to learners.  However, an alternative referencing system (provided it clearly identifies derivative material, and, ideally, enables the reader to trace its source) would be accepted as a defence against the charge of plagiarism (although a deficient referencing method my reduce the credit given for an assessment.).

 

Verification of levels of attainment

 

Where evidence generated by a learner in an assessment for a qualification made available by MAEL is marked by a teaching Centre, MAEL has in place clear and effective arrangements to undertake verification of the assessment. This verification will ensure that:

(a) the assessment remains fit for purpose, and

(b) the criteria against which learners’ performances are differentiated are being applied accurately and consistently by assessors in different Centres, regardless of the identity of the assessor, learner, or centre, and

(c) any such moderation which is undertaken ensures that MAEL is able to make any necessary changes to a Centre’s marking of evidence generated by a learner in an assessment, and

(d) make such a change wherever it is considered necessary in order to ensure that the assessment remains fit for purpose or that the criteria against which learners’ performance is differentiated are being applied accurately and consistently.

Should any verification process necessitate changing the results of an assessment, all assessments must be similarly altered if it is a case of defining standards.  If the change is due to specific circumstances, then all other learners must have their work checked to see if a similar situation applies, in wghich case their marks will also be adjusted.

Following a verification, a statistical analysis (in terms of means and standard errors of those means) will be produced for assessments.  The analysis will be used as the basis of a report into the consistency of assessment(s).  Where inconsistency is apparent, reasonable adjustment to marks of all learners will be made and the relevant learning outcomes checked for consistency and clarity.

 

N.B. Before setting a specified level of attainment for a qualification which it makes available, MAEL will have reviewed the specified levels of attainment set for:

(a) the qualification previously, and

(b) similar qualifications which are made available by MAEL, and

(c) similar qualifications made available by other awarding organisations,

and will have used the results of this monitoring to ensure that the specified level of attainment set for the qualification will promote consistency in measuring the levels of attainment of learners over time and between similar qualifications.

Evidence for this process may include some or all of the following:

* use of shared units with other AOs / participation in collaborative exercises

* expert judgements e.g. Bookmark item-centred analysis [Items in a test (or a subset of them) are ordered by difficulty (e.g., Item Response Theory b-parameters or Classical Test Theory p-values) from easiest to hardest. Subject-matter experts place a “bookmark” in the sequence where they think that the location of the cutoff score should be (“As the test gets harder, where would a participant on the boundary of the performance level not be able to get any more questions correct?”) This method can be used with virtually any question type (e.g., multiple-choice, multiple-response, matching, etc.)]

* review of grade thresholds and distributions from archive material

* external QA reports

Adjudication by regulators of levels of attainment

 

In any case where MAEL makes available a qualification and at least one other awarding organisation makes available a qualification which is viewed by the generality of users of qualifications as being a direct equivalent to that qualification; and each awarding organisation sets the specified level or levels of attainment for the equivalent qualification that they respectively make available; and Ofqual (or any other regulatory body) considers that a specified level of attainment set by an awarding organisation prevents the equivalent qualifications from indicating a consistent level of attainment (including, where Ofqual considers appropriate, over time), and  Ofqual or any other regulatory authority specifies, in writing, requirements in relation to a specified level of attainment for the qualification which is makes available (either for a particular assessment cycle or during a particular time period);

MAEL will ensure that, before the qualification is awarded for that assessment cycle or during that time period, the specified level of attainment for the qualification complies with the requirement(s) provided.

 

 Checking the sufficiency of evidence

 

 

MAEL will take all reasonable steps to ensure that the result of each assessment taken by a learner in relation to an MAEL qualification accurately reflects the level of attainment demonstrated by that learner in the assessment. This will involve checking that the marking of an assessment in relation to, and the awarding of; a qualification takes into account all admissible evidence generated by a Learner as part of that assessment.

 

Where MAEL sets rules as to the quantity or type of evidence generated by Learners which will be admissible in an assessment, it will ensure that:

(a)  the assessment guidelines make the rule(s) clear, and

(b)  the guidelines are provided to all learners in a reasonably timely fashion, and

(c)  the rule is applied to all learners taking the assessment (other than where any reasonable adjustments or special consideration require it to be altered).

 

MAEL sets specific rules as to how the final mark for a qualification will be calculated from marks for different assessments. It will ensure that:

(a) the qualification makes the rules clear, and

(b) the rule is applied to all learners taking the qualification (other than where any reasonable adjustments or special consideration require it to be altered).

Appendix One: General criteria for essays:

Formative essays are marked on a 5-point scale (with intermediate grades allowed).  These are described as follows:

The Major Criteria (usually averaged to give the final grade)

Clear Planning with Logical Argument.

•     * Material relates to task/title set

•     * Clarity of meaning

•     * Logical ordering of material

Comprehensiveness

•     * Coverage of significant points

•     * Explanation of conflicting viewpoints

•     * Additional material is presented over and above what is given in lectures

 Factual Accuracy

•    *  Accuracy in reporting other people’s ideas

•     * Accuracy in interpreting research

•     * Accuracy in the use of quotations

 Ideas Developed and Supported

•     * Ideas explained and developed

•     * Relevant research/reading/examples quoted in support

•     * Conclusion supported by body of essay

 Background Reading

•     * Breadth of relevant reading

•     * Background reading used appropriately

•     * Demonstration of ability to critically evaluate quality of material read

 The Minor Criteria (usually provide the upper limit for a grade)

 Correct Use of Written English

•     * Grammar (including punctuation) correct

•     * Spelling (including typos) correct

•     * Quotations correctly attributed and both references & bibliography laid out consistently in accordance with a recognised method

Excellent                   Work given this grade should not only be error free/complete etc but also have something extra – originality or significant additional material etc.

Normally at least 3 of the 5 major criteria on the marksheet should be ‘excellent’ and the other 2 ‘very good’ (or 4 ‘excellent’ and 1 ‘good’), although markers may use their discretion if a piece of work is outstanding in other ways..

The detailed criteria are:

•     Clear Planning with Logical Argument. – plan clearly apparent AND points follow on from each other in a logical order which relates to the title AND work is outstandingly clear.

•     Comprehensiveness – covers all the significant points AND (if appropriate) different points of view explained clearly AND provides additional relevant/correct material

•     Factual Accuracy – total 100% accuracy in reporting other people’s ideas, interpreting research & use of quotations AND there is a wide range of factual material

•     Ideas Developed and Supported – all ideas well developed AND supported by relevant quotations and/or personal examples and/or other reading AND essay comes to a conclusion clearly based on body of essay, being a legitimate deduction from that

•     Background Reading. – evidence of wide reading from books other than the set texts AND reading used with discretion AND clear evidence of awareness of the quality of both primary & secondary sources of information

•     Correct Use of Written English – Correct English grammar [including punctuation] AND spelling 100% AND no typing mistakes AND references / bibliography totally correct in every particular i.e. both references & bibliography laid out consistently in accordance with a widely recognised method

Please note: ‘excellent’ cannot be given for work submitted late, nor for work which does not display a high standard of written English.

 

Very Good                Better than ‘good’ but not up to ‘excellent’.

Normally at least 3 of the 5 major criteria on the marksheet should be ‘very good’ and the other 2 ‘good’ (or 4 ‘very good’ and 1 ‘satisfactory’), although markers may use their discretion if a piece of work is of a high standard in other ways.   Detailed criteria have not been laid out but the basis of the criteria is that the work should meet all of the ‘good’ criteria and some of the ‘excellent’ ones.

 

 Good                          Work given this grade should be at the pass standard of the first year of a B.A. or B.Ed.

Normally at least 3 of the 5 major criteria should be ‘good’ or better, although markers may use their discretion.   The detailed criteria are:

•     Clear Planning with Logical Argument. – meaning clear AND essay reasonably well planned AND content clearly relates to the title

•     Comprehensiveness – covers the majority of the significant points AND omissions are not crucial to the argument AND (if appropriate) shows an awareness of conflicting viewpoints

•     Factual Accuracy – not more than 2/3 minor errors AND there is a reasonable range of factual material

•     Ideas Developed and Supported – most ideas well developed AND supported by relevant quotations and/or examples AND essay comes to a conclusion based on body of essay

•     Background Reading. – evidence of some reading from books other than the set texts AND reading material used sensibly

•     Correct Use of Written English – substantially correct OR not more than 2/3 errors of grammar/spelling AND quotations correctly attributed with both references & bibliography laid out reasonably consistently

Please note:   neither a ‘very good’ nor a ‘good’ overall grade can be given if there are any ‘weak’ marks at all.

 

Satisfactory             This grade should be used when the work would not merit a ‘good’ grade, but is of a standard that would be accepted at other Montessori training centres.   The detailed criteria are:

•     Clear Planning with Logical Argument. – meaning clear enough AND some evidence of planning OR a logical progression is apparent, i.e. this is not just ‘stream of consciousness’ writing.

•     Comprehensiveness – covers a range of the significant points

•     Factual Accuracy – errors are not crucial (e.g. “Montessori advised the use of reward schemes” would be a critical error) AND any errors do not affect the overall argument

•     Ideas Developed and Supported – some ideas developed AND supported by relevant quotations

•     Background Reading. – evidence of reading from the set texts

•     Correct Use of Written English – not more than 5/6 errors of grammar/spelling (ignore typing errors for this grade unless they are overwhelming).). [N.B. repeated errors count once only] AND sufficient information is given to locate all quotations or references.

Please note: a ‘satisfactory’ grade cannot be given if more than one of the major criteria is marked as ‘weak’.

 

Resubmit                  Work is to be graded for resubmission if it does not meet the criteria for ‘satisfactory’.

Grading Criteria specific to each essay will be given out by the course tutor.   N.B. most assignments have a word limit (typically 1500 minimum to 2500 maximum) which will be printed on the sheet of specific criteria for each assignment.

 

Specific criteria are also provided for all essay and file assessment tasks.

Where there are options as to tasks which may be completed by a learner in an assessment, or as to assessments which may be completed by the learner (including units), MAEL will take steps to ensure that the tasks are of equivalent difficuylty and demand.  If MAEL reasonably concludes that there is a material inconsistency between the level of demand of two optional tasks or assessments, and it is likely that the inconsistency will prejudice a group of learners, then MAEL will make a reasonable alteration to the criteria against which learners’ performance will be differentiated for the optional task or assessment so as to prevent that prejudice from occurring.  If such a reasonable alteration is made for an optional task or assessment, MAEL will ensure that the alteration is applied uniformly in the marking of every task or assessment in relation to which any learner has taken that option.

Initial marks may be released to learners, within a specified time scale, but with the clear proviso that marks may be adjusted later during verification processes.  All assessment task marked by assessors employed directly by MAEL (e.g. essays and examinations) both internal and external verification will take place.   Internal verification will be undertaken, based on an annual statistical analysis of marks, produced by the chief examiner.  The key indicator for action is where either the mean or the standard error are significantly different from the usual range.   The internal verifier’s job description (see here) gives them some discretion as to the course of action to be recommended, but they are answerable to the board of directors.  The external verifer has the main role of ensuring standards overall are maintained.  He/she works to criteria set by the Open University, but will have regard to the policies of MAEL.

Specified forms will be provided for assessors.   As a general rule these are 3-part forms:

* general comments on commendable aspects of the submitted work (invariable)

* specific comments on aspects of the task.  Learner’s work will never be annotated but comments will be referred to page & para no.

* suggestions for improvement (always provided unless the top grade is given.)

Retention of Assessed Work

Formative assessments are returned to the learner (unless submitted electronically, when retention may depend on the parameters of the software) and is not retained by the centre, only the feedback maybe.  Summative assessments should be retained until there is no possibility of any appeal process being initiated (say 6 months) and in any case not more than 3 years (for data protection reasons).   An exception would be samples of work retained for monitoring and/or training purposes, but in such cases the work must be totally anonymitised.

 

Date of last review: 09/06/2016