Planning a Test and Construction of Table of Specifications

 

Steps in Planning a Test

1. Identifying test objectives
2. Deciding on the type of objective test to be prepared
3. Preparing a TOS
4. Constructing the draft test items
5. Try-out and validation

1. Identifying Test Objectives

• An objective test for it to be comprehensive, must cover the various levels of Bloom’s Taxonomy.

2. Deciding on the Type of Objective Test

• The test objectives dictate the kind of tests that will be designed and constructed

3. Preparing a Table of Specifications

Table of Specifications-a test map that guides the teacher in constructing a test

• Ensure balance between items that assess lower level thinking skills and higher level thinking skills, easy and difficult items

4. Constructing the test items

• It is advised that the actual number of items to be constructed in the draft double the desired number of items

5. Item analysis and try-out

• The test draft is tried out to a group of pupils to determine the:
• Item characteristics through item analysis
• Characteristics of the test itself-validity, reliability and practicality

Refresher Course for Licensure Examinations for Teachers: Assessment of Learning Part 4

 Properties of Assessment Methods

A. VALIDITY

• refers to the appropriateness, correctness, meaningfulness and usefulness of the specific conclusions that a teacher reaches regarding the teaching-learning situation
• the degree to which the assessment instrument measures what it intends to measure
• the most important criterion of a good assessment instrument

Types of Validity

1. Content-Validity- refers to the content and format of the instrument

• Do students have adequate experience with the type of task posed by the item?
• Did the teachers cover sufficient materials for most students to be able to answer the item correctly?
• Does the item reflect the degree of emphasis received during instruction?

2. Face Validity- refers to the outward appearance of the test

3. Criterion Validity- refers to the degree to which information provided by a test agrees with information obtained on other, independent test

4. Construct Validity- refers to the degree to which the totality of evidence obtained is consistent with theoretical expectations

B. RELIABILITY

• refers to the consistency, dependability or stability of an assessment method
• refers to the consistency of scores obtained by the same person when retested using the same instrument, its parallel or when compared with others students who took the same test

Methods of Checking Reliability

1. Test-Retest Method - involves administering the same instrument twice to the same group of individuals after a certain time interval has elapsed.

2. Equivalent-Forms Method- involves administering two different, but equivalent, forms of an instrument to the same group of individuals at the same time.

3. Test-retest with Equivalent Forms - involves giving parallel forms of tests with increased time interval between forms

4. Internal-Consistency Method- involves comparing responses to different sets of items that are part of an instrument (Split-half, Kuder-Richardson, Cronbach Alpha)

5. Scoring observer agreement- compare scores obtained by two or more scorers or observers

C. BALANCE

• Involves setting of targets in all domains of learning domains of intelligence
• Makes use of both traditional and alternative assessment

D. FAIRNESS

• involves student’s knowledge of the learning targets and assessment methods to be used; assessment of the learning process rather than the learning object; and freedom from biases
• the key to fairness are as follows:
• students have knowledge of learning targets and assessment
• students are given equal opportunity to learn
• students possess the pre-requisite knowledge and skills
• students are free from teacher stereotypes
• students are free form biased assessment tasks and procedures

E. PRACTICALITY and EFFICIENCY

• includes familiarity, time efficiency and employability of the assessment method 

a. Teacher familiarity with the Method

• The teacher should know the strengths and weaknesses of the method and how to use them

b. Time required

• Includes construction and use of the instrument and the interpretation of the results
• It is desirable to use the shortest assessment time possible that provides valid and reliable results

c. Complexity of the Administration

• Directions and procedures for administrations and procedures are clear and that little time and effort is needed

d. Ease of scoring

• use scoring procedures appropriate to your method and purpose

e. Ease of interpretation

• Interpretation is easier if there was a plan on how to use the results prior to assessment

d. Cost

• The less expense used to gather the information, the better

F. CONTINUITY

• Assessment takes place in all phases of instruction. It could be done before, during and after instruction

G. AUTHENTICITY

• Involves:

a. meaningful performance task

b. clear standards and public criteria

c. quality products and performance

d. positive interaction between assesse and assessor

e. emphasis on meta-cognition and self-evaluation

f. learning that transfers

H. COMMUNICATION

• Assessment targets and standards should be communicated

I. POSITIVE CONSEQUENCES

• Assessment should have positive consequence to students, that is it should motivate them to learn
• Assessment should have a positive consequence on teachers, that is, it should help them improve the effectiveness of their instruction

J. ETHICS 

• Participants in an assessment method should be protected from physical or psychological harm, discomfort or danger that may arise due to the testing procedure
• Test results and assessment results should be confidential and should be known only by the student concerned and the teacher
• There are instances in which it is necessary to conceal the objective of the assessment in order to ensure fair and impartial results. When this is the case, the teacher has a special responsibility to: 

a. determine whether the use of such techniques is justified by the educational value of the assessment, 

b. determine whether alternative procedures are available which do not make use of concealment; and 

c. ensure that students are provided with sufficient explanation as soon as possible.

Assessment should be fair and non-biased.

Refresher Course for Licensure Examinations for Teachers: Assessment of Learning part 3

 B. Appropriateness of Assessment Methods

1. Written-Response Instruments

• include objective tests (multiple-choice, true-false, matching or short answer) tests, essays, examinations, and checklists
• appropriate for assessing the various levels of the hierarchy of educational objectives

2. Product Rating Scales

• used to rate products such as book reports, maps, charts, diagrams, notebooks, essays, and creative endeavors
• to produce a product rating scale, the teacher must possess prototype products over his/her years of experience

3. Performance Tests

• one of the most frequently used performance tests is the checklist which consists of a list of behaviors that make up a certain type of performance

4. Oral Questioning

• appropriate when the objectives are to assess the student’s stock knowledge and to determine the student’s ability to communicate ideas
• several factors need to be considered when using this assessment which could mask the student’s true ability

5. Observation and Self Reports

• are useful supplementary assessment methods when used in conjunction with oral questioning and performance tests
• include tally sheets and self-checklist

Refresher Course for Licensure Examinations for Teachers: Assessment of Learning Part 2

 II. PRINCIPLES OF HIGH QUALITY ASSESSMENT

A. Clarity of Learning Targets

• Assessment can be made precise, accurate and dependable only if what are to be achieved are clearly stated. 
• Learning targets need to be stated in behavioral terms or terms which denote something which can be observed through the behavior of the student.

 Bloom’s Taxonomy of Educational Objectives

1. REMEMBERING

• Recall previous learned information.

Keywords:

defines, describes, identifies, knows, labels, lists, matches, names, outlines, recalls, recognizes, reproduces, selects, states.

Examples:

• Recite a policy. 
• Quote prices from memory to a customer. 
• Knows the safety rules.

2. UNDERSTANDING: 

• Comprehending the meaning, translation, interpolation, and interpretation of instructions and problems. 

Keywords:

comprehends, converts, defends, distinguishes, estimates, explains, extends, generalizes, gives an example, infers, interprets, paraphrases, predicts, rewrites, summarizes, translates.

Examples:

• Rewrites the principles of test writing. 
• Explain in one's own words the steps for performing a complex task. 

3. APPLYING: 

• Use a concept in a new situation or unprompted use of an abstraction. Applies what was learned in the classroom into novel situations in the workplace.

Keywords:

applies, changes, computes, constructs, demonstrates, discovers, manipulates, modifies, operates, predicts, prepares, produces, relates, shows, solves, uses.

Examples:

• Use a manual to calculate an employee's vacation time. 
• Apply laws of statistics to evaluate the reliability of a written test.

4. ANALYZING: 

• Separates material or concepts into component parts so that its organizational structure may be understood. Distinguishes between facts and inferences.

Keywords:

analyzes, breaks down, compares, contrasts, diagrams, deconstructs, differentiates, discriminates, distinguishes, identifies, illustrates, infers, outlines, 

Examples:

• Troubleshoot a piece of equipment by using logical deduction. 
• Recognize logical fallacies in reasoning. 

5. EVALUATING: 

• Make judgments about the value of ideas or materials.

Keywords:

appraises, compares, concludes, contrasts, criticizes, critiques, defends, describes, discriminates, evaluates, explains, interprets, justifies, relates, summarizes, supports.

Examples:

• Select the most effective solution. 
• Hire the most qualified candidate. 
• Explain and justify a new budget.

6. CREATING: 

• Builds a structure or pattern from diverse elements. Put parts together to form a whole, with emphasis on creating a new meaning or structure.

Keywords:

categorizes, combines, compiles, composes, creates, devises, designs, explains, generates, modifies, organizes, plans, rearranges, reconstructs, relates, reorganizes, revises, rewrites, summarizes, tells, writes.

Examples:

• Write a company operations or process manual. 
• Design a machine to perform a specific task. Integrates training from several sources to solve a problem. 

Refresher Course for Licensure Examinations for Teachers: Assessment of Learning

 I.  BASIC CONCEPTS IN ASSESSMENT

MEASUREMENT

• The use of tools or instruments to provide a quantitative description of the progress of students toward desirable educational goals
• A process of quantifying the degree to which someone/something possess a given trait

TEST OR TESTING

• A systematic procedure to determine the presence or absence of certain characteristics or qualities in a learner
• An instrument designed to measure any characteristics, quality, ability, knowledge or skills.
• Comprised of items in the area it is designed to measure

ASSESSMENT

• The process of interpreting the measurement data
• A process of gathering and organizing quantitative or qualitative data into an interpretable form to have a basis for judgment or decision making

EVALUATION

• Implies that measurement and assessment of an educational characteristic had been done and that it is now desired to pass a value judgment on the educational outcome by comparing results with predetermined criteria or objectives.
• Provides a tool for determining the extent to which educational process or program is effective and indicates directions for remediating processes of the curriculum that do not contribute to successful student performance

Types of Measurement

a) Objective measurement

• More stable since repeated measurements of the same quantity or quality of interest produce more or less the same outcome
• Do not depend on the person or individual taking the measurements

b) Subjective measurement

• Are used in certain facets of the quantity or quality of interest that cannot be successfully captured by objective measures
• Often differ from one assessor to the next even if the same quantity or quality is being measured

Forms/Kinds of Assessment

1. Traditional Assessment

• Refers the use of paper-and-pencil test

2. Alternative Assessment

• Refers to the use of methods other than pen-and-paper objective test which includes performance test, projects, portfolios, journals and the like.

3. Authentic Assessment

• Refers to the use of assessment methods that simulate true-to-life situations
• This could be objective tests that reflect real life situations or alternative methods that are parallel to what we experience in real life

Roles of Assessment

a) Diagnostic

• Determines the gaps in learning or learning processes and hopefully to be able to bridge these gaps
• Used to determine students’ recurring or persistent difficulties
• Searches for the underlying causes of student’s learning problems that do not respond to first aid treatment
• Helps formulate a plan for detailed remedial instruction

b) Formative 

• Allows the teacher to redirect and refocus the course of teaching a subject matter
• Teachers continuously monitor the student’s level of attainment of the learning objectives
• The results are communicated clearly and promptly to the students for them to know their strengths and weaknesses and the progress of their learning

c) Summative

• Determines the extent to which the learning objectives are met and why
• Used to certify what students know and can do and  the level of their proficiency or competency
• Reveals whether or not instructions have successfully achieved the curriculum outcomes
• Paves the way for educational reforms

d) Placement

• Determines the appropriate placement of a student both in terms of achievement and aptitude
• Assess the needs of the learners to have basis in planning for relevant instruction
• Used to know what the students are bringing into the learning situation and use this as a starting point for instruction