COURSE: Psych. 335L Psychological Testing Lab
CRN 11283 1 cr Fall
‘09
TIME/PLACE: Wed
Hour 11:00-11:50 Room
AMH 214
INSTRUCTOR: Dr.
Tom Hogan, Professor of Psychology
Office:
AMH 223, Tel: Office 941-4268
e-mail
Thomas.Hogan@Scranton.edu
Office
hours: Tu, Th 2:30-3:30 PM, Wed 10-11 AM
Other
times by arrangement.
Teaching Assistant: Simone Gilpin
REQUIRED
MATERIALS:
Text: Hogan, T. P. (2007). Psychological
testing: A practical introduction (2nd Ed.)
COURSE
OBJECTIVES
This lab provides hands-on experience in the development of tests. Students will develop, try out, conduct standard statistical analysis of, and revise two tests: one a measure of cognitive ability, the other a measure of attitude. Procedures for development of the two measures follow principles described in Chapter 6 and Appendix B of the textbook, as well as parts of Chapters 9 and 15. Students will work in teams of 3-4 when developing the tests.
Tryout
of the tests, followed by item analysis and creation of final forms of the
tests, will employ students in the Psychology department subject pool as well
as students in the lecture section of Psyc 335.
Students in the lab must have completed the CITI
(Collaborative IRB Training Initiative) Program.
Students
in the lab must be concurrently registered in (or have already completed) Psyc
335: Psychological Testing.
The
class meets for one hour per week. Most
of the lab work is completed outside of these meetings.
Specific skills to be developed include
the following:
Ø
Conceptualizing
a measurement target
Ø
Selecting
appropriate measurement format
Ø
Creation
of test items in cognitive and affective domains
Ø
Critiquing
and revision of test items
Ø
Appropriate
formatting of test materials
Ø
Use
of standard (classical test theory) item analysis procedures
Ø
Application
of reliability and validity analyses
Ø
Communication
of final results (test manual)
Ø
Working
cooperatively in a team effort
The
test to be developed in the cognitive domain will be either a verbal or
quantitative measure (team choice) of approximately 30 items and intended to
correlate highly with SAT Verbal or Math scores, which will be used as the
criterion measures for validity. The
instructor is the only one who will see SAT scores associated with student
names.
The
test in the affective domain will measure attitude toward some educational
practice, policy or procedure (team choice).
The measure will not deal with any personnel. The measure will use a Likert format, broadly
defined, including 15-30 items.
GRADING:
1. 20% Quiz
on Ch 6 and App B *
2. 20% Test
manual *
3. 20% Ratings
of team contributions *
4. 20% Ratings
of test quality by tryout examinees **
5. 20% Quality
points for reliability and validity **
* Individual grade. ** Same grade for all team members.
1. The quiz on Ch 6 and Appendix B consists
of 20 multiple-choice items.
2. Each participating student prepares a
brief test manual to be graded by the instructor.
3. Each student rates the quality of
contributions made by each team member.
4. Students completing the tryout forms of
tests rate the appropriateness of the tests.
5. A rubric for levels of reliability and
validity is applied to the results on these features.
ACADEMIC
HONESTY:
See
the University's policy on academic honesty.
A student found cheating or engaging in another form of academic
dishonesty will receive an F for the course.
REVISIONS AND ANNOUNCEMENTS:
The syllabus is subject to revision. Any
revisions will be announced in class. Note also that you will sometimes receive
e-mail announcements through Angel. Make sure you check your University e-mail
account.
COURSE
SCHEDULE Psych 335L
Week Tasks
1 Introduction. Review of syllabus. Formulation of goals. Creation of teams.
2 Review of steps [Chapter
6, Appendix B].
3 Test on Ch 6, Ap B. Background research on the areas to be
measured. Review parts of Chs 9, 15.
4 Decisions on test structures and item formats.
5 Prepare items.
6 Informal tryouts.
7 Critique items, formats.
Revise as needed.
8 Prepare tryout materials.
9 Tryout.
10 Item analysis. Reliability analysis. Validity analysis.
11 Reliability and validity analysis continued.
12 Prepare final instruments.
13 Prepare manuals.
14 Critique of all procedures.
Assessment/evaluation based course improvement mechanisms
At the conclusion of the
lab part of the course, students complete two Likert-based assessments. In the first, students rate their degree
of learning of each of the objectives listed in the syllabus. In the second, students rate the
appropriateness of amount of time devoted to each topic. (The lecture portion of the course features
parallel student ratings for topics covered in the lecture.) The formal ratings are also reviewed with
students in connection with the critique conducted in week 14.
Students’ success in
reaching targets specified in the rubrics for reliability and validity
of the tests constructed also provides an important feedback loop for
evaluating the course and designing improvements to the learning experience.