IPAT :: Choose a City : Introduction to Item-Specific Scoring

Introduction to Item-Specific Scoring

The following introduces forms and processes you can use to score students item by item, using the rubrics expounded in the Choose a City Item-Specific Scoring Guide, which can be downloaded in either Word of PDF formats at the Choose a City download page. The process involves:

Step A: using the Scoring Guide as your reference as you score students item-by-item, recording the scores on the scoring form described below

Step B: converting each score to a simple measure of adequate performance (e.g., whether the student's level of performance on the item was adequate)

Step C: aggregating the scores to obtain composite scores for the students by outcome area (e.g., technology use, reasoning with information, communication)

Step B provides a precise diagnosis of student proficiency. Step C allows you to aggregate performance across items. C is optional.

Use the Scoring Form to score students on Steps A and B. Use the Aggregation Form for Step C.

You may download the forms shown below in either Microsoft Word format or in Adobe Acrobat format. Click "download forms" in the links above to get to them.

Using the Scoring Form

Use the Scoring Form to score each individual student work. If the students are working in pairs or larger groups, you may use the same form for each student who contributed to the work, or you may use a separate form for each. Columns A-E provide item-by-item information about the scoring. Each row is for a different scoring instance. Some items need multiple scoring instances because they yield different evidences of different proficiencies. For example (see rows Q-U), item 5 has five scoring instances, which is another way of saying that it can be scored on five different dimensions. Here are steps to follow for each scoring instance (i.e., each row):

Step 1: Enter into column F the student's raw score. Before you score, consult the Item-Specific Scoring Guide to study the rubric for the scoring instance you are focusing on, and its illustrative examples of student work. Enter "M" for missing if the student did not attempt to answer.

Step 2: Enter into column G whether or not the student's score constitutes adequate performance -- "1" for yes, "0" for no; "M" for missing. This process of extrapolating an adequate score from the raw scores has been developed to permit aggregating scoring instances into composite scores for the three outcome areas (see the Aggregation Form below for more on this). Consult Column E to determine what the minimum score is for adequate performance. For example, the proficiency of finding relevant Web-based information (item 1 -- rows A, D, G, and J) is scored on a 4-point rubric but either a 3 or 4 constitutes adequate performance according to the bar that has been set. (Note: You may alter Step 2 by raising or lowering the bar on what constitutes adequate performance if you think that a different bar would be more appropriate for your student population.)

Item-by-Item Scoring Form

Student 1 Name __________________________________________ Student 2 Name __________________________________________ School __________________________________________ Date __________________________________________
	A		B	C	D	E		F	G
Row	Item-by-Item Scoring Information (for your reference)							Student Scores (fill these in)
	Item #	Outcome area		Proficiency component	Highest possible score		Minimum score for adequate performance	Score (M= missing)		Adequate Performance? (1=yes; 0=no; M=missing)
A	1a	Technology Use; Reasoning with Information		navigating Web to find relevant information	4		3
B	1b	Technology Use		citing URL	3		3
C	1c	Technology Use		citing URL	3		3
D	1d	Technology Use; Reasoning with Information		navigating Web to find relevant information	4		3
E	1e	Technology Use		citing URL	3		3
F	1f	Technology Use		citing URL	3		3
G	1g	Technology Use; Reasoning with Information		navigating Web to find relevant information	4		3
H	1h	Technology Use		citing URL	3		3
I	1i	Technology Use		citing URL	3		3
J	1j	Technology Use; Reasoning with Information		navigating Web to find relevant information	4		3
K	1k	Technology Use		citing URL	3		3
L	1l	Technology Use		citing URL	3		3
M	2	Reasoning with Information		formulating a research problem	3		2
N	2	Communication		expressing research problem in question format	3		2
O	3	Technology Use		formulating targeted Web search phrase	3		2
P	4	Reasoning with Information		critically evaluating Web-based textual information	4		3
Q	5	Reasoning with Information; Commun-ication		formulating and communicating argument (i.e., evidence-based conclusion from data)	4		3
R	5	Commun-ication		displaying adequate organization in written composition	4		3
S	5	Commun-ication		displaying correct mechanics in written composition	4		3
T	5	Technology Use *		using clip graphics in a composition developed on the computer	3		2
U	5	Technology Use *		manipulating productivity tool interface to meet formatting requirements for a composition	3		2

* (Assign these scores only if your students are writing their compositions on the computer using a productivity tool that you have provided for them, such as Word, PowerPoint, HyperStudio, or ClarisWorks. For more about this, click "Home" or "administering the assessment."

Using the Aggregation Form

Following the item-by-item scoring, use the Aggregation Form to consolidate scores into the three general outcome areas. For each outcome area, use the following procedure:

in column C, sum how many non-missing scores you can assign to the student (e.g., the number of items the student attempted)
in column D, sum the number of student's scores that constitute adequate performance
in column E, divide the number of scores you were able to assign (column C) into the number that were adequate (column D) to get a "percent adequate" score for the outcome area represented by the row
in the bottom row, at your option, aggregate across the three outcome areas to get a a total score and percentage (e.g., C4=C1+C2+C3; D4=D1+D2+D3), then divide C4 into D4 to get a total percentage, which you put into E4.

The "percent adequate" score for each outcome area (e.g., row) is the percentage of attempted tasks that the student performed adequately on.

Aggregation Form

Student 1 Name __________________________________________ Student 2 Name __________________________________________ School __________________________________________ Date __________________________________________
	A	B	C	D	E
Row	Outcome area (cells from the table are in parentheses)	Number of score-able instances if the student attempted to answer everything	Number of score-able instances on items the student has attempted to answer	Number of adequate scores	Percent of scores that are adequate
1	Technology use (Column G, Rows A, B, C, D, E, F, G, H, I, J, K, L, O, T, U)	15
2	Reasoning with information (Column G, Rows A, D, G, J, M, P, Q)	7
3	Communication (Column G, Rows N, Q, R, S)	4
4	TOTALS across outcome areas	26

Note: Some scores are aggregated in two outcome areas. For example, the quality of the argument provided in the student's composition is both a reasoning and communication proficiency.

Example of Scoring

The following is an example of how the forms can be used to score a student. All the data about the student are in italics in columns F and G.

Item-by-Item Scoring Form

Student 1 Name __________________________________________ Student 2 Name __________________________________________ School __________________________________________ Date __________________________________________
	A	B		C		D		E		F	G
Row	Item-by-Item Scoring Information (for your reference)									Student Scores (fill these in)
	Item #		Outcome area		Proficiency component		Highest possible score		Minimum score for adequate performance	Score (M= missing)	Adequate Performance? (1=yes; 0=no; M=missing)
A	1a		Technology Use; Reasoning with Information		navigating Web to find relevant information		4		3	2	0
B	1b		Technology Use		citing URL		3		3	3	1
C	1c		Technology Use		citing URL		3		3	3	1
D	1d		Technology Use; Reasoning with Information		navigating Web to find relevant information		4		3	2	0
E	1e		Technology Use		citing URL		3		3	2	0
F	1f		Technology Use		citing URL		3		3	M	M
G	1g		Technology Use; Reasoning with Information		navigating Web to find relevant information		4		3	4	1
H	1h		Technology Use		citing URL		3		3	3	1
I	1i		Technology Use		citing URL		3		3	3	1
J	1j		Technology Use; Reasoning with Information		navigating Web to find relevant information		4		3	3	1
K	1k		Technology Use		citing URL		3		3	2	0
L	1l		Technology Use		citing URL		3		3	3	1
M	2		Reasoning with Information		formulating a research problem		3		2	2	1
N	2		Communication		expressing research problem in question format		3		2	2	1
O	3		Technology Use		formulating targeted Web search phrase		3		2	3	1
P	4		Reasoning with Information		critically evaluating Web-based textual information		4		3	M	M
Q	5		Reasoning with Information; Communication		formulating and communicating argument (i.e., evidence-based conclusion from data)		4		3	3	1
R	5		Communication		displaying adequate organization in written composition		4		3	2	0
S	5		Communication		displaying correct mechanics in written composition		4		3	3	1
T	5		Technology Use		using clip graphics in a composition developed on the computer		3		2	1	0
U	5		Technology Use		using a productivity tool to create an expository composition on the computer		3		2	2	1

Aggregation Form

Student 1 Name __________________________________________ Student 2 Name __________________________________________ School __________________________________________ Date __________________________________________
	A	B	C	D	E
Row	Outcome area (cells from the table are in parentheses)	Number of score-able instances if the student attempted to answer everything	Number of score-able instances on items the student has attempted to answer	Number of adequate scores	Percent of scores that are adequate
1	Technology use (Column G, Rows A, B, C, D, E, F, G, H, I, J, K, L, O, T, U)	15	14	9	64%
2	Reasoning with information (Column G, Rows A, D, G, J, M, P, Q)	7	6	4	67%
3	Communication (Column G, Rows N, Q, R, S)	4	4	3	75%
4	TOTALS across outcome areas	26	24	16	67%