Applying Item Response Theory to Analyzing and Improving the Item Quality of an Online Chinese Reading Assessment

Xinyun Tian; Xiaoxue Han; Hercy N.H. Cheng; Wang Chen Chang; Calvin C.Y. Liao; Jianwen Sun; Xiaoliang Zhu; Sanya Liu

doi:10.1109/IIAI-AAI.2017.100

Applying Item Response Theory to Analyzing and Improving the Item Quality of an Online Chinese Reading Assessment

Xinyun Tian
, Xiaoxue Han
, Hercy N.H. Cheng
, Wang Chen Chang
, Calvin C.Y. Liao
, Jianwen Sun
, Xiaoliang Zhu
, Sanya Liu

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Citations (Scopus)

Abstract

In order to realize the individualized teaching of Chinese language in primary schools, this research has developed an online Chinese reading assessment for primary schools, which aims to record the students test process and analyze the development level of students Chinese reading ability. In this paper, the item response theory (IRT) is applied to the quality analysis of the assessment in terms of measurement attributes (difficulty, discrimination, guessing), item characteristic curve, item information function and test information function. Additionally, this paper further explores the relationship between the various parameters of the items. The results show that the IRT can effectively guide the construction of the reading assessment scale, improve the discrimination degree, reduce the guessing degree, and effectively improve the quality of the item. This paper also proposes a method to find out the unreasonable options and modify items locally through project parameters and option analysis. It is expected that researchers and educators can modify the item more efficiently by quality analysis.

Original language	English
Title of host publication	Proceedings - 2017 6th IIAI International Congress on Advanced Applied Informatics, IIAI-AAI 2017
Editors	Kiyota Hashimoto, Naoki Fukuta, Tokuro Matsuo, Sachio Hirokawa, Masao Mori, Masao Mori
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	754-759
Number of pages	6
ISBN (Electronic)	9781538606216
DOIs	https://doi.org/10.1109/IIAI-AAI.2017.100
Publication status	Published - Nov 15 2017
Externally published	Yes
Event	6th IIAI International Congress on Advanced Applied Informatics, IIAI-AAI 2017 - Hamamatsu, Shizuoka, Japan Duration: Jul 9 2017 → …

Publication series

Name	Proceedings - 2017 6th IIAI International Congress on Advanced Applied Informatics, IIAI-AAI 2017

Conference

Conference	6th IIAI International Congress on Advanced Applied Informatics, IIAI-AAI 2017
Country/Territory	Japan
City	Hamamatsu, Shizuoka
Period	7/9/17 → …

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 4 Quality Education

Keywords

Item response theory
Reading ability
Test quality

ASJC Scopus subject areas

Artificial Intelligence
Computer Networks and Communications
Computer Science Applications
Information Systems
Information Systems and Management

Access to Document

10.1109/IIAI-AAI.2017.100

Cite this

Tian, X., Han, X., Cheng, H. N. H., Chang, W. C., Liao, C. C. Y., Sun, J., Zhu, X., & Liu, S. (2017). Applying Item Response Theory to Analyzing and Improving the Item Quality of an Online Chinese Reading Assessment. In K. Hashimoto, N. Fukuta, T. Matsuo, S. Hirokawa, M. Mori, & M. Mori (Eds.), Proceedings - 2017 6th IIAI International Congress on Advanced Applied Informatics, IIAI-AAI 2017 (pp. 754-759). Article 8113344 (Proceedings - 2017 6th IIAI International Congress on Advanced Applied Informatics, IIAI-AAI 2017). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IIAI-AAI.2017.100

Applying Item Response Theory to Analyzing and Improving the Item Quality of an Online Chinese Reading Assessment. / Tian, Xinyun; Han, Xiaoxue; Cheng, Hercy N.H. et al.
Proceedings - 2017 6th IIAI International Congress on Advanced Applied Informatics, IIAI-AAI 2017. ed. / Kiyota Hashimoto; Naoki Fukuta; Tokuro Matsuo; Sachio Hirokawa; Masao Mori; Masao Mori. Institute of Electrical and Electronics Engineers Inc., 2017. p. 754-759 8113344 (Proceedings - 2017 6th IIAI International Congress on Advanced Applied Informatics, IIAI-AAI 2017).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Tian, X, Han, X, Cheng, HNH, Chang, WC, Liao, CCY, Sun, J, Zhu, X & Liu, S 2017, Applying Item Response Theory to Analyzing and Improving the Item Quality of an Online Chinese Reading Assessment. in K Hashimoto, N Fukuta, T Matsuo, S Hirokawa, M Mori & M Mori (eds), Proceedings - 2017 6th IIAI International Congress on Advanced Applied Informatics, IIAI-AAI 2017., 8113344, Proceedings - 2017 6th IIAI International Congress on Advanced Applied Informatics, IIAI-AAI 2017, Institute of Electrical and Electronics Engineers Inc., pp. 754-759, 6th IIAI International Congress on Advanced Applied Informatics, IIAI-AAI 2017, Hamamatsu, Shizuoka, Japan, 7/9/17. https://doi.org/10.1109/IIAI-AAI.2017.100

Tian X, Han X, Cheng HNH, Chang WC, Liao CCY, Sun J et al. Applying Item Response Theory to Analyzing and Improving the Item Quality of an Online Chinese Reading Assessment. In Hashimoto K, Fukuta N, Matsuo T, Hirokawa S, Mori M, Mori M, editors, Proceedings - 2017 6th IIAI International Congress on Advanced Applied Informatics, IIAI-AAI 2017. Institute of Electrical and Electronics Engineers Inc. 2017. p. 754-759. 8113344. (Proceedings - 2017 6th IIAI International Congress on Advanced Applied Informatics, IIAI-AAI 2017). doi: 10.1109/IIAI-AAI.2017.100

Tian, Xinyun ; Han, Xiaoxue ; Cheng, Hercy N.H. et al. / Applying Item Response Theory to Analyzing and Improving the Item Quality of an Online Chinese Reading Assessment. Proceedings - 2017 6th IIAI International Congress on Advanced Applied Informatics, IIAI-AAI 2017. editor / Kiyota Hashimoto ; Naoki Fukuta ; Tokuro Matsuo ; Sachio Hirokawa ; Masao Mori ; Masao Mori. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 754-759 (Proceedings - 2017 6th IIAI International Congress on Advanced Applied Informatics, IIAI-AAI 2017).

@inproceedings{084d5482cc424b6cbdb762b3c05a1021,

title = "Applying Item Response Theory to Analyzing and Improving the Item Quality of an Online Chinese Reading Assessment",

abstract = "In order to realize the individualized teaching of Chinese language in primary schools, this research has developed an online Chinese reading assessment for primary schools, which aims to record the students test process and analyze the development level of students Chinese reading ability. In this paper, the item response theory (IRT) is applied to the quality analysis of the assessment in terms of measurement attributes (difficulty, discrimination, guessing), item characteristic curve, item information function and test information function. Additionally, this paper further explores the relationship between the various parameters of the items. The results show that the IRT can effectively guide the construction of the reading assessment scale, improve the discrimination degree, reduce the guessing degree, and effectively improve the quality of the item. This paper also proposes a method to find out the unreasonable options and modify items locally through project parameters and option analysis. It is expected that researchers and educators can modify the item more efficiently by quality analysis.",

keywords = "Item response theory, Reading ability, Test quality",

author = "Xinyun Tian and Xiaoxue Han and Cheng, \{Hercy N.H.\} and Chang, \{Wang Chen\} and Liao, \{Calvin C.Y.\} and Jianwen Sun and Xiaoliang Zhu and Sanya Liu",

note = "Funding Information: This work was financially supported by the research funds from Ministry of Education and China Mobile (Grant No. MCM20160401) and the self-determined research funds of Central China Normal University from the colleges{\textquoteright} basic research and operation of Ministry of Education of China (Grant No. CCNU16A02022, CCNU17A06035). Publisher Copyright: {\textcopyright} 2017 IEEE.; 6th IIAI International Congress on Advanced Applied Informatics, IIAI-AAI 2017 ; Conference date: 09-07-2017",

year = "2017",

month = nov,

day = "15",

doi = "10.1109/IIAI-AAI.2017.100",

language = "English",

series = "Proceedings - 2017 6th IIAI International Congress on Advanced Applied Informatics, IIAI-AAI 2017",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "754--759",

editor = "Kiyota Hashimoto and Naoki Fukuta and Tokuro Matsuo and Sachio Hirokawa and Masao Mori and Masao Mori",

booktitle = "Proceedings - 2017 6th IIAI International Congress on Advanced Applied Informatics, IIAI-AAI 2017",

address = "United States",

}

TY - GEN

T1 - Applying Item Response Theory to Analyzing and Improving the Item Quality of an Online Chinese Reading Assessment

AU - Tian, Xinyun

AU - Han, Xiaoxue

AU - Cheng, Hercy N.H.

AU - Chang, Wang Chen

AU - Liao, Calvin C.Y.

AU - Sun, Jianwen

AU - Zhu, Xiaoliang

AU - Liu, Sanya

N1 - Funding Information: This work was financially supported by the research funds from Ministry of Education and China Mobile (Grant No. MCM20160401) and the self-determined research funds of Central China Normal University from the colleges’ basic research and operation of Ministry of Education of China (Grant No. CCNU16A02022, CCNU17A06035). Publisher Copyright: © 2017 IEEE.

PY - 2017/11/15

Y1 - 2017/11/15

N2 - In order to realize the individualized teaching of Chinese language in primary schools, this research has developed an online Chinese reading assessment for primary schools, which aims to record the students test process and analyze the development level of students Chinese reading ability. In this paper, the item response theory (IRT) is applied to the quality analysis of the assessment in terms of measurement attributes (difficulty, discrimination, guessing), item characteristic curve, item information function and test information function. Additionally, this paper further explores the relationship between the various parameters of the items. The results show that the IRT can effectively guide the construction of the reading assessment scale, improve the discrimination degree, reduce the guessing degree, and effectively improve the quality of the item. This paper also proposes a method to find out the unreasonable options and modify items locally through project parameters and option analysis. It is expected that researchers and educators can modify the item more efficiently by quality analysis.

AB - In order to realize the individualized teaching of Chinese language in primary schools, this research has developed an online Chinese reading assessment for primary schools, which aims to record the students test process and analyze the development level of students Chinese reading ability. In this paper, the item response theory (IRT) is applied to the quality analysis of the assessment in terms of measurement attributes (difficulty, discrimination, guessing), item characteristic curve, item information function and test information function. Additionally, this paper further explores the relationship between the various parameters of the items. The results show that the IRT can effectively guide the construction of the reading assessment scale, improve the discrimination degree, reduce the guessing degree, and effectively improve the quality of the item. This paper also proposes a method to find out the unreasonable options and modify items locally through project parameters and option analysis. It is expected that researchers and educators can modify the item more efficiently by quality analysis.

KW - Item response theory

KW - Reading ability

KW - Test quality

UR - https://www.scopus.com/pages/publications/85040548656

UR - https://www.scopus.com/pages/publications/85040548656#tab=citedBy

U2 - 10.1109/IIAI-AAI.2017.100

DO - 10.1109/IIAI-AAI.2017.100

M3 - Conference contribution

AN - SCOPUS:85040548656

T3 - Proceedings - 2017 6th IIAI International Congress on Advanced Applied Informatics, IIAI-AAI 2017

SP - 754

EP - 759

BT - Proceedings - 2017 6th IIAI International Congress on Advanced Applied Informatics, IIAI-AAI 2017

A2 - Hashimoto, Kiyota

A2 - Fukuta, Naoki

A2 - Matsuo, Tokuro

A2 - Hirokawa, Sachio

A2 - Mori, Masao

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 6th IIAI International Congress on Advanced Applied Informatics, IIAI-AAI 2017

Y2 - 9 July 2017

ER -

Applying Item Response Theory to Analyzing and Improving the Item Quality of an Online Chinese Reading Assessment

Abstract

Publication series

Conference

UN SDGs

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this