To evaluate an admission review tool - The reliability and validity of AEP

Objectives: The purposes of our study are to evaluate the reliability, validity and practicality of mAEP (modified Appropriateness Evaluation Protocol) in Taiwan, and try to find the major reasons for inappropriate admission. Methods: We collected 462 appeal claim medical records from Taipei and the Northern branch of the Bureau of National Health Insurance (BNHI). The copies of the medical records were dispensed by a single blind technique to 4 reviewers who used mAEP to judge the appropriateness of admission. Then the result was compared to the BNHI medical review physicians' judgement. The data was analyzed by SPSS 10.0 statistics software. Results: Three of the four reviewers' original Cohen's kappa were around 0.4. However, after reviewers discussion and revision of their appropriateness judgements. The kappa increased between 0.486 to 0.661. This represents a good reproducibility of mAEP. With regard to the sensitivity and specificity, the branch of BNHI medical review physicians and some of our reviewers were around 70%. Therefore, mAEP's validity still has room for improvement. The major reason for inappropriate admission is that the "diagnostic procedures and/or treatment could be done on an outpatient basis". This is the same as the studies in Western countires. The mean time to review the medical records with mAEP was 2.6 minutes. This is similar to Western studies and suggests that mAEP is a fast, easy and timesaving utilization review instrument. Conclusion: The preliminary result proved that the reliability, validity and practicality of mAEP are all good. However, if mAEP could become a tool for the BNHI's medical review and hospitals' admission protocol, further studies are needed.