A study on the evaluation method for effectiveness of data envelopment analysis models in hospital efficiency_West China Medical Journal

Authors：

YANG Wei ¹ , LI Meng ¹ , GUO Shuyan ¹ , YANG Tingting ¹ ,  DONG Siping ^1,2

1. National Institute of Hospital Administration, National Health Commission, Beijing 100044, P. R. China;
2. School of Politics Science & Public Administration, Wuhan University, Wuhan, Hubei 430072, P. R. China;

Corresponding author：

DONG Siping, Email: spingd@163.com

Keywords：

Reliability; Validity; Discrimination; Data envelopment analysis; Hospital efficiency

DOI：

10.7507/1002-0179.202011259

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Objective To compare and evaluate the discrimination, validity, and reliability of different data envelopment analysis (DEA) models for measuring the effectiveness of models by selecting different input and output indicators of the model.Methods Data from health statistical reports and pilot program of diagnosis-related groups of tertiary hospitals in Hubei Province from 2017 to 2018 were used to analyze the discrimination, content and structure validity, and reliability of the models. Six DEA models were established by enriching the details of input and output on the basis of the input and output indicators of the conventional DEA model of hospitals.Results From the view of discrimination, the results of all models were left-skewed, the cost-efficiency model had the lowest left-skewed degree (skewness coefficient: -0.14) and was the flattest (kurtosis coefficient: -1.02). From the view of structure validity, the results of the cost-efficiency model were positively correlated with total weights, outpatient visits, and inpatient visits (r=0.328, 0.329, 0.315; P<0.05). From the perspective of content validity, the interpretation of model was more consistent with theory of production after revision of input and output indicators. From the view of reliability, the cost efficiency model had the largest correlation coefficient between the data of 2017 and 2018 (r=0.880, P<0.05).Conclusions After refining the input and output indicators of the DEA model, the discrimination, validity, and reliability of the model are higher, and the results are more reasonable. Using indicators such as discrimination, validity, and reliability can measure the effectiveness of the DEA model, and then optimize the model by selecting different input and output indicators.

Citation： YANG Wei, LI Meng, GUO Shuyan, YANG Tingting, DONG Siping. A study on the evaluation method for effectiveness of data envelopment analysis models in hospital efficiency. West China Medical Journal, 2020, 35(12): 1435-1440. doi: 10.7507/1002-0179.202011259 Copy

1.	Parkin D, Hollingsworth B. Measuring production efficiency of acute hospitals in Scotland, 1991-94: validity issues in data envelopment analysis. Appl Econ, 1997, 29(11): 1425-1433.
2.	Giuffrida A, Gravelle H. Measuring performance in primary care: econometric analysis and DEA. Appl Econ, 2001, 33(2): 163-175.
3.	Serrano Cinca C, Mar Molinero C, Fuertes Callén Y. Input and output search in DEA: the case of financial institutions// Hwang SN, Lee HS, Zhu J. Handbook of operations analytics using data envelopment analysis. Boston, MA: Springer, 2016: 51-87.
4.	Besstremyannaya G. The impact of Japanese hospital financing reform on hospital efficiency: a difference-in-difference approach. Jpn Eco Rev, 2013, 64(3): 337-362.
5.	Hollingsworth B, Wildman J. The efficiency of health production: re-estimating the WHO panel data using parametric and non-parametric approaches to provide additional information. Health Econ, 2003, 12(6): 493-504.
6.	张俊容, 郭耀煌. 评价指标与DEA有效的关系. 系统工程理论方法应用, 2004, 13(6): 520-523.
7.	Simar L, Wilson PW. Sensitivity analysis of efficiency scores: how to bootstrap in nonparametric frontier models. Manage Sci, 1998, 44(1): 49-61.
8.	Hollingsworth B, Peacock SJ. Efficiency measurement in health and health care. London: Routledge, 2008: 8-9.
9.	董四平, 左玉玲, 陶红兵, 等. 中国医院效率DEA研究分类与投入产出指标分析. 中国卫生政策研究, 2014, 7(10): 40-45.
10.	杨婷婷, 李萌, 郭淑岩, 等. 我国公立二级综合医院运营效率与规模经济状况研究. 华西医学, 2019, 34(12): 1362-1367.
11.	Varabyova Y, Schreyögg J. International comparisons of the technical efficiency of the hospital sector: panel data analysis of OECD countries using parametric and non-parametric approaches. Health Policy, 2013, 112(1/2): 70-79.
12.	Linna M, Häkkinen U, Magnussen J. Comparing hospital cost efficiency between Norway and Finland. Health Policy, 2006, 77(3): 268-278.
13.	Linna M, Häkkinen U, Peltola M, et al. Measuring cost efficiency in the Nordic hospitals--a cross-sectional comparison of public hospitals in 2002. Health Care Manag Sci, 2010, 13(4): 346-357.
14.	杨威, 郭淑岩, 李萌, 等. 基于DRG指标的湖北省三级医院Bootstrap-DEA效率分析. 中华医院管理杂志, 2020, 36(11): 902-906.
15.	杨威, 李萌, 郭淑岩, 等. 公立医院内部管理水平对技术效率的影响研究—基于偏最小二乘结构方程模型方法. 中国卫生政策研究, 2019, 12(12): 48-54.

1. Parkin D, Hollingsworth B. Measuring production efficiency of acute hospitals in Scotland, 1991-94: validity issues in data envelopment analysis. Appl Econ, 1997, 29(11): 1425-1433.
2. Giuffrida A, Gravelle H. Measuring performance in primary care: econometric analysis and DEA. Appl Econ, 2001, 33(2): 163-175.
3. Serrano Cinca C, Mar Molinero C, Fuertes Callén Y. Input and output search in DEA: the case of financial institutions// Hwang SN, Lee HS, Zhu J. Handbook of operations analytics using data envelopment analysis. Boston, MA: Springer, 2016: 51-87.
4. Besstremyannaya G. The impact of Japanese hospital financing reform on hospital efficiency: a difference-in-difference approach. Jpn Eco Rev, 2013, 64(3): 337-362.
5. Hollingsworth B, Wildman J. The efficiency of health production: re-estimating the WHO panel data using parametric and non-parametric approaches to provide additional information. Health Econ, 2003, 12(6): 493-504.
6. 张俊容, 郭耀煌. 评价指标与DEA有效的关系. 系统工程理论方法应用, 2004, 13(6): 520-523.
7. Simar L, Wilson PW. Sensitivity analysis of efficiency scores: how to bootstrap in nonparametric frontier models. Manage Sci, 1998, 44(1): 49-61.
8. Hollingsworth B, Peacock SJ. Efficiency measurement in health and health care. London: Routledge, 2008: 8-9.
9. 董四平, 左玉玲, 陶红兵, 等. 中国医院效率DEA研究分类与投入产出指标分析. 中国卫生政策研究, 2014, 7(10): 40-45.
10. 杨婷婷, 李萌, 郭淑岩, 等. 我国公立二级综合医院运营效率与规模经济状况研究. 华西医学, 2019, 34(12): 1362-1367.
11. Varabyova Y, Schreyögg J. International comparisons of the technical efficiency of the hospital sector: panel data analysis of OECD countries using parametric and non-parametric approaches. Health Policy, 2013, 112(1/2): 70-79.
12. Linna M, Häkkinen U, Magnussen J. Comparing hospital cost efficiency between Norway and Finland. Health Policy, 2006, 77(3): 268-278.
13. Linna M, Häkkinen U, Peltola M, et al. Measuring cost efficiency in the Nordic hospitals--a cross-sectional comparison of public hospitals in 2002. Health Care Manag Sci, 2010, 13(4): 346-357.
14. 杨威, 郭淑岩, 李萌, 等. 基于DRG指标的湖北省三级医院Bootstrap-DEA效率分析. 中华医院管理杂志, 2020, 36(11): 902-906.
15. 杨威, 李萌, 郭淑岩, 等. 公立医院内部管理水平对技术效率的影响研究—基于偏最小二乘结构方程模型方法. 中国卫生政策研究, 2019, 12(12): 48-54.

West China Medical Journal

A study on the evaluation method for effectiveness of data envelopment analysis models in hospital efficiency

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content