From Outcome Measurement to Impact Measurement: Sentiment Analysis to Support Performance Evaluation of Public Healthcare Organizations

Journal title MANAGEMENT CONTROL
Author/s Christian Di Falco, Guido Noto, Gustavo Barresi
Publishing Year 2025 Issue 2025/2
Language Italian Pages 21 P. 157-177 File size 157 KB
DOI 10.3280/MACO2025-002008
DOI is like a bar code for intellectual property: to have more infomation click here

Below, you can see the article first page

If you want to buy this article in PDF format, you can do it, following the instructions to buy download credits

Article preview

FrancoAngeli is member of Publishers International Linking Association, Inc (PILA), a not-for-profit association which run the CrossRef service enabling links to and from online scholarly content.

Performance Management (PM) systems of public healthcare organizations are increasingly focusing on measuring the creation of public value, i.e., the ability to economically meet the social needs of users. To do this, the most recent literature suggests integrating outcome measures – the results achieved in meeting the needs of public service users – into PM systems. However, to fully appreciate the public value created by organizations and recognized by the community, it is also important to measure and evaluate the results in terms of impact they achieve, including for “non-beneficiaries,” a dimension that is still little explored in the literature. The study proposes the use of Sentiment Analysis (SA) to measure this impact by analyzing opinions and sentiments expressed on web platforms. The case study of the G. Martino University Hospital in Messina is used to explore the potential and limitations of SA in measuring, managing, and evaluating organizational performance in the public sector.

Keywords: Performance Management, Sentiment Analysis, Public Value, Healthcare, Artificial Intelligence

  1. Anselmi, L. (1995), Il processo di trasformazione della pubblica amministrazione. Il percorso aziendale, Torino, Giappichelli Editore.
  2. Anthony, R.N. (1965), Planning and Control Systems: A Framework for Analysis, Boston, Harvard University.
  3. Barresi, G. (2005), Dalla competizione alla collaborazione. Nuovi modelli per la gestione dei servizi sanitari, Torino, Giappichelli.
  4. Barretta, A. (2008), The functioning of co-opetition in the health-care sector: An explorative analysis, Scandinavian Journal of Management, 24(3), pp. 209-220.
  5. Barsanti, S., Nuti, S. (2014), The equity lens in the health care performance evaluation system, The International Journal of Health Planning and Management, 29(3), pp. e233-e246.
  6. Baumeister, R. F., Bratslavsky, E., Finkenauer, C., Vohs, K. D. (2001). Bad is stronger than good, Review of General Psychology, 5(4), pp. 323-370.
  7. Bergamaschi, M. (2008), I driver del valore in sanità, Economia e Management, 6, pp. 41-53.
  8. Bianchi, C. (2010), Improving performance and fostering accountability in the public sector through system dynamics modelling: From an ‘external’ to an ‘internal’ perspective, Systems Research and Behavioral Science, 27(4), pp. 361-384.
  9. Birjali, M., Kasri, M., Beni-Hssane, A. (2021), A comprehensive survey on sentiment analysis: Approaches, challenges and trends, Knowledge-Based Systems, 226, 107134.
  10. Bititci, U., Garengo, P., Dörfler, V., Nudurupati, S. (2012), Performance measurement: challenges for tomorrow, International Journal of Management Reviews, 14 (3), pp. 305-327.
  11. Borgonovi, E., Anessi-Pessina, E., Bianchi, C. (2018), Outcome-Based Performance Management in the Public Sector, Cham, Springer.
  12. Cambria, E., Schuller, B., Xia, Y., Havasi, C. (2013), New avenues in opinion mining and sentiment analysis, IEEE Intelligent Systems, 28(2), pp. 15-21. DOI: 10.1109/MIS.2013.30
  13. Capalbo, F., Sorrentino, M., Smarra, M., Ippolito, A. (2023), Il ruolo strategico dei Sistemi di Misurazione delle Performance nelle aziende sanitarie: un caso studio, Management Control, 1, pp. 119-142.
  14. Catturi, G. (2003), Principi di economia aziendale: l’azienda universale: l’idea forza, la morfologia e la fisiologia, Padova, Cedam.
  15. Cepiku, D. (2018), Strategia e performance nelle amministrazioni pubbliche, Milano, Egea Editore.
  16. Christensen, T., Lægreid, P. (2007), The whole-of-government approach to public sector reform, Public Administration Review, 67(6), pp. 1059-1066.
  17. Cuccurullo, C. (2003), Il management strategico nelle aziende sanitarie, Milano, McGraw-Hill.
  18. De Rosis, S., Cerasuolo, D., Nuti, S. (2020), Using patient-reported measures to drive change in healthcare: the experience of the digital, continuous and systematic PREMs observatory in Italy, BMC Health Services Research, 20, pp. 1-17.
  19. Deidda Gagliardo, E. (2015), Il valore pubblico. La nuova frontiera delle performance, Roma, Rirea.
  20. Del Bene, L. (2000), Criteri e strumenti per il controllo gestionale nelle aziende sanitarie, Milano, Giuffrè Editore.
  21. Denecke, K., Deng, Y. (2015), Sentiment analysis in medical settings: New opportunities and challenges, Artificial Intelligence in Medicine, 64(1), pp. 17-27.
  22. Di Falco, C., Noto, G., Marisca, C., Barresi, G. (2024), The contribution of information and communication technologies on performance management and measurement in healthcare: a systematic review of the literature, The TQM Journal, 36(9), pp. 371-391.
  23. Di Vaio, A., Varriale, L., Lepore, L., Mastellone, L. (2024), Digitalization and Performance Management Systems: A Shipping Agency Case Study, Management Control, Suppl. 2, pp. 173-197.
  24. Fasiello, R., Rivenbark, W.C., Bianchi, C. (2022), Exploring the shift from output measures to outcome measures of service delivery: Insights from municipal research, Public Performance & Management Review, 45(2), pp. 428-447.
  25. Ferreira, A., Otley, D. (2009), The design and use of performance management systems: An extended framework for analysis, Management Accounting Research, 20(4), pp. 263-282.
  26. Gohil, S., Vuik, S., Darzi, A. (2018), Sentiment analysis of health care tweets: review of the methods used, JMIR Public Health and Surveillance, 4(2), e5789.
  27. Guarini, E., Magli, F., Martinelli, M. (2022), La prospettiva del ‘Sistema di Controllo Interno Integrato’ nella Pubblica Amministrazione: riflessioni a partire dal caso Regione Lombardia, Management Control, 2, pp. 63-89.
  28. Hood, C. (1991), A public management for all seasons?, Public Administration, 69(1), pp. 3-19.
  29. Ippolito, A., Sorrentino, M., Capalbo, F., Di Pietro, A. (2023), How technological innovations in performance measurement systems overcome management challenges in healthcare, International Journal of Productivity and Performance Management, 72(9), pp. 2584-2604.
  30. Kay, C. (2007), Healthcare marketing: What is salient?, International Journal of Pharmaceutical and Healthcare Marketing, 1(3), pp. 247-263. DOI: 10.1108/17506120710818256
  31. Lega, F. (2020), Economia e management sanitario, Milano, Egea Editore.
  32. Lega, F., Vendramini, E. (2008), Budgeting and performance management in the Italian National Health System (INHS): Assessment and constructive criticism, Journal of Health Organization and Management, 22(1), pp. 11-22.
  33. Leotta, A. (2024), Il problema dell’integrazione: mezzo e fine del controllo di gestione, Management Control, 3, pp. 5-15.
  34. Ligthart, A., Catal, C., Tekinerdogan, B. (2021), Systematic reviews in sentiment analysis: a tertiary study, Artificial Intelligence Review, pp. 1-57.
  35. Macinati, M.S., Rizzo, M.G. (2024), Orientamento al feedback, soddisfazione rispetto al feedback del sistema reporting e performance di budget, Management Control, 3, pp. 179-200.
  36. Magon, A., Caruso, R. (2023), Addressing a potential crisis in the Italian National Health System, The Lancet, 401, 10384, pp. 1262-1263.
  37. Mallarini, E. (2004), Fiducia in sanità: non solo marketing, Milano, Egea.
  38. Marchi, L. (2011), L’evoluzione del controllo di gestione nella prospettiva informativa e gestionale esterna, Management Control, 3, pp. 5-16.
  39. Matsuda, S., Ohtomo, T., Okuyama, M., Miyake, H., Aoki, K (2023), Estimating patient satisfaction through a language processing model: Model development and evaluation, JMIR Formative Research, 7. DOI: 10.2196/48534
  40. Mauro, M., Noto, G., Prenestini, A., Sarto, F. (2024), Digital transformation in healthcare: Assessing the role of digital technologies for managerial support processes, Technological Forecasting and Social Change, 209, 123781.
  41. Mauro, S.G., Cinquini, L., Pianezzi, D. (2021), New Public Management between reality and illusion: Analysing the validity of performance-based budgeting, The British Accounting Review, 53(6), 100825.
  42. Merchant, K. A., Otley, D. (2020). Beyond the systems versus package debate, Accounting, Organizations and Society, 86, 101185.
  43. Moore, M.H. (1995), Creating Public Value: Strategic Management in Government, Cambridge, Harvard University Press.
  44. Moynihan, D.P. (2005), Goal-based learning and the future of performance management, Public Administration Review, 65(2), pp. 203-216.
  45. Mussari, R. (2022), Performance e valore pubblico, Padova, Cedam.
  46. Mussari, R., Melis, A., Monfardini, P. (2012), La creazione di valore: aspetti critici e problematiche di misurazione, Management Control, Suppl. 2, pp. 5-12.
  47. Noto, G. (2024), Il governo della performance dei servizi di prevenzione e tutela della salute, in Strategia, Management e Controllo, Giappichelli, Torino,
  48. Nuti, S., Noto, G., Vola, F., Vainieri, M. (2018), Let’s play the patients’ music: A new generation of performance measurement systems in healthcare, Management Decision, 56(10), pp. 2252-2272.
  49. O’Flynn, J. (2007), From new public management to public value: Paradigmatic change and managerial implications, Australian Journal of Public Administration, 66(3), pp. 353-366.
  50. Oppi, C., Vagnoni, E., Cattaneo, C., Galizzi, G. (2024), Controllo di gestione e relazioni intraorganizzative: la prospettiva del controller, Management Control, 3, pp. 155-178.
  51. Osborne, S.P. (2010), The New Public Governance: Emerging Perspectives on the Theory and Practice of Public Governance, London, Routledge.
  52. Osborne, S.P., Radnor, Z., Nasi, G. (2013), A new theory for public service management? Toward a (public) service-dominant approach, The American Review of Public Administration, 43(2), pp. 135-158.
  53. Otley, D. (2003), Management control and performance management: Whence and whither?, The British Accounting Review, 35(4), pp. 309-326.
  54. Ouchi, W.G. (1979), A conceptual framework for the design of organizational control mechanisms, Management Science, 25(9), pp. 833-848.
  55. Paolini, A. (2022), Gestione integrata dei dati e performance aziendali, Management Control, 2, pp. 5-14.
  56. Piryani, R., Madhavi, D., Singh, V.K. (2017), Analytical mapping of opinion mining and sentiment analysis research during 2000-2015, Information Processing & Management, 53(1), pp. 122-150.
  57. Pollitt, C. (2003), Joined-up government: a survey, Political Studies Review, 1(1), pp. 34-49.
  58. Presti, C. (2022), L’azienda intelligente: opportunità e minacce per la creazione di valore, Management Control, 3, pp. 5-12.
  59. Purcarea, V.L., Gheorghe, I.R., Petrescu, C.M. (2019), The impact of marketing strategies in healthcare systems, Journal of Medicine and Life, 12(2), pp. 93-96.
  60. Riccaboni, A. (2009), Il controllo di gestione: obiettivi, attori e strumenti, in Il controllo di gestione. Metodologie e strumenti, Arezzo, Knowità, pp. 29-52.
  61. Russo, S. (2012), Economia e management delle aziende sanitarie pubbliche, Padova, Cedam.
  62. Sánchez-Rada, J.F., Iglesias, C.A. (2019), Social context in sentiment analysis: Formal definition, overview of current trends and framework for comparison, Information Fusion, 52, pp. 344-356.
  63. Senyapar, H.N. (2024), Healthcare Branding and Reputation Management Strategies for Organizational Success, Technium Social Sciences Journal, 55, pp. 26-53.
  64. Sharma, N., Jain, V. (2021), Evaluation and summarization of student feedback using sentiment analysis, in Hassanien, A., Bhatnagar, R. and Darwish, A. (Eds), Advanced Machine Learning Technologies and Applications (AMLTA 2020), Advances in Intelligent Systems and Computing, 1141, Springer, Singapore, pp. 351-360. DOI: 10.1007/978-981-15-3383-9_35
  65. Smith, P. (1995), On the unintended consequences of publishing performance data in the public sector, International Journal of Public Administration, 18(2-3), pp. 277-310.
  66. Soleymani, M., Garcia, D., Jou, B., Schuller, B., Chang, S.F., Pantic, M. (2017), A survey of multimodal sentiment analysis, Image and Vision Computing, 65.
  67. Spanò, R., Ginesti, G. (2022), Fostering performance management in healthcare: Insights into the role of big data, Meditari Accountancy Research, 30(4), pp. 941-963.
  68. Stolk, E.A., Pickee, S.J., Ament, A.H., Busschbach, J.J. (2005), Equity in health care prioritisation: an empirical inquiry into social value, Health Policy, 74(3), pp. 343-355.
  69. Vagnoni E. (2005), La direzione delle aziende sanitarie: criticità di contesto, economicità e tutela della salute, Milano, FrancoAngeli.
  70. Vainieri, M., Noto, G., Ferre, F., Rosella, L.C. (2020), A performance management system in healthcare for all seasons?, International Journal of Environmental Research and Public Health, 17(15), 5590.
  71. Valentinetti, D., Rea, M. (2023), Intelligenza artificiale e accounting: le possibili relazioni, Management Control, 2, pp. 93-116.
  72. Vinci, S., D’Amico, E., Biscotti, A.M. (2022), I sistemi di management control nella sanità pubblica italiana: dalle strutture verticali ai processi orizzontali, Management Control, 1, pp. 103-119.
  73. Wadmann, S., Johansen, S., Lind, A., Birk, H.O., Hoeyer, K. (2013), Analytical perspectives on performance-based management: an outline of theoretical assumptions in the existing literature, Health Economics, Policy and Law, 8(4), pp. 511-527.
  74. Wankhade, M., Rao, A.C.S., Kulkarni, C. (2022), A survey on sentiment analysis methods, applications, and challenges, Artificial Intelligence Review, 55(7), pp. 5731-5780.
  75. Yadav, A., Vishwakarma, D.K. (2020), Sentiment analysis using deep learning architectures: A review, Artificial Intelligence Review, 53(6), pp. 4335-4385.
  76. Yousif, A., Niu, Z., Tarus, J.K. and Ahmad, A. (2019), A survey on sentiment analysis of scientific citations, Artificial Intelligence Review, 52, pp. 1805-1838.
  77. Yue, L., Chen, W., Li, X., Zuo, W., Yin, M. (2019), A survey of sentiment analysis in social media, Knowledge and Information Systems, 60, pp. 617-663.
  78. Zakkar, M.A., Lizotte, D.J. (2021), Analyzing patient stories on social media using text analytics, Journal of Healthcare Informatics Research, 5(4), pp. 382-400.
  79. Zunic, A., Corcoran, P., Spasic, I. (2020), Sentiment analysis in health and well-being: A systematic review, JMIR Medical Informatics, 8(1), e16023. DOI: 10.2196/16023

Christian Di Falco, Guido Noto, Gustavo Barresi, Dalla misurazione dell’outcome a quella dell’impact: la Sentiment Analysis a supporto della valutazione della performance delle aziende sanitarie pubbliche in "MANAGEMENT CONTROL" 2/2025, pp 157-177, DOI: 10.3280/MACO2025-002008