The dark side of AI in organizational decision-making: a socio-technical systems of opacity, overreliance and ethical vigilance
Avinash ChopraPurpose
The application of artificial intelligence (AI) is increasingly being used in organizational decision-making. Concerns about its “dark side” are mounting and warrant attention. This paper aims to examine the impact of these aspects on ethical vigilance and on the emergence of dysfunctional results in AI-enabled environments.
Design/methodology/approach
The study uses a mixed-methods approach based on the socio-technical systems theory and the technology acceptance model by incorporating theme analysis of interviews with experts to determine gaps in the research and using them to structure equation modeling to analyze survey data consisting of 369 professionals working with Indian tech-enabled companies.
Findings
Algorithmic opacity was found to be positively associated with organizational overreliance and negatively associated with trust in AI systems. Organizational overreliance was found to statistically mediate the association between algorithmic opacity and ethical vigilance.
Practical implications
The paper calls on policymakers to leave behind the discourse of efficiency at the cost of AI and take a holistic, governance-focused approach to harmonize the use of AI with transparency, confidence and moral responsibility. This alignment should be achieved not only by law but also by a strategy for long-term organizational integrity and stakeholder trust.
Originality/value
The study provides preliminary empirical support for ethical vigilance as a relevant construct in AI governance and organizational decision-making. It provides useful insights into how AI systems can be designed to be explainable, accountable and ethically sound to counter rising organizational risks.