This week there was an article that speculated that doctors in the UK could be liable for errors made by AI tools unless rules are changed. The argument is that AI tools should be treated as products, which isn’t necessarily the case right now. Right now doctors may face all the liability instead of AI tool makers having some responsibility. From my reading, this doesn’t appear to say that doctors aren’t still accountable for mistakes, but they aren’t wholly accountable. The Medical Protection Society wants the technology vendors to bear some responsibility.
I have to admit I’m torn here. If doctors are using tools, they should be sure the tools work well and they agree with the recommendations or results. However, I think AI changes things somewhat. If I read a medical device, say one that analyzes blood, I expect that the results are deterministic and accurate. If they aren’t, how could I, as a doctor, know that. The companies the built (or maintain) the device would be culpable for errors.
Using AI assistance, however, is a little different. This is a non-deterministic interpretation of data. In the same situation, if an AI were to interpret blood results and summarize them, perhaps failing to point out a deviation of a value and this impacted patient care, is the AI responsible? The doctor for not realizing there is an issue? Joint responsibility?
I think it’s unclear. I’m not 100% sure how I feel here, and I lean towards both being liable. If someone is training an AI model to help with medicine, then they ought to bear (and feel) accountable for results.
This seems different than software engineering to me. In medicine we have humans overloaded and AI is supposed to help, but has to be responsible. I don’t think the stakes are as high in producing software. It’s an arbitrary decision to say we need to get xx work done with yy resources. AI can help, perhaps amplify the abilities of people to produce code, but the people still have to verify the AI results. Any time pressure is a decision we make, not one brought on by medical issues.
Perhaps this seems hypocritical, and I will admin I’m not 100% sure how I feel, but I’m more concerned with AI technology bearing responsibility in healthcare than software. Do you feel the same way? Or perhaps in either case the AI is just a tool? Let me know what you think,
Steve Jones
Voice of the DBA 
“Using AI assistance, however, is a little different. This is a non-deterministic interpretation of data.”
As software professionals, we know this. But do the general public – including other professionals?
Crucially in my mind, has AI assistance been marketed to the public as a fallible system? AI companies would argue that the tiny “AI can make mistakes” disclaimer buried at the bottom of the screen or in a giant unreadable terms-of-service agreement suffices for this.
But seeing how intelligent professionals have used and continue to use AI as an information source, it is clear to me that this is not sufficient, especially in the context of the massive marketing blitz positioning AI as the solution to all of our informational problems.
LikeLike