A deal between Google’s artificial intelligence firm DeepMind and the UK’s NHS had serious “inadequacies”, an academic paper has suggested.
More than a million patient records were shared with DeepMind to build an app to alert doctors about patients at risk of acute kidney injury (AKI).
Author Hal Hodson said that it was “inexcusable” patients were not told how their data would be used.
Google said that the report contained “major errors”.
It told the BBC that it was commissioning its own analysis and rebuttal, which the authors said they welcomed.
When the deal between London’s Royal Free Hospital and DeepMind became public in February 2016, some three months after the data started to be collected, it caused controversy over the amount of patient information being shared and the lack of public consultation.
Mr Hodson, a New Scientist journalist, and co-author Julia Powles, a Cambridge University academic, said there are still big questions to be answered about the tie-up.
“Why DeepMind, an artificial intelligence company wholly owned by data mining and advertising giant Google, was a good choice to build an app that functions primarily as a data-integrating user interface, has never been adequately explained by either DeepMind or Royal Free,” they wrote.
The app contains no artificial intelligence although DeepMind has said that it was hoping to incorporate AI techniques to create smarter alerts in future.
The criticisms in the paper included:
- Questions over whether DeepMind could be considered a mere data processor when it developed an app – Streams – that had direct impact on patient care
- An absence of oversight or legally binding documents about how the data would be used
- Questions about whether the device was correctly registered with regulators
In response, DeepMind and the Royal Free issued a joint statement: “This paper completely misrepresents the reality of how the NHS uses technology to process data.
“It makes a series of significant factual and analytical errors, assuming that this kind of data agreement is unprecedented.
“In fact, every trust in the country uses IT systems to help clinicians access current and historic information about patients, under the same legal and regulatory regime.”
Investigation
The NHS does have information-sharing agreements with a range of third-party firms, but this is the first such deal with a major US tech firm.
DeepMind’s initial assertion that the NHS had 1,500 other agreements with third-party organisations that process patient data has since been described by the NHS as “inaccurate”. There is no central database on how many there are, the BBC was told.
The app is currently the subject of an investigation by the Information Commissioner’s Office while the National Data Guardian, tasked with safeguarding health data, is also looking at it.
In a statement, the ICO told the BBC: “Our investigation into the sharing of patient information between the Royal Free NHS Trust and Deep Mind is close to conclusion.
“We continue to work with the National Data Guardian and have been in regular contact with the Royal Free and Deep Mind who have provided information about the development of the Streams app.
“This has been subject to detailed review as part of our investigation. It’s the responsibility of businesses and organisations to comply with data protection law.”
The National Data Guardian added: “Our consideration of this matter has required a thorough approach in which the NDG and her panel have kept patients’ rightful expectations of both good care and confidentiality at the forefront of discussions.
“The NDG has provided a view on this matter to assist the ICO’s investigation and looks forward to this being concluded as soon as practicable.”
Improved care
DeepMind has been at pains to make clear that none of the data collected for the app has been shared with parent company Google.
AKI is a serious condition, linked to 40,000 deaths a year in the UK and leading to a range of other health issues from minor kidney dysfunction to the need for dialysis and transplant.
In February, DeepMind published details about how the app was improving patient care.
It revealed that more than 26 doctors and nurses at the Royal Free are now using Streams and that each day it alerts them to 11 patients at risk of AKI.
Sarah Stanley, a consultant nurse who leads the patients at risk and resuscitation team, said: “Streams is saving us a substantial amount of time every day. The instant alerts about some of our most vulnerable patients mean we can get the right care to the right patients much more quickly.”
DeepMind has acknowledged that it could have done better in the way it engaged with patients whose data was being used and, on the back of the criticism, agreed to set up patient forums.
It published a strategy on future patient engagement which opens by saying: “Outcomes are better when patients and clinicians make decisions together.”
Source: http://www.bbc.co.uk/
