AI is evolving so rapidly that it’s almost impossible to keep up with – and many people don’t know what it’s capable of. For employers, a key fear is that their employees are using ChatGPT to help them do their work, without realising that this could be a breach of data protection laws. Elliot Fry, Managing Associate at Cripps and Alice Hunter, Associate at Cripps, look at the dangers of AI – with focus on the data protection angle. They also outline some practical tips on how to mitigate the risks of AI in the workplace and use it both safely and in a way that benefits the business.
Tackling data protection challenges in the AI revolution
With the rise in uptake of artificial intelligence (AI), including chatbots such as OpenAI’s ChatGPT, this technology has become increasingly integrated into various aspects of our lives, including the workplace. To realise the benefits of these tools, AI products must be deployed appropriately and lawfully.
AI is the capability of machines to perform tasks that typically require human intelligence. ChatGPT is based on a large language model which is a type of complex AI system trained on huge volumes of data to learn the patterns and structures of language, which allows the model to generate human-like predictive responses to text-based input. The training data used for ChatGPT relies heavily on publicly available information, including data that is scraped from the internet, as well as data which users have submitted in the past, some of which may be personal data (information that relates to an identified or identifiable individual).
The UK GDPR and Data Protection Act 2018 set out requirements and restrictions regarding the use of personal data that must be followed, and the characteristics of AI can cause tensions with compliance in practice.
Lawful basis for processing personal data
Wherever personal data is processed, there must be a lawful basis for processing. The UK GDPR sets out six lawful bases for processing personal data. Consent is one example, however this is unlikely to be satisfied because most individuals will not consent to having their data provided to ChatGPT. Consent must be specific, and the lack of transparency in how personal data might be used by AI products means that any consent obtained may not be valid. Relying on consent must also involve the ability to withdraw the consent, and it’s difficult to see how that could work in practice related to a large AI model which doesn’t allow for deletion of data which has been incorporated into its learning.
The more likely lawful basis to be relied on is legitimate interest. In order to rely on this basis, you must also show that it’s not overridden by the rights and interests of the affected individuals. In other words, the use of their data shouldn’t have an undue effect on their privacy and other interests. Again, with the complex and opaque ways in which AI products use inputted data, it may be difficult to justify providing personal data to AI products.
Businesses should seriously consider pseudonymising (replacing identifying information with a code) personal data which is used with ChatGPT, or anonymising it altogether.
Even with an identified lawful basis, the legitimate interest condition cannot be used for special category personal data including information such as health data which requires extra levels of protection because it is sensitive. Businesses should be very wary of inputting any special category personal data into AI products unless they can clearly demonstrate a condition under data protection law which they can rely on.
Fairness and transparency
Fairness is about whether personal data is being used in a way which data subjects would reasonably expect. Issues can also occur where the output from an AI system could cause discrimination or other adverse effects (as there are also restrictions in data protection law on the usage of automated decision making without human intervention). Where AI is used to make decisions, those decisions need to be both explainable and justifiable and individuals should be given the right to object to any automated decision making.
The nature of sophisticated AI is inherently opaque and we as humans cannot always understand how it operates. If we don’t know how the AI model is using personal data to arrive at a decision or output, how can we say that the transparency principle is being fulfilled? Using ChatGPT or other AI products to make decisions about how individuals are treated is unlikely to be compliant with data protection law. Doing this could also lead to other legal issues, as the outputs of AI products have often inadvertently been discriminatory.
Data protection law also requires businesses to inform individuals how their data is being used. As mentioned above, the complex ways in which AI products work mean that providing meaningful information in relation to this will be difficult. Even if sufficient information can be provided, existing data sets (collected before that information was provided) shouldn’t be repurposed to feed into AI products unless it’s been made clear to individuals how their existing data will be used.
Accuracy
Any incorrect or misleading personal data should be corrected or deleted without undue delay. The output generated by the AI system may contain personal data. If a chatbot such as ChatGPT is used for customer support this can involve the retrieval customer records, for example, and if it makes a mistake there is likely to be inaccuracies in the personal data. ChatGPT is liable to “hallucinations” where questions are answered incorrectly and information provided about people isn’t accurate due to the predictive nature of the response which can appear convincing. All of this means that using AI products with personal data can lead to incorrect information being produced or recorded, breaching data protection law as well as diminishing consumer trust.
Conclusions
The Italian data protection authority temporarily banned the use of ChatGPT earlier this year, and it’s clear other data protection authorities are also concerned about its privacy impacts.
While AI tools can be used to enhance efficiency in the workplace, it is essential to recognise and mitigate the potential risks associated with their use. SMEs can mitigate the risks of AI in the workplace by:
- Considering and documenting the lawful basis for processing personal data;
- Pseudonymising or anonymising data wherever possible;
- Carrying out a data protection impact assessment (DPIA) to help mitigate the risk of non-compliance before deployment of an AI system;
- Ensuring transparency by making information about the processing available to the public and informing individuals where possible;
- Keeping up-to-date with state-of-the-art security measures where possible;
- Collecting and using only the data that is adequate to fulfil your stated purpose.
