Can AI and User Privacy Coexist?

Synology’s Hanen Lin offers insights on whether AI and user privacy can coexist. This article originally appeared on Solutions Review’s Insight Jam, an enterprise IT community enabling the human conversation on AI.

When a tech giant suffers public backlash as vicious as what Microsoft experienced after announcing their new Recall AI feature, other companies may want to reconsider their approach to customer privacy.  

In May of 2024, Microsoft announced Recall as a feature of their new line of AI PCs. It was intended to regularly take snapshots of screen activity, allowing users to search these snapshots using AI. Despite Microsoft’s assurances that this data would be encrypted, the UK Information Commissioner’s Office (ICO) is now investigating the feature with concerns about compliance and product safety, and privacy advocates have labeled this feature a “privacy nightmare”. 

This incident is just the tip of the iceberg of privacy concerns surrounding AI-powered technologies. Facial recognition technologies such as those found in Google Photos have borne the brunt of public scrutiny, as features intended to help users organize and find photos have raised concerns about the use of customer photos to train this AI and other ways that this data might be used. While companies may protest that data stored on cloud servers is protected by encryption and that users have control, privacy advocates regularly raise concerns about data collection, the potential for misuse, and the creation of facial recognition databases without the users’ knowledge. 

Tech companies must navigate this turbulent sea of controversies and scrutiny if they want to explore the exciting new innovations that are only possible with AI. Among these tech companies is Synology, a private cloud solution provider that produces network-attached storage (NAS) hardware and software. Steven Liang, Synology’s Manager of Generative AI Applications, shared the following insights about how Synology works to respect customer privacy when designing new AI features. 

Step 1: Prioritize Customer Privacy When Training AIs 

Like many other companies, Synology is leveraging AI facial and object recognition to enhance photo management tools, automated security camera monitoring, and surveillance footage archives. With these applications, customer privacy starts all the way back in the development stage. When training these types of AIs to recognize faces and objects, many companies will use the photos and videos that users have placed on their cloud servers. This leads to major concerns about the true security of data that customers had assumed was private. 

Synology also provides its customers with both local and cloud storage solutions, but has ensured that this data can only be accessed by the customer; Synology could not access this data, even if it tried. All Synology users’ data either resides on their private Synology NAS devices, or is encrypted on Synology cloud servers using a key that is only available to the customer. As a result, the AI used for surveillance and photo apps is trained to recognize faces and objects using exclusively publicly available datasets, and only begins analyzing user photos and video when the user intentionally applies this AI feature. 

Step 2: Control AI Usage 

Another feature that customers have come to expect is LLMs (large language models) – the AI tool that acts as a generator and writing assistant for emails, slides, documents, and spreadsheets. While this feature is an incredible time saver for employees generating large amounts of text every day, this exposes organizations to the risk of joining the parade of news stories about AI content found in legal documentation, academic papers, and other high-value content. This issue is referred to as “shadow AI” – when students or employees begin using unsanctioned generative AI tools for tasks they would be expected to accomplish on their own. 

In order to deliver customers the convenience they expect from generative AI tools while also mitigating the risk of shadow AI, Synology introduced an AI admin console for system administrators. This tool allows admins to control the AI models and versions that are allowed within the organization and permissions for those models broken down by group or individual user. This ensures that users within an organization are only utilizing authorized AI tools and reduces the risk of unexpected AI contributions to official content. 

Step 3: Avoid Creating an AI Spy  

The early days of ChatGPT showed the ease with which users could gain access to restricted information. As understanding of how AIs retain information and generate responses increased, IT managers and admins raised concerns about the use of AI tools in professional environments. If employees provide the AI with prompts that include sensitive data, the AI has the potential to regurgitate that data to unauthorized individuals. 

To address the danger of AIs learning and reproducing proprietary data, Synology included a de-identification mechanism in the AI admin console. This mechanism works by recognizing when a user has generated a prompt that includes sensitive information such as emails, social security numbers, or IP addresses. Before the prompt is submitted to the AI, the mechanism performs de-identification, masking the sensitive data with generalized labels such as <ip_address> or <email>. The AI receives the sanitized prompt and produces a response that includes the generalized labels. At that point, the mechanism recognizes the labels and replaces them with the original information before presenting the completed response to the user. This ensures that the AI does not learn sensitive information and that this information does not leave the user’s network while using AI tools. 

Final Thoughts 

Perhaps the most important step to ensuring customer privacy and avoiding public scandal is a continued commitment to prioritizing privacy principles when approaching innovation and development. AI tools are advancing at an unprecedented rate, and companies need to take steps to foresee the valid concerns that can result from the rash application of these emerging technologies. When companies take a privacy-first approach to development, they can enjoy the benefit of AI advancements without compromising customer confidence or suffering public backlash.