Addressing User Concerns About AI Data Use

⚠️Privacy Risks in AI Management: Lessons from Italy’s DeepSeek Ban⚠️ Italy’s recent ban on #DeepSeek over privacy concerns underscores the need for organizations to integrate stronger data protection measures into their AI Management System (#AIMS), AI Impact Assessment (#AIIA), and AI Risk Assessment (#AIRA). Ensuring compliance with #ISO42001, #ISO42005 (DIS), #ISO23894, and #ISO27701 (DIS) guidelines is now more material than ever. 1. Strengthening AI Management Systems (AIMS) with Privacy Controls 🔑Key Considerations: 🔸ISO 42001 Clause 6.1.2 (AI Risk Assessment): Organizations must integrate privacy risk evaluations into their AI management framework. 🔸ISO 42001 Clause 6.1.4 (AI System Impact Assessment): Requires assessing AI system risks, including personal data exposure and third-party data handling. 🔸ISO 27701 Clause 5.2 (Privacy Policy): Calls for explicit privacy commitments in AI policies to ensure alignment with global data protection laws. 🪛Implementation Example: Establish an AI Data Protection Policy that incorporates ISO27701 guidelines and explicitly defines how AI models handle user data. 2. Enhancing AI Impact Assessments (AIIA) to Address Privacy Risks 🔑Key Considerations: 🔸ISO 42005 Clause 4.7 (Sensitive Use & Impact Thresholds): Mandates defining thresholds for AI systems handling personal data. 🔸ISO 42005 Clause 5.8 (Potential AI System Harms & Benefits): Identifies risks of data misuse, profiling, and unauthorized access. 🔸ISO 27701 Clause A.1.2.6 (Privacy Impact Assessment): Requires documenting how AI systems process personally identifiable information (#PII). 🪛 Implementation Example: Conduct a Privacy Impact Assessment (#PIA) during AI system design to evaluate data collection, retention policies, and user consent mechanisms. 3. Integrating AI Risk Assessments (AIRA) to Mitigate Regulatory Exposure 🔑Key Considerations: 🔸ISO 23894 Clause 6.4.2 (Risk Identification): Calls for AI models to identify and mitigate privacy risks tied to automated decision-making. 🔸ISO 23894 Clause 6.4.4 (Risk Evaluation): Evaluates the consequences of noncompliance with regulations like #GDPR. 🔸ISO 27701 Clause A.1.3.7 (Access, Correction, & Erasure): Ensures AI systems respect user rights to modify or delete their data. 🪛 Implementation Example: Establish compliance audits that review AI data handling practices against evolving regulatory standards. ➡️ Final Thoughts: Governance Can’t Wait The DeepSeek ban is a clear warning that privacy safeguards in AIMS, AIIA, and AIRA aren’t optional. They’re essential for regulatory compliance, stakeholder trust, and business resilience. 🔑 Key actions: ◻️Adopt AI privacy and governance frameworks (ISO42001 & 27701). ◻️Conduct AI impact assessments to preempt regulatory concerns (ISO 42005). ◻️Align risk assessments with global privacy laws (ISO23894 & 27701).   Privacy-first AI shouldn't be seen just as a cost of doing business, it’s actually your new competitive advantage.

The EDPB recently published a report on AI Privacy Risks and Mitigations in LLMs.   This is one of the most practical and detailed resources I've seen from the EDPB, with extensive guidance for developers and deployers. The report walks through privacy risks associated with LLMs across the AI lifecycle, from data collection and training to deployment and retirement, and offers practical tips for identifying, measuring, and mitigating risks.   Here's a quick summary of some of the key mitigations mentioned in the report:   For providers: • Fine-tune LLMs on curated, high-quality datasets and limit the scope of model outputs to relevant and up-to-date information. • Use robust anonymisation techniques and automated tools to detect and remove personal data from training data. • Apply input filters and user warnings during deployment to discourage users from entering personal data, as well as automated detection methods to flag or anonymise sensitive input data before it is processed. • Clearly inform users about how their data will be processed through privacy policies, instructions, warning or disclaimers in the user interface. • Encrypt user inputs and outputs during transmission and storage to protect data from unauthorized access. • Protect against prompt injection and jailbreaking by validating inputs, monitoring LLMs for abnormal input behaviour, and limiting the amount of text a user can input. • Apply content filtering and human review processes to flag sensitive or inappropriate outputs. • Limit data logging and provide configurable options to deployers regarding log retention. • Offer easy-to-use opt-in/opt-out options for users whose feedback data might be used for retraining.   For deployers: • Enforce strong authentication to restrict access to the input interface and protect session data. • Mitigate adversarial attacks by adding a layer for input sanitization and filtering, monitoring and logging user queries to detect unusual patterns. • Work with providers to ensure they do not retain or misuse sensitive input data. • Guide users to avoid sharing unnecessary personal data through clear instructions, training and warnings. • Educate employees and end users on proper usage, including the appropriate use of outputs and phishing techniques that could trick individuals into revealing sensitive information. • Ensure employees and end users avoid overreliance on LLMs for critical or high-stakes decisions without verification, and ensure outputs are reviewed by humans before implementation or dissemination. • Securely store outputs and restrict access to authorised personnel and systems.   This is a rare example where the EDPB strikes a good balance between practical safeguards and legal expectations. Link to the report included in the comments.   #AIprivacy #LLMs #dataprotection #AIgovernance #EDPB #privacybydesign #GDPR

Using AI in Research? Transparency Isn’t Optional. As more researchers integrate AI tools for transcription, coding, or analysis, we’re also seeing a rise in participant concerns — and, increasingly, refusals — based on misconceptions about what AI actually does with their data. And honestly? Those concerns are valid. AI introduces new questions about privacy, data flow, and security. Participants deserve clarity, not jargon. Here’s the approach I’ve been championing, grounded in the STRESS Framework™ (Sensitivity, Transparency, Responsibility, Ethics, Skepticism, Security): 🔍 Be transparent: Tell participants when AI is used, what it does and doesn’t do, and how long data is stored. 🛡️ Prioritize security: Use vetted tools, encryption, and clear deletion timelines. 🧭 Stay ethical: Participation should always be voluntary — misconceptions are an opportunity to clarify, not persuade. 🤝 Build trust: Explain that AI assists with tasks like transcription, but human researchers still verify and interpret everything. 📄 Document responsibly: Keep clear records of how AI is used, how decisions are made, and how risks are mitigated. When participants understand the process, they’re more empowered — and our research becomes more ethical, transparent, and trustworthy. If you're looking to strengthen your own AI-use statements, consent materials, or research protocols, the STRESS Framework Assistant is an excellent tool to help you structure responsible AI documentation: 👉 https://lnkd.in/esFZEx34

Before I try any new AI tool, whether for my personal use or for work, I ask their customer support the following security-related questions (feel free to copy/paste): 1. Do you use customer data to train, fine-tune, or evaluate AI models beyond my individual account? * Prevent cross-customer learning. 2. If yes, is that data fully de-identified or aggregated? * Reduce re-identification risk. 3. Are AI models trained internally, by third-party providers, or both? * Know who actually touches the data. 4. Is customer data ever used to improve outputs for other customers? * Avoid silent data sharing. 5. Are AI interactions scoped strictly to my account context, or do models learn across customers? * Ensure my data stays mine. 6. Which third-party AI or ML providers process customer data? * Understand the extended trust chain. 7. Do those providers retain, log, or use customer data for their own training? * Avoid backdoor training use. 8. How long is customer data retained for AI or ML purposes? * Limit long-tail exposure. 9. If I request deletion, is my data removed from all downstream systems, including training or evaluation datasets? * Important one - this is nearly impossible to do once the toothpaste is out of the tube. If they say “yes,” then it’s a warning sign that the rest of their answers aren’t accurate. 10. What technical and contractual safeguards prevent misuse of customer data? Verify enforceable controls, not promises. This isn’t paranoia. It’s baseline data and privacy hygiene. AI is moving fast. Trust still has to be earned deliberately. If a vendor can’t answer these clearly, that’s the answer.

This Stanford study examined how six major AI companies (Anthropic, OpenAI, Google, Meta, Microsoft, and Amazon) handle user data from chatbot conversations.  Here are the main privacy concerns. 👀 All six companies use chat data for training by default, though some allow opt-out 👀 Data retention is often indefinite, with personal information stored long-term 👀 Cross-platform data merging occurs at multi-product companies (Google, Meta, Microsoft, Amazon) 👀 Children's data is handled inconsistently, with most companies not adequately protecting minors 👀 Limited transparency in privacy policies, which are complex and hard to understand and often lack crucial details about actual practices Practical Takeaways for Acceptable Use Policy and Training for nonprofits in using generative AI: ✅ Assume anything you share will be used for training - sensitive information, uploaded files, health details, biometric data, etc. ✅ Opt out when possible - proactively disable data collection for training (Meta is the one where you cannot) ✅ Information cascades through ecosystems - your inputs can lead to inferences that affect ads, recommendations, and potentially insurance or other third parties ✅ Special concern for children's data - age verification and consent protections are inconsistent Some questions to consider in acceptable use policies and to incorporate in any training. ❓ What types of sensitive information might your nonprofit staff  share with generative AI?  ❓ Does your nonprofit currently specifically identify what is considered “sensitive information” (beyond PID) and should not be shared with GenerativeAI ? Is this incorporated into training? ❓ Are you working with children, people with health conditions, or others whose data could be particularly harmful if leaked or misused? ❓ What would be the consequences if sensitive information or strategic organizational data ended up being used to train AI models? How might this affect trust, compliance, or your mission? How is this communicated in training and policy? Across the board, the Stanford research points that developers’ privacy policies lack essential information about their practices. They recommend policymakers and developers address data privacy challenges posed by LLM-powered chatbots through comprehensive federal privacy regulation, affirmative opt-in for model training, and filtering personal information from chat inputs by default. “We need to promote innovation in privacy-preserving AI, so that user privacy isn’t an afterthought." How are you advocating for privacy-preserving AI? How are you educating your staff to navigate this challenge? https://lnkd.in/g3RmbEwD

A recent issue has emerged where private ChatGPT conversations, once shared, have become publicly searchable on Google. This is a huge red flag for HR. Conversations containing sensitive information, like employee personal details from CVs, confidential business plans, or even legal advice, are now potentially exposed. My key takeaways: ▶️ Data Privacy Nightmare: This isn't just a technical glitch; it's a massive data privacy risk. Imagine employee PII, performance review details, or internal strategy documents showing up in a public search. This could lead to serious breaches and legal repercussions under regulations like GDPR or state privacy laws. ▶️ Policy and Training Gap: The root of the problem is a lack of awareness. Employees are using AI tools without fully understanding the privacy and security implications. This is a clear indicator that your AI policy needs to be robust and your training needs to be a top priority. Do your employees know what they should and shouldn't be putting into AI tools, or sharing from them? ▶️ Mitigation is Key: 🔸Audit Your Tools: Review which AI tools your employees are using and what data they might be processing. 🔸Revise Your Policy: Update your acceptable use policy to explicitly address the use of generative AI, including what types of information are strictly forbidden from being inputted or shared. 🔸Train Your People: Conduct urgent training sessions to raise awareness about the risks of sharing conversations from AI tools. This situation highlights the critical need for a proactive approach to AI governance in HR. It's no longer just about the tech; it's about the people using it and the sensitive data they handle. What's your biggest concern about employees using generative AI?

Over the past few weeks, a significant theme has emerged in conversations with customers, prospects, and internally at Relyance AI as we approach 2026: Shadow AI The rapid adoption of AI is outpacing governance efforts. Majority of teams we talk to are utilizing copilots, plug-ins, and SaaS AI features without the knowledge of security, privacy, or legal teams, leading to hidden data flows, compliance risks, and exposure of intellectual property. Key stakeholders who are often concerned include: ✅ Security & CISO teams: unmanaged access to sensitive data ✅ Privacy & Legal: unclear data use, cross-border processing, and regulatory exposure ✅ IT & Engineering: tool sprawl, duplication, and inconsistent controls ✅ Risk & Compliance: lack of inventory and audit trails To enhance AI posture without hindering team productivity, teams leading the pack are considering these simple steps: ✔️ Create a basic AI inventory detailing what tools are being used, by whom, and with what data ✔️ Define approved versus restricted AI use cases ✔️ Monitor data inputs to AI tools, particularly personally identifiable information (PII), source code, and intellectual property ✔️ Embed lightweight guardrails early on to prevent complications as AI becomes more entrenched Shadow AI is not a future issue—it is already present. Companies that succeed will focus on enabling AI safely rather than attempting to ban it.

The California AG issues a useful legal advisory notice on complying with existing and new laws in the state when developing and using AI systems. Here are my thoughts. 👇 📢 𝐅𝐚𝐯𝐨𝐫𝐢𝐭𝐞 𝐐𝐮𝐨𝐭𝐞 ---- “Consumers must have visibility into when and how AI systems are used to impact their lives and whether and how their information is being used to develop and train systems. Developers and entities that use AI, including businesses, nonprofits, and government, must ensure that AI systems are tested and validated, and that they are audited as appropriate to ensure that their use is safe, ethical, and lawful, and reduces, rather than replicates or exaggerates, human error and biases.” There are a lot of great details in this, but here are my takeaways regarding what developers of AI systems in California should do: ⬜ 𝐄𝐧𝐡𝐚𝐧𝐜𝐞 𝐓𝐫𝐚𝐧𝐬𝐩𝐚𝐫𝐞𝐧𝐜𝐲: Clearly disclose when AI is involved in decisions affecting consumers and explain how data is used, especially for training models. ⬜ 𝐓𝐞𝐬𝐭 & 𝐀𝐮𝐝𝐢𝐭 𝐀𝐈 𝐒𝐲𝐬𝐭𝐞𝐦𝐬: Regularly validate AI for fairness, accuracy, and compliance with civil rights, consumer protection, and privacy laws. ⬜ 𝐀𝐝𝐝𝐫𝐞𝐬𝐬 𝐁𝐢𝐚𝐬 𝐑𝐢𝐬𝐤𝐬: Implement thorough bias testing to ensure AI does not perpetuate discrimination in areas like hiring, lending, and housing. ⬜ 𝐒𝐭𝐫𝐞𝐧𝐠𝐭𝐡𝐞𝐧 𝐆𝐨𝐯𝐞𝐫𝐧𝐚𝐧𝐜𝐞: Establish policies and oversight frameworks to mitigate risks and document compliance with California’s regulatory requirements. ⬜ 𝐌𝐨𝐧𝐢𝐭𝐨𝐫 𝐇𝐢𝐠𝐡-𝐑𝐢𝐬𝐤 𝐔𝐬𝐞 𝐂𝐚𝐬𝐞𝐬: Pay special attention to AI used in employment, healthcare, credit scoring, education, and advertising to minimize legal exposure and harm. 𝐂𝐨𝐦𝐩𝐥𝐢𝐚𝐧𝐜𝐞 𝐢𝐬𝐧’𝐭 𝐣𝐮𝐬𝐭 𝐚𝐛𝐨𝐮𝐭 𝐦𝐞𝐞𝐭𝐢𝐧𝐠 𝐥𝐞𝐠𝐚𝐥 𝐫𝐞𝐪𝐮𝐢𝐫𝐞𝐦𝐞𝐧𝐭𝐬—it’s about building trust in AI systems. California’s proactive stance on AI regulation underscores the need for robust assurance practices to align AI systems with ethical and legal standards... at least this is my take as an AI assurance practitioner :) #ai #aiaudit #compliance Khoa Lam, Borhane Blili-Hamelin, PhD, Jeffery Recker, Bryan Ilg, Navrina Singh, Patrick Sullivan, Dr. Cari Miller

🤖Sharing data on an AI platform like ChatGPT can have serious implications, especially when it's PERSONAL DATA. During #CybersecurityAwarenessMonth, let's raise awareness of what data should NOT be put into a public AI platform. ➡️A recent post by Jason highlighted this exact case when "Flood survivors in the Northern Rivers trusted a government agency to help them rebuild. Now their personal details including names, addresses, health info, etc have potentially been exposed… because someone uploaded a spreadsheet into ChatGPT! 🔔Twelve Thousand Rows of DATA. A former contractor shared a live Excel sheet into ChatGPT (using the free version), which is not designed for storing, processing and/or safeguarding sensitive data. This wasn’t a malicious act, this was a gap in training, governance and tooling. And now 3,000 real people in flood zones, people who’ve already lost everything once, are wondering who’s seen their data. #AI tools don’t cause breaches, but the people using them without guardrails do!! AI is now embedded in the daily workflows of almost every professional. If your staff can Google, they can prompt. And if they can prompt, they can leak data by accident or design. The question is no longer “Should we allow staff to use AI?” The questions really are/should be... ❓Have we trained them? ❓Are we monitoring usage? ❓And do we have sandboxes in place for safe exploration? Here’s what every agency, council and company should do today... ⁉️ Block external AI tools from handling sensitive data - Use enterprise grade versions or secure local deployments where prompts aren’t stored. ⁉️ Issue a clear AI Acceptable Use Policy - Not later... NOW! Include examples and limits. ⁉️ Train every staff member and contractor - Especially the ones working with customer or public data. AI literacy isn’t a nice to have anymore! ⁉️ Set up internal prompts systems with privacy by design - If you’re using AI internally, ensure it’s fully logged, encrypted and wiped clean of sensitive content. ⁉️ Create an AI red team - Find the holes before someone else does. This isn’t about bashing the NSW Reconstruction Authority, it’s a warning to all. If it can happen there, it can happen anywhere! Not because we’ve got bad people but because we’ve got good people using powerful tools without a map." #AI is a powerful tool, and with great power comes great responsibility. Do you need help in educating and building your human firewall? Why not reach out to the amazing team at ASE Tech to find out how. #ShiftHappens #ThinkBeforeYouClick

Interesting JAMA research letter that assessed patient trust in use of AI by health systems. The authors surveyed a representative sample of patients and found that 65% did not trust health systems to use AI responsibly and more interesting, 57.7% had low trust of their health systems in general. It's understandable that patients may be concerned about a new technology being used in their care. So physicians and healthcare organizations and systems should endeavor to reduce patient concerns. What can health systems do to promote trust when implementing AI? --Disclose when and how AI is being used for care --Ensure patient data is kept private. Obtain consent and disclose how data is being used. --Engage patients in AI policy decisions --Physicians and other AI users in healthcare should demand testing for validity and safety. Patients have a right to know new technology is reliable and safe. Am I missing anything? #AI #healthcare https://lnkd.in/e9GWCJEX