topic Re: ChatGPT data handling in Governance, Risk, Compliance

ChatGPT data handling

rami99 — Sat, 25 Mar 2023 20:10:52 GMT

What do you think are the risks of generative AIs such as ChatGPT with regards to data storing (user inputs/prompts), data handling e.g ChatGPT storing user inputs / output (where and how and is it concerning?) and how would you go about risk management.

Re: ChatGPT data handling

Steve-Wilme — Mon, 27 Mar 2023 07:58:46 GMT

Until you've found a way of assessing the risk posed, you could block access via your corporate proxy.

Re: ChatGPT data handling

AndreaMoore — Mon, 03 Apr 2023 18:38:06 GMT

We recently had a blog post on the topic of ChatGPT.

Nobody predicted how rapidly AI chatbots would change perceptions of what is possible. Some worry how it might improve phishing attacks. More likely, experts think, will be its effect on targeting. Click to read more on the blog:

ANALYSIS: WILL CHATGPT’S PERFECT ENGLISH CHANGE THE GAME FOR PHISHING ATTACKS?

What are your thoughts?

Re: ChatGPT data handling

Steve-Wilme — Thu, 06 Apr 2023 10:09:42 GMT

The quality of the English is only one of a number of indicators that an email is phishy. It's important to look for the other signs, such as, Are you a named addressee? Does it prompt to act with urgency? Does it attempt to mimic someone senior in your organisation? Does it sound too good to be true?

Re: ChatGPT data handling

JoePete — Thu, 06 Apr 2023 13:39:46 GMT

@Steve-Wilme wrote:
The quality of the English is only one of a number of indicators that an email is phishy. It's important to look for the other signs, such as, Are you a named addressee? Does it prompt to act with urgency? Does it attempt to mimic someone senior in your organisation? Does it sound too good to be true?

Back when WiFi became a thing (I know, dating myself), and people were freaking out about its impact on network security, I remember someone noting that if WiFi had you worried, you were probably living with a false sense of security. A lot of the talk about ChatGPT strikes me the same way: If you see it as some watershed threat, you've probably been more vulnerable than you realize. If misspellings are what your employees look for to tip them off to phishing, yikes! Have you seen what passes for "professional" communications these days?

I welcome the flaming on this statement, but defeating phishing can be solved, not by going forward, but by going back: Stop using HTML email! It doesn't do what most people think it does, but it certainly obfuscates links and creates the distraction that allows such scams to succeed. Now, I'll grant that in limited, intranet-level type communications, it may be OK to use. Regardless, ChatGPT advantage to the bad guys is that it is a longer lever for exerting leverage but not that it is a new category of threat. If AI/ChatGPT is the big bad wolf, then maybe it's finally time to tell grandma to get new glasses, lock her door, and stop letting strangers in the house - things she should have been doing years ago.

Re: ChatGPT data handling

denbesten — Thu, 06 Apr 2023 18:41:23 GMT

@JoePete wrote:

I welcome the flaming on this statement, but defeating phishing can be solved, not by going forward, but by going back: Stop using HTML email!

Email phishing is just the latest evolution of a long-standing problem. Confidence men have been an issue "forever". And official-looking and personalized phishing postal mail still arrives to this day. Upgrading (or downgrading 😏 ) technology is just one more whac in the Whac-A-Mole game.

Truly "solving" phishing requires figuring out how to ensure people to treat others honestly, truthfully and with respect, which is way beyond "Security" (and maybe even mortals).

The only bit I see where IT (not just "security") can help is by throwing light on the tells. I agree that link obfuscation (or more generally, homograph attacks) is the most likely area where we can contribute.

Re: ChatGPT data handling

JoePete — Fri, 07 Apr 2023 12:25:21 GMT

@denbesten wrote:
Truly "solving" phishing requires figuring out how to ensure people to treat others honestly, truthfully and with respect, which is way beyond "Security" (and maybe even mortals).

A variant of this topic is how schools in the US have adopted different curricula meant to teach kids how to spot "good" online sources or even phishing. Good intent, but in my (limited) experience, they do it all wrong. Back in my day, we learned the logical fallacies. Talk about time-tested reasoning. I am fairly convinced that Aristotle would never have been duped by a password reset scam. If anything, the approach I've seen in schools embraces a fallacy (e.g., appeal to authority), but so be it.

The older I've gotten (or at least crankier), the younger the age range I look to as to where the problem is. At least in the US, I think schools have been overwhelmed by technology to the point where it detracts from teaching critical thinking and communication skills. Along with that, I would agree, that we've lost larger ideas. We've become very transactional (input for output). By they way, I knew you were going to call foul on me for my HTML ranting; if nothing else, they'll write on my tombstone "did not send HTML email," and it will be written in plaintext 😉

Re: ChatGPT data handling

denbesten — Sat, 08 Apr 2023 04:01:31 GMT

Happy to oblige, but my true intention was to call out that social engineering isn't a technological issue in the first place. Sure, tech makes it worse, and tech can help a bit, but in the end, it is a behavioral problem that as you point out, is best addressed by teaching critical thinking skills.

Re: ChatGPT data handling

Martinjoe23423 — Sun, 11 May 2025 02:56:13 GMT

ChatGPT takes data handling seriously to ensure user privacy and security. It adheres to strict guidelines for data collection, processing, and storage, with a strong focus on user confidentiality. User interactions are securely processed, and any data is managed under strict privacy policies of PK365. Personal information shared during conversations is not stored or used beyond the session unless explicitly permitted by the user. OpenAI continuously improves data protection measures to maintain high standards of data security and user trust.

Re: ChatGPT data handling

mrsimon0007 — Sun, 23 Nov 2025 10:57:46 GMT

You can block access through your corporate proxy until you have a proper way to assess the risk.

Re: ChatGPT data handling

OliLue — Fri, 12 Dec 2025 08:36:51 GMT

Hi all,

i think blocking KI tool (ChatGPT, Copilot etc) is not the way to handle this topic. I ´m not familiar with all the technical possibilities to control the use of KI.

I tried to list some points to use KI. https://www.cybersecurity-luerssen.com/en/post/ki-meets-regulation

And one important point is the ethical guideline which should be guided everyone in the company.

From my perspective there is not the all in answer, there is a way of learning how to use the KI and handle the data in a responsible way.

I looking forward to this discussion

Re: ChatGPT data handling

mrsimon0007 — Sun, 21 Dec 2025 11:04:06 GMT

Generative AI systems can raise concerns around how user inputs are stored, processed, and potentially used for model improvement. Risks include unintended data retention, exposure of sensitive information, and lack of transparency about storage and access. Effective risk management involves limiting sensitive inputs, strong data governance, clear retention policies, encryption, access controls, and user awareness about how data is handled.

Re: ChatGPT data handling

mfak1122 — Sun, 18 Jan 2026 21:02:41 GMT

Thanks for the info

Re: ChatGPT data handling

Caute_cautim — Wed, 28 Jan 2026 02:53:08 GMT

@mfak1122

Having worked within IBM for 23 years, one of the key issues that came up is their philosophy towards AI, I suggest you research and adopt their approach: This is a great starting point.

IBM's perspective on AI governance is centered on creating trustworthy, transparent, and explainable AI that augments human intelligence rather than replacing it. IBM advocates for a risk-based,, collaborative approach to regulation that focuses on high-risk applications, avoids restrictive licensing regimes, and supports an open-source innovation ecosystem.

Key pillars of IBM's AI governance strategy include:

1. Core Principles for Trustworthy AI

IBM defines Trustworthy AI through five fundamental pillars:

Transparency: Disclosing how AI systems are designed, developed, and trained.
Explainability: Ensuring AI-driven decisions can be interpreted and understood by humans.
Fairness: Actively managing and reducing bias to ensure equitable treatment.
Robustness: Enabling AI to handle unexpected conditions and resist technical or adversarial attacks.
Privacy: Safeguarding consumer data and maintaining data rights.

2. The "Augmentation" Philosophy

IBM believes that AI is intended to augment, not replace, human intelligence. Governance should ensure that AI acts as a tool to enhance human capabilities, with humans remaining in the loop for critical decision-making.

3. Regulatory and Policy Perspective

Regulate Risk, Not Algorithms: IBM argues against licensing regimes that could hinder innovation, advocating instead for regulating the context and use of AI, particularly high-risk scenarios.
Support for the EU AI Act: IBM welcomes the risk-based approach of the EU AI Act.
Data Provenance: IBM emphasises that trustworthy data is the foundation of AI and supports industry-wide data provenance standards to track data origin.

4. Operationalising Governance (watsonx.governance)

IBM emphasises that governance must move from theoretical principles to practical, automated application across the AI lifecycle.

watsonx.governance: A, AI-powered toolkit designed to help organizations monitor, audit, and manage AI models for compliance, bias, and drift.
Internal Governance Structure: IBM utilises the "Responsible Technology Board" (formerly AI Ethics Board) to review AI use cases, supported by an Advocacy Network and Policy Advisory Committee.

5. Commitment to Openness

IBM believes that an open innovation ecosystem is critical for safe, diverse, and rapid AI development. Examples include co-founding The AI Alliance, releasing the Granite models into open source, and collaborating on projects like InstructLab or Qiskit.

IBM's approach to AI governance treats it not as a regulatory burden but as a business enabler that increases confidence in AI, boosts ROI, and strengthens reputation.

Ensure the organisation has an agreed AI Governance and associated strategy, always use the "Enterprise" version and do not allow employees to use the "free" AI models from within the organisation. It will only end up in court cases, data leakage and possible loss of company IP, which has been built up over a long period of time. Above all educate all employees, why you have taken these steps i.e., to protect the organisation, the individuals and to ensure productivity enhancements and efficiencies.

Regards

Caute_Cautim

Re: ChatGPT data handling

pamelat — Sat, 31 Jan 2026 10:48:59 GMT

The risks around generative AI like ChatGPT often come down to data governance, privacy, and model training assumptions. From a risk management perspective, one key consideration is understanding what data is logged, how long it’s retained, and how it’s used in training or inference, especially in enterprise environments where sensitive information may be input into AI systems. Proper policies should define clear boundaries for input sanitization, data classification, and logging practices, and integrate those into existing governance and compliance frameworks so that AI usage doesn’t inadvertently expose confidential data or violate privacy requirements. It’s also important for organizations to conduct periodic risk assessments of any AI service they integrate, updating controls as models evolve and regulatory guidance matures.