@Caute_cautim wrote:
Accordingly, CISOs will need to direct their teams to conduct red team exercises against AI models and AI-enabled applications — just as security teams do with any traditional application, platform, or IT system.
I think the mentality is correct. These tools need to be tested, and we already have an ample supply of case studies where they weren't (or at least not sufficiently) before being sent into a production role.
In an earlier career in higher ed, a colleague and I started working on an academic paper about measuring technology return on investment. We never finished the project, in part, because the data was hard to get to. Organizations do a lot of analysis before a purchase or project, but very rarely do they confirm their original premise - Boards and senior management don't want to look in the mirror. Especially with AI, there is a need to look in the mirror before you send these things live: You think you are building a car, but it turns out you have actually built a four-wheeled bicycle.
The problem is there is a prevalent business misconception that technology of any kind equals efficiency. However, that is rarely the case - tech tends to allow incremental more with exponential more. To do AI well takes a large investment - including testing - but that bumps up against business drivers. In plainer terms, when Dr. Frankenstein finished all his surgery, it was hard for him to resist bringing his creation to life even though he wasn't sure what it would do.
