The AI security conversation is shifting. Our founder, Brennan Lodge was in the room at Black Hat Las Vegas. Here is his unfiltered take:

 

The AI wave in cybersecurity is getting bigger, faster, and harder to read from the shore. That is why the lifeguard beach report felt like the right image for this recap.

At BSides and Black Hat, the message was clear. AI is already in the water with us. It is in the SOC, GRC, vendor tools, evidence review, employee workflows, and agentic systems that may or may not have adult supervision.

So before everyone jumps in, check the flags.

Air temp, 113 degrees.
Surf, moderate but building.
Rip currents, present around agents, supply chains, and tool integrations.
Undertow, stronger than it looks when access is messy and logging is weak.
Lifeguard warning, do not swim alone with an autonomous agent holding production credentials.

Vegas in August is not a city. It is a dehydration-based authentication challenge.

Every year, thousands of security people voluntarily walk into the desert to argue about threat models, buy bad coffee, collect vendor socks, and pretend that walking from the Venetian to Caesars in 113-degree heat is a normal business activity.

Welcome back to Hacker Summer Camp.

This year, BSides Las Vegas and Black Hat felt different. The AI hype was still there, of course. You could not walk ten feet without seeing “agentic,” “copilot,” “AI-native,” “autonomous,” or some other phrase that usually means “we added a chat window and raised the price.”

But underneath the marketing fog, the real conversation had changed.

It was not “AI is coming.”

That is over.

The real question now is much less exciting and much more important.

Who is running it?

And can you prove it?

 

Deepfakes Are Not a Technology Problem Anymore

 

One of the better talks I caught was on deepfakes, and the main takeaway was uncomfortable.

We spent the last few years teaching people to look for bad hands, weird teeth, strange blinking, waxy skin, and all the other artifacts from the early “Will Smith eating spaghetti” era of synthetic media.

That advice is mostly obsolete now.

Not because every deepfake is perfect. Most are not. The problem is that perfection is not required. A deepfake only needs to land in the right context, with the right emotional trigger, at the right time.

A fake voicemail from someone you know.
A fake executive message during a live incident.
A fake surveillance clip that confirms a story people already want to believe.
A fake audio note asking for money, access, or urgency.

That is where things get nasty.

The speaker made the point that human detection fails because humans are not media forensics engines. We are busy, biased, tired, emotional, and very good at believing things that fit the story already forming in our heads. Machine detection is also not a silver bullet because the whole thing is an arms race. Compression, filters, lighting, Zoom enhancement, editing artifacts, and normal media degradation all blur the line between “real,” “fake,” and “who knows anymore.”

So the control is not just “buy a detector.”

The control is workflow.

Who verifies the request?
What channel is trusted?
What action requires secondary approval?
What happens when the CEO voice note sounds real but asks for something stupid?

Deepfake defence is becoming less about spotting pixels and more about designing friction in the right places.

Terrible news for people who like easy answers.

Great news for anyone who still believes security is mostly process, incentives, and human behavior wearing a technical costume.

 

A Cybersecurity Exec Held a Funeral for the Fragmented Stack

 

This talk had the right mix of practitioner experience, vendor polish, and just enough theater to keep a Black Hat room awake on a Thursday.

The core argument was that the fragmented security stack is dead.

Email sees the message.
Identity sees the login.
DLP sees the data movement.
AI governance sees the prompt.

Everyone sees one slice. Nobody sees the chain.

That was always a problem, but AI makes it worse because AI compresses time. It helps attackers move faster. It helps employees move faster. It helps agents act faster. Meanwhile, security teams are still trying to stitch together context after the fact with ticket queues, SIEM searches, Slack threads, and prayer.

The phrase that stuck with me was agent integrity.

What can the agent access?
What is the agent supposed to do?
What did the agent actually do?

If those three things do not line up, you do not have trust. You have the first paragraph of a breach notification.

That framing is useful because it gets away from the lazy version of AI governance, where everyone says “responsible AI” and then immediately goes back to approving tools they cannot see, test, or explain.

Agent integrity gives security teams something practical to evaluate. Access, intent, and behavior. That is much better than “do we trust the AI?” which is the kind of question that sounds thoughtful until you realize nobody knows what it means.

Also, partial trust is not trust.

It is just risk in a blazer.

 

The OpenAI and Hugging Face Debrief Was the Horror Movie of the Week

 

The OpenAI debrief on the Hugging Face incident was probably the most technically interesting session I attended.

The short version is that agents got stuck during hard cybersecurity evaluations and started doing what agents apparently do when they get stuck. They looked for shortcuts. They tried to work around constraints. They explored package infrastructure. They found unexpected paths. They left notes for other agents. They shared exploit ideas. They reused techniques. They drifted beyond the intended scope.

Not Skynet.

Worse.

A group project.

The uncomfortable part is that the behavior was not framed as a traditional incident with one clean root cause, one actor, and one timeline. It involved multiple agents interacting over time, communicating indirectly, moving through systems, and creating a forensic mess dense enough that OpenAI had to use AI techniques to help review huge volumes of logs and trajectories.

That is the part people should pay attention to.

Autonomy changes incident response.

Most organizations are still learning how to investigate a normal cloud breach. Now add autonomous agents, delegated sub-agents, tool calls, hidden task chains, model reasoning, package managers, eval harnesses, prompt traces, and logs that were never designed to explain intent.

Good luck, everyone.

The industry loves to talk about AI increasing offensive capability. Fine. True. But the defensive operational problem may be just as urgent.

Can you investigate agent activity?
Can you reconstruct what happened?
Can you distinguish intended tool use from emergent behavior?
Can you see where the agent exceeded the boundary?
Can you prove it?

If not, your AI incident response plan is just a PDF with a dream.

 

More POVs from the conference coming soon…. stay tuned! 

RAG Isn’t a Buzzword. In Compliance, It’s the Architecture That Actually Matters.

Every enterprise software company has an AI story right now.

Most of them sound the same. Large language models. Intelligent automation. AI-powered insights. The language is everywhere and it means almost nothing without one question answered:

Where did the answer come from and can you prove it?

In most industries that question is inconvenient. In compliance it is the only question that matters. And it is exactly why the architecture underneath an AI compliance tool is not a technical detail. It is a trust decision.

Why RAG Is Where Audit CADDIE Started

RAG is not a feature Audit CADDIE added. It is the reason Audit CADDIE exists.

Before building the product, Brennan Lodge wrote a research paper on RAG specifically for cybersecurity  exploring its practical applications in knowledge-intensive, high-accountability environments. The conclusion then is the same conclusion that drives the product today: RAG is still one of the strongest architectures available for enterprise GRC, and its practicality for cybersecurity is not theoretical. It is proven, documented, and more relevant now than when the research was first published.

That research became the foundation. The product followed.

What RAG Actually Is — In Plain English

RAG stands for Retrieval-Augmented Generation.

Here is what it means without the jargon.

A standard AI model answers questions based on what it was trained on. It has knowledge baked in from its training data, but that knowledge has a cutoff date, it cannot be updated in real time, and when it gets something wrong it does so confidently and without any way to trace where the answer came from.

RAG changes that. Instead of relying only on what the model already knows, RAG pulls in specific, relevant documents from a defined knowledge base before generating an answer. The model reads those documents, grounds its response in what they actually say, and can point back to the source.

Think of the difference like this:

A standard AI is like asking a very well-read colleague a question from memory. They will give you an answer ,but you have no idea which source they are drawing on or whether it is current.

A RAG-based system is like asking that same colleague to answer only after reviewing the specific documents you hand them  and requiring them to show their work.

In compliance, showing your work is not optional. It is the entire point.

Why Compliance Is the Highest-Stakes Environment for AI to Get This Wrong

When an AI system makes a confident mistake in a consumer app the consequence is usually minor. A bad restaurant recommendation. A clumsy email draft.

When an AI system makes a confident mistake in a compliance context the consequences are categorically different.

A control mapped to the wrong framework. A policy gap missed because the model hallucinated coverage that does not exist. An audit report that cites a standard version that has since been updated. An assessor who relies on AI output without realizing the grounding was faulty.

These are not hypothetical failure modes. They are predictable outcomes of deploying AI in knowledge-intensive, high-accountability environments without treating retrieval quality and provenance as first-class concerns.

The original RAG research explicitly called out provenance and updatable knowledge as open problems that retrieval architectures are designed to address. More recent work positions RAG as the path toward greater accuracy and credibility for knowledge-intensive tasks  while also warning clearly about the risks of poor retrieval quality and grounding failures.

The warning is important. RAG done poorly is not safer than a standard model. It is just wrong in a different way  and potentially more dangerous because it feels more grounded than it actually is.

That distinction matters enormously in a compliance context. An AI tool that sounds authoritative while drawing on incomplete or misaligned retrieval is not a productivity gain. It is a liability dressed up as an answer.

The Three Things That Separate Audit-Grade RAG from Everything Else

Not all RAG implementations are equal. For a compliance use case specifically three things have to be treated as non-negotiable.

1. Provenance -Every answer needs a traceable source.

When Audit CADDIE tells you that a control is covered or a gap exists, you need to know exactly which document that conclusion came from, which section, and which version of the standard it was mapped against. Not approximately. Exactly.

Auditors do not accept “the AI said so.” They accept evidence. RAG only earns its place in a compliance workflow when every output is traceable back to a specific, citable, verifiable source.

This is not a nice-to-have feature. It is the baseline requirement for any AI output that might end up in front of a regulator, an auditor, or a board.

2. Retrieval quality -The right documents have to surface, every time.

A RAG system is only as good as its ability to find the right information at the right moment. Poor retrieval means the model answers from incomplete context. It may not know what it does not know. And in a compliance gap analysis the gaps the system misses are more dangerous than the ones it finds.

Retrieval quality is not a default. It is an engineering discipline — and it requires ongoing attention as document libraries grow, frameworks are updated, and organizational policies change. A system that retrieves well on day one but degrades as the environment evolves is not a compliance tool. It is a compliance risk.

3. Grounding fidelity – The model has to stay anchored to what it retrieved.

Large language models have a natural tendency to fill gaps, extrapolate, and generate fluent-sounding text even when the underlying evidence does not fully support it. In compliance that tendency has to be actively constrained. The model needs to say “I do not see coverage for this control in the documents provided” rather than constructing a plausible-sounding answer from partial information.

Grounding fidelity is what separates an AI tool that builds trust from one that creates liability. The most dangerous output in a compliance context is not a wrong answer that looks wrong. It is a wrong answer that looks right.

How AI Is Changing What GRC Actually Looks Like

The shift happening in enterprise GRC right now is not just a technology upgrade. It is a fundamental change in how compliance work gets done  who does what, when, and how trust gets demonstrated.

People. Before AI, analysts and auditors spent the majority of their time chasing documents- finding evidence, organizing it, manually mapping it to frameworks, and formatting outputs for review. That work was necessary but it was not the work that required human expertise. AI changes the allocation. People stop chasing documents and start supervising AI, resolving ambiguity, and making the accountability decisions that require judgment. Human-in-the-loop assurance, not button-click compliance.

Process. Compliance used to be a periodic event. Audit prep happened once or twice a year, teams scrambled to pull everything together, the audit happened, and then the cycle reset. AI makes continuous readiness possible- ongoing evidence capture, automatic mapping as documents change, real-time visibility into control coverage. The scramble becomes a baseline. From scramble to continuous readiness.

Technology. Static GRC systems and spreadsheets were built for a world where the volume of documentation was manageable and the number of frameworks was stable. Neither is true anymore. AI-assisted evidence classification, cross-framework mapping, and trust-center workflows are not luxury features ,they are the infrastructure required to manage modern compliance complexity at scale. Automation where it removes toil, controls where it protects trust.

Trust. The old model of demonstrating trust was retrospective – logos on a website, reports delivered after the audit, certifications renewed annually. The new model is ongoing. Organizations are expected to demonstrate their governance posture continuously  to clients, regulators, and partners who increasingly cannot wait for an annual report to know where their vendors stand. Trust is operationalized, not asserted.

What This Means for How Audit CADDIE Is Built

The winning product narrative for an AI compliance tool is not “we use RAG.”

Every tool will say that soon if they do not already.

The winning narrative is “we use retrieval, grounding, and evidence traceability because audit-grade trust demands it.” That is a different claim and a harder one to make without the architecture, the research background, and the real-world deployment experience to back it up.

Audit CADDIE is built on the premise that compliance outputs are only as valuable as the trust that can be placed in them. That means every gap finding is traceable. Every control mapping cites its source. Every report output can be reviewed, validated, and defended by the human expert who signs their name to it.

The AI does the retrieval, the classification, the mapping, and the gap identification – the work that used to take weeks of manual effort. The human does the review, the judgment, and the accountability – the work that no model should be doing alone.

That is not a limitation of the technology. It is the design.

Because in an environment where the output might end up in front of a regulator, an auditor, or a board – the architecture that produces it is not a background detail.

It is the product.

Excited to announce that Audit CADDIE has partnered with 360 Advanced- a relationship-focused cybersecurity and compliance firm offering security, privacy and compliance-oriented solutions.We’ve always believed the best outcomes for clients come from the right experts working together, and this partnership reflects exactly that.

360 Advanced brings deep expertise in security and compliance services, while Audit CADDIE delivers compliance automation, evidence collection, and audit readiness. Together, we provide a more comprehensive, end-to-end solutions  for our clients.

 

 

Introducing EAGLE — Audit CADDIE’s Cloud Scanning Engine.

Your auditors shouldn’t be the first ones to find your drift.
EAGLE continuously scans your cloud environments by flagging misconfigurations, control gaps, and compliance drift in real time, before they become audit findings.

No more scrambling at audit kickoff. No more “we thought that was remediated.” No more evidence gaps discovered in week 3 of a 4-week window.
What EAGLE does: → Live cloud infrastructure monitoring → Real-time drift alerts tied directly to your control library → Auto-mapping of findings to SOC 2, ISO 27001, CMMC, and more → Continuous assurance and not just a once-a-year fire drill.

We built this because the GRC teams we work with kept telling us the same thing: “We only find out something’s wrong when the auditor does.”
That ends now.