The Hidden Cybersecurity Risk in AI: Why Data Quality May Matter More Than the Model

Every AI system inherits the strengths and the blind spots of the data behind it. Yet boardroom discussions on AI security continue to revolve around model capabilities, jailbreaks, adversarial attacks, and governance frameworks, while a quieter risk receives far less scrutiny: the integrity of the data itself.

For CISOs, that distinction carries strategic weight. An AI model trained on incomplete, poorly governed, or drifting data does not simply produce inaccurate outputs. It can distort threat detection, weaken operational decision-making, expose critical infrastructure to unseen risk, and erode confidence in autonomous systems. Across manufacturing, healthcare, defense, energy, and industrial operations, the quality of AI decisions is inseparable from the quality of the data used to train and evaluate those systems.

This edition of CyberTech Intelligence Executive XChange explores that often-overlooked dimension of enterprise AI. Sudipto Ghosh, Host of the Expert Insights series and Head of Content Intelligence & Audience Growth at Intent Amplify, sits down with Brian Moore, Co-Founder and CEO of Voxel51, to examine why data observability, data sovereignty, and continuous evaluation deserve a permanent place in every enterprise AI security strategy.

The conversation shifts attention from model benchmarks to the questions every CISO should be asking.

Who controls enterprise AI data?

How should AI systems be evaluated before production?

Can data observability become as indispensable as network observability?

And why might proprietary data, rather than the model itself, determine both cyber resilience and long-term competitive advantage?

For security leaders responsible for governing AI at enterprise scale, these are not theoretical questions. They define whether AI becomes a trusted operational capability or an unmanaged source of business risk.

The conversation begins with a deceptively simple question. While the industry continues to pursue larger models and more sophisticated AI capabilities, has it underestimated the role of data quality in determining whether those systems can be trusted in production?

Sudipto: Hi Brian, welcome to CyberTech Intelligence. Enterprises continue to invest heavily in increasingly capable AI models. In your view, is the industry placing enough emphasis on the quality, governance, and observability of the data behind those models?

Brian: Not even close. The industry has spent the last few years in a model-centric mindset: bigger models, more parameters, better benchmarks. But models are rapidly becoming commodities. What isn't a commodity is your data and in physical AI especially, data quality is the single biggest determinant of whether a system ever makes it to production.

We see this every day with teams building autonomous vehicles, robotics, and industrial automation. The gap between a promising demo and a deployed system is almost never the model architecture. It's the long tail of edge cases, annotation errors, sensor drift, and distribution gaps hiding in the data. The teams that win are the ones that treat their datasets as first-class engineering artifacts versioned, evaluated, and continuously improved not as a static pile of files you point a training job at.

Sudipto: Cybersecurity conversations around AI often focus on prompt injection, model attacks, and AI-generated threats. How significant is poor-quality training and evaluation data as an enterprise security risk?

Brian: It's one of the most underrated risks in the field, because it doesn't look like an attack it looks like normal operations. A mislabeled dataset, an unrepresentative evaluation set, or silent data drift can produce a model that passes internal testing and then fails in the real world in ways nobody anticipated. In physical AI, that failure mode isn't a bad chatbot answer. It's a robot arm, a vehicle, or an inspection system making a wrong decision with physical consequences.

There's also a supply-chain dimension. Data poisoning is a real vector, but the more common problem is unintentional: teams ingest data from vendors, open datasets, and synthetic pipelines without the tooling to audit what's actually inside. If you can't inspect your data at scale, you can't secure the system built on top of it.

Sudipto: Physical AI is becoming increasingly important across robotics, manufacturing, autonomous systems, healthcare, and critical infrastructure. What unique security and operational challenges emerge when AI begins interpreting and acting upon the physical world?

Brian: Three things change fundamentally when AI touches the physical world.

First, the cost of failure goes up by orders of magnitude. A hallucinated paragraph is embarrassing; a misperceived pedestrian is catastrophic. That asymmetry demands a much higher evidentiary bar for what "production-ready" means, and that bar can only be met with rigorous, granular evaluation of both models and the data they were trained and tested on.

Second, the data itself becomes the crown jewels. Fleet footage, factory sensor streams, field imagery this data encodes years of operational knowledge that competitors can't replicate. It's the most valuable asset these companies own, which is exactly why enterprises should think hard before it leaves their premises.

Third, the environments are non-stationary. Weather, lighting, seasons, equipment wear the physical world drifts constantly. Continuous data curation and evaluation isn't a nice-to-have; it's the core operational loop.

Sudipto:  Voxel51 has focused on helping organizations better understand and evaluate their datasets. Why do you believe data observability is becoming just as important as model observability?

Brian: Because model observability tells you *that* something went wrong; data observability tells you *why* and what to fix. Model metrics are downstream symptoms. The root causes live in the data: class imbalance, annotation errors, coverage gaps, near-duplicate leakage between train and test sets.

The industry built a mature model-observability stack over the last five years, and it's genuinely valuable. But teams kept hitting the same wall: the dashboard says accuracy dropped, and now what? Without the ability to slice into the underlying data to find the exact samples, scenarios, and failure modes driving the regression you're guessing. We built FiftyOne because visual and multimodal data is where this problem is most acute. You can't grep a million images. You need purpose-built infrastructure to query, visualize, and evaluate at that scale.

Sudipto: AI systems inevitably reflect the strengths and weaknesses of the data used to train and evaluate them. What practical steps can enterprises take to continuously improve data quality and reduce AI-related operational risk?

Brian: A few that we see separate the strong teams from the rest:

Treat evaluation as a data problem, not just a metrics problem. Aggregate accuracy hides everything interesting. Break performance down by scenario, condition, and sample find the specific slices where the model fails and trace them back to gaps in the training data.

Audit before you annotate. Annotation is expensive. Most teams get far more lift from finding and fixing label errors and redundancy in existing data than from labeling more of the same.

Close the loop from production. Every deployment should feed hard cases back into curation. The teams shipping reliable physical AI treat this as a flywheel, not a one-time pipeline.

And keep the loop on your infrastructure. The moment your data-improvement workflow requires shipping proprietary data to a third party, you've traded operational control and possibly competitive position for convenience.

Sudipto:  Looking ahead, do you believe competitive advantage in enterprise AI will come primarily from better models or from organizations that better understand, govern, and continuously improve their proprietary data?

Brian: Data, decisively. Frontier models are converging in capability and are available to everyone, including your competitors. Your proprietary data is the only input to an AI system that is uniquely yours.

But I'd push the point further, because there's a strategic dimension enterprises are only beginning to reckon with. The large AI labs are not just model vendors anymore they're moving up the stack into vertical applications. We've watched labs use exposure to customer data and workflows to build products that compete directly with the entrenched leaders in those verticals. If you're a domain leader in agriculture, logistics, manufacturing, or defense, your operational data is precisely what a lab needs to challenge you in your own market.

So the question isn't just "which model should we use?"

It's "who gets to learn from our data?" The organizations that keep their data on their own terms on their premises, in their own tooling, improving models they control are the ones that convert data advantage into durable market advantage. The ones that pipe it out to third parties are, whether they realize it or not, funding their future competition.

Sudipto:  If you were advising a Chief Information Security Officer evaluating enterprise AI initiatives today, what are the three most important questions they should ask before approving production deployment?

Brian: One: Where does our data go, and what rights attach to it when it gets there? Read the data-use terms of every AI vendor and lab in the pipeline. "We may use your data to improve our services" can mean your proprietary edge is training someone else's product. Insist on architectures where sensitive data stays within your perimeter.

Two: How was this system evaluated, and would that evaluation survive scrutiny? Not "what's the accuracy?" but "on what data, covering which scenarios, with what known gaps?" If the team can't show granular, scenario-level evaluation on data that resembles the deployment environment, it isn't production-ready.

Three: What happens when the world changes? Every AI system degrades as its environment drifts. Who monitors that, on what cadence, with what tooling, and how does new data flow back into improvement? A deployment without a continuous data loop is a liability on a delay timer.

Sudipto:  Finally, what emerging trend in AI security, AI governance, or AI infrastructure do you believe deserves far more attention from enterprise leaders over the next three to five years?

Brian: Data sovereignty in the AI era. Enterprises spent a decade getting comfortable with cloud data governance, and that muscle memory is now being applied incorrectly to AI. Sending data to a SaaS application and sending data to an AI lab are categorically different acts. One is storage and processing; the other can be capability transfer.

Over the next three to five years, I expect the leading enterprises in every physical-world vertical to converge on a common posture: proprietary data stays on-premises or in their private cloud, models come to the data rather than the reverse, and the data engine the infrastructure for curating, evaluating, and improving that data is something they own and operate. The companies that build that muscle early will find their AI advantage compounds. The ones that don't will wake up to find their most valuable asset already priced into a competitor's product.

Rapid Fire

1. One AI misconception you wish would disappear.

Brian: That more data is better data. Volume without curation just scales your blind spots.

2. One technology area every CISO should monitor closely over the next 24 months.

Brian: The data-use and IP terms of foundation model providers. That fine print is where competitive risk now lives

3. The biggest obstacle preventing organizations from building trustworthy AI.

Brian: They evaluate models in aggregate and deploy them in specifics. Trust is built at the level of individual scenarios and edge cases, and most teams lack the tooling to see that granularly.

4. One capability that will define the next generation of enterprise AI platforms.

Brian:  A closed loop from production back to data: every real-world failure automatically becoming curated training and evaluation data – inside the enterprise's own perimeter.

5. One prediction for enterprise AI that you believe will become reality by 2030.

Brian: Every serious physical AI organization will run its own data engine the way every serious software organization runs its own CI/CD. Data infrastructure will be as non-negotiable as version control.

Conclusion

The enterprise AI conversation is entering a different phase. The debate has shifted from whether organizations should adopt AI to how they can deploy it with confidence, resilience, and accountability. That transition places data governance, observability, and sovereignty at the center of every AI security strategy.

As Brian Moore highlights throughout this discussion, trustworthy AI is not defined solely by model performance. It is shaped by the integrity of the data, the rigor of evaluation, and the operational discipline required to adapt as environments evolve. For CISOs, the challenge extends beyond securing AI systems. It includes understanding the provenance of enterprise data, preserving strategic control over proprietary information, and establishing continuous assurance throughout the AI lifecycle.

Those considerations will influence far more than cybersecurity outcomes. They will shape regulatory readiness, operational resilience, intellectual property protection, and the long-term competitiveness of AI-driven enterprises.

Our thanks to Brian Moore for sharing his perspectives and challenging several assumptions surrounding enterprise AI. As security leaders evaluate the next generation of AI platforms, one question deserves to remain at the forefront:

“If data is the foundation of every intelligent system, how much confidence do you have in the data your AI is learning from today?”

I'm Sudipto Ghosh, and this has been another edition of the CyberTech Intelligence Executive XChange. Thank you for joining us. We look forward to bringing you more conversations with the executives, innovators, and security leaders shaping the future of enterprise technology and cybersecurity.

About Brian Moore, Co-Founder and CEO – Voxel51

Dr. Brian Moore is co-Founder and CEO of Voxel51, where he leads the strategy and vision of the company and is actively involved in the design and execution of the core software infrastructure.

Brian holds a Ph.D. in Electrical Engineering from the University of Michigan, where his research focused on efficient algorithms for large-scale machine learning problems, with a particular emphasis on computer vision applications. He has authored dozens of articles in peer-reviewed journals and conference proceedings on various topics in machine learning, statistics, and optimization.