By Lars Hegenberg | Trace3 Senior Innovation Researcher
In an era obsessed with shiny AI tools, everyone claims to understand that a good data posture is the price of admission for AI. Yet very few are willing to roll up their sleeves for the unglamorous groundwork that makes these products work.
Ever since the term “Big Data” crossed over to a broader audience, including executives and journalists, organizations have been trying to catch up to leverage in-house data to inform decision-making and innovation.
The hurdles have always been around:
- Data Curation and Availability: How can we break down data silos and organize and maintain datasets in a way that makes them easily accessible for users and tools across the organization?
-
Data Quality: How do we avoid flawed insights and erroneous decisions derived from inaccurate, inconsistent, and incomplete data?
-
Data Governance: How do we ensure the integrity and safety of data, preventing misuse, and regulation non-compliance?
And while these challenges have been around for a while, the era of generative AI and agentic AI has re-emphasized the importance of high-quality, well-governed, and fresh data. It has left many organizations scrambling to improve their data posture in fear of missing out on the exciting new wave of tools coming to market.
AI has not only raised the stakes, it has shifted the goalposts of data readiness by changing the requirements for usable data.
So, what exact changes have the latest developments in AI brought about, and how are emerging solutions stepping up to help you get AI-ready?
Key Developments in Data
Three major themes are underpinning the changing requirements for Data Readiness. This is also where we see a wave of emerging solutions coming to market that help organizations kick-start their efforts.
AI-fication
Not that long ago, the Modern Data Stack became one of the most exciting concepts in the world of data. It gravitated around the fast-growing cloud data warehouses with vendors both upstream and downstream. However, it focused on structured data, the type of data that fits neatly into rows and columns. Generative AI tools instead are largely fed by unstructured data such as text, images, audio, and video that lack consistent formatting. This requires organizations to widen their scope to cover multiple modalities and invest in tooling that supports these accordingly.
Emerging as the holy grail of automation, agentic AI solutions have seen exceptionally quick rollouts. Tempted by the great upside potential, most organizations underestimate the expanded data requirements and risks that can lead to failed or underperforming initiatives. As such, insufficient metadata management, lineage tracking, or unified access can quickly result in agents making incorrect or untrustworthy decisions.
Addressing the evolving requirements, AI solutions now support multiple tasks throughout the data lifecycle, unlocking opportunities for automating and optimizing the complex workflows required for data readiness. To name a few examples, synthetic data generation bridges the gap where data is scarce, observability tools monitor and fix broken data pipelines, and SQL queries are increasingly executed by AI.
However, automating the data lifecycle demands a careful approach. To make AI-powered queries accurate, data modeling becomes an essential technology to eliminate hallucinations and to guarantee quality. In addition, data observability tools will become more critical as data not only feeds the core BI and analytics layers, but also the AI systems that are key parts of every production application, both internal and external.
Data Management and Governance
As more and more proprietary data is fed into AI systems, data management becomes a critical enabler of data governance, data integrity, and enterprise decision making. The problem is the average data stack has shot up in complexity over the past years, and traditional master data management has struggled to keep up.
The beauty of the modern data stack was the explosion in choice. As the cloud burst onto the scene, the legacy data warehouse was replaced by a collection of fast-moving platforms. In that era, specialization won, at the cost of complexity and new silos of metadata. Each tool in the stack has data models and metadata that are designed to support the unique objective of that tool or use case. This makes commonality across different tools scarce, leading to challenges in accessing consistent metadata across the stack. The pendulum is now swinging back towards consolidation, evident by a series of acquisitions and the convergence of features.
Several solutions, including Syncari, Reltio, Ascend and Dataloop, have come out with AI-powered platforms that unify data management and governance capabilities across the lifecycle to ensure synchronized, high-quality data:
- Data Unification: Data Catalogs provide a unified, searchable view of all enterprise data assets, consolidating metadata and business context so teams and AI systems can easily find, understand, and trust the data they need.
- Data Synchronization: Ensures data is continuously and consistently aligned across all systems, automatically propagating updates, resolving conflicts, and enforcing business rules so every team and AI pipeline works from the same trusted, up-to-date information.
- Data Quality: Without high-quality data, AI models and analytics initiatives fail to deliver meaningful insights. Solutions ensure clean, structured, complete, and reliable data.
- Data Lineage: Provides end-to-end transparency concerning where the data originates and how it is used across all usage scenarios. This will cover lineage from the sources, pipelines, transformations, and various AI models that may have been applied.
In plain terms, Data Management solutions ensure that data is always updated and ready for analytics, automation and AI - thereby boosting system performance and trust in generated output.
Depending on the type of data, the nature of use cases and speed required, a platform may not be agile enough, requiring additional specialized solutions for certain tasks such as data discovery or pipelines.
Speed and Actionability
AI has democratized many data analyst tasks. In theory, dashboards and insights can be generated at the click of a button by any business user, and oftentimes, speed serves as a big competitive advantage. However, the speed and, more importantly, the quality of insights generated are largely dependent on the underlying data architecture and whether actionable data can be unlocked. This is supported by the following technologies and solutions.
Data Pipelines and Transformation: These tools integrate diverse data sources and formats from across the enterprise, minimizing data silos and ensuring data is accessible. Ideally, data is converted from raw data into a clean and structured form, ready to be used downstream. All this is done in a low-code/no-code fashion, reducing reliance on scarce data engineers.
Addressing the rise in unstructured data, Flexor.ai focuses on data that may be locked in emails, customer interactions, call transcripts, documents, support tickets, etc. It extracts the data from various sources and transforms text into clean, annotated and structured data that is ready for BI & Analytics.
Tensorstax integrates with the existing data stack and offers AI agents that respond to natural language to create, maintain, and optimize data pipelines.
A successful data pipeline solution adds value by delivering faster and higher quality insights while freeing data engineers from manual coding, debugging, and maintenance, allowing them to focus on higher impact architecture.
Data Streaming: As evident by IBM’s recent acquisition of Confluent, data streaming capabilities are becoming more critical. Organizations now try to respond to changing business conditions and customer needs in near real-time. Consider use cases like fraud detection, recommendation systems, and operational monitoring where timing directly impacts business outcomes. Modern streaming data architectures support continuous ingestion and transformation of data streams from diverse sources while maintaining data quality and governance standards.
Solutions like Estuary are stepping up to the challenge by unifying batch and streaming into a single, governed platform that gives teams full control over latency, cost, and deployment. Its flow engine abstracts the heavy lifting of streaming and addresses the trade-off of streaming pipelines that are fast but notoriously brittle, expensive, and hard to manage.
Semantic Layer and Headless BI: Historically, data pipelines have served people: complex pipelines to ingest, filter, and transform information in different systems of record such as cloud data warehouses or SIEMs. Teams then interpret these outputs and act upon them. Humans are great at contextual interpretation. When a VP of sales says “revenue”, a CFO can distinguish between bookings, billings, recognized revenue, or ARR. Humans navigate these nuances effortlessly; machines don’t. As it turns out, the end consumers of data are increasingly machine identities.
What happens when an AI agent pulls “customer count” but doesn’t know support counts active users, sales counts paying accounts, and marketing counts every email subscriber? The agent gives one number, but each team meant something different. The coming years will see a major shift as enterprises realize their most valuable digital asset isn’t their data lake or their AI models—it’s the semantic layer that makes those investments meaningful.
Addressing this, Illumex builds a generative semantic fabric mapping structured data, and then analyses usage context to link user queries to business metrics, allowing any question to be answered reliably by AI. Omni tightly couples its semantic layer with the BI interface, enabling real-time feedback and collaboration so AI assistants can reflect evolving business logic while keeping human oversight in the loop. Cube.dev provides a headless, API-driven semantic layer that lets teams define metrics centrally and serve them consistently to any BI or AI tool, reducing misinterpretation.
Data Readiness is Not a Static State
The difficult truth is that data doesn’t become AI-ready until it’s in use. And the first deployments always reveal new issues - gaps in data coverage, misunderstood metrics, broken lineage, and more.
Therefore, data readiness must be treated as an iterative process, supported by continuous monitoring and human feedback. The right strategy and the right tooling will not just allow teams to quickly improve data posture, but also to quickly react to failures and broken processes across the data lifecycle and iterate accordingly.
Data Readiness can only be determined in the context of the specific use case and AI technique. Proof of readiness will come from the ability to continuously meet AI requirements by aligning data to use cases, qualifying the data, and demonstrating appropriate governance.
Don’t think of Data Readiness as a checklist, but rather an operating model for continuously earning and maintaining trust in data and AI systems.
Data teams are critical enablers of trustworthy AI. With the right tooling and resources, they can help their organizations scale AI confidently — and responsibly.
If you’re curious to learn more or want to stay on top of the latest developments in Innovation, feel free to reach out to us at innovation@trace3.com.
Lars Hegenberg is a Senior Innovation Researcher on Trace3's Innovation Team, where he is focused on demystifying emerging trends and technologies across the enterprise IT space. By vetting innovative solutions, and combining insights from leading research and the world's most successful venture capital firms, Lars helps IT leaders navigate through an ever-changing technology landscape.