Why Is Real-Time Data Management Important for Organizational Agility?

Çağda Çelikbilek, IBM Turkey’s Software Country Leader, discussed the importance of real-time data management for organizational agility. Çelikbilek stated the following:

We can think of data manifesting digitally in two main ways. First, there is transactional data. Then, there is the analytical side of data. Transactional data entered our lives first. As data began to multiply and accumulate over time, the need to perform analytics on it arose. Subsequently, data warehouses entered the picture. Because performing analytics where daily operations took place was difficult, we sought to add value to our business by replicating the data, moving it to a separate area, and conducting analytics there. Then, the scope began to expand further. Data lakes entered the scene, and now we are talking about “data lakehouses.” We are gradually shifting toward “data in motion”—data that is actively moving. All of this has accelerated rapidly—reaching an exponential pace—especially with the boom in artificial intelligence. Let me also add this: in the period leading up to perhaps 15 or 20 years ago, the vast majority of data consisted of what we call “structured” data. Today, however, 90% of the data within organizations is “unstructured.” What is unstructured data? It includes emails, PDF documents, Word documents, Excel files, videos, and audio files. Collectively, these now account for 90% of the data within organizations. As for artificial intelligence, it currently accounts for only 1–2% of the data within organizations. In other words, we still have a long way to go in leveraging data and AI within institutions. We undertook an initiative to aggregate data from distributed environments and derive insights from it—utilizing both historical data and live, “data-in-motion” streams to provide actionable intelligence to the organization. As IBM, we want to emphasize two key points here. First, we recognize that the technology landscape has evolved to a stage where no single company can claim to do everything alone; we strongly believe in the value of business partnerships. The solution mentioned earlier—which involved managing data—was a project executed in partnership with “SingleStore,” a key player in the data platform space. Secondly, IBM is a firm believer in open source. Through investments spanning over 25 years, we have made open source an integral part of our operations. We believe we reached a pinnacle in this area with the acquisition of Red Hat in 2018. More recently—about a year and a half after the Red Hat acquisition—IBM brought “HashiCorp” into the fold; this platform has become the *de facto* standard for infrastructure automation in the cloud world. Finally, we also acquired “Confluent,” a company that manages “data in motion” and is rapidly becoming the *de facto* standard in that domain. We place immense importance on the role of streaming data in the future of artificial intelligence.

The Evolution of the Data World

Today, data has become one of the most valuable assets in the digital world. Transactional data and analytical data form the cornerstones of this digital transformation. Transactional data entered our lives first, and over time, the need to utilize this data for analytical purposes emerged. This process led to the development of data warehouses and data lakes. While data lakes facilitate analytics by storing vast amounts of data, the concept of “data in motion” has added a new dimension to the business world by enabling the real-time analysis of data as it moves.

The Rise of Transactional Data

Transactional data consists of data generated and recorded during daily business processes. This data helps businesses monitor and manage their operational activities. Initially, this data was used solely for record-keeping purposes. However, the need to use it for analytics eventually arose, leading to the development of data warehouses to meet this demand.

From Data Warehouses to Data Lakes

Data warehouses are systems where structured data is stored and analyzed. However, with the increase in data volume and the diversification of data types, data lakes emerged. Data lakes offer broader data analysis capabilities by storing unstructured data as well. This enables businesses to conduct more comprehensive and in-depth analyses.

Structured and Unstructured Data

Traditionally, the vast majority of data consisted of structured data. Today, however, 90% of enterprise data is unstructured. Unstructured data—such as emails, PDFs, Word documents, videos, and audio files—has become a vital source of information for businesses. Analyzing this data provides companies with a competitive advantage.

Artificial Intelligence and Data Analysis

Artificial intelligence has revolutionized data analysis processes. However, AI technologies currently account for only 1–2% of enterprise data. This indicates that businesses still have a long way to go in terms of data analysis. AI offers valuable insights to businesses by analyzing both historical data and data in motion.

IBM’s Data Strategy

As a major player in the data landscape, IBM places great importance on business partnerships and open-source technology. Partnerships—such as the one with SingleStore—and acquisitions like Red Hat and HashiCorp are steps that strengthen IBM’s data strategy. Furthermore, the integration of Confluent reinforces IBM’s leadership in managing data in motion.

Data and AI in the Future

Data and artificial intelligence will play a critical role in shaping the business world of the future. While analyzing data in motion enables businesses to make real-time decisions, AI technologies will further accelerate these processes. Through its investments in this field, IBM demonstrates its belief in the future importance of data and AI.

The data landscape has evolved across a broad spectrum, ranging from transactional data to analytical data, and from data warehouses to data lakes. The growth of unstructured data and the ability of artificial intelligence to analyze this data offer significant opportunities for businesses. IBM’s strategic partnerships and acquisitions reinforce its leadership in the fields of data and artificial intelligence. In the future, data and artificial intelligence will continue to be indispensable elements of the business world.