There is a strong trend toward the adoption of in-memory computing (IMC). IMC puts together advanced technology and innovative business processes and applications. In-memory processing is taking action on the data stored in random access memory and is very popular because of the advantages it offers, such as faster speeds, reduced IT costs, and increased efficiency due to the ability to obtain real-time business insights.
In-memory computing is taking data traditionally stored on hard discs and moving it into memory. When using hard disks, processing power is not at full capacity because the data to be processed cannot be retrieved fast enough. Storing data in the random access memory (RAM) reduces latency and speeds up the data analysis process.
Data analysis consists of the storage of data, the processor to perform the calculations and the system that transfers the data between the two. The slowest of the components is the bottleneck and this bottleneck is latency of storage. More specifically, it is the latency of hard disks that’s the problem. The process of data analysis increases exponentially in speed when data is stored in memory and not on hard disks.
In-memory processing means taking action on the data in RAM. This contrasts with other ways of processing data that rely on reading and writing data from slower media like disk drives.
Software that runs on one or more computers manages processing and data in memory. Where there are multiple computers, the processing is divided up into smaller tasks and distributed to each computer where they can run in parallel. In-memory processing is often done using an in-memory data grid (IMDG).
In-memory processing is not limited to a specific technology. It may include data grids, query engines, and databases. Data grids are for running applications that perform actions on data, databases are for storing and retrieving data, and query engines retrieve data from various sources.
The use of RAM data storage and parallel distributed processing allows IMC to offer high speed and scalability. It can reduce the process of data analysis from hours to minutes or even seconds. This makes it possible to leverage the most recent data for analysis and informed decision-making.
Additional performance gains come from intelligent data processing, compressing data and having a built-in calculation engine. This engine can deliver near real-time results to complex data queries. It is possible to process mixed workloads (both transactional and analytical data) within the same architecture. Managing operational applications and data analysis close to real-time on a single database is possible.
The performance gains from using IMC offer the potential for innovative applications to differ from competitors. There are many different use cases where high-speed processing is required, from payment processing and fraud detection, to predictive maintenance, self-driving cars and algorithmic trading. For example, if a company can predict when a car is going to need maintenance, it can have the spares on hand at the right time.
In retail, it is important to ensure high levels of customer experience at all times and IMC offers scalability and real-time analytics on transactional data to offer this.
Financial and insurance companies need to calculate risks, increase automation, achieve consistency and improve the performance of many applications and IMC can help them to do this.
The reduction in layers allows for the simplification of data models. The shift from traditional, hard disk-enabled warehouses to IMC-enabled data warehouses means a reduction in layers on the way from raw data to data analysis results.
In a traditional data warehouse, raw data is stored in the warehouse. Part of the data is extracted to data marts and business intelligence applications can request results stored in the data mart. BI applications can get final results from the IMC-enabled data warehouse. The results are computed near real-time by querying the built-in calculation engine and the layer of data marts is unnecessary. Raw data is frequently updated and transactional applications feed data directly into the IMC-enabled data warehouse.
With this comes quicker creation, less testing, and easier adaption. The reduction of complexity also reduces the sources of potential errors.
The near real-time calculation of analytical results from raw data adds flexibility in terms of integration of additional data sources and modification of the analysis. It is possible to change analytical procedures easily as only a change to a query is necessary.
It is also easy to plug in new data sources as additional sources of information because each calculation starts from raw data. This can significantly reduce total costs, effort to develop new data models, and effort to maintain and develop existing data models.
Common misconceptions about IMC
In-memory computing may speed up current analytical and transactional processes but there’s a misconception that IMC offers a sustainable competitive advantage out of the box. Using it as a competitive tool requires the talent to identify and serve the specific information needs of each business. Data volumes need to be effectively managed as well as the various transactional and analytical processes.
Document management software encompasses many different programs or processes, all of which can help a business store and manage documents more efficiently in one easy-to-access database.
There is a misconception that using IMC means a reduction in IT governance efforts. The impact on IMC performance is a major one if the volume of data and number of database queries increases due to lack of IT governance. Lack of governance will result in more process redundancies and complexities, more maintenance efforts and an increase in data volumes.
Another misconception is that IMC solves all current and future performance problems. The expected continuing exponential growth of data volumes could even out the performance gains if applications make inefficient use of IMC.
In-memory processing contrasts with other ways of processing data as action is taken on data stored in random access memory. There are many use cases where the in-memory performance advantage is significant in meeting business requirements and giving them a competitive edge.