IDC defines Big Data technologies as a new generation of technologies and architectures designed to extract value economically from very large volumes of a wide variety of data by enabling high-velocity capture, discovery, and/or analysis. This encompasses hardware and software that integrates, organizes, manages, analyzes, and presents data that is characterized by the "four Vs": Volume, Variety, Velocity and Value.

Analysis of the Big Data market by IDC's global analyst team covers the technologies, integration strategies, use cases, and go-to-market strategies of vendors in this arena and provides insight into the evolution of these technologies and application of Big Data technologies for economic gain.


Related Services

IDC services research around Big Data shows that BPO vendors find themselves in a favorable position to assist their clients' transformation efforts as enterprises face new sources of competition in this global environment and try to stay competitive by keeping a close check on business performance metrics.

Related Services

At the foundation of the Big Data stack is the infrastructure. The use of industry standard servers, networks, storage, hypervisors, and clustering software has been a major driver of the value component of Big Data. For storage, an evolving trend toward the use of object-enabled, content-addressable, and/or scale-out file systems is becoming the norm for Big Data infrastructures.

From a compute perspective, most data-intensive jobs can be partitioned and can be run efficiently on standard clusters with memories that are physically and logically distributed. A smaller number of problems that are less uniform and more communications dependent.


Related Services

The data organization and management layer of the Big Data technology stack refers to software that processes and prepares all types of structured and unstructured data for analysis. This layer extracts, cleanses, normalizes, tags, and integrates data. IDC's use of the terms data organization and management are intentional and highlight the fact that the technology options in this layer of the stack include not just databases.

These technologies include relational databases, NoSQL databases, key value stores, text analytics, ontologies, categorizers, schema extractors, search indexes, parallel file systems, complex event processing engines, graph databases, and other technologies that organize and manage data at rest and in flight.


Related Services

IDC broadly segments Big Data analysis and discovery into software that supports offline, ad hoc, discovery, and deep analytics and software that supports dynamic, real-time analysis and automated, rules-based transactional decision making. Big Data analytics and discovery tools can also be segmented by the type of data being analyzed.

Another dimension that identifies specific analysis and discovery software is the complexity of analysis. There are tools for highly complex descriptive and predictive analysis and tools that simply help with basic aggregation and access to information. Newer tools such as Hadoop, MapReduce, and Dryad (along with methodologies including graph analysis) are proving adept at expediting searches through the large, irregular data sets that characterize many Big Data problems.


Related Services