Data structure has changed to lose its structure and to add hundreds of formats. The video, audio, spatial data, IoT Sensor data, pure text, Images, PDFs etc., one no longer has to control over the input data format. As new applications and technologies, like AI, ML, Block Chain etc., can be easily incorporated in businesses, new data sources and formats come to life. Structure or centralized data sources can no longer be imposed like in the past in order to keep control over the analysis.
Let’s consider a manufacturing industry to understand what and how velocity is a challenge in Big data. The Manufacturing industry automates its production lines by collecting various information through logs, various sensors and environmental information inside and outside of the plant. This type of data is not expected to expand as violently as other fields, for example, the field of general consumer marketing, but in high velocity (frequency of data occurrence). Managers/Technicians try to control production facilities and machinery by analyzing this data in real time. The high frequency of data occurrence makes it almost impossible to pre-process, discover the pattern and analyze to take appropriate decisions.