Big data architecture may include the following components: Data sources, Big data storage, Real-time message ingestion store, Analytical data store, Big data analytics and reporting.
- Data sources – relational databases, files (e.g., web server log files) produced by applications, real-time data produced by IoT devices.
- Big data storage – NoSQL databases for storing high data volumes of different types before filtering, aggregating and preparing data for analysis.
- Real-time message ingestion store – to capture and store real-time messages for stream processing.
- Analytical data store – relational databases for preparing and structuring big data for further analytical querying.
- Big data analytics and reporting, which may include OLAP cubes, ML tools, self-service BI tools, etc. – to provide big data insights to end users.
