Note: refer appendix added with the Complete alerting and monitoring solution serving future needs serving growing data and performance needs of increasing products.
- Data Sources: Logs and APM metrics are generated by various applications, servers, and infrastructure components of respective products like Razi, Iqraa, law etc.
- Beats: Lightweight shippers like Filebeat, Metricbeat, and Auditbeat are deployed on data source hosts to collect and forward data to Logstash or Elasticsearch.
- Logstash: data processing pipeline that ingests data from various sources, transforms it into a common format, and then sends it to Elasticsearch. It acts as the data ingestion and processing layer.
- Elasticsearch: A distributed, RESTful search and analytics engine capable of storing, searching, and analyzing large volumes of data in near1 real-time. It serves as the central repository for all ingested data.
- Kibana: A visualization and exploration tool that allows users to create dashboards, charts, and graphs to explore and analyze the data stored in Elasticsearch. It provides the user interface for interacting with the data.
