Scalable Telemetry Platform Deployment

Deployed Spark and Airflow on AWS for real-time industrial IoT cybersecurity

Project Overview

Provisioned cloud infrastructure and deployed Apache Spark and Airflow clusters for a cybersecurity platform processing real-time telemetry from industrial IoT devices. Focused on scalability, performance, and robust data workflows, while integrating modern observability tooling.

Key Outcomes

Provisioned scalable Spark and Airflow clusters on cloud-native infrastructure

Integrated real-time monitoring with Grafana and Prometheus

Streamlined telemetry data pipelines with DAG-based orchestration

Ensured high availability and fault tolerance of data workflows

Project Toolstack

GCPKubernetesTerraformHelmPythonApache SparkAirflow

Interested?

Want to learn more about this project or discuss a similar solution for your business?