StreamPulse

Lightweight Camera Stream Health Monitoring Microservice

Overview

StreamPulse is a minimal Python-based microservice for monitoring the operational health of heterogeneous IP camera networks.

It was designed during the expansion of a mixed-infrastructure deployment consisting of low-cost consumer cameras (such as TP-Link Tapo) and custom Raspberry Pi camera nodes running MotionEye. As the number of independent devices increased, conventional NVR monitoring and simple ping checks became insufficient to confirm real video availability or client accessibility.

To address this, StreamPulse implements a two-part architecture:

• Monitor Service – periodically connects to configured RTSP and MJPEG endpoints, captures a frame, and records the success or failure as a heartbeat log in an SQLite database.
• Web GUI Service – provides a Flask-based dashboard for configuration, visualization, and on-demand live frame verification.

A simple YAML configuration defines each stream’s name and URL. Both the monitor and the GUI read from this configuration, ensuring lightweight synchronization without external dependencies.

SQLite is used as the database to minimize hardware requirements and enable deployment on single-board computers (Raspberry Pi, Orange Pi, etc.).

---

Problem Statement

Initial deployments used 9–10 cameras connected to an NVR, which was easy to supervise.

As the network scaled to include numerous standalone IP and MotionEye cameras, monitoring became difficult:

• Each hardware platform supported different stream formats and client limits.
• Network pings could not confirm if a stream was actually functional.
• Commercial NVR solutions were resource-heavy and unsuitable for mixed hardware.

StreamPulse provides a lightweight, hardware-agnostic alternative that records the operational state of each stream as a “heartbeat” with accurate timestamps (NTP-synchronized). This allows technical teams to identify failures—power, network, or configuration—without manual inspection.

---

Key Features

• Supports RTSP and MJPEG streams
• Logs stream reachability with timestamp and latency
• SQLite database backend for minimal resource usage
• Flask-based GUI for real-time monitoring and configuration
• YAML configuration for ease of editing and integration
• Modular two-process design (monitor / GUI) for reliability
• Built for low-spec IoT or edge devices

---

Architecture

             ┌─────────────────────────────┐
             │         Web GUI (Flask)     │
             │ - Dashboard & Config Editor │
             │ - Live Frame Preview        │
             └──────────────┬──────────────┘
                            │
                     REST API / SQLite
                            │
             ┌──────────────┴──────────────┐
             │      Monitor Service        │
             │ - Periodic stream probing   │
             │ - Logs results to database  │
             └─────────────────────────────┘

---

🔢 Version Map

Version	Folder	Description
v2.0	version-2/	Latest stable build with async engine and improved GUI
v1.1	version-1/	First GUI-based microservice (Docker supported)
v0.5	legacy_prototypes/	Early standalone scripts and research prototypes

---

Configuration Example

config.yaml

heartbeat_seconds: 15
timezone: Asia/Kolkata

streams:
  - name: GateCamera
    url: rtsp://user:[email protected]:554/stream1

  - name: LabCam1
    url: http://192.168.1.1:9081

---

Usage

Install dependencies:

pip install -r requirements.txt

Start the monitor service:

python monitor.py

Start the web interface:

python webgui.py

Open http://localhost:8000 in your browser.

Default credentials: admin / admin123

---

🐳 Manual Docker Deployment

To pull and run the container manually:

docker pull devprincekumar/streampulse:1.1

docker run -d -p 8000:8000 -v $(pwd)/data:/data devprincekumar/streampulse:1.1

Once the container is running, open:

 [http://localhost:8000](http://localhost:8000)

Docker Compose Deployment

You can also deploy using Docker Compose for easier management:

docker compose up -d

This will automatically start the StreamPulse web GUI and monitoring services.

---

Persistent Data Storage

All persistent data — including configuration files (config.yaml) and the SQLite database (streams.db) — are stored in the local data/ folder on the host system.

This ensures that all settings, logs, and stream configurations remain intact across container restarts or image updates.

---

Default Access

• URL: http://localhost:8000
• Default Credentials: admin / admin123

---

Test Streams (for First-Time Users)

If you don’t have your own camera setup yet, you can easily test StreamPulse with publicly available live streams.

A great source for open camera feeds:
🔗 Insecam – Public IP Cameras Directory

You can copy any RTSP or MJPEG URL from that site and add it in your config.yaml or through the web GUI.

Example entry in config.yaml:

streams:
  - name: TestCam
    url: http://91.191.213.49:8081/mjpg/video.mjpg

Once added, restart StreamPulse or reload the dashboard — you’ll see the camera’s health status live in the GUI.

Note: These streams are publicly shared by their operators. You can use them responsibly and for testing purposes only.

---

Technical Notes

• Image: devprincekumar/streampulse:1.1
• Built with Python 3.11 and Flask
• Includes sample stream for first-run testing
• Lightweight — suitable for Raspberry Pi or edge devices
• Each stream has a dedicated log table within streams.db.
• Logs are timestamped using NTP-synchronized UTC time and mapped to the configured timezone.
• Designed to run continuously on low-power devices.
• No external database or message broker required.

---

Legacy Prototypes and Evolution

Before StreamPulse became a Flask-based microservice with a GUI and database, it went through several experimental stages — from single-camera stream loggers to multi-threaded network monitors.

These early scripts were the foundation for understanding stream behavior, latency, and reliability under different protocols (RTSP, MJPEG) and hardware setups (NVRs, MotionEye, Raspberry Pi nodes).

You can explore these prototypes in the legacy-prototypes/ folder.
Each script was part of the evolution that led to StreamPulse v1.

File	Description	Key Learnings
1-working-rtsp-log-(cv).py	Early OpenCV-based RTSP logger that captured frames and stored timestamps to CSV.	Validated minimal RTSP connection handling and frame fetch consistency.
2-me-stream-receive.py	MotionEye MJPEG receiver script — saved short video chunks while logging CPU, memory, and network usage.	Tested continuous HTTP-based MJPEG fetching and real-time system monitoring.
3-rtsp_sender_logger.py	RTSP sender prototype using FFmpeg to stream video input with timestamp overlay and NTP sync.	Helped understand stream generation, encoding, and network bitrate behavior.
3-rtsp_receiver_logger.py	Receiver-side analytics tool that calculated latency, jitter, and frame consistency using NTP and CSV logs.	Introduced frame-level analysis and time-sync verification.
4-rtsp_heartbest.py	Multi-threaded RTSP heartbeat system for several cameras at once, storing logs per camera.	Core inspiration for StreamPulse’s parallel stream checking and logging architecture.

These prototypes collectively formed the groundwork for StreamPulse v1, merging lightweight frame checking, NTP time sync, and CSV-based health logs into a unified, database-backed service.

Each version was tested under real IoT and lab conditions, progressively optimized for performance, error handling, and deployment scalability.

---

Current Status

Version 1 implements a functional threaded architecture suitable for up to a few hundred streams under normal heartbeat intervals.
Further improvements and scalability enhancements are planned as subsequent versions are tested and validated in live environments.

---

License

MIT License © 2025 Prince Kumar