HERTZ / OPEN-SOURCE RPPG

rPPG Heart-Rate Estimation from Facial Video

HERTZ is an extensible open-source collection of complementary facial-video rPPG estimators: TinyHR learns pulse waveforms from video, while AdaChrom recovers pulse signals through training-free chrominance-based processing.

Watch demo Code on GitHub Technical report

TINYHR RECORDED DEMO · 12-SECOND PREVIEW

WHY TINYHR

More data. Less model. Faster response.

TinyHR learns across people and recording conditions from four complementary rPPG datasets while keeping the network to just 82K parameters. Once the initial window is ready, compute stays well below one second and the estimate updates every second by default.

4 training datasets
6,307 training videos
82,177 model parameters · 381 KiB ONNX
80 / 6 ms mean CPU / GPU inference · 100 runs each
TinyHR inference: 80 ms mean on CPU, 6 ms mean on GPU.

The reference benchmark ran 100 times per device on an Intel Core i9-13900KF CPU at 3.00 GHz and an NVIDIA H20 GPU in a server environment. Values cover model inference only; video capture, face detection, 160-frame input collection, and update scheduling are excluded. Results vary by runtime environment.

TRAINING DATA

1,288 subjects. 6,307 videos.

Multiple sources broaden the range of people and recording conditions available for learning robust rPPG representations.

Dataset Subjects Videos Usage
VIPL-HR V1851,883Training
VIPL-HR V25002,498Training
V4V103726Training
MCD-rPPG6001,200Training; front-facing only
Total 1,288 6,307 Four-source corpus

SEE IT IN ACTION

From facial video to a pulse waveform.

Watch the recorded TinyHR pipeline: facial video, a predicted rPPG waveform, and heart-rate estimates displayed together.

25-second recording · Silent playback Download demo video

HERTZ ESTIMATOR COLLECTION

One toolkit, multiple paths to the pulse

HERTZ presents each heart-rate estimator as an independent method module. The collection combines a compact learned model with an interpretable training-free pipeline.

LEARNING-BASED · LIGHTWEIGHT

TinyHR

Remote photoplethysmography (rPPG) estimates pulse-related signals from subtle changes in light reflected by facial skin. TinyHR processes a clip of 160 RGB facial frames at 128 × 128 pixels. Its lightweight convolutional pipeline emphasizes differences between neighboring frames, builds compact spatial features, and combines temporal information at multiple scales. The network predicts one rPPG waveform sample per video frame. Heart rate is then calculated from the predicted waveform using filtering and spectral analysis.

Supplied TinyHR pipeline schematic showing facial video input, feature extraction, waveform prediction, and heart-rate post-processing.
Open architecture PDF Download architecture PDF
View PDF inline

  1. 01

    Frame Difference Fusion Stem

    Four neighboring-frame difference maps are concatenated into a 12-channel representation. The final model uses this difference branch without the original RGB appearance branch. A 5 × 5 convolution, batch normalization, ReLU, pooling, and a 3 × 3 convolution produce 16-channel spatial features.

  2. 02

    Spatial Patch Embedding

    A frame-wise 2D convolution with kernel size 4 and stride 4 reduces each feature map from 32 × 32 to 8 × 8, with 32 channels. Batch normalization and ReLU follow. The temporal dimension is preserved.

  3. 03

    Multi-scale Temporal Feature Block

    A single convolutional stage enhances spatial features and pools them into a temporal sequence. Four branches combine information from different temporal offsets, followed by pointwise fusion and a temporal feed-forward network. Residual connections retain the input features; no hierarchical temporal downsampling is used.

  4. 04

    Waveform Predictor

    Spatial global average pooling and two pointwise convolutional layers produce a single-channel rPPG waveform. The output contains one value per input frame: 160 samples for a 160-frame clip. Heart rate is computed after waveform prediction.

INFERENCE

From waveform to heart rate

During inference, a second-order Butterworth band-pass filter retains frequencies from 0.75 to 2.5 Hz. Welch's method estimates the power spectral density of the filtered waveform. The dominant frequency is converted to beats per minute.

Heart rate (BPM) = 60 × dominant frequency (Hz)

Training objectives

Training combines a negative Pearson correlation objective for waveform agreement, a frequency-domain cross-entropy objective, and a KL-divergence objective for heart-rate distributions. The reported weights are 0.2, 1.0, and 1.0, respectively.

L = 0.2 Ltime + LCE + LKL

UNSUPERVISED · SIGNAL PROCESSING

AdaChrom

AdaChrom is an unsupervised rPPG heart-rate estimator that requires no labeled training data. It locates reliable skin regions in facial RGB video, extracts subtle pulse-related color changes, recovers a blood-volume-pulse waveform through CHROM chrominance projection, and estimates heart rate from the dominant FFT frequency, providing a lightweight and interpretable open-source alternative to TinyHR.

AdaChrom preprocessing, BVP extraction, and heart-rate post-processing pipeline

PRE-PROCESSING

Adaptive skin ROI

After face alignment, geometric facial regions are intersected with YCrCb skin constraints. AdaChrom-v4 further retains larger connected components consistent with the face region to reduce isolated false positives.

BVP EXTRACTION

Temporal chrominance traces

Mean BGR signals from valid ROIs are aggregated in a sliding window, then resampled from timestamps, smoothed, and normalized by each channel’s temporal mean.

HEART-RATE ESTIMATION

CHROM and FFT

CHROM chrominance projection recovers the blood-volume-pulse signal. A windowed FFT selects the dominant frequency within an approximately 50–120 BPM band and converts it to heart rate.

EVALUATION

Reported performance

3.88 BPM

Mean absolute error (MAE)

Dataset
VIPL-HR V1
Test split
22 subjects · 485 videos
Evaluation protocol
Held-out test set, excluded from training according to the report
Source
TinyHR technical report, page 7
Read the evaluation details

Explore the project

Find the source code, setup instructions, and technical documentation.

GitHubSource code, setup, and usage instructions. View repository TinyHR technical reportArchitecture, training objectives, and reported evaluation. Download report PDF Architecture diagramThe supplied pipeline schematic in PDF format. Download architecture PDF
Hugging FaceModel distribution and interactive demos are planned. Planned