Back to Projects

Reka AI · 2026

Video training-data pipelines for world models

I build the Ray and PySpark pipelines that turn raw video into clean training data for pretraining, SFT, and distillation.

Member of Technical Staff, Data Platform

Multimodal data

Video

Ray workloads

CPU + GPU

Training stages served

3

Problem

The research team needs a lot of video, prepared the same way every time: extracted, run through batch inference, checked, and packaged so the training stack can stream it. One bad batch that slips through burns GPU hours later, so I wanted to catch it early.

Approach

  • Tuned the PySpark and Ray-on-Kubernetes pipelines that prepare the video.
  • Moved extraction and batch inference onto Ray so CPU and GPU nodes stay busy instead of waiting on each other.
  • Added data-quality checks and QA reports that run before anything reaches training.
  • Sent pipeline health to Slack so I hear about failures right away.
  • Packaged the output as Energon WebDatasets, which the researchers can stream directly.

Results

  • Bad data gets caught before training, not after a wasted run.
  • Research gets ready-to-train WebDatasets for pretraining, SFT, and distillation.
  • Better CPU and GPU utilization across extraction and batch-inference jobs.

Stack

  • Ray (CPU + GPU workers)
  • Kubernetes
  • PySpark
  • Python
  • Energon WebDataset
  • Batch inference
  • Slack alerts