Libin Guo · Senior Software Engineer (Data Platform)

“I have no special talent. I am only passionately curious.”

— Albert Einstein

I'm a Senior Software Engineer (Data Platform) with 9+ years building large-scale data pipelines and data platforms — most recently a 20 PB Iceberg lakehouse at Apple and video training-data pipelines for world models at Reka AI.

Open to Senior Data, Data Platform, AI & Applied AI Engineer rolesAustin, TX

Follow
Libin Guo
PathFrom fraud analytics to petabyte-scale platforms to AI training data — each move closer to the data models learn from.
  1. 2015UT DallasM.S., Information Technology & ManagementWhere it started
  2. 2016QlarantSenior Data AnalystMedicare fraud analytics
  3. 2018HealthmineData EngineerHealthcare ETL pipelines and a Postgres analytics warehouse
  4. 2020JPMorgan ChaseData EngineerReal-time ingestion on AWS
  5. 2021AppleSenior Software Engineer, Data PlatformScaled the lakehouse platform · Modernized legacy pipelines: on‑prem to cloud, pandas to Spark
  6. 2026Reka AIMember of Technical Staff, Data PlatformVideo training data for world models

9+

years building data pipelines and platforms

20 PB

Iceberg lakehouse at Apple

2,000+

Spark jobs run daily

$0.5M

saved per year through Spark tuning

Key skills
  • Spark
  • Python
  • Airflow
  • Kubernetes
  • AWS

What I Do

Focus areas

  1. 01

    Spark data pipelines & Iceberg lakehouses

    Large-scale Spark pipelines on Apache Iceberg — tuned for speed and cost, and run reliably at thousands of jobs a day.

  2. 02

    AI training-data pipelines

    Ray-on-Kubernetes extraction and batch inference, data-quality gates, and WebDataset delivery for pretraining and fine-tuning.

  3. 03

    AI agents for data platforms

    MCP-based agents and LLM evaluation frameworks that automate platform work, like migration assessment.

Selected Work

Production projects

All projects

Hiring for a senior data role?

I'd like to hear about the platform you're building and the data problems behind it.