Research Engineer at Luma AI
I work on multimodal image/video generation models, structured visual generation, post-training, reward modeling, and multimodal evaluation.
I lead Layering, a structured visual generation system that turns flat generated images or design media into editable raster, text, vector, and alpha-aware components for multi-turn human and agent editing.
Before Layering, I worked across Luma's image and video generation models, including Ray3 and Uni-1. I built the experimental RL/post-training workflow for Ray3 video generation, including sample generation loops, reward modeling, VLM-as-judge graders, held-out evaluation, and diffusion RL, DPO-style, and GRPO-style experiments. I also contributed RL/data/eval experiments for Uni-1, including OCR-focused rewards, caption/data ablations, and early evaluation.
Before Luma, I completed my Stanford MSCS with Jiajun Wu, where I led IKEA Manuals at Work, a first-author NeurIPS 2024 project on 4D grounding of real-world assembly videos to 3D structure over time. I received my BEng in Electronics and Computer Science from the University of Edinburgh, ranking 2nd in my cohort.
I lead Layering, which turns flat generated images or design media into editable components across vectorized SVG, text, and alpha-aware pixels. The goal is to make generated media structured enough for people and agents to inspect, revise, verify, and build on across multiple turns.
I worked across Luma's image and video generation models, including Ray3 and Uni-1. For Ray3, I built the experimental RL/post-training workflow for video generation, including sample generation loops, diffusion RL, DPO-style and GRPO-style experiments, reward modeling, VLM-as-judge graders, and held-out evaluation calibrated against human preference.
I build evals that measure visual model behavior: instruction following, language grounding, information preservation, edit success over repeated interventions, and usefulness in real workflows. These evals guide data, reward, conditioning, and model changes.
Ray3 is Luma's next-generation video model, focused on reasoning-driven generation, controllable video-to-video workflows, character reference, keyframes, HDR, and production-grade fidelity.
Uni-1 is Luma's unified multimodal model for interleaved language and image understanding, reasoning, and generation.
This year, I am interested in visual generation as a bridge between pixels, structure, code, feedback, and action. I want generated artifacts to be more than flat outputs: structured states that people and agents can inspect, revise, verify, and reuse in interactive multi-turn workflows. In design, this means layouts, diagrams, interfaces, product visuals, SVG, HTML, UI components, and other code-backed representations. In coding, it means artifacts that agents can read, modify, test, and verify instead of treating visual outputs as opaque images. In video and embodied settings, it points toward visual models that preserve spatial, temporal, and object-level structure well enough to support downstream reasoning and action. Layering is one attempt at this direction, with reward design and repeated-edit evals as the feedback loop around it.
Shijia Yang*, Yunong Liu*, Bohan Zhai*, Ximeng Sun, Zicheng Liu, Emad Barsoum, Manling Li, Chenfeng Xu.
(*equal contribution)Equal-contribution work introducing CaptionQA, a utility-based benchmark with 33,027 multiple-choice questions across Natural, Document, E-commerce, and Embodied AI domains. We show measurable gaps between image and caption utility even for state-of-the-art multimodal models, reflecting my interest in evals that measure whether visual representations preserve the information people actually need.
Seungwoo Kim*, Khai Loong Aw*, Klemen Kotar*, Cristobal Eyzaguirre, Wanhee Lee, Yunong Liu, Jared Watrous, Stefan Stojanov, Juan Carlos Niebles, Jiajun Wu, Daniel L. K. Yamins.
Contributor to a counterfactual probe over video-diffusion logits that extracts optical flow with no labels and no fine-tuning. The method achieves state-of-the-art TAP-Vid results, generalizes to in-the-wild videos, and outperforms specialized optical-flow baselines.
Yunong Liu, Cristobal Eyzaguirre, Manling Li, Shubh Khanna, Juan Carlos Niebles, Vineeth Ravi, Saumitra Mishra, Weiyu Liu*, Jiajun Wu*.
First-author work and sole student lead on the first dataset aligning real-world assembly videos with 3D models and instruction manuals. Built cross-frame optimization with temporal-consistency constraints across 34k+ frames over 98 videos, and coordinated annotation / validation for evaluating whether models can ground human instructions in real visual workflows.
Yunong Liu*, Nikhil Kolluri*, Dhiraj Murthy.
(*equal contribution)Developed hybrid framework combining machine learning with crowdsourced annotations to combat COVID-19 misinformation. Developed systematic comparison framework across classical models (SVM, LR, BNB) and pre-trained models (BERT, RoBERTa, XLNet) on 7 dataset combinations.
Student Tech Intern, Semiconductor Manufacturing Anomaly Detection, NXP Semiconductors, Tianjin, China, May 2021 - Aug 2021
Deep Learning Intern, Wind Farm Performance AI Optimization, Zealen AI (Startup), Beijing, China, Aug 2021 - Sep 2021
Research Assistant - NLP Project, University of Edinburgh, Worked with Prof Bonnie Webber on discourse relation analysis, July 2022 - Jan 2023
Teaching Demonstrator, University of Edinburgh, InfBase, Sep 2022 - Jan 2023
Programme Representative, University of Edinburgh, Sep 2022 - June 2023
Global Buddies Program Mentor, University of Edinburgh, Sep 2021 - June 2022
Volunteer, Sri Lanka Wildlife Conservation Society (SLWCS), Jul 2019 - Aug 2019
DoB: October 1, 2012
Role: The Elegant Lady
Superpowers: Tsundere Queen Cleanliness Lover Fearless Explorer
Mewomewo is the queen of the house since 2012. She acts cool but secretly loves attention. She's super brave, except when it comes to mess. You'll often see her checking if the house is clean and tidy.
DoB: July 17, 2014
Role: The Cuddly Foodie
Superpowers: Cuddle Expert Food Lover Master of Stealth (when scared)
Xiaopang, our big bundle of love, joined us in 2014. His name means "little fat", but really, he's just extra cuddly! He loves snuggles and treats equally. He might get scared easily, but his heart is as big as his appetite.
