Jianfeng Zhang 张健锋

Research Scientist at ByteDance Seed  ·  Lead of Seed3D  ·  Generative 3D & World Models

I am a Research Scientist at ByteDance Seed, where I lead Seed3D. Like humans, machines will need a world to grow up in: a place to see, act, and fail safely before they enter ours. We build generative 3D world models that create such worlds, from objects to scenes to interactions, with the geometry, materials, dynamics, and physics to be acted in rather than just looked at.

Before this, I received my PhD from the National University of Singapore, with research internships at ByteDance AI Lab, Sea AI Lab, and Meta Reality Labs.

We are always looking for great talent (both full-time & intern). Feel free to reach out if interested!

Jianfeng Zhang

News

Open-Source Projects

Models and codebases released by our team, ready to use.

2025CVPR 2025
Dora

Sampling & benchmarking for 3D shape VAEs, the geometry backbone behind Seed3D.

2025CVPR 2025
MagicArticulate

Make any 3D model articulation-ready with auto-generated skeletons and skinning.

2025NeurIPS'25 Spotlight
Puppeteer

Rig and animate your 3D models from a single static asset.

2024CVPR 2024
MagicAnimate

Temporally consistent human image animation (10k+ GitHub stars); powers AI Dancing on TikTok.

Publications

* equal contribution  ·  corresponding author  ·  representative works

2026
ArtiAtlas: A Holistic Ecosystem for Diverse 3D Articulation Modeling

ArtiAtlas: A Holistic Ecosystem for Diverse 3D Articulation Modeling

Junzhe Lu, Jing Lin, Ziang Cao, Haoqian Wang, Ziwei Liu, Jianfeng Zhang
SIGGRAPH Asia 2026

ArtiAtlas: a holistic ecosystem for diverse 3D articulation modeling, turning static 3D assets into articulated, functional objects.

@inproceedings{lu2026artiatlas,
  title={{ArtiAtlas: A Holistic Ecosystem for Diverse 3D Articulation Modeling}},
  author={Lu, Junzhe and Lin, Jing and Cao, Ziang and Wang, Haoqian and Liu, Ziwei and Zhang, Jianfeng},
  booktitle={ACM SIGGRAPH Asia Conference Papers},
  year={2026}
}
SceneSpinner: Taming LLMs for Codematic Indoor Scene Generation

SceneSpinner: Taming LLMs for Codematic Indoor Scene Generation

Yixun Liang, Qianyi Wu, Chuan Fang, Rui Chen, Jiahang Liu, Jianfeng Zhang, Ping Tan
ECCV 2026

SceneSpinner generates 3D indoor scenes as code: an LLM finetuned on a unified corpus of 52K rooms and 280K text-scene pairs, with a language-based planning stage and Conditional Mutual Information regularization for instruction-faithful spatial layouts, producing furnished, editable scenes from free-form text.

@inproceedings{liang2026scenespinner,
  title={{SceneSpinner: Taming LLMs for Codematic Indoor Scene Generation}},
  author={Liang, Yixun and Wu, Qianyi and Fang, Chuan and Chen, Rui and Liu, Jiahang and Zhang, Jianfeng and Tan, Ping},
  booktitle={European Conference on Computer Vision},
  year={2026}
}

Seed3D 2.0: Advancing High-Fidelity Simulation-Ready 3D Content Generation

Seed3D Team, ByteDance Seed (Jianfeng Zhang)
Technical Report

Seed3D 2.0: a new-generation 3D foundation model with a coarse-to-fine geometry pipeline and a unified PBR material model, extended to simulation-ready scene generation via layout planning, part-aware decomposition, and training-free articulation.

TopoMesh: High-Fidelity Mesh Autoencoding via Topological Unification

TopoMesh: High-Fidelity Mesh Autoencoding via Topological Unification

Guan Luo, Xiu Li, Rui Chen, Xuanyu Yi, Jing Lin, Chia-Hao Chen, Jiahang Liu, Song-Hai Zhang, Jianfeng Zhang
CVPR 2026

TopoMesh: a sparse voxel-based mesh VAE that unifies ground-truth and predicted meshes under a shared Dual Marching Cubes topology, enabling explicit mesh-level supervision on topology, vertex positions and face orientations, and reconstructing high-fidelity meshes at 1024³ resolution.

@inproceedings{luo2026topomesh,
  title={{TopoMesh: High-Fidelity Mesh Autoencoding via Topological Unification}},
  author={Luo, Guan and Li, Xiu and Chen, Rui and Yi, Xuanyu and Lin, Jing and Chen, Chia-Hao and Liu, Jiahang and Zhang, Song-Hai and Zhang, Jianfeng},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2026}
}
ViLearn: Accelerating Training Convergence of Image-to-3D Generation via Visibility Learning

ViLearn: Accelerating Training Convergence of Image-to-3D Generation via Visibility Learning

Rui Chen, Jianfeng Zhang, Jing Lin, Xuanyu Yi, Yixun Liang, Guan Luo, Xiu Li, Zeming Li, Ping Tan
CVPR 2026

ViLearn: a training paradigm for image-to-3D latent diffusion that injects visibility structure and positional inductive bias via Visibility Grouping and Visibility-Aware Positional Encoding, accelerating training convergence of VecSet-based models by up to 4.4× with better generation quality.

@inproceedings{chen2026vilearn,
  title={{ViLearn: Accelerating Training Convergence of Image-to-3D Generation via Visibility Learning}},
  author={Chen, Rui and Zhang, Jianfeng and Lin, Jing and Yi, Xuanyu and Liang, Yixun and Luo, Guan and Li, Xiu and Li, Zeming and Tan, Ping},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2026}
}
2025

Seed3D 1.0: From Images to High-Fidelity Simulation-Ready 3D Assets

Seed3D Team, ByteDance Seed (Jianfeng Zhang)
Technical Report

Seed3D 1.0: a 3D generative foundation model that generates simulation-ready assets from single images, enabling scalable physics-based world simulators!

Puppeteer: Rig and Animate Your 3D Models

Puppeteer: Rig and Animate Your 3D Models

NeurIPS 2025Spotlight

What if static 3D models could be automatically rigged and animated? Puppeteer makes it possible, transforming any 3D object into animation-ready assets with stable motion!

@inproceedings{song2025puppeteer,
  title={{Puppeteer: Rig and Animate Your 3D Models}},
  author={Song, Chaoyue and Li, Xiu and Yang, Fan and Xu, Zhongcong and Wei, Jiacheng and Liu, Fayao and Feng, Jiashi and Lin, Guosheng and Zhang, Jianfeng},
  booktitle={Advances in Neural Information Processing Systems},
  year={2025}
}
MS3D: High-Quality 3D Generation via Multi-Scale Representation Modeling

MS3D: High-Quality 3D Generation via Multi-Scale Representation Modeling

Guan Luo, Jianfeng Zhang
ICCV 2025

Sparse views to high-quality 3D meshes in 5 seconds? MS3D makes it real with hierarchical multi-scale magic!

@inproceedings{guan2025ms3d,
  title={{MS3D: High-Quality 3D Generation via Multi-Scale Representation Modeling}},
  author={Luo, Guan and Zhang, Jianfeng},
  booktitle={International Conference on Computer Vision},
  year={2025}
}
MagicArticulate: Make Your 3D Models Articulation-Ready

MagicArticulate: Make Your 3D Models Articulation-Ready

CVPR 2025

MagicArticulate transforms static 3D models into articulation-ready assets with 1) Articulation-XL, a large-scale benchmark of 33K+ annotated models; 2) auto-regressive skeleton generation handling varying bone structures; and 3) functional diffusion for skinning with geodesic priors.

@inproceedings{song2025magicarticulate,
  title={{MagicArticulate: Make Your 3D Models Articulation-Ready}},
  author={Song, Chaoyue and Zhang, Jianfeng and Li, Xiu and Yang, Fan and Chen, Yiwen and Xu, Zhongcong and Liew, Jun Hao and Guo, Xiaoyang and Liu, Fayao and Feng, Jiashi and Lin, Guosheng},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2025}
}
Dora: Sampling and Benchmarking for 3D Shape Variational Auto-Encoders

Dora: Sampling and Benchmarking for 3D Shape Variational Auto-Encoders

CVPR 2025

3D shape VAEs used to lose geometric details with uniform sampling, but Dora's sharp edge sampling preserves them while being 8× smaller.

@inproceedings{chen2025dora,
  title={{Dora: Sampling and Benchmarking for 3D Shape Variational Auto-Encoders}},
  author={Chen, Rui and Zhang, Jianfeng and Liang, Yixun and Luo, Guan and Li, Weiyu and Liu, Jiarui and Li, Xiu and Long, Xiaoxiao and Feng, Jiashi and Tan, Ping},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2025}
}
AvatarStudio: High-fidelity and Animatable 3D Avatar Creation from Text

AvatarStudio: High-fidelity and Animatable 3D Avatar Creation from Text

IJCV 2025

What if you could create animation-ready 3D avatars from just text? AvatarStudio brings it to life via coarse-to-fine modeling and DensePose-guided control.

@article{zhang2025avatarstudio,
  title={Avatarstudio: High-fidelity and animatable 3d avatar creation from text},
  author={Zhang, Xuanmeng and Zhang, Jianfeng and Zhang, Chenxu and Liew, Jun Hao and Zhang, Huichao and Yang, Yi and Feng, Jiashi},
  journal={International Journal of Computer Vision},
  volume={133},
  pages={5178--5196},
  year={2025},
  publisher={Springer}
}
2024
Magic-Boost: Boost 3D Generation with Multi-View Conditioned Diffusion

Magic-Boost: Boost 3D Generation with Multi-View Conditioned Diffusion

arXiv 2024

Coarse 3D generation results plagued by blur and geometry errors? Magic-Boost refines them into high-quality assets in ~15 minutes through multi-view conditioned diffusion.

@article{yang2024magicboost,
  title={{Magic-Boost: Boost 3D Generation with Multi-View Conditioned Diffusion}},
  author={Yang, Fan and Zhang, Jianfeng and Shi, Yichun and Chen, Bowen and Zhang, Chenxu and Zhang, Huichao and Yang, Xiaofeng and Feng, Jiashi and Lin, Guosheng},
  journal={arXiv preprint arXiv:2404.06429},
  year={2024}
}
High Quality Human Image Animation using Regional Supervision and Motion Blur Condition

High Quality Human Image Animation using Regional Supervision and Motion Blur Condition

arXiv 2024

Better faces, hands, and motion in human animation—regional supervision + explicit blur modeling deliver 21% L1 and 57% FVD improvements.

@article{xu2024hia,
  title={{High Quality Human Image Animation using Regional Supervision and Motion Blur Condition}},
  author={Xu, Zhongcong and Song, Chaoyue and Song, Guoxian and Zhang, Jianfeng and Liew, Jun Hao and Xu, Hongyi and Xie, You and Luo, Linjie and Lin, Guosheng and Feng, Jiashi and Shou, Zheng},
  journal={arXiv preprint arXiv:2409.19580},
  year={2024}
}
MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model

MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model

CVPR 2024

A breakthrough in human animation: MagicAnimate delivers the first high-fidelity, temporally consistent results via video diffusion model!

@inproceedings{xu2024magicanimate,
  title={{MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model}},
  author={Xu, Zhongcong and Zhang, Jianfeng and Liew, Jun Hao and Yan, Hanshu and Liu, Jiawei and Zhang, Chenxu and Feng, Jiashi and Shou, Zheng},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2024}
}
2023
MagicAvatar: Multimodal Avatar Generation and Animation

MagicAvatar: Multimodal Avatar Generation and Animation

arXiv 2023

Text prompt? Video clip? A few selfies? MagicAvatar turns them all into animated avatars through explicit motion-based generation.

@article{zhang2023magicavatar,
  title={{MagicAvatar: Multimodal Avatar Generation and Animation}},
  author={Zhang, Jianfeng and Yan, Hanshu and Xu, Zhongcong and Feng, Jiashi and Liew, Jun Hao},
  journal={arXiv preprint arXiv:2308.14748},
  year={2023}
}
MagicEdit: High-Fidelity Temporally Coherent Video Editing

MagicEdit: High-Fidelity Temporally Coherent Video Editing

arXiv 2023

Simple, effective, versatile—MagicEdit disentangles content, structure, and motion to power diverse video editing tasks with high quality.

@article{liew2023magicedit,
  title={{MagicEdit: High-Fidelity Temporally Coherent Video Editing}},
  author={Liew, Jun Hao and Yan, Hanshu and Zhang, Jianfeng and Xu, Zhongcong and Feng, Jiashi},
  journal={arXiv preprint arXiv:2308.14749},
  year={2023}
}
XAGen: 3D Expressive Human Avatars Generation

XAGen: 3D Expressive Human Avatars Generation

NeurIPS 2023

Want expressive 3D avatars with detailed face and hand control? XAGen delivers as the first model to master body, facial, and hand synthesis!

@inproceedings{xu2023xagen,
  title={{XAGen: 3D Expressive Human Avatars Generation}},
  author={Xu, Zhongcong and Zhang, Jianfeng and Liew, Jun Hao and Feng, Jiashi and Shou, Zheng},
  booktitle={Advances in Neural Information Processing Systems},
  year={2023}
}
GETAvatar: Generative Textured Meshes for Animatable Human Avatars

GETAvatar: Generative Textured Meshes for Animatable Human Avatars

ICCV 2023

GETAvatar generates explicit textured 3D meshes that capture rich surface details and enable efficient high-resolution rendering for animatable avatars.

@inproceedings{zhang2023getavatar,
  title={{GETAvatar: Generative Textured Meshes for Animatable Human Avatars}},
  author={Zhang, Xuanmeng and Zhang, Jianfeng and Chacko, Rohan and Xu, Hongyi and Song, Guoxian and Yang, Yi and Feng, Jiashi},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  year={2023}
}
Learning to Augment Poses for 3D Human Pose Estimation in Images and Videos

Learning to Augment Poses for 3D Human Pose Estimation in Images and Videos

TPAMI 2023

Simple yet effective extension of PoseAug, which decomposes video pose augmentation into end frames and intermediate generation for improved cross-dataset generalization.

@article{zhang2023learning,
  title={{Learning to Augment Poses for 3D Human Pose Estimation in Images and Videos}},
  author={Zhang, Jianfeng and Gong, Kehong and Wang, Xinchao and Feng, Jiashi},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  volume={45},
  number={8},
  pages={10012--10026},
  year={2023},
  publisher={IEEE}
}
PV3D: A 3D Generative Model for Portrait Video Generation

PV3D: A 3D Generative Model for Portrait Video Generation

ICLR 2023

PV3D breaks new ground in 3D portrait video generation, enabling multi-view consistent synthesis for animating static portraits and view-consistent editing.

@inproceedings{xu2023pv3d,
  title={{PV3D: A 3D Generative Model for Portrait Video Generation}},
  author={Xu, Zhongcong and Zhang, Jianfeng and Liew, Jun Hao and Zhang, Wenqing and Bai, Song and Feng, Jiashi and Shou, Zheng},
  booktitle={International Conference on Learning Representations},
  year={2023}
}
2022
PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervision

PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervision

Kehong Gong*, Bingbing Li*, Jianfeng Zhang*, Tao Wang*, Jing Huang, Michael Bi Mi, Jiashi Feng, Xinchao Wang
CVPR 2022Oral

How to train 3D pose estimators without 3D data? Our dual-loop framework generates augmented 2D-3D pairs through RL-based physical constraints and motion sequence hallucination!

@inproceedings{gong2022posetriplet,
  title={{PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervision}},
  author={Gong, Kehong and Li, Bingbing and Zhang, Jianfeng and Wang, Tao and Huang, Jing and Mi, Michael Bi and Feng, Jiashi and Wang, Xinchao},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2022}
}
AvatarGen: A 3D Generative Model for Animatable Human Avatars

AvatarGen: A 3D Generative Model for Animatable Human Avatars

ECCV Workshop 2022

Generative 3D humans that are both high-quality and animatable? AvatarGen achieves this through SMPL-guided canonical representation, trained only from 2D images.

@inproceedings{zhang2022avatargen,
  title={{AvatarGen: A 3D Generative Model for Animatable Human Avatars}},
  author={Zhang, Jianfeng and Jiang, Zihang and Yang, Dingdong and Xu, Hongyi and Shi, Yichun and Song, Guoxian and Xu, Zhongcong and Wang, Xinchao and Feng, Jiashi},
  booktitle={European Conference on Computer Vision Workshops},
  year={2022}
}
Geometry-Guided Progressive NeRF for Generalizable and Efficient Neural Human Rendering

Geometry-Guided Progressive NeRF for Generalizable and Efficient Neural Human Rendering

ECCV 2022

GP-NeRF tackles sparse-view human synthesis through geometry-guided multi-view integration for handling occlusions and progressive rendering for 70%+ speedup.

@inproceedings{chen2022geometry,
  title={{Geometry-Guided Progressive NeRF for Generalizable and Efficient Neural Human Rendering}},
  author={Chen, Mingfei and Zhang, Jianfeng and Xu, Xiangyu and Liu, Lijuan and Cai, Yujun and Feng, Jiashi and Yan, Shuicheng},
  booktitle={European Conference on Computer Vision},
  year={2022}
}
2021
Direct Multi-view Multi-person 3D Pose Estimation

Direct Multi-view Multi-person 3D Pose Estimation

NeurIPS 2021

Why use costly volumetric representation or 2D pose reconstruction? MvP directly estimates multi-person 3D poses through transformer decoder and geometry-guided projective attention.

@inproceedings{wang2021direct,
  title={{Direct Multi-view Multi-person 3D Pose Estimation}},
  author={Wang, Tao and Zhang, Jianfeng and Cai, Yujun and Yan, Shuicheng and Feng, Jiashi},
  booktitle={Advances in Neural Information Processing Systems},
  year={2021}
}
PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation

PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation

CVPR 2021Best Paper Finalist

3D pose estimators struggle with generalization? PoseAug solves this through differentiable, online pose augmentation that learns to generate diverse yet plausible training poses!

@inproceedings{gong2021poseaug,
  title={{PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation}},
  author={Gong, Kehong and Zhang, Jianfeng and Feng, Jiashi},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2021}
}
Body Meshes as Points

Body Meshes as Points

CVPR 2021

BMP simplifies multi-person mesh estimation through single-stage point-based representation, concurrently localizing instances and estimating meshes with depth coherence.

@inproceedings{zhang2021body,
  title={{Body Meshes as Points}},
  author={Zhang, Jianfeng and Yu, Dongdong and Liew, Jun Hao and Nie, Xuecheng and Feng, Jiashi},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2021}
}
2020
Inference Stage Optimization for Cross-scenario 3D Human Pose Estimation

Inference Stage Optimization for Cross-scenario 3D Human Pose Estimation

Jianfeng Zhang, Xuecheng Nie, Jiashi Feng
NeurIPS 2020

ISO improves cross-scenario 3D pose generalization by adapting models at inference time through geometry-aware self-supervised learning.

@inproceedings{zhang2020inference,
  title={{Inference Stage Optimization for Cross-scenario 3D Human Pose Estimation}},
  author={Zhang, Jianfeng and Nie, Xuecheng and Feng, Jiashi},
  booktitle={Advances in Neural Information Processing Systems},
  year={2020}
}

Academic Service

Computer Graphics

Conference Reviewer: SIGGRAPH, SIGGRAPH Asia

Journal Reviewer: TVCG, TOG

Computer Vision

Conference Reviewer: CVPR, ECCV, ICCV, NeurIPS, ICLR, 3DV, WACV, BMVC

Journal Reviewer: T-PAMI, IJCV, TIP