VTC: DNN Compilation with Virtual Tensors for Data Movement Elimination (paper)

VTC is a tensor compilation framework, built into PyTorch's Inductor backend (torch.compile), that eliminates unnecessary data movement in DNN workloads. It represents data-movement operators (reshape, concat, expand, slice, permute, ...) as virtual tensors tracked through index mappings, rather than as physical transfers to and from global memory. Evaluation shows that VTC achieves up to 1.9x speed-up and 60% inference memory savings on NVIDIA GPUs.

Environment Preparation

VTC is implemented inside a PyTorch source tree, so installation is the same as building PyTorch from source.

Set up Python Environment

conda create -n torch_vtc python=3.10
conda activate torch_vtc

Clone and Build VTC

First clone the repo, then:

cd torch-vtc
pip install -r requirements.txt
python setup.py develop

Run VTC

VTC is off by default and enabled through a single Inductor config flag:

import torch
import torch._inductor.config as inductor_config

inductor_config.enable_vtc = True

compiled = torch.compile(model)
out = compiled(x)

See example models in vtc_e2e_tests/.

Citation

@inproceedings{hu2026vtc,
  title={VTC: DNN Compilation with Virtual Tensors for Data Movement Elimination},
  author={Hu, Muyan and Gupta, Ahan and Yuan, Jiachen and Gupta, Vima and Kim, Taeksang and Xu, Xin and Kulkarni, Janardhan and Dekel, Ofer and Adve, Vikram and Mendis, Charith},
  booktitle={Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI '26)},
  year={2026}
}

Contact

Muyan Hu: muyanhu2@illinois.edu