This paper treats computer vision as a single “multimodal generation” problem. Instead of building a different model for each vision task, the authors ask one model to follow natural-language instructions and produce either text, images, or both. The idea is that vision tasks can be expressed in the model’s native output formats: words for labels or descriptions, and pixels for dense spatial predictions.

The team built SenseNova‑Vision and a training dataset called the SenseNova‑Vision Corpus. They converted many existing vision annotations into instruction–response examples that pair a plain-language prompt with the desired text or image output. Starting from a publicly available pretrained multimodal model, they trained SenseNova‑Vision mainly on this corpus and mixed in other multimodal data to preserve general capabilities. No task-specific prediction heads or changes to the model architecture were used.

SenseNova‑Vision can handle a wide range of tasks. The paper reports results on object detection, optical character recognition (OCR), keypoint estimation (finding specific points on objects), segmentation (which pixels belong to an object), depth and surface normal prediction (geometry estimates), point maps, and camera pose estimation. The system also supports language-defined variants, for example asking for objects of a certain color or items within a named region.

In experiments, a single unified model matched leading task-specialized systems across structured visual understanding, dense geometric prediction, segmentation, and multi-view visual geometry. If robust, this approach could simplify how vision capabilities are added to larger foundation models. The authors also released both the model and the corpus for others to use.