Vision-Language-Action Models

Vision-Language-Action Models
Models that combine visual observations and language instructions to predict or select robotic actions.
Area	Physical AI and robotics
Applications	Research, engineering and systems operation

Vision-Language-Action Models models that combine visual observations and language instructions to predict or select robotic actions.

How the model operates

A vision-language-action (VLA) model connects images or other observations, natural-language instructions and an action representation. The action may denote motion, gripper control or a discrete skill. Some approaches adapt visual-language models, while others train joint representations using robot trajectories.

Training and evaluation

Training data may mix robot demonstrations with image-language examples. Evaluation should distinguish success on familiar tasks from transfer to new objects, environments and robot platforms. The control frequency, action discretization and permitted operating space strongly influence reported results.

Role in physical AI

VLA systems are one research approach to Physical AI and Robot Foundation Models. They can reduce reliance on hand-coded task definitions, but they still operate through particular sensors, actuators and control interfaces.

Limitations and safety

Language ambiguity, unexpected objects, perception errors and distribution shifts can all lead to unsuitable actions. A learned policy is not a certified safety function; practical systems require independent safety limits and testing.

See also

• RT-2 (robotics model)
• Robot Learning from Demonstration
• Physical AI Benchmarking

Further reading

• RT-2: New model translates vision and language into action — Google DeepMind
• Scaling up learning across many different robot types — Google DeepMind
• AI Risk Management Framework (AI RMF 1.0) — NIST