Physical AI

An industrial robot used for flat-glass handling. Photograph: KUKA Roboter GmbH, Bachmann; Wikimedia Commons, public domain.

Physical artificial intelligence (physical AI) refers to artificial intelligence used in systems that perceive or interact with the physical world. Examples include robots, automated vehicles and systems that interpret camera footage to monitor physical environments. In industry usage, NVIDIA describes physical AI as technology that enables machines to interpret their surroundings and use that information to guide actions.

Physical AI combines AI methods with sensing, computation and, where physical action is required, control systems and mechanical hardware. Its applications overlap with Robotics, Embodied AI and Autonomous Systems. Implementations differ: industrial robot systems can have separate controllers and mechanical components, while research models such as RT-2 learn a direct relationship between visual observations, language instructions and robot actions.

Relationship to robotics and automation

Robotics and physical AI are related concepts, but they describe different aspects of a system. Robotics includes the design and control of physical machines. The International Federation of Robotics, drawing on ISO 8373, defines robots in terms of programmed actuation and a degree of autonomy. That definition does not require a particular AI technique.

Robot motion can be controlled using explicitly designed feedback methods, such as proportional-integral-derivative control, rather than a policy learned from training data. Conversely, physical AI can include camera-based monitoring that has no moving robot body. The presence of a robot therefore does not, by itself, establish that a system uses machine learning.

Core elements

Sensing and perception

Sensors provide information about the environment and the machine itself. Industrial robot systems can use cameras, position sensors and other sensing devices as inputs to their controllers. AI perception models can interpret visual observations, for example by locating an object that a robot is expected to grasp.

Planning, control and actuation

Planning determines a desired movement or sequence of actions. Control converts that objective into commands for the machine and uses feedback to correct deviations. Depending on the task, a controller may regulate position, motion, contact force or a combination of motion and force. Actuators and end-effectors, such as motors and grippers, carry out the physical action.

Learning models

Machine learning can connect observations and instructions to actions. In RT-2, researchers trained vision-language models using both robot demonstrations and internet-derived vision-language data. The resulting vision-language-action model represented robot actions as tokens that the model could predict. The study evaluated this approach on robotic manipulation tasks.

Training and simulation

Robot-learning research uses data collected from physical robots as well as simulated environments. The Open X-Embodiment collaboration assembled datasets from 22 robot embodiments and investigated whether shared training could improve performance across different platforms. Its RT-X experiments reported beneficial transfer between robots, providing evidence that experience collected on one platform can help another within the evaluated tasks.

Simulation can provide training examples without collecting every example on physical equipment. However, simulated observations may differ from those encountered by a real robot. This difference is often called the reality gap. Tobin and colleagues investigated domain randomization, which varies the appearance of simulated scenes during training, and demonstrated transfer to real-world object localization and grasping.

Reinforcement learning is another training approach. A system learns through actions and feedback expressed as rewards. Physical AI development platforms use simulated environments for this process, allowing repeated trials before deployment on hardware.

Applications

Applications and research areas include:

• Industrial manipulation: Robot systems perform tasks such as material handling, assembly, welding and inspection. AI-based manipulation research also investigates grasping objects using visual information and language instructions.
• Warehouse and logistics operations: Autonomous mobile robots use onboard sensing to navigate and respond to obstacles in environments shared with people and equipment.
• Automated driving: Vehicle automation systems perform different portions of the driving task. The U.S. National Highway Traffic Safety Administration distinguishes driver assistance from higher levels of automation. At Level 2, the driver must remain engaged and attentive even when the system assists with both steering and acceleration or braking.
• Assistive and wearable robotics: Research on exoskeletons and other wearable robots examines sensing, human intention recognition and control for rehabilitation and assistance.

Safety and trust

Physical systems require assessment of the complete machine and its operating environment. Hazards include unexpected movement, contact with moving equipment and stored energy. OSHA's industrial robotics guidance recommends risk assessment throughout design, integration, operation and maintenance, with safeguards selected for the identified hazards.

AI models also require evaluation. NIST's AI Risk Management Framework identifies characteristics of trustworthy AI, including validity and reliability, safety, security and resilience, accountability and transparency, explainability, privacy protection and management of harmful bias. The framework is intended for voluntary use across the development and operation of AI systems.

Cybersecurity is relevant when programmable equipment monitors or changes physical processes. NIST's guidance on operational technology security addresses these systems while accounting for their performance, reliability and safety requirements.

Limitations

Performance depends on the task, training data and operating conditions. Research on transferring learning between robots and transferring models from simulation addresses specific evaluation settings; these results do not establish reliable operation in every environment.

Physical control also has its own requirements, including stable responses and acceptable tracking errors. A system that interprets an instruction successfully must still execute the resulting movement within the machine's physical and safety constraints.

See also

• Robotics
• Embodied AI
• Autonomous Systems
• Robot Foundation Models
• Physical AI Identity
• Robot Cybersecurity

Further reading

• Modern Robotics — educational resources on robot mechanics, planning and control.
• NIST AI Risk Management Framework — guidance on managing AI risks.
• Open X-Embodiment research paper — datasets and models for learning across robot platforms.