STATERA Library · Series 03 · Volume 06 · Chapter 10 · EN
Source: https://github.com/Statera-Library/03_Vol_06_Robot_Hand_and_Dexterous_Grasping/blob/main/10_In_Hand_Manipulation/10_In_Hand_Manipulation_EN.md

Volume 06 Robot Hands and Dexterous Grasping

10. In-Hand Manipulation

10.01 Introduction to In-Hand Manipulation

In-hand manipulation is one of the most advanced and challenging capabilities in robotic grasping and dexterous manipulation. While traditional robotic grasping focuses on acquiring and holding an object securely, in-hand manipulation extends this capability by enabling a robot to change the position, orientation, or configuration of an object while it remains within the robot's grasp. This capability is fundamental for achieving human-like dexterity and represents a major step toward general-purpose robotic manipulation. Within the context of dexterous robot hands, in-hand manipulation serves as a bridge between grasping and full task execution, allowing robots to perform complex operations without repeatedly placing and regrasping objects. It occupies a central role in advanced manufacturing, service robotics, medical robotics, logistics automation, and future humanoid systems. The topic forms a major component of Volume 06, Chapter 10 of the Robot Hand and Dexterous Grasping series.

Humans perform in-hand manipulation continuously in everyday life. Activities such as rotating a pen between fingers, adjusting a screwdriver before tightening a screw, repositioning a key before inserting it into a lock, or changing the orientation of a smartphone all rely on sophisticated in-hand manipulation skills. These actions appear effortless because the human hand integrates tactile sensing, proprioception, force control, compliance, and motor planning into a highly coordinated system. Replicating these capabilities in robotic systems requires solving numerous challenges involving mechanics, sensing, planning, control, and artificial intelligence.

The primary objective of in-hand manipulation is to transform an object\'s pose relative to the hand without losing grasp stability. Unlike pick-and-place operations where the object is transported as a rigidly held item, in-hand manipulation intentionally introduces controlled motion between the fingers and the object. This motion may involve rolling contact, sliding contact, pivoting, finger gaiting, regrasping, or coordinated multi-finger motion. During the process, the robot must continuously maintain sufficient grasp forces to prevent accidental object drops while simultaneously allowing controlled object movement.

In-hand manipulation can be broadly categorized into several classes. Translational manipulation changes the position of the object within the hand while maintaining its orientation. Rotational manipulation changes the object\'s orientation while keeping its center of mass approximately fixed. Reorientation tasks involve combined translational and rotational movements to achieve a target pose. Finger gaiting strategies sequentially reposition fingers around the object to expand the reachable manipulation workspace. Regrasping methods intentionally modify contact configurations to achieve otherwise unreachable object poses. Each category requires different planning and control techniques but shares the common goal of maintaining grasp stability throughout the manipulation process.

The mechanics of in-hand manipulation are fundamentally governed by contact interactions between the fingers and the object. These contacts generate forces and moments that determine how the object moves within the grasp. Depending on the contact model, interactions may be represented as point contacts, soft-finger contacts, line contacts, or surface contacts. Friction plays a critical role because it determines whether contacts remain stable, slide intentionally, or experience uncontrolled slip. Accurate modeling of friction cones, contact constraints, and grasp wrench spaces is therefore essential for understanding and controlling in-hand manipulation behavior.

One of the most important concepts in dexterous manipulation is the distinction between intrinsic and extrinsic dexterity. Intrinsic dexterity refers to object manipulation achieved solely through motions of the fingers and joints. The object remains suspended within the hand while the fingers coordinate to move it. Extrinsic dexterity, in contrast, utilizes environmental interactions to assist manipulation. The robot may push the object against a table, wall, fixture, or external surface to achieve reorientation. Extrinsic dexterity often simplifies manipulation tasks because the environment provides additional constraints and support forces that reduce the burden on finger coordination.

Finger gaiting represents one of the most powerful approaches for achieving large object reconfigurations. Inspired by human finger movements, finger gaiting involves temporarily releasing one finger while maintaining object stability with the remaining fingers. The released finger moves to a new contact location and re-establishes contact before another finger begins moving. Through repeated sequences of finger gaiting, a robot can gradually reposition its grasp and manipulate objects far beyond the limits of individual finger motion ranges. This technique greatly expands the effective manipulation workspace of dexterous hands.

Rolling manipulation is another common strategy. During rolling, the object rotates relative to the fingertip while maintaining continuous contact. Rolling minimizes energy loss because it avoids large sliding friction forces. Human fingertips naturally exploit rolling contacts during many everyday manipulation tasks. Robotic implementations require precise control of contact forces and fingertip trajectories to ensure stable rolling motion. Successful rolling manipulation enables smooth object reorientation and often improves manipulation efficiency.

Pivoting manipulation involves rotating an object around one or more contact points. For example, a robotic hand may pivot a screwdriver into a preferred orientation before inserting it into a fastener. Pivoting tasks require accurate estimation of object mass distribution, contact geometry, and gravitational effects. Controlled pivoting often leverages gravity to reduce actuator effort and simplify planning.

Regrasping strategies play a crucial role when desired object poses cannot be reached through continuous contact motions alone. During regrasping, the robot intentionally modifies the grasp configuration by changing contact locations. Regrasp planning involves determining safe intermediate grasps that maintain object security while enabling progression toward the target pose. This process introduces combinatorial complexity because many possible contact sequences may exist. Advanced planning algorithms therefore use optimization techniques, search methods, or learning-based approaches to identify efficient regrasp sequences.

The kinematics of in-hand manipulation differ significantly from conventional robotic arm kinematics. Instead of controlling a single end-effector pose, the system must simultaneously manage multiple finger contacts and object states. The object effectively becomes part of a coupled kinematic system involving the hand and the manipulated object. The resulting configuration space can be extremely high-dimensional, especially for multifinger hands with many degrees of freedom. Planning algorithms must therefore account for both finger joint configurations and object pose variables.

Dynamics further increase the complexity of the problem. In dynamic manipulation tasks, inertial forces influence object behavior. Fast rotations, sudden accelerations, and impact interactions introduce nonlinear effects that must be modeled accurately. Dynamic in-hand manipulation can exploit these effects advantageously. For example, humans often use inertia to flip objects rapidly rather than relying solely on slow, controlled finger movements. Robotic systems are increasingly exploring similar strategies to improve efficiency and speed.

Tactile sensing is a key enabling technology for robust in-hand manipulation. Visual sensing alone often cannot provide sufficient information because contact states are frequently occluded by the fingers themselves. Tactile sensors supply direct measurements of contact pressure distributions, force magnitudes, slip events, and local surface geometries. These measurements allow robots to estimate object motion relative to the fingers and adjust grasp forces accordingly. Advanced tactile sensing systems can detect subtle contact changes that are critical for successful manipulation.

Proprioceptive sensing also contributes significantly to manipulation performance. Joint encoders, motor current sensors, tendon tension sensors, and force-torque sensors provide information about finger configurations and interaction forces. By combining tactile and proprioceptive data, robots can build more accurate internal representations of object states during manipulation. Sensor fusion approaches integrate these multiple modalities to improve robustness under uncertainty.

Planning for in-hand manipulation presents substantial computational challenges. The planner must determine how finger contacts should move, what forces should be applied, and how grasp stability should be maintained throughout the manipulation sequence. Classical planning approaches rely on geometric models and optimization techniques. These methods often formulate manipulation as a constrained optimization problem that minimizes task errors while satisfying friction, force, and kinematic constraints.

Sampling-based planners extend concepts from motion planning to manipulation planning. Methods such as Rapidly Exploring Random Trees and Probabilistic Roadmaps can be adapted to explore object pose spaces and contact transition spaces. These approaches are particularly useful when analytical solutions are difficult to derive due to complex object geometries or contact interactions.

Optimization-based manipulation planning has gained considerable attention in recent years. Such methods formulate manipulation objectives as mathematical optimization problems. Cost functions may include target pose errors, energy consumption, manipulation time, contact stability margins, or force minimization objectives. Modern solvers can compute feasible manipulation trajectories while satisfying complex physical constraints.

Machine learning has emerged as a transformative technology for in-hand manipulation. Reinforcement learning, imitation learning, behavior cloning, and self-supervised learning have demonstrated remarkable capabilities in discovering manipulation strategies that would be difficult to engineer manually. Learning-based methods can automatically identify efficient finger coordination patterns, adaptive force control policies, and robust manipulation strategies from large-scale experience.

One of the most influential demonstrations of learning-based dexterity was the development of robotic systems capable of solving object reorientation tasks using multifinger hands. By training in simulation and transferring policies to physical hardware, researchers showed that robots could learn complex manipulation skills involving dynamic finger coordination, object rotations, and recovery from disturbances. These achievements highlighted the potential of combining dexterous hardware with large-scale machine learning.

Simulation plays an essential role in modern manipulation research because collecting physical training data is often expensive and time-consuming. Physics simulators provide environments for testing manipulation algorithms under controlled conditions. However, accurately simulating contact-rich interactions remains challenging. Small modeling inaccuracies can significantly affect manipulation outcomes. Consequently, sim-to-real transfer techniques, domain randomization, and system identification methods are commonly employed to bridge the gap between simulation and reality.

Benchmarking is critical for evaluating manipulation performance. Researchers measure task success rates, object pose accuracy, manipulation speed, force efficiency, robustness to disturbances, and generalization across object categories. Standardized benchmark datasets and object sets enable meaningful comparisons between different algorithms and hardware platforms. In-hand manipulation benchmarks increasingly focus on real-world tasks that require practical dexterity rather than simplified laboratory demonstrations.

Industrial applications of in-hand manipulation are expanding rapidly. Manufacturing environments benefit from the ability to orient parts without requiring dedicated fixtures. Assembly operations often involve complex object alignment tasks that can be simplified through dexterous manipulation. Electronics manufacturing, automotive assembly, aerospace production, and medical device fabrication all present opportunities for advanced in-hand manipulation systems.

In logistics and warehouse automation, dexterous manipulation enables robots to handle a wider variety of objects with fewer grasping failures. Rather than relying on repeated pick-and-place operations, robots can adjust object poses directly within the grasp, improving efficiency and reducing cycle times. Service robots similarly benefit from dexterous manipulation when handling household objects, kitchen utensils, medical supplies, or consumer products.

Humanoid robots represent perhaps the most demanding application domain for in-hand manipulation. To operate effectively in human-designed environments, humanoids must manipulate objects using capabilities similar to those of human hands. Tasks such as opening containers, using tools, assembling products, operating equipment, and interacting with consumer devices all require sophisticated in-hand manipulation skills. Consequently, dexterous manipulation remains a major research focus for next-generation humanoid systems.

Despite substantial progress, significant challenges remain. Robust manipulation under uncertainty, efficient learning from limited data, reliable tactile perception, real-time planning, and generalization across diverse object categories continue to be active research areas. Future systems will likely integrate foundation models, multimodal perception, tactile intelligence, world models, and large-scale reinforcement learning to achieve higher levels of dexterity.

Ultimately, in-hand manipulation represents one of the defining capabilities that separates simple robotic grippers from truly dexterous robotic hands. It transforms grasping from a static holding operation into a dynamic process of object control and reconfiguration. As sensing technologies, actuation systems, control methods, and artificial intelligence continue to advance, in-hand manipulation will become a foundational capability for autonomous robots operating in complex real-world environments. Its successful realization is essential for achieving human-level manipulation performance and enabling the next generation of intelligent robotic systems.

10.02 Finger Gaiting

Finger gaiting is one of the most important and distinctive mechanisms in dexterous robotic manipulation. It enables a robot hand to change grasp configurations, reposition contact points, and manipulate objects within the hand without releasing them entirely. The concept is inspired directly by human hand behavior. When humans rotate a pen, adjust a screwdriver, reposition a key, or manipulate a small object with their fingertips, they rarely maintain fixed contact locations. Instead, individual fingers sequentially release, move, and re-establish contact while the remaining fingers continue to stabilize the object. This coordinated sequence of contact transitions is known as finger gaiting. In robotic hands, finger gaiting provides a fundamental method for extending manipulation capabilities beyond the physical limitations of finger joint ranges and static grasp configurations. It serves as one of the core techniques that enable true in-hand manipulation and advanced dexterity within multifinger robotic systems.

The term "gaiting" originates from locomotion research. Just as a walking robot moves one leg at a time while maintaining overall balance, a dexterous robotic hand moves one finger at a time while maintaining overall grasp stability. During finger gaiting, one or more fingers temporarily disengage from the object and move to new contact locations. The remaining fingers continue to provide sufficient support forces to prevent the object from slipping or falling. Once the moving finger establishes a new contact, another finger may begin repositioning. Through repeated cycles of these coordinated movements, the hand can gradually alter its grasp configuration and manipulate the object into otherwise unreachable poses.

The necessity of finger gaiting arises from the limited workspace of individual fingers. Every finger possesses finite joint ranges, limited reachability, and constrained contact surfaces. If a robot attempts to manipulate an object using fixed contact locations only, the achievable object motions become severely restricted. Eventually, one or more fingers reach their kinematic limits, preventing further manipulation. Finger gaiting overcomes this limitation by continuously relocating contact points, effectively creating a larger virtual workspace. This capability allows robotic hands to perform extensive object rotations, translations, and reorientations that would be impossible with static grasps.

From a geometric perspective, finger gaiting can be viewed as a process of changing contact topology while preserving grasp stability. A grasp is defined by the set of contact points between the fingers and the object. Each contact point contributes constraints and force transmission capabilities. When a finger changes its contact location, the overall grasp topology changes. The challenge is to ensure that the remaining contacts continue to satisfy stability requirements throughout the transition. Consequently, finger gaiting is fundamentally a problem of coordinated contact management.

A typical finger gaiting sequence consists of several phases. The process begins with a stable initial grasp configuration. A planning algorithm identifies a finger that can be moved without compromising grasp security. The selected finger is then unloaded by gradually transferring its contact force to other fingers. Once the force approaches zero, the finger disengages from the object and moves toward a new target contact location. During this movement phase, the remaining fingers maintain force closure and object stability. After reaching the new location, the finger re-establishes contact and gradually increases its grasp force. The system then reassesses grasp quality and prepares for the next gaiting step.

Force closure plays a critical role during finger gaiting. A robotic hand must ensure that the remaining contacts can resist external disturbances while one finger is moving. If the contact configuration loses force closure, the object may become unstable and slip. Therefore, gait planning algorithms often evaluate grasp wrench spaces and contact constraints before selecting which finger to move. Stability analysis becomes particularly important when manipulating heavy objects, asymmetric objects, or objects with low-friction surfaces.

The effectiveness of finger gaiting depends strongly on the number of available fingers. Three-finger hands can perform basic gaiting operations, but their manipulation flexibility is limited because removing one finger leaves only two active contacts. Four-finger hands provide significantly greater robustness because three contacts remain available during finger relocation. Five-finger anthropomorphic hands offer even greater flexibility, allowing multiple alternative gait sequences and more sophisticated manipulation strategies. As the number of fingers increases, the redundancy of the system grows, enabling more robust planning and control.

Finger gaiting can be classified into several categories based on manipulation objectives. Repositioning gaiting focuses on moving contact points to improve future manipulation opportunities. Reorientation gaiting aims to rotate an object toward a target orientation. Translational gaiting changes the object\'s position within the hand. Precision gaiting emphasizes accurate placement of contact points for delicate tasks. Dynamic gaiting incorporates momentum and inertial effects to accelerate manipulation. Each category places different demands on sensing, planning, and control systems.

One of the most challenging aspects of finger gaiting is contact planning. Determining where a finger should move requires understanding object geometry, grasp quality, reachable workspaces, and future task requirements. The planner must evaluate candidate contact locations and predict their impact on manipulation performance. Poor contact choices can lead to dead-end configurations where further manipulation becomes impossible. Consequently, contact planning is often formulated as an optimization problem involving grasp stability, reachability, dexterity, and task progress.

Kinematic analysis forms the foundation of finger gait planning. Each finger has a reachable workspace defined by its joint limits and mechanical structure. The planner must ensure that target contact points lie within reachable regions while avoiding singularities and collisions. In multifinger hands, kinematic coupling between fingers further complicates the problem because the motion of one finger may affect the available workspace of neighboring fingers. Advanced planners therefore consider the entire hand-object system rather than individual fingers in isolation.

Finger gaiting is closely related to regrasping, but the two concepts are not identical. Regrasping generally refers to changing grasp configurations through any sequence of contact modifications. Finger gaiting represents a specific form of regrasping in which fingers move sequentially while maintaining continuous object support. Traditional regrasping may involve placing an object on a table and acquiring a completely new grasp. Finger gaiting avoids this interruption by preserving object control throughout the process. As a result, it is particularly valuable for tasks requiring continuous manipulation.

The mechanics of finger gaiting depend heavily on friction. During contact transitions, frictional forces help stabilize the object while fingers move. High-friction contacts provide larger stability margins and simplify planning. Low-friction environments increase the risk of slip and often require more conservative force control strategies. Some robotic systems actively adjust grasp forces during gaiting to compensate for varying friction conditions. Others employ tactile sensing to detect incipient slip and adapt accordingly.

Tactile sensing provides essential information for successful finger gaiting. During contact transitions, the robot must know precisely when a finger loses contact and when a new contact is established. Tactile arrays can detect pressure distributions, contact locations, and micro-slip events. This information allows the controller to regulate forces more accurately and reduce uncertainty. Advanced tactile systems can even estimate local object geometry, helping the planner select better contact locations during gaiting operations.

Vision systems complement tactile sensing by providing global information about object pose and hand configuration. Cameras can track object orientation, estimate contact locations, and monitor manipulation progress. However, visual sensing alone is often insufficient because many contacts are occluded by the fingers themselves. Therefore, modern dexterous manipulation systems frequently combine visual perception with tactile sensing and proprioceptive feedback to achieve robust finger gaiting performance.

Control strategies for finger gaiting must coordinate multiple objectives simultaneously. The controller must maintain object stability, execute finger trajectories, regulate grasp forces, avoid collisions, and compensate for disturbances. This multi-objective nature makes the control problem highly complex. Hierarchical control architectures are commonly employed, where high-level planners determine gait sequences while low-level controllers manage force regulation and motion execution.

Optimization-based control methods have become increasingly popular for finger gaiting. These methods formulate manipulation as a constrained optimization problem that balances task objectives against physical constraints. Constraints may include friction cone limits, joint limits, collision avoidance requirements, and force closure conditions. By solving optimization problems in real time, robotic systems can generate adaptive gaiting behaviors that respond to changing environments and object properties.

Learning-based approaches have significantly advanced the state of the art in finger gaiting. Reinforcement learning allows robots to discover effective gaiting strategies through trial-and-error interactions. Instead of relying entirely on analytical models, the robot learns policies that maximize manipulation success rates. Such methods are particularly valuable when dealing with complex contact dynamics that are difficult to model accurately. Simulation environments enable millions of training episodes, allowing robots to acquire sophisticated manipulation skills before deployment on physical hardware.

Behavior cloning and imitation learning provide alternative approaches. Human demonstrations of finger gaiting can be recorded using motion capture systems, data gloves, or teleoperation interfaces. Learning algorithms then extract manipulation policies from these demonstrations. Because humans naturally perform efficient gaiting motions, demonstration-based learning often provides strong initial performance and reduces training time compared to purely reinforcement-based methods.

One of the most influential research directions involves combining reinforcement learning with domain randomization. By exposing simulated robots to a wide range of object shapes, friction coefficients, masses, and contact conditions, learned policies become more robust and transferable to real-world systems. Such approaches have enabled robotic hands to perform impressive finger gaiting tasks, including object rotations, cube reorientation, tool manipulation, and continuous in-hand repositioning.

Finger gaiting is especially important for humanoid robots. Human environments contain countless objects that require continuous adjustment during manipulation. Opening bottles, using tools, operating machinery, assembling products, and handling consumer devices all involve frequent contact reconfiguration. Without finger gaiting, robotic hands would require repeated pick-and-place operations that significantly reduce efficiency. Finger gaiting therefore represents a critical capability for achieving human-like manipulation performance.

Industrial applications of finger gaiting are emerging in advanced manufacturing environments. Assembly operations often require precise orientation adjustments before insertion or fastening. Traditional robotic systems rely on external fixtures to perform these adjustments. Dexterous hands equipped with finger gaiting capabilities can accomplish the same tasks internally, reducing fixture complexity and increasing operational flexibility. This capability is particularly valuable in high-mix, low-volume manufacturing environments where product variations are frequent.

Medical robotics also benefits from finger gaiting. Surgical instruments, catheters, needles, and delicate medical devices often require precise in-hand adjustments. Finger gaiting enables fine manipulations while maintaining stable control of sensitive tools. Similar advantages apply to laboratory automation, pharmaceutical handling, and biomedical assembly tasks.

Performance evaluation of finger gaiting systems involves multiple metrics. Researchers measure grasp stability, manipulation success rate, contact transition efficiency, trajectory accuracy, cycle time, force utilization, and robustness against disturbances. Additional metrics may evaluate energy consumption, computational complexity, and adaptability to novel objects. Standardized benchmark tasks increasingly include finger gaiting challenges because they provide meaningful measures of dexterous manipulation capability.

Despite significant progress, finger gaiting remains an active research area. Challenges include robust contact estimation, real-time planning, uncertainty management, tactile integration, and generalization across diverse object categories. Complex objects with irregular geometries, deformable surfaces, or dynamic behaviors continue to present difficulties. Future research is expected to leverage foundation models, world models, multimodal learning, and advanced tactile intelligence to overcome these limitations.

Ultimately, finger gaiting represents one of the defining characteristics of dexterous robotic hands. It transforms a static grasp into a dynamic and adaptable manipulation system capable of continuously modifying contact configurations. By allowing fingers to reposition themselves while maintaining object stability, finger gaiting dramatically expands the capabilities of robotic hands and brings robotic manipulation closer to the remarkable dexterity exhibited by human hands. As robotic hardware, sensing technologies, and artificial intelligence continue to evolve, finger gaiting will remain a foundational technique for achieving advanced in-hand manipulation and human-level robotic dexterity.

10.03 Rolling and Pivoting

Rolling and pivoting are two of the most fundamental object manipulation strategies used in dexterous robotic hands for achieving in-hand manipulation. While grasping focuses on establishing and maintaining stable contact with an object, rolling and pivoting enable controlled motion of the object relative to the fingers without completely releasing it. These manipulation modes are widely observed in human hands and play a critical role in tasks such as object reorientation, precision positioning, tool usage, assembly operations, and everyday object handling. In robotic systems, rolling and pivoting serve as essential mechanisms for transforming a static grasp into a dynamic manipulation process. They provide efficient alternatives to repeated pick-and-place actions and form important building blocks for advanced dexterous manipulation capabilities. Within the framework of in-hand manipulation, rolling and pivoting are often considered intermediate manipulation primitives that connect simple grasp maintenance with complex object reconfiguration.

Humans continuously use rolling and pivoting during daily activities. When rotating a pen between fingers, turning a key before insertion, adjusting a screwdriver orientation, repositioning a coin, or manipulating a smartphone, the fingers rarely maintain rigid contact. Instead, controlled rolling and pivoting motions occur at the contact interfaces. These motions allow large changes in object pose while minimizing the need for grasp release and reacquisition. The remarkable dexterity of the human hand is partly attributable to its ability to exploit such contact mechanics efficiently. Robotic manipulation research seeks to replicate these capabilities by developing algorithms, sensors, and control systems capable of managing rolling and pivoting interactions.

Rolling manipulation refers to a motion in which an object rotates relative to a contact surface while maintaining continuous contact without significant sliding. The contact point on the object changes continuously, and the corresponding contact point on the finger surface also changes. Ideally, rolling occurs under a no-slip condition where relative tangential velocity at the contact interface remains close to zero. Because rolling minimizes frictional losses and reduces wear, it is generally considered an efficient and stable manipulation strategy.

In a rolling contact, the object effectively moves through a sequence of infinitesimally changing contact locations. This creates a smooth trajectory that can be exploited to reorient or reposition the object. Rolling can occur between a fingertip and an object, between multiple fingers and an object, or between the object and an environmental surface. The geometry of the contact surfaces strongly influences the resulting motion. Curved fingertips, compliant surfaces, and rounded object geometries often facilitate rolling interactions.

The mechanics of rolling can be understood through contact kinematics. At the contact interface, the instantaneous velocity of the object relative to the finger must satisfy rolling constraints. These constraints establish relationships between object motion and finger motion. Rolling contact generates coupled rotational and translational behavior that can be modeled using differential geometry and rigid body kinematics. Accurate modeling is important because even small deviations from ideal rolling can accumulate and significantly affect manipulation outcomes.

One major advantage of rolling manipulation is its ability to generate large object rotations while maintaining stable contact. Instead of repeatedly changing grasp configurations, the robot can exploit rolling motions to gradually achieve a desired orientation. This reduces manipulation complexity and often improves task efficiency. Rolling is particularly useful for cylindrical objects, spherical objects, rounded tools, and many consumer products that naturally support smooth contact transitions.

Another advantage of rolling is force efficiency. Because rolling minimizes sliding friction, less energy is dissipated during manipulation. Lower frictional losses reduce actuator effort and heat generation while improving overall control stability. In robotic hands where power consumption and actuator capacity are constrained, rolling-based manipulation strategies can provide significant operational benefits.

Despite its advantages, rolling manipulation presents several challenges. The contact geometry must be estimated accurately. Small errors in object shape models can lead to incorrect motion predictions. Rolling contacts are also highly sensitive to friction conditions. Excessive friction may prevent rolling and induce sticking behavior, while insufficient friction may cause uncontrolled slip. Furthermore, rolling motions often exhibit nonlinear behavior that complicates planning and control.

Tactile sensing plays a crucial role in rolling manipulation. Tactile arrays can detect contact locations, pressure distributions, and local deformation patterns. By monitoring these signals, the robotic system can estimate whether rolling is occurring as intended or whether slip has begun to develop. Advanced tactile sensors can even reconstruct contact geometry, providing valuable information for manipulation planning.

Visual sensing complements tactile feedback by tracking object pose during rolling operations. Cameras provide global information about object orientation and position. Sensor fusion techniques combine tactile and visual information to improve estimation accuracy. This multimodal perception approach has become increasingly important for robust rolling manipulation in unstructured environments.

Pivoting manipulation differs from rolling in both mechanics and objectives. In pivoting, the object rotates around a relatively fixed contact point or axis. Rather than continuously changing contact locations as in rolling, pivoting maintains one dominant contact region that serves as a rotational center. The object effectively swings or rotates around this pivot while other contacts provide support and control forces.

Pivoting frequently exploits gravity as an active manipulation resource. Consider a robotic hand holding a screwdriver vertically. By slightly relaxing certain grasp forces, the object can pivot under gravitational influence until it reaches a desired orientation. The robot then stabilizes the new pose by adjusting contact forces. Such gravity-assisted pivoting can achieve efficient object reorientation with minimal actuator effort.

The physics of pivoting are governed by torque balance around the pivot point. Gravitational forces, contact forces, inertial effects, and frictional moments collectively determine object motion. Successful pivoting requires maintaining a delicate balance between stability and mobility. Excessive grasp force may prevent motion entirely, while insufficient force may result in object loss. Consequently, force regulation is one of the most critical aspects of pivoting control.

Pivoting is widely used in industrial assembly tasks. Components often need precise orientation adjustments before insertion into fixtures or mating parts. Rather than releasing and regrasping the component, a dexterous robotic hand can pivot the object into alignment. This approach reduces cycle time, improves efficiency, and minimizes the risk of dropping sensitive components.

Many tool-use applications rely heavily on pivoting. Wrenches, screwdrivers, scalpels, surgical instruments, and inspection probes frequently require orientation changes during operation. Human users naturally perform these adjustments through subtle pivoting motions. Replicating such capabilities in robotic systems significantly expands the range of achievable tasks.

Rolling and pivoting are often combined during complex manipulation sequences. A robot may initially roll an object to achieve coarse orientation changes and subsequently employ pivoting for fine adjustments. Conversely, pivoting may position the object into a configuration that facilitates rolling. The integration of these manipulation primitives enables highly flexible object reconfiguration strategies.

The concept of manipulation primitives is central to understanding rolling and pivoting. Rather than treating every manipulation task as a unique problem, researchers decompose complex behaviors into reusable primitive actions. Rolling, pivoting, sliding, finger gaiting, and regrasping represent a set of fundamental primitives that can be combined to achieve sophisticated manipulation objectives. This modular perspective simplifies planning and facilitates skill transfer across tasks.

Planning rolling and pivoting motions presents significant computational challenges. The planner must consider contact geometry, friction constraints, object dynamics, force closure requirements, and task objectives simultaneously. Because contact-rich interactions are inherently nonlinear, many planning algorithms rely on optimization techniques or sampling-based approaches.

Optimization-based planning formulates manipulation as a constrained mathematical problem. The objective may involve minimizing pose error, manipulation time, energy consumption, or force requirements. Constraints enforce contact stability, collision avoidance, and kinematic feasibility. Modern optimization frameworks can generate manipulation trajectories that satisfy complex physical requirements while achieving desired task outcomes.

Sampling-based planning methods explore large configuration spaces by generating candidate motion sequences. Techniques such as Rapidly Exploring Random Trees and Probabilistic Roadmaps have been adapted for manipulation planning. These approaches are particularly useful when analytical models are difficult to construct or when object geometries are highly complex.

Learning-based approaches have emerged as powerful alternatives for rolling and pivoting manipulation. Reinforcement learning enables robotic systems to discover manipulation strategies through repeated interaction with simulated or physical environments. Instead of relying entirely on explicit physical models, the system learns policies that maximize task success. This capability is especially valuable when dealing with uncertain contact conditions and complex object geometries.

Large-scale simulation has accelerated progress in learning-based manipulation. Physics simulators allow robots to practice millions of rolling and pivoting episodes across diverse object categories. Domain randomization techniques expose policies to variations in mass, friction, geometry, and sensor noise. As a result, learned policies become more robust and transferable to real-world hardware.

Imitation learning provides another promising pathway. Human demonstrations captured through teleoperation systems or motion capture devices can reveal effective rolling and pivoting strategies. Learning algorithms extract patterns from these demonstrations and transform them into robotic control policies. Because humans naturally exploit contact mechanics efficiently, demonstration-based learning often accelerates skill acquisition.

The effectiveness of rolling and pivoting depends heavily on hand design. Fingertip geometry influences contact stability and motion controllability. Rounded fingertips generally support smoother rolling interactions, while compliant fingertips improve adaptability to object variations. Soft robotic fingers can conform to object surfaces, increasing contact area and enhancing manipulation robustness. Conversely, rigid fingertips may provide greater positional precision for certain tasks.

Actuation systems also affect manipulation performance. Fully actuated fingers provide independent control over each joint, enabling precise motion generation. Underactuated fingers rely more heavily on passive adaptation mechanisms but may simplify mechanical design. Tendon-driven hands, direct-drive systems, and variable stiffness actuators each offer different advantages for rolling and pivoting applications.

Force and torque sensing are essential for reliable execution. Fingertip force sensors, wrist force-torque sensors, tendon tension measurements, and motor current monitoring all contribute to contact state estimation. Accurate force information allows the controller to maintain stable contact conditions while enabling desired object motion. Without adequate force sensing, rolling and pivoting become significantly more difficult to control.

Benchmarking rolling and pivoting performance requires specialized evaluation metrics. Researchers commonly measure object orientation accuracy, position accuracy, manipulation speed, force efficiency, contact stability, and success rate. Additional metrics may quantify slip occurrence, energy consumption, robustness against disturbances, and adaptability to novel objects. Standardized benchmark tasks facilitate comparison between different robotic hands and control algorithms.

Industrial applications of rolling and pivoting continue to expand. Electronics assembly frequently requires component orientation before insertion. Automotive manufacturing involves manipulating connectors, fasteners, and tools. Aerospace assembly demands precise handling of complex parts. Warehouse automation benefits from object reorientation during sorting and packaging operations. In each of these domains, rolling and pivoting reduce reliance on specialized fixtures and increase system flexibility.

Medical robotics represents another important application area. Surgical instruments often require precise orientation adjustments during procedures. Catheter manipulation, needle insertion, endoscopic tool positioning, and microsurgical operations all involve controlled rolling and pivoting motions. The ability to perform such manipulations safely and accurately can significantly enhance robotic surgical capabilities.

Humanoid robots rely heavily on rolling and pivoting for everyday object interactions. Human environments are designed around the dexterity of human hands, making these manipulation strategies essential for effective operation. Opening containers, operating switches, using tools, handling household objects, and interacting with consumer electronics all involve rolling and pivoting behaviors. Consequently, these manipulation primitives are fundamental building blocks for achieving human-level dexterity.

Despite substantial progress, numerous research challenges remain. Contact modeling inaccuracies, sensor uncertainty, dynamic disturbances, computational complexity, and limited generalization continue to restrict performance. Future developments are expected to integrate tactile intelligence, foundation models, world models, multimodal perception, and large-scale learning frameworks to address these limitations.

Ultimately, rolling and pivoting represent two of the most important contact-based manipulation strategies in dexterous robotics. They enable continuous object reconfiguration without requiring complete grasp release, thereby increasing efficiency, flexibility, and task capability. By exploiting contact mechanics intelligently, robotic systems can perform sophisticated manipulation tasks that more closely resemble human hand behavior. As robotic hardware, sensing technologies, and artificial intelligence continue to advance, rolling and pivoting will remain foundational elements of next-generation in-hand manipulation systems and a critical pathway toward achieving truly human-like robotic dexterity.

10.04 Regrasping Strategies

Regrasping is one of the most important manipulation capabilities in dexterous robotics and serves as a critical bridge between grasp acquisition and advanced in-hand manipulation. While a robot may initially establish a stable grasp on an object, that initial grasp is often not suitable for completing the final task. Objects frequently need to be reoriented, repositioned, or transferred into more advantageous configurations before assembly, insertion, tool use, packaging, or interaction with the environment can occur. Regrasping strategies provide systematic methods for changing the contact configuration between the robotic hand and the object in order to achieve these objectives. Unlike simple grasp maintenance, regrasping intentionally modifies the grasp state while preserving control over the object. It is therefore considered one of the foundational technologies underlying dexterous manipulation, in-hand manipulation, and autonomous robotic task execution. Within the framework of robot hands and dexterous grasping, regrasping occupies a central position because it enables robots to overcome limitations imposed by fixed grasp configurations and restricted finger workspaces.

Human beings perform regrasping continuously and often unconsciously. When a person picks up a screwdriver from a table, the initial grasp is ...