Tactile Sensing Utilizing Human Tactile Perception
Yoshihiro Tanaka Yoshihiro Horita Akihito Sano Hideo Fujimoto
Graduate School of Engineering, Nagoya Institute of Technology
Human tactile perception incorporates two important characteris-tics self-reference and bidirectionality. Self-reference is an im-portant factor when evaluating tactile sensations. Bidirectionalitycontributes to sensing capability enhancement. In this paper, a tac-tile sensor including self-reference and bidirectionality is presented.The sensor with two microphones is mounted on a human finger.Users can apply the sensor while retaining their normal tactile per-ceptions and simultaneously obtaining sound and skin deformationdata based on the mechanical interaction between the finger and theobject. Experimental results on roughness evaluations and convex-ity detections show the validity of the proposed sensing method andits potential for use in various applications.
Keywords: Tactile sensor, self-reference, bidirectionality, humantactile perception
Index Terms: H.1.2 [Models and Principles]: User/MachineSystemsHuman Factors; H.5.2 [Information Interfaces and Pre-sentation]: User InterfacesHaptic I/O
Works by human hands are still necessary in a number of manufac-turing processes developed in todays digital era, and tactile percep-tions continue to play an important role in such work. For example,high quality surfaces have been, and continue to be, demanded fora variety of products, and such surfaces must be inspected to de-tect small surface irregularities and/or to evaluate roughness. Inthe past, this inspection process has most often been performed byexperienced craft workers using their well-trained sense of touch.However, such manual inspection tasks require high skill levels andlong years of experience. They also depend on the subjective judg-ment of the evaluators. Accordingly, an ongoing need exists forefficient and objective inspection technologies that could supple-ment or replace subject judgments. Currently, laser displacementsensors, dial gages, and image processing technologies are gener-ally used for surface profiles measurements. However, it is difficultto introduce such tools to many types of inspection work becausethey are generally inefficient and insufficiently robust. Laser dis-placement sensors and dial gages are unsuitable for scanning wideareas in a short period of time and require time to set up and oper-ate, while image processing is sensitive to surface conditions. Fur-thermore, the objects to be inspected are often small and/or havecurvatures, and there are few generally available devices that aresuitable for evaluating such curved surfaces.
Recently, however, numerous tactile sensors have been devel-oped . For example, a force sensor developed by Saga et al. can measure force applied over a large area with high spatial res-olution and high sensitivity. A slip detection sensor created by
Teshigawara et al.  can be mounted on a robot hand and helpthe robot to grasp an object efficiently. These tactile sensors fo-cus on detecting tactile stimuli applied to the robot or other device.Alternatively, tactile sensors that can be used for evaluating objectproperties have also been developed. Tanaka et al.  developeda polyvinylidene fluoride (PVDF) sensor capable of measuring thetactile sensations of fabrics. The sensor is mounted on a robot fin-ger and sensing is performed by scanning the robot hand across theobject. Zbyszewski et al.  developed a tactile sensor for medicalapplications that utilizes an air cushion and is capable of measuringthe stiffness of body tissues. This paper focuses primarily on tactilesensing used for evaluation.
To measure properties such as roughness, tactile sensors gen-erally require limiting usage conditions for accurate sensing. How-ever, objects often incorporate a variety of factors such as curved ar-eas, small parts, and soft surfaces, which makes it difficult to allowtactile sensors to perform their designed functions. Hand-held orfinger-mounted tactile sensors  can be more easily applied tovarious objects since they are moved by a human being. However,it is difficult to ensure sufficient stability for accurate sensing andthe sensitivity levels decrease if they are not manipulated correctly.Numerous efforts aimed at developing tactile sensors have encoun-tered similar problems between application limitations (strict usageconditions) and ensuring proper sensitivity levels.
Next, let us consider tactile displays (e.g. ) and tactile enhanc-ing devices (e.g. ). Sensors of this type generally do not have theabove-mentioned problems. They do not meet strict usage condi-tions. Users can easily perceive the tactile sensations developers an-ticipate (realistic reproduction is another matter). Accordingly, tac-tile displays and tactile enhancing devices appear to have propertiesthat many tactile sensors lack. We next took note of an importantcharacteristic of human tactile perception, which is active sensing.Human active sensing includes the characteristics of self-referenceand bidirectionality, and it is expected that a tactile sensor that iscapable of including human active sensing characteristics could beapplied with high sensitivity under relaxed conditions.
The purpose of this paper is the development of a tactile sen-sor that includes human active sensing characteristics. First, anoverview of the relevant human active sensing characteristics ispresented and discussed. Following which, the concept of a tac-tile sensing (including human active sensing characteristics) is pro-posed and prototypes of the tactile sensor are shown. Two variantsof a prototype tactile sensor, designed to measure surface properties(roughness and irregularity) have been developed. Experimental re-sults show the potential of the proposed tactile sensing method andconfirm the validity of the developed tactile sensors. Furthermore,the multiplied effects of the proposed tactile sensor are presentedbased on the results of our experiments.
2 HUMAN ACTIVE SENSING
We will focus on two characteristics of human active sensing.
1. Self-reference: our tactile sensations are based on deforma-tions to our skin resulting from mechanical interactions be-tween the skin and the object being evaluated/manipulated.
2. Bidirectionality: our tactile perceptions and sensing motionscannot be separated. They are bi-directionally related.
IEEE World Haptics Conference 201121-24 June, Istanbul, Turkey978-1-4577-0298-3/11/$26.00 2011 IEEE
Self-reference refers to the point that the tactile stimuli provided byobjects will be experienced differently by different persons. Thisis because the mechanical properties of our fingers (such as skinstiffness, fingernail length, among other factors) differ between in-dividuals. Motions related to tactile sensing also differ betweenindividuals. Therefore, our tactile sensations depend on ourselvesas well as the object. This is an important factor to consider whenevaluating tactile sensations and understanding the importance ofan experienced workers well-trained sense of touch. Tactile dis-plays and tactile enhancing devices also provide this characteristicsince the tactile stimuli are transmitted to the users.
Bidirectionality refers to the fact that we adjust our sensing mo-tions to conform to the objects and the tactile sensations. Thus,we can perceive tactile sensations best, and it appears that our tac-tile sensing abilities are optimized by this characteristic. Therefore,when tactile displays and tactile enhancing devices are used, userscan consciously adjust the sensing conditions (such as pressure ex-erted, scanning velocity and other factors) in order to optimize theirperception of tactile sensations. This factor strongly contributesto ensuring adequate performance without imposing limiting con-ditions. Furthermore, bidirectionality is useful when performingdifficult evaluations such as attempting to detect very small differ-ences and large dynamic ranges, and well-trained workers utilizethis characteristic.
3 PROPOSED TACTILE SENSOR
Considering the above-mentioned characteristics, it is expected thata tactile sensor system that includes self-reference and bidirection-ality could achieve a high level of sensitivity without requiring strictusage conditions, and could be used to evaluate tactile sensations atthe individual level. Accordingly, we propose a sensor system thatemploys a users ability to detect his or her own skin deformationsor related information by perceiving tactile sensations. Using thissensor system, we can obtain information on tactile stimuli appliedto ourselves, while also utilizing bidirectionality when performingtactile sensing evaluations.
Ideas of using a human finger as a part of tactile sensors havebeen investigated. Mascaro et al  proposed the nail-mountedtactile sensor capable of measuring the force applied to a finger padby detecting changes to fingernail color. Makino et al.  pro-posed the life log system utilizing a nail-mounted tactile sensor,in which a piezo attached to the nail detects vibrations transmit-ted from the finger pad. They showed that touch motions used inthe life can be estimated from sensor output signals. Iwamoto andShinoda  proposed the finger ring tactile interface, which mea-sures several DOF vibrations propagating along the finger by us-ing accelerometers on the backside of the proximal phalanx. Theyshowed that tapping on distal phalanx and middle phalanx can bediscriminated from sensor output signals. These sensors have thepotential to utilize self-reference and bidirectionality. However,they were not developed from the standpoint of using those charac-teristics, and have not been applied to tactile evaluations of objectproperties. The sensor developed by Mascaro et al. is unsuitable forevaluating tactile stimuli such as roughness (vibration). It seemsthat it is difficult for the sensors developed by Makino et al. andIwamoto and Shinoda to provide tactile information closer to thatreceived by the finger pad because stimuli are obtained in a locationaway from the finger pad (fingernail and backside of the proximalphalanx).
Previous robotic active tactile sensors  have usedscanning motions, but the relation between the motion and the ob-tained information is unidirectional (from the motion to the tac-tile information), not bidirectional. Tactile sensing by robots can
(b) Noise cancellation function type (c) Output center function type
(d) Sensing condition
Figure 1: Sensor photographs
utilize motion control (such as when under a constant load), butthe motion control provided is still grossly inferior to what canbe achieved with human motion control based on bidirectionality.Previously developed hand-held and finger-mounted sensors some-times include bidirectionality in a different way, usually with thetactile information feedback replaced by visual or auditory infor-mation presented through a monitor or speaker. However, a replace-ment for the bidirectionality control has not yet been discussed anda better understanding of the characteristics of human bidirectional-ity is needed if we are to integrate that factor into tactile sensors orother applications. Viewed from these points, our proposed tactilesensing system can be seen as an effective first step.
4 TACTILE SENSOR PROTOTYPE
Based on the proposed tactile human sensing system, we have de-veloped finger-mounted sensors that use microphones. Figure 1shows the developed sensors. The sensors do not interfere withcontact between the finger pad and the object. The microphone thatperforms the sensing faces, and is in contact with, the users skinclose to the finger pad, which will come into contact with an ob-ject. Sound and skin deformation in the microphone contact areaare then detected. The generated sound has a strong relation withthe skin deformation because it is generated by mechanical interac-tion between the skin and the object . The microphones used(Star Micronics Co., Ltd., MAA-03A-L30) are very small (diam-eter: 3 [mm], thickness: 1.5 [mm],) and light (weight: 0.2 [g]).The sensor, which is mounted on the fingertip in front of the distalinterphalangeal (DIP) joint, does not distract users with annoyingsensations. When the finger equipped with the sensor is moved, thecontact location of the microphone does not change. Furthermore,the curvature of the sensor can be adjusted to each user.
Figure 2: Measurement system
0 2 4
Figure 3: Typical sensor output signals
We developed two sensor prototypes. One has a noise cancella-tion function and utilizes two microphones. One of the two micro-phones is in contact with the users skin and provides the sensingelement while the other microphone is mounted on the back of thesensing element microphone (facing away from the skin) and mea-sures ambient sound (noise). By subtracting the outputs of the sec-ond microphone from the first microphone, noise can be canceled.
The other sensor prototype has an output center function. Thissensor also employs two microphones, both of which are used assensing elements. One measures information generated on the rightside of the finger while the other measures information obtainedfrom the left side. By using the outputs from both microphones, anoutput center can be estimated. In our future work, the two newlydeveloped sensor types will be integrated.
The measurement system is shown in Fig. 2. Output signalsfrom the microphones are obtained as voltage using a sampling fre-quency of 40 [kHz] and then processed in real time using a personalcomputer. It is also possible to reproduce the processed output sig-nals using a speaker. In this study, however, the sound reproductionfunction was not addressed since the effects resulting from soundfeedback were not under investigation. In our future work, we in-tend to investigate the effects resulting from sound feedback.
4.3 Sensor Output Signal
Figure 3 shows a section of typical sensor output obtained by a testparticipant sensing the roughness of a sandpaper sample while us-ing the sensor with the noise cancellation function. Here it can beseen that the amplitude of the sensor output signals is large duringsense data gathering of the object, and that the output signals con-sist of vibrations of various frequencies. Furthermore, it can be seenthat sharp sudden pulses sometimes occurred in the output at thestart of the sensing stroke. Such pulses might be caused by stick-slip skin phenomenon. These outputs indicate that the sensor candetect various information and phen...