Vincent Lévesque

Display of Virtual Braille Dots by Lateral Skin Deformation: Feasibility Study

table of contents

2. Virtual Braille Display

2.1 Device

The VBD device consisted of a tactile display mounted on a laterally-moving frictionless slider (see Figure 2) and interfacing control electronics.

Figure 2a: Picture of the VBD's tactile display showing its linear array of 12 actuators.
(a)		 Figure 2b: Picture of the VBD apparatus. The tactile display is mounted on an horizontal slider. An horizontal bar connects the display to a rotary encoder.
(b)
Figure 2: VBD device: (a) STReSS-type tactile display, and (b) display mounted on a slider with rotary encoder.

Reading virtual Braille was done by applying the tip of the index finger against the tactile display and sliding it laterally, as shown in Figure 3. When activated, the tactile display caused lateral deformation to the fingertip skin, that could be varied in response to slider movement. The finger remained in contact with the display and dragged it along the reading surface. Although this principle could allow reading with multiple fingers, the width of the display limited the reader to the use of a single finger.

Figure 3a: Illustration of the strain applied by the VBD. This diagram shows a fingerpad pressed against the tactile display with the finger orthogonal to the linear slider on which the display rests. As the display moves horizontally on the slider, the tip of the tactile display's actuators move sideways along the same axis, stretching or compressing the skin.
(a)		 Figure 3b: Illustration of the contact between a fingerpad and the tactile display. The fingerpad is pressed against the far edge of the tactile display's actuators, near the back, where the actuactors have been cut so as to form a narrow contact line.
(b)		 Figure 3c: Picture of a fingerpad in contact with the tactile display.
(c)
Figure 3: Interaction with the VBD: (a) strain applied during exploration, (b) illustration and (c) picture of finger contact with the VBD.

2.1.1 Tactile Display

The tactile display was made of a stack of twelve 0.38-mm-thick piezoelectric bender plates (Y-poled, 31.8 mm x 12.7 mm, High Performance Bending Motors from Piezo Systems Inc., part number T215-H4CL-303Y), sandwiched at their base between neoprene spacers and clamped between two rigid end-plates using four locating pins and four screws (see Figures 4(a) and 4(b)). The spacers were cut in a 12-mm-high T-shape so that they rested on the locating pins and allowed room for electrical connections (see Figure 4(b)). Once tightly secured, the spatial period ε, or contactor pitch, was approximately 0.7 mm. This assembly method was selected for the convenience of allowing the design parameters such as thickness, length, shape and material of the actuators and spacers to be changed. In the present study, however, only one configuration was used.

The actuators were driven by a bipolar voltage applied between their central electrode and their two electrically-connected external electrodes. Because of the small space between adjacent plates, the electrodes could not be connected using the methods recommended by the motor supplier. Therefore, the actuators were prepared as shown in Figure 4(c). The external electrodes were joined with adhesive electrically conductive tape (3M Corporation, EMI Copper Foil Shielding Tape 1181) running over non-conductive tape on the sides to prevent shorting with the central electrode. One corner of an external electrode was soldered to a ground wire. The other corner was turned into a small electrode pad (isolated from the rest by grinding off the conductive layer) and connected to the central electrode with conductive paint. A wire used for the control voltage was then soldered to this pad. To prevent contact with the adjacent actuator, the solders were kept significantly thinner than the spacers (0.5 mm) and were protected with non-conductive tape. Traces of conductive paint were applied along the length of the electrodes to improve their reliability (not shown). The actuators were then isolated from the metallic locating pins using non-conductive tape. As illustrated in Figure 3(b), the top corners on one side of the blades were beveled to create a narrow linear array of skin contactors (around 0.2 mm2 in area each). Finally, the tips of the actuators were coated with varnish to isolate them from the skin.

Figure 4a: Illustration of the assembly of the VBD's tactile display (see text).
(a)
Figure 4b: Illustration of the assembly of the VBD's tactile display (see text).
(b) 		Figure 4c: Illustration of the fabrication of the tactile display's actuators (see text).
(c)
Figure 4: Assembly of the VBD's tactile display: (a) perspective and (b) frontal views of stacked assembly, and (c) actuator fabrication process.

The display could be used by applying the finger either on the large horizontal contact surface or against the surface formed of the beveled corners of the contactors. The latter surface, as shown in Figure 3, provided a narrower contact area more appropriate for the display of dots and was the only one used in this study.

The deflection of the actuator tips was estimated with the help of a camera. Two sample measurements are shown in Figure 5. However, when not loaded by the finger, the deflection range was estimated to be 0.4 mm. As explained in the next section, the motion was limited in practice to a restricted range of approximately 0.3 mm. The deflection when loaded with the fingerpad could not be quantified but appears to be significantly lower than when unloaded.

Figure 5a (top): Picture showing the tip of 5 actuators, with the middle actuator leaning towards the left.
Figure 5a (bottom): The middle actuator is leaning towards the right.
(a)		 Figure 5b (top): Same as Figure 5a but with a smaller displacement.
Figure 5b (top): Same as Figure 5a but a with smaller displacement.
(b)
Figure 5: Visual estimation of unloaded actuator deflection for (a) full range and (b) restricted range. Adjacent actuators were held deflected away from the actuator under study.

2.1.2 Control System

The position of the linear slider on which the tactile display was mounted was measured by an optical encoder with a nominal resolution of 17 μm. Interfacing electronics were constructed to permit the refresh of the actuators at 500 Hz according to patterns programmed on a personal computer. This enabled us to program the deflection of each actuator with arbitrary functions of space (see Section 2.2).

The interfacing electronics, adapted from a previous project, made use of a Field-Programmable Gate Array (FPGA) development board (Constellation-10KE(TM) from Nova Engineering Inc. operating an Altera FLEX 10KE(TM) chip.) with a Universal Serial Bus (USB) 1.1 interface. It was programmed to convert control frames coming from the computer, or "tactile images", into twelve bender voltages by means of 156-kHz pulse-width modulation (PWM). The same, however, could be accomplished by adopting a variety of other approaches, including the use of micro-controllers or dedicated logic, interfaced to the computer via parallel I/O or high-speed serial I/O.

The computer generated a set of 8-bit actuator control values based on the encoder readout every 2 ms on average. These tactile images were sent to the FPGA by packets of 5 through the USB channel where a FIFO buffer regulated the flow of tactile images to ensure a constant output rate. The logic-level signals were then amplified to a ±40 V range and low-passed by the circuit shown in Figure 6. In order to avoid non-linearities in the signal amplification at extreme PWM duty cycles, the control values were restricted to the range 10 (0x0A) to 250 (0xFA).

Figure 6: Electronic diagram of amplification circuit and low-pass filter.
Figure 6: Electronic circuits: amplification circuit (left) and low-pass filter (right).

2.2 Skin Deformation Patterns

Trial and error led us to select a pattern solely on the basis of the resemblance of the sensation it provided compared to that of actual Braille, as felt by the reference subject. We are however unable to offer a principled explanation as to why this particular pattern creates sensations that resemble Braille dots more than others. The determination of the actual parameters is described in Section 3.

The deflection δi of each actuator i was a function of the slider position xs obtained from the encoder. The actuators followed the same deflection function δ(x), where x was the actuator position along the reading surface, as illustrated in Figure 7. The physical configuration of actuators introduced a spatial phase difference of ε. The first actuator was given a position corresponding to the slider position.

Equation 1: delta sub i of x sub s is equal to delta of x sub s + i times epsilon, where i = 0 to 11. (1)

What we selected was a pattern (Footnote: Movies of the VBD in action can be found online on the Haptic Laboratory's VBD web page.) such that the deflection of each actuator swept the first half-cycle of a sinusoid, starting from the left position, as it scanned a virtual dot, as shown in Figure 7. A small-amplitude, high-frequency sinusoid could also be added to the nominal waveform to enhance contrast. These representations were termed nominal and textured.

Figure 7: Illustration of the deflection of an actuator as it traverses nominal or texture dot (see text).
Figure 7: Actuator deflection as a function of position. The curve shows the actuator deflection function with respect to actuator position for a nominal dot (left) and a textured dot (right). The deflection of actuators is illustrated at discrete points along the virtual reading surface. Texture was always applied either to all dots or none.

These patterns were found to better approximate the sensation of scanning over Braille dots than others that were experimented with, such as triangular or square waves, full-cycle sinusoids, or textured blanks.

The spatial phase difference between actuators resulted in the representation of dots as a traveling wave. Figure 8 and 9 illustrate the movement of actuators as a virtual dot traverses the length of the display. Moving the slider in one direction across a region containing a dot resulted in a wave of actuator deflections traveling at the same speed in the opposite direction on the tactile display, causing the illusion that the reading finger was scanning over stationary Braille dots. Since the deflection function was independent of direction, it caused actuator deflections in the direction of finger movement when reading from left to right, but opposing movement when reading from right to left. The resulting sensations, however, seemed to be similar.

Figure 8a: Illustration of a stimulus that moves from right to left along a fingerpad.
(a)
Figure 8b: Illustration of a the actuator deflection pattern that produces the same sensation (see text).
(b)
Figure 8c: Picture of a the actuator deflection pattern that produces the same sensation.

(c)
Figure 8: Traveling wave representation of a Braille dot at four points in time: (a) finger-dot interaction, (b) depiction of the actuator deflection and corresponding deflection function, and (c) picture of the actuator deflection. The dot center is indicated by arrows.

Figure 9: Illustration of the travelling wave of skin strain that produces the sensation of a sliding dot (see text).
Figure 9: Traveling wave corresponding to a single dot as the slider passes over it. Top-view of actuator deflections and corresponding finger-dot interactions are shown at four locations. Each of the twelve curves indicates the deflection pattern followed by an actuator as the slider moves from left to right.

This pattern had two distinct effects on the skin deformation. The first was to cause a net displacement of a skin region around each contactor. The second was a pattern of compression and expansion of each small region of skin located between two contactors. Patterns of expansion and compression can actually be observed when a finger scans over small shapes [Levesque and Hayward 2003]. The strain variations Δi caused by a pair of actuators is represented in Figure 10. The width ω of a virtual dot is shown relative to the spatial period ε. If ω<ε, then there was no overlap between the deflections of adjacent actuators. If ω>ε, then an overlap existed and there was a continuous transition from expansion to compression. If ω>2ε expansion and compression never reached their maximum values. It is not known whether local displacement or local variations in strain, or both forms of stimulation, caused the illusion of the dot moving under the finger.

Figure 10: Illustration of the displacement of two adjacent actuators and resulting resulting skin strain pattern as a dot is traversed (see text).
Figure 10: Displacement of two consecutive actuators and corresponding skin strain patterns as functions of slider position, for width ω of virtual Braille dots smaller or greater than spatial period ε.

The tactile display could only display a single row of Braille dots. From the Braille character set, the three characters that have dots in row 1 only, or a total absence of dots, could be displayed: 'a', 'c', and ' ', see Figure 11. The fourth possible combination, unused in Braille, was called 'dot #4'.

Figure 11: Illustration of the four Braille characters displayed by the VBD.
Figure 11: Braille characters displayed by the VBD.

continue reading: 3. Parameter Tuning