A system providing various improved calibration techniques for haptic feedback is described. An acoustic field is defined by one or more control points in a space within which the acoustic field may exist. Each control point is assigned an amplitude value equating to a desired amplitude of the acoustic field at the control point. Because complete control of space is not possible, controlling the acoustic field at given points yields erroneous local maxima in the acoustic field levels at other related positions. In relation to mid-air haptic feedback, these can interfere in interactions with the space by creating secondary effects and ghost phenomena that can be felt outside the interaction area. The level and nature of the secondary maxima in the acoustic field is determined by how the space is controlled. By arranging the transducer elements in different ways, unwanted effects on the acoustic field can be limited and controlled.
The present invention concerns a method and apparatus for the modulation of an acoustic field for providing tactile sensations. A method of creating haptic feedback using ultrasound is provided. The method comprises the steps of generating a plurality of ultrasound waves with a common focal point using a phased array of ultrasound transducers, the common focal point being a haptic feedback point, and modulating the generation of the ultrasound waves using a waveform selected to produce little or no audible sound at the haptic feedback point.
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06N 10/20 - Models of quantum computing, e.g. quantum circuits or universal quantum computers
G06N 10/80 - Quantum programming, e.g. interfaces, languages or software-development kits for creating or handling programs capable of running on quantum computersPlatforms for simulating or accessing quantum computers, e.g. cloud-based quantum computing
G08B 6/00 - Tactile signalling systems, e.g. personal calling systems
An acoustic field may be produced from a transducer array having known relative positions and orientations In this acoustic field, one or more control points may be defined. An amplitude may be assigned to the control point. Mid-air haptic effect for a virtual object on a human body part may be generated by moving the control point in a single closed curve comprising a plurality of curve segments. The single closed curve traverses at least one location where the human body part intersects with the virtual object. Additionally, a user may interact with virtual three-dimensional content using the user's hands while a tracking system monitoring the user's hands, a physics engine updates the properties of the virtual three-dimensional content and a haptic feedback system provides haptic information to the user.
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/041 - Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
G06F 3/04815 - Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
G06F 3/0488 - Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
Described herein are techniques for tracking objects (including human body parts such as a hand), namely: 1) two-state transducer interpolation in acoustic phased-arrays; 2) modulation techniques in acoustic phased-arrays; 3) fast acoustic full matrix capture during haptic effects; 4) time-of-flight depth sensor fusion system; 5) phase modulated spherical wave-fronts in acoustic phased-arrays; 6) long wavelength phase modulation of acoustic field for location and tracking; and 7) camera calibration through ultrasonic range sensing.
G01S 15/32 - Systems for measuring distance only using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated
Strategies for managing an “always on” solution for volumes with enhanced interactive haptic feedback and its implications are addressed. Ultrasound transducer arrays may be mounted on a person (such as on a head mounted display or other wearable accessory). This array may utilize some form of 6 degree-of-freedom tracking for both the body and hands of the user. The arrays coordinate to project focused acoustic pressure at specific locations on moving hands such that a touch sensation is simulated. Using wearable microphones, the ultrasonic signal reflected and transmitted into the body can be used for hand and gesture tracking.
G06F 3/04883 - Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
Creating near-instantaneous effects in a phased-array system may occur by splitting the update process of the array state into parts that depend on different update rates. Alternatively, leveraging the uncertainty of physical properties of focusing in a phased-array system may improve the intersection between the body part and the control point. Specifically, by focusing behind or in front of the intended region or at a position with a calculated geometric relationship to the intended interaction region, a larger volume (region) of space is addressed that more certainly contains the body part participating in the interaction. This larger volume is then subjected to the ultrasonic radiative energy flux that encodes the properties desired for the interaction point, which may include haptic and/or audio points.
An acoustic field may be produced from a transducer array having known relative positions and orientations In this acoustic field, one or more control points may be defined. An amplitude may be assigned to the control point. Mid-air haptic effect for a virtual object on a human body part may be generated by moving the control point in a single closed curve comprising a plurality of curve segments. The single closed curve traverses at least one location where the human body part intersects with the virtual object. Additionally, a user may interact with virtual three-dimensional content using the user's hands while a tracking system monitoring the user's hands, a physics engine updates the properties of the virtual three-dimensional content and a haptic feedback system provides haptic information to the user.
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/041 - Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
G06F 3/04815 - Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
G06F 3/0488 - Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
Various techniques for driving phased array systems are described, specifically intended for acoustic phased arrays with applications to mid-air haptics, parametric audio, acoustic levitation and acoustic imaging, including a system: 1) that is capable of mitigating the effect of the changes in the air to provide a consistent haptic experience; 2) that produces trap points in air; 3) that defines phased-array optimization in terms of vectors for the production of more consistent haptic effects; 4) that defines one or more control points or regions in space via a controlled acoustic field; 5) that uses a reduced representation method for the construction of acoustic basis functions; 6) that performs efficient evaluation of complex-valued functions for a large quantity of throughput; 7) that generates a Krylov sub-space of a matrix; and 8) that maximizes an objective described by different control points and/or regions to those used to create the acoustic basis functions.
G10K 11/34 - Sound-focusing or directing, e.g. scanning using electrical steering of transducer arrays, e.g. beam steering
A61B 8/00 - Diagnosis using ultrasonic, sonic or infrasonic waves
B06B 1/06 - Processes or apparatus for generating mechanical vibrations of infrasonic, sonic or ultrasonic frequency making use of electrical energy operating with piezoelectric effect or with electrostriction
G10K 11/00 - Methods or devices for transmitting, conducting or directing sound in generalMethods or devices for protecting against, or for damping, noise or other acoustic waves in general
G10K 11/36 - Devices for manipulating acoustic surface waves
Described is a method for instilling the haptic dimension of texture to virtual and holographic objects using mid-air ultrasonic technology. A set of features is extracted from imported images using their associated displacement maps. Textural qualities such as the micro and macro roughness are then computed and fed to a haptic mapping function together with information about the dynamic motion of the user's hands during holographic touch. Mid-air haptic textures are then synthesized and projected onto the user's bare hands. Further, mid-air haptic technology enables tactile exploration of virtual objects in digital environments. When a user's prior and current expectations and rendered tactile texture differ, user immersion can break. A study aims at mitigating this by integrating user expectations into the rendering algorithm of mid-air haptic textures and establishes a relationship between visual and mid-air haptic roughness.
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06T 7/42 - Analysis of texture based on statistical description of texture using transform domain methods
G06T 7/44 - Analysis of texture based on statistical description of texture using image operators, e.g. filters, edge density metrics or local histograms
G06T 7/45 - Analysis of texture based on statistical description of texture using co-occurrence matrix computation
10.
Blocking plate structure for improved acoustic transmission efficiency
An acoustic matching structure is used to increase the power radiated from a transducing element with a higher impedance into a surrounding acoustic medium with a lower acoustic impedance. The acoustic matching structure consists of a thin, substantially planar cavity bounded by a two end walls and a side wall. The end walls of the cavity are formed by a blocking plate wall and a transducing element wall separated by a short distance (less than one quarter of the wavelength of acoustic waves in the surrounding medium at the operating frequency). The end walls and side wall bound a cavity with diameter approximately equal to half of the wavelength of acoustic waves in the surrounding medium. In operation, a transducing element generates acoustic oscillations in the fluid in the cavity. The transducing element may be an actuator which generates motion of an end wall in a direction perpendicular to the plane of the cavity to excite acoustic oscillations in the fluid in the cavity, and the cavity geometry and resonant amplification increase the amplitude of the resulting pressure oscillation. The cavity side wall or end walls contain at least one aperture positioned away from the center of the cavity to allow pressure waves to propagate into the surrounding acoustic medium.
B06B 1/06 - Processes or apparatus for generating mechanical vibrations of infrasonic, sonic or ultrasonic frequency making use of electrical energy operating with piezoelectric effect or with electrostriction
H04R 1/28 - Transducer mountings or enclosures designed for specific frequency responseTransducer enclosures modified by provision of mechanical or acoustic impedances, e.g. resonator, damping means
The properties of metamaterials are derived both from the inherent properties of their constituent materials and from the geometrical arrangement of those materials. Metamaterials may be stacked or otherwise manipulated to transform substantially monochromatic signal into a second signal having a desired amplitude and phase. Metamaterials may be used with acoustic devices to create haptic feedback with desired properties or to transform the shape of certain devices. Metamaterials may be used in rotating devices with openings that transform a monochromatic signal into a non-monochromatic signal.
G08B 6/00 - Tactile signalling systems, e.g. personal calling systems
G10K 11/178 - Methods or devices for protecting against, or for damping, noise or other acoustic waves in general using interference effectsMasking sound by electro-acoustically regenerating the original acoustic waves in anti-phase
G10K 11/18 - Methods or devices for transmitting, conducting or directing sound
G10K 11/22 - Methods or devices for transmitting, conducting or directing sound for conducting sound through hollow pipes, e.g. speaking tubes
A system providing various improved processing techniques for haptic feedback is described. An acoustic field is defined by one or more control points in a space within which the acoustic field may exist. Each control point is assigned an amplitude value equating to a desired amplitude of the acoustic field at the control point. Transducers are then controlled to create an acoustic field exhibiting the desired amplitude at each of the control points. When human skin interacts with the acoustic field, vibrations of the skin are interpreted by mechanoreceptors being excited and sending signals to the brain via the nervous system. Improved processing techniques allow for more efficient real-world operation.
H04R 1/40 - Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers
13.
Using Iterative 3D-Model Fitting for Domain Adaptation of a Hand-Pose-Estimation Neural Network
Described is a solution for an unlabeled target domain dataset challenge using a domain adaptation technique to train a neural network using an iterative 3D model fitting algorithm to generate refined target domain labels. The neural network supports the convergence of the 3D model fitting algorithm and the 3D model fitting algorithm provides refined labels that are used for training of the neural network. During real-time inference, only the trained neural network is required. A convolutional neural network (CNN) is trained using labeled synthetic frames (source domain) with unlabeled real depth frames (target domain). The CNN initializes an offline iterative 3D model fitting algorithm capable of accurately labeling the hand pose in real depth frames. The labeled real depth frames are used to continue training the CNN thereby improving accuracy beyond that achievable by using only unlabeled real depth frames for domain adaptation.
G06V 10/764 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
G06V 10/82 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
An acoustic field may be produced from a transducer array having known relative positions and orientations In this acoustic field, one or more control points may be defined. An amplitude may be assigned to the control point. Mid-air haptic effect for a virtual object on a human body part may be generated by moving the control point in a single closed curve comprising a plurality of curve segments. The single closed curve traverses at least one location where the human body part intersects with the virtual object. Additionally, a user may interact with virtual three-dimensional content using the user's hands while a tracking system monitoring the user's hands, a physics engine updates the properties of the virtual three-dimensional content and a haptic feedback system provides haptic information to the user.
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/041 - Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
G06F 3/04815 - Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
G06F 3/0488 - Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
A system providing various improved calibration techniques for haptic feedback is described. An acoustic field is defined by one or more control points in a space within which the acoustic field may exist. Each control point is assigned an amplitude value equating to a desired amplitude of the acoustic field at the control point. Because complete control of space is not possible, controlling the acoustic field at given points yields erroneous local maxima in the acoustic field levels at other related positions. In relation to mid-air haptic feedback, these can interfere in interactions with the space by creating secondary effects and ghost phenomena that can be felt outside the interaction area. The level and nature of the secondary maxima in the acoustic field is determined by how the space is controlled. By arranging the transducer elements in different ways, unwanted effects on the acoustic field can be limited and controlled.
Producing multiple independent fields from many phased acoustic transducers represents a difficult computational problem. By first dividing up each field to its own group of transducers and then treating each group as an element with adjustable phase, one can minimize the field-to-field interference through a power iteration solution. These solutions can be further refined by including tracking information from users in the space and how they shadow or reflect the acoustic fields.
Disclosed are methods to manipulate a given parametrized haptic curve in order to yield a smooth phase function for each acoustic transducer which minimizes unwanted parametric audio. Further, the impulse response of a haptic system describes the behavior of the system over time and can be convolved with a given input to simulate a response to that input. To produce a specific response, a deconvolution with the impulse response is necessary to generate an input.
G08B 6/00 - Tactile signalling systems, e.g. personal calling systems
G10K 11/34 - Sound-focusing or directing, e.g. scanning using electrical steering of transducer arrays, e.g. beam steering
H04R 1/40 - Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers
09 - Scientific and electric apparatus and instruments
Goods & Services
Computer hardware, namely, electronic sensor devices,
cameras, projectors and microphones for optically sensing
and tracking motion, and for capturing gesture, facial, and
voice recognition; computer hardware and software systems
for optically sensing and tracking motion and responsively
operating computational and entertainment devices; computer
software for controlling, viewing, accessing, browsing and
utilizing global computer, communication and entertainment
networks; computer game software; computer software for
playing video games and operating sensor devices, computer
hardware, and computer peripherals; computer software for
controlling personal computing and entertainment devices.
19.
Method and apparatus for modulating haptic feedback
The present invention concerns a method and apparatus for the modulation of an acoustic field for providing tactile sensations. A method of creating haptic feedback using ultrasound is provided. The method comprises the steps of generating a plurality of ultrasound waves with a common focal point using a phased array of ultrasound transducers, the common focal point being a haptic feedback point, and modulating the generation of the ultrasound waves using a waveform selected to produce little or no audible sound at the haptic feedback point.
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06N 10/20 - Models of quantum computing, e.g. quantum circuits or universal quantum computers
G06N 10/80 - Quantum programming, e.g. interfaces, languages or software-development kits for creating or handling programs capable of running on quantum computersPlatforms for simulating or accessing quantum computers, e.g. cloud-based quantum computing
G08B 6/00 - Tactile signalling systems, e.g. personal calling systems
20.
Method and apparatus for producing an acoustic field
The present invention concerns a method and apparatus for the modulation of an acoustic field for providing tactile sensations. A method of creating haptic feedback using ultrasound is provided. The method comprises the steps of generating a plurality of ultrasound waves with a common focal point using a phased array of ultrasound transducers, the common focal point being a haptic feedback point, and modulating the generation of the ultrasound waves using a waveform selected to produce little or no audible sound at the haptic feedback point.
G01S 7/52 - Details of systems according to groups , , of systems according to group
G01S 7/539 - Details of systems according to groups , , of systems according to group using analysis of echo signal for target characterisationTarget signatureTarget cross-section
G01S 15/36 - Systems for measuring distance only using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated with phase comparison between the received signal and the contemporaneously transmitted signal
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/04815 - Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
H04S 7/00 - Indicating arrangementsControl arrangements, e.g. balance control
21.
Blocking plate structure for improved acoustic transmission efficiency
An acoustic matching structure is used to increase the power radiated from a transducing element with a higher impedance into a surrounding acoustic medium with a lower acoustic impedance. The acoustic matching structure consists of a thin, substantially planar cavity bounded by a two end walls and a side wall. The end walls of the cavity are formed by a blocking plate wall and a transducing element wall separated by a short distance (less than one quarter of the wavelength of acoustic waves in the surrounding medium at the operating frequency). The end walls and side wall bound a cavity with diameter approximately equal to half of the wavelength of acoustic waves in the surrounding medium. In operation, a transducing element generates acoustic oscillations in the fluid in the cavity. The transducing element may be an actuator which generates motion of an end wall in a direction perpendicular to the plane of the cavity to excite acoustic oscillations in the fluid in the cavity, and the cavity geometry and resonant amplification increase the amplitude of the resulting pressure oscillation. The cavity side wall or end walls contain at least one aperture positioned away from the center of the cavity to allow pressure waves to propagate into the surrounding acoustic medium.
H04R 1/28 - Transducer mountings or enclosures designed for specific frequency responseTransducer enclosures modified by provision of mechanical or acoustic impedances, e.g. resonator, damping means
B06B 1/06 - Processes or apparatus for generating mechanical vibrations of infrasonic, sonic or ultrasonic frequency making use of electrical energy operating with piezoelectric effect or with electrostriction
To resolve an issue related to the calibration of optical cameras in transducer-based mid-air haptic systems, the magnification of the motion induced on an optical camera by an acoustic field modulated at specific frequencies reveals very small temporal variations in video frames. This quantized distortion is used to compare different acoustic fields and to solve the calibration problem in an automatized manner. Further, mechanical resonators may be excited by ultrasound when it is modulated at the resonant frequency. When enough energy is transferred and when operating at the correct frequency, a user in contact with the device can feel vibration near areas of largest displacement. This effect can be exploited to create devices which can produce haptic feedback while not carrying a battery or exciter when in the presence of an ultrasonic source.
G10K 11/34 - Sound-focusing or directing, e.g. scanning using electrical steering of transducer arrays, e.g. beam steering
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/03 - Arrangements for converting the position or the displacement of a member into a coded form
G06F 3/0354 - Pointing devices displaced or positioned by the userAccessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
G06F 3/038 - Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
09 - Scientific and electric apparatus and instruments
Goods & Services
Computer hardware, namely, electronic sensor devices, cameras, projectors and microphones for optically sensing and tracking motion, and for capturing gesture, facial, and voice recognition; computer hardware and software systems downloadable comprised of hardware and software for optically sensing and tracking motion and responsively operating computational and entertainment devices; downloadable computer software for controlling, viewing, accessing, browsing and utilizing global computer, communication and entertainment networks; downloadable computer game software; downloadable computer software for playing video games and operating sensor devices, computer hardware, and computer peripherals; computer software for controlling personal computing and entertainment devices
24.
Method and apparatus for modulating haptic feedback
The present invention concerns a method and apparatus for the modulation of an acoustic field for providing tactile sensations. A method of creating haptic feedback using ultrasound is provided. The method comprises the steps of generating a plurality of ultrasound waves with a common focal point using a phased array of ultrasound transducers, the common focal point being a haptic feedback point, and modulating the generation of the ultrasound waves using a waveform selected to produce little or no audible sound at the haptic feedback point.
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G08B 6/00 - Tactile signalling systems, e.g. personal calling systems
G06N 10/80 - Quantum programming, e.g. interfaces, languages or software-development kits for creating or handling programs capable of running on quantum computersPlatforms for simulating or accessing quantum computers, e.g. cloud-based quantum computing
G06N 10/20 - Models of quantum computing, e.g. quantum circuits or universal quantum computers
25.
Method and apparatus for producing an acoustic field
The present invention concerns a method and apparatus for the modulation of an acoustic field for providing tactile sensations. A method of creating haptic feedback using ultrasound is provided. The method comprises the steps of generating a plurality of ultrasound waves with a common focal point using a phased array of ultrasound transducers, the common focal point being a haptic feedback point, and modulating the generation of the ultrasound waves using a waveform selected to produce little or no audible sound at the haptic feedback point.
G01S 7/52 - Details of systems according to groups , , of systems according to group
H04S 7/00 - Indicating arrangementsControl arrangements, e.g. balance control
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G01S 7/539 - Details of systems according to groups , , of systems according to group using analysis of echo signal for target characterisationTarget signatureTarget cross-section
G01S 15/36 - Systems for measuring distance only using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated with phase comparison between the received signal and the contemporaneously transmitted signal
Phased array systems rely on the production of an exact carrier frequency to function. Reconstructing digital signals by specified amplitude and phase is accomplished explicitly by inducing frequency shifts away from a base frequency implied by phase changes. Shifting the carrier frequency of a digitally controlled phased array while preserving the timing of the individual phase pulses enables more efficient driving of the phased array system when the phase of the drive signals change dynamically in time.
Described herein are techniques for tracking objects (including human body parts such as a hand), namely: 1) two-state transducer interpolation in acoustic phased-arrays; 2) modulation techniques in acoustic phased-arrays; 3) fast acoustic full matrix capture during haptic effects; 4) time-of-flight depth sensor fusion system; 5) phase modulated spherical wave-fronts in acoustic phased-arrays; 6) long wavelength phase modulation of acoustic field for location and tracking; and 7) camera calibration through ultrasonic range sensing.
G01S 15/32 - Systems for measuring distance only using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated
An acoustic field may be produced from a transducer array having known relative positions and orientations In this acoustic field, one or more control points may be defined. An amplitude may be assigned to the control point. Mid-air haptic effect for a virtual object on a human body part may be generated by moving the control point in a single closed curve comprising a plurality of curve segments. The single closed curve traverses at least one location where the human body part intersects with the virtual object. Additionally, a user may interact with virtual three-dimensional content using the user's hands while a tracking system monitoring the user's hands, a physics engine updates the properties of the virtual three-dimensional content and a haptic feedback system provides haptic information to the user.
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/0488 - Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
G06F 3/041 - Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
G06F 3/04815 - Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
A system providing various improved processing techniques for haptic feedback is described. An acoustic field is defined by one or more control points in a space within which the acoustic field may exist. Each control point is assigned an amplitude value equating to a desired amplitude of the acoustic field at the control point. Transducers are then controlled to create an acoustic field exhibiting the desired amplitude at each of the control points. When human skin interacts with the acoustic field, vibrations of the skin are interpreted by mechanoreceptors being excited and sending signals to the brain via the nervous system. Improved processing techniques allow for more efficient real-world operation.
G08B 6/00 - Tactile signalling systems, e.g. personal calling systems
H04R 1/40 - Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers
G06F 3/038 - Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
The present invention concerns a method and apparatus for the modulation of an acoustic field for providing tactile sensations. A method of creating haptic feedback using ultrasound is provided. The method comprises the steps of generating a plurality of ultrasound waves with a common focal point using a phased array of ultrasound transducers, the common focal point being a haptic feedback point, and modulating the generation of the ultrasound waves using a waveform selected to produce little or no audible sound at the haptic feedback point.
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06N 10/20 - Models of quantum computing, e.g. quantum circuits or universal quantum computers
G08B 6/00 - Tactile signalling systems, e.g. personal calling systems
G06N 10/80 - Quantum programming, e.g. interfaces, languages or software-development kits for creating or handling programs capable of running on quantum computersPlatforms for simulating or accessing quantum computers, e.g. cloud-based quantum computing
09 - Scientific and electric apparatus and instruments
Goods & Services
Optical apparatus and instruments; cameras; infrared
cameras; thermal imaging cameras; infrared camera equipment;
thermal and infrared camera systems; infrared thermography
cameras; infrared camera and optical apparatus for
transforming displays into three dimensions; parts and
fittings for all the aforesaid goods.
A phased array of ultrasonic transducers may create arbitrary fields that can be utilized to manipulate fluids. This includes the translation of drops on smooth surfaces as well speeding the evaporation of fluids on wetted hands. Proposed herein is the use airborne ultrasound focused to the surface of the hand. The risk is that coupling directly into the bulk of the hand may cause damage to the cellular material through heating, mechanical stress, or cavitation. Using a phased array, the focus may be moved around, thus preventing acoustic energy from lingering too long on one particular position of the hand. While some signaling may penetrate into the hand, most of the energy (99.9%) is reflected. Also disclosed are methods to couple just to the wetted surface of the hand.
An acoustic field may be produced from a transducer array having known relative positions and orientations. In this acoustic field, one or more control points may be defined. An amplitude may be assigned to the control point. Mid-air haptic effect for a virtual object on a human body part may be generated by moving the control point in a single closed curve comprising a plurality of curve segments. The single closed curve traverses at least one location where the human body part intersects with the virtual object. Additionally, a user may interact with virtual three-dimensional content using the user's hands while a tracking system monitoring the user's hands, a physics engine updates the properties of the virtual three-dimensional content and a haptic feedback system provides haptic information to the user.
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/0488 - Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
G06F 3/041 - Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
G06F 3/04815 - Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
Various techniques for driving phased array systems are described, specifically intended for acoustic phased arrays with applications to mid-air haptics, parametric audio, acoustic levitation and acoustic imaging, including a system: 1) that is capable of mitigating the effect of the changes in the air to provide a consistent haptic experience; 2) that produces trap points in air; 3) that defines phased-array optimization in terms of vectors for the production of more consistent haptic effects; 4) that defines one or more control points or regions in space via a controlled acoustic field; 5) that uses a reduced representation method for the construction of acoustic basis functions; 6) that performs efficient evaluation of complex-valued functions for a large quantity of throughput; 7) that generates a Krylov sub-space of a matrix; and 8) that maximizes an objective described by different control points and/or regions to those used to create the acoustic basis functions.
G10K 11/34 - Sound-focusing or directing, e.g. scanning using electrical steering of transducer arrays, e.g. beam steering
A61B 8/00 - Diagnosis using ultrasonic, sonic or infrasonic waves
B06B 1/06 - Processes or apparatus for generating mechanical vibrations of infrasonic, sonic or ultrasonic frequency making use of electrical energy operating with piezoelectric effect or with electrostriction
G10K 11/00 - Methods or devices for transmitting, conducting or directing sound in generalMethods or devices for protecting against, or for damping, noise or other acoustic waves in general
G10K 11/36 - Devices for manipulating acoustic surface waves
A system providing various improved perceptions techniques for haptic feedback above interactive surfaces that require no contact with either tools, attachments or the surface itself is described. A range of receptors in a perceiving member which is part of the human body is identified to create substantially uniformly perceivable feedback. A vibration frequency that is in the range of the receptors in the perceiving member is chosen and dynamically altered to create substantially uniformly perceivable feedback throughout the receiving member.
H04R 1/40 - Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G08B 6/00 - Tactile signalling systems, e.g. personal calling systems
37.
Blocking plate structure for improved acoustic transmission efficiency
An acoustic matching structure is used to increase the power radiated from a transducing element with a higher impedance into a surrounding acoustic medium with a lower acoustic impedance. The acoustic matching structure consists of a thin, substantially planar cavity bounded by a two end walls and a side wall. The end walls of the cavity are formed by a blocking plate wall and a transducing element wall separated by a short distance (less than one quarter of the wavelength of acoustic waves in the surrounding medium at the operating frequency). The end walls and side wall bound a cavity with diameter approximately equal to half of the wavelength of acoustic waves in the surrounding medium. In operation, a transducing element generates acoustic oscillations in the fluid in the cavity. The transducing element may be an actuator which generates motion of an end wall in a direction perpendicular to the plane of the cavity to excite acoustic oscillations in the fluid in the cavity, and the cavity geometry and resonant amplification increase the amplitude of the resulting pressure oscillation. The cavity side wall or end walls contain at least one aperture positioned away from the center of the cavity to allow pressure waves to propagate into the surrounding acoustic medium.
B06B 1/06 - Processes or apparatus for generating mechanical vibrations of infrasonic, sonic or ultrasonic frequency making use of electrical energy operating with piezoelectric effect or with electrostriction
H04R 1/28 - Transducer mountings or enclosures designed for specific frequency responseTransducer enclosures modified by provision of mechanical or acoustic impedances, e.g. resonator, damping means
A system providing various improved calibration techniques for haptic feedback is described. An acoustic field is defined by one or more control points in a space within which the acoustic field may exist. Each control point is assigned an amplitude value equating to a desired amplitude of the acoustic field at the control point. Because complete control of space is not possible, controlling the acoustic field at given points yields erroneous local maxima in the acoustic field levels at other related positions. In relation to mid-air haptic feedback, these can interfere in interactions with the space by creating secondary effects and ghost phenomena that can be felt outside the interaction area. The level and nature of the secondary maxima in the acoustic field is determined by how the space is controlled. By arranging the transducer elements in different ways, unwanted effects on the acoustic field can be limited and controlled.
09 - Scientific and electric apparatus and instruments
Goods & Services
Optical apparatus and instruments; cameras; infrared
cameras; thermal imaging cameras; infrared camera equipment;
thermal and infrared camera systems; infrared thermography
cameras; parts and fittings for all the aforesaid goods.
09 - Scientific and electric apparatus and instruments
Goods & Services
Electrical apparatus for providing a physical response to a virtual object, computer hardware and software systems for providing physical response to gestures in mid air; gesture recognition software; computer software to enable haptic feedback; computer hardware and software systems for enabling haptic feedback.
42.
Using iterative 3D-model fitting for domain adaptation of a hand-pose-estimation neural network
Described is a solution for an unlabeled target domain dataset challenge using a domain adaptation technique to train a neural network using an iterative 3D model fitting algorithm to generate refined target domain labels. The neural network supports the convergence of the 3D model fitting algorithm and the 3D model fitting algorithm provides refined labels that are used for training of the neural network. During real-time inference, only the trained neural network is required. A convolutional neural network (CNN) is trained using labeled synthetic frames (source domain) with unlabeled real depth frames (target domain). The CNN initializes an offline iterative 3D model fitting algorithm capable of accurately labeling the hand pose in real depth frames. The labeled real depth frames are used to continue training the CNN thereby improving accuracy beyond that achievable by using only unlabeled real depth frames for domain adaptation.
G06V 10/764 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
G06V 10/82 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Described is a solution for an unlabeled target domain dataset challenge using a domain adaptation technique to train a neural network using an iterative 3D model fitting algorithm to generate refined target domain labels. The neural network supports the convergence of the 3D model fitting algorithm and the 3D model fitting algorithm provides refined labels that are used for training of the neural network. During real-time inference, only the trained neural network is required. A convolutional neural network (CNN) is trained using labeled synthetic frames (source domain) with unlabeled real depth frames (target domain). The CNN initializes an offline iterative 3D model fitting algorithm capable of accurately labeling the hand pose in real depth frames. The labeled real depth frames are used to continue training the CNN thereby improving ccuracy beyond that achievable by using only unlabeled real depth frames for domain adaptation.
A system providing various improved processing techniques for haptic feedback is described. An acoustic field is defined by one or more control points in a space within which the acoustic field may exist. Each control point is assigned an amplitude value equating to a desired amplitude of the acoustic field at the control point. Transducers are then controlled to create an acoustic field exhibiting the desired amplitude at each of the control points. When human skin interacts with the acoustic field, vibrations of the skin are interpreted by mechanoreceptors being excited and sending signals to the brain via the nervous system. Improved processing techniques allow for more efficient real-world operation.
G08B 6/00 - Tactile signalling systems, e.g. personal calling systems
H04R 1/40 - Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers
G06F 3/038 - Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
Described is a method for instilling the haptic dimension of texture to virtual and holographic objects using mid-air ultrasonic technology. A set of features is extracted from imported images using their associated displacement maps. Textural qualities such as the micro and macro roughness are then computed and fed to a haptic mapping function together with information about the dynamic motion of the user's hands during holographic touch. Mid-air haptic textures are then synthesized and projected onto the user's bare hands. Further, mid-air haptic technology enables tactile exploration of virtual objects in digital environments. When a user's prior and current expectations and rendered tactile texture differ, user immersion can break. A study aims at mitigating this by integrating user expectations into the rendering algorithm of mid-air haptic textures and establishes a relationship between visual and mid-air haptic roughness.
Described is a method for instilling the haptic dimension of texture to virtual and holographic objects using mid-air ultrasonic technology. A set of features is extracted from imported images using their associated displacement maps. Textural qualities such as the micro and macro roughness are then computed and fed to a haptic mapping function together with information about the dynamic motion of the user's hands during holographic touch. Mid-air haptic textures are then synthesized and projected onto the user's bare hands. Further, mid-air haptic technology enables tactile exploration of virtual objects in digital environments. When a user's prior and current expectations and rendered tactile texture differ, user immersion can break. A study aims at mitigating this by integrating user expectations into the rendering algorithm of mid-air haptic textures and establishes a relationship between visual and mid-air haptic roughness.
09 - Scientific and electric apparatus and instruments
Goods & Services
Optical apparatus and instruments, namely, camera modules being integrated circuit modules for use with cameras, optical lenses; cameras; infrared cameras; infrared camera equipment, namely, camera modules being integrated circuit modules for use with cameras; thermal and infrared camera systems comprised of camera modules being integrated circuit modules for use with cameras, optical lenses; replacement parts and fittings for all the aforesaid goods
48.
Variable phase and frequency pulse-width modulation technique
Phased array systems rely on the production of an exact carrier frequency to function. Reconstructing digital signals by specified amplitude and phase is accomplished explicitly by inducing frequency shifts away from a base frequency implied by phase changes. Shifting the carrier frequency of a digitally controlled phased array while preserving the timing of the individual phase pulses enables more efficient driving of the phased array system when the phase of the drive signals change dynamically in time.
Phased array systems rely on the production of an exact carrier frequency to function. Reconstructing digital signals by specified amplitude and phase is accomplished explicitly by inducing frequency shifts away from a base frequency implied by phase changes. Shifting the carrier frequency of a digitally controlled phased array while preserving the timing of the individual phase pulses enables more efficient driving of the phased array system when the phase of the drive signals change dynamically in time.
An acoustic field may be produced from a transducer array having known relative positions and orientations In this acoustic field, one or more control points may be defined. An amplitude may be assigned to the control point. Mid-air haptic effect for a virtual object on a human body part may be generated by moving the control point in a single closed curve comprising a plurality of curve segments. The single closed curve traverses at least one location where the human body part intersects with the virtual object. Additionally, a user may interact with virtual three-dimensional content using the user's hands while a tracking system monitoring the user's hands, a physics engine updates the properties of the virtual three-dimensional content and a haptic feedback system provides haptic information to the user.
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/0488 - Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
G06F 3/041 - Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
G06F 3/0481 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
A phased array of ultrasonic transducers may create arbitrary fields that can be utilized to manipulate fluids. This includes the translation of drops on smooth surfaces as well speeding the evaporation of fluids on wetted hands. Proposed herein is the use airborne ultrasound focused to the surface of the hand. The risk is that coupling directly into the bulk of the hand may cause damage to the cellular material through heating, mechanical stress, or cavitation. Using a phased array, the focus may be moved around, thus preventing acoustic energy from lingering too long on one particular position of the hand. While some signaling may penetrate into the hand, most of the energy (99.9%) is reflected. Also disclosed are methods to couple just to the wetted surface of the hand.
A phased array of ultrasonic transducers may create arbitrary fields that can be utilized to manipulate fluids. This includes the translation of drops on smooth surfaces as well speeding the evaporation of fluids on wetted hands. Proposed herein is the use airborne ultrasound focused to the surface of the hand. The risk is that coupling directly into the bulk of the hand may cause damage to the cellular material through heating, mechanical stress, or cavitation. Using a phased array, the focus may be moved around, thus preventing acoustic energy from lingering too long on one particular position of the hand. While some signaling may penetrate into the hand, most of the energy (99.9%) is reflected. Also disclosed are methods to couple just to the wetted surface of the hand.
Creating near-instantaneous effects in a phased-array system may occur by splitting the update process of the array state into parts that depend on different update rates. Alternatively, leveraging the uncertainty of physical properties of focusing in a phased-array system may improve the intersection between the body part and the control point. Specifically, by focusing behind or in front of the intended region or at a position with a calculated geometric relationship to the intended interaction region, a larger volume (region) of space is addressed that more certainly contains the body part participating in the interaction. This larger volume is then subjected to the ultrasonic radiative energy flux that encodes the properties desired for the interaction point, which may include haptic and/or audio points.
09 - Scientific and electric apparatus and instruments
42 - Scientific, technological and industrial services, research and design
Goods & Services
Electronic circuits; downloadable and recorded computer software for computer system and application development, deployment and management; Electronic transmitters and receivers for feedback for mid-air gestures or interaction with virtual objects; downloadable and recorded computer software for the development, deployment and management of transmitters and receivers for feedback for mid-air gestures or interaction with virtual objects; Electrical apparatus, namely, computer hardware for providing a physical response to a virtual object; computer systems comprised of computer hardware and software for providing physical response to gestures in mid air Consulting in the field of electrical engineering; Consulting in the field of configuration management for computer hardware and software; Consulting in the field of designing transmitters and receivers for feedback for mid-air gestures or interaction with virtual objects; Consulting in the field of development, deployment and management of computer software for feedback for mid-air gestures or interaction with virtual objects
09 - Scientific and electric apparatus and instruments
Goods & Services
Electrical apparatus for providing a physical response to a
virtual object, computer hardware and software systems for
providing physical response to gestures in mid air; gesture
recognition software.
09 - Scientific and electric apparatus and instruments
Goods & Services
Electrical apparatus for providing a physical response to a
virtual object, computer hardware and software systems for
providing physical response to gestures in mid air; gesture
recognition software.
57.
Method and apparatus for modulating haptic feedback
The present invention concerns a method and apparatus for the modulation of an acoustic field for providing tactile sensations. A method of creating haptic feedback using ultrasound is provided. The method comprises the steps of generating a plurality of ultrasound waves with a common focal point using a phased array of ultrasound transducers, the common focal point being a haptic feedback point, and modulating the generation of the ultrasound waves using a waveform selected to produce little or no audible sound at the haptic feedback point.
09 - Scientific and electric apparatus and instruments
Goods & Services
(1) Electrical haptic devices using ultrasound for electric audio and visual apparatus and instruments to provide a physical response to a virtual object, computer hardware and software systems for providing physical response to gestures in mid air; gesture recognition software.
09 - Scientific and electric apparatus and instruments
Goods & Services
(1) Electrical haptic devices using ultrasound for electric audio and visual apparatus and instruments to provide a physical response to a virtual object, computer hardware and software systems for providing physical response to gestures in mid air; gesture recognition software.
60.
BLOCKING PLATE STRUCTURE FOR IMPROVED ACOUSTIC TRANSMISSION EFFICIENCY
An acoustic matching structure is used to increase the power radiated from a transducing element with a higher impedance into a surrounding acoustic medium with a lower acoustic impedance. The acoustic matching structure consists of a thin, substantially planar cavity bounded by a two end walls and a side wall. The end walls of the cavity are formed by a blocking plate wall and a transducing element wall separated by a short distance (less than one quarter of the wavelength of acoustic waves in the surrounding medium at the operating frequency). The end walls and side wall bound a cavity with diameter approximately equal to half of the wavelength of acoustic waves in the surrounding medium. Inoperation, a transducing element generates acoustic oscillations in the fluid in the cavity. The transducing element may be an actuator which generates motion of an end wall in a direction perpendicular to the plane of the cavity to excite acoustic oscillations in the fluid in the cavity, and the cavity geometry and resonant amplification increase the amplitude of the resulting pressure oscillation. The cavity side wall or end walls contain at least one aperture positioned away from the center of the cavity to allow pressure waves to propagate into the surrounding acoustic medium.
B06B 1/06 - Processes or apparatus for generating mechanical vibrations of infrasonic, sonic or ultrasonic frequency making use of electrical energy operating with piezoelectric effect or with electrostriction
An acoustic matching structure is used to increase the power radiated from a transducing element with a higher impedance into a surrounding acoustic medium with a lower acoustic impedance. The acoustic matching structure consists of a thin, substantially planar cavity bounded by a two end walls and a side wall. The end walls of the cavity are formed by a blocking plate wall and a transducing element wall separated by a short distance (less than one quarter of the wavelength of acoustic waves in the surrounding medium at the operating frequency). The end walls and side wall bound a cavity with diameter approximately equal to half of the wavelength of acoustic waves in the surrounding medium. In operation, a transducing element generates acoustic oscillations in the fluid in the cavity. The transducing element may be an actuator which generates motion of an end wall in a direction perpendicular to the plane of the cavity to excite acoustic oscillations in the fluid in the cavity, and the cavity geometry and resonant amplification increase the amplitude of the resulting pressure oscillation. The cavity side wall or end walls contain at least one aperture positioned away from the center of the cavity to allow pressure waves to propagate into the surrounding acoustic medium.
H04R 1/28 - Transducer mountings or enclosures designed for specific frequency responseTransducer enclosures modified by provision of mechanical or acoustic impedances, e.g. resonator, damping means
A system and method for the development, encapsulation, distribution and deployment of low-latency input-output processing algorithms to mediate human-computer interaction by generating haptic feedback influenced in real time by external stimuli. The invention provides the ability to design algorithms that scale or are otherwise automatically adjusted when deployed to computing environments with different capabilities. The invention also provides features to facilitate the design, sharing and modification of the algorithms, thereby reducing development time/cost.
09 - Scientific and electric apparatus and instruments
Goods & Services
Electrical apparatus, namely, computer hardware for providing a physical response to a virtual object; computer hardware and recorded software systems for providing physical responses to gestures in mid air; downloadable computer software, namely, gesture recognition software
64.
Method and apparatus for producing an acoustic field
The present invention concerns a method and apparatus for the modulation of an acoustic field for providing tactile sensations. A method of creating haptic feedback using ultrasound is provided. The method comprises the steps of generating a plurality of ultrasound waves with a common focal point using a phased array of ultrasound transducers, the common focal point being a haptic feedback point, and modulating the generation of the ultrasound waves using a waveform selected to produce little or no audible sound at the haptic feedback point.
G01S 7/52 - Details of systems according to groups , , of systems according to group
H04S 7/00 - Indicating arrangementsControl arrangements, e.g. balance control
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G01S 7/539 - Details of systems according to groups , , of systems according to group using analysis of echo signal for target characterisationTarget signatureTarget cross-section
G01S 15/36 - Systems for measuring distance only using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated with phase comparison between the received signal and the contemporaneously transmitted signal
A system providing various improved perceptions techniques for haptic feedback above interactive surfaces that require no contact with either tools, attachments or the surface itself is described. A range of receptors in a perceiving member which is part of the human body is identified to create substantially uniformly perceivable feedback. A vibration frequency that is in the range of the receptors in the perceiving member is chosen and dynamically altered to create substantially uniformly perceivable feedback throughout the receiving member.
H04R 1/40 - Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/043 - Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means using propagating acoustic waves
An acoustic field may be produced from a transducer array having known relative positions and orientations in this acoustic field, one or more control points may be defined. An amplitude may be assigned to the control point. Mid-air haptic effect for a virtual object on a human body part may be generated by moving the control point in a single closed curve comprising a plurality of curve segments. The single closed curve traverses at least one location where the human body part intersects with the virtual object. Additionally, a user may interact with virtual three-dimensional content using the user's hands while a tracking system monitoring the user's hands, a physics engine updates the properties of the virtual three-dimensional content and a haptic feedback system provides haptic information to the user.
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/0481 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
G06F 3/0488 - Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
G06F 3/041 - Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
Described herein are techniques for tracking objects (including human body parts such as a hand), namely: 1) two-state transducer interpolation in acoustic phased-arrays; 2) modulation techniques in acoustic phased-arrays; 3) fast acoustic full matrix capture during haptic effects; 4) time-of-flight depth sensor fusion system; 5) phase modulated spherical wave-fronts in acoustic phased-arrays; 6) long wavelength phase modulation of acoustic field for location and tracking; and 7) camera calibration through ultrasonic range sensing.
Strategies for managing an "always on" solution for volumes with enhanced interactive haptic feedback and its implications are addressed. Ultrasound transducer arrays may be mounted on a person (such as on a head mounted display or other wearable accessory). This array may utilize some form of 6 degree-of-freedom tracking for both the body and hands of the user. The arrays coordinate to project focused acoustic pressure at specific locations on moving hands such that a touch sensation is simulated. Using wearable microphones, the ultrasonic signal reflected and transmitted into the body can be used for hand and gesture tracking.
Disclosed are methods to manipulate a given parametrized haptic curve in order to yield a smooth phase function for each acoustic transducer which minimizes unwanted parametric audio. Further, the impulse response of a haptic system describes the behavior of the system over time and can be convolved with a given input to simulate a response to that input. To produce a specific response, a deconvolution with the impulse response is necessary to generate an input.
Disclosed are methods to manipulate a given parametrized haptic curve in order to yield a smooth phase function for each acoustic transducer which minimizes unwanted parametric audio. Further, the impulse response of a haptic system describes the behavior of the system over time and can be convolved with a given input to simulate a response to that input. To produce a specific response, a deconvolution with the impulse response is necessary to generate an input.
G08B 6/00 - Tactile signalling systems, e.g. personal calling systems
H04R 1/40 - Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers
G10K 11/34 - Sound-focusing or directing, e.g. scanning using electrical steering of transducer arrays, e.g. beam steering
A system providing various improved processing techniques for haptic feedback is described. An acoustic field is defined by one or more control points in a space within which the acoustic field may exist. Each control point is assigned an amplitude value equating to a desired amplitude of the acoustic field at the control point. Transducers are then controlled to create an acoustic field exhibiting the desired amplitude at each of the control points. When human skin interacts with the acoustic field, vibrations of the skin are interpreted by mechanoreceptors being excited and sending signals to the brain via the nervous system. Improved processing techniques allow for more efficient real-world operation.
G06F 3/00 - Input arrangements for transferring data to be processed into a form capable of being handled by the computerOutput arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
G08B 6/00 - Tactile signalling systems, e.g. personal calling systems
H04R 1/40 - Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers
G06F 3/038 - Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
Described herein are techniques for tracking objects (including human body parts such as a hand), namely: 1) two-state transducer interpolation in acoustic phased-arrays; 2) modulation techniques in acoustic phased-arrays; 3) fast acoustic full matrix capture during haptic effects; 4) time-of-flight depth sensor fusion system; 5) phase modulated spherical wave- fronts in acoustic phased-arrays; 6) long wavelength phase modulation of acoustic field for location and tracking; and 7) camera calibration through ultrasonic range sensing.
To resolve an issue related to the calibration of optical cameras in transducer- based mid-air haptic systems, the magnification of the motion induced on an optical camera by an acoustic field modulated at specific frequencies reveals very small temporal variations in video frames. This quantized distortion is used to compare different acoustic fields and to solve the calibration problem in an automatized manner. Further, mechanical resonators may be excited by ultrasound when it is modulated at the resonant frequency. When enough energy is transferred and when operating at the correct frequency, a user in contact with the device can feel vibration near areas of largest displacement. This effect can be exploited to create devices which can produce haptic feedback while not carrying a battery or exciter when in the presence of an ultrasonic source.
To resolve an issue related to the calibration of optical cameras in transducer-based mid-air haptic systems, the magnification of the motion induced on an optical camera by an acoustic field modulated at specific frequencies reveals very small temporal variations in video frames. This quantized distortion is used to compare different acoustic fields and to solve the calibration problem in an automatized manner. Further, mechanical resonators may be excited by ultrasound when it is modulated at the resonant frequency. When enough energy is transferred and when operating at the correct frequency, a user in contact with the device can feel vibration near areas of largest displacement. This effect can be exploited to create devices which can produce haptic feedback while not carrying a battery or exciter when in the presence of an ultrasonic source.
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/0354 - Pointing devices displaced or positioned by the userAccessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
G06F 3/03 - Arrangements for converting the position or the displacement of a member into a coded form
G10K 11/34 - Sound-focusing or directing, e.g. scanning using electrical steering of transducer arrays, e.g. beam steering
G06F 3/038 - Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
09 - Scientific and electric apparatus and instruments
Goods & Services
Electrical apparatus for providing a physical response to a virtual object, computer hardware and software systems for providing physical response to gestures in mid air; gesture recognition software.
76.
INTERFERENCE REDUCTION TECHNIQUES IN HAPTIC SYSTEMS
As control points in haptic systems move around, the phase offsets for each transducer change at discrete points in time. These are each expressed as a phase offset combined with a monochromatic carrier frequency. To prevent sharp frequency changes, an algorithm that maintains smooth transitions is used. Further, non-idealities in the implementation of haptic array modulation can create spurs in the frequency response of audio output from the array. Adjusting the signal carrier frequency and the signal modulating frequency may substantially reduce audio noise via a notch filter centered at an interpolation frequency.
Improved algorithm techniques may be used for superior operation of haptic- based systems. An eigensystem may be used to determine for a given spatial distribution of control points with specified output the set of wave phases that are the most efficiently realizable. Reconstructing a modulated pressure field may use emitters firing at different frequencies. An acousiic phased-array device uses a comprehensive reflexive simulation technique. There may be an exchange of information between the users and the transducer control processors having the ability to use that information for optimal haptic generation shadows and the like. Applying mid -air haptic sensations to objects of arbitrary 3D geometry requires that sensation of the object on the user's hand is as close as possible to a realistic depiction of that object. Ultrasonic haptics with multiple and/or large aperture arrays have high-frequency update rates required by the spatio-temporal modulation. More efficient haptic systems require the prevention of a channel of audio unintentionally encoding phase information that may distort its perception.
The present invention concerns a method and apparatus for the modulation of an acoustic field for providing tactile sensations. A method of creating haptic feedback using ultrasound is provided. The method comprises the steps of generating a plurality of ultrasound waves with a common focal point using a phased array of ultrasound transducers, the common focal point being a haptic feedback point, and modulating the generation of the ultrasound waves using a waveform selected to produce little or no audible sound at the haptic feedback point.
G01S 7/00 - Details of systems according to groups , ,
G01S 7/52 - Details of systems according to groups , , of systems according to group
H04S 7/00 - Indicating arrangementsControl arrangements, e.g. balance control
G01S 7/539 - Details of systems according to groups , , of systems according to group using analysis of echo signal for target characterisationTarget signatureTarget cross-section
G01S 15/36 - Systems for measuring distance only using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated with phase comparison between the received signal and the contemporaneously transmitted signal
The present invention concerns a method and apparatus for the modulation of an acoustic field for providing tactile sensations. A method of creating haptic feedback using ultrasound is provided. The method comprises the steps of generating a plurality of ultrasound waves with a common focal point using a phased array of ultrasound transducers, the common focal point being a haptic feedback point, and modulating the generation of the ultrasound waves using a waveform selected to produce little or no audible sound at the haptic feedback point.
A circuit for driving a transducer in a mid-air haptic system includes a voltage source, a voltage sink, a current source, a trickle capacitor, a storage capacitor, a haptic system transducer, a first switch, a second switch, and a third switch. Using these components, a portion of the charge required for switching a transducer is sourced from the decoupling capacitance. When the switching completes, additional charge is transferred immediately from the power supply back into the decoupling capacitance. This acts to lower the peak current by fully utilizing 100% of a switching waveform for transfer of charge from the power supply to capacitors local to the transducer.
G10K 11/34 - Sound-focusing or directing, e.g. scanning using electrical steering of transducer arrays, e.g. beam steering
G06F 3/043 - Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means using propagating acoustic waves
H01G 4/002 - Fixed capacitorsProcesses of their manufacture Details
H02M 1/096 - Circuits specially adapted for the generation of control voltages for semiconductor devices incorporated in static converters for the simultaneous control of series or parallel connected semiconductor devices the power supply of the control circuit being connected in parallel to the main switching element
H02M 1/16 - Means for providing current step on switching, e.g. with saturable reactor
B06B 1/02 - Processes or apparatus for generating mechanical vibrations of infrasonic, sonic or ultrasonic frequency making use of electrical energy
81.
Method and apparatus for providing tactile sensations
The present disclosure concerns a method and apparatus for the creation of an acoustic field for providing tactile sensations. More particularly, but not exclusively, this disclosure concerns a method and apparatus for the creation of an acoustic field providing tactile sensations for use with an interactive device.
The disclosure provides a method of generating a tactile sensation. The method comprises the steps of providing a plurality of acoustic transducers arranged to generate a predetermined distribution of pressure patterns, wherein the pressure patterns comprise a first region providing a first tactile sensation and a second region providing a second, different, tactile sensation.
A circuit for driving a transducer in a mid-air haptic system includes a voltage source, a voltage sink, a current source, a trickle capacitor, a storage capacitor, a haptic system transducer, a first switch, a second switch, and a third switch. Using these components, a portion of the charge required for switching a transducer is sourced from the decoupling capacitance. When the switching completes, additional charge is transferred immediately from the power supply back into the decoupling capacitance. This acts to lower the peak current by fully utilizing 100% of a switching waveform for transfer of charge from the power supply to capacitors local to the transducer.
B06B 1/02 - Processes or apparatus for generating mechanical vibrations of infrasonic, sonic or ultrasonic frequency making use of electrical energy
H01L 41/04 - SEMICONDUCTOR DEVICES; ELECTRIC SOLID STATE DEVICES NOT OTHERWISE PROVIDED FOR - Details thereof - Details of piezo-electric or electrostrictive elements
Various techniques for driving phased array systems are described, specifically intended for acoustic phased arrays with applications to mid-air haptics, parametric audio, acoustic levitation and acoustic imaging, including a system: 1) that is capable of mitigating the effect of the changes in the air to provide a consistent haptic experience; 2) that produces trap points in air; 3) that defines phased-array optimization in terms of vectors for the production of more consistent haptic effects; 4) that defines one or more control points or regions in space via a controlled acoustic field; 5) that uses a reduced representation method for the construction of acoustic basis functions; 6) that performs efficient evaluation of complex- valued functions for a large quantity of throughput; 7) that generates a Krylov sub-space of a matrix; and 8) that maximizes an objective described by different control points and/or regions to those used to create the acoustic basis functions.
Various techniques for driving phased array systems are described, specifically intended for acoustic phased arrays with applications to mid-air haptics, parametric audio, acoustic levitation and acoustic imaging, including a system: 1) that is capable of mitigating the effect of the changes in the air to provide a consistent haptic experience; 2) that produces trap points in air; 3) that defines phased-array optimization in terms of vectors for the production of more consistent haptic effects; 4) that defines one or more control points or regions in space via a controlled acoustic field; 5) that uses a reduced representation method for the construction of acoustic basis functions; 6) that performs efficient evaluation of complex-valued functions for a large quantity of throughput; 7) that generates a Krylov sub-space of a matrix; and 8) that maximizes an objective described by different control points and/or regions to those used to create the acoustic basis functions.
G10K 11/34 - Sound-focusing or directing, e.g. scanning using electrical steering of transducer arrays, e.g. beam steering
B06B 1/06 - Processes or apparatus for generating mechanical vibrations of infrasonic, sonic or ultrasonic frequency making use of electrical energy operating with piezoelectric effect or with electrostriction
A61B 8/00 - Diagnosis using ultrasonic, sonic or infrasonic waves
G10K 11/00 - Methods or devices for transmitting, conducting or directing sound in generalMethods or devices for protecting against, or for damping, noise or other acoustic waves in general
09 - Scientific and electric apparatus and instruments
42 - Scientific, technological and industrial services, research and design
Goods & Services
Computer software; electrical apparatus for providing a physical response to a virtual object, computer hardware and software systems for providing physical response to gestures in mid-air; gesture recognition software; software to enable development of products featuring mid-air tactile feedback; software and hardware and electrical apparatus to allow for control of virtual buttons, sliders and dials; software development tool to enable gesture and sensation recognition. Software as a service; software design; design and development of software to enable gesture and sensation recognition.
09 - Scientific and electric apparatus and instruments
42 - Scientific, technological and industrial services, research and design
Goods & Services
Computer software; electrical apparatus for providing a physical response to a virtual object, computer hardware and software systems for providing physical response to gestures in mid-air; gesture recognition software; software to enable development of products featuring mid-air tactile feedback; software and hardware and electrical apparatus to allow for control of virtual buttons, sliders and dials; software development tool to enable gesture and sensation recognition. Software as a service; software design; design and development of software to enable gesture and sensation recognition.
A system providing various improved perceptions techniques for haptic feedback above interactive surfaces that require no contact with either tools, attachments or the surface itself is described. A range of receptors in a perceiving member which is part of the human body is identified to create substantially uniformly perceivable feedback. A vibration frequency that is in the range of the receptors in the perceiving member is chosen and dynamically altered to create substantially uniformly perceivable feedback throughout the receiving member.
The properties of metamaterials are derived both from the inherent properties of their constituent materials and from the geometrical arrangement of those materials. Metamaterials may be stacked or otherwise manipulated to transform substantially monochromatic signal into a second signal having a desired amplitude and phase. Metamaterials may be used with acoustic devices to create haptic feedback with desired properties or to transform the shape of certain devices. Metamaterials may be used in rotating devices with openings that transform a monochromatic signal into a non-monochromatic signal.
G10K 11/178 - Methods or devices for protecting against, or for damping, noise or other acoustic waves in general using interference effectsMasking sound by electro-acoustically regenerating the original acoustic waves in anti-phase
G10K 11/22 - Methods or devices for transmitting, conducting or directing sound for conducting sound through hollow pipes, e.g. speaking tubes
89.
METAMATERIALS AND ACOUSTIC LENSES IN HAPTIC SYSTEMS
The properties of metamaterials are derived both from the inherent properties of their constituent materials and from the geometrical arrangement of those materials. Metamaterials may be stacked or otherwise manipulated to transform substantially monochromatic signal into a second signal having a desired amplitude and phase. Metamaterials may be used with acoustic devices to create haptic feedback with desired properties or to transform the shape of certain devices. Metamaterials may be used in rotating devices with openings that transform a monochromatic signal into a non-monochromatic signal.
An acoustic field may be produced from a transducer array having known relative positions and orientations In this acoustic field, one or more control points may be defined. An amplitude may be assigned to the control point. Mid-air haptic effect for a virtual object on a human body part may be generated by moving the control point in a single closed curve comprising a plurality of curve segments. The single closed curve traverses at least one location where the human body part intersects with the virtual object. Additionally, a user may interact with virtual three-dimensional content using the user's hands while a tracking system monitoring the user's hands, a physics engine updates the properties of the virtual three-dimensional content and a haptic feedback system provides haptic information to the user.
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/0481 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
G06F 3/0488 - Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
G06F 3/041 - Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
An acoustic field may be produced from a transducer array having known relative positions and orientations. In this acoustic field, one or more control points may be defined. An amplitude may be assigned to the control point. Mid-air haptic effect for a virtual object (120) on a human hand (110) may be generated by moving the control point in a single closed curve comprising a plurality of curve segments. The single closed curve traverses at least one location where the human hand intersects with the virtual object. Additionally, a user may interact with virtual three-dimensional content using the user's hands while a tracking system monitoring the user's hands, a physics engine updates the properties of the virtual three-dimensional content and a haptic feedback system provides haptic information to the user.
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/0488 - Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
G06F 3/041 - Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
Described is a screen surface layer comprising a plurality of light-emitting pixels including a plurality of red pixels, a plurality of green pixels and a plurality of blue pixels; a transducer layer comprising a plurality of acoustic transducers, wherein the transducer layer is affixed below the screen surface layer; and a cover layer having a plurality of cylinder-shaped perforations, wherein the cover layer is affixed above the screen surface layer. In addition, an average “transducer model” describing a phasor distribution in space may be produced describing the output signal at the carrier frequency in the air at a known spatial offset from an averaged transducer. Further, phased array systems may have numerous individual transducer elements that will likely fail before the product incorporating the transducers reaches end-of-life. By detecting such failed transducers, the transducer array may function at peak performance by working around the missing transducer output.
H04R 1/40 - Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers
H04R 3/12 - Circuits for transducers for distributing signals to two or more loudspeakers
Described is a system for producing an acoustic field from a plurality of ultrasonic transducer arrays, each of which has known relative positions and orientations. The acoustic field comprises a carrier wave and a modulated wave. The carrier wave has a plurality of modulated focal areas. A plurality of control points having a known spatial relationship relative to at least one of the plurality of ultrasonic transducer arrays is used. The plurality of ultrasonic transducer arrays are calibrated by using the relative position of each of the plurality of ultrasonic transducer arrays.
Described is a system for producing an acoustic field from a plurality of ultrasonic transducer arrays, each of which has known relative positions and orientations. The acoustic field comprises a carrier wave and a modulated wave. The carrier wave has a plurality of modulated focal areas. A plurality of control points having a known spatial relationship relative to at least one of the plurality of ultrasonic transducer arrays is used. The plurality of ultrasonic transducer arrays are calibrated by using the relative position of each of the plurality of ultrasonic transducer arrays.
09 - Scientific and electric apparatus and instruments
Goods & Services
Electrical apparatus for providing a physical response to a
virtual object, computer hardware and software systems for
providing physical response to gestures in mid air; gesture
recognition software.
97.
Method and apparatus for providing tactile sensations
The present disclosure concerns a method and apparatus for the creation of an acoustic field for providing tactile sensations. More particularly, but not exclusively, this disclosure concerns a method and apparatus for the creation of an acoustic field providing tactile sensations for use with an interactive device.
The disclosure provides a method of generating a tactile sensation. The method comprises the steps of providing a plurality of acoustic transducers arranged to generate a predetermined distribution of pressure patterns, wherein the pressure patterns comprise a first region providing a first tactile sensation and a second region providing a second, different, tactile sensation.
A system providing various improved calibration techniques for haptic feedback is described. An acoustic field is defined by one or more control points in a space within which the acoustic field may exist. Each control point is assigned an amplitude value equating to a desired amplitude of the acoustic field at the control point. Because complete control of space is not possible, controlling the acoustic field at given points yields erroneous local maxima in the acoustic field levels at other related positions. In relation to mid-air haptic feedback, these can interfere in interactions with the space by creating secondary effects and ghost phenomena that can be felt outside the interaction area. The level and nature of the secondary maxima in the acoustic field is determined by how the space is controlled. By arranging the transducer elements in different ways, unwanted effects on the acoustic field can be limited and controlled.
A system providing various improved calibration techniques for haptic feedback is described. An acoustic field is defined by one or more control points in a space within which the acoustic field may exist. Each control point is assigned an amplitude value equating to a desired amplitude of the acoustic field at the control point. Because complete control of space is not possible, controlling the acoustic field at given points yields erroneous local maxima in the acoustic field levels at other related positions. In relation to mid-air haptic feedback, these can interfere in interactions with the space by creating secondary effects and ghost phenomena that can be felt outside the interaction area. The level and nature of the secondary maxima in the acoustic field is determined by how the space is controlled. By arranging the transducer elements in different ways, unwanted effects on the acoustic field can be limited and controlled.
09 - Scientific and electric apparatus and instruments
42 - Scientific, technological and industrial services, research and design
Goods & Services
Computer software used for development of haptic applications; electrical apparatus for providing a physical response to a virtual object, computer hardware and software systems for providing physical response to gestures in mid-air; gesture recognition software; software to enable development of products featuring mid-air tactile feedback; software and hardware and electrical apparatus to allow for control of virtual buttons, sliders and dials; computer software development tool to enable gesture and sensation recognition Software as a service featuring software development tools to enable gesture and sensation recognition; computer software design; design and development of software to enable gesture and sensation recognition
