Abstract

Methods and systems are provided for detecting and processing gestures, expressions (e.g., facial), tone and/or gestures of the user for the purpose of improving the quality and speed of interactions with computer-based systems. Such information may be detected by one or more sensors such as, for example, electromyography (EMG) sensors used to monitor and record electrical activity produced by muscles that are activated. Other sensor types may be used, such as optical, inertial measurement unit (IMU), or other types of bio-sensors. The system may use one or more sensors to detect speech alone or in combination with gestures, expressions (e.g., facial), tone and/or gestures of the user to provide input or control of the system.

IPC Classes  ?

• G10L 13/027 - Concept to speech synthesisersGeneration of natural phrases from machine-based concepts
• G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
• G06N 3/092 - Reinforcement learning
• G06N 20/00 - Machine learning
• G10L 13/033 - Voice editing, e.g. manipulating the voice of the synthesiser
• G10L 13/047 - Architecture of speech synthesisers
• G10L 15/18 - Speech classification or search using natural language modelling
• G10L 15/22 - Procedures used during a speech recognition process, e.g. man-machine dialog
• G10L 15/24 - Speech recognition using non-acoustical features
• G10L 15/25 - Speech recognition using non-acoustical features using position of the lips, movement of the lips or face analysis
• G10L 19/012 - Comfort noise or silence coding
• G10L 19/04 - Speech or audio signal analysis-synthesis techniques for redundancy reduction, e.g. in vocodersCoding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
• G10L 25/18 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band
• G10L 25/78 - Detection of presence or absence of voice signals

Abstract

The present disclosure relates to methods and systems for adjusting a silent speech machine learning model for use with a wearable silent speech device. In some embodiments, a method may include recording speech signals from a user, using a first sensor and a second sensor of a wearable silent speech device. The method may include providing for a silent speech machine learning model for use with the wearable silent speech device, determining whether the silent speech machine learning model is to be adjusted, and in response to determining the silent speech machine learning model is to be adjusted, adjusting the silent speech machine learning model based on at least the speech signals recorded using the first sensor and the second sensor.

IPC Classes  ?

• G10L 15/06 - Creation of reference templatesTraining of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice
• A61B 5/389 - Electromyography [EMG]
• G10L 15/24 - Speech recognition using non-acoustical features
• G10L 25/78 - Detection of presence or absence of voice signals

Abstract

Systems and methods are provided for decoding the silent speech of a user. Speech of the user (e.g., silent, vocalized, etc.) may be detected and captured by a speech input device configured to measure signals indicative of the speech muscle activation patterns of the user. A trained machine learning model configured to decode the speech of the user based at least in part on the signal indicative of the speech muscle activation patterns of the user. To improve accuracy of the model, the trained machine learning model may be trained using training data obtained in at least a subset of sampling contexts of a plurality of sampling contexts. At least one processor may be configured to output the decoded speech of the user.

IPC Classes  ?

• G10L 15/18 - Speech classification or search using natural language modelling
• G10L 15/20 - Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise or of stress induced speech
• G06N 20/00 - Machine learning
• G10L 15/05 - Word boundary detection
• G10L 15/25 - Speech recognition using non-acoustical features using position of the lips, movement of the lips or face analysis

Abstract

Methods and systems are provided for detecting and synthesizing a user's speech, including, for example, vocalized, whispered, and silent speech for the purpose of providing an output substantially in parallel with the user speaking. Such information may be detected by one or more sensors such as, for example, electromyography (EMG) sensors used to monitor and record electrical activity produced by muscles that are activated, for example, speech muscles activated when the user is speaking. Other sensor types may be used, such as audio, optical, inertial measurement unit (IMU), or other types of sensors. The user's speech may be synthesized using one or more machine learning models or a machine learning model in conjunction with other suitable processing devices and methods.

IPC Classes  ?

• G10L 13/027 - Concept to speech synthesisersGeneration of natural phrases from machine-based concepts
• G10L 13/033 - Voice editing, e.g. manipulating the voice of the synthesiser
• G10L 13/047 - Architecture of speech synthesisers
• G10L 15/25 - Speech recognition using non-acoustical features using position of the lips, movement of the lips or face analysis
• G10L 25/18 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band

Abstract

Methods and systems are provided for detecting and processing gestures, expressions (e.g., facial), tone and/or gestures of the user for the purpose of improving the quality and speed of interactions with computer-based systems. Such information may be detected by one or more sensors such as, for example, electromyography (EMG) sensors used to monitor and record electrical activity produced by muscles that are activated. Other sensor types may be used, such as optical, inertial measurement unit (IMU), or other types of bio-sensors. The system may use one or more sensors to detect speech alone or in combination with gestures, expressions (e.g., facial), tone and/or gestures of the user to provide input or control of the system.

IPC Classes  ?

• G10L 13/027 - Concept to speech synthesisersGeneration of natural phrases from machine-based concepts
• G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
• G06N 3/092 - Reinforcement learning
• G06N 20/00 - Machine learning
• G10L 13/033 - Voice editing, e.g. manipulating the voice of the synthesiser
• G10L 13/047 - Architecture of speech synthesisers
• G10L 15/18 - Speech classification or search using natural language modelling
• G10L 15/22 - Procedures used during a speech recognition process, e.g. man-machine dialog
• G10L 15/24 - Speech recognition using non-acoustical features
• G10L 15/25 - Speech recognition using non-acoustical features using position of the lips, movement of the lips or face analysis
• G10L 19/012 - Comfort noise or silence coding
• G10L 19/04 - Speech or audio signal analysis-synthesis techniques for redundancy reduction, e.g. in vocodersCoding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
• G10L 25/18 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band
• G10L 25/78 - Detection of presence or absence of voice signals

Abstract

Systems and methods are provided for decoding the silent speech of a user. Speech of the user (e.g., silent, vocalized, etc.) may be detected and captured by a speech input device configured to measure signals indicative of the speech muscle activation patterns of the user. A trained machine learning model configured to decode the speech of the user based at least in part on the signal indicative of the speech muscle activation patterns of the user. To improve accuracy of the model, the trained machine learning model may be trained using training data obtained in at least a subset of sampling contexts of a plurality of sampling contexts. At least one processor may be configured to output the decoded speech of the user.

IPC Classes  ?

• G10L 19/012 - Comfort noise or silence coding
• G10L 19/04 - Speech or audio signal analysis-synthesis techniques for redundancy reduction, e.g. in vocodersCoding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques

Abstract

Methods and systems are provided for facilitating silent calling over a communication network. Such silent calling may be facilitated using a speech system associated with a first user configured to measure signals associated with the first user's speech muscle activation patterns and a communication interface configured to communicate with a communication device associated with a second user on the communication network. The first user's silent speech may be synthesized based at least in part on the signals associated with the first user's speech muscle activation patterns.

IPC Classes  ?

• G10L 13/027 - Concept to speech synthesisersGeneration of natural phrases from machine-based concepts
• G10L 13/033 - Voice editing, e.g. manipulating the voice of the synthesiser
• G10L 13/047 - Architecture of speech synthesisers
• G10L 15/25 - Speech recognition using non-acoustical features using position of the lips, movement of the lips or face analysis
• G10L 25/18 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band

Abstract

The techniques described herein relate to computerized methods and systems for integrating with a knowledge system. In some embodiments, a user interaction system may include a speech input device wearable on a user and configured to receive an electronic signal indicative of a user's speech muscle activation patterns when the user is speaking. In some embodiments, the electronic signal may include EMG data received from an EMG sensor on the speech input device. The system may include at least one processor configured to use a speech model and the electronic signal as input to the speech model to generate a text prompt. The at least one processor may use a knowledge system to take an action or generate a response based on the text prompt. In some embodiments, the system may provide context to the knowledge system.

IPC Classes  ?

• G10L 15/22 - Procedures used during a speech recognition process, e.g. man-machine dialog
• G10L 15/18 - Speech classification or search using natural language modelling

Abstract

Methods and systems are provided for detecting and processing gestures, expressions (e.g., facial), tone and/or gestures of the user for the purpose of improving the quality and speed of interactions with computer-based systems. Such information may be detected by one or more sensors such as, for example, electromyography (EMG) sensors used to monitor and record electrical activity produced by muscles that are activated. Other sensor types may be used, such as optical, inertial measurement unit (IMU), or other types of bio-sensors. The system may use one or more sensors to detect speech alone or in combination with gestures, expressions (e.g., facial), tone and/or gestures of the user to provide input or control of the system.

IPC Classes  ?

• G06F 3/16 - Sound inputSound output
• G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
• G06N 20/00 - Machine learning