An end-to-end automatic speech recognition (ASR) system can be constructed by fusing a first ASR model with a transformer. The input of the transformer is a learned layer generated by the first ASR model. The fused ASR model and transformer can be treated as a single end-to-end model and trained as a single model. In some embodiments, the end-to-end speech recognition system can be trained using a teacher-student training technique by selectively truncating portions of the first ASR model and/or the transformer components and selectively freezing various layers during the training passes.
Modern automatic speech recognition (ASR) systems can utilize artificial intelligence (AI) models to service ASR requests. The number and scale of AI models used in a modern ASR system can be substantial. The process of configuring and reconfiguring hardware to execute various AI models corresponding to a substantial number of ASR requests can be time consuming and inefficient. Among other features, the described technology utilizes batching of ASR requests, splitting of the ASR requests, and/or parallel processing to efficiently use hardware tasked with executing AI models corresponding to ASR requests. In one embodiment, the compute graphs of ASR tasks are used to batch the ASR requests. The corresponding AI models of each batch can be loaded into hardware, and batches can be processed in parallel. In some embodiments, the ASR requests are split, batched, and processed in parallel.
Systems and methods are disclosed for generating internal state representations of a neural network during processing and using the internal state representations for classification or search. In some embodiments, the internal state representations are generated from the output activation functions of a subset of nodes of the neural network. The internal state representations may be used for classification by training a classification model using internal state representations and corresponding classifications. The internal state representations may be used for search, by producing a search feature from an search input and comparing the search feature with one or more feature representations to find the feature representation with the highest degree of similarity.
G10L 15/16 - Speech classification or search using artificial neural networks
G10L 15/06 - Creation of reference templatesTraining of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice
G06N 3/084 - Backpropagation, e.g. using gradient descent
G10L 25/18 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band
G10L 25/24 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being the cepstrum
G06V 10/44 - Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersectionsConnectivity analysis, e.g. of connected components
G06F 18/214 - Generating training patternsBootstrap methods, e.g. bagging or boosting
G06F 18/2413 - Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
G06N 3/044 - Recurrent networks, e.g. Hopfield networks
Systems and methods are disclosed for generating internal state representations of a neural network during processing and using the internal state representations for classification or search. In some embodiments, the internal state representations are generated from the output activation functions of a subset of nodes of the neural network. The internal state representations may be used for classification by training a classification model using internal state representations and corresponding classifications. The internal state representations may be used for search, by producing a search feature from an search input and comparing the search feature with one or more feature representations to find the feature representation with the highest degree of similarity.
G10L 15/16 - Speech classification or search using artificial neural networks
G10L 15/06 - Creation of reference templatesTraining of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice
G06N 3/084 - Backpropagation, e.g. using gradient descent
G10L 25/18 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band
G10L 25/24 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being the cepstrum
G06V 10/44 - Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersectionsConnectivity analysis, e.g. of connected components
G06F 18/214 - Generating training patternsBootstrap methods, e.g. bagging or boosting
G06F 18/2413 - Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
G06N 3/044 - Recurrent networks, e.g. Hopfield networks
Systems and methods are disclosed for end-to-end neural networks for speech recognition and classification and additional machine learning techniques that may be used in conjunction or separately. Some embodiments comprise multiple neural networks, directly connected to each other to form an end-to-end neural network. One embodiment comprises a convolutional network, a first fully-connected network, a recurrent network, a second fully-connected network, and an output network. Some embodiments are related to generating speech transcriptions, and some embodiments relate to classifying speech into a number of classifications.
G10L 25/18 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band
G10L 25/24 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being the cepstrum
G06V 10/44 - Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersectionsConnectivity analysis, e.g. of connected components
G10L 15/02 - Feature extraction for speech recognitionSelection of recognition unit
G10L 15/22 - Procedures used during a speech recognition process, e.g. man-machine dialog
G10L 15/30 - Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
G10L 15/197 - Probabilistic grammars, e.g. word n-grams
Systems and methods are disclosed for generating internal state representations of a neural network during processing and using the internal state representations for classification or search. In some embodiments, the internal state representations are generated from the output activation functions of a subset of nodes of the neural network. The internal state representations may be used for classification by training a classification model using internal state representations and corresponding classifications. The internal state representations may be used for search, by producing a search feature from an search input and comparing the search feature with one or more feature representations to find the feature representation with the highest degree of similarity.
G06K 9/46 - Extraction of features or characteristics of the image
G10L 25/18 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band
G10L 25/24 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being the cepstrum
G10L 15/02 - Feature extraction for speech recognitionSelection of recognition unit
G10L 15/22 - Procedures used during a speech recognition process, e.g. man-machine dialog
G10L 15/30 - Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
G10L 15/197 - Probabilistic grammars, e.g. word n-grams
Systems and methods are disclosed for end-to-end neural networks for speech recognition and classification and additional machine learning techniques that may be used in conjunction or separately. Some embodiments comprise multiple neural networks, directly connected to each other to form an end-to-end neural network. One embodiment comprises a convolutional network, a first fully-connected network, a recurrent network, a second fully-connected network, and an output network. Some embodiments are related to generating speech transcriptions, and some embodiments relate to classifying speech into a number of classifications.
G06K 9/46 - Extraction of features or characteristics of the image
G10L 25/18 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band
G10L 25/24 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being the cepstrum
G10L 15/02 - Feature extraction for speech recognitionSelection of recognition unit
G10L 15/22 - Procedures used during a speech recognition process, e.g. man-machine dialog
G10L 15/30 - Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
G10L 15/197 - Probabilistic grammars, e.g. word n-grams
Systems and methods are disclosed for customizing a neural network for a custom dataset, when the neural network has been trained on data from a general dataset. The neural network may comprise an output layer including one or more nodes corresponding to candidate outputs. The values of the nodes in the output layer may correspond to a probability that the candidate output is the correct output for an input. The values of the nodes in the output layer may be adjusted for higher performance when the neural network is used to process data from a custom dataset.
Systems and methods are disclosed for generating internal state representations of a neural network during processing and using the internal state representations for classification or search. In some embodiments, the internal state representations are generated from the output activation functions of a subset of nodes of the neural network. The internal state representations may be used for classification by training a classification model using internal state representations and corresponding classifications. The internal state representations may be used for search, by producing a search feature from an search input and comparing the search feature with one or more feature representations to find the feature representation with the highest degree of similarity.
Systems and methods are disclosed for customizing a neural network for a custom dataset, when the neural network has been trained on data from a general dataset. The neural network may comprise an output layer including one or more nodes corresponding to candidate outputs. The values of the nodes in the output layer may correspond to a probability that the candidate output is the correct output for an input. The values of the nodes in the output layer may be adjusted for higher performance when the neural network is used to process data from a custom dataset.
