09 - Scientific and electric apparatus and instruments
42 - Scientific, technological and industrial services, research and design
Goods & Services
Downloadable software development kits (SDK); Computer hardware; Downloadable design intellectual property (IP) for computer hardware and integrated circuits, including intellectual property (IP) cores, reference designs, verification kits, software drivers, and documentation Design and development of artificial intelligence (AI) software; Research in the field of artificial intelligence (AI) technology; Research and development of technology using the study of the human neocortex and neocortical principles to advance artificial intelligence technology; Computer services, namely, AI model training and AI inference provided via a website
2.
Hardware Architecture For Processing Data In Sparse Neural Network
A hardware accelerator that is efficient at performing computations related to a sparse neural network. The sparse neural network may be associated with a plurality of nodes. One of the nodes includes one or more sparse tensors. The accelerator may compress the sparse tensor to a dense tensor. The sparse tensor may also be structured so that the dense locations in the tensor are blocked or partitioned. The accelerator may transpose the weight tensor and align the partitions of the tensor with the hardware architecture. The structured tensor has a balanced number of active values so that the active values can be processed by an efficient number of operating cycles of the accelerator. The accelerator may also perform bitwise and operation to determine the location of dense pairs in two sparse tensors to reduce the number of computations.
An inference system trains and performs inference using a sparse neural network. The sparse neural network may include one or more layers, and each layer may be associated with a set of sparse weights that represent sparse connections between nodes of a layer and nodes of a previous layer. A layer output may be generated by applying the set of sparse weights associated with the layer to the layer output of a previous layer. Moreover, the one or more layers of the sparse neural network may generate sparse layer outputs. By using sparse representations of weights and layer outputs, robustness and stability of the neural network can be significantly improved, while maintaining competitive accuracy.
Embodiments are related to generating a sparsified machine learning model by incrementally sparsifying a machine learning model followed by training of the sparsified machine learning model. The initial machine learning model may be trained as a dense model that includes a large number of active values in its weight tensors. Multiple iterations of sparsifying weights in the weight tensors followed by training of the sparsified machine learning model are performed to gradually increase the sparsity of the weight tensor while recovering or maintaining the accuracy of the output from the machine learning model.
Embodiments relate to an intelligent system that recognizes an object and its state, or affect changes in the state of the object to a target state, based on sensory input. The intelligent system includes sensor processors and learning processors. The sensor processors receives the sensory input from sensors and determines features in the sensory input. The sensor processors also receive poses of the sensors expressed in coordinate systems local to the sensors and converts them into poses expressed in a common coordinate system. Learning processors initialize an evidence value for each hypothesis on a corresponding model, its pose and/or its state, and update the evidence value as additional features are detected or additional signals are received. If none of the hypotheses has an evidence value above a threshold, it is determined that no matching model is found, and hence, a new model is generated and stored in the learning processor.
A common communication protocol (CCP) used across different components of an inference system that recognizes an object and its state, or affect changes in the state of the object to a targeted state, based on sensory input. One or more components may convert information they generate into a format compliant with the CCP for sending to one or more other components. The CCP includes pose information and object information of an object. The pose information indicates the location and the orientation of the object in a common coordinate system, as detected, inferred, predicted or targeted by a component of the inference system. The object information indicates either one or more features of the object, as detected, predicted or targeted, or identification of the object, as inferred or predicted by the component of the inference system.
G06T 7/70 - Determining position or orientation of objects or cameras
H04L 67/12 - Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
7.
Inferencing and Learning Based on Sensorimotor Input Data
One or more multi-layer systems are used to perform inference. A multi-layer system may correspond to a node that receives a set of sensory input data for hierarchical processing, and may be grouped to perform processing for sensory input data. Inference systems at lower layers of a multi-layer system pass representation of objects to inference systems at higher layers. Each inference system can perform inference and form their own versions of representations of objects, regardless of the level and layer of the inference systems. The set of candidate objects for each inference system is updated to those consistent with feature-location representations for the sensors as well as object representations at lower layers. The set of candidate objects is also updated to those consistent with candidate objects from other inference systems, such as inference systems at other layers of the hierarchy or inference systems included in other multi-layer systems.
G06N 5/046 - Forward inferencingProduction systems
G06V 10/80 - Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
G06V 10/94 - Hardware or software architectures specially adapted for image or video understanding
8.
Performing inference and training using sparse neural network
An inference system trains and performs inference using a sparse neural network. The sparse neural network may include one or more layers, and each layer may be associated with a set of sparse weights that represent sparse connections between nodes of a layer and nodes of a previous layer. A layer output may be generated by applying the set of sparse weights associated with the layer to the layer output of a previous layer. Moreover, the one or more layers of the sparse neural network may generate sparse layer outputs. By using sparse representations of weights and layer outputs, robustness and stability of the neural network can be significantly improved, while maintaining competitive accuracy.
Described herein are apparatus and methods for performing inference into the identity of an object. For an object of a plurality of objects, the apparatus receives feature-location information identifying a feature at first location on a first object of the plurality and a feature at a second location on a second object of the plurality. The apparatus activates a first set of location cells that collectively represent the first location on the first object corresponding to a feature on an object of the plurality of objects and a second set of location cells that collectively represent the second location on the second object corresponding to a feature on an object of the plurality of objects. The apparatus activates a set of displacement cells representing displacement of the first set of location cells and the second set of location cells and identifies one or more objects by processing the displacement cells.
An inference system performs inference, such as object recognition, based on sensory inputs generated by sensors and control information associated with the sensory inputs. The sensory inputs describe one or more features of the objects. The control information describes movement of the sensors or known locations of the sensors relative to a reference point. For a particular object, an inference system learns a set of object-location representations of the object. An object-location representation is a unique characterization of an object-centric location relative to the particular object. The inference system also learns a set of feature-location representations associated with the object-location representation that indicate presence of features at the corresponding object-location pair. The inference system can perform inference on an unknown object by identifying candidate object-location representations consistent with feature-location representations observed from the sensory input data and control information.
G06V 10/44 - Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersectionsConnectivity analysis, e.g. of connected components
G06V 10/94 - Hardware or software architectures specially adapted for image or video understanding
G06V 20/80 - Recognising image objects characterised by unique random patterns
11.
Temporal processing scheme and sensorimotor information processing
Embodiments relate to a processing node in a temporal memory system that performs temporal pooling or processing by activating cells where the activation of a cell is maintained longer if the activation of the cell were previously predicted or activation on more than a certain portion of associated cells in a lower node was correctly predicted. An active cell correctly predicted to be activated or an active cell having connections to lower node active cells that were correctly predicted to become active contribute to accurate prediction, and hence, is maintained active longer than cells activated but were not previously predicted to become active. Embodiments also relate to a temporal memory system for detecting, learning, and predicting spatial patterns and temporal sequences in input data by using action information.
A hardware accelerator that is efficient at performing computations related to tensors. The hardware accelerator may store a complementary dense process tensor that is combined from a plurality of sparse process tensors. The plurality of sparse process tensors have non-overlapping locations of active values. The hardware accelerator may perform elementwise operations between the complementary dense process tensor and an activation tensor to generate a product tensor. The hardware accelerator may re-arrange the product tensor based on a permutation logic to separate the products into groups. Each group corresponds to one of the sparse process tensors. Each group may be accumulated separately to generate a plurality of output values. The output values may be selected in an activation selection. The activation selection may be a dense activation or a sparse activation such as k winner activation that set non-winners to zeros.
A hardware accelerator that is efficient at performing computations related to tensors. The hardware accelerator may store a complementary dense process tensor that is combined from a plurality of sparse process tensors. The plurality of sparse process tensors have non-overlapping locations of active values. The hardware accelerator may perform elementwise operations between the complementary dense process tensor and an activation tensor to generate a product tensor. The hardware accelerator may re-arrange the product tensor based on a permutation logic to separate the products into groups. Each group corresponds to one of the sparse process tensors. Each group may be accumulated separately to generate a plurality of output values. The output values may be selected in an activation selection. The activation selection may be a dense activation or a sparse activation such as k winner activation that set non-winners to zeros.
G06F 7/544 - Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using non-contact-making devices, e.g. tube, solid state deviceMethods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using unspecified devices for evaluating functions by calculation
A hardware accelerator that is efficient at performing computations related to tensors. The hardware accelerator may store a complementary dense process tensor that is combined from a plurality of sparse process tensors. The plurality of sparse process tensors have non-overlapping locations of active values. The hardware accelerator may perform elementwise operations between the complementary dense process tensor and an activation tensor to generate a product tensor. The hardware accelerator may re-arrange the product tensor based on a permutation logic to separate the products into groups. Each group corresponds to one of the sparse process tensors. Each group may be accumulated separately to generate a plurality of output values. The output values may be selected in an activation selection. The activation selection may be a dense activation or a sparse activation such as k winner activation that set non-winners to zeros.
A hardware accelerator that is efficient at performing computations related to tensors. The hardware accelerator may store a complementary dense process tensor that is combined from a plurality of sparse process tensors. The plurality of sparse process tensors have non-overlapping locations of active values. The hardware accelerator may perform elementwise operations between the complementary dense process tensor and an activation tensor to generate a product tensor. The hardware accelerator may re-arrange the product tensor based on a permutation logic to separate the products into groups. Each group corresponds to one of the sparse process tensors. Each group may be accumulated separately to generate a plurality of output values. The output values may be selected in an activation selection. The activation selection may be a dense activation or a sparse activation such as k winner activation that set non-winners to zeros.
A sparse neural network is trained such that weights or layer outputs of the neural network satisfy sparsity constraints. The sparsity is controlled by pruning one or more subsets of weights based on their signal-to-noise ratio (SNR). During the training process, an inference system generates outputs for a current layer by applying a set of weights for the current layer to a layer output of a previous layer. The set of weights for the current layer may be modeled as random variables sampled from probability distributions. The inference system determines a loss function and updates the set of weights by backpropagating error terms obtained from the loss function. This process is repeated until a convergence criterion is reached. One or more subsets of weights are then pruned based on their SNR depending on sparsity constraints for the weights of the neural network.
A hardware accelerator that is efficient at performing computations related to a sparse neural network. The sparse neural network may be associated with a plurality of nodes. An artificial intelligence (AI) accelerator stores, at a memory circuit, a weight tenor and an input activation tensor that corresponds to a node of the neural network. The AI accelerator performs a computation such as convolution between the weight tenor and the input activation tensor to generate an output activation tensor. The AI accelerator introduces sparsity to the output activation tensor by reducing the number of active values in the output activation tensor. The sparsity activation may be a K-winner approach, which selects the K-largest values in the output activation tensor and set the remaining values to zero.
G06N 3/063 - Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
G06F 7/08 - Sorting, i.e. grouping record carriers in numerical or other ordered sequence according to the classification of at least some of the information they carry
18.
Hardware architecture for processing data in sparse neural network
A hardware accelerator that is efficient at performing computations related to a sparse neural network. The sparse neural network may be associated with a plurality of nodes. One of the nodes includes one or more sparse tensors. The accelerator may compress the sparse tensor to a dense tensor. The sparse tensor may also be structured so that the dense locations in the tensor are blocked or partitioned. The accelerator may transpose the weight tensor and align the partitions of the tensor with the hardware architecture. The structured tensor has a balanced number of active values so that the active values can be processed by an efficient number of operating cycles of the accelerator. The accelerator may also perform bitwise and operation to determine the location of dense pairs in two sparse tensors to reduce the number of computations.
Embodiments relate to a first processing node that processes an input data having a temporal sequence of spatial patterns by retaining a higher-level context of the temporal sequence. The first processing node performs temporal processing based at least on feedback inputs received from a second processing node. The first processing node determines whether learned temporal sequences are included in the input data based on sequence inputs transmitted within the same level of a hierarchy of processing nodes and the feedback inputs received from an upper level of the hierarchy of processing nodes.
One or more multi-layer systems are used to perform inference. A multi-layer system may correspond to a node that receives a set of sensory input data for hierarchical processing, and may be grouped to perform processing for sensory input data. Inference systems at lower layers of a multi-layer system pass representation of objects to inference systems at higher layers. Each inference system can perform inference and form their own versions of representations of objects, regardless of the level and layer of the inference systems. The set of candidate objects for each inference system is updated to those consistent with feature-location representations for the sensors as well as object representations at lower layers. The set of candidate objects is also updated to those consistent with candidate objects from other inference systems, such as inference systems at other layers of the hierarchy or inference systems included in other multi-layer systems.
G06N 5/046 - Forward inferencingProduction systems
G06V 10/80 - Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
G06V 10/82 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
G06V 10/94 - Hardware or software architectures specially adapted for image or video understanding
21.
Inferencing and learning based on sensorimotor input data
Embodiments relate to performing inference, such as object recognition, based on sensory inputs received from sensors and location information associated with the sensory inputs. The sensory inputs describe one or more features of the objects. The location information describes known or potential locations of the sensors generating the sensory inputs. An inference system learns representations of objects by characterizing a plurality of feature-location representations of the objects, and then performs inference by identifying or updating candidate objects consistent with feature-location representations observed from the sensory input data and location information. In one instance, the inference system learns representations of objects for each sensor. The set of candidate objects for each sensor is updated to those consistent with candidate objects for other sensors, as well as the observed feature-location representations for the sensor.
An inference system trains and performs inference using a sparse neural network. The sparse neural network may include one or more layers, and each layer may be associated with a set of sparse weights that represent sparse connections between nodes of a layer and nodes of a previous layer. A layer output may be generated by applying the set of sparse weights associated with the layer to the layer output of a previous layer. Moreover, the one or more layers of the sparse neural network may generate sparse layer outputs. By using sparse representations of weights and layer outputs, robustness and stability of the neural network can be significantly improved, while maintaining competitive accuracy.
An inference system performs inference, such as object recognition, based on sensory inputs generated by sensors and control information associated with the sensory inputs. The sensory inputs describe one or more features of the objects. The control information describes movement of the sensors or known locations of the sensors relative to a reference point. For a particular object, an inference system learns a set of object-location representations of the object. An object-location representation is a unique characterization of an object-centric location relative to the particular object. The inference system also learns a set of feature-location representations associated with the object-location representation that indicate presence of features at the corresponding object-location pair. The inference system can perform inference on an unknown object by identifying candidate object-location representations consistent with feature-location representations observed from the sensory input data and control information.
G06V 10/44 - Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersectionsConnectivity analysis, e.g. of connected components
G06F 18/22 - Matching criteria, e.g. proximity measures
One or more multi-layer systems are used to perform inference. A multi-layer system may correspond to a node that receives a set of sensory input data for hierarchical processing, and may be grouped to perform processing for sensory input data. Inference systems at lower layers of a multi-layer system pass representation of objects to inference systems at higher layers. Each inference system can perform inference and form their own versions of representations of objects, regardless of the level and layer of the inference systems. The set of candidate objects for each inference system is updated to those consistent with feature- location representations for the sensors as well as object representations at lower layers. The set of candidate objects is also updated to those consistent with candidate objects from other inference systems, such as inference systems at other layers of the hierarchy or inference systems included in other multi-layer systems.
Embodiments relate to a first processing node that processes an input data having a temporal sequence of spatial patterns by retaining a higher-level context of the temporal sequence. The first processing node performs temporal processing based at least on feedback inputs received from a second processing node. The first processing node determines whether learned temporal sequences are included in the input data based on sequence inputs transmitted within the same level of a hierarchy of processing nodes and the feedback inputs received from an upper level of the hierarchy of processing nodes.
A processing node in a temporal memory system includes a spatial pooler and a sequence processor. The spatial pooler generates a spatial pooler signal representing similarity between received spatial patterns in an input signal and stored co-occurrence patterns. The spatial pooler signal is represented by a combination of elements that are active or inactive. Each co-occurrence pattern is mapped to different subsets of elements of an input signal. The spatial pooler signal is fed to a sequence processor receiving and processed to learn, recognize and predict temporal sequences in the input signal. The sequence processor includes one or more columns, each column including one or more cells. A subset of columns may be selected by the spatial pooler signal, causing one or more cells in these columns to activate.
Embodiments relate to a processing node in a temporal memory system that performs temporal pooling or processing by activating cells where the activation of a cell is maintained longer if the activation of the cell were previously predicted or activation on more than a certain portion of associated cells in a lower node was correctly predicted. An active cell correctly predicted to be activated or an active cell having connections to lower node active cells that were correctly predicted to become active contribute to accurate prediction, and hence, is maintained active longer than cells activated but were not previously predicted to become active. Embodiments also relate to a temporal memory system for detecting, learning, and predicting spatial patterns and temporal sequences in input data by using action information.
A processing node in a temporal memory system includes a spatial pooler and a sequence processor. The spatial pooler generates a spatial pooler signal representing similarity between received spatial patterns in an input signal and stored co-occurrence patterns. The spatial pooler signal is represented by a combination of elements that are active or inactive. Each co-occurrence pattern is mapped to different subsets of elements of an input signal. The spatial pooler signal is fed to a sequence processor receiving and processed to learn, recognize and predict temporal sequences in the input signal. The sequence processor includes one or more columns, each column including one or more cells. A subset of columns may be selected by the spatial pooler signal, causing one or more cells in these columns to activate.
One or more multi-layer systems are used to perform inference. A multi-layer system may correspond to a node that receives a set of sensory input data for hierarchical processing, and may be grouped to perform processing for sensory input data. Inference systems at lower layers of a multi-layer system pass representation of objects to inference systems at higher layers. Each inference system can perform inference and form their own versions of representations of objects, regardless of the level and layer of the inference systems. The set of candidate objects for each inference system is updated to those consistent with feature-location representations for the sensors as well as object representations at lower layers. The set of candidate objects is also updated to those consistent with candidate objects from other inference systems, such as inference systems at other layers of the hierarchy or inference systems included in other multi-layer systems.
An inference system performs inference, such as object recognition, based on sensory inputs generated by sensors and control information associated with the sensory inputs. The sensory inputs describe one or more features of the objects. The control information describes movement of the sensors or known locations of the sensors relative to a reference point. For a particular object, an inference system learns a set of object-location representations of the object. An object-location representation is a unique characterization of an object-centric location relative to the particular object. The inference system also learns a set of feature-location representations associated with the object-location representation that indicate presence of features at the corresponding object-location pair. The inference system can perform inference on an unknown object by identifying candidate object-location representations consistent with feature-location representations observed from the sensory input data and control information.
An inference system performs inference, such as object recognition, based on sensory inputs generated by sensors and control information associated with the sensory inputs. The sensory inputs describe one or more features of the objects. The control information describes movement of the sensors or known locations of the sensors relative to a reference point. For a particular object, an inference system learns a set of object-location representations of the object. An object-location representation is a unique characterization of an object-centric location relative to the particular object. The inference system also learns a set of feature-location representations associated with the object-location representation that indicate presence of features at the corresponding object-location pair. The inference system can perform inference on an unknown object by identifying candidate object-location representations consistent with feature-location representations observed from the sensory input data and control information.
G06F 15/18 - in which a program is changed according to experience gained by the computer itself during a complete run; Learning machines (adaptive control systems G05B 13/00;artificial intelligence G06N)
09 - Scientific and electric apparatus and instruments
42 - Scientific, technological and industrial services, research and design
45 - Legal and security services; personal services for individuals.
Goods & Services
Computer software related to hierarchically organized
computer memory systems used to solve problems in machine
vision, language understanding, robotics, and other problems
that require modeling of complex data; computer hardware
related to hierarchically organized computer memory systems
used to solve problems in machine vision, language
understanding, robotics, and other problems that require
modeling of complex data. Consulting services in the field of hierarchically organized
computer memory systems used to solve problems in machine
vision, language understanding, robotics, and other problems
that require modeling of complex data; time-based rental of
computer hardware in the field of hierarchically organized
computer memory systems; design and development of
hierarchically organized computer memory systems used to
solve problems in machine vision, language understanding,
robotics, and other problems that require modeling of
complex data. Licensing of intellectual property in the field of
hierarchically organized computer memory systems used to
solve problems in machine vision, language understanding,
robotics, and other problems that require modeling of
complex data.
33.
Inferencing and learning based on sensorimotor input data
Embodiments relate to performing inference, such as object recognition, based on sensory inputs received from sensors and location information associated with the sensory inputs. The sensory inputs describe one or more features of the objects. The location information describes known or potential locations of the sensors generating the sensory inputs. An inference system learns representations of objects by characterizing a plurality of feature-location representations of the objects, and then performs inference by identifying or updating candidate objects consistent with feature-location representations observed from the sensory input data and location information. In one instance, the inference system learns representations of objects for each sensor. The set of candidate objects for each sensor is updated to those consistent with candidate objects for other sensors, as well as the observed feature-location representations for the sensor.
Embodiments relate to performing inference, such as object recognition, based on sensory inputs received from sensors and location information associated with the sensory inputs. The sensory inputs describe one or more features of the objects. The location information describes known or potential locations of the sensors generating the sensory inputs. An inference system learns representations of objects by characterizing a plurality of feature-location representations of the objects, and then performs inference by identifying or updating candidate objects consistent with feature-location representations observed from the sensory input data and location information. In one instance, the inference system learns representations of objects for each sensor. The set of candidate objects for each sensor is updated to those consistent with candidate objects for other sensors, as well as the observed feature-location representations for the sensor.
Embodiments relate to a first processing node that processes an input data having a temporal sequence of spatial patterns by retaining a higher-level context of the temporal sequence. The first processing node performs temporal processing based at least on feedback inputs received from a second processing node. The first processing node determines whether learned temporal sequences are included in the input data based on sequence inputs transmitted within the same level of a hierarchy of processing nodes and the feedback inputs received from an upper level of the hierarchy of processing nodes.
Embodiments relate to a processing node of a hierarchical temporal memory (HTM) system with a union processor that enables a more stable representation of sequences by unionizing or pooling patterns of a temporal sequence. The union processor biases the HTM system so a learned temporal sequence may be more quickly recognized. The union processor includes union elements that are associated with incoming spatial patterns or with cells that represent temporal relationships between the spatial patterns. A union element of the union processor may be activated if a persistence score of the union element satisfies a predetermined criterion. The persistence score of the detector is updated based on the activation states of the spatial patterns or cells associated with the detector. After activation, the union element remains active for a period longer than a time step for performing the spatial pooling.
09 - Scientific and electric apparatus and instruments
42 - Scientific, technological and industrial services, research and design
45 - Legal and security services; personal services for individuals.
Goods & Services
Computer software related to hierarchically organized computer memory systems used to solve problems in machine vision, language understanding, robotics, and other problems that require modeling of complex data; Computer hardware related to hierarchically organized computer memory systems used to solve problems in machine vision, language understanding, robotics, and other problems that require modeling of complex data Consulting services in the field of hierarchically organized computer memory systems used to solve problems in machine vision, language understanding, robotics, and other problems that require modeling of complex data; Time-based rental of computer hardware in the field of hierarchically organized computer memory systems; Design and development of hierarchically organized computer memory systems used to solve problems in machine vision, language understanding, robotics, and other problems that require modeling of complex data Licensing of intellectual property in the field of hierarchically organized computer memory systems used to solve problems in machine vision, language understanding, robotics, and other problems that require modeling of complex data
38.
Anomaly detection in spatial and temporal memory system
Detecting patterns and sequences associated with an anomaly in predictions made a predictive system. The predictive system makes predictions by learning spatial patterns and temporal sequences in an input data that change over time. As the input data is received, the predictive system generates a series of predictions based on the input data. Each prediction is compared with corresponding actual value or state. If the prediction does not match or deviates significantly from the actual value or state, an anomaly is identified for further analysis. A corresponding state or a series of states of the predictive system before or at the time of prediction are associated with the anomaly and stored. The anomaly can be detected by monitoring whether the predictive system is placed in the state or states that is the same or similar to the stored state or states.
A processing node in a temporal memory system includes a spatial pooler and a sequence processor. The spatial pooler generates a spatial pooler signal representing similarity between received spatial patterns in an input signal and stored co-occurrence patterns. The spatial pooler signal is represented by a combination of elements that are active or inactive. Each co-occurrence pattern is mapped to different subsets of elements of an input signal. The spatial pooler signal is fed to a sequence processor receiving and processed to learn, recognize and predict temporal sequences in the input signal. The sequence processor includes one or more columns, each column including one or more cells. A subset of columns may be selected by the spatial pooler signal, causing one or more cells in these columns to activate.
Embodiments relate to a processing node in a temporal memory system that performs temporal pooling or processing by activating cells where the activation of a cell is maintained longer if the activation of the cell were previously predicted or activation on more than a certain portion of associated cells in a lower node was correctly predicted. An active cell correctly predicted to be activated or an active cell having connections to lower node active cells that were correctly predicted to become active contribute to accurate prediction, and hence, is maintained active longer than cells activated but were not previously predicted to become active. Embodiments also relate to a temporal memory system for detecting, learning, and predicting spatial patterns and temporal sequences in input data by using action information.
A spatial and temporal memory system (STMS) processes input data to detect whether spatial patterns and/or temporal sequences of spatial patterns exist within the data, and to make predictions about future data. The data processed by the STMS may be retrieved from, for example, one or more database fields and is encoded into a distributed representation format using a coding scheme. The performance of the STMS in predicting future data is evaluated for the coding scheme used to process the data as performance data. The selection and prioritization of STMS experiments to perform may be based on the performance data for an experiment. The best fields, encodings, and time aggregations for generating predictions can be determined by an automated search and evaluation of multiple STMS systems.
A web-based hierarchical temporal memory (HTM) system in which one or more client devices communicate with a remote server via a communication network. The remote server includes at least a HTM server for implementing a hierarchical temporal memory (HTM). The client devices generate input data including patterns and sequences, and send the input data to the remote server for processing. The remote server (specifically, the HTM server) performs processing in order to determine the causes of the input data, and sends the results of this processing to the client devices. The client devices need not have processing and/or storage capability for running the HTM but may nevertheless take advantage of the HTM by submitting a request to the HTM server.
Embodiments relate to making predictions for values or states to follow multiple time steps after receiving a certain input data in a spatial and temporal memory system. During a training stage, relationships between states of the spatial and temporal memory system at certain times and spatial patterns of the input data detected a plurality of time steps later after the certain time steps are established. Using the established relationships, the spatial and temporal memory system can make predictions multiple time steps into the future based on the input data received at a current time.
Detecting patterns and sequences associated with an anomaly in predictions made a predictive system. The predictive system makes predictions by learning spatial patterns and temporal sequences in an input data that change over time. As the input data is received, the predictive system generates a series of predictions based on the input data. Each prediction is compared with corresponding actual value or state. If the prediction does not match or deviates significantly from the actual value or state, an anomaly is identified for further analysis. A corresponding state or a series of states of the predictive system before or at the time of prediction are associated with the anomaly and stored. The anomaly can be detected by monitoring whether the predictive system is placed in the state or states that is the same or similar to the stored state or states.
A spatial and temporal memory system (STMS) processes input data to detect whether spatial patterns and/or temporal sequences of spatial patterns exist within the data, and to make predictions about future data. The data processed by the STMS may be retrieved from, for example, one or more database fields and is encoded into a distributed representation format using a coding scheme. The performance of the STMS in predicting future data is evaluated for the coding scheme used to process the data as performance data. The selection and prioritization of STMS experiments to perform may be based on the performance data for an experiment. The best fields, encodings, and time aggregations for generating predictions can be determined by an automated search and evaluation of multiple STMS systems.
G06F 15/18 - in which a program is changed according to experience gained by the computer itself during a complete run; Learning machines (adaptive control systems G05B 13/00;artificial intelligence G06N)
G06F 17/30 - Information retrieval; Database structures therefor
G06F 15/00 - Digital computers in generalData processing equipment in general
G06F 13/12 - Program control for peripheral devices using hardware independent of the central processor, e.g. channel or peripheral processor
G06N 99/00 - Subject matter not provided for in other groups of this subclass
G06F 9/38 - Concurrent instruction execution, e.g. pipeline or look ahead
A spatial and temporal memory system (STMS) processes input data to detect whether spatial patterns and/or temporal sequences of spatial patterns exist within the data, and to make predictions about future data. The data processed by the STMS may be retrieved from, for example, one or more database fields and is encoded into a distributed representation format using a coding scheme. The performance of the STMS in predicting future data is evaluated for the coding scheme used to process the data as performance data. The selection and prioritization of STMS experiments to perform may be based on the performance data for an experiment. The best fields, encodings, and time aggregations for generating predictions can be determined by an automated search and evaluation of multiple STMS systems.
A spatial and temporal memory system (STMS) processes input data to detect whether spatial patterns and/or temporal sequences of spatial patterns exist within the data, and to make predictions about future data. The data processed by the STMS may be retrieved from, for example, one or more database fields and is encoded into a distributed representation format using a coding scheme. The performance of the STMS in predicting future data is evaluated for the coding scheme used to process the data as performance data. The selection and prioritization of STMS experiments to perform may be based on the performance data for an experiment. The best fields, encodings, and time aggregations for generating predictions can be determined by an automated search and evaluation of multiple STMS systems.
G06F 15/18 - in which a program is changed according to experience gained by the computer itself during a complete run; Learning machines (adaptive control systems G05B 13/00;artificial intelligence G06N)
09 - Scientific and electric apparatus and instruments
42 - Scientific, technological and industrial services, research and design
45 - Legal and security services; personal services for individuals.
Goods & Services
Computer software related to hierarchically organized
computer memory systems used to solve problems in machine
vision, language understanding, robotics, and other problems
that require modeling of complex data; computer hardware
related to hierarchically organized computer memory systems
used to solve problems in machine vision, language
understanding, robotics, and other problems that require
modeling of complex data. Consulting services in the field of hierarchically organized
computer memory systems used to solve problems in machine
vision, language understanding, robotics, and other problems
that require modeling of complex data; time-based rental of
computer hardware in the field of hierarchically organized
computer memory systems; design and development of
hierarchically organized computer memory systems used to
solve problems in machine vision, language understanding,
robotics, and other problems that require modeling of
complex data. Licensing of intellectual property in the field of
hierarchically organized computer memory systems used to
solve problems in machine vision, language understanding,
robotics, and other problems that require modeling of
complex data.
49.
Methods, architecture, and apparatus for implementing machine intelligence and hierarchical memory systems
Sophisticated memory systems and intelligent machines may be constructed by creating an active memory system with a hierarchical architecture. Specifically, a system may comprise a plurality of individual cortical processing units arranged into a hierarchical structure. Each individual cortical processing unit receives a sequence of patterns as input. Each cortical processing unit processes the received input sequence of patterns using a memory containing previously encountered sequences with structure and outputs another pattern. As several input sequences are processed by a cortical processing unit, it will therefore generate a sequence of patterns on its output. The sequence of patterns on its output may be passed as an input to one or more cortical processing units in next higher layer of the hierarchy. A lowest layer of cortical processing units may receive sensory input from the outside world. The sensory input also comprises a sequence of patterns.
A web-based hierarchical temporal memory (HTM) system in which one or more client devices communicate with a remote server via a communication network. The remote server includes at least a HTM server for implementing a hierarchical temporal memory (HTM). The client devices generate input data including patterns and sequences, and send the input data to the remote server for processing. The remote server (specifically, the HTM server) performs processing in order to determine the causes of the input data, and sends the results of this processing to the client devices. The client devices need not have processing and/or storage capability for running the HTM but may nevertheless take advantage of the HTM by submitting a request to the HTM server.
A spatio-temporal learning node is a type of HTM node which learns both spatial and temporal groups of sensed input patterns over time. Spatio-temporal learning nodes comprise spatial poolers which are used to determine spatial groups in a set of sensed input patterns. The spatio-temporal learning nodes further comprise temporal poolers which are used to determine groups of sensed input patterns that temporally co-occur. A spatio-temporal learning network is a hierarchical network including a plurality of spatio-temporal learning nodes.
G06F 15/18 - in which a program is changed according to experience gained by the computer itself during a complete run; Learning machines (adaptive control systems G05B 13/00;artificial intelligence G06N)
G06F 9/44 - Arrangements for executing specific programs
G06N 7/02 - Computing arrangements based on specific mathematical models using fuzzy logic
G06N 7/06 - Simulation on general purpose computers
52.
Feedback in group based hierarchical temporal memory system
A Hierarchical Temporal Memory (HTM) network has at least first nodes and a second node at a higher level than the first nodes. The second node provides an inter-node feedback signal to the first nodes for grouping patterns and sequences (or co-occurrences) in input data received at the first nodes at the first nodes. The second node collects forward signals from the first nodes; and thus, the second node has information about the grouping of the patterns and sequences (or co-occurrences) at the first nodes. The second node provides inter-node feedback signals to the first nodes based on which the first nodes may perform the grouping of the patterns and sequences (or co-occurrences) at the first nodes.
G06F 15/18 - in which a program is changed according to experience gained by the computer itself during a complete run; Learning machines (adaptive control systems G05B 13/00;artificial intelligence G06N)
53.
TEMPORAL MEMORY USING SPARSE DISTRIBUTED REPRESENTATION
A processing node in a temporal memory system includes a spatial pooler and a sequence processor. The spatial pooler generates a spatial pooler signal representing similarity between received spatial patterns in an input signal and stored co-occurrence patterns. The spatial pooler signal is represented by a combination of elements that are active or inactive. Each co-occurrence pattern is mapped to different subsets of elements of an input signal. The spatial pooler signal is fed to a sequence processor receiving and processed to learn, recognize and predict temporal sequences in the input signal. The sequence processor includes one or more columns, each column including one or more cells. A subset of columns may be selected by the spatial pooler signal, causing one or more cells in these columns to activate.
A processing node in a temporal memory system includes a spatial pooler and a sequence processor. The spatial pooler generates a spatial pooler signal representing similarity between received spatial patterns in an input signal and stored co-occurrence patterns. The spatial pooler signal is represented by a combination of elements that are active or inactive. Each co-occurrence pattern is mapped to different subsets of elements of an input signal. The spatial pooler signal is fed to a sequence processor receiving and processed to learn, recognize and predict temporal sequences in the input signal. The sequence processor includes one or more columns, each column including one or more cells. A subset of columns may be selected by the spatial pooler signal, causing one or more cells in these columns to activate.
A HTM network that uses supervision signals such as indexes for correct categories of the input patterns to group the co-occurrences detected in the node. In the training mode, the supervised learning node receives the supervision signals in addition to the indexes or distributions from children nodes. The supervision signal is then used to assign the co-occurrences into groups. The groups include unique groups and nonunique groups. The co-occurrences in the unique group appear only when the input data represent certain category but not others. The nonunique groups include patterns that are shared by one or more categories. In an inference mode, the supervised learning node generates distributions over the groups created in the training mode. A top node of the HTM network generates an output based on the distributions generated by the supervised learning node.
G06F 15/18 - in which a program is changed according to experience gained by the computer itself during a complete run; Learning machines (adaptive control systems G05B 13/00;artificial intelligence G06N)
56.
Sequence learning in a hierarchical temporal memory based system
A hierarchy of computing modules is configured to learn a cause of input data sensed over space and time, and is further configured to determine a cause of novel sensed input data dependent on the learned cause. At least one of the computing modules has a sequence learner module configured to associate sequences of input data received by the computing module to a set of causes previously learned in the hierarchy.
A temporal pooler for a Hierarchical Temporal Memory network is provided. The temporal pooler is capable of storing information about sequences of co-occurrences in a higher-order Markov chain by splitting a co-occurrence into a plurality of sub-occurrences. Each split sub-occurrence may be part of a distinct sequence of co-occurrences. The temporal pooler receives the probability of spatial co-occurrences in training patterns and tallies counts or frequency of transitions from one sub-occurrence to another sub-occurrence in a connectivity matrix. The connectivity matrix is then processed to generate temporal statistics data. The temporal statistics data is provided to an inference engine to perform inference or prediction on input patterns. By storing information related to a higher-order Markov model, the temporal statistics data more accurately reflects long temporal sequences of co-occurrences in the training patterns.
G06F 15/18 - in which a program is changed according to experience gained by the computer itself during a complete run; Learning machines (adaptive control systems G05B 13/00;artificial intelligence G06N)
A temporal pooler for a Hierarchical Temporal Memory network is provided. The temporal pooler is capable of storing information about sequences of co-occurrences in a higher-order Markov chain by splitting a co-occurrence into a plurality of sub-occurrences. Each split sub-occurrence may be part of a distinct sequence of co-occurrences. The temporal pooler receives the probability of spatial co-occurrences in training patterns and tallies counts or frequency of transitions from one sub -occurrence to another sub-occurrence in a connectivity matrix. The connectivity matrix is then processed to generate temporal statistics data. The temporal statistics data is provided to an inference engine to perform inference or prediction on input patterns. By storing information related to a higher-order Markov model, the temporal statistics data more accurately reflects long temporal sequences of co-occurrences in the training patterns.
A Hierarchical Temporal Memory (HTM) network has at least first nodes and a second node at a higher level than the first nodes. The second node provides an inter-node feedback signal to the first nodes for grouping patterns and sequences (or co-occurrences) in input data received at the first nodes at the first nodes. The second node collects forward signals from the first nodes; and thus, the second node has information about the grouping of the patterns and sequences (or co-occurrences) at the first nodes. The second node provides inter-node feedback signals to the first nodes based on which the first nodes may perform the grouping of the patterns and sequences (or co-occurrences) at the first nodes. Also, a node in a Hierarchical Temporal Memory (HTM) network comprising a co-occurrence detector and a group learner coupled to the co-occurrence detector. The group learner provides an intra-node feedback signal to the co-occurrence detector including information on the grouping of the co-occurrences. The co-occurrence detector may select co-occurrences to be split, merged, retained or discarded based on the intra-node feedback signals.
G06F 15/18 - in which a program is changed according to experience gained by the computer itself during a complete run; Learning machines (adaptive control systems G05B 13/00;artificial intelligence G06N)
60.
FEEDBACK IN GROUP BASED HIERARCHICAL TEMPORAL MEMORY SYSTEM
A Hierarchical Temporal Memory (HTM) network has at least first nodes and a second node at a higher level than the first nodes. The second node provides an inter-node feedback signal to the first nodes for grouping patterns and sequences (or co-occurrences) in input data received at the first nodes at the first nodes. The second node collects forward signals from the first nodes; and thus, the second node has information about the grouping of the patterns and sequences (or co-occurrences) at the first nodes. The second node provides inter-node feedback signals to the first nodes based on which the first nodes may perform the grouping of the patterns and sequences (or co-occurrences) at the first nodes. Also, a node in a Hierarchical Temporal Memory (HTM) network comprising a co-occurrence detector and a group learner coupled to the co-occurrence detector. The group learner provides an intra- node feedback signal to the co-occurrence detector including information on the grouping of the co-occurrences. The co-occurrence detector may select co-occurrences to be split, merged, retained or discarded based on the intra-node feedback signals.
A set of sequences of sensed input patterns associated with a set of actions is generated by performing at least a first action on data derived from a real-world system. A subset of the sequences of sensed input patterns that form a group associated with the first action is determined. A new sequence of sensed input patterns is received. A first value which indicates the probability that the new sequence of sensed input patterns is associated with the first action based on the subset of sequences of sensed input patterns is determined and stored in a memory associated with the computer system.
G06F 15/18 - in which a program is changed according to experience gained by the computer itself during a complete run; Learning machines (adaptive control systems G05B 13/00;artificial intelligence G06N)
A system and method for automatically analyzing data streams in a hierarchical and temporal network to identify node positions and the network topology in order to generate a hierarchical model of the temporal or spatial data. The system and method receives data streams, identifies a correlation between the data streams, partitions/clusters the data streams based upon the identified correlation and forms a current level of a hierarchical temporal network by having each cluster of data streams be an input to a hierarchical temporal network node. After training the nodes, each of the nodes creates a new data stream and these data streams are correlated and partitioned/clustered and are input into a node at a next level. The process can repeat until a desired portion of the network topology is determined.
A node, a computer program storage medium, and a method for a hierarchical temporal memory (HTM) network where at least one of its nodes generates a top-down message and sends the top-down message to one or more children nodes in the HTM network. The first top-down message represents information about the state of a node and functions as feedback information from a current node to its child node. The node may also maintain history of the input patterns or co-occurrences so that temporal relationships between input patterns or co-occurrences may be taken into account in an inference stage. By providing the top-town message and maintaining history of previous input patterns, the HTM network may, among others, (i) perform more accurate inference based on temporal history, (ii) make predictions, (iii) discriminate between spatial co-occurrences with different temporal histories, (iv) detect 'surprising' temporal patterns, (v) generate examples from a category, and (vi) fill in missing or occluded data.
A node, a computer program storage medium, and a method for a hierarchical temporal memory (HTM) network where at least one of its nodes generates a top-down message and sends the top-down message to one or more children nodes in the HTM network. The first top-down message represents information about the state of a node and functions as feedback information from a current node to its child node. The node may also maintain history of the input patterns or co-occurrences so that temporal relationships between input patterns or co-occurrences may be taken into account in an inference stage. By providing the top-town message and maintaining history of previous input patterns, the HTM network may, among others, (i) perform more accurate inference based on temporal history, (ii) make predictions, (iii) discriminate between spatial co-occurrences with different temporal histories, (iv) detect “surprising” temporal patterns, (v) generate examples from a category, and (vi) fill in missing or occluded data.
G06F 15/18 - in which a program is changed according to experience gained by the computer itself during a complete run; Learning machines (adaptive control systems G05B 13/00;artificial intelligence G06N)
G06F 9/44 - Arrangements for executing specific programs
G06N 7/02 - Computing arrangements based on specific mathematical models using fuzzy logic
G06N 7/06 - Simulation on general purpose computers
65.
HIERARCHICAL TEMPORAL MEMORY (HTM) SYSTEM DEPLOYED AS WEB SERVICE
A web-based hierarchical temporal memory (HTM) system (20) in which one or more client devices (22a, 22b, 22c) communicate with a remote server (24) via a communication network (26). The remote server (24) includes at least a HTM server (29) for implementing a hierarchical temporal memory (HTM). The client devices (22a, 22b, 22c) generate input data including patterns and sequences, and send the input data to the remote server (24) for processing. The remote server (24) (specifically, the HTM server) (29) performs processing in order to determine the causes of the input data, and sends the results of this processing to the client devices (22a, 22b, 22c). The client devices (22a, 22b, 22c) need not have processing and/or storage capability for running the HTM but may nevertheless take advantage of the HTM by submitting a request to the HTM server (29).
A spatio-temporal learning node is a type of HTM node which learns both spatial and temporal groups of sensed input patterns over time. Spatio-temporal learning nodes comprise spatial poolers which are used to determine spatial groups in a set of sensed input patterns. The spatio-temporal learning nodes further comprise temporal poolers which are used to determine groups of sensed input patterns that temporally co-occur. A spatio-temporal learning network is a hierarchical network including a plurality of spatio-temporal learning nodes.
A hierarchy of computing modules is configured to (i) learn a cause of input data sensed over space and time, and (ii) determine a cause of novel sensed input data dependent on the learned cause. When determining the cause of the novel sensed input data, the computing modules determine likely sequences based on observed inputs. Information identifying one or more of those likely sequences and indexes of observed elements in those sequences may then be stored in external memory to facilitate data compression and/or granularity-based searches.
A spatio-temporal learning node is a type of HTM node which learns both spatial and temporal groups of sensed input patterns over time. Spatio-temporal learning nodes comprise spatial poolers which are used to determine spatial groups in a set of sensed input patterns. The spatio-temporal learning nodes further comprise temporal poolers which are used to determine groups of sensed input patterns that temporally co-occur. A spatio-temporal learning network is a hierarchical network including a plurality of spatio-temporal learning nodes.
G06F 15/00 - Digital computers in generalData processing equipment in general
G06F 15/18 - in which a program is changed according to experience gained by the computer itself during a complete run; Learning machines (adaptive control systems G05B 13/00;artificial intelligence G06N)
69.
Episodic memory with a hierarchical temporal memory based system
A hierarchy of computing modules is configured to (i) learn a cause of input data sensed over space and time, and (ii) determine a cause of novel sensed input data dependent on the learned cause. When determining the cause of the novel sensed input data, the computing modules determine likely sequences based on observed inputs. Information identifying one or more of those likely sequences and indexes of observed elements in those sequences may then be stored in external memory to facilitate data compression and/or granularity-based searches.
G06F 15/18 - in which a program is changed according to experience gained by the computer itself during a complete run; Learning machines (adaptive control systems G05B 13/00;artificial intelligence G06N)
70.
Methods, architecture, and apparatus for implementing machine intelligence and hierarchical memory systems
Sophisticated memory systems and intelligent machines may be constructed by creating an active memory system with a hierarchical architecture. Specifically, a system may comprise a plurality of individual cortical processing units arranged into a hierarchical structure. Each individual cortical processing unit receives a sequence of patterns as input. Each cortical processing unit processes the received input sequence of patterns using a memory containing previously encountered sequences with structure and outputs another pattern. As several input sequences are processed by a cortical processing unit, it will therefore generate a sequence of patterns on its output. The sequence of patterns on its output may be passed as an input to one or more cortical processing units in next higher layer of the hierarchy. A lowest layer of cortical processing units may receive sensory input from the outside world. The sensory input also comprises a sequence of patterns.
G06F 15/18 - in which a program is changed according to experience gained by the computer itself during a complete run; Learning machines (adaptive control systems G05B 13/00;artificial intelligence G06N)
71.
Architecture of a hierarchical temporal memory based system
A hierarchical temporal memory (HTM) based system may be provided as a software platform. The software platform includes: a runtime engine arranged to run an HTM network; a first interface accessible by a set of tools to configure, design, modify, train, debug, and/or deploy the HTM network; and a second interface accessible to extend a functionality of the runtime engine.
G06F 15/18 - in which a program is changed according to experience gained by the computer itself during a complete run; Learning machines (adaptive control systems G05B 13/00;artificial intelligence G06N)
G06F 9/44 - Arrangements for executing specific programs
An HTM node learns a plurality of groups of sensed input patterns over time based on the frequency of temporal adjacency of the input patterns. An HTM node receives a new sensed input, the HTM node assigns probabilities as to the likelihood that the new sensed input matches each of the plurality of learned groups. The HTM node then combines this probability distribution (may be normalized) with previous state information to assign probabilities as to the likelihood that the new sensed input is part of each of the learned groups of the HTM node. Then, as described above, the distribution over the set of groups learned by the HTM node is passed to a higher level node. This process is repeated at higher level nodes to infer a cause of the newly sensed input.
A hierarchy of computing modules is configured to (i) learn a cause of input data sensed over space and time, and (ii) determine a cause of novel sensed input data dependent on the learned cause. The hierarchy has a first level of computing modules and a second level of at least one computing module, wherein a computing module in the first level is configured to output to the computing module in the second level a first set of values representing probabilities of possible causes of input data received by the system.
A hierarchy of computing modules is configured to learn a cause of input data sensed over space and time, and is further configured to determine a cause of novel sensed input data dependent on the learned cause. Further, the hierarchy has a first level of computing modules and a second level of at least one computing module, where at least one of the computing modules in the first level operates on a first server, and where the at least one computing module in the second level operates on a second server. The hierarchy also includes a message manager module configured to relay information between the first server and the second server.
09 - Scientific and electric apparatus and instruments
45 - Legal and security services; personal services for individuals.
Goods & Services
Computer software related to hierarchically organized computer memory systems used to solve problems in machine vision, language understanding, robotics, and other problems that require modeling of complex data; Computer hardware related to hierarchically organized computer memory systems used to solve problems in machine vision, language understanding, robotics, and other problems that require modeling of complex data Licensing of intellectual property in the field of hierarchically organized computer memory systems used to solve problems in machine vision, language understanding, robotics, and other problems that require modeling of complex data
42 - Scientific, technological and industrial services, research and design
Goods & Services
Consulting services in the field of hierarchically organized computer memory systems used to solve problems in machine vision, language understanding, robotics, and other problems that require modeling of complex data; Time-based rental of computer hardware in the field of hierarchically organized computer memory systems; Design and development of hierarchically organized computer memory systems used to solve problems in machine vision, language understanding, robotics, and other problems that require modeling of complex data; and Licensing of intellectual property in the field of hierarchically organized computer memory systems used to solve problems in machine vision, language understanding, robotics, and other problems that require modeling of complex data
09 - Scientific and electric apparatus and instruments
Goods & Services
Computer software related to hierarchically organized computer memory systems used to solve problems in machine vision, language understanding, robotics, and other problems that require modeling of complex data; Computer hardware related to hierarchically organized computer memory systems used to solve problems in machine vision, language understanding, robotics, and other problems that require modeling of complex data
