A computer-implemented method of generating road annotation data for annotating an electronic map, the method comprising accessing from persistent storage an electronic map defining a road network; generating a road topology graph encoding a topology of the road network, the road topology graph comprising nodes representing road structure elements and edges representing links between road structure elements; performing a search of the road topology graph for a predetermined graph structure; responsive to identifying a subgraph of the road topology graph exhibiting the predetermined graph structure, generating annotation data for marking in the electronic map a portion of the road network corresponding to the subgraph; and generating in persistent storage an augmented map comprising map data defining the portion of the road network and the annotation data.
A computer-implemented method of planning an ego trajectory for an ego robot in an environment in the presence of an occluding object, the method comprising: determining a trajectory cost function having one or more planning variables that define a planned ego trajectory, the planning variables tuneable to modify the planned ego trajectory, wherein the trajectory cost function is dependent on a predicted occluded region of the environment, as defined by the planned ego trajectory and a predicted state of the occluding object; and tuning the planning variables using an optimizer applied to the trajectory cost function, wherein the trajectory cost function encodes: (i) a progress objective that rewards modifications to the planned ego trajectory that progress the ego robot towards a chosen goal location, and (ii) a visibility objective, dependent on the predicted occluded region, that rewards modifications to the planned ego trajectory that improve visibility of the environment from the perspective of the ego robot, whereby the optimiser is incentivised to tune the planning variables in a manner satisfying the progress objective and the visibility objective.
Systems and method are provided for a driving run performed by a sensor-equipped robot in a driving scene comprising at least one dynamic signalling agent (DSA). DSA data indicating, signalling states of the at least one DSA as a function of time is received, and a graphical user interface (GUI), comprising a schematic representation of the run and at least one DSA state timeline showing a visual indicator of the current signalling state of a corresponding one of the at least one DSA, is rendered on the GUI.
The present disclosure relates to a computer-implemented method of training a generative model to insert an object in spatial sensor data. The method comprises receiving a training sample of spatial sensor data and receiving an indication of a 3D geometric property of an object captured in the training sample. A portion of spatial sensor data corresponding to the object is removed from the training sample, resulting a cropped training sample. The generative model is trained to reconstruct the training sample from the cropped training sample by, providing to the generative model: the cropped training sample as a target input, an indication of the object as a reference input, and the 3D geometric property of the object as a conditioning input. This results in a generated output sample of spatial sensor data. Parameters of the generative model are tuned to reduce a reconstruction error between the training sample and the generated output sample, resulting in a trained generative model configured to insert at inference, in a set of spatial sensor data received as a target input, an object indicated by a reference input with a desired 3D geometric property indicated by a conditioning input.
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
A method of simulating behaviour of an autonomous vehicle to test a control stack for controlling the autonomous vehicle, the method comprising: providing to a simulator a scenario in which an ego vehicle is under observation; initiating operation of the simulator to execute the scenario and simulating an initial phase of a simulation of the scenario with the ego vehicle under the control of a first control stack implemented by the simulator; detecting that a handover condition is met in the scenario; and relinquishing control of the ego vehicle by the first control stack and causing a second control stack to be engaged to control the ego vehicle in the subsequent phase of the simulation, wherein the second control stack is the control stack under test.
A computer-implemented method of assessing performance of perception component, the perception component for interpreting structure in a scene comprises: receiving a set of multiple computed outputs obtained by applying the perception component to the scene, wherein each computed output comprises a confidence score: generating, from the set of multiple computed outputs, multiple pseudo-ground truth sets, wherein each pseudo-ground truth set comprises, for each computed output, a pseudo-ground truth output sampled from a set of possible ground truth outputs based on a probability distribution defined by the confidence score of the computed output; computing a performance score for the perception component applied to the scene with respect to each pseudo-ground truth set, by comparing the set of multiple outputs with that pseudo-ground truth set; and computing an overall performance score for the perception component applied to the scene, by aggregating the performance scores computed with respect to the multiple pseudo-ground truth sets.
G06V 10/776 - ValidationÉvaluation des performances
G06V 10/764 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant la classification, p. ex. des objets vidéo
G06V 10/774 - Génération d'ensembles de motifs de formationTraitement des caractéristiques d’images ou de vidéos dans les espaces de caractéristiquesDispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant l’intégration et la réduction de données, p. ex. analyse en composantes principales [PCA] ou analyse en composantes indépendantes [ ICA] ou cartes auto-organisatrices [SOM]Séparation aveugle de source méthodes de Bootstrap, p. ex. "bagging” ou “boosting”
G06V 10/98 - Détection ou correction d’erreurs, p. ex. en effectuant une deuxième exploration du motif ou par intervention humaineÉvaluation de la qualité des motifs acquis
The present disclosure relates to techniques for locating and modelling a 3D object captured by a mobile robot. A cost function is defined over a set of variables, and is applied to sensor data. The set of variables comprises shape parameters of a 3D object model and a time sequence of poses of the 3D object model. The cost function penalizes inconsistency between the sensor data and the set of variables. The object belongs to a known object class, and the 3D object model or the cost function encodes expected 3D shape information associated with the known object class. The 3D object is modelled by tuning poses of the object and the shape parameters, to optimize the cost function. A visualization of a location of the robot and an object shape representing the 3D object is rendered in a graphical user interface (GUI)
G06T 7/70 - Détermination de la position ou de l'orientation des objets ou des caméras
G06F 3/04815 - Interaction s’effectuant dans un environnement basé sur des métaphores ou des objets avec un affichage tridimensionnel, p. ex. modification du point de vue de l’utilisateur par rapport à l’environnement ou l’objet
G06F 3/0484 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] pour la commande de fonctions ou d’opérations spécifiques, p. ex. sélection ou transformation d’un objet, d’une image ou d’un élément de texte affiché, détermination d’une valeur de paramètre ou sélection d’une plage de valeurs
The present disclosure relates to techniques for training a generative model to insert an object in spatial sensor data. A first training sample of spatial sensor data of a first sensor modality, and a second training sample of spatial sensor data of a second sensor modality are received, the first training sample and the second training sample capture a common object. A first portion of sensor data corresponding to the object is removed from the first training sample, resulting a cropped training sample. A second portion of spatial sensor data corresponding to the common object is extracted from the second training sample. The generative model is trained to reconstruct the first training sample from the cropped training sample by: providing to the generative model: the cropped training sample as a target input, and the second portion of spatial sensor data as a reference input, resulting in a generated output sample of spatial sensor data, and tuning parameters of the generative model to reduce a reconstruction error between the first training sample and the generated output sample. This results in a trained generative model configured to insert at inference, in a first set of spatial sensor data of the first modality received as a target input, an object indicated in a second set of spatial sensor data of the second modality received as a reference input.
The present application relates to techniques for debugging a scenario. A scenario including a static layer and a dynamic layer is received. The static layer includes a road layout defined in a map file, and the dynamic layer defines an intended dynamic behaviour an agent. A visualization of the scenario is rendered, and a behaviour of the agent which deviates from the intended behaviour is exhibited. A user input to an editing interface is received, requesting generation of a scene structure layer that provides a visual indication map file features for the static layer. The scene structure layer is generated and overlaid on visualization. The overlay indicates that the static layer has a map file feature causing a map bug in the scenario. A second user input is received, altering a scenario parameter pertaining to the behaviour of the. Altering the parameter results in a redefined dynamic behaviour of the agent, and removes the map bug from the scenario.
A method of predicting trajectories for agents of a scenario, the method comprising, for each agent generating an agent feature vector based on one or more observed past states of the agent, computing a set of pairwise feature vectors, each computed as a combination of the agent feature vector for that agent with a respective agent feature vector generated for each other agent of the scenario, processing the pairwise feature vectors as independent inputs to one or more interaction layers of a trajectory prediction neural network to generate a pairwise output for each pairwise feature vector, aggregating the pairwise outputs over the other agents of the scenario to generate an interaction-based feature representation for each agent, processing the interaction-based feature representation in one or more prediction layers of the trajectory prediction neural network, and generating, based on the output of the one or more prediction layers, at least one predicted trajectory for each agent.
A computer-implemented method of planning actions for a mobile robot in the presence of occlusion The method comprises determining a first occluded region in a mobile robot field of view at a first time; sampling a first set of agent particles within the first occluded region; planning, using a robotic planner, based on a location and a motion state of a first agent particle of the first set of agent particles a first mobile robot action, the first mobile robot action accounting for predicted motion of the first agent particle; generating a first mobile robot control signal for causing execution of the first mobile robot action; determining a second occluded region in the mobile robot field of view at a second time; computing a propagated set of agent particles for the second time based on the first set of agent particles and a particle motion model; assigning an occlusion weight to each agent particle of the propagated set of agent particles with respect to the second occluded region; sampling a second set of agent particles within the second occluded region based on the propagated set of agent particles and the occlusion weight assigned to each agent particle; planning, using the robotic planner, based on a location and a motion state of a second agent particle of the second set of agent particles a second mobile robot action, the second mobile robot action accounting for predicted motion of the second agent particle; and generating a second mobile robot control signal for causing execution of the second mobile robot action.
A computer system configured to run queries on a static road layout, the computer system comprising: computer storage configured to store the static road layout, the static road layout comprising a section of road having a set of multiple road attributes, each road attribute described throughout the section of road by a describing function that exhibits a change in form at one or more change points along the section of road, the change points of a first of the road attributes exhibiting longitudinal misalignment with respect to the change points of a second of the road attributes; a road partitioning component configured to process the static road layout, and thereby partition the section of road into a sequence of road parts, each road part defined by a longitudinal coordinate interval, in which the describing function of every one of the road attributes has a form that is fixed throughout; a road indexing component configured to generate a road partition index having an entry for each road part, the entry indicating the form of the describing function of each road attribute as fixed throughout the longitudinal coordinate interval of that road part; and a scenario query engine configured to receive a part query, locate the entry in the road partition index for one of the road parts based on the part query, evaluate the describing function of at least one of the road attributes within the road part, and generate a part query response based on the evaluation of the describing function.
The disclosure provides systems and methods for identifying salient test runs involving an autonomous vehicle system. A processor receives sets of run data, each set representative of a driving scenario. For each set, an output set is generated, the output set comprising time-indexed events generated in response to a detected behaviour of at least one challenger agent, and a sequence of decision indicators indicating whether a driving action by an ego agent would be permissible. A data retrieval component is coupled to a results database and retrieves output sets based on the time-indexed events and the sequence of decision indicators. The processor generates the sequence of decision indicators by generating a planned trajectory of the ego agent, and determining whether the predefined driving action by the ego agent would be permissible.
B60W 50/06 - Détails des systèmes d'aide à la conduite des véhicules routiers qui ne sont pas liés à la commande d'un sous-ensemble particulier pour améliorer la réponse dynamique du système d'aide à la conduite, p. ex. pour améliorer la vitesse de régulation, ou éviter le dépassement de la consigne ou l'instabilité
A computer system for generating a scenario to be run in a simulation environment for testing the behaviour of an autonomous vehicle, the computer system comprising: a rendering component configured to: generate display data for causing a display to render a graphical user interface comprising an image of a driving environment and one or more agents within the driving environment; a parameter generator configured to generate in memory a user-defined parameter set responsive to user input defining the parameter set; and an expression manager configured to store in memory a user-defined expression set, responsive to user input defining the expression set, wherein each expression of the expression set is a user-defined function of one or more parameters of the parameter set; and a scenario generator configured to record the scenario in a scenario database; wherein the graphical user interface is configured to provide multiple agent fields for controlling the behaviour of the one or more agents when the scenario is run in a simulation environment, wherein each agent field is modifiable to associate therewith either a parameter of the user-defined parameter set or an expression of the user-defined expression set; and wherein the recorded scenario comprises the driving environment, the one or more agents, the user-defined parameter set, the user-defined expression set, and any user-defined associations between (i) the multiple agent fields and the user-defined parameter set and (ii) the multiple agent fields and the user-defined expression set, wherein each parameter associated with an agent field is controllable to directly modify an agent behaviour, and each parameter that is included in expression associated with an agent field is controllable to indirectly modify an agent behaviour.
A computer-implemented method of defining a scenario to be run in a simulation environment for testing the behaviour of a runtime stack of a sensor-equipped robot is disclosed The method comprises rendering a code editing interface on a display of a computer device, the code editing interface configured to receive an abstract scenario definition in a scenario definition language, the abstract scenario definition defining a scenario to be run in a first simulation environment for testing the behaviour of the runtime stack of the sensor-equipped robot having a role of an ego agent in the scenario; generating a concrete scenario description from the abstract scenario definition and a selected physical context compatible with the abstract scenario definition; providing the concrete scenario description to a second simulation environment; executing the concrete scenario description in the second simulation environment using a lightweight planner not forming part of the runtime stack to act as the ego agent; receiving from the second simulation environment an ego trace representative of actions of the ego agent during execution of the concrete scenario description; rendering, on the display of the computer device, a visualisation interface displaying a visual representation of the selected physical context; and rendering the actions of the ego agent on the visual representation of the selected physical context.
A computer-implemented method of generating lane detector outputs, the method comprising receiving a ground truth lane image containing one or more ground truth lane borders, each ground truth lane border comprising multiple border points; and generating a lane detector output image, by applying horizontal perturbations to the multiple border points of each ground truth lane border, the horizontal perturbations determined using a learned perturbation model, the learned perturbation model constructed to impose mutual correlation in the horizontal perturbations between vertically neighbouring border points of each ground truth lane border, and comprising parameters learned by performing a statistical analysis of lane detector errors computed between computed output images of a modelled lane detector and ground truth lane border annotations corresponding to the computed output images.
G06V 20/56 - Contexte ou environnement de l’image à l’extérieur d’un véhicule à partir de capteurs embarqués
B60W 60/00 - Systèmes d’aide à la conduite spécialement adaptés aux véhicules routiers autonomes
G06N 7/01 - Modèles graphiques probabilistes, p. ex. réseaux probabilistes
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
A directed search method is applied to a parameter space of a scenario for testing the performance of a robotic system in simulation. The directed search method is applied based multiple performance evaluation rules. A performance predictor is trained to probabilistically predict a pass or fail result for each rule at each point in the parameter space. An overall acquisition function is determined as follows: if a pass outcome is predicted at a given, the performance evaluation rule having the highest probability of an incorrect outcome prediction at determines the acquisition function; whereas, if a fail outcome is predicted at a given point for at least one rule, then the acquisition function is determined by the performance evaluation rule for which a fail outcome is predicted with the lowest probability of an incorrect outcome prediction.
G06F 11/34 - Enregistrement ou évaluation statistique de l'activité du calculateur, p. ex. des interruptions ou des opérations d'entrée–sortie
G05B 13/04 - Systèmes de commande adaptatifs, c.-à-d. systèmes se réglant eux-mêmes automatiquement pour obtenir un rendement optimal suivant un critère prédéterminé électriques impliquant l'usage de modèles ou de simulateurs
A directed search method is applied to a parameter space of a scenario for testing the performance of a robotic system in simulation. The directed search method is applied based on at least one performance evaluation rule that returns a performance score that can be numerical or non-numerical. A hierarchical score prediction model is constructed as follows. A score classification model is trained to probabilistically predict whether a point in the parameter space will result in a numerical or non-numerical outcome. A score regression model is trained to probabilistically predict a performance score for a given point, given that the score is numerical. The score classification and regression models are used to guide a directed search of the parameter space towards the most salient instances of the scenario.
G06F 30/27 - Optimisation, vérification ou simulation de l’objet conçu utilisant l’apprentissage automatique, p. ex. l’intelligence artificielle, les réseaux neuronaux, les machines à support de vecteur [MSV] ou l’apprentissage d’un modèle
A computer-implemented method of perceiving structure in an environment comprises steps of: receiving at least one structure observation input pertaining to the environment; processing the at least one structure observation input in a perception pipeline to compute a perception output; determining one or more uncertainty source inputs pertaining to the structure observation input; and determining for the perception output an associated uncertainty estimate by applying, to the one or more uncertainty source inputs, an uncertainty estimation function learned from statistical analysis of historical perception outputs.
G06T 7/593 - Récupération de la profondeur ou de la forme à partir de plusieurs images à partir d’images stéréo
G06T 7/285 - Analyse du mouvement utilisant une séquence de paires d'images stéréo
G06T 7/73 - Détermination de la position ou de l'orientation des objets ou des caméras utilisant des procédés basés sur les caractéristiques
G06V 20/56 - Contexte ou environnement de l’image à l’extérieur d’un véhicule à partir de capteurs embarqués
G06V 20/58 - Reconnaissance d’objets en mouvement ou d’obstacles, p. ex. véhicules ou piétonsReconnaissance des objets de la circulation, p. ex. signalisation routière, feux de signalisation ou routes
A computer-implemented method is provided for generating a trajectory for a first agent of a plurality of agents navigating a mapped area, the method comprising: receiving an observed state of each of the plurality of agents, and map data of the mapped area; generating an initial estimated trajectory for each of the plurality of agents based on the observed state of each agent and the map data; performing a first collision assessment to determine a likelihood of collision between the first agent and each other agent, based on the initial estimated trajectory for the first agent and the initial estimated trajectory for each other agent; and generating a second estimated trajectory for the first agent based on the observed states of each of the plurality of agents, the map data, and the results of the first collision assessment.
Performance testing of mobile robot planners is considered. A particular aim is herein to find a perception error or set of perception errors for a scenario that yields 'good' performance on a predetermined perception performance metric but has an unexpected impact on downstream performance (such as planner performance), in a reduced number of simulated runs. Of particular interest are instances of good perception performance that nevertheless lead to planner failure (e.g. breaking of a safety rule) or some other salient planning outcome. A core aim is to identify perception errors that are salient in this sense in a computationally efficient manner, by maximizing the saliency of perception errors for a given number of simulated runs. This is achieved through a systematic search for salient perception error based on a predetermined perception performance metric in combination with selectively performed simulation rollouts and rules-based planning performance evaluation.
A method of generating at least one trajectory in scenario comprising an agent navigating a mapped area, the method comprising: receiving an observed state of the agent and map data of the mapped area; generating a set of multiple trajectory basis elements from the observed state of the agent based on the map data; processing one or more scenario inputs in a neural network to generate a set of weights, each weight corresponding to one of the trajectory basis elements; and generating a trajectory for the agent by weighting each trajectory basis element by its corresponding weights and combining the weighted trajectory basis elements.
In a first aspect, a computer-implemented method of fusing visual features obtained with multiple sensor modalities comprises extracting from a camera image using a visual feature extractor a plurality of camera feature vectors at respective camera plane locations in a camera image plane; receiving sensor data of a non-camera sensor modality; processing the sensor data to compute a plurality of second feature vectors in a target plane lying non- parallel to the camera image plane in 3D space; and computing a plurality of fused feature vectors at a plurality of target plane locations in the target plane, without applying monocular depth detection to the camera image. In a second aspect, the fused feature vectors are computed using a transformer neural network cross-attention between the camera feature vectors and the second feature vectors.
G06V 10/80 - Fusion, c.-à-d. combinaison des données de diverses sources au niveau du capteur, du prétraitement, de l’extraction des caractéristiques ou de la classification
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
G06V 20/56 - Contexte ou environnement de l’image à l’extérieur d’un véhicule à partir de capteurs embarqués
A computer-implemented method of generating black-box adversarial inputs to a perception component using a surrogate model of the perception component comprises receiving an initial input to the perception component and repeatedly perturbing the initial input until an adversarial input is found that satisfies an attack objective by: performing a primary attack process by perturbing the initial input based on a computed gradient of a surrogate attack loss function of the surrogate model that encodes the attack objective; wherein, if the primary attack process terminates without finding any perturbed input satisfying the promising attack condition, a backup attack process is performed to perform a randomized search of the input space of the perception component, guided by the surrogate model, until a perturbed input satisfying the promising attack condition is found; wherein the primary attack process is repeated based on the perturbed input found by the primary attack process or backup process.
A computer system for rendering a graphical user interface for visualising runs of a driving scenario in which an ego agent navigates a road layout, comprising an input configured to receive a map of the road layout and run data comprising a sequence of timestamped ego agent states and a time-varying numerical score quantifying the performance of the ego agent with respect a set of run evaluation rules; and a rendering component configured to cause a graphical user interface to display, for each rule: a plot of the time-varying numerical score, and a marker denoting a selected time index of the plot, the marker movable along the time axis to change the selected time index; and a scenario visualization comprising a visualization of the run at the selected time index, whereby moving the marker along the time axis causes the scenario visualisation to update as the time index is changed.
G06F 11/3698 - Environnements pour l’analyse, le débogage ou le test de logiciel
G06N 3/006 - Vie artificielle, c.-à-d. agencements informatiques simulant la vie fondés sur des formes de vie individuelles ou collectives simulées et virtuelles, p. ex. simulations sociales ou optimisation par essaims particulaires [PSO]
G07C 5/00 - Enregistrement ou indication du fonctionnement de véhicules
G07C 5/02 - Enregistrement ou indication du temps de circulation, de fonctionnement, d'arrêt ou d'attente uniquement
A computer-implemented method for assessing autonomous vehicle performance comprising receiving, at an input, performance data of at least one autonomous driving run, the performance data comprising at least one time series of perception errors and at least one time series of driving performance results; and generating, at a rendering component, rendering data for rendering a graphical user interface, the graphical user interface for visualizing the performance data and comprising: a perception error timeline, and a driving assessment timeline, wherein the timelines are aligned in time, and divided into multiple time steps of the at least one driving run, wherein, for each time step: the perception timeline comprises a visual indication of whether a perception error occurred at that time step, and the driving assessment timeline comprises a visual indication of driving performance at that time step.
G07C 5/08 - Enregistrement ou indication de données de marche autres que le temps de circulation, de fonctionnement, d'arrêt ou d'attente, avec ou sans enregistrement des temps de circulation, de fonctionnement, d'arrêt ou d'attente
27.
MOTION PREDICTION AND TRAJECTORY GENERATION FOR MOBILE AGENTS
One aspect herein pertains to a computer-implemented method of predicting agent motion comprises receiving a first observed agent state corresponding to a first time instant; determining a set of agent goals; for each agent goal, planning an agent trajectory based on the agent goal and the first observed agent state; receiving a second observed agent state corresponding to a second time instant later than the first time instant; for each goal, comparing the second observed agent state with the at least one agent trajectory planned for the goal, and thereby computing a likelihood of the goal and/or the planned agent trajectory for the goal. Another aspect pertains to trajectory generation, e.g., within a motion planner.
B60W 60/00 - Systèmes d’aide à la conduite spécialement adaptés aux véhicules routiers autonomes
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
G06V 20/56 - Contexte ou environnement de l’image à l’extérieur d’un véhicule à partir de capteurs embarqués
A computer-implemented method of planning a trajectory for an ego agent in the presence of one or more further agents, the method comprising: receiving an observed state of each agent; and, for each agent, generating a predicted distribution over trajectories and adapting the predicted distribution over trajectories to generate a planned distribution over trajectories, the adaptation based on a planner confidence score and a trajectory evaluation score; and selecting an estimated optimal trajectory from the planned distribution over trajectories for the ego agent; wherein the trajectory evaluation score is based on the interaction of each of a set of possible trajectories of that agent with one or more possible trajectories of each other agent; and wherein the planner confidence score is based on a comparison of the observed state of the agent with a previous estimated optimal trajectory of the agent determined by the planner at a previous planning iteration.
The problem of domain shift error in computer vision models and other perception components is addressed. In a label approximation phase, an approximate label distribution is computed for each input of a target batch using a trained machine learning (ML) perception component. In an online label optimization phase, a modified label distribution is assigned to each input of the target batch, via optimization of an unsupervised loss function that (i) penalizes divergence between the approximate label distribution and the modified label distribution for each input of the target batch (ii) penalizes deviation between the modified label distributions assigned to input pairs of the target batch having similar features.
G06V 10/778 - Apprentissage de profils actif, p. ex. apprentissage en ligne des caractéristiques d’images ou de vidéos
G06V 10/75 - Organisation de procédés de l’appariement, p. ex. comparaisons simultanées ou séquentielles des caractéristiques d’images ou de vidéosApproches-approximative-fine, p. ex. approches multi-échellesAppariement de motifs d’image ou de vidéoMesures de proximité dans les espaces de caractéristiques utilisant l’analyse de contexteSélection des dictionnaires
G06V 10/774 - Génération d'ensembles de motifs de formationTraitement des caractéristiques d’images ou de vidéos dans les espaces de caractéristiquesDispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant l’intégration et la réduction de données, p. ex. analyse en composantes principales [PCA] ou analyse en composantes indépendantes [ ICA] ou cartes auto-organisatrices [SOM]Séparation aveugle de source méthodes de Bootstrap, p. ex. "bagging” ou “boosting”
G06V 10/776 - ValidationÉvaluation des performances
G06V 20/40 - ScènesÉléments spécifiques à la scène dans le contenu vidéo
G06V 20/70 - Étiquetage du contenu de scène, p. ex. en tirant des représentations syntaxiques ou sémantiques
A computer-implemented method of generating a test scenario for performance testing a robotic system in a simulation environment, the method comprising: receiving a dynamic element graph comprising a plurality of dynamic nodes representative of dynamic scenario agents, the dynamic element graph including one or more attachment edges connected at one end to one of the plurality of dynamic nodes and unconnected at the other end; searching a static element graph comprising a plurality of static nodes representative of static scenario elements, to identify locations at which the attachment edges can be connected to the static element graph; and storing data representative of the identified locations.
A computer-implemented method of testing an autonomous vehicle stack, comprising: storing a scenario datastore comprising a plurality of scenarios and first metadata describing each scenario, the first metadata having been generated by a first autonomous vehicle planner; retrieving a set of scenarios from the scenario datastore; for each retrieved scenario: executing a test run of the retrieved scenario using a second autonomous vehicle planner different from the first autonomous vehicle planner; generating second metadata describing the retrieved scenario based on the test run; and storing the second metadata in the scenario database.
For generating driving scenarios for testing an autonomous vehicle planner in a simulation environment, a scenario model comprises a scenario variable and a distribution associated with the scenario variable. The scenario variable is a road layout variable. Multiple sampled values of the scenario variable are computed based on the distribution associated the scenario variable. Based on the scenario model, multiple driving scenarios are generated for testing an autonomous vehicle planner in a simulation environment, each driving scenario comprising a road layout generated using a sampled value of said multiple sampled values of the scenario variable.
A computer-implemented method of generating a static layer and/or a dynamic layer for a test scenario for performance testing a robotic system in a simulation environment, the method comprising: inputting to a geometric constraint solver a scenario skeleton defining geometric elements and geometric constraints on the geometric elements, the geometric elements being modifiable based on a set of geometric variables, and the geometric constraints being characterised by one or more constraint parameters, thereby causing the geometric constraint solver to determine one or more functions of the constraint parameters for returning values of the geometric variables that satisfy the geometric constraints; receiving value(s) of the constraint parameter(s); using the received value(s) of the constraint parameter(s) and the one or more functions to compute values of the geometric variables satisfying the geometric constraints; and configuring based on the received value(s) of the constraint parameter(s), the computed value(s) of the geometric variables and the scenario skeleton, the static layer for the test scenario and/or the dynamic layer for the test scenario.
A computer implemented method of generating a scenario to be run in a simulation environment for testing the behaviour of an autonomous vehicle includes rendering, on a display of a computer device, an image of a static scene topology; and rendering on the display an object editing node comprising a set of input fields for receiving user input. The object editing node is for parameterizing an interaction of a challenger object relative to an ego object; and the method includes receiving into the input fields of the object editing node user input defining at least one temporal or relational constraint of the challenger object relative to the ego object. The at least one temporal or relational constraints define an interaction point of a defined interaction stage between the ego object and the challenger object. The method includes storing the set of constraints and defined interaction stage in an interaction container in a computer memory of the computer system; and generating a scenario to be run in a simulation environment, the scenario comprising the defined interaction stage executed on the static scene topology at the interaction point.
A computer system comprising: computer storage configured to store a static road layout; a topological indexing component configured to generate an in-memory topological index of the static road layout, the topological index in the form of a graph of nodes and edges, wherein each node corresponds to a road structure element of the static road layout, and the edges encode topological relationships between the road structure elements; a geometric indexing component configured to generate at least one in-memory geometric index of the static road layout for mapping geometric constraints to road structure elements of the static road layout; and a scenario query engine configured to receive a geometric query, search the geometric index to locate at least one static road element satisfying one or more geometric constraints of the geometric query, and return a descriptor of the at least one road structure element(s), wherein the scenario query engine is configured to receive a topological query comprising a descriptor of at least one road element, search the topological index to locate the corresponding node(s), identify at least one other node satisfying the topological query based on the topological relationships encoded in the edges of the topological index, and return a descriptor of the other node(s) satisfying the topological query.
A computer-implemented method of predicting an external actor trajectory comprises receiving, at a computer, sensor inputs for detecting and tracking an external actor; applying object tracking to the sensor inputs, in order track the external actor, and thereby determine an observed trace of the external actor over a time interval; determining a set of available goals for the external actor; for each of the available goals, determining an expected trajectory model; and comparing the observed trace of the external actor with the expected trajectory model for each of the available goals, to determine a likelihood of that goal.
B60W 60/00 - Systèmes d’aide à la conduite spécialement adaptés aux véhicules routiers autonomes
B60W 40/04 - Calcul ou estimation des paramètres de fonctionnement pour les systèmes d'aide à la conduite de véhicules routiers qui ne sont pas liés à la commande d'un sous-ensemble particulier liés aux conditions ambiantes liés aux conditions de trafic
B60W 50/00 - Détails des systèmes d'aide à la conduite des véhicules routiers qui ne sont pas liés à la commande d'un sous-ensemble particulier
G05B 13/02 - Systèmes de commande adaptatifs, c.-à-d. systèmes se réglant eux-mêmes automatiquement pour obtenir un rendement optimal suivant un critère prédéterminé électriques
G05B 13/04 - Systèmes de commande adaptatifs, c.-à-d. systèmes se réglant eux-mêmes automatiquement pour obtenir un rendement optimal suivant un critère prédéterminé électriques impliquant l'usage de modèles ou de simulateurs
G06V 10/84 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les modèles graphiques de probabilités à partir de caractéristiques d’images ou de vidéos, p. ex. les modèles de Markov ou les réseaux bayésiens
G06V 20/54 - Trafic, p. ex. de voitures sur la route, de trains ou de bateaux
G06V 20/56 - Contexte ou environnement de l’image à l’extérieur d’un véhicule à partir de capteurs embarqués
G08G 1/01 - Détection du mouvement du trafic pour le comptage ou la commande
H04N 7/18 - Systèmes de télévision en circuit fermé [CCTV], c.-à-d. systèmes dans lesquels le signal vidéo n'est pas diffusé
37.
PERFORMANCE TESTING FOR MOBILE ROBOT TRAJECTORY PLANNERS
A computer-implemented method of evaluating the performance of a trajectory planner for a mobile robot in a scenario, in which the trajectory planner is used to control the mobile robot responsive to at least one other agent of the scenario the method comprising: determining a scenario parameter set for the scenario and a likelihood of the scenario parameter set; computing an impact score for a failure event or near failure event between the mobile robot and the other agent occurring in the scenario instance, the impact score quantifying severity of the failure event or near failure event; and computing a risk score for the instance of the scenario based on the impact score and the likelihood of the scenario parameter set.
One aspect herein provides a method of analysing driving behaviour in a data processing computer system, the method comprising: receiving at the data processing computer system driving behaviour data to be analysed, wherein the driving behaviour data records vehicle movements within a monitored driving area; analysing the driving behaviour data to determine a normal driving behaviour model for the monitored driving area; using object tracking to determine driving trajectories of vehicles driving in the monitored driving area; comparing the driving trajectories with the normal driving behaviour model to identify at least one abnormal driving trajectory; and extracting a portion of the driving behaviour data corresponding to a time interval associated with the abnormal driving trajectory.
G08G 1/01 - Détection du mouvement du trafic pour le comptage ou la commande
B60W 40/04 - Calcul ou estimation des paramètres de fonctionnement pour les systèmes d'aide à la conduite de véhicules routiers qui ne sont pas liés à la commande d'un sous-ensemble particulier liés aux conditions ambiantes liés aux conditions de trafic
B60W 50/00 - Détails des systèmes d'aide à la conduite des véhicules routiers qui ne sont pas liés à la commande d'un sous-ensemble particulier
B60W 60/00 - Systèmes d’aide à la conduite spécialement adaptés aux véhicules routiers autonomes
G05B 13/02 - Systèmes de commande adaptatifs, c.-à-d. systèmes se réglant eux-mêmes automatiquement pour obtenir un rendement optimal suivant un critère prédéterminé électriques
G05B 13/04 - Systèmes de commande adaptatifs, c.-à-d. systèmes se réglant eux-mêmes automatiquement pour obtenir un rendement optimal suivant un critère prédéterminé électriques impliquant l'usage de modèles ou de simulateurs
G06V 10/84 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les modèles graphiques de probabilités à partir de caractéristiques d’images ou de vidéos, p. ex. les modèles de Markov ou les réseaux bayésiens
G06V 20/54 - Trafic, p. ex. de voitures sur la route, de trains ou de bateaux
G06V 20/56 - Contexte ou environnement de l’image à l’extérieur d’un véhicule à partir de capteurs embarqués
H04N 7/18 - Systèmes de télévision en circuit fermé [CCTV], c.-à-d. systèmes dans lesquels le signal vidéo n'est pas diffusé
A computer-implemented method of evaluating a candidate trajectory for an ego agent in the presence of at least one external agent, comprising: determining a predicted state of the at least one external agent at a future timestep and the corresponding state of the ego vehicle along the candidate trajectory at the future timestep; determining a headway region for each of the ego agent and the at least one external agent, the headway region for each agent defining the region occupied by that agent within a predefined time duration from the predicted state, based on the predicted state of the agent; performing a first intersection check to determine if the predicted state of the at least one external agent intersects with the headway region of the ego agent; and performing a second intersection check to determine if the predicted state of the ego agent intersects with the headway region of the at least one external agent; wherein, if either of the first or second intersection checks determine that an intersection exists, the candidate trajectory is evaluated as invalid.
The present disclosure provides methods and systems for automatically annotating a second map using annotation data from a first map. A source piece of road structure associated with a first piece of annotation data is identified in the first map, the source piece of road structure is matched with a target piece of road structure in the second map, and a second piece of annotation data for annotating the target piece of road structure in the second map is generated using the first piece of annotation data.
Computer systems and methods for planning a spatial path and a motion profile for an ego agent. The system comprises: an input configured to receive a state of the ego agent, and a trajectory generator configured to determine, based on the state of the ego agent, a first trajectory via a constrained optimization of a cost function encoding one or more soft planning constraints, subject to one or more hard planning constraints; the first trajectory comprising a spatial path and a motion component, the spatial path determined via the constrained optimization in dependence on at least one of a hard planning constraint and a soft planning constraint on the motion component; a motion planner configured to compute, based on the spatial path of the first trajectory, a motion profile for the ego agent satisfying one or more constraints of the motion planner.
A method of detecting one or more objects in a scene using multiple calibrated object detectors, is disclosed. Each calibrated object detector is applied to the scene, resulting in one or more predicted object regions for each calibrated object detector and an uncalibrated confidence score for each predicted object region. For each calibrated detector, its score calibration function is used to determine a calibrated confidence score for each predicted object region. A reduction algorithm is applied to a combined prediction set comprising the one or more predicted object regions for each object detector and their calibrated confidence scores, resulting in a reduced prediction set.
A method of evaluating the performance of a target planner for an ego robot in a scenario. Evaluation data is generated by applying the target planner from an initial scenario state to generate an actual ego trajectory taken by the ego robot. The actual ego trajectory is defined by a target trajectory parameter. Comparison data is generated by applying a comparison planner from the same initial scenario state to generate a comparison ego trajectory for a comparison ego robot, the comparison ego trajectory comprising at least one comparison trajectory parameter. A juncture point at which the comparison trajectory parameter differs from the actual trajectory parameter is determined and a difference between the actual trajectory parameter and the comparison trajectory parameter at the juncture point is determined. A comparison between the determined difference and a threshold value indicates whether the juncture point is significant.
A computer-implemented method of generating black-box adversarial inputs to a perception component comprises computing an adversarial input by applying a perturbation to an original input, the adversarial input satisfying an attack objective when inputted to the perception component. The perturbation is determined by selectively combining component perturbations selected from a predetermined set of component perturbations. Inputs correspond to respective points in an input vector space, and the component perturbations encode principal attack directions in the input vector space for satisfying said attack objective, the principal attack directions having been determined by analyzing: (i) a set of sample attack directions, or (ii) a set of input samples.
G06F 21/57 - Certification ou préservation de plates-formes informatiques fiables, p. ex. démarrages ou arrêts sécurisés, suivis de version, contrôles de logiciel système, mises à jour sécurisées ou évaluation de vulnérabilité
A computer-implemented method of predicting an agent trajectory in a world space, the method comprising: receiving a visual input comprising an image of the agent in an image space, extracting from the visual input using a visual feature extractor a set of visual features, receiving a trajectory history input comprising an observed trajectory of the agent in the world space, extracting from the trajectory history input a set of trajectory features, and generating, based on the set of visual features and the set of trajectory features, a prediction output comprising a predicted trajectory for the agent in the world space.
G06V 10/62 - Extraction de caractéristiques d’images ou de vidéos relative à une dimension temporelle, p. ex. extraction de caractéristiques axées sur le tempsSuivi de modèle
B60W 50/00 - Détails des systèmes d'aide à la conduite des véhicules routiers qui ne sont pas liés à la commande d'un sous-ensemble particulier
B60W 60/00 - Systèmes d’aide à la conduite spécialement adaptés aux véhicules routiers autonomes
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
G06V 20/58 - Reconnaissance d’objets en mouvement ou d’obstacles, p. ex. véhicules ou piétonsReconnaissance des objets de la circulation, p. ex. signalisation routière, feux de signalisation ou routes
Performance of a substitute upstream processing component is tested, in order to determine whether that performance is sufficient to support a downstream processing component, within an autonomous driving system, in place of an existing upstream processing component. The existing upstream processing component and the substitute upstream processing component are mutually interchangeable in so far as they provide the same form of outputs interpretable by the downstream processing component, such that either upstream processing component may be used without modification to the downstream processing component. A direct or indirect metric-based comparison is formulated in terms of the resulting performance of the downstream processing component.
To locate and model a 3D object captured in multiple time-series of sensor data of multiple sensor modalities, a cost function applied to the multiple time-series of sensor data is optimized. The cost function aggregates over time and the multiple sensor modalities, and is defined over a set of variables comprising one or more shape parameters of a 3D object model and a time sequence of poses of the 3D object model. The cost function penalizes inconsistency between the multiple time-series of sensor data and the set of variables. The object belongs to a known object class, and the 3D object model or the cost function encodes expected 3D shape information associated with the known object class, whereby the 3D object is located at multiple time instants and modelled by tuning each pose and the shape parameters with the objective of optimizing the cost function.
G06T 19/20 - Édition d'images tridimensionnelles [3D], p. ex. modification de formes ou de couleurs, alignement d'objets ou positionnements de parties
G06T 7/70 - Détermination de la position ou de l'orientation des objets ou des caméras
G06V 10/764 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant la classification, p. ex. des objets vidéo
A computer-implemented object detection method, comprising: receiving a point cloud and a 2D image having an image plane located at a known location in the point cloud; applying one or more machine learning models to the 2D image to determine a position of a detected object in the 2D image and to determine estimated dimensions and orientation of at least two sides of a bounding box corresponding to the detected object in the point cloud; and generating an initial estimated location of the at least two sides of the bounding box in the point cloud based at least in part on the estimated dimensions and orientation; optimising the initial estimated location of the at least two sides of the bounding box with respect to the point cloud to determine a detected location of the bounding box corresponding to the detected object.
G06V 20/58 - Reconnaissance d’objets en mouvement ou d’obstacles, p. ex. véhicules ou piétonsReconnaissance des objets de la circulation, p. ex. signalisation routière, feux de signalisation ou routes
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
An encoder is trained together with a perception component based on a training set comprising unannotated sensor data sets and annotated sensor data sets in a sequence of multiple training steps. Each training step comprises: in a first phase of the training step, updating the set of encoder parameters based on the unannotated sensor data sets, with the aim of optimizing a self-supervised loss function, without updating the set of task-specific parameters of the perception component, and in a second phase of the training step, updating the set of task-specific parameters based on the annotated sensor data sets, with the aim of optimizing a task-specific loss function, wherein the encoder as updated in the first phase of that training step processes a data representation of each annotated sensor data set to extract features therefrom, wherein the perception component processes the extracted features to compute an output therefrom, and wherein the task-specific loss is defined on the output and the associated annotation for each annotated sensor data set for learning a desired perception task. In performing the sequence of multiple training steps, the method alternates repeatedly between the first phase and the second phase.
G06K 9/00 - Méthodes ou dispositions pour la lecture ou la reconnaissance de caractères imprimés ou écrits ou pour la reconnaissance de formes, p.ex. d'empreintes digitales
G06V 10/44 - Extraction de caractéristiques locales par analyse des parties du motif, p. ex. par détection d’arêtes, de contours, de boucles, d’angles, de barres ou d’intersectionsAnalyse de connectivité, p. ex. de composantes connectées
A computer-implemented method of perceiving structure in a radar point cloud comprises: generating a discretised image representation of the radar point cloud having (i) an occupancy channel indicating whether or not each pixel of the discretised image representation corresponds to a point in the radar point cloud and (ii) a Doppler channel containing, for each occupied pixel, a Doppler velocity of the corresponding point in the radar point cloud; and inputting the discretised image representation to a machine learning (ML) perception component, which has been trained extract information about structure exhibited in the radar point cloud from the occupancy and Doppler channels.
G01S 13/42 - Mesure simultanée de la distance et d'autres coordonnées
G01S 7/41 - Détails des systèmes correspondant aux groupes , , de systèmes selon le groupe utilisant l'analyse du signal d'écho pour la caractérisation de la cibleSignature de cibleSurface équivalente de cible
G01S 13/58 - Systèmes de détermination de la vitesse ou de la trajectoireSystèmes de détermination du sens d'un mouvement
G01S 13/89 - Radar ou systèmes analogues, spécialement adaptés pour des applications spécifiques pour la cartographie ou la représentation
51.
Implementing slowdown manoeuvres in autonomous vehicles
A computer-implemented method of determining control signals for controlling an autonomous vehicle to implement a slowdown manoeuvre, comprising: detecting an obstacle at a distance ahead of the autonomous vehicle; comparing the distance with a threshold value and implementing a slowdown manoeuvre in dependence on the comparison, the slowdown manoeuvre selected from: a first slowdown manoeuvre carried out by a kinematic function, which is a time derivative of acceleration, in which a constraint optimisation has been applied to optimise a cost function of the slowdown manoeuvre subject to a set of hard constraints that require a final acceleration, speed and position to satisfy respective acceleration, speed and position targets, given an initial speed and acceleration of the vehicle, and impose a jerk magnitude upper limit; and a second slowdown manoeuvre implemented in an adaptive cruise control mode which aims to reach a target headway between the autonomous vehicle and the obstacle.
A computer-implemented method of tracking detected objects across multiple image sensors having overlapping fields of view, the method comprising: receiving, for a first image sensor of the multiple image sensors, a first detected object region in a first 2D image plane; receiving, for a second image sensor of the multiple image sensors, a second detected object region in a second 2D image plane, the second detected object region synchronous with the first detected object region; computing, for at least two edge sections of the first detected object region, (i) an outward projection of each edge section into 3D space and (ii) an outward projection into 3D space of a corresponding edge section of the second detected object region; identifying at least four reference points in 3D space, by determining an intersection between the outward projection of each edge section of the first detected object region with the outward projection of the corresponding edge section of the second detected object region; calculating a first extent of overlap in the first 2D image plane between the first detected object region and a first inferred object region defined by the at least four reference points projected into the first image plane; calculating a second extent of overlap in the second 2D image plane between the second detected object region and a second inferred object region defined by the at least four reference points projected into the second image plane; and generating based on the first extent of overlap and the second extent of overlap an object tracking output indicating whether the first detected object region and the second detected object region correspond to the same object.
A computer-implemented method of generating road annotation data for annotating an electronic map, the method comprising accessing from persistent storage an electronic map defining a road network; generating a road topology graph encoding a topology of the road network, the road topology graph comprising nodes representing road structure elements and edges representing links between road structure elements; performing a search of the road topology graph for a predetermined graph structure; responsive to identifying a subgraph of the road topology graph exhibiting the predetermined graph structure, generating annotation data for marking in the electronic map a portion of the road network corresponding to the subgraph; and generating in persistent storage an augmented map comprising map data defining the portion of the road network and the annotation data.
A computer-implemented method of planning an ego trajectory for an ego robot in an environment in the presence of an occluding object, the method comprising: determining, for a current time step, an occluded region of the environment based on a current state of the ego robot and a current state of the occluding object; determining, based on the occluded region at the current time step, a predicted virtual agent collision region at a future time step, using a virtual agent dynamics model applied to the occluded region; and, computing, by a trajectory planner, a planned ego trajectory for the mobile robot from the current time step to the future time step accounting for the predicted virtual agent collision region of the environment.
A computer-implemented method of planning an ego trajectory for an ego robot in an environment in the presence of an occluding object, the method comprising: determining a trajectory cost function having one or more planning variables that define a planned ego trajectory, the planning variables tuneable to modify the planned ego trajectory, wherein the trajectory cost function is dependent on a predicted occluded region of the environment, as defined by the planned ego trajectory and a predicted state of the occluding object; and tuning the planning variables using an optimizer applied to the trajectory cost function, wherein the trajectory cost function encodes: (i) a progress objective that rewards modifications to the planned ego trajectory that progress the ego robot towards a chosen goal location, and (ii) a visibility objective, dependent on the predicted occluded region, that rewards modifications to the planned ego trajectory that improve visibility of the environment from the perspective of the ego robot, whereby the optimiser is incentivised to tune the planning variables in a manner satisfying the progress objective and the visibility objective.
A computer implemented method of evaluating planner performance for an ego robot in a scenario, the method comprising: receiving for each of a set of runs, run evaluation data, wherein the run evaluation data for each run is generated by applying a planner in a scenario of that run to generate an ego trajectory taken by the ego robot in the scenario; determining for the set of runs, an examination category; generating for each run, an indicator of an examination parameter for that run in the examination category; the indicator selected from a group of different indicators in the examination category; and identifying a cluster of runs of the set of runs sharing the same indicator in the examination category.
A computer implemented method of evaluating the performance of a target planner for an ego robot in a scenario, the method comprising: rendering on a display of a graphical user interface of a computer device, a dynamic visualisation of an ego robot moving along a first path in accordance with a first planned trajectory from the target planner and of a comparison ego robot moving along a second path in accordance with a second planned trajectory from a comparison planner; detecting a juncture point at which the first and second trajectories diverge; rendering the ego robot and the comparison ego robot as a single visual object in motion along a common path shared by the first and second paths prior to the juncture point; and rendering the ego robot and the comparison ego robot as separate visual objects on the display along the respective first and second paths from the juncture point.
A computer implemented method of evaluating the performance of at least one component of a planning stack for an autonomous robot, the method comprising: generating first evaluation data of a first run by operating the autonomous robot under the control of a planning stack under test in a scenario; modifying at least one operating parameter of at least one component of the planning stack by applying a variable modification to the operating parameter; generating second evaluation data of a second run by operating the autonomous robot under the control of the planning stack in which the at least one operating parameter has been modified, in the scenario; and comparing the first evaluation data with the second evaluation data using at least one performance metric for the comparison.
Methods are provided for generating a target speed for an autonomous vehicle. Waypoints on a drivable path are detected, and a curvature of the drivable path at each waypoint is determined. Using the curvature at each waypoint and a lateral acceleration limit of the vehicle, a speed at or below which the autonomous vehicle is permitted to drive is computed for each waypoint. It is then determined, for each waypoint, whether the speed can be met while adhering to predefined slowdown criteria. Upon identifying a limiting waypoint at which an allowed speed cannot be met, a target speed is set so that the vehicle attains the allowed speed at the limiting waypoint while adhering to slowdown criteria.
A computer-implemented method of estimating a 3D object pose, the method comprising: receiving 3D data comprising a full or partial view of a 3D object, the 3D object exhibiting reflective symmetry about an unknown 2D symmetry plane; applying symmetry detection to the 3D data, and thereby calculating, in 3D space, an estimated 2D symmetry plane for the 3D object; and applying 3D pose detection to the 3D data based on the estimated 2D symmetry plane, thereby computing a 3D pose estimate of the 3D object that is informed by the reflective symmetry of the 3D object.
G06V 10/44 - Extraction de caractéristiques locales par analyse des parties du motif, p. ex. par détection d’arêtes, de contours, de boucles, d’angles, de barres ou d’intersectionsAnalyse de connectivité, p. ex. de composantes connectées
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
A computer-implemented method of planning ego actions for a mobile robot in the presence of at least one agent, comprising: searching for an optimal ego action in multiple search steps, each comprising: selecting an ego action from a set of possible ego actions, selecting an agent behaviour from a set of possible agent behaviours, running a simulation based on the selected ego action and agent behaviour, determining a possible outcome, and assigning a reward to the selected ego action, based on a reward metric, wherein selection of the ego action in later search steps is biased towards higher reward ego action(s) but selection of the agent behaviour in later search steps is biased towards riskier agent behaviour(s), a risky agent behaviour being, according to earlier search steps, more likely to result in a lower reward outcome and choosing an ego action based on the rewards computed in the search steps.
Methods and systems are provided for evaluating performance of a runtime stack of a sensor-equipped robot in a run of a scenario comprising two or more agents. For each timestep in the run, an extent of compliance of run data with one or more assessment criterion is displayed in a global assessment timeline of a graphical user interface (GUI). A user input selecting one of the agents is received at the GUI. A subset of the run data pertaining to the selected agent is then selected and an agent-specific timeline showing an indication of extent of compliance of the data subset over time is provided on the GUI.
Systems and method are provided for a driving run performed by a sensor-equipped robot in a driving scene comprising at least one dynamic signalling agent (DSA). DSA data indicating, signalling states of the at least one DSA as a function of time is received, and a graphical user interface (GUI), comprising a schematic representation of the run and at least one DSA state timeline showing a visual indicator of the current signalling state of a corresponding one of the at least one DSA, is rendered on the GUI.
A method for testing performance of a stack for planning ego vehicle trajectories in real or simulated driving scenarios applying driving rules to the scenario ground truth for evaluating the performance of the stack in the scenario, and providing output indicating whether each driving rule has been complied with; wherein the driving rules include at least one ODD-based response rule, wherein applying the ODD-based response rule includes processing the scenario ground truth over multiple time steps, to determine whether the scenario is within the defined ODD at each time step, and thereby detecting a change in the scenario that takes the scenario outside the defined ODD, and processing the internal state data, to determine whether a state change occurred within the stack, within a time interval, the output for the at least one ODD-based response rule indicating whether the state change occurred within the time interval.
G07C 5/08 - Enregistrement ou indication de données de marche autres que le temps de circulation, de fonctionnement, d'arrêt ou d'attente, avec ou sans enregistrement des temps de circulation, de fonctionnement, d'arrêt ou d'attente
66.
PERFORMANCE TESTING FOR MOBILE ROBOT TRAJECTORY PLANNERS
A computer-implemented method of evaluating the performance of a trajectory planner for a mobile robot in a real or simulated scenario, comprises receiving scenario ground truth of the scenario, the scenario ground truth generated using the trajectory planner to control an ego agent of the scenario responsive to at least one scenario element of the scenario. One or more performance evaluation rules for the scenario and at least one activation condition for each performance evaluation rule are received. A test oracle processes the scenario ground truth to determine whether the activation condition of each performance evaluation rule is satisfied over multiple time steps of the scenario. Each performance evaluation rule is evaluated by the test oracle, to provide at least one test result, only when its activation condition is satisfied.
A computer implemented method of training an encoder to extract features from sensor data comprises generating a plurality of training examples, each training example comprising at least two data representations of a set of sensor data, the at least two data representations related by a transformation parameterized by at least one numerical transformation value; and training the encoder based on a self-supervised regression loss function applied to the training examples. The encoder extracts respective features from the at least two data representations of each training example, and at least one numerical output value is computed from the extracted features. The self-supervised regression loss function encourages the at least one numerical output value to match the at least one numerical transformation value parameterizing the transformation.
G06V 10/774 - Génération d'ensembles de motifs de formationTraitement des caractéristiques d’images ou de vidéos dans les espaces de caractéristiquesDispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant l’intégration et la réduction de données, p. ex. analyse en composantes principales [PCA] ou analyse en composantes indépendantes [ ICA] ou cartes auto-organisatrices [SOM]Séparation aveugle de source méthodes de Bootstrap, p. ex. "bagging” ou “boosting”
G06V 10/44 - Extraction de caractéristiques locales par analyse des parties du motif, p. ex. par détection d’arêtes, de contours, de boucles, d’angles, de barres ou d’intersectionsAnalyse de connectivité, p. ex. de composantes connectées
G06V 10/766 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant la régression, p. ex. en projetant les caractéristiques sur des hyperplans
G06V 10/776 - ValidationÉvaluation des performances
A method of predicting actions of one or more actor agent in a scenario is implemented by an ego agent in the scenario. A plurality of agent models are used to generate a set of candidate futures, each candidate future providing an expected action of the actor agent. A weighting function is applied to each candidate future to indicate its relevance in the scenario. A group of candidate futures is selected for each actor agent based on the indicated relevance, wherein the plurality of agent models comprises a first model representing a rational goal directed behaviour inferable from the vehicular scene, and at least one second model representing an alternate behaviour not inferable from the vehicular scene.
A computer implemented method of training an encoder to extract features from sensor data comprises training a machine learning (ML) system based on a self-supervised loss function applied to a training set, the ML system comprising the encoder. The training set comprises first data representations and corresponding second data representations, wherein the encoder extracts features from each first and second data representation, and wherein the self-supervised loss function encourages the ML system to associate each first data representation with its corresponding second data representation based on their respective features. Each first data representation and its corresponding second data representation represent a common set of sensor data, and at least the second data representation is generated by: applying a 2D object detector to an image other than the first and second data representations, wherein the image contains or is associated with the common set of sensor data, and transforming the common set of sensor data based one or more objects detected in the image, the second data representation representing the transformed sensor data.
G06V 10/00 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos
G06V 10/77 - Traitement des caractéristiques d’images ou de vidéos dans les espaces de caractéristiquesDispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant l’intégration et la réduction de données, p. ex. analyse en composantes principales [PCA] ou analyse en composantes indépendantes [ ICA] ou cartes auto-organisatrices [SOM]Séparation aveugle de source
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
A method of assessing lateral stability of a moving vehicle in a real or simulated driving scenario comprises: determining a time-varying lateral position signal for the moving vehicle; computing an evolving frequency spectrum of the time-varying lateral position signal over a moving window across the time-varying lateral position signal; and analysing the evolving frequency spectrum to extract a lateral stability signal that indicates an extent to which the moving vehicle is maintaining a stable lateral position.
A computer-implemented method of computer-implemented method of perceiving structure in a point cloud comprises: applying clustering to the point cloud, and thereby identifying at least one moving object cluster within the point cloud, the point cloud comprising time-stamped points captured over a non-zero accumulation window; determining a motion model for the moving object cluster, by fitting one or more parameters of the motion model to the time-stamped points of that cluster; using the motion model to transform the time-stamped points of the moving object cluster to a common reference time; and applying a perception component to the transformed points of the moving object cluster to extract information about structure exhibited in the transformed points.
A computer system receives scenario data generated using a trajectory planner to control an ego agent responsive to at least one other agent in a real or simulated scenario. A test oracle provides predetermined extractor functions for extracting time-varying numerical signals from the scenario data and predetermined assessor functions for assessing the extracted time-varying signals. The test oracle applies, to the scenario data, a rule graph comprising extractor nodes and assessor nodes. Each extractor node applies one of the predetermined extractor functions to the scenario data to extract an output in the form of a time-varying numerical signal. Each assessor node has one or more child nodes, each child node being one of the extractor nodes or another of the assessor nodes, and the assessor node applies one of the predetermined assessor functions to the output(s) of its child node(s). The test oracle provides an output graph comprising the output of at least one of the assessor nodes and the output(s) of at least one of its child node(s).
A computer-implemented method of assessing performance of perception component, the perception component for interpreting structure in a scene comprises: receiving a set of multiple computed outputs obtained by applying the perception component to the scene, wherein each computed output comprises a confidence score; generating, from the set of multiple computed outputs, multiple pseudo-ground truth sets, wherein each pseudo-ground truth set comprises, for each computed output, a pseudo-ground truth output sampled from a set of possible ground truth outputs based on a probability distribution defined by the confidence score of the computed output; computing a performance score for the perception component applied to the scene with respect to each pseudo-ground truth set, by comparing the set of multiple outputs with that pseudo-ground truth set; and computing an overall performance score for the perception component applied to the scene, by aggregating the performance scores computed with respect to the multiple pseudo-ground truth sets.
G06V 20/56 - Contexte ou environnement de l’image à l’extérieur d’un véhicule à partir de capteurs embarqués
G06V 10/98 - Détection ou correction d’erreurs, p. ex. en effectuant une deuxième exploration du motif ou par intervention humaineÉvaluation de la qualité des motifs acquis
G06F 18/21 - Conception ou mise en place de systèmes ou de techniquesExtraction de caractéristiques dans l'espace des caractéristiquesSéparation aveugle de sources
A computer-implemented method of modelling a perception system for perceiving objects captured in sensor data comprises: receiving a plurality of training examples, each comprising a ground truth scene for a set of sensor data and a corresponding perceived scene obtained by applying the perception system to the set of sensor data; fitting to the training examples noise model parameters, encoding a noise distribution over perceived scenes given a misdetection scene, and misdetection model parameters, encoding a misdetection distribution over misdetection scenes given a ground truth scene; computing a perception distribution over perceived scenes for a given ground truth scene by marginalizing the product of noise and misdetection distributions over multiple misdetection scenes, wherein individual objects in the ground truth scene are not associated with individual objects in the perceived scenes; fitting the noise and misdetection model parameters to match the perception distribution to the perceived scene for each training example.
B60W 50/02 - Détails des systèmes d'aide à la conduite des véhicules routiers qui ne sont pas liés à la commande d'un sous-ensemble particulier pour préserver la sécurité en cas de défaillance du système d'aide à la conduite, p. ex. en diagnostiquant ou en palliant à un dysfonctionnement
B60W 60/00 - Systèmes d’aide à la conduite spécialement adaptés aux véhicules routiers autonomes
G06V 10/776 - ValidationÉvaluation des performances
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
G06V 20/56 - Contexte ou environnement de l’image à l’extérieur d’un véhicule à partir de capteurs embarqués
A method of predicting trajectories for agents of a scenario, the method comprising, for each agent generating an agent feature vector based on one or more observed past states of the agent, computing a set of pairwise feature vectors, each computed as a combination of the agent feature vector for that agent with a respective agent feature vector generated for each other agent of the scenario, processing the pairwise feature vectors as independent inputs to one or more interaction layers of a trajectory prediction neural network to generate a pairwise output for each pairwise feature vector, aggregating the pairwise outputs over the other agents of the scenario to generate an interaction-based feature representation for each agent, processing the interaction-based feature representation in one or more prediction layers of the trajectory prediction neural network, and generating, based on the output of the one or more prediction layers, at least one predicted trajectory for each agent.
A computer system for generating a scenario to be run in a simulation environment for testing the behaviour of an autonomous vehicle, the computer system comprising: a rendering component configured to: generate display data for causing a display to render a graphical user interface comprising an image of a driving environment and one or more agents within the driving environment; a parameter generator configured to generate in memory a user-defined parameter set responsive to user input defining the parameter set; and an expression manager configured to store in memory a user-defined expression set, responsive to user input defining the expression set, wherein each expression of the expression set is a user-defined function of one or more parameters of the parameter set; and a scenario generator configured to record the scenario in a scenario database; wherein the graphical user interface is configured to provide multiple agent fields for controlling the behaviour of the one or more agents when the scenario is run in a simulation environment, wherein each agent field is modifiable to associate therewith either a parameter of the user-defined parameter set or an expression of the user-defined expression set; and wherein the recorded scenario comprises the driving environment, the one or more agents, the user-defined parameter set, the user-defined expression set, and any user- defined associations between (i) the multiple agent fields and the user-defined parameter set and (ii) the multiple agent fields and the user-defined expression set, wherein each parameter associated with an agent field is controllable to directly modify an agent behaviour, and each parameter that is included in expression associated with an agent field is controllable to indirectly modify an agent behaviour.
G06F 30/12 - CAO géométrique caractérisée par des moyens d’entrée spécialement adaptés à la CAO, p. ex. interfaces utilisateur graphiques [UIG] spécialement adaptées à la CAO
77.
IDENTIFYING SALIENT TEST RUNS INVOLVING MOBILE ROBOT TRAJECTORY PLANNERS
The disclosure provides systems and methods for identifying salient test runs involving an autonomous vehicle system. A processor receives sets of run data, each set representative of a driving scenario. For each set, an output set is generated, the output set comprising time-indexed events generated in response to a detected behaviour of at least one challenger agent, and a sequence of decision indicators indicating whether a driving action by an ego agent would be permissible. A data retrieval component is coupled to a results database and retrieves output sets based on the time-indexed events and the sequence of decision indicators. The processor generates the sequence of decision indicators by generating a planned trajectory of the ego agent, and determining whether the predefined driving action by the ego agent would be permissible.
A computer-implemented method of processing images for extracting information about known objects comprises the steps of receiving an image containing a view of a known object at a scale dependent on an object distance of the known object from an image capture location of the image; determining, from a world model representing one or more known objects in the vicinity of the image capture location, an object location of the known object, the object location and the image capture location defined in a world frame of reference; and based on the image capture location and the object location in the world frame of reference, applying image scaling to the image, to extract a rescaled image containing a rescaled view of the known object at a scale that is substantially independent of the object distance from the image capture location.
G06T 3/40 - Changement d'échelle d’images complètes ou de parties d’image, p. ex. agrandissement ou rétrécissement
G06T 7/73 - Détermination de la position ou de l'orientation des objets ou des caméras utilisant des procédés basés sur les caractéristiques
G06V 10/774 - Génération d'ensembles de motifs de formationTraitement des caractéristiques d’images ou de vidéos dans les espaces de caractéristiquesDispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant l’intégration et la réduction de données, p. ex. analyse en composantes principales [PCA] ou analyse en composantes indépendantes [ ICA] ou cartes auto-organisatrices [SOM]Séparation aveugle de source méthodes de Bootstrap, p. ex. "bagging” ou “boosting”
G06V 10/94 - Architectures logicielles ou matérielles spécialement adaptées à la compréhension d’images ou de vidéos
H04N 5/262 - Circuits de studio, p. ex. pour mélanger, commuter, changer le caractère de l'image, pour d'autres effets spéciaux
79.
Systems for testing and training autonomous vehicles
A computer implemented method of path verification in a computer system is described. A user provides input to mark a displayed image of a scenario. A path is generated representing the trajectory of a vehicle. Control points along the path are recorded, each control point associated with a vehicle position and target speed. A vehicle position end target speed of two control points is used to calculate at least one path verification parameter which defines how a vehicle travelling along the path would behave. The at least one verification parameter is compared with a corresponding threshold value; and an alert is generated to the user at the user interface when the at least one path verification parameter exceeds the corresponding threshold value.
B60W 60/00 - Systèmes d’aide à la conduite spécialement adaptés aux véhicules routiers autonomes
B60W 50/14 - Moyens d'information du conducteur, pour l'avertir ou provoquer son intervention
G01C 21/36 - Dispositions d'entrée/sortie pour des calculateurs embarqués
G05B 13/02 - Systèmes de commande adaptatifs, c.-à-d. systèmes se réglant eux-mêmes automatiquement pour obtenir un rendement optimal suivant un critère prédéterminé électriques
A computer-implemented method of generating lane detector outputs, the method comprising receiving a ground truth lane image containing one or more ground truth lane borders, each ground truth lane border comprising multiple border points; and generating a lane detector output image, by applying horizontal perturbations to the multiple border points of each ground truth lane border, the horizontal perturbations determined using a learned perturbation model, the learned perturbation model constructed to impose mutual correlation in the horizontal perturbations between vertically neighbouring border points of each ground truth lane border, and comprising parameters learned by performing a statistical analysis of lane detector errors computed between computed output images of a modelled lane detector and ground truth lane border annotations corresponding to the computed output images.
A directed search method is applied to a parameter space of a scenario for testing the performance of a robotic system in simulation. The directed search method is applied based on at least one performance evaluation rule that returns a performance score that can be numerical or non-numerical. A hierarchical score prediction model is constructed as follows. A score classification model is trained to probabilistically predict whether a point in the parameter space will result in a numerical or non-numerical outcome. A score regression model is trained to probabilistically predict a performance score for a given point, given that the score is numerical. The score classification and regression models are used to guide a directed search of the parameter space towards the most salient instances of the scenario.
G05B 13/02 - Systèmes de commande adaptatifs, c.-à-d. systèmes se réglant eux-mêmes automatiquement pour obtenir un rendement optimal suivant un critère prédéterminé électriques
G05B 13/04 - Systèmes de commande adaptatifs, c.-à-d. systèmes se réglant eux-mêmes automatiquement pour obtenir un rendement optimal suivant un critère prédéterminé électriques impliquant l'usage de modèles ou de simulateurs
G06F 30/20 - Optimisation, vérification ou simulation de l’objet conçu
A directed search method is applied to a parameter space of a scenario for testing the performance of a robotic system in simulation. The directed search method is applied based multiple performance evaluation rules. A performance predictor is trained to probabilistically predict a pass or fail result for each rule at each point in the parameter space. An overall acquisition function is determined as follows: if a pass outcome is predicted at a given, the performance evaluation rule having the highest probability of an incorrect outcome prediction at determines the acquisition function; whereas, if a fail outcome is predicted at a given point for at least one rule, then the acquisition function is determined by the performance evaluation rule for which a fail outcome is predicted with the lowest probability of an incorrect outcome prediction.
G06F 11/36 - Prévention d'erreurs par analyse, par débogage ou par test de logiciel
G05B 13/04 - Systèmes de commande adaptatifs, c.-à-d. systèmes se réglant eux-mêmes automatiquement pour obtenir un rendement optimal suivant un critère prédéterminé électriques impliquant l'usage de modèles ou de simulateurs
G05B 13/02 - Systèmes de commande adaptatifs, c.-à-d. systèmes se réglant eux-mêmes automatiquement pour obtenir un rendement optimal suivant un critère prédéterminé électriques
A computer system for analysing driving scenes in relation to an autonomous vehicle (AV) operational design domain (ODD), the computer system comprising: an input configured to receive a definition of the ODD in a formal ontology language; a scene processor configured to receive data of a driving scene and extract a scene representation therefrom, the data comprising an ego trace, at least one agent trace, and environmental data about an environment in which the traces were captured or generated, wherein the scene representation is an ontological representation of both static and dynamic elements of the driving scene extracted from the traces and the environmental data, and expressed in the same formal ontology language as the ODD; and a scene analyzer configured to match the static and dynamic elements of the scene representation with corresponding elements of the ODD, and thereby determine whether or not the driving scene is within the defined ODD.
G06F 30/20 - Optimisation, vérification ou simulation de l’objet conçu
B60W 60/00 - Systèmes d’aide à la conduite spécialement adaptés aux véhicules routiers autonomes
B60W 40/02 - Calcul ou estimation des paramètres de fonctionnement pour les systèmes d'aide à la conduite de véhicules routiers qui ne sont pas liés à la commande d'un sous-ensemble particulier liés aux conditions ambiantes
Abstract: A driving scenario is extracted from real-world driving data captured within a road layout. A simulation is run based on the extracted driving scenario, in which an ego agent and a simulated non-ego agent each exhibit closed-loop behaviour. The closed-loop behaviour of the ego agent is determined by autonomous decisions taken in an AV stack under testing in response to simulated inputs, reactive to the simulated agent. The closed-loop behaviour of the non-ego agent is determined by implementing an inferred goal or behaviour, reactive to the ego agent. The goal or behaviour is inferred from an observed trace of a real-world agent extracted from the real-world driving data.
G06F 11/36 - Prévention d'erreurs par analyse, par débogage ou par test de logiciel
G06F 30/27 - Optimisation, vérification ou simulation de l’objet conçu utilisant l’apprentissage automatique, p. ex. l’intelligence artificielle, les réseaux neuronaux, les machines à support de vecteur [MSV] ou l’apprentissage d’un modèle
85.
GENERATING SIMULATION ENVIRONMENTS FOR TESTING AV BEHAVIOUR
A computer implemented method of generating a scenario to be run in a simulation environment for testing the behaviour of an autonomous vehicle is described. An image is rendered on a display. A user can mark multiple locations to create at least one path for an agent vehicle in the rendered image. A path is generated which passes through the locations and rendered on the display. A user can define at least one behavioural parameter for controlling behaviour of the agent vehicle associated with the at least one path when the scenario is run in a simulation environment. The scenario is recorded for future use.
A computer-implemented method is provided for generating a trajectory for a first agent of a plurality of agents navigating a mapped area, the method comprising: receiving an observed state of each of the plurality of agents, and map data of the mapped area; generating an initial estimated trajectory for each of the plurality of agents based on the observed state of each agent and the map data; performing a first collision assessment to determine a likelihood of collision between the first agent and each other agent, based on the initial estimated trajectory for the first agent and the initial estimated trajectory for each other agent; and generating a second estimated trajectory for the first agent based on the observed states of each of the plurality of agents, the map data, and the results of the first collision assessment.
A method of generating at least one trajectory in scenario comprising an agent navigating a mapped area, the method comprising: receiving an observed state of the agent and map data of the mapped area; generating a set of multiple trajectory basis elements from the observed state of the agent based on the map data; processing one or more scenario inputs in a neural network to generate a set of weights, each weight corresponding to one of the trajectory basis elements; and generating a trajectory for the agent by weighting each trajectory basis element by its corresponding weights and combining the weighted trajectory basis elements.
B60W 30/095 - Prévision du trajet ou de la probabilité de collision
B60W 60/00 - Systèmes d’aide à la conduite spécialement adaptés aux véhicules routiers autonomes
B60W 40/04 - Calcul ou estimation des paramètres de fonctionnement pour les systèmes d'aide à la conduite de véhicules routiers qui ne sont pas liés à la commande d'un sous-ensemble particulier liés aux conditions ambiantes liés aux conditions de trafic
A computer-implemented method of evaluating the performance of a full or partial autonomous vehicle (AV) stack in simulation, the method comprising: applying an optimization algorithm to a numerical performance function defined over a scenario space, wherein the numerical performance function quantifies the extent of success or failure of the AV stack as a numerical score, and the optimization algorithm searches the scenario space for a driving scenario in which the extent of failure of the AV stack is substantially maximized, wherein the optimization algorithm evaluates multiple driving scenarios in the search space over multiple iterations, by running a simulation of each driving scenario in a simulator, in order to provide perception inputs to the AV stack, and thereby generate at least one simulated agent trace and a simulated ego trace reflecting autonomous decisions taken in the AV stack in response to the simulated perception inputs, wherein later iterations of the multiple iterations are guided by the results of previous iterations of the multiple iterations, with the objective of finding the driving scenario for which the extent of failure of the AV stack is maximized.
B60W 60/00 - Systèmes d’aide à la conduite spécialement adaptés aux véhicules routiers autonomes
B60W 50/04 - Détails des systèmes d'aide à la conduite des véhicules routiers qui ne sont pas liés à la commande d'un sous-ensemble particulier pour surveiller le fonctionnement du système d'aide à la conduite
G06F 30/20 - Optimisation, vérification ou simulation de l’objet conçu
An occlusion metric is computed for a target object in a 3D multi-object simulation. The target object is represented in 3D space by a collision surface and a 3D bounding box. In a reference surface defined in 3D space, a bounding box projection is determined for the target object with respect to an ego location. The bounding box projection is used to determine a set of reference points in 3D space. For each reference point of the set of reference points, a corresponding ray is cast based on the ego location, and it is determined whether the ray is an object ray that intersects the collision surface of the target object. For each such object ray, it is determined whether the object ray is occluded. The occlusion metric conveys an extent to which the object rays are occluded.
A method of annotating known objects in road images captured from a sensor-equipped vehicle, the method implemented in an annotation system and comprising: receiving at the annotation system a road image containing a view of a known object; receiving ego localization data, as computed in a map frame of reference, via localization applied to sensor data captured by the sensor-equipped vehicle, the ego localization data indicating an image capture pose of the road image in the map frame of reference; determining, from a predetermined road map, an object location of the known object in the map frame of reference, the predetermined road map representing a road layout the map frame of reference, wherein the known object is one of: a piece of road structure, and an object on or adjacent a road; computing, in an image plane defined by the image capture pose, an object projection, by projecting an object model of the known object from the object location into the image plane; and storing, in an image database, image data of the road image, in association with annotation data of the object projection for annotating the image data with a location of the known object in the image plane.
G06V 20/56 - Contexte ou environnement de l’image à l’extérieur d’un véhicule à partir de capteurs embarqués
G06T 7/70 - Détermination de la position ou de l'orientation des objets ou des caméras
G06V 10/44 - Extraction de caractéristiques locales par analyse des parties du motif, p. ex. par détection d’arêtes, de contours, de boucles, d’angles, de barres ou d’intersectionsAnalyse de connectivité, p. ex. de composantes connectées
G06V 20/70 - Étiquetage du contenu de scène, p. ex. en tirant des représentations syntaxiques ou sémantiques
91.
MOTION PREDICTION AND TRAJECTORY GENERATION FOR MOBILE AGENTS
One aspect herein pertains to a computer-implemented method of predicting agent motion comprises receiving a first observed agent state corresponding to a first time instant; determining a set of agent goals; for each agent goal, planning an agent trajectory based on the agent goal and the first observed agent state; receiving a second observed agent state corresponding to a second time instant later than the first time instant; for each goal, comparing the second observed agent state with the at least one agent trajectory planned for the goal, and thereby computing a likelihood of the goal and/or the planned agent trajectory for the goal. Another aspect pertains to trajectory generation, e.g., within a motion planner.
A computer-implemented method of evaluating the performance of a target planner for an ego robot comprises receiving evaluation data for evaluating the performance of the target planner in the scenario, generated by applying the target planner at incrementing planning steps, to compute a series of ego plans that respond to changes in the scenario and are implemented in the scenario to cause changes in an ego state. The evaluation data includes the ego plan computed by the target planner at one of the planning steps, and a scenario state at a time instant of the scenario. The evaluation data is used to evaluate the target planner by computing a reference plan for said time instant based on the scenario state, the scenario state including the ego state at that time instant, and computing at least one evaluation score for comparing the ego plan with the reference plan.
B60W 50/00 - Détails des systèmes d'aide à la conduite des véhicules routiers qui ne sont pas liés à la commande d'un sous-ensemble particulier
B60W 50/06 - Détails des systèmes d'aide à la conduite des véhicules routiers qui ne sont pas liés à la commande d'un sous-ensemble particulier pour améliorer la réponse dynamique du système d'aide à la conduite, p. ex. pour améliorer la vitesse de régulation, ou éviter le dépassement de la consigne ou l'instabilité
B60W 60/00 - Systèmes d’aide à la conduite spécialement adaptés aux véhicules routiers autonomes
G05B 13/02 - Systèmes de commande adaptatifs, c.-à-d. systèmes se réglant eux-mêmes automatiquement pour obtenir un rendement optimal suivant un critère prédéterminé électriques
Systems and method for updating an accumulated 3D map are disclosed. A current point cloud is received, which is an untwisted lidar point cloud captured over a current interval, each point in the current point cloud associated with feature data indicating a feature type of each point of the current point cloud. Respective subsets of the current point cloud are provided to a plurality of processing threads, wherein each processing thread operates on its point cloud subset in parallel with the other processing thread(s) to perform the following mapping operations: compare each point of its point cloud subset with features of the accumulated 3D map to identify a corresponding feature of the same feature type in the accumulated 3D map, compute a distance between each point of its subset and the corresponding feature in the accumulated point cloud, and determine a derivative of each distance with respect to lidar pose change. The derivatives computed by the plurality of processing threads are used to: (i) calculate a refined pose change estimate over the current time interval, and (ii) augment the accumulated 3D map with the untwisted current point cloud using the refined pose change estimate.
For generating driving scenarios for testing an autonomous vehicle planner in a simulation environment, a scenario model comprises a scenario variable and a distribution associated with the scenario variable. The scenario variable is a road layout variable. Multiple sampled values of the scenario variable are computed based on the distribution associated the scenario variable. Based on the scenario model, multiple driving scenarios are generated for testing an autonomous vehicle planner in a simulation environment, each driving scenario comprising a road layout generated using a sampled value of said multiple sampled values of the scenario variable.
The problem of domain shift error in computer vision models and other perception components is addressed. In a label approximation phase, an approximate label distribution is computed for each input of a target batch using a trained machine learning (ML) perception component. In an online label optimization phase, a modified label distribution is assigned to each input of the target batch, via optimization of an unsupervised loss function that (i) penalizes divergence between the approximate label distribution and the modified label distribution for each input of the target batch (ii) penalizes deviation between the modified label distributions assigned to input pairs of the target batch having similar features.
A computer-implemented method of predicting behaviour of an agent for executing an objective of a mobile robot in the vicinity of the agent, in dependence on the predicted behaviour comprises: determining a reference path, wherein multiple actions are available to the agent, and the reference path relates to one of those actions; projecting a measured velocity vector of the agent onto a reference path, thereby determining a projected speed value for the agent along the reference path; computing predicted agent motion data for the agent along the reference path based on the projected speed value; and generating a series of control signals for controlling a mobile robot to fulfil the objective in dependence on the predicted agent motion data.
A computer-implemented method of evaluating the performance of a trajectory planner for a mobile robot in a scenario, in which the trajectory planner is used to control the mobile robot responsive to at least one other agent of the scenario the method comprising: determining a scenario parameter set for the scenario and a likelihood of the scenario parameter set; computing an impact score for a failure event or near failure event between the mobile robot and the other agent occurring in the scenario instance, the impact score quantifying severity of the failure event or near failure event; and computing a risk score for the instance of the scenario based on the impact score and the likelihood of the scenario parameter set.
In one aspect, hierarchical image segmentation is applied to an image formed of a plurality of pixels, by classifying the pixels according to a hierarchical classification scheme, in which at least some of those pixels are classified by a parent level classifier in relation to a set of parent classes, each of which is associated with a subset of child classes, and each of those pixels is also classified by at least one child level classifier in relation to one of the subsets of child classes, wherein each of the parent classes corresponds to a category of visible structure, and each of the subset of child classes associated with it corresponds to a different type of visible structure within that category.
G06K 9/62 - Méthodes ou dispositions pour la reconnaissance utilisant des moyens électroniques
G06F 18/214 - Génération de motifs d'entraînementProcédés de Bootstrapping, p. ex. ”bagging” ou ”boosting”
G06F 18/243 - Techniques de classification relatives au nombre de classes
G06K 9/46 - Extraction d'éléments ou de caractéristiques de l'image
G06N 3/084 - Rétropropagation, p. ex. suivant l’algorithme du gradient
G06V 10/40 - Extraction de caractéristiques d’images ou de vidéos
G06V 10/44 - Extraction de caractéristiques locales par analyse des parties du motif, p. ex. par détection d’arêtes, de contours, de boucles, d’angles, de barres ou d’intersectionsAnalyse de connectivité, p. ex. de composantes connectées
G06V 10/75 - Organisation de procédés de l’appariement, p. ex. comparaisons simultanées ou séquentielles des caractéristiques d’images ou de vidéosApproches-approximative-fine, p. ex. approches multi-échellesAppariement de motifs d’image ou de vidéoMesures de proximité dans les espaces de caractéristiques utilisant l’analyse de contexteSélection des dictionnaires
G06V 10/764 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant la classification, p. ex. des objets vidéo
G06V 10/771 - Sélection de caractéristiques, p. ex. sélection des caractéristiques représentatives à partir d’un espace multidimensionnel de caractéristiques
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
G06V 20/56 - Contexte ou environnement de l’image à l’extérieur d’un véhicule à partir de capteurs embarqués
A computer system comprising: computer storage configured to store a static road layout; a topological indexing component configured to generate an in-memory topological index of the static road layout, the topological index in the form of a graph of nodes and edges, wherein each node corresponds to a road structure element of the static road layout, and the edges encode topological relationships between the road structure elements; a geometric indexing component configured to generate at least one in-memory geometric index of the static road layout for mapping geometric constraints to road structure elements of the static road layout; and a scenario query engine configured to receive a geometric query, search the geometric index to locate at least one static road element satisfying one or more geometric constraints of the geometric query, and return a descriptor of the at least one road structure element(s), wherein the scenario query engine is configured to receive a topological query comprising a descriptor of at least one road element, search the topological index to locate the corresponding node(s), identify at least one other node satisfying the topological query based on the topological relationships encoded in the edges of the topological index, and return a descriptor of the other node(s) satisfying the topological query.
A computer system configured to run queries on a static road layout, the computer system comprising: computer storage configured to store the static road layout, the static road layout comprising a section of road having a set of multiple road attributes, each road attribute described throughout the section of road by a describing function that exhibits a change in form at one or more change points along the section of road, the change points of a first of the road attributes exhibiting longitudinal misalignment with respect to the change points of a second of the road attributes; a road partitioning component configured to process the static road layout, and thereby partition the section of road into a sequence of road parts, each road part defined by a longitudinal coordinate interval, in which the describing function of every one of the road attributes has a form that is fixed throughout; a road indexing component configured to generate a road partition index having an entry for each road part, the entry indicating the form of the describing function of each road attribute as fixed throughout the longitudinal coordinate interval of that road part; and a scenario query engine configured to receive a part query, locate the entry in the road partition index for one of the road parts based on the part query, evaluate the describing function of at least one of the road attributes within the road part, and generate a part query response based on the evaluation of the describing function.
