The technical problem of reducing the amount of processing involved when searching for customizable media content items that are suitable for incorporating input text is addressed by providing a hybrid search system. In some examples, the hybrid search system executes a rough search first, to determine whether a line of text can be incorporated into a media content item, based on character count conditions associated with the media content item. A more thorough evaluation of the input text with respect to the media content item is executed subsequent to the rough search if the rough search produces a result indicating uncertainty with respect to whether the combination of specific characters included in the input text can or cannot be incorporated into the media content item.
Aspects of the present disclosure involve a system for performing ray tracing between augmented reality (AR) and real-world objects. The system accesses, by the mobile device, a video depicting a first object. The system obtains, by the mobile device, a three-dimensional (3D) model of the first object. The system applies, by the mobile device, a ray tracing process to the 3D model of the first object to estimate an optical effect on a portion of the first object relative to a second object that is depicted in the video. The system modifies a visual property of the portion of the first object based on the optical effect relative to the second object.
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing make-up based augmented reality content. The program and method provide for receiving a request to present augmented reality content in association with a captured image depicting a face of the user; accessing an augmented reality content item associated with applying makeup to the face and configured to generate a mesh for tracking plural regions of the face; receiving user input selecting a region; determining at least one of a range of color values or a range of contrast values relating to available makeup products for the selected region; and presenting an interface element in association with the face, the interface element for user selection of at least one of a color value within the range of color values or a contrast value within the range of contrast values.
A45D 44/00 - Autres articles cosmétiques ou de toilette, p. ex. pour salons de coiffure
G06F 3/04845 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] pour la commande de fonctions ou d’opérations spécifiques, p. ex. sélection ou transformation d’un objet, d’une image ou d’un élément de texte affiché, détermination d’une valeur de paramètre ou sélection d’une plage de valeurs pour la transformation d’images, p. ex. glissement, rotation, agrandissement ou changement de couleur
G06F 3/04847 - Techniques d’interaction pour la commande des valeurs des paramètres, p. ex. interaction avec des règles ou des cadrans
G06F 3/04883 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] utilisant des caractéristiques spécifiques fournies par le périphérique d’entrée, p. ex. des fonctions commandées par la rotation d’une souris à deux capteurs, ou par la nature du périphérique d’entrée, p. ex. des gestes en fonction de la pression exercée enregistrée par une tablette numérique utilisant un écran tactile ou une tablette numérique, p. ex. entrée de commandes par des tracés gestuels pour l’entrée de données par calligraphie, p. ex. sous forme de gestes ou de texte
A media player providing real time rewind playback of a played media file having segments of frames. A last segment N of the played media file is cached and rendered on a device, such as a mobile device, then a previous segment N-1 is cached and rendered, and the process continues until there are no more segments of the played media file to cache and render. Only a segment of the played media file is cached at a time, rather than the whole media file, such that the played media file can be replayed on the fly.
H04N 21/433 - Opération de stockage de contenu, p. ex. opération de stockage en réponse à une requête de pause ou opérations de cache
G06F 3/04883 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] utilisant des caractéristiques spécifiques fournies par le périphérique d’entrée, p. ex. des fonctions commandées par la rotation d’une souris à deux capteurs, ou par la nature du périphérique d’entrée, p. ex. des gestes en fonction de la pression exercée enregistrée par une tablette numérique utilisant un écran tactile ou une tablette numérique, p. ex. entrée de commandes par des tracés gestuels pour l’entrée de données par calligraphie, p. ex. sous forme de gestes ou de texte
H04N 19/177 - Procédés ou dispositions pour le codage, le décodage, la compression ou la décompression de signaux vidéo numériques utilisant le codage adaptatif caractérisés par l’unité de codage, c.-à-d. la partie structurelle ou sémantique du signal vidéo étant l’objet ou le sujet du codage adaptatif l’unité étant un groupe d’images [GOP]
H04N 19/426 - Procédés ou dispositions pour le codage, le décodage, la compression ou la décompression de signaux vidéo numériques caractérisés par les détails de mise en œuvre ou le matériel spécialement adapté à la compression ou à la décompression vidéo, p. ex. la mise en œuvre de logiciels spécialisés caractérisés par les dispositions des mémoires utilisant des procédés de diminution de taille de mémoire
5.
PRESENTING PARTICIPANT REACTIONS WITHIN A VIRTUAL WORKING ENVIRONMENT
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for presenting participant reactions within a virtual working environment. The program and method provide a configuration interface for configuring a virtual working environment for plural participants, the configuration interface for specifying groups of participants, each group comprising respective participants selected from among the plural participants; receive first user input, provided via the configuration interface, specifying a first group of participants; provide, for each participant in the first group, display of a reactions interface with user-selectable buttons to indicate respective reactions for displaying to the first group; receive second user input, provided via the reactions interface, selecting one of the user-selectable buttons to indicate a reaction for displaying to the first group; and provide, for each participant in the first group, display of a reaction icon corresponding to the reaction.
G06F 3/04815 - Interaction s’effectuant dans un environnement basé sur des métaphores ou des objets avec un affichage tridimensionnel, p. ex. modification du point de vue de l’utilisateur par rapport à l’environnement ou l’objet
G06F 3/04817 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] fondées sur des propriétés spécifiques de l’objet d’interaction affiché ou sur un environnement basé sur les métaphores, p. ex. interaction avec des éléments du bureau telles les fenêtres ou les icônes, ou avec l’aide d’un curseur changeant de comportement ou d’aspect utilisant des icônes
A system and method provide feedback to a user, such as a visually impaired user, to guide the user to an object in the field of view of a camera mounted on a frame worn on the head of the user. A processor identifies at least one object and a body part of the user in the field of view of the camera and tracks relative positions of the body part relative to the identified object. The processor also generates and communicates at least one control signal for guiding the body part of the user to the identified object to a user feedback device worn on or adjacent the body part of the user. The feedback device receives the control signal(s) and converts the control signal(s) into at least one of sounds or haptic feedback that guides the body part to the identified object.
A mobile application with an improved user interface facilitates generating stylized media content items including images and videos. An end-user selects a desired visual effect from a set of options. The mobile application captures or accesses an image. The image is processed on a server using a generative neural network pre-trained to apply stylizations based on the selected effect. The server sends back the stylized image to the mobile application for display. The end-user can then save the stylized image or generate a video (e.g., an animation) showing the original image transition to the stylized image. The user interface provides an efficient creative workflow to apply aesthetic enhancements in a visual style chosen by the end-user. Generative machine learning techniques automate stylization to enable accessible media customization and sharing.
G06T 11/60 - Édition de figures et de texteCombinaison de figures ou de texte
G06F 3/0482 - Interaction avec des listes d’éléments sélectionnables, p. ex. des menus
G06F 3/0488 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] utilisant des caractéristiques spécifiques fournies par le périphérique d’entrée, p. ex. des fonctions commandées par la rotation d’une souris à deux capteurs, ou par la nature du périphérique d’entrée, p. ex. des gestes en fonction de la pression exercée enregistrée par une tablette numérique utilisant un écran tactile ou une tablette numérique, p. ex. entrée de commandes par des tracés gestuels
G06T 13/80 - Animation bidimensionnelle [2D], p. ex. utilisant des motifs graphiques programmables
A convolutional neural network processing system includes a data processor to process input feature map data and generate output feature map data in an output feature map storage space. Series of output feature data elements are stored at respective series of mutually successive locations in the output feature map storage space. The data processor identifies an input feature data element in the input feature map data, and accesses a series of mutually successive locations in the output feature map storage space. The data processor updates the output feature map data in an input-centric manner by using an update function to update a set of output feature data elements associated with the input feature data element by a convolution kernel of the update function. The set of output feature data elements is located in the accessed series of mutually successive locations.
Aspects of the present disclosure involve a system and a method for navigating images and AR experiences. The system and method present, by a messaging application, in a scrollable region on top of a live video feed being displayed in a GUI comprising a viewfinder, a first plurality of options associated with previously captured content items and a second plurality of options associated with AR experiences. The system and method scrolls the first plurality of options together with the second plurality of options of the scrollable region to bring one of the first plurality of options or one of the second plurality of options into focus. In response, the system and method: modify a configuration of the GUI in accordance with a first manner or modify the configuration of the GUI in accordance with a second manner.
G06F 3/04845 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] pour la commande de fonctions ou d’opérations spécifiques, p. ex. sélection ou transformation d’un objet, d’une image ou d’un élément de texte affiché, détermination d’une valeur de paramètre ou sélection d’une plage de valeurs pour la transformation d’images, p. ex. glissement, rotation, agrandissement ou changement de couleur
G06F 3/04815 - Interaction s’effectuant dans un environnement basé sur des métaphores ou des objets avec un affichage tridimensionnel, p. ex. modification du point de vue de l’utilisateur par rapport à l’environnement ou l’objet
G06F 3/0482 - Interaction avec des listes d’éléments sélectionnables, p. ex. des menus
G06F 3/0485 - Défilement ou défilement panoramique
G06F 3/0488 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] utilisant des caractéristiques spécifiques fournies par le périphérique d’entrée, p. ex. des fonctions commandées par la rotation d’une souris à deux capteurs, ou par la nature du périphérique d’entrée, p. ex. des gestes en fonction de la pression exercée enregistrée par une tablette numérique utilisant un écran tactile ou une tablette numérique, p. ex. entrée de commandes par des tracés gestuels
H04L 51/04 - Messagerie en temps réel ou quasi en temps réel, p. ex. messagerie instantanée [IM]
A system is disclosed, including a processor and a memory. The memory stores instructions that, when executed by the processor, configure the system to perform operations. Autoexposure (AE) primary camera information is obtained, identifying a first camera as an AE primary camera to be used for computing AE settings of the first camera and a second camera. First camera region of interest (ROI) information and second camera ROI information are obtained, representative of a first number of ROIs within a field of view (FOV) of the first camera and a second number of ROIs within a FOV of the second camera. In response to determining that the first number is zero and the second number is greater than zero, the AE primary camera information is updated to identify the second camera as the AE primary camera, and the second camera ROI information is processed to generate the AE settings.
H04N 23/90 - Agencement de caméras ou de modules de caméras, p. ex. de plusieurs caméras dans des studios de télévision ou des stades de sport
H04N 23/611 - Commande des caméras ou des modules de caméras en fonction des objets reconnus les objets reconnus comprenant des parties du corps humain
H04N 23/73 - Circuits de compensation de la variation de luminosité dans la scène en influençant le temps d'exposition
The present disclosure seeks to address technical problems arising in the field of artificial intelligence (AI) by providing for training of a machine learning model to generate modified images based on an input image and an input instruction. For example, the machine learning model is trained to generate a modified portrait image based on an input portrait image and an input instruction. The machine learning model generates the modified portrait image to depict the input portrait image as modified according to the input instruction while maintaining the identity of a subject depicted in the input portrait image.
G06T 11/60 - Édition de figures et de texteCombinaison de figures ou de texte
G06V 10/774 - Génération d'ensembles de motifs de formationTraitement des caractéristiques d’images ou de vidéos dans les espaces de caractéristiquesDispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant l’intégration et la réduction de données, p. ex. analyse en composantes principales [PCA] ou analyse en composantes indépendantes [ ICA] ou cartes auto-organisatrices [SOM]Séparation aveugle de source méthodes de Bootstrap, p. ex. "bagging” ou “boosting”
13.
DYNAMIC MODEL ADAPTATION CUSTOMIZED FOR INDIVIDUAL USERS
Described is a system for dynamically applying model adaptations customized for individual users by detecting an image of a first real-world object from a camera feed, detecting landmarks on the first real-world object, and processing the landmarks on the first real-world object using a generative machine learning model to generate a first custom image template for the first real-world object where portions of the first custom image template are populated with visual content placed based on the first custom image template. The system then applies a content augmentation based on the first custom image template to the camera feed.
The subject technology receives a set of inputs from multiple input sources. The subject technology determines a set of input features based on the set of inputs from the multiple input sources. The subject technology performs a time window-based aggregation on the set of input features to generate a set of aggregated features. The subject technology performs feature extraction, using a set of modular components of a modular classifier network, on the set of aggregated features to generate a set of extracted features. The subject technology generates, using a pinch detection head, a probability score indicating the likelihood of an occurrence of a pinch gesture based on the set of extracted features. The subject technology determines, using triggering logic, whether a pinch gesture has occurred based at least in part on the probability score. The subject technology provides a pinch detection output based at least in part on the determining.
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06V 10/26 - Segmentation de formes dans le champ d’imageDécoupage ou fusion d’éléments d’image visant à établir la région de motif, p. ex. techniques de regroupementDétection d’occlusion
G06V 10/40 - Extraction de caractéristiques d’images ou de vidéos
G06V 10/764 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant la classification, p. ex. des objets vidéo
G06V 10/80 - Fusion, c.-à-d. combinaison des données de diverses sources au niveau du capteur, du prétraitement, de l’extraction des caractéristiques ou de la classification
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
G06V 40/20 - Mouvements ou comportement, p. ex. reconnaissance des gestes
15.
AUGMENTED REALITY CONTENT GENERATORS FOR IDENTIFYING DESTINATION GEOLOCATIONS AND PLANNING TRAVEL
The subject technology causes display, at the client device, a set of augmented reality content items generated by the first augmented reality content generator. The subject technology receives, at the client device, a second selection of the particular augmented reality content item corresponding to the destination geolocation. The subject technology causes display, at the client device, a second set of augmented reality content items generated by the first augmented reality content generator. The subject technology receives, at the client device, a second selection of the second set of augmented reality content items. The subject technology causes display, at the client device, a third set of augmented reality content items generated by the first augmented reality content generator, the third set of augmented reality content items comprising at least one activity or location associated with the destination geolocation and a selected period of time.
Provided are systems and methods for providing personalized videos featuring multiple persons. An example method includes receiving a video including a plurality of frames including at least one target face, extracting target face parameters associated with the at least one target face, where the target face parameters include facial identity parameters and facial expression parameters, storing the target face parameters as metadata associated with at least one frame of the plurality of frames, receiving an image of a source face, generating source face parameters based on the image of the source face, generating an output face by combining the source face parameters with the facial expression parameters obtained from the metadata associated with the at least one frame, and generating a personalized video by replacing the at least one target face with the output face at least in the at least one frame.
G06V 40/16 - Visages humains, p. ex. parties du visage, croquis ou expressions
G11B 27/02 - Montage, p. ex. variation de l'ordre des signaux d'information enregistrés sur, ou reproduits à partir des supports d'enregistrement ou d'information
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for configuring a three-dimensional (3D) model within a virtual conferencing system. The program and method provide, in association with designing a room for virtual conferencing, an interface for configuring a 3D model; receiving, via the interface, an indication of user input for setting properties for the 3D model, the properties specifying image data for projecting onto the 3D model; and in association with virtual conferencing, providing display of the room based on the properties for the 3D model, and causing the image data to be projected onto the 3D model within the room.
G06F 3/04815 - Interaction s’effectuant dans un environnement basé sur des métaphores ou des objets avec un affichage tridimensionnel, p. ex. modification du point de vue de l’utilisateur par rapport à l’environnement ou l’objet
G06F 3/04845 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] pour la commande de fonctions ou d’opérations spécifiques, p. ex. sélection ou transformation d’un objet, d’une image ou d’un élément de texte affiché, détermination d’une valeur de paramètre ou sélection d’une plage de valeurs pour la transformation d’images, p. ex. glissement, rotation, agrandissement ou changement de couleur
G06F 3/04847 - Techniques d’interaction pour la commande des valeurs des paramètres, p. ex. interaction avec des règles ou des cadrans
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
G06T 19/20 - Édition d'images tridimensionnelles [3D], p. ex. modification de formes ou de couleurs, alignement d'objets ou positionnements de parties
Eyewear providing an interactive augmented reality experience to allow a user of an eyewear device to display a 3D overlay image on a viewed person. The user can select the overlay image from a list of images, such as costumes, stored in memory or generated by the user. The images can be sorted in memory based on common attributes. Registration points of the person are continuously aligned with registration points of the overlay as the person moves such that the user appears to be wearing the 3D costume during movement. By aligning the registration points, the costume adapts to different body types and heights. The coloring of the costume can change based on the environment, such as the lighting, or to contrast with colors viewed in a viewfinder.
Disclosed is a method of providing a music creation interface using a head-mounted device, including displaying first and second geometric loops fixed relative to a location in the real world, the first and second geometric loops each including a plurality of beat indicators. The second geometric loop is spaced apart from the first geometric loop. An interface comprising a plurality of sound or note icons is displayed, and in response to receiving user selection to move a selected sound or note icon to a particular beat indicator on one of the geometric loops, the selected sound or note icon is displayed at the particular beat indicator. In use, the geometric loops are rotated relative to at least one play indicator, and the selected sound or note icon is rendered when it reaches the at least one play indicator.
The subject technology requests a group identifier (ID) based on an item identification indicator to an extension application programming interface (API). The subject technology requests, using the extension API, a set of augmented reality (AR) content generator IDs, based on the item identification indicator, to a shopping AR content generator service. The subject technology receives the set of AR content generator IDs based on a mapping of the item identification indicator to a collection of AR content generator IDs. The subject technology requests, using the extension API, first metadata associated with the set of AR content generators and the item identification indicator to the shopping AR content generator service. The subject technology requests, using the extension API, a set of products based on the first metadata. The subject technology generates, using the extension API, a group ID based on the set of products. The subject technology receives the group ID.
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for performing operations comprising: receiving, from a client device of a first user, a request from the first user to engage in an AR shopping experience curated by a store; identifying a first real-world product available for purchase from the store; receiving an image of a real-world environment of the first user; generating a first AR item that represents the first real-world product; comparing visual attributes of the first AR item to physical layouts of a plurality of real-world objects depicted in the image of the real-world environment; and overlaying the first AR item on a first real-world object of the plurality of real-world objects in the image responsive to comparing the visual attributes of the first AR item to the physical layouts of the plurality of real-world objects.
G06K 7/14 - Méthodes ou dispositions pour la lecture de supports d'enregistrement par radiation électromagnétique, p. ex. lecture optiqueMéthodes ou dispositions pour la lecture de supports d'enregistrement par radiation corpusculaire utilisant la lumière sans sélection des longueurs d'onde, p. ex. lecture de la lumière blanche réfléchie
G06Q 10/087 - Gestion d’inventaires ou de stocks, p. ex. exécution des commandes, approvisionnement ou régularisation par rapport aux commandes
A system and method for generating augmented reality (AR) experiences are disclosed. The system generates source and target indications associated with an image transformation, and generates a first set of source images and first set of target images using a first trained machine learning (ML) model, the source indications, and the target indications. The system trains a second ML model to generate a target image corresponding to a source image based on the first set of source images and the first set of target images, and generates a second set of target images using the second trained ML model and a second set of source images. The system trains a third ML model to generate an additional target image corresponding to an additional source image based on the second set of source images and second set of target images, and generates an AR experience comprising the third trained ML model.
Systems and methods are presented for capturing a video in real-time by an image capture device using a skeletal pose system. The skeletal pose system identifies first pose information in the video, applies a first virtual effect to the video in response to identifying the first pose information, identifies second pose information in the video, and applies a second virtual effect to the video in response to identifying the first pose information.
Apparatuses, systems for electronic wearable devices such as smart glasses are described. The wearable device can comprise a frame, an elongate temple and an articulated joint. The frame can define one or more optical element holders configured to hold respective optical elements for viewing by a user in a viewing direction. The temple can be moveably connected to the frame for holding the frame in position when the device is worn by the user. The articulated joint can connect the temple and the frame to permit movement of the temple relative to the frame between a wearable position in which the temple is generally aligned with the viewing direction, and a collapsed position in which the temple extends generally transversely to the viewing direction. The articulated joint can include a base foot fixed to the frame and oriented transversely to the viewing direction.
A method for managing power resource in an augmented reality (AR) device is described. In one aspect, the method includes configuring a low-power mode to run on a low-power processor of the AR device using a first set of sensor data, and a high-power mode to run on a high-power processor of the AR device using a second set of sensor data, operating, using the low-power processor, a low-power application in the low-power mode based on the first set of sensor data, detecting a request to operate a high-power application at the AR device, in response to detecting the request, activating the second set of sensors of the AR device corresponding to the high-power mode, and operating, using the high-power processor, a high-power application in the high-power mode based on the second set of sensors.
Hierarchical patch-wise diffusion models (HPDMs) use a diffusion paradigm that learns a hierarchical distribution of patches instead of whole videos for efficient patch-wise training of diffusion models. To enforce consistency between the patches, deep context fusion may be used to propagate the context information from low-scale to high-scale patches in a hierarchical manner. To accelerate patch-wise training and inference, adaptive computation also may be used to allocate more computational resources and network capacity towards coarse image details and to cheapen synthesis of high-frequency texture details. All the processing stages are jointly trained to provide spatially aligned global context to the higher levels of the cascade. As a result, the model does not operate on the full-resolution inputs, which allows the model to be trained on high-resolution video datasets in an end-to-end fashion.
G06T 3/4007 - Changement d'échelle d’images complètes ou de parties d’image, p. ex. agrandissement ou rétrécissement basé sur l’interpolation, p. ex. interpolation bilinéaire
G06T 3/4046 - Changement d'échelle d’images complètes ou de parties d’image, p. ex. agrandissement ou rétrécissement utilisant des réseaux neuronaux
G06T 5/60 - Amélioration ou restauration d'image utilisant l’apprentissage automatique, p. ex. les réseaux neuronaux
A messaging system performs image processing to relight objects with neural networks for images provided by users of the messaging system. A method of relighting objects with neural networks includes receiving an input image with first lighting properties comprising an object with second lighting properties and processing the input image using a convolutional neural network to generate an output image with the first lighting properties and comprising the object with third lighting properties, where the convolutional neural network is trained to modify the second lighting properties to be consistent with lighting conditions indicated by the first lighting properties to generate the third lighting properties. The method further includes modifying the second lighting properties of the object to generate the object with modified second lighting properties and blending the third lighting properties with the modified second lighting properties to generate a modified output image comprising the object with fourth lighting properties.
Described is a system for emphasizing XR content based on user intent by gathering interaction data from use of one or more interaction functions by a user, accessing a camera feed of a camera system from the XR device, analyzing a combination of data corresponding to the interaction data and the camera feed using a first machine learning model to identify a priority for individual media content items, and determining that a first subset of media content items are of a higher priority than a second subset of media content items. Then the system displays the media content items on the XR device of the user, the first subset of the media content items displayed differently than the second subset of the media content items based on the identified priority.
Described herein are techniques for facilitating the communication of text-based messages between end-users who are using messaging applications executing on client-based computing devices with different capabilities. Specifically, the messaging system described herein enables a first end-user to add a message element to a text-based message, which, when received by a message recipient using an augmented reality messaging application, will cause a 3-D avatar representing the message sender, to animate in accordance with a specific avatar animation associated with the message element. The message element may be an emoji, or a special sequence of characters, and may be a visible or invisible (e.g., meta-data) element of the text-based message.
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
G06T 13/40 - Animation tridimensionnelle [3D] de personnages, p. ex. d’êtres humains, d’animaux ou d’êtres virtuels
G10L 13/08 - Analyse de texte ou génération de paramètres pour la synthèse de la parole à partir de texte, p. ex. conversion graphème-phonème, génération de prosodie ou détermination de l'intonation ou de l'accent tonique
Examples include a wearable device such as smart glasses having a frame, a temple and onboard electronics components. The frame can define one or more optical element holders for holding respective optical elements within view of a user when the eyewear body is worn. The pair of temples can be connected to the eyewear frame for supporting the eyewear frame in position within view of the user when the eyewear body is worn. The antenna can be incorporated in at least a first of the pair of temples. The antenna can include an exciter circuit, at least a portion of a battery flex of the first of the pair of temples configured as an active element of the antenna and a metal component of the first of the pair of temples configured as a ground of the antenna.
Methods, systems, mobile devices, and non-transitory computer-readable mediums for easily aesthetically enhancing images such as selfies. An example algorithm's input has three parts: image, manipulation magnitude, and text guidance. The algorithm includes two parts: (1) guidance generation based on public and personal aesthetic preferences, and (2) selfie generation. The first part outputs an image to maximize an aesthetic enhancement score (e.g., a beauty score) while following the manipulation input where the output image contains a manipulation direction. The second part is a conditional diffusion model that accepts the rendered output image from the first part and is conditioned on the input image and outputs the final image. The second part is personalized by the user's images.
Systems, methods, and computer readable media that schedules requests for location data of a mobile device, where the methods include selecting a first positioning system based on a power requirement, a latency requirement, and an accuracy requirement, and determining whether a first condition is satisfied for querying the first positioning system. The method further comprises in response to a determination that the first condition is satisfied, querying the first positioning system for first position data. The method further comprises in response to a determination that the first condition is not satisfied, selecting a second positioning system based on the power requirement, the latency requirement, and the accuracy requirement, determining whether a second condition is satisfied for querying the second positioning system, and in response to a determination that the second condition is satisfied, querying the second positioning system for second position data.
G01S 5/00 - Localisation par coordination de plusieurs déterminations de direction ou de ligne de positionLocalisation par coordination de plusieurs déterminations de distance
G01S 5/02 - Localisation par coordination de plusieurs déterminations de direction ou de ligne de positionLocalisation par coordination de plusieurs déterminations de distance utilisant les ondes radioélectriques
G01S 19/48 - Détermination de position en combinant ou en commutant entre les solutions de position dérivées du système de positionnement par satellite à radiophares et les solutions de position dérivées d'un autre système
An optical waveguide device for use in a head up display. The waveguide device provides pupil expansion in two dimensions. The waveguide device comprise a primary waveguide and a secondary waveguide, the secondary waveguide being positioned on a face of the primary waveguide. The secondary waveguide has a diffraction grating on a face opposite to the face which contacts the primary waveguide. The diffraction grating diffracts light into more than diffraction order. Rays diffracted into a non-zero order are trapped in the secondary waveguide by total internal reflection.
An addressable media system for performing operations that include: accessing image data that depicts an object in an environment at a client device; causing display of a presentation of the image data within a graphical user interface at the client device; detecting the display of the object within the presentation of the image data based on at least a portion of the plurality of image features of the display of the object; identifying an object class based on at least the portion of the image features of the display of the object; receiving an input that selects the display of the object from the client device; and associating the object class that corresponds with the object with the user profile in response to the input that selects the display of the object.
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06F 3/0481 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] fondées sur des propriétés spécifiques de l’objet d’interaction affiché ou sur un environnement basé sur les métaphores, p. ex. interaction avec des éléments du bureau telles les fenêtres ou les icônes, ou avec l’aide d’un curseur changeant de comportement ou d’aspect
G06F 3/04842 - Sélection des objets affichés ou des éléments de texte affichés
G06F 3/14 - Sortie numérique vers un dispositif de visualisation
Systems and methods for animating a single image of a human body and applying effects are provided. An example method includes receiving an image including hair, fitting a portion of the image including the hair to a hair model, generating a hair motion control parameter set defining hair animation behavior using one or more pre-set physical parameters, determining deformation of the hair model based on the hair motion control parameter set, generating a hair image based on the deformation of the hair model, and compositing the hair image with a frame of a video generated from the image.
G06T 13/40 - Animation tridimensionnelle [3D] de personnages, p. ex. d’êtres humains, d’animaux ou d’êtres virtuels
G06F 3/0482 - Interaction avec des listes d’éléments sélectionnables, p. ex. des menus
G06F 3/04845 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] pour la commande de fonctions ou d’opérations spécifiques, p. ex. sélection ou transformation d’un objet, d’une image ou d’un élément de texte affiché, détermination d’une valeur de paramètre ou sélection d’une plage de valeurs pour la transformation d’images, p. ex. glissement, rotation, agrandissement ou changement de couleur
G06T 7/194 - DécoupageDétection de bords impliquant une segmentation premier plan-arrière-plan
Methods and systems are disclosed for applying augmented reality animations to an image. The methods and systems access an image and select an augmented reality (AR) experience to apply to the image, the AR experience comprising one or more AR element animations. The methods and systems repeatedly apply the AR experience to the image over a specified time interval to overlay the one or more AR element animations on the image. The methods and systems generate a video having a duration corresponding to the specified time interval in response to repeatedly applying the AR experience to the image, the video depicting the one or more AR element animations on the image.
Display systems, media, and methods. Pixels of an image former form an image. Each pixel has a first color element to propagate a first amount of a first light having a first wavelength, based on a first electrical stimulus, and a second color element to propagate a second amount of a second light having a second wavelength, based on a second electrical stimulus. A display surface presents the image across a plurality of display surface locations. A processor executes instructions to scale the first electrical stimulus of each pixel of the image former by a first light scale factor, scale the second electrical stimulus of each pixel of the image former by a second light scale factor, and apply a pixel shading map to the image former to independently adjust, for each pixel of the display, the first amount relative to the second amount.
G09G 3/20 - Dispositions ou circuits de commande présentant un intérêt uniquement pour l'affichage utilisant des moyens de visualisation autres que les tubes à rayons cathodiques pour la présentation d'un ensemble de plusieurs caractères, p. ex. d'une page, en composant l'ensemble par combinaison d'éléments individuels disposés en matrice
A mobile vision transformer network for use on mobile devices, such as smart eyewear devices and other augmented reality (AR) and virtual reality (VR) devices. The mobile vision transformer network considers factors including number of parameters, latency, and model performance, as they reflect disk storage, mobile frames per second (FPS), and application quality, respectively. The mobile vision transformer network processes images, e.g., for image classification, segmentation, and detection. The mobile vision transformer network has a fine-grained architecture including a search algorithm performing latency-driven slimming that jointly improves model size and speed.
G06V 10/94 - Architectures logicielles ou matérielles spécialement adaptées à la compréhension d’images ou de vidéos
G06V 10/77 - Traitement des caractéristiques d’images ou de vidéos dans les espaces de caractéristiquesDispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant l’intégration et la réduction de données, p. ex. analyse en composantes principales [PCA] ou analyse en composantes indépendantes [ ICA] ou cartes auto-organisatrices [SOM]Séparation aveugle de source
Examples relate to systems and methods for generating an avatar animation. The systems and methods access an audio file comprising speech, spoken by a user, captured by a microphone of a user system, and receive input that selects an avatar associated with the user. The systems and methods process the audio file and the avatar, selected by the received input, by a generative machine learning model to generate an animation of the avatar having lips moving to represent the avatar speaking the speech of the audio file. The systems and methods generate a video comprising a depiction of the generated animation of the avatar speaking the speech of the audio file.
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for performing operations comprising receiving an image that depicts a person, identifying a set of skeletal joints of the person and identifying a pose of the person depicted in the image based on positioning of the set of skeletal joints. The operations also include receiving speech input comprising a request to perform an AR operation and an ambiguous intent, discerning the ambiguous intent of the speech input based on the pose of the person depicted in the image and in response to receiving the speech input, performing the AR operation based on discerning the ambiguous intent of the speech input based on the pose of the person depicted in the image.
Users of a chat system within an interactive platform can suspend the expiration of a plurality of content items. Example methods include generating a chat, the chat comprising an association between a first user account and a second user account, and receiving, from a first user system associated with the first user account, an indication of a plurality of content items and an indication of the chat. The method may further include sending, to a second user account, the plurality of content items and an indication of the chat, and receiving, from the second user account, an indication to save the plurality of content items within the chat. The method may further include setting a saved data field associated with the plurality of content items and the second user account, the saved data field indicating the plurality of content items do not expire within the chat.
Examples in the present disclosure relate to systems and methods for detecting and rejecting a non-user hand in the context of egocentric hand tracking performed by an extended reality (XR) device. While the XR device is worn by a user, the XR device captures at least one image of a hand and processes the at least one image to detect the hand. After detecting the hand, the XR device determines positioning of the hand relative to the XR device or another object in a field of view of the XR device. The XR device detects that the hand is a non-user hand. In response to detecting that the hand is a non-user hand, the XR device excludes the non-user hand from the egocentric hand tracking such that the non-user hand is not tracked for the user.
A case for a portable device like a smartphone includes light sources such as LEDs, which, when illuminated, can be detected and tracked by a head-worn augmented or virtual reality device. The light sources may be located at the corners of the case and may emit infrared light. A relative pose between the smartphone and the head-worn device can be determined based on computer vision techniques performed on images captured by the head-worn device that includes light from the light sources. Relative movement between the smartphone and the head-worn device can be used to provide user input to the head-worn device, as can touch input on the portable device. In some instances, the case is powered inductively from the portable device.
H04M 1/72409 - Interfaces utilisateur spécialement adaptées aux téléphones sans fil ou mobiles avec des moyens de soutien local des applications accroissant la fonctionnalité par interfaçage avec des accessoires externes
H04B 1/3888 - Dispositions pour le transport ou la protection d’émetteurs-récepteurs
H04M 1/72454 - Interfaces utilisateur spécialement adaptées aux téléphones sans fil ou mobiles avec des moyens permettant d’adapter la fonctionnalité du dispositif dans des circonstances spécifiques en tenant compte des contraintes imposées par le contexte ou par l’environnement
Examples in the present disclosure relate generally to techniques for automated organization and presentation of user-selectable features within user interfaces. More particularly, but not by way of limitation, the present disclosure addresses systems and methods for presenting and accessing image modifiers within user interfaces displaying media content items. In some examples, a modifier carousel comprises a set of interface elements, with each interface element within the set of interface elements representing a respective image modifier within the set of image modifiers. Each interface element within the set of interface elements is user-selectable to apply the respective image modifier to the content item.
G06F 3/0482 - Interaction avec des listes d’éléments sélectionnables, p. ex. des menus
G06F 3/04847 - Techniques d’interaction pour la commande des valeurs des paramètres, p. ex. interaction avec des règles ou des cadrans
G06F 3/0488 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] utilisant des caractéristiques spécifiques fournies par le périphérique d’entrée, p. ex. des fonctions commandées par la rotation d’une souris à deux capteurs, ou par la nature du périphérique d’entrée, p. ex. des gestes en fonction de la pression exercée enregistrée par une tablette numérique utilisant un écran tactile ou une tablette numérique, p. ex. entrée de commandes par des tracés gestuels
G06T 11/60 - Édition de figures et de texteCombinaison de figures ou de texte
H04N 21/431 - Génération d'interfaces visuellesRendu de contenu ou données additionnelles
H04N 21/462 - Gestion de contenu ou de données additionnelles, p. ex. création d'un guide de programmes électronique maître à partir de données reçues par Internet et d'une tête de réseau ou contrôle de la complexité d'un flux vidéo en dimensionnant la résolution ou le débit en fonction des capacités du client
46.
EYEWEAR CHARGING CASE USING CHARGE PINS AS COMMUNICATION PATH
A case for an eyewear device having a conductive interface includes a housing that receives the eyewear device. A multi-purpose interface, supported by the housing, includes at least one contact arranged to couple with the conductive interface of the eyewear device when the housing receives the eyewear device. Circuitry is coupled to the at least one contact and includes a processor that detects a connection of the conductive interface of the eyewear device to the multi-purpose interface of the case. The processor performs a charging process during a charge state of the case in which an electrical charge is provided at the multi-purpose interface of the case to the eyewear device. Data is exchanged with the eyewear device during a communication state of the case.
G02F 1/163 - Fonctionnement des cellules électro-chromiques, p. ex. des cellules d’électrodépositionDispositions des circuits à cet effet
G02F 1/17 - Dispositifs ou dispositions pour la commande de l'intensité, de la couleur, de la phase, de la polarisation ou de la direction de la lumière arrivant d'une source lumineuse indépendante, p. ex. commutation, ouverture de porte ou modulationOptique non linéaire pour la commande de l'intensité, de la phase, de la polarisation ou de la couleur basés sur des éléments à absorption variable non prévus dans les groupes
An augmented reality (AR) calibration process for an eyewear device that customizes an AR user interface (UI) for the wearer. Physiological characteristics such as arm length and hand size are determined from images captured by the eyewear device for use in adjusting the UI controls before rendering the UI.
G06F 3/04815 - Interaction s’effectuant dans un environnement basé sur des métaphores ou des objets avec un affichage tridimensionnel, p. ex. modification du point de vue de l’utilisateur par rapport à l’environnement ou l’objet
G06F 3/04845 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] pour la commande de fonctions ou d’opérations spécifiques, p. ex. sélection ou transformation d’un objet, d’une image ou d’un élément de texte affiché, détermination d’une valeur de paramètre ou sélection d’une plage de valeurs pour la transformation d’images, p. ex. glissement, rotation, agrandissement ou changement de couleur
G06F 3/04847 - Techniques d’interaction pour la commande des valeurs des paramètres, p. ex. interaction avec des règles ou des cadrans
G06T 3/40 - Changement d'échelle d’images complètes ou de parties d’image, p. ex. agrandissement ou rétrécissement
Examples relate to systems and methods for generating an avatar animation. The systems and methods access an audio file comprising speech, spoken by a user, captured by a microphone of a user system, and receive input that selects an avatar associated with the user. The systems and methods process the audio file and the avatar, selected by the received input, by a generative machine learning model to generate an animation of the avatar having lips moving to represent the avatar speaking the speech of the audio file. The systems and methods generate a video comprising a depiction of the generated animation of the avatar speaking the speech of the audio file.
G06T 13/40 - Animation tridimensionnelle [3D] de personnages, p. ex. d’êtres humains, d’animaux ou d’êtres virtuels
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
G10L 25/57 - Techniques d'analyse de la parole ou de la voix qui ne se limitent pas à un seul des groupes spécialement adaptées pour un usage particulier pour comparaison ou différentiation pour le traitement des signaux vidéo
G10L 25/63 - Techniques d'analyse de la parole ou de la voix qui ne se limitent pas à un seul des groupes spécialement adaptées pour un usage particulier pour comparaison ou différentiation pour estimer un état émotionnel
49.
Human-machine interaction using EMG speech signals
Methods and systems are disclosed for triggering operations using inner speech. The system detects presence of inner speech in electromyograph (EMG) data. The system adjusts a progress indicator in response to detecting the presence of inner speech. The system determines that a current value of the progress indicator has transgressed a specified threshold value and, in response, triggers an operation.
09 - Appareils et instruments scientifiques et électriques
42 - Services scientifiques, technologiques et industriels, recherche et conception
Produits et services
Downloadable augmented reality and virtual reality software for integrating electronic data with real and virtual world environment; Downloadable software and application programming interface (API) software for use in creating and designing augmented reality and virtual reality experiences; Computer hardware, peripherals, and downloadable software for object recognition, sound recognition, and scanning the environment for augmented reality and virtual reality experiences; Downloadable computer software for tracking motion in, visualizing, manipulating, viewing, transmitting, and displaying images, video, audio, and data for augmented reality and virtual reality experiences; Downloadable computer software and firmware for controlling, configuring, and managing hardware devices and for enabling electronic devices to share data and communicate with each other; Downloadable computer software for cameras, video cameras, video recorders, and visual displays; Downloadable computer software for setting up, configuring, and controlling wearable computer hardware and peripherals; Downloadable computer software and software applications for use in uploading, downloading, capturing, editing, storing, accessing, posting, displaying, tagging, distributing, streaming, linking, sharing, transmitting or otherwise providing photos, videos, images, text, electronic media, photographic and video content, digital data, and information via the Internet, communication networks, and mobile devices; Downloadable multimedia files containing digital photos, video, and audio all in the fields of entertainment, photography, communications, utilities, and online social networking Providing temporary use of non-downloadable software for use in creating and designing augmented reality and virtual reality experiences; Providing temporary use of non-downloadable augmented reality and virtual reality software for use in hardware devices for integrating electronic data with real and virtual world environment; Providing temporary use of non-downloadable software for object recognition, sound recognition, and scanning the environment for augmented reality and virtual reality experiences; Providing temporary use of non-downloadable software for tracking motion in, visualizing, manipulating, viewing, transmitting, and displaying images, video, audio, and data for augmented reality and virtual reality experiences; Providing temporary use of non-downloadable software for managing hardware devices, configuring, and managing controllers and for enabling electronic devices to share data and communicate with each other; Providing temporary use of non-downloadable software for cameras, video cameras, video recorders, and visual displays for enabling augmented reality technology; Providing temporary use of non-downloadable software for setting up, configuring, and controlling wearable computer hardware and peripherals; Providing temporary use of non-downloadable software for use in uploading, downloading, capturing, editing, storing, accessing, posting, displaying, tagging, distributing, streaming, linking, sharing, transmitting or otherwise providing photos, videos, images, text, electronic media, photographic and video content, digital data, or information via the Internet, communication networks, and mobile devices; Providing temporary use of non-downloadable software for accessing multimedia files containing digital photos, video, audio, and other digital data all in the fields of entertainment, photography, communications, utilities, and online social networking
51.
MIRRORING AND NAVIGATING CONTENT IN AUGMENTED REALITY MESSAGING SYSTEMS
The subject technology receives, by one or more hardware processors implementing a local wireless network, a request from a client device to mirror media content displayed on a screen of the client device on a wearable device. In response to the request, the subject technology causes a display of the media content in a mirroring lens of the wearable device. While the media content is being displayed in the mirroring lens of the wearable device, the subject technology tracks hand gestures of a user wearing the wearable device and viewing the media content displayed in the mirroring lens of the wearable device. The subject technology processes navigational or manipulation data based on the tracked hand gestures and sends a navigation or manipulation instruction to the client device or a mirroring lens processor of the wearable device based on the tracked hand gestures.
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06F 3/04815 - Interaction s’effectuant dans un environnement basé sur des métaphores ou des objets avec un affichage tridimensionnel, p. ex. modification du point de vue de l’utilisateur par rapport à l’environnement ou l’objet
G06F 3/04845 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] pour la commande de fonctions ou d’opérations spécifiques, p. ex. sélection ou transformation d’un objet, d’une image ou d’un élément de texte affiché, détermination d’une valeur de paramètre ou sélection d’une plage de valeurs pour la transformation d’images, p. ex. glissement, rotation, agrandissement ou changement de couleur
G06F 3/14 - Sortie numérique vers un dispositif de visualisation
52.
RECOMMENDATION SYSTEM WITH RAPID LEARNING FROM USER FEEDBACK
A system and method that includes retrieving candidate effects to present to a user; computing initial scores for the candidate effects; computing adjusted scores for the candidate effects, each adjusted score for an effect being based on an initial score for the effect and a demotion factor computed based on interaction data associated with the user and the candidate effects; generating a set of recommended effects based on the candidate effects and associated adjusted scores; and causing the set of recommended effects to be presented to the user on a computing device. The demotion factor for the effect is further based on a number of consecutive repeated actions being performed by the user, during a first predetermined interval, with respect to the effect. Effects include lenses, filters, image stylization effects, or video stylization effects, while actions include swipe actions.
The subject technology receives a selection of a first representation of a product from a set of representations of products. The subject technology renders for display the first representation of the product based on metadata using an extension application programming interface (API). The subject technology determines augmented reality (AR) content that is stored remotely using the extension API. The subject technology sends a first request for remote AR content to a camera API using the extension API. The subject technology sends a second request for retrieving the remote AR content using the camera API. The subject technology renders the remote AR content using the camera API. The subject technology receives a notification that the remote AR content was rendered for display from the extension API.
Disclosed are implementations for ranking a feed in a social network. In one aspect, a method of generating a feed includes identifying a group of users associated with a first user, identify notifications for the first user based on the group of users, determine one or more actions that may result from each of the identified notifications, rank the notifications based on the one or more determined actions; and display the notifications to the first user according to the ranking.
Methods and systems are disclosed for building a few-shot logo recognition system that includes accessing an image with several regions of interest and identifying several objects within the regions of interest using a logo detector neural network. For each object, the logo detector neural network indicates whether the object is a logo. The methods and systems also generate a first and second set of image feature data and a first and second ranked list of logos. A final ranked list of logos is generated based on the first and second ranked list of logos and a category associated with each logo in the final ranked list of logos is identified.
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
G06V 10/22 - Prétraitement de l’image par la sélection d’une région spécifique contenant ou référençant une formeLocalisation ou traitement de régions spécifiques visant à guider la détection ou la reconnaissance
G06V 10/764 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant la classification, p. ex. des objets vidéo
Methods and systems are disclosed for enhancing or modifying an image by a diffusion model. The methods and systems receive a first image depicting a real-world scene including a target object and receive input associated with adjusting a zoom level of the first image. The methods and systems, in response to receiving the input, modify the zoom level associated with the first image to generate a second image having a view of the target object that is different from a view of the target object in the first image. The methods and systems analyze the second image using a generative machine learning model to generate an artificial image that modifies portions of the second image to improve the view of the target object relative to the second image.
A terraced battery system is provided that may enhance the battery packaging efficiency within an organic shape of a wearable electronic device such as an electronic eyewear device. The terraced battery includes several stacked cells of different geometries. The terraced battery geometries are selected to better accommodate organic (non-trapezoidal and non-cylindrical) shapes of the battery housing in the wearable electronic device. In an example, the terraced battery geometry is adapted to accommodate the organic shape of a battery housing in the temples of an augmented reality electronic eyewear device. As the number of the battery cells or terraces increases, the battery packaging efficiency can be further improved within an organic shape of the battery housing. The increased packaging efficiency for the battery enables increased battery life within organically shaped enclosures.
A hand-tracking platform generates gesture components for use as user inputs into an application of an Augmented Reality (AR) system. In some examples, the hand-tracking platform generates real-world scene environment frame data based on gestures being made by a user of the AR system using a camera component of the AR system. The hand-tracking platform recognizes a gesture component based on the real-world scene environment frame data and generates gesture component data based on the gesture component. The application utilizes the gesture component data as user input in a user interface of the application.
An eXtended Reality (XR) system is provided that generates an interactive XR user interface including 3D sliders for inputting a value for a variable. The XR system renders a 3D slider using a set of attributes and displays the 3D slider to a user as a component of the XR user interface. The XR system detects a pinch gesture of a hand of the user in proximity to the 3D slider. In response to detecting the pinch gesture, the XR system renders the 3D slider using a set of attributes and re-displays the 3D slider to the user.
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06F 3/04815 - Interaction s’effectuant dans un environnement basé sur des métaphores ou des objets avec un affichage tridimensionnel, p. ex. modification du point de vue de l’utilisateur par rapport à l’environnement ou l’objet
G06F 3/04847 - Techniques d’interaction pour la commande des valeurs des paramètres, p. ex. interaction avec des règles ou des cadrans
60.
RECORDING FOLLOWING BEHAVIORS BETWEEN VIRTUAL OBJECTS AND USER AVATARS IN AR EXPERIENCES
Described are recording tools for generating following behaviors and creating interactive AR experiences. The following recording application enables a user with little or no programming skills to virtually connect virtual objects to other elements, including virtual avatars representing fellow users, thereby creating an interactive story in which multiple elements are apparently and persistently connected. The following interface includes methods for selecting objects and instructions for connecting a virtual object to a target object. In one example, the recording application presents on the display a virtual tether between the objects until a connecting action is detected. The following interface is presented on the display as an overlay, in the foreground relative to the physical environment.
G06F 3/04815 - Interaction s’effectuant dans un environnement basé sur des métaphores ou des objets avec un affichage tridimensionnel, p. ex. modification du point de vue de l’utilisateur par rapport à l’environnement ou l’objet
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06F 3/04842 - Sélection des objets affichés ou des éléments de texte affichés
G06F 3/0488 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] utilisant des caractéristiques spécifiques fournies par le périphérique d’entrée, p. ex. des fonctions commandées par la rotation d’une souris à deux capteurs, ou par la nature du périphérique d’entrée, p. ex. des gestes en fonction de la pression exercée enregistrée par une tablette numérique utilisant un écran tactile ou une tablette numérique, p. ex. entrée de commandes par des tracés gestuels
G06T 7/70 - Détermination de la position ou de l'orientation des objets ou des caméras
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
61.
WAVEGUIDE AND DIFFRACTION GRATING FOR AUGMENTED REALITY OR VIRTUAL REALITY DISPLAY
A waveguide for use in a virtual reality, VR, or augmented reality, AR, device, is disclosed. The waveguide comprising an input region configured to couple light into the waveguide so that it propagates under total internal reflection (TIR) within the waveguide, and an output region comprising optical structures configured to receive image bearing light from the input region. The output region comprises a plurality of zones having different diffraction to each other, the plurality of zones comprising diffraction efficiencies so as to reduce rainbow artefacts.
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing augmented reality content for clothing. The program and method provide for receiving a request to present augmented reality content in association with an image depicting a body part of a user; accessing an augmented reality content item corresponding to an article of clothing, the augmented reality content being configured to track plural points of the body part and to generate a transform based on the plural points, the transform corresponding to the article of clothing and defining plural regions for the article of clothing; presenting the augmented reality content item in association with the body part; receiving user input selecting a region; determining a set of options corresponding to the region; and updating presentation of the augmented reality content item based on the set of options.
An augmented reality system to perform operations that include: accessing image data at a client device; determining a position of a user of the client device based on the image data; causing display of a projection that extends from the position of the user upon a presentation of the image data at the client device; detecting an intersection of the projection and a surface of an object; generating a request that includes an identification of the portion of the surface of the object at the client device; and presenting the portion of the surface of the object based on the graphical property of the projection at the client device in response to the request that includes the identification of the portion of the surface of the object.
G06V 20/20 - ScènesÉléments spécifiques à la scène dans les scènes de réalité augmentée
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06T 7/73 - Détermination de la position ou de l'orientation des objets ou des caméras utilisant des procédés basés sur les caractéristiques
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
G09G 3/00 - Dispositions ou circuits de commande présentant un intérêt uniquement pour l'affichage utilisant des moyens de visualisation autres que les tubes à rayons cathodiques
64.
WAVEGUIDE FOR AN AUGMENTED REALITY OR VIRTUAL REALITY DISPLAY
A planar slab waveguide for use in an augmented reality or virtual reality display is disclosed. The planar slab waveguide comprises a diffractive element comprising optical structures having a different refractive index to a surrounding waveguide medium. The optical structures are arranged in an array parallel to a first major surface and a second major surface of the waveguide. The array is configured to diffract a first portion of the light to provide 2D expansion across the plane of the waveguide and to diffract a second portion of the light out of the plane of the waveguide. At least one of the optical structures has a height that varies along a direction parallel to the plane of the waveguide, thereby causing the second portion of light to be diffracted preferentially from the first major surface of the waveguide, compared to the second major surface of the waveguide.
An extended Reality (XR) display system includes a Light Emitting Diode (LED) display controller, and a Light Emitting Diode (LED) near-eye display element operatively coupled to the LED display driver. The LED near-eye display element includes one or more motors and an LED array operably connected to the one or more motors. During operation, the LED display driver receives video data including a rendered virtual object of an XR experience and generates LED array control signals based on the video data, the LED array control signals causing one or more LEDs of the LED array to be energized in a sequence. The LED display driver also generates synchronized motor control signals and simultaneously communicates the LED array control signals to the LED array and the synchronized motor control signals to the one or more motors causing the LED near-eye display element to display the rendered virtual object.
G09G 3/32 - Dispositions ou circuits de commande présentant un intérêt uniquement pour l'affichage utilisant des moyens de visualisation autres que les tubes à rayons cathodiques pour la présentation d'un ensemble de plusieurs caractères, p. ex. d'une page, en composant l'ensemble par combinaison d'éléments individuels disposés en matrice utilisant des sources lumineuses commandées utilisant des panneaux électroluminescents semi-conducteurs, p. ex. utilisant des diodes électroluminescentes [LED]
Methods and systems are disclosed for using machine learning models to perform pixel-based deformation of fashion items. The methods and systems receive one or more images depicting a person in an individual pose and receive a first source image depicting a first view of a target fashion item and a second source image depicting a second view of the target fashion item. The methods and systems process, using one or more machine learning models, the one or more images that depict the person in the individual pose together with the first and second source images to generate a flow field, the flow field indicating a likelihood of existence and location of each pixel of the one or more images relative to the first and second source images. The methods and systems modify a portion of the one or more images to overlay the target fashion item on the person.
An augmented reality (AR) translation system is provided. The AR translation system may analyze camera data to determine objects included in a field of view of a camera of a user device.
An augmented reality (AR) translation system is provided. The AR translation system may analyze camera data to determine objects included in a field of view of a camera of a user device.
Augmented reality content may be provided that includes a visual translation of an object included in the field of view from a primary language of the user to an additional language. An audible version of the translation may also be provided as part of the augmented reality content. Users may also add an object in the field of view to a listing of translated objects associated with the user based on at least one of touch input, audio input, or gesture input.
G06F 40/58 - Utilisation de traduction automatisée, p. ex. pour recherches multilingues, pour fournir aux dispositifs clients une traduction effectuée par le serveur ou pour la traduction en temps réel
G06F 3/0488 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] utilisant des caractéristiques spécifiques fournies par le périphérique d’entrée, p. ex. des fonctions commandées par la rotation d’une souris à deux capteurs, ou par la nature du périphérique d’entrée, p. ex. des gestes en fonction de la pression exercée enregistrée par une tablette numérique utilisant un écran tactile ou une tablette numérique, p. ex. entrée de commandes par des tracés gestuels
G06T 7/73 - Détermination de la position ou de l'orientation des objets ou des caméras utilisant des procédés basés sur les caractéristiques
A map-based graphical user interface for a social media application displays an interactive map and enables access to ephemeral social media content submitted by multiple users. Ephemeral social media items with geotag data are grouped into location-based collections based on posting locations within common geographic areas, with collection icons displayed at corresponding map locations. Users can perform location-based searches for social media content, with search queries associated with a search area based on the geographical area currently displayed in the interactive map viewport. Search results identify social media items having geotag data indicating posting locations within the search area. Each ephemeral social media item is available for viewing only during a limited time period defined by an availability parameter, after which the item's availability is automatically terminated.
G06F 3/04817 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] fondées sur des propriétés spécifiques de l’objet d’interaction affiché ou sur un environnement basé sur les métaphores, p. ex. interaction avec des éléments du bureau telles les fenêtres ou les icônes, ou avec l’aide d’un curseur changeant de comportement ou d’aspect utilisant des icônes
G06F 3/0482 - Interaction avec des listes d’éléments sélectionnables, p. ex. des menus
G06F 3/04842 - Sélection des objets affichés ou des éléments de texte affichés
G06F 3/0488 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] utilisant des caractéristiques spécifiques fournies par le périphérique d’entrée, p. ex. des fonctions commandées par la rotation d’une souris à deux capteurs, ou par la nature du périphérique d’entrée, p. ex. des gestes en fonction de la pression exercée enregistrée par une tablette numérique utilisant un écran tactile ou une tablette numérique, p. ex. entrée de commandes par des tracés gestuels
G06F 16/248 - Présentation des résultats de requêtes
G06F 16/29 - Bases de données d’informations géographiques
G06F 16/487 - Recherche caractérisée par l’utilisation de métadonnées, p. ex. de métadonnées ne provenant pas du contenu ou de métadonnées générées manuellement utilisant des informations géographiques ou spatiales, p. ex. la localisation
G06F 16/9535 - Adaptation de la recherche basée sur les profils des utilisateurs et la personnalisation
G06F 16/9537 - Recherche à dépendance spatiale ou temporelle, p. ex. requêtes spatio-temporelles
H04L 41/22 - Dispositions pour la maintenance, l’administration ou la gestion des réseaux de commutation de données, p. ex. des réseaux de commutation de paquets comprenant des interfaces utilisateur graphiques spécialement adaptées [GUI]
H04L 41/28 - Restriction de l’accès aux systèmes ou aux fonctions de gestion de réseau, p. ex. en utilisant la fonction d’autorisation pour accéder à la configuration du réseau
H04L 51/52 - Messagerie d'utilisateur à utilisateur dans des réseaux à commutation de paquets, transmise selon des protocoles de stockage et de retransmission ou en temps réel, p. ex. courriel pour la prise en charge des services des réseaux sociaux
H04L 67/12 - Protocoles spécialement adaptés aux environnements propriétaires ou de mise en réseau pour un usage spécial, p. ex. les réseaux médicaux, les réseaux de capteurs, les réseaux dans les véhicules ou les réseaux de mesure à distance
H04L 67/52 - Services réseau spécialement adaptés à l'emplacement du terminal utilisateur
H04W 4/02 - Services utilisant des informations de localisation
H04W 4/029 - Services de gestion ou de suivi basés sur la localisation
H04W 4/18 - Conversion de format ou de contenu d'informations, p. ex. adaptation, par le réseau, des informations reçues ou transmises pour une distribution sans fil aux utilisateurs ou aux terminaux
H04W 4/21 - Signalisation de servicesSignalisation de données auxiliaires, c.-à-d. transmission de données par un canal non destiné au trafic pour applications de réseaux sociaux
H04W 12/02 - Protection de la confidentialité ou de l'anonymat, p. ex. protection des informations personnellement identifiables [PII]
Systems, devices, media, and methods are presented for assigning configuration parameters to unknown devices by comparing characteristics of unknown devices to characteristics of known devices. Characteristics of an unknown device are compared to characteristics of known devices to identify a similar device having known configuration parameters. The unknown device is then assigned the configuration parameters of the identified similar device.
H04L 67/125 - Protocoles spécialement adaptés aux environnements propriétaires ou de mise en réseau pour un usage spécial, p. ex. les réseaux médicaux, les réseaux de capteurs, les réseaux dans les véhicules ou les réseaux de mesure à distance en impliquant la commande des applications des terminaux par un réseau
Devices and methods for dynamic power configuration (e.g., reduction) for thermal management (e.g., mitigation) in a wearable electronic device such as an eyewear device. The wearable electronic device monitors its temperature and, responsive to the temperature, configures the services it provides to operate in different modes for thermal mitigation (e.g., to prevent overheating). For example, based on temperature, the wearable electronic device adjusts sensors (e.g., turns cameras on or off, changes the sampling rate, or a combination thereof) and adjusts display components (e.g., adjusted rate at which a graphical processing unit generates images and a visual display is updated). This enables the wearable electronic device to consume less power when temperatures are too high in order to provide thermal mitigation.
Systems, devices, media, and methods are described for presenting a tutorial in augmented reality on the display of a smart eyewear device. The system includes a marker registration utility for setting a marker on a musical instrument, a localization utility, a virtual object rendering utility for presenting virtual tutorial objects on the display near the instrument, and a hand tracking utility for tracking the performer's finger locations in real time. The virtual tutorial objects, in one example, includes graphical elements presented on a virtual scroll that appears to move toward the instrument at a speed correlated with the song tempo. The hand tracking utility calculates a set of expected fingertip coordinates based on a detected hand shape.
A social media platform provides a map-based graphical user interface (GUI) for accessing social media content submitted for public accessibility via the social media platform supported by the map-based GUI. The GUI includes a map providing interactive location-based searching functionality in that selection of a target location by the user in the GUI, such as by tapping or clicking at the target location, triggers a search for social media content having geo-tag data indicating geographic locations within a geographical search area centered on the target location. A search period for which content is returned is dynamically variable based on the duration for which the tap or click is held.
G06F 16/9038 - Présentation des résultats des requêtes
G06F 3/04817 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] fondées sur des propriétés spécifiques de l’objet d’interaction affiché ou sur un environnement basé sur les métaphores, p. ex. interaction avec des éléments du bureau telles les fenêtres ou les icônes, ou avec l’aide d’un curseur changeant de comportement ou d’aspect utilisant des icônes
G06F 3/04842 - Sélection des objets affichés ou des éléments de texte affichés
G06F 3/0487 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] utilisant des caractéristiques spécifiques fournies par le périphérique d’entrée, p. ex. des fonctions commandées par la rotation d’une souris à deux capteurs, ou par la nature du périphérique d’entrée, p. ex. des gestes en fonction de la pression exercée enregistrée par une tablette numérique
G06F 16/29 - Bases de données d’informations géographiques
A method for a low-power hand-tracking system is described. In one aspect, a method includes polling a proximity sensor of a wearable device to detect a proximity event, the wearable device includes a low-power processor and a high-power processor, in response to detecting the proximity event, operating a low-power hand-tracking application on the low-power processor based on proximity data from the proximity sensor, and ending an operation of the low-power hand-tracking application in response to at least one of: detecting and recognizing a gesture based on the proximity data, detecting without recognizing the gesture based on the proximity data, or detecting a lack of activity from the proximity sensor within a timeout period based on the proximity data.
Examples relate to systems and methods for generating icons based on events. The systems and methods receive a content item from a first user system comprising a digital effect associated with an event and determine whether an event condition is satisfied. The systems and methods, based on determining whether the event condition is satisfied, automatically modify an icon that notifies a second user system about the content item to include an event indicator. The systems and methods, in response to receiving input from the second user system that selects the icon, present the content item comprising the digital effect associated with the event on the second user system.
G06F 3/04817 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] fondées sur des propriétés spécifiques de l’objet d’interaction affiché ou sur un environnement basé sur les métaphores, p. ex. interaction avec des éléments du bureau telles les fenêtres ou les icônes, ou avec l’aide d’un curseur changeant de comportement ou d’aspect utilisant des icônes
G06F 3/0482 - Interaction avec des listes d’éléments sélectionnables, p. ex. des menus
G06F 3/04842 - Sélection des objets affichés ou des éléments de texte affichés
G06F 3/04845 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] pour la commande de fonctions ou d’opérations spécifiques, p. ex. sélection ou transformation d’un objet, d’une image ou d’un élément de texte affiché, détermination d’une valeur de paramètre ou sélection d’une plage de valeurs pour la transformation d’images, p. ex. glissement, rotation, agrandissement ou changement de couleur
G06F 3/0488 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] utilisant des caractéristiques spécifiques fournies par le périphérique d’entrée, p. ex. des fonctions commandées par la rotation d’une souris à deux capteurs, ou par la nature du périphérique d’entrée, p. ex. des gestes en fonction de la pression exercée enregistrée par une tablette numérique utilisant un écran tactile ou une tablette numérique, p. ex. entrée de commandes par des tracés gestuels
75.
SUGGESTING RELEVANT GROUPS AND INDIVIDUALS IN MESSAGE REPLIES
A system and method for suggesting relevant groups and recipients when replying to messages in a messaging application. In response to a first received message, the system identifies groups with membership comprising the sender and receiver. Interface elements representing these mutual groups are displayed as selectable suggestions. The receiving user can choose groups to include in the reply, along with other users. Suggested groups are determined based on recent interactions, mutual connections, and message content. Users can also create new groups from suggestions for ongoing messaging. By recommending shared groups and relevant recipients, the system enables efficient context-based selection when replying. The suggestions aim to streamline recipient picking through intuitive interfaces and machine learning algorithms. This improves the user experience for seamless messaging discussions with appropriate recipients.
H04L 51/216 - Gestion de l'historique des conversations, p. ex. regroupement de messages dans des sessions ou des fils de conversation
G06F 3/0482 - Interaction avec des listes d’éléments sélectionnables, p. ex. des menus
H04L 51/043 - Messagerie en temps réel ou quasi en temps réel, p. ex. messagerie instantanée [IM] en utilisant ou en gérant les informations de présence
H04L 51/52 - Messagerie d'utilisateur à utilisateur dans des réseaux à commutation de paquets, transmise selon des protocoles de stockage et de retransmission ou en temps réel, p. ex. courriel pour la prise en charge des services des réseaux sociaux
76.
SYSTEM FOR USING DIGITAL LIGHT PROJECTORS FOR AUGMENTED REALITY
A method for configuring a digital light projector (DLP) of an augmented reality (AR) display device is described. A light source component of the DLP projector is configured to generate a single red-green-blue color sequence repetition per image frame. The AR display device identifies a color sequence of the light source component of the DLP projector and tracks a motion of the AR display device. The AR display device adjusts an operation of the DLP projector based on the single red-green-blue color sequence repetition, the color sequence of the light source component of the DLP projector, and the motion of the AR display device.
Examples relate to systems and methods for generating icons based on events. The systems and methods receive a content item from a first user system comprising a digital effect associated with an event and determine whether an event condition is satisfied. The systems and methods, based on determining whether the event condition is satisfied, automatically modify an icon that notifies a second user system about the content item to include an event indicator. The systems and methods, in response to receiving input from the second user system that selects the icon, present the content item comprising the digital effect associated with the event on the second user system.
An augmented reality (AR) content system is provided. The AR content system may analyze audio input obtained from a user to generate a search request. The AR content system may obtain search results in response to the search request and determine a layout by which to display the search results. The search results may be displayed in a user interface within an AR environment according to the layout. The AR content system may also analyze audio input to detect commands to perform with respect to content displayed in the user interface.
Various waveguides and image display systems are disclosed herein. In an example, an image display system can include an optical engine configured to generate an image and a waveguide. The waveguide can have a light in-coupling region formed along a peripheral edge of the waveguide, the light in-coupling region including a first surface with a first set of diffraction gratings, and a light exit region formed along a top surface of the waveguide, the light exit region including a second set of diffraction gratings. The first set of diffraction gratings can be configured to diffract light towards the second set of diffraction gratings, and the second set of diffraction gratings can be configured to diffract light towards the user's eye.
A system for creating and presenting enhanced voice notes in augmented reality (AR) environments is disclosed. The system enables users to generate personalized voice notes with visual and audio enhancements on mobile devices, and deliver them to recipients wearing AR devices. Voice notes can be customized with Al-generated voice styles, animated visual representations, and spatial audio effects. Recipients experience immersive playback through AR glasses, with voice notes appearing at specified locations or anchored to body parts. The system leverages computer vision, spatial audio processing, and real-time tracking to create context- aware and spatially relevant communications. This approach transforms traditional voice messaging into an engaging, three-dimensional experience that seamlessly integrates with the user's physical environment.
A system and associated methods display on a user device a non-fungible token (NFT) in an application that displays video data. The methods include storing the NFT of a user in a database accessible to the application, associating an ID of the user for the application to the NFT, and receiving a selection of the NFT during use of the application. Upon selection, the selected NFT is provided to the user device for superimposing the selected NFT over the user’s face, providing the selected NFT as a background image for the video data on the user device, or both. The user device includes a web application that connects the database to a crypto wallet containing the NFT. The NFT may be processed to crop out the eyes and mouth of the NFT for registration with the user’s eyes and mouth when the NFT is superimposed over the user’s face.
G06Q 20/36 - Architectures, schémas ou protocoles de paiement caractérisés par l'emploi de dispositifs spécifiques utilisant des portefeuilles électroniques ou coffres-forts électroniques
G06Q 20/38 - Protocoles de paiementArchitectures, schémas ou protocoles de paiement leurs détails
G06Q 20/40 - Autorisation, p. ex. identification du payeur ou du bénéficiaire, vérification des références du client ou du magasinExamen et approbation des payeurs, p. ex. contrôle des lignes de crédit ou des listes négatives
Systems and methods for reconstructing 3D models of human bodies from 2D images that counts for perspective and/or distortion effects are provided. The systems and methods include reconstructing a three-dimensional model of an object in a three-dimensional scene from a two-dimensional image comprising an image of the object. The systems and methods include determining an absolute depth of a key point of the object in the image; determining, using the absolute depth of the key point, a three-dimensional position of the key point in the three-dimensional scene; generating, using a neural network, a three-dimensional representation of the object, the three-dimensional representation comprising mesh nodes defined in a coordinate system relative to the key point; and positioning the three-dimensional representation of the object in the scene based on the position of the key point by applying a position dependent rotation to the three-dimensional object.
Examples relate to systems and methods for generating digital effects experiences. The system performs operations including accessing a set of instructions that defines a digital effects experience. The system processes the set of instructions by a generative machine learning model to generate one or more digital effects comprising the digital effects experience. The system continuously processes one or more inputs, received by a user device, while the one or more digital effects comprising the digital effects experience are presented on the user device, along with the set of instructions in real time by the generative machine learning model to update presentation of the one or more digital effects comprising the digital effects experience.
An AR or VR display device. First and third input gratings receive light of a first color from first and second projectors, respectively, coupling the light into a first waveguide. Second and fourth input gratings receive light of a second color from the first and second projectors, respectively, coupling the light into a second waveguide. An output diffractive optical element couples light out of the waveguides towards a viewing position. The first and second projectors provide light to the input diffractive optical elements in directions that are at a first and second angle, respectively, to a waveguide normal vector. The output diffractive optical element couples light out of the waveguides in a first range of angles for light from the first projector and in a second range of angles for light from the second projector, the first range of angles and the second range of angles differing but partially overlapping.
Examples relate to systems and methods for generating digital effects experiences. The system performs operations including accessing a set of instructions that defines a digital effects experience. The system processes the set of instructions by a generative machine learning model to generate one or more digital effects comprising the digital effects experience. The system continuously processes one or more inputs, received by a user device, while the one or more digital effects comprising the digital effects experience are presented on the user device, along with the set of instructions in real time by the generative machine learning model to update presentation of the one or more digital effects comprising the digital effects experience.
Marketing, advertising, and promotional services using augmented reality computer software for integrating electronic data with real world environments for the purpose of experiencing, viewing, capturing, recording and editing augmented images, videos, audio and sensory content; Providing online advertising services for others; Dissemination of advertising for others via computer and other communication networks; Targeted marketing services; Advertising and marketing services, namely, promoting the goods and services of others via a mobile application
88.
Charging and data access port for a head-wearable apparatus
In some examples, a head-wearable apparatus for viewing augmented reality (AR) or virtual reality (VR) content is provided. An example the apparatus comprises a frame, an optical assembly including an image display in which the AR or VR content may be viewed by a user, and a user input device operable by the user to navigate through content viewed in the image display, or to invoke a function of the head-wearable apparatus. The user input device includes a body manually engageable by the user to perform a content navigation or function invocation operation and is configured to present at least one contact for accepting a connection to an external charging source, or a connection to an external device.
Examples disclosed herein describe visual-inertial tracking techniques for extended reality (XR) devices. According to some example methods, an XR device is located in, and movable relative to, a vehicle. The XR device generates device tracking data and accesses vehicle tracking data. The vehicle tracking data is generated by an external sensor configured to measure motion of the vehicle. Consolidated tracking data is generated based on the device tracking data and the vehicle tracking data. In some examples, a pose of the XR device is determined by using the consolidated tracking data.
Systems, devices, media, instructions, and methods for computer based automated content generation, curation, and presentation are described. In one embodiment a content collection is generated with a first continuous presentation group by associating a first content element from a first content message of the plurality of content messages and a second content element from a second content message of the plurality of content messages to associate the first content element and the second content element as the first continuous presentation group. Advertising element placement within the presentation order for the first media collection is determined, and adjusted to avoid interrupting the continuous presentation group. In other embodiments, various advertising patterns are used and adjusted based on curated presentation groups within content collections.
Examples relate to processing extended reality (XR) content. A system obtains an unmodified image and a modified image with an XR effect applied. A trained multimodal generative language model generates visual difference text describing differences between the images. Additional text data associated with the XR effect is obtained. The additional text data can include visual text displayed by the XR effect and/or metadata associated with the XR effect. A trained generative language model processes the visual difference text and additional text data to generate output text data descriptive of the XR effect. The output text data may include content tags, location information, and a merged caption. Constrained decoding ensures the output adheres to a predefined structure. The system enables automated understanding and categorization of XR effects for applications like content discovery, recommendations, and moderation.
G06V 20/70 - Étiquetage du contenu de scène, p. ex. en tirant des représentations syntaxiques ou sémantiques
G06V 20/20 - ScènesÉléments spécifiques à la scène dans les scènes de réalité augmentée
G06V 20/30 - ScènesÉléments spécifiques à la scène dans les albums, les collections ou les contenus partagés, p. ex. des photos ou des vidéos issus des réseaux sociaux
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
In a thin film stack deposited on a substrate, a first layer can have a non-uniform thickness. A second layer, disposed so that the first layer is between the substrate and the second layer, can have a non-uniform thickness. A first pattern can be formed on the second layer to define first areas. The second layer can be etched in the first areas to form first holes having varying depths. A second pattern can be formed on the first layer in the first holes to define second areas. Each second area can be smaller than a corresponding first area. The first layer can be etched in the second areas to form second holes having varying depths. An imprint of the etched thin film stack can be formed such that the first holes and the second holes form a plurality of diffractive elements having varying sizes on the imprint.
A content request is received from a user device of a viewing user of an interaction system. An interaction application executing at the user device presents a viewing user interface that includes a content zone and an interaction zone. A third-party content item is presented in the content zone. The interaction zone comprises a first interaction subzone located in a lower section of the viewing user interface, outside of the content zone. The first interaction subzone includes a user-selectable interaction element. The interaction zone further comprises a second interaction subzone positioned adjacent to the first interaction subzone in the lower section of the viewing user interface. The second interaction subzone is located inside of the content zone and includes a user-selectable context element. Responsive to detecting an interaction gesture directed at the interaction zone, the interaction application automatically transitions from a viewing state to an interaction state.
G06F 3/04886 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] utilisant des caractéristiques spécifiques fournies par le périphérique d’entrée, p. ex. des fonctions commandées par la rotation d’une souris à deux capteurs, ou par la nature du périphérique d’entrée, p. ex. des gestes en fonction de la pression exercée enregistrée par une tablette numérique utilisant un écran tactile ou une tablette numérique, p. ex. entrée de commandes par des tracés gestuels par partition en zones à commande indépendante de la surface d’affichage de l’écran tactile ou de la tablette numérique, p. ex. claviers virtuels ou menus
G06F 3/0482 - Interaction avec des listes d’éléments sélectionnables, p. ex. des menus
G06F 3/04883 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] utilisant des caractéristiques spécifiques fournies par le périphérique d’entrée, p. ex. des fonctions commandées par la rotation d’une souris à deux capteurs, ou par la nature du périphérique d’entrée, p. ex. des gestes en fonction de la pression exercée enregistrée par une tablette numérique utilisant un écran tactile ou une tablette numérique, p. ex. entrée de commandes par des tracés gestuels pour l’entrée de données par calligraphie, p. ex. sous forme de gestes ou de texte
Methods and systems are disclosed for performing optical character recognition on augmented images. The system accesses an image depicting a real-world environment augmented with a graphical element. The system recognizes the graphical element in the image by applying a machine learning (ML) model to the image. The system stores a standard code representing the graphical element that has been recognized in association with the image.
G06T 5/50 - Amélioration ou restauration d'image utilisant plusieurs images, p. ex. moyenne ou soustraction
G06V 10/25 - Détermination d’une région d’intérêt [ROI] ou d’un volume d’intérêt [VOI]
G06V 10/774 - Génération d'ensembles de motifs de formationTraitement des caractéristiques d’images ou de vidéos dans les espaces de caractéristiquesDispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant l’intégration et la réduction de données, p. ex. analyse en composantes principales [PCA] ou analyse en composantes indépendantes [ ICA] ou cartes auto-organisatrices [SOM]Séparation aveugle de source méthodes de Bootstrap, p. ex. "bagging” ou “boosting”
A transformer-based network and method for generic face image quality assessment (GFIQA), predicting perceptual scores for face images. The DSL is a self-supervised approach for learning degradation features globally. This network and method effectively captures global degradation representations from both synthetically and naturally degraded images, enhancing the learning process of degradation characteristics. The network's attention is enhanced to salient facial components by integrating facial landmark detection, enabling a holistic quality evaluation that adaptively aggregates local quality assessment across the face.
G06V 10/98 - Détection ou correction d’erreurs, p. ex. en effectuant une deuxième exploration du motif ou par intervention humaineÉvaluation de la qualité des motifs acquis
G06V 10/26 - Segmentation de formes dans le champ d’imageDécoupage ou fusion d’éléments d’image visant à établir la région de motif, p. ex. techniques de regroupementDétection d’occlusion
G06V 10/778 - Apprentissage de profils actif, p. ex. apprentissage en ligne des caractéristiques d’images ou de vidéos
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
G06V 40/16 - Visages humains, p. ex. parties du visage, croquis ou expressions
96.
Display screen or portion thereof with a graphical user interface
A head-worn device system equipped with cameras, display devices, and processors uses stored instructions to facilitate interactions between an augmented reality (AR) device and an external display system. When executed, these instructions establish a communications link between the AR device and the external display. The system identifies the pose of the external display and receives user inputs from the AR device user, relating to interactions with virtual objects in a real-world setting. It also identifies the positions of viewers watching the external display. Based on the external display's pose and the viewer locations, the system generates display data for the virtual object, ensuring it appears correctly on the external display. This display data is then transmitted to the external display system, completing the interaction loop and enhancing the viewing experience for the audience.
A movable assembly with an elastomer bellows forming a cosmetic covering and providing ingress protection of a dynamic opening between a pair of enclosures. On one side of the dynamic opening, the bellows is secured to an enclosure by protrusions of that enclosure that extend into mating openings in the bellows and a retainer that engages those protrusions. An adhesive secures the protrusions to the retainer to form a secure clamped connection of the bellows to the enclosure. On the other side of the dynamic opening, the bellows is secured to another enclosure without the use of fasteners or adhesives. Teeth in openings of the second enclosure are secured in a recessed channel of the bellows created by a protrusion and a flange. As the two enclosures move relative to one another, the bellows is stretched and compressed such that the dynamic opening between the two enclosures is cosmetically covered with ingress protection.
Methods and systems are disclosed for generating an extended reality (XR) try-on experience based on an image produced by a diffusion model. The system receives a first image depicting a real-world object and receives a second image depicting a target fashion item. The system generates a warped image in which pixels of the target fashion item depicted in the second image replace pixels of a portion of the real-world object in the first image and generates one or more segmentation maps corresponding to incomplete portions of the warped image. The system analyzes the warped image and the one or more segmentation maps using a generative machine learning model to generate an artificial image that populates the incomplete portions of the warped image to depict the real-world object wearing the target fashion item.
A system and method for facilitating natural language conversations between customers and vendors for product purchases is provided. The system ingests product catalogs from vendors, normalizes the data, and provides a conversational interface for customers to query the catalogs. In some examples, a chatbot system receives a natural language query from a customer about a product in a chat interface, identifies vendors offering that product by searching uploaded product catalogs, determines available inventory for the product by querying the catalogs, and generates a natural language response to the customer using the vendor and inventory information. The system can extract product details from the query, search based on those details, rank and recommend vendors and products, update user profiles, offer purchase incentives, and complete transactions within the conversation. The system handles the conversational and technical aspects to enable natural dialog between businesses and customers regarding products.
