A composite image generation system segments a face from at least a first image and a second image and generates a composite image comprising the segmented face of at least the first image and the second image. The composite image generation system further generates a text prompt comprising randomized characteristics for a new image an selects a pose from a plurality of predefined poses. Using a machine learning model, the composite image generation system generates a new image based on the text prompt, the composite image, and the selected pose. The new image can be displayed on a computing device, added or posted to a media collection, other otherwise displayed or shared to one or more users.
Examples relate to systems and methods for modifying a video during a video call. The system receives a request to initiate a video call to a recipient. The system, in response to receiving the request, displays, while attempting to establish a connection with the recipient, a video preview interface that includes a plurality of digital effect indicators available for application during the video call. The system enables selection and application of an individual digital effect indicator from the plurality of digital effect indicators to modify the video preview using a corresponding digital effect.
G06F 3/0482 - Interaction avec des listes d’éléments sélectionnables, p. ex. des menus
G06F 3/0484 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] pour la commande de fonctions ou d’opérations spécifiques, p. ex. sélection ou transformation d’un objet, d’une image ou d’un élément de texte affiché, détermination d’une valeur de paramètre ou sélection d’une plage de valeurs
H04L 65/1069 - Établissement ou terminaison d'une session
H04L 65/1089 - Procédures en session en ajoutant des médiasProcédures en session en supprimant des médias
3.
CHARGING AND DATA ACCESS PORT FOR HEAD-WEARABLE APPARATUS
In some examples, a head-wearable apparatus for viewing augmented reality (AR) or virtual reality (VR) content is provided. An example the apparatus comprises a frame, an optical assembly including an image display in which the AR or VR content may be viewed by a user, and a user input device operable by the user to navigate through content viewed in the image display, or to invoke a function of the head-wearable apparatus. The user input device includes a body manually engageable by the user to perform a content navigation or function invocation operation and is configured to present at least one contact for accepting a connection to an external charging source, or a connection to an external device.
Examples described herein relate to hand-based light estimation for extended reality (XR). An image sensor of an XR device is used to obtain an image of a hand in a real-world environment. At least part of the image is processed to detect a pose of the hand. One of a plurality of machine learning models is selected based on the detected pose. At least part of the image is processed via the machine learning model to obtain estimated illumination parameter values associated with the hand. The estimated illumination parameter values are used to render virtual content to be presented by the XR device.
G06T 7/73 - Détermination de la position ou de l'orientation des objets ou des caméras utilisant des procédés basés sur les caractéristiques
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
G06V 10/70 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique
G06V 10/774 - Génération d'ensembles de motifs de formationTraitement des caractéristiques d’images ou de vidéos dans les espaces de caractéristiquesDispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant l’intégration et la réduction de données, p. ex. analyse en composantes principales [PCA] ou analyse en composantes indépendantes [ ICA] ou cartes auto-organisatrices [SOM]Séparation aveugle de source méthodes de Bootstrap, p. ex. "bagging” ou “boosting”
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
G06V 40/10 - Corps d’êtres humains ou d’animaux, p. ex. occupants de véhicules automobiles ou piétonsParties du corps, p. ex. mains
G06V 40/20 - Mouvements ou comportement, p. ex. reconnaissance des gestes
The described system facilitates personalized avatar interactions by dynamically generating and displaying modified garments. The system determines the initiation of an interaction function by a first user with a second user within an interaction platform. It accesses avatar data for the first user, including visual attributes and a garment associated with the first avatar, as well as avatar data for the second user, including visual attributes and a garment associated with the second avatar. An image is generated featuring the first avatar wearing its garment alongside the second avatar wearing its garment. This image is applied to the first avatar's garment, creating a modified version that visually incorporates the second avatar. The system then displays the first avatar wearing the modified garment alongside the second avatar wearing its original garment, enabling enhanced user engagement through visually personalized and contextual representations of user interactions.
A63F 13/213 - Dispositions d'entrée pour les dispositifs de jeu vidéo caractérisées par leurs capteurs, leurs finalités ou leurs types comprenant des moyens de photo-détection, p. ex. des caméras, des photodiodes ou des cellules infrarouges
A63F 13/53 - Commande des signaux de sortie en fonction de la progression du jeu incluant des informations visuelles supplémentaires fournies à la scène de jeu, p. ex. en surimpression pour simuler un affichage tête haute [HUD] ou pour afficher une visée laser dans un jeu de tir
A63F 13/67 - Création ou modification du contenu du jeu avant ou pendant l’exécution du programme de jeu, p. ex. au moyen d’outils spécialement adaptés au développement du jeu ou d’un éditeur de niveau intégré au jeu en s’adaptant à ou par apprentissage des actions de joueurs, p. ex. modification du niveau de compétences ou stockage de séquences de combats réussies en vue de leur réutilisation
A63F 13/795 - Aspects de sécurité ou de gestion du jeu incluant des données sur les joueurs, p. ex. leurs identités, leurs comptes, leurs préférences ou leurs historiques de jeu pour trouver d’autres joueursAspects de sécurité ou de gestion du jeu incluant des données sur les joueurs, p. ex. leurs identités, leurs comptes, leurs préférences ou leurs historiques de jeu pour constituer une équipeAspects de sécurité ou de gestion du jeu incluant des données sur les joueurs, p. ex. leurs identités, leurs comptes, leurs préférences ou leurs historiques de jeu pour fournir une "liste d’amis"
A63F 13/87 - Communiquer avec d’autres joueurs, p. ex. par courrier électronique ou messagerie instantanée
Systems and methods for projecting each of a chronology of images as a sequence of images using a shifting element as part of a near-eye display system are provided for use in virtual reality, augmented reality, or mixed reality systems. In some example embodiments, a chronology of images is received by a peripheral sequencing system. The system divides each image into image portions and generates sequences of image portions to recreate the images based on arrangement data. The system then causes a high-speed display of each sequence of images such that they appear simultaneous to a viewer. In some embodiments, the projection is transmitted to a shifting optical element such as a rotating micromirror that propagates a display to a user. In some embodiments, the system further detects and corrects for image and environmental distortions.
An eXtended Reality (XR) content creation environment is provided. A machine learning model is trained by receiving a set of three-dimensional (3D) assets. The training process includes an alignment filter component that generates aligned 3D assets by orienting each 3D asset to face a consistent predefined direction in global-space. A renderer generates multiview global-space normal maps that maintain consistent orientation independently of camera position for the aligned 3D assets. The renderer also generates corresponding renders that serve as ground truth data. The machine learning model is trained using the global-space normal maps and prompts as conditioning input while using the renders as ground truth, with loss backpropagation applied between generated output textures and the ground truth renders.
Eyewear that includes a frame supporting an optical element. The frame has a first side and a second side. The eyewear also includes a temple adjacent the first side of the frame. The temple includes a first portion adjacent the frame and a second portion releasably connected to the first portion. The eyewear also includes an electrical connector embedded within the first portion of the temple. The second portion conceals the electrical connector from an exterior of the eyewear when the second portion connects to the first portion in a concealed state, and exposes the electrical connector from the exterior of the eyewear when disconnected from the first portion in an exposed state.
Methods and systems are disclosed for performing operations for providing a shared augmented reality experience in a video chat. A video chat can be established between a plurality of client devices. During the video chat, videos of users associated with the client devices can be displayed. During the video chat, a request from a first client device to activate a first AR experience can be received, and in response, and body parts of users depicted in the videos are modified to include one or more AR elements associated with the first AR experience.
H04L 12/18 - Dispositions pour la fourniture de services particuliers aux abonnés pour la diffusion ou les conférences
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
H04L 51/52 - Messagerie d'utilisateur à utilisateur dans des réseaux à commutation de paquets, transmise selon des protocoles de stockage et de retransmission ou en temps réel, p. ex. courriel pour la prise en charge des services des réseaux sociaux
Examples in the present disclosure relate to the prediction of motion of a body part by an extended reality (XR) device. Tracking data is captured by one or more sensors associated with the XR device. The tracking data is processed to track the body part. Based on the tracking of the body part and a kinematic model of the body part, kinematic state tracking data is dynamically updated. The kinematic model and the kinematic state tracking data are used to generate a predicted future kinematic state of the body part. In some examples, operation of the XR device is controlled based on the predicted future kinematic state.
An eXtended Reality (XR) system provides grasp detection of a user grasping a virtual object. The grasp detection may be used as a user input into an XR application. The XR system provides a user interface of the XR application to a user of the XR system, the user interface including one or more virtual objects. The XR system captures video frame tracking data of a pose of a hand of a user while the user interacts with a virtual object of the one or more virtual objects and generates skeletal model data of the hand of the user based on the video frame tracking data. XR system generates grasp detection data based on the skeletal model data and virtual object data of the virtual object, and provides the grasp detection data to the XR application as user input into the XR application.
Augmented reality experiences with an eyewear device including a position detection system and a display system are provided. The eyewear device detects at least one of a hand gesture or movement of the user in the physical environment, and associates the hand gesture with a setting for a virtual object held within the memory of the eyewear device. The eyewear device may then change an attribute of the virtual object based on the detected hand gesture or movement. The eyewear device then provides an output corresponding to one or more attributes of the virtual object. The virtual object may be, for example, a music player or a virtual game piece.
G06F 3/04815 - Interaction s’effectuant dans un environnement basé sur des métaphores ou des objets avec un affichage tridimensionnel, p. ex. modification du point de vue de l’utilisateur par rapport à l’environnement ou l’objet
A method of correcting perspective distortion of a selfie image captured with a short camera-to-face distance by processing the selfie image and generating an undistorted selfie image appearing to be taken with a longer camera-to-face distance. A pre-trained 3D face GAN processes the selfie image, inverts the 3D face GAN to obtain improved face latent code and camera parameters, fine tunes a 3D face GAN generator, and manipulates camera parameters to render a photorealistic face selfie image. The processed selfie image has less distortion in the forehead, nose, cheek bones, jaw line, chin, lips, eyes, eyebrows, ears, hair, and neck of the face.
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, method, and user interface to facilitate a shared control of a virtual object by two or more users. A virtual object is displayed by a first device, for example, as part of an augmented reality experience where the display of the object is overlaid on a real-world environment. User input indicative of a modification to the virtual object is received. The virtual object is modified, and a modified virtual object is displayed by a second device.
The described system facilitates personalized avatar interactions by dynamically generating and displaying modified garments. The system determines the initiation of an interaction function by a first user with a second user within an interaction platform. It accesses avatar data for the first user, including visual attributes and a garment associated with the first avatar, as well as avatar data for the second user, including visual attributes and a garment associated with the second avatar. An image is generated featuring the first avatar wearing its garment alongside the second avatar wearing its garment. This image is applied to the first avatar's garment, creating a modified version that visually incorporates the second avatar. The system then displays the first avatar wearing the modified garment alongside the second avatar wearing its original garment, enabling enhanced user engagement through visually personalized and contextual representations of user interactions.
An artificial intelligence (AI) network or neural network is trained, using a relatively small number of reference images of a target garment, to enable virtual clothing try-ons of the target garment. Example methods include determining a pose for a person depicted in an input image, determining an area of the input image to replace with a target garment, changing values of pixels within the area, and inputting the pose, the area, and a text prompt describing the target garment, into a neural network, to generate an output image, wherein the neural network is trained to generate the target garment. Example methods include training the neural network with images of clothing in a same class or category as the target garment to teach the neural network to shape the target garment in accordance with a pose of the person and to preserve other clothing and the background.
A method for reducing motion-to-photon latency for hand tracking is described. In one aspect, a method includes accessing a first frame from a camera of an Augmented Reality (AR) device, tracking a first image of a hand in the first frame, rendering virtual content based on the tracking of the first image of the hand in the first frame, accessing a second frame from the camera before the rendering of the virtual content is completed, the second frame immediately following the first frame, tracking, using the computer vision engine of the AR device, a second image of the hand in the second frame, generating an annotation based on tracking the second image of the hand in the second frame, forming an annotated virtual content based on the annotation and the virtual content, and displaying the annotated virtual content in a display of the AR device.
Systems and methods are provided for presenting videos. The systems and methods access a video playback graphical user interface (GUI) that automatically plays back a plurality of videos in sequence. The systems and methods determine, by the one or more processors, a current mute state of the video playback GUI, a disabled mute state allowing output of audio associated with the playback of the plurality of videos, and an enabled mute state preventing the output of the audio associated with the playback of the plurality of videos. The systems and methods conditionally present an indicator that visually informs a user that audio is currently in the enabled mute state while an individual video of the plurality of videos is being played back based on the current mute state of the GUI.
H04N 21/439 - Traitement de flux audio élémentaires
H04N 21/472 - Interface pour utilisateurs finaux pour la requête de contenu, de données additionnelles ou de servicesInterface pour utilisateurs finaux pour l'interaction avec le contenu, p. ex. pour la réservation de contenu ou la mise en place de rappels, pour la requête de notification d'événement ou pour la transformation de contenus affichés
19.
AUTOMATICALLY GENERATING DESCRIPTIONS OF AUGMENTED REALITY EFFECTS
A first image and a second image are accessed. The second image is generated by applying an augmented reality (AR) effect to the first image. The first image, the second image, and a prompt are provided to a visual-semantic machine learning model to obtain output describing at least one feature of the AR effect. A description of the AR effect is generated based on the output of the visual-semantic machine learning model. The description of the AR effect is stored in association with an identifier of the AR effect.
Methods and systems are disclosed for generating an extended reality (XR) try-on experience. The methods and systems store, in a multimodal memory, interaction data representing use of one or more interaction functions including data in different modalities. The methods and systems detect an object depicted in an image captured by an interaction client and generate, by a machine learning model, a prompt based on the object depicted in the image and the interaction data in the multimodal memory. The methods and systems generate an artificial texture based on the prompt and modify a texture of the object depicted in the image using the artificial texture that has been generated based on the prompt.
A system for providing context based creative tools and configured to perform operations that include: causing display of first media content at a client device, the first media content comprising a media attribute; detecting the media attribute of the first media content; presenting a graphical icon based on the media attribute of the media content at the client device; receiving an input that selects the graphical icon; and generating second media content at the client device based on the input that selects the graphical icon.
G06F 16/587 - Recherche caractérisée par l’utilisation de métadonnées, p. ex. de métadonnées ne provenant pas du contenu ou de métadonnées générées manuellement utilisant des informations géographiques ou spatiales, p. ex. la localisation
22.
REDUCING POWER CONSUMPTION OF EXTENDED REALITY DEVICES
Examples describe a method performed by an extended reality (XR) device that implements a multi-camera object tracking system. The XR device accesses object tracking data associated with an object in a real-world environment. Based on the object tracking data, the XR device activates a low-power mode of the multi-camera object tracking system. In the low-power mode, a state of the object in the real-world environment is determined by using the multi-camera object tracking system.
G06F 1/3287 - Économie d’énergie caractérisée par l'action entreprise par la mise hors tension d’une unité fonctionnelle individuelle dans un ordinateur
G06V 40/10 - Corps d’êtres humains ou d’animaux, p. ex. occupants de véhicules automobiles ou piétonsParties du corps, p. ex. mains
23.
EXTENDED REALITY EXPERIENCES WITH SHARED HAND TRACKING
Systems and methods in the present disclosure relate to the sharing of hand tracking data in the context of multi-user extended reality (XR) experiences. A first XR device of a first user and a second XR device of a second user participate in a shared XR experience. The first XR device receives, from the second XR device and via a communication link, hand tracking data for a second hand of the second user. The first XR device captures images that include a first hand of the first user and the second hand of the second user. While the shared XR experience is in progress, tracking operations performed by the first XR device are controlled based on the images captured by the first XR device and the hand tracking data it receives from the second XR device.
Described herein are systems and methods for optimizing neural network models for deployment on resource-constrained computing devices through layer-specific quantization. An original neural network model and deployment constraints are received as inputs. The optimization process alternates between a learning phase that updates model weights using task-specific loss functions and a compression phase that determines optimal bitwidth allocations for each layer through multiple-choice knapsack optimization. The compression phase computes quantization errors for different bitwidth options per layer and selects optimal bitwidth combinations while satisfying deployment constraints. The process iteratively updates a penalty parameter and continues until convergence, producing an optimized neural network model with quantized weights and layer-specific bitwidth allocations that maintains performance while meeting size, computational, and latency constraints for the target device.
Systems and methods in the present disclosure relate to the sharing of hand tracking data in the context of multi-user extended reality (XR) experiences. A first XR device of a first user and a second XR device of a second user participate in a shared XR experience. The first XR device receives, from the second XR device and via a communication link, hand tracking data for a second hand of the second user. The first XR device captures images that include a first hand of the first user and the second hand of the second user. While the shared XR experience is in progress, tracking operations performed by the first XR device are controlled based on the images captured by the first XR device and the hand tracking data it receives from the second XR device.
Systems and methods are provided for navigating messaging application interfaces. The systems and methods include operations for: displaying, by a messaging application of a user device, a menu comprising a first set of options relating to a first level in a hierarchy of levels; detecting, by a touch sensor, one finger touch of a first option of the first set of options; in response to detecting the one finger touch of the first option, displaying, by the messaging application, a second set of options related to the first option, the second set of options relating to a second level in the hierarchy of levels; detecting, by the touch sensor, two finger touch of a second option of the second set of options; and in response to detecting the two finger touch of the second option, re-displaying, by the messaging application, the first set of options.
G06F 3/0482 - Interaction avec des listes d’éléments sélectionnables, p. ex. des menus
G06F 3/04847 - Techniques d’interaction pour la commande des valeurs des paramètres, p. ex. interaction avec des règles ou des cadrans
G06F 3/0488 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] utilisant des caractéristiques spécifiques fournies par le périphérique d’entrée, p. ex. des fonctions commandées par la rotation d’une souris à deux capteurs, ou par la nature du périphérique d’entrée, p. ex. des gestes en fonction de la pression exercée enregistrée par une tablette numérique utilisant un écran tactile ou une tablette numérique, p. ex. entrée de commandes par des tracés gestuels
27.
VIRTUAL EVALUATION TOOLS FOR AUGMENTED REALITY EXERCISE EXPERIENCES
Example systems, devices, media, and methods are described for evaluating movements and physical exercises in augmented reality using the display of an eyewear device. A motion evaluation application implements and controls the capturing of frames of motion data using an inertial measurement unit (IMU) on the eyewear device. The method includes presenting virtual targets on the display, localizing the current eyewear device location based on the captured motion data, and presenting virtual indicators on the display. The virtual targets represent goals or benchmarks for the user to achieve using body postures. The method includes detecting determining whether the eyewear device location represents an intersecting posture relative to the virtual targets, based on the IMU data. The virtual indicators display real-time feedback about user posture or performance relative to the virtual targets.
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06F 3/0354 - Dispositifs de pointage déplacés ou positionnés par l'utilisateurLeurs accessoires avec détection des mouvements relatifs en deux dimensions [2D] entre le dispositif de pointage ou une partie agissante dudit dispositif, et un plan ou une surface, p. ex. souris 2D, boules traçantes, crayons ou palets
G06V 40/20 - Mouvements ou comportement, p. ex. reconnaissance des gestes
Method of generating a real-time avatar animation starts with a processor receiving acoustic segments of a real-time acoustic signal. For each of the acoustic segments, processor generates using a music analyzer neural network a tempo value and a dance energy category and selects dance tracks based on the tempo value and the dance energy category. Processor generates using the dance tracks dance sequences for avatars, generates real-time animations for the avatars based on the dance sequences and avatar characteristics for the avatars, and causes to be displayed on a first client device the real-time animations of the avatars. Other embodiments are described herein.
Various embodiments provide for systems, methods, and computer-readable storage media for annotating a collection of media items, such as digital images. According to some embodiments, an annotation system automatically determines one or more annotations for a plurality of media content items, and generates a collection of media content items that associates the determined annotations with the plurality of media content items. Depending on the embodiment, annotations that may be determined for the plurality of media content (and associated with the collection for the media content items) can include, without limitation, a caption, a geographic location, a category, a novelty measurement, an event, and a highlight media content item representing the collection.
G06F 40/169 - Annotation, p. ex. données de commentaires ou notes de bas de page
G06F 16/2457 - Traitement des requêtes avec adaptation aux besoins de l’utilisateur
G06F 16/41 - IndexationStructures de données à cet effetStructures de stockage
G06F 16/48 - Recherche caractérisée par l’utilisation de métadonnées, p. ex. de métadonnées ne provenant pas du contenu ou de métadonnées générées manuellement
G06F 16/487 - Recherche caractérisée par l’utilisation de métadonnées, p. ex. de métadonnées ne provenant pas du contenu ou de métadonnées générées manuellement utilisant des informations géographiques ou spatiales, p. ex. la localisation
30.
INTRINSIC PARAMETERS ESTIMATION IN VISUAL TRACKING SYSTEMS
A method for adjusting camera intrinsic parameters of a multi-camera visual tracking device is described. In one aspect, a method for calibrating the multi-camera visual tracking system includes disabling a first camera of the multi-camera visual tracking system while a second camera of the multi-camera visual tracking system is enabled, detecting a first set of features in a first image generated by the first camera after detecting that the temperature of the first camera is within the threshold of the factory calibration temperature of the first camera, and accessing and correcting intrinsic parameters of the second camera based on the projection of the first set of features in the second image and a second set of features in the second image.
G06T 7/80 - Analyse des images capturées pour déterminer les paramètres de caméra intrinsèques ou extrinsèques, c.-à-d. étalonnage de caméra
G06V 10/44 - Extraction de caractéristiques locales par analyse des parties du motif, p. ex. par détection d’arêtes, de contours, de boucles, d’angles, de barres ou d’intersectionsAnalyse de connectivité, p. ex. de composantes connectées
H04N 17/00 - Diagnostic, test ou mesure, ou leurs détails, pour les systèmes de télévision
H04N 23/61 - Commande des caméras ou des modules de caméras en fonction des objets reconnus
H04N 23/65 - Commande du fonctionnement de la caméra en fonction de l'alimentation électrique
H04N 23/68 - Commande des caméras ou des modules de caméras pour une prise de vue stable de la scène, p. ex. en compensant les vibrations du boîtier de l'appareil photo
31.
MODEL FINE-TUNING FOR AUTOMATED AUGMENTED REALITY DESCRIPTIONS
A second input image is generated by applying a target augmented reality (AR) effect to a first input image. The first input image and the second input image are provided to a first visual-semantic machine learning model to obtain output describing at least one feature of the target AR effect. The first visual-semantic machine learning model is fine-tuned from a second visual-semantic machine learning model by using training samples. Each training sample comprises a first training image, a second training image, and a training description of a given AR effect. The second training image is generated by applying the given AR effect to the first training image. A description of the target AR effect is selected based on the output of the visual-semantic machine learning model. The description of the target AR effect is stored in association with an identifier of the target AR effect.
G06V 10/774 - Génération d'ensembles de motifs de formationTraitement des caractéristiques d’images ou de vidéos dans les espaces de caractéristiquesDispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant l’intégration et la réduction de données, p. ex. analyse en composantes principales [PCA] ou analyse en composantes indépendantes [ ICA] ou cartes auto-organisatrices [SOM]Séparation aveugle de source méthodes de Bootstrap, p. ex. "bagging” ou “boosting”
G06V 20/20 - ScènesÉléments spécifiques à la scène dans les scènes de réalité augmentée
G06V 20/40 - ScènesÉléments spécifiques à la scène dans le contenu vidéo
Examples disclosed herein describe visual-inertial tracking techniques for extended reality (XR) devices. According to some example methods, an XR device is located in, and movable relative to, a vehicle. The XR device generates device tracking data and accesses vehicle tracking data. The vehicle tracking data is generated by an external sensor configured to measure motion of the vehicle. Consolidated tracking data is generated based on the device tracking data and the vehicle tracking data. In some examples, a pose of the XR device is determined by using the consolidated tracking data.
In various embodiments, a waveguide assembly includes first and second waveguide slabs. The first waveguide slab is to receive image-bearing light and enlarge a pupil size of the light parallel to a first axis. The first waveguide slab includes a first input-coupling device to couple image-bearing light into the first waveguide slab under total internal reflection (TIR), and a first out-coupling region to decouple image-bearing light out of the first waveguide slab by reflection. The second waveguide slab is to couple at least a portion of the out-coupled image-bearing light from the first waveguide slab into the second waveguide slab and enlarge the pupil size parallel to a second axis, which is substantially orthogonal to the first axis. The second waveguide slab includes a diffractive in-coupling region and a transmissive diffractive out-coupling region, through which a user can view real world imagery and the out-coupled image-bearing light simultaneously.
A waveguide for use in an augmented reality or virtual reality display comprises a plurality of optical structures in or on a photonic crystal. The plurality of optical structures are arranged in an array to provide two diffractive optical elements overlaid on one another in the waveguide. Each of the two diffractive optical elements is configured to receive light from an input direction and couple it towards the other diffractive optical element which can then act as an output diffractive optical element, providing outcoupled orders towards a viewer. The plurality of optical structures have different respective cross sectional shapes, for a cross section parallel to the plane of the waveguide, at different positions in the array in order to provide different diffraction efficiencies at different positions in the array.
H04M 1/72406 - Interfaces utilisateur spécialement adaptées aux téléphones sans fil ou mobiles avec des moyens de soutien local des applications accroissant la fonctionnalité fournis par mise à jour ou téléchargement de logiciel
36.
AUGMENTED REALITY GAMING USING VIRTUAL EYEWEAR BEAMS
Interactive augmented reality experiences with an eyewear device including a virtual eyewear beam. The user can direct the virtual beam by orienting the eyewear device or the user’s eye gaze or both. The eyewear device may detect the direction of an opponent’s eyewear device or eye gaze of both. The eyewear device may calculate a score based on hits of the virtual beam of the user and the opponent on respective target areas such as the other player’s head or face.
A method to enhance virtual audio capture in Augmented Reality (AR) experience recordings starts with a processor receiving a video from a camera that includes images of a real-world scene and an AR content item. Processor receives acoustic signals from microphones generate acoustic signals using real-world audio and speaker output that including AR audio of the AR content item. Processor receives an audio file associated with the AR audio of the AR content item and generates an enhanced audio using the acoustic signals and the audio file. Processor generates an enhanced video using the video and the enhanced audio. Other examples are described herein.
A system is disclosed, including a processor and a memory. The memory stores instructions that, when executed by the processor, configure the system to perform operations. Raw region of interest (ROI) information is obtained, identifying a raw ROI having a raw ROI size, within a first video frame captured by a camera. Motion information representative of motion of the raw ROI relative to a field of view (FOV) of the camera is obtained. The raw ROI information and the motion information are processed to generate a dynamic ROI having a dynamic ROI size larger than the raw ROI size. A second video frame is captured by the camera. A portion of the second video frame defined by the dynamic ROI is processed to generate autoexposure (AE) settings for the camera.
H04N 23/73 - Circuits de compensation de la variation de luminosité dans la scène en influençant le temps d'exposition
H04N 23/611 - Commande des caméras ou des modules de caméras en fonction des objets reconnus les objets reconnus comprenant des parties du corps humain
H04N 23/72 - Combinaison de plusieurs commandes de compensation
An eXtended Reality (XR) system that determines a non-interaction intent of a user is provided. The XR system captures hand tracking data using tracking sensors that include cameras capable of capturing hand movements and gestures in real-time. The XR system also captures pose data of the head-wearable apparatus using pose sensors including an Inertial Measurement Unit (IMU) and cameras to determine Six Degrees of Freedom (6 DoF) data. The XR system detects non-interaction indicators by analyzing the hand tracking data to identify situations where the user likely does not intend to interact with virtual content. Based on these detected non-interaction indicators, the system modifies virtual interaction capabilities of an XR user interface such as by selectively enabling or disabling virtual cursor feedback while maintaining direct manipulation abilities.
The present disclosure provides a method and system for training a neural network to be executed by a processor system comprising a plurality of processor cores. During training, an aggregate loss is computed that is based on a task loss indicative of an accuracy with which the neural network performs a task and based on a computation performance loss which indicates an estimation of a computation performance parameter. Weights of the neural network are then updated by backpropagation of the aggregated loss.
Examples described herein relate to systems and methods for automatic evaluation of graphical element recommendations, such as sticker recommendations. According to some examples, a system accesses a set of text queries and provides each text query as input to a graphical element recommendation machine learning model. The graphical element recommendation machine learning model is trained to generate, based on a given text query, one or more graphical element recommendations for use in a message in a context of a messaging interface of an interaction application. The system obtains, from the graphical element recommendation machine learning model, at least one graphical element recommendation for each text query. The system generates a model quality score for the graphical element recommendation machine learning model by applying a model quality metric to the graphical element recommendations. Output indicative of the model quality score may be presented at a user device.
An extended Reality (XR) system that determines a non-interaction intent of a user is provided. The XR system captures hand tracking data using tracking sensors that include cameras capable of capturing hand movements and gestures in real-time. The XR system also captures pose data of the head-wearable apparatus using pose sensors including an Inertial Measurement Unit (IMU) and cameras to determine Six Degrees of Freedom (6D0F) data. The XR system detects non-interaction indicators by analyzing the hand tracking data to identify situations where the user likely does not intend to interact with virtual content. Based on these detected non-interaction indicators, the system modifies virtual interaction capabilities of an XR user interface such as by selectively enabling or disabling virtual cursor feedback while maintaining direct manipulation abilities.
A messaging system, which hosts a backend service for an associated messaging client, includes a voice chat system that provides voice chat functionality that enables users to dictate their messages, while delivering the resulting message to the intended recipient as both the associated audio and text content. When a user at a sender client device begins dictating a voice message, the voice chat system starts converting the received audio stream into text and, also, starts communicating the audio content together with the generated text to the recipient client device. The recipient user can listen to the voice message and read the text generated from the audio in real time. It is also possible for the recipient user to consume the voice message in a textual form only, if the sound at the client device is undesirable.
G10L 15/26 - Systèmes de synthèse de texte à partir de la parole
G10L 15/22 - Procédures utilisées pendant le processus de reconnaissance de la parole, p. ex. dialogue homme-machine
G10L 15/30 - Reconnaissance distribuée, p. ex. dans les systèmes client-serveur, pour les applications en téléphonie mobile ou réseaux
G10L 25/51 - Techniques d'analyse de la parole ou de la voix qui ne se limitent pas à un seul des groupes spécialement adaptées pour un usage particulier pour comparaison ou différentiation
G10L 25/90 - Détermination de la hauteur tonale des signaux de parole
H04L 51/04 - Messagerie en temps réel ou quasi en temps réel, p. ex. messagerie instantanée [IM]
Systems and methods herein describe generating user-defined contextual spaces by receiving, by an image of a view of a physical space from a camera, receiving a virtual boundary of a first portion of the physical space, determining a virtual volume of the first portion of the physical space based on the first input, receiving data associated with to the first portion of the physical space, and storing the virtual volume in association with the received data.
A neural network processor is provided comprising a plurality of mutually succeeding neural network processor layers is provided. A neural network processor layer therein comprising a plurality of neural network processor elements (1) having a respective state register (2) for storing a state value (X) indicative for their state, as well as an additional state register (4) for storing a value (Q) of a state value change indicator that is indicative for a direction of a previous state change exceeding a threshold value. Neural network processor elements in a neural network processor layer are configured to selectively transmit differential event messages indicative for a change of their state, dependent both on the change of their state value and on the value of their state value change indicator.
G06N 3/04 - Architecture, p. ex. topologie d'interconnexion
G06F 18/2113 - Sélection du sous-ensemble de caractéristiques le plus significatif en classant ou en filtrant l'ensemble des caractéristiques, p. ex. en utilisant une mesure de la variance ou de la corrélation croisée des caractéristiques
At least one unit of a software application is identified. The at least one unit includes source code. The source code of the at least one unit is analyzed to determine a style of the source code. Metadata is extracted from the at least one unit based on the source code analysis. One or more features of the extracted metadata are classified. A template file is modified based on the extracted metadata and the classified features to create a modified template file.
A display driver device (210) receives a downloadable “sequence” for dynamically reconfiguring displayed image characteristics in an image system. The display driver device comprises one or more storage devices, for example, memory devices, for storing image data (218) and portions of drive sequences (219) that are downloadable and/or updated in real time depending on various inputs (214).
G09G 3/20 - Dispositions ou circuits de commande présentant un intérêt uniquement pour l'affichage utilisant des moyens de visualisation autres que les tubes à rayons cathodiques pour la présentation d'un ensemble de plusieurs caractères, p. ex. d'une page, en composant l'ensemble par combinaison d'éléments individuels disposés en matrice
09 - Appareils et instruments scientifiques et électriques
14 - Métaux précieux et leurs alliages; bijouterie; horlogerie
16 - Papier, carton et produits en ces matières
21 - Ustensiles, récipients, matériaux pour le ménage; verre; porcelaine; faience
25 - Vêtements; chaussures; chapellerie
28 - Jeux, jouets, articles de sport
35 - Publicité; Affaires commerciales
41 - Éducation, divertissements, activités sportives et culturelles
42 - Services scientifiques, technologiques et industriels, recherche et conception
Produits et services
Downloadable computer software, namely, software for augmented reality (AR) and virtual reality (VR) for integrating electronic data with real-world environments and for creating, capturing, editing, and sharing multimedia content and data; downloadable software for creating, editing, storing, and transmitting digital content and communications via the internet, communication networks, and mobile devices; downloadable augmented reality software for virtual product try-on and visualization of consumer products; downloadable software for object and sound recognition and environmental scanning; downloadable artificial intelligence and machine learning software; computer hardware, computer peripherals, and wearable computer hardware and peripherals; computer hardware and peripherals for capturing, transmitting, and displaying images, video, audio, and data; downloadable software for setting up, configuring, and controlling wearable computer hardware and peripherals; downloadable multimedia files containing audio and video content in the fields of entertainment, photography, and social networking; computer software for accessing and transmitting data and content among electronic devices and displays Key chains Stationery; Writing pads; notebooks; stickers; printed greetings cards and printed postcards Containers; Mugs; drinking glasses; thermally insulated beverage containers Clothing; clothing, namely, shirts, sweatshirts, hats Toys and games; plush toys; beach balls; playing cards Retail store services featuring computer hardware, peripherals, cameras, video cameras, and digital media, namely, pre-recorded music, videos, photographs, images, and audiovisual content; facilitating the exchange and sale of goods and services of third parties via the internet and communication networks, namely, facilitating transactions between buyers and sellers through providing buyers with information about sellers, goods, and/or services via the internet and communication networks; advertising, marketing, and promotion services; dissemination of advertising for others via computer and other communication networks; consumer profiling for commercial or marketing purposes; providing commercial consumer information and advice for consumers in the selection of products to buy Entertainment services, namely, providing online non-downloadable multimedia content in the fields of entertainment, photography, and social networking in the nature of avatars, graphic icons, symbols, images representing individuals, fanciful designs, comics, comic series, phrases, and graphical depictions of people, places and things that end users can transmit and receive by means of the Internet or other computer or telecommunication networks, wireless communications networks, or by using computers, laptops, mobile equipment, and handheld digital electronic devices; providing online augmented reality experiences for entertainment purposes; organizing and hosting social entertainment events in the fields of augmented reality and creating, editing, sharing and sending digital photos, videos, images, emojis, avatars, emoticons, text, audio, and video games Providing online non-downloadable software for augmented reality (AR) and virtual reality (VR) for integrating electronic data with real-world environments and for creating, capturing, editing, and sharing multimedia content and data; providing online non-downloadable software for creating, editing, storing, and transmitting digital content and communications via the internet, communication networks, and mobile devices; providing online non-downloadable augmented reality software for virtual product try-on and visualization of consumer products; providing online non-downloadable software for object and sound recognition and environmental scanning; providing online non-downloadable artificial intelligence and machine learning software; providing online non-downloadable software for setting up, configuring, and controlling wearable computer hardware and peripherals; providing online non-downloadable software for accessing and transmitting data and content among electronic devices and displays; platform as a service (PaaS) featuring software for the foregoing purposes
09 - Appareils et instruments scientifiques et électriques
14 - Métaux précieux et leurs alliages; bijouterie; horlogerie
16 - Papier, carton et produits en ces matières
21 - Ustensiles, récipients, matériaux pour le ménage; verre; porcelaine; faience
25 - Vêtements; chaussures; chapellerie
28 - Jeux, jouets, articles de sport
35 - Publicité; Affaires commerciales
41 - Éducation, divertissements, activités sportives et culturelles
42 - Services scientifiques, technologiques et industriels, recherche et conception
Produits et services
Downloadable computer software, namely, software for augmented reality (AR) and virtual reality (VR) for integrating electronic data with real-world environments and for creating, capturing, editing, and sharing multimedia content and data; downloadable software for creating, editing, storing, and transmitting digital content and communications via the internet, communication networks, and mobile devices; downloadable augmented reality software for virtual product try-on and visualization of consumer products; downloadable software for object and sound recognition and environmental scanning; downloadable artificial intelligence and machine learning software; computer hardware, computer peripherals, and wearable computer hardware and peripherals; computer hardware and peripherals for capturing, transmitting, and displaying images, video, audio, and data; downloadable software for setting up, configuring, and controlling wearable computer hardware and peripherals; downloadable multimedia files containing audio and video content in the fields of entertainment, photography, and social networking; computer software for accessing and transmitting data and content among electronic devices and displays Key chains Stationery; Writing pads; notebooks; stickers; printed greetings cards and printed postcards Containers; Mugs; drinking glasses; thermally insulated beverage containers Clothing; clothing, namely, shirts, sweatshirts, hats Toys and games; plush toys; beach balls; playing cards Retail store services featuring computer hardware, peripherals, cameras, video cameras, and digital media, namely, pre-recorded music, videos, photographs, images, and audiovisual content; facilitating the exchange and sale of goods and services of third parties via the internet and communication networks, namely, facilitating transactions between buyers and sellers through providing buyers with information about sellers, goods, and/or services via the internet and communication networks; advertising, marketing, and promotion services; dissemination of advertising for others via computer and other communication networks; consumer profiling for commercial or marketing purposes; providing commercial consumer information and advice for consumers in the selection of products to buy Entertainment services, namely, providing online non-downloadable multimedia content in the fields of entertainment, photography, and social networking in the nature of avatars, graphic icons, symbols, images representing individuals, fanciful designs, comics, comic series, phrases, and graphical depictions of people, places and things that end users can transmit and receive by means of the Internet or other computer or telecommunication networks, wireless communications networks, or by using computers, laptops, mobile equipment, and handheld digital electronic devices; providing online augmented reality experiences for entertainment purposes; organizing and hosting social entertainment events in the fields of augmented reality and creating, editing, sharing and sending digital photos, videos, images, emojis, avatars, emoticons, text, audio, and video games Providing online non-downloadable software for augmented reality (AR) and virtual reality (VR) for integrating electronic data with real-world environments and for creating, capturing, editing, and sharing multimedia content and data; providing online non-downloadable software for creating, editing, storing, and transmitting digital content and communications via the internet, communication networks, and mobile devices; providing online non-downloadable augmented reality software for virtual product try-on and visualization of consumer products; providing online non-downloadable software for object and sound recognition and environmental scanning; providing online non-downloadable artificial intelligence and machine learning software; providing online non-downloadable software for setting up, configuring, and controlling wearable computer hardware and peripherals; providing online non-downloadable software for accessing and transmitting data and content among electronic devices and displays; platform as a service (PaaS) featuring software for the foregoing purposes
50.
ENABLING THE VISUALLY IMPAIRED WITH AR USING FORCE FEEDBACK
A system and method provide feedback to a user, such as a visually impaired user, to guide the user to an object in the field of view of a camera mounted on a frame worn on the head of the user. A processor identifies at least one object and a body part of the user in the field of view of the camera and tracks relative positions of the body part relative to the identified object. The processor also generates and communicates at least one control signal for guiding the body part of the user to the identified object to a user feedback device worn on or adjacent the body part of the user. The feedback device receives the control signal(s) and converts the control signal(s) into at least one of sounds or haptic feedback that guides the body part to the identified object.
The technical problem of reducing the amount of processing involved when searching for customizable media content items that are suitable for incorporating input text is addressed by providing a hybrid search system. In some examples, the hybrid search system executes a rough search first, to determine whether a line of text can be incorporated into a media content item, based on character count conditions associated with the media content item. A more thorough evaluation of the input text with respect to the media content item is executed subsequent to the rough search if the rough search produces a result indicating uncertainty with respect to whether the combination of specific characters included in the input text can or cannot be incorporated into the media content item.
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for presenting participant reactions within a virtual working environment. The program and method provide a configuration interface for configuring a virtual working environment for plural participants, the configuration interface for specifying groups of participants, each group comprising respective participants selected from among the plural participants; receive first user input, provided via the configuration interface, specifying a first group of participants; provide, for each participant in the first group, display of a reactions interface with user-selectable buttons to indicate respective reactions for displaying to the first group; receive second user input, provided via the reactions interface, selecting one of the user-selectable buttons to indicate a reaction for displaying to the first group; and provide, for each participant in the first group, display of a reaction icon corresponding to the reaction.
G06F 3/04815 - Interaction s’effectuant dans un environnement basé sur des métaphores ou des objets avec un affichage tridimensionnel, p. ex. modification du point de vue de l’utilisateur par rapport à l’environnement ou l’objet
G06F 3/04817 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] fondées sur des propriétés spécifiques de l’objet d’interaction affiché ou sur un environnement basé sur les métaphores, p. ex. interaction avec des éléments du bureau telles les fenêtres ou les icônes, ou avec l’aide d’un curseur changeant de comportement ou d’aspect utilisant des icônes
Aspects of the present disclosure involve a system for performing ray tracing between augmented reality (AR) and real-world objects. The system accesses, by the mobile device, a video depicting a first object. The system obtains, by the mobile device, a three-dimensional (3D) model of the first object. The system applies, by the mobile device, a ray tracing process to the 3D model of the first object to estimate an optical effect on a portion of the first object relative to a second object that is depicted in the video. The system modifies a visual property of the portion of the first object based on the optical effect relative to the second object.
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing make-up based augmented reality content. The program and method provide for receiving a request to present augmented reality content in association with a captured image depicting a face of the user; accessing an augmented reality content item associated with applying makeup to the face and configured to generate a mesh for tracking plural regions of the face; receiving user input selecting a region; determining at least one of a range of color values or a range of contrast values relating to available makeup products for the selected region; and presenting an interface element in association with the face, the interface element for user selection of at least one of a color value within the range of color values or a contrast value within the range of contrast values.
A45D 44/00 - Autres articles cosmétiques ou de toilette, p. ex. pour salons de coiffure
G06F 3/04845 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] pour la commande de fonctions ou d’opérations spécifiques, p. ex. sélection ou transformation d’un objet, d’une image ou d’un élément de texte affiché, détermination d’une valeur de paramètre ou sélection d’une plage de valeurs pour la transformation d’images, p. ex. glissement, rotation, agrandissement ou changement de couleur
G06F 3/04847 - Techniques d’interaction pour la commande des valeurs des paramètres, p. ex. interaction avec des règles ou des cadrans
G06F 3/04883 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] utilisant des caractéristiques spécifiques fournies par le périphérique d’entrée, p. ex. des fonctions commandées par la rotation d’une souris à deux capteurs, ou par la nature du périphérique d’entrée, p. ex. des gestes en fonction de la pression exercée enregistrée par une tablette numérique utilisant un écran tactile ou une tablette numérique, p. ex. entrée de commandes par des tracés gestuels pour l’entrée de données par calligraphie, p. ex. sous forme de gestes ou de texte
A media player providing real time rewind playback of a played media file having segments of frames. A last segment N of the played media file is cached and rendered on a device, such as a mobile device, then a previous segment N-1 is cached and rendered, and the process continues until there are no more segments of the played media file to cache and render. Only a segment of the played media file is cached at a time, rather than the whole media file, such that the played media file can be replayed on the fly.
H04N 21/433 - Opération de stockage de contenu, p. ex. opération de stockage en réponse à une requête de pause ou opérations de cache
G06F 3/04883 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] utilisant des caractéristiques spécifiques fournies par le périphérique d’entrée, p. ex. des fonctions commandées par la rotation d’une souris à deux capteurs, ou par la nature du périphérique d’entrée, p. ex. des gestes en fonction de la pression exercée enregistrée par une tablette numérique utilisant un écran tactile ou une tablette numérique, p. ex. entrée de commandes par des tracés gestuels pour l’entrée de données par calligraphie, p. ex. sous forme de gestes ou de texte
H04N 19/177 - Procédés ou dispositions pour le codage, le décodage, la compression ou la décompression de signaux vidéo numériques utilisant le codage adaptatif caractérisés par l’unité de codage, c.-à-d. la partie structurelle ou sémantique du signal vidéo étant l’objet ou le sujet du codage adaptatif l’unité étant un groupe d’images [GOP]
H04N 19/426 - Procédés ou dispositions pour le codage, le décodage, la compression ou la décompression de signaux vidéo numériques caractérisés par les détails de mise en œuvre ou le matériel spécialement adapté à la compression ou à la décompression vidéo, p. ex. la mise en œuvre de logiciels spécialisés caractérisés par les dispositions des mémoires utilisant des procédés de diminution de taille de mémoire
56.
TECHNIQUES FOR GENERATING A STYLIZED MEDIA CONTENT ITEM WITH A GENERATIVE NEURAL NETWORK
A mobile application with an improved user interface facilitates generating stylized media content items including images and videos. An end-user selects a desired visual effect from a set of options. The mobile application captures or accesses an image. The image is processed on a server using a generative neural network pre-trained to apply stylizations based on the selected effect. The server sends back the stylized image to the mobile application for display. The end-user can then save the stylized image or generate a video (e.g., an animation) showing the original image transition to the stylized image. The user interface provides an efficient creative workflow to apply aesthetic enhancements in a visual style chosen by the end-user. Generative machine learning techniques automate stylization to enable accessible media customization and sharing.
G06T 11/60 - Édition de figures et de texteCombinaison de figures ou de texte
G06F 3/0482 - Interaction avec des listes d’éléments sélectionnables, p. ex. des menus
G06F 3/0488 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] utilisant des caractéristiques spécifiques fournies par le périphérique d’entrée, p. ex. des fonctions commandées par la rotation d’une souris à deux capteurs, ou par la nature du périphérique d’entrée, p. ex. des gestes en fonction de la pression exercée enregistrée par une tablette numérique utilisant un écran tactile ou une tablette numérique, p. ex. entrée de commandes par des tracés gestuels
G06T 13/80 - Animation bidimensionnelle [2D], p. ex. utilisant des motifs graphiques programmables
A system and method are described for generating 3D garments from two-dimensional (2D) scribble images drawn by users. The system includes a conditional 2D generator, a conditional 3D generator, and two intermediate media including dimension-coupling color-density pairs and flat point clouds that bridge the gap between dimensions. Given a scribble image, the 2D generator synthesizes dimension-coupling color-density pairs including the RGB projection and density map from the front and rear views of the scribble image. A density-aware sampling algorithm converts the 2D dimension-coupling color-density pairs into a 3D flat point cloud representation, where the depth information is ignored. The 3D generator predicts the depth information from the flat point cloud. Dynamic variations per garment due to deformations resulting from a wearer's pose as well as irregular wrinkles and folds may be bypassed by taking advantage of 2D generative models to bridge the dimension gap in a non-parametric way.
Aspects of the present disclosure involve a system and a method for navigating images and AR experiences. The system and method present, by a messaging application, in a scrollable region on top of a live video feed being displayed in a GUI comprising a viewfinder, a first plurality of options associated with previously captured content items and a second plurality of options associated with AR experiences. The system and method scrolls the first plurality of options together with the second plurality of options of the scrollable region to bring one of the first plurality of options or one of the second plurality of options into focus. In response, the system and method: modify a configuration of the GUI in accordance with a first manner or modify the configuration of the GUI in accordance with a second manner.
G06F 3/04845 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] pour la commande de fonctions ou d’opérations spécifiques, p. ex. sélection ou transformation d’un objet, d’une image ou d’un élément de texte affiché, détermination d’une valeur de paramètre ou sélection d’une plage de valeurs pour la transformation d’images, p. ex. glissement, rotation, agrandissement ou changement de couleur
G06F 3/04815 - Interaction s’effectuant dans un environnement basé sur des métaphores ou des objets avec un affichage tridimensionnel, p. ex. modification du point de vue de l’utilisateur par rapport à l’environnement ou l’objet
G06F 3/0482 - Interaction avec des listes d’éléments sélectionnables, p. ex. des menus
G06F 3/0485 - Défilement ou défilement panoramique
G06F 3/0488 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] utilisant des caractéristiques spécifiques fournies par le périphérique d’entrée, p. ex. des fonctions commandées par la rotation d’une souris à deux capteurs, ou par la nature du périphérique d’entrée, p. ex. des gestes en fonction de la pression exercée enregistrée par une tablette numérique utilisant un écran tactile ou une tablette numérique, p. ex. entrée de commandes par des tracés gestuels
H04L 51/04 - Messagerie en temps réel ou quasi en temps réel, p. ex. messagerie instantanée [IM]
A convolutional neural network processing system includes a data processor to process input feature map data and generate output feature map data in an output feature map storage space. Series of output feature data elements are stored at respective series of mutually successive locations in the output feature map storage space. The data processor identifies an input feature data element in the input feature map data, and accesses a series of mutually successive locations in the output feature map storage space. The data processor updates the output feature map data in an input-centric manner by using an update function to update a set of output feature data elements associated with the input feature data element by a convolution kernel of the update function. The set of output feature data elements is located in the accessed series of mutually successive locations.
A system is disclosed, including a processor and a memory. The memory stores instructions that, when executed by the processor, configure the system to perform operations. Autoexposure (AE) primary camera information is obtained, identifying a first camera as an AE primary camera to be used for computing AE settings of the first camera and a second camera. First camera region of interest (ROI) information and second camera ROI information are obtained, representative of a first number of ROIs within a field of view (FOV) of the first camera and a second number of ROIs within a FOV of the second camera. In response to determining that the first number is zero and the second number is greater than zero, the AE primary camera information is updated to identify the second camera as the AE primary camera, and the second camera ROI information is processed to generate the AE settings.
H04N 23/90 - Agencement de caméras ou de modules de caméras, p. ex. de plusieurs caméras dans des studios de télévision ou des stades de sport
H04N 23/611 - Commande des caméras ou des modules de caméras en fonction des objets reconnus les objets reconnus comprenant des parties du corps humain
H04N 23/73 - Circuits de compensation de la variation de luminosité dans la scène en influençant le temps d'exposition
The present disclosure seeks to address technical problems arising in the field of artificial intelligence (AI) by providing for training of a machine learning model to generate modified images based on an input image and an input instruction. For example, the machine learning model is trained to generate a modified portrait image based on an input portrait image and an input instruction. The machine learning model generates the modified portrait image to depict the input portrait image as modified according to the input instruction while maintaining the identity of a subject depicted in the input portrait image.
G06T 11/60 - Édition de figures et de texteCombinaison de figures ou de texte
G06V 10/774 - Génération d'ensembles de motifs de formationTraitement des caractéristiques d’images ou de vidéos dans les espaces de caractéristiquesDispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant l’intégration et la réduction de données, p. ex. analyse en composantes principales [PCA] ou analyse en composantes indépendantes [ ICA] ou cartes auto-organisatrices [SOM]Séparation aveugle de source méthodes de Bootstrap, p. ex. "bagging” ou “boosting”
Eyewear providing an interactive augmented reality experience to allow a user of an eyewear device to display a 3D overlay image on a viewed person. The user can select the overlay image from a list of images, such as costumes, stored in memory or generated by the user. The images can be sorted in memory based on common attributes. Registration points of the person are continuously aligned with registration points of the overlay as the person moves such that the user appears to be wearing the 3D costume during movement. By aligning the registration points, the costume adapts to different body types and heights. The coloring of the costume can change based on the environment, such as the lighting, or to contrast with colors viewed in a viewfinder.
Described is a system for dynamically applying model adaptations customized for individual users by detecting an image of a first real-world object from a camera feed, detecting landmarks on the first real-world object, and processing the landmarks on the first real-world object using a generative machine learning model to generate a first custom image template for the first real-world object where portions of the first custom image template are populated with visual content placed based on the first custom image template. The system then applies a content augmentation based on the first custom image template to the camera feed.
The subject technology receives a set of inputs from multiple input sources. The subject technology determines a set of input features based on the set of inputs from the multiple input sources. The subject technology performs a time window-based aggregation on the set of input features to generate a set of aggregated features. The subject technology performs feature extraction, using a set of modular components of a modular classifier network, on the set of aggregated features to generate a set of extracted features. The subject technology generates, using a pinch detection head, a probability score indicating the likelihood of an occurrence of a pinch gesture based on the set of extracted features. The subject technology determines, using triggering logic, whether a pinch gesture has occurred based at least in part on the probability score. The subject technology provides a pinch detection output based at least in part on the determining.
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06V 10/26 - Segmentation de formes dans le champ d’imageDécoupage ou fusion d’éléments d’image visant à établir la région de motif, p. ex. techniques de regroupementDétection d’occlusion
G06V 10/40 - Extraction de caractéristiques d’images ou de vidéos
G06V 10/764 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant la classification, p. ex. des objets vidéo
G06V 10/80 - Fusion, c.-à-d. combinaison des données de diverses sources au niveau du capteur, du prétraitement, de l’extraction des caractéristiques ou de la classification
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
G06V 40/20 - Mouvements ou comportement, p. ex. reconnaissance des gestes
66.
AUGMENTED REALITY CONTENT GENERATORS FOR IDENTIFYING DESTINATION GEOLOCATIONS AND PLANNING TRAVEL
The subject technology causes display, at the client device, a set of augmented reality content items generated by the first augmented reality content generator. The subject technology receives, at the client device, a second selection of the particular augmented reality content item corresponding to the destination geolocation. The subject technology causes display, at the client device, a second set of augmented reality content items generated by the first augmented reality content generator. The subject technology receives, at the client device, a second selection of the second set of augmented reality content items. The subject technology causes display, at the client device, a third set of augmented reality content items generated by the first augmented reality content generator, the third set of augmented reality content items comprising at least one activity or location associated with the destination geolocation and a selected period of time.
Provided are systems and methods for providing personalized videos featuring multiple persons. An example method includes receiving a video including a plurality of frames including at least one target face, extracting target face parameters associated with the at least one target face, where the target face parameters include facial identity parameters and facial expression parameters, storing the target face parameters as metadata associated with at least one frame of the plurality of frames, receiving an image of a source face, generating source face parameters based on the image of the source face, generating an output face by combining the source face parameters with the facial expression parameters obtained from the metadata associated with the at least one frame, and generating a personalized video by replacing the at least one target face with the output face at least in the at least one frame.
G06V 40/16 - Visages humains, p. ex. parties du visage, croquis ou expressions
G11B 27/02 - Montage, p. ex. variation de l'ordre des signaux d'information enregistrés sur, ou reproduits à partir des supports d'enregistrement ou d'information
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for configuring a three-dimensional (3D) model within a virtual conferencing system. The program and method provide, in association with designing a room for virtual conferencing, an interface for configuring a 3D model; receiving, via the interface, an indication of user input for setting properties for the 3D model, the properties specifying image data for projecting onto the 3D model; and in association with virtual conferencing, providing display of the room based on the properties for the 3D model, and causing the image data to be projected onto the 3D model within the room.
G06F 3/04815 - Interaction s’effectuant dans un environnement basé sur des métaphores ou des objets avec un affichage tridimensionnel, p. ex. modification du point de vue de l’utilisateur par rapport à l’environnement ou l’objet
G06F 3/04845 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] pour la commande de fonctions ou d’opérations spécifiques, p. ex. sélection ou transformation d’un objet, d’une image ou d’un élément de texte affiché, détermination d’une valeur de paramètre ou sélection d’une plage de valeurs pour la transformation d’images, p. ex. glissement, rotation, agrandissement ou changement de couleur
G06F 3/04847 - Techniques d’interaction pour la commande des valeurs des paramètres, p. ex. interaction avec des règles ou des cadrans
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
G06T 19/20 - Édition d'images tridimensionnelles [3D], p. ex. modification de formes ou de couleurs, alignement d'objets ou positionnements de parties
Disclosed is a method of providing a music creation interface using a head-mounted device, including displaying first and second geometric loops fixed relative to a location in the real world, the first and second geometric loops each including a plurality of beat indicators. The second geometric loop is spaced apart from the first geometric loop. An interface comprising a plurality of sound or note icons is displayed, and in response to receiving user selection to move a selected sound or note icon to a particular beat indicator on one of the geometric loops, the selected sound or note icon is displayed at the particular beat indicator. In use, the geometric loops are rotated relative to at least one play indicator, and the selected sound or note icon is rendered when it reaches the at least one play indicator.
The subject technology requests a group identifier (ID) based on an item identification indicator to an extension application programming interface (API). The subject technology requests, using the extension API, a set of augmented reality (AR) content generator IDs, based on the item identification indicator, to a shopping AR content generator service. The subject technology receives the set of AR content generator IDs based on a mapping of the item identification indicator to a collection of AR content generator IDs. The subject technology requests, using the extension API, first metadata associated with the set of AR content generators and the item identification indicator to the shopping AR content generator service. The subject technology requests, using the extension API, a set of products based on the first metadata. The subject technology generates, using the extension API, a group ID based on the set of products. The subject technology receives the group ID.
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for performing operations comprising: receiving, from a client device of a first user, a request from the first user to engage in an AR shopping experience curated by a store; identifying a first real-world product available for purchase from the store; receiving an image of a real-world environment of the first user; generating a first AR item that represents the first real-world product; comparing visual attributes of the first AR item to physical layouts of a plurality of real-world objects depicted in the image of the real-world environment; and overlaying the first AR item on a first real-world object of the plurality of real-world objects in the image responsive to comparing the visual attributes of the first AR item to the physical layouts of the plurality of real-world objects.
G06K 7/14 - Méthodes ou dispositions pour la lecture de supports d'enregistrement par radiation électromagnétique, p. ex. lecture optiqueMéthodes ou dispositions pour la lecture de supports d'enregistrement par radiation corpusculaire utilisant la lumière sans sélection des longueurs d'onde, p. ex. lecture de la lumière blanche réfléchie
G06Q 10/087 - Gestion d’inventaires ou de stocks, p. ex. exécution des commandes, approvisionnement ou régularisation par rapport aux commandes
A system and method for generating augmented reality (AR) experiences are disclosed. The system generates source and target indications associated with an image transformation, and generates a first set of source images and first set of target images using a first trained machine learning (ML) model, the source indications, and the target indications. The system trains a second ML model to generate a target image corresponding to a source image based on the first set of source images and the first set of target images, and generates a second set of target images using the second trained ML model and a second set of source images. The system trains a third ML model to generate an additional target image corresponding to an additional source image based on the second set of source images and second set of target images, and generates an AR experience comprising the third trained ML model.
Systems and methods are presented for capturing a video in real-time by an image capture device using a skeletal pose system. The skeletal pose system identifies first pose information in the video, applies a first virtual effect to the video in response to identifying the first pose information, identifies second pose information in the video, and applies a second virtual effect to the video in response to identifying the first pose information.
Apparatuses, systems for electronic wearable devices such as smart glasses are described. The wearable device can comprise a frame, an elongate temple and an articulated joint. The frame can define one or more optical element holders configured to hold respective optical elements for viewing by a user in a viewing direction. The temple can be moveably connected to the frame for holding the frame in position when the device is worn by the user. The articulated joint can connect the temple and the frame to permit movement of the temple relative to the frame between a wearable position in which the temple is generally aligned with the viewing direction, and a collapsed position in which the temple extends generally transversely to the viewing direction. The articulated joint can include a base foot fixed to the frame and oriented transversely to the viewing direction.
A method for managing power resource in an augmented reality (AR) device is described. In one aspect, the method includes configuring a low-power mode to run on a low-power processor of the AR device using a first set of sensor data, and a high-power mode to run on a high-power processor of the AR device using a second set of sensor data, operating, using the low-power processor, a low-power application in the low-power mode based on the first set of sensor data, detecting a request to operate a high-power application at the AR device, in response to detecting the request, activating the second set of sensors of the AR device corresponding to the high-power mode, and operating, using the high-power processor, a high-power application in the high-power mode based on the second set of sensors.
Examples include a wearable device such as smart glasses having a frame, a temple and onboard electronics components. The frame can define one or more optical element holders for holding respective optical elements within view of a user when the eyewear body is worn. The pair of temples can be connected to the eyewear frame for supporting the eyewear frame in position within view of the user when the eyewear body is worn. The antenna can be incorporated in at least a first of the pair of temples. The antenna can include an exciter circuit, at least a portion of a battery flex of the first of the pair of temples configured as an active element of the antenna and a metal component of the first of the pair of temples configured as a ground of the antenna.
Methods, systems, mobile devices, and non-transitory computer-readable mediums for easily aesthetically enhancing images such as selfies. An example algorithm's input has three parts: image, manipulation magnitude, and text guidance. The algorithm includes two parts: (1) guidance generation based on public and personal aesthetic preferences, and (2) selfie generation. The first part outputs an image to maximize an aesthetic enhancement score (e.g., a beauty score) while following the manipulation input where the output image contains a manipulation direction. The second part is a conditional diffusion model that accepts the rendered output image from the first part and is conditioned on the input image and outputs the final image. The second part is personalized by the user's images.
Hierarchical patch-wise diffusion models (HPDMs) use a diffusion paradigm that learns a hierarchical distribution of patches instead of whole videos for efficient patch-wise training of diffusion models. To enforce consistency between the patches, deep context fusion may be used to propagate the context information from low-scale to high-scale patches in a hierarchical manner. To accelerate patch-wise training and inference, adaptive computation also may be used to allocate more computational resources and network capacity towards coarse image details and to cheapen synthesis of high-frequency texture details. All the processing stages are jointly trained to provide spatially aligned global context to the higher levels of the cascade. As a result, the model does not operate on the full-resolution inputs, which allows the model to be trained on high-resolution video datasets in an end-to-end fashion.
G06T 3/4007 - Changement d'échelle d’images complètes ou de parties d’image, p. ex. agrandissement ou rétrécissement basé sur l’interpolation, p. ex. interpolation bilinéaire
G06T 3/4046 - Changement d'échelle d’images complètes ou de parties d’image, p. ex. agrandissement ou rétrécissement utilisant des réseaux neuronaux
G06T 5/60 - Amélioration ou restauration d'image utilisant l’apprentissage automatique, p. ex. les réseaux neuronaux
A messaging system performs image processing to relight objects with neural networks for images provided by users of the messaging system. A method of relighting objects with neural networks includes receiving an input image with first lighting properties comprising an object with second lighting properties and processing the input image using a convolutional neural network to generate an output image with the first lighting properties and comprising the object with third lighting properties, where the convolutional neural network is trained to modify the second lighting properties to be consistent with lighting conditions indicated by the first lighting properties to generate the third lighting properties. The method further includes modifying the second lighting properties of the object to generate the object with modified second lighting properties and blending the third lighting properties with the modified second lighting properties to generate a modified output image comprising the object with fourth lighting properties.
Described is a system for emphasizing XR content based on user intent by gathering interaction data from use of one or more interaction functions by a user, accessing a camera feed of a camera system from the XR device, analyzing a combination of data corresponding to the interaction data and the camera feed using a first machine learning model to identify a priority for individual media content items, and determining that a first subset of media content items are of a higher priority than a second subset of media content items. Then the system displays the media content items on the XR device of the user, the first subset of the media content items displayed differently than the second subset of the media content items based on the identified priority.
Described herein are techniques for facilitating the communication of text-based messages between end-users who are using messaging applications executing on client-based computing devices with different capabilities. Specifically, the messaging system described herein enables a first end-user to add a message element to a text-based message, which, when received by a message recipient using an augmented reality messaging application, will cause a 3-D avatar representing the message sender, to animate in accordance with a specific avatar animation associated with the message element. The message element may be an emoji, or a special sequence of characters, and may be a visible or invisible (e.g., meta-data) element of the text-based message.
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
G06T 13/40 - Animation tridimensionnelle [3D] de personnages, p. ex. d’êtres humains, d’animaux ou d’êtres virtuels
G10L 13/08 - Analyse de texte ou génération de paramètres pour la synthèse de la parole à partir de texte, p. ex. conversion graphème-phonème, génération de prosodie ou détermination de l'intonation ou de l'accent tonique
Systems, methods, and computer readable media that schedules requests for location data of a mobile device, where the methods include selecting a first positioning system based on a power requirement, a latency requirement, and an accuracy requirement, and determining whether a first condition is satisfied for querying the first positioning system. The method further comprises in response to a determination that the first condition is satisfied, querying the first positioning system for first position data. The method further comprises in response to a determination that the first condition is not satisfied, selecting a second positioning system based on the power requirement, the latency requirement, and the accuracy requirement, determining whether a second condition is satisfied for querying the second positioning system, and in response to a determination that the second condition is satisfied, querying the second positioning system for second position data.
G01S 5/00 - Localisation par coordination de plusieurs déterminations de direction ou de ligne de positionLocalisation par coordination de plusieurs déterminations de distance
G01S 5/02 - Localisation par coordination de plusieurs déterminations de direction ou de ligne de positionLocalisation par coordination de plusieurs déterminations de distance utilisant les ondes radioélectriques
G01S 19/48 - Détermination de position en combinant ou en commutant entre les solutions de position dérivées du système de positionnement par satellite à radiophares et les solutions de position dérivées d'un autre système
An optical waveguide device for use in a head up display. The waveguide device provides pupil expansion in two dimensions. The waveguide device comprise a primary waveguide and a secondary waveguide, the secondary waveguide being positioned on a face of the primary waveguide. The secondary waveguide has a diffraction grating on a face opposite to the face which contacts the primary waveguide. The diffraction grating diffracts light into more than diffraction order. Rays diffracted into a non-zero order are trapped in the secondary waveguide by total internal reflection.
An addressable media system for performing operations that include: accessing image data that depicts an object in an environment at a client device; causing display of a presentation of the image data within a graphical user interface at the client device; detecting the display of the object within the presentation of the image data based on at least a portion of the plurality of image features of the display of the object; identifying an object class based on at least the portion of the image features of the display of the object; receiving an input that selects the display of the object from the client device; and associating the object class that corresponds with the object with the user profile in response to the input that selects the display of the object.
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06F 3/0481 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] fondées sur des propriétés spécifiques de l’objet d’interaction affiché ou sur un environnement basé sur les métaphores, p. ex. interaction avec des éléments du bureau telles les fenêtres ou les icônes, ou avec l’aide d’un curseur changeant de comportement ou d’aspect
G06F 3/04842 - Sélection des objets affichés ou des éléments de texte affichés
G06F 3/14 - Sortie numérique vers un dispositif de visualisation
Systems and methods for animating a single image of a human body and applying effects are provided. An example method includes receiving an image including hair, fitting a portion of the image including the hair to a hair model, generating a hair motion control parameter set defining hair animation behavior using one or more pre-set physical parameters, determining deformation of the hair model based on the hair motion control parameter set, generating a hair image based on the deformation of the hair model, and compositing the hair image with a frame of a video generated from the image.
G06T 13/40 - Animation tridimensionnelle [3D] de personnages, p. ex. d’êtres humains, d’animaux ou d’êtres virtuels
G06F 3/0482 - Interaction avec des listes d’éléments sélectionnables, p. ex. des menus
G06F 3/04845 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] pour la commande de fonctions ou d’opérations spécifiques, p. ex. sélection ou transformation d’un objet, d’une image ou d’un élément de texte affiché, détermination d’une valeur de paramètre ou sélection d’une plage de valeurs pour la transformation d’images, p. ex. glissement, rotation, agrandissement ou changement de couleur
G06T 7/194 - DécoupageDétection de bords impliquant une segmentation premier plan-arrière-plan
Methods and systems are disclosed for applying augmented reality animations to an image. The methods and systems access an image and select an augmented reality (AR) experience to apply to the image, the AR experience comprising one or more AR element animations. The methods and systems repeatedly apply the AR experience to the image over a specified time interval to overlay the one or more AR element animations on the image. The methods and systems generate a video having a duration corresponding to the specified time interval in response to repeatedly applying the AR experience to the image, the video depicting the one or more AR element animations on the image.
Display systems, media, and methods. Pixels of an image former form an image. Each pixel has a first color element to propagate a first amount of a first light having a first wavelength, based on a first electrical stimulus, and a second color element to propagate a second amount of a second light having a second wavelength, based on a second electrical stimulus. A display surface presents the image across a plurality of display surface locations. A processor executes instructions to scale the first electrical stimulus of each pixel of the image former by a first light scale factor, scale the second electrical stimulus of each pixel of the image former by a second light scale factor, and apply a pixel shading map to the image former to independently adjust, for each pixel of the display, the first amount relative to the second amount.
G09G 3/20 - Dispositions ou circuits de commande présentant un intérêt uniquement pour l'affichage utilisant des moyens de visualisation autres que les tubes à rayons cathodiques pour la présentation d'un ensemble de plusieurs caractères, p. ex. d'une page, en composant l'ensemble par combinaison d'éléments individuels disposés en matrice
A mobile vision transformer network for use on mobile devices, such as smart eyewear devices and other augmented reality (AR) and virtual reality (VR) devices. The mobile vision transformer network considers factors including number of parameters, latency, and model performance, as they reflect disk storage, mobile frames per second (FPS), and application quality, respectively. The mobile vision transformer network processes images, e.g., for image classification, segmentation, and detection. The mobile vision transformer network has a fine-grained architecture including a search algorithm performing latency-driven slimming that jointly improves model size and speed.
G06V 10/94 - Architectures logicielles ou matérielles spécialement adaptées à la compréhension d’images ou de vidéos
G06V 10/77 - Traitement des caractéristiques d’images ou de vidéos dans les espaces de caractéristiquesDispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant l’intégration et la réduction de données, p. ex. analyse en composantes principales [PCA] ou analyse en composantes indépendantes [ ICA] ou cartes auto-organisatrices [SOM]Séparation aveugle de source
Examples relate to systems and methods for generating an avatar animation. The systems and methods access an audio file comprising speech, spoken by a user, captured by a microphone of a user system, and receive input that selects an avatar associated with the user. The systems and methods process the audio file and the avatar, selected by the received input, by a generative machine learning model to generate an animation of the avatar having lips moving to represent the avatar speaking the speech of the audio file. The systems and methods generate a video comprising a depiction of the generated animation of the avatar speaking the speech of the audio file.
Examples relate to systems and methods for generating an avatar animation. The systems and methods access an audio file comprising speech, spoken by a user, captured by a microphone of a user system, and receive input that selects an avatar associated with the user. The systems and methods process the audio file and the avatar, selected by the received input, by a generative machine learning model to generate an animation of the avatar having lips moving to represent the avatar speaking the speech of the audio file. The systems and methods generate a video comprising a depiction of the generated animation of the avatar speaking the speech of the audio file.
G06T 13/40 - Animation tridimensionnelle [3D] de personnages, p. ex. d’êtres humains, d’animaux ou d’êtres virtuels
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
G10L 25/57 - Techniques d'analyse de la parole ou de la voix qui ne se limitent pas à un seul des groupes spécialement adaptées pour un usage particulier pour comparaison ou différentiation pour le traitement des signaux vidéo
G10L 25/63 - Techniques d'analyse de la parole ou de la voix qui ne se limitent pas à un seul des groupes spécialement adaptées pour un usage particulier pour comparaison ou différentiation pour estimer un état émotionnel
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for performing operations comprising receiving an image that depicts a person, identifying a set of skeletal joints of the person and identifying a pose of the person depicted in the image based on positioning of the set of skeletal joints. The operations also include receiving speech input comprising a request to perform an AR operation and an ambiguous intent, discerning the ambiguous intent of the speech input based on the pose of the person depicted in the image and in response to receiving the speech input, performing the AR operation based on discerning the ambiguous intent of the speech input based on the pose of the person depicted in the image.
Users of a chat system within an interactive platform can suspend the expiration of a plurality of content items. Example methods include generating a chat, the chat comprising an association between a first user account and a second user account, and receiving, from a first user system associated with the first user account, an indication of a plurality of content items and an indication of the chat. The method may further include sending, to a second user account, the plurality of content items and an indication of the chat, and receiving, from the second user account, an indication to save the plurality of content items within the chat. The method may further include setting a saved data field associated with the plurality of content items and the second user account, the saved data field indicating the plurality of content items do not expire within the chat.
Examples in the present disclosure relate to systems and methods for detecting and rejecting a non-user hand in the context of egocentric hand tracking performed by an extended reality (XR) device. While the XR device is worn by a user, the XR device captures at least one image of a hand and processes the at least one image to detect the hand. After detecting the hand, the XR device determines positioning of the hand relative to the XR device or another object in a field of view of the XR device. The XR device detects that the hand is a non-user hand. In response to detecting that the hand is a non-user hand, the XR device excludes the non-user hand from the egocentric hand tracking such that the non-user hand is not tracked for the user.
A case for a portable device like a smartphone includes light sources such as LEDs, which, when illuminated, can be detected and tracked by a head-worn augmented or virtual reality device. The light sources may be located at the corners of the case and may emit infrared light. A relative pose between the smartphone and the head-worn device can be determined based on computer vision techniques performed on images captured by the head-worn device that includes light from the light sources. Relative movement between the smartphone and the head-worn device can be used to provide user input to the head-worn device, as can touch input on the portable device. In some instances, the case is powered inductively from the portable device.
H04M 1/72409 - Interfaces utilisateur spécialement adaptées aux téléphones sans fil ou mobiles avec des moyens de soutien local des applications accroissant la fonctionnalité par interfaçage avec des accessoires externes
H04B 1/3888 - Dispositions pour le transport ou la protection d’émetteurs-récepteurs
H04M 1/72454 - Interfaces utilisateur spécialement adaptées aux téléphones sans fil ou mobiles avec des moyens permettant d’adapter la fonctionnalité du dispositif dans des circonstances spécifiques en tenant compte des contraintes imposées par le contexte ou par l’environnement
Examples in the present disclosure relate generally to techniques for automated organization and presentation of user-selectable features within user interfaces. More particularly, but not by way of limitation, the present disclosure addresses systems and methods for presenting and accessing image modifiers within user interfaces displaying media content items. In some examples, a modifier carousel comprises a set of interface elements, with each interface element within the set of interface elements representing a respective image modifier within the set of image modifiers. Each interface element within the set of interface elements is user-selectable to apply the respective image modifier to the content item.
G06F 3/0482 - Interaction avec des listes d’éléments sélectionnables, p. ex. des menus
G06F 3/04847 - Techniques d’interaction pour la commande des valeurs des paramètres, p. ex. interaction avec des règles ou des cadrans
G06F 3/0488 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] utilisant des caractéristiques spécifiques fournies par le périphérique d’entrée, p. ex. des fonctions commandées par la rotation d’une souris à deux capteurs, ou par la nature du périphérique d’entrée, p. ex. des gestes en fonction de la pression exercée enregistrée par une tablette numérique utilisant un écran tactile ou une tablette numérique, p. ex. entrée de commandes par des tracés gestuels
G06T 11/60 - Édition de figures et de texteCombinaison de figures ou de texte
H04N 21/431 - Génération d'interfaces visuellesRendu de contenu ou données additionnelles
H04N 21/462 - Gestion de contenu ou de données additionnelles, p. ex. création d'un guide de programmes électronique maître à partir de données reçues par Internet et d'une tête de réseau ou contrôle de la complexité d'un flux vidéo en dimensionnant la résolution ou le débit en fonction des capacités du client
97.
EYEWEAR CHARGING CASE USING CHARGE PINS AS COMMUNICATION PATH
A case for an eyewear device having a conductive interface includes a housing that receives the eyewear device. A multi-purpose interface, supported by the housing, includes at least one contact arranged to couple with the conductive interface of the eyewear device when the housing receives the eyewear device. Circuitry is coupled to the at least one contact and includes a processor that detects a connection of the conductive interface of the eyewear device to the multi-purpose interface of the case. The processor performs a charging process during a charge state of the case in which an electrical charge is provided at the multi-purpose interface of the case to the eyewear device. Data is exchanged with the eyewear device during a communication state of the case.
G02F 1/163 - Fonctionnement des cellules électro-chromiques, p. ex. des cellules d’électrodépositionDispositions des circuits à cet effet
G02F 1/17 - Dispositifs ou dispositions pour la commande de l'intensité, de la couleur, de la phase, de la polarisation ou de la direction de la lumière arrivant d'une source lumineuse indépendante, p. ex. commutation, ouverture de porte ou modulationOptique non linéaire pour la commande de l'intensité, de la phase, de la polarisation ou de la couleur basés sur des éléments à absorption variable non prévus dans les groupes
An augmented reality (AR) calibration process for an eyewear device that customizes an AR user interface (UI) for the wearer. Physiological characteristics such as arm length and hand size are determined from images captured by the eyewear device for use in adjusting the UI controls before rendering the UI.
G06F 3/04815 - Interaction s’effectuant dans un environnement basé sur des métaphores ou des objets avec un affichage tridimensionnel, p. ex. modification du point de vue de l’utilisateur par rapport à l’environnement ou l’objet
G06F 3/04845 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] pour la commande de fonctions ou d’opérations spécifiques, p. ex. sélection ou transformation d’un objet, d’une image ou d’un élément de texte affiché, détermination d’une valeur de paramètre ou sélection d’une plage de valeurs pour la transformation d’images, p. ex. glissement, rotation, agrandissement ou changement de couleur
G06F 3/04847 - Techniques d’interaction pour la commande des valeurs des paramètres, p. ex. interaction avec des règles ou des cadrans
G06T 3/40 - Changement d'échelle d’images complètes ou de parties d’image, p. ex. agrandissement ou rétrécissement
Methods and systems are disclosed for triggering operations using inner speech. The system detects presence of inner speech in electromyograph (EMG) data. The system adjusts a progress indicator in response to detecting the presence of inner speech. The system determines that a current value of the progress indicator has transgressed a specified threshold value and, in response, triggers an operation.
09 - Appareils et instruments scientifiques et électriques
42 - Services scientifiques, technologiques et industriels, recherche et conception
Produits et services
Downloadable augmented reality and virtual reality software for integrating electronic data with real and virtual world environment; Downloadable software and application programming interface (API) software for use in creating and designing augmented reality and virtual reality experiences; Computer hardware, peripherals, and downloadable software for object recognition, sound recognition, and scanning the environment for augmented reality and virtual reality experiences; Downloadable computer software for tracking motion in, visualizing, manipulating, viewing, transmitting, and displaying images, video, audio, and data for augmented reality and virtual reality experiences; Downloadable computer software and firmware for controlling, configuring, and managing hardware devices and for enabling electronic devices to share data and communicate with each other; Downloadable computer software for cameras, video cameras, video recorders, and visual displays; Downloadable computer software for setting up, configuring, and controlling wearable computer hardware and peripherals; Downloadable computer software and software applications for use in uploading, downloading, capturing, editing, storing, accessing, posting, displaying, tagging, distributing, streaming, linking, sharing, transmitting or otherwise providing photos, videos, images, text, electronic media, photographic and video content, digital data, and information via the Internet, communication networks, and mobile devices; Downloadable multimedia files containing digital photos, video, and audio all in the fields of entertainment, photography, communications, utilities, and online social networking Providing temporary use of non-downloadable software for use in creating and designing augmented reality and virtual reality experiences; Providing temporary use of non-downloadable augmented reality and virtual reality software for use in hardware devices for integrating electronic data with real and virtual world environment; Providing temporary use of non-downloadable software for object recognition, sound recognition, and scanning the environment for augmented reality and virtual reality experiences; Providing temporary use of non-downloadable software for tracking motion in, visualizing, manipulating, viewing, transmitting, and displaying images, video, audio, and data for augmented reality and virtual reality experiences; Providing temporary use of non-downloadable software for managing hardware devices, configuring, and managing controllers and for enabling electronic devices to share data and communicate with each other; Providing temporary use of non-downloadable software for cameras, video cameras, video recorders, and visual displays for enabling augmented reality technology; Providing temporary use of non-downloadable software for setting up, configuring, and controlling wearable computer hardware and peripherals; Providing temporary use of non-downloadable software for use in uploading, downloading, capturing, editing, storing, accessing, posting, displaying, tagging, distributing, streaming, linking, sharing, transmitting or otherwise providing photos, videos, images, text, electronic media, photographic and video content, digital data, or information via the Internet, communication networks, and mobile devices; Providing temporary use of non-downloadable software for accessing multimedia files containing digital photos, video, audio, and other digital data all in the fields of entertainment, photography, communications, utilities, and online social networking
