A display system can include a head-mounted display configured to project light to an eye of a user to display augmented reality image content to the user. The display system can include one or more user sensors configured to sense the user and can include one or more environmental sensors configured to sense surroundings of the user. The display system can also include processing electronics in communication with the display, the one or more user sensors, and the one or more environmental sensors. The processing electronics can be configured to sense a situation involving user focus, determine user intent for the situation, and alter user perception of a real or virtual object within the vision field of the user based at least in part on the user intent and/or sensed situation involving user focus. The processing electronics can be configured to at least one of enhance or de-emphasize the user perception of the real or virtual object within the vision field of the user.
A61B 17/00 - Instruments, dispositifs ou procédés chirurgicaux
A61B 34/00 - Chirurgie assistée par ordinateurManipulateurs ou robots spécialement adaptés à l’utilisation en chirurgie
A61B 34/20 - Systèmes de navigation chirurgicaleDispositifs pour le suivi ou le guidage d'instruments chirurgicaux, p. ex. pour la stéréotaxie sans cadre
A61B 90/00 - Instruments, outillage ou accessoires spécialement adaptés à la chirurgie ou au diagnostic non couverts par l'un des groupes , p. ex. pour le traitement de la luxation ou pour la protection de bords de blessures
A61B 90/50 - Supports pour instruments chirurgicaux, p. ex. bras articulés
B60K 35/10 - Dispositions d'entrée, c.-à-d. de l'utilisateur au véhicule, associées aux fonctions du véhicule ou spécialement adaptées à celles-ci
B60K 35/26 - Dispositions de sortie, c.-à-d. du véhicule à l'utilisateur, associées aux fonctions du véhicule ou spécialement adaptées à celles-ci utilisant une sortie acoustique
B60K 35/28 - Dispositions de sortie, c.-à-d. du véhicule à l'utilisateur, associées aux fonctions du véhicule ou spécialement adaptées à celles-ci caractérisées par le type d’informations de sortie, p. ex. divertissement vidéo ou informations sur la dynamique du véhiculeDispositions de sortie, c.-à-d. du véhicule à l'utilisateur, associées aux fonctions du véhicule ou spécialement adaptées à celles-ci caractérisées par la finalité des informations de sortie, p. ex. pour attirer l'attention du conducteur
B60K 35/60 - Instruments caractérisés par leur emplacement ou leur positionnement relatif dans ou sur les véhicules
B60K 35/80 - Dispositions pour la commande des instruments
B60K 35/90 - Étalonnage d'instruments, p. ex. réglage des paramètres initiaux ou de référenceTest d'instruments, p. ex. détection d'un dysfonctionnement
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06F 3/03 - Dispositions pour convertir sous forme codée la position ou le déplacement d'un élément
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
2.
METHODS AND SYSTEMS FOR DISPLAYING STEREOSCOPY WITH A FREEFORM OPTICAL SYSTEM WITH ADDRESSABLE FOCUS FOR VIRTUAL AND AUGMENTED REALITY
Several unique hardware configurations and methods for freeform optical display systems are disclosed. A freeform display system includes primary freeform optical element(s) and secondary freeform optical element(s) in tiled arrangements to expand the horizontal field of view (FOV) or the vertical field of view. The system may include a variable focusing system that produces intermediate pupil and changes the focal distance of a single focal plane or switches among multiple focal planes for rendering objects in focus while resolving accommodation-convergence conflict. The system may map light samples to appropriate light rays in physical space and use a cluster of projectors to project the mapped light rays to produce the light field of the virtual display content. Methods for making tiled freeform optical display systems and methods for producing virtual content with variable focus freeform optics and rendering light fields are also disclosed.
G02B 3/12 - Lentilles remplies d'un fluide ou à l'intérieur desquelles le vide a été fait
G02B 3/14 - Lentilles remplies d'un fluide ou à l'intérieur desquelles le vide a été fait à distance focale variable
G02B 26/08 - Dispositifs ou dispositions optiques pour la commande de la lumière utilisant des éléments optiques mobiles ou déformables pour commander la direction de la lumière
G02B 30/10 - Systèmes ou appareils optiques pour produire des effets tridimensionnels [3D], p. ex. des effets stéréoscopiques en utilisant des méthodes d'imagerie intégrale
A thin transparent layer can be integrated in a head mounted display device and disposed in front of the eye of a wearer. The thin transparent layer may be configured to output light such that light is directed onto the eye to create reflections therefrom that can be used, for example, for glint based tracking. The thin transparent layer can be configured to reduced obstructions in the field of the view of the user.
A computer implemented method of facilitating communication between first and second users includes displaying, by a first head-worn device, a first virtual object to the first user first user wearing the first head-worn device. The method also includes displaying, by a second head-worn device, a second virtual object to the second user wearing the second head-worn device. The method further includes facilitating, by the first and second head-worn devices, communications between the first and second users using the first and second virtual objects to simulate the first and second users being present in a common environment.
A host device having a first processor executes an application via the first processor. The host device determines a state of the application. A scenegraph is generated corresponding to the state of the application, and the scenegraph is presented to a remote device having a display and a second processor. The remote device is configured to, in response to receiving the scenegraph, render to the display a view corresponding to the scenegraph, without executing the application via the second processor.
Recesses are formed on a front side and a rear side of a waveguide. A solid porogen material is spun onto the front side and the rear side and fills the recesses. First front and rear cap layers are then formed on raised formations of the waveguide and on the solid porogen material. The entire structure is then heated and the solid porogen material decomposes to a porogen gas. The first front and rear cap layers are porous to allow the porogen gas to escape and air to enter into the recesses. The air maximizes a difference in refractive indices between the high-index transparent material of the waveguide and the air to promote reflection in the waveguide from interfaces between the waveguide and the air.
F21V 8/00 - Utilisation de guides de lumière, p. ex. dispositifs à fibres optiques, dans les dispositifs ou systèmes d'éclairage
G02B 6/10 - Guides de lumièreDétails de structure de dispositions comprenant des guides de lumière et d'autres éléments optiques, p. ex. des moyens de couplage du type guide d'ondes optiques
G02B 6/122 - Éléments optiques de base, p. ex. voies de guidage de la lumière
G02B 6/13 - Circuits optiques intégrés caractérisés par le procédé de fabrication
G02B 27/00 - Systèmes ou appareils optiques non prévus dans aucun des groupes ,
A method includes patterning a plurality of first trenches in a surface of a substrate; and etching the plurality of first trenches with an etchant having an etch rate for a first crystalline plane of the substrate that is greater than for a second crystalline plane of the substrate. The etching forms a slanted grating in the substrate.
Examples of systems and methods to facilitate audiovisual presence transitions of virtual objects such as virtual avatars in a mixed reality collaborative environment are disclosed. The systems and methods may be configured to produce different audiovisual presence transitions such as appearance, disappearance and reappearance of the virtual avatars. The virtual avatar audiovisual transitions may be further indicated by various visual and sound effects of the virtual avatars. The transitions may occur based on various colocation or decolocation scenarios.
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for calibrating an augmented reality device using camera and inertial measurement unit data. In some implementations, a bundle adjustment process jointly optimizes or estimates states of the augmented reality device. The process can use, as input, visual and inertial measurements as well as factory-calibrated sensor extrinsic parameters. The process performs bundle adjustment and uses non-linear optimization of estimated states constrained by the measurements and the factory calibrated extrinsic parameters. The process can jointly optimize inertial constraints, IMU calibration, and camera calibrations. Output of the process can include most likely estimated states, such as data for a 3D map of an environment, a trajectory of the device, and/or updated extrinsic parameters of the visual and inertial sensors (e.g., cameras and IMUs).
An augmented reality device (ARD) can present virtual content which can provide enhanced experiences with the user's physical environment. For example, the ARD can detect a linkage between a person in the FOV of the ARD and a physical object (e.g., a document presented by the person) or detect linkages between the documents. The linkages may be used in identity verification or document verification.
Wearable systems and method for operation thereof incorporating headset and controller localization using headset cameras and controller fiducials are disclosed. A wearable system may include a headset and a controller. The wearable system may alternate between performing headset tracking and performing controller tracking by repeatedly capturing images using a headset camera of the headset during headset tracking frames and controller tracking frames. The wearable system may cause the headset camera to capture a first exposure image an exposure above a threshold and cause the headset camera to capture a second exposure image having an exposure below the threshold. The wearable system may determine a fiducial interval during which fiducials of the controller are to flash at a fiducial frequency and a fiducial period. The wearable system may cause the fiducials to flash during the fiducial interval in accordance with the fiducial frequency and the fiducial period.
Techniques for artifact mitigation in an optical system are disclosed. Light associated with a world object is received at the optical system, which is characterized by a world side and a user side. Light associated with a virtual image is projected onto an eyepiece of the optical system, causing a portion of the light associated with the virtual image to propagate toward the user side and light associated with an artifact image to propagate toward the world side. A dimmer of the optical system positioned between the world side and the eyepiece is adjusted to reduce an intensity of the light associated with the artifact image impinging on the dimmer and an intensity of the light associated with the world object impinging on the dimmer.
G02B 27/00 - Systèmes ou appareils optiques non prévus dans aucun des groupes ,
G02B 26/08 - Dispositifs ou dispositions optiques pour la commande de la lumière utilisant des éléments optiques mobiles ou déformables pour commander la direction de la lumière
Disclosed herein is a wearable display system for capturing retraining eye images of an eye of a user for retraining a neural network for eye tracking. The system captures retraining eye images using an image capture device when user interface (UI) events occur with respect to UI devices displayed at display locations of a display. The system can generate a retraining set comprising the retraining eye images and eye poses of the eye of the user in the retraining eye images (e.g., related to the display locations of the UI devices) and obtain a retrained neural network that is retrained using the retraining set.
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06F 3/0346 - Dispositifs de pointage déplacés ou positionnés par l'utilisateurLeurs accessoires avec détection de l’orientation ou du mouvement libre du dispositif dans un espace en trois dimensions [3D], p. ex. souris 3D, dispositifs de pointage à six degrés de liberté [6-DOF] utilisant des capteurs gyroscopiques, accéléromètres ou d’inclinaison
G06F 3/04815 - Interaction s’effectuant dans un environnement basé sur des métaphores ou des objets avec un affichage tridimensionnel, p. ex. modification du point de vue de l’utilisateur par rapport à l’environnement ou l’objet
Techniques related to the computation of gaze vectors of users of wearable devices are disclosed. A neural network may be trained through first and second training steps. The neural network may include a set of feature encoding layers and a plurality of sets of task-specific layers that each operate on an output of the set of feature encoding layers. During the first training step, a first image of a first eye may be provided to the neural network, eye segmentation data may be generated using the neural network, and the set of feature encoding layers may be trained. During the second training step, a second image of a second eye may be provided to the neural network, network output data may be generated using the neural network, and the plurality of sets of task-specific layers may be trained.
G06V 10/26 - Segmentation de formes dans le champ d’imageDécoupage ou fusion d’éléments d’image visant à établir la région de motif, p. ex. techniques de regroupementDétection d’occlusion
G06V 10/774 - Génération d'ensembles de motifs de formationTraitement des caractéristiques d’images ou de vidéos dans les espaces de caractéristiquesDispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant l’intégration et la réduction de données, p. ex. analyse en composantes principales [PCA] ou analyse en composantes indépendantes [ ICA] ou cartes auto-organisatrices [SOM]Séparation aveugle de source méthodes de Bootstrap, p. ex. "bagging” ou “boosting”
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
G06V 40/18 - Caractéristiques de l’œil, p. ex. de l’iris
A head mounted display system for displaying image content to a user comprises at least one display configured to be worn by a user to present virtual content to first and second eyes of a user, one or more inwardly facing sensors or camera configured to monitor one or both of the users eye and processing electronics. This head mounted display system is configured such that virtual content activity can be initiated and/or driven from eye inputs such as gaze direction, eyelid motions (e.g., blinking), and/or other eye gestures.
An optical device such as an augmented reality (AR) display device includes variable optical material that alters at least one of: incident ambient light, spectral content of incident ambient light or direction of incident ambient light through the optical device in response to a stimulus provided by the device. The device can sense intensity and/or spectral characteristics of ambient light and provide appropriate stimulus to various portions of the optical device to activate the variable optical material and alter at least one of: incident ambient light, spectral content of incident ambient light or direction of incident ambient light. In some examples, the variable optical material may be distributed unevenly across the optical device.
F21V 8/00 - Utilisation de guides de lumière, p. ex. dispositifs à fibres optiques, dans les dispositifs ou systèmes d'éclairage
G02B 3/14 - Lentilles remplies d'un fluide ou à l'intérieur desquelles le vide a été fait à distance focale variable
G02B 27/00 - Systèmes ou appareils optiques non prévus dans aucun des groupes ,
G02F 1/00 - Dispositifs ou dispositions pour la commande de l'intensité, de la couleur, de la phase, de la polarisation ou de la direction de la lumière arrivant d'une source lumineuse indépendante, p. ex. commutation, ouverture de porte ou modulationOptique non linéaire
G02F 1/01 - Dispositifs ou dispositions pour la commande de l'intensité, de la couleur, de la phase, de la polarisation ou de la direction de la lumière arrivant d'une source lumineuse indépendante, p. ex. commutation, ouverture de porte ou modulationOptique non linéaire pour la commande de l'intensité, de la phase, de la polarisation ou de la couleur
G02F 1/29 - Dispositifs ou dispositions pour la commande de l'intensité, de la couleur, de la phase, de la polarisation ou de la direction de la lumière arrivant d'une source lumineuse indépendante, p. ex. commutation, ouverture de porte ou modulationOptique non linéaire pour la commande de la position ou de la direction des rayons lumineux, c.-à-d. déflexion
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
Systems and methods for generating a face model for a user of a head-mounted device are disclosed. The head-mounted device can include one or more eye cameras configured to image the face of the user while the user is putting the device on or taking the device off. The images obtained by the eye cameras may be analyzed using a stereoscopic vision technique, a monocular vision technique, or a combination, to generate a face model for the user. The face model can be used to generate a virtual image of at least a portion of the user's face, for example to be presented as an avatar.
In some embodiments, a display system comprising a head-mountable, augmented reality display is configured to perform a neurological analysis and to provide a perception aid based on an environmental trigger associated with the neurological condition. Performing the neurological analysis may include determining a reaction to a stimulus by receiving data from the one or more inwardly-directed sensors; and identifying a neurological condition associated with the reaction. In some embodiments, the perception aid may include a reminder, an alert, or virtual content that changes a property, e.g. a color, of a real object. The augmented reality display may be configured to display virtual content by outputting light with variable wavefront divergence, and to provide an accommodation-vergence mismatch of less than 0.5 diopters, including less than 0.25 diopters.
A display system may include a head-mounted display (HMD) for rendering a three-dimensional virtual object which appears to be located in an ambient environment of a user of the display. One or more eyes of the user may not be in desired positions, relative to the HMD, to receive, or register, image information outputted by the HMD and/or to view an external environment. For example, the HMD-to-eye alignment may vary for different users and/or may change over time (e.g., as the HMD is displaced). The display system may determine a relative position or alignment between the HMD and the user's eyes. Based on the relative positions, the wearable device may determine if it is properly fitted to the user, may provide feedback on the quality of the fit to the user, and/or may take actions to reduce or minimize effects of any misalignment.
G02B 27/00 - Systèmes ou appareils optiques non prévus dans aucun des groupes ,
A61B 3/11 - Appareils pour l'examen optique des yeuxAppareils pour l'examen clinique des yeux du type à mesure objective, c.-à-d. instruments pour l'examen des yeux indépendamment des perceptions ou des réactions du patient pour mesurer la distance interpupillaire ou le diamètre de la pupille
A61B 3/113 - Appareils pour l'examen optique des yeuxAppareils pour l'examen clinique des yeux du type à mesure objective, c.-à-d. instruments pour l'examen des yeux indépendamment des perceptions ou des réactions du patient pour déterminer ou enregistrer le mouvement de l'œil
G02B 30/00 - Systèmes ou appareils optiques pour produire des effets tridimensionnels [3D], p. ex. des effets stéréoscopiques
G02B 30/40 - Systèmes ou appareils optiques pour produire des effets tridimensionnels [3D], p. ex. des effets stéréoscopiques donnant à l’observateur d'une seule image bidimensionnelle [2D] une impression perceptive de profondeur
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06F 3/0346 - Dispositifs de pointage déplacés ou positionnés par l'utilisateurLeurs accessoires avec détection de l’orientation ou du mouvement libre du dispositif dans un espace en trois dimensions [3D], p. ex. souris 3D, dispositifs de pointage à six degrés de liberté [6-DOF] utilisant des capteurs gyroscopiques, accéléromètres ou d’inclinaison
G06F 3/04815 - Interaction s’effectuant dans un environnement basé sur des métaphores ou des objets avec un affichage tridimensionnel, p. ex. modification du point de vue de l’utilisateur par rapport à l’environnement ou l’objet
G06T 3/40 - Changement d'échelle d’images complètes ou de parties d’image, p. ex. agrandissement ou rétrécissement
G06V 10/42 - Extraction de caractéristiques globales par l’analyse du motif entier, p. ex. utilisant des transformations dans le domaine de fréquence ou d’autocorrélation
G06V 10/46 - Descripteurs pour la forme, descripteurs liés au contour ou aux points, p. ex. transformation de caractéristiques visuelles invariante à l’échelle [SIFT] ou sacs de mots [BoW]Caractéristiques régionales saillantes
G06V 10/60 - Extraction de caractéristiques d’images ou de vidéos relative aux propriétés luminescentes, p. ex. utilisant un modèle de réflectance ou d’éclairage
G06V 40/18 - Caractéristiques de l’œil, p. ex. de l’iris
H04N 13/344 - Affichage pour le visionnement à l’aide de lunettes spéciales ou de visiocasques avec des visiocasques portant des affichages gauche et droit
H04N 13/383 - Suivi des spectateurs pour le suivi du regard, c.-à-d. avec détection de l’axe de vision des yeux du spectateur
A wearable device can include an inward-facing imaging system configured to acquire images of a user's periocular region. The wearable device can determine a relative position between the wearable device and the user's face based on the images acquired by the inward-facing imaging system. The relative position may be used to determine whether the user is wearing the wearable device, whether the wearable device is optimally fit to the user, and/or whether an adjustment to a rendering location of a virtual object can be made to compensate for a deviation of the wearable device from its normal resting position relative to the user's face.
G06T 3/20 - Translation linéaire d’images complètes ou de parties d’image, p. ex. panoramique
G06T 11/60 - Édition de figures et de texteCombinaison de figures ou de texte
G06V 10/46 - Descripteurs pour la forme, descripteurs liés au contour ou aux points, p. ex. transformation de caractéristiques visuelles invariante à l’échelle [SIFT] ou sacs de mots [BoW]Caractéristiques régionales saillantes
G06V 20/20 - ScènesÉléments spécifiques à la scène dans les scènes de réalité augmentée
G06V 40/16 - Visages humains, p. ex. parties du visage, croquis ou expressions
G06V 40/18 - Caractéristiques de l’œil, p. ex. de l’iris
A wearable display system includes one or more nanowire LED micro-displays. The nanowire micro-LED displays may be monochrome or full-color. The nanowire LEDs forming the arrays may have an advantageously narrow angular emission profile and high light output. Where a plurality of nanowire LED micro-displays is utilized, the micro-displays may be positioned at different sides of an optical combiner, for example, an X-cube prism which receives light rays from different micro-displays and outputs the light rays from the same face of the cube. The optical combiner directs the light to projection optics, which outputs the light to an eyepiece that relays the light to a user's eye. The eyepiece may output the light to the user's eye with different amounts of wavefront divergence, to place virtual content on different depth planes.
Disclosed herein are systems and methods for distributed computing and/or networking for mixed reality systems. A method may include capturing an image via a camera of a head-wearable device. Inertial data may be captured via an inertial measurement unit of the head-wearable device. A position of the head-wearable device can be estimated based on the image and the inertial data via one or more processors of the head-wearable device. The image can be transmitted to a remote server. A neural network can be trained based on the image via the remote server. A trained neural network can be transmitted to the head-wearable device.
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06F 3/14 - Sortie numérique vers un dispositif de visualisation
G06F 18/214 - Génération de motifs d'entraînementProcédés de Bootstrapping, p. ex. ”bagging” ou ”boosting”
G06V 10/764 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant la classification, p. ex. des objets vidéo
G06V 20/20 - ScènesÉléments spécifiques à la scène dans les scènes de réalité augmentée
G06V 40/16 - Visages humains, p. ex. parties du visage, croquis ou expressions
G06V 40/18 - Caractéristiques de l’œil, p. ex. de l’iris
Antireflection coatings for metasurfaces are described herein. In some embodiments, the metasurface may include a substrate, a plurality of nanostructures thereon, and an antireflection coating disposed over the nanostructures. The antireflection coating may be a transparent polymer, for example a photoresist layer, and may have a refractive index lower than the refractive index of the nanostructures and higher than the refractive index of the overlying medium (e.g., air). Advantageously, the antireflection coatings may reduce or eliminate ghost images in an augmented reality display in which the metasurface is incorporated.
Systems and methods for eye image segmentation and image quality estimation are disclosed. In one aspect, after receiving an eye image, a device such as an augmented reality device can process the eye image using a convolutional neural network with a merged architecture to generate both a segmented eye image and a quality estimation of the eye image. The segmented eye image can include a background region, a sclera region, an iris region, or a pupil region. In another aspect, a convolutional neural network with a merged architecture can be trained for eye image segmentation and image quality estimation. In yet another aspect, the device can use the segmented eye image to determine eye contours such as a pupil contour and an iris contour. The device can use the eye contours to create a polar image of the iris region for computing an iris code or biometric authentication.
G06F 18/2413 - Techniques de classification relatives au modèle de classification, p. ex. approches paramétriques ou non paramétriques basées sur les distances des motifs d'entraînement ou de référence
G06T 7/194 - DécoupageDétection de bords impliquant une segmentation premier plan-arrière-plan
G06V 10/44 - Extraction de caractéristiques locales par analyse des parties du motif, p. ex. par détection d’arêtes, de contours, de boucles, d’angles, de barres ou d’intersectionsAnalyse de connectivité, p. ex. de composantes connectées
G06V 10/56 - Extraction de caractéristiques d’images ou de vidéos relative à la couleur
G06V 10/764 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant la classification, p. ex. des objets vidéo
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
G06V 10/98 - Détection ou correction d’erreurs, p. ex. en effectuant une deuxième exploration du motif ou par intervention humaineÉvaluation de la qualité des motifs acquis
G06V 40/18 - Caractéristiques de l’œil, p. ex. de l’iris
25.
LIGHT OUTPUT SYSTEM WITH REFLECTOR AND LENS FOR HIGHLY SPATIALLY UNIFORM LIGHT OUTPUT
A user may interact and view virtual elements such as avatars and objects and/or real world elements in three-dimensional space in an augmented reality (AR) session. The system may allow one or more spectators to view from a stationary or dynamic camera a third person view of the users AR session. The third person view may be synchronized with the user view and the virtual elements of the user view may be composited onto the third person view.
F21V 13/04 - Combinaisons de deux sortes d'éléments uniquement les éléments étant des réflecteurs et des réfracteurs
F21V 8/00 - Utilisation de guides de lumière, p. ex. dispositifs à fibres optiques, dans les dispositifs ou systèmes d'éclairage
G02B 6/12 - Guides de lumièreDétails de structure de dispositions comprenant des guides de lumière et d'autres éléments optiques, p. ex. des moyens de couplage du type guide d'ondes optiques du genre à circuit intégré
G02B 23/06 - Télescopes ou lunettes d'approche, p. ex. jumellesPériscopesInstruments pour voir à l'intérieur de corps creuxViseursPointage optique ou appareils de visée comprenant des prismes ou des miroirs ayant une action de mise au point, p. ex. miroir parabolique
G02B 30/50 - Systèmes ou appareils optiques pour produire des effets tridimensionnels [3D], p. ex. des effets stéréoscopiques l’image étant construite à partir d'éléments d'image répartis sur un volume 3D, p. ex. des voxels
H04N 13/315 - Reproducteurs d’images pour visionnement sans avoir recours à des lunettes spéciales, c.-à-d. utilisant des affichages autostéréoscopiques utilisant des barrières de parallaxe les barrières de parallaxe variant dans le temps
H04N 13/344 - Affichage pour le visionnement à l’aide de lunettes spéciales ou de visiocasques avec des visiocasques portant des affichages gauche et droit
26.
SELECTING VIRTUAL OBJECTS IN A THREE-DIMENSIONAL SPACE
Systems and methods for interacting with virtual objects in a three-dimensional space using a wearable system are disclosed. The wearable system can be programmed to permit user interaction with interactable objects in a field of regard (FOR) of a user. The FOR includes a portion of the environment around the user that is capable of being perceived by the user via the AR system. The system can determine a group of interactable objects in the FOR of the user and determine a pose of the user. The system can update, based on a change in the pose or a field of view (FOV) of the user, a subgroup of the interactable objects that are located in the FOV of the user and receive a selection of a target interactable object from the subgroup of interactable objects. The system can initiate a selection event on the target interactable object.
G06F 3/0346 - Dispositifs de pointage déplacés ou positionnés par l'utilisateurLeurs accessoires avec détection de l’orientation ou du mouvement libre du dispositif dans un espace en trois dimensions [3D], p. ex. souris 3D, dispositifs de pointage à six degrés de liberté [6-DOF] utilisant des capteurs gyroscopiques, accéléromètres ou d’inclinaison
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06F 3/04815 - Interaction s’effectuant dans un environnement basé sur des métaphores ou des objets avec un affichage tridimensionnel, p. ex. modification du point de vue de l’utilisateur par rapport à l’environnement ou l’objet
G06F 3/0482 - Interaction avec des listes d’éléments sélectionnables, p. ex. des menus
G06F 3/04883 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] utilisant des caractéristiques spécifiques fournies par le périphérique d’entrée, p. ex. des fonctions commandées par la rotation d’une souris à deux capteurs, ou par la nature du périphérique d’entrée, p. ex. des gestes en fonction de la pression exercée enregistrée par une tablette numérique utilisant un écran tactile ou une tablette numérique, p. ex. entrée de commandes par des tracés gestuels pour l’entrée de données par calligraphie, p. ex. sous forme de gestes ou de texte
27.
ECLIPSE CURSOR FOR VIRTUAL CONTENT IN MIXED REALITY DISPLAYS
Systems and methods for displaying a cursor and a focus indicator associated with real or virtual objects in a virtual, augmented, or mixed reality environment by a wearable display device are disclosed. The system can determine a spatial relationship between a user-movable cursor and a target object within the environment. The system may render a focus indicator (e.g., a halo, shading, or highlighting) around or adjacent objects that are near the cursor. When the cursor overlaps with a target object, the system can render the object in front of the cursor (or not render the cursor at all), so the object is not occluded by the cursor. The object can be rendered closer to the user than the cursor. A group of virtual objects can be scrolled, and a virtual control panel can be displayed indicating objects that are upcoming in the scroll.
G06F 3/04817 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] fondées sur des propriétés spécifiques de l’objet d’interaction affiché ou sur un environnement basé sur les métaphores, p. ex. interaction avec des éléments du bureau telles les fenêtres ou les icônes, ou avec l’aide d’un curseur changeant de comportement ou d’aspect utilisant des icônes
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06F 3/04812 - Techniques d’interaction fondées sur l’aspect ou le comportement du curseur, p. ex. sous l’influence de la présence des objets affichés
G06F 3/04815 - Interaction s’effectuant dans un environnement basé sur des métaphores ou des objets avec un affichage tridimensionnel, p. ex. modification du point de vue de l’utilisateur par rapport à l’environnement ou l’objet
G06F 3/0485 - Défilement ou défilement panoramique
A method includes determining an eye gaze location of a user and generating a spatial foveation map based on the eye gaze location. The method also includes receiving an image, forming a spatially foveated image using the image and the spatial foveation map, and transmitting the spatially foveated image to a wearable device. The method further includes spatially defoveating the spatially foveated image to produce a spatially defoveated image and displaying the spatially defoveated image.
An augmented reality (AR) system includes a wearable device including: a frame, a projector coupled to the frame, a display optically coupled to the projector, and an eye tracking system. The AR system also includes a memory and a processor configured to: receive an eye gaze location from the eye tracking system, generate an image, and generate a foveation map based on the eye gaze location. The foveation map includes a first region of the image and a second region of the image. The processor is also configured to compress the first region of the image using a first quality setting and the second region of the image using a second quality setting. The first quality setting (e.g., a setting of 100%) can be greater than the second quality setting.
H04N 13/117 - Transformation de signaux d’images correspondant à des points de vue virtuels, p. ex. interpolation spatiale de l’image les positions des points de vue virtuels étant choisies par les spectateurs ou déterminées par suivi du spectateur
H04N 13/139 - Conversion du format, p. ex. du débit de trames ou de la taille
H04N 13/161 - Encodage, multiplexage ou démultiplexage de différentes composantes des signaux d’images
H04N 13/344 - Affichage pour le visionnement à l’aide de lunettes spéciales ou de visiocasques avec des visiocasques portant des affichages gauche et droit
H04N 13/383 - Suivi des spectateurs pour le suivi du regard, c.-à-d. avec détection de l’axe de vision des yeux du spectateur
An augmented reality (AR) device can be configured to monitor ambient audio data. The AR device can detect speech in the ambient audio data, convert the detected speech into text, or detect keywords such as rare words in the speech. When a rare word is detected, the AR device can retrieve auxiliary information (e.g., a definition) related to the rare word from a public or private source. The AR device can display the auxiliary information for a user to help the user better understand the speech. The AR device may perform translation of foreign speech, may display text (or the translation) of a speaker's speech to the user, or display statistical or other information associated with the speech.
G06F 40/58 - Utilisation de traduction automatisée, p. ex. pour recherches multilingues, pour fournir aux dispositifs clients une traduction effectuée par le serveur ou pour la traduction en temps réel
G06V 20/20 - ScènesÉléments spécifiques à la scène dans les scènes de réalité augmentée
A mixed reality virtual environment is sharable among multiple users through the use of multiple view modes that are selectable by a presenter. Multiple users with wearable display systems may wish to view a common virtual object, which may be presented in a virtual room to any suitable number of users. A presentation may be controlled by a presenter using a presenter wearable system that leads multiple participants through information associated with the virtual object. Use of different viewing modes allows individual users to see different virtual content through their wearable display systems, despite being in a shared viewing space or alternatively, to see the same virtual content in different locations within a shared space.
G09B 5/12 - Matériel à but éducatif à commande électrique avec présentation individuelle d'une information à une pluralité de postes d'élèves différents postes étant capables de présenter des informations différentes simultanément
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
H04L 12/18 - Dispositions pour la fourniture de services particuliers aux abonnés pour la diffusion ou les conférences
32.
METHOD AND SYSTEM FOR PERFORMING IMAGE REPROJECTION USING HEAD POSE INFORMATION
A method includes receiving, at an encoder, virtual content, receiving, at the encoder, a predicted head pose corresponding to the virtual content, and encoding the virtual content based on the predicted head pose. The method also includes producing compressed content, receiving, at a decoder, the compressed content, and receiving, at the decoder, a current head pose. The method also includes decoding the compressed content based on the current head pose and producing reprojected virtual content. In some embodiments, the method further includes compressing the reprojected virtual content to form one or more foveated images, decompressing the one or more foveated images to form a set of output images, and displaying the set of output images on a display.
H04N 13/383 - Suivi des spectateurs pour le suivi du regard, c.-à-d. avec détection de l’axe de vision des yeux du spectateur
H04N 13/139 - Conversion du format, p. ex. du débit de trames ou de la taille
H04N 13/344 - Affichage pour le visionnement à l’aide de lunettes spéciales ou de visiocasques avec des visiocasques portant des affichages gauche et droit
33.
TUNABLE CYLINDRICAL LENSES AND HEAD-MOUNTED DISPLAY INCLUDING THE SAME
Systems include three optical elements arranged along an optical axis each having a different cylinder axis and a variable cylinder refractive power. Collectively, the three elements form a compound optical element having an overall spherical refractive power (SPH), cylinder refractive power (CYL), and cylinder axis (Axis) that can be varied according to a prescription (Rx).
A head mounted display system can include at least one imaging device, a waveguide, and optical elements formed on or in the waveguide, including at least one coupling optical element configured to in-couple, into the waveguide, light from the environment, and at least one out-coupling optical element configured to out-couple, from the waveguide and toward the imaging device(s), the light from the environment, such that the imaging device(s) can image the environment based on the in-coupled light. The waveguide may also include an in-coupling optical element configured to couple, into the waveguide, image light that conveys virtual image content, and another out-coupling optical element that may be separate from the at least one coupling optical element and that is configured to couple the image light out of the waveguide toward the user's eye.
A method of producing a reprojected image includes receiving motion data and determining, based on the motion data, if a motion threshold is exceeded. The method also includes generating a depth-based reprojection if the motion threshold is exceeded or generating a non-depth-based reprojection if the motion threshold is not exceeded. In some embodiments, performing the foveated compression of the depth-based reprojection includes determining an eye gaze location of a user and generating a foveation map based on the eye gaze location. The foveation map includes a first region of the depth-based reprojection and a second region of the depth-based reprojection. Performing the foveated compression of the depth-based reprojection also includes compressing the first region using a first quality setting and the second region using a second quality setting.
Examples of eye-imaging apparatus using diffractive optical elements are provided. For example, an optical device comprises a substrate having a proximal surface and a distal surface, a first coupling optical element disposed on one of the proximal and distal surfaces of the substrate, and a second coupling optical element disposed on one of the proximal and distal surfaces of the substrate and offset from the first coupling optical element. The first coupling optical element can be configured to deflect light at an angle to totally internally reflect (TIR) the light between the proximal and distal surfaces and toward the second coupling optical element, and the second coupling optical element can be configured to deflect at an angle out of the substrate. The eye-imaging apparatus can be used in a head-mounted display such as an augmented or virtual reality display.
F21V 8/00 - Utilisation de guides de lumière, p. ex. dispositifs à fibres optiques, dans les dispositifs ou systèmes d'éclairage
G02B 27/00 - Systèmes ou appareils optiques non prévus dans aucun des groupes ,
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
H04N 13/332 - Affichage pour le visionnement à l’aide de lunettes spéciales ou de visiocasques
H04N 13/344 - Affichage pour le visionnement à l’aide de lunettes spéciales ou de visiocasques avec des visiocasques portant des affichages gauche et droit
H04N 13/383 - Suivi des spectateurs pour le suivi du regard, c.-à-d. avec détection de l’axe de vision des yeux du spectateur
37.
CONDITIONAL, PROBABILISTIC GENERATION OF 3D VIRTUAL ENVIRONMENT BASED ON INCOMPLETE INFORMATION AVAILABLE TO AN EXTENDED REALITY DEVICE
An extended reality system that includes a wearable eyepiece presenting virtual contents to a user, a belt pack operatively coupled to the wearable eyepiece, a processor, and a non-transitory computer readable medium storing thereupon a sequence of instructions which, when executed by a model with the processor, causes the processor to estimate a plurality of keyframes, a plurality of keyframe poses, and depth data from a plurality of captures that is captured by at least the extended reality device, to generate a semantically annotated, manipulatable three-dimensional (3D) representation in a physical environment for perception by the user wearing the wearable eyepiece, and to modify the semantically annotated, manipulatable 3D representation in real-time or nearly real-time in response to a user interaction.
G06T 7/55 - Récupération de la profondeur ou de la forme à partir de plusieurs images
G06T 17/00 - Modélisation tridimensionnelle [3D] pour infographie
G06V 10/762 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant le regroupement, p. ex. de visages similaires sur les réseaux sociaux
A head-mounted display system includes: a head mounted display frame; a first eyepiece supported by the frame, the first eyepiece including a first substrate composed of a crystalline, transparent material having crystallographic axes in a first orientation with respect to the frame, the substrate having a first surface and a second surface opposite the first surface, the first eyepiece further including a first in-coupling element including a grating on the first surface, and a first out-coupling element including a grating on the first surface and/or a grating on the second surface; and a second eyepiece including a second substrate composed of the crystalline, transparent material having crystallographic axes in a second orientation with respect to the frame different from the first orientation, a second in-coupling element on either surface of the second substrate, and a second out-coupling element on either surface of the second substrate.
G02B 1/02 - Éléments optiques caractérisés par la substance dont ils sont faitsRevêtements optiques pour éléments optiques faits de cristaux, p. ex. sel gemme, semi-conducteurs
The invention relates to a viewing system for use in a surgical environment. Various real object detection devices detect locations of real objects in a real environment, such as a patient and body part of patient, medical staff, robots, a cutting tool on a robot, implant transferred by robot into body part, surgical tools, and disposable items. A map generator generates a map that forms a digital representation or a digital twin of the real environment. Various guiding modules including a room setup module, an anatomy registration module, a surgical planning module, and a surgical execution module make use of the digital representation to guide virtual or real objects based on the digital representation.
A61B 34/20 - Systèmes de navigation chirurgicaleDispositifs pour le suivi ou le guidage d'instruments chirurgicaux, p. ex. pour la stéréotaxie sans cadre
A61B 90/00 - Instruments, outillage ou accessoires spécialement adaptés à la chirurgie ou au diagnostic non couverts par l'un des groupes , p. ex. pour le traitement de la luxation ou pour la protection de bords de blessures
A61B 90/50 - Supports pour instruments chirurgicaux, p. ex. bras articulés
A virtual reality (VR) and/or augmented reality (AR) display system is configured to control a display using control information that is embedded in or otherwise included with imagery data to be presented through the display. The control information can indicate depth plane(s) and/or color plane(s) to be used to present the imagery data, depth plane(s) and/or color plane(s) to be activated or inactivated, shift(s) of at least a portion of the imagery data (e.g., one or more pixels) laterally within a depth plane and/or longitudinally between depth planes, other adjustment(s) to the virtual imagery, and/or other suitable controls.
G06F 3/14 - Sortie numérique vers un dispositif de visualisation
G06T 7/579 - Récupération de la profondeur ou de la forme à partir de plusieurs images à partir du mouvement
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
G09G 3/00 - Dispositions ou circuits de commande présentant un intérêt uniquement pour l'affichage utilisant des moyens de visualisation autres que les tubes à rayons cathodiques
G09G 3/20 - Dispositions ou circuits de commande présentant un intérêt uniquement pour l'affichage utilisant des moyens de visualisation autres que les tubes à rayons cathodiques pour la présentation d'un ensemble de plusieurs caractères, p. ex. d'une page, en composant l'ensemble par combinaison d'éléments individuels disposés en matrice
G09G 5/00 - Dispositions ou circuits de commande de l'affichage communs à l'affichage utilisant des tubes à rayons cathodiques et à l'affichage utilisant d'autres moyens de visualisation
H04N 13/344 - Affichage pour le visionnement à l’aide de lunettes spéciales ou de visiocasques avec des visiocasques portant des affichages gauche et droit
H04N 13/395 - Affichages volumétriques, c.-à-d. systèmes où l’image est réalisée à partir d’éléments répartis dans un volume avec échantillonnage de la profondeur, c.-à-d. construction du volume à partir d’un ensemble ou d’une séquence de plans d’image 2D
Very high refractive index (n>2.2) lightguide substrates enable the production of 70° field of view eyepieces with all three color primaries in a single eyepiece layer. Disclosed herein are viewing optics assembly architectures that make use of such eyepieces to reduce size and cost, simplifying manufacturing and assembly, and better-accommodating novel microdisplay designs.
Exemplary systems and methods for creating spatial contents in a mixed reality environment are disclosed. In an example, a location associated with a first user in a coordinate space is determined. A persistent virtual content is generated. The persistent virtual content is associated with the first user's associated location. The first user's associated location is determined and is associated with the persistent virtual content. A location of a second user at a second time in the coordinate space is determined. The persistent virtual content is presented to the second user via a display at a location in the coordinate space corresponding to the first user's associated location.
A head-mounted display system is configured to project light to an eye of a user wearing the head-mounted display system to display content in a vision field of said user. The head-mounted display system comprises at least one diffusive optical element, at least one out-coupling optical element, at least one mask comprising at least one mask opening, at least one illumination in-coupling optical element configured to in-couple light from at least one illumination source into a light-guiding component, an image projector configured to in-couple an image and an at least one illumination source is configured to in-couple light into at least one illumination in-coupling optical element, an eyepiece, a curved light-guiding component, a light-guiding component comprising a portion of a frame, and/or two light-guiding components disposed on opposite sides of at least one out-coupling optical element.
A voice user interface (VUI) and methods for operating the VUI are disclosed. In some embodiments, the VUI configured to receive and process linguistic and non-linguistic inputs. For example, the VUI receives an audio signal, and the VUI determines whether the audio input comprises a linguistic and/or a non-linguistic input. In accordance with a determination that the audio signal comprises a non-linguistic input, the VUI causes a system to perform an action associated with the non-linguistic input.
G10L 15/18 - Classement ou recherche de la parole utilisant une modélisation du langage naturel
G10L 25/63 - Techniques d'analyse de la parole ou de la voix qui ne se limitent pas à un seul des groupes spécialement adaptées pour un usage particulier pour comparaison ou différentiation pour estimer un état émotionnel
45.
SYSTEMS AND METHOD FOR AUDIO PROCESSING IN EXTENDED REALITY
A method for presenting an extended reality experience to a user includes a display subsystem presenting images corresponding to image data to the user. The method also includes a microphone capturing input sound from an ambient acoustic environment and converting the captured input sound to input audio data. The method further includes an audio controller analyzing the input audio data to determine a masking level, rendering output audio data corresponding to source audio data, analyzing the output audio data to determine an energy level of the output audio data, comparing the energy level with the masking level to determine a comparison outcome, and modifying the output audio data based on the comparison outcome to raise the energy level of the output audio data to generate modified output audio data. The method also includes an audio subsystem presenting output sound corresponding to the modified output audio data to the user.
An optical device may include a light turning element. The optical device can include a first surface that is parallel to a horizontal axis and a second surface opposite to the first surface. The optical device may include a light module that includes a plurality of light emitters, and that can be configured to combine light from the emitters. The optical device can further include a light input surface that is between the first and the second surfaces and is disposed with respect to the light module to receive light. The optical device may include an end reflector that is disposed on a side opposite the light input surface. The light coupled into the light turning element may be reflected by the end reflector and/or reflected from the second surface towards the first surface.
G02B 27/00 - Systèmes ou appareils optiques non prévus dans aucun des groupes ,
G02B 27/14 - Systèmes divisant ou combinant des faisceaux fonctionnant uniquement par réflexion
G02B 30/26 - Systèmes ou appareils optiques pour produire des effets tridimensionnels [3D], p. ex. des effets stéréoscopiques en fournissant des première et seconde images de parallaxe à chacun des yeux gauche et droit d’un observateur du type autostéréoscopique
G02B 30/52 - Systèmes ou appareils optiques pour produire des effets tridimensionnels [3D], p. ex. des effets stéréoscopiques l’image étant construite à partir d'éléments d'image répartis sur un volume 3D, p. ex. des voxels le volume 3D étant construit à partir d'une pile ou d'une séquence de plans 2D, p. ex. systèmes d'échantillonnage en profondeur
G02F 1/137 - Dispositifs ou dispositions pour la commande de l'intensité, de la couleur, de la phase, de la polarisation ou de la direction de la lumière arrivant d'une source lumineuse indépendante, p. ex. commutation, ouverture de porte ou modulationOptique non linéaire pour la commande de l'intensité, de la phase, de la polarisation ou de la couleur basés sur des cristaux liquides, p. ex. cellules d'affichage individuelles à cristaux liquides caractérisés par l'effet électro-optique ou magnéto-optique, p. ex. transition de phase induite par un champ, effet d'orientation, interaction entre milieu récepteur et matière additive ou diffusion dynamique
G03B 21/00 - Projecteurs ou visionneuses du type par projectionLeurs accessoires
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06F 3/0482 - Interaction avec des listes d’éléments sélectionnables, p. ex. des menus
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
G06V 20/20 - ScènesÉléments spécifiques à la scène dans les scènes de réalité augmentée
G09G 3/02 - Dispositions ou circuits de commande présentant un intérêt uniquement pour l'affichage utilisant des moyens de visualisation autres que les tubes à rayons cathodiques par traçage ou balayage d'un faisceau lumineux sur un écran
G09G 3/24 - Dispositions ou circuits de commande présentant un intérêt uniquement pour l'affichage utilisant des moyens de visualisation autres que les tubes à rayons cathodiques pour la présentation d'un ensemble de plusieurs caractères, p. ex. d'une page, en composant l'ensemble par combinaison d'éléments individuels disposés en matrice utilisant des sources lumineuses commandées utilisant des filaments incandescents
H04N 13/239 - Générateurs de signaux d’images utilisant des caméras à images stéréoscopiques utilisant deux capteurs d’images 2D dont la position relative est égale ou en correspondance à l’intervalle oculaire
H04N 13/279 - Générateurs de signaux d’images à partir de modèles 3D d’objets, p. ex. des signaux d’images stéréoscopiques générés par ordinateur les positions des points de vue virtuels étant choisies par les spectateurs ou déterminées par suivi
H04N 13/344 - Affichage pour le visionnement à l’aide de lunettes spéciales ou de visiocasques avec des visiocasques portant des affichages gauche et droit
Head-mounted virtual and augmented reality display systems include a light projector with one or more emissive micro-displays having a first resolution and a pixel pitch. The projector outputs light forming frames of virtual content having at least a portion associated with a second resolution greater than the first resolution. The projector outputs light forming a first subframe of the rendered frame at the first resolution, and parts of the projector are shifted using actuators, such that physical positions of light output for individual pixels occupy gaps between the old locations of light output for individual pixels. The projector then outputs light forming a second subframe of the rendered frame. The first and second subframes are outputted within the flicker fusion threshold. Advantageously, an emissive micro-display (e.g., micro-LED display) having a low resolution can form a frame having a higher resolution by using the same light emitters to function as multiple pixels of that frame.
G02B 27/09 - Mise en forme du faisceau, p. ex. changement de la section transversale, non prévue ailleurs
G02B 27/10 - Systèmes divisant ou combinant des faisceaux
G02B 27/14 - Systèmes divisant ou combinant des faisceaux fonctionnant uniquement par réflexion
G02B 27/18 - Systèmes ou appareils optiques non prévus dans aucun des groupes , pour projection optique, p. ex. combinaison de miroir, de condensateur et d'objectif
G02B 27/40 - Moyens optiques auxiliaires pour mise au point
G02B 27/62 - Appareils optiques spécialement adaptés pour régler des éléments optiques pendant l'assemblage de systèmes optiques
G09G 3/00 - Dispositions ou circuits de commande présentant un intérêt uniquement pour l'affichage utilisant des moyens de visualisation autres que les tubes à rayons cathodiques
G09G 3/32 - Dispositions ou circuits de commande présentant un intérêt uniquement pour l'affichage utilisant des moyens de visualisation autres que les tubes à rayons cathodiques pour la présentation d'un ensemble de plusieurs caractères, p. ex. d'une page, en composant l'ensemble par combinaison d'éléments individuels disposés en matrice utilisant des sources lumineuses commandées utilisant des panneaux électroluminescents semi-conducteurs, p. ex. utilisant des diodes électroluminescentes [LED]
H02N 2/02 - Machines électriques en général utilisant l'effet piézo-électrique, l'électrostriction ou la magnétostriction produisant un mouvement linéaire, p. ex. actionneursPositionneurs linéaires
49.
Cross Reality System with Simplified Programming of Virtual Content
A cross reality system that renders virtual content generated by executing native mode applications may be configured to render web-based content using components that render content from native applications. The system may include a Prism manager that provides Prisms in which content from executing native applications is rendered. For rendering web based content, a browser, accessing the web based content, may be associated with a Prism and may render content into its associated Prism, creating the same immersive experience for the user as when content is generated by a native application. The user may access the web application from the same program launcher menu as native applications. The system may have tools that enable a user to access these capabilities, including by creating for a web location an installable entity that, when processed by the system, results in an icon for the web content in a program launcher menu.
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
G06F 3/04815 - Interaction s’effectuant dans un environnement basé sur des métaphores ou des objets avec un affichage tridimensionnel, p. ex. modification du point de vue de l’utilisateur par rapport à l’environnement ou l’objet
G06F 3/04817 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] fondées sur des propriétés spécifiques de l’objet d’interaction affiché ou sur un environnement basé sur les métaphores, p. ex. interaction avec des éléments du bureau telles les fenêtres ou les icônes, ou avec l’aide d’un curseur changeant de comportement ou d’aspect utilisant des icônes
G06F 3/0482 - Interaction avec des listes d’éléments sélectionnables, p. ex. des menus
G06F 16/954 - Navigation, p. ex. en utilisant la navigation par catégories
G06F 16/955 - Recherche dans le Web utilisant des identifiants d’information, p. ex. des localisateurs uniformisés de ressources [uniform resource locators - URL]
50.
COMPENSATING THICKNESS VARIATIONS IN SUBSTRATES FOR OPTICAL DEVICES
This disclosure describes techniques for fabrication of waveguides as optical devices or for use in optical devices, with the waveguides customized to have a desired thickness variation. Techniques can employ inkjet-based lithography to compensate for thickness variations in the substrate used to manufacture the optical devices, and/or create custom variations in the thickness to achieve various optical properties in the resulting device. In some implementations, a curvature can also be applied to one or both surfaces of the substrate, to achieve desired optical performance and/or enhance fit of a wearable optical device. The optical devices created using the techniques described herein are suitable for use in virtual reality, augmented reality, and/or other suitable optical applications. The optical devices may be created on flexible (e.g., polymer) or more rigid (e.g., glass) substrates, with the thickness of the substrate being customizable using a jettable and curable polymer resin or photoresist.
G03F 7/00 - Production par voie photomécanique, p. ex. photolithographique, de surfaces texturées, p. ex. surfaces impriméesMatériaux à cet effet, p. ex. comportant des photoréservesAppareillages spécialement adaptés à cet effet
B29D 11/00 - Fabrication d'éléments optiques, p. ex. lentilles ou prismes
An eye tracking system can include eye-tracking camera(s) configured to obtain image(s) of the eye at different exposure times or different frame rates. For example, image(s) of the eye taken with a longer exposure time can be analyzed to detect eye feature(s) such as iris or pupil features, and image(s) taken with a shorter exposure time can be analyzed to detect glints reflected from the eye. The shorter exposure images may be taken at a higher frame rate than the longer exposure images for more accurate gaze prediction based on glint analysis, e.g., to provide glint locations to subpixel accuracy. Such glint analysis may also take into account estimated location(s) of partially or totally occluded glint(s). The longer exposure images can be analyzed for pupil center, eye center of rotation, or other characteristics. The system can predict gaze direction, e.g., for foveated rendering by a wearable display system.
An optics assembly includes an eyepiece operable to receive virtual image light and project the virtual image light toward an eye side of the optics assembly. The optics assembly includes a rear lens disposed between the eyepiece and the eye side of the optics assembly. The optics assembly includes one or more wedge prisms integrated with the rear lens along a periphery of the rear lens. The optics assembly includes an eye tracking camera disposed outside the periphery of the rear lens in alignment with the one or more wedge prisms and operable to capture light that propagates from the eye side of the optics assembly and is refracted through the one or more wedge prisms.
A wearable computing system that includes a head-mounted display implements a gaze timer feature for enabling the user to temporarily extend the functionality of a handheld controller or other user input device. In one embodiment, when the user gazes at, or in the vicinity of, a handheld controller for a predetermined period of time, the functionality of one or more input elements (e.g., buttons) of the handheld controller is temporarily modified. For example, the function associated with a particular controller button may be modified to enable the user to open a particular menu using the button. The gaze timer feature may, for example, be used to augment the functionality of a handheld controller or other user input device during mixed reality and/or augmented reality sessions.
G06F 3/023 - Dispositions pour convertir sous une forme codée des éléments d'information discrets, p. ex. dispositions pour interpréter des codes générés par le clavier comme codes alphanumériques, comme codes d'opérande ou comme codes d'instruction
G06F 3/048 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI]
G06F 3/0482 - Interaction avec des listes d’éléments sélectionnables, p. ex. des menus
54.
METHOD AND SYSTEM FOR EYE TRACKING WITH REDUCED KEYSTONE DISTORTION
An eye tracking system operable to image an eye of a user includes an illumination source operable to illuminate the eye and an optical system including a first imaging surface, an aperture stop, and a second imaging surface. The optical system is operable to image an object plane corresponding to the eye. The eye tracking system also includes an image sensor having an image plane parallel to the first imaging surface, the second imaging surface, and the object plane.
A61B 3/113 - Appareils pour l'examen optique des yeuxAppareils pour l'examen clinique des yeux du type à mesure objective, c.-à-d. instruments pour l'examen des yeux indépendamment des perceptions ou des réactions du patient pour déterminer ou enregistrer le mouvement de l'œil
A61B 3/14 - Dispositions spécialement adaptées à la photographie de l'œil
G02B 6/00 - Guides de lumièreDétails de structure de dispositions comprenant des guides de lumière et d'autres éléments optiques, p. ex. des moyens de couplage
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
G02B 9/08 - Objectifs optiques caractérisés à la fois par le nombre de leurs composants et la façon dont ceux-ci sont disposés selon leur signe, c.-à-d. + ou — ayant uniquement deux composants deux composants + associés à un diaphragme
G02B 1/00 - Éléments optiques caractérisés par la substance dont ils sont faitsRevêtements optiques pour éléments optiques
55.
METHOD AND SYSTEM FOR EYE-TRACKING USING AN OFF-AXIS MIRROR WITH DISTORTION CORRECTION
Techniques for tracking an eye of a user of an augmented reality headset having an eye tracking system. The eye tracking system can reflect light from the eye of the user and propagate the light through a first refractive optical element. The eye tracking system can then propagate the light through a second refractive optical element, impinge the light on a reflective optical element, and reflect the light from the reflective optical element. The eye tracking system can then propagate the reflected light through the second refractive optical element to a lateral aperture of the second refractive optical element and detect the reflected light at a camera. The eye tracking system can output an eye tracking signal from the camera to track the eye.
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G02B 6/12 - Guides de lumièreDétails de structure de dispositions comprenant des guides de lumière et d'autres éléments optiques, p. ex. des moyens de couplage du type guide d'ondes optiques du genre à circuit intégré
An extended reality display system includes a light source to emit light. The system also includes a waveguide assembly to receive and direct the light. The system further includes a spatial light modulator (SLM) configured to receive the light. The waveguide assembly includes a first waveguide, a second waveguide, and an adhesive layer disposed between and in direct contact with the first and second waveguides.
G02B 6/10 - Guides de lumièreDétails de structure de dispositions comprenant des guides de lumière et d'autres éléments optiques, p. ex. des moyens de couplage du type guide d'ondes optiques
09 - Appareils et instruments scientifiques et électriques
Produits et services
(1) Headset accessories, namely, headset cushions and headset pads used to correct virtual, augmented and mixed reality headsets to the appropriate size of the user.
58.
EXTENDED REALITY (XR) SYSTEM WITH BODY-CENTRIC POSE ESTIMATION USING ALTIMETER RELATIVE ELEVATION
An extended reality (XR) system, comprises a head-mounted display (HMD) configured for displaying virtual content to a user, a first altimeter carried by the HMD, a hand-held control, and a second altimeter carried by the hand-held control. The first altimeter configured for outputting first atmospheric pressure data indicative of an elevation of the HMD, while the second altimeter is configured for outputting second atmospheric pressure data indicative of an elevation of the hand-held control. The XR system further comprises at least one processor configured for determining a relative elevation between the first altimeter and the second altimeter based on the first atmospheric pressure data and the second atmospheric pressure data.
G01C 5/06 - Mesure des hauteursMesure des distances transversales par rapport à la ligne de viséeNivellement entre des points séparésNiveaux à lunette en utilisant des moyens barométriques
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
59.
DISPLAY DEVICE HAVING DIFFRACTION GRATINGS WITH REDUCED POLARIZATION SENSITIVITY
Blazed diffraction gratings provide optical elements in head-mounted display systems to, e.g., incouple light into or out-couple light out of a waveguide. These blazed diffraction gratings may be configured to have reduced polarization sensitivity. Such gratings may, for example, incouple or outcouple light of different polarizations with similar level of efficiency. The blazed diffraction gratings and waveguides may be formed in or on a high refractive index substrate such as lithium niobate. In some implementations, the blazed diffraction gratings may include diffractive features having a feature height or feature depth of 40 nm to 120 nm, for example, 80 nm. In some examples, the diffractive features may be etched into the high index substrate, e.g., lithium niobate.
This disclosure generally describes methods and systems for fabrication of high-quality surface relief waveguides for eyepieces. In particular, this disclosure describes techniques for manufacturing waveguides having surface relief features, such as diffractive gratings to achieve various optical effects, using nanolithographic imprinting techniques that reduce or eliminate the presence of gaps in the imprinted features through use of optimized drop patterns for dispensing photoresist. Moreover, the disclosure also describes techniques for manufacturing surface relief waveguides having a gradation, e.g., a substantially continuous grade or slope, between zones that have different residual layer thicknesses of the dispensed photoresist, and/or between zones having surface features of different height (or depth). Such gradation can reduce or eliminate adverse optical effects that may be caused by a more abrupt transition between zones, and increase the optical efficiency of the completed waveguide.
G03F 7/00 - Production par voie photomécanique, p. ex. photolithographique, de surfaces texturées, p. ex. surfaces impriméesMatériaux à cet effet, p. ex. comportant des photoréservesAppareillages spécialement adaptés à cet effet
61.
OPTICAL SYSTEMS HAVING WAVEGUIDES WITH NANOPOROUS, LOW REFRACTIVE INDEX LAYERS AND METHODS OF MAKING
Augmented reality display systems include an eyepiece with a low refractive index, nanoporous layer, which may be a sol-gel layer formed using a sol-gel synthesis. The eyepiece includes one or more waveguides, which are transmissive to allow the wearer of the system to see the ambient environment and include a grating for out-coupling light propagating within the waveguides. Protrusions form the grating and the sol-gel layer is disposed over the waveguide and in spaces between the protrusions. The protrusions may have differing heights. The sol-gel layer may completely cover all or only some of the protrusions, while other protrusions extend above the sol-gel layer. The sol-gel layer may be part of a stack of sol-gel layers, which may have different compositions, thicknesses, and/or refractive indices. Hie sol-gel layer and grating structure has advantageously low reflectivity and mitigates a rainbow effect otherwise caused by ambient light reflected from the grating.
G02B 27/09 - Mise en forme du faisceau, p. ex. changement de la section transversale, non prévue ailleurs
G02B 6/00 - Guides de lumièreDétails de structure de dispositions comprenant des guides de lumière et d'autres éléments optiques, p. ex. des moyens de couplage
An eyepiece includes a substrate, an input coupling grating on a first side of the substrate, and a morphed grating comprising characteristics of both a primary grating and a secondary grating on at least the first side of the substrate. The primary grating and the secondary grating may differ in pitch, orientation, and dimensions.
An eyepiece waveguide for an augmented reality display system. The eyepiece waveguide can include an input coupling grating (ICG) region. The ICG region can couple an input beam into the substrate of the eyepiece waveguide as a guided beam. A first combined pupil expander-extractor (CPE) grating region can be formed on or in a surface of the substrate. The first CPE grating region can receive the guided beam, create a first plurality of diffracted beams at a plurality of distributed locations, and out-couple a first plurality of output beams. The eyepiece waveguide can also include a second CPE grating region formed on or in the opposite surface of the substrate. The second CPE grating region can receive the guided beam, create a second plurality of diffracted beams at a plurality of distributed locations, and out-couple a second plurality of output beams.
Examples of wearable systems and methods can use multiple inputs (e.g., gesture, head pose, eye gaze, voice, totem, and/or environmental factors (e.g., location)) to determine a command that should be executed and objects in the three-dimensional (3D) environment that should be operated on. The wearable system can detect when different inputs converge together, such as when a user seeks to select a virtual object using multiple inputs such as eye gaze, head pose, hand gesture, and totem input. Upon detecting an input convergence, the wearable system can perform a transmodal filtering scheme that leverages the converged inputs to assist in properly interpreting what command the user is providing or what object the user is targeting.
This disclosure describes in-plane switching mode liquid crystal geometric phase tunable lenses that can be integrated into an eyepiece of an optical device for the correction of non-emmetropic vision, such as in an augmented reality display system. The eyepiece can include an integrated, field-configurable optic arranged with respect to a waveguide used to project digital imagery to the user, the optic being capable of providing a tunable Rx for the user including variable spherical refractive power (SPH), cylinder refractive power, and cylinder axis values. In certain configuration, each tunable eyepiece includes two variable compound lenses: one on the user-side of the waveguide with variable SPH, cylinder power, and axis values; and a second on the world side of the waveguide with variable SPH.
G02C 7/08 - Verres auxiliairesDispositions pour faire varier la distance focale
G02F 1/1347 - Disposition de couches ou de cellules à cristaux liquides dans lesquelles un faisceau lumineux est modifié par l'addition des effets de plusieurs couches ou cellules
G02F 1/29 - Dispositifs ou dispositions pour la commande de l'intensité, de la couleur, de la phase, de la polarisation ou de la direction de la lumière arrivant d'une source lumineuse indépendante, p. ex. commutation, ouverture de porte ou modulationOptique non linéaire pour la commande de la position ou de la direction des rayons lumineux, c.-à-d. déflexion
A method of fabricating a blazed diffraction grating comprises providing a master template substrate and imprinting periodically repeating lines on the master template substrate in a plurality of master template regions. The periodically repeating lines in different ones of the master template regions extend in different directions. The method additionally comprises using at least one of the master template regions as a master template to imprint at least one blazed diffraction grating pattern on a grating substrate.
A wearable system can comprise a display system configured to present virtual content in a three-dimensional space, a user input device configured to receive a user input, and one or more sensors configured to detect a user's pose. The wearable system can support various user interactions with objects in the user's environment based on contextual information. As an example, the wearable system can adjust the size of an aperture of a virtual cone during a cone cast (e.g., with the user's poses) based on the contextual information. As another example, the wearable system can adjust the amount of movement of virtual objects associated with an actuation of the user input device based on the contextual information.
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06F 3/0346 - Dispositifs de pointage déplacés ou positionnés par l'utilisateurLeurs accessoires avec détection de l’orientation ou du mouvement libre du dispositif dans un espace en trois dimensions [3D], p. ex. souris 3D, dispositifs de pointage à six degrés de liberté [6-DOF] utilisant des capteurs gyroscopiques, accéléromètres ou d’inclinaison
A thin transparent layer can be integrated in a head mounted display device and disposed in front of the eye of a wearer. The thin transparent layer may be configured to output light such that light is directed onto the eye to create reflections therefrom that can be used, for example, for glint based tracking. The thin transparent layer can be configured to reduced obstructions in the field of the view of the user.
A display device comprises a waveguide configured to guide light in a lateral direction parallel to an output surface of the waveguide. The waveguide is further configured to outcouple the guided light through the output surface. The display device additionally comprises a broadband adaptive lens assembly configured to incouple and to diffract therethrough the outcoupled light from the waveguide. The broadband adaptive lens assembly comprises a first waveplate lens comprising a liquid crystal (LC) layer arranged such that the waveplate lens has birefringence (Δn) that varies in a radially outward direction from a central region of the first waveplate lens and configured to diffract the outcoupled light at a diffraction efficiency greater than 90% within a wavelength range including at least 450 nm to 630 nm. The broadband adaptive lens assembly is configured to be selectively switched between a plurality of states having different optical powers.
G02B 6/12 - Guides de lumièreDétails de structure de dispositions comprenant des guides de lumière et d'autres éléments optiques, p. ex. des moyens de couplage du type guide d'ondes optiques du genre à circuit intégré
G02B 6/122 - Éléments optiques de base, p. ex. voies de guidage de la lumière
G02F 1/1337 - Orientation des molécules des cristaux liquides induite par les caractéristiques de surface, p. ex. par des couches d'alignement
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
G06V 20/20 - ScènesÉléments spécifiques à la scène dans les scènes de réalité augmentée
G09G 3/00 - Dispositions ou circuits de commande présentant un intérêt uniquement pour l'affichage utilisant des moyens de visualisation autres que les tubes à rayons cathodiques
G09G 5/00 - Dispositions ou circuits de commande de l'affichage communs à l'affichage utilisant des tubes à rayons cathodiques et à l'affichage utilisant d'autres moyens de visualisation
Disclosed herein are systems and methods for generating and presenting virtual audio for mixed reality systems. A method may include determining a collision between a first object and a second object, wherein the first object comprises a first virtual object. A memory storing one or more audio models can be accessed. It can be determined if the one or more audio models stored in the memory comprises an audio model corresponding to the first object. In accordance with a determination that the one or more audio models comprises an audio model corresponding to the first object, an audio signal can be synthesized, wherein the audio signal is based on the collision and the audio model corresponding to the first object, and the audio signal can be presented to a user via a speaker of a head-wearable device. In accordance with a determination that the one or more audio models does not comprise an audio model corresponding to the first object, an acoustic property of the first object can be determined, a custom audio model based on the acoustic property of the first object can be generated, an audio signal can be synthesized, wherein the audio signal is based on the collision and the custom audio model, and the audio signal can be presented, via a speaker of a head-wearable device, to a user.
H04S 7/00 - Dispositions pour l'indicationDispositions pour la commande, p. ex. pour la commande de l'équilibrage
A63F 13/285 - Génération de signaux de retour tactiles via le dispositif d’entrée du jeu, p. ex. retour de force
A63F 13/577 - Simulations de propriétés, de comportement ou de déplacement d’objets dans le jeu, p. ex. calcul de l’effort supporté par un pneu dans un jeu de course automobile utilisant la détermination de la zone de contact entre les personnages ou les objets du jeu, p. ex. pour éviter une collision entre des voitures de course virtuelles
Plasma etching processes for forming patterns in high refractive index glass substrates, such as for use as waveguides, are provided herein. The substrates may be formed of glass having a refractive index of greater than or equal to about 1.65 and having less than about 50 wt % SiO2. The plasma etching processes may include both chemical and physical etching components. In some embodiments, the plasma etching processes can include forming a patterned mask layer on at least a portion of the high refractive index glass substrate and exposing the mask layer and high refractive index glass substrate to a plasma to remove high refractive index glass from the exposed portions of the substrate. Any remaining mask layer is subsequently removed from the high refractive index glass substrate. The removal of the glass forms a desired patterned structure, such as a diffractive grating, in the high refractive index glass substrate.
C03C 15/00 - Traitement de surface du verre, autre que sous forme de fibres ou de filaments, par attaque chimique
C03C 3/097 - Compositions pour la fabrication du verre contenant de la silice avec 40 à 90% en poids de silice contenant du phosphore, du niobium ou du tantale
F21V 8/00 - Utilisation de guides de lumière, p. ex. dispositifs à fibres optiques, dans les dispositifs ou systèmes d'éclairage
G02B 30/26 - Systèmes ou appareils optiques pour produire des effets tridimensionnels [3D], p. ex. des effets stéréoscopiques en fournissant des première et seconde images de parallaxe à chacun des yeux gauche et droit d’un observateur du type autostéréoscopique
A method includes providing a wearable device including a left optical stack and a right optical stack and receiving virtual content to be displayed. The method also includes determining that eye tracking data is available and determining if a user is fixated on the first focal plane or the second focal plane. In response to determining that the user is fixated on the first focal plane, the method includes causing the left optical stack to display the virtual content at the first focal plane and causing the right optical stack to display the virtual content at the first focal plane. In response to determining that the user is fixated on the second focal plane, the method includes causing the left optical stack to display the virtual content at the second focal plane and causing the right optical stack to display the virtual content at the second focal plane.
An article includes: a substrate composed of a first material having a first refractive index at an operative wavelength in the visible spectrum; and a layer of a second material on a surface of the substrate, the layer defining two or more surface relief gratings, the second material having a second refractive index at the operative wavelength. The second material can be an organic material or a hybrid material; the first and second materials can be different; a residual layer thickness of the layer between adjacent surface relief gratings of the two or more surface relief gratings can be 30 nm or less; the first refractive index can be 1.7 or greater; and a difference between the first refractive index and the second refractive index at the operative wavelength can be 0.1 or less.
A method for matching a base mesh to a target mesh incudes obtaining base and target meshes; matching the base mesh to the target mesh by: determining distance differences between at least some of the vertices of the base mesh relative to the target mesh; identifying a set of vertices in the base mesh that have distance differences above a first threshold; applying a rigid transformation to the set of vertices in the base mesh to reduce the distance differences of the vertices in the set of vertices and to produce a first transformed base mesh; and applying a non-rigid deformation to the set of vertices in the first transformed base mesh to further reduce the distance differences of the vertices in the set of vertices and to produce a second transformed base mesh; and providing a blendshape based at least on the second transformed base mesh.
A user may interact and view virtual elements such as avatars and objects and/or real world elements in three-dimensional space in an augmented reality (AR) session. The system may allow one or more spectators to view from a stationary or dynamic camera a third person view of the users AR session. The third person view may be synchronized with the user view and the virtual elements of the user view may be composited onto the third person view.
An apparatus includes a light-emitting diode (LED) driver circuit, one or more LEDs of an LED array, and an electronic switching circuit. The LED driver circuit is configured to generate an electric current. The one or more LEDs are electrically connected to the LED driver circuit. The electronic switching circuit is electrically connected to the one or more LEDs and configured to be placed in one of multiple switching configurations. The electronic switching circuit is further configured to direct a portion of the electric current away from the one or more LEDs, such that a remaining portion of the electric current drives the one or more LEDs. The portion of the electric current corresponds to the one of the multiple switching configurations.
An image sensor suitable for use in an augmented reality system to provide low latency image analysis with low power consumption. The augmented reality system can be compact, and may be small enough to be packaged within a wearable device such as a set of goggles or mounted on a frame resembling ordinary eyeglasses. The image sensor may receive information about a region of an imaging array associated with a movable object and selectively output imaging information for that region. The region may be updated dynamically as the image sensor and/or the object moves. Such an image sensor provides a small amount of data from which object information used in rendering an augmented reality scene can be developed. The amount of data may be further reduced by configuring the image sensor to output indications of pixels for which the measured intensity of incident light changes.
In some embodiments, an interconnect electrical connects a light emitter to wiring on a substrate. The interconnect may be deposited by 3D printing and lays flat on the light emitter and substrate. In some embodiments, the interconnect has a generally rectangular or oval cross-sectional profile and extends above the light emitter to a height of about 50 μm or less, or about 35 μm or less. This small height allows close spacing between an overlying optical structure and the light emitter, thereby providing high efficiency in the injection of light from the light emitter into the optical structure, such as a light pipe.
Display devices include waveguides with metasurfaces as in-coupling and/or out-coupling optical elements. The metasurfaces may be formed on a surface of the waveguide and may include a plurality or an array of sub-wavelength-scale (e.g., nanometer-scale) protrusions. Individual protrusions may include horizontal and/or vertical layers of different materials which may have different refractive indices, allowing for enhanced manipulation of light redirecting properties of the metasurface. Some configurations and combinations of materials may advantageously allow for broadband metasurfaces. Manufacturing methods described herein provide for vertical and/or horizontal layers of different materials in a desired configuration or profile.
Techniques are described for enhanced eye tracking for display systems, such as augmented or virtual reality display systems. The display systems may include a light source configured to output light, a moveable diffractive grating configured to reflect light from the light source, the reflected light forming a scan pattern on the eye of the user, and light detectors to detect light reflected from the eye. The orientation of the diffractive grating can be moved such that the light reflected from the diffractive grating is scanned across the eye according to the scan pattern. Light intensity pattern(s) are obtained via the light detectors, with a light intensity pattern representing a light detector signal obtained by detecting light reflected by the eye as the light is scanned across the eye. One or more physiological characteristics and/or a rotation speed of the eye are determined based on detected light intensity pattern(s).
An augmented reality display system can include an eyepiece waveguide with an input coupling grating (ICG) region. The ICG region can receive a set of input beams of light corresponding to an input image having a corresponding field of view (FOV), and can couple at least a subset of the input beams into the waveguide. The waveguide can also include a multi-directional pupil expander (MPE) region and an exit pupil expander (EPE) region on opposite surfaces of the waveguide, to replicate the light beams propagating in the waveguide. In some examples, the waveguide can include a second ICG region, and the two ICG regions can each be arranged to couple, into the waveguide, subsets of light beams corresponding to different sub-portions of the FOV which together comprise a complete FOV of an input image.
The invention provides a method of initiating code including (i) storing an application having first, second and third functions, the first function being a main function that calls the second and third functions to run the application, (ii) compiling the application to first and second heterogeneous processors to create first and second central processing unit (CPU) instruction set architecture (ISA) objects respectively, (iii) pruning the first and second CPU ISA objects by removing the third function from the first CPU ISA objects and removing first and second functions from the second CPU ISA objects, (iv) proxy inserting first and second remote procedure calls (RPC's) in the first and second CPU ISA objects respectively, and pointing respectively to the third function in the second CPU ISA objects and the second function in the first CPU ISA objects, and (v) section renaming the second CPU ISA objects to common application library.
A wearable display system for a cross reality (XR) system may have a dynamic vision sensor (DVS) camera and a color camera. At least one of the cameras may be a plenoptic camera. The wearable display system may dynamically restrict processing of image data from either or both cameras based on detected conditions and XR function being performed. For tracking an object, image information may be processed for patches of a field of view of either or both cameras corresponding to the object. The object may be tracked based on asynchronously acquired events indicating changes within the patches. Stereoscopic or other types of image information may be used when event-based object tacking yields an inadequate quality metric. The tracked object may be a user's hand or a stationary object in the physical world, enabling calculation of the pose of the wearable display system and of the wearer's head.
A wearable display system including multiple cameras and a processor is disclosed. A greyscale camera and a color camera can be arranged to provide a central view field associated with both cameras and a peripheral view field associated with one of the two cameras. One or more of the two cameras may be a plenoptic camera. The wearable display system may acquire light field information using the at least one plenoptic camera and create a world model using the first light field information and first depth information stereoscopically determined from images acquired by the greyscale camera and the color camera. The wearable display system can track head pose using the at least one plenoptic camera and the world model. The wearable display system can track objects in the central view field and the peripheral view fields using the one or two plenoptic cameras, when the objects satisfy a depth criterion.
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
H04N 23/951 - Systèmes de photographie numérique, p. ex. systèmes d'imagerie par champ lumineux en utilisant plusieurs images pour influencer la résolution, la fréquence d'images ou le rapport de cadre
H04N 23/957 - Caméras ou modules de caméras à champ lumineux ou plénoptiques
H04N 25/46 - Extraction de données de pixels provenant d'un capteur d'images en agissant sur les circuits de balayage, p. ex. en modifiant le nombre de pixels ayant été échantillonnés ou à échantillonner en combinant ou en groupant les pixels
86.
THREE DIMENSIONAL VIRTUAL AND AUGMENTED REALITY DISPLAY SYSTEM
A system may comprise a selectively transparent projection device for projecting an image toward an eye of a viewer from a projection device position in space relative to the eye of the viewer, the projection device being capable of assuming a substantially transparent state when no image is projected; an occlusion mask device coupled to the projection device and configured to selectively block light traveling toward the eye from one or more positions opposite of the projection device from the eye of the viewer in an occluding pattern correlated with the image projected by the projection device; and a zone plate diffraction patterning device interposed between the eye of the viewer and the projection device and configured to cause light from the projection device to pass through a diffraction pattern having a selectable geometry as it travels to the eye.
G02B 30/24 - Systèmes ou appareils optiques pour produire des effets tridimensionnels [3D], p. ex. des effets stéréoscopiques en fournissant des première et seconde images de parallaxe à chacun des yeux gauche et droit d’un observateur du type stéréoscopique impliquant un multiplexage temporel, p. ex. utilisant des obturateurs gauche et droit activés séquentiellement
G02B 30/34 - Stéréoscopes fournissant une paire stéréoscopique d'images séparées correspondant à des vues déplacées parallèlement du même objet, p. ex. visionneuses de diapositives 3D
G02B 30/52 - Systèmes ou appareils optiques pour produire des effets tridimensionnels [3D], p. ex. des effets stéréoscopiques l’image étant construite à partir d'éléments d'image répartis sur un volume 3D, p. ex. des voxels le volume 3D étant construit à partir d'une pile ou d'une séquence de plans 2D, p. ex. systèmes d'échantillonnage en profondeur
G03B 21/00 - Projecteurs ou visionneuses du type par projectionLeurs accessoires
G03B 35/08 - Photographie stéréoscopique par enregistrement simultané
G03B 35/18 - Photographie stéréoscopique par examen simultané
87.
METHOD AND SYSTEM FOR USING CHARACTERIZATION LIGHT TO DETECT FIBER POSITION IN A FIBER SCANNING PROJECTOR
A projector including a cantilever position detection system includes a chassis and an actuator mounted to the chassis. The projector also includes a cantilever light source having a longitudinal axis and mechanically coupled to the actuator. The cantilever light source is operable to transmit light. The projector further includes an optical assembly section operable to receive the light. The optical assembly section includes a polarizing beamsplitter having an incidence surface and an opposing surface. The polarizing beamsplitter is operable to transmit light incident on the incidence surface and reflect at least a portion of light incident on the opposing surface. The projectors includes a position measurement device operable to receive the reflected portion of the light and an optical waveguide disposed between the optical assembly section and the position measurement device. The optical waveguide is operable to transmit at least a second portion of the light.
G02B 26/08 - Dispositifs ou dispositions optiques pour la commande de la lumière utilisant des éléments optiques mobiles ou déformables pour commander la direction de la lumière
H04N 9/31 - Dispositifs de projection pour la présentation d'images en couleurs
88.
Display panel or portion thereof with a transitional mixed reality graphical user interface
A head-mounted display system includes a waveguide configured to guide light from a light projection system coupled into the waveguide; a grating structure optically coupled to the waveguide, the grating structure being configured to couple light from the light projection system into the waveguide. The grating structure includes a grating layer having a grating with multiple ridges having ablaze profile in at least one cross-section, the blaze profile having an anti-blaze angle of 85° or less; and one or more additional layers on the grating layer, the additional layers including a first layer of a material having a refractive index of 1.5 or less at an operative wavelength of the head-mounted display, the first layer being an outermost layer of the grating structure.
A smartphone may be freely moved in three dimensions as it captures a stream of images of an object. Multiple image frames may be captured in different orientations and distances from the object and combined into a composite image representing an three-dimensional image of the object. The image frames may be formed into the composite image based on representing features of each image frame as a set of points in a three dimensional depth map. Coordinates of the points in the depth map may be estimated with a level of certainty. The level of certainty may be used to determine which points are included in the composite image. The selected points may be smoothed and a mesh model may be formed by creating a convex hull of the selected points. The mesh model and associated texture information may be used to render a three-dimensional representation of the object on a two-dimensional display. Additional techniques include processing and formatting of the three-dimensional representation data to be printed by a three-dimensional printer so a three-dimensional model of the object may be formed.
G06T 17/20 - Description filaire, p. ex. polygonalisation ou tessellation
G06T 19/20 - Édition d'images tridimensionnelles [3D], p. ex. modification de formes ou de couleurs, alignement d'objets ou positionnements de parties
H04N 13/211 - Générateurs de signaux d’images utilisant des caméras à images stéréoscopiques utilisant un seul capteur d’images 2D utilisant le multiplexage temporel
H04N 13/221 - Générateurs de signaux d’images utilisant des caméras à images stéréoscopiques utilisant un seul capteur d’images 2D utilisant le mouvement relatif de caméras et de sujets
H04N 13/271 - Générateurs de signaux d’images où les signaux d’images générés comprennent des cartes de profondeur ou de disparité
H04N 13/275 - Générateurs de signaux d’images à partir de modèles 3D d’objets, p. ex. des signaux d’images stéréoscopiques générés par ordinateur
A visual perception device is described. The visual perception device has corrective optics for viewing virtual and real-world content. An insert for the corrective optics is attached using a magnetic set, pins and/or a nose piece. Interchangeable nose pieces allow for height adjustments to accommodate different users. The visual perception device has pliable components to absorb forces exerted on a nose piece and a protective barrier for limiting electric shock or ingress of dirt.
Disclosed herein are systems and methods for presenting an audio signal associated with presentation of a virtual object colliding with a surface. The virtual object and the surface may be associated with a mixed reality environment. Generation of the audio signal may be based on at least one of an audio stream from a microphone and a video stream form a sensor. In some embodiments, the collision between the virtual object and the surface is associated with a footstep on the surface.
G06V 20/40 - ScènesÉléments spécifiques à la scène dans le contenu vidéo
G10L 25/57 - Techniques d'analyse de la parole ou de la voix qui ne se limitent pas à un seul des groupes spécialement adaptées pour un usage particulier pour comparaison ou différentiation pour le traitement des signaux vidéo
A head-mounted display system configured to be worn over eyes of a user includes a display disposed over the eyes of the user and configured to emit light into the eyes of the user, where the display includes a dimmer configured to modify a transparency of the display. The system also includes a light sensor separated from an external environment of the user by the display and configured to measure a transmitted light intensity of light from the external environment of the user transmitted through the dimmer. The system further includes a processor operatively coupled to the display and the light sensor, and configured to control an emitted light intensity of the light emitted by the display based on the transmitted light intensity.
G09G 3/22 - Dispositions ou circuits de commande présentant un intérêt uniquement pour l'affichage utilisant des moyens de visualisation autres que les tubes à rayons cathodiques pour la présentation d'un ensemble de plusieurs caractères, p. ex. d'une page, en composant l'ensemble par combinaison d'éléments individuels disposés en matrice utilisant des sources lumineuses commandées
G09G 3/06 - Dispositions ou circuits de commande présentant un intérêt uniquement pour l'affichage utilisant des moyens de visualisation autres que les tubes à rayons cathodiques pour la présentation d'un seul caractère, soit en sélectionnant un seul caractère parmi plusieurs, soit en composant le caractère par combinaison d'éléments individuels, p. ex. de segments élémentaires utilisant des sources lumineuses commandées
H05B 45/10 - Commande de l'intensité de la lumière
G06F 3/038 - Dispositions de commande et d'interface à cet effet, p. ex. circuits d'attaque ou circuits de contrôle incorporés dans le dispositif
H05B 41/38 - Commande de l'intensité de la lumière
G09G 5/02 - Dispositions ou circuits de commande de l'affichage communs à l'affichage utilisant des tubes à rayons cathodiques et à l'affichage utilisant d'autres moyens de visualisation caractérisés par la manière dont la couleur est visualisée
G09G 3/20 - Dispositions ou circuits de commande présentant un intérêt uniquement pour l'affichage utilisant des moyens de visualisation autres que les tubes à rayons cathodiques pour la présentation d'un ensemble de plusieurs caractères, p. ex. d'une page, en composant l'ensemble par combinaison d'éléments individuels disposés en matrice
94.
METHOD AND SYSTEM FOR ALTERNATING MASK-BASED AND FRAME-BASED DATA COMPRESSION
A method displaying a reconstructed image includes receiving an image, determining an eye gaze location of a user, compressing a primary quality region of the image using a compression ratio of X → Y bits per pixel (bpp) to form a primary quality image, and compressing a secondary quality region of the image using a compression ratio of X → Z bpp, wherein Z < Y, to form a secondary quality image. The method also includes overlaying the primary quality image on the secondary quality image to form the reconstructed image and displaying the reconstructed image.
H04N 19/132 - Échantillonnage, masquage ou troncature d’unités de codage, p. ex. ré-échantillonnage adaptatif, saut de trames, interpolation de trames ou masquage de coefficients haute fréquence de transformée
H04N 19/182 - Procédés ou dispositions pour le codage, le décodage, la compression ou la décompression de signaux vidéo numériques utilisant le codage adaptatif caractérisés par l’unité de codage, c.-à-d. la partie structurelle ou sémantique du signal vidéo étant l’objet ou le sujet du codage adaptatif l’unité étant un pixel
H04N 19/42 - Procédés ou dispositions pour le codage, le décodage, la compression ou la décompression de signaux vidéo numériques caractérisés par les détails de mise en œuvre ou le matériel spécialement adapté à la compression ou à la décompression vidéo, p. ex. la mise en œuvre de logiciels spécialisés
An eyepiece for an augmented reality headset includes an eyepiece waveguide and an incoupling diffractive optical element coupled to the eyepiece waveguide. The incoupling diffractive optical element is disposed at a first lateral location. The eyepiece also includes an outcoupling diffractive optical element coupled to the eyepiece waveguide. The outcoupling diffractive optical element is disposed at a second lateral location different than the first lateral location. The eyepiece further includes an optical structure coupled to the eyepiece waveguide. The optical structure is disposed at a third lateral location between the first lateral location and the second lateral location.
Display systems are described including augmented 1-dimensional pixel arrays and scanning mirrors. In one example, a pixel array includes first and second columns of pixels, relay optics configured to receive incident light and to output the incident light to a viewer, and a scanning mirror disposed to receive the light from the first and second columns of pixels and to reflect the received light toward the relay optics. The scanning mirror may move between a plurality of positions while the first and second columns emit light in temporally spaced pulses so as to form a perceived image at the relay optics having a higher resolution relative to the pixel pitch of the individual columns. Foveated rendering may provide for more efficient use of power and processing resources.
An optical device can include a liquid crystal layer including a first plurality of liquid crystal molecules arranged in a first pattern and a second plurality of liquid crystal molecules arranged in a second pattern. The first and the second patterns may be separated from each other by a suitable distance, e.g., about 20 nm to about 100 nm, along a longitudinal or a transverse axis of the liquid crystal layer. The first and the second pluralities of liquid crystal molecules can be configured as first and second grating structures that can redirect light of visible or infrared wavelengths. In some examples, the optical device includes electrode layers arranged on either side of the liquid crystal layer to control an alignment of the liquid crystal molecules. Methods of fabricating such devices are also described.
G02F 1/1347 - Disposition de couches ou de cellules à cristaux liquides dans lesquelles un faisceau lumineux est modifié par l'addition des effets de plusieurs couches ou cellules
B81C 1/00 - Fabrication ou traitement de dispositifs ou de systèmes dans ou sur un substrat
G02B 6/00 - Guides de lumièreDétails de structure de dispositions comprenant des guides de lumière et d'autres éléments optiques, p. ex. des moyens de couplage
G02B 27/00 - Systèmes ou appareils optiques non prévus dans aucun des groupes ,
G02B 27/10 - Systèmes divisant ou combinant des faisceaux
G02B 30/36 - Stéréoscopes fournissant une paire stéréoscopique d'images séparées correspondant à des vues déplacées parallèlement du même objet, p. ex. visionneuses de diapositives 3D utilisant des éléments optiques réfractifs, p. ex. des prismes, dans le chemin optique entre les images et les observateurs
G02F 1/1334 - Dispositions relatives à la structure basées sur des cristaux liquides dispersés dans un polymère, p. ex. cristaux liquides micro-encapsulés
G02F 1/1337 - Orientation des molécules des cristaux liquides induite par les caractéristiques de surface, p. ex. par des couches d'alignement
G02F 1/29 - Dispositifs ou dispositions pour la commande de l'intensité, de la couleur, de la phase, de la polarisation ou de la direction de la lumière arrivant d'une source lumineuse indépendante, p. ex. commutation, ouverture de porte ou modulationOptique non linéaire pour la commande de la position ou de la direction des rayons lumineux, c.-à-d. déflexion
98.
SYSTEMS AND METHODS FOR PERFORMING A MOTOR SKILLS NEUROLOGICAL TEST USING AUGMENTED OR VIRTUAL REALITY
System and methods for performing a motor skills neurological test using augmented reality which provides an objective assessment of the results of the test. A virtual target is displayed to a user in an AR field of view of an AR system at a target location. The movement of a body part (e.g., a finger) of a user is tracked as the user moves the body part from a starting location to the target location. A total traveled distance of the body part in moving from the starting location to the target location is determined based on the tracking. A linear distance between the starting location and the target location is determined. An efficiency index is then determined which represents an overall quality of movement of the body part from the starting location to the target location based on the total traveled distance and the linear distance.
A method of processing an acoustic signal is disclosed. According to one or more embodiments, a first acoustic signal is received via a first microphone. The first acoustic signal is associated with a first speech of a user of a wearable headgear unit. A first sensor input is received via a sensor. a control parameter is determined based on the sensor input. The control parameter is applied to one or more of the first acoustic signal, the wearable headgear unit, and the first microphone. Determining the control parameter comprises determining, based on the first sensor input, a relationship between the first speech and the first acoustic signal.
Examples of systems and methods for rendering an avatar in a mixed reality environment are disclosed. The systems and methods may be configured to automatically scale an avatar or to render an avatar based on a determined intention of a user, an interesting impulse, environmental stimuli, or user saccade points. The disclosed systems and methods may apply discomfort curves when rendering an avatar. The disclosed systems and methods may provide a more realistic interaction between a human user and an avatar.
