A method of forming a foveated image includes (a) setting dimensions of a first region, (b) receiving an image having a first resolution, and (c) forming the foveated image including a primary quality region having the dimensions of the first region and the first resolution and a secondary quality region having a second resolution less than the first resolution. The method also includes (d) outputting the foveated image, (e) determining an eye gaze location, and (f) determining an eye gaze velocity. If the eye gaze velocity is less than a threshold velocity, the method includes decreasing the dimensions of the primary quality region and repeating (b)-(f). If the eye gaze velocity is greater than or equal to the threshold velocity, the method includes repeating (a)-(f).
G09G 3/20 - Dispositions ou circuits de commande présentant un intérêt uniquement pour l'affichage utilisant des moyens de visualisation autres que les tubes à rayons cathodiques pour la présentation d'un ensemble de plusieurs caractères, p. ex. d'une page, en composant l'ensemble par combinaison d'éléments individuels disposés en matrice
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
2.
LOW MOTION-TO-PHOTON LATENCY ARCHITECTURE FOR AUGMENTED AND VIRTUAL REALITY DISPLAY SYSTEMS
Systems and methods are disclosed for low motion-to-photon latency for augmented and virtual reality systems. Some systems generate rendered frames that are presented to a user by outputting light from a head-mounted display unit. The rendered frames are perceived by the user as virtual content. The head-mounted display unit includes an orientation sensor, a display configured to output light to the user, and processors. The processors receive a rendered frame of virtual content, obtain orientation information from the orientation sensor, and warp or modify the rendered frame of virtual content based on changes to the orientation of the user's head. The warped rendered frame is subsequently outputted from the display using modulated light. The processors and the orientation sensor may be part of a spatial light modulator for modulating the light used to present the warped rendered frame. In addition, the spatial light modulator may be a LED array having low persistence and a high duty cycle.
A dynamic diffractive structure includes an optical waveguide having a surface, a first transparent electrode coupled to the surface, a chamber formed on the surface, and a nanoscale diffraction grating structure including a plurality of ridges formed on the first transparent electrode and within the chamber and a plurality of channels between adjacent ridges of the plurality of ridges. The dynamic diffractive structure also includes a fluid having an index of refraction disposed within the chamber, a second transparent electrode, and a cover coupled to the chamber. In a first state, the fluid substantially fills the plurality of channels, causing the dynamically actuable diffractive grating to be in an “off” state and in a second state, the fluid is substantially absent from at least a portion of the plurality of channels, causing the dynamically actuable diffractive grating to be in an “on” state and diffract incident light.
G02B 26/00 - Dispositifs ou dispositions optiques pour la commande de la lumière utilisant des éléments optiques mobiles ou déformables
G02B 26/08 - Dispositifs ou dispositions optiques pour la commande de la lumière utilisant des éléments optiques mobiles ou déformables pour commander la direction de la lumière
G02B 27/00 - Systèmes ou appareils optiques non prévus dans aucun des groupes ,
An article includes: a waveguide for guiding light at a wavelength, λ, the waveguide extending in a waveguide plane; an input coupling grating (ICG) optically coupled to a surface of the waveguide, the ICG comprising a plurality of metallic strips spaced apart from each other and extending parallel to each other in a first direction parallel to the waveguide plane, the ICG further comprising one or more dielectric materials encapsulating the metallic strips, each metallic strip having a first surface that is (i) oriented at an acute angle with respect to the waveguide plane, (ii) oriented perpendicularly with respect to the waveguide plane, or (iii) curvilinear in shape. The first surface of each metallic strip is configured to redirect incident light at the wavelength λ into the waveguide.
Various methods and apparatus are described herein for enabling one or more users to interface with virtual or augmented reality environments. An example system includes a computing network having computer servers interconnected through high bandwidth interfaces to gateways for processing data and/or for enabling communication of data between the servers and one or more local user interface devices. The servers include memory, processing circuitry, and software for designing and/or controlling virtual worlds, as well as for storing and processing user data and data provided by other components of the system. One or more virtual worlds may be presented to a user through a user device for the user to experience and interact. A large number of users may each use a device to simultaneously interface with one or more digital worlds by using the device to observe and interact with each other and with objects produced within the digital worlds.
A method of operating an eyepiece waveguide of an augmented reality system includes projecting virtual content using a projector assembly and diffracting the virtual content into the eyepiece waveguide via a first order diffraction. A first portion of the virtual content is clipped to produce a remaining portion of the virtual content. The method also includes propagating the remaining portion of the virtual content in the eyepiece waveguide, outcoupling the remaining portion of the virtual content out of the eyepiece waveguide, and diffracting the virtual content into the eyepiece waveguide via a second order diffraction. A second portion of the virtual content is clipped to produce a complementary portion. The method further includes propagating the complementary portion of the virtual content in the eyepiece waveguide and outcoupling the complementary portion of the virtual content out of the eyepiece waveguide.
An augmented reality system includes a projector assembly and a set of imaging optics optically coupled to the projector assembly. The augmented reality system also includes an eyepiece optically coupled to the set of imaging optics. The eyepiece has a world side and a user side opposite the world side and includes one or more eyepiece waveguides. Each of the one or more eyepiece waveguides includes an incoupling interface and an outcoupling interface operable to output virtual content toward the user side. The augmented reality system further includes an optical notch filter disposed on the world side of the eyepiece.
Head-mounted display systems with power saving functionality are disclosed. The systems can include a frame configured to be supported on the head of the user. The systems can also include a head-mounted display disposed on the frame, one or more sensors, and processing electronics in communication with the display and the one or more sensors. In some implementations, the processing electronics can be configured to cause the system to reduce power of one or more components in response to at least in part on a determination that the frame is in a certain position (e.g., upside-down or on top of the head of the user). In some implementations, the processing electronics can be configured to cause the system to reduce power of one or more components in response to at least in part on a determination that the frame has been stationary for at least a threshold period of time.
The present disclosure generally relates to display systems, and more particularly to augmented reality display systems and methods of fabricating the same. A method of fabricating a display device includes providing a substrate comprising a lithium (Li)-based oxide and forming an etch mask pattern exposing regions of the substrate. The method additionally includes plasma etching the exposed regions of the substrate using a gas mixture comprising CHF3 to form a diffractive optical element, wherein the diffractive optical element comprises Li-based oxide features configured to diffract visible light incident thereon.
Methods and systems are disclosed for presenting virtual objects on a limited number of depth planes using, e.g., an augmented reality display system. A farthest one of the depth planes is within a mismatch tolerance of optical infinity. The display system may switch the depth plane on which content is actively displayed so that the content is displayed on the depth plane on which a user is fixating. The impact of errors in fixation tracking is addressed using partially overlapping depth planes. A fixation depth is determined and the display system determines whether to adjust selection of a selected depth plane at which a virtual object is presented. The determination may be based on whether the fixation depth falls within a depth overlap region of adjacent depth planes. The display system may switch the active depth plane depending upon whether the fixation depth falls outside the overlap region.
Disclosed herein is an article including: a waveguide formed from a polymer material and including: an optical coupling structure on a surface of the waveguide, the optical coupling structure configured to couple light incident on the optical coupling structure into the waveguide; an out-coupling surface grating extending over a first region of the surface of the waveguide; and an anti-reflective surface grating extending over a second region of the surface of the waveguide, the second region encircling the optical coupling structure, the anti-reflective surface grating including: a layer of material layered to a thickness on the surface of the waveguide; and a pattern of nanostructures having a width, extending from the layer of material by a height and spaced apart by a pitch.
G02B 1/118 - Revêtements antiréfléchissants ayant des structures de surface de longueur d’onde sous-optique conçues pour améliorer la transmission, p. ex. structures du type œil de mite
G02B 27/00 - Systèmes ou appareils optiques non prévus dans aucun des groupes ,
A display system can include a head-mounted display configured to project light to an eye of a user to display virtual image content at different amounts of divergence and collimation. The display system can include an inward-facing imaging system possibly comprising a plurality of cameras that image the user's eye and glints for thereon and processing electronics that are in communication with the inward-facing imaging system and that are configured to obtain an estimate of a center of rotation of the user's eye using cornea data derived from the glint images. The display system may render virtual image content with a render camera positioned at the determined position of the center of rotation of said eye.
In some implementations, a method includes imprinting an optically-diffractive structure, and imprinting an optically-sub-diffractive structure adjacent to the optically-diffractive structure. For example, the structures can be grating with different characteristics. The imprinted structures can be included in optical devices and display systems.
G03F 7/00 - Production par voie photomécanique, p. ex. photolithographique, de surfaces texturées, p. ex. surfaces impriméesMatériaux à cet effet, p. ex. comportant des photoréservesAppareillages spécialement adaptés à cet effet
14.
ACTIVE NOISE CANCELLATION FOR WEARABLE HEAD DEVICE
Examples of the disclosure describe systems and methods for reducing audio effects of fan noise, specifically, for a wearable system. A method wherein operating a fan of a wearable head device: detecting, with a microphone of the wearable head device, noise generated by the fan generating a fan reference signal. wherein the fan reference signal represents at least one of a speed of the fan, a mode of the fan, a power output of the fan, and a phase of the fan; deriving a transfer function based on the fan reference signal and based further on the detected noise of the fan: generating a compensation signal based on the transfer function: and while operating the fan of the wearable head device, outputting, by a speaker of the wearable head device, an anti-noise signal, wherein the anti-noise signal is based on the compensation signal.
G10K 11/178 - Procédés ou dispositifs de protection contre le bruit ou les autres ondes acoustiques ou pour amortir ceux-ci, en général utilisant des effets d'interférenceMasquage du son par régénération électro-acoustique en opposition de phase des ondes acoustiques originales
15.
IMPROVED EDGE BLACKENING FOR WAVEGUIDE EYEPIECES FOR USE WITH VIRTUAL AND AUGMENTED REALITY DISPLAY SYSTEMS
Improved stacked waveguide assemblies, and methods of making the same. The stacked waveguide assemblies include a plurality of waveguides stacked together and configured to transmit image information to a user's eye. Each of the waveguides is bonded to adjacent waveguides using a stack adhesive applied between the adjacent waveguides proximate an edge of each waveguide. A thin blackening edge layer is applied around the edge of each waveguide and is configured to absorb substantially all visible light bouncing off the edge of each respective waveguide. This design improves image contrast without affecting image sharpness.
A head-mounted display system comprises a projector, one or more eyepieces, and left and right biasing optics. The one or more eyepieces include an incoupling optical element operable to receive virtual light generated by the projector, a waveguide operable to propagate the virtual light toward left and right portions of the one or more eyepieces, and left and right light distributing elements operable to output the virtual light toward left and right eyes of a user. The left and right biasing optics are operable to angle the virtual light outputted from the left and right light distributing elements such that a left virtual light origination point overlaps with a right virtual light origination point.
G02F 1/137 - Dispositifs ou dispositions pour la commande de l'intensité, de la couleur, de la phase, de la polarisation ou de la direction de la lumière arrivant d'une source lumineuse indépendante, p. ex. commutation, ouverture de porte ou modulationOptique non linéaire pour la commande de l'intensité, de la phase, de la polarisation ou de la couleur basés sur des cristaux liquides, p. ex. cellules d'affichage individuelles à cristaux liquides caractérisés par l'effet électro-optique ou magnéto-optique, p. ex. transition de phase induite par un champ, effet d'orientation, interaction entre milieu récepteur et matière additive ou diffusion dynamique
H04N 13/341 - Affichage pour le visionnement à l’aide de lunettes spéciales ou de visiocasques utilisant le multiplexage temporel
G02B 27/28 - Systèmes ou appareils optiques non prévus dans aucun des groupes , pour polariser
G02B 30/24 - Systèmes ou appareils optiques pour produire des effets tridimensionnels [3D], p. ex. des effets stéréoscopiques en fournissant des première et seconde images de parallaxe à chacun des yeux gauche et droit d’un observateur du type stéréoscopique impliquant un multiplexage temporel, p. ex. utilisant des obturateurs gauche et droit activés séquentiellement
A head mounted display system can include a camera, at least one waveguide, at least one coupling optical element that is configured such that light is coupled into said waveguide and guided therein, and at least one out-coupling element. The at least one out-coupling element can be configured to couple light that is guided within said waveguide out of said waveguide and direct said light to said camera. The at least one coupling element may comprise a diffractive optical element having optical power.
A display system may include a wearable display for rendering three-dimensional virtual image content that appears to be located in an environment of a user of the display. The relative positions of the display and one or more eyes of the user may not be in desired positions to receive, or register, image information outputted by the display. For example, the display-to-eye alignment may vary for different users and/or may change over time (e.g., as a user moves or as the display becomes displaced). The wearable device may determine a relative position and/or alignment between the display and the user's eyes by determining whether features of the eye are at certain vertical positions relative to the display. Based on the relative positions, the wearable device may determine if it is properly fitted to the user, and position render camera(s) accordingly to present virtual image content.
Systems and methods are disclosed for operating a head-mounted display system based on user perceptibility. The display system may be an augmented reality display system configured to provide virtual content on a plurality of depth planes by presenting the content with different amounts of wavefront divergence. Some embodiments include obtaining an image captured by an imaging device of the display system. Whether a threshold measure or more of motion blur is determined to be exhibited in one or more regions of the image. Based on a determination that the threshold measure or more of motion blur is exhibited in one or more regions of the image, one or more operating parameters of the wearable display are adjusted. Example operating parameter adjustments comprise adjusting the depth plane on which content is presented (e.g., by switching from a first depth plane to a second depth plane), adjusting a rendering quality, and adjusting power characteristics of the system.
G09G 5/00 - Dispositions ou circuits de commande de l'affichage communs à l'affichage utilisant des tubes à rayons cathodiques et à l'affichage utilisant d'autres moyens de visualisation
20.
POLYMER EYEPIECE ASSEMBLIES FOR AUGMENTED AND MIXED REALITY SYSTEMS
Fabrication of augmented reality (AR) and mixed reality (MR) polymer eyepiece assemblies and the resulting AR/MR polymer eyepiece assemblies may include one or more features, separately or in any appropriate combination, to compensate for expected deformation and to maintain substantially uniform gaps between polymer layers. Such features include fabricating polymer eyepiece assemblies with components having coefficients of thermal expansion (CTE) that are substantially the same; modifying the surface chemistry or structure of one or more polymer layers to increase hydrophobicity or omniphobicity of the polymer layer; disposing adhesive between adjacent polymer layers in continuous and/or extended configurations; and disposing microspheres of different sizes at selected locations between polymer layers.
The systems and methods described can include approaches to calibrate head-mounted displays for improved viewing experiences. Some methods include receiving data of a first target image associated with an undeformed state of a first eyepiece of a head-mounted display device; receiving data of a first captured image associated with deformed state of the first eyepiece of the head-mounted display device; determining a first transformation that maps the first captured image to the image; and applying the first transformation to a subsequent image for viewing on the first eyepiece of the head-mounted display device.
G06T 3/14 - Transformations pour le recalage d’images, p. ex. ajustement ou mappage pour l’alignement d’images
H04N 13/344 - Affichage pour le visionnement à l’aide de lunettes spéciales ou de visiocasques avec des visiocasques portant des affichages gauche et droit
H04N 23/54 - Montage de tubes analyseurs, de capteurs d'images électroniques, de bobines de déviation ou de focalisation
An apparatus for providing gaze tracking in a near-eye display. Certain examples provide an apparatus including a light modulator configured to receive light of a first range of wavelengths and generate an image beam therefrom. The light modulator is further configured to receive light of a second range of wavelengths and generate a probe beam therefrom. The apparatus also includes one or more light guides including one or more in-coupling element areas, and one or more out-coupling element areas. The one or more in-coupling diffractive element areas are configured to receive and in-couple the image beam and the probe beam into the one or more light guides. The one or more out-coupling element areas are configured to out-couple, from the one or more light guides: the image beam to a user's eye for user viewing, and the probe beam to the user's eye for detection of reflection therefrom.
A61B 3/113 - Appareils pour l'examen optique des yeuxAppareils pour l'examen clinique des yeux du type à mesure objective, c.-à-d. instruments pour l'examen des yeux indépendamment des perceptions ou des réactions du patient pour déterminer ou enregistrer le mouvement de l'œil
F21V 8/00 - Utilisation de guides de lumière, p. ex. dispositifs à fibres optiques, dans les dispositifs ou systèmes d'éclairage
H04N 23/11 - Caméras ou modules de caméras comprenant des capteurs d'images électroniquesLeur commande pour générer des signaux d'image à partir de différentes longueurs d'onde pour générer des signaux d'image à partir de longueurs d'onde de lumière visible et infrarouge
H04N 23/56 - Caméras ou modules de caméras comprenant des capteurs d'images électroniquesLeur commande munis de moyens d'éclairage
H04N 23/74 - Circuits de compensation de la variation de luminosité dans la scène en influençant la luminosité de la scène à l'aide de moyens d'éclairage
24.
METHOD AND SYSTEM FOR LARGE FIELD-OF-VIEW AUGMENTED REALITY WAVEGUIDE UTILIZING UNPOLARIZED LIGHT
An augmented reality optical system includes a source of virtual content and an eyepiece waveguide optically coupled to the source of virtual content. The eyepiece waveguide includes an incoupling diffractive element and an outcoupling diffractive optical element. At least one of the incoupling diffractive element or the outcoupling diffractive optical element comprises a polarization volume grating (PVG).
A head-mounted, near-eye display system comprises a stack of waveguides having integral spacers separating the waveguides. The waveguides may each include diffractive optical elements that are formed simultaneously with the spacers by imprinting or casting. The spacers are disposed on one or more major surfaces of the waveguides and define a distance between immediately adjacent waveguides. Adjacent waveguides may be bonded using adhesives on the spacers. The spacers may fit within indentations of overlying waveguides. In some cases, the spacers may form one or more walls of material substantially around a perimeter of an associated waveguide. Vent holes may be provided in the walls to allow gas flow into and out from an interior volume defined by the spacers. Debris trapping structures may be provided between two walls of spacers to trap and prevent debris from entering into the interior volume.
Methods, systems, and apparatus for performing bundling adjustment using epipolar constraints. A method includes receiving image data from a headset for a particular pose. The image data includes a first image from a first camera of the headset and a second image from a second camera of the headset. The method includes identifying at least one key point in a three-dimensional model of an environment at least partly represented in the first image and the second image and performing bundle adjustment. Bundle adjustment is performed by jointly optimizing a reprojection error for the at least one key point and an epipolar error for the at least one key point. Results of the bundle adjustment are used to perform at least one of (i) updating the three-dimensional model, (ii) determining a position of the headset at the particular pose, or (iii) determining extrinsic parameters of the first camera and second camera.
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G01C 21/00 - NavigationInstruments de navigation non prévus dans les groupes
G01C 21/16 - NavigationInstruments de navigation non prévus dans les groupes en utilisant des mesures de la vitesse ou de l'accélération exécutées à bord de l'objet navigantNavigation à l'estime en intégrant l'accélération ou la vitesse, c.-à-d. navigation par inertie
G01C 21/20 - Instruments pour effectuer des calculs de navigation
G02B 27/00 - Systèmes ou appareils optiques non prévus dans aucun des groupes ,
Examples of the disclosure describe systems and methods for sharing perspective views of virtual content. In an example method, a virtual object is presented, via a display, to a first user. A first perspective view of the virtual object is determined, wherein the first perspective view is based on a position of the virtual object and a position of the first user. The virtual object is presented, via a display, to a second user, wherein the virtual object is presented to the second user according to the first perspective view. A second perspective view of the virtual object is determined, wherein the second perspective view is based on an input from the first user. The virtual object is presented, via a display, to the second user, wherein presenting the virtual object to the second user comprises presenting a transition from the first perspective view to the second perspective view.
Various techniques pertaining to methods, systems, and computer program products a spatial persistence process that places a virtual object relative to a physical object for an extended-reality display device based at least in part upon a persistent coordinate frame (PCF). A determination is made to decide whether a drift is detected for the virtual object relative to the physical object. upon or after detection of the drift or deviation, the drift or deviation is corrected at least by updating a tracking map into an updated tracking map and further at least by updating the persistent coordinate frame (PCF) based at least in part upon the updated tracking map, wherein the persistent coordinate frame (PCF) comprises six degrees of freedom relative to the map coordinate system.
A beamsplitter can include a first surface with a diffractive optical element, a second surface normal to the first surface, and a beam splitting surface arranged at an angle to the second surface. The beamsplitter may be configured to illuminate the entire second surface in response to an input beam at the first surface.
An optical master is created by using a nanoimprint alignment layer to pattern a liquid crystal (LC) layer. The nanoimprint alignment layer and the LC layer constitute the optical master. The optical master is positioned above a photoalignment layer. The optical master is illuminated and light propagating through the nanoimprinted alignment layer and the LC layer is diffracted and subsequently strikes the photo-alignment layer. The incident diffracted light causes the pattern in the LC layer to be transferred to the photo-alignment layer. A second LC layer is deposited onto the patterned photo-alignment layer, which subsequently is used to align the molecules of the second LC layer. The second LC layer in the patterned photo-alignment layer may be utilized as a replica optical master or as a diffractive optical element for directing light in optical devices such as augmented reality display devices.
G03H 1/02 - Procédés ou appareils holographiques utilisant la lumière, les infrarouges ou les ultraviolets pour obtenir des hologrammes ou pour en obtenir une imageLeurs détails spécifiques Détails
32.
ELECTROMAGNETIC TRACKING WITH AUGMENTED REALITY SYSTEMS
Head-mounted augmented reality (AR) devices can track pose of a wearer's head to provide a three-dimensional virtual representation of objects in the wearer's environment. An electromagnetic (EM) tracking system can track head or body pose. A handheld user input device can include an EM emitter that generates an EM field, and the head-mounted AR device can include an EM sensor that senses the EM field. EM information from the sensor can be analyzed to determine location and/or orientation of the sensor and thereby the wearer's pose. The EM emitter and sensor may utilize time division multiplexing (TDM) or dynamic frequency tuning to operate at multiple frequencies. Voltage gain control may be implemented in the transmitter, rather than the sensor, allowing smaller and lighter weight sensor designs. The EM sensor can implement noise cancellation to reduce the level of EM interference generated by nearby audio speakers.
G01S 1/68 - Marqueur, balise d'extrémité, indicatif d'appel ou toutes balises analogues transmettant des signaux ne portant pas d'information directionnelle
G01S 1/70 - Radiophares ou systèmes de balisage émettant des signaux ayant une ou des caractéristiques pouvant être détectées par des récepteurs non directionnels et définissant des directions, situations ou lignes de position déterminées par rapport aux émetteurs de radiophareRécepteurs travaillant avec ces systèmes utilisant des ondes électromagnétiques autres que les ondes radio
G01S 5/02 - Localisation par coordination de plusieurs déterminations de direction ou de ligne de positionLocalisation par coordination de plusieurs déterminations de distance utilisant les ondes radioélectriques
G02B 27/00 - Systèmes ou appareils optiques non prévus dans aucun des groupes ,
An eyepiece includes a substrate and an in-coupling grating patterned on a single side of the substrate. A first grating coupler is patterned on the single side of the substrate and has a first grating pattern. The first grating coupler is optically coupled to the in-coupling grating. A second grating coupler is patterned on the single side of the substrate adjacent to the first grating coupler. The second grating coupler has a second grating pattern different from the first grating pattern. The second grating coupler is optically coupled to the in-coupling grating.
G02B 6/293 - Moyens de couplage optique ayant des bus de données, c.-à-d. plusieurs guides d'ondes interconnectés et assurant un système bidirectionnel par nature en mélangeant et divisant les signaux avec des moyens de sélection de la longueur d'onde
This disclosure describes techniques for manufacturing waveguides that include spacer(s) on at least one surface of the waveguide, such that the spacers maintain mechanical stability and separation between the waveguides when the waveguides as assembled into a waveguide stack that is usable as an optical device. The disclosure also describes the various implementations of waveguides and optical devices that include spacers. The spacers may be created using a drop dispenser, in which drops of a (e.g., polymer) fluid are dispensed onto at least one surface of a substrate to be used as a waveguide. After being dispensed, the fluid drops can be cured to create the final, solidified spacers. Curing may also be performed in-flight before the drops reach the surface of the substrate. Partially cured drops may be stacked to create spacers of a particular height.
An augmented reality system includes a light source to generate a virtual light beam, the virtual light beam carrying information for a virtual object. The system also includes a light guiding optical element, the light guiding optical element allowing a first portion of a first real-world light beam to pass therethrough, where the virtual light beam enters the light guiding optical element, propagates through the light guiding optical element by substantially total internal reflection (TIR), and exits the light guiding optical element. The system further includes a lens disposed adjacent and exterior to a surface of the light guiding optical element, the lens comprising a light modulating mechanism to absorb a second portion of the real-world light beam and to allow the first portion of the real-world light to pass through the lens.
G02F 1/01 - Dispositifs ou dispositions pour la commande de l'intensité, de la couleur, de la phase, de la polarisation ou de la direction de la lumière arrivant d'une source lumineuse indépendante, p. ex. commutation, ouverture de porte ou modulationOptique non linéaire pour la commande de l'intensité, de la phase, de la polarisation ou de la couleur
G02F 1/1335 - Association structurelle de cellules avec des dispositifs optiques, p. ex. des polariseurs ou des réflecteurs
An image display system includes an optical subsystem configured to emit a modulated light beam, and a scanning mirror for generating a reflected light beam that is scanned according to randomly selected or pseudo-randomly selected scan patterns to generate multiple image fields of a multiple interlaced scan image. A plurality of different scan patterns can be cycled through, randomly or pseudo-randomly selected, for the different image fields to reduce artifacts that may be observed while viewing a projected image.
G02B 26/08 - Dispositifs ou dispositions optiques pour la commande de la lumière utilisant des éléments optiques mobiles ou déformables pour commander la direction de la lumière
G03B 21/00 - Projecteurs ou visionneuses du type par projectionLeurs accessoires
H04N 9/31 - Dispositifs de projection pour la présentation d'images en couleurs
An eyepiece waveguide for an augmented reality display system. The eyepiece waveguide can include an optically transmissive substrate with an input coupling grating (ICG) region. The ICG region can receive a beam of light and couple the beam into the substrate in a guided propagation mode. The eyepiece waveguide can also include a combined pupil expander-extractor (CPE) grating region that receives the beam of light from the ICG region and alters the propagation direction of the beam with a first interaction and out-couples the beam with a second interaction. The diffractive features of the CPE grating region can be arranged in rows and columns of alternating higher and lower quadrilateral surfaces or the diffractive features can comprise diamond shaped raised ridges. The eyepiece waveguide can also include one or more recycler grating regions.
Methods and systems for depth-based foveated rendering in a display system are disclosed. The display system may be an augmented reality display system configured to provide virtual content on a plurality of depth planes using different wavefront divergence. Some embodiments include monitoring eye orientations of a user of the display system. A fixation point can be determined based on the eye orientations, the fixation point representing a three-dimensional location with respect to a field of view. Location information of virtual object(s) to present is obtained, with the location information including three-dimensional position(s) of the virtual object(s). A resolution of the virtual object(s) can be adjusted based on a proximity of the location(s) of the virtual object(s) to the fixation point. The resolution can also be adjusted based on color, ambient illumination level, and/or other considerations. The virtual object(s) are presented by the display system according to the adjusted resolution(s).
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
H04N 13/279 - Générateurs de signaux d’images à partir de modèles 3D d’objets, p. ex. des signaux d’images stéréoscopiques générés par ordinateur les positions des points de vue virtuels étant choisies par les spectateurs ou déterminées par suivi
H04N 13/341 - Affichage pour le visionnement à l’aide de lunettes spéciales ou de visiocasques utilisant le multiplexage temporel
H04N 13/344 - Affichage pour le visionnement à l’aide de lunettes spéciales ou de visiocasques avec des visiocasques portant des affichages gauche et droit
H04N 13/383 - Suivi des spectateurs pour le suivi du regard, c.-à-d. avec détection de l’axe de vision des yeux du spectateur
H04N 13/395 - Affichages volumétriques, c.-à-d. systèmes où l’image est réalisée à partir d’éléments répartis dans un volume avec échantillonnage de la profondeur, c.-à-d. construction du volume à partir d’un ensemble ou d’une séquence de plans d’image 2D
Disclosed is an improved diffraction structure for 3D display systems. The improved diffraction structure includes an intermediate layer that resides between a waveguide substrate and a top grating surface. The top grating surface comprises a first material that corresponds to a first refractive index value, the underlayer comprises a second material that corresponds to a second refractive index value, and the substrate comprises a third material that corresponds to a third refractive index value. According to additional embodiments, improved approaches are provided to implement deposition of imprint materials onto a substrate, which allow for very precise distribution and deposition of different imprint patterns onto any number of substrate surfaces.
A head-mounted display system includes: a head mounted display frame; a first eyepiece supported by the frame, the first eyepiece including a first substrate composed of a crystalline, transparent material having crystallographic axes in a first orientation with respect to the frame, the substrate having a first surface and a second surface opposite the first surface, the first eyepiece further including a first in-coupling element including a grating on the first surface, and a first out-coupling element including a grating on the first surface and/or a grating on the second surface; and a second eyepiece including a second substrate composed of the crystalline, transparent material having crystallographic axes in a second orientation with respect to the frame different from the first orientation, a second in-coupling element on either surface of the second substrate, and a second out-coupling element on either surface of the second substrate.
G02B 1/02 - Éléments optiques caractérisés par la substance dont ils sont faitsRevêtements optiques pour éléments optiques faits de cristaux, p. ex. sel gemme, semi-conducteurs
41.
SYSTEMS AND METHODS FOR OPERATING A HEAD-MOUNTED DISPLAY SYSTEM BASED ON USER IDENTITY
Systems and methods for depth plane selection in display system such as augmented reality display systems, including mixed reality display systems, are disclosed. A display(s) may present virtual image content via image light to an eye(s) of a user. The display(s) may output the image light to the eye(s) of the user, the image light to have different amounts of wavefront divergence corresponding to different depth planes at different distances away from the user. A camera(s) may capture images of the eye(s). An indication may be generated based on obtained images of the eye(s), indicating whether the user is identified. The display(s) may be controlled to output the image light to the eye(s) of the user, the image light to have the different amounts of wavefront divergence based at least in part on the generated indication indicating whether the user is identified.
Waveguides comprising materials with refractive index greater than or equal to 1.8 and methods of patterning waveguides are disclosed. Patterned waveguides comprising materials with refractive index greater than or equal to 1.8 can be incorporated in display devices, such as, for example wearable display devices to project virtual images to a viewer.
A plurality of waveguide display substrates, each waveguide display substrate having a cylindrical portion having a diameter and a planar surface, a curved portion opposite the planar surface defining a nonlinear change in thickness across the substrate and having a maximum height D with respect to the cylindrical portion, and a wedge portion between the cylindrical portion and the curved portion defining a linear change in thickness across the substrate and having a maximum height W with respect to the cylindrical portion. A target maximum height Dt of the curved portion is 10−7 to 10−6 times the diameter, D is between about 70% and about 130% of Dt, and W is less than about 30% of Dt.
G02B 6/13 - Circuits optiques intégrés caractérisés par le procédé de fabrication
F21V 8/00 - Utilisation de guides de lumière, p. ex. dispositifs à fibres optiques, dans les dispositifs ou systèmes d'éclairage
G02B 6/12 - Guides de lumièreDétails de structure de dispositions comprenant des guides de lumière et d'autres éléments optiques, p. ex. des moyens de couplage du type guide d'ondes optiques du genre à circuit intégré
G02B 6/122 - Éléments optiques de base, p. ex. voies de guidage de la lumière
G02B 26/08 - Dispositifs ou dispositions optiques pour la commande de la lumière utilisant des éléments optiques mobiles ou déformables pour commander la direction de la lumière
44.
VIRTUAL AND AUGMENTED REALITY SYSTEMS AND METHODS USING DISPLAY SYSTEM CONTROL INFORMATION EMBEDDED IN IMAGE DATA
A display system, such as a virtual reality or augmented reality display system, can control a display to present image data including a plurality of color components, on a plurality of depth planes supported by the display. The presentation of the image data through the display can be controlled based on control information that is embedded in the image data, for example to activate or inactivate a color component and/or a depth plane. In some examples, light sources and/or spatial light modulators that relay illumination from the light sources may receive signals from a display controller to adjust a power setting to the light source or spatial light modulator based on control information embedded in an image data frame.
Described herein are systems and methods that provide localized dimming of world light emanating from world light sources. An optical system can include left and right dimmers. The optical system can also include left and right cameras configured to capture a left and right brightness images. The optical system can generate a 3D brightness source map based on the left and right brightness images, and generate left and right 2D brightness maps based on the 3D brightness source map. The optical can compute left and right dimming values for the left and right dimmers based on the left and right 2D brightness maps, and adjust the left and right dimmers to reduce an intensity of the world light.
A two-dimensional waveguide light multiplexer can efficiently multiplex and distribute a light signal in two dimensions. An example of a two-dimensional waveguide light multiplexer can include a waveguide, a first diffraction grating, and a second diffraction grating arranged such that the grating direction of the first diffraction grating is perpendicular to the grating direction of the second diffraction grating. In some examples, the first and second diffraction gratings are on opposite sides of a waveguide. In some examples, the first and second diffraction gratings are on a same side of a waveguide, with the second grating over the first grating.
G02F 1/00 - Dispositifs ou dispositions pour la commande de l'intensité, de la couleur, de la phase, de la polarisation ou de la direction de la lumière arrivant d'une source lumineuse indépendante, p. ex. commutation, ouverture de porte ou modulationOptique non linéaire
G02F 1/1335 - Association structurelle de cellules avec des dispositifs optiques, p. ex. des polariseurs ou des réflecteurs
47.
MISCALIBRATION DETECTION FOR VIRTUAL REALITY AND AUGMENTED REALITY SYSTEMS
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing miscalibration detection. One of the methods includes receiving sensor data from each of multiple sensors of a device in a system configured to provide augmented reality or mixed reality output to a user. Feature values are determined based on the sensor data for a predetermined set of features. The determined feature values are processed using a miscalibration detection model that has been trained, based on examples of captured sensor data from one or more devices, to predict whether a miscalibration condition of one or more of the multiple sensors has occurred. Based on the output of the miscalibration detection model, the system determines whether to initiate recalibration of extrinsic parameters for at least one of the multiple sensors or to bypass recalibration.
G06N 3/126 - Algorithmes évolutionnaires, p. ex. algorithmes génétiques ou programmation génétique
G06N 7/01 - Modèles graphiques probabilistes, p. ex. réseaux probabilistes
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
G06V 10/44 - Extraction de caractéristiques locales par analyse des parties du motif, p. ex. par détection d’arêtes, de contours, de boucles, d’angles, de barres ou d’intersectionsAnalyse de connectivité, p. ex. de composantes connectées
G06V 20/70 - Étiquetage du contenu de scène, p. ex. en tirant des représentations syntaxiques ou sémantiques
H04N 13/239 - Générateurs de signaux d’images utilisant des caméras à images stéréoscopiques utilisant deux capteurs d’images 2D dont la position relative est égale ou en correspondance à l’intervalle oculaire
H04N 13/271 - Générateurs de signaux d’images où les signaux d’images générés comprennent des cartes de profondeur ou de disparité
H04N 13/344 - Affichage pour le visionnement à l’aide de lunettes spéciales ou de visiocasques avec des visiocasques portant des affichages gauche et droit
48.
METHOD AND SYSTEM FOR AUGMENTED REALITY DISPLAY WITH GEOMETRIC-PHASE LENSES
A method of operating an augmented reality display having a world side and a user side includes receiving world light incident on the augmented reality display from the world side, linearly polarizing the world light to produce first linearly polarized light characterized by a first polarization state, and rotating the first linearly polarized light to produce second linearly polarized light characterized by a second polarization state orthogonal to the first polarization state. The method also includes converting the second linearly polarized light to first circularly polarized light having a first handedness, converting the first circularly polarized light to second circularly polarized light having a second handedness, converting the second circularly polarized light to the second linearly polarized light; and blocking the second linearly polarized light.
A display system is configured to direct a plurality of parallactically-disparate intra-pupil images into a viewer's eye. The parallactically-disparate intra-pupil images provide different parallax views of a virtual object, and impinge on the pupil from different angles. The wavefronts of light forming the images approximate a continuous divergent wavefront and provide selectable accommodation cues for the user, depending on the amount of parallax disparity between the intra-pupil images. The images may be formed by an emissive micro-display. Each pixel formed by the micro-display may be formed by one of a group of light emitters, which are at different locations such that the emitted light takes different paths to the eye to provide different amounts of parallax disparity.
G02B 30/24 - Systèmes ou appareils optiques pour produire des effets tridimensionnels [3D], p. ex. des effets stéréoscopiques en fournissant des première et seconde images de parallaxe à chacun des yeux gauche et droit d’un observateur du type stéréoscopique impliquant un multiplexage temporel, p. ex. utilisant des obturateurs gauche et droit activés séquentiellement
G09G 3/00 - Dispositions ou circuits de commande présentant un intérêt uniquement pour l'affichage utilisant des moyens de visualisation autres que les tubes à rayons cathodiques
H04N 13/344 - Affichage pour le visionnement à l’aide de lunettes spéciales ou de visiocasques avec des visiocasques portant des affichages gauche et droit
Examples of wearable systems and methods can use multiple inputs (e.g., gesture, head pose, eye gaze, voice, and/or environmental factors (e.g., location)) to determine a command that should be executed and objects in the three-dimensional (3D) environment that should be operated on. The multiple inputs can also be used by the wearable system to permit a user to interact with text, such as, e.g., composing, selecting, or editing text.
An eyepiece waveguide for an augmented reality display system includes an optically transmissive substrate, a first in-coupling grating (ICG) region, a second ICG region and one or more pupil expander and extraction gratings. The first ICG region can receive input beams of light corresponding to a first color component of an input image, and can couple them into the substrate. The second ICG region can receive input beams of light corresponding to a second color component of the input image, and can couple them into the substrate. The pupil expander and extraction gratings can replicate the in-coupled beams and out-couple them from the substrate. The first and second ICG regions can be provided at angularly separated locations around the substrate. The eyepiece waveguide can be capable of reducing color distortion in an output image.
G02B 1/02 - Éléments optiques caractérisés par la substance dont ils sont faitsRevêtements optiques pour éléments optiques faits de cristaux, p. ex. sel gemme, semi-conducteurs
G02B 1/04 - Éléments optiques caractérisés par la substance dont ils sont faitsRevêtements optiques pour éléments optiques faits de substances organiques, p. ex. plastiques
52.
IMAGING MODIFICATION, DISPLAY AND VISUALIZATION USING AUGMENTED AND VIRTUAL REALITY EYEWEAR
A display system can include a head-mounted display configured to project light to an eye of a user to display augmented reality image content to the user. The display system can include one or more user sensors configured to sense the user and can include one or more environmental sensors configured to sense surroundings of the user. The display system can also include processing electronics in communication with the display, the one or more user sensors, and the one or more environmental sensors. The processing electronics can be configured to sense a situation involving user focus, determine user intent for the situation, and alter user perception of a real or virtual object within the vision field of the user based at least in part on the user intent and/or sensed situation involving user focus. The processing electronics can be configured to at least one of enhance or de-emphasize the user perception of the real or virtual object within the vision field of the user.
A61B 17/00 - Instruments, dispositifs ou procédés chirurgicaux
A61B 34/00 - Chirurgie assistée par ordinateurManipulateurs ou robots spécialement adaptés à l’utilisation en chirurgie
A61B 34/20 - Systèmes de navigation chirurgicaleDispositifs pour le suivi ou le guidage d'instruments chirurgicaux, p. ex. pour la stéréotaxie sans cadre
A61B 90/00 - Instruments, outillage ou accessoires spécialement adaptés à la chirurgie ou au diagnostic non couverts par l'un des groupes , p. ex. pour le traitement de la luxation ou pour la protection de bords de blessures
A61B 90/50 - Supports pour instruments chirurgicaux, p. ex. bras articulés
B60K 35/10 - Dispositions d'entrée, c.-à-d. de l'utilisateur au véhicule, associées aux fonctions du véhicule ou spécialement adaptées à celles-ci
B60K 35/26 - Dispositions de sortie, c.-à-d. du véhicule à l'utilisateur, associées aux fonctions du véhicule ou spécialement adaptées à celles-ci utilisant une sortie acoustique
B60K 35/28 - Dispositions de sortie, c.-à-d. du véhicule à l'utilisateur, associées aux fonctions du véhicule ou spécialement adaptées à celles-ci caractérisées par le type d’informations de sortie, p. ex. divertissement vidéo ou informations sur la dynamique du véhiculeDispositions de sortie, c.-à-d. du véhicule à l'utilisateur, associées aux fonctions du véhicule ou spécialement adaptées à celles-ci caractérisées par la finalité des informations de sortie, p. ex. pour attirer l'attention du conducteur
B60K 35/60 - Instruments caractérisés par leur emplacement ou leur positionnement relatif dans ou sur les véhicules
B60K 35/80 - Dispositions pour la commande des instruments
B60K 35/90 - Étalonnage d'instruments, p. ex. réglage des paramètres initiaux ou de référenceTest d'instruments, p. ex. détection d'un dysfonctionnement
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06F 3/03 - Dispositions pour convertir sous forme codée la position ou le déplacement d'un élément
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
53.
METHODS AND SYSTEMS FOR DISPLAYING STEREOSCOPY WITH A FREEFORM OPTICAL SYSTEM WITH ADDRESSABLE FOCUS FOR VIRTUAL AND AUGMENTED REALITY
Several unique hardware configurations and methods for freeform optical display systems are disclosed. A freeform display system includes primary freeform optical element(s) and secondary freeform optical element(s) in tiled arrangements to expand the horizontal field of view (FOV) or the vertical field of view. The system may include a variable focusing system that produces intermediate pupil and changes the focal distance of a single focal plane or switches among multiple focal planes for rendering objects in focus while resolving accommodation-convergence conflict. The system may map light samples to appropriate light rays in physical space and use a cluster of projectors to project the mapped light rays to produce the light field of the virtual display content. Methods for making tiled freeform optical display systems and methods for producing virtual content with variable focus freeform optics and rendering light fields are also disclosed.
G02B 3/12 - Lentilles remplies d'un fluide ou à l'intérieur desquelles le vide a été fait
G02B 3/14 - Lentilles remplies d'un fluide ou à l'intérieur desquelles le vide a été fait à distance focale variable
G02B 26/08 - Dispositifs ou dispositions optiques pour la commande de la lumière utilisant des éléments optiques mobiles ou déformables pour commander la direction de la lumière
G02B 30/10 - Systèmes ou appareils optiques pour produire des effets tridimensionnels [3D], p. ex. des effets stéréoscopiques en utilisant des méthodes d'imagerie intégrale
A thin transparent layer can be integrated in a head mounted display device and disposed in front of the eye of a wearer. The thin transparent layer may be configured to output light such that light is directed onto the eye to create reflections therefrom that can be used, for example, for glint based tracking. The thin transparent layer can be configured to reduced obstructions in the field of the view of the user.
A computer implemented method of facilitating communication between first and second users includes displaying, by a first head-worn device, a first virtual object to the first user first user wearing the first head-worn device. The method also includes displaying, by a second head-worn device, a second virtual object to the second user wearing the second head-worn device. The method further includes facilitating, by the first and second head-worn devices, communications between the first and second users using the first and second virtual objects to simulate the first and second users being present in a common environment.
A host device having a first processor executes an application via the first processor. The host device determines a state of the application. A scenegraph is generated corresponding to the state of the application, and the scenegraph is presented to a remote device having a display and a second processor. The remote device is configured to, in response to receiving the scenegraph, render to the display a view corresponding to the scenegraph, without executing the application via the second processor.
Recesses are formed on a front side and a rear side of a waveguide. A solid porogen material is spun onto the front side and the rear side and fills the recesses. First front and rear cap layers are then formed on raised formations of the waveguide and on the solid porogen material. The entire structure is then heated and the solid porogen material decomposes to a porogen gas. The first front and rear cap layers are porous to allow the porogen gas to escape and air to enter into the recesses. The air maximizes a difference in refractive indices between the high-index transparent material of the waveguide and the air to promote reflection in the waveguide from interfaces between the waveguide and the air.
F21V 8/00 - Utilisation de guides de lumière, p. ex. dispositifs à fibres optiques, dans les dispositifs ou systèmes d'éclairage
G02B 6/10 - Guides de lumièreDétails de structure de dispositions comprenant des guides de lumière et d'autres éléments optiques, p. ex. des moyens de couplage du type guide d'ondes optiques
G02B 6/122 - Éléments optiques de base, p. ex. voies de guidage de la lumière
G02B 6/13 - Circuits optiques intégrés caractérisés par le procédé de fabrication
G02B 27/00 - Systèmes ou appareils optiques non prévus dans aucun des groupes ,
A method includes patterning a plurality of first trenches in a surface of a substrate; and etching the plurality of first trenches with an etchant having an etch rate for a first crystalline plane of the substrate that is greater than for a second crystalline plane of the substrate. The etching forms a slanted grating in the substrate.
Examples of systems and methods to facilitate audiovisual presence transitions of virtual objects such as virtual avatars in a mixed reality collaborative environment are disclosed. The systems and methods may be configured to produce different audiovisual presence transitions such as appearance, disappearance and reappearance of the virtual avatars. The virtual avatar audiovisual transitions may be further indicated by various visual and sound effects of the virtual avatars. The transitions may occur based on various colocation or decolocation scenarios.
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for calibrating an augmented reality device using camera and inertial measurement unit data. In some implementations, a bundle adjustment process jointly optimizes or estimates states of the augmented reality device. The process can use, as input, visual and inertial measurements as well as factory-calibrated sensor extrinsic parameters. The process performs bundle adjustment and uses non-linear optimization of estimated states constrained by the measurements and the factory calibrated extrinsic parameters. The process can jointly optimize inertial constraints, IMU calibration, and camera calibrations. Output of the process can include most likely estimated states, such as data for a 3D map of an environment, a trajectory of the device, and/or updated extrinsic parameters of the visual and inertial sensors (e.g., cameras and IMUs).
An augmented reality device (ARD) can present virtual content which can provide enhanced experiences with the user's physical environment. For example, the ARD can detect a linkage between a person in the FOV of the ARD and a physical object (e.g., a document presented by the person) or detect linkages between the documents. The linkages may be used in identity verification or document verification.
Wearable systems and method for operation thereof incorporating headset and controller localization using headset cameras and controller fiducials are disclosed. A wearable system may include a headset and a controller. The wearable system may alternate between performing headset tracking and performing controller tracking by repeatedly capturing images using a headset camera of the headset during headset tracking frames and controller tracking frames. The wearable system may cause the headset camera to capture a first exposure image an exposure above a threshold and cause the headset camera to capture a second exposure image having an exposure below the threshold. The wearable system may determine a fiducial interval during which fiducials of the controller are to flash at a fiducial frequency and a fiducial period. The wearable system may cause the fiducials to flash during the fiducial interval in accordance with the fiducial frequency and the fiducial period.
Techniques for artifact mitigation in an optical system are disclosed. Light associated with a world object is received at the optical system, which is characterized by a world side and a user side. Light associated with a virtual image is projected onto an eyepiece of the optical system, causing a portion of the light associated with the virtual image to propagate toward the user side and light associated with an artifact image to propagate toward the world side. A dimmer of the optical system positioned between the world side and the eyepiece is adjusted to reduce an intensity of the light associated with the artifact image impinging on the dimmer and an intensity of the light associated with the world object impinging on the dimmer.
G02B 27/00 - Systèmes ou appareils optiques non prévus dans aucun des groupes ,
G02B 26/08 - Dispositifs ou dispositions optiques pour la commande de la lumière utilisant des éléments optiques mobiles ou déformables pour commander la direction de la lumière
Disclosed herein is a wearable display system for capturing retraining eye images of an eye of a user for retraining a neural network for eye tracking. The system captures retraining eye images using an image capture device when user interface (UI) events occur with respect to UI devices displayed at display locations of a display. The system can generate a retraining set comprising the retraining eye images and eye poses of the eye of the user in the retraining eye images (e.g., related to the display locations of the UI devices) and obtain a retrained neural network that is retrained using the retraining set.
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06F 3/0346 - Dispositifs de pointage déplacés ou positionnés par l'utilisateurLeurs accessoires avec détection de l’orientation ou du mouvement libre du dispositif dans un espace en trois dimensions [3D], p. ex. souris 3D, dispositifs de pointage à six degrés de liberté [6-DOF] utilisant des capteurs gyroscopiques, accéléromètres ou d’inclinaison
G06F 3/04815 - Interaction s’effectuant dans un environnement basé sur des métaphores ou des objets avec un affichage tridimensionnel, p. ex. modification du point de vue de l’utilisateur par rapport à l’environnement ou l’objet
Techniques related to the computation of gaze vectors of users of wearable devices are disclosed. A neural network may be trained through first and second training steps. The neural network may include a set of feature encoding layers and a plurality of sets of task-specific layers that each operate on an output of the set of feature encoding layers. During the first training step, a first image of a first eye may be provided to the neural network, eye segmentation data may be generated using the neural network, and the set of feature encoding layers may be trained. During the second training step, a second image of a second eye may be provided to the neural network, network output data may be generated using the neural network, and the plurality of sets of task-specific layers may be trained.
G06V 10/26 - Segmentation de formes dans le champ d’imageDécoupage ou fusion d’éléments d’image visant à établir la région de motif, p. ex. techniques de regroupementDétection d’occlusion
G06V 10/774 - Génération d'ensembles de motifs de formationTraitement des caractéristiques d’images ou de vidéos dans les espaces de caractéristiquesDispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant l’intégration et la réduction de données, p. ex. analyse en composantes principales [PCA] ou analyse en composantes indépendantes [ ICA] ou cartes auto-organisatrices [SOM]Séparation aveugle de source méthodes de Bootstrap, p. ex. "bagging” ou “boosting”
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
G06V 40/18 - Caractéristiques de l’œil, p. ex. de l’iris
A head mounted display system for displaying image content to a user comprises at least one display configured to be worn by a user to present virtual content to first and second eyes of a user, one or more inwardly facing sensors or camera configured to monitor one or both of the users eye and processing electronics. This head mounted display system is configured such that virtual content activity can be initiated and/or driven from eye inputs such as gaze direction, eyelid motions (e.g., blinking), and/or other eye gestures.
An optical device such as an augmented reality (AR) display device includes variable optical material that alters at least one of: incident ambient light, spectral content of incident ambient light or direction of incident ambient light through the optical device in response to a stimulus provided by the device. The device can sense intensity and/or spectral characteristics of ambient light and provide appropriate stimulus to various portions of the optical device to activate the variable optical material and alter at least one of: incident ambient light, spectral content of incident ambient light or direction of incident ambient light. In some examples, the variable optical material may be distributed unevenly across the optical device.
F21V 8/00 - Utilisation de guides de lumière, p. ex. dispositifs à fibres optiques, dans les dispositifs ou systèmes d'éclairage
G02B 3/14 - Lentilles remplies d'un fluide ou à l'intérieur desquelles le vide a été fait à distance focale variable
G02B 27/00 - Systèmes ou appareils optiques non prévus dans aucun des groupes ,
G02F 1/00 - Dispositifs ou dispositions pour la commande de l'intensité, de la couleur, de la phase, de la polarisation ou de la direction de la lumière arrivant d'une source lumineuse indépendante, p. ex. commutation, ouverture de porte ou modulationOptique non linéaire
G02F 1/01 - Dispositifs ou dispositions pour la commande de l'intensité, de la couleur, de la phase, de la polarisation ou de la direction de la lumière arrivant d'une source lumineuse indépendante, p. ex. commutation, ouverture de porte ou modulationOptique non linéaire pour la commande de l'intensité, de la phase, de la polarisation ou de la couleur
G02F 1/29 - Dispositifs ou dispositions pour la commande de l'intensité, de la couleur, de la phase, de la polarisation ou de la direction de la lumière arrivant d'une source lumineuse indépendante, p. ex. commutation, ouverture de porte ou modulationOptique non linéaire pour la commande de la position ou de la direction des rayons lumineux, c.-à-d. déflexion
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
Systems and methods for generating a face model for a user of a head-mounted device are disclosed. The head-mounted device can include one or more eye cameras configured to image the face of the user while the user is putting the device on or taking the device off. The images obtained by the eye cameras may be analyzed using a stereoscopic vision technique, a monocular vision technique, or a combination, to generate a face model for the user. The face model can be used to generate a virtual image of at least a portion of the user's face, for example to be presented as an avatar.
In some embodiments, a display system comprising a head-mountable, augmented reality display is configured to perform a neurological analysis and to provide a perception aid based on an environmental trigger associated with the neurological condition. Performing the neurological analysis may include determining a reaction to a stimulus by receiving data from the one or more inwardly-directed sensors; and identifying a neurological condition associated with the reaction. In some embodiments, the perception aid may include a reminder, an alert, or virtual content that changes a property, e.g. a color, of a real object. The augmented reality display may be configured to display virtual content by outputting light with variable wavefront divergence, and to provide an accommodation-vergence mismatch of less than 0.5 diopters, including less than 0.25 diopters.
A display system may include a head-mounted display (HMD) for rendering a three-dimensional virtual object which appears to be located in an ambient environment of a user of the display. One or more eyes of the user may not be in desired positions, relative to the HMD, to receive, or register, image information outputted by the HMD and/or to view an external environment. For example, the HMD-to-eye alignment may vary for different users and/or may change over time (e.g., as the HMD is displaced). The display system may determine a relative position or alignment between the HMD and the user's eyes. Based on the relative positions, the wearable device may determine if it is properly fitted to the user, may provide feedback on the quality of the fit to the user, and/or may take actions to reduce or minimize effects of any misalignment.
G02B 27/00 - Systèmes ou appareils optiques non prévus dans aucun des groupes ,
A61B 3/11 - Appareils pour l'examen optique des yeuxAppareils pour l'examen clinique des yeux du type à mesure objective, c.-à-d. instruments pour l'examen des yeux indépendamment des perceptions ou des réactions du patient pour mesurer la distance interpupillaire ou le diamètre de la pupille
A61B 3/113 - Appareils pour l'examen optique des yeuxAppareils pour l'examen clinique des yeux du type à mesure objective, c.-à-d. instruments pour l'examen des yeux indépendamment des perceptions ou des réactions du patient pour déterminer ou enregistrer le mouvement de l'œil
G02B 30/00 - Systèmes ou appareils optiques pour produire des effets tridimensionnels [3D], p. ex. des effets stéréoscopiques
G02B 30/40 - Systèmes ou appareils optiques pour produire des effets tridimensionnels [3D], p. ex. des effets stéréoscopiques donnant à l’observateur d'une seule image bidimensionnelle [2D] une impression perceptive de profondeur
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06F 3/0346 - Dispositifs de pointage déplacés ou positionnés par l'utilisateurLeurs accessoires avec détection de l’orientation ou du mouvement libre du dispositif dans un espace en trois dimensions [3D], p. ex. souris 3D, dispositifs de pointage à six degrés de liberté [6-DOF] utilisant des capteurs gyroscopiques, accéléromètres ou d’inclinaison
G06F 3/04815 - Interaction s’effectuant dans un environnement basé sur des métaphores ou des objets avec un affichage tridimensionnel, p. ex. modification du point de vue de l’utilisateur par rapport à l’environnement ou l’objet
G06T 3/40 - Changement d'échelle d’images complètes ou de parties d’image, p. ex. agrandissement ou rétrécissement
G06V 10/42 - Extraction de caractéristiques globales par l’analyse du motif entier, p. ex. utilisant des transformations dans le domaine de fréquence ou d’autocorrélation
G06V 10/46 - Descripteurs pour la forme, descripteurs liés au contour ou aux points, p. ex. transformation de caractéristiques visuelles invariante à l’échelle [SIFT] ou sacs de mots [BoW]Caractéristiques régionales saillantes
G06V 10/60 - Extraction de caractéristiques d’images ou de vidéos relative aux propriétés luminescentes, p. ex. utilisant un modèle de réflectance ou d’éclairage
G06V 40/18 - Caractéristiques de l’œil, p. ex. de l’iris
H04N 13/344 - Affichage pour le visionnement à l’aide de lunettes spéciales ou de visiocasques avec des visiocasques portant des affichages gauche et droit
H04N 13/383 - Suivi des spectateurs pour le suivi du regard, c.-à-d. avec détection de l’axe de vision des yeux du spectateur
A wearable device can include an inward-facing imaging system configured to acquire images of a user's periocular region. The wearable device can determine a relative position between the wearable device and the user's face based on the images acquired by the inward-facing imaging system. The relative position may be used to determine whether the user is wearing the wearable device, whether the wearable device is optimally fit to the user, and/or whether an adjustment to a rendering location of a virtual object can be made to compensate for a deviation of the wearable device from its normal resting position relative to the user's face.
G06T 3/20 - Translation linéaire d’images complètes ou de parties d’image, p. ex. panoramique
G06T 11/60 - Édition de figures et de texteCombinaison de figures ou de texte
G06V 10/46 - Descripteurs pour la forme, descripteurs liés au contour ou aux points, p. ex. transformation de caractéristiques visuelles invariante à l’échelle [SIFT] ou sacs de mots [BoW]Caractéristiques régionales saillantes
G06V 20/20 - ScènesÉléments spécifiques à la scène dans les scènes de réalité augmentée
G06V 40/16 - Visages humains, p. ex. parties du visage, croquis ou expressions
G06V 40/18 - Caractéristiques de l’œil, p. ex. de l’iris
A virtual, augmented, or mixed reality display system includes a display configured to display virtual, augmented, or mixed reality image data, the display including one or more optical components which introduce optical distortions or aberrations to the image data. The system also includes a display controller configured to provide the image data to the display. The display controller includes memory for storing optical distortion correction information, and one or more processing elements to at least partially correct the image data for the optical distortions or aberrations using the optical distortion correction information.
A wearable display system includes one or more nanowire LED micro-displays. The nanowire micro-LED displays may be monochrome or full-color. The nanowire LEDs forming the arrays may have an advantageously narrow angular emission profile and high light output. Where a plurality of nanowire LED micro-displays is utilized, the micro-displays may be positioned at different sides of an optical combiner, for example, an X-cube prism which receives light rays from different micro-displays and outputs the light rays from the same face of the cube. The optical combiner directs the light to projection optics, which outputs the light to an eyepiece that relays the light to a user's eye. The eyepiece may output the light to the user's eye with different amounts of wavefront divergence, to place virtual content on different depth planes.
Disclosed herein are systems and methods for distributed computing and/or networking for mixed reality systems. A method may include capturing an image via a camera of a head-wearable device. Inertial data may be captured via an inertial measurement unit of the head-wearable device. A position of the head-wearable device can be estimated based on the image and the inertial data via one or more processors of the head-wearable device. The image can be transmitted to a remote server. A neural network can be trained based on the image via the remote server. A trained neural network can be transmitted to the head-wearable device.
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06F 3/14 - Sortie numérique vers un dispositif de visualisation
G06F 18/214 - Génération de motifs d'entraînementProcédés de Bootstrapping, p. ex. ”bagging” ou ”boosting”
G06V 10/764 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant la classification, p. ex. des objets vidéo
G06V 20/20 - ScènesÉléments spécifiques à la scène dans les scènes de réalité augmentée
G06V 40/16 - Visages humains, p. ex. parties du visage, croquis ou expressions
G06V 40/18 - Caractéristiques de l’œil, p. ex. de l’iris
Antireflection coatings for metasurfaces are described herein. In some embodiments, the metasurface may include a substrate, a plurality of nanostructures thereon, and an antireflection coating disposed over the nanostructures. The antireflection coating may be a transparent polymer, for example a photoresist layer, and may have a refractive index lower than the refractive index of the nanostructures and higher than the refractive index of the overlying medium (e.g., air). Advantageously, the antireflection coatings may reduce or eliminate ghost images in an augmented reality display in which the metasurface is incorporated.
Systems and methods for eye image segmentation and image quality estimation are disclosed. In one aspect, after receiving an eye image, a device such as an augmented reality device can process the eye image using a convolutional neural network with a merged architecture to generate both a segmented eye image and a quality estimation of the eye image. The segmented eye image can include a background region, a sclera region, an iris region, or a pupil region. In another aspect, a convolutional neural network with a merged architecture can be trained for eye image segmentation and image quality estimation. In yet another aspect, the device can use the segmented eye image to determine eye contours such as a pupil contour and an iris contour. The device can use the eye contours to create a polar image of the iris region for computing an iris code or biometric authentication.
G06F 18/2413 - Techniques de classification relatives au modèle de classification, p. ex. approches paramétriques ou non paramétriques basées sur les distances des motifs d'entraînement ou de référence
G06T 7/194 - DécoupageDétection de bords impliquant une segmentation premier plan-arrière-plan
G06V 10/44 - Extraction de caractéristiques locales par analyse des parties du motif, p. ex. par détection d’arêtes, de contours, de boucles, d’angles, de barres ou d’intersectionsAnalyse de connectivité, p. ex. de composantes connectées
G06V 10/56 - Extraction de caractéristiques d’images ou de vidéos relative à la couleur
G06V 10/764 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant la classification, p. ex. des objets vidéo
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
G06V 10/98 - Détection ou correction d’erreurs, p. ex. en effectuant une deuxième exploration du motif ou par intervention humaineÉvaluation de la qualité des motifs acquis
G06V 40/18 - Caractéristiques de l’œil, p. ex. de l’iris
77.
LIGHT OUTPUT SYSTEM WITH REFLECTOR AND LENS FOR HIGHLY SPATIALLY UNIFORM LIGHT OUTPUT
A user may interact and view virtual elements such as avatars and objects and/or real world elements in three-dimensional space in an augmented reality (AR) session. The system may allow one or more spectators to view from a stationary or dynamic camera a third person view of the users AR session. The third person view may be synchronized with the user view and the virtual elements of the user view may be composited onto the third person view.
F21V 13/04 - Combinaisons de deux sortes d'éléments uniquement les éléments étant des réflecteurs et des réfracteurs
F21V 8/00 - Utilisation de guides de lumière, p. ex. dispositifs à fibres optiques, dans les dispositifs ou systèmes d'éclairage
G02B 6/12 - Guides de lumièreDétails de structure de dispositions comprenant des guides de lumière et d'autres éléments optiques, p. ex. des moyens de couplage du type guide d'ondes optiques du genre à circuit intégré
G02B 23/06 - Télescopes ou lunettes d'approche, p. ex. jumellesPériscopesInstruments pour voir à l'intérieur de corps creuxViseursPointage optique ou appareils de visée comprenant des prismes ou des miroirs ayant une action de mise au point, p. ex. miroir parabolique
G02B 30/50 - Systèmes ou appareils optiques pour produire des effets tridimensionnels [3D], p. ex. des effets stéréoscopiques l’image étant construite à partir d'éléments d'image répartis sur un volume 3D, p. ex. des voxels
H04N 13/315 - Reproducteurs d’images pour visionnement sans avoir recours à des lunettes spéciales, c.-à-d. utilisant des affichages autostéréoscopiques utilisant des barrières de parallaxe les barrières de parallaxe variant dans le temps
H04N 13/344 - Affichage pour le visionnement à l’aide de lunettes spéciales ou de visiocasques avec des visiocasques portant des affichages gauche et droit
78.
SELECTING VIRTUAL OBJECTS IN A THREE-DIMENSIONAL SPACE
Systems and methods for interacting with virtual objects in a three-dimensional space using a wearable system are disclosed. The wearable system can be programmed to permit user interaction with interactable objects in a field of regard (FOR) of a user. The FOR includes a portion of the environment around the user that is capable of being perceived by the user via the AR system. The system can determine a group of interactable objects in the FOR of the user and determine a pose of the user. The system can update, based on a change in the pose or a field of view (FOV) of the user, a subgroup of the interactable objects that are located in the FOV of the user and receive a selection of a target interactable object from the subgroup of interactable objects. The system can initiate a selection event on the target interactable object.
G06F 3/0346 - Dispositifs de pointage déplacés ou positionnés par l'utilisateurLeurs accessoires avec détection de l’orientation ou du mouvement libre du dispositif dans un espace en trois dimensions [3D], p. ex. souris 3D, dispositifs de pointage à six degrés de liberté [6-DOF] utilisant des capteurs gyroscopiques, accéléromètres ou d’inclinaison
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06F 3/04815 - Interaction s’effectuant dans un environnement basé sur des métaphores ou des objets avec un affichage tridimensionnel, p. ex. modification du point de vue de l’utilisateur par rapport à l’environnement ou l’objet
G06F 3/0482 - Interaction avec des listes d’éléments sélectionnables, p. ex. des menus
G06F 3/04883 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] utilisant des caractéristiques spécifiques fournies par le périphérique d’entrée, p. ex. des fonctions commandées par la rotation d’une souris à deux capteurs, ou par la nature du périphérique d’entrée, p. ex. des gestes en fonction de la pression exercée enregistrée par une tablette numérique utilisant un écran tactile ou une tablette numérique, p. ex. entrée de commandes par des tracés gestuels pour l’entrée de données par calligraphie, p. ex. sous forme de gestes ou de texte
79.
ECLIPSE CURSOR FOR VIRTUAL CONTENT IN MIXED REALITY DISPLAYS
Systems and methods for displaying a cursor and a focus indicator associated with real or virtual objects in a virtual, augmented, or mixed reality environment by a wearable display device are disclosed. The system can determine a spatial relationship between a user-movable cursor and a target object within the environment. The system may render a focus indicator (e.g., a halo, shading, or highlighting) around or adjacent objects that are near the cursor. When the cursor overlaps with a target object, the system can render the object in front of the cursor (or not render the cursor at all), so the object is not occluded by the cursor. The object can be rendered closer to the user than the cursor. A group of virtual objects can be scrolled, and a virtual control panel can be displayed indicating objects that are upcoming in the scroll.
G06F 3/04817 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] fondées sur des propriétés spécifiques de l’objet d’interaction affiché ou sur un environnement basé sur les métaphores, p. ex. interaction avec des éléments du bureau telles les fenêtres ou les icônes, ou avec l’aide d’un curseur changeant de comportement ou d’aspect utilisant des icônes
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06F 3/04812 - Techniques d’interaction fondées sur l’aspect ou le comportement du curseur, p. ex. sous l’influence de la présence des objets affichés
G06F 3/04815 - Interaction s’effectuant dans un environnement basé sur des métaphores ou des objets avec un affichage tridimensionnel, p. ex. modification du point de vue de l’utilisateur par rapport à l’environnement ou l’objet
G06F 3/0485 - Défilement ou défilement panoramique
A method includes determining an eye gaze location of a user and generating a spatial foveation map based on the eye gaze location. The method also includes receiving an image, forming a spatially foveated image using the image and the spatial foveation map, and transmitting the spatially foveated image to a wearable device. The method further includes spatially defoveating the spatially foveated image to produce a spatially defoveated image and displaying the spatially defoveated image.
An augmented reality (AR) system includes a wearable device including: a frame, a projector coupled to the frame, a display optically coupled to the projector, and an eye tracking system. The AR system also includes a memory and a processor configured to: receive an eye gaze location from the eye tracking system, generate an image, and generate a foveation map based on the eye gaze location. The foveation map includes a first region of the image and a second region of the image. The processor is also configured to compress the first region of the image using a first quality setting and the second region of the image using a second quality setting. The first quality setting (e.g., a setting of 100%) can be greater than the second quality setting.
H04N 13/117 - Transformation de signaux d’images correspondant à des points de vue virtuels, p. ex. interpolation spatiale de l’image les positions des points de vue virtuels étant choisies par les spectateurs ou déterminées par suivi du spectateur
H04N 13/139 - Conversion du format, p. ex. du débit de trames ou de la taille
H04N 13/161 - Encodage, multiplexage ou démultiplexage de différentes composantes des signaux d’images
H04N 13/344 - Affichage pour le visionnement à l’aide de lunettes spéciales ou de visiocasques avec des visiocasques portant des affichages gauche et droit
H04N 13/383 - Suivi des spectateurs pour le suivi du regard, c.-à-d. avec détection de l’axe de vision des yeux du spectateur
An augmented reality (AR) device can be configured to monitor ambient audio data. The AR device can detect speech in the ambient audio data, convert the detected speech into text, or detect keywords such as rare words in the speech. When a rare word is detected, the AR device can retrieve auxiliary information (e.g., a definition) related to the rare word from a public or private source. The AR device can display the auxiliary information for a user to help the user better understand the speech. The AR device may perform translation of foreign speech, may display text (or the translation) of a speaker's speech to the user, or display statistical or other information associated with the speech.
G06F 40/58 - Utilisation de traduction automatisée, p. ex. pour recherches multilingues, pour fournir aux dispositifs clients une traduction effectuée par le serveur ou pour la traduction en temps réel
G06V 20/20 - ScènesÉléments spécifiques à la scène dans les scènes de réalité augmentée
A mixed reality virtual environment is sharable among multiple users through the use of multiple view modes that are selectable by a presenter. Multiple users with wearable display systems may wish to view a common virtual object, which may be presented in a virtual room to any suitable number of users. A presentation may be controlled by a presenter using a presenter wearable system that leads multiple participants through information associated with the virtual object. Use of different viewing modes allows individual users to see different virtual content through their wearable display systems, despite being in a shared viewing space or alternatively, to see the same virtual content in different locations within a shared space.
G09B 5/12 - Matériel à but éducatif à commande électrique avec présentation individuelle d'une information à une pluralité de postes d'élèves différents postes étant capables de présenter des informations différentes simultanément
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
H04L 12/18 - Dispositions pour la fourniture de services particuliers aux abonnés pour la diffusion ou les conférences
84.
METHOD AND SYSTEM FOR PERFORMING IMAGE REPROJECTION USING HEAD POSE INFORMATION
A method includes receiving, at an encoder, virtual content, receiving, at the encoder, a predicted head pose corresponding to the virtual content, and encoding the virtual content based on the predicted head pose. The method also includes producing compressed content, receiving, at a decoder, the compressed content, and receiving, at the decoder, a current head pose. The method also includes decoding the compressed content based on the current head pose and producing reprojected virtual content. In some embodiments, the method further includes compressing the reprojected virtual content to form one or more foveated images, decompressing the one or more foveated images to form a set of output images, and displaying the set of output images on a display.
H04N 13/383 - Suivi des spectateurs pour le suivi du regard, c.-à-d. avec détection de l’axe de vision des yeux du spectateur
H04N 13/139 - Conversion du format, p. ex. du débit de trames ou de la taille
H04N 13/344 - Affichage pour le visionnement à l’aide de lunettes spéciales ou de visiocasques avec des visiocasques portant des affichages gauche et droit
85.
TUNABLE CYLINDRICAL LENSES AND HEAD-MOUNTED DISPLAY INCLUDING THE SAME
Systems include three optical elements arranged along an optical axis each having a different cylinder axis and a variable cylinder refractive power. Collectively, the three elements form a compound optical element having an overall spherical refractive power (SPH), cylinder refractive power (CYL), and cylinder axis (Axis) that can be varied according to a prescription (Rx).
A head mounted display system can include at least one imaging device, a waveguide, and optical elements formed on or in the waveguide, including at least one coupling optical element configured to in-couple, into the waveguide, light from the environment, and at least one out-coupling optical element configured to out-couple, from the waveguide and toward the imaging device(s), the light from the environment, such that the imaging device(s) can image the environment based on the in-coupled light. The waveguide may also include an in-coupling optical element configured to couple, into the waveguide, image light that conveys virtual image content, and another out-coupling optical element that may be separate from the at least one coupling optical element and that is configured to couple the image light out of the waveguide toward the user's eye.
A method of producing a reprojected image includes receiving motion data and determining, based on the motion data, if a motion threshold is exceeded. The method also includes generating a depth-based reprojection if the motion threshold is exceeded or generating a non-depth-based reprojection if the motion threshold is not exceeded. In some embodiments, performing the foveated compression of the depth-based reprojection includes determining an eye gaze location of a user and generating a foveation map based on the eye gaze location. The foveation map includes a first region of the depth-based reprojection and a second region of the depth-based reprojection. Performing the foveated compression of the depth-based reprojection also includes compressing the first region using a first quality setting and the second region using a second quality setting.
Examples of eye-imaging apparatus using diffractive optical elements are provided. For example, an optical device comprises a substrate having a proximal surface and a distal surface, a first coupling optical element disposed on one of the proximal and distal surfaces of the substrate, and a second coupling optical element disposed on one of the proximal and distal surfaces of the substrate and offset from the first coupling optical element. The first coupling optical element can be configured to deflect light at an angle to totally internally reflect (TIR) the light between the proximal and distal surfaces and toward the second coupling optical element, and the second coupling optical element can be configured to deflect at an angle out of the substrate. The eye-imaging apparatus can be used in a head-mounted display such as an augmented or virtual reality display.
F21V 8/00 - Utilisation de guides de lumière, p. ex. dispositifs à fibres optiques, dans les dispositifs ou systèmes d'éclairage
G02B 27/00 - Systèmes ou appareils optiques non prévus dans aucun des groupes ,
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
H04N 13/332 - Affichage pour le visionnement à l’aide de lunettes spéciales ou de visiocasques
H04N 13/344 - Affichage pour le visionnement à l’aide de lunettes spéciales ou de visiocasques avec des visiocasques portant des affichages gauche et droit
H04N 13/383 - Suivi des spectateurs pour le suivi du regard, c.-à-d. avec détection de l’axe de vision des yeux du spectateur
89.
CONDITIONAL, PROBABILISTIC GENERATION OF 3D VIRTUAL ENVIRONMENT BASED ON INCOMPLETE INFORMATION AVAILABLE TO AN EXTENDED REALITY DEVICE
An extended reality system that includes a wearable eyepiece presenting virtual contents to a user, a belt pack operatively coupled to the wearable eyepiece, a processor, and a non-transitory computer readable medium storing thereupon a sequence of instructions which, when executed by a model with the processor, causes the processor to estimate a plurality of keyframes, a plurality of keyframe poses, and depth data from a plurality of captures that is captured by at least the extended reality device, to generate a semantically annotated, manipulatable three-dimensional (3D) representation in a physical environment for perception by the user wearing the wearable eyepiece, and to modify the semantically annotated, manipulatable 3D representation in real-time or nearly real-time in response to a user interaction.
G06T 7/55 - Récupération de la profondeur ou de la forme à partir de plusieurs images
G06T 17/00 - Modélisation tridimensionnelle [3D] pour infographie
G06V 10/762 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant le regroupement, p. ex. de visages similaires sur les réseaux sociaux
A head-mounted display system includes: a head mounted display frame; a first eyepiece supported by the frame, the first eyepiece including a first substrate composed of a crystalline, transparent material having crystallographic axes in a first orientation with respect to the frame, the substrate having a first surface and a second surface opposite the first surface, the first eyepiece further including a first in-coupling element including a grating on the first surface, and a first out-coupling element including a grating on the first surface and/or a grating on the second surface; and a second eyepiece including a second substrate composed of the crystalline, transparent material having crystallographic axes in a second orientation with respect to the frame different from the first orientation, a second in-coupling element on either surface of the second substrate, and a second out-coupling element on either surface of the second substrate.
G02B 1/02 - Éléments optiques caractérisés par la substance dont ils sont faitsRevêtements optiques pour éléments optiques faits de cristaux, p. ex. sel gemme, semi-conducteurs
The invention relates to a viewing system for use in a surgical environment. Various real object detection devices detect locations of real objects in a real environment, such as a patient and body part of patient, medical staff, robots, a cutting tool on a robot, implant transferred by robot into body part, surgical tools, and disposable items. A map generator generates a map that forms a digital representation or a digital twin of the real environment. Various guiding modules including a room setup module, an anatomy registration module, a surgical planning module, and a surgical execution module make use of the digital representation to guide virtual or real objects based on the digital representation.
A61B 34/20 - Systèmes de navigation chirurgicaleDispositifs pour le suivi ou le guidage d'instruments chirurgicaux, p. ex. pour la stéréotaxie sans cadre
A61B 90/00 - Instruments, outillage ou accessoires spécialement adaptés à la chirurgie ou au diagnostic non couverts par l'un des groupes , p. ex. pour le traitement de la luxation ou pour la protection de bords de blessures
A61B 90/50 - Supports pour instruments chirurgicaux, p. ex. bras articulés
A virtual reality (VR) and/or augmented reality (AR) display system is configured to control a display using control information that is embedded in or otherwise included with imagery data to be presented through the display. The control information can indicate depth plane(s) and/or color plane(s) to be used to present the imagery data, depth plane(s) and/or color plane(s) to be activated or inactivated, shift(s) of at least a portion of the imagery data (e.g., one or more pixels) laterally within a depth plane and/or longitudinally between depth planes, other adjustment(s) to the virtual imagery, and/or other suitable controls.
G06F 3/14 - Sortie numérique vers un dispositif de visualisation
G06T 7/579 - Récupération de la profondeur ou de la forme à partir de plusieurs images à partir du mouvement
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
G09G 3/00 - Dispositions ou circuits de commande présentant un intérêt uniquement pour l'affichage utilisant des moyens de visualisation autres que les tubes à rayons cathodiques
G09G 3/20 - Dispositions ou circuits de commande présentant un intérêt uniquement pour l'affichage utilisant des moyens de visualisation autres que les tubes à rayons cathodiques pour la présentation d'un ensemble de plusieurs caractères, p. ex. d'une page, en composant l'ensemble par combinaison d'éléments individuels disposés en matrice
G09G 5/00 - Dispositions ou circuits de commande de l'affichage communs à l'affichage utilisant des tubes à rayons cathodiques et à l'affichage utilisant d'autres moyens de visualisation
H04N 13/344 - Affichage pour le visionnement à l’aide de lunettes spéciales ou de visiocasques avec des visiocasques portant des affichages gauche et droit
H04N 13/395 - Affichages volumétriques, c.-à-d. systèmes où l’image est réalisée à partir d’éléments répartis dans un volume avec échantillonnage de la profondeur, c.-à-d. construction du volume à partir d’un ensemble ou d’une séquence de plans d’image 2D
Very high refractive index (n>2.2) lightguide substrates enable the production of 70° field of view eyepieces with all three color primaries in a single eyepiece layer. Disclosed herein are viewing optics assembly architectures that make use of such eyepieces to reduce size and cost, simplifying manufacturing and assembly, and better-accommodating novel microdisplay designs.
Exemplary systems and methods for creating spatial contents in a mixed reality environment are disclosed. In an example, a location associated with a first user in a coordinate space is determined. A persistent virtual content is generated. The persistent virtual content is associated with the first user's associated location. The first user's associated location is determined and is associated with the persistent virtual content. A location of a second user at a second time in the coordinate space is determined. The persistent virtual content is presented to the second user via a display at a location in the coordinate space corresponding to the first user's associated location.
A head-mounted display system is configured to project light to an eye of a user wearing the head-mounted display system to display content in a vision field of said user. The head-mounted display system comprises at least one diffusive optical element, at least one out-coupling optical element, at least one mask comprising at least one mask opening, at least one illumination in-coupling optical element configured to in-couple light from at least one illumination source into a light-guiding component, an image projector configured to in-couple an image and an at least one illumination source is configured to in-couple light into at least one illumination in-coupling optical element, an eyepiece, a curved light-guiding component, a light-guiding component comprising a portion of a frame, and/or two light-guiding components disposed on opposite sides of at least one out-coupling optical element.
A voice user interface (VUI) and methods for operating the VUI are disclosed. In some embodiments, the VUI configured to receive and process linguistic and non-linguistic inputs. For example, the VUI receives an audio signal, and the VUI determines whether the audio input comprises a linguistic and/or a non-linguistic input. In accordance with a determination that the audio signal comprises a non-linguistic input, the VUI causes a system to perform an action associated with the non-linguistic input.
G10L 15/18 - Classement ou recherche de la parole utilisant une modélisation du langage naturel
G10L 25/63 - Techniques d'analyse de la parole ou de la voix qui ne se limitent pas à un seul des groupes spécialement adaptées pour un usage particulier pour comparaison ou différentiation pour estimer un état émotionnel
97.
SYSTEMS AND METHOD FOR AUDIO PROCESSING IN EXTENDED REALITY
A method for presenting an extended reality experience to a user includes a display subsystem presenting images corresponding to image data to the user. The method also includes a microphone capturing input sound from an ambient acoustic environment and converting the captured input sound to input audio data. The method further includes an audio controller analyzing the input audio data to determine a masking level, rendering output audio data corresponding to source audio data, analyzing the output audio data to determine an energy level of the output audio data, comparing the energy level with the masking level to determine a comparison outcome, and modifying the output audio data based on the comparison outcome to raise the energy level of the output audio data to generate modified output audio data. The method also includes an audio subsystem presenting output sound corresponding to the modified output audio data to the user.
An optical device may include a light turning element. The optical device can include a first surface that is parallel to a horizontal axis and a second surface opposite to the first surface. The optical device may include a light module that includes a plurality of light emitters, and that can be configured to combine light from the emitters. The optical device can further include a light input surface that is between the first and the second surfaces and is disposed with respect to the light module to receive light. The optical device may include an end reflector that is disposed on a side opposite the light input surface. The light coupled into the light turning element may be reflected by the end reflector and/or reflected from the second surface towards the first surface.
G02B 27/00 - Systèmes ou appareils optiques non prévus dans aucun des groupes ,
G02B 27/14 - Systèmes divisant ou combinant des faisceaux fonctionnant uniquement par réflexion
G02B 30/26 - Systèmes ou appareils optiques pour produire des effets tridimensionnels [3D], p. ex. des effets stéréoscopiques en fournissant des première et seconde images de parallaxe à chacun des yeux gauche et droit d’un observateur du type autostéréoscopique
G02B 30/52 - Systèmes ou appareils optiques pour produire des effets tridimensionnels [3D], p. ex. des effets stéréoscopiques l’image étant construite à partir d'éléments d'image répartis sur un volume 3D, p. ex. des voxels le volume 3D étant construit à partir d'une pile ou d'une séquence de plans 2D, p. ex. systèmes d'échantillonnage en profondeur
G02F 1/137 - Dispositifs ou dispositions pour la commande de l'intensité, de la couleur, de la phase, de la polarisation ou de la direction de la lumière arrivant d'une source lumineuse indépendante, p. ex. commutation, ouverture de porte ou modulationOptique non linéaire pour la commande de l'intensité, de la phase, de la polarisation ou de la couleur basés sur des cristaux liquides, p. ex. cellules d'affichage individuelles à cristaux liquides caractérisés par l'effet électro-optique ou magnéto-optique, p. ex. transition de phase induite par un champ, effet d'orientation, interaction entre milieu récepteur et matière additive ou diffusion dynamique
G03B 21/00 - Projecteurs ou visionneuses du type par projectionLeurs accessoires
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06F 3/0482 - Interaction avec des listes d’éléments sélectionnables, p. ex. des menus
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
G06V 20/20 - ScènesÉléments spécifiques à la scène dans les scènes de réalité augmentée
G09G 3/02 - Dispositions ou circuits de commande présentant un intérêt uniquement pour l'affichage utilisant des moyens de visualisation autres que les tubes à rayons cathodiques par traçage ou balayage d'un faisceau lumineux sur un écran
G09G 3/24 - Dispositions ou circuits de commande présentant un intérêt uniquement pour l'affichage utilisant des moyens de visualisation autres que les tubes à rayons cathodiques pour la présentation d'un ensemble de plusieurs caractères, p. ex. d'une page, en composant l'ensemble par combinaison d'éléments individuels disposés en matrice utilisant des sources lumineuses commandées utilisant des filaments incandescents
H04N 13/239 - Générateurs de signaux d’images utilisant des caméras à images stéréoscopiques utilisant deux capteurs d’images 2D dont la position relative est égale ou en correspondance à l’intervalle oculaire
H04N 13/279 - Générateurs de signaux d’images à partir de modèles 3D d’objets, p. ex. des signaux d’images stéréoscopiques générés par ordinateur les positions des points de vue virtuels étant choisies par les spectateurs ou déterminées par suivi
H04N 13/344 - Affichage pour le visionnement à l’aide de lunettes spéciales ou de visiocasques avec des visiocasques portant des affichages gauche et droit
Head-mounted virtual and augmented reality display systems include a light projector with one or more emissive micro-displays having a first resolution and a pixel pitch. The projector outputs light forming frames of virtual content having at least a portion associated with a second resolution greater than the first resolution. The projector outputs light forming a first subframe of the rendered frame at the first resolution, and parts of the projector are shifted using actuators, such that physical positions of light output for individual pixels occupy gaps between the old locations of light output for individual pixels. The projector then outputs light forming a second subframe of the rendered frame. The first and second subframes are outputted within the flicker fusion threshold. Advantageously, an emissive micro-display (e.g., micro-LED display) having a low resolution can form a frame having a higher resolution by using the same light emitters to function as multiple pixels of that frame.
G02B 27/09 - Mise en forme du faisceau, p. ex. changement de la section transversale, non prévue ailleurs
G02B 27/10 - Systèmes divisant ou combinant des faisceaux
G02B 27/14 - Systèmes divisant ou combinant des faisceaux fonctionnant uniquement par réflexion
G02B 27/18 - Systèmes ou appareils optiques non prévus dans aucun des groupes , pour projection optique, p. ex. combinaison de miroir, de condensateur et d'objectif
G02B 27/40 - Moyens optiques auxiliaires pour mise au point
G02B 27/62 - Appareils optiques spécialement adaptés pour régler des éléments optiques pendant l'assemblage de systèmes optiques
G09G 3/00 - Dispositions ou circuits de commande présentant un intérêt uniquement pour l'affichage utilisant des moyens de visualisation autres que les tubes à rayons cathodiques
G09G 3/32 - Dispositions ou circuits de commande présentant un intérêt uniquement pour l'affichage utilisant des moyens de visualisation autres que les tubes à rayons cathodiques pour la présentation d'un ensemble de plusieurs caractères, p. ex. d'une page, en composant l'ensemble par combinaison d'éléments individuels disposés en matrice utilisant des sources lumineuses commandées utilisant des panneaux électroluminescents semi-conducteurs, p. ex. utilisant des diodes électroluminescentes [LED]
H02N 2/02 - Machines électriques en général utilisant l'effet piézo-électrique, l'électrostriction ou la magnétostriction produisant un mouvement linéaire, p. ex. actionneursPositionneurs linéaires
