An apparatus for providing gaze tracking in a near-eye display. Certain examples provide an apparatus including a light modulator configured to receive light of a first range of wavelengths and generate an image beam therefrom. The light modulator is further configured to receive light of a second range of wavelengths and generate a probe beam therefrom. The apparatus also includes one or more light guides including one or more in-coupling element areas, and one or more out-coupling element areas. The one or more in-coupling diffractive element areas are configured to receive and in-couple the image beam and the probe beam into the one or more light guides. The one or more out-coupling element areas are configured to out-couple, from the one or more light guides: the image beam to a user's eye for user viewing, and the probe beam to the user's eye for detection of reflection therefrom.
A61B 3/113 - Objective types, i.e. instruments for examining the eyes independent of the patients perceptions or reactions for determining or recording eye movement
F21V 8/00 - Use of light guides, e.g. fibre optic devices, in lighting devices or systems
H04N 23/11 - Cameras or camera modules comprising electronic image sensorsControl thereof for generating image signals from different wavelengths for generating image signals from visible and infrared light wavelengths
H04N 23/56 - Cameras or camera modules comprising electronic image sensorsControl thereof provided with illuminating means
H04N 23/74 - Circuitry for compensating brightness variation in the scene by influencing the scene brightness using illuminating means
2.
METHOD AND SYSTEM FOR LARGE FIELD-OF-VIEW AUGMENTED REALITY WAVEGUIDE UTILIZING UNPOLARIZED LIGHT
An augmented reality optical system includes a source of virtual content and an eyepiece waveguide optically coupled to the source of virtual content. The eyepiece waveguide includes an incoupling diffractive element and an outcoupling diffractive optical element. At least one of the incoupling diffractive element or the outcoupling diffractive optical element comprises a polarization volume grating (PVG).
A head-mounted, near-eye display system comprises a stack of waveguides having integral spacers separating the waveguides. The waveguides may each include diffractive optical elements that are formed simultaneously with the spacers by imprinting or casting. The spacers are disposed on one or more major surfaces of the waveguides and define a distance between immediately adjacent waveguides. Adjacent waveguides may be bonded using adhesives on the spacers. The spacers may fit within indentations of overlying waveguides. In some cases, the spacers may form one or more walls of material substantially around a perimeter of an associated waveguide. Vent holes may be provided in the walls to allow gas flow into and out from an interior volume defined by the spacers. Debris trapping structures may be provided between two walls of spacers to trap and prevent debris from entering into the interior volume.
Methods, systems, and apparatus for performing bundling adjustment using epipolar constraints. A method includes receiving image data from a headset for a particular pose. The image data includes a first image from a first camera of the headset and a second image from a second camera of the headset. The method includes identifying at least one key point in a three-dimensional model of an environment at least partly represented in the first image and the second image and performing bundle adjustment. Bundle adjustment is performed by jointly optimizing a reprojection error for the at least one key point and an epipolar error for the at least one key point. Results of the bundle adjustment are used to perform at least one of (i) updating the three-dimensional model, (ii) determining a position of the headset at the particular pose, or (iii) determining extrinsic parameters of the first camera and second camera.
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G01C 21/00 - NavigationNavigational instruments not provided for in groups
G01C 21/16 - NavigationNavigational instruments not provided for in groups by using measurement of speed or acceleration executed aboard the object being navigatedDead reckoning by integrating acceleration or speed, i.e. inertial navigation
G01C 21/20 - Instruments for performing navigational calculations
G02B 27/00 - Optical systems or apparatus not provided for by any of the groups ,
Examples of the disclosure describe systems and methods for sharing perspective views of virtual content. In an example method, a virtual object is presented, via a display, to a first user. A first perspective view of the virtual object is determined, wherein the first perspective view is based on a position of the virtual object and a position of the first user. The virtual object is presented, via a display, to a second user, wherein the virtual object is presented to the second user according to the first perspective view. A second perspective view of the virtual object is determined, wherein the second perspective view is based on an input from the first user. The virtual object is presented, via a display, to the second user, wherein presenting the virtual object to the second user comprises presenting a transition from the first perspective view to the second perspective view.
A beamsplitter can include a first surface with a diffractive optical element, a second surface normal to the first surface, and a beam splitting surface arranged at an angle to the second surface. The beamsplitter may be configured to illuminate the entire second surface in response to an input beam at the first surface.
Various techniques pertaining to methods, systems, and computer program products a spatial persistence process that places a virtual object relative to a physical object for an extended-reality display device based at least in part upon a persistent coordinate frame (PCF). A determination is made to decide whether a drift is detected for the virtual object relative to the physical object. upon or after detection of the drift or deviation, the drift or deviation is corrected at least by updating a tracking map into an updated tracking map and further at least by updating the persistent coordinate frame (PCF) based at least in part upon the updated tracking map, wherein the persistent coordinate frame (PCF) comprises six degrees of freedom relative to the map coordinate system.
An optical master is created by using a nanoimprint alignment layer to pattern a liquid crystal (LC) layer. The nanoimprint alignment layer and the LC layer constitute the optical master. The optical master is positioned above a photoalignment layer. The optical master is illuminated and light propagating through the nanoimprinted alignment layer and the LC layer is diffracted and subsequently strikes the photo-alignment layer. The incident diffracted light causes the pattern in the LC layer to be transferred to the photo-alignment layer. A second LC layer is deposited onto the patterned photo-alignment layer, which subsequently is used to align the molecules of the second LC layer. The second LC layer in the patterned photo-alignment layer may be utilized as a replica optical master or as a diffractive optical element for directing light in optical devices such as augmented reality display devices.
G03H 1/02 - Holographic processes or apparatus using light, infrared, or ultraviolet waves for obtaining holograms or for obtaining an image from themDetails peculiar thereto Details
10.
ELECTROMAGNETIC TRACKING WITH AUGMENTED REALITY SYSTEMS
Head-mounted augmented reality (AR) devices can track pose of a wearer's head to provide a three-dimensional virtual representation of objects in the wearer's environment. An electromagnetic (EM) tracking system can track head or body pose. A handheld user input device can include an EM emitter that generates an EM field, and the head-mounted AR device can include an EM sensor that senses the EM field. EM information from the sensor can be analyzed to determine location and/or orientation of the sensor and thereby the wearer's pose. The EM emitter and sensor may utilize time division multiplexing (TDM) or dynamic frequency tuning to operate at multiple frequencies. Voltage gain control may be implemented in the transmitter, rather than the sensor, allowing smaller and lighter weight sensor designs. The EM sensor can implement noise cancellation to reduce the level of EM interference generated by nearby audio speakers.
G01S 1/68 - Marker, boundary, call-sign, or like beacons transmitting signals not carrying directional information
G01S 1/70 - Beacons or beacon systems transmitting signals having a characteristic or characteristics capable of being detected by non-directional receivers and defining directions, positions, or position lines fixed relatively to the beacon transmittersReceivers co-operating therewith using electromagnetic waves other than radio waves
G01S 5/02 - Position-fixing by co-ordinating two or more direction or position-line determinationsPosition-fixing by co-ordinating two or more distance determinations using radio waves
G02B 27/00 - Optical systems or apparatus not provided for by any of the groups ,
G06F 1/16 - Constructional details or arrangements
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
An eyepiece includes a substrate and an in-coupling grating patterned on a single side of the substrate. A first grating coupler is patterned on the single side of the substrate and has a first grating pattern. The first grating coupler is optically coupled to the in-coupling grating. A second grating coupler is patterned on the single side of the substrate adjacent to the first grating coupler. The second grating coupler has a second grating pattern different from the first grating pattern. The second grating coupler is optically coupled to the in-coupling grating.
G02B 6/293 - Optical coupling means having data bus means, i.e. plural waveguides interconnected and providing an inherently bidirectional system by mixing and splitting signals with wavelength selective means
This disclosure describes techniques for manufacturing waveguides that include spacer(s) on at least one surface of the waveguide, such that the spacers maintain mechanical stability and separation between the waveguides when the waveguides as assembled into a waveguide stack that is usable as an optical device. The disclosure also describes the various implementations of waveguides and optical devices that include spacers. The spacers may be created using a drop dispenser, in which drops of a (e.g., polymer) fluid are dispensed onto at least one surface of a substrate to be used as a waveguide. After being dispensed, the fluid drops can be cured to create the final, solidified spacers. Curing may also be performed in-flight before the drops reach the surface of the substrate. Partially cured drops may be stacked to create spacers of a particular height.
An augmented reality system includes a light source to generate a virtual light beam, the virtual light beam carrying information for a virtual object. The system also includes a light guiding optical element, the light guiding optical element allowing a first portion of a first real-world light beam to pass therethrough, where the virtual light beam enters the light guiding optical element, propagates through the light guiding optical element by substantially total internal reflection (TIR), and exits the light guiding optical element. The system further includes a lens disposed adjacent and exterior to a surface of the light guiding optical element, the lens comprising a light modulating mechanism to absorb a second portion of the real-world light beam and to allow the first portion of the real-world light to pass through the lens.
G02F 1/01 - Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulatingNon-linear optics for the control of the intensity, phase, polarisation or colour
G02F 1/1335 - Structural association of cells with optical devices, e.g. polarisers or reflectors
An image display system includes an optical subsystem configured to emit a modulated light beam, and a scanning mirror for generating a reflected light beam that is scanned according to randomly selected or pseudo-randomly selected scan patterns to generate multiple image fields of a multiple interlaced scan image. A plurality of different scan patterns can be cycled through, randomly or pseudo-randomly selected, for the different image fields to reduce artifacts that may be observed while viewing a projected image.
G02B 26/08 - Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
G03B 21/00 - Projectors or projection-type viewersAccessories therefor
H04N 9/31 - Projection devices for colour picture display
An eyepiece waveguide for an augmented reality display system. The eyepiece waveguide can include an optically transmissive substrate with an input coupling grating (ICG) region. The ICG region can receive a beam of light and couple the beam into the substrate in a guided propagation mode. The eyepiece waveguide can also include a combined pupil expander-extractor (CPE) grating region that receives the beam of light from the ICG region and alters the propagation direction of the beam with a first interaction and out-couples the beam with a second interaction. The diffractive features of the CPE grating region can be arranged in rows and columns of alternating higher and lower quadrilateral surfaces or the diffractive features can comprise diamond shaped raised ridges. The eyepiece waveguide can also include one or more recycler grating regions.
Methods and systems for depth-based foveated rendering in a display system are disclosed. The display system may be an augmented reality display system configured to provide virtual content on a plurality of depth planes using different wavefront divergence. Some embodiments include monitoring eye orientations of a user of the display system. A fixation point can be determined based on the eye orientations, the fixation point representing a three-dimensional location with respect to a field of view. Location information of virtual object(s) to present is obtained, with the location information including three-dimensional position(s) of the virtual object(s). A resolution of the virtual object(s) can be adjusted based on a proximity of the location(s) of the virtual object(s) to the fixation point. The resolution can also be adjusted based on color, ambient illumination level, and/or other considerations. The virtual object(s) are presented by the display system according to the adjusted resolution(s).
G02B 27/00 - Optical systems or apparatus not provided for by any of the groups ,
G06T 15/00 - 3D [Three Dimensional] image rendering
G06T 19/00 - Manipulating 3D models or images for computer graphics
H04N 13/279 - Image signal generators from 3D object models, e.g. computer-generated stereoscopic image signals the virtual viewpoint locations being selected by the viewers or determined by tracking
H04N 13/341 - Displays for viewing with the aid of special glasses or head-mounted displays [HMD] using temporal multiplexing
H04N 13/344 - Displays for viewing with the aid of special glasses or head-mounted displays [HMD] with head-mounted left-right displays
H04N 13/383 - Image reproducers using viewer tracking for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
H04N 13/395 - Volumetric displays, i.e. systems where the image is built up from picture elements distributed through a volume with depth sampling, i.e. the volume being constructed from a stack or sequence of 2D image planes
Disclosed is an improved diffraction structure for 3D display systems. The improved diffraction structure includes an intermediate layer that resides between a waveguide substrate and a top grating surface. The top grating surface comprises a first material that corresponds to a first refractive index value, the underlayer comprises a second material that corresponds to a second refractive index value, and the substrate comprises a third material that corresponds to a third refractive index value. According to additional embodiments, improved approaches are provided to implement deposition of imprint materials onto a substrate, which allow for very precise distribution and deposition of different imprint patterns onto any number of substrate surfaces.
A head-mounted display system includes: a head mounted display frame; a first eyepiece supported by the frame, the first eyepiece including a first substrate composed of a crystalline, transparent material having crystallographic axes in a first orientation with respect to the frame, the substrate having a first surface and a second surface opposite the first surface, the first eyepiece further including a first in-coupling element including a grating on the first surface, and a first out-coupling element including a grating on the first surface and/or a grating on the second surface; and a second eyepiece including a second substrate composed of the crystalline, transparent material having crystallographic axes in a second orientation with respect to the frame different from the first orientation, a second in-coupling element on either surface of the second substrate, and a second out-coupling element on either surface of the second substrate.
G02B 1/02 - Optical elements characterised by the material of which they are madeOptical coatings for optical elements made of crystals, e.g. rock-salt, semiconductors
19.
SYSTEMS AND METHODS FOR OPERATING A HEAD-MOUNTED DISPLAY SYSTEM BASED ON USER IDENTITY
Systems and methods for depth plane selection in display system such as augmented reality display systems, including mixed reality display systems, are disclosed. A display(s) may present virtual image content via image light to an eye(s) of a user. The display(s) may output the image light to the eye(s) of the user, the image light to have different amounts of wavefront divergence corresponding to different depth planes at different distances away from the user. A camera(s) may capture images of the eye(s). An indication may be generated based on obtained images of the eye(s), indicating whether the user is identified. The display(s) may be controlled to output the image light to the eye(s) of the user, the image light to have the different amounts of wavefront divergence based at least in part on the generated indication indicating whether the user is identified.
Waveguides comprising materials with refractive index greater than or equal to 1.8 and methods of patterning waveguides are disclosed. Patterned waveguides comprising materials with refractive index greater than or equal to 1.8 can be incorporated in display devices, such as, for example wearable display devices to project virtual images to a viewer.
A plurality of waveguide display substrates, each waveguide display substrate having a cylindrical portion having a diameter and a planar surface, a curved portion opposite the planar surface defining a nonlinear change in thickness across the substrate and having a maximum height D with respect to the cylindrical portion, and a wedge portion between the cylindrical portion and the curved portion defining a linear change in thickness across the substrate and having a maximum height W with respect to the cylindrical portion. A target maximum height Dt of the curved portion is 10−7 to 10−6 times the diameter, D is between about 70% and about 130% of Dt, and W is less than about 30% of Dt.
G02B 6/13 - Integrated optical circuits characterised by the manufacturing method
F21V 8/00 - Use of light guides, e.g. fibre optic devices, in lighting devices or systems
G02B 6/12 - Light guidesStructural details of arrangements comprising light guides and other optical elements, e.g. couplings of the optical waveguide type of the integrated circuit kind
G02B 6/122 - Basic optical elements, e.g. light-guiding paths
G02B 26/08 - Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
22.
VIRTUAL AND AUGMENTED REALITY SYSTEMS AND METHODS USING DISPLAY SYSTEM CONTROL INFORMATION EMBEDDED IN IMAGE DATA
A display system, such as a virtual reality or augmented reality display system, can control a display to present image data including a plurality of color components, on a plurality of depth planes supported by the display. The presentation of the image data through the display can be controlled based on control information that is embedded in the image data, for example to activate or inactivate a color component and/or a depth plane. In some examples, light sources and/or spatial light modulators that relay illumination from the light sources may receive signals from a display controller to adjust a power setting to the light source or spatial light modulator based on control information embedded in an image data frame.
A two-dimensional waveguide light multiplexer can efficiently multiplex and distribute a light signal in two dimensions. An example of a two-dimensional waveguide light multiplexer can include a waveguide, a first diffraction grating, and a second diffraction grating arranged such that the grating direction of the first diffraction grating is perpendicular to the grating direction of the second diffraction grating. In some examples, the first and second diffraction gratings are on opposite sides of a waveguide. In some examples, the first and second diffraction gratings are on a same side of a waveguide, with the second grating over the first grating.
G02F 1/00 - Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulatingNon-linear optics
G02F 1/1335 - Structural association of cells with optical devices, e.g. polarisers or reflectors
24.
MISCALIBRATION DETECTION FOR VIRTUAL REALITY AND AUGMENTED REALITY SYSTEMS
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing miscalibration detection. One of the methods includes receiving sensor data from each of multiple sensors of a device in a system configured to provide augmented reality or mixed reality output to a user. Feature values are determined based on the sensor data for a predetermined set of features. The determined feature values are processed using a miscalibration detection model that has been trained, based on examples of captured sensor data from one or more devices, to predict whether a miscalibration condition of one or more of the multiple sensors has occurred. Based on the output of the miscalibration detection model, the system determines whether to initiate recalibration of extrinsic parameters for at least one of the multiple sensors or to bypass recalibration.
G06N 3/126 - Evolutionary algorithms, e.g. genetic algorithms or genetic programming
G06N 7/01 - Probabilistic graphical models, e.g. probabilistic networks
G06T 19/00 - Manipulating 3D models or images for computer graphics
G06V 10/44 - Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersectionsConnectivity analysis, e.g. of connected components
G06V 20/70 - Labelling scene content, e.g. deriving syntactic or semantic representations
H04N 13/00 - Stereoscopic video systemsMulti-view video systemsDetails thereof
H04N 13/239 - Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
A method of operating an augmented reality display having a world side and a user side includes receiving world light incident on the augmented reality display from the world side, linearly polarizing the world light to produce first linearly polarized light characterized by a first polarization state, and rotating the first linearly polarized light to produce second linearly polarized light characterized by a second polarization state orthogonal to the first polarization state. The method also includes converting the second linearly polarized light to first circularly polarized light having a first handedness, converting the first circularly polarized light to second circularly polarized light having a second handedness, converting the second circularly polarized light to the second linearly polarized light; and blocking the second linearly polarized light.
Described herein are systems and methods that provide localized dimming of world light emanating from world light sources. An optical system can include left and right dimmers. The optical system can also include left and right cameras configured to capture a left and right brightness images. The optical system can generate a 3D brightness source map based on the left and right brightness images, and generate left and right 2D brightness maps based on the 3D brightness source map. The optical can compute left and right dimming values for the left and right dimmers based on the left and right 2D brightness maps, and adjust the left and right dimmers to reduce an intensity of the world light.
A display system is configured to direct a plurality of parallactically-disparate intra-pupil images into a viewer's eye. The parallactically-disparate intra-pupil images provide different parallax views of a virtual object, and impinge on the pupil from different angles. The wavefronts of light forming the images approximate a continuous divergent wavefront and provide selectable accommodation cues for the user, depending on the amount of parallax disparity between the intra-pupil images. The images may be formed by an emissive micro-display. Each pixel formed by the micro-display may be formed by one of a group of light emitters, which are at different locations such that the emitted light takes different paths to the eye to provide different amounts of parallax disparity.
G02B 30/24 - Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer’s left and right eyes of the stereoscopic type involving temporal multiplexing, e.g. using sequentially activated left and right shutters
G09G 3/00 - Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
H04N 13/344 - Displays for viewing with the aid of special glasses or head-mounted displays [HMD] with head-mounted left-right displays
Examples of wearable systems and methods can use multiple inputs (e.g., gesture, head pose, eye gaze, voice, and/or environmental factors (e.g., location)) to determine a command that should be executed and objects in the three-dimensional (3D) environment that should be operated on. The multiple inputs can also be used by the wearable system to permit a user to interact with text, such as, e.g., composing, selecting, or editing text.
An eyepiece waveguide for an augmented reality display system includes an optically transmissive substrate, a first in-coupling grating (ICG) region, a second ICG region and one or more pupil expander and extraction gratings. The first ICG region can receive input beams of light corresponding to a first color component of an input image, and can couple them into the substrate. The second ICG region can receive input beams of light corresponding to a second color component of the input image, and can couple them into the substrate. The pupil expander and extraction gratings can replicate the in-coupled beams and out-couple them from the substrate. The first and second ICG regions can be provided at angularly separated locations around the substrate. The eyepiece waveguide can be capable of reducing color distortion in an output image.
G02B 1/02 - Optical elements characterised by the material of which they are madeOptical coatings for optical elements made of crystals, e.g. rock-salt, semiconductors
G02B 1/04 - Optical elements characterised by the material of which they are madeOptical coatings for optical elements made of organic materials, e.g. plastics
30.
IMAGING MODIFICATION, DISPLAY AND VISUALIZATION USING AUGMENTED AND VIRTUAL REALITY EYEWEAR
A display system can include a head-mounted display configured to project light to an eye of a user to display augmented reality image content to the user. The display system can include one or more user sensors configured to sense the user and can include one or more environmental sensors configured to sense surroundings of the user. The display system can also include processing electronics in communication with the display, the one or more user sensors, and the one or more environmental sensors. The processing electronics can be configured to sense a situation involving user focus, determine user intent for the situation, and alter user perception of a real or virtual object within the vision field of the user based at least in part on the user intent and/or sensed situation involving user focus. The processing electronics can be configured to at least one of enhance or de-emphasize the user perception of the real or virtual object within the vision field of the user.
A61B 17/00 - Surgical instruments, devices or methods
A61B 34/00 - Computer-aided surgeryManipulators or robots specially adapted for use in surgery
A61B 34/20 - Surgical navigation systemsDevices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
A61B 90/00 - Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups , e.g. for luxation treatment or for protecting wound edges
A61B 90/50 - Supports for surgical instruments, e.g. articulated arms
B60K 35/10 - Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
B60K 35/26 - Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor using acoustic output
B60K 35/28 - Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor characterised by the type of the output information, e.g. video entertainment or vehicle dynamics informationOutput arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor characterised by the purpose of the output information, e.g. for attracting the attention of the driver
B60K 35/60 - Instruments characterised by their location or relative disposition in or on vehicles
B60K 35/80 - Arrangements for controlling instruments
B60K 35/90 - Calibration of instruments, e.g. setting initial or reference parametersTesting of instruments, e.g. detecting malfunction
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/03 - Arrangements for converting the position or the displacement of a member into a coded form
G06T 19/00 - Manipulating 3D models or images for computer graphics
31.
METHODS AND SYSTEMS FOR DISPLAYING STEREOSCOPY WITH A FREEFORM OPTICAL SYSTEM WITH ADDRESSABLE FOCUS FOR VIRTUAL AND AUGMENTED REALITY
Several unique hardware configurations and methods for freeform optical display systems are disclosed. A freeform display system includes primary freeform optical element(s) and secondary freeform optical element(s) in tiled arrangements to expand the horizontal field of view (FOV) or the vertical field of view. The system may include a variable focusing system that produces intermediate pupil and changes the focal distance of a single focal plane or switches among multiple focal planes for rendering objects in focus while resolving accommodation-convergence conflict. The system may map light samples to appropriate light rays in physical space and use a cluster of projectors to project the mapped light rays to produce the light field of the virtual display content. Methods for making tiled freeform optical display systems and methods for producing virtual content with variable focus freeform optics and rendering light fields are also disclosed.
G02B 3/14 - Fluid-filled or evacuated lenses of variable focal length
G02B 26/08 - Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
G02B 30/10 - Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images using integral imaging methods
A thin transparent layer can be integrated in a head mounted display device and disposed in front of the eye of a wearer. The thin transparent layer may be configured to output light such that light is directed onto the eye to create reflections therefrom that can be used, for example, for glint based tracking. The thin transparent layer can be configured to reduced obstructions in the field of the view of the user.
A computer implemented method of facilitating communication between first and second users includes displaying, by a first head-worn device, a first virtual object to the first user first user wearing the first head-worn device. The method also includes displaying, by a second head-worn device, a second virtual object to the second user wearing the second head-worn device. The method further includes facilitating, by the first and second head-worn devices, communications between the first and second users using the first and second virtual objects to simulate the first and second users being present in a common environment.
A host device having a first processor executes an application via the first processor. The host device determines a state of the application. A scenegraph is generated corresponding to the state of the application, and the scenegraph is presented to a remote device having a display and a second processor. The remote device is configured to, in response to receiving the scenegraph, render to the display a view corresponding to the scenegraph, without executing the application via the second processor.
Recesses are formed on a front side and a rear side of a waveguide. A solid porogen material is spun onto the front side and the rear side and fills the recesses. First front and rear cap layers are then formed on raised formations of the waveguide and on the solid porogen material. The entire structure is then heated and the solid porogen material decomposes to a porogen gas. The first front and rear cap layers are porous to allow the porogen gas to escape and air to enter into the recesses. The air maximizes a difference in refractive indices between the high-index transparent material of the waveguide and the air to promote reflection in the waveguide from interfaces between the waveguide and the air.
F21V 8/00 - Use of light guides, e.g. fibre optic devices, in lighting devices or systems
G02B 6/10 - Light guidesStructural details of arrangements comprising light guides and other optical elements, e.g. couplings of the optical waveguide type
G02B 6/122 - Basic optical elements, e.g. light-guiding paths
G02B 6/13 - Integrated optical circuits characterised by the manufacturing method
G02B 27/00 - Optical systems or apparatus not provided for by any of the groups ,
A method includes patterning a plurality of first trenches in a surface of a substrate; and etching the plurality of first trenches with an etchant having an etch rate for a first crystalline plane of the substrate that is greater than for a second crystalline plane of the substrate. The etching forms a slanted grating in the substrate.
Examples of systems and methods to facilitate audiovisual presence transitions of virtual objects such as virtual avatars in a mixed reality collaborative environment are disclosed. The systems and methods may be configured to produce different audiovisual presence transitions such as appearance, disappearance and reappearance of the virtual avatars. The virtual avatar audiovisual transitions may be further indicated by various visual and sound effects of the virtual avatars. The transitions may occur based on various colocation or decolocation scenarios.
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for calibrating an augmented reality device using camera and inertial measurement unit data. In some implementations, a bundle adjustment process jointly optimizes or estimates states of the augmented reality device. The process can use, as input, visual and inertial measurements as well as factory-calibrated sensor extrinsic parameters. The process performs bundle adjustment and uses non-linear optimization of estimated states constrained by the measurements and the factory calibrated extrinsic parameters. The process can jointly optimize inertial constraints, IMU calibration, and camera calibrations. Output of the process can include most likely estimated states, such as data for a 3D map of an environment, a trajectory of the device, and/or updated extrinsic parameters of the visual and inertial sensors (e.g., cameras and IMUs).
An augmented reality device (ARD) can present virtual content which can provide enhanced experiences with the user's physical environment. For example, the ARD can detect a linkage between a person in the FOV of the ARD and a physical object (e.g., a document presented by the person) or detect linkages between the documents. The linkages may be used in identity verification or document verification.
Wearable systems and method for operation thereof incorporating headset and controller localization using headset cameras and controller fiducials are disclosed. A wearable system may include a headset and a controller. The wearable system may alternate between performing headset tracking and performing controller tracking by repeatedly capturing images using a headset camera of the headset during headset tracking frames and controller tracking frames. The wearable system may cause the headset camera to capture a first exposure image an exposure above a threshold and cause the headset camera to capture a second exposure image having an exposure below the threshold. The wearable system may determine a fiducial interval during which fiducials of the controller are to flash at a fiducial frequency and a fiducial period. The wearable system may cause the fiducials to flash during the fiducial interval in accordance with the fiducial frequency and the fiducial period.
Techniques for artifact mitigation in an optical system are disclosed. Light associated with a world object is received at the optical system, which is characterized by a world side and a user side. Light associated with a virtual image is projected onto an eyepiece of the optical system, causing a portion of the light associated with the virtual image to propagate toward the user side and light associated with an artifact image to propagate toward the world side. A dimmer of the optical system positioned between the world side and the eyepiece is adjusted to reduce an intensity of the light associated with the artifact image impinging on the dimmer and an intensity of the light associated with the world object impinging on the dimmer.
G02B 27/00 - Optical systems or apparatus not provided for by any of the groups ,
G02B 26/08 - Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
Disclosed herein is a wearable display system for capturing retraining eye images of an eye of a user for retraining a neural network for eye tracking. The system captures retraining eye images using an image capture device when user interface (UI) events occur with respect to UI devices displayed at display locations of a display. The system can generate a retraining set comprising the retraining eye images and eye poses of the eye of the user in the retraining eye images (e.g., related to the display locations of the UI devices) and obtain a retrained neural network that is retrained using the retraining set.
G06F 1/16 - Constructional details or arrangements
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/0346 - Pointing devices displaced or positioned by the userAccessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
G06F 3/04815 - Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
Techniques related to the computation of gaze vectors of users of wearable devices are disclosed. A neural network may be trained through first and second training steps. The neural network may include a set of feature encoding layers and a plurality of sets of task-specific layers that each operate on an output of the set of feature encoding layers. During the first training step, a first image of a first eye may be provided to the neural network, eye segmentation data may be generated using the neural network, and the set of feature encoding layers may be trained. During the second training step, a second image of a second eye may be provided to the neural network, network output data may be generated using the neural network, and the plurality of sets of task-specific layers may be trained.
G06V 10/26 - Segmentation of patterns in the image fieldCutting or merging of image elements to establish the pattern region, e.g. clustering-based techniquesDetection of occlusion
G06V 10/774 - Generating sets of training patternsBootstrap methods, e.g. bagging or boosting
G06V 10/82 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
G06V 40/18 - Eye characteristics, e.g. of the iris
A head mounted display system for displaying image content to a user comprises at least one display configured to be worn by a user to present virtual content to first and second eyes of a user, one or more inwardly facing sensors or camera configured to monitor one or both of the users eye and processing electronics. This head mounted display system is configured such that virtual content activity can be initiated and/or driven from eye inputs such as gaze direction, eyelid motions (e.g., blinking), and/or other eye gestures.
An optical device such as an augmented reality (AR) display device includes variable optical material that alters at least one of: incident ambient light, spectral content of incident ambient light or direction of incident ambient light through the optical device in response to a stimulus provided by the device. The device can sense intensity and/or spectral characteristics of ambient light and provide appropriate stimulus to various portions of the optical device to activate the variable optical material and alter at least one of: incident ambient light, spectral content of incident ambient light or direction of incident ambient light. In some examples, the variable optical material may be distributed unevenly across the optical device.
F21V 8/00 - Use of light guides, e.g. fibre optic devices, in lighting devices or systems
G02B 3/14 - Fluid-filled or evacuated lenses of variable focal length
G02B 27/00 - Optical systems or apparatus not provided for by any of the groups ,
G02F 1/00 - Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulatingNon-linear optics
G02F 1/01 - Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulatingNon-linear optics for the control of the intensity, phase, polarisation or colour
G02F 1/29 - Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulatingNon-linear optics for the control of the position or the direction of light beams, i.e. deflection
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
Systems and methods for generating a face model for a user of a head-mounted device are disclosed. The head-mounted device can include one or more eye cameras configured to image the face of the user while the user is putting the device on or taking the device off. The images obtained by the eye cameras may be analyzed using a stereoscopic vision technique, a monocular vision technique, or a combination, to generate a face model for the user. The face model can be used to generate a virtual image of at least a portion of the user's face, for example to be presented as an avatar.
In some embodiments, a display system comprising a head-mountable, augmented reality display is configured to perform a neurological analysis and to provide a perception aid based on an environmental trigger associated with the neurological condition. Performing the neurological analysis may include determining a reaction to a stimulus by receiving data from the one or more inwardly-directed sensors; and identifying a neurological condition associated with the reaction. In some embodiments, the perception aid may include a reminder, an alert, or virtual content that changes a property, e.g. a color, of a real object. The augmented reality display may be configured to display virtual content by outputting light with variable wavefront divergence, and to provide an accommodation-vergence mismatch of less than 0.5 diopters, including less than 0.25 diopters.
A display system may include a head-mounted display (HMD) for rendering a three-dimensional virtual object which appears to be located in an ambient environment of a user of the display. One or more eyes of the user may not be in desired positions, relative to the HMD, to receive, or register, image information outputted by the HMD and/or to view an external environment. For example, the HMD-to-eye alignment may vary for different users and/or may change over time (e.g., as the HMD is displaced). The display system may determine a relative position or alignment between the HMD and the user's eyes. Based on the relative positions, the wearable device may determine if it is properly fitted to the user, may provide feedback on the quality of the fit to the user, and/or may take actions to reduce or minimize effects of any misalignment.
G02B 27/00 - Optical systems or apparatus not provided for by any of the groups ,
A61B 3/11 - Objective types, i.e. instruments for examining the eyes independent of the patients perceptions or reactions for measuring interpupillary distance or diameter of pupils
A61B 3/113 - Objective types, i.e. instruments for examining the eyes independent of the patients perceptions or reactions for determining or recording eye movement
G02B 30/00 - Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images
G02B 30/40 - Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images giving the observer of a single two-dimensional [2D] image a perception of depth
G06F 1/16 - Constructional details or arrangements
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/0346 - Pointing devices displaced or positioned by the userAccessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
G06F 3/04815 - Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
G06T 3/40 - Scaling of whole images or parts thereof, e.g. expanding or contracting
G06V 10/42 - Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
G06V 10/46 - Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]Salient regional features
G06V 10/60 - Extraction of image or video features relating to illumination properties, e.g. using a reflectance or lighting model
G06V 40/18 - Eye characteristics, e.g. of the iris
H04N 13/344 - Displays for viewing with the aid of special glasses or head-mounted displays [HMD] with head-mounted left-right displays
H04N 13/383 - Image reproducers using viewer tracking for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
A wearable device can include an inward-facing imaging system configured to acquire images of a user's periocular region. The wearable device can determine a relative position between the wearable device and the user's face based on the images acquired by the inward-facing imaging system. The relative position may be used to determine whether the user is wearing the wearable device, whether the wearable device is optimally fit to the user, and/or whether an adjustment to a rendering location of a virtual object can be made to compensate for a deviation of the wearable device from its normal resting position relative to the user's face.
G06T 3/20 - Linear translation of whole images or parts thereof, e.g. panning
G06T 11/60 - Editing figures and textCombining figures or text
G06V 10/46 - Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]Salient regional features
G06V 20/20 - ScenesScene-specific elements in augmented reality scenes
G06V 40/16 - Human faces, e.g. facial parts, sketches or expressions
G06V 40/18 - Eye characteristics, e.g. of the iris
A virtual, augmented, or mixed reality display system includes a display configured to display virtual, augmented, or mixed reality image data, the display including one or more optical components which introduce optical distortions or aberrations to the image data. The system also includes a display controller configured to provide the image data to the display. The display controller includes memory for storing optical distortion correction information, and one or more processing elements to at least partially correct the image data for the optical distortions or aberrations using the optical distortion correction information.
A wearable display system includes one or more nanowire LED micro-displays. The nanowire micro-LED displays may be monochrome or full-color. The nanowire LEDs forming the arrays may have an advantageously narrow angular emission profile and high light output. Where a plurality of nanowire LED micro-displays is utilized, the micro-displays may be positioned at different sides of an optical combiner, for example, an X-cube prism which receives light rays from different micro-displays and outputs the light rays from the same face of the cube. The optical combiner directs the light to projection optics, which outputs the light to an eyepiece that relays the light to a user's eye. The eyepiece may output the light to the user's eye with different amounts of wavefront divergence, to place virtual content on different depth planes.
Disclosed herein are systems and methods for distributed computing and/or networking for mixed reality systems. A method may include capturing an image via a camera of a head-wearable device. Inertial data may be captured via an inertial measurement unit of the head-wearable device. A position of the head-wearable device can be estimated based on the image and the inertial data via one or more processors of the head-wearable device. The image can be transmitted to a remote server. A neural network can be trained based on the image via the remote server. A trained neural network can be transmitted to the head-wearable device.
G06F 18/214 - Generating training patternsBootstrap methods, e.g. bagging or boosting
G06V 10/764 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
G06V 20/20 - ScenesScene-specific elements in augmented reality scenes
G06V 40/16 - Human faces, e.g. facial parts, sketches or expressions
G06V 40/18 - Eye characteristics, e.g. of the iris
Antireflection coatings for metasurfaces are described herein. In some embodiments, the metasurface may include a substrate, a plurality of nanostructures thereon, and an antireflection coating disposed over the nanostructures. The antireflection coating may be a transparent polymer, for example a photoresist layer, and may have a refractive index lower than the refractive index of the nanostructures and higher than the refractive index of the overlying medium (e.g., air). Advantageously, the antireflection coatings may reduce or eliminate ghost images in an augmented reality display in which the metasurface is incorporated.
Systems and methods for eye image segmentation and image quality estimation are disclosed. In one aspect, after receiving an eye image, a device such as an augmented reality device can process the eye image using a convolutional neural network with a merged architecture to generate both a segmented eye image and a quality estimation of the eye image. The segmented eye image can include a background region, a sclera region, an iris region, or a pupil region. In another aspect, a convolutional neural network with a merged architecture can be trained for eye image segmentation and image quality estimation. In yet another aspect, the device can use the segmented eye image to determine eye contours such as a pupil contour and an iris contour. The device can use the eye contours to create a polar image of the iris region for computing an iris code or biometric authentication.
G06F 18/2413 - Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
G06V 10/44 - Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersectionsConnectivity analysis, e.g. of connected components
G06V 10/56 - Extraction of image or video features relating to colour
G06V 10/764 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
G06V 10/82 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
G06V 10/98 - Detection or correction of errors, e.g. by rescanning the pattern or by human interventionEvaluation of the quality of the acquired patterns
G06V 40/18 - Eye characteristics, e.g. of the iris
55.
LIGHT OUTPUT SYSTEM WITH REFLECTOR AND LENS FOR HIGHLY SPATIALLY UNIFORM LIGHT OUTPUT
A user may interact and view virtual elements such as avatars and objects and/or real world elements in three-dimensional space in an augmented reality (AR) session. The system may allow one or more spectators to view from a stationary or dynamic camera a third person view of the users AR session. The third person view may be synchronized with the user view and the virtual elements of the user view may be composited onto the third person view.
F21V 13/04 - Combinations of only two kinds of elements the elements being reflectors and refractors
F21V 8/00 - Use of light guides, e.g. fibre optic devices, in lighting devices or systems
G02B 6/12 - Light guidesStructural details of arrangements comprising light guides and other optical elements, e.g. couplings of the optical waveguide type of the integrated circuit kind
G02B 23/06 - Telescopes, e.g. binocularsPeriscopesInstruments for viewing the inside of hollow bodiesViewfindersOptical aiming or sighting devices involving prisms or mirrors having a focusing action, e.g. parabolic mirror
G02B 30/50 - Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images the image being built up from image elements distributed over a 3D volume, e.g. voxels
H04N 13/315 - Image reproducers for viewing without the aid of special glasses, i.e. using autostereoscopic displays using parallax barriers the parallax barriers being time-variant
H04N 13/344 - Displays for viewing with the aid of special glasses or head-mounted displays [HMD] with head-mounted left-right displays
56.
SELECTING VIRTUAL OBJECTS IN A THREE-DIMENSIONAL SPACE
Systems and methods for interacting with virtual objects in a three-dimensional space using a wearable system are disclosed. The wearable system can be programmed to permit user interaction with interactable objects in a field of regard (FOR) of a user. The FOR includes a portion of the environment around the user that is capable of being perceived by the user via the AR system. The system can determine a group of interactable objects in the FOR of the user and determine a pose of the user. The system can update, based on a change in the pose or a field of view (FOV) of the user, a subgroup of the interactable objects that are located in the FOV of the user and receive a selection of a target interactable object from the subgroup of interactable objects. The system can initiate a selection event on the target interactable object.
G06F 3/0346 - Pointing devices displaced or positioned by the userAccessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
G06F 1/16 - Constructional details or arrangements
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/04815 - Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
G06F 3/0482 - Interaction with lists of selectable items, e.g. menus
G06F 3/04883 - Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
57.
ECLIPSE CURSOR FOR VIRTUAL CONTENT IN MIXED REALITY DISPLAYS
Systems and methods for displaying a cursor and a focus indicator associated with real or virtual objects in a virtual, augmented, or mixed reality environment by a wearable display device are disclosed. The system can determine a spatial relationship between a user-movable cursor and a target object within the environment. The system may render a focus indicator (e.g., a halo, shading, or highlighting) around or adjacent objects that are near the cursor. When the cursor overlaps with a target object, the system can render the object in front of the cursor (or not render the cursor at all), so the object is not occluded by the cursor. The object can be rendered closer to the user than the cursor. A group of virtual objects can be scrolled, and a virtual control panel can be displayed indicating objects that are upcoming in the scroll.
G06F 3/04817 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/04812 - Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
G06F 3/04815 - Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
A method includes determining an eye gaze location of a user and generating a spatial foveation map based on the eye gaze location. The method also includes receiving an image, forming a spatially foveated image using the image and the spatial foveation map, and transmitting the spatially foveated image to a wearable device. The method further includes spatially defoveating the spatially foveated image to produce a spatially defoveated image and displaying the spatially defoveated image.
An augmented reality (AR) system includes a wearable device including: a frame, a projector coupled to the frame, a display optically coupled to the projector, and an eye tracking system. The AR system also includes a memory and a processor configured to: receive an eye gaze location from the eye tracking system, generate an image, and generate a foveation map based on the eye gaze location. The foveation map includes a first region of the image and a second region of the image. The processor is also configured to compress the first region of the image using a first quality setting and the second region of the image using a second quality setting. The first quality setting (e.g., a setting of 100%) can be greater than the second quality setting.
H04N 13/117 - Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
H04N 13/139 - Format conversion, e.g. of frame-rate or size
H04N 13/161 - Encoding, multiplexing or demultiplexing different image signal components
H04N 13/344 - Displays for viewing with the aid of special glasses or head-mounted displays [HMD] with head-mounted left-right displays
H04N 13/383 - Image reproducers using viewer tracking for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
An augmented reality (AR) device can be configured to monitor ambient audio data. The AR device can detect speech in the ambient audio data, convert the detected speech into text, or detect keywords such as rare words in the speech. When a rare word is detected, the AR device can retrieve auxiliary information (e.g., a definition) related to the rare word from a public or private source. The AR device can display the auxiliary information for a user to help the user better understand the speech. The AR device may perform translation of foreign speech, may display text (or the translation) of a speaker's speech to the user, or display statistical or other information associated with the speech.
A mixed reality virtual environment is sharable among multiple users through the use of multiple view modes that are selectable by a presenter. Multiple users with wearable display systems may wish to view a common virtual object, which may be presented in a virtual room to any suitable number of users. A presentation may be controlled by a presenter using a presenter wearable system that leads multiple participants through information associated with the virtual object. Use of different viewing modes allows individual users to see different virtual content through their wearable display systems, despite being in a shared viewing space or alternatively, to see the same virtual content in different locations within a shared space.
G09B 5/12 - Electrically-operated educational appliances providing for individual presentation of information to a plurality of student stations different stations being capable of presenting different information simultaneously
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06T 19/00 - Manipulating 3D models or images for computer graphics
H04L 12/18 - Arrangements for providing special services to substations for broadcast or conference
62.
METHOD AND SYSTEM FOR PERFORMING IMAGE REPROJECTION USING HEAD POSE INFORMATION
A method includes receiving, at an encoder, virtual content, receiving, at the encoder, a predicted head pose corresponding to the virtual content, and encoding the virtual content based on the predicted head pose. The method also includes producing compressed content, receiving, at a decoder, the compressed content, and receiving, at the decoder, a current head pose. The method also includes decoding the compressed content based on the current head pose and producing reprojected virtual content. In some embodiments, the method further includes compressing the reprojected virtual content to form one or more foveated images, decompressing the one or more foveated images to form a set of output images, and displaying the set of output images on a display.
Systems include three optical elements arranged along an optical axis each having a different cylinder axis and a variable cylinder refractive power. Collectively, the three elements form a compound optical element having an overall spherical refractive power (SPH), cylinder refractive power (CYL), and cylinder axis (Axis) that can be varied according to a prescription (Rx).
A head mounted display system can include at least one imaging device, a waveguide, and optical elements formed on or in the waveguide, including at least one coupling optical element configured to in-couple, into the waveguide, light from the environment, and at least one out-coupling optical element configured to out-couple, from the waveguide and toward the imaging device(s), the light from the environment, such that the imaging device(s) can image the environment based on the in-coupled light. The waveguide may also include an in-coupling optical element configured to couple, into the waveguide, image light that conveys virtual image content, and another out-coupling optical element that may be separate from the at least one coupling optical element and that is configured to couple the image light out of the waveguide toward the user's eye.
A method of producing a reprojected image includes receiving motion data and determining, based on the motion data, if a motion threshold is exceeded. The method also includes generating a depth-based reprojection if the motion threshold is exceeded or generating a non-depth-based reprojection if the motion threshold is not exceeded. In some embodiments, performing the foveated compression of the depth-based reprojection includes determining an eye gaze location of a user and generating a foveation map based on the eye gaze location. The foveation map includes a first region of the depth-based reprojection and a second region of the depth-based reprojection. Performing the foveated compression of the depth-based reprojection also includes compressing the first region using a first quality setting and the second region using a second quality setting.
Examples of eye-imaging apparatus using diffractive optical elements are provided. For example, an optical device comprises a substrate having a proximal surface and a distal surface, a first coupling optical element disposed on one of the proximal and distal surfaces of the substrate, and a second coupling optical element disposed on one of the proximal and distal surfaces of the substrate and offset from the first coupling optical element. The first coupling optical element can be configured to deflect light at an angle to totally internally reflect (TIR) the light between the proximal and distal surfaces and toward the second coupling optical element, and the second coupling optical element can be configured to deflect at an angle out of the substrate. The eye-imaging apparatus can be used in a head-mounted display such as an augmented or virtual reality display.
An extended reality system that includes a wearable eyepiece presenting virtual contents to a user, a belt pack operatively coupled to the wearable eyepiece, a processor, and a non-transitory computer readable medium storing thereupon a sequence of instructions which, when executed by a model with the processor, causes the processor to estimate a plurality of keyframes, a plurality of keyframe poses, and depth data from a plurality of captures that is captured by at least the extended reality device, to generate a semantically annotated, manipulatable three-dimensional (3D) representation in a physical environment for perception by the user wearing the wearable eyepiece, and to modify the semantically annotated, manipulatable 3D representation in real-time or nearly real-time in response to a user interaction.
G06V 10/762 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
A head-mounted display system includes: a head mounted display frame; a first eyepiece supported by the frame, the first eyepiece including a first substrate composed of a crystalline, transparent material having crystallographic axes in a first orientation with respect to the frame, the substrate having a first surface and a second surface opposite the first surface, the first eyepiece further including a first in-coupling element including a grating on the first surface, and a first out-coupling element including a grating on the first surface and/or a grating on the second surface; and a second eyepiece including a second substrate composed of the crystalline, transparent material having crystallographic axes in a second orientation with respect to the frame different from the first orientation, a second in-coupling element on either surface of the second substrate, and a second out-coupling element on either surface of the second substrate.
G02B 1/02 - Optical elements characterised by the material of which they are madeOptical coatings for optical elements made of crystals, e.g. rock-salt, semiconductors
The invention relates to a viewing system for use in a surgical environment. Various real object detection devices detect locations of real objects in a real environment, such as a patient and body part of patient, medical staff, robots, a cutting tool on a robot, implant transferred by robot into body part, surgical tools, and disposable items. A map generator generates a map that forms a digital representation or a digital twin of the real environment. Various guiding modules including a room setup module, an anatomy registration module, a surgical planning module, and a surgical execution module make use of the digital representation to guide virtual or real objects based on the digital representation.
A61B 34/20 - Surgical navigation systemsDevices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
A61B 90/00 - Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups , e.g. for luxation treatment or for protecting wound edges
A61B 90/50 - Supports for surgical instruments, e.g. articulated arms
A virtual reality (VR) and/or augmented reality (AR) display system is configured to control a display using control information that is embedded in or otherwise included with imagery data to be presented through the display. The control information can indicate depth plane(s) and/or color plane(s) to be used to present the imagery data, depth plane(s) and/or color plane(s) to be activated or inactivated, shift(s) of at least a portion of the imagery data (e.g., one or more pixels) laterally within a depth plane and/or longitudinally between depth planes, other adjustment(s) to the virtual imagery, and/or other suitable controls.
G06T 7/579 - Depth or shape recovery from multiple images from motion
G06T 19/00 - Manipulating 3D models or images for computer graphics
G09G 3/00 - Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
G09G 3/20 - Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix
G09G 5/00 - Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
H04N 13/344 - Displays for viewing with the aid of special glasses or head-mounted displays [HMD] with head-mounted left-right displays
H04N 13/395 - Volumetric displays, i.e. systems where the image is built up from picture elements distributed through a volume with depth sampling, i.e. the volume being constructed from a stack or sequence of 2D image planes
Very high refractive index (n>2.2) lightguide substrates enable the production of 70° field of view eyepieces with all three color primaries in a single eyepiece layer. Disclosed herein are viewing optics assembly architectures that make use of such eyepieces to reduce size and cost, simplifying manufacturing and assembly, and better-accommodating novel microdisplay designs.
Exemplary systems and methods for creating spatial contents in a mixed reality environment are disclosed. In an example, a location associated with a first user in a coordinate space is determined. A persistent virtual content is generated. The persistent virtual content is associated with the first user's associated location. The first user's associated location is determined and is associated with the persistent virtual content. A location of a second user at a second time in the coordinate space is determined. The persistent virtual content is presented to the second user via a display at a location in the coordinate space corresponding to the first user's associated location.
A head-mounted display system is configured to project light to an eye of a user wearing the head-mounted display system to display content in a vision field of said user. The head-mounted display system comprises at least one diffusive optical element, at least one out-coupling optical element, at least one mask comprising at least one mask opening, at least one illumination in-coupling optical element configured to in-couple light from at least one illumination source into a light-guiding component, an image projector configured to in-couple an image and an at least one illumination source is configured to in-couple light into at least one illumination in-coupling optical element, an eyepiece, a curved light-guiding component, a light-guiding component comprising a portion of a frame, and/or two light-guiding components disposed on opposite sides of at least one out-coupling optical element.
A voice user interface (VUI) and methods for operating the VUI are disclosed. In some embodiments, the VUI configured to receive and process linguistic and non-linguistic inputs. For example, the VUI receives an audio signal, and the VUI determines whether the audio input comprises a linguistic and/or a non-linguistic input. In accordance with a determination that the audio signal comprises a non-linguistic input, the VUI causes a system to perform an action associated with the non-linguistic input.
G10L 15/18 - Speech classification or search using natural language modelling
G10L 25/63 - Speech or voice analysis techniques not restricted to a single one of groups specially adapted for particular use for comparison or discrimination for estimating an emotional state
75.
SYSTEMS AND METHOD FOR AUDIO PROCESSING IN EXTENDED REALITY
A method for presenting an extended reality experience to a user includes a display subsystem presenting images corresponding to image data to the user. The method also includes a microphone capturing input sound from an ambient acoustic environment and converting the captured input sound to input audio data. The method further includes an audio controller analyzing the input audio data to determine a masking level, rendering output audio data corresponding to source audio data, analyzing the output audio data to determine an energy level of the output audio data, comparing the energy level with the masking level to determine a comparison outcome, and modifying the output audio data based on the comparison outcome to raise the energy level of the output audio data to generate modified output audio data. The method also includes an audio subsystem presenting output sound corresponding to the modified output audio data to the user.
An optical device may include a light turning element. The optical device can include a first surface that is parallel to a horizontal axis and a second surface opposite to the first surface. The optical device may include a light module that includes a plurality of light emitters, and that can be configured to combine light from the emitters. The optical device can further include a light input surface that is between the first and the second surfaces and is disposed with respect to the light module to receive light. The optical device may include an end reflector that is disposed on a side opposite the light input surface. The light coupled into the light turning element may be reflected by the end reflector and/or reflected from the second surface towards the first surface.
G02B 27/00 - Optical systems or apparatus not provided for by any of the groups ,
G02B 27/14 - Beam splitting or combining systems operating by reflection only
G02B 30/26 - Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer’s left and right eyes of the autostereoscopic type
G02B 30/52 - Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images the image being built up from image elements distributed over a 3D volume, e.g. voxels the 3D volume being constructed from a stack or sequence of 2D planes, e.g. depth sampling systems
G02F 1/137 - Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulatingNon-linear optics for the control of the intensity, phase, polarisation or colour based on liquid crystals, e.g. single liquid crystal display cells characterised by the electro-optical or magneto-optical effect, e.g. field-induced phase transition, orientation effect, guest-host interaction or dynamic scattering
G03B 21/00 - Projectors or projection-type viewersAccessories therefor
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/0482 - Interaction with lists of selectable items, e.g. menus
G06T 19/00 - Manipulating 3D models or images for computer graphics
G06V 20/20 - ScenesScene-specific elements in augmented reality scenes
G09G 3/02 - Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes by tracing or scanning a light beam on a screen
G09G 3/24 - Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix using controlled light sources using incandescent filaments
H04N 13/00 - Stereoscopic video systemsMulti-view video systemsDetails thereof
H04N 13/239 - Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
H04N 13/279 - Image signal generators from 3D object models, e.g. computer-generated stereoscopic image signals the virtual viewpoint locations being selected by the viewers or determined by tracking
H04N 13/344 - Displays for viewing with the aid of special glasses or head-mounted displays [HMD] with head-mounted left-right displays
Head-mounted virtual and augmented reality display systems include a light projector with one or more emissive micro-displays having a first resolution and a pixel pitch. The projector outputs light forming frames of virtual content having at least a portion associated with a second resolution greater than the first resolution. The projector outputs light forming a first subframe of the rendered frame at the first resolution, and parts of the projector are shifted using actuators, such that physical positions of light output for individual pixels occupy gaps between the old locations of light output for individual pixels. The projector then outputs light forming a second subframe of the rendered frame. The first and second subframes are outputted within the flicker fusion threshold. Advantageously, an emissive micro-display (e.g., micro-LED display) having a low resolution can form a frame having a higher resolution by using the same light emitters to function as multiple pixels of that frame.
G02B 27/14 - Beam splitting or combining systems operating by reflection only
G02B 27/18 - Optical systems or apparatus not provided for by any of the groups , for optical projection, e.g. combination of mirror and condenser and objective
G02B 27/62 - Optical apparatus specially adapted for adjusting optical elements during the assembly of optical systems
G09G 3/00 - Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
G09G 3/32 - Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix using controlled light sources using electroluminescent panels semiconductive, e.g. using light-emitting diodes [LED]
H02N 2/02 - Electric machines in general using piezoelectric effect, electrostriction or magnetostriction producing linear motion, e.g. actuatorsLinear positioners
79.
Cross Reality System with Simplified Programming of Virtual Content
A cross reality system that renders virtual content generated by executing native mode applications may be configured to render web-based content using components that render content from native applications. The system may include a Prism manager that provides Prisms in which content from executing native applications is rendered. For rendering web based content, a browser, accessing the web based content, may be associated with a Prism and may render content into its associated Prism, creating the same immersive experience for the user as when content is generated by a native application. The user may access the web application from the same program launcher menu as native applications. The system may have tools that enable a user to access these capabilities, including by creating for a web location an installable entity that, when processed by the system, results in an icon for the web content in a program launcher menu.
G06T 19/00 - Manipulating 3D models or images for computer graphics
G06F 3/04815 - Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
G06F 3/04817 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
G06F 3/0482 - Interaction with lists of selectable items, e.g. menus
This disclosure describes techniques for fabrication of waveguides as optical devices or for use in optical devices, with the waveguides customized to have a desired thickness variation. Techniques can employ inkjet-based lithography to compensate for thickness variations in the substrate used to manufacture the optical devices, and/or create custom variations in the thickness to achieve various optical properties in the resulting device. In some implementations, a curvature can also be applied to one or both surfaces of the substrate, to achieve desired optical performance and/or enhance fit of a wearable optical device. The optical devices created using the techniques described herein are suitable for use in virtual reality, augmented reality, and/or other suitable optical applications. The optical devices may be created on flexible (e.g., polymer) or more rigid (e.g., glass) substrates, with the thickness of the substrate being customizable using a jettable and curable polymer resin or photoresist.
G03F 7/00 - Photomechanical, e.g. photolithographic, production of textured or patterned surfaces, e.g. printed surfacesMaterials therefor, e.g. comprising photoresistsApparatus specially adapted therefor
B29D 11/00 - Producing optical elements, e.g. lenses or prisms
An eye tracking system can include eye-tracking camera(s) configured to obtain image(s) of the eye at different exposure times or different frame rates. For example, image(s) of the eye taken with a longer exposure time can be analyzed to detect eye feature(s) such as iris or pupil features, and image(s) taken with a shorter exposure time can be analyzed to detect glints reflected from the eye. The shorter exposure images may be taken at a higher frame rate than the longer exposure images for more accurate gaze prediction based on glint analysis, e.g., to provide glint locations to subpixel accuracy. Such glint analysis may also take into account estimated location(s) of partially or totally occluded glint(s). The longer exposure images can be analyzed for pupil center, eye center of rotation, or other characteristics. The system can predict gaze direction, e.g., for foveated rendering by a wearable display system.
An optics assembly includes an eyepiece operable to receive virtual image light and project the virtual image light toward an eye side of the optics assembly. The optics assembly includes a rear lens disposed between the eyepiece and the eye side of the optics assembly. The optics assembly includes one or more wedge prisms integrated with the rear lens along a periphery of the rear lens. The optics assembly includes an eye tracking camera disposed outside the periphery of the rear lens in alignment with the one or more wedge prisms and operable to capture light that propagates from the eye side of the optics assembly and is refracted through the one or more wedge prisms.
A wearable computing system that includes a head-mounted display implements a gaze timer feature for enabling the user to temporarily extend the functionality of a handheld controller or other user input device. In one embodiment, when the user gazes at, or in the vicinity of, a handheld controller for a predetermined period of time, the functionality of one or more input elements (e.g., buttons) of the handheld controller is temporarily modified. For example, the function associated with a particular controller button may be modified to enable the user to open a particular menu using the button. The gaze timer feature may, for example, be used to augment the functionality of a handheld controller or other user input device during mixed reality and/or augmented reality sessions.
G06F 3/023 - Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
G06F 3/048 - Interaction techniques based on graphical user interfaces [GUI]
G06F 3/0482 - Interaction with lists of selectable items, e.g. menus
84.
METHOD AND SYSTEM FOR EYE TRACKING WITH REDUCED KEYSTONE DISTORTION
An eye tracking system operable to image an eye of a user includes an illumination source operable to illuminate the eye and an optical system including a first imaging surface, an aperture stop, and a second imaging surface. The optical system is operable to image an object plane corresponding to the eye. The eye tracking system also includes an image sensor having an image plane parallel to the first imaging surface, the second imaging surface, and the object plane.
A61B 3/113 - Objective types, i.e. instruments for examining the eyes independent of the patients perceptions or reactions for determining or recording eye movement
A61B 3/14 - Arrangements specially adapted for eye photography
G02B 9/08 - Optical objectives characterised both by the number of the components and their arrangements according to their sign, i.e. + or – having two components only two + components arranged about a stop
G02B 1/00 - Optical elements characterised by the material of which they are madeOptical coatings for optical elements
85.
METHOD AND SYSTEM FOR EYE-TRACKING USING AN OFF-AXIS MIRROR WITH DISTORTION CORRECTION
Techniques for tracking an eye of a user of an augmented reality headset having an eye tracking system. The eye tracking system can reflect light from the eye of the user and propagate the light through a first refractive optical element. The eye tracking system can then propagate the light through a second refractive optical element, impinge the light on a reflective optical element, and reflect the light from the reflective optical element. The eye tracking system can then propagate the reflected light through the second refractive optical element to a lateral aperture of the second refractive optical element and detect the reflected light at a camera. The eye tracking system can output an eye tracking signal from the camera to track the eye.
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G02B 6/12 - Light guidesStructural details of arrangements comprising light guides and other optical elements, e.g. couplings of the optical waveguide type of the integrated circuit kind
An extended reality display system includes a light source to emit light. The system also includes a waveguide assembly to receive and direct the light. The system further includes a spatial light modulator (SLM) configured to receive the light. The waveguide assembly includes a first waveguide, a second waveguide, and an adhesive layer disposed between and in direct contact with the first and second waveguides.
G02B 6/10 - Light guidesStructural details of arrangements comprising light guides and other optical elements, e.g. couplings of the optical waveguide type
09 - Scientific and electric apparatus and instruments
Goods & Services
(1) Headset accessories, namely, headset cushions and headset pads used to correct virtual, augmented and mixed reality headsets to the appropriate size of the user.
88.
DISPLAY DEVICE HAVING DIFFRACTION GRATINGS WITH REDUCED POLARIZATION SENSITIVITY
Blazed diffraction gratings provide optical elements in head-mounted display systems to, e.g., incouple light into or out-couple light out of a waveguide. These blazed diffraction gratings may be configured to have reduced polarization sensitivity. Such gratings may, for example, incouple or outcouple light of different polarizations with similar level of efficiency. The blazed diffraction gratings and waveguides may be formed in or on a high refractive index substrate such as lithium niobate. In some implementations, the blazed diffraction gratings may include diffractive features having a feature height or feature depth of 40 nm to 120 nm, for example, 80 nm. In some examples, the diffractive features may be etched into the high index substrate, e.g., lithium niobate.
An extended reality (XR) system, comprises a head-mounted display (HMD) configured for displaying virtual content to a user, a first altimeter carried by the HMD, a hand-held control, and a second altimeter carried by the hand-held control. The first altimeter configured for outputting first atmospheric pressure data indicative of an elevation of the HMD, while the second altimeter is configured for outputting second atmospheric pressure data indicative of an elevation of the hand-held control. The XR system further comprises at least one processor configured for determining a relative elevation between the first altimeter and the second altimeter based on the first atmospheric pressure data and the second atmospheric pressure data.
G01C 5/06 - Measuring heightMeasuring distances transverse to line of sightLevelling between separated pointsSurveyors' levels by using barometric means
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
90.
IMPRINTING TECHNIQUES IN NANOLITHOGRAPHY FOR OPTICAL DEVICES
This disclosure generally describes methods and systems for fabrication of high-quality surface relief waveguides for eyepieces. In particular, this disclosure describes techniques for manufacturing waveguides having surface relief features, such as diffractive gratings to achieve various optical effects, using nanolithographic imprinting techniques that reduce or eliminate the presence of gaps in the imprinted features through use of optimized drop patterns for dispensing photoresist. Moreover, the disclosure also describes techniques for manufacturing surface relief waveguides having a gradation, e.g., a substantially continuous grade or slope, between zones that have different residual layer thicknesses of the dispensed photoresist, and/or between zones having surface features of different height (or depth). Such gradation can reduce or eliminate adverse optical effects that may be caused by a more abrupt transition between zones, and increase the optical efficiency of the completed waveguide.
G03F 7/00 - Photomechanical, e.g. photolithographic, production of textured or patterned surfaces, e.g. printed surfacesMaterials therefor, e.g. comprising photoresistsApparatus specially adapted therefor
91.
OPTICAL SYSTEMS HAVING WAVEGUIDES WITH NANOPOROUS, LOW REFRACTIVE INDEX LAYERS AND METHODS OF MAKING
Augmented reality display systems include an eyepiece with a low refractive index, nanoporous layer, which may be a sol-gel layer formed using a sol-gel synthesis. The eyepiece includes one or more waveguides, which are transmissive to allow the wearer of the system to see the ambient environment and include a grating for out-coupling light propagating within the waveguides. Protrusions form the grating and the sol-gel layer is disposed over the waveguide and in spaces between the protrusions. The protrusions may have differing heights. The sol-gel layer may completely cover all or only some of the protrusions, while other protrusions extend above the sol-gel layer. The sol-gel layer may be part of a stack of sol-gel layers, which may have different compositions, thicknesses, and/or refractive indices. Hie sol-gel layer and grating structure has advantageously low reflectivity and mitigates a rainbow effect otherwise caused by ambient light reflected from the grating.
An eyepiece waveguide for an augmented reality display system. The eyepiece waveguide can include an input coupling grating (ICG) region. The ICG region can couple an input beam into the substrate of the eyepiece waveguide as a guided beam. A first combined pupil expander-extractor (CPE) grating region can be formed on or in a surface of the substrate. The first CPE grating region can receive the guided beam, create a first plurality of diffracted beams at a plurality of distributed locations, and out-couple a first plurality of output beams. The eyepiece waveguide can also include a second CPE grating region formed on or in the opposite surface of the substrate. The second CPE grating region can receive the guided beam, create a second plurality of diffracted beams at a plurality of distributed locations, and out-couple a second plurality of output beams.
Examples of wearable systems and methods can use multiple inputs (e.g., gesture, head pose, eye gaze, voice, totem, and/or environmental factors (e.g., location)) to determine a command that should be executed and objects in the three-dimensional (3D) environment that should be operated on. The wearable system can detect when different inputs converge together, such as when a user seeks to select a virtual object using multiple inputs such as eye gaze, head pose, hand gesture, and totem input. Upon detecting an input convergence, the wearable system can perform a transmodal filtering scheme that leverages the converged inputs to assist in properly interpreting what command the user is providing or what object the user is targeting.
This disclosure describes in-plane switching mode liquid crystal geometric phase tunable lenses that can be integrated into an eyepiece of an optical device for the correction of non-emmetropic vision, such as in an augmented reality display system. The eyepiece can include an integrated, field-configurable optic arranged with respect to a waveguide used to project digital imagery to the user, the optic being capable of providing a tunable Rx for the user including variable spherical refractive power (SPH), cylinder refractive power, and cylinder axis values. In certain configuration, each tunable eyepiece includes two variable compound lenses: one on the user-side of the waveguide with variable SPH, cylinder power, and axis values; and a second on the world side of the waveguide with variable SPH.
G02C 7/08 - Auxiliary lensesArrangements for varying focal length
G02F 1/1347 - Arrangement of liquid crystal layers or cells in which the final condition of one light beam is achieved by the addition of the effects of two or more layers or cells
G02F 1/29 - Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulatingNon-linear optics for the control of the position or the direction of light beams, i.e. deflection
A method of fabricating a blazed diffraction grating comprises providing a master template substrate and imprinting periodically repeating lines on the master template substrate in a plurality of master template regions. The periodically repeating lines in different ones of the master template regions extend in different directions. The method additionally comprises using at least one of the master template regions as a master template to imprint at least one blazed diffraction grating pattern on a grating substrate.
A wearable system can comprise a display system configured to present virtual content in a three-dimensional space, a user input device configured to receive a user input, and one or more sensors configured to detect a user's pose. The wearable system can support various user interactions with objects in the user's environment based on contextual information. As an example, the wearable system can adjust the size of an aperture of a virtual cone during a cone cast (e.g., with the user's poses) based on the contextual information. As another example, the wearable system can adjust the amount of movement of virtual objects associated with an actuation of the user input device based on the contextual information.
G06T 19/00 - Manipulating 3D models or images for computer graphics
G06F 1/16 - Constructional details or arrangements
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/0346 - Pointing devices displaced or positioned by the userAccessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
An eyepiece includes a substrate, an input coupling grating on a first side of the substrate, and a morphed grating comprising characteristics of both a primary grating and a secondary grating on at least the first side of the substrate. The primary grating and the secondary grating may differ in pitch, orientation, and dimensions.
A thin transparent layer can be integrated in a head mounted display device and disposed in front of the eye of a wearer. The thin transparent layer may be configured to output light such that light is directed onto the eye to create reflections therefrom that can be used, for example, for glint based tracking. The thin transparent layer can be configured to reduced obstructions in the field of the view of the user.
A display device comprises a waveguide configured to guide light in a lateral direction parallel to an output surface of the waveguide. The waveguide is further configured to outcouple the guided light through the output surface. The display device additionally comprises a broadband adaptive lens assembly configured to incouple and to diffract therethrough the outcoupled light from the waveguide. The broadband adaptive lens assembly comprises a first waveplate lens comprising a liquid crystal (LC) layer arranged such that the waveplate lens has birefringence (Δn) that varies in a radially outward direction from a central region of the first waveplate lens and configured to diffract the outcoupled light at a diffraction efficiency greater than 90% within a wavelength range including at least 450 nm to 630 nm. The broadband adaptive lens assembly is configured to be selectively switched between a plurality of states having different optical powers.
G02B 6/12 - Light guidesStructural details of arrangements comprising light guides and other optical elements, e.g. couplings of the optical waveguide type of the integrated circuit kind
G02B 6/122 - Basic optical elements, e.g. light-guiding paths
Disclosed herein are systems and methods for generating and presenting virtual audio for mixed reality systems. A method may include determining a collision between a first object and a second object, wherein the first object comprises a first virtual object. A memory storing one or more audio models can be accessed. It can be determined if the one or more audio models stored in the memory comprises an audio model corresponding to the first object. In accordance with a determination that the one or more audio models comprises an audio model corresponding to the first object, an audio signal can be synthesized, wherein the audio signal is based on the collision and the audio model corresponding to the first object, and the audio signal can be presented to a user via a speaker of a head-wearable device. In accordance with a determination that the one or more audio models does not comprise an audio model corresponding to the first object, an acoustic property of the first object can be determined, a custom audio model based on the acoustic property of the first object can be generated, an audio signal can be synthesized, wherein the audio signal is based on the collision and the custom audio model, and the audio signal can be presented, via a speaker of a head-wearable device, to a user.
H04S 7/00 - Indicating arrangementsControl arrangements, e.g. balance control
A63F 13/285 - Generating tactile feedback signals via the game input device, e.g. force feedback
A63F 13/577 - Simulating properties, behaviour or motion of objects in the game world, e.g. computing tyre load in a car race game using determination of contact between game characters or objects, e.g. to avoid collision between virtual racing cars
