Depth imaging system implementing the method of any of the previous claims comprising: a. an imaging device (4) comprising a matrix of pixels (1) each pixel comprising a photodetector (9) capable of detecting single photons impinging thereon and optics able to make an image of the field of view on the matrix of pixel (1), said single photon detector having a binary logic status of true when a photon is detected and a logic status of false when no photon is detected in a timeframe; b. a projector (5) able to project a pattern in a time window of less than 10 sec, preferably less than 1 sec; c. a controller synchronising the projector (5) time window and the imaging device timeframe; d. a logic determining, in use, the presence, during the timeframe, of contiguous pixels (11) in the true state, and calculating the depth profile corresponding to said contiguous pixels (11).
H04N 13/25 - Image signal generators using stereoscopic image cameras using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristicsImage signal generators using stereoscopic image cameras using image signals from one sensor to control the characteristics of another sensor
The present invention relates to a high-speed imaging sensor system in which single-photon detectors are provided in an architecture adapted for high-speed processing of the output of the detectors with high reliability to filter out false positives.
The present invention relates to an optical sensing system (1) for depth estimation. The system (1) comprises at least a first optical sensor (2), wherein said first optical sensor (2) comprises a plurality of sensing units (3). Each of said sensing unit (3) comprises a photo detector, wherein said first optical sensor (2) has a first predetermined length and a first predetermined width. Said first optical sensor (2) having a first field of view (6). The system (1) further comprises at least a first projector (4), wherein said first projector (4) is adapted to project a first light beam (8) on said field of view (6). The system (1) further comprises a first scanning means. The system (1) further comprises a first set of optics able to make an image of said first field of view (6) on said first optical sensor (2). The system (1) is characterized in that said first scanning means is adapted to scan said first beam (8) in one angular degree of freedom along or within said first field of view (6).
G01B 11/25 - Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. moiré fringes, on the object
4.
METHOD FOR POINT CLOUD ACQUISITION AND PROCESSING, AND SYSTEM THEREOF
The present invention relates to a method (100) and system (10) for point cloud acquisition and processing using a 3D sensor that acquires a continuous stream of 3D points is provided, the 3D points being spatially distributed over a whole scene. The method comprises the steps of acquiring a continuous stream of points using a resolution scalable 3D sensor, wherein the continuous stream of points is partitioned in real-time, each partition (2a-c) is employed as input to an encoder (3a-c) from the moment the partition (2a-c) becomes available. The points of the continuous stream are acquired such that each subsequent partition provides an increased resolution compared to a previous partition. In a next step encoding of a point cloud of each partition (2a-c) to a feature vector (4a-c) is performed using a deep learning-based point cloud encoder (3a-c), preferably a transformer architecture. Finally there is a decoding the feature vector (4a-c) of each partition (2a-c) to an output prediction using a decoder (7a-c).
G06V 10/44 - Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersectionsConnectivity analysis, e.g. of connected components
G06V 10/82 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
The present invention relates to a method for 3D reconstruction of a scene (2). The method comprises the step of providing a first camera (3) having a known position and orientation. The method further comprises the step of determining a first set of projection lines corresponding to pixels (11) of said first camera (3), said projection lines being defined by the position and orientation of said camera (3) and the corresponding pixel (11). The method further comprises the step of projecting a light beam (13) on the scene (2) by means of a light source (12), thereby defining a dot (1) on said scene (2). The method further comprises the step of determining the pixel corresponding to an image of the dot (1) on said first camera (3), thereby defining a corresponding first projection line (9) from the first set of projection lines. The method further comprises the step of determining a second projection line (10) from a second set of projection lines either corresponding to instantaneous positions and orientations of the light beam (13) or to an image of the dot (1) on a second camera (4). The method further comprises the step of calculating a 3D location (17) of said dot (1) on the scene (2) based on said first and second projection lines (9, 10), wherein said 3D location (17) is defined in a region (18) in which the first projection line (9) and second projection line (10) are closest to each other.
The present invention relates to an eye tracking system (100). The system comprises at least one projection means (3) adapted to project a light beam (4) on a predetermined area of a cornea (2) of an eye (1). The system further comprises at least one optical sensor (6) capable of detecting reflected light (5) from the eye (1) and determining the location of detection thereon. The reflected light (5) from the eye (1) comprises at least one, preferably two, specular reflection, preferably at least a first Purkinje reflection (P1), more preferably at least a first Purkinje reflection (P1) and a fourth Purkinje reflection (P4). The system (100) is adapted to calculate the lateral and/or angular translation of the eye (1), or the change thereof, based on the location of detection and/or the displacement thereof.
The present invention relates to a system, detector element and method for high-speed imaging with SPD arrays, and a calibration routine for SPD arrays.
The present invention relates to an improved system and method for imaging, based on single photon detectors (SPDs), whereby false positives are suppressed.
The present invention relates to an optical sensing system (1) for 3D imaging. The system (1) comprises at least one optical sensor (2) comprising a plurality of sensing units (3), each of said sensing unit (3) comprising a photo detector, wherein said optical sensor (2) further comprises a timing module comprising a plurality of timing means, wherein each of said sensing units (3) corresponds to one of said timing means, or wherein said sensing units (3) are grouped in groups, wherein each group of said sensing units (3) corresponds to one of said timing means,. The system (1) further comprises at least one light source (5). The system (1) further comprises optics (3) able to produce an image of said scene (4) on said optical sensor (2). The system (1) further comprises a reading module. Each timing means is adapted to measure at least one first depth value, wherein said first depth value is mathematically related to the time between emitting a light pulse by the light source (5) and receiving said pulse by the optical sensing unit (3). The reading module is adapted to read said first depth value from said timing means only when the sensing unit (3) or said group of said sensing units (3) corresponding thereto have a positive detection status.
The present invention relates to an optical sensing system (1) for optical sensing. The system (1) comprises at least one optical sensor (30) comprising a plurality of sensing units (2'), preferably in a matrix configuration, wherein each of said sensing units (2') is preferably a pixel sensor (2'), wherein each sensing unit comprising a photo detector, wherein each photo detector is adapted to output a signal upon detection of a photon. The system (1) further comprises optics (3) able to produce an image of a scene (4) on said optical sensor (30). The system (2) further comprises a plurality of processing means (5'), wherein each of said processing means (5') is connected to at least one sensing unit (2') corresponding thereto. Each of said processing means (5') is adapted to receive at least one input corresponding to at least one output (7') of the corresponding sensing unit (2'). Each of said processing means (5') is adapted to integrate the output (7', 7'', 7''') of the corresponding sensing unit (2', 2'', 2''') to obtain a first integrated output (10), wherein said processing means is adapted to produce a first output signal (8) when the first integrated output (10) reaches at least a first predetermined value (9) within a first predetermined time span (∆t).
G01S 17/42 - Simultaneous measurement of distance and other coordinates
G06N 3/049 - Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
G06V 20/40 - ScenesScene-specific elements in video content
H04N 25/75 - Circuitry for providing, modifying or processing image signals from the pixel array
G06V 10/44 - Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersectionsConnectivity analysis, e.g. of connected components
G06V 10/82 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
The present invention relates to an imaging sensor system (26) having a high detection accuracy. The system (26) comprises a plurality of pixel sensors (1) and at least one light source. The pixel sensors (1) are grouped in clusters, wherein each pixel sensor (1) comprises a photodetector (2) and a local controlling circuit (18). The plurality of sensors (1) in each cluster are configured, using local controlling circuits (18), to output a global detection signal (15) when detecting a local detection signal (10) from at least two outputs of the photodetectors (2) of said cluster. The system (26) has a high detection accuracy due to ruling out false positive detections, since each detection requires at least two positive outputs of the photodetectors (2) of said cluster.
H04N 25/40 - Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled
H04N 25/773 - Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components comprising A/D, V/T, V/F, I/T or I/F converters comprising photon counting circuits, e.g. single photon detection [SPD] or single photon avalanche diodes [SPAD]
The present invention relates to an optical sensing system (1) for efficient optical sensing. The system (1) comprises at least one optical sensor (2), and optics (3) able to produce an image of a scene (4) on said optical sensor (2). The system (2) further comprises at least one light source (5), and scanning means (9) adapted to scan a light beam (33) of said light source (5) on said scene (4) along a trajectory (8). The system (1) further comprises a controller (12), wherein said controller (12) is adapted to vary the output optical power of said light source (5) between at least a first predetermined value (11) and a second predetermined value (10) higher than said first predetermined value.
The present invention relates to a method for sensing. The method comprises the step of identifying a second set of locations (2) of points of at least one object (3) in a scene (4) within a second time window (T2), with respect to a first reference point (28), such as preferably a first viewpoint. The method further comprises the step of converting said second set of locations (2) to a second set of 3D positions (6). The method further comprises the step of determining a transformation to said second set of 3D positions (6), such that said transformed second set of 3D positions (6) is a best fit to a first set of 3D positions (5), wherein said first set of 3D positions (5) is denser than said second set of 3D positions (6), and wherein said first set of 3D positions (5) is within a first time window (T1) prior to said second time window (T2), and wherein said second time window (T2) is shorter than said first time window (T1), preferably at least two times shorter.
The present invention relates to an image sensor (1) for efficient optical sensing. The sensor (1) comprises a plurality of pixel sensors (3), each pixel sensor (3) comprising a photo detector (4), each pixel being adapted to output a digital signal. The sensor (1) further comprises processing means, for example a neural network (5), comprising a plurality of neurons (6). Each photo detector (4) is in the vicinity of said processing means. Furthermore, each photo detector (4) is connected to the processing means.
H04N 25/773 - Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components comprising A/D, V/T, V/F, I/T or I/F converters comprising photon counting circuits, e.g. single photon detection [SPD] or single photon avalanche diodes [SPAD]
H04N 25/79 - Arrangements of circuitry being divided between different or multiple substrates, chips or circuit boards, e.g. stacked image sensors
G06N 3/04 - Architecture, e.g. interconnection topology
G06V 10/44 - Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersectionsConnectivity analysis, e.g. of connected components
G06N 3/049 - Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
The present invention relates to a high-speed imaging sensor system in which single-photon detectors are provided in an architecture adapted for high-speed processing of the output of the detectors with high reliability to filter out false positives.
H04N 5/378 - Readout circuits, e.g. correlated double sampling [CDS] circuits, output amplifiers or A/D converters
H04N 25/441 - Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled by partially reading an SSIS array by reading contiguous pixels from selected rows or columns of the array, e.g. interlaced scanning
H04N 25/626 - Reduction of noise due to residual charges remaining after image readout, e.g. to remove ghost images or afterimages
H04N 25/773 - Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components comprising A/D, V/T, V/F, I/T or I/F converters comprising photon counting circuits, e.g. single photon detection [SPD] or single photon avalanche diodes [SPAD]
Depth imaging system implementing the method of any of the previous claims comprising: a. an imaging device (4) comprising a matrix of pixels (1) each pixel comprising a photodetector (9) capable of detecting single photons impinging thereon and optics able to make an image of the field of view on the matrix of pixel (1), said single photon detector having a binary logic status of true when a photon is detected and a logic status of false when no photon is detected in a timeframe; b. a projector (5) able to project a pattern in a time window of less than 10µsec, preferably less than 1 µsec; c. a controller synchronising the projector (5) time window and the imaging device timeframe; d. a logic determining, in use, the presence, during the timeframe, of contiguous pixels (11) in the true state, and calculating the depth profile corresponding to said contiguous pixels (11).
G01B 11/25 - Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. moiré fringes, on the object
The present invention relates to a system, detector element and method for high-speed imaging with SPD arrays, and a calibration routine for SPD arrays.
Methods and systems for tracking an individual's eye, by tracking one or more ocular axes, are presented. The technique comprises the following: (i) illuminating the eye, over an area of the cornea extending over the pupil, with first and second incident light beams having a transverse cross sectional area smaller than a predetermined value with respect to an area of the pupil and propagating coaxially along a first optical path defined by central axes of the first and second incident light beams, wherein said first incident light beam is configured to be reflected from the cornea and said second incident light beam is configured to pass through the cornea and the pupil and to be reflected from a retina region of the eye; (ii) detecting respective first and second reflected light beams; (iii) adjusting the first optical path such that said first reflected light beam propagates along said first optical path and said second reflected light beam propagates along a second optical path having a predetermined spatial relationship with said first optical path whereby said predetermined spatial relationship is indicative of said ocular axis being along at least said first optical path; and (iv) tracking said ocular axis of the eye under changes in gaze direction of said eye by repeating (i) to (iii).
The present invention relates to an improved system and method for imaging, based on single photon detectors (SPDs), whereby false positives are suppressed.
The present invention relates to an imaging sensor system (26) having a high detection accuracy. The system (26) comprises a plurality of pixel sensors (1 ) and at least one light source. The pixel sensors (1 ) are grouped in clusters, wherein each pixel sensor (1 ) comprises a photodetector (2) and a local controlling circuit (18). The plurality of sensors (1 ) in each cluster are configured, using local controlling circuits (18), to output a global detection signal (15) when detecting a local detection signal (10) from at least two outputs of the photodetectors (2) of said cluster. The system (26) has a high detection accuracy due to ruling out false positive detections, since each detection requires at least two positive outputs of the photodetectors (2) of said cluster.
H04N 5/341 - Extracting pixel data from an image sensor by controlling scanning circuits, e.g. by modifying the number of pixels having been sampled or to be sampled
H04N 5/3745 - Addressed sensors, e.g. MOS or CMOS sensors having additional components embedded within a pixel or connected to a group of pixels within a sensor matrix, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components
21.
System and method for socially relevant user engagement indicator in augmented reality devices
Some embodiments of the presently disclosed subject matter are directed to an indicator device of user activity in Augmented Reality (AR). The indicator device comprises an indicator module being configured and operable to generate at least one signal being indicative of at least one social relevant parameter related to the user activity to at least one user's onlooker, and a monitoring module connectable to the indicator module and being configured and operable to receive and process data related to user activity, determine whether the user is in a certain condition corresponding to a certain social relevant parameter, and activate the indicator module upon identification of the certain user condition.
Some embodiments are directed to a system comprising a MEMS based actuator unit and a control electric circuit. The actuator unit comprising one or more MEMS actuators, each comprising a stator and a rotor and configured to define a payload position in response to electric potential between said stator and rotor. The electric circuit comprising one or more amplifiers configured to provide electric control signal to the one or more MEMS actuators to selectively vary position of said payload. The electric circuit comprises a sensing circuit configured for providing an alternating carrier signal and for monitoring said carrier signal to generate data on impedance of said one or more MEMS actuators indicative of position of the rotor with respect to the stator of said one or more MEMS actuators.
G02B 26/08 - Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
The present invention relates to a high-speed imaging sensor system in which single-photon detectors are provided in an architecture adapted for high-speed processing of the output of the detectors with high reliability to filter out false positives.
G01T 1/24 - Measuring radiation intensity with semiconductor detectors
H04N 5/374 - Addressed sensors, e.g. MOS or CMOS sensors
H04L 1/18 - Automatic repetition systems, e.g. Van Duuren systems
H04N 3/14 - Scanning details of television systemsCombination thereof with generation of supply voltages by means not exclusively optical-mechanical by means of electrically scanned solid-state devices
G01B 11/25 - Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. moiré fringes, on the object
H04N 5/335 - Transforming light or analogous information into electric information using solid-state image sensors [SSIS]
G01B 1/00 - Measuring instruments characterised by the selection of material therefor
A system and method for retinal image projection are disclosed. The system includes an image projection module that is configured and operable for projecting at least first and second image portions corresponding respectively to first and second parts of an image to be respectively projected on foveal and peripheral regions of an eye retina; and an optical multiplexing module optically coupled to the image projection module and configured and operable to spatially and/or temporally combine/multiplex the projections of the at least first and second image portions for generating a combined image projection output for propagation along a general common optical path which is to be directed towards said eye retina. The optical multiplexing module is configured and operable to multiplex the projections the first and second image portions such that a lateral extent of the combined projection is somewhat similar to a lateral extent of the second image portion in the combined projection which is intended to fall/cover the parafoveal regions of the eye retina, and the lateral extent of the first image portion in the combined projection being smaller and within the lateral extent of the combined projection. The first and second image portions may be projected with different qualities (different resolutions and/or different color depth) and/or with different framerates in accordance with the respective foveal and parafoveally regions of the retina to which there are intended, to the by optimize the image quality/frame rate, and the associated processing and projection capabilities to the anatomical structure of the retina.
Systems and methods for direct projection of images onto an eye retina including, for example, systems and methods for directing a projection/imaging optical path so as to track a location of the eye in accordance with a gaze direction thereof. This enables for projecting images onto specific/fixed locations on the eye retina, while the gaze direction changes.
A61B 3/113 - Objective types, i.e. instruments for examining the eyes independent of the patients perceptions or reactions for determining or recording eye movement
An image projection device for displaying an image onto a remote surface. The image projection device employs a scanner to project image beams of visible light and tracer beams of light onto a remote surface to form a display of the image. The device also employs a light detector to sense at least the reflections of light from the tracer beam pulses incident on the remote surface. The device employs the sensed tracer beam light pulses to predict the trajectory of subsequent image beam light pulses and tracer beam light pulses that form a display of the image on the remote surface in a pseudo random pattern. The trajectory of the projected image beam light pulses can be predicted so that the image is displayed from a point of view that can be selected by, or automatically adjusted for, a viewer of the displayed image.
G01B 11/14 - Measuring arrangements characterised by the use of optical techniques for measuring distance or clearance between spaced objects or spaced apertures
A display system with video see-through eye display unit is disclosed. The eye display unit includes: at least one camera unit; at least one image forming module; and an optical deflection module including at least one double sided light reflecting optical element which is at least partially reflecting to light arriving from both sides thereof. The camera unit is configured for collecting light arriving from a region of interest of a scene along a first optical path intersecting the optical deflection module, and generating image data indicative of the region of interest. The image forming module is configured for receiving imagery data indicative of images to be projected to an eye of a user, and generating and projecting the received images to propagate along a second optical path intersecting the optical deflection module. The double sided light reflecting optical element of the optical deflection module is arranged in the display system to be located in front of the eye while intersecting the first and second optical paths, and is oriented to define the first optical path (between the camera unit and the scene) and the second optical path (between the image forming unit and the eye), such that the camera unit has a point of view substantially similar to line of sight of the eye.
A system projects a user-viewable, computer-generated or -fed image, wherein a head-mounted projector is used to project an image onto a retro-reflective surface, so only the viewer can see the image. The projector is connected to a computer that contains software to create virtual 2-D and or 3-D images for viewing by the user. Further, one projector each is mounted on either side of the user's head, and, by choosing for example a retro angle of less than about 10 degrees, each eye can only see the image of one of the projectors at a given distance up to 3 meters, in this example, from the retro-reflective screen. The retro angle used may be reduced with larger viewing distance desired. These projectors use lasers to avoid the need for focusing, and in some cases these projectors use instead of lasers highly collimated LED light sources to avoid the need for focusing.
Systems and methods for direct projection of images onto an eye retina including, for example, systems and methods for directing a projection/imaging optical path so as to track a location of the eye in accordance with a gaze direction thereof. This enables for projecting images onto specific/fixed locations on the eye retina, while the gaze direction changes.
A61B 3/113 - Objective types, i.e. instruments for examining the eyes independent of the patients perceptions or reactions for determining or recording eye movement
An eye projection device is provide comprising an eye projection system configured for projecting a light beam to propagate along a propagation path to an eye. The eye projection system comprises: an optical assembly defining a general optical path of light in said optical assembly, the optical assembly comprising a deflector arrangement comprising one or more adjustable optical deflectors arranged along said general optical path and configured to define an adjustable propagation path of the light beam from the optical assembly to the eye; wherein said one or more optical deflectors are configured with at least three adjustable deflection parameters affecting deflection of said propagation path to provide at least three degrees of freedom in adjusting the propagation path of said light beam towards said eye; wherein two of said at least three degrees of freedom are associated with two angular orientations of the propagation path to the eye for compensating over angular changes in a gaze direction of the eye, and at least one of said at least three degrees of freedom are associated with a lateral deflection of the propagation path for compensating over variations in a relative lateral position of said projection system relative to the eye.
G02B 27/00 - Optical systems or apparatus not provided for by any of the groups ,
G09G 3/02 - Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes by tracing or scanning a light beam on a screen
31.
Method, apparatus, and manufacture for a tracking camera or detector with fast asynchronous triggering
An image projection device for displaying an image onto a remote surface. The image projection device employs a scanner to project image beams of visible light and tracer beams of light onto a remote surface to form a display of the image. The device also employs a light detector to sense at least the reflections of light from the tracer beam pulses incident on the remote surface. The device employs the sensed tracer beam light pulses to predict the trajectory of subsequent image beam light pulses and tracer beam light pulses that form a display of the image on the remote surface in a pseudo random pattern. The trajectory of the projected image beam light pulses can be predicted so that the image is displayed from a point of view that can be selected by, or automatically adjusted for, a viewer of the displayed image.
H04N 9/31 - Projection devices for colour picture display
G01B 11/14 - Measuring arrangements characterised by the use of optical techniques for measuring distance or clearance between spaced objects or spaced apertures
32.
Method, apparatus, and manufacture for a tracking camera or detector with fast asynchronous triggering
An image projection device for displaying an image onto a remote surface. The image projection device employs a scanner to project image beams of visible light and tracer beams of light onto a remote surface to form a display of the image. The device also employs a light detector to sense at least the reflections of light from the tracer beam pulses incident on the remote surface. The device employs the sensed tracer beam light pulses to predict the trajectory of subsequent image beam light pulses and tracer beam light pulses that form a display of the image on the remote surface in a pseudo random pattern. The trajectory of the projected image beam light pulses can be predicted so that the image is displayed from a point of view that can be selected by, or automatically adjusted for, a viewer of the displayed image.
H04N 9/31 - Projection devices for colour picture display
G01B 11/14 - Measuring arrangements characterised by the use of optical techniques for measuring distance or clearance between spaced objects or spaced apertures
33.
System and method for 3-D projection and enhancements for interactivity
A system projects a user-viewable, computer-generated or -fed image, wherein a head-mounted projector is used to project an image onto a retro-reflective surface, so only the viewer can see the image. The projector is connected to a computer that contains software to create virtual 2-D and or 3-D images for viewing by the user. Further, one projector each is mounted on either side of the user's head, and, by choosing for example a retro angle of less than about 10 degrees, each eye can only see the image of one of the projectors at a give distance up to 3 meters, in this example, from the retro-reflective screen. The retro angle used may be reduced with larger viewing distance desired. These projectors use lasers to avoid the need for focusing, and in some cases there projectors use instead of lasers highly collimated LED light sources to avoid the need for focusing.
patents
