A stored volumetric scene model of a real scene is generated from data defining digital images of a light field in a real scene containing different types of media. The digital images have been formed by a camera from opposingly directed poses and each digital image contains image data elements defined by stored data representing light field flux received by light sensing detectors in the camera. The digital images are processed by a scene reconstruction engine to form a digital volumetric scene model representing the real scene. The volumetric scene model (i) contains volumetric data elements defined by stored data representing one or more media characteristics and (ii) contains solid angle data elements defined by stored data representing the flux of the light field. Adjacent volumetric data elements form corridors, at least one of the volumetric data elements in at least one corridor represents media that is partially light transmissive. The constructed digital volumetric scene model data is stored in a digital data memory for subsequent uses and applications.
Methods and apparatus for systems using a scene codec are described, where systems are either providers or consumers of multi-way, just-in-time, only-as-needed scene data including subscenes and subscene increments. An example system using a scene codec includes a plenoptic scene database containing one or more digital models of scenes, where representations and organization of representations are distributable across multiple systems such that collectively the multiplicity of systems can represent scenes of almost unlimited detail. The system may further include highly efficient means for the processing of these representation and organizations of representation providing the just-in-time, only-as-needed subscenes and scene increments necessary for ensuring a maximally continuous user experience enabled by a minimal amount of newly provided scene information, where the highly efficient means include a spatial processing unit.
G06T 3/4092 - Image resolution transcoding, e.g. by using client-server architectures
G06T 3/067 - Reshaping or unfolding 3D tree structures onto 2D planes
G06T 3/4076 - Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution using the original low-resolution images to iteratively correct the high-resolution images
Various embodiments of the disclosure are directed to a scene reconstruction and machine learning system. In embodiments, the system comprises a storage medium configured to store image data, one or more scene models, one or more relightable matter fields, and information related to a machine learning model. In one or more embodiments, the system comprises an input circuit configured to receive image data characterizing light in a scene. In embodiments the system includes a processor. In embodiments the processor configured to reconstruct a scene model representing the scene using the image data. In embodiments the processor is configured to extract a relightable matter field from the scene model representing the object, store the scene model and the relightable matter field representing the object in the storage medium, apply the relightable matter field as an input to the machine learning model, and generate an output from the machine learning model.
Various embodiments of the disclosure are directed to a scene reconstruction and machine learning system. In embodiments, the system comprises a storage medium configured to store image data, one or more scene models, one or more relightable matter fields, and information related to a machine learning model. In one or more embodiments, the system comprises an input circuit configured to receive image data characterizing light in a scene. In embodiments the system includes a processor. In embodiments the processor configured to reconstruct a scene model representing the scene using the image data. In embodiments the processor is configured to extract a relightable matter field from the scene model representing the object, store the scene model and the relightable matter field representing the object in the storage medium, apply the relightable matter field as an input to the machine learning model, and generate an output from the machine learning model.
G06V 10/60 - Extraction of image or video features relating to illumination properties, e.g. using a reflectance or lighting model
G06V 10/762 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
G06V 10/764 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
G06V 10/82 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
09 - Scientific and electric apparatus and instruments
42 - Scientific, technological and industrial services, research and design
Goods & Services
Downloadable or recorded computer programs, computer software and computer firmware, and computer hardware, all for extracting information from image or camera data; downloadable or recorded computer programs, computer software and computer firmware, and computer hardware, all for creating or transferring an electronic representation of one or more objects or scenes to a virtual environment including a metaverse, virtual reality world, digital twin environment and extended reality virtual environment Design, development, and consulting services related thereto in the field of computer programs, computer software, computer firmware, and computer hardware all for extracting information from image or camera data; research, design, development and consulting services in the field of artificial intelligence and visual cognition as it relates to processing image or camera data; design, development, and consulting services related thereto in the field of computer programs, computer software, computer firmware, and computer hardware all for creating or transferring an electronic representation of one or more objects or scenes to a virtual environment including a metaverse, virtual reality world, digital twin environment and extended reality virtual environment; research, design, development and consulting services in the field of artificial intelligence and visual cognition as it relates to creating or transferring an electronic representation of one or more objects or scenes to a virtual environment including a metaverse, virtual reality world, digital twin environment and extended reality virtual environment
09 - Scientific and electric apparatus and instruments
Goods & Services
Downloadable or recorded computer programs, computer software and computer firmware, and computer hardware, all for extracting information from image or camera data; downloadable or recorded computer programs, computer software and computer firmware, and computer hardware, all for creating or transferring an electronic representation of one or more objects or scenes to a virtual environment including a metaverse, virtual reality world, digital twin environment and extended reality virtual environment
09 - Scientific and electric apparatus and instruments
42 - Scientific, technological and industrial services, research and design
Goods & Services
Downloadable or recorded computer programs for use in extracting information from camera data; Downloadable or recorded computer software for use in extracting information from camera data; Downloadable or recorded computer firmware for use in extracting information from camera data; Computer hardware for use in extracting information from camera data Providing access to temporary, online non-downloadable, computer programs for use in extracting information from camera data; Providing access to temporary, online non-downloadable computer software for use in extracting information from camera data; Design, development, and consulting services related thereto in the field of computer programs, computer software, computer firmware, and computer hardware all for extracting information from camera data; research and development in the field of artificial intelligence and visual cognition as it relates to processing camera data
8.
Codec for processing scenes of almost unlimited detail
Methods and apparatus for systems using a scene codec are described, where systems are either providers or consumers of multi-way, just-in-time, only-as-needed scene data including subscenes and subscene increments. An example system using a scene codec includes a plenoptic scene database containing one or more digital models of scenes, where representations and organization of representations are distributable across multiple systems such that collectively the multiplicity of systems can represent scenes of almost unlimited detail. The system may further include highly efficient means for the processing of these representation and organizations of representation providing the just-in-time, only-as-needed subscenes and scene increments necessary for ensuring a maximally continuous user experience enabled by a minimal amount of newly provided scene information, where the highly efficient means include a spatial processing unit.
09 - Scientific and electric apparatus and instruments
42 - Scientific, technological and industrial services, research and design
Goods & Services
Downloadable computer programs for extracting information from camera data; Recorded computer programs for extracting information from camera data; Downloadable computer software for extracting information from camera data; Recorded computer software for extracting information from camera data; Downloadable computer firmware for extracting information from camera data; Recorded computer firmware for extracting information from camera data; Computer hardware for use in extracting information from camera data Providing an interactive website featuring technology that enables users to access downloadable, recorded and temporary, online non-downloadable, computer programs for use in extracting information from camera data; Providing an interactive website featuring technology that enables users to access downloadable, recorded and temporary, online non-downloadable computer software for use in extracting information from camera data; Design, development, and consulting services related thereto in the field of computer programs, computer software, computer firmware, and computer hardware all for extracting information from camera data; research and development in the field of artificial intelligence and visual cognition as it relates to processing camera data
A stored volumetric scene model of a real scene is generated from data defining digital images of a light field in a real scene containing different types of media. The digital images have been formed by a camera from opposingly directed poses and each digital image contains image data elements defined by stored data representing light field flux received by light sensing detectors in the camera. The digital images are processed by a scene reconstruction engine to form a digital volumetric scene model representing the real scene. The volumetric scene model (i) contains volumetric data elements defined by stored data representing one or more media characteristics and (ii) contains solid angle data elements defined by stored data representing the flux of the light field. Adjacent volumetric data elements form corridors, at least one of the volumetric data elements in at least one corridor represents media that is partially light transmissive. The constructed digital volumetric scene model data is stored in a digital data memory for subsequent uses and applications.
09 - Scientific and electric apparatus and instruments
42 - Scientific, technological and industrial services, research and design
Goods & Services
Downloadable or recorded computer programs for use in extracting information from camera data; Downloadable or recorded computer software for use in extracting information from camera data; Downloadable or recorded computer firmware for use in extracting information from camera data; Computer hardware for use in extracting information from camera data Providing access to temporary, online non-downloadable, computer programs for use in extracting information from camera data; Providing access to temporary, online non-downloadable computer software for use in extracting information from camera data; Design, development, and consulting services related thereto in the field of computer programs, computer software, computer firmware, and computer hardware all for extracting information from camera data; research and development in the field of artificial intelligence and visual cognition as it relates to processing camera data
12.
A CODEC FOR PROCESSING SCENES OF ALMOST UNLIMITED DETAIL
Methods and apparatus for systems using a scene codec are described, where systems are either providers or consumers of multi-way, just-in-time, only-as-needed scene data including subscenes and subscene increments. An example system using a scene codec comprises aplenoptic scene database containing one or more digital models of scenes, where representations and organization of representations are distributable across multiple systems such that collectively the multiplicity of systems can represent scenes of almost unlimited detail. The system may further include highly efficient means for the processing of these representation and organizations of representation providing the just-in-time, only- as-needed subscenes and scene increments necessary for ensuring a maximally continuous user experience enabled by a minimal amount of newly provided scene information, where the highly efficient means include a spatial processing unit.
Methods and apparatus for systems using a scene codec are described, where systems are either providers or consumers of multi-way, just-in-time, only-as-needed scene data including subscenes and subscene increments. An example system using a scene codec comprises aplenoptic scene database containing one or more digital models of scenes, where representations and organization of representations are distributable across multiple systems such that collectively the multiplicity of systems can represent scenes of almost unlimited detail. The system may further include highly efficient means for the processing of these representation and organizations of representation providing the just-in-time, only- as-needed subscenes and scene increments necessary for ensuring a maximally continuous user experience enabled by a minimal amount of newly provided scene information, where the highly efficient means include a spatial processing unit.
G06T 7/149 - SegmentationEdge detection involving deformable models, e.g. active contour models
G06T 7/44 - Analysis of texture based on statistical description of texture using image operators, e.g. filters, edge density metrics or local histograms
G06T 7/521 - Depth or shape recovery from laser ranging, e.g. using interferometryDepth or shape recovery from the projection of structured light
G06T 7/557 - Depth or shape recovery from multiple images from light fields, e.g. from plenoptic cameras
G06T 7/586 - Depth or shape recovery from multiple images from multiple light sources, e.g. photometric stereo
09 - Scientific and electric apparatus and instruments
42 - Scientific, technological and industrial services, research and design
Goods & Services
Downloadable or recorded computer programs for use in extracting information from camera data; Downloadable or recorded computer software for use in extracting information from camera data; Downloadable or recorded computer firmware for use in extracting information from camera data; Computer hardware for use in extracting information from camera data Providing access to temporary, online non-downloadable, computer programs for use in extracting information from camera data; Providing access to temporary, online non-downloadable computer software for use in extracting information from camera data; Design, development, and consulting services related thereto in the field of computer programs, computer software, computer firmware, and computer hardware all for extracting information from camera data; research and development in the field of artificial intelligence and visual cognition as it relates to processing camera data
A stored volumetric scene model of a real scene is generated from data defining digital images of a light field in a real scene containing different types of media. The digital images have been formed by a camera from opposingly directed poses and each digital image contains image data elements defined by stored data representing light field flux received by light sensing detectors in the camera. The digital images are processed by a scene reconstruction engine to form a digital volumetric scene model representing the real scene. The volumetric scene model (i) contains volumetric data elements defined by stored data representing one or more media characteristics and (ii) contains solid angle data elements defined by stored data representing the flux of the light field. Adjacent volumetric data elements form corridors, at least one of the volumetric data elements in at least one corridor represents media that is partially light transmissive. The constructed digital volumetric scene model data is stored in a digital data memory for subsequent uses and applications.
A stored volumetric scene model of a real scene is generated from data defining digital images of a light field in a real scene containing different types of media. The digital images have been formed by a camera from opposingly directed poses and each digital image contains image data elements defined by stored data representing light field flux received by light sensing detectors in the camera. The digital images are processed by a scene reconstruction engine to form a digital volumetric scene model representing the real scene. The volumetric scene model (i) contains volumetric data elements defined by stored data representing one or more media characteristics and (ii) contains solid angle data elements defined by stored data representing the flux of the light field. Adjacent volumetric data elements form corridors, at least one of the volumetric data elements in at least one corridor represents media that is partially light transmissive. The constructed digital volumetric scene model data is stored in a digital data memory for subsequent uses and applications.
ABSTRACT A stored volumetric scene model of a real scene is generated from data defining digital images of a light field in a real scene containing different types of media. The digital images have been fomied by a camera from opposingly directed poses and each digital image contains image data elements defined by stored data representing light field flux received by light sensing detectors in the camera. The digital images are processed by a scene reconstruction engine to form a digital volumetric scene model representing the real scene. The volumetric scene model (i) contains volumetric data elements defined by stored data representing one or more media characteristics and (ii) contains solid angle data elements defined by stored data representing the flux of the light field. Adjacent volumetric data elements fonn corridors, at least one of the volumetric data elements in at least one corridor represents media that is partially light transmissive. The constructed digital volumetric scene model data is stored in a digital data memory for subsequent uses and applications. Date Recue/Date Received 2023-09-22
A stored volumetric scene model of a real scene is generated from data defining digital images of a light field in a real scene containing different types of media. The digital images have been formed by a camera from opposingly directed poses and each digital image contains image data elements defined by stored data representing light field flux received by light sensing detectors in the camera. The digital images are processed by a scene reconstruction engine to form a digital volumetric scene model representing the real scene. The volumetric scene model (i) contains volumetric data elements defined by stored data representing one or more media characteristics and (ii) contains solid angle data elements defined by stored data representing the flux of the light field. Adjacent volumetric data elements form corridors, at least one of the volumetric data elements in at least one corridor represents media that is partially light transmissive. The constructed digital volumetric scene model data is stored in a digital data memory for subsequent uses and applications.
H04N 23/951 - Computational photography systems, e.g. light-field imaging systems by using two or more images to influence resolution, frame rate or aspect ratio
G06T 7/55 - Depth or shape recovery from multiple images
G06T 7/557 - Depth or shape recovery from multiple images from light fields, e.g. from plenoptic cameras
G01N 21/95 - Investigating the presence of flaws, defects or contamination characterised by the material or shape of the object to be examined
09 - Scientific and electric apparatus and instruments
42 - Scientific, technological and industrial services, research and design
Goods & Services
Computer programs, computer software, computer firmware, and computer hardware all for extracting information from camera data Design, development, and consulting services related thereto in the field of computer programs, computer software, computer firmware, and computer hardware all for extracting information from camera data; research and development in the field of artificial intelligence and visual cognition as it relates to processing camera data
