A stored volumetric scene model of a real scene is generated from data defining digital images of a light field in a real scene containing different types of media. The digital images have been formed by a camera from opposingly directed poses and each digital image contains image data elements defined by stored data representing light field flux received by light sensing detectors in the camera. The digital images are processed by a scene reconstruction engine to form a digital volumetric scene model representing the real scene. The volumetric scene model (i) contains volumetric data elements defined by stored data representing one or more media characteristics and (ii) contains solid angle data elements defined by stored data representing the flux of the light field. Adjacent volumetric data elements form corridors, at least one of the volumetric data elements in at least one corridor represents media that is partially light transmissive. The constructed digital volumetric scene model data is stored in a digital data memory for subsequent uses and applications.
Methods and apparatus for systems using a scene codec are described, where systems are either providers or consumers of multi-way, just-in-time, only-as-needed scene data including subscenes and subscene increments. An example system using a scene codec includes a plenoptic scene database containing one or more digital models of scenes, where representations and organization of representations are distributable across multiple systems such that collectively the multiplicity of systems can represent scenes of almost unlimited detail. The system may further include highly efficient means for the processing of these representation and organizations of representation providing the just-in-time, only-as-needed subscenes and scene increments necessary for ensuring a maximally continuous user experience enabled by a minimal amount of newly provided scene information, where the highly efficient means include a spatial processing unit.
G06T 3/4092 - Transcodage de la résolution de l’image, p. ex. en utilisant des architectures client-serveur
G06T 3/067 - Remodelage ou dépliement de structures en arbre 3D sur des plans 2D
G06T 3/4076 - Changement d'échelle d’images complètes ou de parties d’image, p. ex. agrandissement ou rétrécissement basé sur la super-résolution, c.-à-d. où la résolution de l’image obtenue est plus élevée que la résolution du capteur utilisant les images originales basse résolution pour corriger itérativement les images haute résolution
G06T 9/40 - Codage sous forme arborescente, p. ex. à quatre branches, à huit branches
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
3.
SYSTEMS AND METHODS FOR GENERALIZED SCENE RECONSTRUCTION
Various embodiments of the disclosure are directed to a scene reconstruction and machine learning system. In embodiments, the system comprises a storage medium configured to store image data, one or more scene models, one or more relightable matter fields, and information related to a machine learning model. In one or more embodiments, the system comprises an input circuit configured to receive image data characterizing light in a scene. In embodiments the system includes a processor. In embodiments the processor configured to reconstruct a scene model representing the scene using the image data. In embodiments the processor is configured to extract a relightable matter field from the scene model representing the object, store the scene model and the relightable matter field representing the object in the storage medium, apply the relightable matter field as an input to the machine learning model, and generate an output from the machine learning model.
Various embodiments of the disclosure are directed to a scene reconstruction and machine learning system. In embodiments, the system comprises a storage medium configured to store image data, one or more scene models, one or more relightable matter fields, and information related to a machine learning model. In one or more embodiments, the system comprises an input circuit configured to receive image data characterizing light in a scene. In embodiments the system includes a processor. In embodiments the processor configured to reconstruct a scene model representing the scene using the image data. In embodiments the processor is configured to extract a relightable matter field from the scene model representing the object, store the scene model and the relightable matter field representing the object in the storage medium, apply the relightable matter field as an input to the machine learning model, and generate an output from the machine learning model.
G06V 10/60 - Extraction de caractéristiques d’images ou de vidéos relative aux propriétés luminescentes, p. ex. utilisant un modèle de réflectance ou d’éclairage
G06V 10/762 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant le regroupement, p. ex. de visages similaires sur les réseaux sociaux
G06V 10/764 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant la classification, p. ex. des objets vidéo
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
09 - Appareils et instruments scientifiques et électriques
42 - Services scientifiques, technologiques et industriels, recherche et conception
Produits et services
Downloadable or recorded computer programs, computer software and computer firmware, and computer hardware, all for extracting information from image or camera data; downloadable or recorded computer programs, computer software and computer firmware, and computer hardware, all for creating or transferring an electronic representation of one or more objects or scenes to a virtual environment including a metaverse, virtual reality world, digital twin environment and extended reality virtual environment Design, development, and consulting services related thereto in the field of computer programs, computer software, computer firmware, and computer hardware all for extracting information from image or camera data; research, design, development and consulting services in the field of artificial intelligence and visual cognition as it relates to processing image or camera data; design, development, and consulting services related thereto in the field of computer programs, computer software, computer firmware, and computer hardware all for creating or transferring an electronic representation of one or more objects or scenes to a virtual environment including a metaverse, virtual reality world, digital twin environment and extended reality virtual environment; research, design, development and consulting services in the field of artificial intelligence and visual cognition as it relates to creating or transferring an electronic representation of one or more objects or scenes to a virtual environment including a metaverse, virtual reality world, digital twin environment and extended reality virtual environment
09 - Appareils et instruments scientifiques et électriques
Produits et services
Downloadable or recorded computer programs, computer software and computer firmware, and computer hardware, all for extracting information from image or camera data; downloadable or recorded computer programs, computer software and computer firmware, and computer hardware, all for creating or transferring an electronic representation of one or more objects or scenes to a virtual environment including a metaverse, virtual reality world, digital twin environment and extended reality virtual environment
09 - Appareils et instruments scientifiques et électriques
42 - Services scientifiques, technologiques et industriels, recherche et conception
Produits et services
Downloadable or recorded computer programs for use in extracting information from camera data; Downloadable or recorded computer software for use in extracting information from camera data; Downloadable or recorded computer firmware for use in extracting information from camera data; Computer hardware for use in extracting information from camera data Providing access to temporary, online non-downloadable, computer programs for use in extracting information from camera data; Providing access to temporary, online non-downloadable computer software for use in extracting information from camera data; Design, development, and consulting services related thereto in the field of computer programs, computer software, computer firmware, and computer hardware all for extracting information from camera data; research and development in the field of artificial intelligence and visual cognition as it relates to processing camera data
8.
Codec for processing scenes of almost unlimited detail
Methods and apparatus for systems using a scene codec are described, where systems are either providers or consumers of multi-way, just-in-time, only-as-needed scene data including subscenes and subscene increments. An example system using a scene codec includes a plenoptic scene database containing one or more digital models of scenes, where representations and organization of representations are distributable across multiple systems such that collectively the multiplicity of systems can represent scenes of almost unlimited detail. The system may further include highly efficient means for the processing of these representation and organizations of representation providing the just-in-time, only-as-needed subscenes and scene increments necessary for ensuring a maximally continuous user experience enabled by a minimal amount of newly provided scene information, where the highly efficient means include a spatial processing unit.
09 - Appareils et instruments scientifiques et électriques
42 - Services scientifiques, technologiques et industriels, recherche et conception
Produits et services
Downloadable computer programs for extracting information from camera data; Recorded computer programs for extracting information from camera data; Downloadable computer software for extracting information from camera data; Recorded computer software for extracting information from camera data; Downloadable computer firmware for extracting information from camera data; Recorded computer firmware for extracting information from camera data; Computer hardware for use in extracting information from camera data Providing an interactive website featuring technology that enables users to access downloadable, recorded and temporary, online non-downloadable, computer programs for use in extracting information from camera data; Providing an interactive website featuring technology that enables users to access downloadable, recorded and temporary, online non-downloadable computer software for use in extracting information from camera data; Design, development, and consulting services related thereto in the field of computer programs, computer software, computer firmware, and computer hardware all for extracting information from camera data; research and development in the field of artificial intelligence and visual cognition as it relates to processing camera data
A stored volumetric scene model of a real scene is generated from data defining digital images of a light field in a real scene containing different types of media. The digital images have been formed by a camera from opposingly directed poses and each digital image contains image data elements defined by stored data representing light field flux received by light sensing detectors in the camera. The digital images are processed by a scene reconstruction engine to form a digital volumetric scene model representing the real scene. The volumetric scene model (i) contains volumetric data elements defined by stored data representing one or more media characteristics and (ii) contains solid angle data elements defined by stored data representing the flux of the light field. Adjacent volumetric data elements form corridors, at least one of the volumetric data elements in at least one corridor represents media that is partially light transmissive. The constructed digital volumetric scene model data is stored in a digital data memory for subsequent uses and applications.
09 - Appareils et instruments scientifiques et électriques
42 - Services scientifiques, technologiques et industriels, recherche et conception
Produits et services
Downloadable or recorded computer programs for use in extracting information from camera data; Downloadable or recorded computer software for use in extracting information from camera data; Downloadable or recorded computer firmware for use in extracting information from camera data; Computer hardware for use in extracting information from camera data Providing access to temporary, online non-downloadable, computer programs for use in extracting information from camera data; Providing access to temporary, online non-downloadable computer software for use in extracting information from camera data; Design, development, and consulting services related thereto in the field of computer programs, computer software, computer firmware, and computer hardware all for extracting information from camera data; research and development in the field of artificial intelligence and visual cognition as it relates to processing camera data
12.
A CODEC FOR PROCESSING SCENES OF ALMOST UNLIMITED DETAIL
Methods and apparatus for systems using a scene codec are described, where systems are either providers or consumers of multi-way, just-in-time, only-as-needed scene data including subscenes and subscene increments. An example system using a scene codec comprises aplenoptic scene database containing one or more digital models of scenes, where representations and organization of representations are distributable across multiple systems such that collectively the multiplicity of systems can represent scenes of almost unlimited detail. The system may further include highly efficient means for the processing of these representation and organizations of representation providing the just-in-time, only- as-needed subscenes and scene increments necessary for ensuring a maximally continuous user experience enabled by a minimal amount of newly provided scene information, where the highly efficient means include a spatial processing unit.
Methods and apparatus for systems using a scene codec are described, where systems are either providers or consumers of multi-way, just-in-time, only-as-needed scene data including subscenes and subscene increments. An example system using a scene codec comprises aplenoptic scene database containing one or more digital models of scenes, where representations and organization of representations are distributable across multiple systems such that collectively the multiplicity of systems can represent scenes of almost unlimited detail. The system may further include highly efficient means for the processing of these representation and organizations of representation providing the just-in-time, only- as-needed subscenes and scene increments necessary for ensuring a maximally continuous user experience enabled by a minimal amount of newly provided scene information, where the highly efficient means include a spatial processing unit.
G06T 7/149 - DécoupageDétection de bords impliquant des modèles déformables, p. ex. des modèles de contours actifs
G06T 7/44 - Analyse de la texture basée sur la description statistique de texture utilisant des opérateurs de l'image, p. ex. des filtres, des mesures de densité des bords ou des histogrammes locaux
G06T 7/521 - Récupération de la profondeur ou de la forme à partir de la télémétrie laser, p. ex. par interférométrieRécupération de la profondeur ou de la forme à partir de la projection de lumière structurée
G06T 7/557 - Récupération de la profondeur ou de la forme à partir de plusieurs images à partir des champs de lumière, p. ex. de caméras plénoptiques
G06T 7/586 - Récupération de la profondeur ou de la forme à partir de plusieurs images à partir de plusieurs sources de lumière, p. ex. stéréophotométrie
09 - Appareils et instruments scientifiques et électriques
42 - Services scientifiques, technologiques et industriels, recherche et conception
Produits et services
Downloadable or recorded computer programs for use in extracting information from camera data; Downloadable or recorded computer software for use in extracting information from camera data; Downloadable or recorded computer firmware for use in extracting information from camera data; Computer hardware for use in extracting information from camera data Providing access to temporary, online non-downloadable, computer programs for use in extracting information from camera data; Providing access to temporary, online non-downloadable computer software for use in extracting information from camera data; Design, development, and consulting services related thereto in the field of computer programs, computer software, computer firmware, and computer hardware all for extracting information from camera data; research and development in the field of artificial intelligence and visual cognition as it relates to processing camera data
A stored volumetric scene model of a real scene is generated from data defining digital images of a light field in a real scene containing different types of media. The digital images have been formed by a camera from opposingly directed poses and each digital image contains image data elements defined by stored data representing light field flux received by light sensing detectors in the camera. The digital images are processed by a scene reconstruction engine to form a digital volumetric scene model representing the real scene. The volumetric scene model (i) contains volumetric data elements defined by stored data representing one or more media characteristics and (ii) contains solid angle data elements defined by stored data representing the flux of the light field. Adjacent volumetric data elements form corridors, at least one of the volumetric data elements in at least one corridor represents media that is partially light transmissive. The constructed digital volumetric scene model data is stored in a digital data memory for subsequent uses and applications.
A stored volumetric scene model of a real scene is generated from data defining digital images of a light field in a real scene containing different types of media. The digital images have been formed by a camera from opposingly directed poses and each digital image contains image data elements defined by stored data representing light field flux received by light sensing detectors in the camera. The digital images are processed by a scene reconstruction engine to form a digital volumetric scene model representing the real scene. The volumetric scene model (i) contains volumetric data elements defined by stored data representing one or more media characteristics and (ii) contains solid angle data elements defined by stored data representing the flux of the light field. Adjacent volumetric data elements form corridors, at least one of the volumetric data elements in at least one corridor represents media that is partially light transmissive. The constructed digital volumetric scene model data is stored in a digital data memory for subsequent uses and applications.
H04N 5/76 - Enregistrement du signal de télévision
G06T 7/55 - Récupération de la profondeur ou de la forme à partir de plusieurs images
G06T 7/557 - Récupération de la profondeur ou de la forme à partir de plusieurs images à partir des champs de lumière, p. ex. de caméras plénoptiques
G06T 9/40 - Codage sous forme arborescente, p. ex. à quatre branches, à huit branches
G02B 1/08 - Éléments optiques caractérisés par la substance dont ils sont faitsRevêtements optiques pour éléments optiques faits de substances polarisantes
A stored volumetric scene model of a real scene is generated from data defining digital images of a light field in a real scene containing different types of media. The digital images have been formed by a camera from opposingly directed poses and each digital image contains image data elements defined by stored data representing light field flux received by light sensing detectors in the camera. The digital images are processed by a scene reconstruction engine to form a digital volumetric scene model representing the real scene. The volumetric scene model (i) contains volumetric data elements defined by stored data representing one or more media characteristics and (ii) contains solid angle data elements defined by stored data representing the flux of the light field. Adjacent volumetric data elements form corridors, at least one of the volumetric data elements in at least one corridor represents media that is partially light transmissive. The constructed digital volumetric scene model data is stored in a digital data memory for subsequent uses and applications.
H04N 23/951 - Systèmes de photographie numérique, p. ex. systèmes d'imagerie par champ lumineux en utilisant plusieurs images pour influencer la résolution, la fréquence d'images ou le rapport de cadre
G06T 7/55 - Récupération de la profondeur ou de la forme à partir de plusieurs images
G06T 7/557 - Récupération de la profondeur ou de la forme à partir de plusieurs images à partir des champs de lumière, p. ex. de caméras plénoptiques
G01N 21/95 - Recherche de la présence de criques, de défauts ou de souillures caractérisée par le matériau ou la forme de l'objet à analyser
G06T 9/40 - Codage sous forme arborescente, p. ex. à quatre branches, à huit branches
ABSTRACT A stored volumetric scene model of a real scene is generated from data defining digital images of a light field in a real scene containing different types of media. The digital images have been fomied by a camera from opposingly directed poses and each digital image contains image data elements defined by stored data representing light field flux received by light sensing detectors in the camera. The digital images are processed by a scene reconstruction engine to form a digital volumetric scene model representing the real scene. The volumetric scene model (i) contains volumetric data elements defined by stored data representing one or more media characteristics and (ii) contains solid angle data elements defined by stored data representing the flux of the light field. Adjacent volumetric data elements fonn corridors, at least one of the volumetric data elements in at least one corridor represents media that is partially light transmissive. The constructed digital volumetric scene model data is stored in a digital data memory for subsequent uses and applications. Date Recue/Date Received 2023-09-22
09 - Appareils et instruments scientifiques et électriques
42 - Services scientifiques, technologiques et industriels, recherche et conception
Produits et services
Computer programs, computer software, computer firmware, and computer hardware all for extracting information from camera data Design, development, and consulting services related thereto in the field of computer programs, computer software, computer firmware, and computer hardware all for extracting information from camera data; research and development in the field of artificial intelligence and visual cognition as it relates to processing camera data
