System and method are provided for scaling a 3-D representation of a building structure. The method includes obtaining images of the building structure, including non-camera anchors. The method also includes identifying reference poses for images based on the non-camera anchors. The method also includes obtaining world map data including real-world poses for the images. The method also includes selecting candidate poses from the real-world poses based on corresponding reference poses. The method also includes calculating a scaling factor for a 3-D representation of the building structure based on correlating the reference poses with the selected candidate poses. Some implementations use structure from motion techniques or LiDAR, in addition to augmented reality frameworks, for scaling the 3-D representations of the building structure. In some implementations, the world map data includes environmental data, such as illumination data, and the method includes generating or displaying the 3-D representation.
Methods, systems, and storage media for determining observation coverage of an observed scene by a sensing device are disclosed. A plurality of features are captured by a sensing device from a plurality of poses. An occupancy grid comprising occupancy cells is generated. When an occupancy cell comprises a feature, that cell is designated as occupied, and occupancy cells between the occupied cell and a cell comprising an associated sensing device pose are also designated as occupied.
An example implementation provides a method, including: receiving, using a set of one or more processors, a real-world two-dimensional (2D) image of a scene; obtaining, using the set of one or more processors, a depth map for the real-world 2D image, the depth map comprising depth estimates for pixels of the real-world 2D image; identifying, using the set of one or more processors, a subset of pixels in the real-world 2D image associated with a predetermined geometric feature type associated with a three-dimensional (3D) object of the scene; modifying, using the set of one or more processors, depth estimates for select pixels of the real-world 2D image based on the subset of pixels; and creating, using the set of one or more processors, a representation of the scene based on the modified depth estimates.
Images subject to panoramic image stitching are analyzed for suitability in three dimensional construction. Effects of translation changes between cameras are mitigated. The dioptric relationship of subject content of the images to the camera set identifies camera translation changes, and eligible camera position inputs, permissible to avoid parallax errors. Images within permissible ranges of each other are stitched to create panoramic images of subject content that a camera cannot fully capture with a single image. Registration plane selection based on translation distances between cameras, and permissible translation changes based on detected subject content depth are disclosed.
H04N 23/698 - Commande des caméras ou des modules de caméras pour obtenir un champ de vision élargi, p. ex. pour la capture d'images panoramiques
H04N 23/80 - Chaînes de traitement de la caméraLeurs composants
H04N 23/959 - Systèmes de photographie numérique, p. ex. systèmes d'imagerie par champ lumineux pour l'imagerie à grande profondeur de champ en ajustant la profondeur de champ pendant la capture de l'image, p. ex. en maximisant ou en réglant la portée en fonction des caractéristiques de la scène
Techniques are described for identifying correspondences between images to generate a fundamental matrix for the camera positions related to the images. The resultant fundamental matrix enables epipolar geometry to correlate common features among the images. Correspondences are identified by confirming feature matches across images by applying a homography to data representing features across images. Further techniques are described herein for generating a representation of a boundary of a feature of a structure based on a digital image. In one or more embodiments, generating a representation of a boundary of a particular feature in a digital image comprises determining a portion of the image that corresponds to the structure, and determining a portion of the image that corresponds to the particular feature. One more vanishing points are associated with the portion of the image corresponding to the particular feature. The one or more vanishing points are used to generate a set of bounding lines for the particular feature, based on which the boundary indicator for the feature is generated.
G06T 7/543 - Récupération de la profondeur ou de la forme à partir des lignes dessinées
G06T 7/33 - Détermination des paramètres de transformation pour l'alignement des images, c.-à-d. recalage des images utilisant des procédés basés sur les caractéristiques
G06T 7/73 - Détermination de la position ou de l'orientation des objets ou des caméras utilisant des procédés basés sur les caractéristiques
6.
CALIBRATING A MONOCULAR DEPTH MAP BASED ON A THREE-DIMENSIONAL (3D) MODEL
An example provides a method, including: receiving a real-world two-dimensional (2D) image of a scene (410); obtaining, using a set of one or more processors, a depth map for the real-world 2D image, the depth map being selectively corrected using a second depth map formed using a three-dimensional (3-D) model of the scene; and creating, using the set of one or more processors (420), a representation of the scene based on the depth map selectively corrected by the 3-D model (430).
An example provides a method (100), including: obtaining, using a set of one or more processors, first depth map data and second depth map data derived from respective depth maps generated for different two-dimensional (2D) images of a scene (102); selecting, using the set of one or more processors, a subset of three-dimensional (3D) points derived from the respective depth maps to adjust one or more of the first depth map data and the second depth map data (104); and adjusting, using the set of one or more processors, the one or more of the first depth map data and the second depth map data based on the subset of the 3D points selected (106).
An example provides a method, including: receiving a real-world two-dimensional (2D) image of a scene; obtaining, using a set of one or more processors, a depth map for the real-world 2D image, the depth map being selectively corrected using a second depth map formed using a three-dimensional (3-D) model of the scene; and creating, using the set of one or more processors, a representation of the scene based on the depth map selectively corrected by the 3-D model.
An example provides a method, including: obtaining, using a set of one or more processors, first depth map data and second depth map data derived from respective depth maps generated for different two-dimensional (2D) images of a scene; selecting, using the set of one or more processors, a subset of three-dimensional (3D) points derived from the respective depth maps to adjust one or more of the first depth map data and the second depth map data; and adjusting, using the set of one or more processors, the one or more of the first depth map data and the second depth map data based on the subset of the 3D points selected.
G06T 7/33 - Détermination des paramètres de transformation pour l'alignement des images, c.-à-d. recalage des images utilisant des procédés basés sur les caractéristiques
G06T 7/55 - Récupération de la profondeur ou de la forme à partir de plusieurs images
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
10.
TECHNIQUES FOR ENHANCED IMAGE CAPTURE USING A COMPUTER-VISION NETWORK
Disclosed are techniques for enhancing two-dimensional (2D) image capture of subjects (e.g., a physical structure, such as a residential building) to maximize the feature correspondences available for three-dimensional (3D) model reconstruction. More specifically, disclosed is a computer-vision network configured to provide viewfinder interfaces and analyses to guide the improved capture of an intended subject for specified purposes. Additionally, the computer-vision network can be configured to generate a metric representing a quality of feature correspondences between images of a complete set of images used for reconstructing a 3D model of a physical structure. The computer-vision network can also be configured to generate feedback at or before image capture time to guide improvements to the quality of feature correspondences between a pair of images.
G06T 17/00 - Modélisation tridimensionnelle [3D] pour infographie
G06V 10/26 - Segmentation de formes dans le champ d’imageDécoupage ou fusion d’éléments d’image visant à établir la région de motif, p. ex. techniques de regroupementDétection d’occlusion
G06V 10/44 - Extraction de caractéristiques locales par analyse des parties du motif, p. ex. par détection d’arêtes, de contours, de boucles, d’angles, de barres ou d’intersectionsAnalyse de connectivité, p. ex. de composantes connectées
G06V 30/19 - Reconnaissance utilisant des moyens électroniques
G06V 30/414 - Extraction de la structure géométrique, p. ex. arborescenceDécoupage en blocs, p. ex. boîtes englobantes pour les éléments graphiques ou textuels
H04N 23/63 - Commande des caméras ou des modules de caméras en utilisant des viseurs électroniques
11.
TECHNIQUES FOR ENHANCED IMAGE CAPTURE USING A COMPUTER-VISION NETWORK
Disclosed are techniques for enhancing two-dimensional (2D) image capture of subjects (e.g., a physical structure, such as a residential building) to maximize the feature correspondences available for three-dimensional (3D) model reconstruction. More specifically, disclosed is a computer-vision network configured to provide viewfinder interfaces and analyses to guide the improved capture of an intended subject for specified purposes. Additionally, the computer-vision network can be configured to generate a metric representing a quality of feature correspondences between images of a complete set of images used for reconstructing a 3D model of a physical structure. The computer-vision network can also be configured to generate feedback at or before image capture time to guide improvements to the quality of feature correspondences between a pair of images.
G06T 17/00 - Modélisation tridimensionnelle [3D] pour infographie
G06V 10/26 - Segmentation de formes dans le champ d’imageDécoupage ou fusion d’éléments d’image visant à établir la région de motif, p. ex. techniques de regroupementDétection d’occlusion
G06V 10/44 - Extraction de caractéristiques locales par analyse des parties du motif, p. ex. par détection d’arêtes, de contours, de boucles, d’angles, de barres ou d’intersectionsAnalyse de connectivité, p. ex. de composantes connectées
G06V 30/19 - Reconnaissance utilisant des moyens électroniques
G06V 30/414 - Extraction de la structure géométrique, p. ex. arborescenceDécoupage en blocs, p. ex. boîtes englobantes pour les éléments graphiques ou textuels
H04N 23/63 - Commande des caméras ou des modules de caméras en utilisant des viseurs électroniques
12.
TECHNIQUES FOR ENHANCED IMAGE CAPTURE USING A COMPUTER-VISION NETWORK
Disclosed are techniques for enhancing two-dimensional (2D) image capture of subjects (e.g., a physical structure, such as a residential building) to maximize the feature correspondences available for three-dimensional (3D) model reconstruction. More specifically, disclosed is a computer-vision network configured to provide viewfinder interfaces and analyses to guide the improved capture of an intended subject for specified purposes. Additionally, the computer-vision network can be configured to generate a metric representing a quality of feature correspondences between images of a complete set of images used for reconstructing a 3D model of a physical structure. The computer-vision network can also be configured to generate feedback at or before image capture time to guide improvements to the quality of feature correspondences between a pair of images.
Disclosed are techniques for generating a photorealistic image by augmenting or compositing at least a portion of a physical structure (e.g., a house) depicted in a two- dimensional (2D) image with synthetic image data. Additionally, disclosed are techniques for augmenting the depicted physical structure using a minimum amount of three-dimensional (3D) geometric data and applying a scene effect to the synthetic image data to create a photorealistic effect.
Methods, storage media, and systems for combining disparate 3d models of a common building object are disclosed. Exemplary implementations may: receive a first plurality of images; generate a first 3d model based on the first plurality of images; receive a second plurality of images; generate a second 3d model based on the second plurality of images; and align, in a common 3d coordinate system, the first 3d model with the second 3d model.
G06T 19/20 - Édition d'images tridimensionnelles [3D], p. ex. modification de formes ou de couleurs, alignement d'objets ou positionnements de parties
G06F 30/13 - Conception architecturale, p. ex. conception architecturale assistée par ordinateur [CAAO] relative à la conception de bâtiments, de ponts, de paysages, d’usines ou de routes
G06T 17/00 - Modélisation tridimensionnelle [3D] pour infographie
An image set is refined by selection criteria among captured images, such that images within the set must satisfy criteria such as feature matching among a plurality of frames or positional changes between frame pairs or sufficient overlap of reprojected points of one image into another image such that the reprojected points or features are observed in the frustum or coordinate space of the another image.
G06T 17/00 - Modélisation tridimensionnelle [3D] pour infographie
G06V 10/26 - Segmentation de formes dans le champ d’imageDécoupage ou fusion d’éléments d’image visant à établir la région de motif, p. ex. techniques de regroupementDétection d’occlusion
G06V 10/44 - Extraction de caractéristiques locales par analyse des parties du motif, p. ex. par détection d’arêtes, de contours, de boucles, d’angles, de barres ou d’intersectionsAnalyse de connectivité, p. ex. de composantes connectées
G06V 30/19 - Reconnaissance utilisant des moyens électroniques
G06V 30/414 - Extraction de la structure géométrique, p. ex. arborescenceDécoupage en blocs, p. ex. boîtes englobantes pour les éléments graphiques ou textuels
H04N 23/63 - Commande des caméras ou des modules de caméras en utilisant des viseurs électroniques
A system and method is provided for constructing a labeled and dimensioned multidimensional (e.g., 3D) building model from building object imagery (e.g., ground-level imagery). The method begins by retrieve building object imagery, the building object imagery collected based on directed capture with a mobile device. The method continues by constructing a scaled multi-dimensional building model, the scale based on sizing of at least one selected architectural feature. The method continues by identifying architectural elements within facades of the multi-dimensional building model. The method continues by determining dimensions of at least one of the architectural elements, the dimensions based on the scale. The method continues by determining dimensions (e.g., area) of at least one of the architectural elements. The method continues by labeling each identified architectural element with at least an identifier and by labeling at least one of the architectural elements with the determined dimensions.
G06T 17/00 - Modélisation tridimensionnelle [3D] pour infographie
G06F 3/14 - Sortie numérique vers un dispositif de visualisation
G06F 30/13 - Conception architecturale, p. ex. conception architecturale assistée par ordinateur [CAAO] relative à la conception de bâtiments, de ponts, de paysages, d’usines ou de routes
G06F 30/23 - Optimisation, vérification ou simulation de l’objet conçu utilisant les méthodes des éléments finis [MEF] ou les méthodes à différences finies [MDF]
Disclosed are techniques for generating a photorealistic image by augmenting or compositing at least a portion of a physical structure (e.g., a house) depicted in a two-dimensional (2D) image with synthetic image data. Additionally, disclosed are techniques for augmenting the depicted physical structure and applying a scene effect to the synthetic image data to create a photorealistic effect.
A method and related software are disclosed for processing imagery related to three dimensional models. To display new visual data for select portions of images, an image of a physical structure such as a building with a façade is retrieved with an associated three dimensional model for that physical structure according to common geolocation tags. A scaffolding of surfaces composing the three dimensional model is generated and regions of the retrieved image are registered to the surfaces of the scaffolding to create mapped surfaces for the image. New image data such as texture information is received and applied to select mapped surfaces to give the retrieved image the appearance of having the new texture data at the selected mapped surface.
System and method are provided for scaling a 3-D representation of a building structure. The method includes obtaining world map data including a first track of real-world poses for a plurality of images. The plurality of images comprises non-camera anchors. The method also includes detecting a discrepancy in at least one real-world pose of the first track. The method also includes in response to detecting a discrepancy, generating a new track of real-world poses. The method also includes calculating a scaling factor for a 3-D representation of the building structure based on sampling across a plurality of tracks. The plurality of tracks comprises at least the first track and the new track.
System and method are provided for detecting and correcting drift in camera poses. A method includes obtaining pose data from an augmented reality (AR) system of a mobile device indicative of a position and orientation of a camera in three-dimensional (3-D) space and associated with the capture image(s) of an environment. Geometric-based data for respective images is determined using linear features and the pose data and a comparison is made between the geometric-based data and a predetermined geometry applicable to the environment. The comparison is employed to detect a drift value, where the drift value is based on an increasing error in the pose data. The images are grouped into first and second subsets based on a drift error value associated with respective images, with different corrections applied to the first and second subsets.
G06T 7/70 - Détermination de la position ou de l'orientation des objets ou des caméras
G06V 10/75 - Organisation de procédés de l’appariement, p. ex. comparaisons simultanées ou séquentielles des caractéristiques d’images ou de vidéosApproches-approximative-fine, p. ex. approches multi-échellesAppariement de motifs d’image ou de vidéoMesures de proximité dans les espaces de caractéristiques utilisant l’analyse de contexteSélection des dictionnaires
G06V 20/17 - Scènes terrestres transmises par des avions ou des drones
A method renders photorealistic images in a web browser. The method is performed at a computing device having a general purpose processor and a graphics processing unit (GPU). The method includes obtaining an environment map and images of an input scene. The method also includes computing textures for the input scene including by encoding an acceleration structure of the input scene. The method further includes transmitting the textures to shaders executing on a GPU. The method includes generating samples of the input scene, by performing at least one path tracing algorithm on the GPU, according to the textures. The method also includes lighting or illuminating a sample of the input scene using the environment map, to obtain a lighted scene, and tone mapping the lighted scene. The method includes drawing output on a canvas, in the web browser, based on the tone-mapped scene to render the input scene.
Systems and methods are disclosed for directed image capture of a subject of interest, such as a physical building. Directed image capture can produce higher quality images such as content more centrally located within an image frame (or an associated viewing device or other display), higher quality images have greater value for subsequent uses of captured images such as for information extraction or model reconstruction. Graphical guide(s) overlaid within an image frame can facilitate quality assessments for the content or the image frame itself, such as for pixel distance of the subject of interest to a centroid of the image frame (or an associated viewing device or other display), or the effect of obscuring objects. Quality assessments can further include instructions for improving the quality of the image capture for the content of interest.
H04N 13/221 - Générateurs de signaux d’images utilisant des caméras à images stéréoscopiques utilisant un seul capteur d’images 2D utilisant le mouvement relatif de caméras et de sujets
H04N 23/63 - Commande des caméras ou des modules de caméras en utilisant des viseurs électroniques
24.
METHODS, STORAGE MEDIA, AND SYSTEMS FOR MEASURING AN ANGLE OF A ROOF FACET
Methods, storage media, and systems for measuring an angle of a roof facet are disclosed. Exemplary implementations may include initiating a flight path for an image capture device, the flight path including, at successively greater heights, a starting position, a calibration position, and an orthographic position above the roof facet. A first fiducial is detected as or at the calibration position and a second fiducial is detected concurrent with movement of the image capture device along the flight path towards the orthographic position. An elevation change of the image capture device is measured between the first fiducial and second fiducial. An orthographic image of the roof facet is captured from the orthographic position. An outline of the roof facet is generated from the orthographic image. A pitch of the roof is calculated from the outline of the roof facet and the elevation change.
G05D 1/646 - Suivi d’une trajectoire prédéfinie, p. ex. d’une ligne marquée sur le sol ou d’une trajectoire de vol
G05D 101/15 - Détails des architectures logicielles ou matérielles utilisées pour la commande de la position utilisant des techniques d’intelligence artificielle [IA] utilisant l’apprentissage automatique, p. ex. les réseaux neuronaux
G05D 105/80 - Applications spécifiques des véhicules commandés pour la collecte d’informations, p. ex. recherche universitaire
G06T 7/80 - Analyse des images capturées pour déterminer les paramètres de caméra intrinsèques ou extrinsèques, c.-à-d. étalonnage de caméra
G06T 7/90 - Détermination de caractéristiques de couleur
G06V 10/56 - Extraction de caractéristiques d’images ou de vidéos relative à la couleur
G06V 10/70 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique
Methods, storage media, and systems for constraining color and lighting conditions of generative machine learning (GML) models are disclosed. An image is provided, a simulated calibration object is inserted into a portion of the scene in the image, and color and lighting conditions of the scene in the image are derived based on the at least one simulated calibration object. The image is input into a GML model, which is constrained based on the derived color and lighting conditions. A visualization including new visual data is generated using the constrained GML model.
Identifying a pre-existing three-dimensional (3D) model of a target structure includes receiving at least one two-dimensional (2D) image of a target physical structure; generating a predicted 3D model of the target structure based on the at least one 2D image; generating a search descriptor of the predicted 3D model; querying a data structure storing a plurality of pre-existing descriptors, where each pre-existing descriptor characterizes a previously constructed 3D model of an associated physical structure; and identifying at least one previously constructed 3D model that is substantially similar to the predicted 3D model based on a difference between the search descriptor and the plurality of pre-existing descriptors.
A method generates a three-dimensional (3D) line segment from a two-dimensional (2D) image. A 3D coordinate space along with a first plane in the coordinate space are received. An image with an associated camera pose in the 3D coordinate space is also received. A 2D line segment is detected in the image and then projected into the 3D coordinate space based on the camera pose. The projected 2D line segment is intersected with the first plane, which generates a 3D line segment defined by the intersection. By leveraging the camera pose and first plane, the 3D line segment can be generated from a single 2D image, eliminating the need to triangulate line segments across stereo image pairs.
A method generates a three-dimensional (3D) line segment from a two-dimensional (2D) image. A 3D coordinate space along with a first plane in the coordinate space are received. An image with an associated camera pose in the 3D coordinate space is also received. A 2D line segment is detected in the image and then projected into the 3D coordinate space based on the camera pose. The projected 2D line segment is intersected with the first plane, which generates a 3D line segment defined by the intersection. By leveraging the camera pose and first plane, the 3D line segment can be generated from a single 2D image, eliminating the need to triangulate line segments across stereo image pairs.
Visualizing three dimensional content is complicated by display platforms capable of more degrees of freedom to display the content than interface tools have to navigate that content. Disclosed are methods and systems for displaying select portions of the content and generating virtual camera positions with associated look angles for the select portions, such as planar geometries of a three dimensional building, thereby constraining the degrees of freedom for improved navigation through views of the content. Look angles can be associated with axes of the content and fields of view.
System and method are provided for generating training data for feature matching among images of a building structure. The method includes obtaining a model of a building that includes a camera solution and images used to generate the geometric model. The method also includes, for facades of the model: applying a minimum bounding box to a respective facade to obtain a respective facade slice that is a 2-D plane represented in a 3-D coordinate system of the model; and projecting visual data of at least one camera in the camera solution that viewed the respective facade onto a visibility mask associated with the respective facade slice. The method also includes photo-texturing the projected visual data facade slice to one of the facade slices or the geometric model to generate a visual 3-D representation of the building; and generating a training dataset by perturbing the visual 3-D representation.
Systems and methods are provided for pitch determination. An example method includes obtaining an image depicting a structure, the image being captured via a user device positioned proximate to the structure. The image is segmented to identify, at least, a roof facet of the structure. An eave vector and a rake vector which are associated with the roof facet are determined. A normal vector of the roof facet is calculated based on the eave vector and the rake vector, and compared to a vector indicating a vertical direction such as gravity. The angle made out by the normal and a gravity vector may be utilized to calculate the pitch of the roof facet.
A computer system generates an outline of a roof of a structure based on a set of lateral images depicting the structure. For each image in the set of lateral images, one or more rooflines corresponding to the roof of the structure are determined. The computer system determines how the rooflines connect to one another. Based on the determination, the rooflines are connected to generate an outline of the roof.
Disclosed are systems and method for determining information related to building materials based on determined measurements from a multi-dimensional building model comprising features and elements embodying such materials and measurements. The multi-dimensional model may be based on a plurality of received images, such as ground-based images of a building. The multi-dimensional model may be scaled, or a scale extracted based on data of the model. The multi-dimensional model may comprise architectural elements, and the scale used to determine measurements of such architectural elements. With the scaled measurements of the architectural elements in the model, product information related to multi-dimensional model and its elements may be derived and combined in alternative means.
System and method are provided for improving camera pose accuracy in augmented reality (AR) systems. The method detects and processes inconsistencies in captured camera poses by identifying locally rigid pose groups and matching visual features between images both within and across these groups. The process involves triangulating 3D landmarks within pose groups, establishing correspondences between groups, and performing bundle adjustment to optimize camera poses. This systematic approach enables the generation of accurate 3D models by detecting and correcting pose drift through feature matching, landmark triangulation, and global pose optimization across multiple camera positions and orientations.
An image capture system provides automated prompts for aiding a user in capturing images for use in 3D model creation. While a user is preparing to capture an image, the system provides visual indications that indicate whether a quality-based condition is satisfied. Based on the visual indications, a user can determine whether an image, if captured, would likely be suitable for use in creating a 3D model. Determining if the quality-based condition is satisfied may include monitoring output generated by one or more sensors and comparing the output against a threshold value. Additionally, the system may analyze the visual content or metadata associated with an image to determine if the quality-based condition is satisfied and request user input to further identify certain image features that were identified by the system.
System and method are provided for improving camera pose accuracy in augmented reality (AR) systems. The method detects and processes inconsistencies in captured camera poses by identifying locally rigid pose groups and matching visual features between images both within and across these groups. The process involves triangulating 3D landmarks within pose groups, establishing correspondences between groups, and performing bundle adjustment to optimize camera poses. This systematic approach enables the generation of accurate 3D models by detecting and correcting pose drift through feature matching, landmark triangulation, and global pose optimization across multiple camera positions and orientations.
G06T 7/73 - Détermination de la position ou de l'orientation des objets ou des caméras utilisant des procédés basés sur les caractéristiques
G06T 7/246 - Analyse du mouvement utilisant des procédés basés sur les caractéristiques, p. ex. le suivi des coins ou des segments
G06T 7/33 - Détermination des paramètres de transformation pour l'alignement des images, c.-à-d. recalage des images utilisant des procédés basés sur les caractéristiques
G06T 17/20 - Description filaire, p. ex. polygonalisation ou tessellation
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
G06T 19/20 - Édition d'images tridimensionnelles [3D], p. ex. modification de formes ou de couleurs, alignement d'objets ou positionnements de parties
38.
AUTOMATED GUIDE FOR IMAGE CAPTURING FOR 3D MODEL CREATION
An image capture system provides automated prompts for aiding a user in capturing images for use in 3D model creation. While a user is preparing to capture an image, the system provides visual indications that indicate whether a quality-based condition is satisfied. Based on the visual indications, a user can determine whether an image, if captured, would likely be suitable for use in creating a 3D model. Determining if the quality-based condition is satisfied may include monitoring output generated by one or more sensors and comparing the output against a threshold value. Additionally, the system may analyze the visual content or metadata associated with an image to determine if the quality-based condition is satisfied and request user input to further identify certain image features that were identified by the system.
Systems and methods are provided for pitch determination. An example method includes obtaining an image depicting a structure, the image being captured via a user device positioned proximate to the structure. The image is segmented to identify, at least, a roof facet of the structure. An eave vector and a rake vector which are associated with the roof facet are determined. A normal vector of the roof facet is calculated based on the eave vector and the rake vector, and compared to a vector indicating a vertical direction such as gravity. The angle made out by the normal and a gravity vector may be utilized to calculate the pitch of the roof facet.
Systems and methods are provided for pitch determination. An example method includes obtaining an image depicting a structure, the image being captured via a user device positioned proximate to the structure. The image is segmented to identify, at least, a roof facet of the structure. An eave vector and a rake vector which are associated with the roof facet are determined. A normal vector of the roof facet is calculated based on the eave vector and the rake vector, and compared to a vector indicating a vertical direction such as gravity. The angle made out by the normal and a gravity vector may be utilized to calculate the pitch of the roof facet.
Disclosed are systems and method for automatic building material ordering based on determined measurements from a multi-dimensional building model. The multi-dimensional model may be based on a plurality of received images, such as ground-based images of a building. The multi-dimensional model may be scaled, or a scale extracted based on data of the model. The multi-dimensional model may comprise architectural elements, and the scale used to determine measurements of such architectural elements. With the scaled measurements of the architectural elements in the model, product information related to multi-dimensional model may be processed, such as confirming availability of the sizes and quantities associated with the architectural elements. Ordering manufacturer products may based on the processed information.
A process for receiving, from a computing device, a series of captured building images. The process continues by processing, in real-time, each building image in the series of captured building images to determine if each building image meets a minimum criterion, wherein the minimum criteria includes applicability to be used in constructing a specific digital multi-dimensional building model. The process continues by aggregating each image meeting the minimum criteria, determining when a base set of building images has been aggregated, wherein the base set of building images includes a threshold number images to model at least a partial multi-dimensional building model representing the series of captured building images, determining one or more facades present in the partial multi-dimensional building model, determining preliminary dimensions for one or more architectural features of the one or more facades and returning, incrementally (in real-time), the preliminary dimensions to the computing device.
Systems and methods are disclosed for adjusting plane positions in multi-dimensional models. Disclosed is moving a plane associated with an architectural element based on a scale and a translation positional error, wherein the scaled is determined based on the architectural element, and the translation position error is based on a position of the architectural element, and reconstructing the multi-dimensional building model based on the moved plane.
G06T 3/40 - Changement d'échelle d’images complètes ou de parties d’image, p. ex. agrandissement ou rétrécissement
G06F 16/28 - Bases de données caractérisées par leurs modèles, p. ex. des modèles relationnels ou objet
G06F 16/583 - Recherche caractérisée par l’utilisation de métadonnées, p. ex. de métadonnées ne provenant pas du contenu ou de métadonnées générées manuellement utilisant des métadonnées provenant automatiquement du contenu
G06F 30/00 - Conception assistée par ordinateur [CAO]
G06F 30/13 - Conception architecturale, p. ex. conception architecturale assistée par ordinateur [CAAO] relative à la conception de bâtiments, de ponts, de paysages, d’usines ou de routes
Systems and methods are disclosed for adjusting plane positions in multi-dimensional models. Disclosed is moving a plane associated with an architectural element based on a scale and a translation positional error, wherein the scaled is determined based on the architectural element, and the translation position error is based on a position of the architectural element, and reconstructing the multi-dimensional building model based on the moved plane.
G06T 3/40 - Changement d'échelle d’images complètes ou de parties d’image, p. ex. agrandissement ou rétrécissement
G06F 16/28 - Bases de données caractérisées par leurs modèles, p. ex. des modèles relationnels ou objet
G06F 16/583 - Recherche caractérisée par l’utilisation de métadonnées, p. ex. de métadonnées ne provenant pas du contenu ou de métadonnées générées manuellement utilisant des métadonnées provenant automatiquement du contenu
G06F 30/00 - Conception assistée par ordinateur [CAO]
G06F 30/13 - Conception architecturale, p. ex. conception architecturale assistée par ordinateur [CAAO] relative à la conception de bâtiments, de ponts, de paysages, d’usines ou de routes
Disclosed are systems and method for automatic building material ordering based on determined measurements from a multi-dimensional building model. The multi-dimensional model may be based on a plurality of received images, such as ground-based images of a building. The multi-dimensional model may be scaled, or a scale extracted based on data of the model. The multi-dimensional model may comprise architectural elements, and the scale used to determine measurements of such architectural elements. With the scaled measurements of the architectural elements in the model, product information related to multi-dimensional model may be processed, such as confirming availability of the sizes and quantities associated with the architectural elements. Ordering manufacturer products may based on the processed information.
A system and method is provided for measurements of building façade elements by combining ground-level and orthogonal imagery. The measurements of the dimension of building façade elements are based on ground-level imagery that is scaled and geo-referenced using orthogonal imagery. The method continues by creating a tabular dataset of measurements for one or more architectural elements such as siding (e.g., aluminum, vinyl, wood, brick and/or paint), windows or doors. The tabular dataset can be part of an estimate report.
Systems and methods are provided for scaling a 3-D representation of a building structure by selectively pairing camera poses generated by augmented reality frameworks. The geometric information provided by augmented reality frameworks enables scale for non-augmented reality cameras, such as SLAM derived camera solutions, associated with the augmented reality cameras. To reduce the noise and error that augmented reality frameworks can impart into their camera solves, only reliable augmented reality cameras are used for scale calculations of associated non augmented reality cameras. Reliable augmented reality cameras are identified based on translation distance analyses and comparisons. The method includes obtaining world map data including a first track of real-world poses for a plurality of images. The plurality of images comprises non camera anchors.
Methods, storage media, and systems for generating a three-dimensional line segment are disclosed. Exemplary implementations may: receive a plurality of images: generate a point cloud based on the plurality of images; detect a two-dimensional line segment in a first image: project a set of 3d points of the plurality of 3d points as 2d points in the first image; select projected 3d points that are proximate to 2d points along the 2d line segment: and generate a 3d line segment by connecting 3d points of the point cloud represented by the selected projected 3d points.
G06V 10/764 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant la classification, p. ex. des objets vidéo
H04N 5/74 - Dispositifs de projection pour reproduction d'image, p. ex. eidophor
49.
THREE-DIMENSIONAL BUILDING MODEL GENERATION BASED ON CLASSIFICATION OF IMAGE ELEMENTS
Methods, storage media, and systems for three-dimensional building model generation based on classification of image elements. An example method includes obtaining images depicting a building, with individual images being taken at individual positions about an exterior of the building, and with the images being associated with camera properties reflecting extrinsic and/or intrinsic camera parameters. Semantic labels are determined for the images via a machine learning model, with the labels being associated with elements of the building, and with the semantic labels being associated with two-dimensional positions in the images. Three-dimensional positions associated with the plurality of elements are estimated, with estimating being based on one or more epipolar constraints. A three-dimensional representation of at least a portion of the building is generated, with the portion including a roof of the building.
G06T 17/00 - Modélisation tridimensionnelle [3D] pour infographie
G06T 7/70 - Détermination de la position ou de l'orientation des objets ou des caméras
G06V 10/764 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant la classification, p. ex. des objets vidéo
G06V 20/70 - Étiquetage du contenu de scène, p. ex. en tirant des représentations syntaxiques ou sémantiques
Systems and methods are disclosed for adjusting plane positions in multi-dimensional models. Disclosed is moving a plane associated with an architectural element based on a scale and a translation positional error, wherein the scaled is determined based on the architectural element, and the translation position error is based on a position of the architectural element, and reconstructing the multi-dimensional building model based on the moved plane.
G06T 3/40 - Changement d'échelle d’images complètes ou de parties d’image, p. ex. agrandissement ou rétrécissement
G06F 16/28 - Bases de données caractérisées par leurs modèles, p. ex. des modèles relationnels ou objet
G06F 16/583 - Recherche caractérisée par l’utilisation de métadonnées, p. ex. de métadonnées ne provenant pas du contenu ou de métadonnées générées manuellement utilisant des métadonnées provenant automatiquement du contenu
G06F 30/00 - Conception assistée par ordinateur [CAO]
G06F 30/13 - Conception architecturale, p. ex. conception architecturale assistée par ordinateur [CAAO] relative à la conception de bâtiments, de ponts, de paysages, d’usines ou de routes
A system and method for real-time updating of three-dimensional (3D) building models includes receiving a request to analyze building imagery to detect potential physical changes in or around a first building, receiving the building imagery, the building imagery including one or more images of the building, optionally building a first 3D building model (textured or untextured) based on the building imagery, retrieving, from computer storage, a previously stored version of the first 3D building model, comparing, on a region-by-region basis, the first 3D building model against the previously stored version of the first 3D building model, cataloging in computer storage, based on the comparing, changes to the previously stored version of the first 3D building model, where the changes to the first 3D building model represent physical changes to or around the building occurring since a time of the previous stored version of the 3D building model.
System and methods for generating similar successive predictions to a three-dimensional scene. An example method includes accessing a three-dimensional model associated with a building, the three-dimensional model being associated with geometry which is included in a scene; generating, via one or more machine learning models, a first viewpoint of the scene from a first viewpoint angle, the first viewpoint including textures applied to the three-dimensional model from the first viewpoint angle and additional image content generated based on at least one parameter; extracting, via one or more machine learning models, updated geometry for inclusion in the scene, wherein the updated geometry includes geometry associated with the additional image content at the first viewpoint angle; and generating, via one or more machine learning models, a second viewpoint of the scene from a second viewpoint angle, where generating the second viewpoint is based on the updated geometry.
G06V 10/44 - Extraction de caractéristiques locales par analyse des parties du motif, p. ex. par détection d’arêtes, de contours, de boucles, d’angles, de barres ou d’intersectionsAnalyse de connectivité, p. ex. de composantes connectées
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
55.
METHODS, STORAGE MEDIA, AND SYSTEMS FOR CLASSIFYING AN IMAGE ACCORDING TO AN INTENSITY OF AN OBJECT
Methods, systems, and storage medium for operating a graphics processing unit to evaluate the extent or degree that a real-world image comprises pixels representing a particular classification category are disclosed. The evaluation may involve evaluating a plurality of pixels in the real-world image according to a first classification category using a first layer of an inference network and at least one other classification category according to at least one other layer of the inference network, creating a first intra-layer value based on combining only the predicted values for one or more pixels of the real-world image as provided by the first layer, and classifying the real-world image according to the first intra-layer value. Post-evaluation operations such as training set generation, three-dimensional (3D) reconstruction, inferring un-evaluated classifications, image depth inference, and selective activation of communications with other processing units are disclosed.
G06V 10/44 - Extraction de caractéristiques locales par analyse des parties du motif, p. ex. par détection d’arêtes, de contours, de boucles, d’angles, de barres ou d’intersectionsAnalyse de connectivité, p. ex. de composantes connectées
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
56.
METHODS, STORAGE MEDIA, AND SYSTEMS FOR SELECTING AN OPTIMAL IMAGE FRAME WITHIN A CAPTURE WINDOW
H04N 23/68 - Commande des caméras ou des modules de caméras pour une prise de vue stable de la scène, p. ex. en compensant les vibrations du boîtier de l'appareil photo
57.
Systems and methods for generating three dimensional geometry
Systems and methods are described for creating three dimensional models of building objects by creating a point cloud from a plurality of input images, defining edges of the building object's surfaces represented by the point cloud, creating simplified geometries of the building object's surfaces and constructing a building model based on the simplified geometries. Input images may include ground, orthographic, or oblique images. The resultant model may be scaled according to correlation with select image types and textured.
Methods, storage media, and systems for selecting an optimal two-dimensional image frame within one or more capture windows for a three-dimensional reconstruction pipeline. The method may include generating one or more capture windows. Each capture window may be proximate to a detected actuation, relative to a first image frame, or a combination thereof. A plurality of candidate image frames and sensor data may be captured within each capture window, assigned to each capture window, or a combination thereof. A frame cost for each candidate image frame may be generated based on sensor data, image data, or a combination thereof. An optimal image frame may be selected based on frame cost, sensor data, image data, or a combination thereof. The optimal image frame may be stored. Image frames other than the optimal image frame may be distinguished.
Methods, storage media, and systems for selecting an optimal two-dimensional image frame within a capture window for a three-dimensional reconstruction pipeline. The method includes generating a capture window proximate to a detected actuation. A plurality of candidate image frames and sensor data are captured within the capture window. An optimal image frame is selected based on sensor data, image data, or a combination thereof. The selected optimal image frame is stored.
H04N 23/68 - Commande des caméras ou des modules de caméras pour une prise de vue stable de la scène, p. ex. en compensant les vibrations du boîtier de l'appareil photo
60.
METHODS, STORAGE MEDIA, AND SYSTEMS FOR SELECTING A PAIR OF CONSISTENT REAL-WORLD CAMERA POSES
Disclosed are methods, storage media, and systems for selecting a pair of consistent real-world camera poses for 3D reconstruction. The disclosed processes involve capturing multiple images of an object from various camera poses and analyzing the images and camera poses to select a consistent pair of camera poses. This selection is based on calculating perturbation errors and reprojection errors generated from 2D points or 2D line segments in the images. A weight is calculated for each camera pose pair based on these errors, and the pair with the largest weight, indicative of the highest consistency and stability, is selected.
Disclosed are methods, storage media, and systems for selecting a pair of consistent real-world camera poses for 3D reconstruction. The disclosed processes involve capturing multiple images of an object from various camera poses and analyzing the images and camera poses to select a consistent pair of camera poses. This selection is based on calculating perturbation errors and reprojection errors generated from 2D points or 2D line segments in the images. A weight is calculated for each camera pose pair based on these errors, and the pair with the largest weight, indicative of the highest consistency and stability, is selected.
H04N 23/60 - Commande des caméras ou des modules de caméras
H04N 23/63 - Commande des caméras ou des modules de caméras en utilisant des viseurs électroniques
H04N 23/68 - Commande des caméras ou des modules de caméras pour une prise de vue stable de la scène, p. ex. en compensant les vibrations du boîtier de l'appareil photo
64.
METHODS, STORAGE MEDIA, AND SYSTEMS FOR DETECTING A PERSISTING OR SUSTAINED BLUR CONDITION
The disclosure relates to methods, storage media, and systems of detecting a persisting or sustained blur condition across 2D image frames captured for a 3D reconstruction pipeline. It may include receiving a plurality of image frames captured by a camera of a capture device, receiving sensor data of the capture device, detecting, based on the sensor data of the capture device, a blur condition for at least a threshold number of image frames within a capture window, responsive to detecting the blur condition, providing an augmentation for at least one image frame of the plurality of image frames based on the sensor data, applying the augmentation to the at least one image frame of the plurality of image frames, and displaying, on a display of the capture device, the at least one augmented image frame.
H04N 23/63 - Commande des caméras ou des modules de caméras en utilisant des viseurs électroniques
H04N 23/60 - Commande des caméras ou des modules de caméras
H04N 23/68 - Commande des caméras ou des modules de caméras pour une prise de vue stable de la scène, p. ex. en compensant les vibrations du boîtier de l'appareil photo
65.
Methods, storage media, and systems for detecting a persisting or sustained blur condition
The disclosure relates to methods, storage media, and systems of detecting a persisting or sustained blur condition across 2D image frames captured for a 3D reconstruction pipeline. It may include receiving a plurality of image frames captured by a camera of a capture device, receiving sensor data of the capture device, detecting, based on the sensor data of the capture device, a blur condition for at least a threshold number of image frames within a capture window, responsive to detecting the blur condition, providing an augmentation for at least one image frame of the plurality of image frames based on the sensor data, applying the augmentation to the at least one image frame of the plurality of image frames, and displaying, on a display of the capture device, the at least one augmented image frame.
H04N 23/68 - Commande des caméras ou des modules de caméras pour une prise de vue stable de la scène, p. ex. en compensant les vibrations du boîtier de l'appareil photo
H04N 23/63 - Commande des caméras ou des modules de caméras en utilisant des viseurs électroniques
66.
METHODS, STORAGE MEDIA, AND SYSTEMS FOR INTEGRATING DATA STREAMS TO GENERATE VISUAL CUES
Methods, storage media, and systems for integrating disparate data streams of a current scan to generate visual cues of the current scan are disclosed. Exemplary implementations may: receive, from a data capture device, captured visual data and captured depth data of a current scan of an environment; generate a first plurality of masks based on the captured depth data; generate a depth propagation based on the first plurality of masks; generate augmented visual data based on the captured visual data, the first plurality of masks, and the depth propagation; and display, on a display of the data capture device, the augmented visual data.
Methods, storage media, and systems for augmenting two-dimensional (2D) data, three-dimensional (3D) data, 2D models, or 3D models are disclosed. Exemplary implementations may: receive a first plurality of images: generate a first 3D model based on the first plurality of images; receive a second plurality of images; generate a second 3D model based on the second plurality of images; and augment the first 3D model with the second 3D model.
G06T 19/20 - Édition d'images tridimensionnelles [3D], p. ex. modification de formes ou de couleurs, alignement d'objets ou positionnements de parties
G06T 7/32 - Détermination des paramètres de transformation pour l'alignement des images, c.-à-d. recalage des images utilisant des procédés basés sur la corrélation
G06T 7/33 - Détermination des paramètres de transformation pour l'alignement des images, c.-à-d. recalage des images utilisant des procédés basés sur les caractéristiques
G06T 17/00 - Modélisation tridimensionnelle [3D] pour infographie
Systems and methods are provided for pitch determination. An example method includes obtaining an image depicting a structure, the image being captured via a user device positioned proximate to the structure. The image is segmented to identify, at least, a roof facet of the structure. An eave vector and a rake vector which are associated with the roof facet are determined. A normal vector of the roof facet is calculated based on the eave vector and the rake vector, and compared to a vector indicating a vertical direction such as gravity. The angle made out by the normal and a gravity vector may be utilized to calculate the pitch of the roof facet.
A computer system maintains structure data indicating geometrical constraints for each structure category of a plurality of structure categories. The computer system generates a virtual 3D representation of a structure based on a set of images depicting the structure. For each image in the set of images, one or more landmarks are identified. Based on the landmarks, a candidate structure category is selected. The virtual 3D representation is generated based on the geometrical constraints of the candidate structure category and the landmarks identified in the set of images.
G06F 30/13 - Conception architecturale, p. ex. conception architecturale assistée par ordinateur [CAAO] relative à la conception de bâtiments, de ponts, de paysages, d’usines ou de routes
A method and related software are disclosed for processing imagery related to three dimensional models. To display new visual data for select portions of images, an image of a physical structure such as a building with a façade is retrieved with an associated three dimensional model for that physical structure according to common geolocation tags. A scaffolding of surfaces composing the three dimensional model is generated and regions of the retrieved image are registered to the surfaces of the scaffolding to create mapped surfaces for the image. New image data such as texture information is received and applied to select mapped surfaces to give the retrieved image the appearance of having the new texture data at the selected mapped surface.
Systems and methods for generating or rendering a three-dimensional (3D) representation of a structure based on images of the structure are disclosed. A selectively rendered point cloud is generated based on the images of the structure and real cameras associated with a virtual camera observing the selectively rendered point cloud. Images attributes may be applied to the selectively rendered point cloud.
A method renders photorealistic images in a web browser. The method is performed at a computing device having a general purpose processor and a graphics processing unit (GPU). The method includes obtaining an environment map and images of an input scene. The method also includes computing textures for the input scene including by encoding an acceleration structure of the input scene. The method further includes transmitting the textures to shaders executing on a GPU. The method includes generating samples of the input scene, by performing at least one path tracing algorithm on the GPU, according to the textures. The method also includes lighting or illuminating a sample of the input scene using the environment map, to obtain a lighted scene, and tone mapping the lighted scene. The method includes drawing output on a canvas, in the web browser, based on the tone-mapped scene to render the input scene.
G06F 16/957 - Optimisation de la navigation, p. ex. mise en cache ou distillation de contenus
G06F 15/00 - Calculateurs numériques en généralÉquipement de traitement de données en général
G06F 15/04 - Calculateurs numériques en généralÉquipement de traitement de données en général recevant les programmes en même temps que les données à traiter, p. ex. sur le même support d'enregistrement
G06F 15/08 - Calculateurs numériques en généralÉquipement de traitement de données en général utilisant un tableau de connexion pour la programmation
A system and method is provided for constructing a labeled and dimensioned multidimensional (e.g., 3D) building model from building object imagery (e.g., ground-level imagery). The method begins by retrieve building object imagery, the building object imagery collected based on directed capture with a mobile device. The method continues by constructing a scaled multi-dimensional building model, the scale based on sizing of at least one selected architectural feature. The method continues by identifying architectural elements within facades of the multi-dimensional building model. The method continues by determining dimensions of at least one of the architectural elements, the dimensions based on the scale. The method continues by determining dimensions (e.g., area) of at least one of the architectural elements. The method continues by labeling each identified architectural element with at least an identifier and by labeling at least one of the architectural elements with the determined dimensions.
G06T 17/00 - Modélisation tridimensionnelle [3D] pour infographie
G06F 3/14 - Sortie numérique vers un dispositif de visualisation
G06F 30/13 - Conception architecturale, p. ex. conception architecturale assistée par ordinateur [CAAO] relative à la conception de bâtiments, de ponts, de paysages, d’usines ou de routes
G06F 30/23 - Optimisation, vérification ou simulation de l’objet conçu utilisant les méthodes des éléments finis [MEF] ou les méthodes à différences finies [MDF]
A system and method is provided for measurements of building façade elements by combining ground-level and orthogonal imagery. The measurements of the dimension of building façade elements are based on ground-level imagery that is scaled and geo-referenced using orthogonal imagery. The method continues by creating a tabular dataset of measurements for one or more architectural elements such as siding (e.g., aluminum, vinyl, wood, brick and/or paint), windows or doors. The tabular dataset can be part of an estimate report.
A computer system generates an outline of a roof of a structure based on a set of lateral images depicting the structure. For each image in the set of lateral images, one or more rooflines corresponding to the roof of the structure are determined. The computer system determines how the rooflines connect to one another. Based on the determination, the rooflines are connected to generate an outline of the roof.
G06K 9/00 - Méthodes ou dispositions pour la lecture ou la reconnaissance de caractères imprimés ou écrits ou pour la reconnaissance de formes, p.ex. d'empreintes digitales
Systems and methods are provided for pitch determination. An example method includes obtaining an image depicting a structure, the image being captured via a user device positioned proximate to the structure. The image is segmented to identify, at least, a roof facet of the structure. An eave vector and a rake vector which are associated with the roof facet are determined. A normal vector of the roof facet is calculated based on the eave vector and the rake vector, and compared to a vector indicating a vertical direction such as gravity. The angle made out by the normal and a gravity vector may be utilized to calculate the pitch of the roof facet.
G06F 30/13 - Conception architecturale, p. ex. conception architecturale assistée par ordinateur [CAAO] relative à la conception de bâtiments, de ponts, de paysages, d’usines ou de routes
The present disclosure describes systems, methods, and techniques for predicting plausible, semantic, and structure three-dimensional (3D) representations of a building from one or more images of the building. One or more two-dimensional (2D) images of the same building from different camera perspectives are input and used to predict corresponding 2D semantic representations of the building. Latent codes are iteratively sampled from a learned latent space representing a distribution of building structures and used to infer successive semantic representations based on losses between the predicted representations and the inferred representations until a convergence between the predicted representations and the inferred representations is detected. A resulting latent code is then decoded into a semantic geometry for the building.
G06T 19/20 - Édition d'images tridimensionnelles [3D], p. ex. modification de formes ou de couleurs, alignement d'objets ou positionnements de parties
G06F 30/13 - Conception architecturale, p. ex. conception architecturale assistée par ordinateur [CAAO] relative à la conception de bâtiments, de ponts, de paysages, d’usines ou de routes
An image capture system provides automated prompts for aiding a user in capturing images for use in 3D model creation. While a user is preparing to capture an image, the system provides visual indications that indicate whether a quality-based condition is satisfied. Based on the visual indications, a user can determine whether an image, if captured, would likely be suitable for use in creating a 3D model. Determining if the quality-based condition is satisfied may include monitoring output generated by one or more sensors and comparing the output against a threshold value. Additionally, the system may analyze the visual content or metadata associated with an image to determine if the quality-based condition is satisfied and request user input to further identify certain image features that were identified by the system.
H04N 5/335 - Transformation d'informations lumineuses ou analogues en informations électriques utilisant des capteurs d'images à l'état solide [capteurs SSIS]
G06T 17/00 - Modélisation tridimensionnelle [3D] pour infographie
H04N 23/60 - Commande des caméras ou des modules de caméras
H04N 23/63 - Commande des caméras ou des modules de caméras en utilisant des viseurs électroniques
H04N 23/661 - Transmission des signaux de commande de la caméra par le biais de réseaux, p. ex. la commande via Internet
80.
3-D reconstruction using augmented reality frameworks
System and method are provided for scaling a 3-D representation of a building structure. The method includes obtaining images of the building structure, including non-camera anchors. The method also includes identifying reference poses for images based on the non-camera anchors. The method also includes obtaining world map data including real-world poses for the images. The method also includes selecting candidate poses from the real-world poses based on corresponding reference poses. The method also includes calculating a scaling factor for a 3-D representation of the building structure based on correlating the reference poses with the selected candidate poses. Some implementations use structure from motion techniques or LiDAR, in addition to augmented reality frameworks, for scaling the 3-D representations of the building structure. In some implementations, the world map data includes environmental data, such as illumination data, and the method includes generating or displaying the 3-D representation.
Methods, storage media, and systems for combining disparate 3d models of a common building object are disclosed. Exemplary implementations may: receive a first plurality of images; generate a first 3d model based on the first plurality of images; receive a second plurality of images; generate a second 3d model based on the second plurality of images; and align, in a common 3d coordinate system, the first 3d model with the second 3d model.
G06F 30/13 - Conception architecturale, p. ex. conception architecturale assistée par ordinateur [CAAO] relative à la conception de bâtiments, de ponts, de paysages, d’usines ou de routes
G06F 30/27 - Optimisation, vérification ou simulation de l’objet conçu utilisant l’apprentissage automatique, p. ex. l’intelligence artificielle, les réseaux neuronaux, les machines à support de vecteur [MSV] ou l’apprentissage d’un modèle
G06T 7/70 - Détermination de la position ou de l'orientation des objets ou des caméras
82.
METHODS, STORAGE MEDIA, AND SYSTEMS FOR COMBINING DISPARATE 3D MODELS OF A COMMON BUILDING OBJECT
Methods, storage media, and systems for combining disparate 3d models of a common building object are disclosed. Exemplary implementations may: receive a first plurality of images; generate a first 3d model based on the first plurality of images; receive a second plurality of images; generate a second 3d model based on the second plurality of images; and align, in a common 3d coordinate system, the first 3d model with the second 3d model.
G06F 30/13 - Conception architecturale, p. ex. conception architecturale assistée par ordinateur [CAAO] relative à la conception de bâtiments, de ponts, de paysages, d’usines ou de routes
G06F 30/27 - Optimisation, vérification ou simulation de l’objet conçu utilisant l’apprentissage automatique, p. ex. l’intelligence artificielle, les réseaux neuronaux, les machines à support de vecteur [MSV] ou l’apprentissage d’un modèle
G06T 7/70 - Détermination de la position ou de l'orientation des objets ou des caméras
83.
Method for generating roof outlines from lateral images
A computer system generates an outline of a roof of a structure based on a set of lateral images depicting the structure. For each image in the set of lateral images, one or more rooflines corresponding to the roof of the structure are determined. The computer system determines how the rooflines connect to one another. Based on the determination, the rooflines are connected to generate an outline of the roof.
G06K 9/00 - Méthodes ou dispositions pour la lecture ou la reconnaissance de caractères imprimés ou écrits ou pour la reconnaissance de formes, p.ex. d'empreintes digitales
Exemplary implementations may: receive a 3d model; identify at least first, second, and third images that observe a first 3d line segment of the 3d model; identify a 2d line segment in each of the first, second, and third images that corresponds to the first 3d line segment; triangulate the 2d line segment of the first and second images to create a second 3d line segment; triangulate the 2d line segment of the first and third images to create a third 3d line segment; triangulate the 2d line segment of the second and third images to create a fourth 3d line segment; group pose pairs, into groups, based on a parameter of the second 3d line segment, the third 3d line segment, and the fourth 3d line segment; select poses of pose pairs in a selected group of the groups comprising a largest number of pose pairs.
G06T 7/33 - Détermination des paramètres de transformation pour l'alignement des images, c.-à-d. recalage des images utilisant des procédés basés sur les caractéristiques
G06T 7/73 - Détermination de la position ou de l'orientation des objets ou des caméras utilisant des procédés basés sur les caractéristiques
G06V 10/75 - Organisation de procédés de l’appariement, p. ex. comparaisons simultanées ou séquentielles des caractéristiques d’images ou de vidéosApproches-approximative-fine, p. ex. approches multi-échellesAppariement de motifs d’image ou de vidéoMesures de proximité dans les espaces de caractéristiques utilisant l’analyse de contexteSélection des dictionnaires
Exemplary implementations may: receive a 3d model; identify at least first, second, and third images that observe a first 3d line segment of the 3d model; identify a 2d line segment in each of the first, second, and third images that corresponds to the first 3d line segment; triangulate the 2d line segment of the first and second images to create a second 3d line segment; triangulate the 2d line segment of the first and third images to create a third 3d line segment; triangulate the 2d line segment of the second and third images to create a fourth 3d line segment; group pose pairs, into groups, based on a parameter of the second 3d line segment, the third 3d line segment, and the fourth 3d line segment; select poses of pose pairs in a selected group of the groups comprising a largest number of pose pairs.
G06V 10/764 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant la classification, p. ex. des objets vidéo
86.
METHODS, STORAGE MEDIA, AND SYSTEMS FOR EVALUATING CAMERA POSES
Exemplary implementations may: receive a 3d model; identify at least first, second, and third images that observe a first 3d line segment of the 3d model; identify a 2d line segment in each of the first, second, and third images that corresponds to the first 3d line segment; triangulate the 2d line segment of the first and second images to create a second 3d line segment; triangulate the 2d line segment of the first and third images to create a third 3d line segment; triangulate the 2d line segment of the second and third images to create a fourth 3d line segment; group pose pairs, into groups, based on a parameter of the second 3d line segment, the third 3d line segment, and the fourth 3d line segment; select poses of pose pairs in a selected group of the groups comprising a largest number of pose pairs.
G06T 7/33 - Détermination des paramètres de transformation pour l'alignement des images, c.-à-d. recalage des images utilisant des procédés basés sur les caractéristiques
G06T 7/73 - Détermination de la position ou de l'orientation des objets ou des caméras utilisant des procédés basés sur les caractéristiques
G06V 10/75 - Organisation de procédés de l’appariement, p. ex. comparaisons simultanées ou séquentielles des caractéristiques d’images ou de vidéosApproches-approximative-fine, p. ex. approches multi-échellesAppariement de motifs d’image ou de vidéoMesures de proximité dans les espaces de caractéristiques utilisant l’analyse de contexteSélection des dictionnaires
Systems and methods are described for creating three dimensional models of building objects by creating a point cloud from a plurality of input images, defining edges of the building object's surfaces represented by the point cloud, creating simplified geometries of the building object's surfaces and constructing a building model based on the simplified geometries. Input images may include ground, orthographic, or oblique images. The resultant model may be scaled according to correlation with select image types and textured.
Disclosed are techniques for generating a photorealistic image by augmenting or compositing at least a portion of a physical structure (e.g., a house) depicted in a two-dimensional (2D) image with synthetic image data. Additionally, disclosed are techniques for augmenting the depicted physical structure and applying a scene effect to the synthetic image data to create a photorealistic effect.
Systems and methods are described for creating three dimensional models of building objects by creating a point cloud from a plurality of input images, defining edges of the building object's surfaces represented by the point cloud, creating simplified geometries of the building object's surfaces and constructing a building model based on the simplified geometries. Input images may include ground, orthographic, or oblique images. The resultant model may be scaled according to correlation with select image types and textured.
Visualizing three dimensional content is complicated by display platforms capable of more degrees of freedom to display the content than interface tools have to navigate that content. Disclosed are methods and systems for displaying select portions of the content and generating virtual camera positions with associated look angles for the select portions, such as planar geometries of a three dimensional building, thereby constraining the degrees of freedom for improved navigation through views of the content. Look angles can be associated with axes of the content and fields of view.
Methods, storage media, and systems for measuring an angle of a roof facet are disclosed. Exemplary implementations may include initiating a flight path for an image capture device, the flight path including, at successively greater heights, a starting position, a calibration position, and an orthographic position above the roof facet. A first fiducial is detected as or at the calibration position and a second fiducial is detected concurrent with movement of the image capture device along the flight path towards the orthographic position. An elevation change of the image capture device is measured between the first fiducial and second fiducial. An orthographic image of the roof facet is captured from the orthographic position. An outline of the roof facet is generated from the orthographic image. A pitch of the roof is calculated from the outline of the roof facet and the elevation change.
Methods, storage media, and systems for measuring an angle of a roof facet are disclosed. Exemplary implementations may include initiating a flight path for an image capture device, the flight path including, at successively greater heights, a starting position, a calibration position, and an orthographic position above the roof facet. A first fiducial is detected as or at the calibration position and a second fiducial is detected concurrent with movement of the image capture device along the flight path towards the orthographic position. An elevation change of the image capture device is measured between the first fiducial and second fiducial. An orthographic image of the roof facet is captured from the orthographic position. An outline of the roof facet is generated from the orthographic image. A pitch of the roof is calculated from the outline of the roof facet and the elevation change.
G06T 7/30 - Détermination des paramètres de transformation pour l'alignement des images, c.-à-d. recalage des images
B64C 39/02 - Aéronefs non prévus ailleurs caractérisés par un emploi spécial
G05D 1/00 - Commande de la position, du cap, de l'altitude ou de l'attitude des véhicules terrestres, aquatiques, aériens ou spatiaux, p. ex. utilisant des pilotes automatiques
G06V 10/74 - Appariement de motifs d’image ou de vidéoMesures de proximité dans les espaces de caractéristiques
93.
METHODS, STORAGE MEDIA, AND SYSTEMS FOR GENERATING A THREE- DIMENSIONAL COORDINATE SYSTEM
Methods, storage media, and systems for generating a three-dimensional coordinate system, evaluating quality of camera poses associated with images, scaling a three-dimensional model, and calculating an alignment transformation are disclosed.
G06T 7/246 - Analyse du mouvement utilisant des procédés basés sur les caractéristiques, p. ex. le suivi des coins ou des segments
G06T 7/50 - Récupération de la profondeur ou de la forme
G06V 10/44 - Extraction de caractéristiques locales par analyse des parties du motif, p. ex. par détection d’arêtes, de contours, de boucles, d’angles, de barres ou d’intersectionsAnalyse de connectivité, p. ex. de composantes connectées
Systems and methods are disclosed for directed image capture of a subject of interest, such as a physical building. Directed image capture can produce higher quality images such as content more centrally located within an image frame (or an associated viewing device or other display), higher quality images have greater value for subsequent uses of captured images such as for information extraction or model reconstruction. Graphical guide(s) overlaid within an image frame can facilitate quality assessments for the content or the image frame itself, such as for pixel distance of the subject of interest to a centroid of the image frame (or an associated viewing device or other display), or the effect of obscuring objects. Quality assessments can further include instructions for improving the quality of the image capture for the content of interest.
H04N 13/221 - Générateurs de signaux d’images utilisant des caméras à images stéréoscopiques utilisant un seul capteur d’images 2D utilisant le mouvement relatif de caméras et de sujets
H04N 23/63 - Commande des caméras ou des modules de caméras en utilisant des viseurs électroniques
95.
METHODS, STORAGE MEDIA, AND SYSTEMS FOR GENERATING A THREE-DIMENSIONAL COORDINATE SYSTEM
Methods, storage media, and systems for generating a three-dimensional coordinate system, evaluating quality of camera poses associated with images, scaling a three-dimensional model, and calculating an alignment transformation are disclosed.
Methods, storage media, and systems for generating a three-dimensional coordinate system, evaluating quality of camera poses associated with images, scaling a three-dimensional model, and calculating an alignment transformation are disclosed.
G06V 10/44 - Extraction de caractéristiques locales par analyse des parties du motif, p. ex. par détection d’arêtes, de contours, de boucles, d’angles, de barres ou d’intersectionsAnalyse de connectivité, p. ex. de composantes connectées
Systems and methods are disclosed for adjusting plane positions in multi-dimensional models. Disclosed is moving a plane associated with an architectural element based on a scale and a translation positional error, wherein the scaled is determined based on the architectural element, and the translation position error is based on a position of the architectural element, and reconstructing the multi-dimensional building model based on the moved plane.
G06T 3/40 - Changement d'échelle d’images complètes ou de parties d’image, p. ex. agrandissement ou rétrécissement
G06F 16/28 - Bases de données caractérisées par leurs modèles, p. ex. des modèles relationnels ou objet
G06F 16/583 - Recherche caractérisée par l’utilisation de métadonnées, p. ex. de métadonnées ne provenant pas du contenu ou de métadonnées générées manuellement utilisant des métadonnées provenant automatiquement du contenu
G06F 30/00 - Conception assistée par ordinateur [CAO]
G06F 30/13 - Conception architecturale, p. ex. conception architecturale assistée par ordinateur [CAAO] relative à la conception de bâtiments, de ponts, de paysages, d’usines ou de routes
System and method are provided for generating training data for feature matching among images of a building structure. The method includes obtaining a model of a building that includes a camera solution and images used to generate the geometric model. The method also includes, for facades of the model: applying a minimum bounding box to a respective facade to obtain a respective facade slice that is a 2-D plane represented in a 3-D coordinate system of the model; and projecting visual data of at least one camera in the camera solution that viewed the respective facade onto a visibility mask associated with the respective facade slice. The method also includes photo-texturing the projected visual data facade slice to one of the facade slices or the geometric model to generate a visual 3-D representation of the building; and generating a training dataset by perturbing the visual 3-D representation.
System and method are provided for generating training data for feature matching among images of a building structure. The method includes obtaining a model of a building that includes a camera solution and images used to generate the geometric model. The method also includes, for facades of the model: applying a minimum bounding box to a respective facade to obtain a respective facade slice that is a 2-D plane represented in a 3-D coordinate system of the model; and projecting visual data of at least one camera in the camera solution that viewed the respective facade onto a visibility mask associated with the respective facade slice. The method also includes photo-texturing the projected visual data facade slice to one of the facade slices or the geometric model to generate a visual 3-D representation of the building; and generating a training dataset by perturbing the visual 3-D representation.
A system and method is provided for constructing a labeled and dimensioned multidimensional (e.g., 3D) building model from building object imagery (e.g., ground-level imagery). The method begins by retrieve building object imagery, the building object imagery collected based on directed capture with a mobile device. The method continues by constructing a scaled multi-dimensional building model, the scale based on sizing of at least one selected architectural feature. The method continues by identifying architectural elements within facades of the multi-dimensional building model. The method continues by determining dimensions of at least one of the architectural elements, the dimensions based on the scale. The method continues by determining dimensions (e.g., area) of at least one of the architectural elements. The method continues by labeling each identified architectural element with at least an identifier and by labeling at least one of the architectural elements with the determined dimensions.
G06T 17/00 - Modélisation tridimensionnelle [3D] pour infographie
G06F 3/14 - Sortie numérique vers un dispositif de visualisation
G06F 30/13 - Conception architecturale, p. ex. conception architecturale assistée par ordinateur [CAAO] relative à la conception de bâtiments, de ponts, de paysages, d’usines ou de routes
G06F 30/23 - Optimisation, vérification ou simulation de l’objet conçu utilisant les méthodes des éléments finis [MEF] ou les méthodes à différences finies [MDF]
