A splat generation system directly generates splats to accurately represent a three-dimensional (3D) object or scene from existing 3D image data or 3D primitives of a different non-splat 3D format without multiple iterative passes through a neural network or machine learning process. The system receives the 3D primitives of the non-splat 3D format, selects a set of 3D primitives, generates one or more splats that form a first surface that matches a second surface formed by the set of 3D primitives with an acceptable amount of structural loss, and defines visual characteristics for the one or more splats to match visual characteristics across the surface formed by the set of 3D primitives with an acceptable amount of visual characteristic loss. The system present the one or more splats instead of the set of 3D primitives in response to a request to access the 3D object or scene.
A scene generation system provide graphics enhanced content placement in the quality or fidelity of promoted visual assets in a scene are adjusted relative to other or non-promoted visual assets in the same scene. The different quality or fidelity at which the promoted visual assets are presented draw viewer focus to the promoted visual assets without prominently positioning the promoted visual assets in the scene or otherwise altering the creative intent for the scene. The system receives a request to promote a particular visual asset in a scene, determines a first level-of-detail (LoD) at which to render the visual assets of the scene, selects a second LoD that is greater than the first LoD for the particular visual asset, and presents the scene by rendering the non-promoted visual assets at the first LoD and the particular visual asset at the second LoD.
A splat generation system and associated methods implement a modified splatting pipeline that quantifies loss at each splat generation iteration without rendering the splats generated at each iteration or comparing the resulting visualization against reference images of a 3D asset. The system receives the 3D asset and generates a splat representation of the 3D asset. The system compares geometric similarity between different sets of generated splats in different regions of the splat representation and different sets of 3D primitives in corresponding regions of the 3D asset. The system adjusts a set of splats in response to the geometric similarity between the set of splats and a set of 3D primitives in corresponding regions not satisfying a threshold, and reconstructs the 3D asset with an acceptable amount of loss based on the adjustments made to the splats on a region-by-region basis.
An adaptive streaming system dynamically streams different parts of a three-dimensional (3D) model at different resolutions to maximize visual detail and quality in response to changing network performance and/or client device rendering performance. The system receives a request to view the 3D model from a particular field-of-view. The system associates different priorities to different parts of the 3D model, selects first Gaussian splats associated with nodes at a first level in a tree based on the first Gaussian splats representing the different parts of the 3D model in the particular field-of-view with a first priority, and selects second Gaussian splats associated with nodes at a second level in the tree structure based on the second Gaussians splats representing the different parts of the 3D model in the particular field-of-view with a second priority. The system streams the selected Gaussian splats in response to the request.
A streaming system performs a dynamic densification of streamed content based on tracked user focus. The streaming system streams different content having a common classification to one or more users, and tracks a focus of the one or more users on different parts of the different content. The streaming system receives a request for new content, classifies the new content with the common classification, and streams a first parts of new content with greater detail than second parts of the new content in response to the request based on corresponding first parts from the different parts of the different content receiving more of the focus than corresponding second parts from the different parts of the different content.
H04N 19/85 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
A splat generation system and associated methods perform splatting based on a controlled sampling of a three-dimensional (3D) asset. The controlled sampling ensures minimal coverage the 3D asset primitives and provides enhanced coverage for primitives that have greater detail or variation. The system projects rays in different directions from a set of the primitives towards a bounding volume, and detects points at which some of the projected rays intersect the bounding volume. The system defines virtual cameras in the 3D space of the 3D asset based on a set of the intersection points, and obtains the minimal and/or enhanced coverage based on one or more views of the primitives captured in images taken by the virtual cameras. The system generates a set of splats from the one or more views in the captured images with an acceptable amount of loss and with less total data than the 3D asset.
A real-time streaming system and associated methods are provided for initially presenting three-dimensional (3D) content at a low-fidelity first encoding format so that the initial visualization of the 3D content on the requesting device is presented with no or insignificant delay and for streaming prioritized 3D assets within the initial visualization at higher fidelities in different encoding formats that preserve the real-time responsiveness of the system. The system generates and streams a two-dimensional (2D) image for a first presentation of the 3D content at a first fidelity. The system increases a fidelity of a prioritized 3D asset while the 3D content does not change by streaming 3D primitives for the prioritized 3D asset in a second encoding format that increases the fidelity of the prioritized 3D asset in a second presentation of the 3D content from the first fidelity to a greater second fidelity.
A splat generation system and associated methods generate a splat representation of a three-dimensional (3D) model with varying detail based on an unequal or non-uniform capture of the 3D model. The system receives the 3D model. The system analyzes variation in the position or visual characteristics of the 3D model primitives, attributes different levels-of-importance to different surfaces or regions of the 3D model based on the variation, and defines a non-uniform placement of virtual cameras around the 3D model according to the different levels-of-importance that are attributed to the different surfaces or regions of the 3D model. The system generates the splats for the splat representation that reconstruct the different surfaces or regions with differing amounts of detail that match the detail captured for the different surfaces or regions in images generated from the non-uniform placement of the virtual cameras.
Disclosed is a system that streams true three-dimensional ("3D") image data over a data network in a manner that preserves the dimensionality and detail of a dynamic and changing 3D scene. The system generates the 3D image data to represent the 3D scene, and streams different set of the 3D image data that are within different viewing frustums requested by different devices. The system generates updates to the 3D image data based on changes occurring at different parts of the 3D scene. The system streams a first update to the first device in response to image data updated by the first update being within the first device's viewing frustum, and streams a second update to the second device in response to image data updated by the second update being within the second device's viewing frustum.
H04N 13/275 - Image signal generators from 3D object models, e.g. computer-generated stereoscopic image signals
H04N 13/361 - Reproducing mixed stereoscopic imagesReproducing mixed monoscopic and stereoscopic images, e.g. a stereoscopic image overlay window on a monoscopic image background
Disclosed is a system and associated methods for compressing motion within an animated point cloud. The resulting compressed file encodes different transforms that recreate the motion of different sets of points across different point clouds or frames of the animation in place of the data for the different sets of points from the different point clouds. The compression involves detecting a motion that changes positioning of a set of points between a first point cloud and subsequent point clouds of an uncompressed encoding of two or more frames of an animation. The compression further involves defining a transform that models the motion, and generating a compressed animated point cloud by encoding the data of the first point cloud in the compressed animated point cloud, and by replacing the data for the set of points in the one or more subsequent point clouds with the transform.
Disclosed is a system and associated methods that account for the change in coloring or tint that some wavelengths of light have on materials of an object, and that generate an object model with the accounted for change in coloring or tint removed from the pixels or constructs of that model. The system receives spectral data in different electromagnetic spectrum bands for a particular surface of the object. The system measures a first quality of the light that illuminates the object, and determines a reactivity of the particular surface to the first quality of the light based on the spectral data matching a spectral signature of a material having that reactivity. The system removes the light effects on the particular surface by adjusting the spectral data according to the reactivity to the first quality of the light and measuring the first quality in the light illuminating the object.
Disclosed is a system and associated methods for controlling computer operation through three-dimensional ("3D") menus, 3D toolbars, and other 3D elements that provide an efficient and dynamic organization of application functionality and icons across three dimensions. The system generates a first 3D element with a first number of regions with each region of the first number of regions providing access to different functionality from a first set of functionality, and a second 3D element with a second number of regions with each region of the second number of regions providing access to different functionality from a second set of functionality. The system connects the first 3D element to the second 3D element, and rotates one or more of the first element and the second 3D element to change the functionality that is accessible from the one or more of the first element and the second element.
G06F 9/451 - Execution arrangements for user interfaces
G06F 3/0487 - Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
Disclosed is a security controller for a content distribution system that generates and encodes point clouds with invisible, visually undetectable, and/or tamperproof watermarks. The controller selects a set of data points from the point cloud, and changes at least one specific bit of a set of bits that define a value for a particular non-positional element of the set of data points. The controller encodes point cloud with a first watermark that is unique to a requesting client based on the changing of the at least one specific bit. The controller distributes the file encoded with the first watermark to the requesting client and tracks the set of data points that were encoded with the first watermark in order to subsequently verify that the requesting client was the original recipient of the point cloud encoded with the first watermark.
A system and associated methods for generating a consistently sharp, detailed, and in-focus three-dimensional ("3D") model of an object from two-dimensional ("2D") images that collectively capture all sides of the object with multiple depths-of-field. The system receives a set of 2D images that capture a particular part of the object with different depths-of-field. The system determines a first pixel from a first 2D image and a second pixel from a second 2D image that represent a common point of the object, determines that the first pixel is out of focus based on the first 2D image depth-of-field and that the second pixel is in focus based on the second 2D image depth-of-field, and defines a 3D construct, that represents the common point in a 3D model of the object, using data of the in-focus second pixel instead of data of the out-of-focus first pixel.
Disclosed is a system and associated methods for improving interactions with three-dimensional ("3D") objects in a 3D space by dynamically defining the positioning of the handles used to control the interactions with the camera and/or the 3D objects. The system analyzes the positioning of different constructs that form the 3D objects. From the analysis, the system defines handles at different dynamically determined positions about the 3D objects. The system applies an edit from a first dynamically determined position about a particular 3D object in response to a user interaction with a first handle defined at the first dynamically determined position, and applies the edit from a second dynamically determined position about the particular 3D object in response to a user interaction with a second handle defined at the second dynamically determined position.
G06F 3/04815 - Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
G06F 3/04817 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
G06F 3/04845 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
G06V 10/22 - Image preprocessing by selection of a specific region containing or referencing a patternLocating or processing of specific regions to guide the detection or recognition
G06V 10/48 - Extraction of image or video features by mapping characteristic values of the pattern into a parameter space, e.g. Hough transformation
Disclosed is a computing system and associated methods that use changes in eye focus or the depth at which a user is looking to modify a user interface. The computing system presents a three-dimensional ("3D") environment with different user interface ("UI") elements that are positioned in a foreground or near plane of the 3D environment and that partially or wholly obscure a background or far plane of the 3D environment. The computing system detects a change in user eye focus from the foreground to the background by using a sensor to track changes to the pupil or the amount of light reflecting off the user's eye. The computing system produces an unobstructed view to all or part of the background by adjusting positioning, opacity, or other properties of the UI elements in the foreground in response to detecting the change in the user eye focus.
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/0481 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
G06F 3/04842 - Selection of displayed objects or displayed text elements
G06F 3/04845 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
G06F 3/04847 - Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
H04N 13/279 - Image signal generators from 3D object models, e.g. computer-generated stereoscopic image signals the virtual viewpoint locations being selected by the viewers or determined by tracking
Disclosed is a system and associated methods for the generative drawing and customization of three- dimensional ("3D") objects in 3D space using hand gestures. The system adapts the hand gestures as intuitive controls for rapidly creating and customizing the 3D objects to have a desired artistic effect or a desired look. The system selects a 3D model of a particular object in response to a first user input, sets a position in a virtual space at which to generate the particular object in response to a mapped position of a first hand gesture tracked in a physical space, and generates a first state representation of the particular object at the position in the virtual space in response to a second hand gesture. The first state representation presents the particular object at one of different modeled stages of the particular object lifecycle.
G06T 19/20 - Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
G06F 3/04883 - Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
Disclosed are systems and methods for splat filling a three-dimensional ("3D") model using semi- measured data. The splat filling includes generating measured data points for a 3D representation of a scene with positions that are measured from scanning the scene, and with color values defined from measured color values of used pixels from a two-dimensional ("2D") of the scene. The splat filling includes generating a semi-measured data point based on an unused pixel of the 2D image. The position of the semi-measured data point is derived based on a separation between the unused pixel and one or more used pixels of the 2D image, and based on the positions of measured data points that are defined with the measured color values of the one or more pixels. The color values of the semi-measured data point are defined directly from the measured color values associated with the unused pixel.
A three-dimensional ("3D") interactive system uses the positional data of the point cloud points to identify exactly where user input contacts part of a 3D object represented by the point cloud points, and to generate precise haptic feedback based on the haptic characteristics of the contacted points. Specifically, the system determines that coordinates of the user input match or are within a threshold distance of a particular data point from a set of data points that form the 3D object. The system retrieves the haptic characteristics of the particular data point, and generates the haptic response on a haptic input device based on the haptic characteristics of the particular data point.
G06F 3/038 - Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
G06F 3/0481 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
G06F 3/04842 - Selection of displayed objects or displayed text elements
Disclosed is an editing system that accounts for or leverages the three-dimensional ("3D") positioning of 3D image data to edit attributes of a selected first set of image data based on attributes of an unselected second set of image data that is determined to be a threshold distance from the first set of image data and on the same surface as the first set of image data. The system leverages x, y, and z coordinates as well as surface normals to exclude image data from the editing that is within the threshold distance but that forms part of a different object, surface, or side about an edge of a surface than the first set of image data. The system also modifies the attribute adjustment based on the distance separating the first set of image data from a render position or each instance of the second set of image data.
G06T 7/90 - Determination of colour characteristics
G06T 17/30 - Surface description, e.g. polynomial surface description
G06V 10/36 - Applying a local operator, i.e. means to operate on image points situated in the vicinity of a given pointNon-linear local filtering operations, e.g. median filtering
G06V 10/44 - Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersectionsConnectivity analysis, e.g. of connected components
G06V 10/771 - Feature selection, e.g. selecting representative features from a multi-dimensional feature space
G06V 10/94 - Hardware or software architectures specially adapted for image or video understanding
21.
SYSTEMS AND METHODS FOR EDITING THREE-DIMENSIONAL IMAGE DATA WITH REALISTIC CAMERA AND LENS EFFECTS
Disclosed is an editing system for postprocessing three-dimensional ("3D") image data to realistically recreate the effects associated with viewing or imaging a represented scene with different camera settings or lenses. The system receives an original image and an edit command with a camera setting or a camera lens. The system associates the selection to multiple image adjustments. The system performs a first of the multiple image adjustments on a first set of 3D image data from the original image in response to the first set of 3D image data satisfying specific positional or non-positional values defined for the first image adjustment, and performs a second of the multiple image adjustments on a second set of 3D image data from the original image in response to the second set of 3D image data satisfying the specific positional or non-positional values defined for the second image adjustment.
Disclosed is an encoding and decoding system and associated methods for producing a compressed waveform that encodes data points of a point cloud in a format and size that may be transmitted over a data network, decompressed, decoded, and rendered on a remote device without the buffering or lag associated with transmitting and rendering an uncompressed point cloud. The encoder receives a request from a remote device to access the point cloud, encodes a set of data points from the point cloud as one or more signals derived from values defined for the positional and non-positional elements of each data point from the set of data points, generates one or more compressed waveforms from compressing the one or more signals and transmits the one or more compressed waveforms to the remote device in response to the request for decompression, decoding, and image rendering.
Disclosed is a system for differentiating the selection of three-dimensional ("3D") image data in a 3D space from other unselected 3D image data that may be positioned in front of the selected 3D image data, and for customizing editing operations that are presented in a user interface based on the object or material property represented in the selection. The system selects a set of 3D image data in response to a user input, and adjusts the transparency of unselected 3D image data that is positioned in front of the selected set of 3D image data. The system presents a differentiated visualization by rendering the selected set of 3D image data according to an original size, position, and visual characteristics defined for the selected set of 3D image, and by performing a partial or fully transparent rendering of the unselected 3D image as a result of the transparency adjustment.
Disclosed is a compression system for compressing image data. The compression system receives an uncompressed image file with data points that are defined with absolute values for elements representing the data point position in a space. The compression system stores the absolute values defined for a first data point in a compressed image file, determines a difference between the absolute values of the first data point and the absolute values of a second data point, derives a relative value for the absolute values of the second data point from the difference, and stores the relative value in place of the absolute values of the second data point in the compressed image file.
Systems and methods for generating a regularized image from non-uniformly distributed image data based on a curve derived from the image data. The curve is used to reduce the non-uniformity in the image data without losing detail or changing the overall image. Regularizing the image includes obtaining a tree-based representation of the non-uniformly distributed image data, generating a regularization curve that models a particular distribution of the image data, and applying the regularization curve to the tree- based representation in order to select the nodes of the tree-based representation for the decimated image data to render in place of the original image data and the original image data to preserve and render as part of the regularized image. Specifically, the system render the original image data associated with leaf nodes and the decimated image data associated with parent nodes that intersect or are within the regularization curve.
Disclosed is a rendering system that dynamically changes the resolution at which image data in a field- of-view ("FOV") is rendered based on the proximity or depth of the image data relative to the render position or camera position. The rendering system selects the image data that is within the FOV, and generates a depth map with a set of distance measures for different distances or depths of the image data in the FOV. The rendering system selects a dynamic resolution at which to render the FOV based on the set of distance measures from the depth map, and renders the image data and/or FOV at the dynamic resolution.
Disclosed is an imaging system and associated methods for mapping in-focus image data from two- dimensional ("2D") images of a scene to a three-dimensional ("3D") model of the scene. The imaging system receives the 2D images and the 3D model, determines the depth of field ("DOF") and the field of view ("FOV") for each 2D image, and selects a subset of 3D model constructs that form the FOV and are within the DOF of a particular 2D image. The imaging system determines pixels of the particular 2D image that represent a same set of points in the scene as the subset of 3D model constructs, and maps the visual characteristics from those pixels to non-positional elements of the subset of 3D model constructs.
Disclosed is a system and associated methods for generating a mutable tree to efficiently access data within a three-dimensional ("3D") environment. The system generates the mutable tree with a root node defined at a root node position, a first branch with nodes for each of a first set of subdivided regions that are a first distance from the root node position, and a second branch with nodes for each of a second set of subdivided regions that are a second distance from the root node position. The system sorts the mutable tree in response to a request to access data from a first position within the 3D environment so that the first node in the first branch is the first subtree node that is closest to the first position, and the first node in the second branch is the second subtree node that is closest to the first position.
A graphics system and associated methods produce a continuous presentation and/or visualization from a point cloud with a distributed and disconnected set of data points that otherwise produce a discontinuous presentation and/or visualization of a scene. The graphics system receives the data points, and expands a polygonal mesh from the position of each particular data point such that each side of the polygonal mesh connects to a side of a polygonal mesh that is expanded from the position of each data point of a set of data points that neighbors the particular data point. The polygonal mesh of the particular data point spans a larger area or volume of the space than the particular data point. The graphics system produces the continuous visualization of the scene from rendering the polygonal mesh that is expanded from the position of each particular data point instead of rendering the data points.
Disclosed is a spatial computing system and associated methods that provide multi-modality interactions for precise and imprecise interactions with two-dimensional ("2D") and three-dimensional ("3D") user interfaces ("UI") that are presented in a 3D interactive space. The multi-modality interactions are provided by a dynamic spatial pointer. The dynamic spatial pointer has a first 3D representation for navigating the 3D interactive space and selecting one of the presented UI elements. The dynamic spatial pointer converts from the first 3D representation to a different second 3D representation in response to attaching to one of the 3D UI elements, and converts to a first 2D representation in response to attaching to one of the 2D UI elements. The second 3D representation remains attached to and tracks the 3D form of the 3D UI element, and the first 2D representation remains attached to and tracks the 2D plane of the 2D UI element.
G06F 3/04815 - Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
31.
SYSTEMS AND METHODS FOR PRIORITIZED RENDERING AND STREAMING BASED ON RISK MAPS THAT PREDICT CHANGE IN A THREE-DIMENSIONAL ENVIRONMENT
A system prioritizes the rendering and streaming of image data based on risk maps that predict change in a three-dimensional ("3D") environment. The system receives primitives that are distributed across a 3D space to represent the 3D environment. The system generates a first image based on primitives that fall within a first view frustum, and generates a risk map with a risk value for each particular pixel of the first image. Each risk value quantifies a probability that a pixel of the first image associated with that risk value changes as a result of changing the first view frustum to a second view frustum. The system then performs an out-of-order rendering of primitives that fall within the second view frustum based on the risk value for each first image pixel that is replaced in a second image with a rendered primitive from the second view frustum.
42 - Scientific, technological and industrial services, research and design
Goods & Services
Photogrammetry; Photogrammetry services; Computer graphics design services, namely, creating three-dimensional computer models; Software as a service (SAAS) services featuring software for generating, manipulating, rendering, viewing, or editing point clouds and other three-dimensional image formats; Software as a service (SAAS) services featuring software for integrating depth measurements with color photography to generate digital three-dimensional images; Providing temporary use of on-line non-downloadable software for generating, manipulating, rendering, viewing or editing three-dimensional representations; Providing temporary use of on-line non-downloadable software for integrating depth measurements with color photography to generate digital three-dimensional images; Providing temporary use of on-line non-downloadable software for processing digital images; Providing temporary use of on-line non-downloadable software for generating visual effects or animations in three-dimensional space; Data encryption and decoding services; Providing temporary use of on-line non-downloadable software codecs for the compression and decompression of point cloud images; Providing a website allowing users to download and stream spatial image data; Providing a website for the electronic storage and distribution of spacial image data
33.
SYSTEMS AND METHODS FOR PREDICTIVE STREAMING OF IMAGE DATA FOR SPATIAL COMPUTING
Disclosed are systems and methods for the out-of-order predictive streaming of elements from a three-dimensional ("3D") image file so that a recipient device is able to produce a first visualization of at least a first streamed element from a particular perspective, similar to the instant transfer of two-dimensional ("2D") images, while the additional elements and perspectives of the 3D image are streamed. The sending device prioritizes the 3D image elements based on a predicted viewing order, streams a particular element from a particular perspective with a priority that is greater than a priority associated with other elements and other perspectives, determines a next element to stream after the particular element based on the next element being positioned adjacent to the particular element and having a priority that is greater than adjacent elements, and streams the next element to the recipient device.
H04N 13/117 - Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
H04N 13/354 - Multi-view displays for displaying three or more geometrical viewpoints without viewer tracking for displaying sequentially
34.
SYSTEMS AND METHODS FOR SYNCHRONIZING THE SCANNING OF A SCENE WITH THE VIEWING OF THE SCAN RESULTS RELATIVE TO A LIVE VIEW OF THE SCENE
Disclosed is a three-dimensional ("3D") scanning system that synchronizes the scanning of a scene with the viewing of the scan results relative to a live view of the scene. The system includes a first device that scans a first set of surfaces that are exposed to the first device from a first position. The system further includes a second device that receives the scan data as it is generated for each scanned surface of the first set of surfaces. The second device augments a visualization of a second set of surfaces, within a field-of-view of the second device from a second position, with the scan data that is generated for a subset of scanned surfaces from the first position corresponding to one or more surfaces of the second set of surfaces visualized from the second position.
An image postprocessor enhances the quality and detail of a rendered image by applying a photosite demosaicing technique directly to the pixels stored in the viewport of the rendered image. The image postprocessor detects a first subset of the viewport pixels having color values that deviate by more than a threshold amount from color values of a neighboring second subset of the viewport pixels. The image postprocessor maps the viewport pixels to a set of emulated photosites based on a position of each pixel in the viewport, and demosaics and/or interpolates the color values associated with a subset of the emulated photosites in order to generate new pixels for the gap. The image postprocessor replaces the first subset of pixels in the viewport with the new pixels, and presents a second visualization based on the modified pixels of the viewport.
Disclosed is a system and associated methods for generating and rendering a polyhedral point cloud that represents a scene with multi-faceted primitives. Each multi-faceted primitive stores multiple sets of values that represent different non-positional characteristics that are associated with a particular point in the scene from different angles. For instance, the system generates a multi-faceted primitive for a particular point of the scene that is captured in first capture from a first position and a second capture from a different second position. Generating the multi- faceted primitive includes defining a first facet with a first surface normal oriented towards the first position and first non-positional values based on descriptive characteristics of the particular point in the first capture, and defining a second facet with a second surface normal orientated towards the second position and second non-positional values based on different descriptive characteristics of the particular point in the second capture.
An editing system and associated methods execute a workflow involving a sequence of two-dimensional/three-dimensional ("2D/3D") data manipulations and/or associated operations to 2D/3D data of different formats by calling functions of different applications that are compatible with the changing format of the 2D/3D data throughout the workflow. The system determines that the 2D/3D data provided as input to a first workflow node is in a first format, and executes the first node by invoking a function of a first application that implements the operations associated with the first node on the 2D/3D data in the first format. Execution of the first node converts the 2D/3D data to a different second format that is passed to a second workflow node. The system executes a second workflow node by invoking a function of a second application that implements the operations associated with the second node on the 2D/3D data in the second format.
Disclosed is a system and associated methods for generating a composite image from scans or images that are aligned using invisible fiducials. The invisible fiducial is a transparent substance or a projected specific wavelength that is applied to and changes reflectivity of a surface at the specific wavelength without interfering with a capture of positions or visible color characteristics across the surface. The system performs first and second capture of a scene with the surface, and detects a position of the invisible fiducial in each capture based on values measured across the specific wavelength that satisfy a threshold associated with the invisible fiducial. The system aligns the first capture with the second capture based on the detected positions of the invisible fiducial, and generates a composite image by merging or combining the positions or visible color characteristics from the aligned captures.
G06V 10/24 - Aligning, centring, orientation detection or correction of the image
G06V 10/50 - Extraction of image or video features by performing operations within image blocksExtraction of image or video features by using histograms, e.g. histogram of oriented gradients [HoG]Extraction of image or video features by summing image-intensity valuesProjection analysis
39.
SYSTEMS AND METHODS FOR LOSSLESS LINE FIT DATA COMPRESSION
A multi-dimensional encoder ("MDE") receives a file with different data points. Each data point is defined with uncompressed multi-dimensional data. The multi-dimensional data may include multiple elements for defining a data point position (e.g., a multi-dimensional position including x, y, and/or z coordinates), color values of the data point (e.g., red, green, and blue), and/or other data point attributes. The MDE assigns an index to each data point or each data point element, and maps the data points to a frequency domain based on a frequency with which values occur in two or more of the elements (e.g., multiple dimensions). The MDE generates a line that represents the frequency, and provides a compressed file format to a requesting device that includes the line and different sets of indices that are associated with different frequencies represented by the line.
Disclosed is a system and associated methods for generating a point-accurate three-dimensional ("3D") model of a scene with point-accurate color information from a non-cosited capture of the scene. The system may define a first model based on positional measurements obtained from a first device. The system may capture images of the scene with a different set of devices, and may generate a second model from a photogrammetry modeling of the images. The system may align the models in order to determine positioning of the set of devices relative to the first model, and may select a particular device that captures descriptive characteristics from a position in the scene that is represented by the positional elements of a particular data point. The system may define non-positional elements of the particular data point based on the descriptive characteristics captured by the particular device.
An encoder is disclosed that uses hyperspectral data to produce a unified three-dimensional ("3D") scan that incorporates depth for various points, surfaces, and features within a scene. The encoder may scan a particular point of the scene using frequencies from different electromagnetic spectrum bands, may determine spectral properties of the particular point based on returns measured across a first set of bands, may measure a distance of the particular point using frequencies of another band that does not interfere with the spectral properties at each of the first set of bands, and may encode the spectral properties and the distance of the particular point in a single hyperspectral dataset. The spectral signature encoded within the dataset may be used to classify the particular point or generate a point cloud or other visualization that accurately represents the spectral properties and distances of the scanned points.
Disclosed is a rigging system for animating the detached and non-uniformly distributed data points of a point cloud. In response to a selection of a region of space in which a first set of data points are located, the system may identify commonality in the positional or non-positional elements of a first subset of the first set of data points, and may determine that a second subset of the first set of data points lack the commonality. The system may refine the first set of data points to a second set of data points that includes the first subset of data points and that excludes the second subset of data points. The system may link the second set of data points to a bone of a skeletal framework, and may animate the second set of data points based on an animation that is defined for the bone.
An augmented reality system enhances or augments the view of a real-world object by replacing different portions of the object with visualizations of non-visible data encoded for those different portions. The system may receive an image that captures visible properties of an object from a particular perspective. The system may select a model with data points that are generated from scanning the object and that encode visible and non-visible properties of the object. The system may align the model to match the object at the particular perspective, may map the non-visible properties from a set of the data points to an object part that is represented and aligned with the set of data points, and may present an augmented reality view in which the visible properties at the object part from the particular perspective are replaced with a visualization of the non-visible properties from the set of data points.
G06T 19/00 - Manipulating 3D models or images for computer graphics
G06V 10/44 - Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersectionsConnectivity analysis, e.g. of connected components
44.
SYSTEMS AND METHODS FOR ENCODING HYPERSPECTRAL DATA WITH VARIABLE BAND RESOLUTIONS
An encoder may perform a dynamic encoding that adapts the encoding of hyperspectral data according to the number of bands of the electromagnetic spectrum that are captured by different imaging devices, the amount of data that is contained in each band, and/or encoding criteria that are specified by a user or that are automatically generated by the encoder for an optimal encoding of the hyperspectral data. The encoder may receive hyperspectral data for different electromagnetic spectrum bands. The encoder may determine an encoding resolution based on one or more of a number of bands and a maximum resolution within the received bands. The encoder may configure a block size for a file format that is used to store an encoding of the hyperspectral data based on the encoding resolution, and may encode the hyperspectral data contained within each band to at least one block of the block size.
Disclosed are three-dimensional ("3D") graphical user interface ("GUI") elements for improving user interactions with a digital environment or a device by simplifying access to different data, functionality, and operations of the digital environment or the device. A 3D GUI element may include first visual information at a first position and second visual information at a second position within the 3D space represented by the 3D GUI element. In response to first input directed to the first visual information, the 3D GUI or system may perform a first action that is mapped to the first input and the first visual information within the 3D GUI element. In response to second input directed to the second visual information, the 3D GUI or system may perform a second action that is mapped to the second input and the second visual information within the 3D GUI element.
G06F 3/0487 - Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
G06F 3/04815 - Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
G06F 3/04817 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
G06F 9/451 - Execution arrangements for user interfaces
46.
SYSTEMS AND METHODS FOR LIDAR-BASED CAMERA METERING, EXPOSURE ADJUSTMENT, AND IMAGE POSTPROCESSING
Disclosed is Light Detection and Ranging ("LiDAR")-based camera metering, exposure adjustment, and image postprocessing. The LiDAR-based exposure adjustment may include emitting a laser from an imaging device, obtaining one or more measurements based on the laser reflecting off one or more objects in a scene and returning to the imaging device, adjusting exposure settings of the imaging device based on the one or more measurements, and capturing an image of the scene using the exposure settings. The LiDAR-based image postprocessing may include receiving an image of a scene and measurements or outputs from a LiDAR scan of the scene, and performing different adjustments to color values, contrast, brightness, saturation, levels, and other visual characteristics of different sets of pixels in the image based on different distance, material property, and/or other measurements obtained by the LiDAR for objects represented by the different sets of pixels.
A rendering system adapts textures used by a Graphics Processing Unit ("GPU") for the purpose of transferring data points of a point cloud to the GPU for processing and rendering. The rendering system determines the GPU constraints, and obtains a tree-based representation for the point cloud data points. The rendering system selects nodes from the tree-based representation that represent at least a first set of data points within the field-of-view with a first level of detail, that represent a second set of data points within the field-of-view with a different second level of detail, and that collectively satisfy the GPU constraints. The rendering system partitions a texture into chunks, encodes each chunk with data from different selected nodes, and presents an image based on rendered partial images generated by the GPU for the texture chunks.
Disclosed are editing tools for manipulating a three-dimensional ("3D") data file or point cloud. An editing application may generate a visualization of the 3D data file or point cloud, and a user may invoke an editing tool over a particular region of the visualization that is rendered based on the positional and non-positional values of a first data point set and a second data point set from the 3D data file or point cloud. The editing tool may differentiate the first data point set from the second data point set based on unique commonality in the positional and/or non-positional values of the first data point set, and may edit less than all of the particular region by adjusting one or more of the positional and/or non-positional values of the first data point set while retaining the positional and non-positional values of the second data point set.
Disclosed is a heap memory manager that manages the entry and removal of data from a single allocated block of memory. The heap memory manager may receive a set of nodes from a tree-based representation of a point cloud or other image, may allocate a single block of memory for processing and/or rendering the set of nodes, may assign each node of the set of nodes to an exclusive range of addresses within the single block of memory, and may upload the data of each of the set of nodes to the single block of memory in the exclusive range of addresses assigned to each node. A controller may then invoke a compute kernel with one or more compute kernel instances processing an address range that is different than the exclusive range of addresses assigned to each node of the set of nodes.
Disclosed is a system that receives a point cloud, and that generates a Bounding Volume Hierarchy ("BVH") based on the point cloud data points. The BVH includes leaf nodes and parent nodes at one or more levels above the leaf nodes. The leaf nodes correspond to the point cloud data points. The system may receive input for adjusting a first set of elements of data points that are identified based on values specified for a second set of elements, and may locate those data points by traversing the BVH to arrive at a particular parent node that encompasses the values specified for the second set of elements. The system may then modify, based on the input, the first set of elements of a set of data points that correspond to a set of leaf nodes from the BVH that are directly or indirectly linked to the particular parent node.
Disclosed is a system for dynamically classifying different data point sets within a point cloud with different classifications that may alter how data point sets with different classifications are processed, edited, and/or rendered. The system may generate a model based on a first set of relationships between a first set of data point elements that result in the first classification, and a second set of relationships between a second set of data point elements that result in the second classification. The system may compare the data point elements from unclassified data point sets against the first set of relationships and the second set of relationships in the model, and may assign the first classification to a particular unclassified data point set in response to the data point elements of the particular data point set having a threshold amount of the first set of relationships.
G06V 10/764 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
52.
SYSTEMS AND METHODS FOR DEFINING, BONDING, AND EDITING POINT CLOUD DATA POINTS WITH PRIMITIVES
Disclosed are systems and methods for defining, bonding, and editing point cloud data points with primitives. In particular, a system may receive a point cloud with data points that are defined by positional and non-positional elements. The system may associate primitives to each data point based on a material or a property associated with the surface, feature, or article at a position in an imaged three-dimensional ("3D") environment that corresponds to a particular position of the data point in the point cloud. The system may receive input for a first adjustment to at least one of the data point elements, and may edit point cloud by defining a second adjustment that modifies the first adjustment according to a rule of a particular primitive, and by applying the second adjustment to the at least one element of a first set of data points that are associated with the particular primitive.
Disclosed is a system to add photorealistic detail and motion to an image based on a first material property associated with a first set of data points of an incomplete first object, and a second material property associated with a second set of data points of an incomplete second object in the image. The system may generate first artificial data points amongst the first set of data points that completes a first arrangement defined for the first material property, and may generate second artificial data points amongst the second set of data points that completes a second arrangement defined for the second material property. The system may then output an enhanced image of the completed first object based on first set of data points and the first artificial data points, and of the completed second object based on the second set of data points and the second artificial data points.
H04N 19/895 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving methods or arrangements for detection of transmission errors at the decoder in combination with error concealment
H04N 19/85 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
H04N 19/65 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using error resilience
G06F 17/00 - Digital computing or data processing equipment or methods, specially adapted for specific functions
G06T 15/00 - 3D [Three Dimensional] image rendering
54.
SYSTEMS AND METHODS FOR DYNAMICALLY RENDERING IMAGES OF THREE-DIMENSIONAL DATA WITH VARYING DETAIL TO EMULATE HUMAN VISION
Disclosed is a system and associated methods for dynamically rendering an image with varying detail that emulates human vision and that provides a dynamic resolution or level of detail at each layer of the image that is equal to or greater than the resolvable detail that can be detected by human vision within each layer. The system may adjust a non-linear function based on one or more of a display size, a display resolution, and a viewer distance from a display. The system may determine a dynamic resolution or level of detail for each layer of the image based on the adjusted non-linear function. The system may render the image data at or greater than the dynamic resolution or level of detail determined for each layer.
Disclosed is a checksum generation and validation system and associated methods for dynamically generating and validating checksums with customizable levels of integrity verification. The system receives a file with data points defined with positional values and non-positional values, and differentiates a first set of the data points from a second set of the data points. The system generates a checksum based on a combination of two or more values from the positional values and the non-positional values of each data point from the first set of data points, and further based on exclusion of the positional values and the non-positional values of the second set of data points from the checksum. The system may use the checksum to verify the integrity of the data associated with the first set of data points.
Disclosed is a system for efficiently accessing a point cloud via a multi-tree deconstruction of the point cloud. The system may receive the point cloud, may differentiate different sets of data points from the point cloud using differentiation criteria, and may generate different trees with each tree having leaf nodes corresponding to one of the differentiated sets of data points and parent nodes defined according to commonality in values of two or more leaf nodes. The system may receive a request to render the 3D environment, load a first tree into memory, generate a first image from the first tree data points, flush the first tree from the memory, load a second tree into the memory, generate a second image from the second tree data points, and present a composite image by combining at least the first image with the second image.
Disclosed is a system and method for rendering point clouds via a hybrid data point and construct visualization. The system receives a point cloud of a three-dimensional ("3D") environment, and differentiates a first set of the point cloud data points from a second set of the data points based on a position of each data point relative to a specified render position. The system generates a first visualization from values of each of the first set of data points, and a second visualization from values of a set of constructs that replace the second set of data points. Each construct has a polygonal shape and a singular set of values defined from the values of two or more of the second set of data points. The system presents a final render of the 3D environment from the render position by combining the first visualization with the second visualization.
Disclosed is a system and method for rendering point clouds via a hybrid data point and construct visualization. The system receives a point cloud of a three-dimensional ("3D") environment, and differentiates a first set of the point cloud data points from a second set of the data points based on a position of each data point relative to a specified render position. The system generates a first visualization from values of each of the first set of data points, and a second visualization from values of a set of constructs that replace the second set of data points. Each construct has a polygonal shape and a singular set of values defined from the values of two or more of the second set of data points. The system presents a final render of the 3D environment from the render position by combining the first visualization with the second visualization.
Disclosed is an imaging system that uses point cloud data point depth values and/or positional data to accurately differentiate and extract features from a point cloud representation of a three-dimensional ("3D") environment without loss of extremely fine details of the extracted features, and without reliance on green screens or other chroma keying techniques. The imaging system differentiate and extract a set of data points, that represent a particular feature in the 3D environment, from other data points of a first point cloud based on the positional data of the set of data points being within depth values specified for feature extraction. The imaging system may generate the 3D environment with at least one visual effect by inserting the set of data points into a second point cloud with one or more data points of the second point cloud differing from the data points of the first point cloud.
