Abstract

The interface is to receive an indication to execute an optimize command. The processor is to receive a file name; determine whether adding a file of the file name to a current bin causes the current bin to exceed a threshold; associate the file with the current bin in response to determining that adding the file does not cause the current bin to exceed the bin threshold; in response to determining that adding the file to the current bin causes the current bin to exceed the bin threshold: associate the file with a next bin, indicate that the current bin is closed, and add the current bin to a batch of bins; determine whether a measure of the batch of bins exceeds a batch threshold; and in response to determining that the measure exceeds the batch threshold, provide the batch of bins for processing.

IPC Classes  ?

• G06F 16/172 - Caching, prefetching or hoarding of files
• G06F 16/22 - IndexingData structures thereforStorage structures

Goods & Services

(1) Software; downloadable software; downloadable software for big data processing and analytics; downloadable software for accessing, managing and connecting to data warehouses, data assets, data files, data sources; downloadable software for application database integration; downloadable software for use in ETL (extract, transform, load) data processing; downloadable software for compiling, organizing, visualizing, sharing and analyzing business intelligence; desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; downloadable software featuring libraries for data science training; downloadable software featuring libraries for data analytics; downloadable software featuring libraries for machine learning training; downloadable software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; downloadable software featuring artificial intelligence and machine learning technology; downloadable software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; downloadable software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; downloadable software for modifying and enabling transmission of images, audio, audio visual and video content and data; downloadable software for processing images, graphics, audio, video, and text; downloadable software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; downloadable software for use in data centers and data storage; downloadable software for use as an application programming interface (API); downloadable software development tools; downloadable software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; downloadable software development tools for building designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; downloadable software development kits (SDKs); downloadable software for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; downloadable software for use in data governance, namely, software featuring artificial intelligence that allows users to define and manage policies for gathering, querying, storing, processing, sharing, accessing and disposing of data, create data clean rooms, and track data lineage; downloadable software for retrieval augmented generation (RAG); downloadable software for retrieval augmented generation (RAG) for use in generative AI applications; downloadable software for retrieval augmented generation (RAG) for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; downloadable podcasts in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence (1) Database management; database management consultancy; business consulting in the field of artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; business data analysis; business data consultancy; providing an online marketplace featuring downloadable software applications, data sets, data notebooks, artificial intelligence models, machine learning models; data management and processing, namely, data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, and data science (2) Providing online non-downloadable software; providing online non-downloadable software for big data processing and analytics; providing online non-downloadable software for accessing, managing and connecting to data warehouses, data assets, data files, data sources; providing online non-downloadable software for application database integration; providing online non-downloadable software for use in ETL (extract, transform, load) data processing; providing online non-downloadable software for compiling, organizing, visualizing, sharing and analyzing business intelligence; providing online non-downloadable desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; providing online non-downloadable software featuring libraries for data science training; providing online non-downloadable software featuring libraries for data analytics; providing online non-downloadable software featuring libraries for machine learning training; providing online non-downloadable software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing online non-downloadable software featuring artificial intelligence and machine learning technology; providing online non-downloadable software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing online non-downloadable software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; providing online non-downloadable software for modifying and enabling transmission of images, audio, audio visual and video content and data; providing online non-downloadable software for processing images, graphics, audio, video, and text; providing online non-downloadable software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; providing online non-downloadable software for use in data centers and data storage; providing online non-downloadable software for use as an application programming interface (API); providing online non-downloadable software development tools; providing online non-downloadable software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; providing online non-downloadable software development tools for building designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing online non-downloadable software development kits (SDKs); software as a service (SAAS) services; software as a service (SAAS) services featuring software for big data processing and analytics; software as a service (SAAS) services, namely, featuring software for accessing, managing and connecting to data warehouses, data assets, data files, data sources; software as a service (SAAS) services featuring software for application database integration; software as a service (SAAS) services featuring software for use in ETL (extract, transform, load) data processing; software as a service (SAAS) services featuring software for compiling, organizing, visualizing, sharing and analyzing business intelligence; software as a service (SAAS) services featuring desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; software as a service (SAAS) services featuring software featuring libraries for data science training; software as a service (SAAS) services featuring software featuring libraries for data analytics; software as a service (SAAS) services featuring software featuring libraries for machine learning training; software as a service (SAAS) services featuring software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; software as a service (SAAS) services featuring software featuring artificial intelligence and machine learning technology; software as a service (SAAS) services featuring software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; software as a service (SAAS) services featuring software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; software as a service (SAAS) services featuring software for modifying and enabling transmission of images, audio, audio visual and video content and data; software as a service (SAAS) services featuring software for processing images, graphics, audio, video, and text; software as a service (SAAS) services featuring software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; software as a service (SAAS) services featuring software for use in data centers and data storage; software as a service (SAAS) services featuring software for use as an application programming interface (API); software as a service (SAAS) services featuring software development tools; software as a service (SAAS) services featuring software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; software as a service (SAAS) services featuring software development tools for building designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; software as a service (SAAS) services featuring software development kits (SDKs); platform as a services (PAAS) services; platform as a services (PAAS) services featuring software for big data processing and analytics; platform as a services (PAAS) services, namely, featuring software for accessing, managing and connecting to data warehouses, data assets, data files, data sources; platform as a services (PAAS) services featuring software for use in ETL (extract, transform, load) data processing; platform as a services (PAAS) services featuring software for compiling, organizing, visualizing, sharing and analyzing business intelligence; platform as a services (PAAS) services featuring desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; platform as a services (PAAS) services featuring software featuring libraries for data science training; platform as a services (PAAS) services featuring software featuring libraries for data analytics; platform as a services (PAAS) services featuring software featuring libraries for machine learning training; platform as a services (PAAS) services featuring software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; platform as a services (PAAS) services featuring software featuring artificial intelligence and machine learning technology; platform as a services (PAAS) services featuring software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; platform as a services (PAAS) services featuring software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; platform as a services (PAAS) services featuring software for modifying and enabling transmission of images, audio, audio visual and video content and data; platform as a services (PAAS) services featuring software for processing images, graphics, audio, video, and text; platform as a services (PAAS) services featuring software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; platform as a services (PAAS) services featuring software for use in data centers and data storage; platform as a services (PAAS) services featuring software for use as an application programming interface (API); platform as a services (PAAS) services featuring software development tools; platform as a services (PAAS) services featuring software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; platform as a services (PAAS) services featuring software development tools for building designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; platform as a services (PAAS) services featuring software development kits (SDKs); data mining; computer services, namely, hosting of search platforms on the Internet to allow users to index, integrate, warehouse, mine, process, share, collect, interpret, research, query, visualize, and analyze data; custom design and development of computer software; software engineering services for data processing; development and creation of computer programs for data processing and analysis; providing a website featuring information in the fields of technology, computers, computer software, computer networks, web services, mobile computing and artificial intelligence; technology consulting in the field of artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing of online non-downloadable software for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; providing online non-downloadable software featuring artificial intelligence for use in data governance, namely, software that allows users to define and manage policies for gathering, querying, storing, processing, sharing, accessing and disposing of data, create data clean rooms, and track data lineage; providing online non-downloadable software for retrieval augmented generation (RAG); providing online non-downloadable software for retrieval augmented generation (RAG) for use in generative AI applications; providing online non-downloadable software for retrieval augmented generation (RAG) for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; software as a service (SAAS) services, namely, featuring software use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance; platform as a services (PAAS) services, namely, featuring software for data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance; application service provider services, namely, hosting, managing, developing and maintaining applications, and software of others in the fields of big data processing and analytics, data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science; computer services, namely, hosting of artificial intelligence models, machine learning models, and large language models (LLMs); computer services, namely, hosting of artificial intelligence models, machine learning models, and large language models (LLMs) to allow users to perform search queries and develop inference; technology consulting in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, and data science

Goods & Services

Software; downloadable software; downloadable software for big data processing and analytics; downloadable software for accessing, managing and connecting to data warehouses, data assets, data files, data sources; downloadable software for application database integration; downloadable software for use in ETL (extract, transform, load) data processing; downloadable software for compiling, organizing, visualizing, sharing and analyzing business intelligence; desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; downloadable software featuring libraries for data science training; downloadable software featuring libraries for data analytics; downloadable software featuring libraries for machine learning training; downloadable software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; downloadable software featuring artificial intelligence and machine learning technology; downloadable software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; downloadable software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; downloadable software for modifying and enabling transmission of images, audio, audio visual and video content and data; downloadable software for processing images, graphics, audio, video, and text; downloadable software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; downloadable software for use in data centers and data storage; downloadable software for use as an application programming interface (API); downloadable software development tools; downloadable software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; downloadable software development tools for building designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; downloadable software development kits (SDKs); downloadable software for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; downloadable software for use in data governance, namely, software featuring artificial intelligence that allows users to define and manage policies for gathering, querying, storing, processing, sharing, accessing and disposing of data, create data clean rooms, and track data lineage; downloadable software for retrieval augmented generation (RAG); downloadable software for retrieval augmented generation (RAG) for use in generative AI applications; downloadable software for retrieval augmented generation (RAG) for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; downloadable podcasts in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence. Database management; database management consultancy; business consulting in the field of artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; business data analysis; business data consultancy; providing an online marketplace featuring downloadable software applications, data sets, data notebooks, artificial intelligence models, machine learning models; data management and processing, namely, data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, and data science. Providing online non-downloadable software; providing online non-downloadable software for big data processing and analytics; providing online non-downloadable software for accessing, managing and connecting to data warehouses, data assets, data files, data sources; providing online non-downloadable software for application database integration; providing online non-downloadable software for use in ETL (extract, transform, load) data processing; providing online non-downloadable software for compiling, organizing, visualizing, sharing and analyzing business intelligence; providing online non-downloadable desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; providing online non-downloadable software featuring libraries for data science training; providing online non-downloadable software featuring libraries for data analytics; providing online non-downloadable software featuring libraries for machine learning training; providing online non-downloadable software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, , predictive analytics and business intelligence; providing online non-downloadable software featuring artificial intelligence and machine learning technology; providing online non-downloadable software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing online non-downloadable software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; providing online non-downloadable software for modifying and enabling transmission of images, audio, audio visual and video content and data; providing online non-downloadable software for processing images, graphics, audio, video, and text; providing online non-downloadable software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; providing online non-downloadable software for use in data centers and data storage; providing online non-downloadable software for use as an application programming interface (API); providing online non-downloadable software development tools; providing online non-downloadable software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; providing online non-downloadable software development tools for building designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing online non-downloadable software development kits (SDKs); software as a service (SAAS) services; software as a service (SAAS) services featuring software for big data processing and analytics; software as a service (SAAS) services, namely, featuring software for accessing, managing and connecting to data warehouses, data assets, data files, data sources; software as a service (SAAS) services featuring software for application database integration; software as a service (SAAS) services featuring software for use in ETL (extract, transform, load) data processing; software as a service (SAAS) services featuring software for compiling, organizing, visualizing, sharing and analyzing business intelligence; software as a service (SAAS) services featuring desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; software as a service (SAAS) services featuring software featuring libraries for data science training; software as a service (SAAS) services featuring software featuring libraries for data analytics; software as a service (SAAS) services featuring software featuring libraries for machine learning training; software as a service (SAAS) services featuring software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; software as a service (SAAS) services featuring software featuring artificial intelligence and machine learning technology; software as a service (SAAS) services featuring software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; software as a service (SAAS) services featuring software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; software as a service (SAAS) services featuring software for modifying and enabling transmission of images, audio, audio visual and video content and data; software as a service (SAAS) services featuring software for processing images, graphics, audio, video, and text; software as a service (SAAS) services featuring software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; software as a service (SAAS) services featuring software for use in data centers and data storage; software as a service (SAAS) services featuring software for use as an application programming interface (API); software as a service (SAAS) services featuring software development tools; software as a service (SAAS) services featuring software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; software as a service (SAAS) services featuring software development tools for building designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; software as a service (SAAS) services featuring software development kits (SDKs); platform as a services (PAAS) services; platform as a services (PAAS) services featuring software for big data processing and analytics; platform as a services (PAAS) services, namely, featuring software for accessing, managing and connecting to data warehouses, data assets, data files, data sources; platform as a services (PAAS) services featuring software for use in ETL (extract, transform, load) data processing; platform as a services (PAAS) services featuring software for compiling, organizing, visualizing, sharing and analyzing business intelligence; platform as a services (PAAS) services featuring desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; platform as a services (PAAS) services featuring software featuring libraries for data science training; platform as a services (PAAS) services featuring software featuring libraries for data analytics; platform as a services (PAAS) services featuring software featuring libraries for machine learning training; platform as a services (PAAS) services featuring software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; platform as a services (PAAS) services featuring software featuring artificial intelligence and machine learning technology; platform as a services (PAAS) services featuring software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; platform as a services (PAAS) services featuring software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; platform as a services (PAAS) services featuring software for modifying and enabling transmission of images, audio, audio visual and video content and data; platform as a services (PAAS) services featuring software for processing images, graphics, audio, video, and text; platform as a services (PAAS) services featuring software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; platform as a services (PAAS) services featuring software for use in data centers and data storage; platform as a services (PAAS) services featuring software for use as an application programming interface (API); platform as a services (PAAS) services featuring software development tools; platform as a services (PAAS) services featuring software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; platform as a services (PAAS) services featuring software development tools for building designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; platform as a services (PAAS) services featuring software development kits (SDKs); data mining; computer services, namely, hosting of search platforms on the Internet to allow users to index, integrate, warehouse, mine, process, share, collect, interpret, research, query, visualize, and analyze data; custom design and development of computer software; software engineering services for data processing; development and creation of computer programs for data processing and analysis; providing a website featuring information in the fields of technology, computers, computer software, computer networks, web services, mobile computing and artificial intelligence; technology consulting in the field of artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing of online non-downloadable software for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; providing online non-downloadable software featuring artificial intelligence for use in data governance, namely, software that allows users to define and manage policies for gathering, querying, storing, processing, sharing, accessing and disposing of data, create data clean rooms, and track data lineage; providing online non-downloadable software for retrieval augmented generation (RAG); providing online non-downloadable software for retrieval augmented generation (RAG) for use in generative AI applications; providing online non-downloadable software for retrieval augmented generation (RAG) for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; software as a service (SAAS) services, namely, featuring software use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance; pr platform as a services (PAAS) services, namely, featuring software for data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance; application service provider services, namely, hosting, managing, developing and maintaining applications, and software of others in the fields of big data processing and analytics, data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science; computer services, namely, hosting of artificial intelligence models, machine learning models, and large language models (LLMs); computer services, namely, hosting of artificial intelligence models, machine learning models, and large language models (LLMs) to allow users to perform search queries and develop inference; technology consulting in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, and data science.

Abstract

Various embodiments of the present technology generally relate to management of big data storage and the physical removal of data via data access systems for large data processing environments having multiple application services and multiple storage services. In some embodiments, a method of physically removing data from a storage system provides for identifying one or more files needing data removal treatment, determining that a file needing data removal treatment should be queued, and populating a queue with the file. Determining that a file should be queued is based, at least in part, on a staleness tolerance. The method further provides for treating the file and replacing a previous version of the file in storage with the updated file. In some implementations, treating the file includes removing data from the file to create an updated file and may further include additional changes to the file.

IPC Classes  ?

• G06F 16/16 - File or folder operations, e.g. details of user interfaces specifically adapted to file systems
• G06F 9/54 - Interprogram communication
• G06F 16/17 - Details of further file system functions
• G06F 16/23 - Updating
• G06F 21/62 - Protecting access to data via a platform, e.g. using keys or access control rules

Abstract

A data processing service generates for iteratively applying a geospatial function to geospatial data. The generated code includes at least a first iterative loop and a second iterative loop. The data processing service compiles the generated code to generate compiled code that vectorized at least the second iterative loop. The data processing service receives a request from a client device to perform one or more data processing operations including applying the geospatial function to a data table of geospatial cell indices. The data processing service compiles the request into one or more tasks including at least a vectorized operation based on the compiled code and executes the one or more tasks by at least invoking the vectorized operation on the set of worker nodes.

IPC Classes  ?

• G06F 8/41 - Compilation

Abstract

A data processing system accesses a dataset from a data source and generates a set of embedding vectors representing the dataset in a latent space. The system splits the dataset into a set of data chunks and generates the embedding vectors. Each embedding vector represents a data chunk. The system may store the generated set of embedding vectors in a vector database that includes a plurality of embedding vectors. The system updates the embedding vectors by detecting a change to a first dataset that is represented by a first set of embedding vectors in the vector database, determining that the change to the first dataset is related to a first data chunk of the first set of data chunks included in the first dataset, updating a first embedding vector representing the first data chunk with the detected change; and storing the updated first embedding vector in the vector database.

IPC Classes  ?

• G06F 16/22 - IndexingData structures thereforStorage structures

Abstract

A system manages access control policies for accessing data stored in a plurality of data platforms. The system receives access control policy specification describing an access control policy that controls access to a set of datasets by a set of users. The set of datasets is defined based on a condition based on the data tags representing attributes of the datasets. The system compiles the access control policy specification to generate a platform independent access control representation of the access control policy. The platform independent access control representation comprises a set of tuples. Each tuple identifies a particular set of users, a particular set of datasets, and a particular action. The system further generates data platform specific instructions for each data platform of the plurality of data platforms.

IPC Classes  ?

• G06F 21/62 - Protecting access to data via a platform, e.g. using keys or access control rules

Abstract

A configuration/method/system for cross-platform capture/replay of an emulation of a circuit design. The system receives, during an emulation of a circuit design performed on a first platform, time-annotated ready bits messages (TARBMs), and generates a first set of TARBM files, each corresponding to a particular FPGA in the first set of FPGAs. The system also generates a first set of bitmap file for the first platform, and a second set of bitmap file for a second platform. Each bitmap file records a bit index of each ready bit of an FPGA in a corresponding platform. The system then generates a second set of TARBM files based in part on the first set of TARBM files and the two sets of bitmap files. The emulation of the circuit design can then be replayed on the second platform based in part on the second set of TARBM files.

IPC Classes  ?

• G06F 30/327 - Logic synthesisBehaviour synthesis, e.g. mapping logic, HDL to netlist, high-level language to RTL or netlist
• G06F 30/3312 - Timing analysis

Abstract

A data processing service monitors an activity status of a first set of computing devices that executes code in a notebook. The service may determine that the activity status of the first set of computing devices meets a termination condition. The service may generate, prior to termination of the first set of computing devices, a snapshot recording a current code execution progress for the notebook. To generate the snapshot, the service may determine serialized variables included in the current code execution progress and store the generated snapshot in a data store. The generated snapshot may include the determined serialized variables. The service may terminate the first set of computer devices from executing the code in the notebook.

IPC Classes  ?

• G06F 11/14 - Error detection or correction of the data by redundancy in operation, e.g. by using different operation sequences leading to the same result

Abstract

A system for multipass sort includes a communication interface and a processor. The communication interface is configured to receive from a client device a request to sort a dataset that includes a plurality of rows. The processor is configured to perform a first sort pass on the dataset in part by: extracting prefixes associated with a first schema element associated with the dataset for the plurality of rows; and sorting the extracted prefixes utilizing an integer sort algorithm based on a sort order included in the request to sort the dataset, where sorting the extracted prefixes includes utilizing NULL values to resolve a tied range that includes at least two rows of the plurality of rows having a same extracted prefix.

IPC Classes  ?

• G06F 16/22 - IndexingData structures thereforStorage structures
• G06F 16/2455 - Query execution

Abstract

A database system performs pipelined execution of queries that process batches of streaming data. The database system compiles a database query to generate an execution plan and determines a set of stages based on the execution plan. The database query processes streaming data comprising batches. A scheduler schedules pipelined execution stages of the database query. Accordingly, the database system performs execution of a particular stage processing a batch of the streaming data in parallel with subsequent stages of the database query processing previous batches of the streaming data. The system further maintains watermarks for different stages of the database query.

IPC Classes  ?

• G06F 16/2455 - Query execution
• G06F 9/48 - Program initiatingProgram switching, e.g. by interrupt
• G06F 16/2453 - Query optimisation

Goods & Services

Software; downloadable software; downloadable software for big data processing and analytics; downloadable software for accessing, managing and connecting to data warehouses, data assets, data files, data sources; downloadable software for application database integration; downloadable software for use in ETL (extract, transform, load) data processing; downloadable software for compiling, organizing, visualizing, sharing and analyzing business intelligence; desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; downloadable software featuring libraries for data science training; downloadable software featuring libraries for data analytics; downloadable software featuring libraries for machine learning training; downloadable software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; downloadable software featuring artificial intelligence and machine learning technology; downloadable software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; downloadable software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; downloadable software for modifying and enabling transmission of images, audio, audio visual and video content and data; downloadable software for processing images, graphics, audio, video, and text; downloadable software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; downloadable software for use in data centers and data storage; downloadable software for use as an application programming interface (API); downloadable software development tools; downloadable software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; downloadable software development tools for building, designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; downloadable software development kits (SDKs); downloadable software for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; downloadable software for use in data governance, namely, software featuring artificial intelligence that allows users to define and manage policies for gathering, querying, storing, processing, sharing, accessing and disposing of data, create data clean rooms, and track data lineage; downloadable software for retrieval augmented generation (RAG); downloadable software for retrieval augmented generation (RAG) for use in generative AI applications; downloadable software for retrieval augmented generation (RAG) for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; downloadable podcasts in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence Database management; database management consultancy; business consulting in the field of artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; business data analysis; business data consultancy; providing an online marketplace featuring downloadable software applications, data sets, data notebooks, artificial intelligence models, machine learning models; data management and processing, namely, data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, and data science Educational services; conducting conferences, seminars, classes workshops, courses, and webinars; training services; providing training for certification; non-downloadable podcasts; conducting conferences, seminars, classes workshops, courses, and webinars in field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; training services in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing training for certification in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; non-downloadable publications and blogs in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; non-downloadable podcasts in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; non-downloadable publications and blogs Providing online non-downloadable software; providing online non-downloadable software for big data processing and analytics; providing online non-downloadable software for accessing, managing and connecting to data warehouses, data assets, data files, data sources; providing online non-downloadable software for application database integration; providing online non-downloadable software for use in ETL (extract, transform, load) data processing; providing online non-downloadable software for compiling, organizing, visualizing, sharing and analyzing business intelligence; providing online non-downloadable desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; providing online non-downloadable software featuring libraries for data science training; providing online non-downloadable software featuring libraries for data analytics; providing online non-downloadable software featuring libraries for machine learning training; providing online non-downloadable software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing online non-downloadable software featuring artificial intelligence and machine learning technology; providing online non-downloadable software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing online non-downloadable software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; providing online non-downloadable software for modifying and enabling transmission of images, audio, audio visual and video content and data; providing online non-downloadable software for processing images, graphics, audio, video, and text; providing online non-downloadable software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; providing online non-downloadable software for use in data centers and data storage; providing online non-downloadable software for use as an application programming interface (API); providing online non-downloadable software development tools; providing online non-downloadable software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; providing online non-downloadable software development tools for building, designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing online non-downloadable software development kits (SDKs); software as a service (SAAS) services; software as a service (SAAS) services featuring software for big data processing and analytics; software as a service (SAAS) services, namely, featuring software for accessing, managing and connecting to data warehouses, data assets, data files, data sources; software as a service (SAAS) services featuring software for application database integration; software as a service (SAAS) services featuring software for use in ETL (extract, transform, load) data processing; software as a service (SAAS) services featuring software for compiling, organizing, visualizing, sharing and analyzing business intelligence; software as a service (SAAS) services featuring desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; software as a service (SAAS) services featuring software featuring libraries for data science training; software as a service (SAAS) services featuring software featuring libraries for data analytics; software as a service (SAAS) services featuring software featuring libraries for machine learning training; software as a service (SAAS) services featuring software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; software as a service (SAAS) services featuring software featuring artificial intelligence and machine learning technology; software as a service (SAAS) services featuring software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; software as a service (SAAS) services featuring software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; software as a service (SAAS) services featuring software for modifying and enabling transmission of images, audio, audio visual and video content and data; software as a service (SAAS) services featuring software for processing images, graphics, audio, video, and text; software as a service (SAAS) services featuring software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; software as a service (SAAS) services featuring software for use in data centers and data storage; software as a service (SAAS) services featuring software for use as an application programming interface (API); software as a service (SAAS) services featuring software development tools; software as a service (SAAS) services featuring software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; software as a service (SAAS) services featuring software development tools for building, designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; software as a service (SAAS) services featuring software development kits (SDKs); platform as a services (PAAS) services; platform as a services (PAAS) services featuring software for big data processing and analytics; platform as a services (PAAS) services, namely, featuring software for accessing, managing and connecting to data warehouses, data assets, data files, data sources; platform as a services (PAAS) services featuring software for use in ETL (extract, transform, load) data processing; platform as a services (PAAS) services featuring software for compiling, organizing, visualizing, sharing and analyzing business intelligence; platform as a services (PAAS) services featuring desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; platform as a services (PAAS) services featuring software featuring libraries for data science training; platform as a services (PAAS) services featuring software featuring libraries for data analytics; platform as a services (PAAS) services featuring software featuring libraries for machine learning training; platform as a services (PAAS) services featuring software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; platform as a services (PAAS) services featuring software featuring artificial intelligence and machine learning technology; platform as a services (PAAS) services featuring software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; platform as a services (PAAS) services featuring software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; platform as a services (PAAS) services featuring software for modifying and enabling transmission of images, audio, audio visual and video content and data; platform as a services (PAAS) services featuring software for processing images, graphics, audio, video, and text; platform as a services (PAAS) services featuring software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; platform as a services (PAAS) services featuring software for use in data centers and data storage; platform as a services (PAAS) services featuring software for use as an application programming interface (API); platform as a services (PAAS) services featuring software development tools; platform as a services (PAAS) services featuring software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; platform as a services (PAAS) services featuring software development tools for building, designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; platform as a services (PAAS) services featuring software development kits (SDKs); data mining; computer services, namely, hosting of search platforms on the Internet to allow users to index, integrate, warehouse, mine, process, share, collect, interpret, research, query, visualize, and analyze data; custom design and development of computer software; software engineering services for data processing; development and creation of computer programs for data processing and analysis; providing a website featuring information in the fields of technology, computers, computer software, computer networks, web services, mobile computing and artificial intelligence; technology consulting in the field of artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing of online non-downloadable software for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; providing online non-downloadable software featuring artificial intelligence for use in data governance, namely, software that allows users to define and manage policies for gathering, querying, storing, processing, sharing, accessing and disposing of data, create data clean rooms, and track data lineage; providing online non-downloadable software for retrieval augmented generation (RAG); providing online non-downloadable software for retrieval augmented generation (RAG) for use in generative AI applications; providing online non-downloadable software for retrieval augmented generation (RAG) for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; software as a service (SAAS) services, namely, featuring software use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; platform as a services (PAAS) services, namely, featuring software for data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; application service provider services, namely, hosting, managing, developing and maintaining applications, and software of others in the fields of big data processing and analytics, data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science; computer services, namely, hosting of artificial intelligence models, machine learning models, and large language models (LLMs); computer services, namely, hosting of artificial intelligence models, machine learning models, and large language models (LLMs) to allow users to perform search queries and develop inference; technology consulting in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, and data science

Goods & Services

Software; downloadable software; downloadable software for big data processing and analytics; downloadable software for accessing, managing and connecting to data lakes, data warehouses, data assets, data files, data sources; downloadable software for application database integration; downloadable software for use in ETL (extract, transform, load) data processing; downloadable software for compiling, organizing, visualizing, sharing and analyzing business intelligence; desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; downloadable software featuring libraries for data science training; downloadable software featuring libraries for data analytics; downloadable software featuring libraries for machine learning training; downloadable software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; downloadable software featuring artificial intelligence and machine learning technology; downloadable software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; downloadable software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; downloadable software for modifying and enabling transmission of images, audio, audio visual and video content and data; downloadable software for processing images, graphics, audio, video, and text; downloadable software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; downloadable software for use in data centers and data storage; downloadable software for use as an application programming interface (API); downloadable software development tools; downloadable software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; downloadable software development tools for building, designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; downloadable software development kits (SDKs); downloadable software for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; downloadable software for use in data governance, namely, software featuring artificial intelligence that allows users to define and manage policies for gathering, querying, storing, processing, sharing, accessing and disposing of data, create data clean rooms, and track data lineage; downloadable software for retrieval augmented generation (RAG); downloadable software for retrieval augmented generation (RAG) for use in generative AI applications; downloadable software for retrieval augmented generation (RAG) for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; downloadable podcasts in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence. Database management; database management consultancy; business consulting in the field of artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; business data analysis; business data management consultancy; providing an online marketplace featuring downloadable software applications, data sets, data notebooks, artificial intelligence models, machine learning models; data management and processing, namely, business data analytics and interpretation, compiling and systemization of data into computer databases, data collection, management of data lakes, and administrative support services in the fields of data wrangling, data visualization, data governance, data science. Educational services; conducting conferences, seminars, classes, workshops, courses, and webinars; training services; providing training for certification; educational services featuring podcasts; conducting conferences, seminars, classes, workshops, courses, and webinars in field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; training services in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing training for certification in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing non-downloadable electronic publications and on-line publication of journals or diaries [blog services] in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; educational services featuring podcasts in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing non-downloadable electronic publications and on-line publication of journals or diaries [blog services]. Providing online non-downloadable software; providing online non-downloadable software for big data processing and analytics; providing online non-downloadable software for accessing, managing and connecting to data lakes, data warehouses, data assets, data files, data sources; providing online non-downloadable software for application database integration; providing online non-downloadable software for use in ETL (extract, transform, load) data processing; providing online non-downloadable software for compiling, organizing, visualizing, sharing and analyzing business intelligence; providing online non-downloadable desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; providing online non-downloadable software featuring libraries for data science training; providing online non-downloadable software featuring libraries for data analytics; providing online non-downloadable software featuring libraries for machine learning training; providing online non-downloadable software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing online non-downloadable software featuring artificial intelligence and machine learning technology; providing online non-downloadable software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing online non-downloadable software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; providing online non-downloadable software for modifying and enabling transmission of images, audio, audio visual and video content and data; providing online non-downloadable software for processing images, graphics, audio, video, and text; providing online non-downloadable software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; providing online non-downloadable software for use in data centers and data storage; providing online non-downloadable software for use as an application programming interface (API); providing online non-downloadable software development tools; providing online non-downloadable software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; providing online non-downloadable software development tools for building, designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing online non-downloadable software development kits (SDKs); software as a service (SAAS) services; software as a service (SAAS) services featuring software for big data processing and analytics; software as a service (SAAS) services, namely, featuring software for accessing, managing and connecting to data lakes, data warehouses, data assets, data files, data sources; software as a service (SAAS) services featuring software for application database integration; software as a service (SAAS) services featuring software for use in ETL (extract, transform, load) data processing; software as a service (SAAS) services featuring software for compiling, organizing, visualizing, sharing and analyzing business intelligence; software as a service (SAAS) services featuring desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; software as a service (SAAS) services featuring software featuring libraries for data science training; software as a service (SAAS) services featuring software featuring libraries for data analytics; software as a service (SAAS) services featuring software featuring libraries for machine learning training; software as a service (SAAS) services featuring software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; software as a service (SAAS) services featuring software featuring artificial intelligence and machine learning technology; software as a service (SAAS) services featuring software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; software as a service (SAAS) services featuring software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; software as a service (SAAS) services featuring software for modifying and enabling transmission of images, audio, audio visual and video content and data; software as a service (SAAS) services featuring software for processing images, graphics, audio, video, and text; software as a service (SAAS) services featuring software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; software as a service (SAAS) services featuring software for use in data centers and data storage; software as a service (SAAS) services featuring software for use as an application programming interface (API); software as a service (SAAS) services featuring software development tools; software as a service (SAAS) services featuring software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; software as a service (SAAS) services featuring software development tools for building, designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; software as a service (SAAS) services featuring software development kits (SDKs); platform as a services (PAAS) services; platform as a services (PAAS) services featuring software for big data processing and analytics; platform as a services (PAAS) services, namely, featuring software for accessing, managing and connecting to data lakes, data warehouses, data assets, data files, data sources; platform as a services (PAAS) services featuring software for use in ETL (extract, transform, load) data processing; platform as a services (PAAS) services featuring software for compiling, organizing, visualizing, sharing and analyzing business intelligence; platform as a services (PAAS) services featuring desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; platform as a services (PAAS) services featuring software featuring libraries for data science training; platform as a services (PAAS) services featuring software featuring libraries for data analytics; platform as a services (PAAS) services featuring software featuring libraries for machine learning training; platform as a services (PAAS) services featuring software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; platform as a services (PAAS) services featuring software featuring artificial intelligence and machine learning technology; platform as a services (PAAS) services featuring software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; platform as a services (PAAS) services featuring software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; platform as a services (PAAS) services featuring software for modifying and enabling transmission of images, audio, audio visual and video content and data; platform as a services (PAAS) services featuring software for processing images, graphics, audio, video, and text; platform as a services (PAAS) services featuring software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; platform as a services (PAAS) services featuring software for use in data centers and data storage; platform as a services (PAAS) services featuring software for use as an application programming interface (API); platform as a services (PAAS) services featuring software development tools; platform as a services (PAAS) services featuring software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; platform as a services (PAAS) services featuring software development tools for building, designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; platform as a services (PAAS) services featuring software development kits (SDKs); data mining; computer services, namely, hosting of search platforms on the Internet to allow users to index, integrate, warehouse, mine, process, share, collect, interpret, research, query, visualize, and analyze data; custom design and development of computer software; software engineering services for data processing; development and creation of computer programs for data processing and analysis; providing technological information in the fields of technology, computers, computer software, computer networks, web services, mobile computing and artificial intelligence via a website; technology consulting in the field of artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing of online non-downloadable software for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; providing online non-downloadable software featuring artificial intelligence for use in data governance, namely, software that allows users to define and manage policies for gathering, querying, storing, processing, sharing, accessing and disposing of data, create data clean rooms, and track data lineage; providing online non-downloadable software for retrieval augmented generation (RAG); providing online non-downloadable software for retrieval augmented generation (RAG) for use in generative AI applications; providing online non-downloadable software for retrieval augmented generation (RAG) for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; software as a service (SAAS) services, namely, featuring software for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance; providing online non-downloadable software for retrieval augmented generation (RAG) for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; platform as a services (PAAS) services, namely, featuring software for data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance; providing online non-downloadable software for retrieval augmented generation (RAG) for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; application service provider services, namely, hosting, managing, developing and maintaining applications, and software of others in the fields of big data processing and analytics, data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science; computer services, namely, hosting of artificial intelligence models, machine learning models, and large language models (LLMs); computer services, namely, hosting of artificial intelligence models, machine learning models, and large language models (LLMs) to allow users to perform search queries and develop inference; technology consulting in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, and building, design and management of data lakes; data migration; data mining; data warehousing; development and design of data lakes.

Abstract

A data processing service uses minor compactions for committing transactions to a data table. The service may receive requests to commit transactions to a data table and write metadata for the transactions to log files, and generate a checkpoint file aggregating the transactions described in the log files to compute a data table state at a first time. The service may receive requests to commit a set of transactions and write metadata for the set of transactions to a set of log files. The service may determine that a number of log files in the set of log files reaches a threshold commit number, generate a minor compaction file aggregating the set of transactions, and generate a second checkpoint file aggregating the data table state at the first time with information from the minor compaction file to compute the data table state at a second time.

IPC Classes  ?

• G06F 16/23 - Updating
• G06F 16/174 - Redundancy elimination performed by the file system

Abstract

A cloud computation system configured to 1) receive a first request to read a first set of query results stored in a cloud based data storage; 2) transmit a first subset of the first set of query results in response to the first request; 3) transmit a second subset of the first set of query results in response to the first request; 4) receive a second request to read a second set of query results stored in the cloud based data storage; 5) transmit a first subset of the second set of query results in response to the second request. 6) transmit a second subset of the second set of query results in response to the second request.

IPC Classes  ?

• G06F 16/2458 - Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
• G06F 11/34 - Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation
• G06F 16/242 - Query formulation
• G06F 16/25 - Integrating or interfacing systems involving database management systems

Abstract

A data processing service generates synthetic captions for uncaptioned images of a set of training data. The data processing service applies a pre-trained I2T model to the uncaptioned images, generating synthetic captions as output. The data processing service uses the training data to train a T2I model to produce images from text.

IPC Classes  ?

• G06V 20/70 - Labelling scene content, e.g. deriving syntactic or semantic representations
• G06T 3/40 - Scaling of whole images or parts thereof, e.g. expanding or contracting
• G06V 10/774 - Generating sets of training patternsBootstrap methods, e.g. bagging or boosting

Abstract

Disclosed herein provides enhancements for operating a data access system for large data processing environments. In one implementation, a method provides for maintaining a data structure comprising a plurality of customized code configurations each associated with a data request rule for each of the multiple application services. A code configuration query from a user is then received indicating a data request rule. The code configuration query requests code configurations for data retrieval from at least one of the multiple storage services over the data access system. The data structure is queried for one or more customized code configurations for each of the multiple application services associated with the indicated data request rule. The user is then provided with the one or more customized code configurations for each of the multiple application services associated with the indicated data request rule.

IPC Classes  ?

• G06F 16/22 - IndexingData structures thereforStorage structures
• G06F 8/70 - Software maintenance or management
• G06F 9/54 - Interprogram communication
• G06F 16/2455 - Query execution
• G06F 16/2457 - Query processing with adaptation to user needs
• G06F 16/248 - Presentation of query results
• G06F 16/28 - Databases characterised by their database models, e.g. relational or object models

Abstract

A multi-cluster computing system which includes a query result caching system is presented. The multi-cluster computing system may include a data processing service and client devices communicatively coupled over a network. The data processing service may include a control layer and a data layer. The data layer may include a data storage system, which further includes a remote query result cache Store. The query result cache store may include a cloud storage query result cache which stores data associated with results of previously executed requests. As such, when a cluster encounters a previously executed request, the cluster may efficiently retrieve the cached result of the request from the in-memory query result cache or the cloud storage query result cache.

IPC Classes  ?

• G06F 16/2453 - Query optimisation
• G06F 16/25 - Integrating or interfacing systems involving database management systems
• G06F 16/28 - Databases characterised by their database models, e.g. relational or object models

Goods & Services

Software; downloadable software; downloadable software for big data processing and analytics; downloadable software for accessing, managing and connecting to data lakes, data warehouses, data assets, data files, data sources; downloadable software for application database integration; downloadable software for use in ETL (extract, transform, load) data processing; downloadable software for compiling, organizing, visualizing, sharing and analyzing business intelligence; desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; downloadable software featuring libraries for data science training; downloadable software featuring libraries for data analytics; downloadable software featuring libraries for machine learning training; downloadable software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; downloadable software featuring artificial intelligence and machine learning technology; downloadable software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; downloadable software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; downloadable software for modifying and enabling transmission of images, audio, audio visual and video content and data; downloadable software for processing images, graphics, audio, video, and text; downloadable software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; downloadable software for use in data centers and data storage; downloadable software for use as an application programming interface (API); downloadable software development tools; downloadable software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; downloadable software development tools for building, designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; downloadable software development kits (SDKs); downloadable software for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; downloadable software for use in data governance, namely, software featuring artificial intelligence that allows users to define and manage policies for gathering, querying, storing, processing, sharing, accessing and disposing of data, create data clean rooms, and track data lineage; downloadable software for retrieval augmented generation (RAG); downloadable software for retrieval augmented generation (RAG) for use in generative AI applications; downloadable software for retrieval augmented generation (RAG) for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; downloadable podcasts in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence Database management; database management consultancy; business consulting in the field of artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; business data analysis; business data consultancy; providing an online marketplace featuring downloadable software applications, data sets, data notebooks, artificial intelligence models, machine learning models; data management and processing, namely, data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, and building, design and management of data lakes Educational services; conducting conferences, seminars, classes workshops, courses, and webinars; training services; providing training for certification; non-downloadable podcasts; conducting conferences, seminars, classes workshops, courses, and webinars in field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; training services in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing training for certification in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; non-downloadable publications and blogs in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; non-downloadable podcasts in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; non-downloadable publications and blogs Providing online non-downloadable software; providing online non-downloadable software for big data processing and analytics; providing online non-downloadable software for accessing, managing and connecting to data lakes, data warehouses, data assets, data files, data sources; providing online non-downloadable software for application database integration; providing online non-downloadable software for use in ETL (extract, transform, load) data processing; providing online non-downloadable software for compiling, organizing, visualizing, sharing and analyzing business intelligence; providing online non-downloadable desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; providing online non-downloadable software featuring libraries for data science training; providing online non-downloadable software featuring libraries for data analytics; providing online non-downloadable software featuring libraries for machine learning training; providing online non-downloadable software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing online non-downloadable software featuring artificial intelligence and machine learning technology; providing online non-downloadable software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing online non-downloadable software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; providing online non-downloadable software for modifying and enabling transmission of images, audio, audio visual and video content and data; providing online non-downloadable software for processing images, graphics, audio, video, and text; providing online non-downloadable software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; providing online non-downloadable software for use in data centers and data storage; providing online non-downloadable software for use as an application programming interface (API); providing online non-downloadable software development tools; providing online non-downloadable software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; providing online non-downloadable software development tools for building, designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing online non-downloadable software development kits (SDKs); software as a service (SAAS) services; software as a service (SAAS) services featuring software for big data processing and analytics; software as a service (SAAS) services, namely, featuring software for accessing, managing and connecting to data lakes, data warehouses, data assets, data files, data sources; software as a service (SAAS) services featuring software for application database integration; software as a service (SAAS) services featuring software for use in ETL (extract, transform, load) data processing; software as a service (SAAS) services featuring software for compiling, organizing, visualizing, sharing and analyzing business intelligence; software as a service (SAAS) services featuring desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; software as a service (SAAS) services featuring software featuring libraries for data science training; software as a service (SAAS) services featuring software featuring libraries for data analytics; software as a service (SAAS) services featuring software featuring libraries for machine learning training; software as a service (SAAS) services featuring software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; software as a service (SAAS) services featuring software featuring artificial intelligence and machine learning technology; software as a service (SAAS) services featuring software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; software as a service (SAAS) services featuring software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; software as a service (SAAS) services featuring software for modifying and enabling transmission of images, audio, audio visual and video content and data; software as a service (SAAS) services featuring software for processing images, graphics, audio, video, and text; software as a service (SAAS) services featuring software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; software as a service (SAAS) services featuring software for use in data centers and data storage; software as a service (SAAS) services featuring software for use as an application programming interface (API); software as a service (SAAS) services featuring software development tools; software as a service (SAAS) services featuring software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; software as a service (SAAS) services featuring software development tools for building, designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; software as a service (SAAS) services featuring software development kits (SDKs); platform as a services (PAAS) services; platform as a services (PAAS) services featuring software for big data processing and analytics; platform as a services (PAAS) services, namely, featuring software for accessing, managing and connecting to data lakes, data warehouses, data assets, data files, data sources; platform as a services (PAAS) services featuring software for use in ETL (extract, transform, load) data processing; platform as a services (PAAS) services featuring software for compiling, organizing, visualizing, sharing and analyzing business intelligence; platform as a services (PAAS) services featuring desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; platform as a services (PAAS) services featuring software featuring libraries for data science training; platform as a services (PAAS) services featuring software featuring libraries for data analytics; platform as a services (PAAS) services featuring software featuring libraries for machine learning training; platform as a services (PAAS) services featuring software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; platform as a services (PAAS) services featuring software featuring artificial intelligence and machine learning technology; platform as a services (PAAS) services featuring software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; platform as a services (PAAS) services featuring software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; platform as a services (PAAS) services featuring software for modifying and enabling transmission of images, audio, audio visual and video content and data; platform as a services (PAAS) services featuring software for processing images, graphics, audio, video, and text; platform as a services (PAAS) services featuring software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; platform as a services (PAAS) services featuring software for use in data centers and data storage; platform as a services (PAAS) services featuring software for use as an application programming interface (API); platform as a services (PAAS) services featuring software development tools; platform as a services (PAAS) services featuring software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; platform as a services (PAAS) services featuring software development tools for building, designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; platform as a services (PAAS) services featuring software development kits (SDKs); data mining; computer services, namely, hosting of search platforms on the Internet to allow users to index, integrate, warehouse, mine, process, share, collect, interpret, research, query, visualize, and analyze data; custom design and development of computer software; software engineering services for data processing; development and creation of computer programs for data processing and analysis; providing a website featuring information in the fields of technology, computers, computer software, computer networks, web services, mobile computing and artificial intelligence; technology consulting in the field of artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing of online non-downloadable software for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; providing online non-downloadable software featuring artificial intelligence for use in data governance, namely, software that allows users to define and manage policies for gathering, querying, storing, processing, sharing, accessing and disposing of data, create data clean rooms, and track data lineage; providing online non-downloadable software for retrieval augmented generation (RAG); providing online non-downloadable software for retrieval augmented generation (RAG) for use in generative AI applications; providing online non-downloadable software for retrieval augmented generation (RAG) for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; software as a service (SAAS) services, namely, featuring software use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; platform as a services (PAAS) services, namely, featuring software for data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; application service provider services, namely, hosting, managing, developing and maintaining applications, and software of others in the fields of big data processing and analytics, data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science; computer services, namely, hosting of artificial intelligence models, machine learning models, and large language models (LLMs); computer services, namely, hosting of artificial intelligence models, machine learning models, and large language models (LLMs) to allow users to perform search queries and develop inference; technology consulting in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, and building, design and management of data lakes

Abstract

The present application discloses a method, system, and computer system for managing a data in a storage system. The method includes receiving a first transaction that modifies or deletes first data stored in a storage system, determining that the first data is subject to an intervening re-arrangement transaction, and in response to determining that the first data is subject to the intervening re-arrangement transaction, rolling back the re-arrangement transaction at least with respect to the first data and committing the first transaction.

IPC Classes  ?

• G06F 16/174 - Redundancy elimination performed by the file system

Abstract

A data processing service trains a transformer model in two stages. In a first stage, for a first number of iterations, the data processing service trains the model without computing moving average parameters. In a second stage, for a second number of iterations, the data processing service trains the model using parameters that follow a moving average of the training parameters. In the second stage, the data processing service obtains moving average parameters for a current iteration and generates training parameters for the current iteration. The data processing service computes moving average parameters for a next iteration by combining the training parameters for the current iteration and the moving average parameters for the current iteration. The data processing service updates the moving average parameters for the next iteration as the moving average parameters for the current iteration.

IPC Classes  ?

• G06N 20/00 - Machine learning

Abstract

A method for generating a trust credential for an AI-driven application is presented. The method includes identifying one or more risk factors, receiving a request to generate a trust credential for an AI-driven application, and receiving the AI-driven application and associated data, wherein the AI-driven application has one or more subcomponents. The method includes applying a risk determination function to each of the one or more subcomponents of the AI-driven application and the associated data to generate a risk score for each of the one or more subcomponents. The method further includes applying a weighting function to the risk score of each subcomponent to generate a trust score for each of the one or more subcomponents, and generating the trust credential for the AI-driven application based on the trust scores of each of the one or more subcomponents.

IPC Classes  ?

• G06Q 10/0635 - Risk analysis of enterprise or organisation activities

Abstract

The present application discloses a method, system, and computer system for managing a plurality of features and storing lineage information pertaining to the features. The method includes obtaining one or more datasets, determining a first feature, wherein the first feature is determined based at least in part on the one or more datasets, and storing the first feature in a feature store. The first feature is stored in association with a dataset indication of the one or more datasets from which the first feature is determined. The feature store comprises a plurality of features.

IPC Classes  ?

• G06F 16/28 - Databases characterised by their database models, e.g. relational or object models
• G06F 30/27 - Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model

Abstract

A system manages access control for database queries in accordance with access control policies. The techniques may be used for access control of managed clusters in cloud platforms used for data processing, for example, for MapReduce operations. According to an embodiment, the access control policies are fine grained access control policies that allow a user to access a subset of datasets including data of a dataset. The system receives and compiles a database query to generate a query plan for processing the database query. The query plan includes: one or more data access operators, one or more data processing operators, and one or more access control filters. The system executes the query plan using an executor process and a helper process. The helper process executes a data access operator and a corresponding data access filter and the executor process executes one or more data processing operators.

IPC Classes  ?

• G06F 21/62 - Protecting access to data via a platform, e.g. using keys or access control rules
• G06F 16/2453 - Query optimisation

Abstract

A data processing service accesses data files from data streams, each data file including samples to be processed for training a machine-learning model. The service converts the data files to discrete shard files, each shard file comprising a subset of samples. Each sample ID is mapped to a shard index of a respective shard file that includes the sample. The service may generate partition tensors that partition sample index spaces into a number of physical nodes, devices, workers, and batches. The service may shuffle the shard files and divide the sample IDs into a number of logical nodes and shuffle the sample IDs. The service generates shuffled sample ID arrays that map the sample indices to the sample IDs. During training, workers download disjoint shard files and map the assigned sample indices to corresponding batches of sample IDs based on the shuffled sample ID arrays.

IPC Classes  ?

• G06F 16/16 - File or folder operations, e.g. details of user interfaces specifically adapted to file systems
• G06F 16/11 - File system administration, e.g. details of archiving or snapshots
• G06F 16/176 - Support for shared access to filesFile sharing support

Abstract

The present application discloses a method, system, and computer system for providing an integrated workspace file system. The method includes receive a receiving a request to access a first file, determining a first user associated with the request to access the first file, determining whether the first user is authorized to access the first file, and in response to determining that the first user is authorized to access the first file: generating a uniform resource identifier (URI) associated with the first file, wherein the URI comprises a credential for accessing the first file, wherein the credential is based on a user credential and a first file access authorization, and providing the URI to a user system.

IPC Classes  ?

• G06F 21/62 - Protecting access to data via a platform, e.g. using keys or access control rules

Abstract

A system trains a machine learning model, such as a language model for a set of iterations using a learning rate that is a piecewise function comprising: (1) a first range of inputs for which the learning rate is linearly increasing in value with the number of iterations, (2) a second range of inputs after the first range of inputs for which the learning rate comprises: a first term that varies as inverse square root of the number of iterations, and a second term that has a constant value with respect to the number of iterations, and (3) a third range of inputs for which the learning rate is linearly decreasing in value with the number of iterations. The system evaluates the trained language model and determines based on the evaluation, whether the trained language model should be deployed.

IPC Classes  ?

• G06N 20/00 - Machine learning

Abstract

A system performs a batch selection process on a small model and uses the results of batch selection to train a large language model (LLM). The system receives training examples and splits the training examples into a holdout set and an evaluation set. Each training example corresponds to a label. The system uses trains a small model using the training examples of the holdout set. The system evaluates the small model on the training examples of the evaluation set, generating a prediction for each training example and computing a loss between the prediction and the training example's label. The system generates an LLM training set by selecting a set of training examples from the evaluation set with the highest loss. The system trains the LLM using the LLM training set.

IPC Classes  ?

• G06N 3/0455 - Auto-encoder networksEncoder-decoder networks
• G06N 3/084 - Backpropagation, e.g. using gradient descent

Abstract

A system manages access control policies for accessing data stored in a plurality of data platforms. The system receives access control policy specification describing an access control policy that controls access to a set of datasets by a set of users. The set of datasets is defined based on a condition based on the data tags representing attributes of the datasets. The system compiles the access control policy specification to generate a platform independent access control representation of the access control policy. The platform independent access control representation comprises a set of tuples. Each tuple identifies a particular set of users, a particular set of datasets, and a particular action. The system further generates data platform specific instructions for each data platform of the plurality of data platforms.

IPC Classes  ?

• G06F 21/62 - Protecting access to data via a platform, e.g. using keys or access control rules

Abstract

A data processing service performs a training process to train a transformer architecture including a set of decoders coupled to receive a set of inputs and generate a set of outputs. At least one decoder or encoder includes an attention block coupled to receive a query, a key, and a value and generate an attention output. For one or more iterations, the data processing service obtains a batch of training instances for a current iteration. The parameters of the transformer architecture for the current iteration are applied to a set of inputs obtained from the batch of training instances to generate a set of estimated outputs. The applying includes obtaining a query, a key, and a value from the set of inputs, and applying a clipping function to values of the query, the key, the value.

IPC Classes  ?

• G06N 3/084 - Backpropagation, e.g. using gradient descent

Abstract

A data processing service uses minor compactions for committing transactions to a data table. The service may receive requests to commit transactions to a data table and write metadata for the transactions to log files, and generate a checkpoint file aggregating the transactions described in the log files to compute a data table state at a first time. The service may receive requests to commit a set of transactions and write metadata for the set of transactions to a set of log files. The service may determine that a number of log files in the set of log files reaches a threshold commit number, generate a minor compaction file aggregating the set of transactions, and generate a second checkpoint file aggregating the data table state at the first time with information from the minor compaction file to compute the data table state at a second time.

IPC Classes  ?

• G06F 16/23 - Updating
• G06F 16/174 - Redundancy elimination performed by the file system

Abstract

A system performs efficient startup of executors of a distributed computing engine used for processing queries, for example, database queries. The system starts an executor node and processes a set of queries using the executor node to warm up the executor node. The system performs a checkpoint of the warmed-up executor node to create an image. The image is restored in the target executor nodes. The system may store a checkpoint image for each configuration of an executor node. The configuration is determined based on various factors including the hardware of the executor node, memory allocation of the processes, and so on. The user or restore based on checkpoint images improves efficiency of execution of the startup of executor nodes.

IPC Classes  ?

• G06F 16/2453 - Query optimisation

Abstract

The present application discloses a method, system, and computer system for automatically scaling task-processing capacity. The method includes obtaining, at a data layer, a current measure of queued tasks and/or a task-processing capacity, obtaining, by one or more processors, a cost-prioritized criterion or a latency-prioritized criterion, determining a set of tasks to process using the task-processing capacity based at least in part on a set of an input data to process, and automatically scaling the task-processing capacity based at least in part on the current measure of queued tasks and/or the task-processing capacity and either (i) the cost-prioritized criterion or (ii) the latency-prioritized criterion.

IPC Classes  ?

• G06F 9/48 - Program initiatingProgram switching, e.g. by interrupt
• G06F 9/355 - Indexed addressing
• G06F 9/445 - Program loading or initiating

Abstract

A system receives ETL specification for processing stream data, including a transform operation represented using a database query specification. The system generates a dataflow graph of a sequence of database queries by decomposing the database query into a first database query that generates an intermediate results table, and a second database query that receives as input the intermediate results table and outputs data used for performing the transform operation. The system executes the sequence of database queries for performing the transform operation on stream data received from the source. When receiving an incremental data set, the system determines an output change set based on the received incremental data set by traversing an execution plan and processing each operator in the execution plan, and computing a change set of a particular operator from the change sets output by the one or more other operators based on the incremental data set.

IPC Classes  ?

• G06F 16/25 - Integrating or interfacing systems involving database management systems
• G06F 16/2455 - Query execution

Abstract

A system receives ETL specification for processing stream data, including a transform operation represented using a database query specification. The system generates a dataflow graph of a sequence of database queries by decomposing the database query into a first database query that generates an intermediate results table, and a second database query that receives as input the intermediate results table and outputs data used for performing the transform operation. The system executes the sequence of database queries for performing the transform operation on stream data received from the source. When receiving an incremental data set, the system determines an output change set based on the received incremental data set by traversing an execution plan and processing each operator in the execution plan, and computing a change set of a particular operator from the change sets output by the one or more other operators based on the incremental data set.

IPC Classes  ?

• G06F 16/00 - Information retrievalDatabase structures thereforFile system structures therefor
• G06F 16/2453 - Query optimisation
• G06F 16/2455 - Query execution
• G06F 16/25 - Integrating or interfacing systems involving database management systems

Abstract

The present application discloses a method, system, and computer system for starting up and maintaining a cluster in a warmed up state, and/or allocating clusters from a warmed up state. The method includes instantiating a set of virtual machines, wherein instantiating the set of virtual machines includes setting a temporary security credential for each virtual machine of the set of virtual machines, receiving a virtual machine allocation request associated with a workspace, a customer, or a tenant, in response to the virtual machine allocation request: allocating a virtual machine, wherein allocating the virtual machine comprises replacing the temporary security credential with a security credential associated with the workspace, the customer, or the tenant.

IPC Classes  ?

• G06F 9/50 - Allocation of resources, e.g. of the central processing unit [CPU]
• G06F 21/45 - Structures or tools for the administration of authentication

Abstract

The present application discloses a method, system, and computer system for providing access to information stored on system for data storage. The method includes receiving a data request from a user, determining data corresponding to the data request, determining whether the user has requisite permissions to access the data, and in response to determining that the user has requisite permissions to access the data: determining a manner by which to provide access to the data, wherein the data comprises a filtered subset of stored data, and generating a token based at least in part on the user and the manner by which access to the data is to be provided.

IPC Classes  ?

• G06F 21/62 - Protecting access to data via a platform, e.g. using keys or access control rules
• G06F 3/06 - Digital input from, or digital output to, record carriers

Abstract

The present application discloses a method, system, and computer system for processing data. The method includes obtaining a query plan for processing input data in response to a query, obtaining the input data, selecting a batch of the input data, creating a metadata structure for the batch, allocating one or more contiguous parts of a memory for processing the batch, processing the batch in accordance with the metadata structure to generate resulting data, and storing each array of the resulting data for the batch in one of the one or more contiguous parts of the memory.

IPC Classes  ?

• G06F 16/23 - Updating
• G06F 16/2453 - Query optimisation
• G06F 16/2455 - Query execution

Abstract

A data processing service builds a container for a customer to run a trained large language model (LLM). The data processing service receives a trained LLM and a desired configuration from a user of a client device. Based on the desired configuration, the data processing service selects a hardware configuration and structures weights of the trained LLM based on the hardware configuration. The data processing service generates a container image reflecting the hardware configuration, registers the container image to a container registry, and generates a container from the container image as well as an application programming interface (API) endpoint for the container. The data processing service deploys the trained LLM in the API endpoint using the container such that the trained LLM is accessible through API calls.

IPC Classes  ?

• G06F 9/445 - Program loading or initiating

Abstract

A data processing service facilitates the creation and processing of data processing pipelines that process data processing jobs defined with respect to a set of tasks in a sequence and with data dependencies associated with each separate task such that the output from one task is used as input for a subsequent task. In various embodiments, the set of tasks include at least one cleanroom task that is executed in a cleanroom station and at least one non-cleanroom task executed in an execution environment of a user where each task is configured to read one or more input datasets and transform the one or more input datasets into one or more output datasets.

IPC Classes  ?

• G06F 21/62 - Protecting access to data via a platform, e.g. using keys or access control rules

Abstract

A database system performs pipelined execution of queries that process batches of streaming data. The database system compiles a database query to generate an execution plan and determines a set of stages based on the execution plan. The database query processes streaming data comprising batches. A scheduler schedules pipelined execution stages of the database query. Accordingly, the database system performs execution of a particular stage processing a batch of the streaming data in parallel with subsequent stages of the database query processing previous batches of the streaming data. The system further maintains watermarks for different stages of the database query.

IPC Classes  ?

• G06F 16/2455 - Query execution
• G06F 9/48 - Program initiatingProgram switching, e.g. by interrupt
• G06F 16/2453 - Query optimisation

Abstract

A system for monitoring job execution includes an interface and a processor. The interface is configured to receive an indication to start a cluster processing job. The processor is configured to determine whether processing a data instance associated with the cluster processing job satisfies a watchdog criterion; and in the event that processing the data instance satisfies the watchdog criterion, cause the processing of the data instance to be killed.

IPC Classes  ?

• G06F 11/07 - Responding to the occurrence of a fault, e.g. fault tolerance
• G06F 11/30 - Monitoring
• G06F 11/34 - Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation

Goods & Services

(1) Software; downloadable software; downloadable software for big data processing and analytics; downloadable software for accessing, managing and connecting to data lakes, data warehouses, data assets, data files, data sources; downloadable software for application database integration; downloadable software for use in ETL (extract, transform, load) data processing; downloadable software for compiling, organizing, visualizing, sharing and analyzing business intelligence; desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; downloadable software featuring libraries for data science training; downloadable software featuring libraries for data analytics; downloadable software featuring libraries for machine learning training; downloadable software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; downloadable software featuring artificial intelligence and machine learning technology; downloadable software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; downloadable software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; downloadable software for modifying and enabling transmission of images, audio, audio visual and video content and data; downloadable software for processing images, graphics, audio, video, and text; downloadable software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; downloadable software for use in data centers and data storage; downloadable software for use as an application programming interface (API); downloadable software development tools; downloadable software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; downloadable software development tools for building, designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; downloadable software development kits (SDKs); downloadable software for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; downloadable software for use in data governance, namely, software featuring artificial intelligence that allows users to define and manage policies for gathering, querying, storing, processing, sharing, accessing and disposing of data, create data clean rooms, and track data lineage; downloadable software for retrieval augmented generation (RAG); downloadable software for retrieval augmented generation (RAG) for use in generative AI applications; downloadable software for retrieval augmented generation (RAG) for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; downloadable podcasts in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence. (1) Database management; database management consultancy; business consulting in the field of artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; business data analysis; business data management consultancy; providing an online marketplace featuring downloadable software applications, data sets, data notebooks, artificial intelligence models, machine learning models; data management and processing, namely, business data analytics and interpretation, compiling and systemization of data into computer databases, data collection, management of data lakes, and administrative support services in the fields of data wrangling, data visualization, data governance, data science. (2) Educational services; conducting conferences, seminars, classes, workshops, courses, and webinars; training services; providing training for certification; educational services featuring podcasts; conducting conferences, seminars, classes, workshops, courses, and webinars in field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; training services in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing training for certification in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing non-downloadable electronic publications and on-line publication of journals or diaries [blog services] in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; educational services featuring podcasts in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing non-downloadable electronic publications and on-line publication of journals or diaries [blog services]. (3) Providing online non-downloadable software; providing online non-downloadable software for big data processing and analytics; providing online non-downloadable software for accessing, managing and connecting to data lakes, data warehouses, data assets, data files, data sources; providing online non-downloadable software for application database integration; providing online non-downloadable software for use in ETL (extract, transform, load) data processing; providing online non-downloadable software for compiling, organizing, visualizing, sharing and analyzing business intelligence; providing online non-downloadable desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; providing online non-downloadable software featuring libraries for data science training; providing online non-downloadable software featuring libraries for data analytics; providing online non-downloadable software featuring libraries for machine learning training; providing online non-downloadable software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing online non-downloadable software featuring artificial intelligence and machine learning technology; providing online non-downloadable software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing online non-downloadable software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; providing online non-downloadable software for modifying and enabling transmission of images, audio, audio visual and video content and data; providing online non-downloadable software for processing images, graphics, audio, video, and text; providing online non-downloadable software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; providing online non-downloadable software for use in data centers and data storage; providing online non-downloadable software for use as an application programming interface (API); providing online non-downloadable software development tools; providing online non-downloadable software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; providing online non-downloadable software development tools for building, designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing online non-downloadable software development kits (SDKs); software as a service (SAAS) services; software as a service (SAAS) services featuring software for big data processing and analytics; software as a service (SAAS) services, namely, featuring software for accessing, managing and connecting to data lakes, data warehouses, data assets, data files, data sources; software as a service (SAAS) services featuring software for application database integration; software as a service (SAAS) services featuring software for use in ETL (extract, transform, load) data processing; software as a service (SAAS) services featuring software for compiling, organizing, visualizing, sharing and analyzing business intelligence; software as a service (SAAS) services featuring desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; software as a service (SAAS) services featuring software featuring libraries for data science training; software as a service (SAAS) services featuring software featuring libraries for data analytics; software as a service (SAAS) services featuring software featuring libraries for machine learning training; software as a service (SAAS) services featuring software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; software as a service (SAAS) services featuring software featuring artificial intelligence and machine learning technology; software as a service (SAAS) services featuring software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; software as a service (SAAS) services featuring software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; software as a service (SAAS) services featuring software for modifying and enabling transmission of images, audio, audio visual and video content and data; software as a service (SAAS) services featuring software for processing images, graphics, audio, video, and text; software as a service (SAAS) services featuring software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; software as a service (SAAS) services featuring software for use in data centers and data storage; software as a service (SAAS) services featuring software for use as an application programming interface (API); software as a service (SAAS) services featuring software development tools; software as a service (SAAS) services featuring software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; software as a service (SAAS) services featuring software development tools for building, designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; software as a service (SAAS) services featuring software development kits (SDKs); platform as a services (PAAS) services; platform as a services (PAAS) services featuring software for big data processing and analytics; platform as a services (PAAS) services, namely, featuring software for accessing, managing and connecting to data lakes, data warehouses, data assets, data files, data sources; platform as a services (PAAS) services featuring software for use in ETL (extract, transform, load) data processing; platform as a services (PAAS) services featuring software for compiling, organizing, visualizing, sharing and analyzing business intelligence; platform as a services (PAAS) services featuring desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; platform as a services (PAAS) services featuring software featuring libraries for data science training; platform as a services (PAAS) services featuring software featuring libraries for data analytics; platform as a services (PAAS) services featuring software featuring libraries for machine learning training; platform as a services (PAAS) services featuring software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; platform as a services (PAAS) services featuring software featuring artificial intelligence and machine learning technology; platform as a services (PAAS) services featuring software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; platform as a services (PAAS) services featuring software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; platform as a services (PAAS) services featuring software for modifying and enabling transmission of images, audio, audio visual and video content and data; platform as a services (PAAS) services featuring software for processing images, graphics, audio, video, and text; platform as a services (PAAS) services featuring software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; platform as a services (PAAS) services featuring software for use in data centers and data storage; platform as a services (PAAS) services featuring software for use as an application programming interface (API); platform as a services (PAAS) services featuring software development tools; platform as a services (PAAS) services featuring software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; platform as a services (PAAS) services featuring software development tools for building, designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; platform as a services (PAAS) services featuring software development kits (SDKs); data mining; computer services, namely, hosting of search platforms on the Internet to allow users to index, integrate, warehouse, mine, process, share, collect, interpret, research, query, visualize, and analyze data; custom design and development of computer software; software engineering services for data processing; development and creation of computer programs for data processing and analysis; providing technological information in the fields of technology, computers, computer software, computer networks, web services, mobile computing and artificial intelligence via a website; technology consulting in the field of artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing of online non-downloadable software for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; providing online non-downloadable software featuring artificial intelligence for use in data governance, namely, software that allows users to define and manage policies for gathering, querying, storing, processing, sharing, accessing and disposing of data, create data clean rooms, and track data lineage; providing online non-downloadable software for retrieval augmented generation (RAG); providing online non-downloadable software for retrieval augmented generation (RAG) for use in generative AI applications; providing online non-downloadable software for retrieval augmented generation (RAG) for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; software as a service (SAAS) services, namely, featuring software for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance; providing online non-downloadable software for retrieval augmented generation (RAG) for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; platform as a services (PAAS) services, namely, featuring software for data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance; providing online non-downloadable software for retrieval augmented generation (RAG) for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; application service provider services, namely, hosting, managing, developing and maintaining applications, and software of others in the fields of big data processing and analytics, data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science; computer services, namely, hosting of artificial intelligence models, machine learning models, and large language models (LLMs); computer services, namely, hosting of artificial intelligence models, machine learning models, and large language models (LLMs) to allow users to perform search queries and develop inference; technology consulting in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, and building, design and management of data lakes; data migration; data mining; data warehousing; development and design of data lakes.

Goods & Services

Downloadable software for big data processing and analysis; downloadable software for accessing, managing and connecting to data lakes, data warehouses, data assets, data files; downloadable software for application database integration; downloadable software for use in ETL (extract, transform, load) data processing; downloadable software for compiling, organizing, visualizing, sharing and analyzing business intelligence data; desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; downloadable software featuring libraries for data science training; downloadable software featuring libraries for data analytics; downloadable software featuring libraries for machine learning training; downloadable software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; downloadable software featuring artificial intelligence and machine learning technology for deep learning, high performance computing, distributed computing, virtualization, natural language generation, statistical learning, supervised learning, un-supervised learning, data mining, predictive analytics and business intelligence; downloadable software for development and implementation of artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; downloadable software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; downloadable software for modifying and enabling transmission of images, audio, audio visual and video content and data; downloadable software for processing images, graphics, audio, video, and text; downloadable software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; downloadable software for use in operating data centers and data storage; downloadable software for use as an application programming interface (API); downloadable computer software development tools; downloadable software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; downloadable software development tools for building designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; downloadable software development kits (SDKs); downloadable software for use in performing data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; downloadable software for use in data governance, namely, software featuring artificial intelligence that allows users to define and manage policies for gathering, querying, storing, processing, sharing, accessing and disposing of data, create data clean rooms, and track data lineage; downloadable software for retrieval augmented generation (RAG); downloadable software for retrieval augmented generation (RAG) for use in generative AI applications; downloadable software for retrieval augmented generation (RAG) for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; downloadable podcasts in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence Database management; database management consultancy for business purposes; business consulting in the field of artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; business data analysis; business data consultancy; providing an online marketplace featuring downloadable software applications, data sets, data notebooks, artificial intelligence models, machine learning models; business data management and processing, namely, data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, and building, design and management of data lakes Conducting conferences, seminars, classes workshops, courses, and webinars in field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; training services in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing training for certification in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; non-downloadable publications and blogs in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; non-downloadable podcasts in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence providing online non-downloadable software for big data processing and analysis; providing online non-downloadable software for accessing, managing and connecting to data lakes, data warehouses, data assets, data files, data sources; providing online non-downloadable software for application database integration; providing online non-downloadable software for use in ETL (extract, transform, load) data processing; providing online non-downloadable software for compiling, organizing, visualizing, sharing and analyzing business intelligence data; providing online non-downloadable desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; providing online non-downloadable software featuring libraries for data science training; providing online non-downloadable software featuring libraries for data analytics; providing online non-downloadable software featuring libraries for machine learning training; providing online non-downloadable software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, , predictive analytics and business intelligence; providing online non-downloadable software featuring artificial intelligence and machine learning technology; providing online non-downloadable software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing online non-downloadable software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; providing online non-downloadable software for modifying and enabling transmission of images, audio, audio visual and video content and data; providing online non-downloadable software for processing images, graphics, audio, video, and text; providing online non-downloadable software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; providing online non-downloadable software for use in data centers and data storage; providing online non-downloadable software for use as an application programming interface (API); providing online non-downloadable software development tools; providing online non-downloadable software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; providing online non-downloadable software development tools for building designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing online non-downloadable software development kits (SDKs); software as a service (SAAS) services featuring software for big data processing and analytics; software as a service (SAAS) services, namely, featuring software for accessing, managing and connecting to data lakes, data warehouses, data assets, data files, data sources; software as a service (SAAS) services featuring software for application database integration; software as a service (SAAS) services featuring software for use in ETL (extract, transform, load) data processing; software as a service (SAAS) services featuring software for compiling, organizing, visualizing, sharing and analyzing business intelligence; software as a service (SAAS) services featuring desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; software as a service (SAAS) services featuring software featuring libraries for data science training; software as a service (SAAS) services featuring software featuring libraries for data analytics; software as a service (SAAS) services featuring software featuring libraries for machine learning training; software as a service (SAAS) services featuring software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; software as a service (SAAS) services featuring software featuring artificial intelligence and machine learning technology for deep learning, high performance computing, distributed computing, virtualization, natural language generation, statistical learning, supervised learning, un-supervised learning, data mining, predictive analytics and business intelligence; software as a service (SAAS) services featuring software for development and implementation of artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; software as a service (SAAS) services featuring software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; software as a service (SAAS) services featuring software for modifying and enabling transmission of images, audio, audio visual and video content and data; software as a service (SAAS) services featuring software for processing images, graphics, audio, video, and text; software as a service (SAAS) services featuring software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; software as a service (SAAS) services featuring software for use in data centers and electronic data storage; software as a service (SAAS) services featuring software for use as an application programming interface (API); software as a service (SAAS) services featuring software development tools; software as a service (SAAS) services featuring software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; software as a service (SAAS) services featuring software development tools for building designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; software as a service (SAAS) services featuring software development kits (SDKs); platform as a services (PAAS) services; platform as a services (PAAS) services featuring software for big data processing and analysis; platform as a services (PAAS) services, featuring software for accessing, managing and connecting to data lakes, data warehouses, data assets, data files, data sources; platform as a services (PAAS) services featuring software for use in ETL (extract, transform, load) data processing; platform as a services (PAAS) services featuring software for compiling, organizing, visualizing, sharing and analyzing business intelligence; platform as a services (PAAS) services featuring desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; platform as a services (PAAS) services featuring software featuring libraries for data science training; platform as a services (PAAS) services featuring software featuring libraries for data analytics; platform as a services (PAAS) services featuring software featuring libraries for machine learning training; platform as a services (PAAS) services featuring software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; platform as a services (PAAS) services featuring software featuring artificial intelligence and machine learning technology for deep learning, high performance computing, distributed computing, virtualization, natural language generation, statistical learning, supervised learning, un-supervised learning, data mining, predictive analytics and business intelligence; platform as a services (PAAS) services featuring software for development and implementation of artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; platform as a services (PAAS) services featuring software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; platform as a services (PAAS) services featuring software for modifying and enabling transmission of images, audio, audio visual and video content and data; platform as a services (PAAS) services featuring software for processing images, graphics, audio, video, and text; platform as a services (PAAS) services featuring software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; platform as a services (PAAS) services featuring software for use in data centers and data storage; platform as a services (PAAS) services featuring software for use as an application programming interface (API); platform as a services (PAAS) services featuring software development tools; platform as a services (PAAS) services featuring software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; platform as a services (PAAS) services featuring software development tools for building designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; platform as a services (PAAS) services featuring software development kits (SDKs); data mining; computer services, namely, hosting of search platforms on the Internet to allow users to index, integrate, warehouse, mine, process, share, collect, interpret, research, query, visualize, and analyze data; custom design and development of computer software; software engineering services for data processing; development and creation of computer programs for data processing and analysis; providing a website featuring information in the fields of technology, computers, computer software, computer networks, web services, mobile computing and artificial intelligence; technology consulting in the field of artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, un-supervised learning, predictive analytics and business intelligence; providing of online non-downloadable software for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; providing online non-downloadable software featuring artificial intelligence for use in data governance, namely, software that allows users to define and manage policies for gathering, querying, storing, processing, sharing, accessing and disposing of data, create data clean rooms, and track data lineage; providing online non-downloadable software for retrieval augmented generation (RAG); providing online non-downloadable software for retrieval augmented generation (RAG) for use in generative AI applications; providing online non-downloadable software for retrieval augmented generation (RAG) for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; software as a service (SAAS) services, namely, featuring software use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance; providing online non-downloadable software for retrieval augmented generation (RAG) for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; platform as a services (PAAS) services, namely, featuring software for data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance; • providing online non-downloadable software for retrieval augmented generation (RAG) for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; application service provider services, namely, hosting, managing, developing and maintaining applications, and software of others in the fields of big data processing and analytics, data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science; computer services, namely, hosting of artificial intelligence models, machine learning models, and large language models (LLMs); computer services, namely, hosting of artificial intelligence models, machine learning models, and large language models (LLMs) to allow users to perform search queries and develop inference; technology consulting in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, and building, design and management of data lakes

Abstract

A cluster computing system maintains a first set of queues for short queries and a set second set for longer queries. The first set is allocated a majority of the cluster's processing resources and processes queries on a first in first out basis. The second set is allocated a minority of the cluster's processing resources which are shared among queries in the second set. Accordingly, the system assigns each query to the first set of queues for a fixed amount of resource time. While a query is processing, the system monitors the query's resource time and reassigns the query to the second set of queues if the query has not completed within the allotted amount of resource time. Thus, short queries receive the necessary resources to complete quickly without getting stuck behind longer queries while ensuring that longer queries continue to make progress.

IPC Classes  ?

• G06F 16/2453 - Query optimisation
• G06F 9/48 - Program initiatingProgram switching, e.g. by interrupt
• G06F 11/34 - Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation
• G06F 16/28 - Databases characterised by their database models, e.g. relational or object models

Abstract

A system for retrieving and caching metadata from a remote data source is described. The system may receive a request from a client device. The request is to perform a query operation on a set of data objects stored in the remote data source. The system may access a metadata cache storing metadata information on one or more data objects of the remote data source and identify metadata corresponding to the set of data objects for the query operation in the metadata cache. The system may determine whether the identified metadata for the set of data objects meets an update condition. In response to the identified metadata meeting the update condition, the system may fetch updated metadata for at least the set of data objects from the remote data source, and store the updated metadata in the metadata cache.

IPC Classes  ?

• G06F 16/23 - Updating
• G06F 16/2455 - Query execution

Abstract

A system includes an interface and a processor. The interface is configured to receive a table indication of a data table and to receive a transaction indication to perform a transaction. The processor is configured to determine a current position N in a transaction log; determine a current state of the metadata; determine a read set associated with a transaction; attempt to write an update to the transaction log associated with a next position N+1; in response to a transaction determination that a simultaneous transaction associated with the next position N+1 already exists, determine a set of updated files; and in response to a determination that there is not an overlap between the read set associated with the current transaction and the set of updated files associated with the simultaneous transaction, attempt to write the update to the transaction to the transaction log associated with a further position N+2.

IPC Classes  ?

• G06F 16/23 - Updating
• G06F 16/14 - Details of searching files based on file metadata
• G06F 16/22 - IndexingData structures thereforStorage structures

Abstract

Disclosed herein is a method, system, or non-transitory computer readable medium for evaluating a query on a columnar dataset comprising one or more dictionaries associated with columns in the dataset. The method includes receiving a request to perform a query comprising at least an operator for a columnar dataset on cloud storage. At least one column in the dataset is based on a dictionary, and the dictionary maps one or more values for a column to one or more respective identifiers. The method evaluates the operator on one or more values of the dictionary to generate an updated dictionary comprising updated values. The method may decode the updated dictionary into an updated column comprising updated data values.

IPC Classes  ?

• G06F 16/2455 - Query execution
• G06F 11/34 - Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation
• G06F 16/22 - IndexingData structures thereforStorage structures

Abstract

The disclosed configurations provide a method (and/or a computer-readable medium or system) for determining, from a table schema describing keys of a data table, one or more clustering keys that can be used to cluster data files of a data table. The method includes generating features for the data table, generating tokens from the features, generating a prediction for each token by applying to the token a machine-learned transformer model trained to predict a likelihood that the key associated with the token is a clustering key for the data table, determining clustering keys based on the predictions, and clustering data records of the data table into data files based on key-values for the clustering keys.

IPC Classes  ?

• G06F 16/28 - Databases characterised by their database models, e.g. relational or object models
• G06F 16/21 - Design, administration or maintenance of databases
• G06F 16/22 - IndexingData structures thereforStorage structures

Abstract

A system for multipass sort with subsplitting includes a communication interface and a processor. The communication interface is configured to receive from a client device a request to sort a dataset that includes a plurality of rows, where the size of the dataset is greater than a threshold size. The processor is configured to: subdivide the dataset into a plurality of data subsets; sort each of the plurality of data subsets; merge the plurality of sorted data subsets utilizing a binary merge tree to generate a sorted dataset; and provide the sorted dataset to the client device.

IPC Classes  ?

• G06F 16/22 - IndexingData structures thereforStorage structures
• G06F 16/2453 - Query optimisation

Abstract

A data processing service receives indication that a recipient will request access to data assets of a provider and provides a request for credentials from a recipient governance module. The recipient governance module stores a recipient metastore including an object for a provider metastore. In response to determining that the assets are associated with the provider metastore, the service provides a request for credentials to a provider governance module. The provider governance module stores the provider metastore describing data assets of the provider and permissions for accessing data assets. The provider metastore includes a recipient object attached to the data assets with an identifier for the recipient metastore. In response to verifying that the recipient was provided access to the data assets, the service provides a token to the recipient governance module. The service then provides the token to a computing resource to provide access to the data assets.

IPC Classes  ?

• G06F 21/31 - User authentication

Abstract

The present application discloses a method, system, and computer system for providing access to data. The method includes receiving, by a data manager service from a data requesting service, a request using an identifier for a high-level data object to access a set of data associated with the high-level data object, determining, by the data manager service, low-level data object(s) corresponding to the set of data based on the identifier for the high-level data object, determining whether a user associated with the request has permission to access at least a subset of the low-level data object(s), and in response to determining that the user associated has permission to access the at least the subset of the low-level data object(s), generating, by the data manager service, a uniform resource locator (URL) via which the at least the subset of the one or more low-level data objects is accessible by the user.

IPC Classes  ?

• G06F 21/62 - Protecting access to data via a platform, e.g. using keys or access control rules
• G06F 21/60 - Protecting data

Abstract

Disclosed is a configuration for managing the organization of data tables in cloud-based storage. The configuration receives metrics for data processing operations on the data table. Metrics include at least one of a size of the data table, a size of each file in the data table, and metadata describing the data table. The configuration automatically executes a cost-benefit analysis based on the one or more metrics for each candidate maintenance operation in a plurality of candidate maintenance operations. The configuration automatically selects a maintenance operation from the candidate maintenance operations to automate based on the cost-benefit analysis of the one or more candidate maintenance operations. The selected maintenance operation is automated and scheduled on the data table.

IPC Classes  ?

• G06F 16/21 - Design, administration or maintenance of databases
• G06F 11/34 - Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation
• G06F 16/22 - IndexingData structures thereforStorage structures

Goods & Services

Software; downloadable software; downloadable software for big data processing and analytics; downloadable software for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; downloadable software for accessing, managing and connecting to data lakes, data warehouses, data assets, data files, data sources; downloadable software for application database integration; downloadable software for use in ETL (extract, transform, load) data processing; downloadable software for compiling, organizing, visualizing, sharing and analyzing business intelligence; downloadable software for use in data governance, namely, software featuring artificial intelligence that allows users to define and manage policies for gathering, querying, storing, processing, sharing, accessing and disposing of data, create data clean rooms, and track data lineage; desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; downloadable software featuring libraries for data science training; downloadable software featuring libraries for data analytics; downloadable software featuring libraries for machine learning training; downloadable software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, unsupervised learning, predictive analytics and business intelligence; downloadable software featuring artificial intelligence and machine learning technology; downloadable software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, unsupervised learning, predictive analytics and business intelligence; downloadable software for retrieval augmented generation (RAG); downloadable software for retrieval augmented generation (RAG) for use in generative AI applications; downloadable software for retrieval augmented generation (RAG) for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; downloadable software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; downloadable software for modifying and enabling transmission of images, audio, audio visual and video content and data; downloadable software for processing images, graphics, audio, video, and text; downloadable software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; downloadable software for use in data centers and data storage; downloadable software for use as an application programming interface (API); downloadable software development tools; downloadable software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; downloadable software development tools for building, designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, unsupervised learning, predictive analytics and business intelligence; downloadable software development kits (SDKs); downloadable podcasts in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, unsupervised learning, predictive analytics and business intelligence. Database management; Database management consultancy; data management and processing, namely, data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, and building, design and management of data lakes; business consulting in the field of artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, unsupervised learning, predictive analytics and business intelligence; Business data analysis; Business data consultancy; providing an online marketplace featuring downloadable software applications, data sets, data notebooks, artificial intelligence models, machine learning models. Educational services; Conducting conferences, seminars, classes workshops, courses, and webinars; Conducting conferences, seminars, classes workshops, courses, and webinars in field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, unsupervised learning, predictive analytics and business intelligence; Training services; Training services in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, unsupervised learning, predictive analytics and business intelligence; Providing training for certification; Providing training for certification in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, unsupervised learning, predictive analytics and business intelligence; Non-downloadable publications and blogs; Non-downloadable publications and blogs in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, unsupervised learning, predictive analytics and business intelligence; Non-downloadable podcasts; Non-downloadable podcasts in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science, artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, unsupervised learning, predictive analytics and business intelligence. providing online non-downloadable software; providing online non-downloadable software for big data processing and analytics; providing of online non-downloadable software for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; providing online non-downloadable software for accessing, managing and connecting to data lakes, data warehouses, data assets, data files, data sources; providing online non-downloadable software for application database integration; providing online non-downloadable software for use in ETL (extract, transform, load) data processing; providing online non-downloadable software for compiling, organizing, visualizing, sharing and analyzing business intelligence; providing online non-downloadable software featuring artificial intelligence for use in data governance, namely, software that allows users to define and manage policies for gathering, querying, storing, processing, sharing, accessing and disposing of data, create data clean rooms, and track data lineage; providing online non-downloadable desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; providing online non-downloadable software featuring libraries for data science training; providing online non-downloadable software featuring libraries for data analytics; providing online non-downloadable software featuring libraries for machine learning training; providing online non-downloadable software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, unsupervised learning, predictive analytics and business intelligence; providing online non-downloadable software featuring artificial intelligence and machine learning technology; providing online non-downloadable software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, unsupervised learning, predictive analytics and business intelligence; providing online non-downloadable software for retrieval augmented generation (RAG); providing online non-downloadable software for retrieval augmented generation (RAG) for use in generative AI applications; providing online non-downloadable software for retrieval augmented generation (RAG) for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; providing online non-downloadable software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; providing online non-downloadable software for modifying and enabling transmission of images, audio, audio visual and video content and data; providing online non-downloadable software for processing images, graphics, audio, video, and text; providing online non-downloadable software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; providing online non-downloadable software for use in data centers and data storage; providing online non-downloadable software for use as an application programming interface (API); providing online non-downloadable software development tools; providing online non-downloadable software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; providing online non-downloadable software development tools for building, designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, unsupervised learning, predictive analytics and business intelligence; providing online non-downloadable software development kits (SDKs); software as a service (SAAS) services; software as a service (SAAS) services featuring software for big data processing and analytics; software as a service (SAAS) services, namely, featuring software use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; software as a service (SAAS) services, namely, featuring software for accessing, managing and connecting to data lakes, data warehouses, data assets, data files, data sources; software as a service (SAAS) services featuring software for application database integration; software as a service (SAAS) services featuring software for use in ETL (extract, transform, load) data processing; software as a service (SAAS) services featuring software for compiling, organizing, visualizing, sharing and analyzing business intelligence; software as a service (SAAS) services featuring software featuring artificial intelligence for use in data governance, namely, software that allows users to define and manage policies for gathering, querying, storing, processing, sharing, accessing and disposing of data, create data clean rooms, and track data lineage; software as a service (SAAS) services featuring desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; software as a service (SAAS) services featuring software featuring libraries for data science training; software as a service (SAAS) services featuring software featuring libraries for data analytics; software as a service (SAAS) services featuring software featuring libraries for machine learning training; software as a service (SAAS) services featuring software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, unsupervised learning, , predictive analytics and business intelligence; software as a service (SAAS) services featuring software featuring artificial intelligence and machine learning technology; software as a service (SAAS) services featuring software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, unsupervised learning, predictive analytics and business intelligence; software as a service (SAAS) services featuring software for retrieval augmented generation (RAG); software as a service (SAAS) services featuring software for retrieval augmented generation (RAG) for use in generative AI applications; software as a service (SAAS) services featuring software for retrieval augmented generation (RAG) for use in data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; software as a service (SAAS) services featuring software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; software as a service (SAAS) services featuring software for modifying and enabling transmission of images, audio, audio visual and video content and data; software as a service (SAAS) services featuring software for processing images, graphics, audio, video, and text; software as a service (SAAS) services featuring software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; software as a service (SAAS) services featuring software for use in data centers and data storage; software as a service (SAAS) services featuring software for use as an application programming interface (API); software as a service (SAAS) services featuring software development tools; software as a service (SAAS) services featuring software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; software as a service (SAAS) services featuring software development tools for building, designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, unsupervised learning, predictive analytics and business intelligence; software as a service (SAAS) services featuring software development kits (SDKs); platform as a services (PAAS) services; platform as a services (PAAS) services featuring software for big data processing and analytics; platform as a services (PAAS) services, namely, featuring software for data analytics, data migration, data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data governance, data science; platform as a services (PAAS) services, namely, featuring software for accessing, managing and connecting to data lakes, data warehouses, data assets, data files, data sources; platform as a services (PAAS) services featuring software for use in ETL (extract, transform, load) data processing; platform as a services (PAAS) services featuring software for compiling, organizing, visualizing, sharing and analyzing business intelligence; platform as a services (PAAS) services featuring desktop and mobile computing and operating platforms consisting of data transceivers, wireless networks and gateways, for collection, analysis, sharing, interpretation and management of data; platform as a services (PAAS) services featuring software featuring libraries for data science training; platform as a services (PAAS) services featuring software featuring libraries for data analytics; platform as a services (PAAS) services featuring software featuring libraries for machine learning training; platform as a services (PAAS) services featuring software featuring libraries for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, unsupervised learning, , predictive analytics and business intelligence; platform as a services (PAAS) services featuring software featuring artificial intelligence and machine learning technology; platform as a services (PAAS) services featuring software for artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, unsupervised learning, predictive analytics and business intelligence; platform as a services (PAAS) services featuring software for collecting, managing, editing, organizing, modifying, transmitting, sharing, and storing of data; platform as a services (PAAS) services featuring software for modifying and enabling transmission of images, audio, audio visual and video content and data; platform as a services (PAAS) services featuring software for processing images, graphics, audio, video, and text; platform as a services (PAAS) services featuring software for transmitting, sharing, receiving, downloading, displaying, interacting with and transferring content, text, visual works, audio works, audiovisual works, literary works, data, files, documents and electronic works; platform as a services (PAAS) services featuring software for use in data centers and data storage; platform as a services (PAAS) services featuring software for use as an application programming interface (API); platform as a services (PAAS) services featuring software development tools; platform as a services (PAAS) services featuring software development tools for building, designing, deploying, and monitoring applications for use in data integration, data warehousing, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, and data analytics; platform as a services (PAAS) services featuring software development tools for building, designing, deploying, and monitoring applications featuring artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, unsupervised learning, predictive analytics and business intelligence; platform as a services (PAAS) services featuring software development kits (SDKs); Data mining; computer services, namely, hosting of search platforms on the Internet to allow users to index, integrate, warehouse, mine, process, share, collect, interpret, research, query, visualize, and analyze data; computer services, namely, hosting of artificial intelligence models, machine learning models, and large language models (LLMs); computer services, namely, hosting of artificial intelligence models, machine learning models, and large language models (LLMs) to allow users to perform search queries and develop inference; application service provider services, namely, hosting, managing, developing and maintaining applications, and software of others in the fields of big data processing and analytics, data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science; custom design and development of computer software; Software engineering services for data processing; development and creation of computer programs for data processing and analysis; Providing a website featuring information in the fields of technology, computers, computer software, computer networks, web services, mobile computing and artificial intelligence; Technology consulting in the field of data analytics, data migration, data importing, data wrangling, data mining, data processing, data sharing, data collection, data interpretation, data queries, data visualization, data integration, data warehousing, data processing, data governance, data science and building, design and management of data lakes; Technology consulting in the field of artificial intelligence, machine learning, deep learning, large language models (LLMs), natural language generation, statistical learning, supervised learning, unsupervised learning, predictive analytics and business intelligence.

Abstract

A method for evaluating textual output of one or more machine-learned language models is presented. The method includes receiving, from a user of a client device, a first prompt for input to one or more machine-learned language models, providing the first prompt to the one or more models for execution, and receiving a set of generated responses to the first prompt from the one or more models. The method further includes generating a user interface (UI) on the client device displaying the first prompt and generated responses as a table user interface element. The method applies a selected evaluation function to the generated response to evaluate the response with respect to an evaluation objective and identifies words that influence the evaluation. The method generates one or more UI elements on the UI to display the results of the evaluation for the generated responses.

IPC Classes  ?

• G06F 40/40 - Processing or translation of natural language
• G06F 40/103 - Formatting, i.e. changing of presentation of documents
• G06F 40/30 - Semantic analysis

Abstract

A disclosed configuration receives a first indication that a first transaction is committed to update a first subset of records in a data table at a first version to generate a second version of the data table and receiving a second indication to commit a second transaction to update a second subset of records in a data file of the data table at the first version. The configuration determines a logical prerequisite based on whether the first subset of records changes content of one or more records in the second subset of records and determining a physical prerequisite on whether the second subset of records corresponds to respective data records in data files of the second version of the data table. The configuration commits the second transaction to generate a third version of the data table by updating elements of the deletion vector if the prerequisites are satisfied.

IPC Classes  ?

• G06F 16/23 - Updating

Abstract

A data processing service facilitates the creation and processing of data processing pipelines that process data processing jobs defined with respect to a set of tasks in a sequence and with data dependencies associated with each separate task such that the output from one task is used as input for a subsequent task. In various embodiments, the set of tasks include at least one cleanroom task that is executed in a cleanroom station and at least one non-cleanroom task executed in an execution environment of a user where each task is configured to read one or more input datasets and transform the one or more input datasets into one or more output datasets.

IPC Classes  ?

• G06F 21/00 - Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
• G06F 21/62 - Protecting access to data via a platform, e.g. using keys or access control rules

Abstract

A resource management configuration may receive an API request from an API server. The API request specifies task information from a plurality of tenants. The configuration transmits status information of a plurality of VMs to the API server to assign tasks to one or more VMs based on the task information and the status information. Tasks assigned to a VM of the plurality of VMs are for one tenant of the plurality of tenants. The configuration configures on an untrusted network, network security groups for managing communications of tenants such that a network security group configured for a tenant permits communications between VMs assigned to the same tenant but prevents communications between VMs assigned to different tenants. The configuration pins each assigned VM of the one or more assigned VMs to perform the task based on the task information of the corresponding tenant.

IPC Classes  ?

• G06F 9/455 - EmulationInterpretationSoftware simulation, e.g. virtualisation or emulation of application or operating system execution engines
• G06F 9/54 - Interprogram communication

Abstract

A system for executing a streaming query includes an interface and a processor. The interface is configured to receive a logical query plan. The processor is configured to determine a physical query plan based at least in part on the logical query plan. The physical query plan comprises an ordered set of operators. Each operator of the ordered set of operators comprises an operator input mode and an operator output mode. The processor is further configured to execute the physical query plan using the operator input mode and the operator output mode for each operator of the query.

IPC Classes  ?

• G06F 16/2453 - Query optimisation
• G06F 16/2455 - Query execution

Abstract

A system for clustering data into corresponding files comprises one or more processors and a memory. The one or more processors is/are configured to: 1) determine to cluster a set of data into a set of files; 2) determine a set of split points in a corresponding set of dimensions of the set of data to determine the set of files, wherein each file of the set of files has an approximate target size; and 3) store one or more items of the set of data into a corresponding file of the set of files based at least in part on the set of split points. The memory is coupled to the one or more processors and configured to provide the processor with instructions.

IPC Classes  ?

• G06F 16/22 - IndexingData structures thereforStorage structures
• G06F 16/27 - Replication, distribution or synchronisation of data between databases or within a distributed database systemDistributed database system architectures therefor
• G06F 16/28 - Databases characterised by their database models, e.g. relational or object models

Abstract

The present application discloses a method, system, and computer system for starting up and maintaining a cluster in a warmed up state, and/or allocating clusters from a warmed up state. The method includes instantiating a set of virtual machines, wherein instantiating the set of virtual machines includes setting a temporary security credential for each virtual machine of the set of virtual machines, receiving a virtual machine allocation request associated with a workspace, a customer, or a tenant, in response to the virtual machine allocation request: allocating a virtual machine, wherein allocating the virtual machine comprises replacing the temporary security credential with a security credential associated with the workspace, the customer, or the tenant.

IPC Classes  ?

• G06F 9/50 - Allocation of resources, e.g. of the central processing unit [CPU]
• G06F 21/45 - Structures or tools for the administration of authentication

Abstract

The present application discloses a method, system, and computer system for managing lineage data for data entities. The method includes generating lineage data, wherein generating the lineage data, and storing and indexing, in a data structure, the lineage data in association with the selected data entity. The generating the lineage data includes selecting a selected data entity, obtaining a query tree that was used to generate the selected data entity, and determining lineage data for the selected data entity based at least in part on the query tree.

IPC Classes  ?

• G06F 16/28 - Databases characterised by their database models, e.g. relational or object models
• G06F 11/07 - Responding to the occurrence of a fault, e.g. fault tolerance
• G06F 16/215 - Improving data qualityData cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• G06F 16/22 - IndexingData structures thereforStorage structures
• G06F 16/23 - Updating
• G06F 16/906 - ClusteringClassification
• G06F 17/18 - Complex mathematical operations for evaluating statistical data

Abstract

The present application discloses a method, system, and computer system for building a model associated with a dataset. The method includes receiving a data set, the dataset comprising a plurality of keys and a plurality of key-value relationships, determining a plurality of models to build based at least in part on the dataset, wherein determining the plurality of models to build comprises using the dataset format information to identify the plurality of models, building the plurality of models, and optimizing at least one of the plurality of models.

IPC Classes  ?

• G06N 20/00 - Machine learning
• G06F 18/20 - Analysing
• G06F 18/2132 - Feature extraction, e.g. by transforming the feature spaceSummarisationMappings, e.g. subspace methods based on discrimination criteria, e.g. discriminant analysis

Abstract

A data processing service performs a rebalancing process for rebalancing stateful tasks on a cluster computing system. In one instance, the method for rebalancing stateful tasks is performed such that the per-operator partitions are spread across available executors of a cluster of the cluster computing system with respect to one or more statistics of the tasks. In one instance, the method for rebalancing stateful tasks is also performed such that the total number of stateful tasks are balanced per executor as long as this rebalancing does not imbalance the per-operator placements. In this way, the processing of stateful tasks can be spread across multiple executors in a relatively uniform manner, even though there may be an upfront cost of breaking the local caching on an executor.

IPC Classes  ?

• G06F 16/27 - Replication, distribution or synchronisation of data between databases or within a distributed database systemDistributed database system architectures therefor
• G06F 16/2455 - Query execution

Abstract

A system performs efficient startup of executors of a distributed computing engine used for processing queries, for example, database queries. The system starts an executor node and processes a set of queries using the executor node to warm up the executor node. The system performs a checkpoint of the warmed-up executor node to create an image. The image is restored in the target executor nodes. The system may store a checkpoint image for each configuration of an executor node. The configuration is determined based on various factors including the hardware of the executor node, memory allocation of the processes, and so on. The user or restore based on checkpoint images improves efficiency of execution of the startup of executor nodes.

IPC Classes  ?

• G06F 16/2453 - Query optimisation

Abstract

The disclosed configurations provide a method (and/or a computer-readable medium or system) for determining, from a table schema describing keys of a data table, one or more clustering keys that can be used to cluster data files of a data table. The method includes generating features for the data table, generating tokens from the features, generating a prediction for each token by applying to the token a machine-learned transformer model trained to predict a likelihood that the key associated with the token is a clustering key for the data table, determining clustering keys based on the predictions, and clustering data records of the data table into data files based on key-values for the clustering keys.

IPC Classes  ?

• G06F 16/28 - Databases characterised by their database models, e.g. relational or object models
• G06F 16/21 - Design, administration or maintenance of databases
• G06F 16/22 - IndexingData structures thereforStorage structures

Abstract

A device for using message identifiers for Publish/subscribe messaging deduplication is described. The system may fetch one or more sets of data records from a data source, and each data record is associated with a message identifier. The system may store the one or more sets of data records in a data file, which is associated with a metadata comprising the message identifier, a file path and a row number for each data record. The system may determine whether one or more of the data records are duplicated based on the associated message identifiers. In response to determining that the one or more data records are duplicated, the system may generate a second metadata comprising the file paths and row numbers associated with the duplicated data records.

IPC Classes  ?

• G06F 16/174 - Redundancy elimination performed by the file system
• G06F 16/14 - Details of searching files based on file metadata
• G06F 16/16 - File or folder operations, e.g. details of user interfaces specifically adapted to file systems

Abstract

A system includes an interface, a processor, and a memory. The interface is configured to receive a version of a model from a model registry. The processor is configured to store the version of the model, start a process running the version of the model, and update a proxy with version information associated with the version of the model, wherein the updated proxy indicates to redirect an indication to invoke the version of the model to the process. The memory is coupled to the processor and configured to provide the processor with instructions.

IPC Classes  ?

• G06F 16/00 - Information retrievalDatabase structures thereforFile system structures therefor
• G06F 16/21 - Design, administration or maintenance of databases
• G06F 16/955 - Retrieval from the web using information identifiers, e.g. uniform resource locators [URL]
• G06N 5/022 - Knowledge engineeringKnowledge acquisition

Abstract

A data processing service receives a request from a first collaborator to create a clean room for data sharing collaboration with at least a second collaborator. In response, the data processing service creates an execution environment separate from the data environment of the first collaborator and the second collaborator. The first and second collaborators can then add content into the clean room in the form of data tables and executable notebooks. Approval from each collaborator is required before a notebook can be executed using any data table shared into the clean room. Upon receiving notebook approval from each collaborator, the data processing service creates a notebook job to execute the notebook on one or more cluster computing resources of the data processing service to generate an output.

IPC Classes  ?

• G06F 16/00 - Information retrievalDatabase structures thereforFile system structures therefor
• G06F 16/21 - Design, administration or maintenance of databases

Abstract

A method, system, and computer system for performing an operation with respect to a target table are disclosed. The method includes performing first and second jobs, obtaining one or more other resulting files based at least in part on unmatched rows, and obtaining a set of processed files based at least in part on performing a post-processing operation with respect to the set of resulting files. The set of processed files has less files than the set of resulting files. Performing the first job includes determining a set of matching target table files and storing target table information indicating for each of the set of matching target table files, a particular set of rows having matching rows. Performing the second job includes performing a matching action based on matched rows and obtaining the second job resulting file(s).

IPC Classes  ?

• G06F 16/2453 - Query optimisation
• G06F 11/34 - Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation
• G06F 16/22 - IndexingData structures thereforStorage structures
• G06F 16/28 - Databases characterised by their database models, e.g. relational or object models

Abstract

A data processing service generates a data classifier tree for managing data files of a data table. The data classifier tree may be configured as a KD-classifier tree and includes a plurality of nodes and edges. A node of the data classifier tree may represent a splitting condition with respect to key-values for a respective key. A node of the data classifier tree may be associated with one or more data files assigned to the node. The data files assigned to the node each include a subset of records having key-values that satisfy the conditions represented by the node and parent nodes of the node. The data processing service may efficiently cluster the data in the data table while reducing the number of data files that are rewritten when data is modified or added to the data table.

IPC Classes  ?

• G06F 16/10 - File systemsFile servers
• G06F 16/13 - File access structures, e.g. distributed indices
• G06F 16/16 - File or folder operations, e.g. details of user interfaces specifically adapted to file systems
• G06F 16/22 - IndexingData structures thereforStorage structures
• G06F 16/28 - Databases characterised by their database models, e.g. relational or object models

Abstract

A data tree for managing data files of a data table and performing one or more transaction operations to the data table is described. The data tree is configured as a KD-epsilon tree and includes a plurality of nodes and edges. A node of the data tree may represent a splitting condition with respect to key-values for a respective key. A leaf node of the data tree may correspond to a data file for a data table that includes a subset of records having key-values that satisfy the condition for the node and conditions associated with parent nodes of the node. A parent node may correspond to a file including a buffer that stores changes to data files reachable by this parent node, and also includes dedicated storage to pointers of the child nodes. By using the data tree, the data processing system may efficiently cluster the data in the data table while reducing the number of data files that are rewritten.

IPC Classes  ?

• G06F 16/20 - Information retrievalDatabase structures thereforFile system structures therefor of structured data, e.g. relational data
• G06F 16/22 - IndexingData structures thereforStorage structures
• G06F 16/23 - Updating
• G06F 16/245 - Query processing
• G06F 16/2453 - Query optimisation
• G06F 16/28 - Databases characterised by their database models, e.g. relational or object models

Abstract

Disclosed herein provides enhancements for operating a data access application service executing on a data access server system and an external computing system. In the data access server system, a request is received from a client device executing at least one of multiple application services for a dataset from one or more of multiple storage systems. In the data access server system, a data retrieval instruction is generated for the client device to access the dataset from the one or more of the multiple storage systems. The data retrieval instruction comprises task descriptions and a temporary credential. The data retrieval instruction is transferred to the external computing system via the client device and the requested dataset is retrieved and deployed based on the task descriptions and the temporary credential from the one or more of the multiple storage systems.

IPC Classes  ?

• G06F 16/27 - Replication, distribution or synchronisation of data between databases or within a distributed database systemDistributed database system architectures therefor
• G06F 9/54 - Interprogram communication
• G06F 16/2455 - Query execution
• G06F 16/25 - Integrating or interfacing systems involving database management systems
• G06F 16/28 - Databases characterised by their database models, e.g. relational or object models

Abstract

A system for dataflow graph processing comprises a communication interface and a processor. The communication interface is configured receive an indication to generate a dataflow graph, wherein the indication includes a set of queries. The processor is coupled to the communication interface and is configured to: determine dependencies of each query in the set of queries on another query; determine a DAG of nodes based at least in part on the dependencies; insert a node in the DAG of nodes to generate an updated DAG to enforce an expectation; determine a dataflow graph based on the updated DAG; and provide the dataflow graph.

IPC Classes  ?

• G06F 16/901 - IndexingData structures thereforStorage structures
• G06F 16/215 - Improving data qualityData cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• G06F 16/22 - IndexingData structures thereforStorage structures
• G06F 16/245 - Query processing

Abstract

The present application discloses a method, system, and computer system for providing access to data. The method includes receiving, by a data manager service from a data requesting service, a request using an identifier for a high-level data object to access a set of data associated with the high-level data object, determining, by the data manager service, low-level data object(s) corresponding to the set of data based on the identifier for the high-level data object, determining whether a user associated with the request has permission to access at least a subset of the low-level data object(s), and in response to determining that the user associated has permission to access the at least the subset of the low-level data object(s), generating, by the data manager service, a uniform resource locator (URL) via which the at least the subset of the one or more low-level data objects is accessible by the user.

IPC Classes  ?

• G06F 21/62 - Protecting access to data via a platform, e.g. using keys or access control rules
• G06F 21/00 - Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
• G06F 21/60 - Protecting data

Abstract

A system performs training and execution of machine learning models that use on-demand features using feature functions. The system receives commands for registering metadata associated with a machine learning model. The machine learning model may process a set of features including on-demand features as well as other features such as batch features. The system executes the command by storing an association between the machine learning model and the feature functions associated with any on-demand features processed by the machine learning model. The feature functions are executed using an end point of a data asset service. The use of the data asset service for invoking the feature functions ensures that the same set of instructions is executed during model training and model inferencing, thereby avoiding model skew.

IPC Classes  ?

• G06N 20/00 - Machine learning

Abstract

The system is configured to: 1) receive a client request; 2) determine executor(s) to generate a response to the user request; 3) provide each of the executor(s) with an indication; 4) receive for each indication a response including an output of either a cloud output or an in-line output to generate a group of in-line outputs and a group of cloud outputs; 5) determine whether the group of in-line outputs comprises all outputs; and 6) in response to the group of in-line outputs not comprising all the outputs for the client request: a) convert the group of in-line outputs to a converted group of cloud outputs; b) generate metadata for the converted group of cloud outputs and the group of cloud outputs; and c) provide response to the client request including the metadata for the converted group of cloud outputs and the group of cloud outputs.

IPC Classes  ?

• G06F 16/2458 - Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
• G06F 11/34 - Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation
• G06F 16/242 - Query formulation
• G06F 16/25 - Integrating or interfacing systems involving database management systems

Abstract

A system includes a plurality of computing units. A first computing unit of the plurality of computing units comprises: a communication interface configured to receive an indication to roll up data in a data table; and a processor coupled to the communication interface and configured to: build a preaggregation hash table based at least in part on a set of columns and the data table by aggregating input rows of the data table; for each preaggregated hash table entry of the preaggregated hash table: provide the preaggregated hash table entry to a second computing unit of the plurality of computing units based at least in part on a distribution hash value; receive a set of received entries from computing units of the plurality of computing units; and build an aggregation hash table based at least in part on the set of received entries by aggregating the set of received entries.

IPC Classes  ?

• G06F 16/00 - Information retrievalDatabase structures thereforFile system structures therefor
• G06F 16/13 - File access structures, e.g. distributed indices
• G06F 16/22 - IndexingData structures thereforStorage structures
• G06F 16/242 - Query formulation
• G06F 16/2455 - Query execution
• G06F 16/28 - Databases characterised by their database models, e.g. relational or object models

Abstract

The present application discloses a method, system, and computer system for providing access to data. The method includes receiving, by a data manager service from a data requesting service, a request using an identifier for a high-level data object to access a set of data associated with the high-level data object, determining, by the data manager service, low-level data object(s) corresponding to the set of data based on the identifier for the high-level data object, determining whether a user associated with the request has permission to access at least a subset of the low-level data object(s), and in response to determining that the user associated has permission to access the at least the subset of the low-level data object(s), generating, by the data manager service, a uniform resource locator (URL) via which the at least the subset of the one or more low-level data objects is accessible by the user.

IPC Classes  ?

• G06F 21/62 - Protecting access to data via a platform, e.g. using keys or access control rules
• G06F 21/00 - Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
• G06F 21/60 - Protecting data

Abstract

Disclosed is a configuration for managing the organization of data tables in cloud-based storage. The configuration receives metrics for data processing operations on the data table. Metrics include at least one of a size of the data table, a size of each file in the data table, and metadata describing the data table. The configuration automatically executes a cost-benefit analysis based on the one or more metrics for each candidate maintenance operation in a plurality of candidate maintenance operations. The configuration automatically selects a maintenance operation from the candidate maintenance operations to automate based on the cost-benefit analysis of the one or more candidate maintenance operations. The selected maintenance operation is automated and scheduled on the data table.

IPC Classes  ?

• G06F 16/21 - Design, administration or maintenance of databases
• G06F 11/34 - Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation
• G06F 16/22 - IndexingData structures thereforStorage structures

Abstract

A cluster computing system maintains a first set of queues for short queries and a set second set for longer queries. The first set is allocated a majority of the cluster's processing resources and processes queries on a first in first out basis. The second set is allocated a minority of the cluster's processing resources which are shared among queries in the second set. Accordingly, the system assigns each query to the first set of queues for a fixed amount of resource time. While a query is processing, the system monitors the query's resource time and reassigns the query to the second set of queues if the query has not completed within the allotted amount of resource time. Thus, short queries receive the necessary resources to complete quickly without getting stuck behind longer queries while ensuring that longer queries continue to make progress.

IPC Classes  ?

• G06F 16/24 - Querying
• G06F 9/48 - Program initiatingProgram switching, e.g. by interrupt
• G06F 11/34 - Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation
• G06F 16/2453 - Query optimisation
• G06F 16/28 - Databases characterised by their database models, e.g. relational or object models

Abstract

A system for retrieving and caching metadata from a remote data source is described. The system may receive a request from a client device. The request is to perform a query operation on a set of data objects stored in the remote data source. The system may access a metadata cache storing metadata information on one or more data objects of the remote data source and identify metadata corresponding to the set of data objects for the query operation in the metadata cache. The system may determine whether the identified metadata for the set of data objects meets an update condition. In response to the identified metadata meeting the update condition, the system may fetch updated metadata for at least the set of data objects from the remote data source, and store the updated metadata in the metadata cache.

IPC Classes  ?

• G06F 16/00 - Information retrievalDatabase structures thereforFile system structures therefor
• G06F 16/23 - Updating
• G06F 16/2455 - Query execution

Abstract

A system for multipass sort includes a communication interface and a processor. The communication interface is configured to receive from a client device a request to sort a dataset that includes a plurality of rows. The processor is configured to perform a first sort pass on the dataset in part by: extracting prefixes associated with a first schema element associated with the dataset for the plurality of rows; and sorting the extracted prefixes utilizing an integer sort algorithm based on a sort order included in the request to sort the dataset, where sorting the extracted prefixes includes utilizing NULL values to resolve a tied range that includes at least two rows of the plurality of rows having a same extracted prefix.

IPC Classes  ?

• G06F 16/00 - Information retrievalDatabase structures thereforFile system structures therefor
• G06F 16/22 - IndexingData structures thereforStorage structures
• G06F 16/2455 - Query execution

Abstract

The interface is to receive an indication to execute an optimize command. The processor is to receive a file name; determine whether adding a file of the file name to a current bin causes the current bin to exceed a threshold; associate the file with the current bin in response to determining that adding the file does not cause the current bin to exceed the bin threshold; in response to determining that adding the file to the current bin causes the current bin to exceed the bin threshold: associate the file with a next bin, indicate that the current bin is closed, and add the current bin to a batch of bins; determine whether a measure of the batch of bins exceeds a batch threshold; and in response to determining that the measure exceeds the batch threshold, provide the batch of bins for processing.

IPC Classes  ?

• G06F 16/172 - Caching, prefetching or hoarding of files
• G06F 16/22 - IndexingData structures thereforStorage structures

Abstract

A data tree for managing data files of a data table and performing one or more transaction operations to the data table is described. The data tree is configured as a KD-epsilon tree and includes a plurality of nodes and edges. A node of the data tree may represent a splitting condition with respect to key-values for a respective key. A leaf node of the data tree may correspond to a data file for a data table that includes a subset of records having key-values that satisfy the condition for the node and conditions associated with parent nodes of the node. A parent node may correspond to a file including a buffer that stores changes to data files reachable by this parent node, and also includes dedicated storage to pointers of the child nodes. By using the data tree, the data processing system may efficiently cluster the data in the data table while reducing the number of data files that are rewritten.

IPC Classes  ?

• G06F 16/20 - Information retrievalDatabase structures thereforFile system structures therefor of structured data, e.g. relational data
• G06F 16/22 - IndexingData structures thereforStorage structures
• G06F 16/23 - Updating
• G06F 16/245 - Query processing
• G06F 16/28 - Databases characterised by their database models, e.g. relational or object models

Abstract

The present application discloses a method, system, and computer system for managing a data in a storage system. The method includes receiving a first transaction that modifies or deletes first data stored in a storage system, determining that the first data is subject to an intervening re-arrangement transaction, and in response to determining that the first data is subject to the intervening re-arrangement transaction, rolling back the re-arrangement transaction at least with respect to the first data and committing the first transaction.

IPC Classes  ?

• G06F 16/174 - Redundancy elimination performed by the file system

Abstract

A multi-cluster computing system which includes a query result caching system is presented. The multi-cluster computing system may include a data processing service and client devices communicatively coupled over a network. The data processing service may include a control layer and a data layer. The control layer may be configured to receive and process requests from the client devices and manage resources in the data layer. The data layer may be configured to include instances of clusters of computing resources for executing jobs. The data layer may include a data storage system, which further includes a remote query result cache Store. The query result cache store may include a cloud storage query result cache which stores data associated with results of previously executed requests. As such, when a cluster encounters a previously executed request, the cluster may efficiently retrieve the cached result of the request from the in-memory query result cache or the cloud storage query result cache.

IPC Classes  ?

• G06F 16/24 - Querying
• G06F 16/2453 - Query optimisation
• G06F 16/25 - Integrating or interfacing systems involving database management systems
• G06F 16/28 - Databases characterised by their database models, e.g. relational or object models

Abstract

A multi-cluster computing system which includes a query result caching system is presented. The multi-cluster computing system may include a data processing service and client devices communicatively coupled over a network. The data processing service may include a control layer and a data layer. The control layer may be configured to receive and process requests from the client devices and manage resources in the data layer. The data layer may be configured to include instances of clusters of computing resources for executing jobs. The data layer may include a data storage system, which further includes a remote query result cache Store. The query result cache store may include a cloud storage query result cache which stores data associated with results of previously executed requests. As such, when a cluster encounters a previously executed request, the cluster may efficiently retrieve the cached result of the request from the in-memory query result cache or the cloud storage query result cache.

IPC Classes  ?

• G06F 16/24 - Querying
• G06F 16/2453 - Query optimisation
• G06F 16/25 - Integrating or interfacing systems involving database management systems
• G06F 16/28 - Databases characterised by their database models, e.g. relational or object models

Abstract

A system executes database queries specified using a declarative database query language such as the structured query language (SQL). The system determines whether a runtime error is encountered during execution of a query, for example, a division by zero error, resource usage errors such as out of memory error, time out error, and so on. The system reports such runtime errors encountered during execution of a database query. The system identifies one or more origins of the runtime error in the database query. The origin identifies a portion of the database query that represents a cause of the runtime error. Reporting the origin of a runtime error in the database query simplifies the task of development and testing of database queries.

IPC Classes  ?

• G06F 16/21 - Design, administration or maintenance of databases
• G06F 16/24 - Querying

Abstract

Disclosed herein is a method for determining whether to apply a lazy materialization technique to a query run. The method includes receiving a request to perform a new query in a columnar database containing a plurality of columns. A step in the method includes accessing a set of data in a column of the plurality of columns based on the query. The method includes generating an input to a machine-learned model comprising characteristics of the set of data in the column. From the machine-learned model, the method includes generating a likelihood value indicative of whether a filter of a first portion of the set of data in the column has greater efficiency than a download followed by a filter of the set of data in the column. The method further includes comparing the likelihood value to a threshold value. Based on the comparison, the method includes filtering the first portion of the set of data before downloading the set of data if the likelihood value is equal to or above the threshold value.

IPC Classes  ?

• G06F 16/2453 - Query optimisation
• G06F 16/22 - IndexingData structures thereforStorage structures

Abstract

Disclosed herein is a method for determining whether to apply a lazy materialization technique to a query run. A data processing service receives a request to perform a query identifying a filter column and a non-filter column in a columnar database. The data processing service accesses a first task of contiguous rows in the filter column from a cloud-based object storage. The data processing service applies a filter defined by the query to the first task. The data processing service generates filter results for the first task that may include a percentage of the first task discarded and a run-time. The data processing service determines, based on the filter results for the first task, a likelihood value that indicates a likelihood of gaining a performance benefit by applying the lazy materialization technique to a second task of the query.

IPC Classes  ?

• G06F 16/2453 - Query optimisation
• G06F 11/34 - Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation
• G06F 16/22 - IndexingData structures thereforStorage structures

Abstract

Disclosed herein is a method, system, or non-transitory computer readable medium for evaluating a query on a columnar dataset comprising one or more dictionaries associated with columns in the dataset. The method includes receiving a request to perform a query comprising at least an operator for a columnar dataset on cloud storage. At least one column in the dataset is based on a dictionary, and the dictionary maps one or more values for a column to one or more respective identifiers. The method evaluates the operator on one or more values of the dictionary to generate an updated dictionary comprising updated values. The method may decode the updated dictionary into an updated column comprising updated data values.

IPC Classes  ?

• G06F 16/2455 - Query execution
• G06F 11/34 - Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation
• G06F 16/22 - IndexingData structures thereforStorage structures

Abstract

Disclosed herein is a method, system, or non-transitory computer readable medium for evaluating a query on a columnar dataset comprising one or more dictionaries associated with columns in the dataset. The method includes receiving a request to perform a query comprising at least a operator and a request to return information about a value of interest in a columnar dataset stored on cloud storage. At least one column in the columnar dataset is based on a dictionary. The dictionary maps one or more values for a column to one or more respective identifiers. The method determines whether to perform dictionary filtering for the query by calculating a metric based on one or more factors. Responsive to the metric being below a threshold, which may be predetermined, the method performs the dictionary filtering.

IPC Classes  ?

• G06F 16/24 - Querying
• G06F 11/34 - Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation
• G06F 16/22 - IndexingData structures thereforStorage structures
• G06F 16/2455 - Query execution

Abstract

A system executes user defined functions (UDFs) invoked by database queries. The UDF includes UDF code specified using a programing language distinct from a database query language. A hash value from the UDF code provided by a client application for creating the UDF is compared with a hash value generated from UDF code invoked by database queries to determine whether the two UDF codes match. If the two hash values fail to match, the system takes an action, for example, storing an indication of UDF code mismatch or disabling subsequent executions of the database queries invoking the UDF. The system may use encoded UDF code that is decoded by the system at runtime using a key obtained from a separate system such as the client application. The client application can disable execution of database queries executing the UDF code by refusing to provide the key.

IPC Classes  ?

• G06F 16/242 - Query formulation
• G06F 16/22 - IndexingData structures thereforStorage structures
• G06F 21/60 - Protecting data

Abstract

Disclosed herein is a method for resource management in a web-based container orchestrating environment. A disclosed method includes initializing a set of micro-virtual machines (VMs) within a macro-VM environment. The method each container within a micro-VM based sandbox. The method assigns a virtual central processing unit (vCPU) to a micro-VM based on an estimated memory required by the micro-VM and the estimated available memory associated with the vCPU. The method pins the vCPU with a physical CPU based on the pod location of the physical CPU and an estimated available memory associated with the vCPU and an available local memory of the physical CPU. The method maintains a state of the vCPU and the physical CPU in a resource manager.

IPC Classes  ?

• G06F 9/50 - Allocation of resources, e.g. of the central processing unit [CPU]
• G06F 9/455 - EmulationInterpretationSoftware simulation, e.g. virtualisation or emulation of application or operating system execution engines

Abstract

A system executes database queries specified using a declarative database query language such as the structured query language (SQL). The system determines whether a runtime error is encountered during execution of a query, for example, a division by zero error, resource usage errors such as out of memory error, time out error, and so on. The system reports such runtime errors encountered during execution of a database query. The system identifies one or more origins of the runtime error in the database query. The origin identifies a portion of the database query that represents a cause of the runtime error. Reporting the origin of a runtime error in the database query simplifies the task of development and testing of database queries.

IPC Classes  ?

• G06F 11/3604 - Analysis of software for verifying properties of programs
• G06F 16/25 - Integrating or interfacing systems involving database management systems
• G06F 16/901 - IndexingData structures thereforStorage structures

Abstract

A disclosed configuration receives a first indication that a first transaction is committed to update a first subset of records in a data table at a first version to generate a second version of the data table and receiving a second indication to commit a second transaction to update a second subset of records in a data file of the data table at the first version. The configuration determines a logical prerequisite based on whether the first subset of records changes content of one or more records in the second subset of records and determining a physical prerequisite on whether the second subset of records corresponds to respective data records in data files of the second version of the data table. The configuration commits the second transaction to generate a third version of the data table by updating elements of the deletion vector if the prerequisites are satisfied.

IPC Classes  ?

• G06F 16/00 - Information retrievalDatabase structures thereforFile system structures therefor
• G06F 16/23 - Updating

Abstract

A data processing service performs a rebalancing process for rebalancing stateful tasks on a cluster computing system. In one instance, the method for rebalancing stateful tasks is performed such that the per-operator partitions are spread across available executors of a cluster of the cluster computing system with respect to one or more statistics of the tasks. In one instance, the method for rebalancing stateful tasks is also performed such that the total number of stateful tasks are balanced per executor as long as this rebalancing does not imbalance the per-operator placements. In this way, the processing of stateful tasks can be spread across multiple executors in a relatively uniform manner, even though there may be an upfront cost of breaking the local caching on an executor.

IPC Classes  ?

• G06F 16/27 - Replication, distribution or synchronisation of data between databases or within a distributed database systemDistributed database system architectures therefor
• G06F 16/2455 - Query execution

Abstract

Various embodiments of the present technology generally relate to management of big data storage and data access control systems. In some embodiments, a data access system for use in multiple application service and multiple storage service environments comprises a sandbox database for users, wherein the sandbox database is a virtual database environment via which a user may access datasets according to one or more access policies. In some embodiments, the data access system receives a user request to access a dataset stored in a database into the sandbox environment, wherein the database is associated with the data access system. In response to the request, the data access system may retrieve the corresponding data from the database, determine any associated sandbox access policies, and generate an anonymized data table in the sandbox environment.

IPC Classes  ?

• G06F 21/53 - Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems during program execution, e.g. stack integrity, buffer overflow or preventing unwanted data erasure by executing in a restricted environment, e.g. sandbox or secure virtual machine
• G06F 16/248 - Presentation of query results
• G06F 21/62 - Protecting access to data via a platform, e.g. using keys or access control rules

Abstract

A system includes an interface, a processor, and a memory. The interface is configured to receive a version of a model from a model registry. The processor is configured to store the version of the model, start a process running the version of the model, and update a proxy with version information associated with the version of the model, wherein the updated proxy indicates to redirect an indication to invoke the version of the model to the process. The memory is coupled to the processor and configured to provide the processor with instructions.

IPC Classes  ?

• G06F 16/00 - Information retrievalDatabase structures thereforFile system structures therefor
• G06F 16/21 - Design, administration or maintenance of databases
• G06F 16/955 - Retrieval from the web using information identifiers, e.g. uniform resource locators [URL]
• G06N 5/022 - Knowledge engineeringKnowledge acquisition