Provided are a neural processor, a neural processing device, and a clock gating method thereof, which perform clock gating for a plurality of compute units based on a data flow architecture, in which the neural processor includes at least one neural core that processes at least one task, and a clock controller that selectively gates, according to a data flow architecture of the at least one task, a clock signal provided to the at least one neural core.
A load balancing method in an Artificial Intelligence (AI) service providing system, comprising: obtaining load balancing information of a plurality of servers, generating a load balancing table based on the load balancing information of the plurality of servers, obtaining an inference task request message for an AI service from a user device, deriving at least one target server among the plurality of servers based on the inference task message for the AI service and the load balancing table, and performing load balancing for an inference task of the AI service on the derived target server based on a preset load balancing algorithm, wherein the load balancing information includes connection information, AI model information, and supported hardware information of each server.
Provided are a memory device and a method for dynamically mapping address thereof, in which the memory device includes a memory including a plurality of memory regions, an address register in which at least one of a memory device different from the memory device or a host stores a device address for accessing the memory, and an address translator configured to receive a first interrupt, and in response to the first interrupt, convert a physical address of the memory mapped to the device address.
Provided are a processing device and a method for secure booting thereof, in which the processing device includes a security core that operates a Root of Trust and sequentially performs an integrity check on first firmware and second firmware through the Root of Trust, a main core that sequentially operates the first firmware and the second firmware, a non-volatile memory storing the first firmware and the second firmware, and a first volatile memory that loads the first firmware and the second firmware from a main core domain of the main core and operates the loaded firmware with the main core.
G06F 21/57 - Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
A method for high frequency trading is provided, which is performed by one or more processors, and includes generating input data based on market data for a target item, generating prediction data for the target item for each of a plurality of future time points by inputting the generated input data to a machine learning model, and generating order data for the target item based on the generated prediction data.
A neural processing device and transaction tracking method thereof are provided. The neural processing device comprises a first set of a plurality of neural cores, a shared memory shared by the first set of the plurality of neural cores, and a programmable hardware transactional memory (PHTM) configured to receive a memory access request directed to the shared memory from the first set of the plurality of neural cores and configured to commit or buffer the memory access request.
The present disclosure relates to a data processing device. The data processing device includes: a storage module configured to store a plurality of entries included in a first matrix and position information associated with each of the plurality of entries; a data load module configured to receive the plurality of entries and position information from the storage module, generate a determination result as to whether each of the received entries is zero, and generate an instruction sequence based on the determination result and the position information; and a processing unit configured to generate an operation result by using some of the plurality of entries in accordance with the instruction sequence.
The present disclosure relates to a method for communicating between chiplets in a chiplet system. The chiplet system includes a first chiplet and a second chiplet, and the method includes, by the first chiplet, generating a die-to-die interface flit from a first protocol type transaction based on conversion information, by the first chiplet, transmitting the die-to-die interface flit to the second chiplet, and, by the second chiplet, generating a second protocol type transaction from the die-to-die interface flit based on the conversion information.
Provided is an operating method of a memory access device, the operating method including allowing an access of a first processing element to a plurality of data lines of a memory based on a first command, identifying a number of accesses of the first processing element to the plurality of data lines, and when a number of accesses of the first processing element to a first data line among the plurality of data lines reaches a predetermined number of accesses, allowing an access of a second processing element to the first data line based on a second command.
A method for confidential computing is provided, which is performed by a security core including one or more processor, and includes storing first encrypted data associated with a first tenant in a first memory, in which the first encrypted data is obtained by performing encryption of the first plaintext data using a first encryption key associated with the first tenant, in response to receiving a request to access the first plaintext data, decrypting the first encrypted data using the first encryption key so as to generate the first plaintext data, and providing the first plaintext data to a main core that processes data stored in the first memory.
Provided is a method for shifting data within a memory, which is performed by a direct memory access (DMA) controller, and which includes receiving a task associated with an operation of shifting target data stored in a first area of a memory connected to the DMA controller to a second area of the memory, based on the task, generating sub-tasks associated with operations of shifting data chunks divided from the target data to the second area, determining data chunk groups each including at least one data chunk by grouping the data chunks, determining a priority of each of the data chunk groups based on an address of the first area and an address of the second area, and shifting the data chunk groups to the second area in sequence according to the determined priority.
G06F 13/28 - Handling requests for interconnection or transfer for access to input/output bus using burst mode transfer, e.g. direct memory access, cycle steal
Provided is a method for shifting data within a memory, which is performed by a direct memory access (DMA) controller, and which includes receiving a task associated with an operation of shifting target data stored in a first area of a memory connected to the DMA controller to a second area of the memory, receiving the target data from the memory, determining a priority of each of a plurality of data items divided from the target data based on an address of the first area and an address of the second area, generating a plurality of write requests corresponding to the plurality of data items, and transmitting the plurality of generated write requests sequentially to a memory controller associated with the memory, so that the plurality of data items are stored in the second area in sequence according to the determined priority.
G06F 13/18 - Handling requests for interconnection or transfer for access to memory bus with priority control
G06F 13/16 - Handling requests for interconnection or transfer for access to memory bus
G06F 13/28 - Handling requests for interconnection or transfer for access to input/output bus using burst mode transfer, e.g. direct memory access, cycle steal
13.
NEURAL PROCESSING DEVICE AND METHOD FOR SYNCHRONIZATION THEREOF
A neural processing device is provided. The neural processing device comprises a plurality of neural processors, a shared memory shared by the plurality of neural processors, a plurality of semaphore memories, and global interconnection. The plurality of neural processors generates a plurality of L3 sync targets, respectively. Each semaphore memory is associated with a respective one of the plurality of neural processors, and the plurality of semaphore memories receive and store the plurality of L3 sync targets, respectively. Synchronization of the plurality of neural processors is performed according to the plurality of L3 sync targets. The global interconnection connects the plurality of neural processors with the shared memory, and comprises an L3 sync channel through which an L3 synchronization signal corresponding to at least one L3 sync target is transmitted.
G06N 3/063 - Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
G06F 5/06 - Methods or arrangements for data conversion without changing the order or content of the data handled for changing the speed of data flow, i.e. speed regularising
G06F 9/38 - Concurrent instruction execution, e.g. pipeline or look ahead
G06F 9/52 - Program synchronisationMutual exclusion, e.g. by means of semaphores
14.
NEURAL PROCESSING DEVICE, NON-TRANSITORY COMPUTER-READABLE RECORDING MEDIUM AND METHOD FOR TRANSFERRING OWNERSHIP OF THE NEURAL PROCESSING DEVICE THEREOF
A neutral processing device, a method and a non-transitory computer-readable recording medium for transferring ownership of the neural processing device are described herein. In an implementation, the method comprising steps of executing a first stage boot loader, loading and verifying a second stage boot loader based on the first stage boot loader, executing the second stage boot loader after verification, loading and verifying ownership transfer firmware based on the second stage boot loader, executing the ownership transfer firmware after verification, transferring ownership of the neural processing device from a first user to a second user based on the ownership setting information of the neural processing device, and encrypting or decrypting data stored in the neural processing device based on an encryption key of the second user.
G06F 21/72 - Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure computing or processing of information in cryptographic circuits
G06F 21/57 - Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
H04L 9/32 - Arrangements for secret or secure communicationsNetwork security protocols including means for verifying the identity or authority of a user of the system
15.
NEURAL CORE, NEURAL PROCESSING DEVICE INCLUDING SAME, AND METHOD FOR LOADING DATA OF NEURAL PROCESSING DEVICE
A neural core, a neural processing device including same and a method for lauding data of a neural processing device are provided. The neural core comprises a processing unit configured to perform operations, an L0 memory configured to store input data and an LSU configured to perform a load task and a store task of data between the processing unit and the L0 memory, wherein the LSU comprises a local memory load unit configured to transmit the input data in the L0 memory to the processing unit, and the local memory load unit comprises a target decision module configured to identify and retrieve the input data in the L0 memory, a transformation logic configured to transform the input data and thereby generate transformed data and an output FIFO configured to receive the transformed data and transmit the transformed data to the processing unit in the received order.
A neural processing device and a method for managing tasks thereof are provided. The neural processing device includes a neural core configured to perform a task and generate a completion signal for completion of the task, a core global configured to transfer task information for the task to the neural core and receive the completion signal of the task from the neural core, and a task manager configured to generate and transmit the task information to the core global, receive the completion signal from the core global, generate a completion report, and transmit the completion report.
A processing device comprises multiplier circuitry configured to output a product of mantissas represented by first and second signals in response to a mode signal and output a product of an integer represented by the first signal and a mantissa represented by the second signal in response to the mode signal. The processing device further includes an aligning circuit configured to shift a mantissa part of a third signal based on an exponent part of the third signal and an exponent of a product of the first and second signal to generate and output a shifted signal. The processing device further includes an arithmetic logic circuit configured to output a mantissa of a sum of a product of the first and second signals and the third signal in response to an output signal of the aligning circuit and an output signal of the multiplier circuitry.
G06F 5/01 - Methods or arrangements for data conversion without changing the order or content of the data handled for shifting, e.g. justifying, scaling, normalising
G06F 7/544 - Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using non-contact-making devices, e.g. tube, solid state deviceMethods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using unspecified devices for evaluating functions by calculation
42 - Scientific, technological and industrial services, research and design
Goods & Services
Software as a service [SaaS]; cloud computing services; consulting services in the field of cloud computing; providing virtual computer systems through cloud computing; product research, custom design and testing for new product development regarding semiconductors; research services relating to semiconductors; design of semiconductors; semiconductor design services; integrated circuit design services; design and testing of semiconductors; research in the area of semiconductor processing technology; design of semiconductor chips; design of integrated circuits; development of computer hardware; computer hardware design.
42 - Scientific, technological and industrial services, research and design
Goods & Services
Software as a service [SaaS]; cloud computing services; consulting services in the field of cloud computing; providing virtual computer systems through cloud computing; product research, custom design and testing for new product development regarding semiconductors; research services relating to semiconductors; design of semiconductors; design of semiconductors or integrated circuits; design and testing of semiconductors; research in the area of semiconductor processing technology; design of semiconductor chips; design of integrated circuits; development of computer hardware; computer hardware design
22.
TASK MANAGER, PROCESSING DEVICE, AND METHOD FOR CHECKING TASK DEPENDENCIES THEREOF
A task manager, a neural processing device, and a method for checking task dependencies thereof are provided. The task manager includes a task buffer configured to receive first and second tasks of different first and second types, a first queue configured to receive a first task descriptor for the first task from the task buffer, a second queue configured to receive a second task descriptor for the second task from the task buffer, a dependency checker configured to check dependencies of the first and second task descriptors, a third queue configured to receive the first task descriptor from the dependency checker, and a fourth queue configured to receive the second task descriptor from the dependency checker.
Provided is an electronic device, including a first chiplet including a first bus interface, a first interconnect management module, and a first interconnect module, and a second chiplet connected to the first chiplet through the first interconnect module, wherein, in response to an occurrence of a request transaction associated with the second chiplet, the first interconnect management module stores, in a register, first information associated with the request transaction.
G06F 13/28 - Handling requests for interconnection or transfer for access to input/output bus using burst mode transfer, e.g. direct memory access, cycle steal
G06F 13/36 - Handling requests for interconnection or transfer for access to common bus or bus system
A data operation device is disclosed. The data operation device comprises at least one memory configured to store a first data set represented as a first sparse matrix and a second data set represented as a second matrix, a vector unit configured to perform a row-wise product-based matrix multiplication operation based on the first sparse matrix and the second matrix and output a third data set represented as a third matrix, and a memory load unit configured to load into the vector unit first vector data associated with a row of the first sparse matrix from the first data set, and second vector data associated with a row of the second matrix that corresponds to an order of non-zero vector elements included in the first vector data from the second data set.
Neural processing device, non-transitory computer-readable recording medium and method for transferring ownership of the neural processing device thereof
A neural processing device, a method and a non-transitory computer-readable recording medium for transferring ownership of the neural processing device are described herein. In an implementation, the method comprising steps of executing a first stage boot loader, loading and verifying a second stage boot loader based on the first stage boot loader, executing the second stage boot loader after verification, loading and verifying ownership transfer firmware based on the second stage boot loader, executing the ownership transfer firmware after verification, transferring ownership of the neural processing device from a first user to a second user based on the ownership setting information of the neural processing device, and encrypting or decrypting data stored in the neural processing device based on an encryption key of the second user.
G06F 21/72 - Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure computing or processing of information in cryptographic circuits
H04L 9/32 - Arrangements for secret or secure communicationsNetwork security protocols including means for verifying the identity or authority of a user of the system
G06F 21/57 - Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
26.
NEURAL PROCESSING DEVICE AND METHOD FOR TRANSMITTING DATA THEREOF
A processing device comprises processors, a first memory shared by the processors, and a cache comprising a second memory comprising a plurality of memory units, each of the plurality of memory units in the second memory being associated with a respective one of a plurality of request identifiers. The cache receives a memory read request including a request identifier and a memory address from at least one of the processors, identifies an allocated memory address identifier for the memory address, accesses the first memory to read data of the memory address, obtains one or more request identifiers which requested data of the memory address from the second memory based on the allocated memory address identifier, and transmitting the data of the memory address to one or more processors which requested data of the memory address based on the one or more request identifiers.
An apparatus comprising neural processors, a command processor, and a shared memory. The command processor receives a context start signal indicating a start of a context of a neural network model from a host system. The command processor determines whether neural network model data is entirely or partially updated based on the context start signal. The command processor updates the neural network model data in the shared memory based on a determination on whether neural network model data is entirely or partially updated based on the context start signal. The command processor generates a plurality of task descriptors describing neural network model tasks based on the neural network model data, and distributes the plurality of task descriptors to the neural processors.
A data operation device is disclosed. The data operation device comprises at least one memory configured to store a first data set represented as a first sparse matrix and a second data set represented as a second matrix, a vector unit configured to perform a row-wise product-based matrix multiplication operation based on the first sparse matrix and the second matrix and output a third data set represented as a third matrix, and a memory load unit configured to load into the vector unit first vector data associated with a row of the first sparse matrix from the first data set, and second vector data associated with a row of the second matrix that corresponds to an order of non-zero vector elements included in the first vector data from the second data set.
A processing device may include multiplier circuitry configured to output a product of an integer represented by the first signal and a mantissa represented by the second signal. The processing device may further include a dynamic shifting circuit configured to shift a first shifted signal generated by shifting a mantissa part of a third signal based on the integer part of the first signal to generate and output a second shifted signal, and shift an output signal of the multiplier circuitry based on the integer part of the first signal to generate and output a third shifted signal. The processing device may further include an arithmetic logic circuit configured to output an signal representing a mantissa of a sum of a product of the first and second signals and the third signal based on output signals of the dynamic shifting circuit.
G06F 5/01 - Methods or arrangements for data conversion without changing the order or content of the data handled for shifting, e.g. justifying, scaling, normalising
G06F 7/544 - Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using non-contact-making devices, e.g. tube, solid state deviceMethods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using unspecified devices for evaluating functions by calculation
G06F 7/57 - Arithmetic logic units [ALU], i.e. arrangements or devices for performing two or more of the operations covered by groups or for performing logical operations
Provided is an electronic device, including a first chiplet including a system bus, a first interconnect module, and a second interconnect module, and a second chiplet connected to the first chiplet through at least one of a first interconnect interface connected to the first interconnect module or second interconnect interface connected to the second interconnect module, in which, in response to determining that a communication failure occurs between the first chiplet and the second chiplet, at least a part of a transfer path through which information is transmitted from the first chiplet to the second chiplet is changed.
H01L 23/538 - Arrangements for conducting electric current within the device in operation from one component to another the interconnection structure between a plurality of semiconductor chips being formed on, or in, insulating substrates
H01L 25/065 - Assemblies consisting of a plurality of individual semiconductor or other solid-state devices all the devices being of a type provided for in a single subclass of subclasses , , , , or , e.g. assemblies of rectifier diodes the devices not having separate containers the devices being of a type provided for in group
Provided is an electronic device, in which a first management module of a first chiplet generates a first request transaction for measuring a latency between the first chiplet and a second chiplet, and transmits the generated first request transaction to the second chiplet through a first interconnect module of the first chiplet, a second management module of the second chiplet generates a first response transaction corresponding to the first request transaction, and transmits the generated first response transaction to the first chiplet through a second interconnect module of the second chiplet, and the latency between the first chiplet and the second chiplet is determined based on a first time at which the first request transaction is generated in the first chiplet and a second time at which the first chiplet receives the first response transaction.
Provided is a chiplet including an interconnect module for connecting to another chiplet, a bus interface for connecting to at least one functional module in the chiplet, and an address remapper block connected to the interconnect module and the bus interface, in which the address remapper block receives a transaction and remaps a destination address in the transaction.
Provided is a chiplet including an interconnect module for connecting to another chiplet, a bus interface for connecting to at least one functional module in the chiplet, and a save and forward module connected to the interconnect module and the bus interface, in which the save and forward module includes a slave port that receives a transaction from one of the interconnect module or the bus interface, a data buffer that temporarily stores at least a portion of the transaction, and a master port that transmits the transaction stored in the data buffer to the other one of the interconnect module or the bus interface, and the transaction is divided into predetermined units and transmitted.
G06F 13/16 - Handling requests for interconnection or transfer for access to memory bus
G06F 13/28 - Handling requests for interconnection or transfer for access to input/output bus using burst mode transfer, e.g. direct memory access, cycle steal
Provided is an electronic device, including a first chiplet including a first bus interface, a first interconnect management module, and a first interconnect module, and a second chiplet connected to the first chiplet through the first interconnect module, wherein, in response to an occurrence of a request transaction associated with the second chiplet, the first interconnect management module stores, in a register, first information associated with the request transaction.
G06F 13/28 - Handling requests for interconnection or transfer for access to input/output bus using burst mode transfer, e.g. direct memory access, cycle steal
G06F 13/36 - Handling requests for interconnection or transfer for access to common bus or bus system
An electronic device comprises a main chiplet including a first memory and at least one sub-chiplet including a second memory, wherein the main chiplet is configured to initialize a first interface for inter-chiplet connection based on first boot firmware stored in the first memory in response to receiving booting signal, acquire third boot firmware stored in an external memory, initialize a second interface for communication between an external device and the main chiplet based on the third boot firmware, set a configuration for interconnection between the main chiplet and the at least one sub-chiplet, initialize a third memory included in the main chiplet, and load at least one of an application firmware or an operating system to the third memory, and the at least one sub-chiplet is configured to initialize the first interface based on second boot firmware stored in the second memory in response to receiving the booting signal.
G06F 11/14 - Error detection or correction of the data by redundancy in operation, e.g. by using different operation sequences leading to the same result
G06F 13/42 - Bus transfer protocol, e.g. handshakeSynchronisation
G06F 21/57 - Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
36.
RECONFIGURABLE ON-CHIP MEMORY BANK, RECONFIGURABLE ON-CHIP MEMORY, SYSTEM-ON-CHIP HAVING SAME MOUNTED THEREON, AND METHOD FOR USING RECONFIGURABLE ON-CHIP MEMORY
A reconfigurable on-chip memory bank, reconfigurable on-chip memory, system-on-chip having same mounted thereon, and method for using reconfigurable on-chip memory is provided. The reconfigurable on-chip memory bank comprises a cell array in which a plurality of cells are arranged and data are stored, a first path unit configured to move data in the cell array, a second path unit configured to move data in the cell array and a bank controller operably coupled with the first and second path units, and configured to determine, according to a path control signal during runtime, data path which data are to be transmitted to or received from, to use a first address system when the first path unit is determined to be the data path, and to use a second address system different from the first address system when the second path unit is determined to be the data path.
The disclosure relates to an apparatus for high frequency trading. The apparatus includes one or more memories, at least one reconfigurable processor coupled to the one or more memories, and a dedicated accelerator preconfigured for the machine learning model. The one or more processors receive market-related information from one or more market-related information servers and generates market prediction reference data based on the market-related information. The dedicated accelerator performs operations for the machine learning model with the market prediction reference data to generate future market prediction data. The at least one reconfigurable processor generates an order signal based on the future market prediction data and transmits the order signal to a target exchange server.
A Field Programmable Gate Array (FPGA) system includes a main FPGA and one or more sub-FPGAs connected to the main FPGA. The main FPGA is configured to detect a positive edge of a pulse included in a user clock using a sampling clock of the main FPGA, generate a flag using the detected positive edge, generate a clock packet indicating the generated flag, and provide the generated clock packet to any one of the one or more sub-FPGAs.
A command processor determines whether a command descriptor describing a current command is in a first format or in a second format, wherein the first format includes a source memory address pointing to a memory area in a shared memory having a binary code to be accessed according to direct memory access (DMA) scheme, and the second format includes one or more object indices, a respective one of the one or more object indices indicating an object in an object database. If the command descriptor describing the current command is in the second format, the command processor converts a format of the command descriptor to the first format, generates one or more task descriptors describing neural network model tasks based on the command descriptor in the first format, and distributes the one or more task descriptors to the one or more neural processors.
A neural processing device, a processing element included therein and a method for operating various formats of the neural processing device are provided. The neural processing device includes at least one neural processor, a shared memory shared by the at least one neural processor, and a global interconnection configured to transmit data between the at least one neural processor and the shared memory, wherein each of the at least one neural processor comprises at least one processing element, each of the at least one processing element receives an input in a first format and thereby performs an operation, and receives an input in a second format that is different from the first format and thereby performs an operation if a format conversion signal is received, and the first format and the second format have a same number of bits.
A neural processing device is provided. The neural processing device comprises: an activation buffer in which first and second input activations are stored, an activation compressor configured to generate a first compressed input activation by using the first and second input activations, and a tensor unit configured to perform two-dimensional calculations using the first compressed input activation, wherein the first compressed input activation comprises first input row data comprising at least a portion of the first input activation and at least a portion of the second input activation, and first metadata corresponding to the first input row data.
A processing device comprises multiplier circuitry configured to output a product of mantissas represented by first and second signals in response to a mode signal and output a product of an integer represented by the first signal and a mantissa represented by the second signal in response to the mode signal. The processing device further includes an aligning circuit configured to shift a mantissa part of a third signal based on an exponent part of the third signal and an exponent of a product of the first and second signal to generate and output a shifted signal. The processing device further includes an arithmetic logic circuit configured to output a mantissa of a sum of a product of the first and second signals and the third signal in response to an output signal of the aligning circuit and an output signal of the multiplier circuitry.
G06F 5/01 - Methods or arrangements for data conversion without changing the order or content of the data handled for shifting, e.g. justifying, scaling, normalising
G06F 7/544 - Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using non-contact-making devices, e.g. tube, solid state deviceMethods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using unspecified devices for evaluating functions by calculation
A processing device may include multiplier circuitry configured to output a product of an integer represented by the first signal and a mantissa represented by the second signal. The processing device may further include a dynamic shifting circuit configured to shift a first shifted signal generated by shifting a mantissa part of a third signal based on the integer part of the first signal to generate and output a second shifted signal, and shift an output signal of the multiplier circuitry based on the integer part of the first signal to generate and output a third shifted signal. The processing device may further include an arithmetic logic circuit configured to output an signal representing a mantissa of a sum of a product of the first and second signals and the third signal based on output signals of the dynamic shifting circuit.
G06F 5/01 - Methods or arrangements for data conversion without changing the order or content of the data handled for shifting, e.g. justifying, scaling, normalising
G06F 7/544 - Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using non-contact-making devices, e.g. tube, solid state deviceMethods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using unspecified devices for evaluating functions by calculation
G06F 7/57 - Arithmetic logic units [ALU], i.e. arrangements or devices for performing two or more of the operations covered by groups or for performing logical operations
44.
NEURAL PROCESSING DEVICE AND METHOD FOR CONVERTING DATA THEREOF
A neural processing device and a method for converting data thereof are provided. The neural processing device comprises a first compute unit configured to receive first input data in first precision and generate first output data in the first precision by performing calculations, a second compute unit configured to receive second input data in second precision which is different from the first precision and generate second output data in the second precision by performing calculation, and a first converting buffer configured to receive and store the first output data, generate the second input data by converting the first output data into the second precision, and transmit the second input data to the second compute unit.
G06F 5/00 - Methods or arrangements for data conversion without changing the order or content of the data handled
G06F 3/06 - Digital input from, or digital output to, record carriers
G06F 5/06 - Methods or arrangements for data conversion without changing the order or content of the data handled for changing the speed of data flow, i.e. speed regularising
45.
ELECTRONIC DEVICE INCLUDING A PLURALITY OF CHIPLETS AND METHOD FOR TRANSMITTING TRANSACTION THEREOF
An electronic device comprising a plurality of chiplets is disclosed. The electronic device comprises a first chiplet that generates a transaction, a second chiplet that receives the transaction, and at least one third chiplet that relays the transaction, wherein the first chiplet determines a route path for the transaction that passes through the at least one third chiplet, and transmits the transaction through the determined route path for the transaction.
G06F 13/28 - Handling requests for interconnection or transfer for access to input/output bus using burst mode transfer, e.g. direct memory access, cycle steal
46.
CHIPLET SYSTEM AND METHOD FOR COMMUNICATING BETWEEN CHIPLETS IN CHIPLET SYSTEM
The present disclosure relates to a method for communicating between chiplets in a chiplet system. The chiplet system includes a first chiplet and a second chiplet, the first chiplet includes a controller including a protocol layer, and the method includes, by the protocol layer of the first chiplet, receiving first data, by the protocol layer of the first chiplet, receiving conversion information from the second chiplet, and, by the protocol layer of the first chiplet, generating second data based on the received first data and conversion information.
The present disclosure relates to a method for communicating between chiplets in a chiplet system. The chiplet system includes a first chiplet and a second chiplet, and the method includes, by the first chiplet, generating a die-to-die interface flit from a first protocol type transaction based on conversion information, by the first chiplet, transmitting the die-to-die interface flit to the second chiplet, and, by the second chiplet, generating a second protocol type transaction from the die-to-die interface flit based on the conversion information.
A neural processing device is provided. The neural processing device comprises: a processing unit configured to perform calculations, an L0 memory configured to receive data from the processing unit and provide data to the processing unit, and an LSU (Load/Store Unit) configured to perform load and store operations of the data, wherein the LSU comprises: a neural core load unit configured to issue a load instruction of the data, a neural core store unit configured to issue a store instruction for transmitting and storing the data, and a sync ID logic configured to provide a sync ID to the neural core load unit and the neural core store unit to thereby cause a synchronization signal to be generated for each sync ID.
An electronic device comprises a main chiplet including a first memory and at least one sub-chiplet including a second memory, wherein the main chiplet is configured to initialize a first interface for inter-chiplet connection based on first boot firmware stored in the first memory in response to receiving booting signal, acquire third boot firmware stored in an external memory, initialize a second interface for communication between an external device and the main chiplet based on the third boot firmware, set a configuration for interconnection between the main chiplet and the at least one sub-chiplet, initialize a third memory included in the main chiplet, and load at least one of an application firmware or an operating system to the third memory, and the at least one sub-chiplet is configured to initialize the first interface based on second boot firmware stored in the second memory in response to receiving the booting signal.
G06F 11/14 - Error detection or correction of the data by redundancy in operation, e.g. by using different operation sequences leading to the same result
G06F 21/57 - Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
G06F 13/42 - Bus transfer protocol, e.g. handshakeSynchronisation
50.
METHOD AND SYSTEM FOR RECOVERING DATA ASSOCIATED WITH ARTIFICIAL INTELLIGENCE CALCULATION
Provided is a method for recovering data for artificial intelligence calculation, which is performed by one or more processors and which includes extracting, from a command queue, a descriptor associated with a target job, that is a job to be executed, of a plurality of jobs, executing at least one command associated with the extracted descriptor to execute artificial intelligence calculation associated with the target job, resetting the command queue if an error occurs while executing the artificial intelligence calculation, determining at least one descriptor to be recovered, based on tracking data associated with at least one job, and recovering the determined at least one descriptor to the reset command queue.
G06F 11/14 - Error detection or correction of the data by redundancy in operation, e.g. by using different operation sequences leading to the same result
G06F 11/07 - Responding to the occurrence of a fault, e.g. fault tolerance
A method for controlling operations of a machine learning model is performed by one or more processors and includes determining a threshold for skipping an operation, acquiring an activation value associated with at least one layer included in the machine learning model, determining whether the activation value is less than the threshold, and if the activation value is less than the threshold, controlling the operations of the machine learning model such that an operation associated with the activation value is skipped in the machine learning model.
A method and a device for quantizing parameters of a neural network are disclosed. According to one aspect of the present invention, a computer-implemented method for quantizing parameters of a neural network including batch normalization parameters, the method comprising obtaining parameters in a second layer connected to a first layer; removing at least one parameter among the parameters based on either any one of output values of the first layer or batch normalization parameters applied to the parameters; and quantizing the parameters in the second layer based on parameters that have survived the removing, is provided.
A neural processing device and a method of updating translation lookaside buffer thereof are provided. The neural processing device includes at least one processor module each of which includes at least one micro translation lookaside buffer (TLB), a hierarchical memory that is accessed by the at least one micro TLB, and a command processor configured to update the at least one micro TLB in a push mode by generating a first update signal which indicates update of the at least one micro TLB and transmitting the first update signal to the at least one micro TLB.
A neural core, a neural processor, and a data processing method thereof are provided. The neural core includes a processing unit configured to generate output activation by performing calculation of input activation and a weight, a memory load/store unit configured to generate a memory calculation request requesting one-dimensional calculation of summing the output activation and target data, and a memory configured to receive the output activation and the memory calculation request, import the target data from a memory address in response to the memory calculation request, generate output data by performing the one-dimensional calculation of the target data and the output activation, and store the generated output data in the memory address.
09 - Scientific and electric apparatus and instruments
Goods & Services
Large scale integrated circuits; microchips; electronic components in the nature of semiconductors; semiconductor devices; semiconductors; semi-conductor memory units; semiconductor wafers; system on a chip (SoC); electronic semi-conductors; electronic control circuits for electric devices and for computers featuring AI software; electronic circuits; integrated circuits; chips for electric devices and for computers featuring AI software; chipsets
57.
Neural processing device and transaction tracking method thereof
A neural processing device and transaction tracking method thereof are provided. The neural processing device comprises a first set of a plurality of neural cores, a shared memory shared by the first set of the plurality of neural cores, and a programmable hardware transactional memory (PHTM) configured to receive a memory access request directed to the shared memory from the first set of the plurality of neural cores and configured to commit or buffer the memory access request.
Provided is a method for recovering data for artificial intelligence calculation, which is performed by one or more processors and which includes extracting, from a command queue, a descriptor associated with a target job, that is a job to be executed, of a plurality of jobs, executing at least one command associated with the extracted descriptor to execute artificial intelligence calculation associated with the target job, resetting the command queue if an error occurs while executing the artificial intelligence calculation, determining at least one descriptor to be recovered, based on tracking data associated with at least one job, and recovering the determined at least one descriptor to the reset command queue.
G06F 11/07 - Responding to the occurrence of a fault, e.g. fault tolerance
G06F 11/14 - Error detection or correction of the data by redundancy in operation, e.g. by using different operation sequences leading to the same result
An apparatus comprising neural processors, a command processor, and a shared memory is provided. The command processor, in response to receiving a context start signal indicating a start of a context of a neural network model from a host system, directly accesses a memory in the host system to read neural network model data for the context of the neural network model. The command processor, based on a determination on whether the plurality of task descriptors for the previous context of the neural network model are not allowed to be reused for the plurality of task descriptors for the current context of the neural network model, generates the plurality of task descriptors for the current context of the neural network model.
Provided are a neural processor, a neural processing device, and a clock gating method thereof, which perform clock gating for a plurality of compute units based on a data flow architecture, in which the neural processor includes at least one neural core that processes at least one task, and a clock controller that selectively gates, according to a data flow architecture of the at least one task, a clock signal provided to the at least one neural core.
An apparatus comprising neural processors, a command processor, and a shared memory. The command processor receives a context start signal indicating a start of a context of a neural network model from a host system. The command processor determines whether neural network model data is entirely or partially updated based on the context start signal. The command processor updates the neural network model data in the shared memory based on a determination on whether neural network model data is entirely or partially updated based on the context start signal. The command processor generates a plurality of task descriptors describing neural network model tasks based on the neural network model data, and distributes the plurality of task descriptors to the neural processors.
A command processor determines whether a command descriptor describing a current command is in a first format or in a second format, wherein the first format includes a source memory address pointing to a memory area in a shared memory having a binary code to be accessed according to direct memory access (DMA) scheme, and the second format includes one or more object indices, a respective one of the one or more object indices indicating an object in an object database. If the command descriptor describing the current command is in the second format, the command processor converts a format of the command descriptor to the first format, generates one or more task descriptors describing neural network model tasks based on the command descriptor in the first format, and distributes the one or more task descriptors to the one or more neural processors.
An apparatus comprising neural processors, a command processor, and a shared memory is provided. The command processor, in response to receiving a context start signal indicating a start of a context of a neural network model from a host system, directly accesses a memory in the host system to read command stream data for the neural network model. The command processor selects a current command and determines whether the current command is a branch command or a command describing neural network model tasks. The command processor selects a command among commands in the command stream data as a next command to be executed, based on a determination on whether the current command is the branch command or the command describing neural network model tasks.
A neural processing device is provided, which includes a first block that operates at a first operating frequency and at a second operating frequency different from the first operating frequency, a second block operates at the first operating frequency, and a data communication mode determiner that controls data communication between the first block and the second block, and determines a first data communication mode for a first interface between the first block and the second block.
A method for processing tasks in parallel is performed by at least one processor, and includes performing a first task associated with a first instruction, determining whether the first instruction is a burst load instruction, in response to determining that the first instruction is the burst load instruction, acquiring a second instruction, and performing a second task associated with the acquired second instruction, in which the first task and the second task are performed in parallel.
A method for allocating on-chip memory of a neural processing unit is performed by one or more processors, and includes deallocating an allocated chunk in an on-chip memory area, which is finished with the use of the memory, and converting it into a cached chunk, receiving an on-chip memory allocation request for specific data, determining whether there is a cached chunk of one or more cached chunks that is allocable for the specific data, based on a comparison between a size of the specific data and the size of the one or more cached chunks, and based on a result of determining whether there is the cached chunk that is allocable for the specific data, allocating the specific data to a specific cached chunk of the one or more cached chunks, or allocating the specific data to at least a portion of the free chunk.
A neural processor and method for assigning config ID for neural core included in the same are provided. The neural processor includes a core array comprising a first neural core, a second neural core, a first data line connecting the first neural core and the second neural core in series, and a config line connecting the first neural core and the second neural core in series, an ID config manager configured to assign a first config ID to the first neural core and a second config ID to the second neural core via the config line, and a memory configured to input and output data to and from the core array via the first data line.
A neural processing device and a method for managing tasks thereof are provided. The neural processing device includes a neural core configured to perform a task and generate a completion signal for completion of the task, a core global configured to transfer task information for the task to the neural core and receive the completion signal of the task from the neural core, and a task manager configured to generate and transmit the task information to the core global, receive the completion signal from the core global, generate a completion report, and transmit the completion report.
A method and an apparatus are disclosed for a function approximation using a multi-level lookup table. The function approximation device and the method approximate function values for a function in multiple stages using a multi-level lookup table (LUT) in approximating nonlinear function values based on piecewise linear approximation or piecewise polynomial approximation. In the multi-level LUT, a segment length is set differently depending on the amount of change in the function in order to reduce approximation errors.
G06F 17/17 - Function evaluation by approximation methods, e.g. interpolation or extrapolation, smoothing or least mean square method
G06F 7/483 - Computations with numbers represented by a non-linear combination of denominational numbers, e.g. rational numbers, logarithmic number system or floating-point numbers
70.
METHOD AND DEVICE FOR DETERMINING SATURATION RATIO-BASED QUANTIZATION RANGE FOR QUANTIZATION OF NEURAL NETWORK
A method and a device for determining a quantization range based on a saturation ratio for quantization of an artificial neural network are disclosed.
A method and a device for determining a quantization range based on a saturation ratio for quantization of an artificial neural network are disclosed.
According to one aspect of the present invention, there is provided a computer-implemented method and a device for determining a quantization range for tensors of an artificial neural network, comprising observing a saturation ratio at a current iteration from the tensors of the artificial neural network and the quantization range; and adjusting the quantization range so that the observed saturation ratio follows a predetermined target saturation ratio.
A task manager, a neural processing device, and a method for checking task dependencies thereof are provided. The task manager includes a task buffer configured to receive first and second tasks of different first and second types, a first queue configured to receive a first task descriptor for the first task from the task buffer, a second queue configured to receive a second task descriptor for the second task from the task buffer, a dependency checker configured to check dependencies of the first and second task descriptors, a third queue configured to receive the first task descriptor from the dependency checker, and a fourth queue configured to receive the second task descriptor from the dependency checker.
A method for confidential computing is provided, which is performed by a security core including one or more processor, and includes storing first encrypted data associated with a first tenant in a first memory, in which the first encrypted data is obtained by performing encryption of the first plaintext data using a first encryption key associated with the first tenant, in response to receiving a request to access the first plaintext data, decrypting the first encrypted data using the first encryption key so as to generate the first plaintext data, and providing the first plaintext data to a main core that processes data stored in the first memory.
A computing system and a transposition method therefor. One aspect of the present disclosure provides a computing system that variably determines an input/output unit of a memory depending on a processing unit or an output unit of a processor, and a transposition method for supporting same.
A neural processing device processing circuitry comprising and method for controlling the same are provided. The neural processing device comprises at least one processing engine group each of which includes at least one processing engines, a first memory shared by the at least one processing engine group, and an interconnection configured to exchange data between the at least one processing engine group and the first memory. The processing circuitry is configured to monitor the at least one processing engine to check performance related to the at least one processing engine, and provide hardware resources to at least one of the first memory, the interconnection or the at least one processing engine, according to the performance.
A method for runtime integrity check, performed by a security core including one or more processors includes storing a first output value, which is generated by using a one-way encryption algorithm based on first data and a first encryption key managed by an encryption key manager accessible by the security core, in a main memory that is a volatile memory in association with the first data, generating a second output value for the first data based on the first data and the first encryption key by using the one-way encryption algorithm, and checking for possible tampering of the first data stored in the main memory by comparing the first output value with the generated second output value.
G06F 21/74 - Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure computing or processing of information operating in dual or compartmented mode, i.e. at least one secure mode
G06F 21/78 - Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure storage of data
76.
Neural processor, neural processing device including the same, and method for determining data communication mode of neural processing device
A neural processing device is provided, which includes a first block that operates at a first operating frequency and at a second operating frequency different from the first operating frequency, a second block operates at the first operating frequency, and a data communication mode determiner that controls data communication between the first block and the second block, and determines a first data communication mode for a first interface between the first block and the second block.
A processing device comprises a first set of processors comprising a first processor and a second processor, each of which comprises at least one controllable port, a first memory operably coupled to the first set of processors, at least one forward data line configured for one-way transmission of data in a forward direction between the first set of processors, and at least one backward data line configured for one-way transmission of data in a backward direction between the first set of processors. wherein the first set of processors are operably coupled in series via the at least one forward data line and the at least one backward data line.
G06F 15/80 - Architectures of general purpose stored program computers comprising an array of processing units with common control, e.g. single instruction multiple data processors
G06F 13/36 - Handling requests for interconnection or transfer for access to common bus or bus system
G06N 3/063 - Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
G06N 3/084 - Backpropagation, e.g. using gradient descent
78.
Neural processing device, processing element included therein and method for operating various formats of neural processing device
A neural processing device, a processing element included therein and a method for operating various formats of the neural processing device are provided. The neural processing device includes at least one neural processor, a shared memory shared by the at least one neural processor, and a global interconnection configured to transmit data between the at least one neural processor and the shared memory, wherein each of the at least one neural processor comprises at least one processing element, each of the at least one processing element receives an input in a first format and thereby performs an operation, and receives an input in a second format that is different from the first format and thereby performs an operation if a format conversion signal is received, and the first format and the second format have a same number of bits.
A method for allocating on-chip memory of a neural processing unit is performed by one or more processors, and includes deallocating an allocated chunk in an on-chip memory area, which is finished with the use of the memory, and converting it into a cached chunk, receiving an on-chip memory allocation request for specific data, determining whether there is a cached chunk of one or more cached chunks that is allocable for the specific data, based on a comparison between a size of the specific data and the size of the one or more cached chunks, and based on a result of determining whether there is the cached chunk that is allocable for the specific data, allocating the specific data to a specific cached chunk of the one or more cached chunks, or allocating the specific data to at least a portion of the free chunk.
A neural processing device and a method of updating translation lookaside buffer thereof are provided. The neural processing device includes at least one processor module each of which includes at least one micro translation lookaside buffer (TLB), a hierarchical memory that is accessed by the at least one micro TLB, and a command processor configured to update the at least one micro TLB in a push mode by generating a first update signal which indicates update of the at least one micro TLB and transmitting the first update signal to the at least one micro TLB.
A neural core, a neural processor, and a data processing method thereof are provided. The neural core includes a processing unit configured to generate output activation by performing calculation of input activation and a weight, a memory load/store unit configured to generate a memory calculation request requesting one-dimensional calculation of summing the output activation and target data, and a memory configured to receive the output activation and the memory calculation request, import the target data from a memory address in response to the memory calculation request, generate output data by performing the one-dimensional calculation of the target data and the output activation, and store the generated output data in the memory address.
A processing device comprises processors, a first memory shared by the processors, and a cache comprising a second memory comprising a plurality of memory units, each of the plurality of memory units in the second memory being associated with a respective one of a plurality of request identifiers. The cache receives a memory read request including a request identifier and a memory address from at least one of the processors, identifies an allocated memory address identifier for the memory address, accesses the first memory to read data of the memory address, obtains one or more request identifiers which requested data of the memory address from the second memory based on the allocated memory address identifier, and transmitting the data of the memory address to one or more processors which requested data of the memory address based on the one or more request identifiers.
A neural core, a neural processing device including same and a method for lauding data of a neural processing device are provided. The neural core comprises a processing unit configured to perform operations, an L0 memory configured to store input data and an LSU configured to perform a load task and a store task of data between the processing unit and the L0 memory, wherein the LSU comprises a local memory load unit configured to transmit the input data in the L0 memory to the processing unit, and the local memory load unit comprises a target decision module configured to identify and retrieve the input data in the L0 memory, a transformation logic configured to transform the input data and thereby generate transformed data and an output FIFO configured to receive the transformed data and transmit the transformed data to the processing unit in the received order.
A neural processor and a method for fetching instructions thereof are provided. The neural processor includes a local memory in which weights, input activations, and partial sums are stored, a processing unit configured to compute the weights, the input activations, and the partial sums, and a local memory load unit configured to load the weights, the input activations, and the partial sums from the local memory into the processing unit, wherein the local memory load unit includes an instruction fetch unit configured to fetch instructions included in a program of the local memory load unit for loading any one of the weights, the input activations, or the partial sums from the local memory, and an instruction execution unit configured to generate control signals for executing instructions fetched by the instruction fetch unit.
A neural processor and method for assigning config ID for neural core included in the same are provided. The neural processor includes a core array comprising a first neural core, a second neural core, a first data line connecting the first neural core and the second neural core in series, and a config line connecting the first neural core and the second neural core in series, an ID config manager configured to assign a first config ID to the first neural core and a second config ID to the second neural core via the config line, and a memory configured to input and output data to and from the core array via the first data line.
A method for confidential computing is provided, which is performed by a security core including one or more processor, and includes storing first encrypted data associated with a first tenant in a first memory, in which the first encrypted data is obtained by performing encryption of the first plaintext data using a first encryption key associated with the first tenant, in response to receiving a request to access the first plaintext data, decrypting the first encrypted data using the first encryption key so as to generate the first plaintext data, and providing the first plaintext data to a main core that processes data stored in the first memory.
A neural processing device is provided. The neural processing device includes a plurality of processing engine groups, a first memory shared by the plurality of engine groups, a first interconnection configured to transmit data between the first memory and the plurality of processing engine groups. The neural processing device is configured to provide hardware resource to the plurality of processing engine groups. At least one of the plurality of processing engine groups includes a plurality of processing engines, each of the plurality of processing engines includes an array of a plurality of processing elements interconnected by a mesh style network, the processing elements being reconfigurable. The neural processing device includes a second memory shared by the plurality of processing engines, and a second interconnection configured to transmit data between the second memory and the plurality of processing engines.
A Field Programmable Gate Array (FPGA) system includes a main FPGA and one or more sub-FPGAs connected to the main FPGA. The main FPGA is configured to detect a positive edge of a pulse included in a user clock using a sampling clock of the main FPGA, generate a flag using the detected positive edge, generate a clock packet indicating the generated flag, and provide the generated clock packet to any one of the one or more sub-FPGAs.
A method for measuring performance of neural processing devices and devices for measuring performance are provided. The method for measuring performance of neural processing devices comprises receiving hardware information of a neural processing device, modeling hardware components according to the hardware information as agents, dividing a calculation task by events for the agents and modeling the calculation task, thereby generating an event model which includes nodes corresponding to the agents and edges corresponding to the events and measuring a total duration of the calculation task through simulation of the event model.
Provided are a processing device and a method for secure booting thereof, in which the processing device includes a security core that operates a Root of Trust and sequentially performs an integrity check on first firmware and second firmware through the Root of Trust, a main core that sequentially operates the first firmware and the second firmware, a non-volatile memory storing the first firmware and the second firmware, and a first volatile memory that loads the first firmware and the second firmware from a main core domain of the main core and operates the loaded firmware with the main core.
G06F 21/57 - Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
A neural processing device and a method for managing tasks thereof are provided. The neural processing device includes a neural core configured to perform a task and generate a completion signal for completion of the task, a core global configured to transfer task information for the task to the neural core and receive the completion signal of the task from the neural core, and a task manager configured to generate and transmit the task information to the core global, receive the completion signal from the core global, generate a completion report, and transmit the completion report.
A task manager, a neural processing device, and a method for checking task dependencies thereof are provided. The task manager includes a task buffer configured to receive first and second tasks of different first and second types, a first queue configured to receive a first task descriptor for the first task from the task buffer, a second queue configured to receive a second task descriptor for the second task from the task buffer, a dependency checker configured to check dependencies of the first and second task descriptors, a third queue configured to receive the first task descriptor from the dependency checker, and a fourth queue configured to receive the second task descriptor from the dependency checker.
Provided is a method for converting order book data into 2D data for a machine learning model, which is performed by one or more processors, in which the method includes acquiring order book data including data items for a plurality of bid prices and data items for a plurality of ask prices for a stock traded in a first stock exchange according to time from a time point in the past that is a predetermined period of time earlier, to a current time point, calculating a mid price based on a highest bid price of the plurality of bid prices and a lowest ask price of the plurality of ask prices, and generating data in tensor form having a first axis of times at certain time intervals and a second axis of prices in units of tick.
A processing device comprises a first set of processors comprising a first processor and a second processor, each of which comprises at least one controllable port, a first memory operably coupled to the first set of processors, at least one forward data line configured for one-way transmission of data in a forward direction between the first set of processors, and at least one backward data line configured for one-way transmission of data in a backward direction between the first set of processors. wherein the first set of processors are operably coupled in series via the at least one forward data line and the at least one backward data line.
G06F 15/80 - Architectures of general purpose stored program computers comprising an array of processing units with common control, e.g. single instruction multiple data processors
G06F 13/36 - Handling requests for interconnection or transfer for access to common bus or bus system
G06N 3/063 - Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
G06N 3/084 - Backpropagation, e.g. using gradient descent
95.
Neural processing device and method for transmitting data thereof
A processing device comprises processors, a first memory shared by the processors, and a cache comprising a second memory comprising a plurality of memory units, each of the plurality of memory units in the second memory being associated with a respective one of a plurality of request identifiers. The cache receives a memory read request including a request identifier and a memory address from at least one of the processors, identifies an allocated memory address identifier for the memory address, accesses the first memory to read data of the memory address, obtains one or more request identifiers which requested data of the memory address from the second memory based on the allocated memory address identifier, and transmitting the data of the memory address to one or more processors which requested data of the memory address based on the one or more request identifiers.
The present disclosure discloses a processing element and a neural processing device including the processing element. The processing element includes a weight register configured to store a weight, an input activation register configured to store input activation, a flexible multiplier configured to generate result data by performing a multiplication operation of the weight and the input activation by using a first multiplier of a first precision or using both the first multiplier and a second multiplier of the first precision in response to a calculation mode signal and a saturating adder configured to generate a partial sum by using the result data.
G06F 7/533 - Reduction of the number of iteration steps or stages, e.g. using the Booth algorithm, log-sum, odd-even
G06F 7/544 - Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using non-contact-making devices, e.g. tube, solid state deviceMethods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using unspecified devices for evaluating functions by calculation
97.
Neural processing device, processing element included therein and method for operating various formats of neural processing device
A neural processing device, a processing element included therein and a method for operating various formats of the neural processing device are provided. The neural processing device includes at least one neural processor, a shared memory shared by the at least one neural processor, and a global interconnection configured to transmit data between the at least one neural processor and the shared memory, wherein each of the at least one neural processor comprises at least one processing element, each of the at least one processing element receives an input in a first format and thereby performs an operation, and receives an input in a second format that is different from the first format and thereby performs an operation if a format conversion signal is received, and the first format and the second format have a same number of bits.
A method for high frequency trading is provided, which is performed by one or more processors, and includes calculating a latency for a market order for each of a plurality of candidate batch sizes, selecting a batch size from among the plurality of candidate batch sizes based on the calculated latency, generating input data corresponding to the selected batch size using market data for a target item, using a machine learning model, generating prediction data for the target item at a future time point associated with the selected batch size, based on the generated input data, and generating order data for the target item based on the generated prediction data.
A neural processor and a method for fetching instructions thereof are provided. The neural processor includes a local memory in which weights, input activations, and partial sums are stored, a processing unit configured to compute the weights, the input activations, and the partial sums, and a local memory load unit configured to load the weights, the input activations, and the partial sums from the local memory into the processing unit, wherein the local memory load unit includes an instruction fetch unit configured to fetch instructions included in a program of the local memory load unit for loading any one of the weights, the input activations, or the partial sums from the local memory, and an instruction execution unit configured to generate control signals for executing instructions fetched by the instruction fetch unit.
A neural processing device and a method for job scheduling are provided. The neural processing device configured to receive, by an address space ID (ASID) manager, first and second requests from at least one context, respectively, and determine whether ASIDs are allocated, store jobs of contexts to which the ASIDs have not been allocated from the ASID manager in entities, schedule, by a job scheduler, an execution order of the jobs stored in the entities and cause the ASID manager to allocate the ASIDs to the contexts to which the ASIDs have not been allocated among the at least one context, and sequentially receive, by a command queue, jobs of contexts to which the ASIDs have been allocated, store the jobs as standby jobs, and sequentially execute the standby jobs.
