Is this your business? Claim it to manage your IP and profile
The ONNC Calibrator is engineered to ensure high precision in AI System-on-Chips using post-training quantization (PTQ) techniques. This tool enables architecture-aware quantization, which helps maintain 99.99% precision even with fixed-point architecture, such as INT8. Designed for diverse heterogeneous multicore setups, it supports multiple engines within a single chip architecture and employs rich entropy calculation techniques. A major advantage of the ONNC Calibrator is its efficiency; it significantly reduces the time required for quantization, taking only seconds to process standard computer vision models. Unlike re-training methods, PTQ is non-intrusive, maintains network topology, and adapts based on input distribution to provide quick and precise quantization suitable for modern neural network frameworks such as ONNX and TensorFlow. Furthermore, the Calibrator's internal precision simulator uses hardware control registers to maintain precision, demonstrating less than 1% precision drop in most computer vision models. It adapts flexibly to various hardware through its architecture-aware algorithms, making it a powerful tool for maintaining the high performance of AI systems.
The ONNC Compiler is a sophisticated tool designed for AI-on-chip implementations, facilitating the transformation of neural networks into machine-specific instructions. Its architecture is particularly advantageous for heterogeneous multicore SoCs, accommodating configurations like big.LITTLE ARM and various DSPs. The compiler supports a modular parser that leverages MLIR frameworks, facilitating support for popular deep learning frameworks like PyTorch and TensorFlow. It includes both single and multiple backend modes, ensuring adaptability across broad AI system-on-chip architectures, including support for PCIe accelerators and application processors in smartphones. One of the standout features of ONNC is its capability to manage fragmented memory spaces, allowing efficient data flow and optimization across complex systems. Its design addresses intricate memory configurations with non-linear, often fragmented memory spaces, and it offers a high-dimensional memory allocation system that minimizes RAM usage. ONNC enhances performance by optimizing data movement with techniques such as software pipelining and DMA allocation, ultimately aiming to maximize processing element utilization and reduce memory overhead. The ONNC Compiler has been engineered to be modular and retargetable, allowing it to cater to diverse hardware architectures and optimize both performance and resource use. Key optimizations such as software pipelining, DMA scheduling, and memory management support are built into its backend, making ONNC an attractive choice for those looking to maximize efficiency in AI system design.
Forest Runtime gives developers the flexibility to execute compiled neural networks across various hardware platforms, providing a seamless interface with its C++, C, and Python APIs. The system is designed with modular architecture that caters to datacenter, mobile, and TinyML applications, offering retargetability which simplifies integration across different hardware. Forest Runtime stands out with its 'hot batching' technology, which allows for dynamic model partitioning, enhancing throughput and minimizing response times without runtime compilation transformations. It is engineered to be highly scalable with features allowing merging of models to reduce synchronization load between CPUs and NPUs, making it adaptable for data centers and mobile devices. Its advanced design allows for flexibility in application scales and contexts, employing model fusion techniques and context switching to optimize resource use across systems with multiple accelerator cards. The setup exhibits robust scalability by leveraging software pipelining to enhance performance and manage memory allocation efficiently.
EdgeThought represents a shift in deploying large language models (LLMs) at the device level, optimizing for high performance and cost-effectiveness. This innovative solution combines extensive memory bandwidth with deterministic response times, tailored for decoding high-capacity transformer models such as LLaMA and Mistral. The unique architecture offers programmability and model flexibility, enabling efficient deployment across various edge environments. EdgeThought supports a partition of 7-13 billion parameter models, requiring minimal MAC operations while leveraging modular instruction sets for extensive coverage of transformer models. It significantly reduces response times, making EdgeThought well-suited for applications demanding rapid processing speeds and high data throughput. This system's ecosystem readiness is underscored by its compatibility with prominent LLM frameworks such as HuggingFace Transformers and Nvidia Triton Inference Server. By integrating fine-tuning and retrieval-augmented generation tools, EdgeThought fosters seamless adaptation and enrichment of AI applications across diverse hardware infrastructures, from data centers to edge computing devices.
Join the world's most advanced semiconductor IP marketplace!
It's free, and you'll get all the tools you need to discover IP, meet vendors and manage your IP workflow!
To evaluate IP you need to be logged into a buyer profile. Select a profile below, or create a new buyer profile for your company.
