By Jarrod Piper / 5 Apr 2024 / Topics: Generative AI Devices Device lifecycle Data and AI
AI and generative AI solutions are emerging everywhere. As a result, Original Equipment Manufacturers (OEMs) are beginning to rethink how devices are designed to support the intense workloads required by these technologies. In fact, the projected demand for this new generation of PCs is so great, IDC forecasts AI PCs will account for nearly 60% of all PC shipments by 2027.
If you procured devices in 2020 (during the mass transition to remote work) or have devices that won’t support an upgrade to Windows 11, now is the time to plan a device refresh and get ahead of the demand curve. Making smart, future-forward purchasing decisions begins with understanding the nuances of emerging AI devices — what are the differences between AI-capable and AI-ready? What is a next-gen AI device? As intelligent tech continues transforming how we live and work, these enhanced devices will become essential to driving innovation and growth.
A next-gen, on-device AI is capable of performing AI tasks locally, minimizing server or cloud connectivity needs. Data can be more efficiently processed, with less latency and without compromising privacy.
Technological advancement in the processor is made possible by using specialized hardware components like Neural Processing Units (NPUs) designed to accelerate AI workloads.
One of the most essential factors in developing and evaluating next-gen AI devices is measuring their performance, which is where Trillions or Tera Operations Per Second (TOPS) comes in. TOPS scores indicate how many computing operations a processing unit can perform in one second. The higher the TOPS value, the faster the device can perform AI tasks.
While CPUs are general-purpose processors that can handle a wide range of tasks, they may not be optimized for AI workloads. GPUs are designed for parallel processing tasks, making them slightly more suitable for AI tasks. Still, they may not be as power-efficient or specialized as NPUs.
The NPU is a unique discrete or integrated processor component explicitly designed for executing machine learning algorithms. NPUs are optimized to perform the matrix calculations required for deep learning algorithms, providing a faster execution of complex AI operations. As generative AI becomes pervasive in software, devices built with NPUs will be critical to running applications at their best.
In the processing realm, ARM (Advanced RISC Machine) technology will also play a critical role in developing next-gen AI devices (although not all next-gen AI PCs will use ARM technology). With highly efficient and scalable designs, ARM processors have lower power consumption levels, making them ideal for mobile and IoT devices.
To understand the nuances, let's examine the most basic category: devices considered "AI-capable." These have been on the market for nearly a decade and can produce up to 10 TOPS with AI accelerators on the CPU and GPU. They can run on-device AI workloads with lower efficiency, making them an experimental choice for simple AI tasks.
Moving up a level, "AI-ready" devices have recently been released to incorporate the first generation of AI processing and are called different things depending on the manufacturer. These devices can produce up to 30 TOPS and were available in the previous and current processor generations at the high end. They offer a significant performance boost compared to "AI-capable" devices and are acceptable for more complex AI workloads.
Finally, we have "next-gen" devices. These integrate advanced technology and components, including a dedicated, next-gen NPU, to deliver unmatched machine performance. With combined processing speeds exceeding 40 TOPS, these devices are a game-changer for end users by executing some AI workloads local to the device. They offer faster response times, enhanced productivity and a superior user experience, making them differentiators for your modern IT environment.
The value of next-gen AI lies in its capabilities for enhancing productivity, device speed and operational efficiency at scale. While traditional hardware obsolescence and end-of-life support have influenced device investments, the lifecycle refresh is now driven by the deeper breadth of capabilities associated with on-device AI.
Next-gen AI PCs offer a host of benefits, including:
It's becoming increasingly clear that older devices may struggle to keep up with the modern demands of AI workloads. Slower chips and reduced efficiency will limit AI productivity, hindering an organization’s success in this competitive, AI-driven technology environment.
Vice President of IT at Insight Matt Skaff suggests, “If you really think gen AI is going to take off — and all indications are that it is — then you need to future-proof your organization. You don't want to be sitting on the last month of the year trying to figure out Y2K and changing all your code, right? You want to be ahead of it.”
With AI devices, organizations can use AI to its full extent and gain an edge over the competition. This new wave of super-fast device processing presents a timely opportunity for organizations to position themselves at the forefront of innovation and achieve unprecedented levels of performance and differentiation.
By updating your current device fleet and adding next-gen AI to your long-term plan, you can benefit from the latest AI breakthroughs and future-proof your environment. Need more assistance with your device update? Insight can help you pick the best devices for your user requirements and simplify your device strategy.
Brand Marketing Specialist, Insight
Jarrod is a results-driven brand marketing specialist with a wealth of experience in campaign management, behavioral analysis and business development. He has a proven track record of executing successful marketing initiatives that engage audiences and drive business growth. Over the years, he has also provided strategic direction for cybersecurity and related IT disciplines for organizations across diverse industries.
