• Top AI Single Board Computers for 2024

    Top AI Single Board Computers for 2024

    AI support in Single Boards computers is getting more popular. But what are the real options? And how do you get value for your money to support your TensorFlow, PyTorch and other workflows?

    We’re going over the boards with the best AI support, starting with the top board which is a winner in our eyes: the Raspberry Pi 5.

    Winner - Raspberry Pi 5 with AI HAT

    13TOPS at $11.5 per TOPS

    Image Courtesy of Raspberry Pi

     
    Despite being one of the top selling SBCs, the Raspberry Pi never had much of AI support. With the new Raspberry Pi 5, there’s now support for the Hailo-8L M.2 AI accelerator. This provides 13TOPS of performance, which is some of the highest performance seen, topping most other AI capable SBCs. The board will set you back $70 (for both the M.2 adapter and the Hailo-8L itself), making the total around $150 for an AI capable device.
     

    Orange Pi 5 Pro

    6TOPS at $16.1 per TOPS

    Image Courtesy of Orange Pi

     
    Based on the Rockchip RK3588, Orange Pi 5 Pro is has 6TOPS of performance at a cost of $97, meaning you’re paying $16 per TOPS. This is one of the better prices, and it helps that the Orange Pi 5 Pro comes in a Raspberry Pi form factor, with a lot of connectivity and a high performance eight core processor.
     

    BeagleV-Ahead

    4TOPS at $18.1 per TOPS

    Image Courtesy of Beagle Board

     
    Based on the Alibaba TH1520, the BeagleV-Ahead has 4TOPS(INT8) of performance at a cost of $148.75 meaning you’re paying $37 per TOPS. This is the highest price per TOPS we’ve seen in our analysis, as well as the fact the performance is for Integers, as opposed to float or double means it’s an even worse choice
     

    BeagleBone AI-64

    8TOPS at $23.4 per TOPS

    Image Courtesy of Beagle Board

     
    As its name indicates, the BeagleBone AI-64 is an AI focused board with 8TOPS (at 8 bits) along with DSPs and some other interesting capabilities that enable its use in high end drones, robotics and other applications due to its image processing capabilities.  It’s on the pricier end costing around $187. Another downside is that it has a large heatsink on the bottom, although most high end AI boards will have even active cooling, like the LattePanda Delta 3.
     

    BeagleY AI

    4TOPS at $18.1 per TOPS

    Image Courtesy of Beagle Board

     
    Focused on TI’s AM67A with 4TOPS of performance, the BeagleY is a great balance between performance and cost. You only pay $72.54 and get 4TOPs. However, the per TOPS cost is around $18.1 which is higher than other options. 
    We have a review of the BeagleY AI board so you can see whether the other features in this board make sense.
     

    NVIDIA Jetson Nano Dev Kit

    40TOPS at $12.5 per TOPS

    Image Courtesy of NVIDIA

     

    There’s no doubt that NVIDIA is the king of AI. Its H100 GPU is the backbone of AI, so it’s not surprising NVIDIA offers SBCs.  The NVIDIA Jetson Nano Dev Kit which we previously reviewed only had 472 GFLOPS which is 0.472TOPS (assuming floating operations). The NVIDIA Jetson Orin Nano takes it to the next level by supporting 20TOPs (4GB Version) and 40TOPs (8GB Version). The downside is it costs $500, so unless you’re looking for the top of the line, it is an expensive way to get into AI as you’re paying $12.5 per 1 TOPs.
     

    The original SBCs all put a huge focus on the processing power of the CPU, typically an ARM processor. But as AI applications have become popular, we’ve started to see Neural Processing units being added. Those units are important to accelerate any AI workload you may want to run.

    Why AI Specific Accelerators?

    You can run AI on a general purpose processor (CPU). And the faster that CPU, the more performance you’ll get. Of course that means that you’ll also dissipate a lot more heat and consume power. However, it’s even worse. a generic CPU can significantly underperform a dedicated accelerator, by a huge amount. How bad is it?

    Here’s a benchmark for the ResNet-50 v1 comparing the Coral TPU that’s attached via PCIe and provides about 8TOPS of performances vs a $1200 (or so) Intel Xeon Gold 6154. The Coral TPU outperforms the Intel Xeon Gold 6154 by 10x and costs 48x less (we’ve seen even higher price points). 

    What’s more, the Coral TPU consumes a few watts of power at most as opposed to the power of the Intel Xeon processor. Clearly, you want a dedicated accelerator if you’re running neural networks and AI workloads.