A4.1.2 — Hardware Requirements

Key Hardware Components

GPU (Graphics Processing Unit): Originally for graphics rendering. Highly effective for ML because of its parallel processing capability. Can perform thousands of calculations simultaneously. Essential for training deep learning models.
TPU (Tensor Processing Unit): Google's custom ASIC designed specifically for neural network workloads. Accelerates tensor calculations. Optimised for TensorFlow. Used in large-scale cloud environments.
ASIC (Application-Specific Integrated Circuit): Custom chip designed for one specific task. High speed and low power consumption. Used in mobile devices and embedded systems where efficiency is critical.
FPGA (Field-Programmable Gate Array): Semiconductor device that can be reprogrammed after manufacture. Used where flexibility is required. Accelerates specific ML inference workloads and data flow processing.
Edge devices: Hardware that processes data near the source rather than sending it to a central server. Often uses ASICs or small GPUs. Used for real-time applications requiring low latency (facial recognition, autonomous vehicles).
Cloud-based platforms: Virtualised, scalable resources on demand including CPUs, GPUs, and TPUs. No physical infrastructure to maintain. Ideal for variable workloads, experimental projects, and bursty data loads.

Scenario	Hardware	Key reason
Development and testing	Standard laptop or desktop. Multi-core CPU. 8 to 16 GB RAM.	Small models and datasets. No heavy compute needed.
Data processing and feature engineering	High-performance workstation. Powerful CPU. Large SSD (1 TB+) or RAID.	Larger datasets require fast storage and processing. More complex transformations.
Model training and deep learning	Dedicated GPU server. High-end GPU(s). High-capacity SSD.	Deep networks require parallel computation. GPUs handle thousands of matrix operations simultaneously.
Large-scale deployment and production	High-end servers or cloud. Scalable CPUs and GPUs/TPUs. Enterprise distributed storage.	Must handle continuous high load, real-time processing, and scale with demand.
Edge computing	Compact energy-efficient device. ARM CPU. Small GPU or FPGA/ASIC. Flash storage.	Must operate locally with low latency, limited power, and without cloud connectivity.

Hardware	Best for
Standard laptop or desktop	Beginners, students, small projects. Basic models and small datasets.
High-performance workstation	Professional developers. Complex models that do not yet need a full server setup.
GPU-enhanced system	Deep learning and large-scale ML. Complex neural networks with massive datasets.
Dedicated AI server	Industry and academia. High-demand, continuous ML tasks. May include multiple GPUs or TPUs and up to 1 TB RAM.
Edge device	Consumer devices and IoT. Real-time inference at the network edge. Low power, limited storage.

Ninja Notes:

GPUs are used for training because ML involves massive parallel matrix multiplication. A CPU does this sequentially. A GPU does thousands at once.
TPUs are like GPUs but purpose-built for tensor operations. Faster and more efficient for TensorFlow workloads.
ASICs and FPGAs are used at the edge because they are small, efficient, and fast. ASICs are fixed. FPGAs can be reprogrammed.
Edge devices sacrifice raw power for low latency and low energy. They run pre-trained models locally, not training from scratch.