What is SIMD? The Basic Concept

SIMD (Single Instruction, Multiple Data) is a type of parallel data processing in which a single instruction is executed simultaneously on multiple data elements.

A simple analogy: Imagine you have 4 pairs of numbers, and you need to add each pair.

• Scalar approach (SISD): You take the first pair (A1, B1), execute the "add" command and get C1. Then you take the second pair (A2, B2), execute "add" again and get C2, and so on. 4 instructions for 4 operations.
• Efficiency: The text emphasizes that SIMD can provide an "order of magnitude increase in efficiency" (work per instruction) compared to scalar code.
• SIMD approach: You load all 4 pairs of numbers into a special wide register. Then you execute ONE command "add all pairs simultaneously". As a result, you immediately get 4 results (C1, C2, C3, C4). 1 instruction for 4 operations.

This allows for significant acceleration in tasks where it is necessary to apply the same operation to a large array of data.

1. Key Characteristics of SIMD

• Data Level Parallelism: SIMD does not create parallel command threads (like in multithreading), but uses the parallelism of the data itself. All processed elements go through the same operation "pipeline".

Let's go into more detail if it's not clear ⬇️⬇️⬇️

Data Parallelism (SIMD) vs Task Parallelism (Multithreading)

TASK Parallelism: Different workers do DIFFERENT things with different data. Example: 4 chefs cook 4 different dishes.

DATA Parallelism: One boss gives ONE command, which everyone executes SIMULTANEOUSLY on their own data. Example: Coach to the team: "EVERYONE squat!" — 20 people squat at once.

The Essence: SIMD is not "divide and conquer", but "do everything at once".

• Not to be confused with ISA: SIMD is an architectural principle that can be implemented as an instruction set (like SSE or AVX), but it is not the entire instruction set architecture as a whole.

What is ISA? ⬇️⬇️⬇️

ISA (Instruction Set Architecture) is the language of communication between the processor and the programmer.

In a nutshell: ISA is the list of commands that the processor understands.

The Essence in Three Points:

1. "Contract" or "Interface" ISA is not the processor itself, but its "instruction manual". It says: "Here are the commands (addition, copy, branch) I understand and here are the fast memory cells (registers) I have".

2. Separates "What" from "How"

   • ISA defines WHAT the processor can do (which set of commands is available).
   • Microarchitecture defines HOW it does it (the internal design, which can be different for different manufacturers).

   Example: Intel and AMD processors have the same ISA (x86), but different internal design.

3. Compatibility Guarantee Thanks to ISA, the same program will work on any processor that understands this "language", even if these processors are made by different companies or have different performance.

Analogy: ISA is like traffic rules. All drivers (programs) know that a "Stop" sign (processor command) means to stop. It doesn't matter which exact car (microarchitecture) you are driving - Mercedes or BMW - the rule is the same.

but, let's see how exactly ISA defines something there

ISA = Table of Operation Codes (OPcode)

OPcode is a number that tells the processor WHAT TO DO

(simplified, figuratively)

How it works:

The processor sees 001000 → looks up the ISA table → understands: "need to add a register with a number"

ISA is a dictionary:

• 001000 = "addi"
• 000000 = "add"
• 100011 = "lw"

let's say there is also this

The processor doesn't think — it simply executes actions according to a pre-defined table (ISA)

2. Hardware

SIMD is implemented in most modern processors in the form of special execution units and registers.

• Registers: These are special, very wide registers that can store multiple values simultaneously.
  • Example: A 128-bit SSE register can store four 32-bit floating-point numbers or sixteen 8-bit integers.
  • Modern standards, such as AVX-512, use 512-bit registers, allowing work with 16 floating-point numbers simultaneously.
• Examples of SIMD extensions:
  • Intel: MMX, SSE, SSE2, SSE3, AVX, AVX2, AVX-512.
  • AMD: 3DNow!, and now also supports AVX.
  • ARM: NEON (very common in mobile processors), Scalable Vector Extension (SVE).
  • PowerPC/IBM: AltiVec.
• Other devices: SIMD is actively used in GPUs (graphics processing units), specialized processors for video processing (for example, the Cell processor in PlayStation 3) and AI accelerators (for example, the Neural Engine in Apple chips).

3. Software / Programmer Interface

A programmer does not need to write code in assembly to use SIMD. There are more convenient ways:

• Intrinsics: These are special functions in C/C++ languages that compile directly into SIMD instructions. They look like ordinary functions but give the programmer low-level control.
  • Example: _mm_add_ps() for adding four floating-point numbers in SSE.
• Auto-vectorization: Modern compilers (GCC, Clang, MSVC, ICC) can automatically analyze loops and transform scalar code into SIMD code. Compilation flags are often used for this, for example, -O3 or -ftree-vectorize.
• Language Extensions and Libraries:
  • In C/C++ there is an experimental header <std::simd>.
  • In Rust there is the packed_simd crate.
  • In .NET there is the System.Numerics.Vector namespace.
  • OpenMP provides the #pragma omp simd directive to hint to the compiler.

4. Multi-versioning (SIMD Multi-versioning)

Since different generations of SIMD instructions exist, a program must work on different processors. For this, a method is used where several versions of the same code are created:

• FMV (Function Multi-versioning): The same function is compiled multiple times for different instruction sets (for example, one version for SSE4, another for AVX2). During runtime, the program checks the CPU capabilities and selects the appropriate version.
• LMV (Library Multi-versioning): An entire library is compiled in several variants, and the operating system or program loads the required version.

5. Disadvantages

• Not for all tasks: SIMD is effective only for tasks with a high degree of data parallelism. Algorithms with strong dependencies between steps (for example, traversing a linked list) are poorly vectorized.
• Complexity of manual optimization: Although compilers handle auto-vectorization well, achieving maximum performance often requires manual tuning using intrinsics, which requires deep knowledge (but this is a plus, not a minus).

6. Where is it used? (Commercial Applications)

SIMD is critically important for tasks requiring intensive computations on large data arrays:

• Image and Video Processing: Changing brightness, contrast, applying filters, encoding/decoding (MPEG, H.264).
• Audio Processing: Mixing, applying effects, equalizers.
• Scientific Computing: Numerical modeling, matrix and vector calculations.
• Games: 3D vertex transformations, physics calculations, artificial intelligence.
• Cryptography: Many encryption and hashing algorithms can be accelerated using SIMD.
• Artificial Intelligence: Matrix multiplication and convolutions in neural networks.

7. Confusion with SIMT

• SIMD: A single instruction controls multiple processing elements in a CPU.
• SIMT (Single Instruction, Multiple Threads): An architecture used in GPUs (NVIDIA). Here, a single instruction is also executed on multiple data elements, but it is delivered to multiple independent threads, which can have their own execution path (for example, conditional if/else statements are handled differently). The text from Wikipedia clarifies that SIMT is a subcategory of SIMD according to Flynn's taxonomy.

Summary

SIMD is a fundamental acceleration technology that has become an integral part of modern processors. It allows for a significant increase in performance in multimedia, scientific, and engineering tasks by parallelizing the execution of the same operation on a set of data. Despite some programming complexities, thanks to automatic vectorization in compilers and convenient libraries, the benefits of SIMD are becoming available to a wider range of developers.