Reduced heat generation

Fewer errors

Faster processing speed

Simplified programming

A part of the motherboard that connects to the CPU

A single processing unit that can independently execute tasks

The storage capacity of the CPU

The cooling system for the processor

Symmetric Multi-Processing

Single Memory Processing

Serial Multi-Processing

Shared Memory Protocol

Concurrency involves tasks running at the exact same time on multiple processors, while parallel processing involves switching between tasks on a single processor.

Concurrency involves managing multiple tasks that are in progress, but not necessarily running at the same time, while parallel processing involves tasks being executed simultaneously on multiple processors.

Concurrency always requires multiple processors, whereas parallel processing only uses a single processor.

Concurrency and parallel processing are two terms for the same concept.

The ability to perform multiple tasks at the exact same time on different processors

The execution of a single task using multiple processors

The use of a single processor to execute a single task at a time

The ability to start, pause, and resume multiple tasks without necessarily performing them at the same time

Multi-Processor Program

Massively-Programmed Process

Memory Portion Parallelism

Massively Parallel Processing

Non-Uniform Multi-Application

New Under Machine Architecture

Non-Uniform Memory Access

Network-United Memory Arithmetic

Integrated Program Computing

Internal Parallel Communication

Interrupt Processing Controller

Inter-Process Communication

NUM

UMA

GPU

Cluster

Memory access speed

Code scalability

Task serialization

Power consumption

Each processor runs its own operating system.

Tasks are divided based on processor load and availability.

Each processor executes the same task in parallel.

Tasks are distributed randomly.

A single processor can handle multiple threads.

Processors share a common memory space.

Each processor has its own operating system.

SMP systems cannot scale beyond two processors.

High memory latency

Increased power consumption

Complexity in load balancing

Limited scalability due to shared memory contention

All processors access memory at the same speed.

Access time depends on the location of the memory relative to the processor.

Memory access is only through a shared bus.

Each processor has an identical memory access speed.

Reducing the number of processors

Localizing memory access by allocating memory near the processing unit

Using a single shared memory space

Eliminating memory caching

Memory access is uniform across processors.

Balancing memory allocation to minimize latency is complex.

NUMA systems cannot run multithreaded applications.

It is impossible to predict memory access patterns.

To create a single high-performance computer

To provide redundancy and fault tolerance

To improve the performance of single-threaded applications

To distribute workloads across multiple servers

USB connections

Shared memory

High-speed networks (e.g., Ethernet, InfiniBand)

Wireless connections

They are less expensive than single supercomputers.

They have lower energy consumption.

They provide unlimited scalability with no performance degradation.

They are easier to maintain than traditional systems.

CPUs have more cores than GPUs.

GPUs are optimized for sequential processing, while CPUs are optimized for parallel processing.

GPUs have a higher number of simpler cores designed for parallel tasks.

CPUs can only perform arithmetic operations, while GPUs can perform logic operations.

MPI (Message Passing Interface)

OpenMP (Open Multi-Processing)

CUDA (Compute Unified Device Architecture)

POSIX Threads

Lack of available memory

Difficulty in handling branching and complex control logic

Inefficient parallelism

High power consumption

Data dependency and synchronization issues

High clock speeds

Simplified algorithms

Reduced memory usage

The maximum speedup of a process is limited by the sequential portion of the task.

It describes the efficiency of a CPU's cache memory.

It indicates the relationship between clock speed and performance.

It evaluates the scalability of memory in a computing system.

Dividing tasks evenly among processors to avoid bottlenecks

The process of optimizing memory access

Minimizing energy consumption across processors

Ensuring that all processors run at the same clock speed

The speed of data transfer between processors

The time delay before a transfer of data begins following an instruction

The amount of data that can be processed simultaneously

The total time taken to complete a task

To simplify code

To utilize multiple processors to improve performance

To reduce the cost of hardware

To increase the size of data sets