Course Details

1. Classify parallel architectures based on the Flynn’s classification and propose other parameters that are essential for the classification of modern parallel processing systems.
2. Justify the need and describe the methodologies employed for synchronization and memory consistency and cache coherence in shared memory systems.
3. Describe and compare the different types of interconnects employed in parallel processing systems.
4. Outline and analyse the features of microarchitecture parallel systems such as superscalar, VLIW, vector, multithreading, CMP multi-core and tile processors.
5. Describe how the performance of a parallel system can be measured, list possible sources for performance losses and propose ways to improve the performance of a system.
6. Write efficient programs for message passing and shared memory parallel processing platforms using MPI and OpenMP.
7. Outline the present developments in the field of parallel processing and show familiarity with the forefront of the relevant knowledge.

Introduction to Parallel Processing: Historic evolution of parallel processing. Motivation for parallel processing. Parallel processing applications case studies.

Parallel Computer Models and Systems: Classification of parallel computer architectures, SIMD and MIMD systems. Shared address space, message passing and data parallel processing. Networks of Workstations, clusters, SANs, MPP.

Performance Metrics: Workloads and benchmarks, execution time, throughput, speedup and efficiency. Latency tolerance techniques, load balancing and data locality issues.

Interconnection Networks: Communication performance, latency and bandwidth. Interconnection organization, links, switches and interconnection topologies. Switch design, routing and flow control. High speed LANs.

Shared Memory Multiprocessors: The cache coherence problem, snoop-based and directory based cache coherence. Memory consistency and replication. Synchronization mechanisms, mutual exclusion, event and global synchronization.

Microarchitecture Parallelism: Levels of parallelism, DLP, ILP, TLP. The temperature and power wall problem, clock skewing and soft errors.  Single-chip parallel processing architectures such as multithreaded processors, CMP and multi-core processors, and tile architectures. Networks-on-Chip, Heterogeneous architectures.

Parallel Programming: The parallelization process, decomposition, assignment, orchestration and load balancing. Message passing programming using MPI, basic instructions, point to point communication, collective communication and barrier Synchronization. Shared memory programming using OpenMP, parallel regions, worksharing, data environment, and synchronization.

• M. Quinn, "Parallel Programming in C with MPI and OpenMP", McGraw Hill, 2004

• D. Culler, P. Singh, A. Gupta, Parallel Computer Architecture: A Hardware Software Approach, Morgan Kufmann, 1999

The taught part of course is delivered to the students by means of lectures, conducted with the help of both computer presentations and traditional means. Practical examples and exercises are included in the lectures to enhance the material learning process. Lecture notes and presentations are available through the web for students to use in combination with the textbooks.

Lectures are supplemented with laboratory work curried out on parallel programming. During laboratory sessions, students develop, test , debug programs in C using OpenMP and MPI.