Programming Multi-core and Many-core Systems (2018)


Course code 5284PMCM6Y
Format Lecture course with lab sessions and workshops
Curriculum MSc Computer Science (joint degree UvA/VU)
Block 2a (Feb 5 - Mar 30, 2018)
Status Compulsory for track Parallel Computing Systems
Constrained choice Programming
ECTS 6 credits
Language English
Study Guide Click here.
Objectives

To develop an understanding of the opportunities, challenges and limits of parallel computing and to gain practical familiarity with state-of-the-art programming models for contemporary concurrent multi-core and many-core computing systems.

Contents

The course provides a comprehensive introduction into state-of-the-art programming models for concurrent computing systems from multi-core processors in everyday laptops to large-scale server systems and high-end accelerators.

We start with instruction-level parallelism and vectorisation. Then we continue with multithreaded programming models for shared address space systems, where we look both into OpenMP compiler directives and into more low-level Posix threads before we discuss advanced topics and common pitfalls of shared memory parallel programming.

In the second half of the course we focus on general-purpose graphics accelerators (GPGPUs) using NVidia's programming model CUDA and end with advanced topics such as directive-based GPU programming, Intel Xeon Phi programming and cross-architecture programming using the standard-driven OpenCL programming model.

The lectures are complemented by labs where participants gain first-hand experience with the various programming models and by group discussion workshops (werkcolleges) where participants present their work and discuss their achievements with each other as well as with the lecturers and lab assistants.

The course is complementary to the VU courses Programming Large-scale Parallel Systems and the corresponding project course in that it looks into node-level concurrency, whereas the VU courses focus on systems that are made up of many nodes.

Lecturers Dr Clemens Grelck (ccordinator)
Dr Ana Varbanescu
Assistant Giulio Stramondo
Lectures Tue, 13-15, D1.116
Fri, 09-11, A1.24
Labs Tue, 15-17, D1.113
Fri, 11-13, A1.24
Workshops Fri, Feb 16, 11-13, A1.24
Fri, Mar 02, 11-13, A1.24
Fri, Mar 16, 11-13, A1.24
Participation in workshops is compulsory!!
Exam Tue, Mar 27, 15-17, G3.02
Grading We have 4 bi-weekly assignments and an exam, each counts for 20% of the final grade. Submission of assignments is in groups of two. The (individual) exam must be passed with a minimum grade of 5.0.
If we have reason to believe that the workload and contributions in a group are not somewhat equal, we reserve the right to conduct individual interviews.
Slides Will be made available on Blackboard after each lecture.
Contents
  • Chapter 1: Introduction and Motivation
  • Chapter 2: SIMD Parallelism and Vectorisation
  • Chapter 3: OpenMP Compiler Directives
  • Chapter 4: Multithreading with PThreads
  • Chapter 5: Common Pitfalls in Shared Memory Parallel Processing
  • Chapter 6: GPGPU Programming with CUDA
  • Chapter 7: GPGPU Programming with OpenCL and OpenACC
  • Chapter 8: Heterogeneous Computing
  • Chapter 9: Performance Metrics and Models
Assignments The assignments will be made available on Blackboard.
Mailing List The UvA mailing list PMMS2018 serves as the primary medium of communication between lecturers and participants as well as among participants themselves. Subscription to the mailing list is controlled and restricted to participants of the course. Traffic on the mailing list is archived; access to the archives is restricted to participants. Please, subscribe here.
Literature and Resources The course does not follow any specific text book. The following list summarises a number of interesting textbooks on various aspects of the course.

Parallel Programming in General
  • Kevin Dowd, Charles Severance: High Performance Computing, O'Reilly, 1998.
  • Ananth Grama, Anshul Gupta, George Karypis, Vipin Kumar: Introduction to Parallel Computing, Addison-Wesley, 2003.
  • Gregory R. Andrews: Foundations of Multithreaded, Parallel, and Distributed Programming, Addison Wesley, 2000.
  • Michael J. Quinn: Parallel Programming in C with MPI and OpenMP, McGraw-Hill, 2003.

OpenMP
  • Rohit Chandra, Leo Dagum, Dave Kohr, Dror Maydan, Jeff McDonald, Ramesh Menon: Parallel Programming in OpenMP, Morgan Kaufmann, 2000.
  • Barbara Chapman, Gabriele Jost, Ruud van der Pas: Using OpenMP, MIT Press, 2007.
  • www.openmp.org

OpenACC
NVidia CUDA

Valid HTML 4.01!     Valid CSS!             Dr Clemens Grelck