Programming Multi-core and Many-core Systems (2020)


Course code 5284PMCM6Y
Format Lecture course with lab sessions and workshops
Curriculum MSc Computer Science (joint degree UvA/VU)
Block 4
Status Compulsory for track Parallel Computing Systems
Constrained choice Programming
ECTS 6 credits
Language English
Study Guide Click here.
Objectives

To develop an understanding of the opportunities, challenges and limits of parallel computing and to gain practical familiarity with state-of-the-art programming models for contemporary concurrent multi-core and many-core computing systems.

Contents

The course provides a comprehensive introduction into state-of-the-art programming models for concurrent computing systems from multi-core processors in everyday laptops to large-scale server systems and high-end accelerators.

We start with instruction-level parallelism and vectorisation. Then we continue with multithreaded programming models for shared address space systems, where we look both into OpenMP compiler directives and into more low-level Posix threads before we discuss advanced topics and common pitfalls of shared memory parallel programming.

In the second half of the course we focus on general-purpose graphics accelerators (GPGPUs) using NVidia's programming model CUDA and end with advanced topics such as directive-based GPU programming, Intel Xeon Phi programming and cross-architecture programming using the standard-driven OpenCL programming model.

The lectures are complemented by labs where participants gain first-hand experience with the various programming models and by group discussion workshops (werkcolleges) where participants present their work and discuss their achievements with each other as well as with the lecturers and lab assistants.

The course is complementary to the VU courses Programming Large-scale Parallel Systems and the corresponding project course in that it looks into node-level concurrency, whereas the VU courses focus on systems that are made up of many nodes.

Lecturers Dr Clemens Grelck (ccordinator)
Dr Ana Varbanescu
Assistants Julius Roeder
Misha Mesarcik
Lectures Mon, 13-15
Thu, 09-11
See Datanose for all details and exceptions.
Labs Mon, 15-17
Thu, 11-13
See Datanose for all details and exceptions.
Workshops Thu, Feb 11, 11-13
Thu, Feb 25, 11-13
Thu, Mar 11, 11-13
Thu, Mar 25, 11-13
Participation in workshops is compulsory!!
Exam Mon, Mar 22, 09-11
Grading Four bi-weekly assignments and an exam count for 20% each. Submission of assignments is in groups of two. The (individual) exam must be passed with a minimum grade of 5.0.
If we have reason to believe that the workload and contributions in a group are not reasonably balanced, we reserve the right to conduct individual interviews.
Slides Will be made available on Canvas after each lecture.
Contents
  • Introduction and Motivation
  • SIMD Parallelism and Vectorisation
  • OpenMP Compiler Directives
  • Common Pitfalls in Shared Memory Parallel Processing
  • Multithreading with PThreads
  • Performance Metrics and Models
  • GPGPU Programming with CUDA
  • GPGPU Programming with OpenCL and OpenACC
  • Heterogeneous Computing
Assignments Will be made available on Canvas.
Mailing List The UvA mailing list PMMS2021-Team serves as a convenient way to reach all members of the teaching team at once. Participants of the course are permitted to post to this mailing list, but are not subscribed to it.
Slack For Everyday communication we use the Slack workspace UvA-PMMS2021.
Literature and Resources The course does not follow any specific text book. The following list summarises a number of interesting textbooks on various aspects of the course.

Parallel Programming in General
  • Kevin Dowd, Charles Severance: High Performance Computing, O'Reilly, 1998.
  • Ananth Grama, Anshul Gupta, George Karypis, Vipin Kumar: Introduction to Parallel Computing, Addison-Wesley, 2003.
  • Gregory R. Andrews: Foundations of Multithreaded, Parallel, and Distributed Programming, Addison Wesley, 2000.
  • Michael J. Quinn: Parallel Programming in C with MPI and OpenMP, McGraw-Hill, 2003.

OpenMP
  • Rohit Chandra, Leo Dagum, Dave Kohr, Dror Maydan, Jeff McDonald, Ramesh Menon: Parallel Programming in OpenMP, Morgan Kaufmann, 2000.
  • Barbara Chapman, Gabriele Jost, Ruud van der Pas: Using OpenMP, MIT Press, 2007.
  • www.openmp.org

OpenACC
NVidia CUDA

Valid HTML 4.01!     Valid CSS!             Dr Clemens Grelck