Programming Multi-core and Many-core Systems (2020)


Course code 5284PMCM6Y
Format Lecture course with lab sessions and workshops
Curriculum MSc Computer Science (joint degree UvA/VU)
Block 4 (Feb 3 - Mar 27, 2020)
Status Compulsory for track Parallel Computing Systems
Constrained choice Programming
ECTS 6 credits
Language English
Study Guide Click here.
Objectives

To develop an understanding of the opportunities, challenges and limits of parallel computing and to gain practical familiarity with state-of-the-art programming models for contemporary concurrent multi-core and many-core computing systems.

Contents

The course provides a comprehensive introduction into state-of-the-art programming models for concurrent computing systems from multi-core processors in everyday laptops to large-scale server systems and high-end accelerators.

We start with instruction-level parallelism and vectorisation. Then we continue with multithreaded programming models for shared address space systems, where we look both into OpenMP compiler directives and into more low-level Posix threads before we discuss advanced topics and common pitfalls of shared memory parallel programming.

In the second half of the course we focus on general-purpose graphics accelerators (GPGPUs) using NVidia's programming model CUDA and end with advanced topics such as directive-based GPU programming, Intel Xeon Phi programming and cross-architecture programming using the standard-driven OpenCL programming model.

The lectures are complemented by labs where participants gain first-hand experience with the various programming models and by group discussion workshops (werkcolleges) where participants present their work and discuss their achievements with each other as well as with the lecturers and lab assistants.

The course is complementary to the VU courses Programming Large-scale Parallel Systems and the corresponding project course in that it looks into node-level concurrency, whereas the VU courses focus on systems that are made up of many nodes.

Lecturers Dr Clemens Grelck (ccordinator)
Dr Ana Varbanescu
Assistants Julius Roeder
Misha Mesarcik
Lectures Mon, 15-17
Thu, 09-11
See Datanose for all details and exceptions.
Labs Mon, 17-19
Thu, 11-13
See Datanose for all details and exceptions.
Workshops Thu, Feb 13, 11-13, A1.28
Thu, Feb 27, 11-13, F1.02
Thu, Mar 12, 11-13, C1.110
Thu, Mar 26, 11-13, C1.112
Participation in workshops is compulsory!!
Exam Mon, Mar 23, 13-15, C1.110
Grading Four bi-weekly assignments and an exam count for 20% each. Submission of assignments is in groups of two. The (individual) exam must be passed with a minimum grade of 5.0.
If we have reason to believe that the workload and contributions in a group are not reasonably balanced, we reserve the right to conduct individual interviews.
Slides Will be made available on Canvas after each lecture.
Contents
  • Introduction and Motivation
  • SIMD Parallelism and Vectorisation
  • OpenMP Compiler Directives
  • Common Pitfalls in Shared Memory Parallel Processing
  • Multithreading with PThreads
  • Performance Metrics and Models
  • GPGPU Programming with CUDA
  • GPGPU Programming with OpenCL and OpenACC
  • Heterogeneous Computing
Assignments Will be made available on Canvas.
Mailing List The UvA mailing list PMMS2020 serves as the primary medium of communication between lecturers and participants as well as among participants themselves. Subscription to the mailing list is controlled and restricted to participants of the course. Traffic on the mailing list is archived; access to the archives is restricted to participants. Please, subscribe!!
The UvA mailing list PMMS2020-Team serves as a convenient way to reach all members of the teaching team at once. Participants of the course are permitted to post to this mailing list, but are not subscribed to it.
Guest lecture

March 16:
Dr Ben van Werkhoven
Netherlands eScience Center, Amsterdam:
GPU Computing at the Netherlands eScience Center

Abstract:

Guest lecturer:
Following a Master in Parallel and Distributed Computer Systems (PDCS) from 2008 to 2010 at the Vrije Universiteit, Ben van Werkhoven continued to work on his PhD in the group of Henri Bal, still at the Vrije Universiteit. In 2014 Ben obtained his PhD for a dissertation entitled Scientific Supercomputing with Graphics Processing Units. Since 2014 he has been working as eScience Research Coordinator at the Netherlands eScience Center and has been involved in various scientific research projects.

Literature and Resources The course does not follow any specific text book. The following list summarises a number of interesting textbooks on various aspects of the course.

Parallel Programming in General
  • Kevin Dowd, Charles Severance: High Performance Computing, O'Reilly, 1998.
  • Ananth Grama, Anshul Gupta, George Karypis, Vipin Kumar: Introduction to Parallel Computing, Addison-Wesley, 2003.
  • Gregory R. Andrews: Foundations of Multithreaded, Parallel, and Distributed Programming, Addison Wesley, 2000.
  • Michael J. Quinn: Parallel Programming in C with MPI and OpenMP, McGraw-Hill, 2003.

OpenMP
  • Rohit Chandra, Leo Dagum, Dave Kohr, Dror Maydan, Jeff McDonald, Ramesh Menon: Parallel Programming in OpenMP, Morgan Kaufmann, 2000.
  • Barbara Chapman, Gabriele Jost, Ruud van der Pas: Using OpenMP, MIT Press, 2007.
  • www.openmp.org

OpenACC
NVidia CUDA

Valid HTML 4.01!     Valid CSS!             Dr Clemens Grelck