Programming Multi-core and Many-core Systems (2017)


Course code 5284PMCM6Y
Format Lecture course with lab sessions and workshops
Curriculum MSc Computer Science (joint degree UvA/VU)
Block 2a (Feb 6 - Mar 31, 2017)
Status Compulsory for track Parallel Computing Systems
Constrained choice Programming
ECTS 6 credits
Language English
Study Guide Click here.
Objectives

To develop an understanding of the opportunities, challenges and limits of parallel computing and to gain practical familiarity with state-of-the-art programming models for contemporary concurrent multi-core and many-core computing systems.

Contents

The course provides a comprehensive introduction into state-of-the-art programming models for concurrent computing systems from multi-core processors in everyday laptops to large-scale server systems and high-end accelerators.

We start with instruction-level parallelism and vectorisation. Then we continue with multithreaded programming models for shared address space systems, where we look both into OpenMP compiler directives and into more low-level Posix threads before we discuss advanced topics and common pitfalls of shared memory parallel programming.

In the second half of the course we focus on general-purpose graphics accelerators (GPGPUs) using NVidia's programming model CUDA and end with advanced topics such as directive-based GPU programming, Intel Xeon Phi programming and cross-architecture programming using the standard-driven OpenCL programming model.

The lectures are complemented by labs where participants gain first-hand experience with the various programming models and by group discussion workshops (werkcolleges) where participants present their work and discuss their achievements with each other as well as with the lecturers and lab assistants.

The course is complementary to the VU courses Programming Large-scale Parallel Systems and the corresponding project course in that it looks into node-level concurrency, whereas the VU courses focus on systems that are made up of many nodes.

Lecturers Dr Clemens Grelck (ccordinator)
Dr Ana Varbanescu
Assistant Giulio Stramondo
Lectures Tue, 13-15, B0.203
Fri, 09-11, D1.110
Labs Tue, 15-17, B0.207
Fri, 11-13, D1.110
Workshops Fri, Feb 17, 11-13, D1.110
Fri, Mar 03, 11-13, D1.110
Fri, Mar 17, 11-13, D1.110
Fri, Mar 31, 11-13, D1.110
Participation in workshops is mandatory!!
Exam We have no written exam; the final grade is determined by the four bi-weekly assignments.
Grading Each assignment counts for 25% of the final grade. Submission is in groups of two.
If we have reason to believe that the workload and contributions in a group are not somewhat equal, we reserve the right to conduct individual interviews.
Slides Will be made available on Blackboard after the lecture.
Contents
  • Chapter 1: Introduction and Motivation
  • Chapter 2: SIMD Parallelism and Vectorisation
  • Chapter 3: OpenMP Compiler Directives
  • Chapter 4: Multithreading with PThreads
  • Chapter 5: Common Pitfalls in Shared Memory Parallel Processing
  • Chapter 6: GPGPU Programming with CUDA
  • Chapter 7: GPGPU Programming with OpenCL and OpenACC
  • Chapter 8: Heterogeneous Computing
  • Chapter 9: Performance Metrics and Models
Assignments The assignments will be made available on Blackboard.
Mailing List The UvA mailing list PMMS2017 serves as the primary medium of communication between lecturers and participants as well as among participants themselves. Subscription to the mailing list is controlled and restricted to participants of the course. Traffic on the mailing list is archived; access to the archives is restricted to participants. Please, subscribe here.
Literature and Resources The course does not follow any specific text book. The following list summarises a number of interesting textbooks on various aspects of the course.

Parallel Programming in General
  • Kevin Dowd, Charles Severance: High Performance Computing, O'Reilly, 1998.
  • Ananth Grama, Anshul Gupta, George Karypis, Vipin Kumar: Introduction to Parallel Computing, Addison-Wesley, 2003.
  • Gregory R. Andrews: Foundations of Multithreaded, Parallel, and Distributed Programming, Addison Wesley, 2000.
  • Michael J. Quinn: Parallel Programming in C with MPI and OpenMP, McGraw-Hill, 2003.

OpenMP
  • Rohit Chandra, Leo Dagum, Dave Kohr, Dror Maydan, Jeff McDonald, Ramesh Menon: Parallel Programming in OpenMP, Morgan Kaufmann, 2000.
  • Barbara Chapman, Gabriele Jost, Ruud van der Pas: Using OpenMP, MIT Press, 2007.
  • www.openmp.org

OpenACC
NVidia CUDA

Valid HTML 4.01!     Valid CSS!             Dr Clemens Grelck