High Performance Parallel Programming

The course can be part of the following programmes:

1. Computer Science, Master's Programme (N2COS)
2. Software Engineering and Management, Master's Programme (N2SOF)
3. Applied Data Science, Master's Programme (N2ADS)

The course is a also a single-subject course at Gothenburg University.

To be eligible for the course, students should have successfully completed courses corresponding to 105 credits within the subject of Computer Science, Mathematics, Software Engineering, or equivalent.
Including a 7.5 credits course in machine-oriented programming (e.g., DIT153 Machine Oriented Programming, or equivalent).

Applicants must prove knowledge of English: English 6/English level 2 or the equivalent level of an internationally recognized test, for example TOEFL, IELTS.

This course looks at parallel programming models, efficient programming methodologies and performance tools with the objective of developing highly efficient parallel programs.

The course consists of a set of lectures and laboratory sessions. The lectures start with an overview of parallel computer architectures and parallel programming models and paradigms. An important part of the discussion are mechanisms for synchronization and data exchange. Next, performance analysis of parallel programs is covered. The course proceeds with a discussion of tools and techniques for developing parallel programs in shared address spaces. This section covers popular programming environments such as pthreads and OpenMP. Next the course discusses the development of parallel programs for distributed address space. The focus in this part is on the Message Passing Interface (MPI). Finally, we discuss programming approaches for executing applications on accelerators such as GPUs. This part introduces the CUDA (Compute Unified Device Architecture) programming environment.

The lectures are complemented with a set of laboratory sessions in which participants explore the topics introduced in the lectures. During the lab sessions, participants parallelize sample programs over a variety of parallel architectures, and use performance analysis tools to detect and remove bottlenecks in the parallel implementations of the programs.

On successful completion of the course the student will be able to:

Knowledge and understanding

• List the different types of parallel computer architectures, programming models and paradigms, as well as different schemes for synchronization and communication.
• List the typical steps to parallelize a sequential algorithm
• List different methods for analyses methodologies of parallel program systems

Competence and skills

• Apply performance analysis methodologies to determine the bottlenecks in the execution of a parallel program
• Predict the upper limit to the performance of a parallel program

Judgement and approach

• Given a particular software, specify what performance bottlenecks are limiting the efficiency of parallel code and select appropriate strategies to overcome these bottlenecks given a certain software.
• Design energy-aware parallelization strategies based on a specific algorithm structure and computing system organization
• Argue for which performance analysis methods that are important given a specific context.

The teaching consists of theory-oriented lectures and lab sessions in which the participants develop code for different types of parallel computer systems.

Language of instruction: English

The course is examined by an individual written exam that is carried out in an exam hall and a laboratory report written in groups of two.

If a student who has been failed twice for the same examination element wishes to change examiner before the next examination session, such a request is to be granted unless there are specific reasons to the contrary (Chapter 6 Section 22 HF).

If a student has received a certificate of disability study support from the University of Gothenburg with a recommendation of adapted examination and/or adapted forms of assessment, an examiner may decide, if this is consistent with the course’s intended learning outcomes and provided that no unreasonable resources would be needed, to grant the student adapted examination and/or adapted forms of assessment.

If a course has been discontinued or undergone major changes, the student must be offered at least two examination sessions in addition to ordinary examination sessions. These sessions are to be spread over a period of at least one year but no more than two years after the course has been discontinued/changed. The same applies to placement and internship (VFU) except that this is restricted to only one further examination session.

If a student has been notified that they fulfil the requirements for being a student at Riksidrottsuniversitetet (RIU student), to combine elite sports activities with studies, the examiner is entitled to decide on adaptation of examinations if this is done in accordance with the Local rules regarding RIU students at the University of Gothenburg.

Sub-courses

1. Written hall examination, 4.5 credits
   Grading scale: Pass with Distinction (VG), Pass (G) and Fail (U)
2. Laboratory, 3 credits
   Grading scale: Pass (G) and Fail (U)

The grading scale comprises: Pass with Distinction (VG), Pass (G) and Fail (U).

A Pass grade (G) for the entire course requires at least a Pass grade for all sub-courses.

A Pass with Distinction grade (VG) for the entire course requires a Pass with Distinction (VG) on the Written hall examination.

The course is evaluated through meetings both during and after the course between teachers and student representatives. Further, an anonymous questionnaire is used to ensure written information. The outcome of the evaluations serves to improve the course by indication which parts could be added, improved, changed or removed.

Knowledge of concurrent programming is recommended, e.g., from the course DIT391 Principles of Concurrent Programming, or equivalent.

The course is a joint course together with Chalmers.

The course replaces DIT430 Parallel programming for high performance, 7.5 credits. This course cannot be included in a degree containing DIT430. Nor can it be part of a degree based on another degree that includes DIT430.