CS4823/CS6643 Parallel Programming, Spring 2024
Course Objectives and Schedule

Course Objectives

  1. Understand parallel computer architectures.
  2. Learn the Operating System and Middle-ware support for parallel computation.
  3. Understand parallel algorithm designs.
  4. Learn to program parallel applications with various software libraries on different parallel architectures.
  5. Understand parallel performance theory and learn to analyze and optimize parallel applications.
  6. Be able to program non-CPU parallel processors and understand the major research issues in Parallel Computing.

Syllabus

The following syllabus gives you a rough idea of the time spent on each topic. The syllabus may change depending on how quickly or slowly we move. Regardless, the tests and exams will be on the dates shown. Make any travel plans accordingly.

Week Topics Assignments & Exams Supplement Materials
Week 1: 01/15 Course introduction;

A machine-learning problem for parallelization;
Week 2: 01/22 Parallel algorithm design, part 1;

 Assignment 1: Linux Programming Due
Week 3: 01/29 Parallel algorithm design, part2;

Case Study Sparse Matrix Parallelization;

Shared Memory Programming Basics;

Assignment 2: A simple machine-learning problem due.

Week 4: 02/05 Shared-memory parallel programming: OpenMP, Part 0 (intro);

Shared-memory parallel programming: OpenMP, Part 1;

OpenMP Examples
Week 5: 02/12 Parallel Performance Models
Week 6: 02/19 Shared-memory parallel programming: Pthreads, part 1;

 Midterm Exam 1 on Feb. 21th, in class. Midterm Preparation

Pthreads Examples
Week 7: 02/26 Shared-memory parallel programming: Pthreads, part 2;

Shared-memory parallel programming: Pthreads, part 3; 		Assignment 3: OpenMP due;
Week 8: 03/04 Parallel Architecture: Distributed-memory systems;

Distributed-memory parallel programming: MPI, part 1;

Assignment 4: Pthread due;

MPI Examples
Week 9: 03/11 Spring break, no class;

Week 10: 03/18 Distributed-memory parallel programming: MPI, part 2;

Assignment 5: MPI due;

Week 11: 03/25 Distributed-memory parallel programming: MPI, part 3;

Shared Memory Architecture;
Week 12: 04/01 Parallel Architecture: GPU and SIMD;

GPU programming, part 1;

CUDA Examples
Week 13: 04/08 GPU programming, part 2;

Advanced GPU programming topics;

Week 14: 04/15 In-class coding session;

 Assignment 6: GPU due;
Week 15: 04/22 MapReduce, Hadoop, and Spark;

Assignment 7: Parallel Model Training.
Final Programming Exam/Excercise Mon 04/29th, 2024, Saturday, during class time.