CS 415 Parallel Programming

Credit Hours:	4
Course Coordinator:	Jingke Li
Course Description:	An introduction to parallel programming concepts and techniques. Topics include: parallel programming models and languages, share-memory programming, message-passing programming, performance models and analysis techniques, domain-specific parallel algorithms. Prerequisites: CS 321 and CS 333.
Prerequisites:	Working knowledge of computer architectures, compilers, and operatingsystems. Adequate programming skills in C. No prior parallelcomputation background is necessary.
Goals:	Understand the fundamental issues in parallel programming and learn programming skills on actual parallel computing systems. Upon the successful completion of this course students will be able to: Explain major programming issues on shared-memory multiprocessors and message-passing multicomputers, respectively. Write simple fork/join-style parallel programs with PTHREAD routines. Write simple message-passing parallel programs with PVM routines. Measure and analyze runtime performance of parallel programs. Apply a set of basic parallel algorithms (e.g. sorting, matrix multiplication, search) to actual problems. Compare trade-offs among different parallel algorithms for the same problem.
Textbooks:	Parallel Programming: Techniques and Applications Using Networked Workstations and Parallel Computers, by Barry Wilkinson and Michael Allen, Prentice Hall, 1999.
References:	PVM: Parallel Virtual Machine, Al Geist et al, MIT Press, 1994. Parallel Programming with MPI, Peter S. Pacheco, Morgan Kaufmann, 1997.
Major Topics:	- Basic concepts (3 hours) - Parallel architectures (3 hours) - Programming with shared memory (multi-threading, synchronization) (4.5 hours) - Programming with message passing (data decomposition, communication) (6 hours) - Programming models and languages (PRAM, Fortran 90, others) (4.5 hours) - Programming Tools (PVM, MPI) (3 hours) - Performance analysis (3 hours) - Parallel algorithms (6 hours)
Laboratory Exercises:	- Implementing a simple application on a shared-memory system using the Pthread package (2 weeks) - Implementing a simple application on a message-passing system with PVM and/or MPI (2 weeks) - Implementing a larger application on a message-passing system with PVM and/or MPI and study its performance (3 weeks)

CAC Category Credits	Core	Advanced
Data Structures		0.5
Algorithms		0.5
Software Design		1.0
Computer Architecture		1.0
Programming Languages		1.0

Oral and Written Communications:	Every student is required to submit at least __3___ written reports (not including exams, tests, quizzes, or commented programs) of typically __2-5__ pages
Social and Ethical Issues:	None.
Theoretical Content:	Theoretical parallel machine models, performance analysis. (About 20% of the course.)
Problem Analysis:	Analysis is a component of many sections of this course, e.g. analysis on parallel algorithms, analysis on interconnection topologies of parallel computers, performance analysis of actual parallel programs.
Solution Design:	Students are required to design implementation strategies for three applications.