Syllabus for

Computer Science 668/742

Parallel Computing --- Fall 2007


Time:         TuTh 11:00-12:15 PM
Professor:  Fred Annexstein

Phone:

...(513)-556-1807

FAX:

...(513)-556-7326

Email:

... fred.annexstein@uc.edu

Address:

... 889 Rhodes Hall

 

... Department of CS

 

... College of Engineering

 

... University of Cincinnati

 

... Cincinnati, Ohio 45221

 

 

 

Web: Information on this course will be posted on the BLACKBOARD system and will be regularly updated. You should regularly check BLACKBOARD for announcements, updated syllabus, homework assignments, etc. 

 

Syllabus:

 

 

Objectives: This class is designed as an introduction to the concepts and practice of Parallel Computing. In this class students will be introduced to some of these tools, techniques, and methods of analysis in parallel computing. We will do a number of programming projects during the term.

 

Grading Scheme: 

The grading for this class will be based on two lecture exams and approximately 6 homework assignments, including written assignments, programming assignments, a research report, and a final project report. Your final grade will be a weighted average for these two aspects - Homeworks 60% and Lecture Exams 40%.
Late homework or makeup exams are not allowed unless pre-arranged with the instructor.

 

Academic Dishonesty

  The standard penalty for any form of academic dishonesty in this course is failure in the course. Providing or receiving help during exams, or submitting the work of another as your own both constitute academic dishonesty. There are no "small" offenses to this policy. 

 

Course Description:

This course is designed as a dual level/ senior undergraduate level course covering the programming and algorithmic design issues arising in parallel computing. The course is designed to meet the competencies for Area 6 for the Minor Program in Computational Science of the Ralph Regula School of Computational Science. The following competencies are addressed in this course:

6.1 Describe the fundamental concepts of parallel programming and related architectures
6.2 Demonstrate parallel programming concepts using MPI
6.3 Demonstrate knowledge of parallel scalability
6.4 Demonstrate knowledge of parallel programming libraries

 

Textbook:
Parallel Programming in C With Mpi and Openmp
: Michael J. Quinn
Street Price $75.30

 

Reference Texts:

 Parallel Programming With Mpi: Peter Pacheco
Street Price $25.00
Introduction to Parallel Computing: Design and Analysis of Algorithms: Ananth Grama, Anshul Gupta, George Karpis, Vipin Kumar
Street Price $64.00

Using MPI - 2nd Edition: Portable Parallel Programming with the Message Passing Interface (Scientific and Engineering Computation) by William Gropp, Ewing Lusk, and Anthony Skjellu $34.00



Topics and Schedule:

Lecture Topics

Readings

Lecture Notes

Introduction to Parallel Computing

Chapter 1, 1.1-1.8

cs668-lec0-Welcome.ppt

Parallel Architectures

Chapter 2, 2.3-2.7

cs668-lec1-Parallel Arch.ppt

Introduction to MPI

Chapter 4, 4.1-4.7

cs668-lec2-MPIIntro.ppt

More about MPI

Chapter 5, 5.1-5.10

lcs668-lec3-MPI.ppt

Floyd's Algorithm and Parallelization

Chapter 6, 6.1-6.8

cs668-lec4-Floyds.ppt

Design and Analysis

Chapter 3, 3.1-3.5

cs668-lec6-Analysis.ppt

Speedup, Amdahl's Law, Analysis, Iso-Efficiency, Scalability

Chapter 7, 7.1-7.7

cs668-lec5-Amdahls.ppt

Interconnection Networks, Distributed Memory, Distributed Shared Memory

Chapter 2, 2.1-2.2

cs668-lec7-Interconnection.ppt

Manger-Worker Paradigms, Document Classification, Parallel K-means

Chapter 9, 9.1-9.6

cs668-lec8-WorkerClassification.pp

Linear Algebra, Matrix-Vector Multiplication

Chapter 8, 8.1-8.7

cs668-lec9-VectorMultiplication.ppt

Monte Carlo Methods, and Parallel Simulation Algorithms

Chapter 10, 10.1-10.4

cs668-lec10-MonteCarlo.ppt

More on Monte Carlo Methods, Case Studies

Chapter 10, 10.5-10.6

cs668-lec11-MonteCarlo2.ppt

Finite Difference Methods

Chapter 13, 13.1-13.5

cs668-lec12-FiniteDiff.ppt

Combinatorial Search

Chapter 16, 16.1-16.10

cs668-lec13-CombSearch.ppt

Shared Memory Parallel Programming, Scheduling and Work-stealing

Chapter17 17.1-17.3, Cilk Notes

cilk.pdf

cs668-lec14-SMP-Cilk.ppt

Profiling Tools, Performance Tools, Parallel Libraries

Notes

www.netlib.org/scalapack

cs668-lec15-Profiling.ppt

Bridging Models, Bulk-Synchronous Programming

Notes

 

 

 

 

Programming Assignments

 

ItemProgramming Assignment #1: Due Tuesday Oct 2
Using the C code in figure 4.8 of Quinn's book (see below), write an MPI parallel version of the code to compute pi using the rectangle rule with 1,000,000 intervals. Benchmark your program on various numbers of processors and produce a graphic such as we saw in class. Extra credit for doing same with Simpson's Rule.

#include <stdio.h>
#define INTERVALS 1000000
int main(int argc, char* argv[])
{
  double area; /* The final answer */
  double ysum; /* Sum of rectangle heights */
  double xi;   /* Midpoint of interval */
  int i;
  ysum= 0.0;
  for(i=0;i<INTERVALS;i++)
    {
      xi= ((1.0/INTERVALS)*(i+0.5)); /* Midpoint of interval */
      ysum += 4.0/(1.0 + xi*xi);
    }
      area=ysum*(1.0/INTERVALS);
      printf("Area is %13.11f\n",area);
  return 0;
}


ItemProgramming Assignment #2 : Due Oct 15
Complete Exercise 6.13 from Quinn.
The assignment is to implement a well-known cellular automata simulation called the "Game of Life".
Each cell in a 2-D array is either live or dead. During each iteration a dead cell with 3 live neighbors is reborn. A live cell with less than 2 neighbors or more than three neighbors becomes dead. All updates are synchronized. Write a parallel program with command line parameters m,n,j,k where the m by n 2-D array is read from standard input. The simulation runs for j iterations, and every kth iteration the array is printed. You are also to complete a theoretical analysis of a parallel implementation of the problem.

Final Project Resources

Final Project: Parallel Application Performance Study Fall 2007

 

Web Resources

Item

MPI Information Links
Index: http://mpi.deino.net/mpi_functions/index.htm
Testsome: http://mpi.deino.net/mpi_functions/MPI_Testsome.html

--  from Ryan McGovern

 

Item

Midterm Study Sheet
CS668-Midterm-Study.doc (21 Kb)

 

 

 

 

Item


Profiling: Joe and Mike recommend Valgrind.
http://valgrind.org/

 

Item

Scalapack Project

http://www.netlib.org/scalapack/index.html

OSC Notes:
Using Parallel Numerical Libraries (focus on ScaLAPACK)

http://www.osc.edu/supercomputing/training/parlib/
 

 

Item

Article on State of Computer Chess

From Adam G.: Here is the article I mentioned to you today that compares modern day computers to Deep Blue. It also goes into some of the heuristics used in the software. It also says there are roughly 10120 possible board positions in chess.  

  http://www.extremetech.com/article2/0,1558,1915527,00.asp 

.

 

Item

Final Study Sheet: Final Study Sheet