CS 410 Top: Experimental Design

Credit Hours:	4
Course Coordinator:	N/A
Course Description:	To be added
Prerequisites:	Introduction to Probability & Statistics
Goals:	Students will learn how to design experiments for evaulating hardware or software systems. The techniques studied will include factorial designs and the related statistical evaluation methods.
Textbooks:	Raj Jain, "The art of computer systems performance analysis", John Wiley & Sons, Inc., 1991.
References:	None.
Major Topics:	1. Overview of performance evaluation techniques (simulation, modeling, measurement). 2. Measurement techniques and tools: Selecting metrics, selecting workloads, characterizing workloads, benchmarks. 3. Statistical techniques: Mean/mode/median, confidence intervals, testing for a zero mean, linear regression models, non-linear regression models. 4. Experimental design & analysis: 2^k factorial designs, 2^kr factorial designs, 2^k-p fractional factorial designs, one-factor & two-factor full factorial designs. 5. Simulation: overview, discrete event simulation, random number generation.
Laboratory Exercises:	None.

Oral and Written Communications:	Every student is required to submit at least one written report (not including exams, tests, quizzes, or commented programs) of typically 5 pages.
Social and Ethical Issues:	How to interpret data correctly and honestly. How to avoid ratio games. 2 1:30 hour classes. Ratio games are easy to make one system look better than another. Students are taught different such ways to misrepresent results and to then develop skills to avoid these mistakes. This is an integral part of the classwork.
Theoretical Content:	Statistical analysis. Approximately 35%.
Problem Analysis:	The students are required to develop experimental approaches for measuring the performance of a diverse set of systems.
Solution Design:	None.