Decision Making Methods

Decision Making Methods

C

/ 26.11.2019

• The ELECTRE methods

• The PROMETHEE methods

• Group decision making

• Sensitivity analysis

Introduction to Decision Making Methods, János Fülöp Laboratory of Operations Research and Decision Systems, Computer and Automation Institute, Hungarian Academy of Sciences http://academic.evergreen.edu/projects/bdei/documents/decisionmakingmethods.pdf

Some values of the multiattribute decision models are often subjective. The weights of the criteria and the scoring values of the alternatives against the subjective (judgmental) criteria contain always some uncertainties.

It is therefore an important question how the final ranking or the ranking values of the alternatives are sensitive to the changes of some input parameters of the decision model.

The simplest case is when the value of the weight of a single criterion is allowed to vary. For additive multiattribute models, the ranking values of the alternatives are simple linear functions of this single variable and attractive graphical tools can be applied to present a simple sensitivity analysis to a user.

(Forman and Selly, 2001).

… 1. Sensitivity analysis

For a wide class of multiattribute decision models Mareschal, 1988 showed how to determine the stability intervals or regions for the weights of different criteria. These consist of the values that the weights of one or more criteria can take without altering the results given by the initial set of weights, all other weights being kept constant.

Wolters and Mareschal, 1995 proposed a linear programming model to find the minimum modification of the weights required to make a certain alternative ranked first.

Triantaphyllou and Sanchez, 1997 presented an approach of a more complex

sensitivity analysis with the change of the scores of the alternatives against the

criteria, as well.

A methodology was presented by Mészáros and Rapcsák, 1996 for a wide class of MAUT models where the aggregation is based on generalized means, including so the additive and multiplicative models as well. In this approach the weights and the scores of the alternatives against the criteria can change simultaneously in given intervals. The following questions were addressed:

− What are the intervals of the final ranking values of the alternatives with the restriction that the intervals of the weights and scores are given?

− What are the intervals of the weights and scores with the restriction that the final ranking of the alternatives does not change?

− Consider a subset of alternatives whose ranking values are allowed to change in an interval. In what intervals are the weights and scores allowed to vary, and how will these modifications effect the ranking values of the entire set of alternatives?

Mészáros and Rapcsák, 1996 pointed out that these questions lead to the optimization of linear fractional functions over rectangles and proposed an efficient technique to solve these problems. Some of the results of Mészáros and Rapcsák, 1996 were recently extended by Ekárt and Németh, 2005 for more general decision functions.

About

… 1. Sensitivity analysis

A technique used to determine how different values of an independent variable will impact a particular dependent variable under a given set of assumptions. This technique is used within specific boundaries that will depend on one or more input variables, such as the effect that changes in interest rates will have on a bond's price.

Sensitivity analysis is a way to predict the outcome of a decision if a situation turns out to be different compared to the key prediction(s).

For example, an analyst might create a financial model that will value a company's equity (the dependent variable) given the amount of earnings per share (an independent variable) the company reports at the end of the year and the company's price-to-earnings multiple (another independent variable) at that time. The analyst can create a table of predicted price-to-earnings multiples and a corresponding value of the company's equity based on different values for each of the independent variables.

Sensitivity analysis is very useful when attempting to determine the impact the actual outcome of a particular variable will have if it differs from what was previously assumed. By creating a given set of scenarios, the analyst can determine how changes in one variable(s) will impact the target variable.

Investopedia explains Sensitivity Analysis :

Sensitivity analysis (SA) is the study of how the variation (uncertainty) in the output of a mathematical model can be apportioned, qualitatively or quantitatively, to different sources of variation in the input of a model [1].

1. a b Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D. Saisana, M., and Tarantola, S., 2008, Global Sensitivity Analysis. The Primer, John Wiley & Sons.

The problem setting in sensitivity analysis has strong similarities with design of experiments. In design of experiments one studies the effect of some process or intervention (the 'treatment') on some objects (the 'experimental units'). In sensitivity analysis one looks at the effect of varying the inputs of a mathematical model on the output of the model itself. In both disciplines one strives to obtain information from the system with a minimum of physical or numerical experiments.

In more general terms uncertainty and sensitivity analyses investigate the robustness of a study when the study includes some form of mathematical modelling. While uncertainty analysis studies the overall uncertainty in the conclusions of the study, sensitivity analysis tries to identify what source of uncertainty weights more on the study's conclusions. For example, several guidelines for modelling (see e.g. one from the US EPA) or for impact assessment (see one from the European Commission) prescribe sensitivity analysis as a tool to ensure the quality of the modelling / assessment.

More about

… 1. Sensitivity analysis

In uncertainty and sensitivity analysis there is a crucial trade off between how scrupulous an analyst is in exploring the input assumptions and how wide the resulting inference may be. The point is well illustrated by the econometrician Edward E. Leamer (1990) [2]:

I have proposed a form of organized sensitivity analysis that I call ‘global sensitivity analysis’ in which a neighborhood of alternative assumptions is selected and the corresponding interval of inferences is identified. Conclusions are judged to be sturdy only if the neighborhood of assumptions is wide enough to be credible and the corresponding interval of inferences is narrow enough to be useful.

2. Leamer, E., (1990) Let's take the con out of econometrics, and Sensitivity analysis would help.

In C. Granger (ed.), Modelling Economic Series. Oxford: Clarendon Press 1990.

Note Leamer’s emphasis is on the need for 'credibility' in the selection of assumptions. The easiest way to invalidate a model is to demonstrate that it is fragile with respect to the uncertainty in the assumptions or to show that its assumptions have not been taken 'wide enough'. The same concept is expressed by Jerome R. Ravetz, for whom bad modeling is when uncertainties in inputs must be suppressed lest outputs become indeterminate.[3]

3. Ravetz, J.R., 2007, No-Nonsense Guide to Science, New Internationalist Publications Ltd..

In modern econometrics the use of sensitivity analysis to anticipate criticism is the subject of one of the ten commandments of applied econometrics (from Kennedy, 2007[4] ):

Thou shall confess in the presence of sensitivity. Corollary: Thou shall anticipate criticism [···] When reporting a sensitivity analysis, researchers should explain fully their specification search so that the readers can judge for themselves how the results may have been affected. This is basically an ‘honesty is the best policy’ approach, advocated by Leamer, (1978[5]).

4. Ravetz, J.R., 2007, No-Nonsense Guide to Science, New Internationalist Publications Ltd.

5. Kennedy, P. (2007). A guide to econometrics, Fifth edition. Blackwell Publishing..

The use of mathematical modelling can be the subject of controversies, see Nassim Nicholas Taleb[6] in Economics, and Orrin H. Pilkey and Linda Pilkey Jarvis[7] in Environmental Sciences. As noted by the latter Authors, this increases the relevance of sensitivity analysis in today's modelling practice[1] .

6. Taleb, N. N., (2007) The Black Swan: The Impact of the Highly Improbable, Random House.

7. Pilkey, O. H. and L. Pilkey-Jarvis (2007), Useless Arithmetic. Why Environmental Scientists Can't Predict the Future. New York: Columbia University Press..

Sensitivity Analysis

There are a number of questions that could be asked concerning the sensitivity of an optimal solution to changes in the data.

Every commercial linear-programming system provides this elementary sensitivity analysis, since the calculations are easy to perform using the tableau associated with an optimal solution. There are two variations in the data that invariably are reported: objective function and right

hand-side ranges. The objective-function ranges refer to the range over which an individual coefficient of the objective function can vary, without changing the basis associated with an optimal solution. In essence, these are the ranges on the objective-function coefficients over which we can be sure the values of the decision variables in an optimal solution will remain unchanged. The right-hand-side ranges refer to the range over which an individual right-hand-side value can vary, again without changing the basis associated with an optimal solution. These are the ranges on the right-hand-side values over which we can be sure the values of the shadow prices and reduced costs will remain unchanged.

Further, associated with each range is information concerning how the basis

would change if the range were exceeded. These concepts will become clear if

we deal with a specific example.

We will consider for concreteness the custom-molder example in order to increase the complexity somewhat, let us add a third alternative to the production possibilities.

Suppose that, besides the six-ounce juice glasses x1 and the ten-ounce cocktail glasses x2, our molder is approached by a new customer to produce a champagne glass.

The champagne glass is not difficult to produce except that it must be molded in two separate pieces—the bowl with stem and then base.

As a result, the production time for the champagne glass is 8 hours per hundred cases, which is greater than either of the other products.

The storage space required for the champagne glasses is 1000 cubic feet per hundred cases; and the contribution is $6.00 per case, which is higher than either of the other products.

There is no limit on the demand for champagne glasses. Now what is the

optimal product mix among the three alternatives?

An example for analysis ~ http://web.mit.edu/15.053/www/AMP-Chapter-03.pdf

The formulation of the custom-molding example, including the new activity of producing champagne glasses, is straightforward. We have exactly the same capacity limitations—hours of production capacity, cubic feet of warehouse capacity, and limit on six-ounce juice-glass demand—and one additional decision

variable for the production of champagne glasses. Letting

x1 = Number of cases of six-ounce juice glasses, in hundreds;

x2 = Number of cases of ten-ounce cocktail glasses, in hundreds;

x3 = Number of cases of champagne glasses, in hundreds;

and measuring the contribution in hundreds of dollars, we have the following formulation of our custommolder example:

Maximize z = 5x1 + 4.5x2 + 6x3, (hundreds of dollars)

subject to:

6x1 + 5x2 + 8x3 ≤ 60, (production capacity; hours)

10x1 + 20x2 + 10x3 ≤ 150, (warehouse capacity; hundreds of sq. ft.)

x1 ≤ 8, (demand for 6 oz. glasses; hundreds of cases)

x1 ≥ 0, x2 ≥ 0, x3 ≥ 0.

If we add one slack variable in each of the less-than-or-equal-to constraints, the problem will be in the

following canonical form for performing the simplex method:

6x1 + 5x2 + 8x3 + x4 = 60, (2)

10x1 + 20x2 + 10x3 + x5 = 150, (3)

x1 + x6 = 8, (4) 5x1 + 4.5x2 + 6x3 − z = 0. (5)

The corresponding initial table:

BV CV x1 x2 x3 x4 x5 x6

x4 60 6 5 8 1

x5 150 10 20 10 0 1

x6 8 1 0 0 1

-z 0 5 4.5 6

BV CV x1 x2 x3 x4 x5 x6

x2 4 2/7 0 1 - 2/7 - 1/7 3/35 0

x6 1 4/7 0 0 -1 4/7 - 2/7 1/14 1

x1 6 3/7 1 0 1 4/7 2/7 - 1/14 0

-z -51 3/7 0 0 - 4/7 - 11/14 - 1/35 0

The final table

(computed by formula):

An example for analysis ~ http://web.mit.edu/15.053/www/AMP-Chapter-03.pdf

We wish to analyze the effect on the optimal solution of changing various elements of the problem data without re-solving the linear program or having to remember any of the intermediate tableaus generated in solving the problem by the simplex method. The type of results that can be derived in this way are conservative, in the sense that they provide sensitivity analysis for changes in the problem data small enough so that the same decision variables remain basic, but not for larger changes in the data.

In the sensitivity analysis discussed thus far, we have restricted our presentation to changes in the problem data that can be made without changing the optimal basis.

Consequently, what we have been able to say is fairly conservative. We did go so far as to indicate the variable that would enter the basis and the variable that would leave the basis when a boundary of a range was encountered. Further, in the case of alternative optimal solutions and alternative optimal shadow prices, the indicated pivot was completed at least far enough to exhibit the particular alternative. One important point in these discussions was the ease with which we could determine the pivot to be performed at a boundary of a range. This seems to indicate that it is relatively easy to make systematic calculations beyond the indicated objective-function or righthand-side ranges. This, in fact, is the case;

and the procedure by which these systematic calculations are made is called parametric programming.

Glass Problem ~

60 Hours production/week 15,000 cubic feet storage space

Constraints

72.57142857 <= 60

165.7142857 <= 150

6.428571429 <= 8

Warehouse capacity must not exceed 150 Demand for 6oz glasses must be less than 8 Production capacity must not exceed 60 hours

60.85714286 Total Contribution

Juice 6.428571

Cocktail 4.285714 Champagne 1.571429 Number of Glasses

Hundreds of cases per week Prod. Time (hrs) Storage Contribution Limit

Juice 100 6 10 $5 <800

Cocktail 100 5 20 $4.50 N/A

Champagne 100 8 10 $6 N/A

An example for analysis ~ http://web.mit.edu/15.053/www/AMP-Chapter-03.pdf

Glass Problem ~

Prod. Time (hrs) Storage Contribution Limit

Juice 100 6 10 $5 <800

Cocktail 100 5 20 $4.50 N/A

Champagne 100 8 10 $6 N/A

Number of Glasses

Hundreds of cases per week Juice 6.428571

Cocktail 4.285714

Champagne 0

60 Hours production/week 15,000 cubic feet storage space

Constraints

60 <= 60 Production capacity must not exceed 60 hours 150 <= 150 Warehouse capacity must not exceed 150 6.428571429 <= 8 Demand for 6oz glasses must be less than 8

Total Contribution 51.42857143

Glass Problem ~

Microsoft Excel 9.0 Sensitivity Report Worksheet: [Glass, pg 110.xls]Optimum Report Created: 7/18/01 5:58:51 PM

Adjustable Cells

Final Reduced Objective Allowable Allowable

Cell Name Value Cost Coefficient Increase Decrease

$I$6 Juice 6.428571429 0 5 0.4 0.363636364

$I$7 Cocktail 4.285714286 0 4.5 2 0.333333333

$I$8 Champagne 0 -0.571428571 6 0.571428571 1E+30

Constraints

Final Shadow Constraint Allowable Allowable

Cell Name Value Price R.H. Side Increase Decrease

$A$14 Constraints 60 0.785714286 60 5.5 22.5

$A$15 Constraints 150 0.028571429 150 90 22

$A$16 Constraints 6.428571429 0 8 1E+30 1.571428571

References

• Baker, D., Bridges, D., Hunter, R., Johnson, G., Krupa, J., Murphy, J. and Sorenson, K. (2002) Guidebook to Decision- Making Methods, WSRC-IM-2002-00002, Department of Energy,

USA. http://emi-web.inel.gov/Nissmg/Guidebook_2002.pdf

• Brans, J.P. and Vincke, Ph. (1985) "A preference ranking organization method", Management Science, 31, 647-656.

• Triantaphyllou, E. and Sanchez, A. (1997) "A sensitivity analysis approach for some deterministic multi-criteria decision making methods", Decision Sciences, 28, 151-194.

• Triantaphyllou, E. (2000) Multi-Criteria Decision Making Methods: A Comparative Study, Kluwer Academic Publishers, Dordrecht.

• Wolters, W.T.M. and Mareschal, B. (1995) .Novel types of sensitivity analysis for additive MCDM methods., European Journal of Operational Research, 81, 281-290.

• Andrea Saltelli, Global Sensitivity Analysis: An Introduction - European Commission, Joint Research Centre of Ispra, Italy [email protected]

• Andrea Saltelli, Marco Ratto, Stefano Tarantola and Francesca Campolongo, SENSITIVITY ANALYSIS PRACTICES. STRATEGIES FOR MODEL-BASED INFERENCE -

European Commission, Joint Research Centre of Ispra (I).

• Stephen L Nelson, Jr M.D., PH.D., Stephen L. Nelson, - Excel 2007 Data Analysis For Dummies - The sensitivity Report - Page 11-27

http://www.mindtools.com/pages/main/newMN_TED.htm

•Investopedia explains Sensitivity Analysis, http://www.investopedia.com/terms/s/sensitivityanalysis.asp

•Sensitivity analysis, http://en.wikipedia.org/wiki/Sensitivity_analysis

•Lucia Breierova, Mark Choudhari, An Introduction to Sensitivity Analysis,

http://sysdyn.clexchange.org/sdep/Roadmaps/RM8/D-4526-2.pdf

•David J. Pannell , Sensitivity analysis: strategies, methods, concepts, examples ,

http://cyllene.uwa.edu.au/~dpannell/dpap971f.htm

•D. G. Cacuci, Sensitivity and Uncertainty Analysis: Theory ______________

•Andrea Saltelli, Sensitivity analysis in practice: a guide to _______

assessing scientific models

• Andrea Saltelli, Sensitivity analysis ________________

•Lisa Gayle Davis Stanley, Dawn L. Stewart, _ Design sensitivity analysis: computational issues of sensitivity equation methods

Sensitivity analysis (SA) is “ the study of how the variation in the output of a model (numerical or otherwise) can be apportioned, qualitatively or quantitatively, to different sources of variation”.[1] However, when the assumptions are uncertain, and/or there are alternative sets of assumptions to chose from, the inference will also be also uncertain. Investigating the uncertainty in the inference (regardless of its source) goes under the name of Uncertainty analysis.

Sensitivity Analysis tries to identify those assumptions which weight the most in determining the uncertainty in the inference ('screening' sensitivity analysis).

'Quantitative' sensitivity analysis tries not only to identify but also to quantify the relative importance the influential assumptions. In the preceding discussion the term 'factor' if often used instead of 'assumption' - implying that assumptions have been translated into factors entering the model, e.g. with defined numerical values possibly drawn from factor-value distributions - while 'model output' can be used instead of inference.

Definition

• A mathematical model is defined by a series of equations, input factors, parameters, and variables aimed to characterize the process being investigated.

• Input is subject to many sources of uncertainty including errors of measurement, absence of information and poor or partial understanding of the driving forces and mechanisms. This uncertainty imposes a limit on our confidence in the response or output of the model. Further, models may have to cope with the natural intrinsic variability of the system, such as the occurrence of stochastic events.

• Good modeling practice requires that the modeler provides an evaluation of the confidence in the model, possibly assessing the uncertainties associated with the modeling process and with the outcome of the model itself. Uncertainty and Sensitivity Analysis offer valid tools for characterizing the uncertainty associated with a model.

• In models involving many input variables sensitivity analysis is an essential ingredient of model building and quality assurance. National and international agencies involved in impact assessment studies have included section devoted to sensitivity analysis in their guidelines. Examples are the European Commission, the White House Office for Budget and Management, the Intergovernmental Panel on Climate Change and the US Environmental Protection Agency.

• There are several possible procedures to perform uncertainty (UA) and sensitivity analysis (SA). The most common sensitivity analysis is sampling-based. A sampling- based sensitivity is one in which the model is executed repeatedly for combinations of values sampled from the distribution (assumed known) of the input factors. Sampling based methods can also be used to decompose the variance of the model output (see references).

Methodology

• In general, UA and SA are performed jointly by executing the model repeatedly for combination of factor values sampled with some probability distribution. The following steps can be listed:

1. Specify the target function and select the input of interest 2. Assign a distribution function to the selected factors

3. Generate a matrix of inputs with that distribution(s) through an appropriate design 4. Evaluate the model and compute the distribution of the target function

5. Select a method for assessing the influence or relative importance of each input factor on the target function.

• Sensitivity Analysis can be used to determine:

1. The model resemblance with the process under study 2. The quality of model definition

3. Factors that mostly contribute to the output variability

4. The region in the space of input factors for which the model variation is maximum 5. Optimal - or instability - regions within the space of factors for use in a subsequent

calibration study

6. Interactions between factors

Sensitivity Analysis is popular in financial applications, risk analysis, signal processing, neural networks and any area where models are developed. SA can also be used in model- based policy assessment studies see e.g. [1].

• In both cases sensitivity analysis may help understanding the contribution of the various sources of uncertainty to the model output uncertainty and system performance in general. In these cases, depending on model complexity, different sampling strategies may be advisable and traditional sensitivity indexes have to be generalized to cover multivariate sensitivity analysis, heteroskedastic effects and correlated inputs.

• Computer environmental models are increasingly used in a wide variety of studies and applications. For example global climate model are used for both short term weather forecasts and long term climate change.

• Moreover, computer models are increasingly used for environmental decision making at a local scale, for example for assessing the impact of a waste water treatment plant on a river flow, or for assessing the behavior and life length of bio-filters for contaminated waste water.

Environmental

quantities for which we need to acquire better knowledge in order to make an informed decision. On the other hand, some quantities have no influence on the predictions, so that we can save resources at no loss in accuracy by relaxing some of the conditions. See Corporate finance: Quantifying uncertainty.

Sensitivity analysis can help in a variety of other circumstances which can be handled by the settings illustrated below:

• to identify critical assumptions or compare alternative model structures

• guide future data collections

• detect important criteria

• optimize the tolerance of manufactured parts in terms of the uncertainty in the parameters

• optimize resources allocation

• model simplification or model lumping, etc.

However there are also some problems associated with sensitivity analysis in the business context:

• Variables are often interdependent, which makes examining them each individually unrealistic, e.g.: changing one factor such as sales volume, will most likely affect other factors such as the selling price.

• Often the assumptions upon which the analysis is based are made by using past experience/data which may not hold in the future.

• Assigning a maximum and minimum (or optimistic and pessimistic) value is open to subjective interpretation. For instance one persons 'optimistic' forecast may be more conservative than that of another person performing a different part of the analysis. This sort of subjectivity can adversely affect the accuracy and overall objectivity of the

* Cacuci, Dan G., Mihaela Ionescu-Bujor, Michael Navon, 2005, Sensitivity And Uncertainty Analysis:

Applications to Large-Scale Systems (Volume II), Chapman & Hall.

* Fassò A. (2007) Statistical sensitivity analysis and water quality. In Wymer L. Ed, Statistical Framework for Water Quality Criteria and Monitoring. Wiley, New York.

* Fassò A., Esposito E., Porcu E., Reverberi A.P., Vegliò F. (2003) Statistical Sensitivity Analysis of Packed Column Reactors for Contaminated Wastewater. Environmetrics. Vol. 14, n.8, 743 - 759.

* Fassò A., Perri P.F. (2002) Sensitivity Analysis. In Abdel H. El-Shaarawi and Walter W. Piegorsch (eds) Encyclopedia of Environmetrics, Volume 4, pp 1968–1982, Wiley.

* J.C. Helton, J.D. Johnson, C.J. Salaberry, and C.B. Storlie. Survey of sampling based methods for uncertainty and sensitivity analysis. Reliability Engineering and System Safety, 91:1175{1209, 2006.

* Homma, T. and A. Saltelli (1996). Importance measures in global sensitivity analysis of nonlinear models.

Reliability Engineering and System Safety, 52, 1–17.

* Kennedy, P. (2007). A guide to econometrics, Fifth edition. Blackwell Publishing.

* Morris, M. D. (1991). Factorial sampling plans for preliminary computational experiments. Technometrics, 33, 161–174.

* Rabitz, H. (1989). System analysis at molecular scale. Science, 246, 221–226.

* Saltelli, A., S. Tarantola, and K. Chan (1999). Quantitative model-independent method for global sensitivity analysis of model output. Technometrics 41(1), 39–56.

* Cacuci, Dan G. Sensitivity & Uncertainty Analysis, Volume 1: Theory, Chapman & Hall, 2003.

Bibliography

* Saltelli, A., K. Chan, and M. Scott (Eds.) (2000). Sensitivity Analysis. Wiley Series in Probability and Statistics. New York: John Wiley and Sons.

* Saltelli, A. and S. Tarantola (2002). On the relative importance of input factors in mathematical models: safety assessment for nuclear waste disposal. Journal of American Stat. Association, 97, 702–709.

* Santner, T. J.; Williams, B. J.; Notz, W.I. Design and Analysis of Computer Experiments; Springer-Verlag, 2003.

* Saltelli, A., S. Tarantola, F. Campolongo, and M. Ratto (2004). Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models. John Wiley and Sons.

* Saltelli, A., M. Ratto, S. Tarantola and F. Campolongo (2005) Sensitivity Analysis for Chemical Models, Chemical Reviews, 105(7) pp 2811 – 2828.

* Saisana M., Saltelli A., Tarantola S., 2005, Uncertainty and Sensitivity analysis techniques as tools for the quality assessment of composite indicators, Journal Royal Stat. Society A, 168 (2), 307-323.

* Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D. Saisana, M., and Tarantola, S., 2007, Global Sensitivity Analysis. The Primer, John Wiley & Sons. See A forum on sensitivity analysis for more information.

* Sobol’, I. (1990). Sensitivity estimates for nonlinear mathematical models. Matematicheskoe Modelirovanie 2, 112–118. in Russian, translated in English in Sobol’ , I. (1993). Sensitivity analysis for non-linear mathematical models. Mathematical Modeling & Computational Experiment (Engl. Transl.), 1993, 1, 407–414.

* Sobol’, I. M. Mathematical Modeling & Computational Experiment (Engl. Transl.), 1993, 1, 407.

To start with, we propose the following material :

• Cookbook

• Tutorial 1

• Tutorial 2

• Tutorial 3

• Book on Sensitivity Analysis (Preface)

A REAL TEST CASE

We also suggest some bibliographic material, where the reader will find the grater part of sensitivity analysis studies, together with applications and reviews:

• Sensitivity Analysis for Chemical Models

• Composite Indicators

• Archer, G.; Saltelli, A.; Sobol’, I. M. Journal of Statistical Computation and Simulation 1997, 58, 99

Tutorial on Sensitivity Analysis

* Box, G. E. P.; Hunter, W. G.; Hunter, J. S. In Statistics for experimenters; John Wiley and Sons, New York, 1978 * Bratley, P.; Bennet, L.F. ACM Transactions on Mathematical Software 1988, 14, 88

* Cacuci, D. G. In Sensitivity & Uncertainty Analysis, Volume 1: Theory; Chapman & Hall, 2003 * Cacuci D. G.; Ionesco-Bujor, M. Nuclear Science and Engineering 2004, 147, 204

* Campolongo, F.; Saltelli, A.; Jensen, N. R.; Wilson, J.; Hjorth, J. Journal of Atmospheric Chemistry 1999, 32, 327 * Campolongo, F.; Tarantola, S.; Saltelli, A. Computer Physics Communications 1999, 117, 75

* Campolongo, F.; Kleijnen, J.; Andres, T., In Sensitivity Analysis; Chan, K., Scott, M., Eds.; John Wiley & Sons, Chichester, 2000; p 65 * Capaldo, K. P.; Pandis, S. N. Journal of Geophysical Research 1997, 102, 23, 251

* Cawlfield, J. D. In Sensitivity Analysis; Chan, K., Scott, M., Eds.; John Wiley & Sons, Chichester, 2000; p 155

* Chan, K.; Tarantola, S.; Saltelli, A.; Sobol', I. M. In Sensitivity Analysis; Chan, K., Scott, M., Eds.; John Wiley & Sons, Chichester, 2000; p 167

* Chance, E.; Curtis, A.; Jones, I.; Kirby, C. Report, AERE-B8775, Harwell, 1977

* Cukier, R. I.; Fortuin, C. M.; Shuler, K. E.; Petschek, A. G.; Schaibly, J. H. The Journal of Chemical Physics 1973 , 59, 3873 * Cukier, R. I.; Schaibly, J. H.; Shuler, K. E. The Journal of Chemical Physics 1975, 63, 1140

* Cukier, R. I.; Levine, H. B.; Shuler, K. E. Journal of Computational Physics 1978, 26, 1

* Draper, N. R.; Smith, H. In Applied Regression Analysis; John Wiley & Sons, New York,1981 * Frey, H. C., Ed. Risk Analysis 2002, 22

* Goldsmith, C. H. In Encyclopaedia of Biostatistics; Armitage, P., Colton, T., Eds.; Wiley, New York, 1998 * Grievank, A., In Evaluating derivatives, Principles and techniques of algorithmic differentiation; SIAM, 2000 * Hakami, A.; Odman M. T.; Russel, A. G. Environmental Science and Technology 2003, 37, 2442

* Hamby, D.M. Environmental Monitoring and Assessment 1994, 32, 135 * Helton, J. C. Reliability Engineering and System Safety 1993, 42, 327

* Helton, J. C.; Davis, F.J. Reliability Engineering and System Safety 2003, 81, 23

* Helton, J. C.; McKay, M. D.; Cooke, R.; Saltelli, A. Reliability Engineering and System Safety to appear spring 2005

Bibliographic material:

* Homma, T.; Saltelli, A. Reliability Engineering and System Safety 1996, 52, 1 * Hora, S. C.; Iman, R. L. Sandia Laboratories Report 1986, SAND85-2839

* Hornberger, G. M.; Spear, R. C. Journal of Environmental management 1981, 12, 7 * Iman, R. L.; Conover, W. J. Technometrics 1979, 21, 499

* Iman R. L.; Conover W. J. Communications in Statistics: Theory and Methods 1980, A9, 1749 * Iman, R. L.; S. C. Hora., Risk Analysis 1990, 10, 401

* Ionesco-Bujor, M.; Cacuci D. G. Nuclear Science and Engineering 2004, 147, 189

* Ishigami, T.; Homma, T. In Proceedings of the ISUMA ’90. First International Symposium on Uncertainty Modelling and Analysis, University of

* Kioutsioukis, I.; Melas, D.; Zerefos, C.; Ziomas, I. Computer Physics Communications 2005, in press * Kleijnen, J. P. C. In Handbook of Simulation; Banks, J. Ed.; Wiley, New York, 1998

* Kleijnen, J. P.; Helton, J. C. Reliability Engineering and System Safety 1999, 65, 147

* Koda, M.; McRae, G. J.; Seinfeld, J. H. International Journal of Chemical Kinetics 1979, 11, 427 * Koda M.; Dogru A. H.; Seinfeld J. H. Journal of Computational Physics 1979, 30, 259

* Koda, M. Atmospheric Environment 1982, 16, 2595

* Koda M.; Seinfeld J. H. IEEE Transactions on Automatic Control 1982 , 27, 951

* Krzykacz-Hausmann, B. Technical Report GRS-A-1700, Gesellschaft fuer Reaktor Sicherheit (GRS) MbH, Garching 1990 * Krzykacz-Hausmann, B. In Proceedings of SAMO2001; Prado, P.; Bolado, R., Eds.; CIEMAT, Madrid, 2001; p 31

* Le Bras, G.; Barnes¸ I.; Hjorth, J.; Zetzsch, C.; Martinez E.; Mihalopoulos, N. Report EUR 19569 EN of the European Commission, Luxembourg., 2000

* Le Dimet, F.-X. ; Navon, I.M ; Daescu, D.N. Montly Weather Review 2002, 130, 629 * Liepmann, D.; Stephanopoulos, G. Ecological Modelling 1985, 30, 13

* Mallet, V.;Sportisse, B. Atmospheric Chemistry and Physics Discussions 2004, 4, 1371 * Maryland, USA, December 3-5; 1990, p 398

* McKay, M. D.; Beckman, R. J.; Conover, W. J. Technometrics 1979, 21, 239

* McRae, G. J.; Tilden, J. W.; Seinfeld, J. H. Computers and Chemical Engineering 1982, 6, 15 * Morbidelli M.; Varma A., Chemical Engineering Science 1988, 43,91

* Morris, M. D. Technometrics 1991, 33, 161

* Oakley, J.; O'Hagan, A. Journal of the Royal Statistical Society, Series B 66 2004, 751 * O'Hagan, A.; Kennedy, M. C.; Oakley J. E. Bayesian Statistics 1999, 6, 503

* Pandis, S. N.; Seinfeld, J. H. Journal of Geophysical Research 1989, 94, 1105 * Pierce, T. H; Cukier, R. I. Journal of Computational Physics 1981, 41, 427

* Rabitz, H.; Kramer, M.; Dacol, D. Annal Review of Physical Chemistry 1983 34, 419 * Rabitz, H. Science, 1989, 246, 221

* Rabitz, H.; Alis, Ö., Journal of Mathematical Chemistry 1999, 25, 197

* Rabitz, H.; Aliş, Ö. F.; Shorter, J.; Shim, K. Computer Physics Communications 1999, 117, 11

* Rabitz, H.; Aliş, Ö. F. In Sensitivity Analysis; Chan, K., Scott, M., Eds.; John Wiley & Sons, Chichester, 2000; p 199 * Ratto, M.; Tarantola, S.; Saltelli, A. Computer Physycs Communications 2001, 136, 212.

* Ratto, M.; Saltelli, A.; Tarantola, S.; Young, P. Journal of the Royal Statistical Society - B 2004, submitted

* Rosen, R., In A Comprehensive Inquiry into the Nature, Origin, and Fabrication of Life; Columbia University Press, New York, 1991 * Sacks, J.; Welch, W. J.; Mitchell, T. J.; Wynn, H. P. Statistical Science 1989, 4, 409

* Saltelli, A. Computer Physics Communications 2002, 145, 280

* Saltelli, A.; Andres, T. H.; Homma, T. Computational Statistics and Data Analysis 1993, 15, 211 * Saltelli, A.; Hjorth, J. Journal of Atmospheric Chemistry 1995, 21, 187

* Saltelli, A.; Sobol', I. M. Reliability Engineering and System Safety 1995, 50, 225 * Saltelli, A.; Bolado, R. Computational Statistics and Data Analysis 1998, 26, 445 * Saltelli, A. Journal of Geophysical Research, 1999, 104, 3789; 1999, 104, 24013 * Saltelli, A.; Chan, K.; Scott, M., Eds. Computer Physics Communications 1999, 117

Bibliographic material:………

* Saltelli, A.; Tarantola S.; Chan, K. Technometrics 1999, 41, 39

* Saltelli A.; Tarantola, S. Journal of American Statistical Association 2002, 97, 702

* Saltelli, A.; Tarantola, S.; Campolongo, F.; Ratto, M. In Sensitivity Analysis in Practice. A Guide to Assessing Scientific Models; John Wiley & Sons publishers, 2004.

* Saltelli, A., In Sensitivity Analysis; Chan, K., Scott, M., Eds.; John Wiley & Sons, Chichester, 2000.

* Saltelli, A.; Tarantola, S.; Campolongo, F. Statistical Science 2000, 15, 377

* Santner, T. J.; Williams, B. J.; Notz, W.I. In Design and Analysis of Computer Experiments; Springer-Verlag, 2003 * Schaibly, J. H.; Shuler, K. E. The Journal of Chemical Physics 1973, 59, 3879

* Scott, M.; Saltelli, A., Eds. Journal of Statistical Computation and Simulation 1997, 57 * Shim, K.; Rabitz, H. Physical Review B 1998, 58, 1940

* Scire, J.J.; Dryer, F.L.; Yetter, R.A. International Journal of Chemical Kinetics 2001, 33, 784 * Shorter, J. A.; Ip, P. C.; Rabitz, H. Journal of Physical Chemistry A 1999, 103, 7192

* Shorter, J. A.; Ip, P. C.; Rabitz, H. Geophysical Research Letters 2000, 27, 3485

* Sobol', I.M. USSR Computational Mathematics and Mathematical Physics 1976, 16(5), 1332 * Sobol’, I. M. Mathematical Modelling & Computational Experiment (Engl. Transl.) 1993, 1, 407 * Tarantola, S.; Saltelli, A., Eds. Reliability Engineering and System Safety 2003, 79

* Tarantola, S.; Gatelli, D.; Mara, T. Reliability Engineering and System Safety 2005, forthcoming

* Tomlin, A. S.;Turányi, T. in Low temperature Combustion and Autoignition, Pilling M.J; Hancock G. Eds, Elsevier 1997, 293 * Turányi, T. Journal of Mathematical Chemistry, 1990, 5, 203

* Turányi, T.; Rabitz, H. In Sensitivity Analysis; Chan, K., Scott, M., Eds.; John Wiley & Sons, Chichester, 2000; p 81 * Turányi, T.; Zalotai, L.; Dóbé, S.; Bercé, T. Physical Chemistry Chemical Physiscs 2002, 4, 2568

* Varma, A.; Morbidelli M.; Wu H. In Parametric Sensitivity in Chemical Systems; Cambridge, 1999 * Vuilieumier, L.; Harley , R.; Brown, N. J. Environmental Science and Technology 1997, 31, 1206

* Welch, W. J.; Buck, R. J.; Sacks, J.; Wynn, H. P.; Mitchell, T. J.; Morris, M. D. Technometrics 1992, 34 , 15 * Young, P. C.; Parkinson, S. D.; Lees, M. Journal of Applied Statistics 1996, 23, 165

* Young, P. Computer Physics Communication 1999, 117, 113

* http://www.reactiondesign.com/; see also CHEMKIN page at SANDIA Labs. http://www.ca.sandia.gov/chemkin/

* simlab.jrc.ec.europa.eu

Prepared for the

MIT System Dynamics in Education Project Under the Supervision of

Dr. Jay W. Forrester

by

Lucia Breierova Mark Choudhari September 6, 1996

Vensim Examples added October 2001

7.1 LEMONADE STAND MODEL 7.2 EPIDEMICS MODEL

7.3 COFFEEHOUSE MODEL

Lemonade Stand

In the first exploration, let’s look at a lemonade stand located on a college campus.

As usual, we are particularly interested in the behavior of the stock, the number of cups of lemonade that are ready to be sold to customers. The stand is open eight hours every day.

Howard, the owner, is the only person working in the stand.

Coffeehouse

We now return to Howard, the owner of the lemonade stand on a college campus.

Howard realized that it could be more profitable for him to sell coffee because students tend to drink more coffee than lemonade, and they drink it at any time of the day and night. Therefore, he closed his lemonade stand and opened a 24-hour Coffeehouse.

Howard bases the Coffeehouse model on the model he used in his lemonade stand to model the number of cups of “Coffee ready.” We will run the simulation over a

Epidemics

In the second exploration we look at an epidemics model. The model was already used in a previous chapter in Road Maps, so it is possible that you have already built it.

Specific parameter values can change the appearance of the graphs representing the behavior of the system. But significant changes in behavior do not occur for all parameters. System dynamics models are in general insensitive to many parameter changes. It is the structure of the system, and not the parameter values, that has most influence on the behavior of the system.

Sensitivity analysis is an important tool in the model building process. By showing that the system does not react greatly to a change in a parameter value, it reduces the modeler’s uncertainty in the behavior. In addition, it gives an opportunity for a better understanding of the dynamic behavior of the system.

We encourage you to experiment with the three models from this paper (as well as any other models that you have built) on your own. For example, try to change several parameters at the same time, observe the behavior produced, and compare it to the conclusions in this paper. Can you suggest any parameter values that would produce the

“optimal,” or most desirable behavior? The use of sensitivity analysis in such policy analysis will be explored in a later paper in this series.

References

1. Batty, M. and Xie, Y. (1994a) "Modelling inside GIS: Part 2. Selecting and calibrating urban models using Arc/Info," International Journal of Geographical Information Systems, vol. 8, no. 5, pp. 429-450.

2. Batty M, and Xie Y, 1994b, "From Cells to Cities" Environment and Planning B 21 S31-S48.

3. Bell, C., Acevedo, W. and J.T. Buchanan. (1995) "Dynamic mapping of urban regions:

Growth of the San Francisco Sacramento region," Proceedings, Urban and Regional Information Systems Association, San Antonio, pp. 723-734. (Appendix 11.3)

4. Clarke, K. C. Brass, J. A. and Riggan, P. (1995) "A cellular automaton model of wildfire propagation and extinction" Photogrammetric Engineering and Remote Sensing, vol. 60, no. 11, pp. 1355-1367.

5. Clarke, K.C., Gaydos, L., Hoppen, S., (1996) "A self-modifying cellular automaton model of historical urbanization in the San Francisco Bay area," Environment and Planning B. (in press).

6. Couclelis H, 1985, "Cellular worlds: a framework for modeling micro-macro dynamics"

Environment and Planning A 17 585-596

7. Debaeke, Ph., Loague, K., Green, R.E., 1991, "Statistical and graphical methods for evaluating solute transport models: overview and application," Journal of Contaminant Hydrology 7 51-73 .

change," Proceedings of the International Cartographic Association Conference, Barcelona, Spain, International Cartographic Association, pp. 1174-1178.

2. Kirkby, M.J., Naden, P.S., Burt, T.P., Butcher, D.P. (1987) Computer Simulation in Physical Geography. John Wiley & Sons.

3. Kirtland, D., Gaydos, L. Clarke, K. DeCola, L., Acevedo, W. and Bell, C. (1994) An Analysis of Human-Induced Land Transformations in the San Francisco Bay/Sacramento Area. World

Resources Review, vol. 6, no. 2, pp. 206-217.

4. Oreskes, N., Shrader-Frechette, K., Belitz, K., (1994) Verification, Validation, and Confirmation of Numerical Models in the Earth Sciences. Science, vol. 263, pp. 641-646.

5. United States Geological Survey (1994) Human Induced Land Transformations Home Page:

http://geo.arc.nasa.gov/usgs/HILTStart http://geo.arc.nasa.gov/usgs/HILTStart

6. White, R. and Engelen, G. (1992) Cellular automata and fractal urban form: a cellular modelling approach to the evolution of urban land use patterns, Working Paper no. 9264, Research Institute for Knowledge Systems (RIKS), Maastricht, The Netherlands.

7. Keith C. Clarke, Department of Geology and Geography, Hunter College- City University of New York, and the City University of New York Graduate School and University Center.

8. Stacy Hoppen, Graduate Student, Department of Geology and Geography, Hunter College-City University of New York.

9. Leonard J. Gaydos, US Geological Survey, EROS Data Center, NASA¬Ames Research Center, Moffett Field, CA.

Sensitivity analysis

Sensitivity analysis (SA) is the study of how the variation (uncertainty) in the output of a mathematical model can be apportioned, qualitatively or quantitatively, to different sources of variation in the input of a model [1]. Put another way, it is a technique for systematically changing parameters in a model to determine the effects of such changes.

In more general terms uncertainty and sensitivity analyses investigate the robustness of a study when the study includes some form ofmathematical modelling. Sensitivity analysis can be useful to computer modellers for a range of purposes[2], including:

• support decision making or the development of recommendations for decision makers (e.g.

testing the robustness of a result);

• enhancing communication from modellers to decision makers (e.g. by making recommendations more credible, understandable, compelling or persuasive);

• increased understanding or quantification of the system (e.g. understanding relationships between input and output variables); and

• model development (e.g. searching for errors in the model).

While uncertainty analysis studies the overall uncertainty in the conclusions of the study, sensitivity analysis tries to identify what source of uncertainty weights more on the study's conclusions. For example, several guidelines for modelling (see e.g. one from the US EPA) or forimpact assessment (see one from the European Commission) prescribe sensitivity analysis as a tool to ensure the quality of the modelling/assessment.

The problem setting in sensitivity analysis has strong similarities with design of experiments. In design of experiments one studies the effect of some process or intervention (the 'treatment') on some objects (the 'experimental units'). In sensitivity analysis one looks at the effect of varying the inputs of a mathematical model on the output of the model itself.

In both disciplines one strives to obtain information from the system with a minimum of physical or numerical experiments.

In uncertainty and sensitivity analysis there is a crucial trade off between how scrupulous an analyst is in exploring the input assumptions and how wide the resulting inference may be. The point is well illustrated by the econometrician Edward E. Leamer (1990) [3]:

I have proposed a form of organized sensitivity analysis that I call ‘global sensitivity analysis’ in which a neighborhood of alternative assumptions is selected and the corresponding interval of inferences is identified. Conclusions are judged to be sturdy only if the neighborhood of assumptions is wide enough to be credible and the corresponding interval of inferences is narrow enough to be useful.

Note Leamer’s emphasis is on the need for 'credibility' in the selection of assumptions. The easiest way to invalidate a model is to demonstrate that it is fragile with respect to the uncertainty in the assumptions or to show that its assumptions have not been taken 'wide enough'. The same concept is expressed by Jerome R. Ravetz, for whom bad modeling is when uncertainties in inputs must be suppressed lest outputs become indeterminate.[4]

In modern econometrics the use of sensitivity analysis to anticipate criticism is the subject of one of the ten commandments of applied econometrics (from Kennedy, 2007[5] ):

… Sensitivity analysis

Thou shall confess in the presence of sensitivity. Corollary: Thou shall anticipate criticism [···] When reporting a sensitivity analysis, researchers should explain fully their specification search so that the readers can judge for themselves how the results may have been affected. This is basically an ‘honesty is the best policy’ approach, advocated by Leamer, (1978[6]).

The use of mathematical modelling can be the subject of controversies, see Nassim Nicholas Taleb[7] in Economics, and Orrin H. Pilkey and Linda Pilkey Jarvis[8] in Environmental Sciences. As noted by the latter Authors, this increases the relevance of sensitivity analysis in today's modelling practice[1] .

Mathematical problems met in social, economic or natural sciences may entail the use of mathematical models, which generally do not lend themselves to a straightforward understanding of the relationship between input factors (what goes into the model) and output (the model’s dependent variables). Such an appreciation, i.e. the understanding of how the model behaves in response to changes in its inputs, is of fundamental importance to ensure a correct use of the models.

• A mathematical model is defined by a series of equations, input factors, parameters, and variables aimed to characterize the process being investigated.

Ideal scheme of a possibly sampling-based sensitivity analysis. Uncertainty arising from different sources — errors in the data, parameter estimation procedure, alternative model structures — are propagated through the model for uncertainty analysis and their relative importance is quantified via sensitivity analysis.

… Sensitivity analysis

Errors

In sensitivity analysis Type I error is assessing as important a non important factor, and Type II error assessing as non important an important factor. Type III error corresponds to analyzing the wrong problem, e.g. via an incorrect specification of the input uncertainties.

Possible pitfalls in sensitivity analysis are:

•Unclear purpose of the analysis. Different statistical tests and measures are applied to the problem and different factors rankings are obtained. The test should instead be tailored to the purpose of the analysis, e.g. one uses Monte Carlo filtering if one is interested in which factors are most responsible for generating high/low values of the output.

•Too many model outputs are considered. This may be acceptable for quality assurance of sub-models but should be avoided when presenting the results of the overall analysis.

•Piecewise sensitivity. This is when one performs sensitivity analysis on one sub-model at a time. This approach is non conservative as it might overlook interactions among factors in different sub-models (Type II error).

• Cruz, J. B., editor, (1973) System Sensitivity Analysis, Dowden, Hutchinson & Ross, Stroudsburg, PA.

• Cruz, J. B. and Perkins, W.R., (1964), A New Approach to the Sensitivity Problem in Multivariable Feedback System Design, IEEE TAC, Vol. 9, 216-223.

• Fassò A. (2007) Statistical sensitivity analysis and water quality. In Wymer L.

Ed, Statistical Framework for Water Quality Criteria and Monitoring. Wiley, New York.

• Fassò A., Esposito E., Porcu E., Reverberi A.P., Vegliò F. (2003) Statistical Sensitivity Analysis of Packed Column Reactors for Contaminated Wastewater. Environmetrics.

Vol. 14, n.8, 743 - 759.

• Fassò A., Perri P.F. (2002) Sensitivity Analysis. In Abdel H. El-Shaarawi and Walter W.

Piegorsch (eds) Encyclopedia of Environmetrics, Volume 4, pp 1968–1982, Wiley.

• Saltelli, A., S. Tarantola, and K. Chan (1999). Quantitative model-independent method for global sensitivity analysis of model output.Technometrics 41(1), 39–56.

• Santner, T. J.; Williams, B. J.; Notz, W.I. Design and Analysis of Computer Experiments;

Springer-Verlag, 2003.

• Haug, Edward J.; Choi, Kyung K.; Komkov, Vadim (1986) Design sensitivity analysis of structural systems. Mathematics in Science and Engineering, 177. Academic Press, Inc., Orlando, FL.

Sensitivity Analysis ~ http://www.investopedia.com/terms/s/sensitivityanalysis.asp

What Does Sensitivity Analysis Mean?

A technique used to determine how different values of an independent variable will impact a particular dependent variable under a given set of assumptions. This technique is used within specific boundaries that will depend on one or more input variables, such as the effect that changes in interest rates will have on a bond's price.

Sensitivity analysis is a way to predict the outcome of a decision if a situation turns out to be different compared to the key prediction(s).

Investopedia explains Sensitivity Analysis

Sensitivity analysis is very useful when attempting to determine the impact the

actual outcome of a particular variable will have if it differs from what was previously assumed. By creating a given set of scenarios, the analyst can determine how changes in one variable(s) will impact the target variable.

For example, an analyst might create a financial model that will value a company's equity (the dependent variable) given the amount of earnings per share (an independent

variable) the company reports at the end of the year and the company's price-to-earnings multiple (another independent variable) at that time. The analyst can create a table of predicted price-to-earnings multiples and a corresponding value of the company's equity based on different values for each of the independent variables.

Spreadsheets and the Case Projects

The Dynamic Strategic Planning workbook is accompanied by a number of spreadsheet-based tools for data analysis. We have supplied these tools so that the users of this workbook can concentrate upon the use and implementation of decision analysis and strategic planning, rather than focusing upon the mechanics of the mathematics underlying their use.

The current form of the spreadsheets is a consequence of a combination of factors:

academic research, pedagogical design, and in-class experiences. Based upon new developments, they are being routinely improved.

However, no amount of care in tool design can substitute for expertise on the part of the user.

The case projects have been designed assuming that these tools will be used effectively. The purpose of this document is to assure that you, the user of these tools, are prepared to exploit them to their fullest - specifically, that you are able to make use of spreadsheet sensitivity analysis tools.

http://msl1.mit.edu/rdn/d_table.pdf

Sensitivity Analysis Using Excel

The main goal of sensitivity analysis is to gain insight into which assumptions are critical, i.e., which assumptions affect choice. The process involves various ways of changing input values of the model to see the effect on the output value. In some decision situations you can use a single model to investigate several alternatives. In other cases, you may use a separate spreadsheet model for each alternative.

MANUAL WHAT-IF ANALYSIS

Using this approach, you enter values into cells C4:C6 and see what the effect is on net cash flow.

For example, with the predetermined price of $29, you may think that Units Sold will be in the range between 500 and 900 units.

Keeping other input assumptions at base case, the corresponding Net Cash Flows are

$–1,500 and $6,900. When we vary a single input assumption, keeping all other input assumptions at their base case values, we say we are doing "one at a time" or

"singlefactor“ sensitivity analysis.

http://www.sce.carleton.ca/faculty/chinneck/po/Chapter6.pdf

References: Sensitivity Analysis

Global Sensitivity AnalysisThe Primer

Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D. Saisana, M., and Tarantola, S., 2008, John Wiley & Sons (ISBN: 978- 0-470-05997-5)Who needs Sensitivity Analysis

Tutorial on Sensitivity Analysis

SimLab Software for Sensitivity Analysis

What's New

•Sixth International Conferenceon Sensitivity Analysis of Model Output, Bocconi University of Milan, 19-22 July 2010

•Sixth Summer School onSensitivity Analysis of Model Output, Villa La Stella, Fiesole - Florence, 14-17 September 2010

Multi-disciplinary Design Optimization

Centre for Aerospace Systems Design & Engineering Department of Aerospace Engineering

Indian Institute of Technology Mumbai 400 076

sensitivity.ppt (67 slides)

Sensitivity Analysis – ^other References

• Supporting Financial Statements - The Handbook of Business Planning ~

http://www.jian.com/software/business-plan/sensitivity-analysis.pdf

• Sensitivity analysis: strategies, methods, concepts, examples, David J. Pannell, School of Agricultural and Resource Economics, University of Western Australia, Crawley 6009, Australia ~

• SENSITIVITY AND RISK ANALYSES ~

http://www.adb.org/documents/handbooks/water_supply_projects/Chap7-r6.PDF

• Sensitivity Analysis of LP ~ http://www.youtube.com/watch?v=rACFwIt2szk

• What is sensitivity analysis ~

http://www.medicine.ox.ac.uk/bandolier/painres/download/whatis/What_is_sens_analy.pdf

• Sensitivity Analysis ~ http://web.mit.edu/15.053/www/AMP-Chapter-03.pdf

• Tutorial_09_Sensitivity_Analysis ~

http://www.rocscience.com/downloads/slide/webhelp/pdf_files/tutorials/Tutorial_09_Sensitivity_Anal ysis.pdf

• Cash Flow Sensitivity Analysis ~

www.jaxworks.com/Cash%20Flow%20Sensitivity%20Analysis.xls