Nash–Sutcliffe model efficiency coefficient

From HandWiki
Short description: Used to assess the predictive power of hydrological models

The Nash–Sutcliffe model efficiency coefficient (NSE) is used to assess the predictive skill of hydrological models. It is defined as:

[math]\displaystyle{ NSE=1-\frac {\sum_{t=1}^T\left(Q_o^t-Q_m^t\right)^2} {\sum_{t=1}^T\left(Q_o^t-\overline{Q}_o\right)^2} }[/math]

where [math]\displaystyle{ \overline{Q}_o }[/math] is the mean of observed discharges, and [math]\displaystyle{ Q_m }[/math] is modeled discharge. [math]\displaystyle{ Q_o^t }[/math] is observed discharge at time t.[1]

The Nash–Sutcliffe efficiency is calculated as one minus the ratio of the error variance of the modeled time-series divided by the variance of the observed time-series. In the situation of a perfect model with an estimation error variance equal to zero, the resulting Nash–Sutcliffe Efficiency equals 1 (NSE = 1). Conversely, a model that produces an estimation error variance equal to the variance of the observed time series results in a Nash–Sutcliffe Efficiency of 0.0 (NSE = 0). In reality, NSE = 0 indicates that the model has the same predictive skill as the mean of the time-series in terms of the sum of the squared error. In the case of a modeled time series with an estimation error variance that is significantly larger than the variance of the observations, the NSE becomes negative. An efficiency less than zero (NSE < 0) occurs when the observed mean is a better predictor than the model. Values of the NSE nearer to 1, suggest a model with more predictive skill. Subjective application of different NSE values as thresholds of sufficiency have been suggested by several authors.[2][3][4][5] For the application of NSE in regression procedures (i.e. when the total sum of squares can be partitioned into error and regression components), the Nash–Sutcliffe efficiency is equivalent to the coefficient of determination (R2), thus ranging between 0 and 1.

In some applications such as automatic calibration or machine learning, the NSE lower limit of (-∞) creates problems. To eliminate this problem and re-scale the NSE to lie solely within the range of {0,1} normalization, use the following equation that yields a Normalized Nash–Sutcliffe Efficiency (NNSE) [6][7]

[math]\displaystyle{ NNSE = \frac{1}{2-NSE} }[/math]

Note that NSE=1 corresponds to NNSE=1, NSE=0 corresponds to NNSE=0.5, and NSE=-∞ corresponds to NNSE=0. This convenient re-scaling of the NSE allows for easier interpretation, and use of the NSE measure in parameter estimation schemes used in model calibration.

The NSE coefficient is sensitive to extreme values and might yield sub-optimal results when the dataset contains large outliers. To address this a modified version of NSE has been suggested where the sums of squares in the numerator and denominator of NSE are raised to 1 instead of 2 and the resulting modified NSE values compared to the original NSE values to assess the potential effect of extreme values.[8] Importantly, this modification relies on the absolute value in lieu of the square power:

[math]\displaystyle{ NSE_1=1-\frac {\sum_{t=1}^T\left|Q_o^t-Q_m^t\right|} {\sum_{t=1}^T\left|Q_o^t-\overline{Q}_o\right|} }[/math]

Many scientists apply a logarithmic transformation to the observed and simulated data prior to calculating the NSE, and this is referred to as the LNSE.[9] This is helpful when the emphasis is on simulating low flows, as it increases the relative weight of small observations. Note that the log-transform should not be used with the related Kling-Gupta Efficiency (KGE), as the results will depend on the units and not be meaningful.

A test significance for NSE to assess its robustness has been proposed whereby the model can be objectively accepted or rejected based on the probability value of obtaining NSE greater than some subjective threshold.

Nash–Sutcliffe efficiency can be used to quantitatively describe the accuracy of model outputs other than discharge. This indicator can be used to describe the predictive accuracy of other models as long as there is observed data to compare the model results to. For example, Nash–Sutcliffe efficiency has been reported in scientific literature for model simulations of discharge; water quality constituents such as sediment, nitrogen, and phosphorus loading.[5] Other applications are the use of Nash–Sutcliffe coefficients to optimize parameter values of geophysical models, such as models to simulate the coupling between isotope behavior and soil evolution.[10]

Criticism

The Nash–Sutcliffe Coefficient masks important behaviors that if re-cast can aid in the interpreted as the different sources of model behavior in terms of bias, random, and other components.[11] The alternate Kling-Gupta efficiency is intended to improve upon NSE by incorporating bias and variance terms.[12]

See also

References

  1. ↑ Nash, J. E.; Sutcliffe, J. V. (1970). "River flow forecasting through conceptual models part I — A discussion of principles". Journal of Hydrology 10 (3): 282–290. doi:10.1016/0022-1694(70)90255-6. Bibcode: 1970JHyd...10..282N. 
  2. ↑ McCuen, R.H.; Knight, Z; Cutter, A.G. (2006). "Evaluation of the Nash–Sutcliffe efficiency index". Journal of Hydrologic Engineering 11 (6): 597–602. doi:10.1061/(ASCE)1084-0699(2006)11:6(597). 
  3. ↑ Criss, R.E; Winston, W.E (2008). "Do Nash values have value? Discussion and alternate proposals". Hydrological Processes 22 (14): 2723–2725. doi:10.1002/hyp.7072. Bibcode: 2008HyPr...22.2723C. 
  4. ↑ Ritter, A.; Muñoz-Carpena, R. (2013). "Performance evaluation of hydrological models: statistical significance for reducing subjectivity in goodness-of-fit assessments". Journal of Hydrology 480 (1): 33–45. doi:10.1016/j.jhydrol.2012.12.004. Bibcode: 2013JHyd..480...33R. 
  5. ↑ 5.0 5.1 Moriasi, D. N.; Arnold, J. G.; Van Liew, M. W.; Bingner, R. L.; Harmel, R. D.; Veith, T. L. (2007). "Model Evaluation Guidelines for Systematic Quantification of Accuracy in Watershed Simulations". Transactions of the ASABE 50 (3): 885–900. doi:10.13031/2013.23153. http://swat.tamu.edu/media/1312/moriasimodeleval.pdf. 
  6. ↑ Mathevet, Thibault; Michel, Claude; Andréassian, Vazken; Perrin, Charles (2006). "A bounded version of the Nash-Sutcliffe criterion for better model assessment on large sets of basins". IHS Publication 307: 211–220. 
  7. ↑ Nossent, J; Bauwens, W (2012). "Application of a normalized Nash–Sutcliffe efficiency to improve the accuracy of the Sobol'sensitivity analysis of a hydrological model". EGUGA: 237. Bibcode: 2012EGUGA..14..237N. 
  8. ↑ Legates, D.R.; McCabe, G.J. (1999). "Evaluating the use of "goodness-of-fit" measures in hydrologic and hydroclimatic model validation". Water Resour. Res. 35 (1): 233–241. doi:10.1029/1998WR900018. Bibcode: 1999WRR....35..233L. 
  9. ↑ Lamontagne, Jonathan R.; Barber, Caitline A.; Vogel, Richard M. (September 2020). "Improved Estimators of Model Performance Efficiency for Skewed Hydrologic Data". Water Resources Research 56 (9). doi:10.1029/2020WR027101. ISSN 0043-1397. https://onlinelibrary.wiley.com/doi/10.1029/2020WR027101. Retrieved 2021-07-07. 
  10. ↑ Campforts, Benjamin; Vanacker, Veerle; Vanderborght, Jan; Baken, Stijn; Smolders, Erik; Govers, Gerard (2016). "Simulating the mobility of meteoric 10 Be in the landscape through a coupled soil-hillslope model (Be2D)". Earth and Planetary Science Letters 439: 143–157. doi:10.1016/j.epsl.2016.01.017. ISSN 0012-821X. Bibcode: 2016E&PSL.439..143C. 
  11. ↑ Gupta, H.V.; Kling, H (2011). "On typical range, sensitivity, and normalization of Mean Squared Error and Nash‐Sutcliffe Efficiency type metrics". Water Resources Research 47 (10): W10601. doi:10.1029/2011WR010962. Bibcode: 2011WRR....4710601G. 
  12. ↑ Knoben, W.J; Freer, J.E.; Woods, R.A. (2019). "Inherent benchmark or not? Comparing Nash–Sutcliffe and Kling-Gupta efficiency scores". Hydrology and Earth System Sciences 23 (10): 4323–4331. doi:10.5194/hess-23-4323-2019. 



Retrieved from "https://handwiki.org/wiki/index.php?title=Nash–Sutcliffe_model_efficiency_coefficient&oldid=3373300"