View source for Nash–Sutcliffe model efficiency coefficient

{{Short description|Used to assess the predictive power of hydrological models}}
The '''Nash–Sutcliffe model efficiency coefficient (NSE)''' is used to assess the predictive skill of [[Earth:Hydrology|hydrological]] models. It is defined as:

: <math>
NSE=1-\frac
{\sum_{t=1}^T\left(Q_o^t-Q_m^t\right)^2}
{\sum_{t=1}^T\left(Q_o^t-\overline{Q}_o\right)^2}
</math>

where <math display="inline">\overline{Q}_o</math> is the mean of observed discharges, and <math display="inline">Q_m</math> is modeled discharge. <math display="inline"> Q_o^t</math> is observed discharge at time ''t''.<ref>{{cite journal |last1=Nash |first1=J. E. |first2=J. V. |last2=Sutcliffe |year=1970 |title=River flow forecasting through conceptual models part I — A discussion of principles |journal=Journal of Hydrology |volume=10 |issue=3 |pages=282–290 |doi=10.1016/0022-1694(70)90255-6 |bibcode=1970JHyd...10..282N }}</ref>

The Nash–Sutcliffe efficiency is calculated as one minus the ratio of the error variance of the modeled time-series divided by the variance of the observed time-series. In the situation of a perfect model with an estimation error variance equal to zero, the resulting Nash–Sutcliffe Efficiency equals 1 (''NSE''&nbsp;=&nbsp;1). Conversely, a model that produces an estimation error variance equal to the variance of the observed time series results in a Nash–Sutcliffe Efficiency of 0.0 (''NSE''&nbsp;=&nbsp;0).  In reality, ''NSE''&nbsp;=&nbsp;0  indicates that the model has the same predictive skill as the mean of the time-series in terms of the sum of the squared error.  In the case of a modeled time series with an estimation error variance that is significantly larger than the variance of the observations, the NSE becomes negative.  An efficiency less than zero (''NSE''&nbsp;<&nbsp;0) occurs when the observed mean is a better predictor than the model. Values of the NSE nearer to 1, suggest a model with more predictive skill. Subjective application of different NSE values as thresholds of sufficiency have been suggested by several authors.<ref>{{Cite journal|last1=McCuen|first1=R.H.|last2=Knight|first2=Z|last3=Cutter|first3=A.G.|date=2006|title=Evaluation of the Nash–Sutcliffe efficiency index|journal=Journal of Hydrologic Engineering|volume=11|issue=6|pages=597–602|doi=10.1061/(ASCE)1084-0699(2006)11:6(597)}}</ref><ref>{{cite journal|last1=Criss|first1=R.E|last2=Winston|first2=W.E|date=2008|title=Do Nash values have value? Discussion and alternate proposals|journal=Hydrological Processes|volume=22|number=14|pages=2723–2725|doi=10.1002/hyp.7072|bibcode=2008HyPr...22.2723C}}</ref><ref name=":0">{{Cite journal|last1=Ritter|first1=A.|last2=Muñoz-Carpena|first2=R.|date=2013|title=Performance evaluation of hydrological models: statistical significance for reducing subjectivity in goodness-of-fit assessments|journal=Journal of Hydrology|volume=480|issue=1|pages=33–45|doi=10.1016/j.jhydrol.2012.12.004|bibcode=2013JHyd..480...33R}}</ref><ref name=":1" />  For the application of NSE in regression procedures (i.e. when the total sum of squares can be partitioned into error and regression components), the Nash–Sutcliffe efficiency is equivalent to the [[Coefficient of determination|coefficient of determination]] (''R''<sup>2</sup>), thus ranging between 0 and 1.

In some applications such as automatic calibration or machine learning, the NSE lower limit of (-∞) creates problems.  To eliminate this problem and re-scale the NSE to lie solely within the range of {0,1} normalization, use the following equation that yields a Normalized Nash–Sutcliffe Efficiency (NNSE)
<ref>{{Cite journal| pages = 211–220| last1 = Mathevet| first1 = Thibault| last2 = Michel| first2 = Claude| last3 = Andréassian| first3 = Vazken| last4 = Perrin| first4 = Charles| title = A bounded version of the Nash-Sutcliffe criterion for better model assessment on large sets of basins| journal = IHS Publication 307| date = 2006}}</ref><ref>{{Cite journal|last1=Nossent|first1=J|last2=Bauwens|first2=W|date=2012|title=Application of a normalized Nash–Sutcliffe efficiency to improve the accuracy of the Sobol'sensitivity analysis of a hydrological model|journal=EGUGA|pages=237|bibcode=2012EGUGA..14..237N}}</ref>

:<math>
NNSE = \frac{1}{2-NSE}
</math>

Note that NSE=1 corresponds to NNSE=1, NSE=0 corresponds to NNSE=0.5, and NSE=-∞ corresponds to NNSE=0.  This convenient re-scaling of the NSE allows for easier interpretation, and use of the NSE measure in parameter estimation schemes used in model calibration.

The NSE coefficient is sensitive to extreme values and might yield sub-optimal results when the dataset contains large outliers. To address this a modified version of ''NSE'' has been suggested where the sums of squares in the numerator and denominator of ''NSE'' are raised to 1 instead of 2 and the resulting modified ''NSE'' values compared to the original ''NSE'' values to assess the potential effect of extreme values.<ref>{{Cite journal|last1=Legates|first1=D.R.|last2=McCabe|first2=G.J.|date=1999|title=Evaluating the use of "goodness-of-fit" measures in hydrologic and hydroclimatic model validation|journal=Water Resour. Res.|volume=35|issue=1|pages=233–241|doi=10.1029/1998WR900018|bibcode=1999WRR....35..233L|doi-access=}}</ref> Importantly, this modification relies on the absolute value in lieu of the square power:

: <math>
NSE_1=1-\frac
{\sum_{t=1}^T\left|Q_o^t-Q_m^t\right|}
{\sum_{t=1}^T\left|Q_o^t-\overline{Q}_o\right|}
</math>

Many scientists apply a logarithmic transformation to the observed and simulated data prior to calculating the NSE, and this is referred to as the LNSE.<ref>{{Cite journal| doi = 10.1029/2020WR027101| issn = 0043-1397| volume = 56| issue = 9| last1 = Lamontagne| first1 = Jonathan R.| last2 = Barber| first2 = Caitline A.| last3 = Vogel| first3 = Richard M.| title = Improved Estimators of Model Performance Efficiency for Skewed Hydrologic Data| journal = Water Resources Research| accessdate = 2021-07-07| date = September 2020| url = https://onlinelibrary.wiley.com/doi/10.1029/2020WR027101}}</ref> This is helpful when the emphasis is on simulating low flows, as it increases the relative weight of small observations. Note that the log-transform should not be used with the related Kling-Gupta Efficiency (KGE), as the results will depend on the units and not be meaningful.

A test significance for ''NSE'' to assess its robustness has been proposed whereby the model can be objectively accepted or rejected based on the probability value of obtaining ''NSE'' greater than some subjective threshold.

Nash–Sutcliffe efficiency can be used to quantitatively describe the accuracy of model outputs other than discharge.  This indicator can be used to describe the predictive accuracy of other models as long as there is observed data to compare the model results to. For example, Nash–Sutcliffe efficiency has been reported in scientific literature for model simulations of discharge; water quality constituents such as [[Earth:Sediment|sediment]], nitrogen, and phosphorus loading.<ref name=":1">{{cite journal |last1=Moriasi |first1=D. N. |last2=Arnold |first2=J. G. |last3=Van Liew |first3=M. W. |last4=Bingner |first4=R. L. |last5=Harmel |first5=R. D. |last6=Veith |first6=T. L. |year=2007 |title=Model Evaluation Guidelines for Systematic Quantification of Accuracy in Watershed Simulations |journal=Transactions of the ASABE |volume=50 |issue=3 |pages=885–900 |url=http://swat.tamu.edu/media/1312/moriasimodeleval.pdf |doi=10.13031/2013.23153 }}</ref> Other applications are the use of Nash–Sutcliffe coefficients to optimize parameter values of geophysical models, such as models to simulate the coupling between isotope behavior and soil evolution.<ref>{{Cite journal|last1=Campforts|first1=Benjamin|last2=Vanacker|first2=Veerle|last3=Vanderborght|first3=Jan|last4=Baken|first4=Stijn|last5=Smolders|first5=Erik|last6=Govers|first6=Gerard|date=2016|title=Simulating the mobility of meteoric 10 Be in the landscape through a coupled soil-hillslope model (Be2D)|journal=Earth and Planetary Science Letters|volume=439|pages=143–157|doi=10.1016/j.epsl.2016.01.017|bibcode=2016E&PSL.439..143C|issn=0012-821X}}</ref>

== Criticism ==

The Nash–Sutcliffe Coefficient masks important behaviors that if re-cast can aid in the interpreted as the different sources of model behavior in terms of bias, random, and other components.<ref>{{cite journal|last1=Gupta|first1=H.V.|last2=Kling|first2=H|date=2011|title=On typical range, sensitivity, and normalization of Mean Squared Error and Nash‐Sutcliffe Efficiency type metrics|journal=Water Resources Research|volume=47|number=10|pages=W10601|doi=10.1029/2011WR010962|bibcode=2011WRR....4710601G|doi-access=}}</ref>  The alternate Kling-Gupta  efficiency is intended to improve upon NSE by incorporating bias and variance terms.<ref>{{cite journal|last1=Knoben|first1=W.J|last2=Freer|first2=J.E.|last3=Woods|first3=R.A.|date=2019|title=Inherent benchmark or not? Comparing Nash–Sutcliffe and Kling-Gupta efficiency scores|journal=Hydrology and Earth System Sciences|volume=23|number=10|pages=4323–4331|doi=10.5194/hess-23-4323-2019|doi-access=free|hdl=1983/ecc5e89d-80b6-4d5d-a782-286c1b05f55b|hdl-access=free}}</ref>

== See also ==
* [[Coefficient of determination]]
* Kling-Gupta efficiency

== References ==
{{Reflist}}

{{DEFAULTSORT:Nash-Sutcliffe model efficiency coefficient}}
[[Category:Point estimation performance]]

{{Sourceattribution|Nash–Sutcliffe model efficiency coefficient}}