Details on the October ENSO Diagnostic Discussion: Trust, but verify.

October 9, 2014

Do we sound like a broken record? The CPC/IRI El Niño-Southern Oscillation forecast released today is essentially unchanged from last month, with around 60-65% chance of El Niño, starting in October-November. Sea surface temperatures in the Niño3.4 region are +0.3°C over the last week, a downwelling Kelvin wave continues to transport warm water toward the eastern equatorial Pacific, and global climate models continue to call for the development of a weak El Niño.

Just how good are these models, though? In our last post, Tony discussed ENSO forecasts over the last few years, including prediction for El Niño in the fall of 2012 that never materialized. Here, I’ll take a look at one forecast system to see how it performed compared to the past 33 years of observed ENSO episodes.

While dynamical climate modeling has been around for decades, many of the current models have only been in existence for a few years (for details on dynamical prediction models, see the footnotes from Tony’s post). For example, NOAA’s operational climate model, the NCEP Climate Forecast System version 2 (CFSv2), started producing forecasts in 2011.  It is difficult to evaluate model performance with only a few years of forecasts. So, in order to get more years to study, we create forecasts based on historical records.

For example, we start with the observed state of the atmosphere and ocean in September 1997, and then let the computer model make a forecast out to 9 months (for the rest of 1997 and into 1998).  Since we already know what happened in 1997-1998, we can compare the observations against this “retrospective forecast” (also known as a “hindcast”) to determine how well the model performed.

One of the tools we use in making the ENSO forecast is the North American Multi-Model Ensemble (NMME) forecasting system, a project that incorporates several global climate models, including the CFSv2. I looked at the September forecasts for October-November-December (OND) from 1982 – 2013, to see how the forecast tracked with the observations (Fig. 1). This system has been active since 2011, so I used 29 years of hindcasts (1982 – 2010) and three years of archived real-time forecasts (2011 – 2013). The exact same versions of the models are used for both the hindcasts and the forecasts, allowing for a continuous data set.

Model forecast and observed ONI value

Forecast from the NMME (lighter colored bars) and observed Oceanic Nino Index values (darker bars). Red/blue bars indicate ENSO events; gray bars correspond to neutral conditions. The four "missed" forecasts, and the one false alarm (2012) are highlighted with asterisks. Figure by from CPC analysis.

The forecast I looked at was the area-averaged sea surface temperature in the Niño3.4 region, averaged over a three-month period (this is also called the Oceanic Niño Index (ONI), NOAA’s indicator of ENSO). In the 1982 – 2013 period, we had 10 El Niños winters and 12 La Niña winters; the rest were neutral years. The NMME forecast I used is the combination of all the models (“ensemble mean,” the red line in Fig. 2).


NMME ensemble forecast for Nino3.4 SST

Forecast from the North American Multi-Model Ensemble for the Nino3.4 index. The red line is the average of all the forecasts. Figure from NOAA, using CPC data.

To get a “hit”, the system had to correctly forecast an ENSO event, based on five consecutive seasons with an ONI greater than +0.5°C (El Niño) or less than-0.5°C (La Niña.) The other possible outcomes are “miss”, when an ENSO event happened, but wasn’t forecast, and “false alarm”, when the forecast system called for an ENSO event, but it didn’t occur.

In the past 33 years, grouping all ENSO events together, the September forecast for October-November-December had 18 hits, 4 misses, and one false alarm (2012). The rest of the forecasts were “correct negatives” – forecasts for neutral conditions. The forecasts correlate with the observed ONI at a 0.95 coefficient. While this is a simplistic way of taking stock of the forecasts, it does give an idea of why we tend to trust the model forecasts made this time of the year, especially at such a short lead time.  At longer lead times, for example, the June forecasts for October-November-December, there are 13 hits, 8 misses, and 3 false alarms.

A few other elements are giving forecasters the confidence to stick with the 60-65% chance of El Niño. One, the CFSv2 appears to have forecast many of the changes in subsurface ocean temperatures since May (i.e. oceanic Kelvin waves). Also, there are currently some westerly wind anomalies in the western Pacific, which may encourage more eastward heat transport across the tropical Pacific.

Finally, atmospheric conditions, as represented by the Southern Oscillation Index, remained in a borderline El-Niño-friendly state throughout September (SOI = -0.7 at the end of September.) So, while this year’s ENSO forecasts may sound like a broken record as we wait to see if the forecast hits, the track record of model forecasts for past El Niños and the current state of the atmosphere and ocean tell us that the odds are still for an appearance of El Niño this year.

**Editor's note: A draft version of this post was accidentally posted at 11 a.m. EST. The post was updated with the final version at 12:23 p.m.