Weather Forecasting Model Evaluation Processes

After we develop a weather forecasting model, we need to evaluate the performance of the model to know if it’s fulfill our needs. In this article, I discuss the processes to evaluate a weather forecasting model.

Step 1. Know our requirement

As Gordon reported in 2000, we need to know what’s we are trying to simulate through the forecasting model. We can analyze the weather system to figure the characteristics in different spatial and timely domains. We can use clustering or eigenvalue technique to stand out the pattern of the weather system. When we have more clear pattern of the weather system, we may exam the model more precisely.

Step 2. Run the model to show the evolution of the weather system

Dissect the weather system and plot the charts in different spatial and timely resolution with different weather parameters. Wind, temperature, and humidity fields are the very basic characters to a weather system. In the same time, we can compare the model output with observation or reanalysis data to see the difference. The different is what we are going to reduce to achieve a perfect model.

Step 3. Track the weather system

Focus on the key area(s), such as the eye of typhoon, to grab the data there. We will use this information to analyze time series and path trajectory to compare the model performance.

Step 4. Plot time series of the key area

This is the time series analysis to compare the different weather factors from different model and observation for the weather system.

Step 5. Calculate the statistical metrics

To quantify the evaluation, there are different statistical metrics, such as root mean square error (RMSE), correlation coefficient etc. to represent the performance of a model.

Step 6. Sensitivity analysis

To optimize the model, using different configuration, initial data, boundary condition can help us to review the performance in different exams.

In summary, with the modeling requirements in mind, we can display the differences of weather characters between different model exams and observation. Use the difference to calculate the statistical metrics will let us assess the performance of the weather forecasting model.

Reference:

Gordon, N. D., & Shaykewich, J. (2000). Guidelines on performance assessment of public weather services. World Meteorological Organization.