REALISTIC SPECIFICATIONS AND MODEL PREDICTABILITY: TESTING THE PERFORMANCE OF A STOCHASTIC CGE MODEL WITH REGIONALLY CORRELATED YIELD VARIABILITY IN THE WHEAT AND RICE SECTORS

Tetsuji Tanaka¹⁺ --- Jin Guo²

^1,2Department of Economics, Setsunan University, Ikedanaka, Neyagawa, Japan.

Keywords:Model validation, Simulation, Agricultural productivity, Producer price.

JEL Classification:Q11; C52; C68.

ARTICLE HISTORY: Received: 12 November 2019, Revised: 16 December 2019, Accepted: 21 January 2020, Published: 2 March 2020

Contribution/ Originality: The study contributes in the existing literature by testing the forecastability improvement of considering regional correlation of agricultural productivity shocks in a multi-regional CGE model.

1. INTRODUCTION

With several advantageous features, computable general equilibrium (CGE) modeling has gained immense popularity over the past decades since the first model was developed by Johansen (1960) having been applied to a broad spectrum of research fields such as international trade, the environment, development, agriculture, energy, transport, and tourism (e.g., (Burnett, Cutler, & Thresher, 2007; Khan, Zada, & Mukhopadhyay, 2018; Li & Su, 2017; Liu, Hertel, Taheripour, Zhu, & Ringler, 2014; Meng, Siriwardana, McNeill, & Nelson, 2018; Raihan, Osmani, & Khalily, 2017)). Other archetypal methods such as econometric models or dynamic stochastic general equilibrium (DSGE) models require a large amount of data, whereas just a single-year social accounting matrix (SAM) is needed to construct a CGE model, which enables analysis of a wide range of developing areas where economists frequently encounter data deficiency. Nevertheless, this broad applicability is a double-edged sword, as a simple parameter estimation – namely, point estimation – raises questions and criticisms regarding the reliance of CGE simulation results.

Considering the enormous volume of empirical studies with CGE models, few ex-post performance checks have been conducted. Kehoe, Polo, and Sancho (1995) addressed this by quoting Whalley (1985): “Since the essence of theory is simplification which in an exact sense must be wrong, the constraints of tractability perhaps dictate that economic theory can ultimately be only an organizational framework for thinking about economic problems.” This may suggest that the ceteris paribus assumption incapacitates the comparison between historical data and simulated results. Still, some validation exercises seem to have been successfully made for CGE models – e.g., by Kehoe et al. (1995); Kehoe. (2003); Valenzuela, Hertel, Keeney, and Reimer (2007); Beckman, Hertel, and Tyner (2011) and Guivarch, Hallegatte, and Crassous (2009).    

Deterministic models have been extended to stochastic models in some ways. Harris and Robinson (2001) and Tanaka and Hosoe (2011) integrated the Monte Carlo method with a deterministic and static model to convert it into a stochastic framework with repetitive simulations. The Gaussian quadrature (GQ) technique developed by Stroud (1957) and Haber (1970) was applied to a CGE model by Arndt (1996) as an alternative and efficient approach to the Monte Carlo method in order to curtail computational costs. This technique was introduced to simplify robustness tests against parametric uncertainty, but Valenzuela et al. (2007) and Beckman et al. (2011) applied this approach to empirical analyses. Even though earlier studies assumed the independence of distributions for variable parameters, Erhan, Karapinar, and Tanaka (2017) emphasized the importance of incorporating regional correlations of agricultural productivity shocks, which suggests that extreme outcomes such as good or poor harvests could be marginalized if crop yield correlations between regions are not considered.

This article identifies the replication capability of endogenous producer prices of wheat and rice in multi-regional standard CGE models through an empirical example where annual wheat yield variability occurs in individual countries as exogenous shocks. More specifically, the simulated short-run volatility of producer prices for large exporting and importing nations refers to historical data, employing models integrated with regional correlations of wheat or rice yield shocks. Although agricultural models often incorporate land-use modules as part of a supply-side mechanism, this study does not consider any modification in either the demand- or supply-side structure, instead concentrating on testing the performances of a standard model used by many economists, as different specifications create different outcomes, which dilute the meanings of our validation exercises. Advancing the understanding of prediction performance generated by a common framework would provide useful information on reshaping the basic architecture.

2. LITERATURE REVIEW

This paper is inspired by Valenzuela et al. (2007) who undertook validation exercises for the GTAP model, focusing on producer price volatilities of wheat caused by productivity changes based on actual data. The autoregressive–moving-average (ARMA) model was used to gauge the yield variations of each country, with experiments conducted by the GQ numerical integration technique. The model was found to perform well, and the explanatory power was improved by incorporating price transmission elasticities into the model. However, the stochastic exercises did not consider regional correlations in the productivity shocks, assuming the independent and identically distributed (i.i.d.) normal distributions, which implies that extreme market jolts synthesized by positive or negative simultaneous shocks in multi-regions were ignored.

Kehoe et al. (1995) compared the model outputs and observed data of Spain’s economy under the fiscal reform policy with a static general equilibrium model. The model fit was broadly satisfactory in both the macro- and micro-level indicators. While the production activity levels were not well predicted with anaround 0.1, the sectoral prices of producer and consumer were accounted for by about 65–79%. The values for the labor market, the basic macroeconomic variables, and the fiscal variables were 0.67, 0.89 and 0.65, respectively.¹

Kehoe. (2003) analyzed the effects of the North American Free Trade Agreement (NAFTA) to validate a multi-regional CGE model. In contrast to Kehoe et al. (1995) the model for the liberalization agreement did not fit well with the historical data. The author implied that the reason why Spain’s model forecasted better than NAFTA’s could be because, as economists, they comprehend public finance more than global trade issues, and the time period for Spain’s model covered only one or two years, while the trade analysis was predicted over a decade.  

Beckman et al. (2011) carried out a GTAP-E model validation analysis on petroleum price distributions with the inputs of both demand- and supply-side factors by comparing to historical outcomes. In the stochastic simulations, it was found that the GTAP-E with the original parameters did not explain the price volatility of oil well, while the model with the revised parameters from various earlier works reproduced much lower price volatility than the observed data. The deterministic validation exercise to examine the degree of changes in oil consumption implied that the model with revised elasticities more closely predicted historical oil consumption than the one with original parameters.

Guivarch et al. (2009) tested the replicability of a global energy-environment CGE model with the focus on oil price shocks to macro-aspects of India’s economy between 2003 and 2006. With the original settings, the model performance was poor in predicting India’s GDP growth rates, but was significantly improved with the consideration of mechanisms identified in the IMF country report: (1) an increase in the export of refined oil goods, (2) a capital influx from foreign regions, and (3) a sluggish price transmission between local international and domestic markets.   

As shown above, the performance verification studies for CGE are extremely limited considering its widespread usage. Valenzuela et al. (2007) provide the only work in the field of agriculture. Erhan et al. (2017) developed a stochastic model into which productivity correlations between regions are introduced, but its performance has never been tested to ascertain its replicability. This is the knowledge gap to be filled by this article. 

3. METHOD

In this study, a global-scale CGE model was constructed for wheat and rice individually, with the GTAP database version 7, the base year of which is 2004, to compare simulated price volatilities with actual price volatilities between 1992 and 2006. We selected this limited period of time to exclude the 2007–08 food crisis and miscellaneous driving forces such as commodity speculation, biofuel production and trade restriction policy measures, which hamper our model performance verification, as this analysis concentrates on the associated relationships between yield variability and producer price volatility. Also, yield data for the former Soviet Union countries are not available before 1992. One of the standard general equilibrium models built by Devarajan, Lewis, and Robinson (1990) is extended to a multi-regional framework, introducing a stochastic element with the Monte Carlo method.

3.1. Model

We construct a model for each cereal – wheat and rice – with identical model structure, sectoral and factor aggregations, and elasticity parameters. However, they are heterogeneous with respect to regional aggregation and, accordingly, productivity shocks.

An individual region has nine sectors and five production factors Table 1. ² Each representative firm behaves as a perfectly competitive profit-maximizing firm following the Leontief production form Figure 1. Value-added factors of production are aggregated to create a value-added composite good with a constant elasticity of substitution (CES) production function. Only unskilled labor is assumed to be mobile across sectors, while the other factors are fixed to estimate short-run effects. We assume factors of production are fully employed.

Table-1. Sectoral and factor aggregations.

Figure-1. Overview of the model structure.

Sectoral outputs produced by representative firms are distributed between foreign and domestic markets, using a constant elasticity of transformation (CET) technology. The domestic goods and composite imports are aggregated to make composite goods with a CES function (Armington, 1969). Import composite goods comprise imported goods from various regions, and composite exports are disaggregated into exports of individual regions. In the structure of the standard model, a single nest system is employed, with the Cobb–Douglas form for household consumption Figure 2.

Figure-2. Household consumption structure.

3.2. Estimation of Yield Volatility

To generate random yield shocks, we estimate the volatilities of wheat and rice productivity from historical records for each region, using the FAOSTAT spanning from 1992 to 2006. The yield is defined as harvest quantity per acre (production divided by area harvested). With technological advancements, wheat productivity grew steadily across many countries, exhibiting time trends, which can cause overestimation of the volatility. ARMA models are fitted to filter autocorrelation and non-stationarity. ³ The model used for this is described as follows:

 The Akaike information criterion (AIC) is used for model selection; the results and the standard deviations of yield volatility obtained from the residuals are summarized in Tables 2 and 3. We assume that yield shocks randomly occur to the total factor productivity (TFP) of the gross output production function for the wheat industry.

Table-3. ARMA results for rice yield.

We create two sets of 1000 randomized yield draws for each grain with/without regional correlations, using the covariance matrix estimated from the prediction errors of the ARMA models. The non-correlated series follows an i.i.d.and the other follows i.i.d. with regional correlations. The Cholesky decomposition of this matrix is used to convert the non-correlated Monte Carlo draws into correlated shocks. Our 1000 noncorrelated Monte Carlo runs for 16 wheat regions and 15 rice regions are described as the matrix Z, the dimensions of which are 1000 by 16 or 15.  Then, we take the Cholesky decomposition (C) of the covariance matrix Σ, such that Our covariance matrix Σ and its Cholesky decomposition C are now 16×16 or 15×15 matrices. Finally, we have the correlated Monte Carlo draws (X) by the following transformation:

where each column of Z is 1000 Monte Carlo runs from N (0,1). The correlation pairs are displayed in Tables 4 and 5.

3.3. Evaluation Method

Two methods, correlation and are used to gauge the comprehensive measures of model fittingness following Kehoe et al. (1995). ⁴ We examine the validity of three price volatilities: producer, export and import prices. The specification takes the form:

where signify the weights measuring the relative size of production, export, or import quantity, observed data, and simulated price volatilities assessed in coefficient variation (CV) ⁵ for country i, respectively. This measure reveals the correctness of the sign (positive or negative) and the linearity between historical and simulated results, which do not necessarily inform the predictability. In this study, we focus on major wheat-exporting and -importing countries, and the rest of the world (ROW) is ignored to avoid the obscurity in calculating the two indicators, since the ROW has a wide range of price fluctuations and various global regions. The ability to forecast is gauged by the weighted which is specified as follows:

Both benchmarks assume a linear relationship between actual and predicted variables.

3.4. Scenarios

We establish four scenarios to investigate the degree of improvement or deterioration of CGE performance on wheat and rice producer price forecastability with/without taking regional correlations into account. NCR-W and CR-W represent non-correlated (independent) and correlated productivity shocks, respectively, for wheat. NCR-R and CR-R signify non-correlated (independent) and correlated yield shocks, respectively, for rice. 

4. RESULTS

This section compares historical data and simulated outputs of price volatility of wheat and rice in order to identify the performance of CGE models constructed based on the GTAP database version 7 with a base year of 2004. Identical experiments but with the GTAP database version 9 (the benchmark year is 2011) are executed as robustness tests for the period 1992–2013. In addition, we exhibit results of analyses of sensitivity to variations in the Armington elasticities and the elasticities of substitution between factors that often influence modeling results to a great extent.  In the NCR-W Scenario, the model explains 65% of the real producer price fluctuations (see the weighted R2 in Table 6, which is enhanced to 72% when considering regionally correlated yield variabilities. The model prediction estimates for the large wheat producers of the former Soviet Union, such as Kazakhstan, Russia and Ukraine, are overestimated, while those for major producers in Europe, such as France and Germany, are undervalued. The countries with relatively small prediction errors are Australia, Brazil, China, Japan, South Korea, and Nigeria, with all but Australia being net importing nations. This is primarily because domestic price is less attributed to local production in a net importing country in comparison with a net exporting country – that is, the price movements of a large importer rely more on the variations of imported wheat compared with a net exporting state.

Table-4. The correlations of wheat yield across regions.

Table-5. The correlations of rice yield across regions.

Table-6. Performance of the models on real producer price of wheat.

Note: EX and IM indicate exporter and importer, respectively

Another point to be highlighted is that the simulated prices for all the regions in scenario CR-W are more volatile than those in scenario NCR-W, which is due to the synchrony of regional yield shocks generated by correlated random draws in the model. In other words, crop failures or good harvests in different countries more frequently coincide to convulse local markets, mitigating underestimation and intensifying overestimation. The price fluctuations in Kazakhstan between scenarios NCR-W and CR-W are smaller than, for instance, those in Canada or Ukraine, which implies that the wheat market in Kazakhstan is well segregated from foreign markets, and in contrast, local markets in Canada and Ukraine are more closely integrated into the global market.  (check FAO data like trade exposure).

Thirty-seven percent of the volatility of the historical producer price of rice is accounted for by simulated counterparts Table 7. For both scenarios NCR-R and CR-R, the weighted R2 value is 0.37, and the CVs for both regions are similar, which suggests that regionally correlated yield shocks do not significantly affect foreign markets. This is because international rice tradability is relatively limited compared to other crops such as wheat or maize (Tanaka & Hosoe, 2011) – that is, most rice is consumed in the country in which it is produced, and therefore only a small portion is traded on the global market. The modeling results for rice are consistently underestimated, while the signs of prediction errors for wheat are mixed. Even though there might be various reasons for this difference, one of the primary causes behind it is that elasticities offered by the GTAP database, such as the Armington elasticity and the elasticity of substitution between factors, may not be for assessing short-term effects whose values are normally smaller (less elastic) than elasticities for the long-term. While the explanatory powers for rice are estimated as lower than those of the wheat price, the weighted correlations for rice are evidently higher than those for wheat, which implies that the associated relationship between yield variability and producer price for rice presents more trustworthiness compared with the linkages for wheat.                                   

Table-7. Model prediction performance on real producer price of rice.

Note: EX and IM indicate exporter and importer, respectively.

4.1. Robustness Tests with the Extended Period 1992–2013

In the above experiments, the time period from 1992 to 2006 was used to assess the accuracy of model predictions on price volatilities to circumvent noises from other influential factors, such as oil prices, speculation, and biofuel production, considered as driving forces of food prices around 2008 (Tanaka., Hosoe, & Qiu, 2012). To check the robustness of the results, models structured identically to the one used in the main simulations but based on the benchmark year of 2011 with GTAP version 9 are constructed. The data period in this sensitivity analysis consequently spans from 1992 to 2013. The CV for yield shocks and the volatilities of historical producer prices are re-estimated for the extended time period.⁶  

As anticipated, the for the producer price of wheat in the revised models indicates slightly lower values than those in the 2004-based models Table 8. One of the primary reasons for the differences is that the food crisis period, which encompasses a wide variety of factors that affect domestic price, is incorporated into the original time period. Therefore, other factors (non-productivity factors) shock the producer price of wheat, which aggravates the explanatory powers. Nevertheless, a main conclusion obtained in the original analysis stating that productivity shocks with regional correlations improve forecastability holds true. The weighted correlations for the two scenarios are reduced with the revised model but are still slightly improved if regional correlations are considered. For the producer price of rice, lower R2 values are exhibited with the extended period to 2013 Table 9, which is coherent with the wheat analysis shown above. All the simulated volatilities are underestimated, and the weighted correlations are high. These results are compatible with the original simulation exercises.

Table-8. Robustness tests for wheat price against the extended time period.

Table-9. Robustness tests for rice price against the extended time period.

4.2. Robustness Tests Regarding the Armington Elasticities

Table 10 presents sensitivity results regarding the Armington elasticities. Even with a ±30% alteration in the elasticity values, the R2 values and correlations are improved when considering regional correlations. The volatilities of the producer price for the former Soviet Union countries, such as Russia, Ukraine and Kazakhstan, tend to be overestimated, while those for France and Germany are underestimated. With this evidence, we find the original results robust against variations in the Armington elasticities. Concerning the Armington elasticities for rice, it is found that the R2 and correlations remain unchanged, although insignificant divergence from the results of the original analysis is confirmed in the simulated volatilities of individual countries Table 11.

Table-10. Robustness results for wheat price against the Armington elasticity.

Table-11. Robustness results for rice price against the Armington elasticity.

5. CONCLUSION

This article attempted to identify the predictive performance of CGE models with correlated or non-correlated Monte Carlo draws for wheat and rice markets. The primary findings are summarized as follows: (1) applying regional correlations to Monte Carlo random productivity shocks enhances the explanatory power and the correlations for wheat, (2) these indices for rice remain unchanged due to the thinness of the global rice market, and (3) rice producer price volatilities tend to be underestimated by yield changes.

The causes of the last finding concerning the undervaluation of the rice price remain a mystery. A reasonable hypothesis is that the elasticities from the GTAP database may be too large for evaluating short-term impacts, and therefore the simulated price volatilities were coherently smaller than the actual counterparts. In our standard CGE model, both the elasticities of substitution for value added and the Armington elasticities are quoted from the GTAP database, which makes it extremely difficult to identify the problem, which is beyond the scope of this paper.      

This study concentrated only on wheat and rice sectors, therefore omitting influences from other markets. Yet, in reality, there is interaction from a myriad of markets and factors. Incorporating other potential factors could raise the R2 values of the models. In the experiments for 1992–2013, the R2 values declined compared with those for 1992–2006, which was likely caused by factors such as oil price, biofuel from food materials, the variation of wheat reserves, and financial speculation. Those factors together with wheat or rice yield could also mitigate the estimation errors.     

It is indisputable that validation test analysis for CGE models has not been sufficiently extensive to support their heavy usage. The models have been applied not solely to the agricultural field but also to a broad scope of societal issues, such as transport, trade policy, climate change, and so forth. However, published articles have attempted to justify the predictive ability of CGE models solely in agriculture, energy, fiscal reform, and trade liberalization. More efforts need to be made to enhance the trustworthiness of models, analyzing various issues and regions from different approaches.

REFERENCES

Armington, P. S. (1969). A theory of demand for products distinguished by place of production. Staff Papers, 16(1), 159-178. Available at: https://doi.org/10.2307/3866403.

Arndt, C. (1996). An introduction to systematic sensitivity analysis via Gaussian quadrature. GTAP Technical Paper, No. 2. Purdue University, West Lafayette.

Beckman, J., Hertel, T., & Tyner, W. (2011). Validating energy-oriented CGE models. Energy Economics, 33(5), 799-806. Available at: https://doi.org/10.1016/j.eneco.2011.01.005.

Burnett, P., Cutler, H., & Thresher, R. (2007). The impact of tourism for a small city: A CGE approach. The Journal of Regional Analysis & Policy, 37(3), 233–242.

Devarajan, S., Lewis, J. D., & Robinson, S. (1990). Policy lessons from trade-focused, two sector models. Journal of Policy Modelling, 12(4), 625–657. Available at: https://doi.org/10.1016/0161-8938(90)90002-v.

Erhan, U. G., Karapinar, B., & Tanaka, T. (2017). Welfare-at-risk and extreme dependency of regional wheat yields: Implications of a stochastic CGE model. Journal of Agricultural Economics, 69(1), 18-34.

Guivarch, C., Hallegatte, S., & Crassous, R. (2009). The resilience of the Indian economy to rising oil prices as a validation test for a global energy–environment–economy CGE model. Energy Policy, 37(11), 4259-4266. Available at: https://doi.org/10.1016/j.enpol.2009.05.025.

Haber, S. (1970). Numerical evaluation of multiple integrals. SIAM Review, 12(4), 481-526. Available at: https://doi.org/10.1137/1012102.

Harris, R. L., & Robinson, S. (2001). Economy-wide effects of El Niño/Southern Oscillation (ENSO) in Mexico and the role of improved forecasting and technological change. TMD Discussion Paper No.83.

Johansen, L. (1960). A multisectoral study of economic growth. Amsterdam: North-Holland Publishing Company.

Kehoe, T. J., Polo, C., & Sancho, F. (1995). An evaluation of the performance of an applied general equilibrium model of the Spanish economy. Economic Theory, 6(1), 115-141. Available at: https://doi.org/10.1007/bf01213943.

Kehoe., T. J. (2003). An evaluation of the performance of applied general equilibrium models of the impact of NAFTA. Federal Reserve Bank of Minneapolis Research Department Staff Report 320.

Khan, M. A., Zada, N., & Mukhopadhyay, K. (2018). Economic implications of the comprehensive and progressive agreement for trans-pacific partnership (CPTPP) on Pakistan: A CGE approach. Journal of Economic Structures, 7(2), 1-20. Available at: https://doi.org/10.1186/s40008-017-0103-x.

Li, Y., & Su, B. (2017). The impacts of carbon pricing on coastal megacities: A CGE analysis of Singapore. Journal of Cleaner Production, 165, 1239-1248. Available at: https://doi.org/10.1016/j.jclepro.2017.07.206.

Liu, J., Hertel, T. W., Taheripour, F., Zhu, T., & Ringler, C. (2014). International trade buffers the impact of future irrigation shortfalls. Global Environmental Change, 29, 22-31. Available at: https://doi.org/10.1016/j.gloenvcha.2014.07.010.

Meng, S., Siriwardana, M., McNeill, J., & Nelson, T. (2018). The impact of an ETS on the Australian energy sector: An integrated CGE and electricity modelling approach. Energy Economics, 69, 213-224. Available at: https://doi.org/10.1016/j.eneco.2017.11.017.

Raihan, S., Osmani, S., & Khalily, M. B. (2017). The macro impact of microfinance in Bangladesh: A CGE analysis. Economic Modelling, 62, 1-15. Available at: https://doi.org/10.1016/j.econmod.2017.01.002.

Stroud, A. H. (1957). Remarks on the disposition of points in numerical integration formulas. Mathematical Tables and Other Aids to Computation, 11(60), 257-261. Available at: https://doi.org/10.1090/s0025-5718-1957-0093911-3.

Tanaka, T., & Hosoe, N. (2011). Does agricultural trade liberalization increase risks of supply-side uncertainty? Effects of productivity shocks and export restrictions on welfare and food supply in Japan. Food Policy, 36(3), 368-377. Available at: https://doi.org/10.1016/j.foodpol.2011.01.002.

Tanaka., T., Hosoe, N., & Qiu, H. (2012). Risk assessment of food supply: A computable general equilibrium approach: Cambridge Scholars Publishing.

Valenzuela, E., Hertel, T. W., Keeney, R., & Reimer, J. J. (2007). Assessing global computable general equilibrium model validity using agricultural price volatility. American Journal of Agricultural Economics, 89(2), 383-397. Available at: https://doi.org/10.1111/j.1467-8276.2007.00977.x.

Whalley, J. (1985). Hidden challenges in recent applied general equilibrium exercises. In New developments in applied general equilibrium analysis (J. Piggott and J. Whalley. Eds.) (pp. 25–41). Cambridge: Cambridge University Press.

Appendix:Algebraic description of the model.  

A full specification of the standard CGE used in the analysis is as follows.

Views and opinions expressed in this article are the views and opinions of the author(s), Asian Journal of Economic Modelling shall not be responsible or answerable for any loss, damage or liability etc. caused in relation to/arising out of the use of the content.

---

Footnotes:

1. This paper covered two time periods (1985–1986 and 1985–1987), and the better-predicted value is shown here.

2. See the Appendix for the full model description.

3. We do not employ a generalized autoregressive conditional heteroskedasticity (GARCH) or EGARCH model in this analysis, as yield volatility is supposed to be independently determined in each year.

4. In Kehoe et al. (1995) sectoral size is used for the weights of the metrics.

5. The coefficient of variation is defined as the standard deviation divided by the mean.

6. Figures for estimation errors are available upon request.