Abstract
The artificial neural network (ANN) test of Lee et al. (Journal of Econometrics 56, 269–290, 1993) uses the ability of the ANN activation functions in the hidden layer to detect neglected functional misspecification. As the estimation of the ANN model is often quite difficult, LWG suggested activate the ANN hidden units based on randomly drawn activation parameters. To be robust to the random activations, a large number of activations is desirable. This leads to a situation for which regularization of the dimensionality is needed by techniques such as principal component analysis (PCA), Lasso, Pretest, partial least squares (PLS), among others. However, some regularization methods can lead to selection bias in testing if the dimensionality reduction is conducted by supervising the relationship between the ANN hidden layer activations of inputs and the output variable. This paper demonstrates that while these supervised regularization methods such as Lasso, Pretest, PLS, may be useful for forecasting, they may not be used for testing because the supervised regularization would create the post-sample inference or post-selection inference (PoSI) problem. Our Monte Carlo simulation shows that the PoSI problem is especially severe with PLS and Pretest while it seems relatively mild or even negligible with Lasso. This paper also demonstrates that the use of unsupervised regularization does not lead to the PoSI problem. Lee et al. (Journal of Econometrics 56, 269–290, 1993) suggested a regularization by principal components, which is a unsupervised regularization. While the supervised regularizations may be useful in forecasting, regularization should not be supervised in inference.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
For radial basis functions, see (e.g.) Campbell, Lo and Mackinlay (1997, p. 517).
- 2.
- 3.
At 5% level, since the p-value is Bernoulli distributed with success probability of 0.05, the standard error of the p-value from the 1,000 Monte Carlo replication is \(\sqrt{\left (0.05 \times 0.95 \right ) /1000} \approx 0.0069\). The 95% confidence interval is 0.05 ± 1.96 × 0.0069 = (0.0365, 0.0635). At 10% level, the standard error of the p-value is \(\sqrt{\left (0.1 \times 0.9 \right ) /1000} = 0.0095\), and the 95 % confidence interval is 0.10 ± 1.96 × 0.0095 = (0.0814, 0.1186).
References
Aït-Sahalia, Y., P. J. Bickel and T. M. Stoker (2001), “Goodness-of-fit Tests for Kernel Regression with an Application to Option Implied Volatilities,” Journal of Econometrics 105(2), 363–412.
Andrews, D. W. K. (1991), “Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation,” Econometrica 59(3), 817–858.
Andrews, D. W. K. and W. Ploberger (1994), “Optimal Tests when a Nuisance Parameter is Present Only under the Alternative,” Econometrica 62, 1383–1414.
Bai, J. and Ng, S. (2008), “Forecasting Economic Time Series Using Targeted Predictors”, Journal of Econometrics 146, 304–317.
Bair, E., Hastie, T., Paul, D. and Tibshirani, R. (2006), “Prediction by Supervised Principal Components,” Journal of the American Statistical Association 101(473), 119–137.
Berk, R., L. Brown, A. Buja, K. Zhang and L. Zhao (2011), “Valid Post-Selection Inference,” The Wharton School, University of Pennsylvania, Working Paper. Submitted to the Annals of Statistics.
Bierens, H. J. (1982), “Consistent Model Specification Tests,” Journal of Ecomometrics 20, 105–134.
Bierens, H. J. (1990), “A Consistent Conditional Moment Test of Functional Form,” Econometrica 58, 1443–1458.
Bierens, H. J. and W. Ploberger (1997), “Asymptotic Theory of Integrated Conditional Moment Tests,” Econometrica 65, 1129–1151.
Blake, A. P. and G. Kapetanios (2000), “A Radial Basis Function Artificial Neural Network Test for ARCH,” Economics Letters 69, 15–23.
Blake, A. P. and G. Kapetanios (2003), “A Radial Based Function Artificial Neural Network Test for Neglected Nonlinearity,” Econometrics Journal 6, 356–372.
Bradley, E., H., Trevor j. Iain and T., Robert (2004), Annals of Statistics 32(2), 407–499.
Bradley, R. and R. McClelland (1996), “A Kernel Test for Neglected Nonlinearity,” Studies in Nonlinear Dynamics and Econometrics 1(2), 119–130.
Cai, Z., J. Fan and Q. Yao (2000), “Functional-Coefficient Regression Models for Nonlinear Time Series,” Journal of the American Statistical Association 95(451), 941–956.
Campbell, J. Y., A. W. Lo and A. C. Mackinlay (1997), The Economometrics of Financial Markets, Princeton University Press.
Corradi, V. and N. R. Swanson (2002), “A Consistent Test for Nonlinear Out of Sample Predictive Accuracy,” Journal of Econometrics 110, 353–381.
Dahl, C. M. (2002), “An Investigation of Tests for Linearity and the Accuracy of Likelihood Based Inference using Random Fields,” Econometrics Journal 1, 1–25.
Dahl, C. M. and G. Gonzalez-Rivera (2003), “Testing for Neglected Nonlinearity in Regression Models: A Collection of New Tests,” Journal of Econometrics 114(1), 141–164.
Davies, R. B. (1977), “Hypothesis Testing when a Nuisance Parameter is Present Only under the Alternative,” Biometrika 64, 247–254.
Davies, R. B. (1987), “Hypothesis Testing when a Nuisance Parameter is Present Only under the Alternative,” Biometrika 74, 33–43.
Eubank, R. L. and C. H. Spiegelman (1990), “Testing the Goodness of Fit of a Linear Model Via Nonparametric Regression Techniques,” Journal of the American Statistical Association 85(410), 387–392.
Fan, Y. and Q, Li (1996), “Consistent Model Specification Tests: Omitted Variables and Semi-parametric Functional Forms,” Econometrica 64, 865–890.
Fan, J. and J, Lv (2008), “Sure Independence Screening for Ultrahigh Dimensional Feature Space,” Journal of the Royal Statistical Society Series B 70, 849–911.
Fan, J. and R. Li (2001), “Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties,” Journal of the American Statistical Association 96, 1348–1360.
Frank, I.E., and J.H. Friedman (1993), “A Statistical View of Some Chemometrics Regression Tools,” Technometrics, 35, 109–148.
Granger, C. W. J. and T. Teräsvirta (1993), Modelling Nonlinear Economic Relationships, Oxford University Press: New York.
Hamilton, J. D. (2001), “A Parametric Approach to Flexible Nonlinear Inference,” Econometrica 69(3): 537–573.
Hansen, B. E. (1996), “Inference When a Nuisance Parameter Is Not Identified Under the Null Hypothesis,” Ecomometrica 64: 413–430.
Härdle, W. and E. Mammen (1993), “Comparing Nonparametric versus Parametric Regression Fits,” Annals of Statistics 21: 1926–1947.
Hillebrand, E., H. Huang, T.-H. Lee, and Canlin Li (2011), “Using the Yield Curve in Forecasting Output Growth and Inflation”, Manuscript. Aarhus University and UC Riverside.
Hjellvik, V. and D. Tjøstheim (1995), “Nonparametric Tests of Linearity for Time Series,” Biometrika 82(2), 351–368.
Hjellvik, V. and D. Tjøstheim (1996), “Nonparametric Statistics for Testing of Linearity and Serial Independence,” Journal of Nonparametric Statistics 6, 223–251.
Hjellvik, V., Q. Yao and D. Tjøstheim (1998), “Linearity Testing Using Local Polynomial Approximation,” Journal of Statistical Planning and Inference 68, 295–321.
Hong, Y. and H. White (1995), “Consistent Specification Testing via Nonparametric Series Regression,” Econometrica 63, 1133–1160.
Hoover, K.D. (2012), “The Role of Hypothesis Testing in the Molding of Econometric Models,” Duke University, Center for the History of Political Economy (CHOPE) Working Paper No. 2012–03.
Hornik, K., M. Stinchcombe, and H. White (1989), “Multi-Layer Feedforward Networks Are Universal Approximators,” Neural Network 2, 359–366.
Huang, J., J. Horowitz and Ma, S. (2008), “Asymptotic Properties of Bridge Estimators in Sparse High-dimensional Regression Models,” Annals of Statistics 36, 587–613.
Huang, H. and T.-H. Lee (2010), “To Combine Forecasts or To Combine Information?” Econometric Reviews 29, 534–570.
Inoue, A. and Kilian, L. (2008), “How Useful is Bagging in Forecasting Economic Time Series? A Case Study of U.S. CPI Inflation,” Journal of the American Statistical Association 103(482), 511–522.
Kock, A. B. (2011), “Forecasting with Universal Approximators and a Learning Algorithm,”Journal of Time Series Econometrics 3, 1–30.
Kock, A. B. and T. Teräsvirta (2011), “Forecasting Macroeconomic Variables using Neural Network Models and Three Automated Model Selection Techniques,” CREATES Research Paper 27.
Lee, T.-H. (2001), “Neural Network Test and Nonparametric Kernel Test for Neglected Nonlinearity in Regression Models,” Studies in Nonlinear Dynamics and Econometrics 4(4), 169–182.
Lee, T.-H. and A. Ullah (2001), “Nonparametric Bootstrap Tests for Neglected Nonlinearity in Time Series Regression Models,” Journal of Nonparametric Statistics 13, 425–451.
Lee, T.-H. and A. Ullah (2003), “Nonparametric Bootstrap Specification Testing in Econometric Model,” Computer-Aided Econometrics, Chapter 15, edited by David Giles, Marcel Dekker, New York, pp. 451–477.
Lee, T.-H., H. White and C. W. J. Granger (1993), “Testing for Neglected Nonlinearity in Time Series Models: A Comparison of Neural Network Methods and Alternative Tests,” Journal of Econometrics 56, 269–290.
Lee, T.-H., Z. Xi and R. Zhang (2012), “Testing for Neglected Nonlinearity Using Artificial Neural Networks with Many Random Hidden Unit Activations,” UCR, manuscript.
Leeb, H. and B.M. Pötscher (2003), “The finite-sample distribution of post model-selection estimators and uniform versus nonuniform approximations,” Econometric Theory 19, 100–142.
Leeb, H. and B.M. Pötscher (2005), “Model selection and inference: Facts and fiction,” Econometric Theory 21, 21–59.
Leeb, H. and B.M. Pötscher (2006), “Performance limits for estimators of the risk or distribution of shrinkage-type estimators, and some general lower risk-bound results,” Econometric Theory 22, 21–59.
Leeb, H. and B.M. Pötscher (2008), “Can One Estimate The Unconditional Distribution Of Post-Model-Selection Estimators?,” Econometric Theory 24(2), 338–376.
Li, Q. and S. Wang (1998), “A Simple Consistent Bootstrap Test for a Parametric Regression Function,” Journal of Econometrics 87, 145–165.
Liu, R. Y. (1988), “Bootstrap Procedures under Some Non-iid Models,” Annals of Statistics 16: 1697–1708.
Matsuda, Y. (1999), “A Test of Linearity Against Functional-Coefficient Autoregressive Models,” Communications in Statistics, Theory and Method 28(11), 2539–2551.
Newey, W. K. (1985), “Maximum Likelihood Specification Testing and Conditional Moment Tests,” Econometrica 53, 1047–1070.
Poggi, J. M. and B. Portier (1997), “A Test of Linearity for Functional Autoregressive Models,” Journal of Time Series Analysis 18(6), 615–639.
Pötscher, B.M. and H. Leeb, (2009), “On the Distribution of Penalized Maximum Likelihood Estimators: The LASSO, SCAD, and Thresholding,” Journal of Multivariate Analysis 100(9), 2065–2082.
Stengos, T. and Y. Sun (2001), “Consistent Model Specification Test for a Regression Function Based on Nonparametric Wavelet Estimation,” Econometric Reviews 20(1), 41–60.
Stinchcombe, M. and H. White (1998), “Consistent Specification Testing with Nuisance Parameters Present only under the Alternative,” Econometric Theory 14, 295–325.
Tauchen, G. (1985), “Diagnostic Testing and Evaluation of Maximum Likelihood Models,” Journal of Econometrics 30, 415–443.
Tibshirani, R. (1996), “Regression Shrinkage and Selection via the Lasso,” Journal of Royal Statistical Society, B, 58, 267–288.
Teräsvirta, T. (1996), “Power Properties of Linearity Tests for Time Series,” Studies in Nonlinear Dynamics and Econometrics 1(1), 3–10.
Teräsvirta, Timo, C.-F. Lin and C. W. J. Granger (1993), “Power of the Neural Network Linearity Test,” Journal of Time Series Analysis 14(2), 209–220.
Tjøstheim, D. (1999), “Nonparametric Specification Procedures for Time Series,” in S. Ghosh (ed.), Asymptotics, Nonparametrics, and Time Series: A Tribute to M.L. Puri, Marcel Dekker.
Ullah, A. (1985), “Specification Analysis of Econometric Models,” Journal of Quantitative Economics 1: 187–209.
Whang, Y.-J. (2000), “Consistent Bootstrap Tests of Parametric Regression Functions,” Journal of Econometrics 98, 27–46.
White, H. (1980), “A Heteroskedasticity Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity,” Econometrica 48, 817–838.
White, H. (1987), “Specification Testing in Dynamic Models,” T.F. Bewley (ed.), Advances in Econometrics, Fifth World Congress, Vol 1, New York: Cambridge University Press, 1–58.
White, H. (1989), “An Additional Hidden Unit Tests for Neglected Nonlinearity in Multilayer Feedforward Networks,” Proceedings of the International Joint Conference on Neural Networks, Washington, DC. (IEEE Press, New York, NY), II: 451–455.
White, H. (1994), Estimation, Inference, and Specification Analysis, Cambridge University Press.
White, H. (2006), “Approximate Nonlinear Forecasting Methods,” Handbook of Economic Forecasting 1, 459–512.
Wu, C. F. J. (1986), “Jackknife, Bootstrap, and Other Resampling Methods in Regression Analysis,” Annals of Statistics 14: 1261–1350.
Zheng, J. X. (1996), “A Consistent Test of Functional Form via Nonparametric Estimation Techniques,” Journal of Econometrics 75, 263–289.
Zou, H. (2006), “The Adaptive Lasso and Its Oracle Properties,” Journal of the American Statistical Association 101(476), 1418–1429.
Zou, H. and H. Zhang (2009), “On The Adaptive Elastic-Net with a Diverging Number of Parameters, ” The Annals of Statistics 37(4), 1773–1751.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer Science+Business Media New York
About this chapter
Cite this chapter
Lee, TH., Xi, Z., Zhang, R. (2014). Testing for Neglected Nonlinearity Using Regularized Artificial Neural Networks. In: Ma, J., Wohar, M. (eds) Recent Advances in Estimating Nonlinear Models. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-8060-0_3
Download citation
DOI: https://doi.org/10.1007/978-1-4614-8060-0_3
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4614-8059-4
Online ISBN: 978-1-4614-8060-0
eBook Packages: Business and EconomicsEconomics and Finance (R0)