Sparse estimation of generalized linear models (GLM) via approximated information criteria

Xiaogang Su, Juanjuan Fan, Richard A. Levine, Martha E. Nunn, Chih Ling Tsai

Research output: Contribution to journalArticle

Abstract

We propose a sparse estimation method, termed MIC (Minimum approximated Information Criterion), for generalized linear models (GLM) in fixed dimensions. What is essentially involved in MIC is the approximation of the 0-norm by a continuous unit dent function. A reparameterization step is devised to enforce sparsity in parameter estimates while maintaining the smoothness of the objective function. MIC yields superior performance in sparse estimation by optimizing the approximated information criterion without reducing the search space and is computationally advantageous since no selection of tuning parameters is required. Moreover, the reparameterization tactic leads to valid significance testing results free of post-selection inference. We explore the asymptotic properties of MIC, and illustrate its usage with simulated experiments and empirical examples.

Original languageEnglish (US)
Pages (from-to)1561-1581
Number of pages21
JournalStatistica Sinica
Volume28
Issue number3
DOIs
StatePublished - Jul 1 2018

Fingerprint

Information Criterion
Generalized Linear Model
Reparameterization
Parameter Tuning
Sparsity
Search Space
Asymptotic Properties
Smoothness
Objective function
Information criterion
Generalized linear model
Valid
Norm
Testing
Unit
Approximation
Estimate
Experiment

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Cite this

Sparse estimation of generalized linear models (GLM) via approximated information criteria. / Su, Xiaogang; Fan, Juanjuan; Levine, Richard A.; Nunn, Martha E.; Tsai, Chih Ling.

In: Statistica Sinica, Vol. 28, No. 3, 01.07.2018, p. 1561-1581.

Research output: Contribution to journalArticle

Su, Xiaogang ; Fan, Juanjuan ; Levine, Richard A. ; Nunn, Martha E. ; Tsai, Chih Ling. / Sparse estimation of generalized linear models (GLM) via approximated information criteria. In: Statistica Sinica. 2018 ; Vol. 28, No. 3. pp. 1561-1581.
@article{77ccfa331b3e47bf9e886085f108c7d6,
title = "Sparse estimation of generalized linear models (GLM) via approximated information criteria",
abstract = "We propose a sparse estimation method, termed MIC (Minimum approximated Information Criterion), for generalized linear models (GLM) in fixed dimensions. What is essentially involved in MIC is the approximation of the 0-norm by a continuous unit dent function. A reparameterization step is devised to enforce sparsity in parameter estimates while maintaining the smoothness of the objective function. MIC yields superior performance in sparse estimation by optimizing the approximated information criterion without reducing the search space and is computationally advantageous since no selection of tuning parameters is required. Moreover, the reparameterization tactic leads to valid significance testing results free of post-selection inference. We explore the asymptotic properties of MIC, and illustrate its usage with simulated experiments and empirical examples.",
author = "Xiaogang Su and Juanjuan Fan and Levine, {Richard A.} and Nunn, {Martha E.} and Tsai, {Chih Ling}",
year = "2018",
month = "7",
day = "1",
doi = "10.5705/ss.202016.0353",
language = "English (US)",
volume = "28",
pages = "1561--1581",
journal = "Statistica Sinica",
issn = "1017-0405",
publisher = "Institute of Statistical Science",
number = "3",

}

TY - JOUR

T1 - Sparse estimation of generalized linear models (GLM) via approximated information criteria

AU - Su, Xiaogang

AU - Fan, Juanjuan

AU - Levine, Richard A.

AU - Nunn, Martha E.

AU - Tsai, Chih Ling

PY - 2018/7/1

Y1 - 2018/7/1

N2 - We propose a sparse estimation method, termed MIC (Minimum approximated Information Criterion), for generalized linear models (GLM) in fixed dimensions. What is essentially involved in MIC is the approximation of the 0-norm by a continuous unit dent function. A reparameterization step is devised to enforce sparsity in parameter estimates while maintaining the smoothness of the objective function. MIC yields superior performance in sparse estimation by optimizing the approximated information criterion without reducing the search space and is computationally advantageous since no selection of tuning parameters is required. Moreover, the reparameterization tactic leads to valid significance testing results free of post-selection inference. We explore the asymptotic properties of MIC, and illustrate its usage with simulated experiments and empirical examples.

AB - We propose a sparse estimation method, termed MIC (Minimum approximated Information Criterion), for generalized linear models (GLM) in fixed dimensions. What is essentially involved in MIC is the approximation of the 0-norm by a continuous unit dent function. A reparameterization step is devised to enforce sparsity in parameter estimates while maintaining the smoothness of the objective function. MIC yields superior performance in sparse estimation by optimizing the approximated information criterion without reducing the search space and is computationally advantageous since no selection of tuning parameters is required. Moreover, the reparameterization tactic leads to valid significance testing results free of post-selection inference. We explore the asymptotic properties of MIC, and illustrate its usage with simulated experiments and empirical examples.

UR - http://www.scopus.com/inward/record.url?scp=85048716007&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85048716007&partnerID=8YFLogxK

U2 - 10.5705/ss.202016.0353

DO - 10.5705/ss.202016.0353

M3 - Article

AN - SCOPUS:85048716007

VL - 28

SP - 1561

EP - 1581

JO - Statistica Sinica

JF - Statistica Sinica

SN - 1017-0405

IS - 3

ER -