The regularization tag has no wiki summary.
5
votes
3answers
125 views
What does “degree of freedom” mean in neural networks?
In Bishop's book "Pattern Classification and Machine Learning", it describes a technique for regularization in the context of neural networks. However, I don't understand a paragraph describing that ...
1
vote
0answers
46 views
Forward Stepwise selection
I am assuming the following model:
$Y = \beta X + \epsilon$
Here both $X$ and $Y$ are matrices. I fit the least squares model without any regularization and get the matrix $\beta$. I would like to ...
2
votes
1answer
73 views
Parameter estimate for linear regression with regularization
For given cost function $S(\beta) = (Y - X \beta)^T(Y - X \beta) + \lambda \beta^T \beta$, where $\lambda$ is regularization parameter, the $\beta$ that minimizes the given cost function is $\beta = ...
0
votes
1answer
42 views
Alternatives to glmnet for feature selection on data with lots of NAs
I have a surgical database in which there are approximately 100,000 observations and 200 features. Each observation corresponds to a separate patient while the features correspond to either ...
0
votes
0answers
56 views
Ridge regression on subset of variables using SVD
I am trying to figure out an algorithm using singular value decomposition to run a modification of ridge regression in which only some of the variables are penalized.
I want the output to match the ...
3
votes
1answer
48 views
Question on the usage of regularization in applied statistics/science
I was reading the paper ``A significance test for the lasso'' by Lockhart, Tibshirani et al and was considering the issue of applying regularization in the applied sciences (for example, behavioral ...
4
votes
1answer
299 views
Gradient descent and elastic-net logistic regression
I'm currently in the process of trying to understand the paper Regularization Paths for Generalized Linear Models via Coordinate Descent by Friedman et al. with regard to the regularization of ...
12
votes
2answers
520 views
Fitting an ARIMAX model with regularization or penalization (e.g. with the lasso, elastic net, or ridge regression)
I use the auto.arima() function in the forecast package to fit ARMAX models with a variety of covariates. However, I often have a large number of variables to select from and usually end up with a ...
1
vote
0answers
103 views
Classification with 3 groups, repeated measurements, missing values, more predictors than subjects
I am working on a classification problem with the following characteristics:
Individuals belong to one of three groups. The groups are "somewhat ordinal": controls, subclinical and clinical group.
...
0
votes
0answers
193 views
matlab gmdistribution.fit 'Regularize' - what regularization method?
I am wondering what is behind matlab 'Regularize' option for method gmdistribution.fit. If it is simply adding a 'little' value to diagonal elements of covariance matrix, so as to make covariance ...
2
votes
0answers
81 views
Can the bias introduced by lasso change the sign of a coefficient?
L1 penalized regression introduces a bias on your regression model but decreases the variance. When this bias is introduced, is it possible that the coefficient of $B$ changes sign? This would ...
0
votes
1answer
66 views
Reducing the dimensionality of a problem
My particular application needs me to build a linear model with a strong correlation structure amongst the independent variables. The dimensions of the problem are high, for instance 1million X 200. ...
1
vote
0answers
42 views
kernelized l1 norm and the representer theorem
I'm trying to derive a kernel-ized $l_1$ penalty for logistic regression. I have been looking at the slides Learning with Sparsity Inducing Norms along with the slides on Regularization and Variable
...
1
vote
1answer
68 views
SVM optimization problem
I think I understand the main idea in support vector machines. Let us assume that we have two linear separable classes and want to apply SVMs. What SVM is doing is that it searches a hyperplane ...
1
vote
0answers
37 views
Representer theorem for vector-valued functions
Is there a representer theorem for loss-functions of the form $\sum_{i}(f(x_i \mathbb{.}),y_i)$ of the form where the output of $f(.)$ is a vector and the domain is also a vector. Also, there is a ...
