www-ai.cs.tu-dortmund.de/de/PERSONAL/MORIK/papers/pdf/Learning_Low_Rank_Document_Embeddings_with_Weighted_Nuclear_Norm_Regularization.pdf
Learning Low-Rank Document Embeddings with Weighted Nuclear Norm Regularization
We rewrite X = U(1)Σ(1)V
T (1) + U(2)Σ(2)V
T (2) where U(1) ∈
Rm×i0 ,Σ(1) ∈ Ri0×i0 , V(1) ∈ Rn×i0 contain the first i0 columns of U,Σ, V respectively and U(2),Σ(2), V(2) contain the remaining entries. A [...] X(t+ 1 2 ) = X(t) − α∇f(X(t)) (10)
X(t+1) = proxα,ε(X (t+ 1
2 )) (11)
The proximal operator for stepsize α is defined as
proxα,ε(Y ) = arg min X
1
2α ||X − Y ||2F + ||X||ε (12)
= arg min X
1
2 ||X − Y [...] for t = 1, ...
s(t) = 1 +
√ 1 + (2s(t−1))2
2 (14)
Y (t) = X(t) +
( s(t−1) − 1
s(t)
) (X(t) −X(t−1)) (15)
X(t+ 1 2 ) = Y (t) − α∇f(Y (t)) (16)
X(t+1) = proxα,ε(X (t+ 1
2 )) (17)
This modified algorithm is …
