End-to-End Trainable Weakly Non-Negative Factorization

Non-negative matrix factorization (NMF) is widely applied to analyze pattern data in an unsupervised manner. It imposes hard non-negativity constraints on factors to extract intrinsic characteristics from an input matrix, though demanding complicated optimization techniques which hinder the general applicability. Toward flexible formulation, we propose weakly non-negative factorization. In contrast to the strict non-negative approach, our method permits factors to contain small amount of negative values. The relaxation theoretically leads to an efficient factorization formulation which can be implemented by means of off-the-shelf techniques used in a deep learning literature. Thus, the method is flexibly applicable to versatile factorization tasks, such as deep NMF and structured NMF. In the experiments on the NMF-related tasks, we demonstrate that the weak non-negativity produces effective factors similarly to NMF and the method exhibits favorable performance in comparison to the other approaches.