最优化算法软件列表及比较，机器学习算法软件

1. 通用型
1.1. 无约束优化：

1). l-bfgs matlab wrapper to fortran code
2). minimize.m for Conjugate Gradient
3). fminunc median scale
4). fminunc large scale

比较
Function: 100-D rosenbrock
下图横轴是各个程序宣称的 iteration，纵坐标是 function value


照这个结果 matlab 的程序还是不错的。

 1.2. bound-constrained
lbfgsb， linux 下编译通过

1.3. IpOpt

貌似很通用
目标

min     f(x)
x in R^n

s.t.       g_L <= g(x) <= g_U
           x_L <=  x   <= x_U

where f(x): R^n --> R is the objective function, and g(x): R^n --> R^m are the constraint functions. The vectors g_L and g_U denote the lower and upper bounds on the constraints, and the vectors x_L and x_U are the bounds on the variables x. The functions f(x) and g(x) can be nonlinear and nonconvex, but should be twice continuously differentiable.

2. 具体问题
2.1 l1 regularized least squared., also sparse representation
l1_ls is developed for large problems. It can solve large sparse problems with a million variables with high accuracy in a few tens of minutes on a PC. It can also efficiently solve very large dense problems, that arise in sparse signal recovery with orthogonal transforms, by exploiting fast algorithms for these transforms.


2.2 linear classifier, logistic regression
LIBLINEAR -- A Library for Large Linear Classification
and LIBSVM

LIBLINEAR is a linear classifier for data with millions of instances and features. It supports L2-regularized logistic regression (LR), L2-loss linear SVM, and L1-loss linear SVM. 

Logistic regression:
L1 Regularized: An Interior-Point Method for Large-Scale l1-Regularized Logistic Regression. 有源码，linux 平台，支持 matlab, 需要 BLAS, LAPACK
Efficient L1 Regularized Logistic Regression Matlab, Matlab/C++ hybrid
Trunk Regularized (L2？): Logistic regression for binary classification
无 Regularized, IRLS 算法, Matlab

这篇文章用 lr 做 movie rating，通俗易懂，可以看看。注意数据的格式，feature 的含义。

3 machine learnning
3.1 Markov Random Fields:
MRF2.0, 参见 NOTE