959 General feature have not tried 2023-02-17 02:08 2023-02-17 02:08 SergeyB SergeyB normal assigned none none open 0 xcore-for-arm todo
958 Interpolation tweak have not tried 2023-02-16 21:56 2023-02-16 21:56 SergeyB SergeyB normal assigned none none open 0 C2 continuous BlockLLS add notice to comments that BlockLLS is C2 continuous, and that FastDDM is not
957 Interpolation major have not tried 2023-02-10 03:51 2023-02-10 03:51 SergeyB SergeyB normal assigned none none open 0 potential bug in FastDDM bicubic spline fitting routine The fitter produces merely C1 continuous models whilst it must produce C2. Possible bug in the model recombination.
952 Interpolation feature have not tried 2022-12-13 21:03 2022-12-13 21:04 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: far field (fast multipoles) acceleration of biharmonic RBF construction and evaluation New version of ALGLIB includes accelerated biharmonic RBFs which utilize far field expansion (AKA fast multipoles method) for approximate computation of the model. This method is heavily utilized by both model construction and model evaluation phases.

Biharmonic RBF construction time was now improved to O(N*logN) instead of O(N^2). We want to note that even previous time was still better than O(N^3) achieved by textbook methods.

Approximate biharmonic RBF evaluation time with rbffastcalc() now shows several-fold speed-up when compared with exact rbfcalc().
951 Optimization feature have not tried 2022-12-07 21:28 2022-12-07 21:28 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: LP/QP presolver Following kinds of presolve are implemented at this moment:
* fixed vars
* empty cols
* empty rows
* nonbinding rows
* singleton rows
* explicit slack variables
* implicit slack variables
* free column singletons
950 General feature have not tried 2022-11-27 20:11 2022-11-27 21:28 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: OpenSSL as an entropy source ALGLIB for C++ now can use OpenSSL as a secure randomness source (as opposed to stdlib rand).
949 Optimization feature have not tried 2022-07-14 11:55 2022-07-14 11:57 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: parallel interior point LP/QP New parallel sparse Cholesky also increased performance of the interior point LP/QP.
948 Optimization feature have not tried 2022-07-14 11:54 2022-07-14 11:54 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: parallel sparse Cholesky/LDLT factorization Implemented: parallel sparse Cholesky/LDLT factorization.
947 Optimization feature N/A 2022-06-29 13:15 2022-06-29 13:15 n.collins normal new none none 3.19.0 open 0 SIMD support for Arm Hello,

we wondered if you have plans to provide a NEON SIMD kernel implementation for arm processors? In particular we are interested in whether you plan to provide a SIMD implementation for Apple Silicon. Our software targets all desktop platforms. It would be useful to know if this is a feature we can expect in the future.

Kind regards,
Nathan
946 Interpolation feature have not tried 2022-06-28 15:31 2022-06-28 15:31 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: bilinear/bicubic spline with missing cells Bilinear/bicubic spline interpolation with missing nodes and cells.

If a node is marked as missing, then all spline cells that are adjacent to this node are marked as missing ones. Any attempt to compute spline value at the missing cell will return NAN.

This function is intended for better handling of grids with big areas of irrelevant nodes. It is not intended for scattered data interpolation - use RBFs or IDW for such problems.
945 Interpolation feature have not tried 2022-06-12 19:32 2022-06-14 13:46 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: fast multiquadric spline interpolation and fitting New large-scale multiquadric algorithm with O(N) memory requirements and O(N^2) running time, utilizing fast domain decomposition method for the solution of linear systems.

New algorithm supports interpolation and fitting (smoothing), with ability to handle more than 100.000 points on a modest workstation.
943 Interpolation feature have not tried 2022-06-12 19:30 2022-06-14 13:45 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: fast thin plate spline interpolation and fitting New large-scale thin plate spline algorithm with O(N) memory requirements and O(N^2) running time, utilizing fast domain decomposition method for the solution of linear systems.

New algorithm supports interpolation and fitting (smoothing), with ability to handle more than 100.000 points on a modest workstation.
944 Interpolation feature have not tried 2022-06-12 19:31 2022-06-12 19:33 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: fast biharmonic spline interpolation and fitting New large-scale biharmonic spline algorithm with O(N) memory requirements and O(N^2) running time, utilizing fast domain decomposition method for the solution of linear systems.

New algorithm supports interpolation and fitting (smoothing), with ability to handle more than 100.000 points on a modest workstation.
942 Optimization minor have not tried 2022-05-24 09:51 2022-05-24 09:51 SergeyB SergeyB normal resolved none none fixed 0 FIXED: instability in SLP solver SLP nonlinear programming solver was sometimes destabilized by problems having "flipping" constraints (ones that activate/deactivate often). Such constraints confused its estimates of Lagrange multipliers maximums (these maximums are used to penalize constrain violations).

Introduced new conservative update strategy for Lagrange multiplier maximums, the problem solved.
941 General minor have not tried 2022-05-21 22:10 2022-05-21 22:10 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: red zones for C++ version Red Zone is a fixed size area added before and after each dynamically allocated block. Red Zone is filled by the special red zone control value during its allocation. When the dynamically allocated block is freed, its control value is checked. Any change means that someone (either ALGLIB or user code that works with ALGLIB-allocated arrays) performed an out-of-bounds write.

Red Zones are essential for finding memory access errors that silently corrupt your data and/or crash your program.
935 Optimization minor have not tried 2021-10-24 00:45 2021-10-24 00:45 SergeyB SergeyB normal resolved none none fixed 0 FIXED: relaxed too strict decrease conditions in nonsmooth AGS optimizer Too strict line search conditions resulted in optimizer sometimes performing too short steps. Relaxed.
913 General feature have not tried 2020-12-10 22:24 2021-10-04 15:50 SergeyB SergeyB normal assigned none none open 0 better handling of nonlinear constraints in SLP/SQP drop inequality ones whose linearization is definitely inactive within trust region
905 Optimization feature have not tried 2020-11-01 23:49 2021-10-04 15:50 SergeyB SergeyB normal assigned none none open 0 inertia for SLP trust radius parameter growth todo
918 Diff.equations minor have not tried 2021-04-02 22:31 2021-10-02 18:52 SergeyB SergeyB normal resolved none none fixed 0 FIXED: ODE solver test hangs sometimes Error in the step size selection sometimes led to an eternal loop.
423 Linear algebra feature have not tried 2012-01-16 12:35 2021-09-30 18:46 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: sparse GMRES for linear equations Linear solver for square non-symmetric problems.
921 Optimization feature have not tried 2021-06-13 14:47 2021-09-24 16:03 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: minqpaddsparsefromdense Convenience function which adds constraint to the sparse constraint matrix, with constraint itself being passed as a dense array.
883 General feature have not tried 2019-12-20 12:56 2021-09-23 23:05 SergeyB SergeyB normal assigned none none open 0 Delphi and CPython tracing todo
930 Linear algebra feature have not tried 2021-07-28 15:42 2021-09-23 23:04 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: AVX2/FMA-capable sparse Cholesky/LDLT decomposition Sparse Cholesky/LDLT decompositions now have optimized SIMD kernels which may use AVX2 and FMA intrinsics.
885 General feature have not tried 2019-12-20 13:38 2021-09-23 23:02 SergeyB SergeyB normal assigned none none open 0 reference manual section for tracing todo
929 Linear algebra feature have not tried 2021-07-28 15:41 2021-09-17 21:58 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: C# GEMM (matrix-matrix product) can now utilize AVX2/FMA NET 5 intrinsics C# implementation of GEMM (generalized matrix-matrix product) can now be compiled in two modes:
* fully managed C# code
* unsafe C# code which makes use of NET 5 hardware intrinsics

Because the GEMM kernel is widely used across the library, many linear algebra, interpolation and optimization functions became much faster due to this change.
933 General feature have not tried 2021-09-17 21:50 2021-09-17 21:50 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: ALGLIB for C# supports NET 5, including hardware intrinsics 1. NET 5 compatibility officially guaranteed
2. Ability to utilize NET 5 hardware intrinsics when compiled in the unsafe mode with ALGLIB_USE_SIMD conditional compilation symbol being defined.
882 Optimization feature have not tried 2019-12-18 18:47 2021-08-28 23:15 SergeyB SergeyB normal assigned none none open 0 improved AUL rho growth.
large-scale problems.
859 Optimization minor have not tried 2019-07-01 17:06 2021-08-28 23:07 SergeyB SergeyB normal assigned none none open 0 perform pivot selection in simplex solver taking into account |rho| todo
743 Optimization feature have not tried 2017-10-24 20:05 2021-08-28 22:48 SergeyB SergeyB normal assigned none none open 0 linearly constrained MinLM improvements reuse Lagrange multipliers and working set
tests for huge amount of inequality constraints
562 Optimization feature have not tried 2013-09-13 12:12 2021-08-28 22:45 SergeyB SergeyB normal assigned none none open 0 SQP version of MinNLC Needs following improvements:
* there should convenient way to generate low-rank LBFGS matrices
* QP solver should be able to work with low-rank QP models (L-BFGS QP?)
* QP solver should be able to perform efficient restarts (fast activation of previously active constraints)
926 Linear algebra feature have not tried 2021-07-22 17:27 2021-08-28 22:42 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: AMD ordering for matrices with dense rows Implemented new AMD ordering algorithm for matrices with dense/nearly dense rows, set as default one for sparse Cholesky/LDLT.

Details: original AMD ordering algorithm has O(N^2) running time for a matrix with just one dense row. Improved algorithm postpones ordering of rows with too many nonzero elements (including ones which are not dense, but still has more nonzeros than typical row) and handles them later, at the second ordering round.
931 Linear algebra feature have not tried 2021-08-06 19:10 2021-08-06 19:11 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: sparse matrix serialization Implemented: sparsematrix structure can be serialized/unserialized using standard ALGLIB serialization interface (supports movement of data between C++, C#, other languages).
920 Optimization minor have not tried 2021-06-13 14:33 2021-06-13 14:33 SergeyB SergeyB normal resolved none none fixed 0 FIXED: nonsmooth AGS optimizer sometimes stopped before reaching optimal point AGS optimizer sometimes stopped before reaching optimal point due to incorrect sampling under active constraints. Fixed sampling algorithm.
919 Optimization minor always 2021-04-27 16:44 2021-04-27 16:44 YitziK normal new none none 3.17.0 open 0 Default stopping conditions for MINBLEIC algorithm are too stringent When passing in all 0s to minbleicsetcond, it is (according to documentation) supposed to lead to automatic stopping criterion selection. However, when this is done in the attached file, the result is a termination code of 7: "stopping conditions are too stringent, further improvement is impossible", making it difficult, for more complex cases, to determine if the issue is due to overly stringent automatic stopping criteria, or an issue with the problem setup that might also cause a significantly suboptimal point to be selected. The attached file is significantly simplified from the case in which I first encountered it, but sufficient to reproduce the issue (though the include path for alglib will likely need to be modified for your own setup). For simplicity, it is optimizing the radius (or, more properly, the radius squared; i.e. sum of squares of the parameters). There are 14 parameters, with a simple box constraint (all parameters must be nonnegative), and one linear constraint that divides the parameters into 3 categories: The first 10 parameters have contribution 1, the next two have contribution -1, and the final 2 have contribution 0.5. The total value of the constraint is required to equal exactly 8, and the initial position does so by assigning a value of 1 to the first 8 parameters and 0 to the others. main.cpp (2 KB) 2021-04-27 16:44
746 General feature have not tried 2017-10-30 15:24 2021-04-02 14:36 SergeyB SergeyB normal assigned none none open 0 blocked random matrix operations todo
741 General feature have not tried 2017-10-24 13:17 2021-04-02 14:36 SergeyB SergeyB normal assigned none none open 0 advanced affinity management for C/C++ core todo

document -DAE_OS=AE_LINUX

decide on default status of affinity management, allow to change in on the fly
728 Linear algebra feature have not tried 2017-09-25 19:01 2021-04-02 14:36 SergeyB SergeyB normal assigned none none open 0 New sparse storage format - COO initializer Faster initialization
565 Data analysis feature have not tried 2013-09-29 11:33 2021-04-02 14:36 SergeyB SergeyB normal assigned none none open 0 Improved Naive Bayes Improvements:
* different ways of estimating class probabilities (see skikit)
* sparsity support
* complementary Bayes
* sparse datasets

Look at:
* http://scikit-learn.org/stable/modules/naive_bayes.html

Papers:
* http://people.csail.mit.edu/jrennie/papers/icml03-nb.pdf (Tackling the Poor Assumptions of Naive Bayes Text Classi ers)
748 Optimization feature have not tried 2017-10-31 12:14 2021-04-02 14:36 SergeyB SergeyB normal assigned none none open 0 Better warm start support for dense-aul and related minlm Todo
886 Optimization feature have not tried 2019-12-23 16:46 2021-04-02 14:34 SergeyB SergeyB normal assigned none none open 0 QP/LP presolver todo
880 Optimization feature have not tried 2019-12-18 18:46 2021-04-02 14:34 SergeyB SergeyB normal assigned none none open 0 Lagrange multipliers for NLP todo
879 Optimization feature have not tried 2019-12-11 14:12 2021-04-02 14:34 SergeyB SergeyB normal assigned none none open 0 MinNLC needs modern setlc2() family of functions and setnlc2() todo
872 Linear algebra feature have not tried 2019-11-26 16:40 2021-04-02 14:34 SergeyB SergeyB normal assigned none none open 0 INCOMPATIBLE: standardize linear solver output parameters todo
848 Interpolation feature have not tried 2018-12-25 11:54 2021-04-02 14:34 SergeyB SergeyB normal assigned none none open 0 1d spline serialization todo
843 Optimization feature have not tried 2018-11-18 13:30 2021-04-02 14:34 SergeyB SergeyB normal assigned none none open 0 "paranoid SLP" optimizer detailed investigation of the target function properties
830 Optimization feature have not tried 2018-10-30 15:20 2021-04-02 14:34 SergeyB SergeyB normal assigned none none open 0 autoscaling for minlm/lsfit todo
804 Interpolation feature have not tried 2018-03-27 17:39 2021-04-02 14:34 SergeyB SergeyB normal assigned none none open 0 lsfitsetscaleauto() todo
799 Solvers feature have not tried 2018-03-05 16:27 2021-04-02 14:34 SergeyB SergeyB normal assigned none none open 0 LSQR solver - better handling of LambdaV explicitly integrate it in the formula
782 Interpolation feature have not tried 2018-01-08 14:17 2021-04-02 14:34 SergeyB SergeyB high assigned none none open 0 Inverse distance weighting, improved version Parallel algorithm
Optional distance-based smoothing
767 General feature have not tried 2017-12-11 18:38 2021-04-02 14:34 SergeyB SergeyB normal assigned none none open 0 better C#-to-native memory management * transient allocations (session object)
* move from fixed statement to GCHandle object; freely choose between pinned/attached storage and independent native storage for input arguments
756 General feature have not tried 2017-11-07 21:10 2021-04-02 14:34 SergeyB SergeyB normal assigned none none open 0 nonsmooth BFGS todo

A combined BFGS-Gradient Sampling method with convergence
theory (F.E. Curtis and X. Que, 2015)

A successive quadratic programming (SQP) BFGS method
applied to challenging problems in static-output-feedback control
design (F.E. Curtis, T. Mitchell and M.L.O., 2015).
Although there are no theoretical results, it is much more
efficient and effective than the SQP Gradient Sampling method
which does have convergence results.


==== review

A successive quadratic programming gradient sampling method
with convergence theory.
F.E. Curtis and M.L.O., SIOPT, 2012.
755 Interpolation feature have not tried 2017-11-07 20:13 2021-04-02 14:34 SergeyB SergeyB normal assigned none none open 0 switch circumscribed/inscribed circle fitting to new SLP-NLC todo
708 General feature have not tried 2017-06-08 18:23 2021-04-02 14:34 SergeyB SergeyB normal assigned none none open 0 ability to control RNG seed for HPC core todo
685 Linear algebra feature have not tried 2017-02-07 14:34 2021-04-02 14:34 SergeyB SergeyB normal assigned none none open 0 SYMM-based code for dense subspace iteration and for SSA Now it uses temporary buffer to unroll matrix.
646 Linear algebra feature have not tried 2015-03-13 17:00 2021-04-02 14:34 SergeyB SergeyB high assigned none none open 0 Faster XBLAS, faster iterative refinement todo
684 Linear algebra feature have not tried 2017-02-07 14:33 2021-04-02 14:29 SergeyB SergeyB normal assigned none none open 0 Implement RMatrixSYMM/CMatrixHEMM todo
884 Optimization feature have not tried 2019-12-20 12:57 2021-04-02 14:26 SergeyB SergeyB normal assigned none none open 0 interchangeable scalar/vector callbacks in optimizers todo
917 Optimization minor have not tried 2021-03-22 18:37 2021-03-22 18:37 SergeyB SergeyB normal resolved none none fixed 0 FIXED: better estimate of penalty coefficients for SLP merit function Better estimate of penalty coefficients for SLP merit function improves solver convergence.
916 Optimization minor have not tried 2021-03-22 18:34 2021-03-22 18:35 SergeyB SergeyB normal resolved none none fixed 0 FIXED: better detection of trust region stagnation in the SQP optimizer Better detection of trust region stagnation in the SQP optimizer.
908 Solvers feature have not tried 2020-11-07 21:51 2020-12-19 17:49 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: sparse symmetric positive definite direct solver Implemented linear solver based on supernodal Cholesky with fill-in reducing ordering
914 Solvers tweak have not tried 2020-12-19 17:19 2020-12-19 17:47 SergeyB SergeyB normal resolved none none implemented 0 INCOMPATIBLE: changed signatures and names of some sparse direct solvers Names and signatures of some sparse solvers are changed to make API more uniform. First, we removed matrix size from the method signature because sparse matrix structure already stores its size. Second, we changed order of output parameters: instead of "report,x" we now return "x,report" as does the rest of the library. We also changed names of several methods to conform to uniform naming convention used by direct dense solvers.

Following functions were changed:
* sparsesolvesks(a,n,isupper,b,rep,x) => sparsespdsolvesks(a,isupper,b,x,rep)
* sparsecholeskysolvesks(a,n,isupper,b,rep,x) => sparsespdcholeskysolve(a,isupper,b,x,rep)
* sparsesolve(a,n,b,x,rep) => sparsesolve(a,b,x,rep)
* sparselusolve(a,p,q,n,b,x,rep) => sparselusolve(a,p,q,b,x,rep)
911 Optimization feature have not tried 2020-12-07 22:15 2020-12-07 22:15 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: sparse interior point LP solver Sparse large-scale interior point (barrier) linear programming solver
910 General feature have not tried 2020-11-28 16:09 2020-11-28 16:09 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: generating random vectors/matrices with normal entries Implemented hqrndnormalv() and hqrndnormalm()
909 Optimization feature have not tried 2020-11-08 12:51 2020-11-08 12:51 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: sparse interior point QP solver Interior point QP engine now supports both dense (medium scale) and sparse (large scale) problems. The latter ones are solved using new supernodal Cholesky decomposition routine with AMD ordering applied to (M+N)*(M+N) quasidefinite KKT system (Vanderbei-style QP).
907 Linear algebra feature have not tried 2020-11-07 16:40 2020-11-07 16:40 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: sparse supernodal Cholesky and LDLT factorizations with fill-in reducing ordering (AMD) Sparse supernodal factorization (both positive definite Cholesky and indefinite LDLT with diagonal D) is implemented. It supports fill-in reducing ordering (approximate minimum degree).
906 Linear algebra feature have not tried 2020-11-07 12:46 2020-11-07 12:46 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: optimized sparse matrix transposition Improved pattern of memory accesses, no unnecessary dynamically allocated temporaries (all memory allocated during copy+transpose is reused by the target).
904 Linear algebra feature have not tried 2020-10-08 12:31 2020-10-08 12:31 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: SparseExists() for quick tests for element existence This function checks that element exists in the sparse matrix structure (returns True even for elements that are numerically zero, but still have slot allocated for them)
903 Linear algebra feature have not tried 2020-10-07 22:04 2020-10-07 22:04 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: symmetric permutation of sparse symmetric matrix SparseSymmPermTbl() allows to perform symmetric (two-sided) permutation of symmetric matrix given by its upper or lower triangle.
902 Interpolation minor have not tried 2020-08-02 23:05 2020-08-02 23:05 SergeyB SergeyB normal resolved none none fixed 0 FIXED: interior point method for LP/QP was destabilized by huge bounds (like x<1.0E10) Fixed. Adaptive algorithm performs early detection of huge and irrelevant bounds, drops them and restarts computation.
901 Optimization feature have not tried 2020-08-02 23:00 2020-08-02 23:00 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: improved handling of sparse pivot row in simplex solver Improved sparsity support, ~30% improvement on some problems.
891 Optimization feature have not tried 2020-01-08 11:30 2020-01-08 11:30 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: stabilized Hessian updates for SQP solver with periodic resets 1. Stabilized Hessian updates with better guards against degenerate line searches

2. Periodic Hessian resets in order to improve algorithm behavior on highly nonquadratic problems with varying curvature. Behavior on quadratic problems is almost same (a few percents of performance drop).
890 Optimization feature have not tried 2019-12-30 17:06 2020-01-08 11:28 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: stronger OptGuard integrity checking Improved detection of nonsmoothness in nonlinear constraints. More detailed report (reports iteration index in addition to line search log).
889 Optimization minor have not tried 2019-12-23 16:48 2019-12-23 16:49 SergeyB SergeyB normal resolved none none fixed 0 FIXED: tolerance thresholds in the infeasibility deteciton code were too strict Fixed. Original code sometimes reported infeasibility on feasible problems.
888 Optimization minor have not tried 2019-12-23 16:47 2019-12-23 16:47 SergeyB SergeyB normal resolved none none fixed 0 FIXED: suboptimal decision during initial point selection in IPM method Original code made suboptimal decision during initial point generation. Sometimes algorithm took too much time to converge.
824 Optimization minor always 2018-09-04 11:57 2019-12-18 15:54 rmanthorpe SergeyB normal resolved none none 3.14.0 fixed 0 FIXED: breaking change in exception handling in user callbacks Fixed. Now exceptions generated in user callbacks are forwarded to external code.

---------------------------------------------------------------

The example below should print "user error" but instead prints "ALGLIB: exception generated in user callback". This change happened between 3.12 and 3.13. It's now no longer possible to find out what caused optimisation to fail, only that an exception was thrown.

test.cpp
---------------------------------------------------------------
#include "optimization.h"
#include <cstdio>

void callback(const alglib::real_1d_array&, alglib::real_1d_array&, void*)
{
    throw "user error";
}

int main()
{
    alglib::minlmstate state;
    alglib::real_1d_array x;
    x.setlength(1);
    alglib::minlmcreatev(1, x, 1e-5, state);
    try
    {
        alglib::minlmoptimize(state, &callback);
    }
    catch (const char* e)
    {
        std::printf(e);
    }
    catch (alglib::ap_error& e)
    {
        std::printf(e.msg.c_str());
    }
    return 0;
}
858 Optimization minor have not tried 2019-07-01 17:05 2019-12-13 18:18 SergeyB SergeyB normal resolved none none fixed 0 IMPLEMENTED: more precise choice of pivot element in LP solver Now it uses AlphaR[Q] instead of AlphaQ[R] (ideally, both are same, but in the presence of numerical rounding former is preferable).
878 Optimization feature have not tried 2019-12-11 14:11 2019-12-11 14:11 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: SQP nonlinear programming solver Implemented SQP nonlinear programming solver
877 Linear algebra feature have not tried 2019-12-10 18:20 2019-12-10 18:20 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: sparsegemv() Generalized matrix-vector product y := Alpha*op(S)*x + Beta*y, can be computed for CRS and SKS matrices.
875 Optimization feature have not tried 2019-11-28 13:53 2019-11-28 13:53 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: QP solvers now report Lagrange multipliers DENSE-AUL, DENSE-IPM and SPARSE-IPM solvers now report Lagrange multipliers for box and general linear constraints.
863 General feature have not tried 2019-10-23 18:11 2019-11-27 13:48 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: trace infrastructure for ALGLIB solvers and algorithms Now ALGLIB solvers and algorithms can performed detailed and tunable tracing of their progress, with trace log being saved to file.

Presently only MINNLC.SLP and MINQP.IPM (dense and sparse) solvers support tracing, but additional traceable algorithms are expected in next releases.
873 Solvers feature have not tried 2019-11-26 20:20 2019-11-26 20:20 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: sparse nonsymmetric real solvers Implemented sparsesolve(), nonsymmetric real solver which utilizes sparse LU factorization, and sparselusolve(), which accepts already factorized matrix representation.
871 Optimization feature have not tried 2019-11-24 13:16 2019-11-24 13:16 SergeyB SergeyB normal resolved none none implemented 0 INCOMPATIBLE: changed parameters of minqpsetlcmixed() Since version 3.16.0 ALGLIB treats mixed sparse/dense linear constraints as a set of sparse ones followed by dense ones. Specific ordering of constraints is important when we want to tie Lagrange multipliers to them.

Previous versions of ALGLIB accepted constraints in reverse order, which was reflected in signature of minqpsetlcmixed() - dense constraints were specified first, followed by sparse ones.

Since 3.16.0, old minqpsetlcmixed() was renamed to minqpsetlcmixedlegacy(), and new version of this function accepts constraints in the correct order.
870 Optimization feature have not tried 2019-11-24 13:08 2019-11-24 13:08 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: QP optimizer now supports two-sided linear constraints Now you can specify constraints of the form AL<=A*x<=AU. Such constraints are handled more efficiently by the modern IPM solver.
868 Optimization feature have not tried 2019-11-19 19:26 2019-11-19 19:26 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: dense interior point QP solver Implemented dense interior point QP solver for convex and semidefinite QP problems with moderate (up to several thousands) variables count.

New algorithm is order of magnitude faster than BLEIC-QP and DENSE-AUL-QL solvers on general QP problems (although some special cases are still better handled by BLEIC/DENSE-AUL).
867 Spec.functions feature have not tried 2019-11-19 13:20 2019-11-19 13:20 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: bivariate normal CDF and PDF Implemented probability density function and cumulative distribution function for bivariate normals.
862 Optimization feature have not tried 2019-09-03 17:29 2019-10-23 18:12 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: improved SLP solver Improved stability of the SLP solver, at the cost of some decrease in running time. Now solver is less prone to stalling in bad points (more exhaustive search for good conjugate directions is performed).
737 Interpolation feature have not tried 2017-10-16 19:07 2019-09-03 16:19 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: fast fitting by penalized cubic spline Implemented fast fitting by penalized cubic spline. New function, spline1dfit(), replaces obsolete spline1dfitpenalized(). It has O(N*logN) running time and O(N) memory requirements, which is much faster than previous O(N^3) time / O(N^2) space.
861 Data analysis feature have not tried 2019-08-20 16:04 2019-08-20 16:04 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: variable importance for random forests Implemented three variable importance algorithms:
* MDI (mean decrease in impurity)
* OOB-MDI, an out-of-bag version of MDI
* MDA (mean decrease in accuracy), also known as permutation importance
484 Data analysis feature have not tried 2012-08-12 17:43 2019-08-20 16:02 SergeyB SergeyB normal assigned none none open 0 Decision forest improvements * features are replaced by their ranks before learning (faster and easier training)
* ability to merge forests
* other forest construction algos: boosting
860 Data analysis feature have not tried 2019-07-29 13:35 2019-07-29 13:35 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: compressed storage format for random forests (3.7x-5.7x memory usage reduction) New compressed storage format for random forests allows to achieve 3.7x-5.7x reduction in RAM usage due to several binary compression techniques being applied.

The only downsides of the approach are (a) slight reduction in model accuracy due to rounding to 24-bit floating point format, and (b) ~1.5x reduction in the inference speed.
855 Data analysis feature have not tried 2019-02-21 16:09 2019-02-21 16:09 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: added dfprocess0() and dfclassify() convenience wrappers Added convenience wrappers around dfprocess() inference function.

Former is intended for regression and binary classification problems (function result is a scalar instead of vector), latter is intended for multiclass problems (function returns most probable class index).
854 Data analysis feature have not tried 2019-02-19 17:31 2019-02-19 17:31 SergeyB SergeyB normal assigned none none open 0 regression forests with multiple dependend vars todo
851 Data analysis feature have not tried 2018-12-27 18:44 2019-02-19 16:38 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: k-nn classification/regression k-nn classification/regression using ALGLIB implementation of kd-trees
853 Data analysis feature have not tried 2019-02-18 12:58 2019-02-18 12:58 SergeyB SergeyB normal assigned none none open 0 nca and lmnn improvements to knn classifier * neighborhood component analysis
* large margin nearest neighbors
852 Optimization feature have not tried 2019-01-31 21:39 2019-01-31 21:39 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: revised dual simplex method Implemented: large-scale sparse dual simplex method, with DSE pricing, long dual step (aka bounds flipping ratio test) and Forest-Tomlin updates.
825 Linear algebra feature have not tried 2018-09-10 13:00 2019-01-16 19:38 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: sparse LU factorization with complete (row/col) and partial (row) pivoting Implemented sparse LU factorization with partial (row) pivoting for stability, and complete pivoting for combined stability and sparsity.
838 Optimization feature have not tried 2018-11-09 15:19 2018-12-27 18:12 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: MinNLC optimizer now reports violation of constraints Maximum violation of box, linear and nonlinear constraints (separately for each constraint type) is reported as well as index of the most violated constraint.
822 Optimization minor have not tried 2018-08-20 17:49 2018-12-27 18:12 SergeyB SergeyB normal resolved none none fixed 0 FIXED: gradient checking functionality in nonlinear optimizers does not handle scaling properly === ORIGINAL REPORT BY USER ===
Problem: When I run minlbfgssetgradientcheck(state, 1e-4) to verify that the numerical gradient is accurate enough, I only get a good result when the scale is set to 1 for each variable. If I first set the scale the way I think is desirable using minlbfgssetscale(state, s), I always get the -7 status. My hypothesis is that the gradient check does not take the scale into account properly. More specifically, I think that the line

 

    if( !derivativecheck(state->fm1, state->fp1, state->fm2, state->fp2, state->f, state->g.ptr.p_double[i], 2*state->teststep, _state) )

 

should be:

 

    if( !derivativecheck(state->fm1, state->fp1, state->fm2, state->fp2, state->f, state->g.ptr.p_double[i], 2*state->teststep * state->s.ptr.p_double[i], _state) )

 

to take the scaling properly into account.

=== SOLUTION ===

Switched everything to new OptGuard API which has this error fixed.
810 Optimization feature have not tried 2018-04-10 18:00 2018-12-27 12:23 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: SLP solver now is default option in MinNLC MinNLC optimizer now uses SLP solver by default as the most robust one (although slower than AUL).
841 Optimization feature have not tried 2018-11-17 23:04 2018-12-26 18:16 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: termination requests for MinNLC Now MinNLC subpackage supports requests for immediate termination of the optimizer just as the rest of the Optimization package.
850 Optimization feature have not tried 2018-12-26 16:27 2018-12-26 16:27 SergeyB SergeyB normal resolved none none fixed 0 FIXED: improved stability of the gradient checking code Better handling of numerical noise, lesser probability of the false positives for bad analytic gradient.
849 Interpolation feature have not tried 2018-12-26 15:53 2018-12-26 15:53 SergeyB SergeyB normal assigned none none open 0 gridcalc2/gridcalc3 for IDW todo
834 Interpolation minor have not tried 2018-11-01 20:31 2018-12-26 14:10 SergeyB SergeyB normal resolved none none fixed 0 FIXED: logisticfit4 not giving expected results 4PL/5PL logistic fitting functions failed to achieve best results on some skewed datasets due to excessive regularization applied during fitting. Fixed.

See http://forum.alglib.net/viewtopic.php?f=2&t=3871 for discussion and more info.
837 Interpolation feature have not tried 2018-11-07 16:53 2018-12-26 13:30 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: multilayer inverse distance weighting Fast interpolation algorithm with O(N*logN) model construction phase. Builds multilayer IDW model which overcomes limitations of the traditional Shepard's method.
842 Optimization feature have not tried 2018-11-18 13:29 2018-12-24 17:01 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: new OptGuard API for all optimizers (LBFGS, CG, BC, BLEIC, LM, NLC) New OptGuard API allows ALGLIB users to catch common coding and problem statement errors like:
* bad analytic gradient (usually: result of the coding errors)
* nonsmooth target function (usually: result of the problem statement errors)
* discontinuous target function (usually: result of the problem statement errors)

Similar API is provided for all ALGLIB optimizers.

Upon discovering bad analytic gradient OptGuard integrity checker returns both user-provided (bad) gradient/Jacobian and its reference value obtained via numerical differentiation. Upon discovering nonsmooth target both short report and detailed line search logs are returned.
847 Optimization feature have not tried 2018-12-20 15:57 2018-12-20 15:57 SergeyB SergeyB normal assigned none none open 0 nonsmooth optimizers: bundle methods, proximal bundle methods convex/nonconvex
840 Interpolation feature have not tried 2018-11-17 19:40 2018-11-17 19:40 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: HRBF models now supports progress reports and termination requests Hierarchical RBFs (HRBFs) model construction now supports two important features:
* progress reports - you can peek into current progress from some other thread with rbfpeekprogress() function
* termination requests - you can smoothly terminate RBF model construction from some other thread with rbfrequesttermination() function
839 Solvers feature have not tried 2018-11-16 18:19 2018-11-16 18:20 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: LSQR solver now supports progress reports and requests for termination New asynchronous functionality:
* ability to peek into LSQR solver progress (get current iterations count)
* ability to submit request for immediate termination of the running solver
835 General feature have not tried 2018-11-02 16:40 2018-11-02 16:40 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: kd-trees now support unordered distance-based queries Unordered distance-based queries (all neighbors within user-specified distance) are faster than standard ones which sort points by their distance. Specific speed-up is about 10% for small queries (a few points are returned) but can be higher for larger queries.
833 General feature have not tried 2018-11-01 14:01 2018-11-01 14:01 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: better optimized C/C++ implementation of kd-trees Optimized floating-point comparisons in kd-tree code and tagged heap code, results in significant increase in performance of the tree construction (25-50% reduction in build time) and queries (10% for 1-NN queries, larger gains for larger queries).
823 Linear algebra feature have not tried 2018-08-29 15:02 2018-11-01 13:52 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: faster sparsetrsv() New version of sparse TRSV (triangular solve) performs overflow checks only once per call - at the end of the function. On highly sparse matrices it results in the significant difference in performance.
826 General feature have not tried 2018-09-11 12:25 2018-09-11 12:25 SergeyB normal new none none open 0 LM solver has to decrease step upon encountering infinity/nan todo
821 Linear algebra feature have not tried 2018-07-23 13:30 2018-07-23 13:30 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: copying with transposition of sparse CRS matrices In addition to in-place transposition of sparse CRS matrices it is now possible to perform copy with transposition.
820 Data analysis feature have not tried 2018-05-31 16:14 2018-05-31 16:14 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: improved random forests construction algorithm ALGLIB 3.14 introduces new random forests interface (fully compatible with previous releases) and better construction algorithms.

Following improvements were achieved over previous version of the algorithm:
* 2x...10x faster forest training
* significantly improved inference speed
* orders of magnitude smaller trees due to early leaf creation
* slightly lower classification errors due to deeper split scans
819 Linear algebra feature have not tried 2018-05-17 14:34 2018-05-17 14:34 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: sparsemv() and sparsemtv() now can use Intel MKL Implemented.
818 Data analysis feature have not tried 2018-05-17 14:21 2018-05-17 14:21 SergeyB SergeyB normal assigned none none open 0 sparse GEMM usage in sparse PCA todo
817 Data analysis feature have not tried 2018-05-16 17:55 2018-05-17 13:02 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: sparse truncated PCA Truncated PCA (a few top values are extracted) for sparse datasets.
727 Data analysis feature have not tried 2017-09-25 18:15 2018-05-16 16:46 SergeyB SergeyB normal assigned none none open 0 Gradient boosting trees todo

* http://fastml.com/what-is-better-gradient-boosted-trees-or-random-forest/
* An Empirical Comparison of Supervised Learning Algorithms Using Different Performance Metrics (2005) by Rich Caruana , Alexandru Niculescu-Mizil
* An empirical evaluation of supervised learning in high dimensions (2008)
by Rich Caruana , Nikos Karampatziakis , Ainur Yessenalina
* Obtaining Calibrated Probabilities from Boosting, Alexandru Niculescu-Mizil, Rich Caruana
816 Data analysis feature have not tried 2018-05-14 19:03 2018-05-14 19:03 SergeyB SergeyB normal assigned none none open 0 kernel PCA todo
776 General minor have not tried 2017-12-24 13:19 2018-05-08 12:33 SergeyB SergeyB normal resolved none none fixed 0 FIXED: memory leak in the heavily threaded code utilizing Intel MKL === Problem description ===

Intel MKL in its default configuration allocates thread-local buffer storage which is used to accelerate computations (create aligned copies of data). The problem is that this buffer is not deallocated after exit from MKL - just for the case same thread will call MKL one more time.

When application spawns many threads, it results in creation of multiple thread-local buffers; when many threads are spawned constantly (as it happens with some threading frameworks, including NET Task Parallel Library) it results in steadily growing memory consumption.

To be exact, memory is not actually lost - it can be freed with just one call of mkl_free_buffers(). However, such behavior is still undesirable.

=== Solution ==

A watchdog thread starts on MKL startup which performs periodic (once in a few seconds) cleanup of unused buffers.
696 Interpolation feature have not tried 2017-04-29 19:13 2018-05-07 17:34 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: fitting least squares circle/sphere to NX-dimensional data New circle/sphere fitting function, fitspherels(), which fits sphere minimizing sum of squared distances.
813 Optimization minor always 2018-04-24 11:06 2018-04-27 16:06 rmanthorpe SergeyB normal resolved none none 3.13.0 fixed 0 FIXED: minlmoptimize enters an infinite loop when fvec throws an exception on Windows x64 In recent versions of MSVC setjmp/longjmp used by ALGLIB internally interferes with throw/catch also used by ALGLIB to catch user exceptions. MSVC does not support longjmp() out of catch() clause (even though jump itself seems to be legitimate). Code starts looping forever.

New version of ALGLIB exception handling code better separates these two strategies, so no eternal loop takes place.

=== ORIGINAL MESSAGE ===================================================

The example below should exit with status 1 but never completes on Windows x64 using Visual Studio 2015 Update 3. It works as expected on Windows x86 and Linux.

test.cpp
---------------------------------------------------------------
#include "optimization.h"

void callback(const alglib::real_1d_array&, alglib::real_1d_array&, void*)
{
    throw 0;
}

int main()
{
    alglib::minlmstate state;
    alglib::real_1d_array x;
    x.setlength(1);
    alglib::minlmcreatev(1, x, 1e-5, state);
    try
    {
        alglib::minlmoptimize(state, &callback);
    }
    catch(...)
    {
        return 1;
    }
    return 0;
}
SergeyB 2018-04-26 15:51 OK, confirmed issue, started to debug
SergeyB 2018-04-26 18:06 It turned out to be easy - in recent versions of MSVC setjmp/longjmp used by ALGLIB internally interferes with throw/catch also used by ALGLIB to catch user exceptions. Will be fixed in the upcoming 3.14.0.

Thank you for the report!
812 General feature have not tried 2018-04-19 18:51 2018-04-26 17:56 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: support for IDisposable interface and using semantics by ALGLIB for C# All objects in ALGLIB for C# now implement IDisposable interface.

In managed edition Dispose() call does nothing, in HPC edition (native core) it performs immediate deallocation of unmanaged memory allocated by ALGLIB.
800 General feature have not tried 2018-03-06 13:34 2018-04-26 14:24 SergeyB SergeyB normal resolved none none implemented 0 INCOMPATIBLE: new approach to multithreading ALGLIB 3.13.0 and earlier versions used single-threaded code by default. Multithreaded processing had to be activated manually by calling "smp_" versions of ALGLIB functions, with default option to utilize all cores except for one.

Since ALGLIB 3.14.0, there is no "smp_"-prefixed versions of its functions. By default library uses serial processing, but you can indicate that you want to perform parallel execution either by enabling parallelism at global level with setglobalthreading() function/method, or by enabling it at call-local level by passing alglib::parallel with other function parameters.

By default library tries to utilize all cores (when parallel processing is activated). You can change number of worker threads with setnworkers() function/method.
806 Interpolation minor have not tried 2018-03-30 16:06 2018-04-06 13:08 SergeyB SergeyB normal resolved none none fixed 0 FIXED: LSFit solver sometimes returns incorrect error estimates (HPC edition of ALGLIB for C#) LSFit solver performs actions which violate reverse communication protocol (reallocates one of the communication fields during optimization process). Although solution itself is detected/returned correctly, corrupt error estimates can be returned as result.

This issue is relevant only for HPC edition of ALGLIB for C# (commercially licensed managed wrapper around native computational core). Managed edition and C++ edition can tolerate protocol violation.
809 Optimization feature have not tried 2018-04-03 18:26 2018-04-03 18:26 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: new nonlinear programming solver - SLP New nonlinearly constrained optimizer: SLP (Successive Linear Programming) solver with second order convergence properties. Supports (non)convex problems with a mix of box, linear and nonlinear constraints.
807 Optimization feature have not tried 2018-04-03 17:04 2018-04-03 17:04 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: improved tolerance to rounding noise in BLEIC optimizer Improved iteration code more robust against numerical noise in BLEIC and QP-BLEIC optimizers.
781 General minor have not tried 2018-01-08 12:09 2018-04-03 14:01 SergeyB SergeyB normal resolved none none fixed 0 FIXED: added #include ctype.h in ap.cpp Some old compilers complain about tolower() function not present in default includes. Fixed.
783 General minor have not tried 2018-01-15 18:15 2018-04-03 13:56 SergeyB SergeyB normal resolved none none fixed 0 FIXED: removed dependency on InterlockedCompareExchange64 (not present on 32-bit Windows XP systems) Done
801 General major have not tried 2018-03-08 21:50 2018-03-08 21:51 SergeyB SergeyB normal resolved none none fixed 0 FIXED: bug in minbleicsetlc() and related functions BUG: subsequent call of minbleicsetlc() with non-zero constraint count, and later with zero constraint count, does not correctly reset constraints (some internal structures retain constraint information).
797 Interpolation feature have not tried 2018-02-28 19:03 2018-02-28 19:04 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: ~2x faster bicubic spline evaluation/differentiation Spline evaluation code was heavily optimized, now it is much more efficient.
796 Interpolation feature have not tried 2018-02-28 19:02 2018-02-28 19:02 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: serialization of 2D splines (bilinear/bicubic) Now it is possible to serialize/unserialize 2D spline.
795 Interpolation feature have not tried 2018-02-28 19:01 2018-02-28 19:01 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: fitting irregular data by penalized bicubic spline Implemented two fitting algorithms:
* BlockLLS, efficient direct sparse solver which supports nonlinearity penalty and works on grids with up to 512x512 nodes.
* FastDDM, large-scale solver intended for problems with huge amounts of points (supports grids with up to 10000x10000 nodes)

Both solvers support parallelism, although only FastDDM shows nearly-linear scaling.
793 Linear algebra feature have not tried 2018-01-30 14:30 2018-01-30 14:30 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: transpose functionality for CRS matrices sparsetransposecrs() function allows to perform transposition of the sparse matrix stored in CRS format.
643 Data analysis feature have not tried 2015-01-22 16:01 2018-01-25 18:07 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: faster k-means clusterization (blocked, parallel) New incarnation of k-means is much faster than previous version due to several improvements:
* initialization algorithm was remastered - we can choose between random initialization, k-means++ (high-quality but slower and non-parallelizable), and "fast-greedy" algorithm which gives reasonably good results while being cache-efficient and parallelizable.
* main k-means iteration is now blocked and parallelized, which significantly improves performance on large-scale datasets
400 AlgoPascal minor have not tried 2011-06-24 10:39 2018-01-03 08:47 armin SergeyB normal assigned none none 3.3.0 open 0 Compiler warning in ap.h Using gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5), I get the following warning:

ap.h:719: warning: type qualifiers ignored on function return type
ap.h:720: warning: type qualifiers ignored on function return type

These are the two lines:
const bool operator==(const alglib::complex& lhs, const alglib::complex& rhs);
const bool operator!=(const alglib::complex& lhs, const alglib::complex& rhs);

I guess the const flag can be savely removed for bool return values.
zaytsev 2018-01-03 08:47 It doesn't have anything to do with booleans per se, but rather with non-class (POD) rvalues:

https://stackoverflow.com/questions/1607188/why-is-a-type-qualifier-on-a-return-type-meaningless

But yes, the bottom line is that the const qualifier can be safely removed in this situation.
496 Solvers feature have not tried 2012-11-16 10:51 2018-01-02 17:23 SergeyB SergeyB normal assigned none none open 0 Flexible preconditioned conjugate gradient http://en.wikipedia.org/wiki/Conjugate_gradient_method#The_flexible_preconditioned_conjugate_gradient_method

Easy addition, seems to be useful.
429 Optimization feature have not tried 2012-01-17 10:41 2018-01-02 17:23 SergeyB SergeyB normal assigned none none open 0 Acceleration support for LM optimizer Return it back, now using BLEIC optimizer for inner preconditioned iterations.

This feature requires remastering of the BLEIC/CG optimizers - CG must support equality constrained preconditioning (i.e. preconditioner which respects equality constraints).
428 General feature have not tried 2012-01-16 13:00 2018-01-02 17:23 SergeyB SergeyB normal assigned none none open 0 Better support for lambda functions C++ and C# versions of ALGLIB must be able to accept lambda functions as their parameters. As for C++, we may need to accept std::function.

Doctests may need modification...
426 General feature have not tried 2012-01-16 12:57 2018-01-02 17:23 SergeyB SergeyB normal assigned none none open 0 Automatic differentiation in NLEQ solver Like it was implemented in L-BFGS and others.
417 Linear algebra feature have not tried 2011-12-12 11:12 2018-01-02 17:23 SergeyB SergeyB normal assigned none none open 0 LinLSQR - reorthogonalization Partial reorthogonalization.
RBF algorithm must use reorthogonalized version of LSQR.
519 Spec.functions major always 2013-04-23 17:24 2018-01-02 17:23 zaytsev SergeyB normal assigned none none 3.7.0 open 0 Exponential integral Ei(x) returns 0 for x <= 0 The Ei(x) in ALGLIB returns 0 for all x <= 0 . This is not documented on the website:

http://www.alglib.net/specialfunctions/exponentialintegrals.php

In the source code, however, it says "Not defined for x <= 0", which, of course, explains this behavior, but doesn't really help.

Anyways, the function Ei(x) is defined on all real axis [1] and I see no reason why you would silently return 0 for x <= 0, while many other special function packages allow one to compute Ei(x) for negative arguments.

I need fast and accurate routines to compute the exponential integrals and would highly appreciate if you could address this problem.

Thanks!

[1]: http://mathworld.wolfram.com/ExponentialIntegral.html		 I'm looking specifically at the C/C++ version of ALGLIB, but it shouldn't really matter.
SergeyB 2013-04-23 22:04 According to Wiki [2], "definition above can be used for positive values of x, but the integral has to be understood in terms of the Cauchy principal value due to the singularity of the integrand at zero. For complex values of the argument, the definition becomes ambiguous due to branch points at 0 and .[1] In general, a branch cut is taken on the negative real axis".

So values at negative axis are not well-defined because of branch cut.

[2] http://en.wikipedia.org/wiki/Exponential_integral
zaytsev 2013-04-23 23:38 Hi Sergey,

The discussion about the ambiguity that you quoted above relates to the function of the complex argument (it starts with "For complex values of the argument..."), however, we are talking about a real-valued function of a real argument here.

Just have a look at the integral, it's well-defined for x < 0, however, at 0 the divergence happens and then, in order to use the same definition for x >= 0, you have to consider it as a VP.

Right now, I've switched to Boost [3], and I'm totally delighted about the accuracy and speed (4x faster then SPECFUN, didn't measure against CEPHES, because I need negative values).

However, Boost is a couple of hundreds of megabytes of source code, which sounds a bit too much if you only want one function out of it :-)

[3]: http://www.boost.org/doc/libs/1_53_0/libs/math/doc/sf_and_dist/html/math_toolkit/special/expint/expint_i.html

Z.
526 Linear algebra minor have not tried 2013-06-23 14:35 2018-01-02 17:23 zerop SergeyB normal assigned none none open 0 inaccurate q is it my computer problem?
i use a = [1,2,3;4,5,6;7,8,9;10,11,12];

compare matlab result and F# result

some values are different
473 Data analysis feature have not tried 2012-07-15 13:28 2018-01-02 17:23 SergeyB SergeyB normal assigned none none open 0 RNN CLINK/SLINK algorithm Sources:
* http://www.cs.ucf.edu/csdept/faculty/lang/pubs/spie99.pdf
* http://en.wikipedia.org/wiki/Nearest-neighbor_chain_algorithm

Interface:
* supports CLINK, SLINK
* called with ClusterizerRunLargeScaleCLINK() or ClusterizerRunLargeScaleSLINK()
* compare performance against matrix-based O(N^2)-space algorithm
573 Spec.functions major always 2013-10-21 15:33 2018-01-02 17:22 higuerap SergeyB normal assigned none none open 0 Wrong result for Jacobian elliptic function 'Dn' Going for:

alglib::jacobianellipticfunctions(u, m, sn, cn, dn, ph);

with: u = -13.3077 and m = 0.9162 yields:

sn alglib -1
cn alglib 3.06162e-16
dn alglib 0.147015

Sn and Cn are correct, while Dn, which should be sqrt(1-m*sn*sn) is 0.147015 instead of the expected 0.289482
636 Linear algebra feature have not tried 2014-12-08 16:03 2018-01-02 17:22 SergeyB SergeyB normal assigned none none open 0 decide on nvBLAS todo - easy linking and distribution.
599 Optimization feature have not tried 2014-02-24 22:06 2018-01-02 17:22 nlagerr SergeyB normal assigned none none 3.8.1 open 0 sactivesets.sasrebuildbasis could be optimized for sparse constraint matrices We are using the Alglib QP solver and were experiencing some performance issues. I profiled the application and it turns out most time was spent in the code that reorthogonalizes the constraint matrices. The documentation already points out the algorithm does not scale well in the number of constraints.

For our particular QP, the constraint matrices are very sparse. I managed to get significant performance improvements by rewriting sasrebuildbasis to exploit this fact. I implemented a vector that:
- Inserts elements in O(1)
- Removes elements in O(1)
- Computes the dot product in O(nr-non-zeros)
This data structure does use more memory than normal matrices to achieve these characteristics.

If you are interested in implementing the above functionality and would like to have a reference implementation in C#, please contact me via email.
nlagerr 2014-06-02 17:37 I am wondering, did my feature request give rise to 0000607? I think you are right that we ought to be using BLEIC-QP instead of Cholesky-QP. As soon as I have time I will experiment with it. But I was under the impression both solvers needed to run sasrebuildbasis, am I right?
597 Linear algebra feature have not tried 2014-02-24 14:39 2018-01-02 17:22 SergeyB SergeyB normal assigned none none open 0 Root polishing for polynomial solver todo
544 Data analysis feature have not tried 2013-07-29 17:20 2018-01-02 17:22 SergeyB SergeyB normal assigned none none open 0 Statistics - correct handling of degenerate cases * RMatrixSolveLS for NRows=2 and NCols=100.000 (we need RMatrixSolveLSCompact)
* PCA for NPoints=10 and NVars=100.000
* LDA for NPoints=10 and NVars=100.000
543 Data analysis feature have not tried 2013-07-29 10:29 2018-01-02 17:22 SergeyB SergeyB normal assigned none none open 0 Statistics: ANOVA and others Targets for one of the next releases.

Models:
* ANOVA
* Generalized linear model
* Logit (mixed)
* Probit multinomial, ordered)
* Multilevel models
* Fixed effects, random effects, mixed effects
* Least squares - partial, total, generalized, weighted, iteratively reweighted, LASSO
509 Linear algebra feature have not tried 2013-04-04 14:52 2018-01-02 17:22 SergeyB SergeyB normal assigned none none open 0 Efficient TRSM and SYRK basecase code todo
482 Data analysis feature have not tried 2012-08-03 19:23 2018-01-02 17:22 SergeyB SergeyB normal assigned none none open 0 PMML output generate models in PMML format
479 General feature have not tried 2012-07-26 13:44 2018-01-02 17:22 SergeyB SergeyB normal assigned none none open 0 KD-trees - improvements High priority:
* ability to modify kd-tree (add points, remove points... we may need ability to create composite leafs in the tree)

Low priority:
* approximate search in high-dimensional space, FLANN, see attachment		 flann.pdf (380 KB) 2012-08-01 10:06
475 Data analysis feature have not tried 2012-07-16 15:41 2018-01-02 17:22 SergeyB SergeyB normal assigned none none open 0 Support vector machines todo
472 Data analysis feature have not tried 2012-07-12 19:10 2018-01-02 17:22 SergeyB SergeyB normal assigned none none open 0 Cluster analysis improvements SSE-based distance matrix calculation for L1 and L-inf norms.

Sparse input matrices

RNN algorithm and other improvements:
* http://www.daimi.au.dk/~zxr/papers/quadtreeUPGMA.pdf
* http://thames.cs.rhul.ac.uk/~fionn/old-articles/complexities/

Divisive clustering (can be easily parallelized).

Better k-means:
* smart iteration: http://www.siam.org/proceedings/datamining/2010/dm10_012_hamerlyg.pdf

Penalized k-means and penalized-weighted kmeans?
* George C. Tseng, articles

Different clustering techniques from http://en.wikipedia.org/wiki/Cluster_analysis :
* EM and Gaussian
* DBSCAN and OPTICS
* hard vs. soft
* other exotic varietes
* coefficients

Different cluster validation coefficients:
http://scikit-learn.org/stable/modules/clustering.html
http://www.cs.kent.edu/~jin/DM08/ClusterValidation.pdf
http://web.itu.edu.tr/sgunduz/courses/verimaden/paper/validity_survey.pdf
448 Statistics feature have not tried 2012-02-10 11:51 2018-01-02 17:22 SergeyB SergeyB normal assigned none none open 0 Sample covariation improvements 1. N and N-1 forms of the finite sample covariation (correlation is not needed). We can add one more parameter with default value (to have backward compatibility).

2. POSSIBLE: shrinkage estimation, see http://en.wikipedia.org/wiki/Estimation_of_covariance_matrices#Shrinkage_estimation and http://cran.r-project.org/web/packages/corpcor/index.html
442 Linear algebra feature have not tried 2012-01-29 13:54 2018-01-02 17:22 SergeyB SergeyB normal assigned none none open 0 Efficiently handle zero elements during triangular/orthogonal factorization Dense Level 2 LU/QR/... performance on sparse matrices stored in the dense form can be significantly improved.

The idea is to skip rows of rank-1 updates which have zero coefficients. It can be done with just one "if", its cost is insignificant, and it can have major performance consequences on some types of the matrices.
437 Optimization feature have not tried 2012-01-17 15:09 2018-01-02 17:22 SergeyB SergeyB normal assigned none none open 0 Feasibility solver Solver which finds feasible point subject to boundary and linear constraints:
* two forms - canonical (A*x=b) and general BLEIC constraints
* may be, solver for a problem with general (nonlinear) constraints
* scaling of the inputs and constraint matrix - we must rescale problem in order to have good convergence
* use LQ or bidiagonalization instead of SVD. Or, maybe, orthogonalization for rows of the matrix...
* use algorithm similar to one used by MinQP. Not sure that we should use MinQP internally, because MinQP may need same feasibility detection code. Maybe we should separate internal QP solver from MinQP subpackage, and call this solver from feasibility solver AND from MinQP
* efficient hot starts
424 Linear algebra feature have not tried 2012-01-16 12:37 2018-01-02 17:22 SergeyB SergeyB normal assigned none none open 0 Sparse GMRES for least squares GMRES for sparse linear least squares, like described in www.nii.ac.jp/TechReports/07-009E.pdf and www.nii.ac.jp/TechReports/04-006E.pdf
641 General feature have not tried 2014-12-17 12:50 2018-01-02 17:19 SergeyB SergeyB normal assigned none none open 0 Generation of discrete numbers with required probabilities todo hqrnd
581 Data analysis feature have not tried 2014-01-04 13:48 2018-01-02 17:19 SergeyB SergeyB normal assigned none none open 0 reorganize neural networks unit move obsolete functions to comp.ap
rework comments on commercial vs free
674 General minor have not tried 2016-06-06 11:18 2018-01-02 17:18 SergeyB SergeyB normal assigned none none open 0 t-sne todo
659 Data analysis feature have not tried 2015-10-28 15:28 2018-01-02 17:18 SergeyB SergeyB normal assigned none none open 0 linreg improvements full MKL support
645 Statistics feature have not tried 2015-02-03 13:03 2018-01-02 17:18 SergeyB SergeyB normal assigned none none open 0 Compact correlation/covariance which does not allocate a copy of input matrix.

How to test? Some branches of algorithm should work only for really large data...
489 Statistics feature have not tried 2012-10-09 16:01 2018-01-02 17:18 SergeyB SergeyB normal assigned none none open 0 Multiple correlation coefficient http://en.wikipedia.org/wiki/Multiple_correlation

improve docs on correlation
661 Linear algebra feature have not tried 2015-11-04 20:16 2018-01-02 17:17 SergeyB SergeyB normal assigned none none open 0 complex SVD and least squares solver todo.
decide on least squares part - what solvers exactly to implement?
640 Linear algebra feature have not tried 2014-12-12 13:25 2018-01-02 17:17 SergeyB SergeyB normal assigned none none open 0 MKL-powered sparse linear algebra todo
608 Linear algebra feature have not tried 2014-05-26 16:40 2018-01-02 17:17 SergeyB SergeyB normal assigned none none open 0 Sparse matrix improvements * supernodal Cholesky
* YAMD reordering
* profile reduction reordering
517 Linear algebra feature have not tried 2013-04-18 14:48 2018-01-02 17:17 SergeyB SergeyB normal assigned none none open 0 Optimize RMatrixEnforceSymmetricity need cache oblivious code
491 Linear algebra feature have not tried 2012-10-29 10:10 2018-01-02 17:17 SergeyB SergeyB normal assigned none none open 0 Marix inversion - add pre-scaling Pre-scaling is needed for better handling of badly-scaled matrices which become well-conditioned after scaling. Such matrices should not be treated as ill-conditioned.
616 Optimization feature have not tried 2014-06-25 11:23 2018-01-02 17:15 SergeyB SergeyB normal assigned none none open 0 Sparsity support for optimizers * sparse linear constraints (MinQP, MinBLEIC, MinNLC)
* sparse nonlinear constraints (MinNLC)
606 Optimization feature have not tried 2014-05-15 11:13 2018-01-02 17:15 SergeyB SergeyB normal assigned none none open 0 QQP improvements * Newton step for sparse QP with general linear constraints
* investigate feasibility of updating Cholesky decomposition for sparse QP problems. It is possible, but sometimes it is faster to perform sparse Cholesky from scratch than to update it. At least, for large batches of updates...
* permutation of variables (should be performed during initial scaling of the problem) for faster sparse Cholesky
* support for sparse Cholesky solvers other than SKS-based one
460 Optimization feature have not tried 2012-06-27 12:16 2018-01-02 17:15 SergeyB SergeyB normal assigned none none open 0 MinQP improvements * ability to handle different models of target function: dense A, dense (A'*A+diag), sparse A, sparse (A'*A+diag), diagonal A
* better feasibility detection code - damped version
* hot starts from feasible point (when we detect initial feasible point, we reuse active set left from previous optimization)
736 Linear algebra feature have not tried 2017-10-16 18:02 2017-12-29 13:02 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: new set of public level 2 BLAS functions New set of publicly available level 2 BLAS functions replacing obsolete less general ones:
* generalized real matrix-vector product implemented by rmatrixgemv()
* real rank-1 update implemented by rmatrixger()
* generalized symmetric matrix-vector product implemented by rmatrixsymv()
* symmetric vector-matrix-vector product x'*A*x functionality implemented by rmatrixsyvmv()
* triangular matrix-vector solve implemented by rmatrixtrsv()
702 General feature have not tried 2017-05-24 13:43 2017-12-25 20:36 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: debug allocation counter in C core and C++ version for POSIX systems Now allocation counter is supported for the following combination:
* Intel CPU, with AE_CPU=AE_INTEL defined
* GCC compiler
* AE_OS=AE_POSIX defined
769 General feature have not tried 2017-12-11 18:51 2017-12-25 20:36 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: ALGLIB for CPP - additional tests for correct handling of malloc errors Added tests to test_x.cpp which are activated by combination of AE_USE_ALLOC_COUNTER and AE_CPU=AE_INTEL definitions. These tests check that malloc() failure (including failures during initialization) does not crash program and does not result in memory leak.
772 Optimization feature have not tried 2017-12-12 19:27 2017-12-25 17:35 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: MinQP solver now supports automatic scaling QP solver can automatically deduce variable scales from the diagonal of the quadratic term; this feature works only for matrices with strictly positive diagonals.
751 Solvers minor have not tried 2017-11-05 16:58 2017-12-25 14:52 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: direct sparse solver using Skyline (SKS) storage format Direct sparse solver for symmetric positive definite systems stored in SKS format.
779 Linear algebra feature have not tried 2017-12-24 18:59 2017-12-25 12:27 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: sparseset() now works with SKS matrices Done.
778 Linear algebra feature have not tried 2017-12-24 18:58 2017-12-24 18:58 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: band matrix creation with sparsecreatesksband() Added sparsecreatesksband(), convenience wrapper for sparsecreatesks(), which allows to specify matrix bandwidth as single parameter (instead of D and U variable profile arrays).

Also added sparsecreatesksbandbuf() - buffered version which tried to reuse previously allocated space as much as possible.
777 General feature have not tried 2017-12-24 15:13 2017-12-24 15:13 SergeyB SergeyB normal assigned none none open 0 INCOMPATIBLE: remove "out info" parameter from all the functions we already have "out Rep" convention, no need in info parameter
764 General minor have not tried 2017-12-05 18:20 2017-12-05 18:20 SergeyB SergeyB normal resolved none none fixed 0 FIXED: potential memory leak in the native ALGLIB (C++, native core for C#) Potential memory leak in the internal smart pointer object. Under rare circumstances subsequent assignment to same pointer object may leave some memory unfreed.
763 Data analysis feature have not tried 2017-11-29 18:29 2017-11-29 18:30 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: singular spectrum analysis algorithm Implemented:
* trend extraction and prediction with SSA
* two algorithms - precise direct solver, fast real-time solver with incremental updates
762 General feature have not tried 2017-11-28 18:51 2017-11-28 18:52 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: exception-free error handling for C++ Since 3.13.0, ALGLIB for C++ supports exception-free error handling, which is sometimes used in embedded systems.

When compiled with AE_NO_EXCEPTIONS preprocessor symbol, ALGLIB uses global (thread unsafe) flag variable for error reporting. See ALGLIB Reference Manual for more information.
759 General feature have not tried 2017-11-17 19:27 2017-11-17 19:27 SergeyB SergeyB normal assigned none none open 0 numerics.complex for C# todo
758 Linear algebra feature have not tried 2017-11-12 18:46 2017-11-12 18:46 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: warm start for subspace eigensolver Subspace eigensolver now supports warm starts - if asked, it can reuse previous subspace as initial point for new eigenproblem.
757 General feature have not tried 2017-11-09 14:37 2017-11-09 14:37 SergeyB SergeyB normal assigned none none open 0 enums for constants strong C++ enums in C++11 and newever, automatically defer to untyped enums in older code
752 Data analysis feature have not tried 2017-11-07 11:29 2017-11-07 11:29 SergeyB SergeyB normal assigned none none open 0 INCOMPATIBLE: decision trees storage format trees need better storage and internal processing format
689 General minor always 2017-03-14 12:49 2017-10-24 13:19 smani SergeyB normal assigned none none 3.10.0 open 0 GCC7: Test failure on i686 The ReportCorrectnessTest of the linlsqr test fails on i686 when building with GCC7 (7.0.1 20170309) on Fedora rawhide. The failure occurs in the below section of test_c.cpp (some debug prints were added). The output of this snippet is:

ReportCorrectnessTest::Fail
1 0
TNorm=8107.5250330521303113;RNorm=8107.5250330521294018;S.R2=8107.525033052130;

-----------
/*
 * check, that RNorm is't more than S.R2
 * and difference between S.R2 and TNorm
 * is't more than 'eps'(here S.R2=||rk||,
 * calculated by the algorithm for LSQR, and
 * TNorm=||A*S.x-b||, calculated by test function).
 */
if( ae_fp_greater(s.r2,rnorm)||ae_fp_greater(ae_fabs(s.r2-tnorm, _state),eps) )
{
    if( !silent )
    {
        printf("ReportCorrectnessTest::Fail\n");
        printf("%d %d\n", ae_fp_greater(s.r2,rnorm), ae_fp_greater(ae_fabs(s.r2-tnorm, _state),eps));
        printf("TNorm=%0.16f;RNorm=%0.16f;S.R2=%0.12f;\n",
            (double)(tnorm),
            (double)(rnorm),
            (double)(s.r2));
    }
    result = ae_true;
    ae_frame_leave(_state);
    return result;
}
-----------

By the way, the comment is also a bit confusing. If "is't" actually means "isn't", then the comment states that the test should verify:
rNorm < s.r2 <=> s.r2 > rNorm
|s.r2 - tNorm| < eps

but as far as I see the code fails if:
a) ae_fp_greater(s.r2,rnorm) <=> s.r2 > rNorm
b) ae_fp_greater(ae_fabs(s.r2-tnorm, _state),eps) <=> |s.r2 - tnorm| > eps

So a) matches the comment, but b) does not (and if "is't" means "is", then the opposite is true).


The build is using the following flags:
-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m32 -march=i686 -fasynchronous-unwind-tables -fPIC

Build log is here:
https://koji.fedoraproject.org/koji/getfile?taskID=18371905&name=build.log

Other arches build correctly:
https://koji.fedoraproject.org/koji/taskinfo?taskID=18371900
734 Optimization feature have not tried 2017-10-10 15:46 2017-10-10 15:46 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: orders of magnitude faster QP-DENSE-AUL solver New working set expansion/eviction strategy greatly accelerates handling of QP problem with large (more than N) number of inequality constraints. Acceleration ranges from few times to tens of times.
732 General feature have not tried 2017-10-09 17:49 2017-10-09 17:49 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: up to 4x faster BLEIC and QP-BLEIC solvers Significantly improved handling of large sets of linear inequality constraints in BLEIC and QP-BLEIC solvers. Nonlinear and QP problems with large sets of linear inequality constraints can be solved up to 4x faster.
723 General feature have not tried 2017-09-13 13:41 2017-09-30 17:16 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: attach_to_ptr() for C++ vectors/matrices Changed: ALGLIB for C++

attach_to_ptr() function allows to attach double precision vector/matrix to already allocated array, without copying data into internal storage.
722 General feature have not tried 2017-09-12 13:38 2017-09-12 13:38 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: ALGLIB for Delphi/FreePascal - Linux support New version of ALGLIB for Delphi/FreePascal is now shipped with x64 Linux binaries!
721 General feature have not tried 2017-09-12 13:32 2017-09-12 13:32 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: ALGLIB for C# - Linux HPC binaries Native edition of commercial ALGLIB for C# now supports both Windows and Linux platforms (C# wrapper around HPC core correctly detects OS it is running on and loads appropriate shared library).

NOTE: NET edition is still portable anywhere NET is present.
717 Linear algebra tweak have not tried 2017-08-10 16:46 2017-08-16 12:43 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: deterministic seed for internaldstein() inverse iteration initializer Now algorithm is completely deterministic
714 Optimization minor have not tried 2017-08-09 17:06 2017-08-09 17:07 SergeyB SergeyB normal resolved none none fixed 0 FIXED: minor error in QQP solver - incorrect automatic scaling Fixed.
713 Interpolation feature have not tried 2017-08-09 17:02 2017-08-09 17:03 SergeyB SergeyB normal resolved none none implemented 0 TWEAK: rbfsetpoints() - added check for NAN/INF in the inputs Implemented integrity check
706 Data analysis feature have not tried 2017-06-08 16:14 2017-06-08 16:14 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: deterministic seed for k-means clustering algorithm clusterizersetseed() function allows to control behavior of k-means algorithm - to switch between deterministic and random initialization.
690 Optimization feature have not tried 2017-04-12 14:07 2017-05-11 21:53 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: linear constraints for MinLM/LSFit optimizers MinLM/LSFit optimizers now support optimization subject to linear equality/inequality constraints.
700 Optimization tweak have not tried 2017-05-03 15:20 2017-05-03 15:20 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: additional regularization of MinNLC optimizer Additional regularization of MinNLC optimizer helps it to be more robust on ill-conditioned constrained optimization problems.
678 General minor have not tried 2016-09-01 19:52 2017-04-29 16:20 SergeyB SergeyB normal resolved none none fixed 0 FIXED: ae_matrix_attach_to_x() ignores stride in X-object Code which attaches ae_matrix to X-object is unable to handle stride!=columns. We did not change it, just placed appropriate assertion. Current version of ALGLIB does not need such functionality anyway.
692 Interpolation feature have not tried 2017-04-18 16:58 2017-04-18 16:59 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: fitting minimum circumscribed, minimum zone, maximum inscribed circles to N-dimensional data Implemented fitting minimum circumscribed (MCC), minimum zone (MZC), maximum inscribed (MIC) circles/spheres to N-dimensional data.

Two algorithms are provided:
* robust (although less efficient) NLC one, based on ALGLIB nonlinearly constrained solver. This algorithm shows good convergence properties.
* fast inexact SLP (sequential linear programming) one, based on ALGLIB linearly constrained solver.

NOTE: SLP approach to MCC/MZC/MIC is recommended by many authors; however, empirical testing demonstrated that linearization of the problem often breaks down near true solution (validated in MATLAB; it is deficiency of linearization, not solver used for such model). Thus, SLP solver often fails to converge to more than 3-5 digits of precision. However, it is 10-20 times faster than NLC, and on some problems it works good enough. That's why we included it as non-standard option.
691 Optimization feature have not tried 2017-04-14 20:38 2017-04-14 20:44 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: "exact robust" preconditioner mode for MinNLC optimizer New preconditioner which behaves robustly independently of constraint count K (whether it is zero, less than N, more than N). The only drawback is that it always requires at least O(N^3) time to build, even for K<<N.

New preconditioner is now default one (pre-3.11 ALGLIB used inexact preconditioner as default option).

For comparison, "exact low rank" preconditioner behaves well on problems with constraint number K less than problem N, but fails for K>=N, although for small K's it requires just O(N*K^2) time to be built.
683 Optimization feature have not tried 2017-02-01 17:17 2017-02-16 13:19 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: box-constrained optimizer Specialized optimizer for box-only constraints. It allows activation of multiple constraints per step, so it can be several times faster than BLEIC on large-scale problems.
688 General feature have not tried 2017-02-14 23:15 2017-02-14 23:16 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: streaming serialization/unserialization in C# ALGLIB for C# (uncluding version with native computational core) can serialize/unserialize directly to/from System.IO.Stream (in addition to string serialization)
687 General feature have not tried 2017-02-13 13:36 2017-02-14 23:15 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: streaming serialization/unserialization in C++ Now ALGLIB for C++ can serialize/unserialize to/from std::iostream (in addition to std::string serialization)
664 General feature have not tried 2016-02-04 14:48 2017-02-13 13:34 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: IronPython and VB.Net wrappers are now included into default distribution of ALGLIB for C# Restructured archives, no functionality changes
669 Optimization feature have not tried 2016-04-06 12:08 2017-02-09 15:41 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: integrity check for NAN/INF for lsfit and minlm solvers Now minlm optimizer can detect incorrect values (infinities or NANs) in user-supplied function vector and/or Jacobian. In such cases optimization is terminated with dedicated error code -8.
682 Linear algebra feature have not tried 2017-01-24 14:53 2017-02-08 20:09 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: subspace iteration direct/sparse/out-of-core symmetric/nonsymmetric eigensolver Subspace iteration for top K eigenvalues/vectors for dense/sparse/out-of-core problems. For nonsymmetric eigenproblems invariant subspace is returned.
681 Linear algebra feature have not tried 2017-01-24 14:52 2017-02-08 18:26 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: truncated PCA Fast principal component analysis with just a few top principal vectors (as opposed to full N-dimensional basis).
676 Interpolation minor have not tried 2016-06-20 13:16 2017-02-02 15:04 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: hierarchical RBF New scattered interpolation algorithm - hierarchical RBFs - with following features:
* ability to handle arbitrary number of dimensions, starting from NX=1
* ability to work with vector-valued functions
* order of magnitude faster than previous version, multilayer RBFs
* well-suited for large-scale problems because of higher speed and smaller memory footprint during model construction (and, less important, evaluation)
670 Spec.functions minor have not tried 2016-04-07 13:22 2016-12-23 13:41 SergeyB SergeyB normal resolved none none fixed 0 FIXED: some statistical functions are prone to integer overflow Running statistical test (t-test, MWU, WSR) with both samples larger than 32768 resulted in 32-bit overflow. Reproducible in ALGLIB for C#, in 32-bit ALGLIB for C++.

Initial report by Jury Gasnikov mentions unequalvariancettest, but several other functions were affected too.
677 General trivial have not tried 2016-08-18 14:11 2016-12-21 16:04 SergeyB SergeyB normal resolved none none fixed 0 FIXED: minor typo in C# docs C# manual mentioned ae_int_t datatype (used by C++ edition of ALGLIB).

Fixed.
665 Optimization feature have not tried 2016-02-08 12:15 2016-12-16 16:37 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: new QP solver for linearly constrained QP problems Augmented Lagrangian QP solver for equality/inequality constrained QP problems.
672 General feature have not tried 2016-05-14 16:22 2016-05-14 16:22 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: kd-trees now support box queries New functions allow to perform query for all points in the user-specified box.
671 Interpolation feature have not tried 2016-05-11 17:00 2016-05-11 17:01 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: parallel RBF evaluation on regular 3D grids New parallel algorithm.
668 General feature have not tried 2016-03-23 16:34 2016-03-23 16:34 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: thread-safe version of kd-tree object New version of kd-tree search structure allows to use external buffers for temporaries, which helps to use tree structure in multithreaded programs.
658 Optimization minor have not tried 2015-10-12 13:40 2015-10-12 13:40 SergeyB SergeyB normal resolved none none fixed 0 FIXED: BLEIC optimizer hangs in rare cases due to roundoff errors Fixed.
656 Optimization minor have not tried 2015-07-31 16:55 2015-07-31 17:02 SergeyB SergeyB normal resolved none none fixed 0 FIXED: minor bug in the handling of inconsistent linear constraints (BLEIC, QP optimizer) Under rare conditions, findfeasiblepoint() functions used by many optimizers incorrectly handled inconsistent linear constraints. Instead of returning -3 error code it was caught in the infinite loop.
621 Statistics minor always 2014-09-23 18:41 2015-07-24 17:09 pflaquerre SergeyB normal resolved none none 3.8.1 fixed 0 FIXED: mann-whitney u-test can return probabilities greater than 1 I noticed that alglib::mannwhitneyutest can sometimes return p values greater than 1. For example, the following program:

#include "alglib/statistics.h"
#include <iostream>

int main(int argc, char *argv[])
{
  const alglib::real_1d_array x("[4.46, 1.58, 3.1300001, 0.64999998, 2.0899999, 5.8899999, 1.17, 1.49, 1.77, 0.62]");
  const alglib::real_1d_array y("[4.46, 1.58, 3.1300001, 0.64999998, 2.0899999, 5.8899999, 1.17, 1.49, 1.77, 0.62]");
  const size_t nx = x.length();
  const size_t ny = y.length();
  
  double bt(0), lt(0), rt(0);
  alglib::mannwhitneyutest(x, nx, y, ny,
                           bt, lt, rt);

  std::cout << "mann whitney bt, lt, rt " << bt << " " << lt << " " << rt << std::endl;

  return 0;
}

outputs "mann whitney bt, lt, rt 1.02949 0.51483 0.514744", whereas one would expect bt to be 1.
546 Statistics tweak have not tried 2013-08-19 10:23 2015-07-24 16:37 SergeyB SergeyB normal resolved none none fixed 0 FIXED: corrected documentation for Jarque-Bera test Fixed minor error in documentation (mentions non-existent parameters of the function).
654 Data analysis minor have not tried 2015-06-02 12:50 2015-07-24 16:27 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: better memory reuse in clustering functions Some users reported that multiple calls to clustering functions caused memory fragmentation under heavy loads. Improved version of AHC and k-means++ reuses previously allocated memory, which results in better behavior in native and NET environments.
652 General feature have not tried 2015-05-15 16:31 2015-07-24 12:31 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: reading CSV files Support for reading CSV files into 2D real array under C++ and C#.
650 General minor have not tried 2015-04-09 15:23 2015-07-21 16:03 SergeyB SergeyB normal resolved none none fixed 0 FIXED: two race conditions in initializer of HPC ALGLIB for C# Both race conditions were noticed rarely and only under very heavy load, when ALGLIB functions were called from multiple simultaneously started threads.

First race condition appeared during LoadALGLIBFunctions() call - globally accessible hAlglibDL variable was set before initialization of external native DLL was done.

Second race condition was found in addsearchpath() and resulted in out-of-bounds exception during initialization.
615 Optimization feature have not tried 2014-06-14 18:52 2015-07-21 15:11 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: Adaptive Gradient Sampling algorithm for nonsmooth nonconvex constrained optimization State-of-the-art optimizer which may handle nonsmooth nonconvex optimization problems with arbitrary mix of box, linear and nonlinear (nonsmooth) constraints.
511 Fast transforms minor have not tried 2013-04-04 15:33 2015-07-20 17:46 SergeyB SergeyB normal assigned none none open 0 Improved FFT, FHT, convolution and correlation * parallelization support
* faster Rader, perform several Rader transforms simultaneously
* 2D transforms
576 General minor have not tried 2013-11-05 17:28 2015-07-20 13:44 SergeyB SergeyB normal assigned none none open 0 Signed NET assemblies todo
655 Data analysis minor have not tried 2015-06-11 16:59 2015-06-11 17:00 SergeyB SergeyB normal resolved none none fixed 0 FIXED: lrbuild() crashes when called from 64-bit HPC ALGLIB for C# The reason for this bug is that lrreport.cvdefects array is filled by trash which does not fit into 32-bit integer type used by C#. Fixed bug by explicitly filling array by -1's.
651 Data analysis feature have not tried 2015-05-12 11:49 2015-05-12 11:49 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: Ward's method for hierarchical clustering Wards's method for agglomerative hierarchical clustering. Implemented as part of 'clustering' subpackage of 'DataAnalysis' package.
647 Solvers feature have not tried 2015-03-16 16:14 2015-03-18 18:13 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: faster linear solvers For each kind of linear solver present in ALGLIB (real/SPD/complex/HPD, factorized/unfactorized matrix, one right part or multiple right parts) a new, lightweight version was created, which does not perform condition number estimation and iterative refinement. For some problems it resulted in orders of magnitude improvement in speed.
578 Interpolation feature have not tried 2013-11-11 11:27 2014-12-12 12:54 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: parametric/nonparametric RDP algorithm Parametric/nonparametric curve approximation with Ramer-Douglas-Peucker algorithm. Two types of curves are supported (nonparametric 2-dimensional Y(x) and parametric D-dimensional X(t)) and two modes (approximation with desired precision - or approximation by required number of piecewise linear sections).
638 Linear algebra feature have not tried 2014-12-09 17:26 2014-12-10 15:45 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: improved row extraction for CRS/SKS sparse matrices Two changes:
* sparsegetrow() now supports both CRS and SKS matrices
* new function introduced - sparsegetcompressedrow() - which extracts row from CRS/SKS matrix in compressed format (only symbolically nonzero elements are returned)
598 General feature have not tried 2014-02-24 15:11 2014-12-09 17:55 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: version information for NET assemblies and Windows DLL's Versioning information is included into:
* ALGLIB source code
* NET assemblies
* Windows DLL with native core for C# version
* Windows DLL with MKL extensions for C++ version
635 Solvers feature have not tried 2014-12-04 13:09 2014-12-04 13:10 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: polynomial solver EVD-based algorithm without root polishing.
633 General major always 2014-11-25 12:48 2014-12-04 12:34 PeterZajac SergeyB normal resolved none none 3.8.1 fixed 0 FIXED: wrong 32/64 bit integer typedefs for MSVC compiler The integer typedefs for the Microsoft Visual C++ compiler located in lines 159 and 174 of "ap.h" are using deprecated type names. The typedefs use the "_int32" and "_int64" types (prefixed with a single underscore), which are only defined when compiling with Microsoft C++ Language extensions enabled (which is the default). Disabling these extensions explicitly by supplying the "/Za" compiler option leads to the following errors:

ap.h(159): error C2146: syntax error : missing ';' before identifier 'ae_int32_t'
ap.h(159): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
ap.h(174): error C2146: syntax error : missing ';' before identifier 'ae_int64_t'
ap.h(174): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int


This problem can be resolved easily by replacing the (single-underscore) "_int32"/"_int64" type names by the (double-underscore) "__int32"/"__int64" types, resp., see:
http://msdn.microsoft.com/en-us/library/29dh1w7z.aspx
634 Interpolation minor have not tried 2014-12-01 16:54 2014-12-04 12:04 SergeyB SergeyB normal resolved none none fixed 0 FIXED: instability in PolynomialBar2Pow() function Function accumulated rounding errors when magnitude of abscissas was too large or too small (larger or smaller than S - scale parameter). The problem was in the inadequate placement of sampling points - too spread or too close. Now algorithm places sampling points according to spread of nodes of barycentric model, which results in more stable algorithm.
624 Linear algebra minor have not tried 2014-10-17 13:43 2014-11-24 16:46 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: cmatrixsyrk() was renamed to cmatrixherk() CHANGE: cmatrixsyrk() function was renamed to cmatrixherk().

RATIONALE: its old name was incorrect - it is HErmitian rank-k update, not SYmmetric.

NOTES: this change does not break backward compatibility. We left old version here.
580 Linear algebra feature have not tried 2013-12-09 16:47 2014-11-24 12:24 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: MKL-powered EVD All EVD routines are MKL-powered now.

Following types of EVD are accelerated by MKL:
* tridiagonal EVD (all values/vectors)
* symmetric EVD (all values/vectors)
* nonsymmetric EVD (all values/vectors)
632 Data analysis feature have not tried 2014-11-21 17:38 2014-11-21 17:41 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: better LDA algorithm Linear algebra part of LDA algorithm (Linear Discriminant Analysis) was significantly improved: 2x higher performance in open source edition, even better speed-up in commercial ALGLIB.
617 Optimization feature have not tried 2014-07-15 08:28 2014-10-15 14:53 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: improved BLEIC algorithm Boundary and linearly constrained optimizer (BLEIC) was significantly improved - now it better handles noisy constrained problems - new strategy for activation of constraints is used. Performance on noise-free problems was slightly improved.
545 Optimization feature have not tried 2013-08-12 19:23 2014-10-09 17:20 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: interface to stop running nonlinear optimizer Previously it was impossible to stop running optimizer from its internals. Say, after you started LBFGS optimizer, you have no other choice except for waiting for one of the stopping criteria being met.

Now, CG, LBFGS, BLEIC and LM optimizers allow user to terminate optimization process from user-specified callback function. Family of functions called like MinLBFGSRequestTermination() tells optimizer to smoothly stop at the current point and return control back to user.
589 Optimization feature have not tried 2014-02-02 11:37 2014-10-08 12:14 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: all optimizers can detect NAN/INF in the target function Previously, ALGLIB optimizers were unable to react to situation when, because of numerical overflow or programming error, target function value becomes NAN or INF. They usually started to loop forever.

Now almost all nonlinear optimizers (CG, LBFGS, NLC, BLEIC) monitor iterations and terminate them with error code -8 as soon as NAN or +-infinity is detected in target function value or gradient.
609 General tweak have not tried 2014-05-26 17:56 2014-10-01 18:15 SergeyB SergeyB normal resolved none none fixed 0 IMPLEMENTED: memory barrier in ae_cpuid() Memory barrier was added in ae_cpuid() function immediately before "_ae_cpuid_initialized = ae_true" to make sure that no thread will see partially initialized state as completely initialized one.
561 Optimization feature have not tried 2013-09-13 11:52 2014-10-01 14:07 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: nonlinearly constrained optimizer (Augmented Lagrangian) Implemented constrained optimizer which supports arbitrary combination of (a) boundary, (b) linear equality/inequality, (c) nonlinear constraints. Optimizer is based on preconditioned Augmented Lagrangian approach and includes two different preconditioners: (a) inexact fast one (assumes approximate orthogonality of non-boundary constraints), and (b) exact low rank one (assumes moderate number of non-boundary constraints).
525 Data analysis feature have not tried 2013-06-20 12:24 2014-06-03 16:31 SergeyB SergeyB normal assigned none none open 0 Neural network improvements Neural structure:
* layered
* complex interactions between layers
* several activation functions: tanh(), tanh()+linear, fast sigmoid

SGD:
* algorithm without learning rates: http://yann.lecun.com/exdb/publis/pdf/schaul-icml-13.pdf
* page 72 of http://learning.stat.purdue.edu/mlss/_media/mlss/bottou.pdf - important trick
* important info on BP acceleration http://yann.lecun.com/exdb/publis/pdf/lecun-98b.pdf
* ADADELTA seems to be best method - http://www.matthewzeiler.com/pubs/googleTR2012/googleTR2012.pdf
* ADAGRAD???
* "On the importance of initialization and momentum in deep learning", http://jmlr.org/proceedings/papers/v28/sutskever13.pdf

Improvements:
* shortcut layer, see "Deep Learning Made Easier by Linear Transformations in Perceptrons". Maybe - pre-training linear layer separately.

Decide on:
* minibatch training
* bagging for ensembles

* parallel errors for ensembles
* sparse errors for ensembles
* subset errors for ensembles


* decay in ensemble training?
* investigate ensemble tendency to overfit on GLASS dataset

* mini-batch LBFGS training
* approximate Hessian preconditioning
* FindBestDecay
* FindBestNetwork

* "ensemble selection", better way of constructing ensemble

* model compression
* sparse autoencoders?
* stacked autoencoders/autodecoders?

Convolutional Neural Networks (weight sharing = constraints and projections):
* http://yann.lecun.com/exdb/publis/pdf/lecun-98.pdf
582 General tweak have not tried 2014-01-06 13:35 2014-05-28 17:06 SergeyB SergeyB normal resolved none none fixed 0 FIXED: C# version of ae_dispose_task() now clears all pointers owned by the object It is required in order to allow garbage collection of objects passed as parameters to spawned functions (having reference to object in the disposed task will force this object to persist in memory approximately forever).
594 General feature have not tried 2014-02-07 12:41 2014-05-28 17:06 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: additional integrity test for copying of objects This X-test checks that ALGLIB objects can be correctly copied with copy/assignment constructor (C++) and make_copy() function (C#).
602 General tweak have not tried 2014-03-19 13:10 2014-05-28 16:41 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: explicit cast from ae_int_t to double in C core (required to avoid warnings) Reported at http://forum.alglib.net/viewtopic.php?f=2&t=1587
611 General feature have not tried 2014-05-27 17:43 2014-05-27 17:43 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: deep copy constructor for C# objects C# objects created by ALGLIB can now be copied with make_copy() function which works both in managed and HPC editions of ALGLIB for C#.
591 Data analysis major have not tried 2014-02-06 12:50 2014-05-27 11:04 SergeyB SergeyB normal resolved none none fixed 0 FIXED: bug in the shared pool copying code ALGLIB for C++ crashes when object containing shared pool as one of its fields is copied with copy constructor. The reason is unneeded assertion in ae_shared_pool_init_copy() function.
586 Linear algebra feature have not tried 2014-01-17 17:14 2014-05-26 16:39 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: Skyline Cholesky factorizer Cholesky factorization function which works with sparse matrices in Skyline (SKS) format. This solver is most efficient for low-profile matrices.
607 Optimization tweak have not tried 2014-05-23 14:35 2014-05-23 14:35 SergeyB SergeyB normal resolved none none implemented 0 INCOMPATIBLE: default solver used by MinQP subpackage was changed. THIS CHANGE MAY BREAK BACKWARD COMPATIBILITY! IF YOU USE MINQP SUBPACKAGE, CAREFULLY READ TEXT BELOW BEFORE APPLYING UPDATE.

DESCRIPTION:
* changed behavior of MinQP object when no QP solver was chosen by user.

BEFORE:
* Cholesky-QP algorithm was used.

AFTER:
* QuickQP algorithm with default stopping criteria is used, if no general linear constraints is present (only boundary ones). BLEIC-QP algorithm with default stopping criteria is used, if general linear constraints are present.

RATIONALE:
* Cholesky-QP algorithm was excessively slow on large-scale tasks. Some users were unaware of presence of better solver.

INFLUENCE:
* affects only users who did not set QP solver explicitly.
* because default stopping criteria are used, these stopping criteria may be too relaxed for some problems. Thus, default algorithm may solve some problems with less-than-desired precision.
* if you update to new version, you may (a) explicitly choose Cholesky-QP solver with MinQPSetAlgoCholesky() call, or (b) set on of better solvers (QQP, BLEIC-QP) and make sure that algorithm converges with desired precision.
566 Optimization feature have not tried 2013-09-30 15:24 2014-05-23 14:13 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: new constrained QP solver - QuickQP MinQP subpackage now includes new bound constrained QP solver, which is several times faster on constrained problems than BLEIC-QP (and orders of magnitude faster than deprecated Cholesky-QP). The only drawback of new solver is that it supports only boundary constraints.

QuickQP solver can solve both dense and sparse QP problems.
595 Interpolation feature have not tried 2014-02-24 11:29 2014-05-14 11:46 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: 4-parameter and 5-parameter logistic fitting Five fitting functions:
* LogisticFit4 - implements 4PL (four parameter logistic) fitting
* LogisticFit5 - implements 5PL (five parameter logistic) fitting
* LogisticFit4EC - implements 4PL (four parameter logistic) fitting with equality constraints
* LogisticFit5EC - implements 5PL (five parameter logistic) fitting with equality constraints
* LogisticFit45X - "expert" functions which implements 4PL and 5PL (depending on control flags) and allows better control of fitting process.

Two function to calculate 4PL/5PL model:
* LogisticCalc4 - to calculate 4PL model with correct handling of degenerate cases
* LogisticCalc5 - to calculate 5PL model with correct handling of degenerate cases
588 Linear algebra feature have not tried 2014-01-24 17:26 2014-02-12 14:48 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: additional sparse matrix functions New sparse linear algebra functions:
* SparseTRMV - triangular matrix-vector multiply (CRS and SKS formats)
* SparseTRSV - triangular solver (CRS and SKS formats)
* SparseVSMV - triangular symmetric x'*A*x product (CRS and SKS formats)
* SparseTransposeSKS - in-place transposition of SKS matrix

New informational functions:
* SparseGetUpperCount() - counts symbolically non-zero strictly upper triangular elements in the matrix
* SparseGetLowerCount() - same for lower triangle

Improved functions:
* all sparse matrix-vector and matrix-matrix functions now support SKS format
590 Linear algebra feature have not tried 2014-02-05 11:35 2014-02-05 11:36 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: Cholesky updates (rank-1, fixing variables) Several functions which calculate:
* Cholesky decomposition of A_mod = A + u*u', for some vector u, given Cholesky decomposition of original A
* Cholesky decomposition of A_mod, obtained from A by "fixing" some variables (rows/columns are set to zeros, diagonal entries are set to 1.0), given Cholesky decomposition of original A
585 Linear algebra feature have not tried 2014-01-16 10:27 2014-01-16 11:11 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: additional sparse conversion functions Following SKS-capable functions were implemented:
* SparseCopyToSKS()
* SparseConvertToSKS()

Additionally, there were implemented several "buffered" versions of sparse functions, i.e. functions which reuse storage already allocated in target:
* SparseCopyToSKSBuf()
* SparseCopyToCRSBuf()
* SparseCopyToHashBuf()

Finally, a special function to swap two sparse matrices was implemented: SparseSwap.
584 Linear algebra feature have not tried 2014-01-10 16:10 2014-01-16 10:25 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: skyline (SKS) sparse matrix storage format Sparse matrices support skyline storage format.
574 General feature have not tried 2013-10-25 16:26 2013-10-25 16:31 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: C# interface to high performance native core Now ALGLIB users can access high-performance native computational core (ALGLIB+MKL) from C# - same interface and functionality as in 100% managed ALGLIB, but different performance.

Limitations:
* for now, can be used only under Windows
* distributed in binary-only form
575 General feature have not tried 2013-10-25 16:29 2013-10-25 16:31 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: prototype of ALGLIB-MKL interface Native computational core (C++ version, or C# version with access to native core) now can use Intel MKL to accelerate linear algebra. In this first prototype only several matrix-related functions were accelerated.

Limitations:
* ALGLIB+MKL combination is distributed in binary-only form
* for now, ALGLIB+MKL combination can be used only under Windows
569 General minor have not tried 2013-10-08 11:32 2013-10-24 13:20 SergeyB SergeyB normal resolved none none fixed 0 FIXED: non-ANSI characters in comments Two ALGLIB functions had non-ANSI characters in their comments:
* pcabuildbasis
* ellipticintegralkhighprecision

Non-ANSI characters were accidentally introduced due to typing error. Not exactly bug, but it could be source of inconvenience.
563 General tweak have not tried 2013-09-13 21:45 2013-10-24 13:16 SergeyB SergeyB normal resolved none none fixed 0 FIXED: incompatibility with recent versions of BCB C++ include file names (list.h/vector.h) were incompatible with recent version of CodeGears development tools. Now ALGLIB should be able to detect newever version of CodeGears environment and use appropriate names for include files.

This fix was suggested by Marenkov Dmitry.
570 Optimization minor always 2013-10-09 12:52 2013-10-14 15:30 SergeyB SergeyB normal resolved none none fixed 0 FIXED: MinBLEIC optimizer hangs MinBLEIC optimizer hangs on problems where very small steps (less than 1E-50) are required to activate constraints.
512 Data analysis feature have not tried 2013-04-07 11:32 2013-10-05 15:04 SergeyB SergeyB normal assigned none none open 0 Parallel clustering k-means * parallel k-means (first phase - centers) - http://theory.stanford.edu/~sergei/papers/vldb12-kmpar.pdf
* parallel k-means (second phase)

NOTE:
* prior art for subsample k-means:
** Paul S. Bradley and Usama M. Fayyad. Refining initial
points for k-means clustering. In Jude W. Shavlik,
editor, ICML, pages 91–99. Morgan Kaufmann, 1998.
** Fredrik Farnstrom, James Lewis, and Charles
Elkan. Scalability for clustering algorithms revisited.
SIGKDD Explorations, 2(1):51–57, 2000.
547 General minor have not tried 2013-08-19 10:32 2013-09-12 17:09 SergeyB SergeyB normal resolved none none fixed 0 FIXED: name conflict with some math libraries Internal function Log2 conflicts with some C++ standard math libraries. Renamed to LogBase2.
503 Optimization tweak have not tried 2013-02-10 11:44 2013-07-24 12:22 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: uphill steps in BLEIC optimizer In constrained optimization setting sometimes we have to perform uphill steps. It can be necessary when constraints force us to perform very small step, with function change being dominated by numerical noise.

Most optimizers will detect this step as uphill and stop optimization process. Improved BLEIC can handle such situation by allowing small number of short uphill steps, which greatly improves robustness of the optimizer.
539 Optimization feature have not tried 2013-07-22 19:12 2013-07-22 19:12 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: new QP solver (QP-BLEIC) New quadratic programming solver which:
* supports sparse matrices and can efficiently solve sparse QP problems
* can find local minimums of non-convex QP problems (if they have solution subject to constraints)
534 Solvers text have not tried 2013-07-08 14:09 2013-07-15 13:37 SergeyB SergeyB normal resolved none none fixed 0 FIXED: minor fix in documentation on RMatrixSolveLS Minor fix in the documentation (it omits behavior of the solver on degenerate matrices).
528 General minor have not tried 2013-06-24 15:48 2013-07-15 13:30 SergeyB SergeyB normal resolved none none fixed 0 FIXED: memory leak in shared pool and smart pointer (C++) Shared pool and smart pointer (internal ALGLIB objects) may leak memory under some circumstances. This issue was fixed in 3.8.0.

Reported by ilectra http://forum.alglib.net/viewtopic.php?f=2&t=871
478 General feature have not tried 2012-07-26 13:43 2013-07-15 11:03 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: refactoring of high-quality random numbers generator HQRND unit was refactored:
* added support for negative seeds
* added support for integers larger than 2^32
538 Fast transforms feature have not tried 2013-07-11 11:33 2013-07-11 11:34 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: improved FFT Performance of FFT was significantly improved:
* added radix-6 transforms
* medium-sized FFT's (len<=1024) are performed without transpositions
537 Data analysis feature have not tried 2013-07-10 11:19 2013-07-10 11:20 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: parallel training of neural ensembles Neural network ensembles can be trained in SMP mode. This feature is implemented in commercial version of ALGLIB.
536 Data analysis feature have not tried 2013-07-10 11:15 2013-07-10 11:16 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: parallel cross-validation for neural networks Commercial version of ALGLIB features parallel cross-validation for neural networks.
535 Data analysis feature have not tried 2013-07-10 11:14 2013-07-10 11:15 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: parallel training of neural network Commercial version of ALGLIB features following improvements in the training of neural networks:
* parallel calculation of gradient
* parallel multiple restarts from random positions
531 Data analysis feature have not tried 2013-06-27 14:23 2013-06-27 14:23 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: better weight initialization for MLP's Better weight initialization algorithm which prevents stagnation in early phases of the training.
524 Interpolation minor have not tried 2013-05-28 11:57 2013-06-07 12:06 SergeyB SergeyB normal resolved none none fixed 0 FIZED: possible division by zero in estimator for errors-in-parameters When Total Sum-of-Squares (TSS) is zero, it is impossible to calculate R2 (coefficient of determination). Past versions of ALGLIB returned NAN in this case (internally they performed division 0/0).

After the fix, ALGLIB monitors TSS and returns R2=1.0 when TSS is zero.
523 Interpolation feature have not tried 2013-05-23 10:00 2013-06-07 11:47 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: Akima spline for N<5 Spline1DBuildAkima() function now supports N=2, 3, 4 (previously it supported only N>=5).
505 General minor always 2013-02-20 16:07 2013-05-21 17:16 SergeyB SergeyB normal resolved none none fixed 0 FIXED: minor bug in the implementation of proxy array/matrix object Fixed minor bug which appeared when user of the ALGLIB optimizers tried to reallocate array which was passed to the callback function. Optimizer was unaware of the new array which was allocated and was unable to track changes in the function gradient (and thus - to converge).

This bug was marked as minor, because a) it is very rare use case, b) it is impossible to overlook (optimizer does not work at all 100% of the time).

==================

Reported by Eden (http://forum.alglib.net/viewtopic.php?f=2&t=772)

0. This bug appears in ALL C++ code which works with proxy array/matrix objects which are passed to user-defined callbacks

1. grad parameter is a reference to proxy real_1d_array object. This proxy object redirects all accesses into the memory utilized by ALGLIB computational core. The only thing you should can do is to access individual elements of the array - both reads and writes are possible. However, any attempt to resize it (setlength) should result in exception, because proxy guards itself against anything which may change internal data structures.

2. however, there is a loophole in the implementation of the real_1d_array class (and other similar classes). When you assign string constant to proxy object, it effectively breaks connection between proxy and internal memory used by ALGLIB. New independent array is created, and all operations with grad are performed with this new array. Optimizer is unaware of changes and its copy of grad is full of trash data.


=============================================
void function_cx_1_func(const alglib::real_1d_array &c, const alglib::real_1d_array &x, double &func, void *ptr){
   func = c[0]+c[1]*x[0];
}

void function_cx_1_grad(const alglib::real_1d_array &c, const alglib::real_1d_array &x, double &func, alglib::real_1d_array &grad, void *ptr) {
   func = c[0]+c[1]*x[0];
   grad = "[0, 0]";
   grad[0] = 1;
   grad[1] = x[0];
}


int main(int argc, char **argv){

   alglib::real_2d_array x = "[[1], [2], [3], [4], [5]]";
   alglib::real_1d_array y = "[10, 15, 20, 25, 30]";
   alglib::real_1d_array c = "[1, 1]";;

   double epsf = 0;
    double epsx = 0.00001;
    alglib::ae_int_t maxits = 0;
    alglib::ae_int_t info;
    alglib::lsfitstate state;
    alglib::lsfitreport rep;

    //
    // Fitting without weights
    //
    alglib::lsfitcreatefg(x, y, c, true, state);
    alglib::lsfitsetcond(state, epsf, epsx, maxits);
    alglib::lsfitfit(state, function_cx_1_func, function_cx_1_grad);
    alglib::lsfitresults(state, info, c, rep);
   std::cout<<(int)info<<" "<<rep.rmserror<<" "<<c.tostring(1)<<std::endl;

   return 0;
}
522 Data analysis feature have not tried 2013-05-14 15:46 2013-05-14 15:46 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: parallel hierarchical clustering Fully parallelized calculation of the distance matrix. However, algorithm itself is inherently sequential (almost impossible to parallelize).
518 Statistics tweak have not tried 2013-04-19 16:14 2013-05-14 12:32 SergeyB SergeyB normal resolved none none fixed 0 FIXED: minor bug in Student's t-test Previous versions of Student's t-test incorrectly handled degeneracy in the input data (when one/both of the samples had zero variance).

New version fixes behavior of:
* StudentTTest1
* StudentTTest2
* UnequalVarianceTTest

==============================
Reported by Andy Giese <gieseanw@gmail.com>:

The t-tests improperly return all tails = 1.0 when either variance for x or variance for y is 0. This is a mistake.

To reproduce: test against sets
x = {1, 1, 1, 1, 1}
y = {0.45, 0.56, 0.83, 0.92, 0.87}

Alglib says the probability of the null is 1.0 in all tests, which is clearly incorrect.
504 Statistics major have not tried 2013-02-16 11:00 2013-05-13 14:47 SergeyB SergeyB normal resolved none none fixed 0 FIXED: minor bug in the sign test (two sided alternative) Sign test returns incorrect value for BothTails (slightly larger than 1.0) when sample has same number of values at both sides of supposed median value.

This bug was reported by Julin Urbano:
> double[] data = new double[] { 1, 2, 3, -4, -5, -6 };
> alglib.onesamplesigntest(data, data.Length, 0, out both, out left, out right);
> and you'll see that the 2-tailed p-value (variable both) is 1.3125,
> which can't happen :-)
521 General minor have not tried 2013-05-12 13:27 2013-05-13 11:38 SergeyB SergeyB normal resolved none none fixed 0 FIXED: stack overflow in kd-tree construction routine Stack overflow was raised when input array contained a LOT of duplicate vectors. After the fix algorithm may handle arbitrary number of duplicates in the input data without causing overflow.

This issue was reported by jreyniers ( http://forum.alglib.net/viewtopic.php?f=2&t=840 ).
SergeyB 2013-05-13 11:30 Issue was confirmed.
516 Statistics feature have not tried 2013-04-18 11:00 2013-04-25 09:34 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: ranking functionality Ranking functionality:
* RankData() processes MxN matrix, replaces each value by its within-row rank.
* RankDataCentered() - same functionality, but returns centered rank (sum is zero)
* both functions support multithreaded execution
513 Statistics feature have not tried 2013-04-07 11:42 2013-04-25 09:31 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: parallel correlation/covariation functions Correlation/covariation function can be run in multithreaded mode.
510 Linear algebra minor have not tried 2013-04-04 14:55 2013-04-04 14:56 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: parallel linear algebra functions (real/complex TRSM, GEMM, SYRK) New parallel core allows to efficiently parallelize these operations. Parallel code was integrated into upcoming SMP edition of ALGLIB (commercial branch). Open Source edition of ALGLIB replaces all parallel calls by single-threaded stubs.
469 Linear algebra feature have not tried 2012-07-11 11:38 2013-04-04 14:53 SergeyB SergeyB high resolved none none implemented 0 IMPLEMENTED: efficient basecase GEMM code for C# Performance of real/complex GEMM in C# was significantly improved.
502 Optimization tweak have not tried 2013-01-17 16:42 2013-01-17 16:42 SergeyB SergeyB normal resolved none none implemented 0 INCOMPATIBLE: MinBLEICSetCond() replaces three other functions from MinBLEIC Previous version of MinBLEIC used active set algorithm with inner/outer loops. Stopping criteria for this algorithm were specified with MinBLEICSetInnerCond()/MinBLEICSetOuterCond()/MinBLEICSetMaxIts() functions.

However, new algorithm has no nested loops and uses only one set of stopping criteria, which are specified by MinBLEICSetCond(). Thus, deprecated functions were removed from MinBLEIC subpackage to avoid confusion.

This tweak is backward incompatible and requires present code which works with MinBLEIC to be slightly rewritten.
438 Optimization feature have not tried 2012-01-20 10:41 2013-01-14 13:43 SergeyB SergeyB high resolved none none implemented 0 IMPLEMENTED: refactoring of the BLEIC optimizer New version of the BLEIC optimizer has following improvements:
* it uses new three-stage active set algorithm suggested by Elvira Illarionova, which combines gradient projection with iterations of equality constrained L-BFGS
* because we do not have nested inner/outer iterations, more transparent stopping criteria can be specified for algorithm
* it uses new constraint activation/deactivation strategy which can correctly handle degenerate constraints
480 Data analysis feature have not tried 2012-08-03 11:04 2012-12-06 13:53 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: neural network improvements ALGLIB neural networks now include:
* new training interface which greatly simplifies training of multiple networks with same settings/datasets
* support for training sets specified by sparse matrices
* better documentation (several examples)
474 Data analysis feature have not tried 2012-07-16 11:41 2012-11-27 16:50 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: k-means clustering was merged with new clustering unit k-means++ algorithm now provide new-fashion clusterizaton interface, similar to that of new hierarchical clustering unit. Old interface was left for backward compatibility.

The most important change is that new version of algorithm stores centers in ROWS of the resulting matrix instead of columns. However, old code still continue to work because old version of the algorithm was left unchanged.
499 General feature have not tried 2012-11-23 23:27 2012-11-27 10:46 SergeyB SergeyB normal resolved none none implemented 0 DOCS: added important information on support for very large arrays in NET 4.5 Added information on support for arrays larger than 2GB in NET 4.5.
493 Statistics minor always 2012-11-07 17:14 2012-11-21 16:38 karuzo SergeyB normal resolved none none 3.6.0 fixed 0 FIXED: bug in Student t-tests Issue fixed. studenttest1/studenttest2 now return correct values, depending on sign of difference between means of X and Y.

ORIGINAL MESSAGE:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
studenttest1 returns all p-value=1 when all values of sample X are equal and different from assumed Mean.

Similarly studenttest2 returns all p-value=1 when all values of sample X are equal and different from equal values of sample Y.

I suppose this is wrong, as rather all p-values should be zero.		 Dim bt As Double, rt As Double, lt As Double
Call alglib.studentttest1({1, 1, 1, 1, 1}, 5, 0, bt, rt, lt)
Console.WriteLine("bothTails={0}, rightTail={1}, leftTail={2}", bt, rt, lt)
Call alglib.studentttest2({1, 1, 1, 1, 1}, 5, {2, 2, 2, 2, 2}, 5, bt, rt, lt)
Console.WriteLine("bothTails={0}, rightTail={1}, leftTail={2}", bt, rt, lt)


Output:
bothTails=1, rightTail=1, leftTail=1
bothTails=1, rightTail=1, leftTail=1
495 Solvers feature have not tried 2012-11-16 10:49 2012-11-21 15:22 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: automatic diagonal precondtitioner for linear CG and LSQR solvers LinCGSolveSparse() and LinLSQRSolveSparse() now use diagonal of the system matrix as preconditioner, which allows to solve systems with badly scaled variables without explicit preconditioning. This preconditioner is active by default.
470 Interpolation feature have not tried 2012-07-12 11:44 2012-10-30 10:31 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: linear/nonlinear least squares solvers (lsfit subpackage) now report errors in the coefficients Following information is reported by solvers:
* Rep.CovPar - covariance matrix
* Rep.ErrPar - errors in the parameters =sqrt(diag(CovPar))
* Rep.ErrCurve - error in the curve (per-point deviations of empirical curve from "ideal best fit curve")
* Rep.Noise - per-point estimates of noise level
* Rep.R2 - coefficient of determination

Following solvers report this information:
* nonlinear solver (LSFitState object)
* LSFitLinear
* LSFitLinearW
* LSFitLinearC
* LSFitLinearWC
467 Data analysis feature have not tried 2012-07-09 10:06 2012-10-24 11:54 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: hierarchical cluster analysis algorithm Implemented hierarchical cluster analysis algorithm:
* several distance metrics (Euclidean, 1-norm, infinity-norm, Pearson and Spearman correlation-based metrics, cosine distance)
* several linkage types (single linkage, complete linkage, average linkage)
486 Optimization minor always 2012-08-31 15:32 2012-10-24 10:59 nlagerr SergeyB normal resolved none none 3.6.0 fixed 0 FIXED: bug in the constraint activation/deactivation code of QP solver QP solver incorrectly activated/deactivated linear constraints in some tricky cases (multiple constraints which form degenerate system). New version of the solver uses another algorithm for constraint activation/deactivation which is more robust and should not experience this sort of bugs.

Below is an original report from one of the users:

> I have included two NUnit tests in which the QP solver of
> version 3.6.0 does not find the correct optimum. The QP
> solver does return the success termination type. I found
> that if I do not supply a starting point to the solver,
> this problem does not occur and the correct minimum is
> found.		 AlglibQPSolverTest.cs (4 KB) 2012-08-31 15:32
SergeyB 2012-09-04 16:30 I've started investigating this issue and will inform you about results within several days. Thanks for the bug report!
SergeyB 2012-09-04 16:58 issue confirmed.
487 Data analysis minor have not tried 2012-09-06 09:48 2012-09-25 11:18 SergeyB SergeyB normal resolved none none fixed 0 FIXED: infinite loop in k-Means when called for degenerate data From http://forum.alglib.net/viewtopic.php?f=2&t=611 by andreas:

I have a situation where kmeansgenerate seems to be stuck in an infinite loop (C# version of alglib 3.6.0). The parameters are as follows (copied from the debugger):

xy: {double[57, 1]}
 [0, 0]: 12.244689632138986
 [1, 0]: 12.244689632138982
 [2, 0]: 12.244689632138986
 [3, 0]: 12.244689632138982
 [4, 0]: 12.244689632138986
 [5, 0]: 12.244689632138986
 [6, 0]: 12.244689632138986
 [7, 0]: 12.244689632138986
 [8, 0]: 12.244689632138986
 [9, 0]: 12.244689632138986
 [10, 0]: 12.244689632138989
 [11, 0]: 12.244689632138984
 [12, 0]: 12.244689632138986
 [13, 0]: 12.244689632138986
 [14, 0]: 12.244689632138989
 [15, 0]: 12.244689632138986
 [16, 0]: 12.244689632138986
 [17, 0]: 12.244689632138986
 [18, 0]: 12.244689632138986
 [19, 0]: 12.244689632138989
 [20, 0]: 12.244689632138972
 [21, 0]: 12.244689632138986
 [22, 0]: 12.244689632138986
 [23, 0]: 12.244689632138986
 [24, 0]: 12.244689632138984
 [25, 0]: 12.244689632138982
 [26, 0]: 12.244689632138986
 [27, 0]: 12.244689632138986
 [28, 0]: 12.244689632138986
 [29, 0]: 12.244689632138986
 [30, 0]: 12.244689632138986
 [31, 0]: 12.244689632138986
 [32, 0]: 12.244689632138986
 [33, 0]: 12.244689632138986
 [34, 0]: 12.244689632138986
 [35, 0]: 12.244689632138982
 [36, 0]: 12.244689632138989
 [37, 0]: 12.244689632138986
 [38, 0]: 12.244689632138986
 [39, 0]: 12.244689632138986
 [40, 0]: 12.244689632138986
 [41, 0]: 12.244689632138986
 [42, 0]: 12.244689632138986
 [43, 0]: 12.244689632138986
 [44, 0]: 12.244689632138986
 [45, 0]: 12.244689632138986
 [46, 0]: 12.244689632138986
 [47, 0]: 12.244689632138986
 [48, 0]: 12.244689632138986
 [49, 0]: 12.244689632138986
 [50, 0]: 12.244689632138984
 [51, 0]: 12.244689632138986
 [52, 0]: 12.244689632138986
 [53, 0]: 12.244689632138986
 [54, 0]: 12.244689632138986
 [55, 0]: 12.244689632138986
 [56, 0]: 12.244689632138986

nPoints: 57
nVars: 1
k: 4
restarts: 1

Can you reproduce this situation?

I understand that this doesn't really constitute "good data", but unfortunately, I don't have a lot of control on the input values, they can be as meaningless as in this case. Currently I'm just filtering when there are less distinct values than clusters. I'm probably changing this to require some higher minimum range between the values and hope to avoid such situations.

I'm running the program in Vista 64bit and use VS2010 and .Net 4.
481 Optimization minor always 2012-08-03 14:37 2012-08-03 15:02 GrigoriyF SergeyB normal assigned none none 3.6.0 open 0 minbleicrestartfrom(state, x) does not assign state.x The function alglib.minbleicrestartfrom(state, x) does not assign a value x to state.x, so it is not possible to launch BLEIC optimization with a prespecified starting point.
SergeyB 2012-08-03 15:02 No, it assigns s.startx (field which is used when we start optimization). So algorithm should restart correctly from specified point.

Can you post code which reproduces this bug?
462 Linear algebra feature have not tried 2012-06-29 18:06 2012-07-26 14:52 SergeyB Leo normal resolved none none implemented 0 IMPLEMENTED: sparse matrix improvements New sparse matrix functions
* SparseGetRow() allows to read I-th row from CRS representation
* SparseConvertToHash() allows to perform in-place conversion of the sparse matrix to Hash table storage format
* SparseCopyToHash() and SparseCopyToCRS() allow to perform out-of-place conversion of the sparse matrix to another storage format
* SparseGetType(), SparseIsHash(), SparseIsCRS() functions allow to determine sparse storage type

Performance improvements:
* SparseRewriteExisting() now has O(logN) performance for CRS matrices
456 General feature have not tried 2012-05-15 10:06 2012-07-26 13:45 SergeyB SergeyB normal assigned none none open 0 Add support for NumPy arrays Support for NumPy arrays xalglib.py_v2.tar.bz2 (37 KB) 2012-05-22 15:07
427 Optimization feature have not tried 2012-01-16 12:59 2012-07-05 14:27 SergeyB Leo normal resolved none none implemented 0 IMPLEMENTED: verification of the user-supplied gradient (LSFit, MinLM, MinBLEIC, MinCG, MinLBFGS) Optional verification of the user-supplied gradient, allows to detect errors in the analytic derivatives (quite common problem in the optimization).
465 Interpolation feature have not tried 2012-07-03 15:05 2012-07-04 16:54 SergeyB Leo normal resolved none none implemented 0 IMPLEMENTED: 3D splines Implemented:
* trilinear 3D splines
* support for scalar and vector-valued models
* trilinear resampling (grid conversion)
464 Interpolation feature have not tried 2012-07-03 15:05 2012-07-03 17:13 SergeyB Leo normal resolved none none implemented 0 IMPLEMENTED: vector-valued 2D splines Spline2D unit now supports vector-valued 2D splines (both bilinear and bicubic ones).
455 Interpolation tweak have not tried 2012-05-15 10:05 2012-07-03 15:04 SergeyB SergeyB normal resolved none none fixed 0 FIXED: incorrect handling of multilayer RBF models without nonlinear layers RBF-ML algorithm builds combined model, which has one linear layer and NLayers nonlinear layers.

Unfixed version of the algorithm built zero (empty) model when called with NLayers=0 (no nonlinear layers). However, even when we have NLayers=0, there is linear layer which MUST be built (which was not done in the unfixed algorithm).

After this fix algorithm builds pure linear model when called with NLayers=0.
457 Linear algebra feature have not tried 2012-05-24 16:21 2012-06-29 18:34 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: new functionality for sparse matrices New functionality:
* efficient O(logN) SparseGet() for matrices stored in CRS format
* SparseRewriteExisting() - function for thread-safe modification of the already existing (non-zero) elements of the sparse matrix
* SparseEnumerate() function for efficient enumeration of non-zero elements
458 Linear algebra feature have not tried 2012-06-18 10:35 2012-06-29 10:53 SergeyB Leo normal resolved none none implemented 0 IMPLEMENTED: minor tweak of the random matrix generator Random matrix generator (matgen subpackage of the LinAlg package) now enforces strict symmetry (conjugate symmetry) of the matrices being generated, up to the last significant bit.
454 Spec.functions major always 2012-05-08 20:43 2012-06-29 10:33 exp_1 SergeyB normal resolved none none fixed 0 FIXED: big in the ErrorFunction() (about 1E-10 in magnitude) Erf function returns incorrect results. Error is about 1e-10.
Sing is lost in constant 0.288805137207594084924010.
Must be -0.288805137207594084924010.

For example, in Delphi file normaldistr.pas

Wrong code:
P := 0.288805137207594084924010+XSq*P;

Correct code:
P := -0.288805137207594084924010+XSq*P;

Files affected:
cpp\src\specialfunctions.cpp
csharp\src\specialfunctions.cs
ipython\csharp-src\specialfunctions.cs
vbnet\csharp-src\specialfunctions.cs
cpython\core\src\normaldistr.c
vb6\src\normaldistr.bas
freepascal\src\normaldistr.pas
delphi\src\normaldistr.pas
 Proof:
See:
http://www.wolframalpha.com/input/?i=erf%284%2F10%29

Now erf(0.4) =0.42839235536425169
Must be 0.4283923550466684551036038453201724441218629285225903...
Correcting =0.42839235504666845
459 Linear algebra minor always 2012-06-18 17:05 2012-06-28 18:10 SergeyB SergeyB normal resolved none none 3.5.0 fixed 0 FIXED: minor inefficiency in the norm estimator Algorithm always (independently of user settings) used one initial vector instead of trying several vectors to choose better.

Fixed in the new version of ALGLIB. Several tests were added which performed statistical analysis of the algorithm performance to detect this issue.
461 Interpolation feature have not tried 2012-06-28 17:20 2012-06-28 17:20 SergeyB Leo normal resolved none none implemented 0 IMPLEMENTED: monotone spline interpolation Spline construction subrouitne which builds monotone cubic Hermite interpolant. This interpolant is monotonic in [x(0),x(n-1)] and is constant outside of this interval. In case y[] form non-monotonic sequence, piecewise monotonic interpolant is constructed.

Say, for x=(0,1,2,3,4) and y=(0,1,2,1,0) interpolant will monotonically grow at [0..2] and monotonically decrease at [2..4].
449 Optimization feature have not tried 2012-02-29 14:01 2012-06-27 12:22 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: MinQP optimizer with support for general linear equality/inequality constraints Improved version of MinQP optimizer:
* support for arbitrary combination of boundary, linear equality, linear inequality constraints
* combination of the active set method and augmented Lagrangian is used to solve QP subproblems
430 Interpolation feature have not tried 2012-01-17 11:01 2012-03-27 10:30 SergeyB Leo normal resolved none none implemented 0 IMPLEMENTED: RBF model RBF model for interpolation in 2 and 3-dimensional spaces, which:
* supports Gaussian RBF's with additional linear term
* supports scattered interpolation with two algorithms:
** RBF-QNN - suited for approximately uniformly distributed points, perfect fit
** RBF-ML - multilayer RBF model which is able to fit any distribution of points, and can solve problems in the least squares sense (imperfect fit) with controllable amount of smoothing
* supports interpolation of scalar and vector functions
* can calculate function values at arbitrary points and on regular grids
* supports serialization
* uses efficient O(N*logN) algorithms to build model, suited for interpolation of very large datasets (100000 points and even more)
451 General feature have not tried 2012-03-05 12:01 2012-03-23 13:08 SergeyB SergeyB normal resolved none none fixed 0 Fix issue with documentation Clarify that C++ version needs compilation of .cpp files - not inclusion.
450 Optimization minor have not tried 2012-03-05 10:24 2012-03-05 10:24 SergeyB SergeyB normal resolved none none fixed 0 FIXED: bug in the MinQP optimizer Previous version of the MinQP optimizer with non-guaranteed descent was auick, but failed to solve some problems because of eternal cycle. New version of the optimizer guarantees at least small progress per iteration and should find solution within finite number of iterations.
416 Linear algebra feature have not tried 2011-12-12 11:11 2012-02-15 10:22 SergeyB SergeyB normal assigned none none open 0 LinCG and LinLSQR: support for callbacks Add ability to work with matrices given by matrix-vector product operator.
446 Linear algebra feature have not tried 2012-02-09 14:50 2012-02-14 11:29 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: matrix norm estimator Iterative estimator for norms of the sparse real matrices. Can be used to estimate norms of both square and rectangular matrices.
436 Data analysis minor have not tried 2012-01-17 11:35 2012-02-08 16:47 SergeyB SergeyB normal resolved none none fixed 0 FIXED: issue in MLPTrainES() Failure to converge to the optimal point
404 Interpolation minor always 2011-10-17 15:48 2012-02-03 14:05 SergeyB SergeyB normal resolved none none fixed 0 FIXED: minor bugs in Spline1D subpackage Fixed two bugs:
* Spline1DLinTransX() did not correctly handled non-smooth (piecewise linear) splines
* Spline1DBuildCubic() with periodic boundary conditions did not correctly handled input data with omitted last Y value (rare case, but should be handled correctly)
445 Data analysis feature have not tried 2012-02-01 17:29 2012-02-01 17:29 SergeyB SergeyB normal resolved none none implemented 0 TWEAK: support for empty KD-trees Added support for zero-sized KD-trees, i.e. trees which were built from zero dataset.
420 Optimization major have not tried 2012-01-05 23:45 2012-01-20 10:39 SergeyB SergeyB normal resolved none none fixed 0 FIXED: BLEIC optimizer can fail when started from infeasible point Feasibility solver used by BLEIC had flaw in the algorithm which prevented it from finding feasible points in some circumstances. It was replaced by better algorithm with guaranteed convergence (unless problem is so ill-conditioned that it is impossible to find feasible point because of rounding errors).
435 Linear algebra feature have not tried 2012-01-17 11:31 2012-01-17 11:31 SergeyB Leo normal resolved none none implemented 0 IMPLEMENTED: sparse LSQR solver Sparse solver which works with matrices represented by SparseMatrix structure and can solve:
* square non-symmetric non-degenerate problems
* non-square non-degenerate problems (least squares solution is returned)

This solvers avoid squaring of condition number when working with least squares problem. Thus, it is better suited for solution of least squares problems than CG on normal equations. However, algorithm should be used with caution on rank-deficient or ill conditioned problems.
434 Linear algebra feature have not tried 2012-01-17 11:26 2012-01-17 11:27 SergeyB Leo normal resolved none none implemented 0 IMPLEMENTED: sparse linear CG optimizer Sparse linear CG optimizer, which works with matrices stored in the SparseMatrix structure.
410 Linear algebra feature have not tried 2011-10-24 17:53 2012-01-17 11:22 SergeyB Leo normal resolved none none implemented 0 IMPLEMENTED: sparse matrices Sparse matrix structure which supports:
* improved CRS format (for memory efficient operations) and Hash-Table representation (easy to use format, intended to simplify matrix creation)
* basic operations with elements: Set, Get, Add
* linear matrix-vector operations: A*x, A'*x, simultaneous calculation of both products
* linear matrix-matrix operations
* operations with symmetric matrices given by lower or upper triangle only
403 Statistics feature N/A 2011-10-02 23:45 2012-01-17 11:21 SergeyB Leo normal resolved none none implemented 0 IMPLEMENTED: separate functions for mean, variance, standard deviation Separate functions for calculations of mean, variance and standard deviation. See http://forum.alglib.net/viewtopic.php?f=2&t=452 for rationale.
433 General feature have not tried 2012-01-17 11:20 2012-01-17 11:20 SergeyB Leo normal resolved none none implemented 0 IMPLEMENTED: generation of random values from continuous/discrete distribution given by finite samples Two new functions - HQRNDDiscrete and HQRNDContinuous - which can be used to generate random values from continuous/discrete distribution given by finite samples.
432 Data analysis feature have not tried 2012-01-17 11:16 2012-01-17 11:17 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: SMA, EMA, LRMA filters Filters used to smooth variations in time series data:
* simple moving averages
* exponential moving averages
* linear regression moving averages
419 Linear algebra minor have not tried 2011-12-22 11:52 2011-12-22 11:53 SergeyB SergeyB normal resolved none none fixed 0 FIXED: inconvergence of the low-accuracy version of bidiagonal SVD Bidiagonal QR SVD provided by LAPACK has two version - high accuracy but slow one (TOL>0), and moderate accuracy but quick one (TOL<0). Quick version in some very rare circumstances fails to converge.

Currently it is unknown whether it is bug or intrinsic property of the algorithm. In order to make all SVD-dependent algorithms more stable quick version is silently replaced by the slower, but more robust one.
251 Statistics feature have not tried 2009-12-27 12:10 2011-11-02 15:45 SergeyB Leo none assigned none none open 0 Generation of random variables... ...from distributions represented by random sample.

Discrete distribution - we select one element from sample
Continuous - we generate random real R from 0 to 1, and calculate R-quantile of distribution.

Generate:
* both one value and vector

Add this to HQRND unit.
412 Linear algebra feature have not tried 2011-10-26 17:16 2011-10-26 17:16 SergeyB Leo normal assigned none none open 0 User-friendly BLAS in development
405 General feature have not tried 2011-10-17 15:58 2011-10-18 11:05 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: serialization of data analysis structures Implemented:
* serialization of decision forests
* serialization of KD-trees
* serialization neural networks

Serialization is implemented in the portable manner, i.e. serialized structures can be moved between different machines and even versions of ALGLIB for different languages. Development of this addition was funded by IDx LLC - a company which used ALGLIB classification features for analysis of medical data.
408 General feature have not tried 2011-10-18 11:04 2011-10-18 11:05 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: SSE2 support for GCC/SunStudio ALGLIB for C++ now supports SSE2 under GCC and Sun Studio (previous version supported MSVC only).
409 Data analysis feature have not tried 2011-10-18 11:05 2011-10-18 11:05 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: Markov Chains for Population Data New data analysis algorithm was added - Markov Chains for Population/proportional Data (MCPD) solver. This solver can be used for portfolio analysis, when you want to restore hidden transition matrix using aggregated portfolio data.
407 Optimization feature have not tried 2011-10-17 16:11 2011-10-17 16:11 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: numerical differentiation support All optimizers (BLEIC, CG, L-BFGS, LM) now support numerical differentiation. Only fixed step size is supported, but it can be different for different variables. Numerical differentiation allows fast prototyping when solving small and moderately dimensioned problems.
406 Optimization feature have not tried 2011-10-17 16:10 2011-10-17 16:10 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: optimization of functions with singularities Optimization algorithms now can find extrema of functions with singularities (as long as correct values are returned near the singular point, and +INF is returned exactly at the singularity). Special post-processing is applied to the function value in order to prevent overflow.
401 Optimization minor always 2011-08-05 11:03 2011-10-17 15:56 SergeyB SergeyB normal resolved none none implemented 0 FIXED: bug in LM optimizer Numerical differentiation version of the LM optimizer implicitly assumed that if you have all variables constrained to one fixed value with constraints like ai<=xi<=ai, then initial value of xi MUST be equal to ai. When this assumption was violated, algorithm converged to the wrong point.
365 Interpolation feature have not tried 2010-07-31 23:01 2011-10-17 15:51 SergeyB Leo normal assigned none none open 0 New 1D spline operations * list of local extremums
* list of zeros
399 Integration crash always 2011-05-11 12:23 2011-05-14 17:54 Gideon SergeyB normal resolved none none 3.3.0 fixed 0 FIXED: autogkintegrate crashes on certain function spaces for no apparent reason I am using alglib.autogksmooth and autogkintegrate to integrate a function f(x). f(x) is smooth and continuous. Autogkintegrate runs out of memory on certain values of x but exceeds on most other values. For example x=3.34 works within 255 function evaluations, x=3.35 crashes (i.e. endless function evaluations), x=3.5 works.

I will send a sample app to illustrate the error.
SergeyB 2011-05-14 17:54 Function being integrated contains small numerical noise which confuses criteria used to stop algorithm once it achieves desired accuracy.

In order to fix this bug two changes were introduced:
a) inner stopping condition was made less strict
b) upper limit on number of iterations was imposed
398 Optimization feature have not tried 2011-04-25 21:06 2011-04-25 21:06 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: BLEIC optimizer uses faster and more robust algo BLEIC (boundary and linearly constrained optimizer) was switched to more robust algorithm from active set family of methods.
397 Optimization feature have not tried 2011-04-25 21:04 2011-04-25 21:04 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: LSFit improvements LSFit subpackage now supports:
* boundary constraints
* scaling
396 Optimization feature have not tried 2011-04-25 21:03 2011-04-25 21:03 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: MinLM improvements MinLM subpackage now supports:
* boundary constraints
* scaling
395 Optimization feature have not tried 2011-04-25 21:01 2011-04-25 21:01 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: quadratic programming support Dense convex quadratic programming with boundary constraints.
394 Optimization minor have not tried 2011-04-25 20:59 2011-04-25 21:00 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: L-BFGS improvements Following features were implemented:
* diagonal preconditioner
* scaling
393 Optimization feature have not tried 2011-04-25 20:58 2011-04-25 20:58 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: MinCG improvements Following features were implemented:
* diagonal preconditioner
* scaling
* automatic reset after N iterations
392 Linear algebra feature have not tried 2010-12-29 13:28 2010-12-29 13:29 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: limited support for SSE2 ALGLIB for C++ now supports SSE2 intrinsics. SSE2 support is still in the experimental phase, in the current release it is accessible to the MSVC users only.
391 Optimization feature have not tried 2010-12-29 13:27 2010-12-29 13:27 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: nonlinear CG improvements Improvements in the nonlinear CG optimizer:
* user can "suggest" initial length for a step
* algorithm is much more better at detection of eternal loops caused by too stringent stopping conditions
390 Optimization feature have not tried 2010-12-29 13:24 2010-12-29 13:24 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: optimization with linear equality/inequality constraints BLEIC (Boundary, Linear Equality and Inequality Constrained) optimizer can be used for optimization subject to arbitrary combination of boundary, linear equality and linear inequality constraints.
357 Linear algebra feature have not tried 2010-06-28 20:38 2010-11-28 22:53 SergeyB SergeyB normal assigned none none open 0 User-friendly BLAS * operations with matrices with much more simpler interface than that of ABLAS unit: add, sub, FMA
* high-level operations: sin/cos/exp/pow/...
373 Linear algebra feature have not tried 2010-08-24 23:06 2010-11-28 22:52 SergeyB SergeyB normal assigned none none open 0 Pseudoinverse real
389 Optimization feature have not tried 2010-11-28 22:52 2010-11-28 22:52 SergeyB SergeyB normal assigned none none open 0 Optimization improvements * ability to report errors on unbounded functions (CG, L-BFGS, ASA, MINLM, BLEIC)
* BLEIC: exponential modified barrier functions and adaptive control over Mu (slower decrease in case we've accidentally moved deeply into infeasible area)
385 Interpolation feature have not tried 2010-10-28 17:01 2010-11-01 14:03 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: new curve fitting algorithm IMPLEMENTED: curve fitting by penalized regression splines. This algorithm is more robust, tunable and easier to use than cubic/Hermite regression splines without penalty function.
388 Statistics feature have not tried 2010-11-01 14:02 2010-11-01 14:03 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: new covariance/correlation functions New functions for calculation of covariance/correlation:
* calculation of covariance/correlation matrices (both Pearson and Spearman matrices can be calculated).
* calculation of cross-covariance/cross-correlation matrices (both Pearson and Spearman matrices can be calculated).

Cache-oblivious code is used in both cases which can efficiently work with large matrices which do not fit into CPU cache.
387 Optimization feature have not tried 2010-10-28 17:09 2010-10-28 17:10 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: Cholesky preconditioner for L-BFGS algorithm L-BFGS optimizer now supports dense Cholesky preconditioners (can be useful if you have approximate quadratic model of a function and want to do L-BFGS optimization using it).
386 Interpolation feature have not tried 2010-10-28 17:06 2010-10-28 17:06 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: improved resampling with cubic splines IMPLEMENTED: grid conversion functions spline1dcubicconv(), spline1dcubicconvdiff() and spline1dcubicconvdiff2() can now convert from large grids to small ones (previous version supported conversion to grids of same size or larger, but not reverse conversion).
384 Interpolation feature have not tried 2010-10-28 16:55 2010-10-28 16:55 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: improvements in the nonlinear least squares solver NLS solver now can solve gradient-free problems. User supplies function being fitted, ALGLIB automatically calculates derivatives using combination of numerical differentiation and secant updates.
383 Optimization feature have not tried 2010-10-28 16:52 2010-10-28 16:53 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: improvements in the Levenberg-Marquardt optimizer New optimization mode: "V", Jacobian-free optimization. Functions vector only is used; combination of numerical differentiation and secant updates is used to calculate accurate and quick estimates of Jacobian.

We've also introduced two modes - "VJ" and "VGJ", which are intended to replace "FJ" and "FGJ" modes (they are considered obsolete, but still supported). The only difference is that former ones accept function vector while latter use merit function (squared norm of function vector).

We've decided to add new modes because function vector is more informative than merit function, and we can apply more optimizations and acceleration techniques with this kind of information about function.
382 Interpolation feature have not tried 2010-10-28 16:46 2010-10-28 16:46 SergeyB SergeyB normal resolved none none fixed 0 IMPLEMENTED: barycentric-power conversion, barycentric-Chebyshev conversion IMPLEMENTED: functions for conversions between barycentric and power basis representations, and between barycentric and Chebyshev representations. Both forward and backward conversions are supported.
381 General minor have not tried 2010-10-28 16:44 2010-10-28 16:44 SergeyB SergeyB normal resolved none none fixed 0 TWEAK: better detection of IEEE special quantities One more bug in old GCC that was triggered by code for detection of architecture features. Although it was compiler bug, not ALGLIB bug, we've decided to rewrite ALGLIB code for detection of IEEE special quantities.
379 General tweak have not tried 2010-09-30 10:39 2010-09-30 10:39 SergeyB SergeyB normal resolved none none fixed 0 FIXED: minor tweak in detection of IEEE special quantities Detection of IEEE special quantities in the 3.0.0.RC1 was unreliable when ALGLIB was compiled with heavy optimization settings (-xO5 for Sun Studio, -Os for GCC). In both cases subtle bugs in compilers was discovered (incorrect handling of volatile modifier; bugs when dealing with pointers to local variables).

In order to avoid triggering these bugs ALGLIB code was rewritten in a manner which was perfectly understandable by both compilers.
378 Optimization feature have not tried 2010-09-30 10:33 2010-09-30 10:33 SergeyB SergeyB normal resolved none none implemented 0 OPTIMIZATION: better step length selection for CG-based algorithms CG-based algorithms (ASA and CG) use better step selection procedure (step length from previous iteration is used as initial estimate).
377 General feature have not tried 2010-09-30 10:28 2010-09-30 10:28 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: internal sorting improvements All interpolation algorithms which sort nodes have become slightly faster.
376 Interpolation feature have not tried 2010-09-30 10:26 2010-09-30 10:26 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: grid conversion and differentiation functions for 1-dimensional splines A bunch of new functions:
* Spline1DGridDiff() and Spline1DGridDiff2() - for fast and convenient calculation of spline derivatives in the grid nodes
* Spline1DConv(), Spline1DConvDiff(), Spline1DConvDiff2() - for fast and convenient conversion from one grid to another with cubic splines (with optional differentiation)
354 Interpolation feature have not tried 2010-06-23 21:08 2010-09-20 12:43 SergeyB SergeyB normal assigned none none open 0 Fitting functions for important special cases * fitting by sphere
* total least squares straight line
* fitting by Gaussian (with offset, without offset, with prescribed coefficients)
* 4-PL and 5-PL logistic curves
375 Interpolation feature have not tried 2010-09-02 11:12 2010-09-02 11:12 SergeyB SergeyB normal resolved none none implemented 0 INCOMPATIBLE: nonlinear fitting functions were renamed LSFitNonlinear???() functions were renamed to LSFit???() to make their names compatible with naming conventions used by Optimization package.
371 Solvers feature have not tried 2010-08-24 22:13 2010-08-24 22:14 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: nonlinear solver New nonlinear Levenberg-Marquardt-like solver was implemented. Its essential features are:
* weak requirements on the system being solved (Jacobian non-singularity assumption is dropped)
* global convergence
* quadratic convergence in the neighborhood of solution
* same set of additional options as for optimization algorithms: reports, limit on step size, efficient restarts
370 Optimization feature have not tried 2010-08-24 22:08 2010-08-24 22:09 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: buffered functions for optimization algorithms Optimization algorithms have functions with common names like 'ALGONAMEResults' which were used to get algorithm result. These functions have output parameter X - array, which gets reallocated on each call (because of semantics of these functions).

This fix adds new family of functions - 'ALGONAMEResultsBuf', which takes preallocated buffer (and reallocates it, if too small). It allows to save time when algorithm is used from inner cycle of another performance critical algorithm.
369 General feature have not tried 2010-08-24 22:01 2010-08-24 22:02 SergeyB SergeyB normal resolved none none implemented 0 INCOMPATIBLE: changes in 'nearestneighbor' unit Backward compatible changes:
* buffered query functions were added. These functions can use preallocated buffer (which is automatically reallocated, if it is too small), which reduces number of dynamic allocations and increases overall performance
* better detection of errors in input parameters

Incompatible changes:
* functions from 'KDTreeQueryResults' family do NOT return result size K anymore. This value is already returned by functions from 'KDTreeQuery' family, so there are no reasons to duplicate it.
353 General feature have not tried 2010-06-15 18:57 2010-08-24 21:52 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: improved support for IEEE special quantities Several units now support detection of IEEE special quantities in the input data. These units are polint, ratint, matdet, matinv.
358 Linear algebra feature have not tried 2010-06-28 21:12 2010-08-24 21:49 SergeyB SergeyB high resolved none none implemented 0 INCOMPATIBLE: support for LDLT factorization is dropped from ALGLIB 3.0 LDLT factorization is dropped from ALGLIB 3.0
367 TODOs minor have not tried 2010-08-24 08:58 2010-08-24 21:46 Peter Cusack SergeyB normal resolved none none fixed 0 FIXED: documentation bug The description of the complex number routines in visual basic defines the addition routines as multiplications.

"Public Function C_Add(Z1 As Complex Z2 As Complex):Complex
Public Function C_AddR(Z1 As Complex R As Double):Complex
Calculate Z1*Z2 or Z1*R."
SergeyB 2010-08-24 21:46 Thanks, it is fixed.

alglib-2.6.0.vb6.zip now contains fixed manual.
348 Optimization feature have not tried 2010-06-03 13:26 2010-08-01 13:29 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: RestartFrom() functionality for optimizers All optimizers can be restarted from new point without allocating new optimizer object. It allows to significantly increase performance when solving many sequential optimization problems with same sizes.
363 Statistics major have not tried 2010-07-30 16:19 2010-07-30 16:20 SergeyB SergeyB normal resolved none none fixed 0 FIXED: bug in UnequalVarianceTTest Degrees of freedom for t-statistic were incorrectly calculated.

Thanks to Gerhard Vogt for pointing out this bug.
360 Spec.functions tweak have not tried 2010-07-01 21:09 2010-07-01 21:09 SergeyB SergeyB normal resolved none none implemented 0 INCOMPATIBLE: renamed several special functions Due to name conflicts with C standard library following names were changed:
* Gamma => GammaFunction
* Erf => ErrorFunction
* ErfC => ErrorFunctionC
282 Linear algebra tweak have not tried 2010-01-29 21:44 2010-06-28 12:29 SergeyB SergeyB normal assigned none none open 0 Symmetric random matrix must be exactly symmetric it must be checked in unit test
333 AlgoPascal feature have not tried 2010-05-18 19:59 2010-06-16 10:52 artyomstv SergeyB normal assigned none none open 0 ap::complex power It is useful to implement pow method for ap::complex class, like:

complex& operator^=(const double& p) {
    double r = pow(sqrt(x*x + y*y), p);
    double phi = p * (x >= 0. ? atan(y/x) : (y >= 0. ? atan(y/x) + M_PI : atan(y/x) - M_PI));
    x = r * cos(phi);
    y = r * sin(phi);
    return *this;
};
SergeyB 2010-05-19 08:45 Something like this will be added in the next release, thanks!
214 Statistics feature have not tried 2009-12-03 18:32 2010-06-16 10:52 SergeyB SergeyB normal assigned none none open 0 PCA using correlation matrix todo
346 Interpolation feature have not tried 2010-05-31 18:38 2010-05-31 18:39 SergeyB SergeyB normal resolved none none implemented 0 INCOMPATIBLE: Spline1DSerialize/Spline1DUnserialize were removed Motivation: upcoming ALGLIB 3.0 release requires clean-up of ALGLIB source. ALGLIB 3.0 will contain totally new serialization framework which will be incompatible with old-styled serialization function.
344 Interpolation feature have not tried 2010-05-31 18:11 2010-05-31 18:11 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: parametric splines 2-dimensional and 3-dimensional parametric splines were implemented:
* spline types - Catmull-Rom and cubic splines
* boundary conditions - parabolically terminated or periodic (closed curves in 2D/3D)
* parameterizations - uniform, chord length, centripetal
* ability to calculate position corresponding to parameter value, tangents, derivatives of X/Y/Z-components with respect to parameter
* ability to calculate arc length
338 Interpolation feature have not tried 2010-05-27 19:29 2010-05-27 19:30 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: cubic spline improvements * Catmull-Rom splines with periodic or non-periodic boundary conditions
* Cubic splines now support periodic boundary conditions too
332 General major have not tried 2010-05-18 19:47 2010-05-26 12:34 artyomstv SergeyB normal resolved none none fixed 0 FIXED: Mistake in C++ implementation of ap::complex::operator/=(complex& z) Wrong code:

if( fabs(z.y)<fabs(z.x) ){
    e = z.y/z.x;
    f = z.x+z.y*e;
    result.x = (z.x+z.y*e)/f;
    result.y = (z.y-z.x*e)/f;
} else {
    e = z.x/z.y;
    f = z.y+z.x*e;
    result.x = (z.y+z.x*e)/f;
    result.y = (-z.x+z.y*e)/f;
}

Correct code:

if( fabs(z.y)<fabs(z.x) ){
    e = z.y/z.x;
    f = z.x+z.y*e;
    result.x = (x+y*e)/f;
    result.y = (y-x*e)/f;
} else {
    e = z.x/z.y;
    f = z.y+z.x*e;
    result.x = (y+x*e)/f;
    result.y = (y*e-x)/f;
}		 Have not checked any other operators. Maybe it is neccessary to check them.
SergeyB 2010-05-19 08:38 Fixed.

Thanks for your bug report!
329 Spec.functions major always 2010-05-07 22:52 2010-05-26 12:33 gighi SergeyB normal resolved none none 2.5.0 fixed 0 FIXED: K-Means++: 'multiple restarts' doesn't work correctly Hi,
I have a problem regarding the K-Means++ implementation contained in ALGLIB.
Please have a look at kmeans.cpp:238:

if( ap::fp_less(e,ebest) )
{
    copymatrix(ct, 0, k-1, 0, nvars-1, ctbest, 0, k-1, 0, nvars-1);
}

Now, if "e" is less then "ebest", the current table containing centroids is saved as the "best centroids set", but, as you can easily see, the variable "ebest" is never updated, so its value will always be "ap::maxrealnumber". As a consequence of this, the "number of restarts" of the algorithm is completely useless, because only the last iteration will be considered, even if it's not the best one.
Am I wrong? This produces an output which is not the best one computed by the algorithm.
SergeyB 2010-05-10 23:01 Yes, you are perfectly right. Thanks for bug report.

This issue will be fixed in the next release, which is scheduled for June, 1st.
gighi 2010-05-14 21:42 Another thing: I think you should also add something like "xyc_best", because at this time the "xyc" vector is overwritten at every iteration, and so only the last one is reported as output, and that's not good for the same initial reason.
SergeyB 2010-05-26 12:33 * issue is fixed in 2.6.0
* added unit test which checks for statistically significant difference between Restarts=1 and Restarts>1 (solution qualities are compared).
327 Optimization feature have not tried 2010-04-12 11:23 2010-04-12 13:10 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: improved Levenberg-Marquardt optimizer * improved interface (backward incompatible changes), default values are used to initialize algorithm parameters. Once MinLMState is initialized, parameters may be tuned with functions from MinLMSetXXXXXX() family.
* MinLMXXX() is renamed to MinLMCreateXXX(). LMState is renamed to MinLMState. LMReport is renamed to MinLMReport.
* ability to set maximum step (useful when minimizing fast growing functions)
326 Optimization feature have not tried 2010-04-12 11:22 2010-04-12 13:09 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: improved L-BFGS optimizer * improved interface (backward incompatible changes), default values are used to initialize algorithm parameters. Once MinLBFGSState is initialized, parameters may be tuned with functions from MinLBFGSSetXXXXXX() family.
* MinLBFGS() is renamed to MinLBFGSCreateXXX(). LBFGSState is renamed to MinLBFGSState. LBFGSReport is renamed to MinLBFGSReport. Unit name is changed from LBFGS to MinLBFGS.
* ability to set maximum step (useful when minimizing fast growing functions)
319 Optimization feature have not tried 2010-03-26 21:46 2010-04-12 11:24 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: nonlinear conjugate gradient optimizer Features:
* two CG methods: DY and hybrid HS-DY
* global convergence even on non-convex functions under mild conditions: grad(f) must be Lipschitz continuous in a neighborhood of a level set f(x)<f(x0).
321 Optimization feature have not tried 2010-03-31 16:10 2010-04-12 11:24 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: bound constrained nonlinear optimizer Bound constrained nonlinear optimizer:
* both convex and nonconvex functions may be minimized
* only Lipschitz continuity of the gradient is required
* interleaved CG/GPA iterations are used. Gradient Projection Algorithm is used to identify (add/drop) active constraints, Conjugate Gradient method is used to optimized once constraints are identified
325 Interpolation feature have not tried 2010-04-12 10:53 2010-04-12 10:53 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: improved inverse distance weighting interpolation IDWINT unit contains one more interpolation mode: constant nodal functions + "all points within specified radius" selection criterion. It may be used with nonuniform distributions of points.
324 Interpolation feature have not tried 2010-04-12 10:49 2010-04-12 10:49 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: improved nonlinear fitting Two important improvements:
* simpler interface (although slightly incompatible with previous one). Now you don't have to specify all optimization parameters in one call of LSFitNonlinearXXX(). All parameters are initialized by default values, which can be tuned later (if you really need it).
* ability to set maximum step (duplicate of 0000322)
322 Optimization crash always 2010-04-01 16:07 2010-04-12 10:42 senyai SergeyB normal resolved none none 2.4.0 fixed 0 FIXED: crash in lsfitnonlineariteration function ap::ap_error when bad values specified in vector of parameters being fitted. When c(2) at line 40 is 1.5 all is ok, when it is 5.5 there is a crash. Looks like this issue was fixed, but I'm not sure. nonlinear_fit_test.cpp (2 KB) 2010-04-01 16:07
SergeyB 2010-04-02 14:52 The error is caused by target function which has too wild gradient, which shoots algorithm too far from starting point, which in turn leads to overflow. More precisely, algorithm tries step about 1E24 in magnitude so it raises overflow on the second attempt to calculate target function.

The only way to solve this problem is let algorithm restrict its steps so it won't cause overflow in one step. It will be implemented in the upcoming release, which will feature new L-BFGS/Levenberg-Marquardt API and new optimization algorithms (CG and bound constrained CG).
SergeyB 2010-04-12 10:40 Fixed. Maximum step can be specified, so if your function have exponents inside which knocks current point too far, you can call SetStpMax() to guard algorithm steps.
320 Linear algebra minor have not tried 2010-03-30 11:26 2010-03-30 11:27 SergeyB SergeyB normal resolved none none fixed 0 FIXED: minor error in trfac.cpp unit An out-of-bounds pointer was created by incorrect array access in several functions of trfac.cpp unit (this error is specific for C++ version). This pointer wasn't used for reads or writes, so error went unnoticed.

However, when ALGLIB was compiled with AP_ASSERT symbol defined (usually it is turned off for performance reasons), an ap::ap_error exception was generated during attempt to do out-of-bounds operation with array (although this operation was just pointer generation - without actually using it).
311 Interpolation feature have not tried 2010-03-04 17:13 2010-03-04 17:13 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: scattered interpolation/approximation using inverse distance weighting Modified Shepard's algorithm was implemented The most important features are:
* Local interpolation (interpolant value depends on nearest neighbors only)
* Scalability: O(N*logN) construction time, O(logN) interpolation time
* Ability to process scattered datasets; no special distribution of nodes is required
* C1-continuity of interpolant (not guaranteed, but almost always)
310 Data analysis feature have not tried 2010-03-04 17:06 2010-03-04 17:06 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: nearest neighbor search using kd-trees Essential features are:
* O(N*logN) construction time, O(logN) query time
* several query types: k nearest neighbors, all neighbors within R-sphere, k approximate nearest neighbors
* several norm types: 1-norm, 2-norm, inf-norm
* ability to store both points themselves and labels: Y-values or integer tags
308 Linear algebra minor have not tried 2010-02-23 11:24 2010-02-23 11:24 SergeyB SergeyB normal resolved none none fixed 0 FIXED: minor bug in 'rotations' unit Division by zero when passing to generaterotation() numbers whose squares are less than underflow threshold.
306 Linear algebra feature have not tried 2010-02-22 17:38 2010-02-22 17:38 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: improved real/complex QR Improved real/complex QR:
* Level 3 cache oblivious code
* real/complex decompositions
297 Linear algebra feature have not tried 2010-02-10 17:27 2010-02-10 17:28 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: new matrix inversion subroutines New matrix inversion subroutines:
* real/complex/SPD/HPD matrices
* matrices may be given "as is" or by corresponding triangular factorization (LU/Cholesky)
* overflow checks
* built-in condition number estimation
* cache-oblivious code relying on Level 3 ALGLIB BLAS
292 Linear algebra feature have not tried 2010-02-05 15:52 2010-02-10 17:26 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: condition number estimation for triangular matrices Matrix types:
* upper or lower triangular
* triangular or unitriangular
293 Linear algebra crash always 2010-02-07 22:26 2010-02-08 10:38 wvd_vegt SergeyB normal resolved none none 2.3.0 no change required 0 SVD Algorithhm crashes on array bounds Hi,

When i try to run an SVD the routine crashes on an array index out of bounds in

Test method LsaTest.LsaTests.TestDecompose threw exception: System.IndexOutOfRangeException: De index ligt buiten de matrixgrenzen..

The error comes from the loop

                for(i_=1; i_<=nmip1;i_++)
                {
                    t[i_] = a[i,i_+i1_];
                }

where the i_ index runs from 0 to 25 (in the alglib svd test code) where the array is only 25 elements and start at 0 (so the index should run from 0..24).		 Callstack:

alglib.lq.lqdecomposition(Double[,]& a, Int32 m, Int32 n, Double[]& tau) in Alglib\lq.cs: line 285

alglib.svd.svddecomposition(Double[,] a, Int32 m, Int32 n, Int32 uneeded, Int32 vtneeded, Int32 additionalmemory, Double[]& w, Double[,]& u, Double[,]& vt) in
Alglib\svd.cs: line 517

--<rest is from my test probject.
Lsa.NET.Lsa.Decompose() in C:\Documents and Settings\veg\Mijn documenten\Visual Studio 2008\Projects\Lsa\lsa\Lsa.cs: line 942
LsaTest.LsaTests.TestDecompose() in C:\Documents and Settings\veg\Mijn documenten\Visual Studio 2008\Projects\Lsa\lsatest\LsaTests.cs: line 175


As you can see I compiled and ran a vs2k8 unit test (copied the code from the alglib test into it).
SergeyB 2010-02-08 10:38 It is not a bug.

alglib.svd.svddecomposition is an obsolete routine which uses old 1-based notation from LAPACK. It accepts matrix with [1..N,1..N] numeration, not with [0..N-1,0..N-1] indexes. It was left for backward compatibility, new users should use rmatrixsvd which uses modern 0-based array indexes.

P.S. looks like it is better to purge it from next release...
265 Optimization major have not tried 2010-01-22 11:33 2010-01-29 21:37 SergeyB SergeyB normal resolved none none fixed 0 FIXED: Initial step problem in L-BFGS When L-BFGS starts from the minimum vicinity, it may take step which is too large. Now starting step is limited from above.
231 Linear algebra feature have not tried 2009-12-15 10:49 2010-01-28 16:11 SergeyB SergeyB immediate resolved none none implemented 0 IMPLEMENTED: improved real/Hermitian Cholesky decomposition Improved Cholesky decomposition:
* uses ALGLIB BLAS (future improvements in ALGLIB BLAS will improve Cholesky performance)
* implemented in cache-oblivious manner: performance (measured in FLOPS) GROWS with matrix size; no tuning for specific CPU is required to obtain good performance.

Other changes:
* CHOLESKY and HCHOLESKY units are merged with TRFAC unit
232 Linear algebra feature have not tried 2009-12-15 10:52 2010-01-28 16:10 SergeyB SergeyB immediate resolved none none implemented 0 IMPLEMENTED: improved complex/real LU decomposition Improvements:
* new level-3 ALGLIB BLAS is used (future improvements in ALGLIB BLAS will increase LU performance)
* cache-oblivious algorithm allows to obtain good performance without tuning for specific CPU
* automatic scaling (to avoid overflow when dealing with near-overflow matrices)

Other changes:
* CLU and LU units are merged, all code moved to TRFAC unit
233 Linear algebra feature have not tried 2009-12-15 10:55 2010-01-28 00:07 SergeyB SergeyB immediate resolved none none implemented 0 IMPLEMENTED: improved linear solvers for real/complex/SPD/HPD systems Dense solvers for linear systems:
* general real, general complex, symmetric positive definite, Hermitian positive definite matrices
* system matrix may be given explicitly or by its triangular factorization (LU, Cholesky)
* one right part or multiple right parts
* iterative improvement
242 Linear algebra feature have not tried 2009-12-15 12:54 2010-01-25 22:22 SergeyB SergeyB immediate resolved none none implemented 0 IMPLEMENTED: ALGLIB BLAS Improved cache-oblivious level 3 BLAS with:
* real/complex transpositions
* real/complex rank-1 updates
* real/complex TRSM updates
* real/complex SYRK updates
* real/complex GEMM updates

C++ version contains improved base cases implementation (no low-level or processor-specific optimizations implemented yet, just algorithmic improvements and reduced OOP overhead).
241 Linear algebra feature have not tried 2009-12-15 12:18 2010-01-25 22:21 SergeyB SergeyB immediate resolved none none implemented 0 IMPLEMENTED: improved condition number estimation Improvements:
* new implementation of RCOND traverses matrices by rows (which is significantly faster than previous implementation)
* all operations are performed with original matrix when possible (no excessive matrix copying)
* new Hermitian condition number estimation code added
260 Optimization minor have not tried 2010-01-18 10:39 2010-01-18 10:40 SergeyB SergeyB normal resolved none none fixed 0 FIXED: bug in Levenberg-Marquardt optimizer Hybrid Jacobian-based algorithm sometimes failed on highly nonlinear functions due to incorrect assertion.
245 Linear algebra tweak have not tried 2009-12-15 20:01 2009-12-15 20:01 SergeyB SergeyB normal resolved none none fixed 0 FIXED: minor tweak in random matrix generator Random matrix generator returns Hermitian matrix with small but non-zero imaginary components of diagonal elements. These non-zero elements appear due to floating point roundoff. Sometimes it is acceptable, sometimes it is not.

Now matrices returned by HPDMatrixRndCond/HMatrixRndCond have strictly real diagonal elements
135 General feature have not tried 2009-09-09 14:45 2009-12-14 10:48 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: script-based compilation/testing system Set of scripts:
* 'build' - ALGLIB compilation with different compilers
* 'check' - execution of unit tests
* 'example' - execution of examples

Each script is provided in two versions with identical functionality: batch file for Windows users and Bash script for *nix users.
105 Linear algebra feature N/A 2009-08-14 15:55 2009-12-14 10:48 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: new solver for real linear systems Dense solver for real linear systems:
* solves task in usual sense and in the least-squares sense
* iterative improvement allows to solve system with very high precision
103 Interpolation feature N/A 2009-08-10 17:13 2009-12-14 10:42 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: new interface for 2-dimensional splines Changes/features are mostly cosmetic - new interface is more consistent with the rest of the ALGLIB.
194 Interpolation feature have not tried 2009-11-12 23:18 2009-12-14 10:40 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: polynomial interpolation/fitting New functionality for polynomials in barycentric form:
* construction from values on general and special (equidistant, Chebyshev) grids
* unconstrained and constrained fitting
* same functions as for rational functions (calculation, differentiation, ...)
205 General minor have not tried 2009-11-27 11:18 2009-12-14 10:37 SergeyB SergeyB normal resolved none none fixed 0 FIXED: non-IEEE floating point comparison in C++, C#, Delphi, FreePascal x86 FPU uses 80-bit floating point arithmetics for internal calculations, while in-memory data are stored in 64-bit precision.

Sometimes it leads to unexpected results, for example: you may assign sqrt(3) to A, and still get A<>sqrt(3) (just because A is stored in memory and has double precision, and second sqrt(3) is stored in FPU and has long double precision).

Several unit tests were (erroneously) failed due to floating point inconsistency.

Now all sources use IEEE-compliant floating point comparisons independently of compiler/platform.
223 Linear algebra feature have not tried 2009-12-09 17:59 2009-12-09 17:59 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: random matrix generator Random matrix generator.

Supports:
* generation of random real, random complex, random symmetric, random Hermitian, random SPD, random HPD matrices with prescribed condition number
* generation of random orthogonal real/complex matrices
* multiplication from the left/right by random orthogonal real/complex matrices
* symmetric/Hermitian multiplication by random orthogonal real/complex matrices
212 Statistics feature have not tried 2009-12-02 11:40 2009-12-02 11:40 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: high-quality random number generator High-quality random number generator based on code by Pierre L'Ecuyer ('Efficient and Portable Combined Random Number Generators', Communications of the ACM, 31 (1988), 742--749 and 774, http://www.iro.umontreal.ca/~lecuyer/papers.html).

HQRND unit supports:
1. uniform real and integer random numbers (basic building blocks)
2. normal and exponential distributions
204 Spec.functions major have not tried 2009-11-25 22:34 2009-11-25 22:35 SergeyB SergeyB normal resolved none none implemented 0 CHANGED: 'gammaf' unit renamed to 'gammafunc' This fix is due to name conflict between:
* gammaf namespace in multiple precision ALGLIB, and
* gammaf function in mathcalls.h, 32/64 GCC.

This conflict became apparent when new unit testing infrastructure with both Windows and Linux tests began to roll out.

Fix is backward-incompatible (C# users will notice that class name is changed, under C++/Pascal unit name will be changed). However, we have no choice...
198 General tweak have not tried 2009-11-18 15:55 2009-11-18 15:56 SergeyB SergeyB normal resolved none none fixed 0 FIXED: error reports in multiple precision reflections.cpp MPFR does not support subnormal numbers (like IEEE arithmetics do). However, several units are being tested for ability to correctly handle subnormals. These tests were successfully passed in IEEE arithmetics, but sometimes they were failed in MPFR arithmetics - just because of absence of subnormals.

getAlgoPascalMaxNumber()/getAlgoPascalMinNumber() were tweaked (increased by getAlgoPascalEpsilon() factor) to avoid erroneous reports in such cases.
153 General major have not tried 2009-10-13 23:14 2009-10-13 23:17 SergeyB SergeyB normal resolved none none fixed 0 FIXED: compiler error under GCC ap.cpp was successfully compiled under MinGW and other Windows compilers, but there were problems with original GCC under Linux - several standard functions like strlen and strcmp required additional #include's.

Due to ease of fixing this error went unreported until recent message from Chong Luo:

> When I compile the c++ version of svd algorithm using
> "g++ -c *.cpp", I get the following errors:
> ap.cpp:432: error: 'strlen' was not declared in this scope
> ap.cpp:438: error: 'strlen' was not declared in this scope
> ap.cpp:455: error: 'strncmp' was not declared in this scope

However, with new unit testing infrastructure rolled out (next ALGLIB release will be tested under 32/64-bit Windows and Linux systems) such errors shouldn't happen again.
147 Linear algebra minor have not tried 2009-09-29 20:45 2009-10-13 09:51 SergeyB SergeyB normal resolved none none fixed 0 FIXED: incorrect handling of near-undeflow/subnormals in GenerateReflection() From: Bela Ujhazi

> I'm a bit stucked up with the SVD lib in ALGLIB,
> I used it for some matrices before, but now I've
> had to use it for some bigger ones (one is
> approx. 3000*10000, second is 950*4500) and the
> results were not satisfying. All the values in
> the 2 mentioned matrices were '1.#QNAN' unlike
> from the little ones (700*1300 was the biggest)
> I tried before.
SergeyB 2009-10-05 22:55 Looks like this error is caused by bug in the GenerateReflection() subroutine - it does not correctly handle subnormalized input vectors.

This bug is not specific to the SVD lib - it can be found almost in any subroutine which relies on GenerateReflection(). There are two ways to trigger this bug:
1. you can trigger it with a very large degenerate matrix
2. or you can trigger it with a moderate size matrix with near-underflow or subnormal elements

Now investigating possible workarounds....

SergeyB 2009-10-13 09:49 Fixed. Separate release will be issued for this patch in 5 days.
144 Data analysis major always 2009-09-21 20:54 2009-09-21 22:30 SergeyB SergeyB normal resolved none none 2.1.0 fixed 0 FIXED: bug in mlpbase unit "TanH" activation function was incorrectly calculated for arguments less than -100 (it was +1 instead of -1). This error was unlikely to trigger because of inputs normalization and task regularization, which make hard to achive such low inputs during normal conditions.

This bug was introduced in the ALGLIB 2.1.0.

Thanks to Andrei Rizhkovich for pointing out this bug.
130 Interpolation feature N/A 2009-09-09 12:41 2009-09-09 12:47 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: improved 1-dimensional splines Changes/features include:
1. New interface (more consistent with the rest of the ALGLIB)
2. Construction of linear/natural/Hermite/Akima splines (interpolation)
3. Least squares fitting by natural/Hermite splines, with/without constraints on spline values/derivatives
4. Evaluation/differentiation/integration of spline interpolant
5. Linear operations with spline
122 Diff.equations feature N/A 2009-09-04 12:36 2009-09-04 12:37 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: adaptive ODE solver New adaptive ODE solver based on Cash-Karp formula.
121 General minor always 2009-09-04 00:16 2009-09-04 00:16 SergeyB SergeyB normal resolved none none fixed 0 FIXED: miscompilation of ap.cpp under BCB 6 #include <stdio.h> was required for successful compilation of ap.cpp (MSVC, GCC, BCB 5 were satisfied with just stdlib.h, but BCB 6 was not).
92 Fast transforms feature N/A 2009-07-24 12:59 2009-08-30 19:57 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: fast complex cross-corrrelation Circular and non-circular fast complex cross-corrrelation.

FFT-based algorithm with O(N*logN) complexity for any N (composite or prime) is used to calculate general length corrrelations. Two important special cases are handled differently:
* when response is very short - straightforward O(M*N) formula is used
* when responce is relatively long, but significantly sorter than signal - overlap-save algorithm is used.
93 Fast transforms feature N/A 2009-07-24 13:00 2009-08-30 19:57 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: fast real cross-corrrelation Circular and non-circular fast real cross-corrrelation.

FFT-based algorithm with O(N*logN) complexity for any N (composite or prime) is used to calculate general length corrrelations. Two important special cases are handled differently:
* when response is very short - straightforward O(M*N) formula is used
* when responce is relatively long, but significantly sorter than signal - overlap-save algorithm is used.
108 Data analysis minor always 2009-08-22 20:20 2009-08-22 20:21 SergeyB SergeyB normal resolved none none fixed 0 FIXED: overflows in neural networks under some compilers Neural networks in 2.0.1 use tanh as activation function. Some compilers (Delphi 7, for example) have buggy standard libraries that can't calculate TanH of large arguments because of overflow (although final result is "1"). So if neural network is provided with inputs that are too large, such network will crash during training. Strictly speaking, it is Delphi bug, but we'll fix it anyway :)

This problem is fixed by using special code for inputs that are larger than 100.
107 Interpolation feature N/A 2009-08-19 12:44 2009-08-19 12:44 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: rational interpolation/fitting New functionality for rational functions in barycentric form:
* construction from barycentric coefficients
* calculation
* differentiation of barycentric interpolant (first/second derivatives)
* linear translation of argument/function
* rational interpolation using Floater-Hormann algorithm (interpolation without real poles)
* rational fitting using Floater-Hormann basis (rational fitting without real poles)
106 Interpolation feature N/A 2009-08-17 15:41 2009-08-17 15:45 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: Improved linear least squares fitting Improved linear least squares fitting subroutines:
* separate subroutines for tasks without weights and with individual weights
* improved solver - fast QR-based solver for well-conditioned tasks, robust SVD-based solver for ill-conditioned tasks
* automatic solver selection depending on condition number
99 Interpolation feature N/A 2009-07-30 09:46 2009-08-17 15:45 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: Nonlinear least squares fitting using Levenberg-Marquardt algorithm Nonlinear fitting subroutine. Three fitting modes are supported:
* original Levenberg-Marquardt algorithm using function gradient only
* improved Levenberg-Marquardt algorithm for function with cheap O(M) gradients (less matrix inversions, more gradient calculations)
* Hessian-based improved Levenberg-Marquardt algorithm
49 General block N/A 2009-05-01 00:12 2009-07-29 12:52 SergeyB SergeyB normal resolved none none fixed 0 FIXED: MPFR-version bugs Bugs which were not noticed until recent time because of automated testing system failure. Mostly miscompilation of some units.
52 Fast transforms feature N/A 2009-05-18 12:21 2009-07-27 14:22 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: complex FFT for general N (composite/prime) Transforms implemented:
* forward, complex
* backward, complex

N's supported - arbitrary:
* composite N's are handled with Cooley-Tukey algorithm
* small prime-factors are handled with built-in codelets
* large prime-factors are handled with Bluestein's algorithm

Complexity: N*log(N) for composite/prime N's.
61 Fast transforms feature N/A 2009-05-29 14:31 2009-07-27 14:22 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: real FFT for general N (composite/prime) Transforms implemented:
* forward, real
* backward, real

N's supported - arbitrary:
* even-sized tasks are solved using efficient reduction to complex FFT with factor of two savings
* odd-sized tasks are solved using straightforward reduction to complex FFT (factor of two inefficient)

Complexity: N*log(N) for composite/prime N's
91 Fast transforms feature N/A 2009-07-24 12:52 2009-07-27 14:20 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: Fast real convolution Circular and non-circular fast real convolution.

Transforms implemented:
* forward (convolution)
* backward (deconvolution)

FFT-based algorithm with O(N*logN) complexity for any N (composite or prime) is used to calculate general length convolutions. Two important special cases are handled differently:
* when response is very short - straightforward O(M*N) formula is used
* when responce is relatively long, but significantly sorter than signal - overlap-add algorithm is used.
53 Fast transforms feature N/A 2009-05-18 12:22 2009-07-27 14:20 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: Fast complex convolution Circular and non-circular fast complex convolution.

Transforms implemented:
* forward (convolution)
* backward (deconvolution)

FFT-based algorithm with O(N*logN) complexity for any N (composite or prime) is used to calculate general length convolutions. Two important special cases are handled differently:
* when response is very short - straightforward O(M*N) formula is used
* when responce is relatively long, but significantly sorter than signal - overlap-add algorithm is used.
35 Integration feature N/A 2009-04-07 11:58 2009-07-23 12:46 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: adaptive integrator Adaptive integrator for smooth functions and functions with integrable singularities at the ends of the interval. Reverse communication interface.
10 Spec.functions major always 2009-03-10 21:02 2009-07-23 12:42 SergeyB SergeyB normal resolved none none fixed 0 FIXED: modified Besel function returns incorrect results From: Kors van der Werf < korsvanderwerf@yahoo.com >

Hi,

Very nice, ythis ALGLib. I use the VB version for now, and it seems the Modified BesselI funtion for orders of negative half integers do not give proper results, can you have a look?

Already order = -0.5 fails, and can be compared to the well known sqrt(2/(PI x)) cosh(x)

For x=1 it should be 1.2312, I get 0.293525.

Could you ahve a look please.
SergeyB 2009-03-11 10:21 Confirmed: should be 1.2312, returns 0.2935
SergeyB 2009-03-11 12:15 Fixed.

There was an error in original Cephes code when calculating 1F1(v,2*v,2*x) with v=0. Straightforward formula for 1F1 may return 1.0 or NAN (depending on implementation, 1.0 in Cephes), more elaborate calculations gives us 0.5*(1+exp(2*x)).

This error can only happen when v=-0.5, should be OK with other values of v.

SergeyB 2009-03-11 23:36 Other negative half-integer values should be fixed also...
SergeyB 2009-04-19 11:07 Looks like that this subroutine can't calculate modified Bessel functions of negative orders. It is not a bug - it is a limitation of formula used.

I've modified documentation...
77 General major always 2009-07-06 16:47 2009-07-23 12:41 hamlet77 SergeyB normal resolved none none 2.0 fixed 0 FIXED: Additional ; after namespace in ap.h When compiling with
g++ -ansi -pedantic -Wall
additional ; after closing brace } on line 656 causes an error.
75 General major sometimes 2009-06-29 11:43 2009-07-23 12:41 SergeyB SergeyB normal resolved none none fixed 0 FIXED: Uninitialized locals and reverse communication interface Description: saving uninitialized locals between reverse communication calls leads to compilation errors/warnings under some compilers.

Solution: locals in RCOMM subroutines are initialized by values determined during source code generation phase.

Benefits: higher performance (in comparison with initialization by random values), better detectability of bugs related to unititialized variables (in comparison with initialization by zeros) - "new unit test" = "new code generation" = "new initial values".

Drawbacks: source code changes with each release.
51 General feature have not tried 2009-05-01 00:23 2009-07-23 12:41 SergeyB SergeyB normal resolved none none fixed 0 FIXED: VBA math functions 1. More accurate Pi()
2. Overflow-safe TanH
50 General minor N/A 2009-05-01 00:17 2009-07-23 12:41 SergeyB SergeyB normal resolved none none fixed 0 FIXED: non thread safe random number generator in C# C# random number generator (in ap.cs) wasn't thread safe. Now it is (using lock statement).
27 General feature have not tried 2009-03-21 23:37 2009-07-23 12:41 SergeyB SergeyB normal resolved none none fixed 0 FIXED: Improved real RNG in AP/C++ Improved real random number generator in C++ version of AP library. Numbers returned are more fine-grained now.
79 Linear algebra trivial always 2009-07-06 17:03 2009-07-23 12:40 hamlet77 SergeyB normal resolved none none 2.0 fixed 0 FIXED: unused variables in hsschur unit ../src/capd/alglib/hsschur.cpp:177: warning: unused variable ‘p3’
../src/capd/alglib/hsschur.cpp:178: warning: unused variable ‘p4’
78 Linear algebra trivial always 2009-07-06 16:57 2009-07-23 12:40 hamlet77 SergeyB normal resolved none none 2.0 fixed 0 FIXED: unused variables in 'hessenberg' unit In function ‘void rmatrixhessenberg(ap::real_2d_array&, int, ap::real_1d_array&)’:
unused variable ‘aii’
                                                                
In function ‘void rmatrixhessenbergunpackh(const ap::real_2d_array&, int, ap::real_2d_array&)’:
unused variables ‘ip1’, ‘nmi’

In function ‘void toupperhessenberg(ap::real_2d_array&, int, ap::real_1d_array&)’: unused variable ‘aii’
                                                               
In function ‘void unpackhfromupperhessenberg(const ap::real_2d_array&, int, const ap::real_1d_array&, ap::real_2d_array&)’: unused variable ‘ip1’, ‘nmi’
Compiler log:
../src/capd/alglib/hessenberg.cpp: In function ‘void rmatrixhessenberg(ap::real_2d_array&, int, ap::real_1d_array&)’:
../src/capd/alglib/hessenberg.cpp:79: warning: unused variable ‘aii’
../src/capd/alglib/hessenberg.cpp: In function ‘void rmatrixhessenbergunpackh(const ap::real_2d_array&, int, ap::real_2d_array&)’:
../src/capd/alglib/hessenberg.cpp:210: warning: unused variable ‘ip1’
../src/capd/alglib/hessenberg.cpp:211: warning: unused variable ‘nmi’
../src/capd/alglib/hessenberg.cpp: In function ‘void toupperhessenberg(ap::real_2d_array&, int, ap::real_1d_array&)’:
../src/capd/alglib/hessenberg.cpp:239: warning: unused variable ‘aii’
../src/capd/alglib/hessenberg.cpp: In function ‘void unpackhfromupperhessenberg(const ap::real_2d_array&, int, const ap::real_1d_array&, ap::real_2d_array&)’:
../src/capd/alglib/hessenberg.cpp:356: warning: unused variable ‘ip1’
../src/capd/alglib/hessenberg.cpp:357: warning: unused variable ‘nmi’
25 Linear algebra minor have not tried 2009-03-21 00:29 2009-07-23 12:40 SergeyB SergeyB normal resolved none none fixed 0 FIXED: Inefficient Cholesky decomposition Current implementation uses column operations in its inner cycle. It is terribly inefficient.
56 Statistics minor always 2009-05-21 21:26 2009-07-23 12:40 SergeyB SergeyB normal resolved none none fixed 0 FIXED: minor bug in WilcoxonSignedRankTest() subroutine Description: incorrect p-values were calculated for samples with tied values (like 0.11, 0.12, 0.13, 0.13, 0.13, 0.14, 0.15).

Thanks to Paolo Castiglioni for reporting this issue.
87 Integration feature N/A 2009-07-20 17:07 2009-07-23 12:39 SergeyB SergeyB normal resolved none none implemented 0 IMPLEMENTED: more reliable Gauss and Gauss-Kronrod quadrature generators More reliable Gauss quadrature generators (Legendre, Jacobi, Hermite, Laguerre weight functions, general weight function given by three-term recurrence relation). Gauss-Kronrod quadrature generators.
72 Interpolation tweak always 2009-06-12 12:57 2009-07-23 12:38 SergeyB SergeyB