|
Mantis - ALGLIB
|
|||||
| Viewing Issue Advanced Details | |||||
|
|
|||||
| ID: | Category: | Severity: | Reproducibility: | Date Submitted: | Last Update: |
| 525 | Data analysis | feature | have not tried | 2013-06-20 12:24 | 2014-06-03 16:31 |
|
|
|||||
| Reporter: | SergeyB | Platform: | |||
| Assigned To: | SergeyB | OS: | |||
| Priority: | normal | OS Version: | |||
| Status: | assigned | Product Version: | |||
| Product Build: | Resolution: | open | |||
| Projection: | none | ||||
| ETA: | none | Fixed in Version: | |||
| Programming language: | Unspecified | ||||
|
|
|||||
| Summary: | 0000525: Neural network improvements | ||||
| Description: |
Neural structure: * layered * complex interactions between layers * several activation functions: tanh(), tanh()+linear, fast sigmoid SGD: * algorithm without learning rates: http://yann.lecun.com/exdb/publis/pdf/schaul-icml-13.pdf * page 72 of http://learning.stat.purdue.edu/mlss/_media/mlss/bottou.pdf - important trick * important info on BP acceleration http://yann.lecun.com/exdb/publis/pdf/lecun-98b.pdf * ADADELTA seems to be best method - http://www.matthewzeiler.com/pubs/googleTR2012/googleTR2012.pdf * ADAGRAD??? * "On the importance of initialization and momentum in deep learning", http://jmlr.org/proceedings/papers/v28/sutskever13.pdf Improvements: * shortcut layer, see "Deep Learning Made Easier by Linear Transformations in Perceptrons". Maybe - pre-training linear layer separately. Decide on: * minibatch training * bagging for ensembles * parallel errors for ensembles * sparse errors for ensembles * subset errors for ensembles * decay in ensemble training? * investigate ensemble tendency to overfit on GLASS dataset * mini-batch LBFGS training * approximate Hessian preconditioning * FindBestDecay * FindBestNetwork * "ensemble selection", better way of constructing ensemble * model compression * sparse autoencoders? * stacked autoencoders/autodecoders? Convolutional Neural Networks (weight sharing = constraints and projections): * http://yann.lecun.com/exdb/publis/pdf/lecun-98.pdf |
||||
| Steps To Reproduce: | |||||
| Additional Information: | |||||
| Relationships | |||||
| Attached Files: | |||||
|
|
|||||
| Issue History | |||||
| Date Modified | Username | Field | Change | ||
| 2013-06-20 12:24 | SergeyB | New Issue | |||
| 2013-06-20 12:24 | SergeyB | Status | new => assigned | ||
| 2013-06-20 12:24 | SergeyB | Assigned To | => SergeyB | ||
| 2013-06-20 12:24 | SergeyB | Programming language | => Unspecified | ||
| 2013-06-20 13:59 | SergeyB | Description Updated | |||
| 2013-06-21 10:42 | SergeyB | Description Updated | |||
| 2013-06-22 14:40 | SergeyB | Description Updated | |||
| 2013-06-22 14:44 | SergeyB | Description Updated | |||
| 2013-06-22 14:56 | SergeyB | Description Updated | |||
| 2013-06-22 16:23 | SergeyB | Description Updated | |||
| 2013-06-22 17:09 | SergeyB | Description Updated | |||
| 2013-06-23 23:44 | SergeyB | Description Updated | |||
| 2013-06-26 14:46 | SergeyB | Description Updated | |||
| 2013-06-26 15:12 | SergeyB | Description Updated | |||
| 2013-06-27 14:19 | SergeyB | Description Updated | |||
| 2013-06-30 13:42 | SergeyB | Issue Monitored: SergeyB | |||
| 2013-06-30 13:42 | SergeyB | Issue End Monitor: SergeyB | |||
| 2013-06-30 13:42 | SergeyB | Description Updated | |||
| 2013-07-02 16:42 | SergeyB | Description Updated | |||
| 2013-07-04 15:20 | SergeyB | Description Updated | |||
| 2013-07-08 12:50 | SergeyB | Description Updated | |||
| 2013-10-05 15:04 | SergeyB | Target Version | Next release => Next 'Data mining' release | ||
| 2014-03-21 09:46 | SergeyB | Description Updated | |||
| 2014-03-30 11:06 | SergeyB | Description Updated | |||
| 2014-05-10 11:25 | SergeyB | Description Updated | |||
| 2014-06-03 12:23 | SergeyB | Description Updated | |||
| 2014-06-03 16:24 | SergeyB | Description Updated | |||
| 2014-06-03 16:31 | SergeyB | Description Updated | |||
| There are no notes attached to this issue. |