R Chalapathy, AK Menon, S Chawla - arXiv preprint arXiv:1802.06360, 2018 - arxiv.org

Paper link: https://arxiv.org/pdf/1802.06360.pdf

One-Class Neural Network is a unsupervised anomaly detection model, based on One-Class SVM. OC-NN paper shows how the OC-SVM can be replaced with the standard neural network's internal representation as its kernel function. OC-NN also uses autoencoder to encode the features into most salient representation and then feeding it on a simple feed-forward neural network with one class. The loss function used is different from the standard cross-entropy or similar one, here the loss function is inspired from the one-class-svm.

The ON-SVM objective function try to map the feature vectors to higher dimensional space and separate them from "origin" as much as possible. The origin act as positive class and all other training samples act as negative sample. OC-SVM's objective function identifies maximum separation point between origin and all data-points. We have set of hyper parameters in objective function which can be used to tune the specificity and sensitivity of the model.
The paper claims the results it produce beats the one-class-svm and also
current state-of-the art methods.

Model architecture:-

Data => auto-encoder

auto-encoder(encoded layer) => oc-nn
@article{chalapathy2018anomaly,
  title={Anomaly Detection using One-Class Neural Networks},
  author={Chalapathy, Raghavendra and Menon, Aditya Krishna and Chawla, Sanjay},
  journal={arXiv preprint arXiv:1802.06360 },
  year={2018}
}

Reference implementation on Keras/Tensorflow