Group Anomaly Detection using Deep Generative Models

Unlike conventional anomaly detection research that focuses on point anomalies, our goal is to detect anomalous collections of individual data points. In particular, we perform group anomaly detection (GAD) with an emphasis on irregular group distributions (e.g. irregular mixtures of image pixels). GAD is an important task in detecting unusual and anomalous phenomena in real-world applications such as high energy particle physics, social media, and medical imaging. In this paper, we take a generative approach by proposing deep generative models: Adversarial autoencoder (AAE) and variational autoencoder (VAE) for group anomaly detection. Both AAE and VAE detect group anomalies using point-wise input data where group memberships are known a priori. We conduct extensive experiments to evaluate our models on real-world datasets. The empirical results demonstrate that our approach is effective and robust in detecting group anomalies.

μ-cuDNN: Accelerating Deep Learning Frameworks with Micro-Batching

NVIDIA cuDNN is a low-level library that provides GPU kernels frequently used in deep learning. Specifically, cuDNN implements several equivalent convolution algorithms, whose performance and memory footprint may vary considerably, depending on the layer dimensions. When an algorithm is automatically selected by cuDNN, the decision is performed on a per-layer basis, and thus it often resorts to slower algorithms that fit the workspace size constraints. We present {\mu}-cuDNN, a transparent wrapper library for cuDNN, which divides layers’ mini-batch computation into several micro-batches. Based on Dynamic Programming and Integer Linear Programming, {\mu}-cuDNN enables faster algorithms by decreasing the workspace requirements. At the same time, {\mu}-cuDNN keeps the computational semantics unchanged, so that it decouples statistical efficiency from the hardware efficiency safely. We demonstrate the effectiveness of {\mu}-cuDNN over two frameworks, Caffe and TensorFlow, achieving speedups of 1.63x for AlexNet and 1.21x for ResNet-18 on P100-SXM2 GPU. These results indicate that using micro-batches can seamlessly increase the performance of deep learning, while maintaining the same memory footprint.

Asynchronous Parallel Sampling Gradient Boosting Decision Tree

With the development of big data technology, Gradient Boosting Decision Tree, i.e. GBDT, becomes one of the most important machine learning algorithms for its accurate output. However, the training process of GBDT needs a lot of computational resources and time. In order to accelerate the training process of GBDT, the asynchronous parallel sampling gradient boosting decision tree, abbr. asynch-SGBDT is proposed in this paper. Via introducing sampling, we adapt the numerical optimization process of traditional GBDT training process into stochastic optimization process and use asynchronous parallel stochastic gradient descent to accelerate the GBDT training process. Meanwhile, the theoretical analysis of asynch-SGBDT is provided by us in this paper. Experimental results show that GBDT training process could be accelerated by asynch-SGBDT. Our asynchronous parallel strategy achieves an almost linear speedup, especially for high-dimensional sparse datasets.

A Scalable Shared-Memory Parallel Simplex for Large-Scale Linear Programming

We present a shared-memory parallel implementation of the Simplex tableau algorithm for dense large-scale Linear Programming (LP) problems. We present the general scheme and explain each parallelization step of the standard simplex algorithm, emphasizing important solutions for solving performance bottlenecks. We analyzed the speedup and the parallel efficiency for the proposed implementation relative to the standard Simplex algorithm using a shared-memory system with 64 processing cores. The experiments were performed for several different problems, with up to 8192 variables and constraints, in their primal and dual formulations. The results show that the performance is mostly much better when we use the formulation with more variables than inequality constraints. Also, they show that the parallelization strategies applied to avoid bottlenecks caused the implementation to scale well with the problem size and the core count up to a certain limit of problem size. Further analysis showed that this was an effect of resource limitation. Even though, our implementation was able to reach speedups in the order of 19x.

Efficient algorithms for tensor scaling, quantum marginals and moment polytopes

We present a polynomial time algorithm to approximately scale tensors of any format to arbitrary prescribed marginals (whenever possible). This unifies and generalizes a sequence of past works on matrix, operator and tensor scaling. Our algorithm provides an efficient weak membership oracle for the associated moment polytopes, an important family of implicitly-defined convex polytopes with exponentially many facets and a wide range of applications. These include the entanglement polytopes from quantum information theory (in particular, we obtain an efficient solution to the notorious one-body quantum marginal problem) and the Kronecker polytopes from representation theory (which capture the asymptotic support of Kronecker coefficients). Our algorithm can be applied to succinct descriptions of the input tensor whenever the marginals can be efficiently computed, as in the important case of matrix product states or tensor-train decompositions, widely used in computational physics and numerical mathematics. We strengthen and generalize the alternating minimization approach of previous papers by introducing the theory of highest weight vectors from representation theory into the numerical optimization framework. We show that highest weight vectors are natural potential functions for scaling algorithms and prove new bounds on their evaluations to obtain polynomial-time convergence. Our techniques are general and we believe that they will be instrumental to obtain efficient algorithms for moment polytopes beyond the ones consider here, and more broadly, for other optimization problems possessing natural symmetries.

Distribution Regression Network

We introduce our Distribution Regression Network (DRN) which performs regression from input probability distributions to output probability distributions. Compared to existing methods, DRN learns with fewer model parameters and easily extends to multiple input and multiple output distributions. On synthetic and real-world datasets, DRN performs similarly or better than the state-of-the-art. Furthermore, DRN generalizes the conventional multilayer perceptron (MLP). In the framework of MLP, each node encodes a real number, whereas in DRN, each node encodes a probability distribution.

Understanding Community Structure in Layered Neural Networks

A layered neural network is now one of the most common choices for the prediction of high-dimensional practical data sets, where the relationship between input and output data is complex and cannot be represented well by simple conventional models. Its effectiveness is shown in various tasks, however, the lack of interpretability of the trained result by a layered neural network has limited its application area. In our previous studies, we proposed methods for extracting a simplified global structure of a trained layered neural network by classifying the units into communities according to their connection patterns with adjacent layers. These methods provided us with knowledge about the strength of the relationship between communities from the existence of bundled connections, which are determined by threshold processing of the connection ratio between pairs of communities. However, it has been difficult to understand the role of each community quantitatively by observing the modular structure. We could only know to which sets of the input and output dimensions each community was mainly connected, by tracing the bundled connections from the community to the input and output layers. Another problem is that the finally obtained modular structure is changed greatly depending on the setting of the threshold hyperparameter used for determining bundled connections. In this paper, we propose a new method for interpreting quantitatively the role of each community in inference, by defining the effect of each input dimension on a community, and the effect of a community on each output dimension. We show experimentally that our proposed method can reveal the role of each part of a layered neural network by applying the neural networks to three types of data sets, extracting communities from the trained network, and applying the proposed method to the community structure.

Adversarial Clustering: A Grid Based Clustering Algorithm Against Active Adversaries

Nowadays more and more data are gathered for detecting and preventing cyber attacks. In cyber security applications, data analytics techniques have to deal with active adversaries that try to deceive the data analytics models and avoid being detected. The existence of such adversarial behavior motivates the development of robust and resilient adversarial learning techniques for various tasks. Most of the previous work focused on adversarial classification techniques, which assumed the existence of a reasonably large amount of carefully labeled data instances. However, in practice, labeling the data instances often requires costly and time-consuming human expertise and becomes a significant bottleneck. Meanwhile, a large number of unlabeled instances can also be used to understand the adversaries’ behavior. To address the above mentioned challenges, in this paper, we develop a novel grid based adversarial clustering algorithm. Our adversarial clustering algorithm is able to identify the core normal regions, and to draw defensive walls around the centers of the normal objects utilizing game theoretic ideas. Our algorithm also identifies sub-clusters of attack objects, the overlapping areas within clusters, and outliers which may be potential anomalies.

Runge-Kutta Theory and Constraint Programming

There exist many Runge-Kutta methods (explicit or implicit), more or less adapted to specific problems. Some of them have interesting properties, such as stability for stiff problems or symplectic capability for problems with energy conservation. Defining a new method suitable to a given problem has become a challenge. The size, the complexity and the order do not stop growing. This informal challenge to implement the best method is interesting but an important unsolved problem persists. Indeed, the coefficients of Runge-Kutta methods are harder and harder to compute, and the result is often expressed in floating-point numbers, which may lead to erroneous integration schemes. Here, we propose to use interval analysis tools to compute Runge-Kutta coefficients. In particular, we use a solver based on guaranteed constraint programming. Moreover, with a global optimization process and a well chosen cost function, we propose a way to define some novel optimal Runge-Kutta methods.

The unreasonable effectiveness of the forget gate

Given the success of the gated recurrent unit, a natural question is whether all the gates of the long short-term memory (LSTM) network are necessary. Previous research has shown that the forget gate is one of the most important gates in the LSTM. Here we show that a forget-gate-only version of the LSTM with chrono-initialized biases, not only provides computational savings but outperforms the standard LSTM on multiple benchmark datasets and competes with some of the best contemporary models. Our proposed network, the JANET, achieves accuracies of 99% and 92.5% on the MNIST and pMNIST datasets, outperforming the standard LSTM which yields accuracies of 98.5% and 91%.

Monitoring and Executing Workflows in Linked Data Environments

The W3C’s Web of Things working group is aimed at addressing the interoperability problem on the Internet of Things using Linked Data as uniform interface. While Linked Data paves the way towards combining such devices into integrated applications, traditional solutions for specifying the control flow of applications do not work seamlessly with Linked Data. We therefore tackle the problem of the specification, execution, and monitoring of applications in the context of Linked Data. We present a novel approach that combines workflows, semantic reasoning, and RESTful interaction into one integrated solution. We contribute to the state of the art by (1) defining an ontology for describing workflow models and instances, (2) providing operational semantics for the ontology that allows for the execution and monitoring of workflow instances, (3) presenting a benchmark to evaluate our solution. Moreover, we showcase how we used the ontology and the operational semantics to monitor pilots executing workflows in virtual aircraft cockpits.

Geometric Consistency for Self-Supervised End-to-End Visual Odometry
DeepFM: An End-to-End Wide & Deep Learning Framework for CTR Prediction
3D G-CNNs for Pulmonary Nodule Detection
An Approximation to Proof of the Circulant Hadamard Conjecture
Connectivity in Random Annulus Graphs and the Geometric Block Model
Fast approaches for Bayesian estimation of size of hard-to-reach populations using Network Scale-up
Bounds on the largest Kronecker and induced multiplicities of finite groups
Cross-Domain Visual Recognition via Domain Adaptive Dictionary Learning
Topological data analysis and diagnostics of compressible MHD turbulence
On the largest Kronecker and Littlewood–Richardson coefficients
A Variational U-Net for Conditional Appearance and Shape Generation
Efficient Model Identification for Tensegrity Locomotion
Stein kernels and moment maps
Representations of monomial matrices and restriction from $GL_n$ to $S_n$
Fast Prefix Search in Little Space, with Applications
Network-based protein structural classification
Jet Lag Recovery: Synchronization of Circadian Oscillators as a Mean Field Game
MPSM: Multi-prospective PaaS Security Model
Multimodal Unsupervised Image-to-Image Translation
Large deviations for fast transport stochastic RDEs with applications to the exit problem
Joint Estimation of Topology \& Injection Statistics in Distribution Grids with Missing Nodes
Analytically Modeling Unmanaged Intersections with Microscopic Vehicle Interactions
Per-Corpus Configuration of Topic Modelling for GitHub and Stack Overflow Collections
Machine Learning Peeling and Loss Modelling of Time-Domain Reflectometry
MOVI: A Model-Free Approach to Dynamic Fleet Management
RIPEx: Extracting malicious IP addresses from security forums using cross-forum learning
On the Efficiency of Localized Work Stealing
Optimizing Bloom Filter: Challenges, Solutions, and Comparisons
A Hybrid Model for Identity Obfuscation by Face Replacement
FishEyeRecNet: A Multi-Context Collaborative Deep Network for Fisheye Image Rectification
Deep Motion Boundary Detection
Talking Face Generation by Conditional Recurrent Adversarial Network
Unavoidable Subtournaments in Tournaments with Large Chromatic Number
Successful Nash Equilibrium Agent for a 3-Player Imperfect-Information Game
Fast, Parameter free Outlier Identification for Robust PCA
Precise Temporal Action Localization by Evolving Temporal Proposals
Learning Deep Sketch Abstraction
MSnet: Mutual Suppression Network for Disentangled Video Representations
Offline and Online calibration of Mobile Robot and SLAM Device for Navigation
Cooperative Strategies for {UAV}-Enabled Small Cell Networks Sharing Unlicensed Spectrum
Spline Error Weighting for Robust Visual-Inertial Fusion
On analogues of C.R.Rao’s theorems for locally compact Abelian groups
Non-binary Code Correcting Single b-Burst of Insertions or Deletions
On Deep Learning-based Massive MIMO Indoor User Localization
Quasi-random multilinear polynomials
Learning Warped Guidance for Blind Face Restoration
Erasure Correcting Codes by Using Shift Operation and Exclusive OR
Edwards field theory for glasses and granular matter
Shifted Coded Slotted ALOHA
An Ontology-Based Dialogue Management System for Banking and Finance Dialogue Systems
Robust 1-Bit Compressed Sensing via Hinge Loss Minimization
On the detection of low rank matrices in the high-dimensional regime
Probabilistic reconstruction of genealogies for polyploid plant species
On Cannon cone types and vector-valued multiplicative functions for genus-two-surface-group
Towards tight impossibility and possibility results for string stability
Infinite dimensional adaptive MCMC for Gaussian processes
An Online Approach to D2D Trajectory Utility Maximization Problem
An Optimal Dividend Problem with Capital Injections over a Finite Horizon
BodyNet: Volumetric Inference of 3D Human Body Shapes
Learning Contracting Vector Fields For Stable Imitation Learning
Learning to Exploit the Prior Network Knowledge for Weakly-Supervised Semantic Segmentation
Scalable and Interpretable One-class SVMs with Deep Learning and Random Fourier features
Absence of Dobrushin states for $2d$ long-range Ising models
Neural Trajectory Analysis of Recurrent Neural Network In Handwriting Synthesis
Partitions of hypergraphs under variable degeneracy constraints
On the Minimal Overcompleteness Allowing Universal Sparse Representation
On a Feynman-Kac approach to growth-fragmentation semigroups and their asymptotic behaviors
A Mean Field Game of Optimal Portfolio Liquidation *
Some deterministic structured population models which are limit of stochastic individual based models
Large Sample Properties of Partitioning-Based Series Estimators
Integration with respect to the Hermitian fractional Brownian motion
Distributed Collaborative Hashing and Its Applications in Ant Financial
Pose estimation of a single circle using default intrinsic calibration
Absence of hyperuniformity in amorphous hard-sphere packings of nonvanishing complexity
Dynamic Low-Stretch Trees via Dynamic Low-Diameter~Decompositions
Electronic Structure Topology Associated Domain and Its Physical Implication to Minimize Hybrid QM/MM Boundary Charge Transfer Effects
Evaluation of the Gradient Boosting of Regression Trees Method on Estimating the Car Following Behavior
Improving the Representation and Conversion of Mathematical Formulae by Considering their Textual Context
Introducing a Cost-Effective Approach for Improving the Arterial Traffic Performance Operating Under the Semi-Actuated Coordinated Signal Control
CNN-based Landmark Detection in Cardiac CTA Scans
An efficient deep convolutional laplacian pyramid architecture for CS reconstruction at low sampling ratios
Online Fall Detection using Recurrent Neural Networks
A Sequential Algorithm to Detect Diffusion Switching along Intracellular Particle Trajectories
Eigenvalues of random graphs with cycles
Convolutional Neural Networks for Skull-stripping in Brain MR Imaging using Consensus-based Silver standard Masks
Boolean approximate counting CSPs with weak conservativity, and implications for ferromagnetic two-spin
Estimates of the transition densities for the reflected Brownian motion on simple nested fractals
Roster Evaluation Based on Classifiers for the Nurse Rostering Problem
On the Taylor Expansion of Value Functions
Representing smooth functions as compositions of near-identity functions with implications for deep network optimization
Large-scale diversity estimation through surname origin inference
Incorporating Dictionaries into Deep Neural Networks for the Chinese Clinical Named Entity Recognition
Comparatives, Quantifiers, Proportions: A Multi-Task Model for the Learning of Quantities from Vision
A Deep Learning Approach to Fast, Format-Agnostic Detection of Malicious Web Content
The $γ$-Vectors of Pascal-like Triangles Defined by Riordan Arrays
Pieces of Eight: 8-bit Neural Machine Translation
Mitigating Docker Security Issues
Stable Outcomes in Modified Fractional Hedonic Games
Unsupervised Sparse Dirichlet-Net for Hyperspectral Image Super-Resolution
Machine Learning in Astronomy: A Case Study in Quasar-Star Classification
5G Wireless Network Slicing for eMBB, URLLC, and mMTC: A Communication-Theoretic View