Fast Robust Methods for Singular State-Space Models

State-space models are used in a wide range of time series analysis formulations. Kalman filtering and smoothing are work-horse algorithms in these settings. While classic algorithms assume Gaussian errors to simplify estimation, recent advances use a broader range of optimization formulations to allow outlier-robust estimation, as well as constraints to capture prior information. Here we develop methods on state-space models where either innovations or error covariances may be singular. These models frequently arise in navigation (e.g. for `colored noise’ models or deterministic integrals) and are ubiquitous in auto-correlated time series models such as ARMA. We reformulate all state-space models (singular as well as nonsinguar) as constrained convex optimization problems, and develop an efficient algorithm for this reformulation. The convergence rate is {\it locally linear}, with constants that do not depend on the conditioning of the problem. Numerical comparisons show that the new approach outperforms competing approaches for {\it nonsingular} models, including state of the art interior point (IP) methods. IP methods converge at superlinear rates; we expect them to dominate. However, the steep rate of the proposed approach (independent of problem conditioning) combined with cheap iterations wins against IP in a run-time comparison. We therefore suggest that the proposed approach be the {\it default choice} for estimating state space models outside of the Gaussian context, regardless of whether the error covariances are singular or not.

Compact Representations of Event Sequences

We introduce a new technique for the efficient management of large sequences of multidimensional data, which takes advantage of regularities that arise in real-world datasets and supports different types of aggregation queries. More importantly, our representation is flexible in the sense that the relevant dimensions and queries may be used to guide the construction process, easily providing a space-time tradeoff depending on the relevant queries in the domain. We provide two alternative representations for sequences of multidimensional data and describe the techniques to efficiently store the datasets and to perform aggregation queries over the compressed representation. We perform experimental evaluation on realistic datasets, showing the space efficiency and query capabilities of our proposal.

Learning SMaLL Predictors

We present a new machine learning technique for training small resource-constrained predictors. Our algorithm, the Sparse Multiprototype Linear Learner (SMaLL), is inspired by the classic machine learning problem of learning k-DNF Boolean formulae. We present a formal derivation of our algorithm and demonstrate the benefits of our approach with a detailed empirical study.

A Reductions Approach to Fair Classification

We present a systematic approach for achieving fairness in a binary classification setting. While we focus on two well-known quantitative definitions of fairness, our approach encompasses many other previously studied definitions as special cases. Our approach works by reducing fair classification to a sequence of cost-sensitive classification problems, whose solutions yield a randomized classifier with the lowest (empirical) error subject to the desired constraints. We introduce two reductions that work for any representation of the cost-sensitive classifier and compare favorably to prior baselines on a variety of data sets, while overcoming several of their disadvantages.

Multiple Kernel $k$-means Clustering using Min-Max Optimization with $l_2$ Regularization

As various types of biomedical data become available, multiple kernel learning approaches have been proposed to incorporate abundant yet diverse information collected from multiple sources (or views) to facilitate disease prediction and pattern recognition. Although supervised multiple kernel learning has been extensively studied, until recently, only a few unsupervised approaches have been proposed. Moreover, the existing unsupervised approaches are unable to effectively utilize useful and complementary information especially when signals in some views are weak. We propose a novel multiple kernel k-means clustering method which aims to effectively use complementary information from multiple views to identify clusters. It is achieved by optimizing the unsupervised problem using a \min_{\mathbf{H}}\max_{\mathbf{\theta}} formulation, such that more weights can be assigned to views having weak signal for cluster identification. Moreover, our method avoids dismissing views with informative but weak signals by imposing l_2 constraint. Additionally, it allows to distill biological prior knowledge on the clustering by imposing a linear constraint on the kernel coefficients. To evaluate our method, we compare it with seven other clustering approaches on simulated multiview data. The simulation results show that our method outperforms existing clustering approaches especially when there is noise and redundancy in the data.

Gaussian Process Latent Variable Alignment Learning

We present a model that can automatically learn alignments between high-dimensional data in an unsupervised manner. Learning alignments is an ill-constrained problem as there are many different ways of defining a good alignment. Our proposed method casts alignment learning in a framework where both alignment and data are modelled simultaneously. We derive a probabilistic model built on non-parametric priors that allows for flexible warps while at the same time providing means to specify interpretable constraints. We show results on several datasets, including different motion capture sequences and show that the suggested model outperform the classical algorithmic approaches to the alignment task.

Sketching for Principal Component Regression

Principal component regression (PCR) is a useful method for regularizing linear regression. Although conceptually simple, straightforward implementations of PCR have high computational costs and so are inappropriate when learning with large scale data. In this paper, we propose efficient algorithms for computing approximate PCR solutions that are, on one hand, high quality approximations to the true PCR solutions (when viewed as minimizer of a constrained optimization problem), and on the other hand entertain rigorous risk bounds (when viewed as statistical estimators). In particular, we propose an input sparsity time algorithms for approximate PCR. We also consider computing an approximate PCR in the streaming model, and kernel PCR. Empirical results demonstrate the excellent performance of our proposed methods.

Sklar’s Omega: A Gaussian Copula-Based Framework for Assessing Agreement

The statistical measurement of agreement is important in a number of fields, e.g., content analysis, education, computational linguistics, biomedical imaging. We propose Sklar’s Omega, a Gaussian copula-based framework for measuring intra-coder, inter-coder, and inter-method agreement as well as agreement relative to a gold standard. We demonstrate the efficacy and advantages of our approach by applying it to both simulated and experimentally observed datasets, including data from two medical imaging studies. Application of our proposed methodology is supported by our open-source R package, sklarsomega, which is available for download from the Comprehensive R Archive Network.

Deep Back-Projection Networks For Super-Resolution

The feed-forward architectures of recently proposed deep super-resolution networks learn representations of low-resolution inputs, and the non-linear mapping from those to high-resolution output. However, this approach does not fully address the mutual dependencies of low- and high-resolution images. We propose Deep Back-Projection Networks (DBPN), that exploit iterative up- and down-sampling layers, providing an error feedback mechanism for projection errors at each stage. We construct mutually-connected up- and down-sampling stages each of which represents different types of image degradation and high-resolution components. We show that extending this idea to allow concatenation of features across up- and down-sampling stages (Dense DBPN) allows us to reconstruct further improve super-resolution, yielding superior results and in particular establishing new state of the art results for large scaling factors such as 8x across multiple data sets.

HENet:A Highly Efficient Convolutional Neural Networks Optimized for Accuracy, Speed and Storage

In order to enhance the real-time performance of convolutional neural networks(CNNs), more and more researchers are focusing on improving the efficiency of CNN. Based on the analysis of some CNN architectures, such as ResNet, DenseNet, ShuffleNet and so on, we combined their advantages and proposed a very efficient model called Highly Efficient Networks(HENet). The new architecture uses an unusual way to combine group convolution and channel shuffle which was mentioned in ShuffleNet. Inspired by ResNet and DenseNet, we also proposed a new way to use element-wise addition and concatenation connection with each block. In order to make greater use of feature maps, pooling operations are removed from HENet. The experiments show that our model’s efficiency is more than 1 times higher than ShuffleNet on many open source datasets, such as CIFAR-10/100 and SVHN.

Transfer Automatic Machine Learning

Building effective neural networks requires many design choices. These include the network topology, optimization procedure, regularization, stability methods, and choice of pre-trained parameters. This design is time consuming and requires expert input. Automatic Machine Learning aims automate this process using hyperparameter optimization. However, automatic model building frameworks optimize performance on each task independently, whereas human experts leverage prior knowledge when designing a new network. We propose Transfer Automatic Machine Learning, a method to accelerate network design using knowledge of prior tasks. For this, we build upon reinforcement learning architecture design methods to support parallel training on multiple tasks and transfer the search strategy to new tasks. Tested on NLP and Image classification tasks, Transfer Automatic Machine Learning reduces convergence time over single-task methods by almost an order of magnitude on 13 out of 14 tasks. It achieves better test set accuracy on 10 out of 13 tasks NLP tasks and improves performance on CIFAR-10 image recognition from 95.3% to 97.1%.

Fast Dawid-Skene

Many real world problems can now be effectively solved using supervised machine learning. A major roadblock is often the lack of an adequate quantity of labeled data for training. A possible solution is to assign the task of labeling data to a crowd, and then infer the true label using aggregation methods. A well-known approach for aggregation is the Dawid-Skene (DS) algorithm, which is based on the principle of Expectation-Maximization (EM). We propose a new simple, yet effective, EM-based algorithm, which can be interpreted as a ‘hard’ version of DS, that allows much faster convergence while maintaining similar accuracy in aggregation. We also show how the proposed method can be extended to settings when there are multiple labels as well as for online vote aggregation. Our experiments on standard vote aggregation datasets show a significant speedup in time taken for convergence – upto \sim8x over Dawid-Skene and \sim6x over other fast EM methods, at competitive accuracy performance.

Gaussian optimizers for entropic inequalities in quantum information
The equivariant volumes of the permutahedron
Fast Cylinder and Plane Extraction from Depth Cameras for Visual Odometry
Multimodal Emoji Prediction
Game Theoretic Analysis of Road User Safety Scenarios Involving Autonomous Vehicles
Arbitrary Discrete Sequence Anomaly Detection with Zero Boundary LSTM
Visualizing Convolutional Neural Network Protein-Ligand Scoring
Self-reporting and screening: Data with current-status and censored observations
PI-VIO: Robust and Efficient Stereo Visual Inertial Odometry using Points and Lines
On the parameterized complexity of manipulating Top Trading Cycles
Quantum algorithm for energy matching in hard optimization problems
Securing Untrusted Full-Duplex Relay Channels in the Presence of Multiple External Cluster-Based Eavesdroppers
Almost Sure Uniqueness of a Global Minimum Without Convexity
Masked Conditional Neural Networks for Audio Classification
Matched Filters for Noisy Induced Subgraph Detection
On Nonlinear Dimensionality Reduction, Linear Smoothing and Autoencoding
Extracting useful information from Basic Safety Message Data: An empirical study of driving volatility measures and crash frequency at intersections
Categorical Mixture Models on VGGNet activations
Quantum Circuit Designs for Gate-Model Quantum Computer Architectures
Subgradient methods for sharp weakly convex functions
A Poisson Model for Entanglement Optimization in the Quantum Internet
Decision-making processes in the Cognitive Theory of True Conditions
Scaling Structured Multigrid to 500K+ Cores through Coarse-Grid Redistribution
A Nonlinear Bregman Primal-Dual Framework for Optimizing Nonconvex Infimal Convolutions
Estimation of edge density in noisy networks
Combinatorics of $\mathcal{X}$-variables in finite type cluster algebras
Discontinuity-Sensitive Optimal Control Learning by Mixture of Experts
Towards a Data-driven IoT Software Architecture for Smart City Utilities
A linear algorithm for optimization over directed graphs with geometric convergence
Exponential Discriminative Metric Embedding in Deep Learning
On the subgraphs of percolated random geometric graphs and the associated random complexes
An Application of HodgeRank to Online Peer Assessment
Staircases to analytic sum-sides for many new integer partition identities of Rogers-Ramanujan type
Sequential Maximum Margin Classifiers for Partially Labeled Data
Rigid Point Registration with Expectation Conditional Maximization
A proof of the GM-MDS conjecture
Differential Expression Analysis of Dynamical Sequencing Count Data with a Gamma Markov Chain
Bayesian nonparametric regression using complex wavelets
Sparse Adversarial Perturbations for Videos
Packing chromatic number of subdivisions of cubic graphs
Population stability: regulating size in the presence of an adversary
Visual Explanations From Deep 3D Convolutional Neural Networks for Alzheimer’s Disease Classification
Pyramid Person Matching Network for Person Re-identification
An iALM-ICA-based Anti-Jamming DS-CDMA Receiver for LMS Systems
Broadcast domination and multipacking: bounds and the integrality gap
Extracting Domain Invariant Features by Unsupervised Learning for Robust Automatic Speech Recognition
Graph Learning from Filtered Signals: Graph System and Diffusion Kernel Identification
Object cosegmentation using deep Siamese network
Multi-Channel Pyramid Person Matching Network for Person Re-Identification
Energy Efficiency of an Unlicensed Wireless Network in the Presence of Retransmissions
Decoupled Spatial Neural Attention for Weakly Supervised Semantic Segmentation
Consensus over evolutionary graphs
Submodular maximization with uncertain knapsack capacity
Scalable Stochastic Kriging with Markovian Covariances
Generating goal-directed visuomotor plans based on learning using a predictive coding type deep visuomotor recurrent neural network model
Concurrent Spatial and Channel Squeeze & Excitation in Fully Convolutional Networks
Bursty Human Dynamics
On the Efficiency of Nash Equilibria in Charging Games
Single-molecular and Ensemble-level Oscillations of Cyanobacterial Circadian Clock
GPSP: Graph Partition and Space Projection based Approach for Heterogeneous Network Embedding
Revisiting differentially private linear regression: optimal and adaptive prediction & estimation in unbounded domain
The Ising distribution as a latent variable model
Fast in-database cross-matching of high-cadence, high-density source lists with an up-to-date sky model
Successive Wyner-Ziv Coding for the Binary CEO Problem under Log-Loss
Single View Stereo Matching
A Novel Canonical Duality Theory for Solving 3-D Topology Optimization Problems
Partition games are pure breaking games
3D Human Pose Estimation in RGBD Images for Robotic Task Learning
TRLG: Fragile blind quad watermarking for image tamper detection and recovery by providing compact digests with quality optimized using LWT and GA
Smaller Universes for Uniform Sampling of 0,1-matrices with fixed row and column sums
A new Multifractional Process with Random Exponent
Inferencing Based on Unsupervised Learning of Disentangled Representations
Extracting Action Sequences from Texts Based on Deep Reinforcement Learning
Garside combinatorics for Thompson’s monoid $F^+$ and a hybrid with the braid monoid $B\_\infty^+$
Learning Spectral-Spatial-Temporal Features via a Recurrent Convolutional Neural Network for Change Detection in Multispectral Imagery
Mean field repulsive Kuramoto models: Phase locking and spatial signs
Solving large-scale general phase retrieval problems via a sequence of convex relaxations
Law equivalence of Ornstein–Uhlenbeck processes driven by a Lévy process
A Neural Network Approach to Missing Marker Reconstruction
A limit theorem for the six-length of random functional graphs with a fixed degree sequence
A Suboptimality Approach to Distributed Linear Quadratic Optimal Control
Bialgebra Coverings and Transfer of Structure
Chance-Constrained Optimization for Non-Linear Network Flow Problems
On kissing numbers and spherical codes in high dimensions
In absence of long chordless cycles, large tree-width becomes a local phenomenon
A finest balancing score algorithm to avoid common pitfalls of propensity score matching
Frontier improvement in the DEA models
International Arms Trade: A Dynamic Separable Network Model With Heterogeneity Components
Generating Contradictory, Neutral, and Entailing Sentences
Exponential Lyapunov Stability Analysis of a Drilling Mechanism
Varying Coefficient Panel Data Model with Interactive Fixed Effects
Optimal performances of a global allocation of radio resources in heterogeneous networks system
Byzantine Preferential Voting
Towards the Creation of a Large Corpus of Synthetically-Identified Clinical Notes
Frequency and Quadrature Amplitude Modulation for 5G Networks
Single cell and multi-cell performance analysis of OFDM index modulation
Some binary BCH codes with length $n=2^m+1$
Massive MIMO performance with imperfect channel reciprocity and channel estimation error
Neural network feedback controller for inertial platform
Nonparametric Estimation of Probability Density Functions of Random Persistence Diagrams
Genetic Algorithm Assisted Hybrid Beamforming for Wireless Fronthaul
Efficient Synchronization of State-based CRDTs
Aspiration-based Perturbed Learning Automata
Interference Management via Space and Frequency Domain Resource Partitioning
FQAM-FBMC Design and Its Application to Machine Type Communication
RTSeg: Real-time Semantic Segmentation Comparative Study
Placebo inference on treatment effects when the number of clusters is small
Improvements on the distribution of maximal segmental scores in a Markovian sequence
A bag-to-class divergence approach to multiple-instance learning
Fast and Accurate Semantic Mapping through Geometric-based Incremental Segmentation
A Deep Learning Algorithm for One-step Contour Aware Nuclei Segmentation of Histopathological Images
Loynes construction for the extended bipartite matching
Lozenge tilings of hexagons with central holes and dents
Optimal Threshold-Based Control Policies for Persistent Monitoring on Graphs
Long-branch attraction in species tree estimation: inconsistency of partitioned likelihood and topology-based summary methods
Flexible and Efficient Algorithms for Abelian Matching in Strings
OntoWind: An Improved and Extended Wind Energy Ontology
The size of the giant component in random hypergraphs: a short proof
Accelerated Methods for Deep Reinforcement Learning
Addendum to Pontryagin’s maximum principle for dynamic systems on time scales
Sever: A Robust Meta-Algorithm for Stochastic Optimization
Stochastic nonlinear Schrödinger equations on tori
Imitate or innovate: competition of strategy updating attitudes in spatial social dilemma games