COBRAS-TS: A new approach to Semi-Supervised Clustering of Time Series

Clustering is ubiquitous in data analysis, including analysis of time series. It is inherently subjective: different users may prefer different clusterings for a particular dataset. Semi-supervised clustering addresses this by allowing the user to provide examples of instances that should (not) be in the same cluster. This paper studies semi-supervised clustering in the context of time series. We show that COBRAS, a state-of-the-art semi-supervised clustering method, can be adapted to this setting. We refer to this approach as COBRAS-TS. An extensive experimental evaluation supports the following claims: (1) COBRAS-TS far outperforms the current state of the art in semi-supervised clustering for time series, and thus presents a new baseline for the field; (2) COBRAS-TS can identify clusters with separated components; (3) COBRAS-TS can identify clusters that are characterized by small local patterns; (4) a small amount of semi-supervision can greatly improve clustering quality for time series; (5) the choice of the clustering algorithm matters (contrary to earlier claims in the literature).

An Evaluation of Classification and Outlier Detection Algorithms

This paper evaluates algorithms for classification and outlier detection accuracies in temporal data. We focus on algorithms that train and classify rapidly and can be used for systems that need to incorporate new data regularly. Hence, we compare the accuracy of six fast algorithms using a range of well-known time-series datasets. The analyses demonstrate that the choice of algorithm is task and data specific but that we can derive heuristics for choosing. Gradient Boosting Machines are generally best for classification but there is no single winner for outlier detection though Gradient Boosting Machines (again) and Random Forest are better. Hence, we recommend running evaluations of a number of algorithms using our heuristics.

Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review

The framework of reinforcement learning or optimal control provides a mathematical formalization of intelligent decision making that is powerful and broadly applicable. While the general form of the reinforcement learning problem enables effective reasoning about uncertainty, the connection between reinforcement learning and inference in probabilistic models is not immediately obvious. However, such a connection has considerable value when it comes to algorithm design: formalizing a problem as probabilistic inference in principle allows us to bring to bear a wide array of approximate inference tools, extend the model in flexible and powerful ways, and reason about compositionality and partial observability. In this article, we will discuss how a generalization of the reinforcement learning or optimal control problem, which is sometimes termed maximum entropy reinforcement learning, is equivalent to exact probabilistic inference in the case of deterministic dynamics, and variational inference in the case of stochastic dynamics. We will present a detailed derivation of this framework, overview prior work that has drawn on this and related ideas to propose new reinforcement learning and control algorithms, and describe perspectives on future research.

A Simple Discrete-Time Survival Model for Neural Networks

There is currently great interest in applying neural networks to prediction tasks in medicine. It is important for predictive models to be able to use survival data, where each patient has a known follow-up time and event/censoring indicator. This avoids information loss when training the model and enables generation of predicted survival curves. In this paper, we describe a discrete-time survival model that is designed to be used with neural networks. The model is trained with the maximum likelihood method using minibatch stochastic gradient descent (SGD). The use of SGD enables rapid training speed. The model is flexible, so that the baseline hazard rate and the effect of the input data can vary with follow-up time. It has been implemented in the Keras deep learning framework, and source code for the model and several examples is available online. We demonstrated the high performance of the model by using it as part of a convolutional neural network to predict survival for over 10,000 patients with metastatic cancer, using the full text of 1,137,317 provider notes. The model’s C-index on the validation set was 0.71, which was superior to a linear baseline model (C-index 0.69).

Decision Tree Design for Classification in Crowdsourcing Systems

In this paper, we present a novel sequential paradigm for classification in crowdsourcing systems. Considering that workers are unreliable and they perform the tests with errors, we study the construction of decision trees so as to minimize the probability of mis-classification. By exploiting the connection between probability of mis-classification and entropy at each level of the decision tree, we propose two algorithms for decision tree design. Furthermore, the worker assignment problem is studied when workers can be assigned to different tests of the decision tree to provide a trade-off between classification cost and resulting error performance. Numerical results are presented for illustration.

Structured Analysis Dictionary Learning for Image Classification

We propose a computationally efficient and high-performance classification algorithm by incorporating class structural information in analysis dictionary learning. To achieve more consistent classification, we associate a class characteristic structure of independent subspaces and impose it on the classification error constrained analysis dictionary learning. Experiments demonstrate that our method achieves a comparable or better performance than the state-of-the-art algorithms in a variety of visual classification tasks. In addition, our method greatly reduces the training and testing computational complexity.

Deep Perm-Set Net: Learn to Predict Sets with Unknown Permutation and Cardinality Using Deep Neural Networks

We present a novel approach for learning to predict sets with unknown permutation and cardinality using deep neural networks. Even though the output of many real-world problems, e.g. object detection, are naturally expressed as sets of entities, existing deep learning architectures hinder a trivial extension to deal with this unstructured output. Even deep architectures that handle sequential data, such as recurrent neural networks, can only output an ordered set and may not guarantee a valid solution, i.e. a set with unique elements. In this paper, we derive a mathematical formulation for set prediction using feed-forward neural networks, where the output has unknown and unfixed cardinality and permutation. Specifically, in our formulation we incorporate the permutation as unobservable variable and estimate its distribution during the learning process using alternating optimization. We demonstrate the validity of this formulation on two relevant problems including object detection and a complex CAPTCHA test.

Text to Image Synthesis Using Generative Adversarial Networks

Generating images from natural language is one of the primary applications of recent conditional generative models. Besides testing our ability to model conditional, highly dimensional distributions, text to image synthesis has many exciting and practical applications such as photo editing or computer-aided content creation. Recent progress has been made using Generative Adversarial Networks (GANs). This material starts with a gentle introduction to these topics and discusses the existent state of the art models. Moreover, I propose Wasserstein GAN-CLS, a new model for conditional image generation based on the Wasserstein distance which offers guarantees of stability. Then, I show how the novel loss function of Wasserstein GAN-CLS can be used in a Conditional Progressive Growing GAN. In combination with the proposed loss, the model boosts by 7.07% the best Inception Score (on the Caltech birds dataset) of the models which use only the sentence-level visual semantics. The only model which performs better than the Conditional Wasserstein Progressive Growing GAN is the recently proposed AttnGAN which uses word-level visual semantics as well.

Compressed Dictionary Learning

In this paper we show that the computational complexity of the Iterative Thresholding and K-Residual-Means (ITKrM) algorithm for dictionary learning can be significantly reduced by using dimensionality reduction techniques based on the Johnson-Lindenstrauss Lemma. We introduce the Iterative Compressed-Thresholding and K-Means (IcTKM) algorithm for fast dictionary learning and study its convergence properties. We show that IcTKM can locally recover a generating dictionary with low computational complexity up to a target error \tilde{\varepsilon} by compressing d-dimensional training data into m < d dimensions, where m is proportional to \log d and inversely proportional to the distortion level \delta incurred by compressing the data. Increasing the distortion level \delta reduces the computational complexity of IcTKM at the cost of an increased recovery error and reduced admissible sparsity level for the training data. For generating dictionaries comprised of K atoms, we show that IcTKM can stably recover the dictionary with distortion levels up to the order \delta \leq O(1/\sqrt{\log K}). The compression effectively shatters the data dimension bottleneck in the computational cost of the ITKrM algorithm. For training data with sparsity levels S \leq O(K^{2/3}), ITKrM can locally recover the dictionary with a computational cost that scales as O(d K \log(\tilde{\varepsilon}^{-1})) per training signal. We show that for these same sparsity levels the computational cost can be brought down to O(\log^5 (d) K \log(\tilde{\varepsilon}^{-1})) with IcTKM, a significant reduction when high-dimensional data is considered. Our theoretical results are complemented with numerical simulations which demonstrate that IcTKM is a powerful, low-cost algorithm for learning dictionaries from high-dimensional data sets.

Markov Chain Neural Networks

In this work we present a modified neural network model which is capable to simulate Markov Chains. We show how to express and train such a network, how to ensure given statistical properties reflected in the training data and we demonstrate several applications where the network produces non-deterministic outcomes. One example is a random walker model, e.g. useful for simulation of Brownian motions or a natural Tic-Tac-Toe network which ensures non-deterministic game behavior.

Characterizing Question Facets for Complex Answer Retrieval

Complex answer retrieval (CAR) is the process of retrieving answers to questions that have multifaceted or nuanced answers. In this work, we present two novel approaches for CAR based on the observation that question facets can vary in utility: from structural (facets that can apply to many similar topics, such as ‘History’) to topical (facets that are specific to the question’s topic, such as the ‘Westward expansion’ of the United States). We first explore a way to incorporate facet utility into ranking models during query term score combination. We then explore a general approach to reform the structure of ranking models to aid in learning of facet utility in the query-document term matching phase. When we use our techniques with a leading neural ranker on the TREC CAR dataset, our methods rank first in the 2017 TREC CAR benchmark, and yield up to 26% higher performance than the next best method.

Construction and redundancy of codes for correcting deletable errors
Machine Learning for Exam Triage
A Hidden Resource in Wireless Channel Capacity: Dependence Control in Action
‘I ain’t tellin’ white folks nuthin’: A quantitative exploration of the race-related problem of candour in the WPA slave narratives
Regularly varying nonstationary second-order Galton-Watson processes with immigration
Adversarial adaptive 1-D convolutional neural networks for bearing fault diagnosis under varying working condition
Image Denoising via Collaborative Dual-Domain Patch Filtering
Fast counting with tensor networks
The Chiral anomaly without Landau levels: from the quantum to the classical regime
Adapting Mask-RCNN for Automatic Nucleus Segmentation
Novel Active Disturbance Rejection Control Based on Nested Linear Extended State Observers
Adaptive View Planning for Aerial 3D Reconstruction of Complex Scenes
Spiking Neural Algorithms for Markov Process Random Walk
Design and Implementation of a Low-Cost Secure Vehicle Tracking System
On the Schur expansion of Jack polynomials
Securing Social Media User Data – An Adversarial Approach
Direct Runge-Kutta Discretization Achieves Acceleration
OEDIPUS: An Experiment Design Framework for Sparsity-Constrained MRI
Hamiltonicity of $2$-block intersection graphs of ${\rm{TS}}(v,λ)$: $v\equiv 0$ or $4\pmod{12}$
Mutual Clustering Coefficient-based Suspicious-link Detection approach for Online Social Networks
Scalable Importance Tempering and Bayesian Variable Selection
Weakly Supervised Attention Learning for Textual Phrases Grounding
Transporting inferences from a randomized trial to a new target population
Exploring Conversational Language Generation for Rich Content about Hotels
Generating Synthetic X-ray Images of a Person from the Surface Geometry
A general framework for modelling zero inflation
SAGE: Percipient Storage for Exascale Data Centric Computing
Accurate, Fast and Scalable Kernel Ridge Regression on Parallel and Distributed Systems
Solid Harmonic Wavelet Scattering for Predictions of Molecule Properties
Planar diagrams for local invariants of graphs in surfaces
Secure Face Matching Using Fully Homomorphic Encryption
Greedy Bipartite Matching in Random Type Poisson Arrival Model
Convolutional-Recurrent Neural Networks for Speech Enhancement
Structure-sensitive Multi-scale Deep Neural Network for Low-Dose CT Denoising
Primal Dual Interior Point Methods for Type-2 Second Order Cone Optimization
The relation between Parabolic Hecke modules and $W$-graph ideal modules in Kazhdan-Lusztig theory
Placement Delivery Array Design via Attention-Based Deep Neural Network
Parity duality for the amplituhedron
Bi-directional Graph Structure Information Model for Multi-Person Pose Estimation
Text-Independent Speaker Verification Using Long Short-Term Memory Networks
Deterministic and Randomized Actuator Scheduling With Guaranteed Performance Bounds
Towards Interpretable Face Recognition
How Flajolet Processed Streams with Coin Flips
$\ell_1$-regression with Heavy-tailed Distributions
Multimodal Utterance-level Affect Analysis using Visual, Audio and Text Features
On and Off-Blockchain Enforcement Of Smart Contracts
Understand Urban Human Mobility through Crowdsensed Data
Residential Transformer Overloading Risk Assessment Using Clustering Analysis
Accelerating Neural Transformer via an Average Attention Network
Interpretable Fully Convolutional Classification of Intrapapillary Capillary Loops for Real-Time Detection of Early Squamous Neoplasia
A Probabilistic Extension of Action Language BC+
A Deep Network for Arousal-Valence Emotion Prediction with Acoustic-Visual Cues
The hydrodynamics limit for local mean-field dynamics with unbounded spins
Translating LPOD and CR-Prolog2 into Standard Answer Set Programs
Flexible Density Tempering Approaches for State Space Models with an Application to Factor Stochastic Volatility Models
MX-LSTM: mixing tracklets and vislets to jointly forecast trajectories and head poses
Convolutional Sequence to Sequence Model for Human Dynamics
Distributed Big-Data Optimization via Block-Iterative Convexification and Averaging
Functional ASP with Intensional Sets: Application to Gelfond-Zhang Aggregates
Endogenous growth – A dynamic technology augmentation of the Solow model
Chance Constrained Optimal Power Flow Using the Inner-Outer Approximation Approach
BUDAMAF: Data Management in Cloud Federations
ADMM-MCP Framework for Sparse Recovery with Global Convergence
Improved Tactile Resonance Sensor for Robotic Assisted Surgery
Measuring the Noise Cumulative Distribution Function Using Quantized Data
Measure Control of a Semilinear Parabolic Equation with a Nonlocal Time Delay
Weyl almost periodic solutions to abstract linear and semilinear equations with Weyl almost periodic coefficients
Subcritical phase of $d$-dimensional Poisson-Boolean percolation and its vacant set
Information Loss in the Human Auditory System
On the primal-dual dynamics of Support Vector Machines
Utilizing Device-level Demand Forecasting for Flexibility Markets – Full Version
Investigating Audio, Visual, and Text Fusion Methods for End-to-End Automatic Personality Prediction
On the Structure of Interlinked Cycle Structures with Interlocked Outer Cycles
Energy-Efficient Wireless Powered Secure Transmission with Cooperative Jamming for Public Transportation
On Poincar{é} and logarithmic Sobolev inequalities for a class of singular Gibbs measures
Differential passivity like properties for a class of nonlinear systems
CrisisMMD: Multimodal Twitter Datasets from Natural Disasters
Transmit Precoding and Receive Power Splitting for Harvested Power Maximization in MIMO SWIPT Systems
Entropy-based randomisation of rating networks
The Variable-Order Fractional Calculus of Variations
Joint Surgical Gesture and Task Classification with Multi-Task and Multimodal Learning
Emulation of utility functions over a set of permutations: sequencing reliability growth tasks
Evolving Mario Levels in the Latent Space of a Deep Convolutional Generative Adversarial Network
Exploring Emoji Usage and Prediction Through a Temporal Variation Lens
Alternative passive maps in the Brayton-Moser framework: Implications on control and optimization
Multiscale socio-ecological networks in the age of information
Distributed watermarking for secure control of microgrids under replay attacks
KNPTC: Knowledge and Neural Machine Translation Powered Chinese Pinyin Typo Correction
Group Secret-Key Generation using Algebraic Rings in Three-User Wireless Networks
Under Frequency Load Shedding based on PMU Estimates of Frequency and ROCOF
Avalon: Building an Operating System for Robotcenter
From the Periphery to the Center: Information Brokerage in an Evolving Network
Gaussian Process Forecast with multidimensional distributional entries
TR-SVD: Fast and Memory Efficient Method for Time Ranged Singular Value Decomposition
Milling and meandering: Flocking dynamics of stochastically interacting agents with a field of view
Aspect Term Extraction with History Attention and Selective Transformation
Karlin-McGregor mutational occupancy problem revisited
$L^p$ bounds for boundary-to-boundary transport densities, and $W^{1,p}$ bounds for the BV least gradient problem in 2D
Breaking the $\tildeΩ(\sqrt{n})$ Barrier: Fast Consensus under a Late Adversary
Unsupervised Features for Facial Expression Intensity Estimation over Time
Pricing European option with the short rate under Subdiffusive fractional Brownian motion regime
Independence Posets
Impacts of Opinion Leaders on Social Contagions
Largest Weight Common Subtree Embeddings with Distance Penalties
Selection of proposal distributions for generalized importance sampling estimators
Time-discretization of stochastic 2-D Navier–Stokes equations with a penalty-projection method
Learnable PINs: Cross-Modal Embeddings for Person Identity
The design of courier transportation networks with a nonlinear zero-one programming model
Uniquely restricted matchings in subcubic graphs
Greening Internet of Things for Smart Everythings with A Green-Environment Life: A Survey and Future Prospects
Optimal and Robust Controller Synthesis: using Energy Timed Automata with Uncertainty
Nonconventional moderate deviations theorems and exponential concentration inequalities
Fast and accurate simulation of particle detectors using generative adversarial neural networks
Convergence of metric two-level measure spaces
A new analysis of Work Stealing with latency
Maximum cuts in edge-colored graphs
Erasure code-based low storage blockchain node
Modelling cross-dependencies between Spain’s regional tourism markets with an extension of the Gaussian process regression model
Spectral clustering algorithms for the detection of clusters in block-cyclic and block-acyclic graphs
Negative moments for Gaussian multiplicative chaos on fractal sets
A Dynamic Model for Traffic Flow Prediction Using Improved DRN
Approximate Temporal Difference Learning is a Gradient Descent for Reversible Policies
Undersampled dynamic X-ray tomography with dimension reduction Kalman filter
Modelling tourism demand to Spain with machine learning techniques. The impact of forecast horizon on model selection
Unsupervised Cross-Lingual Information Retrieval using Monolingual Data Only
Duality theory for multi-marginal optimal transport with repulsive costs in metric spaces
On executing aggressive quadrotor attitude tracking maneuvers under actuator constraints
Generalising Cost-Optimal Particle Filtering
Controlling the uncertain response of real multiplex networks to random damage
On the Design of Hybrid Pose and Velocity-bias Observers on Lie Group SE(3)
AI safety via debate
Images & Recipes: Retrieval in the cooking context
Optimal corrector estimates on percolation clusters
Computing tensor Z-eigenvectors with dynamical systems
SynTF: Synthetic and Differentially Private Term Frequency Vectors for Privacy-Preserving Text Mining
Altered Fingerprints: Detection and Localization
Fast Directional Self-Attention Mechanism
Negotiation Strategies for Agents with Ordinal Preferences
Neural networks as Interacting Particle Systems: Asymptotic convexity of the Loss Landscape and Universal Scaling of the Approximation Error
Lidar Cloud Detection with Fully Convolutional Networks
Multi-Resolution Multi-Modal Sensor Fusion For Remote Sensing Data With Label Uncertainty
Exploring the Limits of Weakly Supervised Pretraining