Functional ANOVA with Multiple Distributions: Implications for the Sensitivity Analysis of Computer Experiments

The functional ANOVA expansion of a multivariate mapping plays a fundamental role in statistics. The expansion is unique once a unique distribution is assigned to the covariates. Recent investigations in the environmental and climate sciences show that analysts may not be in a position to assign a unique distribution in realistic applications. We offer a systematic investigation of existence, uniqueness, orthogonality, monotonicity and ultramodularity of the functional ANOVA expansion of a multivariate mapping when a multiplicity of distributions is assigned to the covariates. In particular, we show that a multivariate mapping can be associated with a core of probability measures that guarantee uniqueness. We obtain new results for variance decomposition and dimension distribution under mixtures. Implications for the global sensitivity analysis of computer experiments are also discussed.

Compositional Correlation for Detecting Real Associations Among Time Series

Correlation remains to be one of the most widely used statistical tools for assessing the strength of relationships between data series. This paper presents a novel compositional correlation method for detecting linear and nonlinear relationships by considering the averages of all parts of all possible compositions of the data series instead of considering the averages of the whole series. The approach enables cumulative contribution of all local associations to the resulting correlation value. The method is applied on two different datasets: a set of four simple nonlinear polynomial functions and the expression time series data of 4381 budding yeast (saccharomyces cerevisiae) genes. The obtained results show that the introduced compositional correlation method is capable of determining real direct and inverse linear, nonlinear and monotonic relationships. Comparisons with Pearson’s correlation, Spearman’s correlation, distance correlation and the simulated annealing genetic algorithm maximal information coefficient (SGMIC) have shown that the presented method is capable of detecting important associations which were not detected by the compared methods.

Decision-Feedback Detection Strategy for Nonlinear Frequency-Division Multiplexing

By exploiting a causality property of the nonlinear Fourier transform, a novel decision-feedback detection strategy for nonlinear frequency-division multiplexing (NFDM) systems is introduced. The performance of the proposed strategy is investigated both by simulations and by theoretical bounds and approximations, showing that it achieves a considerable performance improvement compared to previously adopted techniques in terms of Q-factor. The obtained improvement demonstrates that, by tailoring the detection strategy to the peculiar properties of the nonlinear Fourier transform, it is possible to boost the performance of NFDM systems and overcome current limitations imposed by the use of more conventional detection techniques suitable for the linear regime.

Time Series Segmentation through Automatic Feature Learning

Internet of things (IoT) applications have become increasingly popular in recent years, with applications ranging from building energy monitoring to personal health tracking and activity recognition. In order to leverage these data, automatic knowledge extraction – whereby we map from observations to interpretable states and transitions – must be done at scale. As such, we have seen many recent IoT data sets include annotations with a human expert specifying states, recorded as a set of boundaries and associated labels in a data sequence. These data can be used to build automatic labeling algorithms that produce labels as an expert would. Here, we refer to human-specified boundaries as breakpoints. Traditional changepoint detection methods only look for statistically-detectable boundaries that are defined as abrupt variations in the generative parameters of a data sequence. However, we observe that breakpoints occur on more subtle boundaries that are non-trivial to detect with these statistical methods. In this work, we propose a new unsupervised approach, based on deep learning, that outperforms existing techniques and learns the more subtle, breakpoint boundaries with a high accuracy. Through extensive experiments on various real-world data sets – including human-activity sensing data, speech signals, and electroencephalogram (EEG) activity traces – we demonstrate the effectiveness of our algorithm for practical applications. Furthermore, we show that our approach achieves significantly better performance than previous methods.

Deep Canonically Correlated LSTMs

We examine Deep Canonically Correlated LSTMs as a way to learn nonlinear transformations of variable length sequences and embed them into a correlated, fixed dimensional space. We use LSTMs to transform multi-view time-series data non-linearly while learning temporal relationships within the data. We then perform correlation analysis on the outputs of these neural networks to find a correlated subspace through which we get our final representation via projection. This work follows from previous work done on Deep Canonical Correlation (DCCA), in which deep feed-forward neural networks were used to learn nonlinear transformations of data while maximizing correlation.

An Integration-Oriented Ontology to Govern Evolution in Big Data Ecosystems

Big Data architectures allow to flexibly store and process heterogeneous data, from multiple sources, in their original format. The structure of those data, commonly supplied by means of REST APIs, is continuously evolving. Thus data analysts need to adapt their analytical processes after each API release. This gets more challenging when performing an integrated or historical analysis. To cope with such complexity, in this paper, we present the Big Data Integration ontology, the core construct to govern the data integration process under schema evolution by systematically annotating it with information regarding the schema of the sources. We present a query rewriting algorithm that, using the annotated ontology, converts queries posed over the ontology to queries over the sources. To cope with syntactic evolution in the sources, we present an algorithm that semi-automatically adapts the ontology upon new releases. This guarantees ontology-mediated queries to correctly retrieve data from the most recent schema version as well as correctness in historical queries. A functional and performance evaluation on real-world APIs is performed to validate our approach.

MORF: A Framework for MOOC Predictive Modeling and Replication At Scale

The MOOC Replication Framework (MORF) is a novel software system for feature extraction, model training/testing, and evaluation of predictive dropout models in Massive Open Online Courses (MOOCs). MORF makes large-scale replication of complex machine-learned models tractable and accessible for researchers, and enables public research on privacy-protected data. It does so by focusing on the high-level operations of an \emph{extract-train-test-evaluate} workflow, and enables researchers to encapsulate their implementations in portable, fully reproducible software containers which are executed on data with a known schema. MORF’s workflow allows researchers to use data in analysis without providing them access to the underlying data directly, preserving privacy and data security. During execution, containers are sandboxed for security and data leakage and parallelized for efficiency, allowing researchers to create and test new models rapidly, on large-scale multi-institutional datasets that were previously inaccessible to most researchers. MORF is provided both as a Python API (the MORF Software), for institutions to use on their own MOOC data) or in a platform-as-a-service (PaaS) model with a web API and a high-performance computing environment (the MORF Platform).

Learning Features For Relational Data

Feature engineering is one of the most important but tedious tasks in data science projects. This work studies automation of feature learning for relational data. We first theoretically proved that learning relevant features from relational data for a given predictive analytics problem is NP-hard. However, it is possible to empirically show that an efficient rule based approach predefining transformations as a priori based on heuristics can extract very useful features from relational data. Indeed, the proposed approach outperformed the state of the art solutions with a significant margin. We further introduce a deep neural network which automatically learns appropriate transformations of relational data into a representation that predicts the target variable well instead of being predefined as a priori by users. In an extensive experiment with Kaggle competitions, the proposed methods could win late medals. To the best of our knowledge, this is the first time an automation system could win medals in Kaggle competitions with complex relational data.

Topic Modeling on Health Journals with Regularized Variational Inference

Topic modeling enables exploration and compact representation of a corpus. The CaringBridge (CB) dataset is a massive collection of journals written by patients and caregivers during a health crisis. Topic modeling on the CB dataset, however, is challenging due to the asynchronous nature of multiple authors writing about their health journeys. To overcome this challenge we introduce the Dynamic Author-Persona topic model (DAP), a probabilistic graphical model designed for temporal corpora with multiple authors. The novelty of the DAP model lies in its representation of authors by a persona — where personas capture the propensity to write about certain topics over time. Further, we present a regularized variational inference algorithm, which we use to encourage the DAP model’s personas to be distinct. Our results show significant improvements over competing topic models — particularly after regularization, and highlight the DAP model’s unique ability to capture common journeys shared by different authors.

Panel Data Quantile Regression with Grouped Fixed Effects

This paper introduces grouped latent heterogeneity in panel data quantile regression. More precisely, we assume that the observed individuals come from a heterogeneous population with an unknown, finite number of types. The number of types and group membership is not assumed to be known in advance and is estimated by means of a convex optimization problem. We provide conditions under which group membership is estimated consistently and establish asymptotic normality of the resulting estimators.

DKVF: A Framework for Rapid Prototyping and Evaluating Distributed Key-value Stores

We present our framework DKVF that enables one to quickly prototype and evaluate new protocols for key-value stores and compare them with existing protocols based on selected benchmarks. Due to limitations of CAP theorem, new protocols must be developed that achieve the desired trade-off between consistency and availability for the given application at hand. Hence, both academic and industrial communities focus on developing new protocols that identify a different (and hopefully better in one or more aspect) point on this trade-off curve. While these protocols are often based on a simple intuition, evaluating them to ensure that they indeed provide increased availability, consistency, or performance is a tedious task. Our framework, DKVF, enables one to quickly prototype a new protocol as well as identify how it performs compared to existing protocols for pre-specified benchmarks. Our framework relies on YCSB (Yahoo! Cloud Servicing Benchmark) for benchmarking. We demonstrate DKVF by implementing four existing protocols –eventual consistency, COPS, GentleRain and CausalSpartan– with it. We compare the performance of these protocols against different loading conditions. We find that the performance is similar to our implementation of these protocols from scratch. And, the comparison of these protocols is consistent with what has been reported in the literature. Moreover, implementation of these protocols was much more natural as we only needed to translate the pseudocode into Java (and add the necessary error handling). Hence, it was possible to achieve this in just 1-2 days per protocol. Finally, our framework is extensible. It is possible to replace individual components in the framework (e.g., the storage component).

Reblur2Deblur: Deblurring Videos via Self-Supervised Learning

Motion blur is a fundamental problem in computer vision as it impacts image quality and hinders inference. Traditional deblurring algorithms leverage the physics of the image formation model and use hand-crafted priors: they usually produce results that better reflect the underlying scene, but present artifacts. Recent learning-based methods implicitly extract the distribution of natural images directly from the data and use it to synthesize plausible images. Their results are impressive, but they are not always faithful to the content of the latent image. We present an approach that bridges the two. Our method fine-tunes existing deblurring neural networks in a self-supervised fashion by enforcing that the output, when blurred based on the optical flow between subsequent frames, matches the input blurry image. We show that our method significantly improves the performance of existing methods on several datasets both visually and in terms of image quality metrics. The supplementary material is

Variational Recurrent Neural Machine Translation

Partially inspired by successful applications of variational recurrent neural networks, we propose a novel variational recurrent neural machine translation (VRNMT) model in this paper. Different from the variational NMT, VRNMT introduces a series of latent random variables to model the translation procedure of a sentence in a generative way, instead of a single latent variable. Specifically, the latent random variables are included into the hidden states of the NMT decoder with elements from the variational autoencoder. In this way, these variables are recurrently generated, which enables them to further capture strong and complex dependencies among the output translations at different timesteps. In order to deal with the challenges in performing efficient posterior inference and large-scale training during the incorporation of latent variables, we build a neural posterior approximator, and equip it with a reparameterization technique to estimate the variational lower bound. Experiments on Chinese-English and English-German translation tasks demonstrate that the proposed model achieves significant improvements over both the conventional and variational NMT models.

OneNet: Joint Domain, Intent, Slot Prediction for Spoken Language Understanding

In practice, most spoken language understanding systems process user input in a pipelined manner; first domain is predicted, then intent and semantic slots are inferred according to the semantic frames of the predicted domain. The pipeline approach, however, has some disadvantages: error propagation and lack of information sharing. To address these issues, we present a unified neural network that jointly performs domain, intent, and slot predictions. Our approach adopts a principled architecture for multitask learning to fold in the state-of-the-art models for each task. With a few more ingredients, e.g. orthography-sensitive input encoding and curriculum training, our model delivered significant improvements in all three tasks across all domains over strong baselines, including one using oracle prediction for domain detection, on real user data of a commercial personal assistant.

Sequences, yet Functions: The Dual Nature of Data-Stream Processing

Data-stream processing has continuously risen in importance as the amount of available data has been steadily increasing over the last decade. Besides traditional domains such as data-center monitoring and click analytics, there is an increasing number of network-enabled production machines that generate continuous streams of data. Due to their continuous nature, queries on data-streams can be more complex, and distinctly harder to understand then database queries. As users have to consider operational details, maintenance and debugging become challenging. Current approaches model data-streams as sequences, because this is the way they are physically received. These models result in an implementation-focused perspective. We explore an alternate way of modeling datastreams by focusing on time-slicing semantics. This focus results in a model based on functions, which is better suited for reasoning about query semantics. By adapting the definitions of relevant concepts in stream processing to our model, we illustrate the practical useful- ness of our approach. Thereby, we link data-streams and query primitives to concepts in functional programming and mathematics. Most noteworthy, we prove that data-streams are monads, and show how to derive monad definitions for current data-stream models. We provide an abstract, yet practical perspective on data- stream related subjects based on a sound, consistent query model. Our work can serve as solid foundation for future data-stream query-languages.

A Bayesian Conjugate Gradient Method

A fundamental task in numerical computation is the solution of large linear systems. The conjugate gradient method is an iterative method which offers rapid convergence to the solution, particularly when an effective preconditioner is employed. However, for more challenging systems a substantial error can be present even after many iterations have been performed. The estimates obtained in this case are of little value unless further information can be provided about the numerical error. In this paper we propose a novel statistical model for this numerical error set in a Bayesian framework. Our approach is a strict generalisation of the conjugate gradient method, which is recovered as the posterior mean for a particular choice of prior. The estimates obtained are analysed with Krylov subspace methods and a contraction result for the posterior is presented. The method is then analysed in a simulation study as well as being applied to a challenging problem in medical imaging.

StressedNets: Efficient Feature Representations via Stress-induced Evolutionary Synthesis of Deep Neural Networks

The computational complexity of leveraging deep neural networks for extracting deep feature representations is a significant barrier to its widespread adoption, particularly for use in embedded devices. One particularly promising strategy to addressing the complexity issue is the notion of evolutionary synthesis of deep neural networks, which was demonstrated to successfully produce highly efficient deep neural networks while retaining modeling performance. Here, we further extend upon the evolutionary synthesis strategy for achieving efficient feature extraction via the introduction of a stress-induced evolutionary synthesis framework, where stress signals are imposed upon the synapses of a deep neural network during training to induce stress and steer the synthesis process towards the production of more efficient deep neural networks over successive generations and improved model fidelity at a greater efficiency. The proposed stress-induced evolutionary synthesis approach is evaluated on a variety of different deep neural network architectures (LeNet5, AlexNet, and YOLOv2) on different tasks (object classification and object detection) to synthesize efficient StressedNets over multiple generations. Experimental results demonstrate the efficacy of the proposed framework to synthesize StressedNets with significant improvement in network architecture efficiency (e.g., 40x for AlexNet and 33x for YOLOv2) and speed improvements (e.g., 5.5x inference speed-up for YOLOv2 on an Nvidia Tegra X1 mobile processor).

Low-Shot Learning from Imaginary Data

Humans can quickly learn new visual concepts, perhaps because they can easily visualize or imagine what novel objects look like from different views. Incorporating this ability to hallucinate novel instances of new concepts might help machine vision systems perform better low-shot learning, i.e., learning concepts from few examples. We present a novel approach to low-shot learning that uses this idea. Our approach builds on recent progress in meta-learning (‘learning to learn’) by combining a meta-learner with a ‘hallucinator’ that produces additional training examples, and optimizing both models jointly. Our hallucinator can be incorporated into a variety of meta-learners and provides significant gains: up to a 6 point boost in classification accuracy when only a single training example is available, yielding state-of-the-art performance on the challenging ImageNet low-shot classification benchmark.

A Matrix Positivstellensatz with lifting polynomials
Fast Uplink Grant for Machine Type Communications: Challenges and Opportunities
Dynamic compensation and homeostasis: a feedback control perspective
Emergent Planarity in two-dimensional Ising Models with finite-range Interactions
What Level of Quality can Neural Machine Translation Attain on Literary Text?
Changing and unchanging of the domination number of a graph: Path addition numbers
Two-Stage LASSO ADMM Signal Detection Algorithm For Large Scale MIMO
Smoothing splines on Riemannian manifolds, with applications to 3D shape space
On the Complexity of the Weighted Fussed Lasso
Vehicle Routing with Subtours
Two-stack-sorting with pop stacks
Divide and Recombine for Large and Complex Data: Model Likelihood Functions using MCMC
Robust port-Hamiltonian representations of passive systems
An octree cells occupancy geometric dimensionality descriptor for massive on-server point cloud visualisation and classification
Global Convergence of Policy Gradient Methods for Linearized Control Problems
Student Beats the Teacher: Deep Neural Networks for Lateral Ventricles Segmentation in Brain MR
Centralized ‘big science’ communities more likely generate non-replicable results
Resistance growth of branching random networks
Latent nested nonparametric priors
Conceptualizing and Evaluating Replication Across Domains of Behavioral Research
Multi-Label Learning from Medical Plain Text with Convolutional Residual Models
Circular Antenna Array Design for Breast Cancer Detection
A Finite Block Length Achievability Bound for Low Probability of Detection Communication
A Human-Grounded Evaluation Benchmark for Local Explanations of Machine Learning
One Way Function Candidate based on the Collatz Problem
Real-time Road Traffic Information Detection Through Social Media
Reed-Muller Sequences for 5G Grant-free Massive Access
Inferring Semantic Layout for Hierarchical Text-to-Image Synthesis
On the Analysis of Puncturing for Finite-Length Polar Codes: Boolean Function Approach
Grounded Language Understanding for Manipulation Instructions Using GAN-Based Classification
Boolean Function Analogs of Covering Systems
On the I/O Costs of Some Repair Schemes for Full-Length Reed-Solomon Codes
Throughput Maximization in Cloud Radio Access Networks using Network Coding
Factor graph fragmentization of expectation propagation
Exact Error and Erasure Exponents for the Asymmetric Broadcast Channel
Generalized Reed-Muller codes over Galois rings
Steady-state analysis of the Join the Shortest Queue model in the Halfin-Whitt regime
Asynchronous Bidirectional Decoding for Neural Machine Translation
Localization-Aware Active Learning for Object Detection
Round- and Message-Optimal Distributed Part-Wise Aggregation
Understanding the Disharmony between Dropout and Batch Normalization by Variance Shift
Total dominator coloring of central graphs
Image denoising and restoration with CNN-LSTM Encoder Decoder with Direct Attention
An Accurate and Real-time Self-blast Glass Insulator Location Method Based On Faster R-CNN and U-net with Aerial Images
Algorithms for Computing Wiener Indices of Acyclic and Unicyclic Graphs
Adversarial Learning for Chinese NER from Crowd Annotations
Constraint-free Natural Image Reconstruction from fMRI Signals Based on Convolutional Neural Network
On derived equivalences for categories of generalized intervals of a finite poset
Empirical Explorations in Training Networks with Discrete Activations
GitGraph – Architecture Search Space Creation through Frequent Computational Subgraph Mining
Embedding a $θ$-invariant code into a complete one
On Hamiltonian and Hamilton-connected digraphs
Universal disorder-induced broadening of phonon bands: from disordered lattices to glasses
Deep Multi-Spectral Registration Using Invariant Descriptor Learning
Fully Convolutional Multi-scale Residual DenseNets for Cardiac Segmentation and Automated Cardiac Diagnosis using Ensemble of Classifiers
A theorem on even pancyclic bipartite digraphs
On the Kernel of $\mathbb{Z}_{2^s}$-Linear Hadamard Codes
Sparsity Preserving Optimal Control of Discretized PDE Systems
Multicolour containers, extremal entropy and counting
The cross-index of a complete graph based on a hamiltonian cycle
Scaling Laws and Warning Signs for Bifurcations of SPDEs
Lower bounds for Combinatorial Algorithms for Boolean Matrix Multiplication
Forward-Invariance and Wong-Zakai Approximation for Stochastic Moving Boundary Problems
A Multi-Agent Neural Network for Dynamic Frequency Reuse in LTE Networks
Enabling Quality-Driven Scalable Video Transmission over Multi-User NOMA System
Dual vibration configuration interaction (DVCI). A novel factorisation of molecular Hamiltonian for high performance infrared spectrum computation
Calculating $p$-values and their significances with the Energy Test for large datasets
Device-to-Device Aided Multicasting
Three-dimensional chimera patterns in networks of spiking neuron oscillators
A Survey of Physical Layer Security Techniques for 5G Wireless Networks and Challenges Ahead
de Finetti reductions for partially exchangeable probability distributions
Rank Selection of CP-decomposed Convolutional Layers with Variational Bayesian Matrix Factorization
Assessing Bayesian Nonparametric Log-Linear Models: an application to Disclosure Risk estimation
Simplified Versions of the Conditional Gradient Method
A probabilistic proof of Perron’s theorem
A new characterization of endogeny
Robust sustainable development assessment with composite indices aggregating interacting dimensions: the hierarchical-SMAA-Choquet integral approach
Long-term Visual Localization using Semantically Segmented Images
Unsupervised Representation Learning with Laplacian Pyramid Auto-encoders
Joint registration and synthesis using a probabilistic model for alignment of MRI and histological sections
Bounds on the Effective-length of Optimal Codes for Interference Channel with Feedback
Social Network based Short-Term Stock Trading System
The Frechet distribution: Estimation and Application an Overview
Critical exponents of infinite balanced words
Re-ID done right: towards good practices for person re-identification
Joint CSI Estimation, Beamforming and Scheduling Design for Wideband Massive MIMO System
Learning Deep Features for One-Class Classification
A note on Harris’ ergodic theorem, controllability and perturbations of harmonic networks
Subword complexity and power avoidance
Rooted tree maps and the Kawashima relations for multiple zeta values
Coexistence of 5G mmWave Users with Incumbent Fixed Stations over 70 and 80 GHz
On the Direction of Discrimination: An Information-Theoretic Analysis of Disparate Impact in Machine Learning
Ambulance Emergency Response Optimization in Developing Countries
Interference Mitigation Techniques for Coexistence of 5G mmWave Users with Incumbents at 70 and 80 GHz
Expectation Propagation for Approximate Inference: Free Probability Framework
An Automated System for Epilepsy Detection using EEG Brain Signals based on Deep Learning Approach
Combinatorial Preconditioners for Proximal Algorithms on Graphs