Latent Dirichlet Allocation (LDA) for Topic Modeling of the CFPB Consumer Complaints

A text mining approach is proposed based on latent Dirichlet allocation (LDA) to analyze the Consumer Financial Protection Bureau (CFPB) consumer complaints. The proposed approach aims to extract latent topics in the CFPB complaint narratives, and explores their associated trends over time. The time trends will then be used to evaluate the effectiveness of the CFPB regulations and expectations on financial institutions in creating a consumer oriented culture that treats consumers fairly and prioritizes consumer protection in their decision making processes. The proposed approach can be easily operationalized as a decision support system to automate detection of emerging topics in consumer complaints. Hence, the technology-human partnership between the proposed approach and the CFPB team could certainly improve consumer protections from unfair, deceptive or abusive practices in the financial markets by providing more efficient and effective investigations of consumer complaint narratives.

Anomaly Detection for Water Treatment System based on Neural Network with Automatic Architecture Optimization

We continue to develop our neural network (NN) based forecasting approach to anomaly detection (AD) using the Secure Water Treatment (SWaT) industrial control system (ICS) testbed dataset. We propose genetic algorithms (GA) to find the best NN architecture for a given dataset, using the NAB metric to assess the quality of different architectures. The drawbacks of the F1-metric are analyzed. Several techniques are proposed to improve the quality of AD: exponentially weighted smoothing, mean p-powered error measure, individual error weight for each variable, disjoint prediction windows. Based on the techniques used, an approach to anomaly interpretation is introduced.

Preventing Poisoning Attacks on AI based Threat Intelligence Systems

As AI systems become more ubiquitous, securing them becomes an emerging challenge. Over the years, with the surge in online social media use and the data available for analysis, AI systems have been built to extract, represent and use this information. The credibility of this information extracted from open sources, however, can often be questionable. Malicious or incorrect information can cause a loss of money, reputation, and resources; and in certain situations, pose a threat to human life. In this paper, we use an ensembled semi-supervised approach to determine the credibility of Reddit posts by estimating their reputation score to ensure the validity of information ingested by AI systems. We demonstrate our approach in the cybersecurity domain, where security analysts utilize these systems to determine possible threats by analyzing the data scattered on social media websites, forums, blogs, etc.

Statistical Model Compression for Small-Footprint Natural Language Understanding

In this paper we investigate statistical model compression applied to natural language understanding (NLU) models. Small-footprint NLU models are important for enabling offline systems on hardware restricted devices, and for decreasing on-demand model loading latency in cloud-based systems. To compress NLU models, we present two main techniques, parameter quantization and perfect feature hashing. These techniques are complementary to existing model pruning strategies such as L1 regularization. We performed experiments on a large scale NLU system. The results show that our approach achieves 14-fold reduction in memory usage compared to the original models with minimal predictive performance impact.

Analyzing Hypersensitive AI: Instability in Corporate-Scale Machine Learning

Predictive geometric models deliver excellent results for many Machine Learning use cases. Despite their undoubted performance, neural predictive algorithms can show unexpected degrees of instability and variance, particularly when applied to large datasets. We present an approach to measure changes in geometric models with respect to both output consistency and topological stability. Considering the example of a recommender system using word2vec, we analyze the influence of single data points, approximation methods and parameter settings. Our findings can help to stabilize models where needed and to detect differences in informational value of data points on a large scale.

Semantic Parsing: Syntactic assurance to target sentence using LSTM Encoder CFG-Decoder

Semantic parsing can be defined as the process of mapping natural language sentences into a machine interpretable, formal representation of its meaning. Semantic parsing using LSTM encoder-decoder neural networks have become promising approach. However, human automated translation of natural language does not provide grammaticality guarantees for the sentences generate such a guarantee is particularly important for practical cases where a data base query can cause critical errors if the sentence is ungrammatical. In this work, we propose an neural architecture called Encoder CFG-Decoder, whose output conforms to a given context-free grammar. Results are show for any implementation of such architecture display its correctness and providing benchmark accuracy levels better than the literature.

Linear Programming Approximations for Index Coding

Index coding, a source coding problem over broadcast channels, has been a subject of both theoretical and practical interest since its introduction (by Birk and Kol, 1998). In short, the problem can be defined as follows: there is an input \textbf{x} \triangleq (\textbf{x}_1, \dots, \textbf{x}_n), a set of n clients who each desire a single symbol \textbf{x}_i of the input, and a broadcaster whose goal is to send as few messages as possible to all clients so that each one can recover its desired symbol. Additionally, each client has some predetermined ‘side information,’ corresponding to certain symbols of the input \textbf{x}, which we represent as the ‘side information graph’ \mathcal{G}. The graph \mathcal{G} has a vertex v_i for each client and a directed edge (v_i, v_j) indicating that client i knows the jth symbol of the input. Given a fixed side information graph \mathcal{G}, we are interested in determining or approximating the ‘broadcast rate’ of index coding on the graph, i.e. the fewest number of messages the broadcaster can transmit so that every client gets their desired information. Using index coding schemes based on linear programs (LPs), we take a two-pronged approach to approximating the broadcast rate. First, extending earlier work on planar graphs, we focus on approximating the broadcast rate for special graph families such as graphs with small chromatic number and disk graphs. In certain cases, we are able to show that simple LP-based schemes give constant-factor approximations of the broadcast rate, which seem extremely difficult to obtain in the general case. Second, we provide several LP-based schemes for the general case which are not constant-factor approximations, but which strictly improve on the prior best-known schemes.

A Projection Pursuit Forest Algorithm for Supervised Classification

This paper presents a new ensemble learning method for classification problems called projection pursuit random forest (PPF). PPF uses the PPtree algorithm introduced in Lee et al. (2013). In PPF, trees are constructed by splitting on linear combinations of randomly chosen variables. Projection pursuit is used to choose a projection of the variables that best separates the classes. Utilizing linear combinations of variables to separate classes takes the correlation between variables into account which allows PPF to outperform a traditional random forest when separations between groups occurs in combinations of variables. The method presented here can be used in multi-class problems and is implemented into an R (R Core Team, 2018) package, PPforest, which is available on CRAN, with development versions at https://…/PPforest.

Imparting Interpretability to Word Embeddings

As an ubiquitous method in natural language processing, word embeddings are extensively employed to map semantic properties of words into a dense vector representation. They capture semantic and syntactic relations among words but the vector corresponding to the words are only meaningful relative to each other. Neither the vector nor its dimensions have any absolute, interpretable meaning. We introduce an additive modification to the objective function of the embedding learning algorithm that encourages the embedding vectors of words that are semantically related a predefined concept to take larger values along a specified dimension, while leaving the original semantic learning mechanism mostly unaffected. In other words, we align words that are already determined to be related, along predefined concepts. Therefore, we impart interpretability to the word embedding by assigning meaning to its vector dimensions. The predefined concepts are derived from an external lexical resource, which in this paper is chosen as Roget’s Thesaurus. We observe that alignment along the chosen concepts is not limited to words in the Thesaurus and extends to other related words as well. We quantify the extent of interpretability and assignment of meaning from our experimental results. We also demonstrate the preservation of semantic coherence of the resulting vector space by using word-analogy and word-similarity tests. These tests show that the interpretability-imparted word embeddings that are obtained by the proposed framework do not sacrifice performances in common benchmark tests.

ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech

In this work, we propose an alternative solution for parallel wave generation by WaveNet. In contrast to parallel WaveNet (Oord et al., 2018), we distill a Gaussian inverse autoregressive flow from the autoregressive WaveNet by minimizing a novel regularized KL divergence between their highly-peaked output distributions. Our method computes the KL divergence in closed-form, which simplifies the training algorithm and provides very efficient distillation. In addition, we propose the first text-to-wave neural architecture for speech synthesis, which is fully convolutional and enables fast end-to-end training from scratch. It significantly outperforms the previous pipeline that connects a text-to-spectrogram model to a separately trained WaveNet (Ping et al., 2017). We also successfully distill a parallel waveform synthesizer conditioned on the hidden representation in this end-to-end model.

Bounded Information Rate Variational Autoencoders

This paper introduces a new member of the family of Variational Autoencoders (VAE) that constrains the rate of information transferred by the latent layer. The latent layer is interpreted as a communication channel, the information rate of which is bound by imposing a pre-set signal-to-noise ratio. The new constraint subsumes the mutual information between the input and latent variables, combining naturally with the likelihood objective of the observed data as used in a conventional VAE. The resulting Bounded-Information-Rate Variational Autoencoder (BIR-VAE) provides a meaningful latent representation with an information resolution that can be specified directly in bits by the system designer. The rate constraint can be used to prevent overtraining, and the method naturally facilitates quantisation of the latent variables at the set rate. Our experiments confirm that the BIR-VAE has a meaningful latent representation and that its performance is at least as good as state-of-the-art competing algorithms, but with lower computational complexity.

Attend and Rectify: a Gated Attention Mechanism for Fine-Grained Recovery

We propose a novel attention mechanism to enhance Convolutional Neural Networks for fine-grained recognition. It learns to attend to lower-level feature activations without requiring part annotations and uses these activations to update and rectify the output likelihood distribution. In contrast to other approaches, the proposed mechanism is modular, architecture-independent and efficient both in terms of parameters and computation required. Experiments show that networks augmented with our approach systematically improve their classification accuracy and become more robust to clutter. As a result, Wide Residual Networks augmented with our proposal surpasses the state of the art classification accuracies in CIFAR-10, the Adience gender recognition task, Stanford dogs, and UEC Food-100.

FuzzerGym: A Competitive Framework for Fuzzing and Learning

Fuzzing is a commonly used technique designed to test software by automatically crafting program inputs. Currently, the most successful fuzzing algorithms emphasize simple, low-overhead strategies with the ability to efficiently monitor program state during execution. Through compile-time instrumentation, these approaches have access to numerous aspects of program state including coverage, data flow, and heterogeneous fault detection and classification. However, existing approaches utilize blind random mutation strategies when generating test inputs. We present a different approach that uses this state information to optimize mutation operators using reinforcement learning (RL). By integrating OpenAI Gym with libFuzzer we are able to simultaneously leverage advancements in reinforcement learning as well as fuzzing to achieve deeper coverage across several varied benchmarks. Our technique connects the rich, efficient program monitors provided by LLVM Santizers with a deep neural net to learn mutation selection strategies directly from the input data. The cross-language, asynchronous architecture we developed enables us to apply any OpenAI Gym compatible deep reinforcement learning algorithm to any fuzzing problem with minimal slowdown.

Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizer

Autoencoders provide a powerful framework for learning compressed representations by encoding all of the information needed to reconstruct a data point in a latent code. In some cases, autoencoders can ‘interpolate’: By decoding the convex combination of the latent codes for two datapoints, the autoencoder can produce an output which semantically mixes characteristics from the datapoints. In this paper, we propose a regularization procedure which encourages interpolated outputs to appear more realistic by fooling a critic network which has been trained to recover the mixing coefficient from interpolated data. We then develop a simple benchmark task where we can quantitatively measure the extent to which various autoencoders can interpolate and show that our regularizer dramatically improves interpolation in this setting. We also demonstrate empirically that our regularizer produces latent codes which are more effective on downstream tasks, suggesting a possible link between interpolation abilities and learning useful representations.

Rearranging the Familiar: Testing Compositional Generalization in Recurrent Networks

Systematic compositionality is the ability to recombine meaningful units with regular and predictable outcomes, and it’s seen as key to humans’ capacity for generalization in language. Recent work has studied systematic compositionality in modern seq2seq models using generalization to novel navigation instructions in a grounded environment as a probing tool, requiring models to quickly bootstrap the meaning of new words. We extend this framework here to settings where the model needs only to recombine well-trained functional words (such as ‘around’ and ‘right’) in novel contexts. Our findings confirm and strengthen the earlier ones: seq2seq models can be impressively good at generalizing to novel combinations of previously-seen input, but only when they receive extensive training on the specific pattern to be generalized (e.g., generalizing from many examples of ‘X around right’ to ‘jump around right’), while failing when generalization requires novel application of compositional rules (e.g., inferring the meaning of ‘around right’ from those of ‘right’ and ‘around’).

A Hand-Held Multimedia Translation and Interpretation System with Application to Diet Management
Minimizing convex quadratic with variable precision Krylov methods
Guess who Multilingual approach for the automated generation of author-stylized poetry
Clinical Text Classification with Rule-based Features and Knowledge-guided Convolutional Neural Networks
Signal Alignment for Humanoid Skeletons via the Globally Optimal Reparameterization Algorithm
Real-Time Stereo Vision for Road Surface 3-D Reconstruction
Eigenspace-Based Minimum Variance Combined with Delay Multiply and Sum Beamformer: Application to Linear-Array Photoacoustic Imaging
High-Mobility Wideband Massive MIMO Communications: Doppler Compensation, Analysis and Scaling Law
A Fixed-Parameter Linear-Time Algorithm to Compute Principal Typings of Planar Flow Networks
Entanglement Transitions from Holographic Random Tensor Networks
Universal Scaling Theory of the Boundary Geometric Tensor in Disordered Metals
Ricci curvature for parametric statistics via optimal transport
Comparative study of Discrete Wavelet Transforms and Wavelet Tensor Train decomposition to feature extraction of FTIR data of medicinal plants
Weakly Monotone Fock Space and Monotone Convolution of the Wigner Law
NIP omega-categorical structures: the rank 1 case
Hierarchical Multi Task Learning With CTC
Real-time digital signal recovery for a low-pass transfer function system with multiple complex poles
The trinomial transform triangle
The classification of homogeneous finite-dimensional permutation structures
Continuous approximation of $(M_t,M_t, 1)$ distributions with application to production
A Holistic Approach to Forecasting Wholesale Energy Market Prices
Reconstructing Latent Orderings by Spectral Clustering
Datamining a medieval medical text reveals patterns in ingredient choice that reflect biological activity against the causative agents of specified infections
Distributed Second-order Convex Optimization
A Scalable MCEM Estimator for Spatio-Temporal Autoregressive Models
Representational efficiency outweighs action efficiency in human program induction
Fast and Deterministic Approximations for $k$-Cut
CT Image Enhancement Using Stacked Generative Adversarial Networks and Transfer Learning for Lesion Segmentation Improvement
Minimum distance computation of linear codes via genetic algorithms with permutation encoding
Take a Look Around: Using Street View and Satellite Images to Estimate House Prices
Approximation Schemes for Low-Rank Binary Matrix Approximation Problems
What kind of content are you prone to tweet Multi-topic Preference Model for Tweeters
Once reinforced random walk on $\mathbb{Z}\times Γ$
How Consumer Empathy Assist Power Grid in Demand Response
Automatic Identification of Ineffective Online Student Questions in Computing Education
A $φ$-Competitive Algorithm for Scheduling Packets with Deadlines
Efficient Power Flow Management and Peak Shaving in a Microgrid-PV System
A Novel Scheme for Support Identification and Iterative Sampling of Bandlimited Graph Signals
Tomlinson-Harashima Precoded Rate-Splitting for Multiuser MIMO Systems
Evaluating Word Embeddings in Multi-label Classification Using Fine-grained Name Typing
Efficient Training on Very Large Corpora via Gramian Estimation
A Tale of Santa Claus, Hypergraphs and Matroids
Tracking Sparse mmWave Channel: Performance Analysis under Intra-Cluster Angular Spread
Is the SIC Outcome There When Nobody Looks
Achievable Rate maximization by Passive Intelligent Mirrors
Asymptotically Optimal Estimation Algorithm for the Sparse Signal with Arbitrary Distributions
Performance, Power, and Area Design Trade-offs in Millimeter-Wave Transmitter Beamforming Architectures
Few-Shot Adaptation for Multimedia Semantic Indexing
Negative Imaginary State Feedback Control with a Prescribed Degree of Stability
Coexistence of scale invariant and rhythmic behavior in self-organized criticality
A Machine Learning Approach for Detecting Students at Risk of Low Academic Achievement
Isolating effects of age with fair representation learning when assessing dementia
Disorder-robust entanglement transport
Efficient Sampling of Bandlimited Graph Signals
Exponential Stabilization for Ito Stochastic Systems with Multiple Input Delays
Monocular Object Orientation Estimation using Riemannian Regression and Classification Networks
Convex Relaxations in Power System Optimization: A Brief Introduction
Stability of generalized Petersen graphs
UAV-Based in-band Integrated Access and Backhaul for 5G Communications
Cooperative Adaptive Cruise Control for Connected Autonomous Vehicles by Factoring Communication-Related Constraints
Optimal estimation of Gaussian mixtures via denoised method of moments
Limiting spectral distribution of the product of truncated Haar unitary matrices
ArticulatedFusion: Real-time Reconstruction of Motion, Geometry and Segmentation Using a Single Depth Camera
Chest X-rays Classification: A Multi-Label and Fine-Grained Problem
Normalization of ternary generalized pseudostandard words
Ricci-flat graphs with girth four
Towards Explainable and Controllable Open Domain Dialogue Generation with Dialogue Acts
Visual Domain Adaptation with Manifold Embedded Distribution Alignment
Machine Learning Based Featureless Signalling
A hybrid algorithm for the two-trust-region subproblem
On the modular Erdös-Burgess constant
Simple robust genomic prediction and outlier detection for a multi-environmental field trial
Searching for network modules
In pixels we trust: From Pixel Labeling to Object Localization and Scene Categorization
Label Aggregation via Finding Consensus Between Models
Deep Sequential Multi-camera Feature Fusion for Person Re-identification
Mr. DLib’s Living Lab for Scholarly Recommendations
SPDEs with Space-Mean Dynamics
Deep Adaptive Proposal Network for Object Detection in Optical Remote Sensing Images
Quantifying Volatility Reduction in German Day-ahead Spot Market in the Period 2006 through 2016
Sequence to Logic with Copy and Cache
On the Phase Tracking Reference Signal (PT-RS) Design for 5G New Radio (NR)
QoS and Coverage Aware Dynamic High Density Vehicle Platooning (HDVP)
Birkhoff-von Neumann Graphs that are PM-compact
Automated Phenotyping of Epicuticular Waxes of Grapevine Berries Using Light Separation and Convolutional Neural Networks
Indexing Execution Patterns in Workflow Provenance Graphs through Generalized Trie Structures
Generative Adversarial Networks for MR-CT Deformable Image Registration
Can We Assess Mental Health through Social Media and Smart Devices Addressing Bias in Methodology and Evaluation
MITK-ModelFit: generic open-source framework for model fits and their exploration in medical imaging – design, implementation and application on the example of DCE-MRI
Test-time augmentation with uncertainty estimation for deep learning-based medical image segmentation
Stochastic Quantization for the Edwards Measure of Fractional Brownian Motion with $Hd=1$
Green function of a random walk in a cone
Speeding up the Hyperparameter Optimization of Deep Convolutional Neural Networks
Revisiting Cross Modal Retrieval
On some special classes of contact $B_0$-VPG graphs
An entropy generation formula on $RCD(K,\infty)$ spaces
Fuzzy quantification for linguistic data analysis and data mining
ISIC 2018-A Method for Lesion Segmentation
Localization of disordered harmonic chain with long-range correlation
Image Reconstruction via Variational Network for Real-Time Hand-Held Sound-Speed Imaging
Delay and Communication Tradeoffs for Blockchain Systems with Lightweight IoT Clients
Modeling Visual Context is Key to Augmenting Object Detection Datasets
Semi-Dense 3D Reconstruction with a Stereo Event Camera
Selective Zero-Shot Classification with Augmented Attributes
On the almost-principal minors of a symmetric matrix
Can Artificial Intelligence Reliably Report Chest X-Rays : Radiologist Validation of an Algorithm trained on 1.2 Million X-Rays
On the Sweep Map for Fuss Rational Dyck Paths
Two algorithms for a fully coupled and consistently macroscopic PDE-ODE system modeling a moving bottleneck on a road
Conditional Random Fields as Recurrent Neural Networks for 3D Medical Imaging Segmentation
Stochastic Model Predictive Control with Discounted Probabilistic Constraints
Guided Upsampling Network for Real-Time Semantic Segmentation
Three for one and one for three: Flow, Segmentation, and Surface Normals
Prophet Secretary Through Blind Strategies
Robust Oil-spill Forensics and Petroleum Source Differentiation using Quantized Peak Topography Maps
A Microservice-enabled Architecture for Smart Surveillance using Blockchain Technology
Edge colourings and topological graph polynomials
Improving Simple Models with Confidence Profiles
Finding Minimum Volume Circumscribing Ellipsoids Using Copositive Programming
A Strategy of MR Brain Tissue Images’ Suggestive Annotation Based on Modified U-Net
Harmonic functions on mated-CRT maps
Hybrid scene Compression for Visual Localization
An invariance principle for ergodic scale-free random environments
Exact Algorithms for Finding Well-Connected 2-Clubs in Real-World Graphs: Theory and Experiments
Using Deep Neural Networks to Translate Multi-lingual Threat Intelligence
Exact asymptotics for Duarte and supercritical rooted kinetically constrained models
Bio-Measurements Estimation and Support in Knee Recovery through Machine Learning
Emulating malware authors for proactive protection using GANs over a distributed image visualization of the dynamic file behavior
Optimal Las Vegas Approximate Near Neighbors in $\ell_p$
Self-Organizing Maps as a Storage and Transfer Mechanism in Reinforcement Learning
Limited Memory Kelley’s Method Converges for Composite Convex and Submodular Objectives
Attention-Guided Curriculum Learning for Weakly Supervised Classification and Localization of Thoracic Diseases on Chest Radiographs
Positional Value in Soccer: Expected League Points Added above Replacement
An expansion formula for type A and Kronecker quantum cluster algebras
A unified theory of adaptive stochastic gradient descent as Bayesian filtering
Partial recovery bounds for clustering with the relaxed $K$means
Realization Spaces of Uniform Phased Matroids
A geometric integration approach to nonsmooth, nonconvex optimisation
Transfer Learning for Action Unit Recognition
Capsule Networks against Medical Imaging Data Challenges
Compositional GAN: Learning Conditional Image Composition
Nested Covariance Determinants and Restricted Trek Separation in Gaussian Graphical Models
A linear-time algorithm for generalized trust region problems