A framework is introduced for solving a sequence of slowly changing optimization problems, including those arising in regression and classification applications, using optimization algorithms such as stochastic gradient descent (SGD). The optimization problems change slowly in the sense that the minimizers change at either a fixed or bounded rate. A method based on estimates of the change in the minimizers and properties of the optimization algorithm is introduced for adaptively selecting the number of samples needed from the distributions underlying each problem in order to ensure that the excess risk, i.e., the expected gap between the loss achieved by the approximate minimizer produced by the optimization algorithm and the exact minimizer, does not exceed a target level. Experiments with synthetic and real data are used to confirm that this approach performs well.

Which programming error messages are the most common? We investigate this question, motivated by writing error explanations for novices. We consider large data sets in Python and Java that include both syntax and run-time errors. In both data sets, after grouping essentially identical messages, the error message frequencies empirically resemble Zipf-Mandelbrot distributions. We use a maximum-likelihood approach to fit the distribution parameters. This gives one possible way to contrast languages or compilers quantitatively.

IllinoisSL is a Java library for learning structured prediction models. It supports structured Support Vector Machines and structured Perceptron. The library consists of a core learning module and several applications, which can be executed from command-lines. Documentation is provided to guide users. In Comparison to other structured learning libraries, IllinoisSL is efficient, general, and easy to use.

The generalized regression model is an important semiparametric generalization to the linear regression model. It assumes there exist unknown monotone increasing link functions connecting the response to a single index of explanatory variables . The generalized regression model covers a lot of well-exploited statistical models. It is appealing in many applications where regression models are regularly employed. In low dimensions, rank-based M-estimators are recommended, giving root- consistent estimators of . However, their applications to high dimensional data are questionable. This is mainly due to the discontinuity of loss function . In detail, (i) computationally, because of ‘s non-smoothness, the optimization problem is intractable; (ii) theoretically, the discontinuity of renders difficulty for analysis in high dimensions. In contrast, this paper suggests a simple, yet powerful, smoothing approach for rank-based estimators. A family of smoothing functions is provided, and the amount of smoothing necessary for efficient inference is carefully calculated. We show the resulting estimators are scaling near-optimal, i.e., they are consistent estimators of as long as are within a near-optimal range (Here represents the sparsity degree). These are the first such results in the literature. The proposed approaches’ power is further verified empirically.