Title: On Casting Importance Weighted Autoencoder to an EM Algorithm to Learn Deep Generative Models

Abstract:

We propose a new and general approach to learn deep generative models. Our approach is based on a new observation that the importance weighted autoencoders (IWAE) can be understood as a procedure of estimating the MLE with an EM algorithm. Utilizing this interpretation, we develop a new learning algorithm called importance weighted EM algorithm (IWEM). IWEM is an EM algorithm with importance sampling (IS) where the proposal distribution is carefully selected to reduce the variance due to IS. In addition, we devise an annealing strategy to stabilize the learning algorithm. For missing data problems, we propose a modified IWEM algorithm called miss-IWEM. Using multiple benchmark datasets, we demonstrate empirically that our proposed methods outperform IWAE with significant margins for both fully-observed and missing data cases.

Title: Nonconvex Sparse Regularization For Deep Neural Networks and its Optimal Property

Abstract:

Sparsity is a key ingredient in the success of learners both theoretically and computationally. This is also the case for deep neural networks (DNNs). A number of empirical observations show that sparse DNNs can dramatically reduce computation time and memory without appreciably harming prediction power. Furthermore, recent theoretical studies proved that DNN estimators with a certain sparsity constraint can attain optimal convergence rates for regression and classification problems. However, they only considered the empirical risk minimizer under the sparsity constraint, where optimization is almost impossible in practice due to its discrete nature. In this research, we propose a novel penalized empirical risk minimization method for estimating sparse DNNs with a scalable computation algorithm. The proposed method yields sparse DNNs that can achieve optimal convergence rates of excess risks for various learning problems including regression and binary classification. We demonstrate the empirical performance of the proposed method and compare it with other competitors for various benchmark datasets.

Title: Predicting location of suitable groundwater for brewing coffee using tree-based ensemble machine learning

Abstract:

Recently, groundwater is used as a source of drinking purposes such as water, coffee, beer and other beverages. The range of water quality for producing high-quality of beverages is different from each usage, so it is important to find suitable groundwater location for each purpose in the aspect of water industry. This study was conducted to predict the suitable location for brewing coffee in Gangwon Province, South Korea using tree-based ensemble machine learning. Appropriate water quality standard for brewing coffee is known as TDS of 75~250 mg/L and calcium hardness of 17~85 mg/L from recent research. Boosted Regression Trees (BRT), Random Forests (RF) and Extremely Randomized Trees (ERT) were used as tree-based ensemble method. Response indicating suitable or unsuitable groundwater for brewing coffee was determined by 254 wells’ water quality data, and predictor variables were composed of slope, altitude, drainage grade, effective soil depth, soil composition, land use, and hydrogeology based on GIS data. Applying models to the test data, all three models showed the area under a curve (AUC) and accuracy more than 0.85 and 0.80, respectively, which indicates high reliability in prediction. Threshold of dividing suitable or unsuitable for brewing coffee is found from Receiver Operating Characteristic (ROC) curve. The BRT showed the highest AUC and accuracy among the three models, therefore, potential map of suitable groundwater location for brewing coffee was suggested by the BRT model. In the condition of lack of water quality data, this research can help to determine location of suitable groundwater for several usages. Keyword: potable groundwater · boosted regression tree · random forest · extremely randomized tree Acknowledgement: This research was supported by the National Research Council of Science and Technology(NST) grant funded by the Korea government(MSIP) (No. CAP-17-05-KIGAM)

Title: Application of neural network model to predict and to evaluate the groundwater level fluctuation

Abstract:

Groundwater is one of valuable water resources used for the various purposes in our life, but it is not limitless. For this reason, prediction and management of the groundwater are very essential work for sustainable use of it. Physics-based models are usually applied on prediction of groundwater level, but they are hard to be implemented successfully when there is any unknown physical property or when subterranean structure is very complicated. In this research, prediction of the groundwater level at riverside area in Yangpyeong, Korea was carried out using neural network model instead. In study area where several natural and anthropogenic factors affect the groundwater level fluctuation, groundwater levels at 8 monitoring wells were well predicted with low range of RMSE errors. Moreover, monthly contributions, which indicate the impact of input variables, were computed to figure out the seasonal variance of influencing factors. This study could suggest another option to predict the groundwater level, and help understanding spatial and temporal variation of impacts of factors affecting the groundwater level fluctuation.

Acknowledgement: This work was supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIP) (No. 2017R1A2B3002119)

Title: 복소 산술 신경망을 통한 물리적 기계학습모형의 건설과 물리적 불변량의 추출