Xiaoge Deng

Behavior Tree Generation with LLM-MCTS-BT as a Pre-Planner Bridging Priors and Uncertainty

Haoming Wang, xueying wang, Xiaoge Deng, Bin Li et autres

While Behavior Trees (BTs) offer modular control for robotics, their manual construction requires significant expertise. Current Large Language Model (LLM) approaches to automation often struggle to leverage BT structure, lacking transparency, systematic exploration, and independence from prior knowledge. We propose LLM-MCTS-BT, a …

Toward Understanding the Generalizability of Delayed Stochastic Gradient Descent

Xiaoge Deng, Li Shen, Shengwei Li, Dacheng Tao

Stochastic gradient descent (SGD) performed in an asynchronous manner plays a crucial role in training large-scale machine learning models. However, the generalization performance of asynchronous delayed SGD, which is an essential metric for assessing machine learning algorithms, has rarely been explored. Existing …

cn, sg (code pays fourni par la source)

Sharpness-Aware Minimization with Adaptive Regularization for Training Deep Neural Networks

Jinping Zou, Xiaoge Deng, Tao Sun

Sharpness-Aware Minimization (SAM) has proven highly effective in improving model generalization in machine learning tasks. However, SAM employs a fixed hyperparameter associated with the regularization to characterize the sharpness of the model. Despite its success, research on adaptive regularization methods based on …

cn (code pays fourni par la source)

Sharpness-Aware Minimization with Adaptive Regularization for Training Deep Neural Networks

Jinping Zou, Xiaoge Deng, Tong Sun

Sharpness-Aware Minimization (SAM) has proven highly effective in improving model generalization in machine learning tasks. However, SAM employs a fixed hyperparameter associated with the regularization to characterize the sharpness of the model. Despite its success, research on adaptive regularization methods based on …

Federated Prediction-Powered Inference from Decentralized Data

Ping Luo, Xiaoge Deng, Ziqing Wen, Tao Sun et autres

In various domains, the increasing application of machine learning allows researchers to access inexpensive predictive data, which can be utilized as auxiliary data for statistical inference. Although such data are often unreliable compared to gold-standard datasets, Prediction-Powered Inference (PPI) has been proposed …

Communication-Efficient Distributed Learning via Sparse and Adaptive Stochastic Gradient

Xiaoge Deng, Tao Sun, Xicheng Lu

Gradient-based optimization methods implemented on distributed computing architectures are increasingly used to tackle large-scale machine learning applications. A key bottleneck in such distributed systems is the high communication overhead for exchanging information, such as stochastic gradients, between workers. The inherent causes of …

cn (code pays fourni par la source)

Decentralized stochastic sharpness-aware minimization algorithm

S. Chen, Xiaoge Deng, Dongpo Xu, Tao Sun et autres

cn (code pays fourni par la source)

Stability and Generalization of Asynchronous SGD: Sharper Bounds Beyond Lipschitz and Smoothness

Xiaoge Deng, Tao Sun, Shengwei Li, Xicheng Lu

Normalized Stochastic Heavy Ball with Adaptive Momentum1

Ziqing Wen, Xiaoge Deng, Tao Sun, Dongsheng Li

The heavy ball momentum technique is widely used in accelerating the machine learning training process, which has demonstrated significant practical success in optimization tasks. However, most heavy ball methods require a preset hyperparameter that will result in excessive tuning, and a calibrated …

cn (code pays fourni par la source)

Towards Understanding the Generalizability of Delayed Stochastic Gradient Descent

Xiaoge Deng, Li Shen, Shengwei Li, Tao Sun et autres

Stochastic gradient descent (SGD) performed in an asynchronous manner plays a crucial role in training large-scale machine learning models. However, the generalization performance of asynchronous delayed SGD, which is an essential metric for assessing machine learning algorithms, has rarely been explored. Existing …

Stability-Based Generalization Analysis of the Asynchronous Decentralized SGD

Xiaoge Deng, Tong Sun, Shengwei Li, Dongsheng Li

The generalization ability often determines the success of machine learning algorithms in practice. Therefore, it is of great theoretical and practical importance to understand and bound the generalization error of machine learning algorithms. In this paper, we provide the first generalization results …

cn (code pays fourni par la source)

Accelerate distributed deep learning with cluster-aware sketch quantization

Keshi Ge, Yiming Zhang, Yongquan Fu, Zhiquan Lai et autres

cn (code pays fourni par la source)