methods: 我们使用 basic Newton method with gradient regularization 来解决这个问题。在无约束情况下，这种算法只需要在每步完成一个简单的矩阵反转操作（解决一个线性系统）。我们证明了这种算法在全球Linear rate 上具有快速的性能，与信任区间算法的复杂性 bound 相同，而我们的方法更加简单实现。
results: 我们发现，使用 quasi-self-concordant 函数的 Newton method 可以在多个实际问题中实现快速的全球Linear rate，无需进一步假设强或平均凸性 для目标函数。这些问题包括Logistic Regression、Soft Maximum 和 Matrix Scaling。Abstract
We study the composite convex optimization problems with a Quasi-Self-Concordant smooth component. This problem class naturally interpolates between classic Self-Concordant functions and functions with Lipschitz continuous Hessian. Previously, the best complexity bounds for this problem class were associated with trust-region schemes and implementations of a ball-minimization oracle. In this paper, we show that for minimizing Quasi-Self-Concordant functions we can use instead the basic Newton Method with Gradient Regularization. For unconstrained minimization, it only involves a simple matrix inversion operation (solving a linear system) at each step. We prove a fast global linear rate for this algorithm, matching the complexity bound of the trust-region scheme, while our method remains especially simple to implement. Then, we introduce the Dual Newton Method, and based on it, develop the corresponding Accelerated Newton Scheme for this problem class, which further improves the complexity factor of the basic method. As a direct consequence of our results, we establish fast global linear rates of simple variants of the Newton Method applied to several practical problems, including Logistic Regression, Soft Maximum, and Matrix Scaling, without requiring additional assumptions on strong or uniform convexity for the target objective.
摘要
我们研究复合凸优化问题中具有半自相关函数组成部分。这个问题类型天然地 interpolates between经典的自相关函数和具有 lipschitz 连续导数的函数。在过去，这个问题类型的最佳复杂性界限与信任区间算法和实现球形最小化器相关。在这篇论文中，我们证明可以使用基本的新顿方法与梯度规则化来减少 quasi-self-concordant 函数的最小化问题。无需进行额外的假设或假设，我们证明了这种算法在全球linear rate具有相同的复杂性界限，而且该算法的实现非常简单。然后，我们引入对应的 dual 新顿方法，并基于它，开发了对这个问题类型的加速新顿方案，进一步改善了基本方法的复杂性因子。作为直接结论，我们建立了基本新顿方法应用于一些实际问题的快速全球线性率，包括逻辑回归、软最大值和矩阵压缩，无需进行额外的假设或假设强或均匀凸性。
Diversified Ensemble of Independent Sub-Networks for Robust Self-Supervised Representation Learning
paper_authors: Amirhossein Vahidi, Lisa Wimmer, Hüseyin Anil Gündüz, Bernd Bischl, Eyke Hüllermeier, Mina Rezaei for: 这个论文主要目标是提高深度学习模型的性能、估计不确定性和稳定性。methods: 这个论文使用了一种新的自我超vised训练方法，利用一个 ensemble of 独立的子网络，并使用一个新的损失函数来鼓励多样性。results: 这个论文的实验结果表明，使用这种方法可以efficiently建立一个多样化的sub-model ensemble，从而实现了高度准确的预测估计和模型不确定性的良好均衡。这种方法在多种任务中表现出色，包括随机数据生成、数据腐坏检测、 semi-supervised setting等。Abstract
Ensembling a neural network is a widely recognized approach to enhance model performance, estimate uncertainty, and improve robustness in deep supervised learning. However, deep ensembles often come with high computational costs and memory demands. In addition, the efficiency of a deep ensemble is related to diversity among the ensemble members which is challenging for large, over-parameterized deep neural networks. Moreover, ensemble learning has not yet seen such widespread adoption, and it remains a challenging endeavor for self-supervised or unsupervised representation learning. Motivated by these challenges, we present a novel self-supervised training regime that leverages an ensemble of independent sub-networks, complemented by a new loss function designed to encourage diversity. Our method efficiently builds a sub-model ensemble with high diversity, leading to well-calibrated estimates of model uncertainty, all achieved with minimal computational overhead compared to traditional deep self-supervised ensembles. To evaluate the effectiveness of our approach, we conducted extensive experiments across various tasks, including in-distribution generalization, out-of-distribution detection, dataset corruption, and semi-supervised settings. The results demonstrate that our method significantly improves prediction reliability. Our approach not only achieves excellent accuracy but also enhances calibration, surpassing baseline performance across a wide range of self-supervised architectures in computer vision, natural language processing, and genomics data.
摘要
ensemble 一种广泛应用的方法是增强模型性能，估计不确定性，并提高深度学习中的稳定性。然而，深度集成常常带来高计算成本和内存需求。此外，集成学习的效率与集成成员之间的多样性有直接关系，而大型、过参数化的深度神经网络中实现多样性是一项挑战。此外，集成学习还尚未得到广泛的应用，而且在无监督或自监督学习中实现集成学习是一项挑战。为了解决这些挑战，我们提出了一种新的自监督训练方法，该方法利用了多个独立的子网络，并且使用了一种新的损失函数，以促进多样性。我们的方法可以高效地建立一个多样性较高的子模型集成，从而获得高度准确的模型不确定性估计，而且与传统的深度自监督集成相比，计算成本减少了较多。为了评估我们的方法的效果，我们在各种任务上进行了广泛的实验，包括内部概率泛化、外部泛化检测、数据损害和半监督设置。结果表明，我们的方法可以显著提高预测可靠性。我们的方法不仅达到了出色的准确率，还可以提高准确性的抽象，在计算机视觉、自然语言处理和生物数据中的各种自监督架构上都达到了或超过了基eline性能。
Hybrid PLS-ML Authentication Scheme for V2I Communication Networks
results: 我们的实验结果表明，使用我们的方案可以减少False报警和遗漏检测的可能性，并且在 missed detection 方面表现更好于基eline方案。Abstract
Vehicular communication networks are rapidly emerging as vehicles become smarter. However, these networks are increasingly susceptible to various attacks. The situation is exacerbated by the rise in automated vehicles complicates, emphasizing the need for security and authentication measures to ensure safe and effective traffic management. In this paper, we propose a novel hybrid physical layer security (PLS)-machine learning (ML) authentication scheme by exploiting the position of the transmitter vehicle as a device fingerprint. We use a time-of-arrival (ToA) based localization mechanism where the ToA is estimated at roadside units (RSUs), and the coordinates of the transmitter vehicle are extracted at the base station (BS).Furthermore, to track the mobility of the moving legitimate vehicle, we use ML model trained on several system parameters. We try two ML models for this purpose, i.e., support vector regression and decision tree. To evaluate our scheme, we conduct binary hypothesis testing on the estimated positions with the help of the ground truths provided by the ML model, which classifies the transmitter node as legitimate or malicious. Moreover, we consider the probability of false alarm and the probability of missed detection as performance metrics resulting from the binary hypothesis testing, and mean absolute error (MAE), mean square error (MSE), and coefficient of determination $\text{R}^2$ to further evaluate the ML models. We also compare our scheme with a baseline scheme that exploits the angle of arrival at RSUs for authentication. We observe that our proposed position-based mechanism outperforms the baseline scheme significantly in terms of missed detections.
摘要
Our scheme uses a time-of-arrival (ToA) based localization mechanism, where the ToA is estimated at roadside units (RSUs) and the coordinates of the transmitter vehicle are extracted at the base station (BS). Additionally, we use ML models to track the mobility of the moving legitimate vehicle. We test two ML models for this purpose: support vector regression and decision tree.To evaluate our scheme, we conduct binary hypothesis testing on the estimated positions with the help of the ground truths provided by the ML model. This classifies the transmitter node as legitimate or malicious. We also consider the probability of false alarm and the probability of missed detection as performance metrics, as well as mean absolute error (MAE), mean square error (MSE), and coefficient of determination $\text{R}^2$ to evaluate the ML models.We compare our scheme with a baseline scheme that exploits the angle of arrival at RSUs for authentication and find that our proposed position-based mechanism outperforms the baseline scheme in terms of missed detections. Our scheme provides a more secure and effective way to authenticate vehicles in vehicular communication networks.