results: numerical studies表明，QOCO算法可以高效地利用边缘节点的计算资源，可以完成14%更多的任务，降低任务延迟和能量消耗，减少9%和6%。这些改进共同带来了最少37%的QoE提升。Abstract
In the realm of mobile edge computing (MEC), efficient computation task offloading plays a pivotal role in ensuring a seamless quality of experience (QoE) for users. Maintaining a high QoE is paramount in today's interconnected world, where users demand responsive and reliable services. This challenge stands as one of the most primary key factors contributing to handling dynamic and uncertain mobile environment. In this study, we delve into computation offloading in MEC systems, where strict task processing deadlines and energy constraints can adversely affect the system performance. We formulate the computation task offloading problem as a Markov decision process (MDP) to maximize the long-term QoE of each user individually. We propose a decentralized QoE-oriented computation offloading (QOCO) algorithm based on deep reinforcement learning (DRL) that empowers mobile devices to make their offloading decisions without requiring knowledge of decisions made by other devices. Through numerical studies, we evaluate the performance of QOCO. Simulation results validate that the QOCO algorithm efficiently exploits the computational resources of edge nodes. Consequently, it can complete 14% more tasks and reduce task delay and energy consumption by 9% and 6%, respectively. These together contribute to a significant improvement of at least 37% in average QoE compared to an existing algorithm.
摘要
在移动边缘计算（MEC）领域，有效地卸载计算任务是保证用户无缝体验质量（QoE）的关键因素。在今天的全球化社会中，用户对服务的响应速度和可靠性有高度的要求。这种挑战是MEC系统中处理动态和不确定的 mobilenvionment的一个Primary key factor。在这种研究中，我们探讨MEC系统中的计算任务卸载问题，其中严格的任务处理截止时间和能量限制可能会对系统性能产生负面影响。我们将计算任务卸载问题表示为Markov决策过程（MDP），以最大化每个用户的长期QoE。我们提出了一种基于深度学习（DRL）的QoE- ориентирован的计算卸载算法（QOCO），该算法让移动设备通过不需要知道其他设备的决策来做出卸载决策。通过数字实验，我们评估了QOCO算法的性能。计算结果表明，QOCO算法可以有效地利用边缘节点的计算资源，可以完成14%更多的任务，同时降低任务延迟和能量消耗的9%和6%。这些因素共同带来了至少37%的QoE提高。
Forward $χ^2$ Divergence Based Variational Importance Sampling
results: VIS 在各种流行的 latent variable 模型中表现出色，包括杂合模型、variational auto-encoders 和部分可见 generalized linear models。结果表明，我们的方法在 log-likelihood 和模型参数估计方面都能够提高 state-of-the-art 基eline。Abstract
Maximizing the log-likelihood is a crucial aspect of learning latent variable models, and variational inference (VI) stands as the commonly adopted method. However, VI can encounter challenges in achieving a high log-likelihood when dealing with complicated posterior distributions. In response to this limitation, we introduce a novel variational importance sampling (VIS) approach that directly estimates and maximizes the log-likelihood. VIS leverages the optimal proposal distribution, achieved by minimizing the forward $\chi^2$ divergence, to enhance log-likelihood estimation. We apply VIS to various popular latent variable models, including mixture models, variational auto-encoders, and partially observable generalized linear models. Results demonstrate that our approach consistently outperforms state-of-the-art baselines, both in terms of log-likelihood and model parameter estimation.
摘要
maximizing the log-likelihood is a crucial aspect of learning latent variable models, and variational inference (VI) stands as the commonly adopted method. However, VI can encounter challenges in achieving a high log-likelihood when dealing with complicated posterior distributions. In response to this limitation, we introduce a novel variational importance sampling (VIS) approach that directly estimates and maximizes the log-likelihood. VIS leverages the optimal proposal distribution, achieved by minimizing the forward $\chi^2$ divergence, to enhance log-likelihood estimation. We apply VIS to various popular latent variable models, including mixture models, variational auto-encoders, and partially observable generalized linear models. Results demonstrate that our approach consistently outperforms state-of-the-art baselines, both in terms of log-likelihood and model parameter estimation.Here is the translation in Traditional Chinese:最大化对应概率是学习隐藏变量模型的重要方面，并且对于复杂的 posterior distribution 这个问题，Variational Inference (VI) 是通常的运用方法。然而，VI 可能在处理复杂的 posterior distribution 时遇到高度限制，导致 log-likelihood 的估计受到影响。为了解决这个限制，我们提出了一个新的 Variational Importance Sampling (VIS) 方法，直接估计和最大化 log-likelihood。VIS 利用了最佳的提案分布，通过最小化前方 $\chi^2$ 构成函数，从而提高 log-likelihood 的估计。我们将 VIS 应用到各种流行的隐藏变量模型，包括混合模型、variational auto-encoder 和部分可观 generalized linear models。结果显示，我们的方法在 log-likelihood 和模型参数估计方面均有所提高，并且比预设的基准方法表现更好。
LocoMuJoCo: A Comprehensive Imitation Learning Benchmark for Locomotion
paper_authors: Firas Al-Hafez, Guoping Zhao, Jan Peters, Davide Tateo
For: The paper is written for researchers and developers working on imitation learning (IL) for locomotion in embodied agents.* Methods: The paper presents a novel benchmark for evaluating and comparing IL algorithms, which includes a diverse set of environments, comprehensive datasets, and handcrafted metrics.* Results: The paper provides a robust and easy-to-use benchmark for advancing research in IL for locomotion, and includes state-of-the-art baseline algorithms for evaluation.Here’s the information in Simplified Chinese text:* For: 本文是为适用于身体机器人的启发学习（IL）步行控制研究者和开发者而写的。* Methods: 本文提出了一个新的评价和比较IL算法的benchmark，包括了多种环境，如四足、二足和人体模型，每个环境都有完整的数据集，如真实噪音捕捉数据、专家数据和优化数据，以及多种部分可见任务来训练代理。* Results: 本文提供了一个可靠且易用的benchmark，可以帮助推进IL控制领域的研究，并包含了现有的基线算法以便快速评价。Abstract
Imitation Learning (IL) holds great promise for enabling agile locomotion in embodied agents. However, many existing locomotion benchmarks primarily focus on simplified toy tasks, often failing to capture the complexity of real-world scenarios and steering research toward unrealistic domains. To advance research in IL for locomotion, we present a novel benchmark designed to facilitate rigorous evaluation and comparison of IL algorithms. This benchmark encompasses a diverse set of environments, including quadrupeds, bipeds, and musculoskeletal human models, each accompanied by comprehensive datasets, such as real noisy motion capture data, ground truth expert data, and ground truth sub-optimal data, enabling evaluation across a spectrum of difficulty levels. To increase the robustness of learned agents, we provide an easy interface for dynamics randomization and offer a wide range of partially observable tasks to train agents across different embodiments. Finally, we provide handcrafted metrics for each task and ship our benchmark with state-of-the-art baseline algorithms to ease evaluation and enable fast benchmarking.
摘要
自适应学习（IL）对具有机器人体的敏捷行走具有很大的承诺。然而，许多现有的行走标准套件主要集中在简单的玩具任务上，经常不能捕捉到实际世界情况的复杂性，导致研究向不实际的领域发展。为推动IL行走研究的进步，我们提出了一个新的标准套件，用于促进IL算法的严格评价和比较。这个标准套件包括了四足、二足和人体模型等多种环境，每个环境都有完整的数据集，如真噪动 capture数据、专家真实数据和优化数据，以及不同难度水平的评价方法。此外，我们还提供了动力随机化的易用接口，以及多种部分可见任务，用于训练不同的机器人体。最后，我们提供了专门设计的任务 metric，并将我们的标准套件与当前的状态略式基eline算法一起发布，以便评价和快速比较。
Uncertainty Quantification in Multivariable Regression for Material Property Prediction with Bayesian Neural Networks
paper_authors: Longze li, Jiang Chang, Aleksandar Vakanski, Min Xian for:This paper is written for researchers and practitioners in the field of material science and machine learning, specifically those interested in uncertainty quantification (UQ) for predicting material properties.methods:The paper proposes an approach for UQ within physics-informed Bayesian Neural Networks (BNNs), which integrates knowledge from governing laws in material modeling to guide the models toward physically consistent predictions. The approach uses Markov Chain Monte Carlo (MCMC) approximation of the posterior distribution of network parameters to produce accurate point and uncertainty estimates.results:The paper presents case studies for predicting the creep rupture life of steel alloys using the proposed approach. Experimental validation with three datasets of collected measurements from creep tests demonstrates the ability of BNNs to produce accurate point and uncertainty estimates that are competitive or exceed the performance of the conventional method of Gaussian Process Regression. Additionally, the paper evaluates the suitability of BNNs for UQ in an active learning application and reports competitive performance.Abstract
With the increased use of data-driven approaches and machine learning-based methods in material science, the importance of reliable uncertainty quantification (UQ) of the predicted variables for informed decision-making cannot be overstated. UQ in material property prediction poses unique challenges, including the multi-scale and multi-physics nature of advanced materials, intricate interactions between numerous factors, limited availability of large curated datasets for model training, etc. Recently, Bayesian Neural Networks (BNNs) have emerged as a promising approach for UQ, offering a probabilistic framework for capturing uncertainties within neural networks. In this work, we introduce an approach for UQ within physics-informed BNNs, which integrates knowledge from governing laws in material modeling to guide the models toward physically consistent predictions. To evaluate the effectiveness of this approach, we present case studies for predicting the creep rupture life of steel alloys. Experimental validation with three datasets of collected measurements from creep tests demonstrates the ability of BNNs to produce accurate point and uncertainty estimates that are competitive or exceed the performance of the conventional method of Gaussian Process Regression. Similarly, we evaluated the suitability of BNNs for UQ in an active learning application and reported competitive performance. The most promising framework for creep life prediction is BNNs based on Markov Chain Monte Carlo approximation of the posterior distribution of network parameters, as it provided more reliable results in comparison to BNNs based on variational inference approximation or related NNs with probabilistic outputs. The codes are available at: https://github.com/avakanski/Creep-uncertainty-quantification.
摘要
随着数据驱动方法和机器学习技术在材料科学中的广泛应用，对预测变量的可靠 uncertainty quantification (UQ) 的重要性不可遗憾。在材料性能预测中，UQ 带来了一系列挑战，包括高级材料的多级和多物理性质、因素之间的复杂交互和模型训练数据的有限性等。在最近几年，权值神经网络 (BNNs) 已经出现为 UQ 的一种有希望的方法，具有捕捉不确定性的 probabilistic 框架。在这项工作中，我们提出了基于物理法律的 BNNs для UQ，具有引导模型生成物理合理预测的能力。为评估这种方法的效果，我们在钢合金的塑性破坏生命中进行了实验 validate，结果显示，BNNs 可以生成高精度的点估计和不确定度估计，与传统 Gaussian Process Regression 方法相当或超过其性能。此外，我们还评估了 BNNs 在活动学习应用中的适用程度，并发现其表现竞争力强。基于 Markov Chain Monte Carlo 方法 approximation posterior distribution 的网络参数，BNNs 提供了更可靠的结果，与基于 variational inference approximation 或相关的NNs WITH probabilistic outputs 相比。代码可以在以下 GitHub 上获取：https://github.com/avakanski/Creep-uncertainty-quantification。
Individualized Policy Evaluation and Learning under Clustered Network Interference
methods: 本文提出一种可以评估ITR的估计器，该估计器可以考虑干扰效应。我们展示该估计器比标准的反杂度权重估计器更高效。我们还 derivates the finite-sample regret bound for a learned ITR，显示使用我们的有效估计器可以提高学习策略的性能。
results: 我们通过 simulations和实际研究示出了我们的方法的优势。我们的结果表明，使用我们的方法可以更好地评估和学习ITR，并且可以避免干扰的影响。Abstract
While there now exists a large literature on policy evaluation and learning, much of prior work assumes that the treatment assignment of one unit does not affect the outcome of another unit. Unfortunately, ignoring interference may lead to biased policy evaluation and yield ineffective learned policies. For example, treating influential individuals who have many friends can generate positive spillover effects, thereby improving the overall performance of an individualized treatment rule (ITR). We consider the problem of evaluating and learning an optimal ITR under clustered network (or partial) interference where clusters of units are sampled from a population and units may influence one another within each cluster. Under this model, we propose an estimator that can be used to evaluate the empirical performance of an ITR. We show that this estimator is substantially more efficient than the standard inverse probability weighting estimator, which does not impose any assumption about spillover effects. We derive the finite-sample regret bound for a learned ITR, showing that the use of our efficient evaluation estimator leads to the improved performance of learned policies. Finally, we conduct simulation and empirical studies to illustrate the advantages of the proposed methodology.
摘要
现存有大量关于政策评估和学习的文献，但大多数之前的工作假设单位减法分配不会影响另一个单位的结果。可惜忽略干扰可能导致政策评估偏向和学习的策略无效。例如，对影响多个单位的个体进行个性化治疗规则（ITR）可能产生正面副作用，从而提高整体性能。我们对集群网络（或部分）干扰下的优化ITR评估问题进行研究。在这种模型下，我们提出一种可用于评估ITR实际性的估计器。我们证明这种估计器比标准的逆权重估计器更有效，后者没有假设干扰效应。我们 derivates finite-sample regret bound for a learned ITR, showing that the use of our efficient evaluation estimator leads to improved performance of learned policies。最后，我们在模拟和实际研究中ILLUSTRATE了我们的方法的优势。
results: 实验结果表明，GMIL在单元细胞数据中比现有的MIL模型表现更好，揭示新的相关性和解释生物学机制，并且可以提高计算效率。Abstract
Predicting patient features from single-cell data can unveil cellular states implicated in health and disease. Linear models and average cell type expressions are typically favored for this task for their efficiency and robustness, but they overlook the rich cell heterogeneity inherent in single-cell data. To address this gap, we introduce GMIL, a framework integrating Generalized Linear Mixed Models (GLMM) and Multiple Instance Learning (MIL), upholding the advantages of linear models while modeling cell-state heterogeneity. By leveraging predefined cell embeddings, GMIL enhances computational efficiency and aligns with recent advancements in single-cell representation learning. Our empirical results reveal that GMIL outperforms existing MIL models in single-cell datasets, uncovering new associations and elucidating biological mechanisms across different domains.
摘要
预测病人特征从单元数据可以揭示健康和疾病的 cellular 状态。通常，线性模型和平均单元类型表达被选择为这项任务的有效性和可靠性的原因，但它们忽略了单元数据中的细胞多样性。为解决这个差距，我们介绍 GMIL，一种将 Generalized Linear Mixed Models (GLMM) 和 Multiple Instance Learning (MIL) 集成的框架，同时保留线性模型的优点，模型单元状态的多样性。通过利用预定的单元嵌入，GMIL 提高计算效率，与最新的单元表示学习技术相吻合。我们的实验结果表明，GMIL 在单元数据集上表现出色，超过现有的 MIL 模型，揭示新的相关性和描述生物学机制的多个领域。
results: 基于两种特定的预期静态总 regret定义， deriv 了 $O(T^{8/9})$ 减少复杂性，并应用于解决一个长期（风险）约束river pollutant source identification问题，证明了理论结果并与现有方法相比表现出色。Abstract
In this paper, a novel Follow-the-Perturbed-Leader type algorithm is proposed and analyzed for solving general long-term constrained optimization problems in online manner, where the objective and constraints are not necessarily convex. In each period, random linear perturbation and strongly concave perturbation are incorporated in primal and dual directions, respectively, to the offline oracle, and a global minimax point is searched as solution. Based on two particular definitions of expected static cumulative regret, we derive the first sublinear $O(T^{8/9})$ regret complexity for this class of problems. The proposed algorithm is applied to tackle a long-term (risk) constrained river pollutant source identification problem, demonstrating the validity of the theoretical results and exhibiting superior performance compared to existing method.
摘要
本文提出了一种新的追随受扰领导者类算法，用于解决总是存在约束的长期优化问题， objective 和约束不一定是凸函数。每个时期，线性受扰和强凹受扰被 incorporated 到 primal 和 dual 方向中，并在全局最小最大点上进行搜索。基于两个特定的预期 static 总 regret 定义，我们 deriv 出了第一个 $O(T^{8/9})$ regret complexity。这种算法被应用于解决一个长期（风险）约束的河流污染源标识问题，并证明了理论结果的有效性，与现有方法相比表现更好。Note: "Simplified Chinese" is a romanization of Chinese characters, which is used to represent the pronunciation of Chinese characters in the Latin alphabet. It is not a translation of the text into Chinese characters, but rather a way of representing the text in a more phonetic way.
Payoff-based learning with matrix multiplicative weights in quantum games
paper_authors: Kyriakos Lotidis, Panayotis Mertikopoulos, Nicholas Bambos, Jose Blanchet
For: The paper studies the problem of learning in quantum games with scalar, payoff-based feedback, and develops new methods that require minimal information from the players.* Methods: The paper introduces a suite of minimal-information matrix multiplicative weights (3MW) methods tailored to different information frameworks, and uses ideas from bandit convex optimization to design a zeroth-order gradient sampler adapted to the semidefinite geometry of the problem.* Results: The paper shows that the 3MW method with deterministic payoff feedback retains the $\mathcal{O}(1/\sqrt{T})$ convergence rate of the vanilla MMW algorithm, and provides a 3MW method that only requires players to observe a random realization of their payoff observable and converges to equilibrium at an $\mathcal{O}(T^{-1/4})$ rate. Additionally, the paper shows that a regularized variant of the proposed 3MW method guarantees local convergence with high probability to all equilibria that satisfy a certain first-order stability condition.Abstract
In this paper, we study the problem of learning in quantum games - and other classes of semidefinite games - with scalar, payoff-based feedback. For concreteness, we focus on the widely used matrix multiplicative weights (MMW) algorithm and, instead of requiring players to have full knowledge of the game (and/or each other's chosen states), we introduce a suite of minimal-information matrix multiplicative weights (3MW) methods tailored to different information frameworks. The main difficulty to attaining convergence in this setting is that, in contrast to classical finite games, quantum games have an infinite continuum of pure states (the quantum equivalent of pure strategies), so standard importance-weighting techniques for estimating payoff vectors cannot be employed. Instead, we borrow ideas from bandit convex optimization and we design a zeroth-order gradient sampler adapted to the semidefinite geometry of the problem at hand. As a first result, we show that the 3MW method with deterministic payoff feedback retains the $\mathcal{O}(1/\sqrt{T})$ convergence rate of the vanilla, full information MMW algorithm in quantum min-max games, even though the players only observe a single scalar. Subsequently, we relax the algorithm's information requirements even further and we provide a 3MW method that only requires players to observe a random realization of their payoff observable, and converges to equilibrium at an $\mathcal{O}(T^{-1/4})$ rate. Finally, going beyond zero-sum games, we show that a regularized variant of the proposed 3MW method guarantees local convergence with high probability to all equilibria that satisfy a certain first-order stability condition.
摘要
在这篇论文中，我们研究了量子游戏学习问题以及其他类型的半definite游戏的问题，使用托管的均值（payoff-based feedback）。为了更加准确，我们专注于广泛使用的matrix multiplicative weights（MMW）算法，而不需要玩家们具有游戏完整信息（或者对方选择的状态信息）。我们则提出了一系列基于最小信息的matrix multiplicative weights（3MW）方法，适用于不同的信息框架。在这种设定下，最大的困难在于，与 classical finite games不同，量子游戏有无穷多个纯状态（量子等价的纯策略），因此标准的重要性评价技术无法应用。相反，我们借鉴了bandit convex optimization的想法，并设计了零次规格梯度抽样器，适应semidefinite geometry问题。作为第一个结果，我们证明了3MW方法与deterministic payoff feedback可以保持$\mathcal{O}(1/\sqrt{T})$的 converges rate，即vanilla, full information MMW算法在量子最小最大游戏中的 converges rate，即使玩家只知道一个整数。然后，我们进一步降低了算法的信息需求，并提供了一种3MW方法，只需要玩家观察其支付 observable的一个随机实现，并可以在$\mathcal{O}(T^{-1/4})$的速度达到均分。最后，我们超越了零和游戏，并证明了一种Regularized variant的3MW方法，可以在所有满足一定的首次稳定条件的均分中提供本地均分的高概率 garantue。
The equivalence of dynamic and strategic stability under regularized learning in games
paper_authors: Victor Boone, Panayotis Mertikopoulos
for: 本研究探讨了归一化学习在有限游戏中的长期行为。
methods: 本研究使用了规范学习和抑制学习来研究 игроков的实际策略的演化。
results: 研究发现，在正规学习下， игроks的实际策略会逐渐接近游戏的均衡点，并且这个过程的速度可以通过不同的规范学习方法来控制。Abstract
In this paper, we examine the long-run behavior of regularized, no-regret learning in finite games. A well-known result in the field states that the empirical frequencies of no-regret play converge to the game's set of coarse correlated equilibria; however, our understanding of how the players' actual strategies evolve over time is much more limited - and, in many cases, non-existent. This issue is exacerbated further by a series of recent results showing that only strict Nash equilibria are stable and attracting under regularized learning, thus making the relation between learning and pointwise solution concepts particularly elusive. In lieu of this, we take a more general approach and instead seek to characterize the \emph{setwise} rationality properties of the players' day-to-day play. To that end, we focus on one of the most stringent criteria of setwise strategic stability, namely that any unilateral deviation from the set in question incurs a cost to the deviator - a property known as closedness under better replies (club). In so doing, we obtain a far-reaching equivalence between strategic and dynamic stability: a product of pure strategies is closed under better replies if and only if its span is stable and attracting under regularized learning. In addition, we estimate the rate of convergence to such sets, and we show that methods based on entropic regularization (like the exponential weights algorithm) converge at a geometric rate, while projection-based methods converge within a finite number of iterations, even with bandit, payoff-based feedback.
摘要
在这篇论文中，我们研究了常规化、不后悔学习在 finite games 中的长期行为。一个广泛知道的结果 states that the empirical frequencies of no-regret play converge to the game's set of coarse correlated equilibria; however, our understanding of how the players' actual strategies evolve over time is much more limited - and, in many cases, non-existent. This issue is exacerbated further by a series of recent results showing that only strict Nash equilibria are stable and attracting under regularized learning, thus making the relation between learning and pointwise solution concepts particularly elusive. In lieu of this, we take a more general approach and instead seek to characterize the \emph{setwise} rationality properties of the players' day-to-day play. To that end, we focus on one of the most stringent criteria of setwise strategic stability, namely that any unilateral deviation from the set in question incurs a cost to the deviator - a property known as closedness under better replies (club). In so doing, we obtain a far-reaching equivalence between strategic and dynamic stability: a product of pure strategies is closed under better replies if and only if its span is stable and attracting under regularized learning. In addition, we estimate the rate of convergence to such sets, and we show that methods based on entropic regularization (like the exponential weights algorithm) converge at a geometric rate, while projection-based methods converge within a finite number of iterations, even with bandit, payoff-based feedback.
BarcodeBERT: Transformers for Biodiversity Analysis
paper_authors: Pablo Millan Arias, Niousha Sadjadi, Monireh Safari, ZeMing Gong, Austin T. Wang, Scott C. Lowe, Joakim Bruslund Haurum, Iuliia Zarubiieva, Dirk Steinke, Lila Kari, Angel X. Chang, Graham W. Taylor
results: 研究发现，在较简单的数据集和任务下，支持学习的卷积神经网络或受训练的基础模型表现较佳，但是面对具有挑战性的物种水平识别任务时，需要一个新的自动化预训练方法。因此，本研究提出了BarcodeBERT，一个首创的自动化预训练方法，利用了150万个无脊椎动物DNA条码参考库。Abstract
Understanding biodiversity is a global challenge, in which DNA barcodes - short snippets of DNA that cluster by species - play a pivotal role. In particular, invertebrates, a highly diverse and under-explored group, pose unique taxonomic complexities. We explore machine learning approaches, comparing supervised CNNs, fine-tuned foundation models, and a DNA barcode-specific masking strategy across datasets of varying complexity. While simpler datasets and tasks favor supervised CNNs or fine-tuned transformers, challenging species-level identification demands a paradigm shift towards self-supervised pretraining. We propose BarcodeBERT, the first self-supervised method for general biodiversity analysis, leveraging a 1.5 M invertebrate DNA barcode reference library. This work highlights how dataset specifics and coverage impact model selection, and underscores the role of self-supervised pretraining in achieving high-accuracy DNA barcode-based identification at the species and genus level. Indeed, without the fine-tuning step, BarcodeBERT pretrained on a large DNA barcode dataset outperforms DNABERT and DNABERT-2 on multiple downstream classification tasks. The code repository is available at https://github.com/Kari-Genomics-Lab/BarcodeBERT
摘要
translate into Simplified Chinese:理解生物多样性是全球挑战，DNA编码 - 短段DNA序列归类到物种水平 - 扮演着关键角色。特别是无脊椎动物，这个非常多样化和未探索的组分，表现出独特的分类复杂性。我们研究机器学习方法，比较使用supervised CNNs、精制基模型和DNA编码特定的遮盾策略，在不同复杂度的数据集上进行比较。而 simpler数据集和任务更倾向于使用supervised CNNs或精制transformers，但是挑战性的种类水平识别需要一种思维方式的转变，强调自我超vised预训练。我们提出了BarcodeBERT，首个针对普通生物多样性分析的自我超vised方法，利用1.5万个无脊椎动物DNA编码参考库。这项工作探讨了数据集特点和覆盖率对模型选择的影响，并强调了自我超vised预训练在达到高精度DNA编码基于识别的物种和属水平的重要性。实际上，不包括精制步骤，BarcodeBERT预训练在大量DNA编码数据集上表现出优于DNABERT和DNABERT-2在多个下游分类任务上。代码仓库可以在https://github.com/Kari-Genomics-Lab/BarcodeBERT中找到。
Entropy Aware Training for Fast and Accurate Distributed GNN
for: 这 paper 的目的是提高分布式图 neural network 的表现，并且解决分布式图 partitioning 生成不均匀和类别偏好的问题。
methods: 这 paper 使用了 Edge-Weighted partitioning 技术来减少总 entropy，并在每个计算机主上进行异步个性化阶段以适应本地数据分布。它还使用了类别偏好抽样法来加速收敛。
results: 在 DistDGL 框架上实现的这些训练技术比标准基elines 2-3x 快，并在 5 个大型图 benchmark 上平均提高了 4% 的微average F1 分数。Abstract
Several distributed frameworks have been developed to scale Graph Neural Networks (GNNs) on billion-size graphs. On several benchmarks, we observe that the graph partitions generated by these frameworks have heterogeneous data distributions and class imbalance, affecting convergence, and resulting in lower performance than centralized implementations. We holistically address these challenges and develop techniques that reduce training time and improve accuracy. We develop an Edge-Weighted partitioning technique to improve the micro average F1 score (accuracy) by minimizing the total entropy. Furthermore, we add an asynchronous personalization phase that adapts each compute-host's model to its local data distribution. We design a class-balanced sampler that considerably speeds up convergence. We implemented our algorithms on the DistDGL framework and observed that our training techniques scale much better than the existing training approach. We achieved a (2-3x) speedup in training time and 4\% improvement on average in micro-F1 scores on 5 large graph benchmarks compared to the standard baselines.
摘要
To address these challenges, we develop several techniques to improve training time and accuracy. First, we propose an Edge-Weighted partitioning technique that minimizes the total entropy to improve the micro average F1 score (accuracy). Additionally, we introduce an asynchronous personalization phase that adapts each compute-host's model to its local data distribution. We also design a class-balanced sampler that significantly speeds up convergence.We implement our algorithms on the DistDGL framework and observe that our training techniques scale much better than the existing training approach. Specifically, we achieve a 2-3x speedup in training time and a 4% improvement on average in micro-F1 scores on 5 large graph benchmarks compared to the standard baselines.
NeuroEvoBench: Benchmarking Evolutionary Optimizers for Deep Learning Applications
paper_authors: Robert Tjarko Lange, Yujin Tang, Yingtao Tian
for: This paper aims to address the lack of understanding and best practices for evolutionary optimization (EO) methods in deep learning, and to provide a new benchmark for evaluating EO methods tailored towards deep learning applications.
methods: The paper uses a variety of EO methods, including traditional and meta-learned EO, and investigates their performance on a new benchmark called NeuroEvoBench. The authors also explore core scientific questions such as resource allocation, fitness shaping, normalization, regularization, and scalability of EO.
results: The paper presents the results of the authors’ experiments on NeuroEvoBench, which demonstrate the effectiveness of EO methods for solving hard optimization problems in deep learning. The authors also show that their new benchmark provides practical insights for deep learning applications and can help to accelerate the adoption of EO methods in the field.Abstract
Recently, the Deep Learning community has become interested in evolutionary optimization (EO) as a means to address hard optimization problems, e.g. meta-learning through long inner loop unrolls or optimizing non-differentiable operators. One core reason for this trend has been the recent innovation in hardware acceleration and compatible software - making distributed population evaluations much easier than before. Unlike for gradient descent-based methods though, there is a lack of hyperparameter understanding and best practices for EO - arguably due to severely less 'graduate student descent' and benchmarking being performed for EO methods. Additionally, classical benchmarks from the evolutionary community provide few practical insights for Deep Learning applications. This poses challenges for newcomers to hardware-accelerated EO and hinders significant adoption. Hence, we establish a new benchmark of EO methods (NeuroEvoBench) tailored toward Deep Learning applications and exhaustively evaluate traditional and meta-learned EO. We investigate core scientific questions including resource allocation, fitness shaping, normalization, regularization & scalability of EO. The benchmark is open-sourced at https://github.com/neuroevobench/neuroevobench under Apache-2.0 license.
摘要
results: 研究发现，在 ambient manifold 和 gradient 信息抽象函数的假设下，这些策略在任意初始状态下避免瑕疵点 / 子抽象空间的概率为 1。这个结果为使用梯度方法在抽象空间进行优化提供了重要的健康检查，因为它表明，大多数情况下，梯度方法的限制状态都是本地最小值。Abstract
Many modern machine learning applications - from online principal component analysis to covariance matrix identification and dictionary learning - can be formulated as minimization problems on Riemannian manifolds, and are typically solved with a Riemannian stochastic gradient method (or some variant thereof). However, in many cases of interest, the resulting minimization problem is not geodesically convex, so the convergence of the chosen solver to a desirable solution - i.e., a local minimizer - is by no means guaranteed. In this paper, we study precisely this question, that is, whether stochastic Riemannian optimization algorithms are guaranteed to avoid saddle points with probability 1. For generality, we study a family of retraction-based methods which, in addition to having a potentially much lower per-iteration cost relative to Riemannian gradient descent, include other widely used algorithms, such as natural policy gradient methods and mirror descent in ordinary convex spaces. In this general setting, we show that, under mild assumptions for the ambient manifold and the oracle providing gradient information, the policies under study avoid strict saddle points / submanifolds with probability 1, from any initial condition. This result provides an important sanity check for the use of gradient methods on manifolds as it shows that, almost always, the limit state of a stochastic Riemannian algorithm can only be a local minimizer.
摘要
许多现代机器学习应用 - 从在线主成分分析到covariance矩阵识别和词库学习 - 都可以表述为在里曼尼投影上的最小化问题，通常使用里曼尼泛化 gradient 方法（或其变种）解决。然而，在许多实际应用中，得到的最小化问题通常不是曲线 convex，因此选择的解决方案的 converges 是不能保证的。在这篇论文中，我们研究了这个问题，即里曼尼泛化优化算法是否能够避免陷阱点的可能性。为了保持一致性，我们研究了一家 retraction-based 方法，这种方法不仅可能比里曼尼泛化 gradient descent 更加低效，还包括了其他广泛使用的算法，如自然政策梯度方法和 mirror descent 在几何空间中。在这个总体设定下，我们证明了，对于 ambient manifold 和 gradient 信息来源的假设满足某些轻量级的条件，则 policies 在研究中避免精确的陷阱点 / 子抽象空间的可能性为 1，从任何初始状态开始。这个结果提供了对使用梯度方法在 manifold 上进行优化的重要的健康性检查，因为它表明，大多数情况下，里曼尼泛化优化算法的极限状态只能是一个本地最小值。
From Trojan Horses to Castle Walls: Unveiling Bilateral Backdoor Effects in Diffusion Models
paper_authors: Zhuoshi Pan, Yuguang Yao, Gaowen Liu, Bingquan Shen, H. Vicky Zhao, Ramana Rao Kompella, Sijia Liu
for: This paper investigates the vulnerability of state-of-the-art diffusion models (DMs) to backdoor attacks, specifically whether generating backdoor attacks can be as simple as BadNets in image classification.
methods: The paper uses a more realistic backdoor setting, where the training dataset is contaminated without tampering the original diffusion process, and uncovers bilateral backdoor effects that can be used for both adversarial and defensive purposes.
results: The paper shows that a BadNets-like backdoor attack remains effective in DMs for producing incorrect images, and that backdoored DMs exhibit an increased ratio of backdoor triggers, which can be used to enhance the detection of backdoor-poisoned training data. Additionally, the paper establishes a linkage between backdoor attacks and the phenomenon of data replications by exploring DMs’ inherent data memorization tendencies.Abstract
While state-of-the-art diffusion models (DMs) excel in image generation, concerns regarding their security persist. Earlier research highlighted DMs' vulnerability to backdoor attacks, but these studies placed stricter requirements than conventional methods like 'BadNets' in image classification. This is because the former necessitates modifications to the diffusion sampling and training procedures. Unlike the prior work, we investigate whether generating backdoor attacks in DMs can be as simple as BadNets, i.e., by only contaminating the training dataset without tampering the original diffusion process. In this more realistic backdoor setting, we uncover bilateral backdoor effects that not only serve an adversarial purpose (compromising the functionality of DMs) but also offer a defensive advantage (which can be leveraged for backdoor defense). Specifically, we find that a BadNets-like backdoor attack remains effective in DMs for producing incorrect images (misaligned with the intended text conditions), and thereby yielding incorrect predictions when DMs are used as classifiers. Meanwhile, backdoored DMs exhibit an increased ratio of backdoor triggers, a phenomenon we refer to as `trigger amplification', among the generated images. We show that this latter insight can be used to enhance the detection of backdoor-poisoned training data. Even under a low backdoor poisoning ratio, studying the backdoor effects of DMs is also valuable for designing anti-backdoor image classifiers. Last but not least, we establish a meaningful linkage between backdoor attacks and the phenomenon of data replications by exploring DMs' inherent data memorization tendencies. The codes of our work are available at https://github.com/OPTML-Group/BiBadDiff.
摘要
当前最先进的扩散模型（DM）在图像生成方面表现出色，但security问题仍然存在。 Earlier research highlighted DMs的易受到后门攻击的问题，但这些研究假设了与传统方法 like 'BadNets' in image classification不同的需求。 This is because the former requires modifications to the diffusion sampling and training procedures. Unlike prior work, we investigate whether generating backdoor attacks in DMs can be as simple as BadNets, i.e., by only contaminating the training dataset without tampering the original diffusion process. In this more realistic backdoor setting, we uncover bilateral backdoor effects that not only serve an adversarial purpose (compromising the functionality of DMs) but also offer a defensive advantage (which can be leveraged for backdoor defense). Specifically, we find that a BadNets-like backdoor attack remains effective in DMs for producing incorrect images (misaligned with the intended text conditions), and thereby yielding incorrect predictions when DMs are used as classifiers. Meanwhile, backdoored DMs exhibit an increased ratio of backdoor triggers, a phenomenon we refer to as 'trigger amplification', among the generated images. We show that this latter insight can be used to enhance the detection of backdoor-poisoned training data. Even under a low backdoor poisoning ratio, studying the backdoor effects of DMs is also valuable for designing anti-backdoor image classifiers. Last but not least, we establish a meaningful linkage between backdoor attacks and the phenomenon of data replications by exploring DMs' inherent data memorization tendencies. The codes of our work are available at .
results: 在使用 LibriCount 数据集进行广泛评估中，TaCNet 的平均准确率为 74.18%，在 11 个类别中表现出色，包括中文和波斯语应用场景。这种跨语言适应性表明其 universality 和可能的影响。Abstract
In this paper, we introduce the Temporal Audio Source Counting Network (TaCNet), an innovative architecture that addresses limitations in audio source counting tasks. TaCNet operates directly on raw audio inputs, eliminating complex preprocessing steps and simplifying the workflow. Notably, it excels in real-time speaker counting, even with truncated input windows. Our extensive evaluation, conducted using the LibriCount dataset, underscores TaCNet's exceptional performance, positioning it as a state-of-the-art solution for audio source counting tasks. With an average accuracy of 74.18 percentage over 11 classes, TaCNet demonstrates its effectiveness across diverse scenarios, including applications involving Chinese and Persian languages. This cross-lingual adaptability highlights its versatility and potential impact.
摘要
在这篇论文中，我们介绍了Temporal Audio Source Counting Network（TaCNet），一种创新的架构，用于解决音频来源计数任务中的限制。TaCNet直接操作 raw 音频输入，从而消除复杂的预处理步骤，简化工作流程。尤其是在实时speaker计数任务中，TaCNet表现出色，即使输入窗口被截断。我们对利用 LibriCount 数据集进行了广泛的评估，并证明 TaCNet 在多种场景下表现出优秀的性能，包括使用中文和波斯语。这种跨语言适应性表明 TaCNet 的多样性和影响力。
MATA*: Combining Learnable Node Matching with A* Algorithm for Approximate Graph Edit Distance Computation
results: 经验表明，MATA* 对于大型图进行 aproximate GED 计算具有显著优势，可以高效地解决现有的搜索和学习方法的缺陷。Abstract
Graph Edit Distance (GED) is a general and domain-agnostic metric to measure graph similarity, widely used in graph search or retrieving tasks. However, the exact GED computation is known to be NP-complete. For instance, the widely used A* algorithms explore the entire search space to find the optimal solution which inevitably suffers scalability issues. Learning-based methods apply graph representation techniques to learn the GED by formulating a regression task, which can not recover the edit path and lead to inaccurate GED approximation (i.e., the predicted GED is smaller than the exact). To this end, in this work, we present a data-driven hybrid approach MATA* for approximate GED computation based on Graph Neural Networks (GNNs) and A* algorithms, which models from the perspective of learning to match nodes instead of directly regressing GED. Specifically, aware of the structure-dominant operations (i.e.,node and edge insertion/deletion) property in GED computation, a structure-enhanced GNN is firstly designed to jointly learn local and high-order structural information for node embeddings for node matchings. Second, top-k candidate nodes are produced via a differentiable top-k operation to enable the training for node matchings, which is adhering to another property of GED, i.e., multiple optimal node matchings. Third, benefiting from the candidate nodes, MATA* only performs on the promising search directions, reaching the solution efficiently. Finally, extensive experiments show the superiority of MATA* as it significantly outperforms the combinatorial search-based, learning-based and hybrid methods and scales well to large-size graphs.
摘要
图文编辑距离（GED）是一个通用和领域不依赖的度量图像相似性，广泛用于图像搜索或检索任务。然而，GED的准确计算是知道NP完备的。例如，通用的A*算法探索整个搜索空间以找到优化解决方案，不可避免的Scalability问题。学习基于方法采用图像表示技术来学习GED，可以不能恢复编辑路径，导致不准确的GED估计（即预测的GED小于实际）。为此，在这项工作中，我们提出了一种数据驱动的混合方法MATA*，基于图像神经网络（GNNs）和A*算法，用于粗略GED计算。具体来说，我们注意到GED计算中结构多 Operation（即节点和边插入/删除）性质，因此我们首先设计了结构增强的GNN来同时学习本地和高阶结构信息以获得节点匹配。其次，通过可导的top-k操作生成top-k候选节点，以便在匹配过程中训练节点匹配。第三，由于候选节点，MATA*仅在有前景的搜索方向上进行搜索，以达到解决方案的目的。最后，我们进行了广泛的实验，并证明MATA*在相比 combinatorial search-based、学习基本和混合方法的情况下表现出色，并且可以在大规模图像上执行。
Sample Complexity of Opinion Formation on Networks
methods: 基于recognized opinion formation game，每个代理的意见视为数据 derive的模型参数，而不仅仅是先前研究中的实数。这种扩展可以更深入地理解意见形成，与联邦学习密切相关。通过这种形式ulation，我们确定了样本复杂性 bound for any network，并显示了特定网络结构的上下文 bound。
results: 发现优化策略通常将样本分配给代理 inverse proportion to their degree，这对政策产生了重要的含义。我们的发现被验证了在 sinthezied 和实际世界网络上。Abstract
Consider public health officials aiming to spread awareness about a new vaccine in a community interconnected by a social network. How can they distribute information with minimal resources, ensuring community-wide understanding that aligns with the actual facts? This concern mirrors numerous real-world situations. In this paper, we initialize the study of sample complexity in opinion formation to solve this problem. Our model is built on the recognized opinion formation game, where we regard each agent's opinion as a data-derived model parameter, not just a real number as in prior studies. Such an extension offers a wider understanding of opinion formation and ties closely with federated learning. Through this formulation, we characterize the sample complexity bounds for any network and also show asymptotically tight bounds for specific network structures. Intriguingly, we discover optimal strategies often allocate samples inversely to the degree, hinting at vital policy implications. Our findings are empirically validated on both synthesized and real-world networks.
摘要
公共卫生官员想推广新疫苗的知识在社交媒体上，如何尽可能地分散信息，使整个社区都能够理解，同时与实际情况保持一致？这个问题与现实生活中的许多情况有着很大的相似性。在这篇论文中，我们开始研究样本复杂性在意见形成中的问题。我们在已知的意见形成游戏中使用每个代理的意见作为数据获得的模型参数，而不仅仅是一个实数，这种扩展可以更好地理解意见形成，并与联邦学习 closely 相关。通过这种形式ulation，我们定义了任何网络的样本复杂性bound，以及特定网络结构的上下文bound。我们发现，优化策略通常会尽可能地分配样本，与代理的度量成正比，这对政策有着重要的含义。我们的发现得到了Synthesized和实际世界网络的验证。
Understanding the Natural Language of DNA using Encoder-Decoder Foundation Models with Byte-level Precision
results: 在不同的下游任务中，包括识别激活器、 promote 和 slice сайты、识别 genomic 序列的生物功能注释、识别 base call mismatches 和 insertion/deletion 错误、以及生成Influenza 病毒的变异，ENBED 模型都表现出了显著的提升，相比 existed 状态的艺术结果。Abstract
This paper presents the Ensemble Nucleotide Byte-level Encoder-Decoder (ENBED) foundation model, analyzing DNA sequences at byte-level precision with an encoder-decoder Transformer architecture. ENBED uses a sub-quadratic implementation of attention to develop an efficient model capable of sequence-to-sequence transformations, generalizing previous genomic models with encoder-only or decoder-only architectures. We use Masked Language Modeling to pre-train the foundation model using reference genome sequences and apply it in the following downstream tasks: (1) identification of enhancers, promotors and splice sites, (2) identification of biological function annotations of genomic sequences, (3) recognition of sequences containing base call mismatches and insertion/deletion errors, an advantage over tokenization schemes involving multiple base pairs, which lose the ability to analyze with byte-level precision, and (4) generating mutations of the Influenza virus using the encoder-decoder architecture and validating them against real-world observations. In each of these tasks, we demonstrate significant improvement as compared to the existing state-of-the-art results.
摘要
Identification of enhancers, promoters, and splice sites: The ENBED model outperforms existing methods in identifying these functional elements in DNA sequences.2. Identification of biological function annotations of genomic sequences: The ENBED model accurately predicts the biological functions of genomic sequences, including the presence of transcription factor binding sites and other regulatory elements.3. Recognition of sequences containing base call mismatches and insertion/deletion errors: The ENBED model is able to identify sequences with base call errors and insertions/deletions, which is an advantage over tokenization schemes that lose the ability to analyze at the byte level.4. Generating mutations of the Influenza virus using the encoder-decoder architecture and validating them against real-world observations: The ENBED model is used to generate mutations of the Influenza virus and the generated mutations are validated against real-world observations, demonstrating the potential of the model for drug resistance analysis and vaccine design.In each of these tasks, the ENBED model achieves significant improvement over existing state-of-the-art results, demonstrating its effectiveness in analyzing DNA sequences at the byte level.
An Operator Learning Framework for Spatiotemporal Super-resolution of Scientific Simulations
paper_authors: Valentin Duruisseaux, Amit Chakraborty for:* 这篇论文是为了解决高维度解析方法在数学模型中的计算限制问题，即使在小尺度下可以更好地捕捉实际动态。methods:* 这篇论文使用机器学习技术进行超Resolution，从低维度估算中重建高维度数值解。results:* 这篇论文提出了一种名为Super Resolution Operator Network（SROpNet）的新方法，可以在各种实际问题中提供更高精度的解决方案。Abstract
In numerous contexts, high-resolution solutions to partial differential equations are required to capture faithfully essential dynamics which occur at small spatiotemporal scales, but these solutions can be very difficult and slow to obtain using traditional methods due to limited computational resources. A recent direction to circumvent these computational limitations is to use machine learning techniques for super-resolution, to reconstruct high-resolution numerical solutions from low-resolution simulations which can be obtained more efficiently. The proposed approach, the Super Resolution Operator Network (SROpNet), frames super-resolution as an operator learning problem and draws inspiration from existing architectures to learn continuous representations of solutions to parametric differential equations from low-resolution approximations, which can then be evaluated at any desired location. In addition, no restrictions are imposed on the locations of (the fixed number of) spatiotemporal sensors at which the low-resolution approximations are provided, thereby enabling the consideration of a broader spectrum of problems arising in practice, for which many existing super-resolution approaches are not well-suited.
摘要
在许多Context中，高分辨率解决方案是必要的，以捕捉小时空尺度下的重要动力学行为。然而，使用传统方法可能会很慢和困难，因为计算资源有限。一种新的方向是使用机器学习技术来实现超解析，从低分辨率的 simulations 中重建高分辨率的数学解。我们的方法，称为 Super Resolution Operator Network (SROpNet)，将超解析视为一个操作学习问题， drew inspiration from existing architectures to learn continuous representations of solutions to parametric differential equations from low-resolution approximations, which can then be evaluated at any desired location.此外，我们不假设仅有一定数量的空间时间感知器的位置，因此可以考虑更多的实际问题，其中许多现有的超解析方法并不适用。
Self-Supervised Learning of Representations for Space Generates Multi-Modular Grid Cells
For: 解决空间问题的映射、定位和导航，哺乳动物的后代发展出了突出的空间表示。* Methods: 使用了四种方法：编码理论、动力系统、功能优化和监督深度学习。* Results: 提出了一种新的自监学习（SSL）框架，能够在不需要指导位信息或工程特定的读出表示的情况下，使多个网格细胞模块出现在训练后进行泛化。Abstract
To solve the spatial problems of mapping, localization and navigation, the mammalian lineage has developed striking spatial representations. One important spatial representation is the Nobel-prize winning grid cells: neurons that represent self-location, a local and aperiodic quantity, with seemingly bizarre non-local and spatially periodic activity patterns of a few discrete periods. Why has the mammalian lineage learnt this peculiar grid representation? Mathematical analysis suggests that this multi-periodic representation has excellent properties as an algebraic code with high capacity and intrinsic error-correction, but to date, there is no satisfactory synthesis of core principles that lead to multi-modular grid cells in deep recurrent neural networks. In this work, we begin by identifying key insights from four families of approaches to answering the grid cell question: coding theory, dynamical systems, function optimization and supervised deep learning. We then leverage our insights to propose a new approach that combines the strengths of all four approaches. Our approach is a self-supervised learning (SSL) framework - including data, data augmentations, loss functions and a network architecture - motivated from a normative perspective, without access to supervised position information or engineering of particular readout representations as needed in previous approaches. We show that multiple grid cell modules can emerge in networks trained on our SSL framework and that the networks and emergent representations generalize well outside their training distribution. This work contains insights for neuroscientists interested in the origins of grid cells as well as machine learning researchers interested in novel SSL frameworks.
摘要
为解决地图、位置Localization和导航问题，哺乳动物的演化历史中发展出了突出的空间表示。一种重要的空间表示是诺贝尔奖获得的格子细胞：神经元表示自己的位置，是一个本地和不规则的量，似乎具有奇怪的非本地和空间周期性的活动模式。为什么哺乳动物演化出这种怪异的格子表示？数学分析表明这些多周期的表示具有出色的算法代码性和内置的错误修复特性，但到目前为止，没有满意的核心原则的合成。在这项工作中，我们开始 by identifying key insights from four families of approaches to answering the grid cell question: coding theory, dynamical systems, function optimization and supervised deep learning. We then leverage our insights to propose a new approach that combines the strengths of all four approaches. Our approach is a self-supervised learning (SSL) framework - including data, data augmentations, loss functions and a network architecture - motivated from a normative perspective, without access to supervised position information or engineering of particular readout representations as needed in previous approaches. We show that multiple grid cell modules can emerge in networks trained on our SSL framework and that the networks and emergent representations generalize well outside their training distribution. This work contains insights for neuroscientists interested in the origins of grid cells as well as machine learning researchers interested in novel SSL frameworks.
methods: 使用一种新的特征值算法 called $\mathsf{Thresholded~Deflated\text{-}HeteroPCA}$，然后使用approx $k$-means来获取层次结构
results: 提供了一种可靠地实现tensor clustering的算法，并且在多种设置下比现有算法表现出更高的可靠性和精度。Abstract
Tensor clustering, which seeks to extract underlying cluster structures from noisy tensor observations, has gained increasing attention. One extensively studied model for tensor clustering is the tensor block model, which postulates the existence of clustering structures along each mode and has found broad applications in areas like multi-tissue gene expression analysis and multilayer network analysis. However, currently available computationally feasible methods for tensor clustering either are limited to handling i.i.d. sub-Gaussian noise or suffer from suboptimal statistical performance, which restrains their utility in applications that have to deal with heteroskedastic data and/or low signal-to-noise-ratio (SNR). To overcome these challenges, we propose a two-stage method, named $\mathsf{High\text{-}order~HeteroClustering}$ ($\mathsf{HHC}$), which starts by performing tensor subspace estimation via a novel spectral algorithm called $\mathsf{Thresholded~Deflated\text{-}HeteroPCA}$, followed by approximate $k$-means to obtain cluster nodes. Encouragingly, our algorithm provably achieves exact clustering as long as the SNR exceeds the computational limit (ignoring logarithmic factors); here, the SNR refers to the ratio of the pairwise disparity between nodes to the noise level, and the computational limit indicates the lowest SNR that enables exact clustering with polynomial runtime. Comprehensive simulation and real-data experiments suggest that our algorithm outperforms existing algorithms across various settings, delivering more reliable clustering performance.
摘要
tensor clustering，它旨在从含有噪声的张量观察中提取下面的底层结构，在过去几年内获得了越来越多的关注。一种广泛研究的张量 clustering 模型是张量块模型，它假设每个模式中存在层次结构，并在多个领域，如多组织表达分析和多层网络分析中发现了广泛的应用。然而，目前可用的计算可行的张量 clustering 方法 Either 是处理 i.i.d. 子 Gaussian 噪声的限制，或者受到不佳的统计性能的限制，这限制了它们在应用中处理不均匀数据和/或低信号响应比例 (SNR) 的能力。为了解决这些挑战，我们提出了一种两stage方法，名为 $\mathsf{High\text{-}order~HeteroClustering}$ ($\mathsf{HHC}$)，它首先通过一种新的спектраль算法 called $\mathsf{Thresholded~Deflated\text{-}HeteroPCA}$ 进行张量子空间估计，然后使用approx $k$-means 获取集群节点。鼓舞人的是，我们的算法可以在 SNR 超过计算限制 (忽略对数因素) 的情况下，提供正确的划分结果，其中 SNR 是对比两个节点之间的差异与噪声水平的比率，而计算限制则是最低的 SNR 可以使用的计算时间的下限。在广泛的 simulate 和实际数据实验中，我们的算法比现有的算法在不同的设置下表现出更高的可靠性。
Contrastive Multi-Modal Representation Learning for Spark Plug Fault Diagnosis
results: 这个研究的结果显示，使用了Denosing Multi-Modal Autoencoder的方法可以实现高效的多感知量融合，并且可以在感知量失效时继续运行，不需要更改现有的感知量组态。此外，这种方法可以实现更cost-effective的状态监控系统，不需要增加更多的感知量。Abstract
Due to the incapability of one sensory measurement to provide enough information for condition monitoring of some complex engineered industrial mechanisms and also for overcoming the misleading noise of a single sensor, multiple sensors are installed to improve the condition monitoring of some industrial equipment. Therefore, an efficient data fusion strategy is demanded. In this research, we presented a Denoising Multi-Modal Autoencoder with a unique training strategy based on contrastive learning paradigm, both being utilized for the first time in the machine health monitoring realm. The presented approach, which leverages the merits of both supervised and unsupervised learning, not only achieves excellent performance in fusing multiple modalities (or views) of data into an enriched common representation but also takes data fusion to the next level wherein one of the views can be omitted during inference time with very slight performance reduction, or even without any reduction at all. The presented methodology enables multi-modal fault diagnosis systems to perform more robustly in case of sensor failure occurrence, and one can also intentionally omit one of the sensors (the more expensive one) in order to build a more cost-effective condition monitoring system without sacrificing performance for practical purposes. The effectiveness of the presented methodology is examined on a real-world private multi-modal dataset gathered under non-laboratory conditions from a complex engineered mechanism, an inline four-stroke spark-ignition engine, aiming for spark plug fault diagnosis. This dataset, which contains the accelerometer and acoustic signals as two modalities, has a very slight amount of fault, and achieving good performance on such a dataset promises that the presented method can perform well on other equipment as well.
摘要
results: 本文预测了机器学习的未来发展趋势，并提出了新的机会和挑战，以帮助机器学习在广泛的行业中得到更广泛的应用和利用。Abstract
Machine learning is expected to enable the next Industrial Revolution. However, lacking standardized and automated assembly networks, ML faces significant challenges to meet ever-growing enterprise demands and empower broad industries. In the Perspective, we argue that ML needs to first complete its own Industrial Revolution, elaborate on how to best achieve its goals, and discuss new opportunities to enable rapid translation from ML's innovation frontier to mass production and utilization.
摘要
机器学习预计会推动下一个工业革命。然而，由于缺乏标准化和自动化的组装网络，机器学习面临着满足永不减少的企业需求和推广到多个行业的重大挑战。在我们的视角中，机器学习需要先完成自己的工业革命，详细说明如何最好实现目标，并讨论新的机会来快速将机器学习的创新前沿翻译成大规模生产和应用。Note: Simplified Chinese is used here, as it is more widely used in mainland China and is the standard language for most online content. Traditional Chinese is used in Taiwan and Hong Kong, and it has some differences in grammar and vocabulary compared to Simplified Chinese.