paper_authors: Adam R. Klivans, Konstantinos Stavropoulos, Arsen Vasilyan
for: 本研究重新强调了学习 distribuion shift 问题，即学习者只有 training distribution $D$ 上标注的样本和 test distribution $D’$ 上的无标注样本，并且需要输出一个具有低测试错误率的分类器。传统的方法是通过 bounding 损失来bounds 分类器的性能，但这些距离们显得困难计算，无法导致高效的算法。
methods: 我们采用了一种新的模型 called testable learning with distribution shift，可以获得可靠的算法来证明分类器在 test distribution $D’$ 上的性能。learner 输出一个具有低测试错误率的分类器，当 samples from $D$ 和 $D’$ 通过相关的测试时。此外，测试还需要接受，如果 marginal of $D$ 与 marginal of $D’$ 相同。我们提供了一些正面的结果，包括在半空间、半空间的交集和决策树上学习的情况下，当 marginal of $D$ 是 Gaussian 或 uniform on ${\pm 1}^d$ 时。在这些基本情况下，没有高效的算法，除非 $D’$ 具有强大的假设。
results: 我们 obtianed several positive results, including the ability to learn well-studied concept classes such as halfspaces, intersections of halfspaces, and decision trees when the marginal of $D$ is Gaussian or uniform on ${\pm 1}^d$. Additionally, we developed a moment-matching approach combined with ideas from active learning to simulate an efficient oracle for estimating disagreement regions for halfspaces in the realizable case. For the non-realizable setting, we applied recent work from testable (agnostic) learning. Furthermore, we proved that any function class with low-degree $L_2$-sandwiching polynomial approximators can be learned in our model. We used constructions from the pseudorandomness literature to obtain the required approximators.Abstract
We revisit the fundamental problem of learning with distribution shift, in which a learner is given labeled samples from training distribution $D$, unlabeled samples from test distribution $D'$ and is asked to output a classifier with low test error. The standard approach in this setting is to bound the loss of a classifier in terms of some notion of distance between $D$ and $D'$. These distances, however, seem difficult to compute and do not lead to efficient algorithms. We depart from this paradigm and define a new model called testable learning with distribution shift, where we can obtain provably efficient algorithms for certifying the performance of a classifier on a test distribution. In this model, a learner outputs a classifier with low test error whenever samples from $D$ and $D'$ pass an associated test; moreover, the test must accept if the marginal of $D$ equals the marginal of $D'$. We give several positive results for learning well-studied concept classes such as halfspaces, intersections of halfspaces, and decision trees when the marginal of $D$ is Gaussian or uniform on $\{\pm 1\}^d$. Prior to our work, no efficient algorithms for these basic cases were known without strong assumptions on $D'$. For halfspaces in the realizable case (where there exists a halfspace consistent with both $D$ and $D'$), we combine a moment-matching approach with ideas from active learning to simulate an efficient oracle for estimating disagreement regions. To extend to the non-realizable setting, we apply recent work from testable (agnostic) learning. More generally, we prove that any function class with low-degree $L_2$-sandwiching polynomial approximators can be learned in our model. We apply constructions from the pseudorandomness literature to obtain the required approximators.
摘要
我们重新探讨了学习对于分布差异的基本问题，在这个问题中，学者被提供了训练分布$D$中的标签样本，测试分布$D'$中的无标签样本，并要求他们生成一个低测试错误的分类器。标准的方法在这个设定中是通过将损失函数绑定到$D$和$D'$之间的一些距离之上，这些距离 however 似乎很难计算，并不会导致有效的算法。我们从这个概念中独立，定义了一个名为“测试学习对于分布差异”的新模型，在这个模型中，学者输出一个低测试错误的分类器，其中标签样本从$D$和$D'$中通过一个相关的测试而获得批准。此外，这个测试还需要确保$D$和$D'$的聚合相同。我们提供了多个正面的结果，包括在对半空间、半空间的交集和决策树等常见的概念类中学习时，当$D$的聚合为高斯或 uniform on $\{\pm 1\}^d$时，可以取得有效的算法。在这些基本情况下，以前没有无效的算法，不需要强制假设$D'$的假设。对半空间情况（其存在一个半空间与 both $D$ 和 $D'$ 兼容），我们结合了一个积分匹配方法与活动学习的想法，实现了一个高效的伪oracle，用于估计不同区域。为了扩展到非可行情况，我们应用了最近的测试（agnostic）学习的成果。更一般地，我们证明任何具有低度$L_2$矩阵拓扑的函数类别可以在我们的模型中学习。我们从 pseudorandomness 文献中取得了需要的拓扑器。
Multi-fidelity Constrained Optimization for Stochastic Black Box Simulators
methods: 使用 Stochastic Constrained Optimization for N dimensions（Scout-Nd）算法，能够有效地估算梯度，减少梯度估计的噪声，并应用多 fidelti 策略进一步减少计算努力。
results: 在标准准例中进行了 validate，demonstrating 能够有效地优化参数，表现比现有方法更好。Abstract
Constrained optimization of the parameters of a simulator plays a crucial role in a design process. These problems become challenging when the simulator is stochastic, computationally expensive, and the parameter space is high-dimensional. One can efficiently perform optimization only by utilizing the gradient with respect to the parameters, but these gradients are unavailable in many legacy, black-box codes. We introduce the algorithm Scout-Nd (Stochastic Constrained Optimization for N dimensions) to tackle the issues mentioned earlier by efficiently estimating the gradient, reducing the noise of the gradient estimator, and applying multi-fidelity schemes to further reduce computational effort. We validate our approach on standard benchmarks, demonstrating its effectiveness in optimizing parameters highlighting better performance compared to existing methods.
摘要
<>转换文本到简化中文。<>模拟器参数的Constrained优化在设计过程中发挥关键作用。这些问题在模拟器是随机的、计算成本高的和参数空间高维的情况下变得挑战性更大。只有通过利用参数与之关系的梯度来进行优化，但这些梯度在许多传统的黑盒代码中不可见。我们介绍了Scout-Nd（随机Constrained优化for N维）算法，以解决上述问题，效率地估计梯度，减少梯度估计的噪声，并应用多级准则来进一步减少计算努力。我们对标准准例进行验证，示出了我们的方法在优化参数时的效果，比传统方法更好。
Modelling wildland fire burn severity in California using a spatial Super Learner approach
paper_authors: Nicholas Simafranca, Bryant Willoughby, Erin O’Neil, Sophie Farr, Brian J Reich, Naomi Giertych, Margaret Johnson, Madeleine Pascolini-Campbell
for: 预测加州西部野火燃烧严重程度
methods: 使用机器学习模型，使用预 Feuer remotely sensed data 来预测火灾后燃烧严重程度
results: 比较标准线性回传方法，Super Learner 算法在所有组合中都表现出色，能够准确预测燃烧严重程度的主要驱动因素，包括绿色、高度和火灾天气变量，这些发现可以提供实际的几何资讯，帮助社区制定缓解措施，例如早期火灾探测系统、预火季节 vegetation 清理活动和紧急 Response 资源配置。Abstract
Given the increasing prevalence of wildland fires in the Western US, there is a critical need to develop tools to understand and accurately predict burn severity. We develop a machine learning model to predict post-fire burn severity using pre-fire remotely sensed data. Hydrological, ecological, and topographical variables collected from four regions of California - the sites of the Kincade fire (2019), the CZU Lightning Complex fire (2020), the Windy fire (2021), and the KNP Fire (2021) - are used as predictors of the difference normalized burn ratio. We hypothesize that a Super Learner (SL) algorithm that accounts for spatial autocorrelation using Vecchia's Gaussian approximation will accurately model burn severity. In all combinations of test and training sets explored, the results of our model showed the SL algorithm outperformed standard Linear Regression methods. After fitting and verifying the performance of the SL model, we use interpretable machine learning tools to determine the main drivers of severe burn damage, including greenness, elevation and fire weather variables. These findings provide actionable insights that enable communities to strategize interventions, such as early fire detection systems, pre-fire season vegetation clearing activities, and resource allocation during emergency responses. When implemented, this model has the potential to minimize the loss of human life, property, resources, and ecosystems in California.
摘要
due to the increasing prevalence of wildland fires in the Western US, there is a critical need to develop tools to understand and accurately predict burn severity. we develop a machine learning model to predict post-fire burn severity using pre-fire remotely sensed data. hydrological, ecological, and topographical variables collected from four regions of California - the sites of the Kincade fire (2019), the CZU Lightning Complex fire (2020), the Windy fire (2021), and the KNP Fire (2021) - are used as predictors of the difference normalized burn ratio. we hypothesize that a Super Learner (SL) algorithm that accounts for spatial autocorrelation using Vecchia's Gaussian approximation will accurately model burn severity. in all combinations of test and training sets explored, the results of our model showed the SL algorithm outperformed standard linear regression methods. after fitting and verifying the performance of the SL model, we use interpretable machine learning tools to determine the main drivers of severe burn damage, including greenness, elevation, and fire weather variables. these findings provide actionable insights that enable communities to strategize interventions, such as early fire detection systems, pre-fire season vegetation clearing activities, and resource allocation during emergency responses. when implemented, this model has the potential to minimize the loss of human life, property, resources, and ecosystems in California.Note: Please note that the translation is in Simplified Chinese, which is the standard form of Chinese used in mainland China and Singapore. If you need the translation in Traditional Chinese, please let me know.
Speech-Based Blood Pressure Estimation with Enhanced Optimization and Incremental Clustering
results: 研究结果表明，combined outcome of these clustering techniques enables robust BP estimation，并且通过 интегрирование高级血压估算技术和 YouTube 视频的情感维度，本研究拓宽了我们对现代媒体环境对健康影响的理解。Abstract
Blood Pressure (BP) estimation plays a pivotal role in diagnosing various health conditions, highlighting the need for innovative approaches to overcome conventional measurement challenges. Leveraging machine learning and speech signals, this study investigates accurate BP estimation with a focus on preprocessing, feature extraction, and real-time applications. An advanced clustering-based strategy, incorporating the k-means algorithm and the proposed Fact-Finding Instructor optimization algorithm, is introduced to enhance accuracy. The combined outcome of these clustering techniques enables robust BP estimation. Moreover, extending beyond these insights, this study delves into the dynamic realm of contemporary digital content consumption. Platforms like YouTube have emerged as influential spaces, presenting an array of videos that evoke diverse emotions. From heartwarming and amusing content to intense narratives, YouTube captures a spectrum of human experiences, influencing information access and emotional engagement. Within this context, this research investigates the interplay between YouTube videos and physiological responses, particularly Blood Pressure (BP) levels. By integrating advanced BP estimation techniques with the emotional dimensions of YouTube videos, this study enriches our understanding of how modern media environments intersect with health implications.
摘要
血压（BP）估算在诊断各种健康状况中扮演着关键角色，高亮了需要采用创新的测量方法来解决传统测量挑战。利用机器学习和语音信号，本研究探讨了准确的BP估算方法，并将重点放在预处理、特征提取和实时应用方面。本研究提出了一种基于归一化策略的高级归一化策略，结合k-means算法和提出的Fact-Finding Instructor优化算法，以提高准确性。这些归一化策略的结合效果使得BP估算更加稳定。此外，本研究还推广了这些视频的感知效果，探讨了YouTube视频平台上的视频内容对血压水平的影响。通过将高级BP估算技术与YouTube视频的情感维度结合起来，本研究扩展了我们对现代媒体环境与健康影响的理解。
AugmentTRAJ: A framework for point-based trajectory data augmentation
For: The paper is written for researchers and practitioners working with mobility data analysis, particularly those interested in leveraging data augmentation techniques to improve the performance and generalization of their models.* Methods: The paper introduces AugmenTRAJ, an open-source Python3 framework designed for trajectory data augmentation. The framework offers a variety of data augmentation techniques, including point-wise augmentation, to generate synthetic trajectories that preserve the inherent characteristics of the original data.* Results: The paper showcases the effectiveness of AugmenTRAJ in enhancing the performance and generalization of mobility data analysis models. The framework is found to be reliable and versatile, providing researchers with a practical tool for augmenting trajectory data and expanding the potential applications of data augmentation in this domain.Abstract
Data augmentation has emerged as a powerful technique in machine learning, strengthening model robustness while mitigating overfitting and under-fitting issues by generating diverse synthetic data. Nevertheless, despite its success in other domains, data augmentation's potential remains largely untapped in mobility data analysis, primarily due to the intricate nature and unique format of trajectory data. Additionally, there is a lack of frameworks capable of point-wise data augmentation, which can reliably generate synthetic trajectories while preserving the inherent characteristics of the original data. To address these challenges, this research introduces AugmenTRAJ, an open-source Python3 framework designed explicitly for trajectory data augmentation. AugmenTRAJ offers a reliable and well-controlled approach for generating synthetic trajectories, thereby enabling the harnessing of data augmentation benefits in mobility analysis. This thesis presents a comprehensive overview of the methodologies employed in developing AugmenTRAJ and showcases the various data augmentation techniques available within the framework. AugmenTRAJ opens new possibilities for enhancing mobility data analysis models' performance and generalization capabilities by providing researchers with a practical and versatile tool for augmenting trajectory data, Its user-friendly implementation in Python3 facilitates easy integration into existing workflows, offering the community an accessible resource to leverage the full potential of data augmentation in trajectory-based applications.
摘要
<>TRANSLATE_TEXT大数据增强技术在机器学习中得到广泛应用，提高模型的鲁棒性，同时避免过拟合和下适应问题，通过生成多样化的 sintetic 数据。然而，在其他领域的成功不withstanding，数据增强在行动数据分析中的潜力仍然未得到充分开发利用，主要是因为行动数据的特殊性和 Format。此外，Point-wise 数据增强的框架缺乏，这些框架可以可靠地生成 sintetic 行动轨迹，同时保持原始数据的基本特征。为解决这些挑战，这项研究提出了 AugmenTRAJ，一个开源的 Python3 框架，专门用于行动数据增强。AugmenTRAJ 提供了一种可靠和受控的 sintetic 行动轨迹生成方法，使得行动数据分析模型的性能和泛化能力得到进一步提高。这份论文提供了开发 AugmenTRAJ 的方法和技术的全面概述，并展示了框架中的多种数据增强技术。AugmenTRAJ 为研究人员提供了一个实用和灵活的工具，可以帮助他们在行动数据分析中更好地利用数据增强的优势。其Python3 实现的易用性使得它可以轻松地与现有的 workflow 集成，为社区提供了一个可访问的资源，以便在行动数据分析中充分发挥数据增强的作用。<>
Where2Start: Leveraging initial States for Robust and Sample-Efficient Reinforcement Learning
paper_authors: Pouya Parsa, Raoof Zare Moayedi, Mohammad Bornosi, Mohammad Mahdi Bejani
for: The paper aims to improve the performance of reinforcement learning agents by leveraging the knowledge captured by trajectories.
methods: The proposed Where2Start algorithm selects the initial state in a way that maximizes the instability of the agent in the vicinity of that state, leading to a decrease in the number of trajectories needed for the agent to reach an acceptable reward.
results: The experiments show that Where2Start can improve sample efficiency up to 8 times and can be combined with most state-of-the-art algorithms to improve robustness and sample efficiency significantly.Abstract
The reinforcement learning algorithms that focus on how to compute the gradient and choose next actions, are effectively improved the performance of the agents. However, these algorithms are environment-agnostic. This means that the algorithms did not use the knowledge that has been captured by trajectory. This poses that the algorithms should sample many trajectories to train the model. By considering the essence of environment and how much the agent learn from each scenario in that environment, the strategy of the learning procedure can be changed. The strategy retrieves more informative trajectories, so the agent can learn with fewer trajectory sample. We propose Where2Start algorithm that selects the initial state so that the agent has more instability in vicinity of that state. We show that this kind of selection decreases number of trajectories that should be sampled that the agent reach to acceptable reward. Our experiments shows that Where2Start can improve sample efficiency up to 8 times. Also Where2Start can combined with most of state-of-the-art algorithms and improve that robustness and sample efficiency significantly.
摘要
эти reinforcement learning 算法，关注计算梯度和选择下一步行为，有效地提高了代理人的性能。然而，这些算法是环境无关的，这意味着这些算法不使用路径中所捕捉的知识。这会导致算法需要采样多个路径来训练模型。通过考虑环境的本质和代理人在那里学习的程度，我们可以更改学习策略。我们提出了 Where2Start 算法，该算法选择初始状态，使得代理人在该状态附近有更大的不稳定性。我们展示了这种选择可以降低代理人需要采样的轨迹数量，以达到acceptable reward。我们的实验表明，Where2Start 可以提高样本效率，达到8倍以上。此外，Where2Start 可以与大多数当前的state-of-the-art算法结合使用，并在robustness和样本效率方面提供显著改善。
A GPU-based Hydrodynamic Simulator with Boid Interactions
for: simulations of virtual agent behaviors and navigation inside a smoothed particle hydrodynamical (SPH) fluid environment
methods: GPU compute shaders of DirectX, parallel smoothed particle hydrodynamics model, distributed boid model, surface reconstruction using marching cubes algorithm
results: real-time water mesh surface reconstruction, interaction between SPH and virtual boid agents, versatility for underwater navigation and remote control engineering purposesAbstract
We present a hydrodynamic simulation system using the GPU compute shaders of DirectX for simulating virtual agent behaviors and navigation inside a smoothed particle hydrodynamical (SPH) fluid environment with real-time water mesh surface reconstruction. The current SPH literature includes interactions between SPH and heterogeneous meshes but seldom involves interactions between SPH and virtual boid agents. The contribution of the system lies in the combination of the parallel smoothed particle hydrodynamics model with the distributed boid model of virtual agents to enable agents to interact with fluids. The agents based on the boid algorithm influence the motion of SPH fluid particles, and the forces from the SPH algorithm affect the movement of the boids. To enable realistic fluid rendering and simulation in a particle-based system, it is essential to construct a mesh from the particle attributes. Our system also contributes to the surface reconstruction aspect of the pipeline, in which we performed a set of experiments with the parallel marching cubes algorithm per frame for constructing the mesh from the fluid particles in a real-time compute and memory-intensive application, producing a wide range of triangle configurations. We also demonstrate that our system is versatile enough for reinforced robotic agents instead of boid agents to interact with the fluid environment for underwater navigation and remote control engineering purposes.
摘要
我们提出了基于 DirectX GPU compute shaders 的 hydrodynamic simulation 系统，用于虚拟代理行为和在流体环境中的导航。现有的 SPH 文献中有些包括 SPH 和不同的网格之间的交互，但几乎不包括 SPH 和虚拟代理之间的交互。我们的系统的贡献在于将平行简化particle hydrodynamics模型与分布式 boid 模型结合起来，使代理能够与流体交互。基于 boid 算法的代理影响流体粒子的运动，而流体粒子的力也影响代理的移动。为实现真实的流体渲染和模拟，在 particle-based 系统中构建一个网格是非常重要的。我们的系统还贡献了 surface reconstruction 方面的研究，在每帧使用并行的 marching cubes 算法来从流体粒子中构建网格，生成了一系列 triangle 配置。我们还示出了我们的系统可以用于水下导航和远程控制工程应用，并且可以使用 reinforced robotic agents 代替 boid agents 与流体环境交互。
Large Catapults in Momentum Gradient Descent with Warmup: An Empirical Study
results: 研究发现，使用大量的学习率和学习率温存的梯度下降with momentum可以使模型迅速 converges to flatter minima，比传统的梯度下降更好。此外，研究还提供了理论 intuition，表明梯度下降with momentum可以通过“增强”自身稳定效果来解释这种现象。Abstract
Although gradient descent with momentum is widely used in modern deep learning, a concrete understanding of its effects on the training trajectory still remains elusive. In this work, we empirically show that momentum gradient descent with a large learning rate and learning rate warmup displays large catapults, driving the iterates towards flatter minima than those found by gradient descent. We then provide empirical evidence and theoretical intuition that the large catapult is caused by momentum "amplifying" the self-stabilization effect (Damian et al., 2023).
摘要
Training a Hopfield Variational Autoencoder with Equilibrium Propagation
results: 该研究表明，通过使用Equilibrium Propagation法则和套管网络的合并模型，可以有效地降低VAE的计算成本，并且保持模型的性能。Abstract
On dedicated analog hardware, equilibrium propagation is an energy-efficient alternative to backpropagation. In spite of its theoretical guarantees, its application in the AI domain remains limited to the discriminative setting. Meanwhile, despite its high computational demands, generative AI is on the rise. In this paper, we demonstrate the application of Equilibrium Propagation in training a variational autoencoder (VAE) for generative modeling. Leveraging the symmetric nature of Hopfield networks, we propose using a single model to serve as both the encoder and decoder which could effectively halve the required chip size for VAE implementations, paving the way for more efficient analog hardware configurations.
摘要
在专门的分析逻辑硬件上，均衡传播是一种能效的替代方案，它可以在AI领域中提高能效性。尽管它具有理论保证，但在推理设置中的应用仍然受限。而在这篇论文中，我们展示了使用均衡传播来训练一个变量自动编码器（VAE） для生成模型。利用惯性网络的对称性，我们提议使用单个模型作为编码器和解码器，这可以减少VAE实现所需的芯片大小，并且开创了更有效的分析硬件配置的道路。
Satellite-based feature extraction and multivariate time-series prediction of biotoxin contamination in shellfish
paper_authors: Sergio Tavares, Pedro R. Costa, Ludwig Krippahl, Marta B. Lopes for: 这个研究的目的是evaluate the integration of satellite data in forecasting models for predicting toxin concentrations in shellfish given forecasting horizons up to four weeks.methods: 这个研究使用了Sentinel-3 satellite imagery for marine surveillance， along with shellfish biotoxin contamination data from various production areas along Portugal’s western coastline， collected by Portuguese official control. Unsupervised feature extraction was performed using autoencoders able to handle non-valid pixels caused by factors like cloud cover, land, or anomalies. Several Artificial Neural Networks models were applied to compare univariate (contamination only) and multivariate (contamination and satellite data) time-series forecasting.results: 研究发现，包含这些特征可以提高预测，特别是在lagoon production areas (RIAV) 和L5B area (oceanic) 的1-week和2-week前 horizon。这种方法可以充分利用高维数据源如Remote Sensing的信息，而不会影响预测模型的准确性。Abstract
Shellfish production constitutes an important sector for the economy of many Portuguese coastal regions, yet the challenge of shellfish biotoxin contamination poses both public health concerns and significant economic risks. Thus, predicting shellfish contamination levels holds great potential for enhancing production management and safeguarding public health. In our study, we utilize a dataset with years of Sentinel-3 satellite imagery for marine surveillance, along with shellfish biotoxin contamination data from various production areas along Portugal's western coastline, collected by Portuguese official control. Our goal is to evaluate the integration of satellite data in forecasting models for predicting toxin concentrations in shellfish given forecasting horizons up to four weeks, which implies extracting a small set of useful features and assessing their impact on the predictive models. We framed this challenge as a time-series forecasting problem, leveraging historical contamination levels and satellite images for designated areas. While contamination measurements occurred weekly, satellite images were accessible multiple times per week. Unsupervised feature extraction was performed using autoencoders able to handle non-valid pixels caused by factors like cloud cover, land, or anomalies. Finally, several Artificial Neural Networks models were applied to compare univariate (contamination only) and multivariate (contamination and satellite data) time-series forecasting. Our findings show that incorporating these features enhances predictions, especially beyond one week in lagoon production areas (RIAV) and for the 1-week and 2-week horizons in the L5B area (oceanic). The methodology shows the feasibility of integrating information from a high-dimensional data source like remote sensing without compromising the model's predictive ability.
摘要
欧洲南部地区著名的贝壳生产业对当地经济起着重要作用，但贝壳生物毒素污染会对公众健康和经济造成重大风险。因此，预测贝壳毒素污染水平具有很大的潜在价值，以提高生产管理和保护公众健康。在我们的研究中，我们使用了多年的Sentinel-3卫星图像数据，以及从葡萄牙西海岸不同生产区收集的贝壳毒素污染数据，并通过葡萄牙官方监测数据来验证我们的模型。我们的目标是评估在四周前的预测模型，以预测贝壳毒素污染水平，并从事件序列预测问题出发，利用历史污染水平和卫星图像数据来预测贝壳毒素污染水平。尽管污染测量每周进行一次，但卫星图像可以在每周多次获取。我们使用自动Encoder来处理非有效像素，以避免因云层、陆地或异常而导致的问题。最后，我们应用了多个人工神经网络模型，以比较单variate（污染水平）和多variate（污染水平和卫星数据）时间序列预测。我们的结果显示，将这些特征集成到预测模型中可以提高预测精度，特别是在里亚瓦（RIAV）和L5B区域（海洋）的1周和2周预测水平。我们的方法表明，可以不COMPROMISE模型预测能力来集成高维数据源如远程感知数据。
Eliminating Domain Bias for Federated Learning in Representation Space
results: 实验结果显示，使用DBE的联合学习方法可以大幅提高已有的个性化联合学习方法的一致性和个性化能力，并且可以大幅超越了现有的个性化联合学习方法。Abstract
Recently, federated learning (FL) is popular for its privacy-preserving and collaborative learning abilities. However, under statistically heterogeneous scenarios, we observe that biased data domains on clients cause a representation bias phenomenon and further degenerate generic representations during local training, i.e., the representation degeneration phenomenon. To address these issues, we propose a general framework Domain Bias Eliminator (DBE) for FL. Our theoretical analysis reveals that DBE can promote bi-directional knowledge transfer between server and client, as it reduces the domain discrepancy between server and client in representation space. Besides, extensive experiments on four datasets show that DBE can greatly improve existing FL methods in both generalization and personalization abilities. The DBE-equipped FL method can outperform ten state-of-the-art personalized FL methods by a large margin. Our code is public at https://github.com/TsingZ0/DBE.
摘要
最近，联合学习（FL）在隐私保护和合作学习能力方面受到欢迎。然而，在统计上不均衡的场景下，我们发现客户端上的数据领域偏好导致代表性偏衡现象，从而降低了服务器和客户端之间的普适表示能力。为解决这些问题，我们提出了一个通用框架域偏衡纠正器（DBE） для FL。我们的理论分析表明，DBE可以推动服务器和客户端之间的双向知识传递，因为它减少了服务器和客户端之间的领域差异在表示空间中。此外，我们在四个数据集上进行了广泛的实验，发现 DBE 可以大幅提高现有 FL 方法的普适和个性化能力。DBE 配置的 FL 方法可以在普适和个性化能力方面超越了十个当前顶尖个性化 FL 方法。我们的代码可以在 上找到。
Selective Inference for Changepoint detection by Recurrent Neural Network
methods: 这个研究使用了Selective Inference（SI）框架，通过conditioning on the event of hypothesis selection来避免选择偏见。
results: 研究通过人工和实际数据实验表明，提出的方法可以有效地控制 false detection 的风险，并且可以在时间序列中检测到复杂动态的变化点。Abstract
In this study, we investigate the quantification of the statistical reliability of detected change points (CPs) in time series using a Recurrent Neural Network (RNN). Thanks to its flexibility, RNN holds the potential to effectively identify CPs in time series characterized by complex dynamics. However, there is an increased risk of erroneously detecting random noise fluctuations as CPs. The primary goal of this study is to rigorously control the risk of false detections by providing theoretically valid p-values to the CPs detected by RNN. To achieve this, we introduce a novel method based on the framework of Selective Inference (SI). SI enables valid inferences by conditioning on the event of hypothesis selection, thus mitigating selection bias. In this study, we apply SI framework to RNN-based CP detection, where characterizing the complex process of RNN selecting CPs is our main technical challenge. We demonstrate the validity and effectiveness of the proposed method through artificial and real data experiments.
摘要
在这种研究中，我们研究了使用循环神经网络（RNN）来评估时间序列中检测到的变化点（CP）的统计可靠性。由于RNN的灵活性，它有可能有效地识别时间序列中的复杂动态中的CP。然而，随机噪声波动的风险增加了false检测的风险。我们的主要目标是通过提供有效的p值来控制false检测的风险。为此，我们提出了基于选择性推理（SI）框架的一种新方法。SI允许有效的推理，通过对假设选择事件进行条件，从而减少选择偏见。在这种研究中，我们将SI框架应用于RNN基于CP检测，并且我们的主要技术挑战是characterizing RNN在选择CP时的复杂过程。我们通过人工和实际数据实验 validate the effectiveness and validity of our proposed method。
Identification of morphological fingerprint in perinatal brains using quasi-conformal mapping and contrastive learning
results: 研究使用了 682 名新生儿的横截面 estructural MRI 数据，并通过数据扩展和参数调整来训练模型。模型在 30 名 longitudinal scanned infant 数据上进行验证，实现了remarkable Top1 和 Top5 准确率为 71.37% 和 84.10%， respectively。感觉和视觉 cortices 被识别为个体识别中最重要的区域。此外，折叠 morphology 显示出更高的分类能力，可能作为新生儿脑中的 morphological fingerprint。Abstract
The morphological fingerprint in the brain is capable of identifying the uniqueness of an individual. However, whether such individual patterns are present in perinatal brains, and which morphological attributes or cortical regions better characterize the individual differences of ne-onates remain unclear. In this study, we proposed a deep learning framework that projected three-dimensional spherical meshes of three morphological features (i.e., cortical thickness, mean curvature, and sulcal depth) onto two-dimensional planes through quasi-conformal mapping, and employed the ResNet18 and contrastive learning for individual identification. We used the cross-sectional structural MRI data of 682 infants, incorporating with data augmentation, to train the model and fine-tuned the parameters based on 60 infants who had longitudinal scans. The model was validated on 30 longitudinal scanned infant data, and remarkable Top1 and Top5 accuracies of 71.37% and 84.10% were achieved, respectively. The sensorimotor and visual cortices were recognized as the most contributive regions in individual identification. Moreover, the folding morphology demonstrated greater discriminative capability than the cortical thickness, which could serve as the morphological fingerprint in perinatal brains. These findings provided evidence for the emergence of morphological fingerprints in the brain at the beginning of the third trimester, which may hold promising implications for understanding the formation of in-dividual uniqueness in the brain during early development.
摘要
Brain中的形态指纹可以识别个体的唯一性。然而，在生前脑中是否存在这样的个体特征，以及哪些形态特征或 cortical 区域更好地描述新生儿的唯一性，还未得到清楚的答案。在这项研究中，我们提出了一种深度学习框架，将三维球面的三个形态特征（即 cortical thickness、mean curvature 和 sulcal depth）映射到二维平面上，并使用 ResNet18 和对比学习来实现个体识别。我们使用了682名新生儿的cross-sectional structural MRI数据，并在数据增强后，根据60名新生儿的长期扫描数据进行参数调整。模型被证明在30名长期扫描新生儿数据上的 Validation 中具有remarkable Top1 和 Top5 准确率为71.37% 和 84.10%，分别。感知动作和视觉 cortices 被识别为个体识别中最重要的区域。此外，折叠 morphology 表现出更高的分类能力，可能作为新生儿脑中的形态指纹。这些发现提供了脑开始第三 trimester 时的形态指纹的诞生证据，可能对早期脑发展中个体独特性的形成产生深远的影响。
Robust Graph Neural Networks via Unbiased Aggregation
methods: 本研究使用了代表性的Robust GNNs，并提供了一种约新颖iterative reweighted least squares算法来解决估计问题，这种算法具有理论上的收敛保证。
results: 实验表明，提议的模型具有强大的Robustness，而ablation study还提供了深入的理解其优势。Abstract
The adversarial robustness of Graph Neural Networks (GNNs) has been questioned due to the false sense of security uncovered by strong adaptive attacks despite the existence of numerous defenses. In this work, we delve into the robustness analysis of representative robust GNNs and provide a unified robust estimation point of view to understand their robustness and limitations. Our novel analysis of estimation bias motivates the design of a robust and unbiased graph signal estimator. We then develop an efficient Quasi-Newton iterative reweighted least squares algorithm to solve the estimation problem, which unfolds as robust unbiased aggregation layers in GNNs with a theoretical convergence guarantee. Our comprehensive experiments confirm the strong robustness of our proposed model, and the ablation study provides a deep understanding of its advantages.
摘要
Graph Neural Networks (GNNs) 的对抗Robustness 被质疑，因为强大的 adaptive 攻击破坏了许多防御措施的假保障。在这项工作中，我们进行了代表性的 GNNs robustness 分析，并提供了一种统一的 robust 估计视角来理解它们的强度和局限性。我们的新的估计偏见分析motivates 了设计一种强健和无偏的图信号估计器。然后，我们开发了一种高效的 quasi-Newton 迭代最小二乘算法来解决估计问题，它在 GNNs 中展开为一种robust 不偏的聚合层，并有理论上的收敛保证。我们的广泛的实验证明了我们提出的模型的强大对抗能力，并且ablation 研究提供了深入的理解。
results: 在杜卷方程中展示了方法的有效性，并建议其适用于类似结构的PDE和ODE系统。Abstract
We introduce a generalizable approach that combines perturbation method and one-shot transfer learning to solve nonlinear ODEs with a single polynomial term, using Physics-Informed Neural Networks (PINNs). Our method transforms non-linear ODEs into linear ODE systems, trains a PINN across varied conditions, and offers a closed-form solution for new instances within the same non-linear ODE class. We demonstrate the effectiveness of this approach on the Duffing equation and suggest its applicability to similarly structured PDEs and ODE systems.
摘要
我们提出一种通用的方法，将扰动方法和一次转移学习结合用于解决非线性偏微分方程（ODE），使用物理学 Informed Neural Networks（PINN）。我们的方法将非线性ODE转化为线性ODE系统，训练PINN在不同条件下，并提供新的实例的关闭形解。我们在杜勃方程中进行了示范，并建议其适用于类似结构的偏微分方程和ODE系统。
A latent linear model for nonlinear coupled oscillators on graphs
results: 作者发现，可以通过将subgraph-level的动态分解成一些基本的元素动态模式，来预测整个图上的同步状态。这种方法可以与基准和黑盒分类算法竞争，即使其架构简单明了。Abstract
A system of coupled oscillators on an arbitrary graph is locally driven by the tendency to mutual synchronization between nearby oscillators, but can and often exhibit nonlinear behavior on the whole graph. Understanding such nonlinear behavior has been a key challenge in predicting whether all oscillators in such a system will eventually synchronize. In this paper, we demonstrate that, surprisingly, such nonlinear behavior of coupled oscillators can be effectively linearized in certain latent dynamic spaces. The key insight is that there is a small number of `latent dynamics filters', each with a specific association with synchronizing and non-synchronizing dynamics on subgraphs so that any observed dynamics on subgraphs can be approximated by a suitable linear combination of such elementary dynamic patterns. Taking an ensemble of subgraph-level predictions provides an interpretable predictor for whether the system on the whole graph reaches global synchronization. We propose algorithms based on supervised matrix factorization to learn such latent dynamics filters. We demonstrate that our method performs competitively in synchronization prediction tasks against baselines and black-box classification algorithms, despite its simple and interpretable architecture.
摘要
翻译结果：系统中的各自振荡器受到邻近振荡器的同步倾向的本地驱动，但可能在整个图上表现出非线性行为。了解这种非线性行为是预测系统是否会全局同步的关键挑战。在这篇论文中，我们发现有一些“隐藏动态空间”，其中每个空间具有特定的同步和不同步动态特征，并且可以将图上的任何动态 aproximated为这些基本动态模式的线性组合。通过 ensemble 的 subgraph 级别预测，可以获得可理解的预测器，以确定整个图上是否会达到全局同步。我们基于supervised matrix factorization提出了一种学习这些隐藏动态空间的算法。我们示出了我们的方法在同步预测任务中与基准和黑obox分类算法相比，具有竞争力，即使其架构简单明了。
results: 这个方法有potential用于提供复杂疾病机制的更深入的理解。Abstract
The package mvlearnR and accompanying Shiny App is intended for integrating data from multiple sources or views or modalities (e.g. genomics, proteomics, clinical and demographic data). Most existing software packages for multiview learning are decentralized and offer limited capabilities, making it difficult for users to perform comprehensive integrative analysis. The new package wraps statistical and machine learning methods and graphical tools, providing a convenient and easy data integration workflow. For users with limited programming language, we provide a Shiny Application to facilitate data integration anywhere and on any device. The methods have potential to offer deeper insights into complex disease mechanisms. Availability and Implementation: mvlearnR is available from the following GitHub repository: https://github.com/lasandrall/mvlearnR. The web application is hosted on shinyapps.io and available at: https://multi-viewlearn.shinyapps.io/MultiView_Modeling/
摘要
package mvlearnR 和附加的 Shiny App 用于 integrating 数据从多个源或视图或模式（例如 genomics, proteomics, clinical 和 demographic 数据）。大多数现有的软件包 для多视图学习是分散的，提供有限的能力，使用者难以进行全面的 integrative 分析。新包 wrapping 统计和机器学习方法和图形工具，提供一个便捷的数据集成 workflow。对不具备编程语言技能的用户，我们提供了一个 Shiny 应用程序，以便在任何设备上进行数据集成。这些方法具有可能为复杂疾病机理提供更深入的理解。可用性和实施：mvlearnR 可以从以下 GitHub 仓库获取：https://github.com/lasandrall/mvlearnR。 web 应用程序被HOST在 shinyapps.io 上，可以通过以下链接访问：https://multi-viewlearn.shinyapps.io/MultiView_Modeling/
Support Vector Machine Implementation on MPI-CUDA and Tensorflow Framework
results: 实验结果表明，使用MPI-CUDA实现的SVM算法在不同数据集上实现了速度提高，而TensorFlow实现提供了跨平台解决方案，可以快速移植到其他硬件组件。Abstract
Support Vector Machine (SVM) algorithm requires a high computational cost (both in memory and time) to solve a complex quadratic programming (QP) optimization problem during the training process. Consequently, SVM necessitates high computing hardware capabilities. The central processing unit (CPU) clock frequency cannot be increased due to physical limitations in the miniaturization process. However, the potential of parallel multi-architecture, available in both multi-core CPUs and highly scalable GPUs, emerges as a promising solution to enhance algorithm performance. Therefore, there is an opportunity to reduce the high computational time required by SVM for solving the QP optimization problem. This paper presents a comparative study that implements the SVM algorithm on different parallel architecture frameworks. The experimental results show that SVM MPI-CUDA implementation achieves a speedup over SVM TensorFlow implementation on different datasets. Moreover, SVM TensorFlow implementation provides a cross-platform solution that can be migrated to alternative hardware components, which will reduces the development time.
摘要
支持向量机 (SVM) 算法需要高度的计算成本（ Both 内存和时间）来解决复杂的quadratic programming (QP) 优化问题 durante el entrenamiento proces. 因此，SVM 需要高度的计算硬件能力。 CPU 频率不能增加 due to physical limitations in the miniaturization process. 然而，可用的并行多架构，包括多核 CPU 和高可扩展 GPU， emerges as a promising solution to enhance algorithm performance. 因此，there is an opportunity to reduce the high computational time required by SVM for solving the QP optimization problem. This paper presents a comparative study that implements the SVM algorithm on different parallel architecture frameworks. The experimental results show that SVM MPI-CUDA implementation achieves a speedup over SVM TensorFlow implementation on different datasets. Moreover, SVM TensorFlow implementation provides a cross-platform solution that can be migrated to alternative hardware components, which will reduce the development time.Note: I've kept the original sentence structure and vocabulary as much as possible, but I've made some adjustments to make it more idiomatic and natural in Simplified Chinese.
LLM-Assisted Code Cleaning For Training Accurate Code Generators
results: 我们在两个复杂的算法代码生成 benchmark 上评估了我们的方法，发现 Fine-tuning CodeLLaMa-7B 在我们修改后的模块化程序上进行了最多30%的提高，并且发现使用较小的高质量数据可以达到更好的性能，甚至超过了关闭源模型。Abstract
Natural language to code generation is an important application area of LLMs and has received wide attention from the community. The majority of relevant studies have exclusively concentrated on increasing the quantity and functional correctness of training sets while disregarding other stylistic elements of programs. More recently, data quality has garnered a lot of interest and multiple works have showcased its importance for improving performance. In this work, we investigate data quality for code and find that making the code more structured and readable leads to improved code generation performance of the system. We build a novel data-cleaning pipeline that uses these principles to transform existing programs by 1.) renaming variables, 2.) modularizing and decomposing complex code into smaller helper sub-functions, and 3.) inserting natural-language based plans via LLM based transformations. We evaluate our approach on two challenging algorithmic code generation benchmarks and find that fine-tuning CodeLLaMa-7B on our transformed modularized programs improves the performance by up to 30% compared to fine-tuning on the original dataset. Additionally, we demonstrate improved performance from using a smaller amount of higher-quality data, finding that a model fine-tuned on the entire original dataset is outperformed by a model trained on 15% of our cleaned dataset. Even in comparison to closed-source models, our models outperform the much larger AlphaCoder models.
摘要
自然语言到代码生成是LLM的重要应用领域，社区内部广泛关注。大多数相关研究均围绕增加训练集的量和功能正确性而围绕，而忽视其他程序的风格元素。在这种情况下，我们发现了代码质量的重要性，并展示了使代码更加结构化和可读性的改进代码生成系统的效果。我们构建了一个新的数据清洁管道，使用这些原则来转换现有的程序，包括：1）重命名变量，2）归并和分解复杂代码为小型帮助子函数，3）通过LLM基于语言模型的转换插入自然语言计划。我们在两个挑战性代码生成标准 benchmark 上评估了我们的方法，发现在我们对模块化程序进行微调CodeLLaMa-7B时，性能提高了30%。此外，我们还发现使用较少量但高质量数据进行微调，模型在15%的清洁数据集上进行微调比模型在原始数据集上进行微调更高性能。而与关闭源代码模型相比，我们的模型仍然表现出优异。
A unified framework for learning with nonlinear model classes from arbitrary linear samples
results: 提供了一系列学习保证，其中包括对模型类的变化的研究，以及对各种已知问题的推广和改进In more detail, the paper is focused on the problem of learning an unknown object from training data using a given model class. The authors introduce a unified framework that allows for objects in arbitrary Hilbert spaces, general types of (random) linear measurements as training data, and general types of nonlinear model classes. They establish a series of learning guarantees for this framework, including explicit relations between the amount of training data and properties of the model class to ensure near-best generalization bounds. The paper also introduces the key notion of the variation of a model class with respect to a distribution of sampling operators, and demonstrates the versatility of the framework by showing that it can accommodate many different types of well-known problems of interest, including matrix sketching by random sampling, compressed sensing with isotropic vectors, active learning in regression, and compressed sensing with generative models.Abstract
This work considers the fundamental problem of learning an unknown object from training data using a given model class. We introduce a unified framework that allows for objects in arbitrary Hilbert spaces, general types of (random) linear measurements as training data and general types of nonlinear model classes. We establish a series of learning guarantees for this framework. These guarantees provide explicit relations between the amount of training data and properties of the model class to ensure near-best generalization bounds. In doing so, we also introduce and develop the key notion of the variation of a model class with respect to a distribution of sampling operators. To exhibit the versatility of this framework, we show that it can accommodate many different types of well-known problems of interest. We present examples such as matrix sketching by random sampling, compressed sensing with isotropic vectors, active learning in regression and compressed sensing with generative models. In all cases, we show how known results become straightforward corollaries of our general learning guarantees. For compressed sensing with generative models, we also present a number of generalizations and improvements of recent results. In summary, our work not only introduces a unified way to study learning unknown objects from general types of data, but also establishes a series of general theoretical guarantees which consolidate and improve various known results.
摘要
这个工作考虑了学习未知对象从培训数据中学习的基本问题，使用给定的模型类。我们提出了一个统一框架，允许对任意希尔伯特空间中的对象进行学习，并且支持random linear measurements作为培训数据，以及一般类型的非线性模型类。我们建立了一系列学习保证，这些保证提供了培训数据的量和模型类型之间的直接关系，以确保近似最佳泛化评估。在这过程中，我们还引入了关键的模型类变化概念，即对于分布 sampling 算子的变化。我们通过展示这个框架可以涵盖许多已知的问题，如矩阵笔记、压缩感知、活动学习、压缩感知和生成模型。在这些例子中，我们显示了如何使用我们的一般学习保证，从而得到已知结果的直接推论。此外，我们还对压缩感知与生成模型进行了一些扩展和改进。总之，我们的工作不仅提出了一个统一的方法来学习未知对象，还建立了一系列一般的理论保证，这些保证总结了和改进了许多已知的结果。
Projected Off-Policy Q-Learning (POP-QL) for Stabilizing Offline Reinforcement Learning
results: 在标准参考 зада例中竞争性表现，并在资料收集策略是明显不理想的情况下表现出色。Abstract
A key problem in off-policy Reinforcement Learning (RL) is the mismatch, or distribution shift, between the dataset and the distribution over states and actions visited by the learned policy. This problem is exacerbated in the fully offline setting. The main approach to correct this shift has been through importance sampling, which leads to high-variance gradients. Other approaches, such as conservatism or behavior-regularization, regularize the policy at the cost of performance. In this paper, we propose a new approach for stable off-policy Q-Learning. Our method, Projected Off-Policy Q-Learning (POP-QL), is a novel actor-critic algorithm that simultaneously reweights off-policy samples and constrains the policy to prevent divergence and reduce value-approximation error. In our experiments, POP-QL not only shows competitive performance on standard benchmarks, but also out-performs competing methods in tasks where the data-collection policy is significantly sub-optimal.
摘要
主要问题在完全离线RL中是状态和动作访问的分布差异，这个问题在完全离线设置中更加突出。主要方法是通过重要性抽样，导致高差值的梯度。其他方法，如保守性或行为规范，将策略正则化，但会导致性能下降。在这篇论文中，我们提出了一种稳定的离线Q学习方法。我们的方法，即投影式离线Q学习（POP-QL），是一种新的actor-critic算法，同时对离线样本进行重新权重并将策略限制，以防止偏离和降低价值近似错误。在我们的实验中，POP-QL不仅在标准准 benchmark上达到竞争性性能，而且在数据采集策略是非常低效的情况下也超过竞争方法。