results: despite the structural differences between the two datasets, the predictability of variables of interest for inverse molecular design is high. This is exemplified with models estimating the number of atoms of the molecule from both the original properties and from lower dimensional embedding spaces.Abstract
This paper explores the internal structure of two quantum mechanics datasets (QM7b, QM9), composed of several thousands of organic molecules and described in terms of electronic properties. Understanding the structure and characteristics of this kind of data is important when predicting the atomic composition from the properties in inverse molecular designs. Intrinsic dimension analysis, clustering, and outlier detection methods were used in the study. They revealed that for both datasets the intrinsic dimensionality is several times smaller than the descriptive dimensions. The QM7b data is composed of well defined clusters related to atomic composition. The QM9 data consists of an outer region predominantly composed of outliers, and an inner core region that concentrates clustered, inliner objects. A significant relationship exists between the number of atoms in the molecule and its outlier/inner nature. Despite the structural differences, the predictability of variables of interest for inverse molecular design is high. This is exemplified with models estimating the number of atoms of the molecule from both the original properties, and from lower dimensional embedding spaces.
摘要
The QM7b dataset is composed of well-defined clusters related to atomic composition, while the QM9 dataset has an outer region primarily consisting of outliers and an inner core region with clustered, linear objects. There is a significant correlation between the number of atoms in the molecule and its outlier/inner nature. Despite the structural differences, the predictability of variables of interest for inverse molecular design is high, as demonstrated by models estimating the number of atoms of the molecule from both the original properties and lower-dimensional embedding spaces.
Towards a statistical theory of data selection under weak supervision
paper_authors: Germain Kolossov, Andrea Montanari, Pulkit Tandon
for: 选择一个小于原始样本大小的子样本，以优化数据预处理和机器学习计算复杂性。
methods: 使用代理模型预测样本标签，然后选择一个子样本，并使用这些标签进行模型训练。
results: 数据选择可以非常有效，在某些情况下甚至可以超过使用整个样本集来训练模型。另外，一些受欢迎的数据选择方法（如偏向重样本或影响函数基于的子样本选择）可能会很差。Abstract
Given a sample of size $N$, it is often useful to select a subsample of smaller size $n摘要
Given a sample size $N$, it is often useful to select a subsample of smaller size $n
Disruption Detection for a Cognitive Digital Supply Chain Twin Using Hybrid Deep Learning
paper_authors: Mahmoud Ashraf, Amr Eltawil, Islam Ali
For: The paper aims to provide an effective and efficient tool for mitigating the impact of disruptive events on global supply chains by introducing a hybrid deep learning approach for disruption detection within a cognitive digital supply chain twin framework.* Methods: The proposed approach uses a deep autoencoder neural network combined with a one-class support vector machine algorithm to detect disruptions in real-time. Long-short term memory neural network models are also developed to identify the disrupted echelon and predict time-to-recovery from the disruption effect.* Results: The proposed approach can help decision-makers and supply chain practitioners make appropriate decisions aiming at minimizing negative impact of disruptive events based on real-time disruption detection data. The results demonstrate the trade-off between disruption detection model sensitivity, encountered delay in disruption detection, and false alarms.Here are the three key points in Simplified Chinese text:
results: 该方法可以帮助决策者和供应链实践者根据实时干扰检测数据进行合适的决策，以减轻突发事件的负面影响。结果表明干扰检测模型的敏感度、遇到延迟的干扰检测和假阳性的负面关系。这种方法在当前文献中很少被使用。Abstract
Purpose: Recent disruptive events, such as COVID-19 and Russia-Ukraine conflict, had a significant impact of global supply chains. Digital supply chain twins have been proposed in order to provide decision makers with an effective and efficient tool to mitigate disruption impact. Methods: This paper introduces a hybrid deep learning approach for disruption detection within a cognitive digital supply chain twin framework to enhance supply chain resilience. The proposed disruption detection module utilises a deep autoencoder neural network combined with a one-class support vector machine algorithm. In addition, long-short term memory neural network models are developed to identify the disrupted echelon and predict time-to-recovery from the disruption effect. Results: The obtained information from the proposed approach will help decision-makers and supply chain practitioners make appropriate decisions aiming at minimizing negative impact of disruptive events based on real-time disruption detection data. The results demonstrate the trade-off between disruption detection model sensitivity, encountered delay in disruption detection, and false alarms. This approach has seldom been used in recent literature addressing this issue.
摘要
目的：latest disruptive events, such as COVID-19 and Russia-Ukraine conflict, have had a significant impact on global supply chains. Digital supply chain twins have been proposed to provide decision makers with an effective and efficient tool to mitigate the impact of disruptions.方法：this paper introduces a hybrid deep learning approach for disruption detection within a cognitive digital supply chain twin framework to enhance supply chain resilience. The proposed disruption detection module uses a deep autoencoder neural network combined with a one-class support vector machine algorithm. In addition, long-short term memory neural network models are developed to identify the disrupted echelon and predict time-to-recovery from the disruption effect.结果：the obtained information from the proposed approach will help decision-makers and supply chain practitioners make appropriate decisions aiming at minimizing the negative impact of disruptive events based on real-time disruption detection data. The results demonstrate the trade-off between disruption detection model sensitivity, encountered delay in disruption detection, and false alarms. This approach has seldom been used in recent literature addressing this issue.Here's the translation in Traditional Chinese:目的：最近的干扰事件，如COVID-19和俄乌战争，对全球供应链造成了巨大的影响。数字供应链双生物被提议，以提供决策者具有更高效和更高效的工具，以mitigate干扰的影响。方法：本篇文章介绍了一种混合深度学习方法，用于干扰检测在认知数字供应链双生物框架中，以提高供应链可靠性。提议的干扰检测模组使用深度自动Encoder神经网络，与一类支持向量机器学习算法结合。此外，长期快速传统神经网络模型也被开发，以识别受到干扰的层次，并预测干扰影响的时间回复。结果：取得的信息将助决策者和供应链实践者做出适当的决策，以减少干扰事件的负面影响。结果显示出干扰检测模型的敏感度、遭遇延误的干扰检测时间和误干扰的负面影响之间的贸易。这种方法在最近的文献中 rarely 被使用以解决这个问题。
Cluster-based Method for Eavesdropping Identification and Localization in Optical Links
paper_authors: Haokun Song, Rui Lin, Andrea Sgambelluri, Filippo Cugini, Yajie Li, Jie Zhang, Paolo Monti
for: 检测和定位光线系统中的窃听事件
methods: 基于集群方法检测和定位窃听事件
results: 通过收集发送器端的光性能监测数据可以检测小功率损失引起的窃听事件，而通过利用线上监测数据可以有效地定位窃听事件。Abstract
We propose a cluster-based method to detect and locate eavesdropping events in optical line systems characterized by small power losses. Our findings indicate that detecting such subtle losses from eavesdropping can be accomplished solely through optical performance monitoring (OPM) data collected at the receiver. On the other hand, the localization of such events can be effectively achieved by leveraging in-line OPM data.
摘要
我们提出了一种基于集群的方法，用于检测和定位光纤系统中的窃听事件。我们的发现表明，通过收集接收端的光性能监测（OPM）数据，可以寻查到这些微量损失。然而，通过利用线上OPM数据，可以有效地确定这些事件的位置。
Detach-ROCKET: Sequential feature selection for time series classification with random convolutional kernels
results: 根据UCR archive的试验结果，SFD方法可以将时间序列分类模型的特征数量从原本的1000多个降至10%左右，同时提高模型的测试精度0.2%。此外， Detach-ROCKET方法可以对最大的 binary UCR 数据集进行最佳化，从而提高测试精度0.6%，同时删除98.9%的特征。Abstract
Time Series Classification (TSC) is essential in many fields, such as medicine, environmental science and finance, enabling tasks like disease diagnosis, anomaly detection, and stock price analysis. Machine learning models for TSC like Recurrent Neural Networks and InceptionTime, while successful in numerous applications, can face scalability limitations due to intensive computational requirements. To address this, efficient models such as ROCKET and its derivatives have emerged, simplifying training and achieving state-of-the-art performance by utilizing a large number of randomly generated features from time series data. However, due to their random nature, most of the generated features are redundant or non-informative, adding unnecessary computational load and compromising generalization. Here, we introduce Sequential Feature Detachment (SFD) as a method to identify and prune these non-essential features. SFD uses model coefficients to estimate feature importance and, unlike previous algorithms, can handle large feature sets without the need for complex hyperparameter tuning. Testing on the UCR archive demonstrates that SFD can produce models with $10\%$ of the original features while improving the accuracy $0.2\%$ on the test set. We also present an end-to-end procedure for determining an optimal balance between the number of features and model accuracy, called Detach-ROCKET. When applied to the largest binary UCR dataset, Detach-ROCKET is able to improve test accuracy by $0.6\%$ while reducing the number of features by $98.9\%$. Thus, our proposed procedure is not only lightweight to train and effective in reducing model size and enhancing generalization, but its significant reduction in feature count also paves the way for feature interpretation.
摘要
时序分类（TSC）在医学、环境科学和金融等领域具有重要意义，可以实现疾病诊断、异常检测和股票价格分析等任务。机器学习模型 для TSC，如循环神经网络和InceptionTime，虽然在多个应用中取得成功，但可能会面临扩展性限制，因为计算需求很高。为解决这问题，有效的模型如ROCKET和其 derivates出现了，使得训练更加简单，并在多个时序数据上实现了状态机器学习性能。然而，由于这些生成的特征 Random，大多数生成的特征都是 redundant或非指导的，这会增加计算负担并降低泛化性。在这种情况下，我们介绍了时序特征分离（SFD）方法，可以识别和剔除非关键的特征。SFD使用模型系数来估计特征重要性，与之前的算法不同之处在于可以处理大量特征集 ohne需要复杂的超参数调整。在UCRL архиivos上进行测试，SFD可以生成模型，其中90%的特征是非关键的，而测试集上的准确率提高0.2%。我们还提出了一种从头到尾的过程，可以确定最佳的特征数量和模型准确率之间的平衡，称为Detach-ROCKET。当应用于最大的二进制UCRL数据集时，Detach-ROCKET可以提高测试准确率0.6%，同时减少特征数量98.9%。因此，我们的提出的过程不仅轻量级训练，效果减小模型大小和提高泛化性，而且它的重要减少特征计数也为特征解释开辟了道路。
results: 这个论文表明了 \texttt{Zo-RASA} 算法可以使用单个样本或常数级批处理在每个迭代中实现 $\epsilon$-近似首Order站点解。此外，论文还引入了一种新的几何条件，即 bounded second fundamental form，可以用于精度地 approximate parallel transport。Abstract
We present Zeroth-order Riemannian Averaging Stochastic Approximation (\texttt{Zo-RASA}) algorithms for stochastic optimization on Riemannian manifolds. We show that \texttt{Zo-RASA} achieves optimal sample complexities for generating $\epsilon$-approximation first-order stationary solutions using only one-sample or constant-order batches in each iteration. Our approach employs Riemannian moving-average stochastic gradient estimators, and a novel Riemannian-Lyapunov analysis technique for convergence analysis. We improve the algorithm's practicality by using retractions and vector transport, instead of exponential mappings and parallel transports, thereby reducing per-iteration complexity. Additionally, we introduce a novel geometric condition, satisfied by manifolds with bounded second fundamental form, which enables new error bounds for approximating parallel transport with vector transport.
摘要
我们提出了零预orde Riemannian Averaging Stochastic Approximation（\texttt{Zo-RASA）}算法，用于在里曼尼抽象上进行数学估计。我们证明了\texttt{Zo-RASA}可以在每个迭代中使用单一样本或常量组数据，实现 $\epsilon$-近似首先稳定解的生成。我们的方法使用里曼尼运动平均梯度估计器，并使用一种新的里曼尼- Lyapunov 分析技术进行对准性分析。我们通过使用抽像和向量运输，而不是对称对映和平行运输，将每次迭代的复杂性降低。此外，我们也提出了一个新的几何条件，这个条件是在具有受限第二funamental form的抽象上存在的，它允许我们给出新的错误上限，用于近似平行运输。
Uncertainty Aware Deep Learning for Particle Accelerators
for: 这篇论文是为了提出一种用Deep Gaussian Process Approximation（DGPA）方法进行误束预测和不确定性评估的方法。
methods: 这篇论文使用了Deep Gaussian Process Approximation（DGPA）方法，该方法可以捕捉复杂系统动态，但是需要考虑误束和不确定性。
results: 这篇论文在SNS加速器中进行了误束预测和不确定性评估，并提供了一个不确定性意识的模型。Abstract
Standard deep learning models for classification and regression applications are ideal for capturing complex system dynamics. However, their predictions can be arbitrarily inaccurate when the input samples are not similar to the training data. Implementation of distance aware uncertainty estimation can be used to detect these scenarios and provide a level of confidence associated with their predictions. In this paper, we present results from using Deep Gaussian Process Approximation (DGPA) methods for errant beam prediction at Spallation Neutron Source (SNS) accelerator (classification) and we provide an uncertainty aware surrogate model for the Fermi National Accelerator Lab (FNAL) Booster Accelerator Complex (regression).
摘要
标准的深度学习模型可以很好地捕捉复杂系统的动态。但是，它们的预测结果可能无法实际地准确，尤其是对于与训练数据不同的输入数据。在这篇文章中，我们使用深度 Gaussian Process Approximation（DGPA）方法进行误偏照射预测（分类），并提供了不对称 uncertainty 意识模型（重回应）。
Unveiling the Potential of Deep Learning Models for Solar Flare Prediction in Near-Limb Regions
methods: 我们使用了三种well-known deep learning architecture——AlexNet、VGG16和ResNet34进行了转移学习，并对三个模型进行了比较和评估，使用了真实技能统计（TSS）和Heidke技能分数（HSS），以及计算了回快分数，以理解预测敏感性在中心和近似limb区域中。
results: 我们的研究发现，使用AlexNet基于的模型在全体性能方面表现最高，其TSS和HSS分别为0.53和0.37；而进一步的空间分析回快分数显示，在近似limb事件中，VGG16和ResNet34基于的模型具有更高的预测敏感性。最佳结果是使用ResNet34基于的模型，其near-limb预测回快率为0.59（X-和M-class预测回快率分别为0.81和0.56）。Abstract
This study aims to evaluate the performance of deep learning models in predicting $\geq$M-class solar flares with a prediction window of 24 hours, using hourly sampled full-disk line-of-sight (LoS) magnetogram images, particularly focusing on the often overlooked flare events corresponding to the near-limb regions (beyond $\pm$70$^{\circ}$ of the solar disk). We trained three well-known deep learning architectures--AlexNet, VGG16, and ResNet34 using transfer learning and compared and evaluated the overall performance of our models using true skill statistics (TSS) and Heidke skill score (HSS) and computed recall scores to understand the prediction sensitivity in central and near-limb regions for both X- and M-class flares. The following points summarize the key findings of our study: (1) The highest overall performance was observed with the AlexNet-based model, which achieved an average TSS$\sim$0.53 and HSS$\sim$0.37; (2) Further, a spatial analysis of recall scores disclosed that for the near-limb events, the VGG16- and ResNet34-based models exhibited superior prediction sensitivity. The best results, however, were seen with the ResNet34-based model for the near-limb flares, where the average recall was approximately 0.59 (the recall for X- and M-class was 0.81 and 0.56 respectively) and (3) Our research findings demonstrate that our models are capable of discerning complex spatial patterns from full-disk magnetograms and exhibit skill in predicting solar flares, even in the vicinity of near-limb regions. This ability holds substantial importance for operational flare forecasting systems.
摘要
这项研究的目标是评估深度学习模型在预测24小时内的$\geq$M级太阳风暴事件的性能，使用每小时采样的全盘线性图像，特别是关注太阳盘面外(-70度以上)的快速风暴事件。我们使用了三种已知的深度学习架构——AlexNet、VGG16和ResNet34进行转移学习，并对三个模型进行比较和评估，使用真实技能统计（TSS）和海德ке技能分数（HSS），并计算了中心和近缘区域的回快率以了解预测敏感度。研究的主要发现包括：1. AlexNet基于的模型在整体性能方面表现最高，其TSS和HSS分别为0.53和0.37；2. 空间分析回快率表明，近缘区域内的风暴事件预测敏感度最高，VGG16和ResNet34基于的模型在近缘区域内表现出色，特别是ResNet34基于的模型，其平均回快率为0.59，X级和M级风暴事件的回快率分别为0.81和0.56；3. 这些研究发现表明，我们的模型可以从全盘线性图像中提取复杂的空间特征，并在靠近近缘区域的风暴事件预测中展现出能力。这种能力对于实际风暴预测系统具有重要意义。
results: 实验结果显示，LogGPT在三个 datasets 上表现出色，较 existing state-of-the-art 方法有更高的检测精度。Abstract
Detecting system anomalies based on log data is important for ensuring the security and reliability of computer systems. Recently, deep learning models have been widely used for log anomaly detection. The core idea is to model the log sequences as natural language and adopt deep sequential models, such as LSTM or Transformer, to encode the normal patterns in log sequences via language modeling. However, there is a gap between language modeling and anomaly detection as the objective of training a sequential model via a language modeling loss is not directly related to anomaly detection. To fill up the gap, we propose LogGPT, a novel framework that employs GPT for log anomaly detection. LogGPT is first trained to predict the next log entry based on the preceding sequence. To further enhance the performance of LogGPT, we propose a novel reinforcement learning strategy to finetune the model specifically for the log anomaly detection task. The experimental results on three datasets show that LogGPT significantly outperforms existing state-of-the-art approaches.
摘要
检测计算机系统中的异常 based on 日志数据是保持安全和可靠性的重要任务。最近，深度学习模型在日志异常检测中得到了广泛的应用。核心思想是模型日志序列为自然语言，采用深度序列模型，如LSTM或Transformer，来编码正常的日志序列模式 via 语言模型化。但是，语言模型化和异常检测之间存在一个差距，因为训练深度序列模型via语言模型化损失并不直接相关于异常检测。为填充这个差距，我们提出了 LogGPT，一种新的框架，它采用 GPT 进行日志异常检测。LogGPT 首先通过预测下一个日志条目基于前一个序列来训练。为了进一步提高 LogGPT 的性能，我们提出了一种新的强化学习策略，用于特定地 finetune 模型为日志异常检测任务。实验结果表明，LogGPT 与现有状态的方法相比，在三个数据集上显著地提高了性能。
Skilog: A Smart Sensor System for Performance Analysis and Biofeedback in Ski Jumping
results: 研究获得了92.7%的中心质量预测精度（脊梁偏移、中立位和腹股偏移），并在低功耗的RISC-V架构上进行了实时推导和反馈（0.0109ms/推导）。Abstract
In ski jumping, low repetition rates of jumps limit the effectiveness of training. Thus, increasing learning rate within every single jump is key to success. A critical element of athlete training is motor learning, which has been shown to be accelerated by feedback methods. In particular, a fine-grained control of the center of gravity in the in-run is essential. This is because the actual takeoff occurs within a blink of an eye ($\sim$300ms), thus any unbalanced body posture during the in-run will affect flight. This paper presents a smart, compact, and energy-efficient wireless sensor system for real-time performance analysis and biofeedback during ski jumping. The system operates by gauging foot pressures at three distinct points on the insoles of the ski boot at 100Hz. Foot pressure data can either be directly sent to coaches to improve their feedback, or fed into a ML model to give athletes instantaneous in-action feedback using a vibration motor in the ski boot. In the biofeedback scenario, foot pressures act as input variables for an optimized XGBoost model. We achieve a high predictive accuracy of 92.7% for center of mass predictions (dorsal shift, neutral stand, ventral shift). Subsequently, we parallelized and fine-tuned our XGBoost model for a RISC-V based low power parallel processor (GAP9), based on the PULP architecture. We demonstrate real-time detection and feedback (0.0109ms/inference) using our on-chip deployment. The proposed smart system is unobtrusive with a slim form factor (13mm baseboard, 3.2mm antenna) and a lightweight build (26g). Power consumption analysis reveals that the system's energy-efficient design enables sustained operation over multiple days (up to 300 hours) without requiring recharge.
摘要
在跳台滑雪中，低重复率的跳跃限制了训练的效iveness。因此，在每次跳跃中提高学习率是关键到success。运动员训练中的核心元素是 дви作学习，已经证明可以通过反馈方法加速。特别是在具有细致控制中心重力的跑道上是关键。因为实际的起飞只需要几十毫秒（约300ms），所以任何不平衡的身体姿势会影响飞行。本文介绍了一种智能、卷积、能效的无线传感器系统，用于实时性表现分析和生物反馈 durante跳台滑雪。该系统通过在跳板底部的三个点检测脚压力，每秒100次获取数据。脚压力数据可以直接给教练提供反馈，或者通过一个机器学习模型给运动员实时反馈，使用跳板内置的振荡机。在生物反馈场景中，脚压力作为输入变量，用于优化的XGBoost模型。我们实现了中心质量预测的高预测精度（92.7%）。然后，我们将XGBoost模型并行化和优化，基于RISC-V架构的低功耗并行处理器（GAP9）。我们实现了实时探测和反馈（0.0109ms/推导），并在芯片上部署。提案的智能系统轻便，减少了跳板的尺寸（13mm基板、3.2mm天线）和重量（26g）。能源消耗分析表明，该系统的能效设计可以持续运行多天（最多300小时）无需充电。
Learning dislocation dynamics mobility laws from large-scale MD simulations
results: 在 BCC 钨中示出了准确地复制了真实 MD simulations 中的压缩/张力偏见，并且在低刺激速度下预测了流体压力， demonstrating 了方法的能力学习扭轧动力学的Physics。Abstract
The computational method of discrete dislocation dynamics (DDD), used as a coarse-grained model of true atomistic dynamics of lattice dislocations, has become of powerful tool to study metal plasticity arising from the collective behavior of dislocations. As a mesoscale approach, motion of dislocations in the DDD model is prescribed via the mobility law; a function which specifies how dislocation lines should respond to the driving force. However, the development of traditional hand-crafted mobility laws can be a cumbersome task and may involve detrimental simplifications. Here we introduce a machine-learning (ML) framework to streamline the development of data-driven mobility laws which are modeled as graph neural networks (GNN) trained on large-scale Molecular Dynamics (MD) simulations of crystal plasticity. We illustrate our approach on BCC tungsten and demonstrate that our GNN mobility implemented in large-scale DDD simulations accurately reproduces the challenging tension/compression asymmetry observed in ground-truth MD simulations while correctly predicting the flow stress at lower straining rate conditions unseen during training, thereby demonstrating the ability of our method to learn relevant dislocation physics. Our DDD+ML approach opens new promising avenues to improve fidelity of the DDD model and to incorporate more complex dislocation motion behaviors in an automated way, providing a faithful proxy for dislocation dynamics several orders of magnitude faster than ground-truth MD simulations.
摘要
计算方法的粗化扭变动力学（DDD）模型，作为真实原子动力学扭变动力学的粗化模型，已成为金属塑形力学的研究powerful工具。作为中规模方法，DDD模型中扭变线的运动是通过 mobilicity 法规定的，这是一个指定扭变线应对驱动力的函数。然而，开发传统手动设计 mobilicity 法可能是一项繁琐的任务，并且可能会带来不利的简化。在这里，我们引入机器学习（ML）框架，以数据驱动的方式开发出更加简单的 mobilicity 法。我们使用Graph Neural Networks（GNN）模型，在大规模的分子动力学（MD） simulation 中训练这些 mobilicity 法。我们在 BCC 钴中实现了我们的 GNN mobilicity，并在大规模的 DDD simulations 中证明了我们的方法可以准确地复制真实 MD simulation 中的困难的压缩/扩展不均勋，并且可以正确地预测低剪力环境下的流体压缩强度。这表明我们的方法可以学习扭变物理学。我们的 DDD + ML 方法打开了新的可能性，以提高 DDD 模型的准确性，并自动地包含更加复杂的扭变动力学行为，提供一个 faithful 的扭变动力学代理，在训练中未达到的低剪力环境下可以正确地预测流体压缩强度。
results: 这篇论文的结果是证明任何量子内核都可以表示为量子特征状态的内积，并且提出了一些新的、未探索的量子内核家族，其中是否还有效的嵌入量子内核还需要进一步研究。Abstract
One of the most natural connections between quantum and classical machine learning has been established in the context of kernel methods. Kernel methods rely on kernels, which are inner products of feature vectors living in large feature spaces. Quantum kernels are typically evaluated by explicitly constructing quantum feature states and then taking their inner product, here called embedding quantum kernels. Since classical kernels are usually evaluated without using the feature vectors explicitly, we wonder how expressive embedding quantum kernels are. In this work, we raise the fundamental question: can all quantum kernels be expressed as the inner product of quantum feature states? Our first result is positive: Invoking computational universality, we find that for any kernel function there always exists a corresponding quantum feature map and an embedding quantum kernel. The more operational reading of the question is concerned with efficient constructions, however. In a second part, we formalize the question of universality of efficient embedding quantum kernels. For shift-invariant kernels, we use the technique of random Fourier features to show that they are universal within the broad class of all kernels which allow a variant of efficient Fourier sampling. We then extend this result to a new class of so-called composition kernels, which we show also contains projected quantum kernels introduced in recent works. After proving the universality of embedding quantum kernels for both shift-invariant and composition kernels, we identify the directions towards new, more exotic, and unexplored quantum kernel families, for which it still remains open whether they correspond to efficient embedding quantum kernels.
摘要
（一些）自然的量子机器学习和经典机器学习之间的连接在内核方法上已经得到了证明。内核方法 rely on 内核，它们是特征向量生活在大特征空间的内积。量子内核通常通过明确构建量子特征状态来评估，然后计算它们的内积，这被称为嵌入量子内核。由于经典内核通常不直接使用特征向量，我们所思考嵌入量子内核的表达能力如何。在这项工作中，我们提出了一个基本问题：可以所有的量子内核都表示为量子特征状态的内积吗？我们的第一个结果是正的：通过计算 universality，我们发现了对于任何内核函数，都存在一个对应的量子特征映射和嵌入量子内核。对于更操作性的问题，我们在第二部分中正式化了嵌入量子内核的 universality 问题。对于不变内核，我们使用随机傅里埃特性来证明它们是 universality 的，并将其扩展到一个新的 composition kernels 类型，这类型包括已知的 projected quantum kernels。在证明嵌入量子内核的 universality 之后，我们确定了新、更有趣、未探索的量子内核家族的方向，其中是否存在效果嵌入量子内核还未知。
Provable advantages of kernel-based quantum learners and quantum preprocessing based on Grover’s algorithm
paper_authors: Till Muser, Elias Zapusek, Vasilis Belis, Florentin Reiter
for: 该研究目的是提高学习问题的计算效率，特别是应用量子计算机在支持向量机中的速度优势。
methods: 该研究使用了Shor的算法和Grover的算法来实现量子支持向量机的速度优势。
results: 研究发现，通过在支持向量机的kernel中使用量子计算机，可以获得 exponential 的速度优势。此外，通过将量子计算机与类传统的分类方法结合使用，可以进一步提高分类器的性能。Abstract
There is an ongoing effort to find quantum speedups for learning problems. Recently, [Y. Liu et al., Nat. Phys. $\textbf{17}$, 1013--1017 (2021)] have proven an exponential speedup for quantum support vector machines by leveraging the speedup of Shor's algorithm. We expand upon this result and identify a speedup utilizing Grover's algorithm in the kernel of a support vector machine. To show the practicality of the kernel structure we apply it to a problem related to pattern matching, providing a practical yet provable advantage. Moreover, we show that combining quantum computation in a preprocessing step with classical methods for classification further improves classifier performance.
摘要
有一个持续进行的努力是找到量子速度减少学习问题。最近，李宇等人（Nat. Phys. $\textbf{17}$, 1013--1017 (2021)）已经证明了量子支持向量机的加速，通过利用戈Vor的算法速度。我们在这个结果基础上进一步扩展，并证明了使用格罗弗尔算法在支持向量机的kernel中获得加速。为证明实用性，我们应用了这种结构到一个相关的模式匹配问题，并提供了实用却可证明的优势。此外，我们还证明了结合量子计算在预处理步骤中与经典方法结合，可以进一步提高分类器性能。
Tasks Makyth Models: Machine Learning Assisted Surrogates for Tipping Points
paper_authors: Gianluca Fabiani, Nikolaos Evangelou, Tianqi Cui, Juan M. Bello-Rivas, Cristina P. Martin-Linares, Constantinos Siettos, Ioannis G. Kevrekidis
results: 该论文通过使用这些方法对高维时空数据进行压缩， constructions 缩写模型来描述不同的级别的 emergent 动力学，并且可以准确地预测突变点和罕见事件的可能性。Abstract
We present a machine learning (ML)-assisted framework bridging manifold learning, neural networks, Gaussian processes, and Equation-Free multiscale modeling, for (a) detecting tipping points in the emergent behavior of complex systems, and (b) characterizing probabilities of rare events (here, catastrophic shifts) near them. Our illustrative example is an event-driven, stochastic agent-based model (ABM) describing the mimetic behavior of traders in a simple financial market. Given high-dimensional spatiotemporal data -- generated by the stochastic ABM -- we construct reduced-order models for the emergent dynamics at different scales: (a) mesoscopic Integro-Partial Differential Equations (IPDEs); and (b) mean-field-type Stochastic Differential Equations (SDEs) embedded in a low-dimensional latent space, targeted to the neighborhood of the tipping point. We contrast the uses of the different models and the effort involved in learning them.
摘要
我们提出了一个基于机器学习（ML）的框架，它将拓扑学学习、神经网络、高斯过程和无方程多尺度模型绑定在一起，用于检测复杂系统的 emergent 行为中的跌宕点，以及在其附近的罕见事件的概率Characterization。我们的示例是一个事件驱动的随机 Agent-Based Model（ABM），描述了金融市场中的模拟行为。给出高维空间时间数据（由随机 ABM 生成），我们构建了不同级别的减少模型，用于描述不同级别的 emergent 动力学：（a） mesoscopic Integro-Partial Differential Equations（IPDEs）；和（b） Mean-field-type Stochastic Differential Equations（SDEs），其embedded在一个低维的隐藏空间中，targeted to the neighborhood of the tipping point。我们对不同模型的使用和学习努力进行了对比。
pLMFPPred: a novel approach for accurate prediction of functional peptides integrating embedding from pre-trained protein language model and imbalanced learning
results: 在一个验证的独立测试集上，pLMFPPred实现了精度、接收操作特征曲线值和F1值的0.974、0.99和0.974，分别。 comparative experiments show that pLMFPPred outperforms current methods for predicting functional peptides。实验结果表明，提案的方法（pLMFPPred）可以在预测功能肽方面提供更好的性能，并代表一种新的计算方法。Abstract
Functional peptides have the potential to treat a variety of diseases. Their good therapeutic efficacy and low toxicity make them ideal therapeutic agents. Artificial intelligence-based computational strategies can help quickly identify new functional peptides from collections of protein sequences and discover their different functions.Using protein language model-based embeddings (ESM-2), we developed a tool called pLMFPPred (Protein Language Model-based Functional Peptide Predictor) for predicting functional peptides and identifying toxic peptides. We also introduced SMOTE-TOMEK data synthesis sampling and Shapley value-based feature selection techniques to relieve data imbalance issues and reduce computational costs. On a validated independent test set, pLMFPPred achieved accuracy, Area under the curve - Receiver Operating Characteristics, and F1-Score values of 0.974, 0.99, and 0.974, respectively. Comparative experiments show that pLMFPPred outperforms current methods for predicting functional peptides.The experimental results suggest that the proposed method (pLMFPPred) can provide better performance in terms of Accuracy, Area under the curve - Receiver Operating Characteristics, and F1-Score than existing methods. pLMFPPred has achieved good performance in predicting functional peptides and represents a new computational method for predicting functional peptides.
摘要
功能蛋白质有很大的治疗潜力。它们的良好的治疗效果和低度的致病性使得它们成为理想的药物代用品。通过人工智能基于计算的方法，可以快速地从蛋白质序列集中获取新的功能蛋白质和其不同的功能。我们使用蛋白质语言模型基于嵌入（ESM-2），开发了一个名为pLMFPPred（蛋白质语言模型基于功能蛋白质预测器）的工具，用于预测功能蛋白质和识别毒蛋白质。我们还使用SMOTE-TOMEK数据合成抽样和Shapley值基于特征选择技术来解决数据均衡问题和降低计算成本。在一个验证的独立测试集上，pLMFPPred实现了精度、接收操作特征曲线值和F1分数的值为0.974、0.99和0.974，分别。相比之下，相关的方法实现了较差的效果。实验结果表明，提案的方法（pLMFPPred）可以在预测功能蛋白质方面提供更好的表现，并代表了一种新的计算方法。
A Unified Framework for Uniform Signal Recovery in Nonlinear Generative Compressed Sensing
paper_authors: Junren Chen, Jonathan Scarlett, Michael K. Ng, Zhaoqiang Liu
For: The paper is written to study the problem of generative compressed sensing (GCS) with nonlinear measurements, and to provide uniform recovery guarantees for this problem.* Methods: The paper uses a unified framework that combines the observation model with the generative model to derive uniform recovery guarantees for nonlinear GCS. The framework accommodates GCS with 1-bit/uniformly quantized observations and single index models as canonical examples.* Results: The paper shows that using a single realization of the sensing ensemble and generalized Lasso, all $\mathbf{x}^*\in G(\mathbb{B}_2^k(r))$ can be recovered up to an $\ell_2$-error at most $\epsilon$ using roughly $\tilde{O}({k}/{\epsilon^2})$ samples, with omitted logarithmic factors typically being dominated by $\log L$. This is almost coincident with existing non-uniform guarantees up to logarithmic factors, indicating that the uniformity comes with a very small cost.Abstract
In generative compressed sensing (GCS), we want to recover a signal $\mathbf{x}^* \in \mathbb{R}^n$ from $m$ measurements ($m\ll n$) using a generative prior $\mathbf{x}^*\in G(\mathbb{B}_2^k(r))$, where $G$ is typically an $L$-Lipschitz continuous generative model and $\mathbb{B}_2^k(r)$ represents the radius-$r$ $\ell_2$-ball in $\mathbb{R}^k$. Under nonlinear measurements, most prior results are non-uniform, i.e., they hold with high probability for a fixed $\mathbf{x}^*$ rather than for all $\mathbf{x}^*$ simultaneously. In this paper, we build a unified framework to derive uniform recovery guarantees for nonlinear GCS where the observation model is nonlinear and possibly discontinuous or unknown. Our framework accommodates GCS with 1-bit/uniformly quantized observations and single index models as canonical examples. Specifically, using a single realization of the sensing ensemble and generalized Lasso, {\em all} $\mathbf{x}^*\in G(\mathbb{B}_2^k(r))$ can be recovered up to an $\ell_2$-error at most $\epsilon$ using roughly $\tilde{O}({k}/{\epsilon^2})$ samples, with omitted logarithmic factors typically being dominated by $\log L$. Notably, this almost coincides with existing non-uniform guarantees up to logarithmic factors, hence the uniformity costs very little. As part of our technical contributions, we introduce the Lipschitz approximation to handle discontinuous observation models. We also develop a concentration inequality that produces tighter bounds for product processes whose index sets have low metric entropy. Experimental results are presented to corroborate our theory.
摘要
在生成式压缩感知（GCS）中，我们想要从 $m$ 测量 ($m \ll n$) 中还原一个信号 $\mathbf{x}^* \in \mathbb{R}^n$ 使用生成模型 $G$，其中 $G$ 是一个 Typically $L$-Lipschitz 连续的生成模型，$\mathbb{B}_2^k(r)$ 表示半径-$r$ $\ell_2$-球在 $\mathbb{R}^k$ 中。在非线性测量下，大多数先前结果是非均匀的，即它们在固定 $\mathbf{x}^*$ 上持有高概率而不是所有 $\mathbf{x}^*$ 上同时持有。在这篇论文中，我们构建了一个统一的框架，用于 derive 均匀的恢复保证，对于非线性 GCS，测量模型可能是不连续或未知的。我们的框架可以涵盖 GCS 与 1-bit/均匀量化观测和单index模型作为 kanonikus 例子。具体来说，使用单个感知ensemble和普通lasso，所有 $\mathbf{x}^*\in G(\mathbb{B}_2^k(r))$ 可以在 $\ell_2$ 误差不超过 $\epsilon$ 的情况下，使用约 $\tilde{O}({k}/{\epsilon^2})$ 个样本进行恢复，忽略 logs 因子通常是由 $\log L$ 控制。这与现有的非均匀保证相差只有 logs 因子，因此均匀性的代价很低。在我们的技术贡献中，我们引入了 Lipschitz approximation 来处理不连续测量模型。我们还开发了一种集中不等式，用于生成产品过程中的指标集 whose 度量 entropy 较低。实验结果用于证明我们的理论。
Futility and utility of a few ancillas for Pauli channel learning
For: 本文 revisits one of the prototypical tasks for characterizing the structure of noise in quantum devices, estimating the eigenvalues of an $n$-qubit Pauli noise channel.* Methods: 本文使用 exponential lower bounds to show the limitations of algorithms for estimating the eigenvalues of the noise channel. These lower bounds hold even for the easier hypothesis testing problem of determining whether the underlying channel is completely depolarizing or has exactly one other nontrivial eigenvalue.* Results: 本文 gets the following results: 1. Any algorithm without quantum memory must make $\Omega(2^n/\epsilon^2)$ measurements to estimate each eigenvalue within error $\epsilon$. 2. Any algorithm with $\le k$ ancilla qubits of quantum memory must make $\Omega(2^{(n-k)/3})$ queries to the unknown channel. 3. With only $k=2$ ancilla qubits of quantum memory, there is an algorithm that solves the hypothesis testing task with high probability using a single measurement.I hope this helps! Let me know if you have any other questions.Abstract
In this paper we revisit one of the prototypical tasks for characterizing the structure of noise in quantum devices, estimating the eigenvalues of an $n$-qubit Pauli noise channel. Prior work (Chen et al., 2022) established exponential lower bounds for this task for algorithms with limited quantum memory. We first improve upon their lower bounds and show: (1) Any algorithm without quantum memory must make $\Omega(2^n/\epsilon^2)$ measurements to estimate each eigenvalue within error $\epsilon$. This is tight and implies the randomized benchmarking protocol is optimal, resolving an open question of (Flammia and Wallman, 2020). (2) Any algorithm with $\le k$ ancilla qubits of quantum memory must make $\Omega(2^{(n-k)/3})$ queries to the unknown channel. Crucially, unlike in (Chen et al., 2022), our bound holds even if arbitrary adaptive control and channel concatenation are allowed. In fact these lower bounds, like those of (Chen et al., 2022), hold even for the easier hypothesis testing problem of determining whether the underlying channel is completely depolarizing or has exactly one other nontrivial eigenvalue. Surprisingly, we show that: (3) With only $k=2$ ancilla qubits of quantum memory, there is an algorithm that solves this hypothesis testing task with high probability using a single measurement. Note that (3) does not contradict (2) as the protocol concatenates exponentially many queries to the channel before the measurement. This result suggests a novel mechanism by which channel concatenation and $O(1)$ qubits of quantum memory could work in tandem to yield striking speedups for quantum process learning that are not possible for quantum state learning.
摘要
在这篇论文中，我们回顾了一个杜理量化器设备噪声结构的典型任务，即估算 $n$- Quint Pauli 噪声通道的 eigenvalues。先前的工作（Chen et al., 2022）已经证明了这个任务的下界，我们首先提高了这个下界并证明： 1. 没有量子储存的算法必须做 $\Omega(2^n/\epsilon^2)$ 测量来估算每个含误值。这是最佳的和 Flammia 和 Wallman（2020）的问题的解。 2. 具有 $\le k$ ancilla qubits 的量子储存算法必须做 $\Omega(2^{(n-k)/3})$ 请求来 unknown 通道。这个下界不仅如 Chen et al.（2022）的下界，而且允许任意适应控制和通道 concatenation。 事实上，这些下界也适用于另一个更容易的假设测试问题，即判断 underlying 通道是完全depolarizing 或者有 exactly one 其他非特性含误值。我们证明： 3. 只有 $k=2$ ancilla qubits 的量子储存算法可以使用单个测量来高probability 解决这个假设测试问题。这个结果表明，通过 concatenating exponentially many queries to the channel before the measurement, it is possible to achieve striking speedups for quantum process learning that are not possible for quantum state learning.
Small-scale proxies for large-scale Transformer training instabilities
paper_authors: Mitchell Wortsman, Peter J. Liu, Lechao Xiao, Katie Everett, Alex Alemi, Ben Adlam, John D. Co-Reyes, Izzeddin Gur, Abhishek Kumar, Roman Novak, Jeffrey Pennington, Jascha Sohl-dickstein, Kelvin Xu, Jaehoon Lee, Justin Gilmer, Simon Kornblith
methods: 本文使用了两种源引起训练不稳定性的研究：在注意层中增长的logits（Dehghani et al., 2023）和输出logits与输出概率的分化（Chowdhery et al., 2022）。通过测量学习率和损失之间的关系，我们显示这些不稳定性也在小模型中出现，并且在高学习率训练中使用了先前在大规模训练中使用的缓解方法可以达到相似的损失值。
results: 本文研究了一些已知优化器和模型调整的影响，包括温存、Weight decay和$\mu$Param（Yang et al., 2022）。我们发现可以通过组合这些技术来训练小模型，以实现在不同学习率下的损失值之间的相似性。最后，我们研究了两种可以预测训练不稳定性的情况：模型活动和梯度 нор的扩展行为。Abstract
Teams that have trained large Transformer-based models have reported training instabilities at large scale that did not appear when training with the same hyperparameters at smaller scales. Although the causes of such instabilities are of scientific interest, the amount of resources required to reproduce them has made investigation difficult. In this work, we seek ways to reproduce and study training stability and instability at smaller scales. First, we focus on two sources of training instability described in previous work: the growth of logits in attention layers (Dehghani et al., 2023) and divergence of the output logits from the log probabilities (Chowdhery et al., 2022). By measuring the relationship between learning rate and loss across scales, we show that these instabilities also appear in small models when training at high learning rates, and that mitigations previously employed at large scales are equally effective in this regime. This prompts us to investigate the extent to which other known optimizer and model interventions influence the sensitivity of the final loss to changes in the learning rate. To this end, we study methods such as warm-up, weight decay, and the $\mu$Param (Yang et al., 2022), and combine techniques to train small models that achieve similar losses across orders of magnitude of learning rate variation. Finally, to conclude our exploration we study two cases where instabilities can be predicted before they emerge by examining the scaling behavior of model activation and gradient norms.
摘要
团队已经训练过大型Transformer模型时报告了训练不稳定的问题，而这些问题在相同的超参数下不会出现。虽然这些不稳定的原因具有科学兴趣，但是需要资源来调查。在这项工作中，我们寻找一种在小规模上重现和研究训练稳定性和不稳定性。我们首先关注以下两种训练不稳定的来源：在注意层中 logged 的生长（Dehghani et al., 2023）和输出 logits 与 log probability 的分化（Chowdhery et al., 2022）。通过测量学习率和损失之间的关系，我们表明这些不稳定也会在小型模型中出现，并且在这种情况下，已经在大规模上使用的 Mitigation 也是有效的。这使得我们想 investigate 其他知道的优化器和模型 intervención 对最终损失响应于学习率变化的敏感性。为此，我们研究了温存、重量 decay 和 $\mu$Param（Yang et al., 2022）等方法，并将这些方法结合使用来训练小模型，以实现在不同学习率下的相同损失水平。最后，我们研究了两种可以预测训练不稳定之前的情况：模型活动和梯度 norms 的扩散行为。
Lifelong Robot Learning with Human Assisted Language Planners
results: 研究人员通过实验和实际应用，证明了该方法可以帮助机器人在不同任务中快速学习和应用新的技能，并且可以在未来的任务中重用已经学习的技能。I hope that helps! Let me know if you have any other questions.Abstract
Large Language Models (LLMs) have been shown to act like planners that can decompose high-level instructions into a sequence of executable instructions. However, current LLM-based planners are only able to operate with a fixed set of skills. We overcome this critical limitation and present a method for using LLM-based planners to query new skills and teach robots these skills in a data and time-efficient manner for rigid object manipulation. Our system can re-use newly acquired skills for future tasks, demonstrating the potential of open world and lifelong learning. We evaluate the proposed framework on multiple tasks in simulation and the real world. Videos are available at: https://sites.google.com/mit.edu/halp-robot-learning.
摘要
大型语言模型（LLM）已经被证明可以 acted like 观察者，将高水平的指令分解为可执行的指令序列。但现有的 LLM-based 观察者只能运行固定的技能。我们解决了这个极限，并提出了使用 LLM-based 观察者来查询新技能并教育机器人这些技能，在数据和时间效率下进行弹性物件抓取。我们的系统可以重复 newly acquired 技能，以便在未来任务中重复使用，这显示了开放世界和一生学习的潜力。我们在多个任务中进行了评估，并在网站上提供了详细的视频：https://sites.google.com/mit.edu/halp-robot-learning。
A post-selection algorithm for improving dynamic ensemble selection methods
results: 实验结果显示，使用精度作为选择组件方法的评估指标，PS-DES方法比单一的DES方法表现更好。Here’s the translation in English:
for: The purpose of this research is to select the best Multiple Classifier Systems (MCS) method to improve accuracy.
methods: The method used in this research is the Post-Selection Dynamic Ensemble Selection (PS-DES) method, which selects the best ensemble method in the selection phase.
results: Experimental results show that using accuracy as a metric to select the ensembles, PS-DES outperforms individual DES techniques.I hope that helps!Abstract
Dynamic Ensemble Selection (DES) is a Multiple Classifier Systems (MCS) approach that aims to select an ensemble for each query sample during the selection phase. Even with the proposal of several DES approaches, no particular DES technique is the best choice for different problems. Thus, we hypothesize that selecting the best DES approach per query instance can lead to better accuracy. To evaluate this idea, we introduce the Post-Selection Dynamic Ensemble Selection (PS-DES) approach, a post-selection scheme that evaluates ensembles selected by several DES techniques using different metrics. Experimental results show that using accuracy as a metric to select the ensembles, PS-DES performs better than individual DES techniques. PS-DES source code is available in a GitHub repository
摘要
<> translate "Dynamic Ensemble Selection (DES) is a Multiple Classifier Systems (MCS) approach that aims to select an ensemble for each query sample during the selection phase. Even with the proposal of several DES approaches, no particular DES technique is the best choice for different problems. Thus, we hypothesize that selecting the best DES approach per query instance can lead to better accuracy. To evaluate this idea, we introduce the Post-Selection Dynamic Ensemble Selection (PS-DES) approach, a post-selection scheme that evaluates ensembles selected by several DES techniques using different metrics. Experimental results show that using accuracy as a metric to select the ensembles, PS-DES performs better than individual DES techniques. PS-DES source code is available in a GitHub repository" into Simplified Chinese.Here's the translation:<>多个类ifier系统（MCS）方法之一是动态ensemble选择（DES），它在选择阶段为每个查询样本选择一个ensemble。尽管已经提出了多种DES方法，但是没有一个特定的DES技术适合所有问题。因此，我们提出了在每个查询实例中选择最佳DES方法的想法，以提高准确率。为了评估这个想法，我们引入了后期选择的动态ensemble选择（PS-DES）方法，该方法使用不同的度量评估由多种DES技术选择的ensemble。实验结果显示，使用准确率作为度量选择ensemble时，PS-DES方法perform Better than个 DES技术。PS-DES源代码可以在GitHub存储库中找到。
Improved Algorithms for Stochastic Linear Bandits Using Tail Bounds for Martingale Mixtures
paper_authors: Hamish Flynn, David Reeb, Melih Kandemir, Jan Peters
for: 这个论文targets the stochastic linear bandit problem, and proposes improved algorithms with worst-case regret guarantees.
methods: 该论文使用了一种新的tail bound for adaptive martingale mixtures to construct confidence sequences, which are suitable for stochastic bandits. These confidence sequences allow for efficient action selection via convex programming.
results: 该论文提供了一种基于 confidence sequences的 linear bandit algorithm, which is guaranteed to achieve competitive worst-case regret. Additionally, the authors show that their confidence sequences are tighter than competitors, both empirically and theoretically, and demonstrate improved performance in several hyperparameter tuning tasks.Abstract
We present improved algorithms with worst-case regret guarantees for the stochastic linear bandit problem. The widely used "optimism in the face of uncertainty" principle reduces a stochastic bandit problem to the construction of a confidence sequence for the unknown reward function. The performance of the resulting bandit algorithm depends on the size of the confidence sequence, with smaller confidence sets yielding better empirical performance and stronger regret guarantees. In this work, we use a novel tail bound for adaptive martingale mixtures to construct confidence sequences which are suitable for stochastic bandits. These confidence sequences allow for efficient action selection via convex programming. We prove that a linear bandit algorithm based on our confidence sequences is guaranteed to achieve competitive worst-case regret. We show that our confidence sequences are tighter than competitors, both empirically and theoretically. Finally, we demonstrate that our tighter confidence sequences give improved performance in several hyperparameter tuning tasks.
摘要
我们提出了改进的算法，带有最坏情况的悔检保证，用于Stochastic Linear Bandit问题。通过“面对不确定性的optimism”原则，将随机bandit问题转化为建立不确定奖金函数的信任序列。算法的性能取决于信任序列的大小，小的信任序列对实际性能和悔检保证具有更好的效果。在这项工作中，我们使用了一种新的尾部 bound for adaptive martingale mixtures来构建信任序列，这些信任序列适用于随机bandits。这些信任序列使得可以通过几何编程进行高效的动作选择。我们证明了一个基于我们的信任序列的线性bandit算法能够实现竞争性最坏情况的悔检保证。我们还证明了我们的信任序列比竞争者更紧， tantoempirically和理论上。最后，我们示出了我们的紧密信任序列可以提高一些超参数调整任务的性能。
paper_authors: Lisa Kuhn, Bernhard Klein, Holger Fröning
for: 这种研究旨在探讨分析计算中的缺失精度问题，以及这些问题对机器学习任务的影响。
methods: 该研究使用了一个简单的模型来示例出实际的分析处理器中的排序效应。
results: 结果表明，忽略排序可能会导致准确率下降substantially。Abstract
The energy efficiency of analog forms of computing makes it one of the most promising candidates to deploy resource-hungry machine learning tasks on resource-constrained system such as mobile or embedded devices. However, it is well known that for analog computations the safety net of discretization is missing, thus all analog computations are exposed to a variety of imperfections of corresponding implementations. Examples include non-linearities, saturation effect and various forms of noise. In this work, we observe that the ordering of input operands of an analog operation also has an impact on the output result, which essentially makes analog computations non-associative, even though the underlying operation might be mathematically associative. We conduct a simple test by creating a model of a real analog processor which captures such ordering effects. With this model we assess the importance of ordering by comparing the test accuracy of a neural network for keyword spotting, which is trained based either on an ordered model, on a non-ordered variant, and on real hardware. The results prove the existence of ordering effects as well as their high impact, as neglecting ordering results in substantial accuracy drops.
摘要
“Analog计算的能源效率使得它成为部署资源受限的移动或嵌入式设备上耗费资源的最佳候选人。然而，所有的analog计算都缺乏精度的保障，因此它们暴晒于实现中的各种不稳定性，如非线性、饱和效应和各种噪声。在这个工作中，我们发现了输入操作的顺序也对输出结果产生影响，从而使analog计算变得非关联的，即使其下面的运算可能是数学上关联的。我们创建了一个模型，用于捕捉这些顺序效应。通过这个模型，我们评估了顺序的重要性，并发现忽略顺序会导致准确性下降。”Note: The translation is in Simplified Chinese, which is the standard writing system used in mainland China. If you need Traditional Chinese, please let me know.
DECORAIT – DECentralized Opt-in/out Registry for AI Training
paper_authors: Kar Balan, Alex Black, Simon Jenni, Andrew Gilbert, Andy Parsons, John Collomosse
For: The paper aims to address the data governance challenge faced by content creators who want to share their work openly without sanctioning its use for training AI models, and to ensure fair recognition and reward for their contributions.* Methods: The paper proposes a decentralized registry called DECORAIT, which uses hierarchical clustering and a combination of on/off-chain storage to trace the provenance of GenAI training data and determine training consent. The registry leverages distributed ledger technology (DLT) and visual fingerprinting, and is built on the emerging C2PA standard.* Results: The paper reports a prototype of DECORAIT, which demonstrates the feasibility of using a decentralized registry to trace the provenance of GenAI training data and ensure fair recognition and reward for content creators. The prototype combines the strengths of DLT and visual fingerprinting to create a secure, open registry that can be used to express consent and data ownership for GenAI.Abstract
We present DECORAIT; a decentralized registry through which content creators may assert their right to opt in or out of AI training as well as receive reward for their contributions. Generative AI (GenAI) enables images to be synthesized using AI models trained on vast amounts of data scraped from public sources. Model and content creators who may wish to share their work openly without sanctioning its use for training are thus presented with a data governance challenge. Further, establishing the provenance of GenAI training data is important to creatives to ensure fair recognition and reward for their such use. We report a prototype of DECORAIT, which explores hierarchical clustering and a combination of on/off-chain storage to create a scalable decentralized registry to trace the provenance of GenAI training data in order to determine training consent and reward creatives who contribute that data. DECORAIT combines distributed ledger technology (DLT) with visual fingerprinting, leveraging the emerging C2PA (Coalition for Content Provenance and Authenticity) standard to create a secure, open registry through which creatives may express consent and data ownership for GenAI.
摘要
我们介绍DECORAIT，一个去中心化的数据库，让内容创作者可以选择是否参与人工智能训练，并获得创作所获得的回馈。生成型人工智能（GenAI）可以使用基于大量公开资料的人工智能模型生成图像。如果模型和内容创作者想要公开分享他们的作品而不授权其用于训练，他们面临资料管理挑战。此外，确定GenAI训练资料的起源是重要的，以确保创作者获得公平的认可和奖励。我们报告DECORAIT的原型，它使用嵌入式数据和分支链技术（DLT），并与可识别的视觉指纹（Visual Fingerprinting）集成，以创建一个可靠、公开的数据库，让创作者表达同意和资料所有权 для GenAI。
Learning Risk-Aware Quadrupedal Locomotion using Distributional Reinforcement Learning
results: 本研究在 simulate 和 ANYmal quadruped robot上实现了 emergent 风险敏感的行动行为，并且可以通过控制一个参数来调整机器人的行为风格，从而实现风险敏感性。Abstract
Deployment in hazardous environments requires robots to understand the risks associated with their actions and movements to prevent accidents. Despite its importance, these risks are not explicitly modeled by currently deployed locomotion controllers for legged robots. In this work, we propose a risk sensitive locomotion training method employing distributional reinforcement learning to consider safety explicitly. Instead of relying on a value expectation, we estimate the complete value distribution to account for uncertainty in the robot's interaction with the environment. The value distribution is consumed by a risk metric to extract risk sensitive value estimates. These are integrated into Proximal Policy Optimization (PPO) to derive our method, Distributional Proximal Policy Optimization (DPPO). The risk preference, ranging from risk-averse to risk-seeking, can be controlled by a single parameter, which enables to adjust the robot's behavior dynamically. Importantly, our approach removes the need for additional reward function tuning to achieve risk sensitivity. We show emergent risk sensitive locomotion behavior in simulation and on the quadrupedal robot ANYmal.
摘要
<>对危险环境部署需要机器人理解其动作和移动的风险，以避免意外。现有的步行控制器并未显式地考虑这些风险。在这项工作中，我们提出一种带有安全考虑的步行训练方法，使用分布式再增强学习来考虑安全。而不是仅仅依靠值期望，我们估算整个值分布，以考虑机器人与环境的互动不确定性。这个值分布被消耗到风险度量来提取风险敏感的价值估计。这些估计被 integrate 到 proximal policy optimization（PPO）中，得到我们的方法：分布式 proximal policy optimization（DPPO）。风险偏好，从不敢风险到敢风险，可以通过一个参数控制，以 dynamically 调整机器人的行为。这种方法可以消除需要额外奖励函数调整以实现风险敏感。我们在 simulated 和 ANYmal 四足机器人上实现了 emergent 风险敏感的步行行为。
results: 我们的方法可以在具有高噪音比例的数据上提供有效的学习和决策，并且可以在训练和测试阶段处理不相关的数据。Abstract
Learning and decision-making in domains with naturally high noise-to-signal ratio, such as Finance or Healthcare, is often challenging, while the stakes are very high. In this paper, we study the problem of learning and acting under a general noisy generative process. In this problem, the data distribution has a significant proportion of uninformative samples with high noise in the label, while part of the data contains useful information represented by low label noise. This dichotomy is present during both training and inference, which requires the proper handling of uninformative data during both training and testing. We propose a novel approach to learning under these conditions via a loss inspired by the selective learning theory. By minimizing this loss, the model is guaranteed to make a near-optimal decision by distinguishing informative data from uninformative data and making predictions. We build upon the strength of our theoretical guarantees by describing an iterative algorithm, which jointly optimizes both a predictor and a selector, and evaluates its empirical performance in a variety of settings.
摘要
学习和决策在具有自然高噪声比例的领域，如金融或医疗，经常是一项挑战，而且风险很高。在这篇论文中，我们研究学习和行动在一个通用的噪声生成过程下的问题。在这个问题中，数据分布中有一定的无用样本，具有高噪声的标签，而其中一部分数据具有低噪声的有用信息。这种分化存在于训练和测试阶段，需要正确处理无用数据。我们提出了一种基于选择学习理论的新方法，通过最小化这种损失函数，使模型能够做出最佳决策。我们在理论保证的基础上描述了一种迭代算法，它同时优化一个预测器和一个选择器，并评估其实际性能。
Predicting environment effects on breast cancer by implementing machine learning
paper_authors: Muhammad Shoaib Farooq, Mehreen Ilyas
for: 本研究旨在探讨环境因素对乳腺癌的发生和进程中的作用，以及这些因素对乳腺癌预后的影响。
methods: 本研究使用了机器学习算法，包括逻辑回归、随机森林、KNN算法、Support Vector Machine和Extra Tree Classifier，以表达预测。
results: 研究发现，Random Forest算法的准确率为0.91%，ROC曲线为0.901%，表示这些机器学习算法在乳腺癌存活分析中具有良好的准确性，这些技术可能成为乳腺癌预后预测的新选择。Abstract
The biggest Breast cancer is increasingly a major factor in female fatalities, overtaking heart disease. While genetic factors are important in the growth of breast cancer, new research indicates that environmental factors also play a substantial role in its occurrence and progression. The literature on the various environmental factors that may affect breast cancer risk, incidence, and outcomes is thoroughly reviewed in this study report. The study starts by looking at how lifestyle decisions, such as eating habits, exercise routines, and alcohol consumption, may affect hormonal imbalances and inflammation, two important factors driving the development of breast cancer. Additionally, it explores the part played by environmental contaminants such pesticides, endocrine-disrupting chemicals (EDCs), and industrial emissions, all of which have been linked to a higher risk of developing breast cancer due to their interference with hormone signaling and DNA damage. Algorithms for machine learning are used to express predictions. Logistic Regression, Random Forest, KNN Algorithm, SVC and extra tree classifier. Metrics including the confusion matrix correlation coefficient, F1-score, Precision, Recall, and ROC curve were used to evaluate the models. The best accuracy among all the classifiers is Random Forest with 0.91% accuracy and ROC curve 0.901% of Logistic Regression. The accuracy of the multiple algorithms for machine learning utilized in this research was good, which is important and indicates that these techniques could serve as replacement forecasting techniques in breast cancer survival analysis, notably in the Asia region.
摘要
最大的乳癌是在女性死亡中日益占据主导地位，超越心血管疾病。虽然遗传因素在乳癌增长中扮演重要角色，但新研究表明环境因素也在乳癌发生和进程中扮演了重要角色。本研究报告 thorougly reviewed the literature on various environmental factors that may affect breast cancer risk, incidence, and outcomes. The study begins by examining how lifestyle decisions, such as dietary habits, exercise routines, and alcohol consumption, may affect hormonal imbalances and inflammation, two key factors driving the development of breast cancer. Additionally, it explores the role played by environmental pollutants such as pesticides, endocrine-disrupting chemicals (EDCs), and industrial emissions, all of which have been linked to a higher risk of developing breast cancer due to their interference with hormone signaling and DNA damage. The study used machine learning algorithms, including logistic regression, random forest, KNN algorithm, SVC, and extra tree classifier, to express predictions. Metrics including confusion matrix, correlation coefficient, F1-score, precision, recall, and ROC curve were used to evaluate the models. The best accuracy among all the classifiers was Random Forest with 0.91% accuracy and ROC curve 0.901% of logistic regression. The accuracy of the multiple machine learning algorithms used in this research was good, indicating that these techniques could serve as replacement forecasting techniques in breast cancer survival analysis, particularly in the Asia region.
Guess & Sketch: Language Model Guided Transpilation
results: 根据实验结果，Guess & Sketch 可以成功转换 57.6% 更多的 Assembly code 示例，比 GPT-4 和手动编写的转换器更高效。Abstract
Maintaining legacy software requires many software and systems engineering hours. Assembly code programs, which demand low-level control over the computer machine state and have no variable names, are particularly difficult for humans to analyze. Existing conventional program translators guarantee correctness, but are hand-engineered for the source and target programming languages in question. Learned transpilation, i.e. automatic translation of code, offers an alternative to manual re-writing and engineering efforts. Automated symbolic program translation approaches guarantee correctness but struggle to scale to longer programs due to the exponentially large search space. Their rigid rule-based systems also limit their expressivity, so they can only reason about a reduced space of programs. Probabilistic neural language models (LMs) produce plausible outputs for every input, but do so at the cost of guaranteed correctness. In this work, we leverage the strengths of LMs and symbolic solvers in a neurosymbolic approach to learned transpilation for assembly code. Assembly code is an appropriate setting for a neurosymbolic approach, since assembly code can be divided into shorter non-branching basic blocks amenable to the use of symbolic methods. Guess & Sketch extracts alignment and confidence information from features of the LM then passes it to a symbolic solver to resolve semantic equivalence of the transpilation input and output. We test Guess & Sketch on three different test sets of assembly transpilation tasks, varying in difficulty, and show that it successfully transpiles 57.6% more examples than GPT-4 and 39.6% more examples than an engineered transpiler. We also share a training and evaluation dataset for this task.
摘要
维护遗传软件需要很多软件和系统工程时间。 Assembly 程式，它们需要低层次控制电脑机器状态，并没有变数名称，对人类分析而言特别困难。现有的传统程式翻译器可以保证正确性，但是它们是手工设计的源和目标程式码语言的。学习型翻译，即自动翻译程式码，提供了一个人工重新写程式码的替代方案。自动 симвоlic 程式翻译方法可以保证正确性，但是它们在更长的程式码中缺乏扩展性，因为搜索空间是指数增长的。它们的僵化规则系统也限制了它们的表达力，只能理解一个受限的程式空间。概率神经语言模型（LM）可以生成可能的输出，但是它们在交互时需要付出正确性的代价。在这个工作中，我们利用 LM 和符号方法的优点，在 assembly 程式中实现了学习型翻译。 assembly 程式可以被分成更短的非分支基本块，适合使用符号方法。 Guess & Sketch 首先从 LM 中提取对适合性和信任度的资讯，然后将其转交给符号方法以解决这个翻译任务的内涵相等性。我们在三个不同的 assembly 翻译任务上进行测试，发现 Guess & Sketch 成功翻译了 57.6% 更多的例子，比 GPT-4 和手工设计的翻译器更高。我们还提供了这个任务的训练和评估数据集。
Learning Restricted Boltzmann Machines with greedy quantum search
results: 对于这两种类型的RBMs，量子算法比类型的纯类型算法具有 polynomial 速度增长。Abstract
Restricted Boltzmann Machines (RBMs) are widely used probabilistic undirected graphical models with visible and latent nodes, playing an important role in statistics and machine learning. The task of structure learning for RBMs involves inferring the underlying graph by using samples from the visible nodes. Specifically, learning the two-hop neighbors of each visible node allows for the inference of the graph structure. Prior research has addressed the structure learning problem for specific classes of RBMs, namely ferromagnetic and locally consistent RBMs. In this paper, we extend the scope to the quantum computing domain and propose corresponding quantum algorithms for this problem. Our study demonstrates that the proposed quantum algorithms yield a polynomial speedup compared to the classical algorithms for learning the structure of these two classes of RBMs.
摘要
restrictive Boltzmann machines (RBMs) 是一种广泛使用的可能性图模型，具有可见节点和隐藏节点，在统计学和机器学习中扮演着重要角色。structure learning问题的解决方法是使用可见节点的样本来推断图结构。特别是，了解每个可见节点的两步邻居，可以推断出图结构。先前的研究已经对特定类型的 RBMs 进行了结构学习问题的研究， specifically ferromagnetic 和 locally consistent RBMs。在这篇论文中，我们将这个问题推广到量子计算领域，并提出相应的量子算法来解决这个问题。我们的研究表明，提议的量子算法与类icial算法相比，对于这两种类型的 RBMs 的结构学习问题，具有Polynomial Speedup。
Federated Learning Under Restricted User Availability
results: 实验结果显示，提出的方法可以与标准联合学习相比，在不同的设定下均表现出较好的性能。Abstract
Federated Learning (FL) is a decentralized machine learning framework that enables collaborative model training while respecting data privacy. In various applications, non-uniform availability or participation of users is unavoidable due to an adverse or stochastic environment, the latter often being uncontrollable during learning. Here, we posit a generic user selection mechanism implementing a possibly randomized, stationary selection policy, suggestively termed as a Random Access Model (RAM). We propose a new formulation of the FL problem which effectively captures and mitigates limited participation of data originating from infrequent, or restricted users, at the presence of a RAM. By employing the Conditional Value-at-Risk (CVaR) over the (unknown) RAM distribution, we extend the expected loss FL objective to a risk-aware objective, enabling the design of an efficient training algorithm that is completely oblivious to the RAM, and with essentially identical complexity as FedAvg. Our experiments on synthetic and benchmark datasets show that the proposed approach achieves significantly improved performance as compared with standard FL, under a variety of setups.
摘要
Designing and evaluating an online reinforcement learning agent for physical exercise recommendations in N-of-1 trials
paper_authors: Dominik Meier, Ipek Ensari, Stefan Konigorski
For: The paper is written to explore the feasibility and effectiveness of using an online reinforcement learning agent to implement personalized adaptive interventions in clinical settings.* Methods: The paper uses a novel study on physical exercise recommendations to reduce pain in endometriosis as an illustration, and describes the design of a contextual bandit recommendation agent. The agent is evaluated in simulation studies.* Results: The results show that adaptive interventions can add complexity to the design and implementation process, but have the potential to improve patients’ benefits even with limited observations. The approach is expected to be transferable to other interventions and clinical settings.Here is the information in Simplified Chinese text:* For: 本研究是为了探讨个性化适应性治疗在临床设置中的可行性和效果。* Methods: 本研究使用了一个新的物理运动推荐算法来降低悬股症的痛症，并对这种算法进行了评估。* Results: 结果显示个性化适应性治疗可能会增加设计和实施过程的复杂性，但是它们可以在有限的观察数据下提高患者的效果。这种方法预期可以在其他治疗和临床设置中传递应用。Abstract
Personalized adaptive interventions offer the opportunity to increase patient benefits, however, there are challenges in their planning and implementation. Once implemented, it is an important question whether personalized adaptive interventions are indeed clinically more effective compared to a fixed gold standard intervention. In this paper, we present an innovative N-of-1 trial study design testing whether implementing a personalized intervention by an online reinforcement learning agent is feasible and effective. Throughout, we use a new study on physical exercise recommendations to reduce pain in endometriosis for illustration. We describe the design of a contextual bandit recommendation agent and evaluate the agent in simulation studies. The results show that adaptive interventions add complexity to the design and implementation process, but have the potential to improve patients' benefits even if only few observations are available. In order to quantify the expected benefit, data from previous interventional studies is required. We expect our approach to be transferable to other interventions and clinical interventions.
摘要
个人化适应 intervención 可以提高病人的效果，但是规划和实施中存在挑战。如果实施了个人化适应 intervención，是否比静态黄金标准 intervención 更有效？在这篇论文中，我们介绍了一种新的 N-of-1 试验研究设计，用于测试个人化 intervención 是否可行和有效。我们使用了一项新的 физи exercise 推荐算法来减轻疼痛的研究，以 illustrate 我们的方法。我们描述了一种上下文 bandit 推荐代理的设计，并在模拟研究中评估了代理。结果显示，个人化 intervención 可以增加病人的效果，但是设计和实施过程可能会加入复杂性。为了量化预期的效果，需要对前一次的 intervenational 研究数据进行分析。我们预计我们的方法可以应用于其他 intervenational 和临床研究。
Extragradient Type Methods for Riemannian Variational Inequality Problems
paper_authors: Zihao Hu, Guanghui Wang, Xi Wang, Andre Wibisono, Jacob Abernethy, Molei Tao for: 这个论文主要研究的是偏微分方程问题（Monotone Riemannian Variational Inequality Problems，简称RVIPs）的最优化问题。methods: 这个论文提出了两种新的算法：Riemannian extragradient（REG）方法和Riemannian past extragradient（RPEG）方法，这两种方法都可以在几何扩展空间上实现最优化问题的解。results: 这个论文的结果表明，REG和RPEG方法的最后迭代都会收敛到RVIPs的解，并且这个收敛速率是$O\left(\frac{1}{\sqrt{T}\right)$。此外，这个论文还证明了REG和RPEG方法的平均迭代收敛速率是$O\left(\frac{1}{T}\right)$，这与欧几里得空间中的观察相一致。Abstract
Riemannian convex optimization and minimax optimization have recently drawn considerable attention. Their appeal lies in their capacity to adeptly manage the non-convexity of the objective function as well as constraints inherent in the feasible set in the Euclidean sense. In this work, we delve into monotone Riemannian Variational Inequality Problems (RVIPs), which encompass both Riemannian convex optimization and minimax optimization as particular cases. In the context of Euclidean space, it is established that the last-iterates of both the extragradient (EG) and past extragradient (PEG) methods converge to the solution of monotone variational inequality problems at a rate of $O\left(\frac{1}{\sqrt{T}\right)$ (Cai et al., 2022). However, analogous behavior on Riemannian manifolds remains an open question. To bridge this gap, we introduce the Riemannian extragradient (REG) and Riemannian past extragradient (RPEG) methods. We demonstrate that both exhibit $O\left(\frac{1}{\sqrt{T}\right)$ last-iterate convergence. Additionally, we show that the average-iterate convergence of both REG and RPEG is $O\left(\frac{1}{T}\right)$, aligning with observations in the Euclidean case (Mokhtari et al., 2020). These results are enabled by judiciously addressing the holonomy effect so that additional complications in Riemannian cases can be reduced and the Euclidean proof inspired by the performance estimation problem (PEP) technique or the sum-of-squares (SOS) technique can be applied again.
摘要
“里曼尼安 convex 优化和最小最大优化在最近吸引了广泛关注。它们的吸引力在于它们可以有效地处理非凸函数和约束的非凸性在欧几何上。在这篇文章中，我们深入研究幂等里曼尼变量不等式问题（RVIPs），它们包括里曼尼 convex 优化和最小最大优化为特殊情况。在欧几何空间中，已经证明了extragradient（EG）和过去extragradient（PEG）方法的最后迭代都会 converge到变量不等式问题的解的 $O\left(\frac{1}{\sqrt{T}\right)$ 速率（Cai et al., 2022）。然而，在里曼尼拓扑上的相似行为仍然是一个未解决的问题。为了桥接这个差距，我们引入里曼尼extragradient（REG）和里曼尼过去extragradient（RPEG）方法。我们证明了这两种方法的最后迭代都会 converge于 $O\left(\frac{1}{\sqrt{T}\right)$ 速率。此外，我们还证明了REG和RPEG的平均迭代速率为 $O\left(\frac{1}{T}\right)$，与欧几何空间中观察到的（Mokhtari et al., 2020）相一致。这些结果是通过谨慎地处理启动效应，使得里曼尼拓扑上的额外复杂性可以被减少，并且可以再次应用欧几何空间中的性能估计问题（PEP）技术或准则集（SOS）技术来实现。”
One-Class Classification for Intrusion Detection on Vehicular Networks
for: 防护 vehicular networks 中的 Controller Area Network bus 系统免受现代黑客攻击
methods: 使用机器学习方法进行检测和报告攻击
results: 试验了多种state-of-the-art一类分类方法对 Controller Area Network bus 流量中的射预攻击的效果，发现Subspace Support Vector Data Description 方法在normal operation 和被攻击时都能够最高效，Gmean 约85%Abstract
Controller Area Network bus systems within vehicular networks are not equipped with the tools necessary to ward off and protect themselves from modern cyber-security threats. Work has been done on using machine learning methods to detect and report these attacks, but common methods are not robust towards unknown attacks. These methods usually rely on there being a sufficient representation of attack data, which may not be available due to there either not being enough data present to adequately represent its distribution or the distribution itself is too diverse in nature for there to be a sufficient representation of it. With the use of one-class classification methods, this issue can be mitigated as only normal data is required to train a model for the detection of anomalous instances. Research has been done on the efficacy of these methods, most notably One-Class Support Vector Machine and Support Vector Data Description, but many new extensions of these works have been proposed and have yet to be tested for injection attacks in vehicular networks. In this paper, we investigate the performance of various state-of-the-art one-class classification methods for detecting injection attacks on Controller Area Network bus traffic. We investigate the effectiveness of these techniques on attacks launched on Controller Area Network buses from two different vehicles during normal operation and while being attacked. We observe that the Subspace Support Vector Data Description method outperformed all other tested methods with a Gmean of about 85%.
摘要
控制器网络攻击系统在交通网络中没有具备防御modern cyber安全攻击的工具。工作已经在使用机器学习方法检测和报告这些攻击，但通用方法不够鲜硬度Unknown攻击。这些方法通常需要充分的攻击数据来表征攻击的分布，但可能缺乏数据或者攻击分布太多样化，导致无法充分表征。使用一类分类方法可以解决这问题，只需要正常数据来训练模型来检测异常情况。研究表示一类支持向量数据描述法和一类支持向量分类器在检测插入攻击方面表现出色，但还没有在交通网络中进行测试。本文 investigate了一些当前顶尖一类分类方法在Controller Area Network总线上检测插入攻击的性能。我们对两辆不同的车辆在正常运行和遭受攻击时的Controller Area Network总线上的攻击进行了测试。我们发现Subspace Support Vector Data Description法的性能高于所有测试过的方法，Gmean约85%。
Driving behavior-guided battery health monitoring for electric vehicles using machine learning
paper_authors: Nanhua Jiang, Jiawei Zhang, Weiran Jiang, Yao Ren, Jing Lin, Edwin Khoo, Ziyou Song
for: 提供了一种基于特征的机器学习管道，用于准确和可靠地监测电池健康状态。
methods: 使用了多种健康指标（HI）的合理选择和融合，以及考虑了实际驾驶行为。
results: 提供了一种能够考虑实际驾驶行为的功能特征选择和融合方法，以提高电池健康监测的准确性和实用性。Abstract
An accurate estimation of the state of health (SOH) of batteries is critical to ensuring the safe and reliable operation of electric vehicles (EVs). Feature-based machine learning methods have exhibited enormous potential for rapidly and precisely monitoring battery health status. However, simultaneously using various health indicators (HIs) may weaken estimation performance due to feature redundancy. Furthermore, ignoring real-world driving behaviors can lead to inaccurate estimation results as some features are rarely accessible in practical scenarios. To address these issues, we proposed a feature-based machine learning pipeline for reliable battery health monitoring, enabled by evaluating the acquisition probability of features under real-world driving conditions. We first summarized and analyzed various individual HIs with mechanism-related interpretations, which provide insightful guidance on how these features relate to battery degradation modes. Moreover, all features were carefully evaluated and screened based on estimation accuracy and correlation analysis on three public battery degradation datasets. Finally, the scenario-based feature fusion and acquisition probability-based practicality evaluation method construct a useful tool for feature extraction with consideration of driving behaviors. This work highlights the importance of balancing the performance and practicality of HIs during the development of feature-based battery health monitoring algorithms.
摘要
《 accurately estimating the state of health (SOH) of batteries is crucial for ensuring the safe and reliable operation of electric vehicles (EVs). feature-based machine learning methods have shown great potential for rapidly and precisely monitoring battery health status. however, using various health indicators (HIs) simultaneously may weaken estimation performance due to feature redundancy. furthermore, ignoring real-world driving behaviors can lead to inaccurate estimation results as some features are rarely accessible in practical scenarios. to address these issues, we proposed a feature-based machine learning pipeline for reliable battery health monitoring, enabled by evaluating the acquisition probability of features under real-world driving conditions. we first summarized and analyzed various individual HIs with mechanism-related interpretations, which provide insightful guidance on how these features relate to battery degradation modes. moreover, all features were carefully evaluated and screened based on estimation accuracy and correlation analysis on three public battery degradation datasets. finally, the scenario-based feature fusion and acquisition probability-based practicality evaluation method construct a useful tool for feature extraction with consideration of driving behaviors. this work highlights the importance of balancing the performance and practicality of HIs during the development of feature-based battery health monitoring algorithms.》Note: Please note that the translation is in Simplified Chinese, and the sentence structure and wording may be different from the original text.
Physics-Informed Solution of The Stationary Fokker-Plank Equation for a Class of Nonlinear Dynamical Systems: An Evaluation Study
paper_authors: Hussam Alhussein, Mohammed Khasawneh, Mohammed F. Daqaq for: This paper aims to present a data-free, physics-informed neural network (PINN) framework to solve the Fokker-Planck (FP) equation for a class of nonlinear stochastic dynamical systems.methods: The PINN framework uses a neural network to approximate the solution of the FP equation, without requiring any data from the system.results: The paper demonstrates the ability and accuracy of the PINN framework in predicting the probability density function (PDF) under the combined effect of additive and multiplicative noise, capturing P-bifurcations of the PDF, and effectively treating high-dimensional systems. The computational time associated with the PINN solution can be substantially reduced by using transfer learning.Abstract
The Fokker-Planck (FP) equation is a linear partial differential equation which governs the temporal and spatial evolution of the probability density function (PDF) associated with the response of stochastic dynamical systems. An exact analytical solution of the FP equation is only available for a limited subset of dynamical systems. Semi-analytical methods are available for larger, yet still a small subset of systems, while traditional computational methods; e.g. Finite Elements and Finite Difference require dividing the computational domain into a grid of discrete points, which incurs significant computational costs for high-dimensional systems. Physics-informed learning offers a potentially powerful alternative to traditional computational schemes. To evaluate its potential, we present a data-free, physics-informed neural network (PINN) framework to solve the FP equation for a class of nonlinear stochastic dynamical systems. In particular, through several examples concerning the stochastic response of the Duffing, Van der Pol, and the Duffing-Van der Pol oscillators, we assess the ability and accuracy of the PINN framework in $i)$ predicting the PDF under the combined effect of additive and multiplicative noise, $ii)$ capturing P-bifurcations of the PDF, and $iii)$ effectively treating high-dimensional systems. Through comparisons with Monte-Carlo simulations and the available literature, we show that PINN can effectively address all of the afore-described points. We also demonstrate that the computational time associated with the PINN solution can be substantially reduced by using transfer learning.
摘要
《福克-普朗克方程》是一个线性偏微分方程，其控制了杂态征函数（PDF）的时间和空间演化，该PDF与杂态动力系统的响应相关。唯一的精确分析解是仅适用于有限个动力系统中。半分析方法可以用于更大的子集，而传统计算方法，如finite element和finite difference，需要将计算Domain分成一个离散点的网格，这会带来高维系统的计算成本很高。物理学 Informed learning提供了一种可能有力的替代方案。为了评估其潜力，我们提出了一个数据自由、物理学 Informed neural network（PINN）框架，用于解决非线性杂态动力系统的福克-普朗克方程。具体来说，通过DUFFING、VAN der POL和DUFFING-VAN der POL振荡器的 einige examples，我们评估了PINN框架在下列方面的能力和准确性：1. 对于添加itive和乘数噪声的PDF预测。2. 捕捉PDF的P-分岔。3. 对高维系统的有效处理。通过与 Monte-Carlo 仿真和已有文献的比较，我们显示了PINN可以有效地解决上述问题。此外，我们还示出了使用传输学习可以将PINN解的计算时间显著减少。
for: 这 paper 的目的是提出一种可靠、多任务、多模态的机器学习模型，能够在不同的模式下进行预测和检测。
methods: 这 paper 使用了一种名为 MultiModN 的多模态、模块化网络，通过序列化多种数据类型的Feature Space来提高预测性能和可解释性。
results: experiments 表明，MultiModN 在多个 benchmark 数据集上表现出色，能够在不同的模式下进行预测和检测，而且在面临不够数据时并不会出现 catastrophic failure。Abstract
Predicting multiple real-world tasks in a single model often requires a particularly diverse feature space. Multimodal (MM) models aim to extract the synergistic predictive potential of multiple data types to create a shared feature space with aligned semantic meaning across inputs of drastically varying sizes (i.e. images, text, sound). Most current MM architectures fuse these representations in parallel, which not only limits their interpretability but also creates a dependency on modality availability. We present MultiModN, a multimodal, modular network that fuses latent representations in a sequence of any number, combination, or type of modality while providing granular real-time predictive feedback on any number or combination of predictive tasks. MultiModN's composable pipeline is interpretable-by-design, as well as innately multi-task and robust to the fundamental issue of biased missingness. We perform four experiments on several benchmark MM datasets across 10 real-world tasks (predicting medical diagnoses, academic performance, and weather), and show that MultiModN's sequential MM fusion does not compromise performance compared with a baseline of parallel fusion. By simulating the challenging bias of missing not-at-random (MNAR), this work shows that, contrary to MultiModN, parallel fusion baselines erroneously learn MNAR and suffer catastrophic failure when faced with different patterns of MNAR at inference. To the best of our knowledge, this is the first inherently MNAR-resistant approach to MM modeling. In conclusion, MultiModN provides granular insights, robustness, and flexibility without compromising performance.
摘要
多任务多模式（MM）模型目的是抽取多种数据类型的共同预测潜力，创建具有不同大小和类型的输入数据中共同含义的共享特征空间。大多数当前的MM架构使用平行融合这些表示，不仅限制了它们的可解释性，还受到数据类型可用性的限制。我们介绍了MultiModN，一种多模态、模块化网络，可以在任意数量、组合或类型的模态中融合干ARN表示，并提供精细的实时预测反馈。MultiModN的可组合管道是设计可解释的，同时也是自然多任务和鲁棒于基本问题的偏见缺失。我们在多个MM数据集上进行了四个实验，测试了MultiModN在10个实际任务（预测医疾诊断、学术表现和天气）上的性能。结果显示，MultiModN的顺序MM融合不会Compromise performance相比基线Parallel融合。通过模拟偏见缺失（MNAR）的挑战，这个工作表明，与MultiModN不同，平行融合基elines会在不同的MNAR挑战时erroneously learn MNAR并遭受极端的失败。在我们知道的范围内，这是首个自然具有MNAR抗性的MM模型。因此，MultiModN提供了细化的洞察、鲁棒性和灵活性，无需牺牲性能。
HyperTrack: Neural Combinatorics for High Energy Physics
results: 作者通过 partiicle tracking simulations 表明了这种前导AI方法的有效性。代码可以在线获取。Abstract
Combinatorial inverse problems in high energy physics span enormous algorithmic challenges. This work presents a new deep learning driven clustering algorithm that utilizes a space-time non-local trainable graph constructor, a graph neural network, and a set transformer. The model is trained with loss functions at the graph node, edge and object level, including contrastive learning and meta-supervision. The algorithm can be applied to problems such as charged particle tracking, calorimetry, pile-up discrimination, jet physics, and beyond. We showcase the effectiveness of this cutting-edge AI approach through particle tracking simulations. The code is available online.
摘要
高能物理中的 combinatorial inverse problems 涉及到庞大的算法挑战。这项工作提出了一种新的深度学习驱动 clustering 算法，使用空间-时非本地可学习图构建器、图神经网络和集Transformer。该模型通过图节、边和对象层的损失函数进行训练，包括对比学习和超级监督。该算法可以应用于荷电粒子跟踪、calorimetry、堆积排除、jet物理和更多。我们通过粒子跟踪模拟显示了这种前沿人工智能方法的效果。代码可在线获取。
paper_authors: Georgios N. Yannakakis, David Melhart
for: 这篇论文探讨了现代情感计算原理、方法和工具在游戏领域的应用，即情感游戏计算。
methods: 论文通过四个核心phasis of the affective loop：游戏情感诱发、游戏情感感知、游戏情感检测和游戏情感适应来进行了评查。
results: 论文提供了情感游戏计算领域的一份综述，并在这一领域中进行了一系列分析和评估。Abstract
This paper surveys the current state of the art in affective computing principles, methods and tools as applied to games. We review this emerging field, namely affective game computing, through the lens of the four core phases of the affective loop: game affect elicitation, game affect sensing, game affect detection and game affect adaptation. In addition, we provide a taxonomy of terms, methods and approaches used across the four phases of the affective game loop and situate the field within this taxonomy. We continue with a comprehensive review of available affect data collection methods with regards to gaming interfaces, sensors, annotation protocols, and available corpora. The paper concludes with a discussion on the current limitations of affective game computing and our vision for the most promising future research directions in the field.
摘要
We then review available affect data collection methods for gaming interfaces, sensors, annotation protocols, and corpora. Finally, we discuss the current limitations of affective game computing and outline the most promising future research directions in the field.Here is the text in Simplified Chinese:这篇论文介绍了现代情感计算原则、方法和工具在游戏领域的应用。我们通过分析四个核心阶段的情感循环来评估这个新兴领域：游戏情感诱发、游戏情感感知、游戏情感检测和游戏情感适应。此外，我们提供了情感循环中不同阶段的术语、方法和approaches的分类，并将该领域置于这种分类中。接下来，我们进行了有关游戏界面、感知器、注释协议和可用数据集的情感数据收集方法的全面回顾。最后，我们讨论了情感游戏计算的当前局限性，并提出了未来研究的最有前途的方向。
Tracking Control for a Spherical Pendulum via Curriculum Reinforcement Learning
results: 学习策略与优化基线相当，在真实系统上达到了类似于最优控制策略的性能。Abstract
Reinforcement Learning (RL) allows learning non-trivial robot control laws purely from data. However, many successful applications of RL have relied on ad-hoc regularizations, such as hand-crafted curricula, to regularize the learning performance. In this paper, we pair a recent algorithm for automatically building curricula with RL on massively parallelized simulations to learn a tracking controller for a spherical pendulum on a robotic arm via RL. Through an improved optimization scheme that better respects the non-Euclidean task structure, we allow the method to reliably generate curricula of trajectories to be tracked, resulting in faster and more robust learning compared to an RL baseline that does not exploit this form of structured learning. The learned policy matches the performance of an optimal control baseline on the real system, demonstrating the potential of curriculum RL to jointly learn state estimation and control for non-linear tracking tasks.
摘要
On the Benefit of Optimal Transport for Curriculum Reinforcement Learning
paper_authors: Pascal Klink, Carlo D’Eramo, Jan Peters, Joni Pajarinen
for: solves complex tasks by generating a tailored sequence of learning tasks
methods: uses interpolations between task distributions to generate curricula
results: improves upon existing CRL methods and achieves high performance in various tasksAbstract
Curriculum reinforcement learning (CRL) allows solving complex tasks by generating a tailored sequence of learning tasks, starting from easy ones and subsequently increasing their difficulty. Although the potential of curricula in RL has been clearly shown in various works, it is less clear how to generate them for a given learning environment, resulting in various methods aiming to automate this task. In this work, we focus on framing curricula as interpolations between task distributions, which has previously been shown to be a viable approach to CRL. Identifying key issues of existing methods, we frame the generation of a curriculum as a constrained optimal transport problem between task distributions. Benchmarks show that this way of curriculum generation can improve upon existing CRL methods, yielding high performance in various tasks with different characteristics.
摘要
使用简化中文翻译文本。学习补充课程（CRL）可以解决复杂任务，通过生成适应性较高的学习任务序列，从易于学习的任务开始，然后逐渐增加Difficulty。虽然CRL的潜力已经在不同的研究中得到了证明，但是如何为给定的学习环境生成课程，还是一个不够清楚的问题，因此有多种方法试图自动化这个任务。在这个工作中，我们将关注将课程框架为 interpolations between task distributions，这种方法在过去已经被证明是CRL的可能的方法。从exististing方法的角度，我们将生成课程的问题定义为constrained optimal transport problem between task distributions。 benchmark表明，这种方法可以超越现有的CRL方法，在不同的任务特点下实现高性能。
results: 实验表明，BiSinger系统可以在英语和中文普通话之间进行自由融合，同时保持中文歌曲表现。音频样本可以在https://bisinger-svs.github.io中找到。Abstract
Although Singing Voice Synthesis (SVS) has made great strides with Text-to-Speech (TTS) techniques, multilingual singing voice modeling remains relatively unexplored. This paper presents BiSinger, a bilingual pop SVS system for English and Chinese Mandarin. Current systems require separate models per language and cannot accurately represent both Chinese and English, hindering code-switch SVS. To address this gap, we design a shared representation between Chinese and English singing voices, achieved by using the CMU dictionary with mapping rules. We fuse monolingual singing datasets with open-source singing voice conversion techniques to generate bilingual singing voices while also exploring the potential use of bilingual speech data. Experiments affirm that our language-independent representation and incorporation of related datasets enable a single model with enhanced performance in English and code-switch SVS while maintaining Chinese song performance. Audio samples are available at https://bisinger-svs.github.io.
摘要
尽管Singing Voice Synthesis（SVS）已经在文本到语音（TTS）技术方面做出了很大的进步，但多语言歌声模型还尚未得到充分的探索。这篇论文介绍了BiSinger，一个拥有英语和中文普通话的双语PoP SVS系统。现有系统需要separate的模型来处理不同的语言，而这会导致不能准确地表示英语和中文，从而限制了码换SVS。为解决这个难题，我们设计了共享表示 между英语和中文歌声，通过使用CMU字典和映射规则来实现。我们将单语言歌声数据与开源的歌声voice转换技术相结合，以生成双语歌声，同时也在探索使用双语言言言言数据。实验证明了我们的语言独立表示和相关数据的 incorporation 使得单个模型在英语和码换SVS中具有提高的性能，同时保持中文歌曲表现。音频示例可以在https://bisinger-svs.github.io中找到。
REPA: Client Clustering without Training and Data Labels for Improved Federated Learning in Non-IID Settings
results: 对三个不同的数据集进行实验分析，显示REPA可以提供最佳模型性能，同时扩展联合学习的应用范围，覆盖之前未经考虑的用 caso。Abstract
Clustering clients into groups that exhibit relatively homogeneous data distributions represents one of the major means of improving the performance of federated learning (FL) in non-independent and identically distributed (non-IID) data settings. Yet, the applicability of current state-of-the-art approaches remains limited as these approaches cluster clients based on information, such as the evolution of local model parameters, that is only obtainable through actual on-client training. On the other hand, there is a need to make FL models available to clients who are not able to perform the training themselves, as they do not have the processing capabilities required for training, or simply want to use the model without participating in the training. Furthermore, the existing alternative approaches that avert the training still require that individual clients have a sufficient amount of labeled data upon which the clustering is based, essentially assuming that each client is a data annotator. In this paper, we present REPA, an approach to client clustering in non-IID FL settings that requires neither training nor labeled data collection. REPA uses a novel supervised autoencoder-based method to create embeddings that profile a client's underlying data-generating processes without exposing the data to the server and without requiring local training. Our experimental analysis over three different datasets demonstrates that REPA delivers state-of-the-art model performance while expanding the applicability of cluster-based FL to previously uncovered use cases.
摘要
clustering 客户端到组合显示相对同质数据分布的组合是非独立和同样分布（非-IID）数据设置中提高联邦学习（FL）性能的一种主要方法。然而，现有的现状之前方法的应用范围仍然受限，因为这些方法基于本地模型参数的演化获取信息来分组客户端。在另一方面，有一个需要使联邦学习模型可用于无法进行训练的客户端，因为他们没有训练所需的处理能力或者只想使用模型而不参与训练。此外，现有的备用方法都需要每个客户端都具备足够量的标注数据，即每个客户端都是一名数据注释者。在这篇论文中，我们提出了一种不需要训练也不需要标注数据的客户端分组方法，称为REPA。REPA使用一种新的监督式自动encoder方法来创建嵌入，这些嵌入 profiling客户端的下游数据生成过程，无需服务器暴露数据，也无需本地训练。我们在三个不同的数据集上进行了实验分析，结果表明，REPA可以提供状态之前的模型性能，同时扩大了基于分组的联邦学习的应用范围。
Diversify and Conquer: Bandits and Diversity for an Enhanced E-commerce Homepage Experience
paper_authors: Sangeet Jaiswal, Korah T Malayil, Saif Jawaid, Sreekanth Vempati
For: 本研究旨在提高电子商务平台上的推荐广告和产品的效果，具体来说是通过vertical widget reordering来个性化推荐widget。* Methods: 本研究使用了contextual multi-arm bandit问题和增强层来实现个性化推荐。* Results: 通过在Myntra proprietary数据上进行线上和线下A/B测试，研究发现该方法可以提高推荐的效果。Abstract
In the realm of e-commerce, popular platforms utilize widgets to recommend advertisements and products to their users. However, the prevalence of mobile device usage on these platforms introduces a unique challenge due to the limited screen real estate available. Consequently, the positioning of relevant widgets becomes pivotal in capturing and maintaining customer engagement. Given the restricted screen size of mobile devices, widgets placed at the top of the interface are more prominently displayed and thus attract greater user attention. Conversely, widgets positioned further down the page require users to scroll, resulting in reduced visibility and subsequent lower impression rates. Therefore it becomes imperative to place relevant widgets on top. However, selecting relevant widgets to display is a challenging task as the widgets can be heterogeneous, widgets can be introduced or removed at any given time from the platform. In this work, we model the vertical widget reordering as a contextual multi-arm bandit problem with delayed batch feedback. The objective is to rank the vertical widgets in a personalized manner. We present a two-stage ranking framework that combines contextual bandits with a diversity layer to improve the overall ranking. We demonstrate its effectiveness through offline and online A/B results, conducted on proprietary data from Myntra, a major fashion e-commerce platform in India.
摘要
在电商领域中，流行的平台通过Widget来推荐广告和产品给他们的用户。然而，由于移动设备的使用，导致了屏幕空间的限制，这意味着Widget的位置变得非常重要，以确保维持用户的兴趣。由于移动设备的屏幕尺寸有限，位于页面顶部的Widget更加抢耳，因此吸引更多的用户注意力。相反，位于页面底部的Widget需要用户滚动，这会导致它们的可见性减少，并最终导致吸引率下降。因此，需要将相关的Widget放在顶部。然而，选择需要显示的Widget是一项困难的任务，因为Widget可以是不同的，并且可以在任何时候从平台中引入或删除。在这种情况下，我们模型了垂直Widget重新排序为Contextual多臂抽象问题。我们的目标是个性化排序垂直Widget。我们提出了一个两个阶段的排名框架， combinatesContextual bandits with a diversity layer，以提高总体排名的效果。我们通过在Myntra，一家主要的印度电商平台上进行的实验和在线A/B测试，证明了我们的方法的有效性。
Hierarchical Imitation Learning for Stochastic Environments
paper_authors: Maximilian Igl, Punit Shah, Paul Mougin, Sirish Srinivasan, Tarun Gupta, Brandyn White, Kyriacos Shiarlis, Shimon Whiteson
for: 该论文旨在提高自适应学习中代理人的行为模型，以便在训练数据中生成完整的行为分布。
methods: 该论文提出了Robust Type Conditioning（RTC）方法，通过对于随机类型的 adversarial 训练来消除环境噪声导致的分布变换。
results: 实验结果表明，RTC 方法可以提高代理人的行为模型的分布实际性，同时保持或改善任务性能，对于两个领域的大规模实验结果都表现出色。Abstract
Many applications of imitation learning require the agent to generate the full distribution of behaviour observed in the training data. For example, to evaluate the safety of autonomous vehicles in simulation, accurate and diverse behaviour models of other road users are paramount. Existing methods that improve this distributional realism typically rely on hierarchical policies. These condition the policy on types such as goals or personas that give rise to multi-modal behaviour. However, such methods are often inappropriate for stochastic environments where the agent must also react to external factors: because agent types are inferred from the observed future trajectory during training, these environments require that the contributions of internal and external factors to the agent behaviour are disentangled and only internal factors, i.e., those under the agent's control, are encoded in the type. Encoding future information about external factors leads to inappropriate agent reactions during testing, when the future is unknown and types must be drawn independently from the actual future. We formalize this challenge as distribution shift in the conditional distribution of agent types under environmental stochasticity. We propose Robust Type Conditioning (RTC), which eliminates this shift with adversarial training under randomly sampled types. Experiments on two domains, including the large-scale Waymo Open Motion Dataset, show improved distributional realism while maintaining or improving task performance compared to state-of-the-art baselines.
摘要
(Simplified Chinese translation)许多学习借风溯行情景中需要智能机器人生成训练数据中全部行为分布的完整分布。例如，评估自动驾驶车辆在模拟环境中的安全性，需要准确且多样化的其他路用者行为模型。现有的方法通常通过层次政策来提高这种分布真实性，这些政策根据目标或人格类型来生成多模样式的行为。然而，这些方法在随机环境中不适用，因为在训练过程中探测到的未来轨迹会影响智能机器人的行为，因此需要在类型编码中分离内部和外部因素的贡献。编码未来环境信息会导致测试时的不合适机器人反应，因为未来是未知的，类型必须从实际未来中独立地采样。我们将这种挑战称为环境随机性导致的类型 conditional distribution 的分布偏移。我们提议Robust Type Conditioning（RTC），通过对随机类型进行对抗训练来消除这种偏移。实验结果表明，RTC在两个领域中提高了分布真实性，同时保持或提高了任务性能，相比于现有的基elines。
Identification of Mixtures of Discrete Product Distributions in Near-Optimal Sample and Time Complexity
methods: 本研究使用了一种类似于 robust tensor decomposition 的方法，并利用了一种新的约束矩阵的condition number bounding方法，称为 Hadamard extensions。
results: 本研究显示，对于任何 $n\geq 2k-1$，可以在 $(1/\zeta)^{O(k)}$ 的样本复杂度和时间复杂度下识别混合型杂度分布 $X_1,\ldots,X_n$。此外，我们还扩展了 $e^{\Omega(k)}$ 的下界，使其与我们的上界相匹配，并且这种下界适用于各种不同的 $\zeta$。Abstract
We consider the problem of identifying, from statistics, a distribution of discrete random variables $X_1,\ldots,X_n$ that is a mixture of $k$ product distributions. The best previous sample complexity for $n \in O(k)$ was $(1/\zeta)^{O(k^2 \log k)}$ (under a mild separation assumption parameterized by $\zeta$). The best known lower bound was $\exp(\Omega(k))$. It is known that $n\geq 2k-1$ is necessary and sufficient for identification. We show, for any $n\geq 2k-1$, how to achieve sample complexity and run-time complexity $(1/\zeta)^{O(k)}$. We also extend the known lower bound of $e^{\Omega(k)}$ to match our upper bound across a broad range of $\zeta$. Our results are obtained by combining (a) a classic method for robust tensor decomposition, (b) a novel way of bounding the condition number of key matrices called Hadamard extensions, by studying their action only on flattened rank-1 tensors.
摘要
我们考虑一个统计方面的问题，即从分布统计中识别 $X_1,\ldots,X_n$ 是一个由 $k$ 个产品分布组成的混合分布。最好的前一个样本复杂性为 $(1/\zeta)^{O(k^2 \log k)}$（受到 $\zeta$ 的宽度假设），而最佳知识下界为 $\exp(\Omega(k))$。我们知道 $n\geq 2k-1$ 是必要和充分的条件。我们示出，对于任何 $n\geq 2k-1$，可以实现样本复杂性和运行时间复杂性 $(1/\zeta)^{O(k)}$。我们还扩展了知识下界，使其与我们的上界在广泛的 $\zeta$ 范围内匹配。我们的结果来自于将（a）纯粹的稳定矩阵分解方法与（b）一种约束矩阵的condition数 bounds的新方法结合在一起，通过研究这些矩阵在扁平 rank-1 张量上的行为来获得 Hadamard 扩展。
A Novel Approach for Effective Multi-View Clustering with Information-Theoretic Perspective
results: 在理论分析和多个多视图数据集上，SUMVC 方法表现出优于传统方法，提供了一种新的多视图数据分析的视角.Abstract
Multi-view clustering (MVC) is a popular technique for improving clustering performance using various data sources. However, existing methods primarily focus on acquiring consistent information while often neglecting the issue of redundancy across multiple views. This study presents a new approach called Sufficient Multi-View Clustering (SUMVC) that examines the multi-view clustering framework from an information-theoretic standpoint. Our proposed method consists of two parts. Firstly, we develop a simple and reliable multi-view clustering method SCMVC (simple consistent multi-view clustering) that employs variational analysis to generate consistent information. Secondly, we propose a sufficient representation lower bound to enhance consistent information and minimise unnecessary information among views. The proposed SUMVC method offers a promising solution to the problem of multi-view clustering and provides a new perspective for analyzing multi-view data. To verify the effectiveness of our model, we conducted a theoretical analysis based on the Bayes Error Rate, and experiments on multiple multi-view datasets demonstrate the superior performance of SUMVC.
摘要
多视图划分（MVC）是一种广泛使用的技术，以提高划分性能使用多种数据源。然而，现有方法主要强调获取一致信息，经常忽略多视图之间的重复性问题。本研究提出了一种新的方法，即足够多视图划分（SUMVC），它从信息论角度探讨多视图划分框架。我们的提议方法包括两部分：首先，我们开发了一种简单可靠的多视图划分方法SCMVC（简单一致多视图划分），使用变量分析生成一致信息。其次，我们提出了一种足够表示下界，以增强一致信息并最小化多视图之间的无用信息。提出的 SUMVC 方法提供了多视图划分问题的有效解决方案，并提供了新的多视图数据分析的视角。为证明我们的模型的有效性，我们基于 bayes 错误率进行了理论分析，并在多个多视图数据集上进行了实验，展示了 SUMVC 的超越性。
Physics-Driven ML-Based Modelling for Correcting Inverse Estimation
methods: 使用模拟和性能指标 guid by physical laws， flag 机器学习估算失败并提出一种新的方法GEESE，通过优化来实现低误差和高效率。GEESE的关键设计包括（1）一种混合模拟错误模型，以减少模拟成本和启用基于错误反馈的梯度循环，以及（2）两种生成模型，用于模拟候选状态的演示和探索行为。所有三个模型均为神经网络。
results: GEESE在三个真实的 SAE 逆问题上进行测试，与一些现有的优化/搜索方法进行比较。结果表明，GEESE最少失败次数，并且通常需要物理评估更少次。Abstract
When deploying machine learning estimators in science and engineering (SAE) domains, it is critical to avoid failed estimations that can have disastrous consequences, e.g., in aero engine design. This work focuses on detecting and correcting failed state estimations before adopting them in SAE inverse problems, by utilizing simulations and performance metrics guided by physical laws. We suggest to flag a machine learning estimation when its physical model error exceeds a feasible threshold, and propose a novel approach, GEESE, to correct it through optimization, aiming at delivering both low error and high efficiency. The key designs of GEESE include (1) a hybrid surrogate error model to provide fast error estimations to reduce simulation cost and to enable gradient based backpropagation of error feedback, and (2) two generative models to approximate the probability distributions of the candidate states for simulating the exploitation and exploration behaviours. All three models are constructed as neural networks. GEESE is tested on three real-world SAE inverse problems and compared to a number of state-of-the-art optimization/search approaches. Results show that it fails the least number of times in terms of finding a feasible state correction, and requires physical evaluations less frequently in general.
摘要
A hybrid surrogate error model to provide fast error estimations and reduce simulation cost, enabling gradient-based backpropagation of error feedback.2. Two generative models to approximate the probability distributions of the candidate states for simulating the exploitation and exploration behaviors. All three models are constructed as neural networks.GEESE is tested on three real-world SAE inverse problems and compared to several state-of-the-art optimization/search approaches. Results show that it fails the least number of times in terms of finding a feasible state correction, and requires physical evaluations less frequently in general.
Newton Method-based Subspace Support Vector Data Description
results: 实验结果表明，提出的优化策略比 gradient-based S-SVDD 在大多数情况下表现更好。Abstract
In this paper, we present an adaptation of Newton's method for the optimization of Subspace Support Vector Data Description (S-SVDD). The objective of S-SVDD is to map the original data to a subspace optimized for one-class classification, and the iterative optimization process of data mapping and description in S-SVDD relies on gradient descent. However, gradient descent only utilizes first-order information, which may lead to suboptimal results. To address this limitation, we leverage Newton's method to enhance data mapping and data description for an improved optimization of subspace learning-based one-class classification. By incorporating this auxiliary information, Newton's method offers a more efficient strategy for subspace learning in one-class classification as compared to gradient-based optimization. The paper discusses the limitations of gradient descent and the advantages of using Newton's method in subspace learning for one-class classification tasks. We provide both linear and nonlinear formulations of Newton's method-based optimization for S-SVDD. In our experiments, we explored both the minimization and maximization strategies of the objective. The results demonstrate that the proposed optimization strategy outperforms the gradient-based S-SVDD in most cases.
摘要
在这篇论文中，我们介绍了新顿方法在SVDD（Subspace Support Vector Data Description）优化中的应用。S-SVDD的目标是将原始数据映射到优化一类分类的子空间，而S-SVDD的迭代优化过程中的数据映射和描述逻辑依赖于梯度下降。然而，梯度下降只使用一阶信息，可能会导致优化结果不佳。为了解决这种限制，我们利用新顿方法来增强数据映射和描述，从而提高子空间学习基于一类分类的优化。通过利用这些辅助信息，新顿方法在子空间学习中提供了更高效的优化策略，比梯度下降更有效。文章讨论了梯度下降的局限性和新顿方法在子空间学习中的优势，并提供了线性和非线性形式的新顿方法基于优化方法。我们在实验中探索了最小化和最大化目标的两种策略。结果表明，我们提出的优化策略在大多数情况下超越了梯度下降基于S-SVDD的优化。
Beam Enumeration: Probabilistic Explainability For Sample Efficient Self-conditioned Molecular Design
for: This paper aims to improve the explainability and sample efficiency of generative molecular design.
methods: The paper proposes Beam Enumeration, a method that exhaustively enumerates the most probable sub-sequences from language-based molecular generative models and extracts meaningful molecular substructures.
results: The proposed method improves the performance of the recently reported Augmented Memory algorithm, achieving better sample efficiency and generating more high-reward molecules.Here’s the Chinese translation:
results: 提议的方法可以提高最近报道的Augmented Memory算法的性能，实现更好的样本效率和产生更多的高质量分子。Abstract
Generative molecular design has moved from proof-of-concept to real-world applicability, as marked by the surge in very recent papers reporting experimental validation. Key challenges in explainability and sample efficiency present opportunities to enhance generative design to directly optimize expensive high-fidelity oracles and provide actionable insights to domain experts. Here, we propose Beam Enumeration to exhaustively enumerate the most probable sub-sequences from language-based molecular generative models and show that molecular substructures can be extracted. When coupled with reinforcement learning, extracted substructures become meaningful, providing a source of explainability and improving sample efficiency through self-conditioned generation. Beam Enumeration is generally applicable to any language-based molecular generative model and notably further improves the performance of the recently reported Augmented Memory algorithm, which achieved the new state-of-the-art on the Practical Molecular Optimization benchmark for sample efficiency. The combined algorithm generates more high reward molecules and faster, given a fixed oracle budget. Beam Enumeration is the first method to jointly address explainability and sample efficiency for molecular design.
摘要
生成分子设计已经从证明阶段积极应用到实际应用阶段，这是在最近几年的论文中 reporting 实验验证。关键的挑战是解释性和样本效率，这些挑战可以增强生成设计，直接优化昂贵的高精度观测器和提供可行的专业意见。我们提出了 Beam Enumeration，将 exhaustively enumerate 最有可能的子序列从语言基于的分子生成模型，并证明分子结构可以被提取。当与循环学习搭配时，提取的分子结构会具有意义，提供解释性和提高样本效率的自我条件生成。Beam Enumeration 适用于任何语言基于的分子生成模型，并且进一步提高 Augmented Memory 算法的性能，该算法已经在 Practical Molecular Optimization 问题上实现新的顶峰性，具体是 Sample Efficiency。联合算法可以更快地生成更高的奖励分子， givent a fixed oracle budget。Beam Enumeration 是第一个同时解释性和样本效率的分子设计方法。Simplified Chinese:生成分子设计已经从证明阶段普及到实际应用阶段，这是最近几年的论文中 reporting 实验验证。关键的挑战是解释性和样本效率，这些挑战可以增强生成设计，直接优化昂贵的高精度观测器和提供可行的专业意见。我们提出了 Beam Enumeration，将 exhaustively enumerate 最有可能的子序列从语言基于的分子生成模型，并证明分子结构可以被提取。当与循环学习搭配时，提取的分子结构会具有意义，提供解释性和提高样本效率的自我条件生成。Beam Enumeration 适用于任何语言基于的分子生成模型，并且进一步提高 Augmented Memory 算法的性能。这种算法已经在 Practical Molecular Optimization 问题上实现新的顶峰性，具体是 Sample Efficiency。联合算法可以更快地生成更高的奖励分子， givent a fixed oracle budget。Beam Enumeration 是第一个同时解释性和样本效率的分子设计方法。
Deep Reinforcement Learning for the Heat Transfer Control of Pulsating Impinging Jets
methods: 研究使用了 vanilla Deep Q-Network（DQN）方法进行热控制，并对不同的 DRL 变体进行了比较。
results: 结果表明，soft Double 和 Duel DQN 在所有变体中表现最好，具有高效学习和动作优先级能力。soft Double DQN 超过 hard Double DQN。此外，soft Double 和 Duel 能够在控制周期内维持温度在所需的阈值内超过 98% 的时间。这些发现表明 DRL 在热控制系统中具有扎实的潜力。Abstract
This research study explores the applicability of Deep Reinforcement Learning (DRL) for thermal control based on Computational Fluid Dynamics. To accomplish that, the forced convection on a hot plate prone to a pulsating cooling jet with variable velocity has been investigated. We begin with evaluating the efficiency and viability of a vanilla Deep Q-Network (DQN) method for thermal control. Subsequently, a comprehensive comparison between different variants of DRL is conducted. Soft Double and Duel DQN achieved better thermal control performance among all the variants due to their efficient learning and action prioritization capabilities. Results demonstrate that the soft Double DQN outperforms the hard Double DQN. Moreover, soft Double and Duel can maintain the temperature in the desired threshold for more than 98% of the control cycle. These findings demonstrate the promising potential of DRL in effectively addressing thermal control systems.
摘要
这个研究项目探讨了深度强化学习（DRL）在计算流体动力学中的应用性。为了实现这一目标，我们 investigate了一种受到脉冲冷风的热板，其中冷风速度是可变的。我们首先评估了普通的深度Q网络（DQN）方法的效率和可行性。然后，我们进行了不同变体的DRL比较。 results indicate that soft Double DQN和Duel DQN在所有变体中表现最佳，它们具有高效学习和动作优先级能力。此外，soft Double DQN超过了hard Double DQN。此外，soft Double和Duel可以在控制ecycle中维持温度在所需的阈值上超过98%的时间。这些发现表明DRL在thermal控制系统中具有扎实的潜力。
Local and Global Trend Bayesian Exponential Smoothing Models
results: 比较其他竞赛算法和参照值，本研究在M3竞赛数据集上得到了最佳的结果，从而在Literature中实现了最佳单variate方法的result。Abstract
This paper describes a family of seasonal and non-seasonal time series models that can be viewed as generalisations of additive and multiplicative exponential smoothing models. Their development is motivated by fast-growing, volatile time series, and facilitated by state-of-the-art Bayesian fitting techniques. When applied to the M3 competition data set, they outperform the best algorithms in the competition as well as other benchmarks, thus achieving to the best of our knowledge the best results of univariate methods on this dataset in the literature.
摘要
Characterising User Transfer Amid Industrial Resource Variation: A Bayesian Nonparametric Approach
results: 实验结果表明，CLUSTER 模型能够准确预测用户传递响应资源变化，并且能够Quantify uncertainty for reliable decision-making。此外，CLUSTER 模型能够独立地函数于个人可 identificable 信息，保护用户隐私。Abstract
In a multitude of industrial fields, a key objective entails optimising resource management whilst satisfying user requirements. Resource management by industrial practitioners can result in a passive transfer of user loads across resource providers, a phenomenon whose accurate characterisation is both challenging and crucial. This research reveals the existence of user clusters, which capture macro-level user transfer patterns amid resource variation. We then propose CLUSTER, an interpretable hierarchical Bayesian nonparametric model capable of automating cluster identification, and thereby predicting user transfer in response to resource variation. Furthermore, CLUSTER facilitates uncertainty quantification for further reliable decision-making. Our method enables privacy protection by functioning independently of personally identifiable information. Experiments with simulated and real-world data from the communications industry reveal a pronounced alignment between prediction results and empirical observations across a spectrum of resource management scenarios. This research establishes a solid groundwork for advancing resource management strategy development.
摘要
在多种工业领域中，一个关键目标是优化资源管理，同时满足用户需求。但是资源管理的实践可能导致用户负载的投递式传输，这种现象的准确描述是非常困难和重要。这项研究发现用户群，这些群体捕捉了资源变化下的用户传输模式。我们提议CLUSTER，一种可解释性强的树状贝叶拟合模型，能够自动确定用户群和用户传输响应资源变化。此外，CLUSTER还可以对不确定性进行评估，以便更加可靠地做出决策。我们的方法可以保护用户隐私，不需要个人可识别信息。实验表明，CLUSTER在通信业中的仿真数据和实际数据上具有杰出的一致性，在资源管理方案的多种场景中具有广泛的应用前景。这项研究为资源管理策略的发展提供了坚实的基础。
results: 通过对多个 benchmark 进行了广泛的实验，本研究发现 POT 可以一直提高 GCL 的性能，并且可以作为一个可靠的插件使用。Abstract
Graph Contrastive Learning (GCL) has emerged as a popular training approach for learning node embeddings from augmented graphs without labels. Despite the key principle that maximizing the similarity between positive node pairs while minimizing it between negative node pairs is well established, some fundamental problems are still unclear. Considering the complex graph structure, are some nodes consistently well-trained and following this principle even with different graph augmentations? Or are there some nodes more likely to be untrained across graph augmentations and violate the principle? How to distinguish these nodes and further guide the training of GCL? To answer these questions, we first present experimental evidence showing that the training of GCL is indeed imbalanced across all nodes. To address this problem, we propose the metric "node compactness", which is the lower bound of how a node follows the GCL principle related to the range of augmentations. We further derive the form of node compactness theoretically through bound propagation, which can be integrated into binary cross-entropy as a regularization. To this end, we propose the PrOvable Training (POT) for GCL, which regularizes the training of GCL to encode node embeddings that follows the GCL principle better. Through extensive experiments on various benchmarks, POT consistently improves the existing GCL approaches, serving as a friendly plugin.
摘要
graph contrastive learning (GCL) 已经成为无标签学习节点嵌入的受欢迎训练方法。 despite the well-established key principle of maximizing the similarity between positive node pairs while minimizing it between negative node pairs, some fundamental problems are still unclear. considering the complex graph structure, are some nodes consistently well-trained and following this principle even with different graph augmentations? or are there some nodes more likely to be untrained across graph augmentations and violate the principle? how to distinguish these nodes and further guide the training of GCL?to answer these questions, we first present experimental evidence showing that the training of GCL is indeed imbalanced across all nodes. to address this problem, we propose the metric "node compactness", which is the lower bound of how a node follows the GCL principle related to the range of augmentations. we further derive the form of node compactness theoretically through bound propagation, which can be integrated into binary cross-entropy as a regularization. to this end, we propose the PrOvable Training (POT) for GCL, which regularizes the training of GCL to encode node embeddings that follows the GCL principle better. through extensive experiments on various benchmarks, POT consistently improves the existing GCL approaches, serving as a friendly plugin.
Evaluating Classification Systems Against Soft Labels with Fuzzy Precision and Recall
results: 研究发现，使用提议的计算方法可以准确地评估音 Event detection系统的性能，并且可以避免因数据二进制化而导致的错误解释。Abstract
Classification systems are normally trained by minimizing the cross-entropy between system outputs and reference labels, which makes the Kullback-Leibler divergence a natural choice for measuring how closely the system can follow the data. Precision and recall provide another perspective for measuring the performance of a classification system. Non-binary references can arise from various sources, and it is often beneficial to use the soft labels for training instead of the binarized data. However, the existing definitions for precision and recall require binary reference labels, and binarizing the data can cause erroneous interpretations. We present a novel method to calculate precision, recall and F-score without quantizing the data. The proposed metrics extend the well established metrics as the definitions coincide when used with binary labels. To understand the behavior of the metrics we show simple example cases and an evaluation of different sound event detection models trained on real data with soft labels.
摘要
<>translate "Classification systems are normally trained by minimizing the cross-entropy between system outputs and reference labels, which makes the Kullback-Leibler divergence a natural choice for measuring how closely the system can follow the data. Precision and recall provide another perspective for measuring the performance of a classification system. Non-binary references can arise from various sources, and it is often beneficial to use the soft labels for training instead of the binarized data. However, the existing definitions for precision and recall require binary reference labels, and binarizing the data can cause erroneous interpretations. We present a novel method to calculate precision, recall and F-score without quantizing the data. The proposed metrics extend the well established metrics as the definitions coincide when used with binary labels. To understand the behavior of the metrics we show simple example cases and an evaluation of different sound event detection models trained on real data with soft labels." into Simplified Chinese.翻译文本为Simplified Chinese：通常，分类系统通过最小化系统输出与参考标签之间的cross-entropy来训练，这使得庒啄-利卜征函数成为衡量系统如何准确地跟踪数据的自然选择。精度和回归提供了另一种视角来衡量分类系统的性能。非二进制参考可以从多种来源 arise，并且在训练时使用软标签可以是有利的。然而，现有的精度和回归定义都需要二进制参考标签，并且将数据二进制化可能会导致错误的解释。我们提出了一种新的方法来计算精度、回归和F-score，而无需将数据二进制化。我们的 metric 扩展了现有的 metric，因为在使用二进制标签时，定义协调。为了理解metric的行为，我们给出了简单的例子cases和使用实际数据和软标签训练不同的音Event检测模型的评估。
SAMN: A Sample Attention Memory Network Combining SVM and NN in One Architecture
for: This paper aims to combine Support Vector Machines (SVM) and Neural Networks (NN) to create a more powerful function for multi-classification tasks.
methods: The proposed method, called Sample Attention Memory Network (SAMN), incorporates a sample attention module, class prototypes, and a memory block into NN to effectively combine SVM and NN.
results: Extensive experiments show that SAMN achieves better classification performance than single SVM or single NN with similar parameter sizes, as well as the previous best model for combining SVM and NN.Here’s the same information in Simplified Chinese:
methods: 提议的方法是将样本注意力模块、类型评估模块和记忆块 incorporated into NN，以实现SVM和NN的有效结合。
results: 广泛的实验表明，SAMN比单个 SVM 或单个 NN 相同参数大小下的性能更好，以及之前最佳结合 SVM 和 NN 的模型。Abstract
Support vector machine (SVM) and neural networks (NN) have strong complementarity. SVM focuses on the inner operation among samples while NN focuses on the operation among the features within samples. Thus, it is promising and attractive to combine SVM and NN, as it may provide a more powerful function than SVM or NN alone. However, current work on combining them lacks true integration. To address this, we propose a sample attention memory network (SAMN) that effectively combines SVM and NN by incorporating sample attention module, class prototypes, and memory block to NN. SVM can be viewed as a sample attention machine. It allows us to add a sample attention module to NN to implement the main function of SVM. Class prototypes are representatives of all classes, which can be viewed as alternatives to support vectors. The memory block is used for the storage and update of class prototypes. Class prototypes and memory block effectively reduce the computational cost of sample attention and make SAMN suitable for multi-classification tasks. Extensive experiments show that SAMN achieves better classification performance than single SVM or single NN with similar parameter sizes, as well as the previous best model for combining SVM and NN. The sample attention mechanism is a flexible module that can be easily deepened and incorporated into neural networks that require it.
摘要
支持向量机(SVM)和神经网络(NN)具有强大的补做性。SVM关注样本之间的内部运算，而NN关注样本中特征之间的运算。因此，将SVM和NN结合起来可能提供一个更强大的函数，而不需要增加参数量。然而，现有的SVM和NN结合方法缺乏真正的集成。为此，我们提议一种叫做样本注意力储存网络(SAMN)，它有效地结合了SVM和NN。SVM可以看作是一种样本注意机器。它允许我们将样本注意模块添加到NN中，以实现SVM的主要功能。类型范例是所有类型的代表，它们可以看作是支持向量的替代品。储存块用于存储和更新类型范例。类型范例和储存块可以有效减少样本注意的计算成本，使SAMN适用于多类分类任务。广泛的实验表明，SAMN在类比单独使用SVM或NN时，达到了更好的分类性能，同时也比前一个最佳结合SVM和NN的模型更好。样本注意机制是一种灵活的模块，可以轻松地深入 incorporated into neural networks 中，当需要时。
results: 在模拟和实际数据上，BPLS方法在面临过拟合和confirmation bias问题时表现出优于传统的 PLS 方法。此外，这个方法还可以使得 PLS 更加鲁棒地对待模型假设。Abstract
Pseudo-Labeling is a simple and effective approach to semi-supervised learning. It requires criteria that guide the selection of pseudo-labeled data. The latter have been shown to crucially affect pseudo-labeling's generalization performance. Several such criteria exist and were proven to work reasonably well in practice. However, their performance often depends on the initial model fit on labeled data. Early overfitting can be propagated to the final model by choosing instances with overconfident but wrong predictions, often called confirmation bias. In two recent works, we demonstrate that pseudo-label selection (PLS) can be naturally embedded into decision theory. This paves the way for BPLS, a Bayesian framework for PLS that mitigates the issue of confirmation bias. At its heart is a novel selection criterion: an analytical approximation of the posterior predictive of pseudo-samples and labeled data. We derive this selection criterion by proving Bayes-optimality of this "pseudo posterior predictive". We empirically assess BPLS for generalized linear, non-parametric generalized additive models and Bayesian neural networks on simulated and real-world data. When faced with data prone to overfitting and thus a high chance of confirmation bias, BPLS outperforms traditional PLS methods. The decision-theoretic embedding further allows us to render PLS more robust towards the involved modeling assumptions. To achieve this goal, we introduce a multi-objective utility function. We demonstrate that the latter can be constructed to account for different sources of uncertainty and explore three examples: model selection, accumulation of errors and covariate shift.
摘要
假标注是一种简单而有效的 semi-supervised learning 方法。它需要一些导向选择假标注数据的标准。这些标准有助于减少假标注的泛化性能。然而，它们的性能通常取决于初始模型适应 labels 数据。早期过度适应可能会传递到最终模型，通常被称为 confirmation bias。在两篇最近的论文中，我们展示了 pseudo-label 选择（PLS）可以自然地被嵌入到决策理论中。这种方法可以减轻 confirmation bias 的问题。PLS 的核心是一个新的选择标准：一种 Analytical 预测 pseudo-samples 和标注数据的 posterior 预测。我们 derive 这个选择标准通过证明 Bayes-优化这个 "pseudo posterior predictive"。我们在 simulated 和实际数据上进行了 Empirical 评估，发现在面临数据泛化和高度可能性 confirmation bias 的情况下，BPLS 比传统的 PLS 方法更好。决策理论的嵌入还使得 PLS 更加抗性待命模型假设。为了实现这一目标，我们引入了一个多目标价值函数。我们示例了这个价值函数可以考虑不同的不确定性来源，并 explore 三个例子：模型选择、积累错误和变量转移。
Sample Complexity of Neural Policy Mirror Descent for Policy Optimization on Low-Dimensional Manifolds
results: 研究发现，NPMD算法可以在高维策略优化问题中减轻维度着色问题，并可以在有限样本下找到$\epsilon$-优的策略，其样本复杂性为$\widetilde{O}(\epsilon^{-{\frac{d}{\alpha}-2})$。Abstract
Policy-based algorithms equipped with deep neural networks have achieved great success in solving high-dimensional policy optimization problems in reinforcement learning. However, current analyses cannot explain why they are resistant to the curse of dimensionality. In this work, we study the sample complexity of the neural policy mirror descent (NPMD) algorithm with convolutional neural networks (CNN) as function approximators. Motivated by the empirical observation that many high-dimensional environments have state spaces possessing low-dimensional structures, such as those taking images as states, we consider the state space to be a $d$-dimensional manifold embedded in the $D$-dimensional Euclidean space with intrinsic dimension $d\ll D$. We show that in each iteration of NPMD, both the value function and the policy can be well approximated by CNNs. The approximation errors are controlled by the size of the networks, and the smoothness of the previous networks can be inherited. As a result, by properly choosing the network size and hyperparameters, NPMD can find an $\epsilon$-optimal policy with $\widetilde{O}(\epsilon^{-\frac{d}{\alpha}-2})$ samples in expectation, where $\alpha\in(0,1]$ indicates the smoothness of environment. Compared to previous work, our result exhibits that NPMD can leverage the low-dimensional structure of state space to escape from the curse of dimensionality, providing an explanation for the efficacy of deep policy-based algorithms.
摘要
Matrix Factorization in Tropical and Mixed Tropical-Linear Algebras
results: 我们提出了一个改进的TMF算法，可以避免许多地方最佳解；此外，我们还提出了一个新的组合矩阵分解方法，具有与多个用户学习 utility 函数的 interessante解释。我们还呈现了一些数据，证明我们的方法的有效性，并实现了一个推荐系统的应用，获得了显著的结果。Abstract
Matrix Factorization (MF) has found numerous applications in Machine Learning and Data Mining, including collaborative filtering recommendation systems, dimensionality reduction, data visualization, and community detection. Motivated by the recent successes of tropical algebra and geometry in machine learning, we investigate two problems involving matrix factorization over the tropical algebra. For the first problem, Tropical Matrix Factorization (TMF), which has been studied already in the literature, we propose an improved algorithm that avoids many of the local optima. The second formulation considers the approximate decomposition of a given matrix into the product of three matrices where a usual matrix product is followed by a tropical product. This formulation has a very interesting interpretation in terms of the learning of the utility functions of multiple users. We also present numerical results illustrating the effectiveness of the proposed algorithms, as well as an application to recommendation systems with promising results.
摘要
矩阵因式（MF）在机器学习和数据挖掘中找到了许多应用，包括共享推荐系统、维度减少、数据可视化和社区检测。受推荐系统的最近成功而受欢迎的泛洋算术和几何，我们 investigate two 矩阵因式问题，其中一个是已经在文献中研究的极地矩阵因式（TMF），我们提出了一种改进的算法，可以避免许多地方最佳点。第二个形式是对给定矩阵的approximate decompositions into the product of three matrices，其中一个是 usual matrix product followed by a tropical product。这个形式有非常有趣的学习多个用户的实用函数的解释。我们还present numerical results demonstrating the effectiveness of the proposed algorithms, as well as an application to recommendation systems with promising results。
Follow-ups Also Matter: Improving Contextual Bandits via Post-serving Contexts
for: 提高在线学习效率，解决 contectual bandit problem 中 valuable 的后服务上下文信息不可见的问题。
methods: 提出了一种新的contextual bandit problem 模型，利用后服务上下文信息进行学习，并设计了一种新的算法poLinUCB，可以在标准假设下实现紧凑的征逐 regret。
results: 对 synthetic 和实际数据进行了广泛的实验测试，证明了利用后服务上下文信息可以提高学习效率，以及poLinUCB 算法的综合性和可靠性。Abstract
Standard contextual bandit problem assumes that all the relevant contexts are observed before the algorithm chooses an arm. This modeling paradigm, while useful, often falls short when dealing with problems in which valuable additional context can be observed after arm selection. For example, content recommendation platforms like Youtube, Instagram, Tiktok also observe valuable follow-up information pertinent to the user's reward after recommendation (e.g., how long the user stayed, what is the user's watch speed, etc.). To improve online learning efficiency in these applications, we study a novel contextual bandit problem with post-serving contexts and design a new algorithm, poLinUCB, that achieves tight regret under standard assumptions. Core to our technical proof is a robustified and generalized version of the well-known Elliptical Potential Lemma (EPL), which can accommodate noise in data. Such robustification is necessary for tackling our problem, and we believe it could also be of general interest. Extensive empirical tests on both synthetic and real-world datasets demonstrate the significant benefit of utilizing post-serving contexts as well as the superior performance of our algorithm over the state-of-the-art approaches.
摘要
通常的上下文抽剂问题假设所有相关的上下文都可以在算法选择器之前观察。这种模型虽然有用，但在处理不可见上下文的问题时，它经常失足。例如，内容推荐平台 like Youtube、Instagram 和 Tiktok 可以在推荐后观察有价值的用户奖励信息（例如用户停留时间、用户播放速度等）。为了在这些应用中提高在线学习效率，我们研究了一种新的上下文抽剂问题，即 post-serving 上下文，并设计了一个新的算法 poLinUCB。我们的技术证明基于一种更加稳健和泛化的 Elliptical Potential Lemma (EPL)，可以承受数据噪声。这种稳健性是我们问题的必要条件，并且我们认为这也可能对总体有利。我们的实验表明，利用 post-serving 上下文和我们的算法的优秀性，可以在实验室和实际数据上达到显著的改善。
Graph Representation Learning Towards Patents Network Analysis
results: 研究结果显示，通过使用Graph representation learning和文本挖掘技术，可以实现专利资料分析和探索新领域，并且可以预防重复申请专利、熟悉相似和相连的发明、了解法律实体支持专利和研究人员在特定领域的知识。Abstract
Patent analysis has recently been recognized as a powerful technique for large companies worldwide to lend them insight into the age of competition among various industries. This technique is considered a shortcut for developing countries since it can significantly accelerate their technology development. Therefore, as an inevitable process, patent analysis can be utilized to monitor rival companies and diverse industries. This research employed a graph representation learning approach to create, analyze, and find similarities in the patent data registered in the Iranian Official Gazette. The patent records were scrapped and wrangled through the Iranian Official Gazette portal. Afterward, the key entities were extracted from the scrapped patents dataset to create the Iranian patents graph from scratch based on novel natural language processing and entity resolution techniques. Finally, thanks to the utilization of novel graph algorithms and text mining methods, we identified new areas of industry and research from Iranian patent data, which can be used extensively to prevent duplicate patents, familiarity with similar and connected inventions, Awareness of legal entities supporting patents and knowledge of researchers and linked stakeholders in a particular research field.
摘要
具有广泛应用前景的专利分析技术已经在全球范围内被大型公司广泛应用，以帮助这些公司更好地了解不同行业的竞争情况。这种技术被视为发展中国家的短cut，因为它可以快速加速技术发展。因此，通过监测竞争对手和多个行业的专利分析，这种技术可以帮助公司更好地了解自己的市场环境。本研究采用了图表学习方法来创建、分析和找出专利数据库中的相似性。我们从伊朗官方报纸网站上抓取了专利笔记，然后使用新的自然语言处理和实体解决技术提取了关键实体。最后，我们通过使用新的图算法和文本挖掘技术，在伊朗专利数据中找到了新的行业和研究领域，这些领域可以用于避免重复专利、熟悉相似和相连的发明、了解法定机构支持专利、了解研究人员和相关的投资者在特定研究领域的知识。
Can Class-Priors Help Single-Positive Multi-Label Learning?
paper_authors: Biao Liu, Jie Wang, Ning Xu, Xin Geng
for: solves the problem of single-positive multi-label learning (SPMLL) with class-prior differences in real-world scenarios.
methods: proposes a novel framework called Class-pRiors Induced Single-Positive multi-label learning, which includes a class-priors estimator and an unbiased risk estimator for classification.
results: experiments on ten MLL benchmark datasets demonstrate the effectiveness and superiority of the proposed method over existing SPMLL approaches.Abstract
Single-positive multi-label learning (SPMLL) is a typical weakly supervised multi-label learning problem, where each training example is annotated with only one positive label. Existing SPMLL methods typically assign pseudo-labels to unannotated labels with the assumption that prior probabilities of all classes are identical. However, the class-prior of each category may differ significantly in real-world scenarios, which makes the predictive model not perform as well as expected due to the unrealistic assumption on real-world application. To alleviate this issue, a novel framework named {\proposed}, i.e., Class-pRiors Induced Single-Positive multi-label learning, is proposed. Specifically, a class-priors estimator is introduced, which could estimate the class-priors that are theoretically guaranteed to converge to the ground-truth class-priors. In addition, based on the estimated class-priors, an unbiased risk estimator for classification is derived, and the corresponding risk minimizer could be guaranteed to approximately converge to the optimal risk minimizer on fully supervised data. Experimental results on ten MLL benchmark datasets demonstrate the effectiveness and superiority of our method over existing SPMLL approaches.
摘要
单正向多标签学习（SPMLL）是一种 Typical weakly supervised multi-label learning 问题，每个训练示例只有一个正确标签。现有的 SPMLL 方法通常将 pseudo-labels 赋给未标记的标签，假设所有类别的先验概率相同。然而，实际应用中每个类别的类别先验可能很大不同，这会使 predictive 模型不能如预期那样表现，因为这是不真实的假设。为解决这个问题，我们提出了一个新的框架，即 Class-pRiors Induced Single-Positive multi-label learning，简称为 \proposed。 Specifically, 我们引入了一个类别先验估计器，可以估算类别先验，并且这些估计器 theoretically guaranteed to converge to the ground-truth class-priors。此外，基于估计器，我们 derive 了一个不偏的风险估计器 для 分类，并且可以 guarantee that the corresponding risk minimizer could approximately converge to the optimal risk minimizer on fully supervised data。实验结果表明，我们的方法在十个 MLL benchmark 数据集上表现出色，比现有的 SPMLL 方法更有效。
Estimating Treatment Effects Under Heterogeneous Interference
for: The paper is written for estimating individual treatment effects (ITEs) in the presence of interference, specifically in online applications where units are associated and interference can be heterogeneous.
methods: The paper proposes a novel approach to model heterogeneous interference by developing a new architecture that aggregates information from diverse neighbors, using graph neural networks, a mechanism to aggregate information from different views, and attention mechanisms.
results: The proposed method significantly outperforms existing methods for ITE estimation in experiments on multiple datasets with heterogeneous interference, confirming the importance of modeling heterogeneous interference.Abstract
Treatment effect estimation can assist in effective decision-making in e-commerce, medicine, and education. One popular application of this estimation lies in the prediction of the impact of a treatment (e.g., a promotion) on an outcome (e.g., sales) of a particular unit (e.g., an item), known as the individual treatment effect (ITE). In many online applications, the outcome of a unit can be affected by the treatments of other units, as units are often associated, which is referred to as interference. For example, on an online shopping website, sales of an item will be influenced by an advertisement of its co-purchased item. Prior studies have attempted to model interference to estimate the ITE accurately, but they often assume a homogeneous interference, i.e., relationships between units only have a single view. However, in real-world applications, interference may be heterogeneous, with multi-view relationships. For instance, the sale of an item is usually affected by the treatment of its co-purchased and co-viewed items. We hypothesize that ITE estimation will be inaccurate if this heterogeneous interference is not properly modeled. Therefore, we propose a novel approach to model heterogeneous interference by developing a new architecture to aggregate information from diverse neighbors. Our proposed method contains graph neural networks that aggregate same-view information, a mechanism that aggregates information from different views, and attention mechanisms. In our experiments on multiple datasets with heterogeneous interference, the proposed method significantly outperforms existing methods for ITE estimation, confirming the importance of modeling heterogeneous interference.
摘要
干预效果估计可以帮助在电商、医疗和教育等领域进行有效的决策。一种受欢迎的应用之一是估计干预（例如推广）对单元（例如商品）的结果（例如销售）的影响，known as 个体干预效果（ITE）。在许多在线应用程序中，单元的结果可能受到其他单元的干预，这是因为单元经常相关，这被称为干扰。例如，在一个在线购物网站上，一个商品的销售会受到其推广的影响。先前的研究尝试了模型干扰，以便准确地估计ITE，但它们通常假设了同质干扰，即单元之间只有一种视角。然而，在实际应用中，干扰可能是多质，即单元之间有多种视角。例如，一个商品的销售通常受到其推广和浏览的影响。我们认为，如果不正确地模型多质干扰，ITE估计就会不准确。因此，我们提出了一种新的方法，用于模型多质干扰。我们的提议方法包括图 neural networks 来聚合同视角信息，一种机制来聚合不同视角信息，以及注意机制。在我们对多个数据集上进行的实验中，我们的提议方法在存在多质干扰的情况下显著超过了现有的ITE估计方法，确认了模型多质干扰的重要性。
Diffusion Conditional Expectation Model for Efficient and Robust Target Speech Extraction
methods: 本文使用了Diffusion Conditional Expectation Model (DCEM)，可以处理多 speaker和单 speaker场景，并且可以在各种噪音和清晰Condition下进行处理。
results: comparing with传统方法，本文的方法在非侵入和侵入 metric 中表现出色，并且具有高效的推理速度和对未看到任务的稳定性。Audio例子可以在线上预览（https://vivian556123.github.io/dcem）。Abstract
Target Speech Extraction (TSE) is a crucial task in speech processing that focuses on isolating the clean speech of a specific speaker from complex mixtures. While discriminative methods are commonly used for TSE, they can introduce distortion in terms of speech perception quality. On the other hand, generative approaches, particularly diffusion-based methods, can enhance speech quality perceptually but suffer from slower inference speed. We propose an efficient generative approach named Diffusion Conditional Expectation Model (DCEM) for TSE. It can handle multi- and single-speaker scenarios in both noisy and clean conditions. Additionally, we introduce Regenerate-DCEM (R-DCEM) that can regenerate and optimize speech quality based on pre-processed speech from a discriminative model. Our method outperforms conventional methods in terms of both intrusive and non-intrusive metrics and demonstrates notable strengths in inference efficiency and robustness to unseen tasks. Audio examples are available online (https://vivian556123.github.io/dcem).
摘要
target speech extraction (tse)是speech processing中关键的任务，它的目标是从复杂的混合中分离出清晰的speaker的speech。although discriminative methods are commonly used for tse, they can introduce distortion in terms of speech perception quality. on the other hand, generative approaches, particularly diffusion-based methods, can enhance speech quality perceptually but suffer from slower inference speed. we propose an efficient generative approach named diffusion conditional expectation model (dcem) for tse. it can handle multi- and single-speaker scenarios in both noisy and clean conditions. additionally, we introduce regenerate-dcem (r-dcem) that can regenerate and optimize speech quality based on pre-processed speech from a discriminative model. our method outperforms conventional methods in terms of both intrusive and non-intrusive metrics and demonstrates notable strengths in inference efficiency and robustness to unseen tasks. audio examples are available online (https://vivian556123.github.io/dcem).Here's the translation breakdown:* target speech extraction (tse) = 目标语音采样 (tse)* speech processing = 语音处理* discriminative methods = 分类方法* generative approaches = 生成方法* diffusion-based methods = 扩散基于方法* Diffusion Conditional Expectation Model (DCEM) = 扩散 conditional expectation model (DCEM)* Regenerate-DCEM (R-DCEM) = 重新生成-DCEM (R-DCEM)* pre-processed speech = 预处理的语音* inference efficiency = 推理效率* robustness to unseen tasks = 对未经见任务的Robustness* audio examples = 音频示例Please note that the translation is done in Simplified Chinese, which is the standard form of Chinese used in mainland China and Singapore. If you need the translation in Traditional Chinese, please let me know.
Statistical Perspective of Top-K Sparse Softmax Gating Mixture of Experts
methods: 作者使用了 Gaussian mixture of experts 来设计一个简单的模型，并通过定义新的损失函数来捕捉输入空间不同区域的行为。
results: 研究发现，当知道真实的专家数 $k_{\ast}$ 时，随着样本大小增加，权重补做的散度和参数估计的速度都是 Parametric。但是，当真实模型超出了 $k_{\ast}$ 的情况下，选择从顶层-$K$ sparse softmax权重函数中的专家数量必须大于某些 Voronoi 细胞与真实参数之间的总体 cardinality，以确保权重补做的散度估计 converge。此外，虽然散度估计的速度仍然是 Parametric，但参数估计速度受到权重补做和专家函数之间的内在交互的影响，导致很慢。Abstract
Top-K sparse softmax gating mixture of experts has been widely used for scaling up massive deep-learning architectures without increasing the computational cost. Despite its popularity in real-world applications, the theoretical understanding of that gating function has remained an open problem. The main challenge comes from the structure of the top-K sparse softmax gating function, which partitions the input space into multiple regions with distinct behaviors. By focusing on a Gaussian mixture of experts, we establish theoretical results on the effects of the top-K sparse softmax gating function on both density and parameter estimations. Our results hinge upon defining novel loss functions among parameters to capture different behaviors of the input regions. When the true number of experts $k_{\ast}$ is known, we demonstrate that the convergence rates of density and parameter estimations are both parametric on the sample size. However, when $k_{\ast}$ becomes unknown and the true model is over-specified by a Gaussian mixture of $k$ experts where $k > k_{\ast}$, our findings suggest that the number of experts selected from the top-K sparse softmax gating function must exceed the total cardinality of a certain number of Voronoi cells associated with the true parameters to guarantee the convergence of the density estimation. Moreover, while the density estimation rate remains parametric under this setting, the parameter estimation rates become substantially slow due to an intrinsic interaction between the softmax gating and expert functions.
摘要
Top-K 稀疏软max权重混合专家已经广泛应用于扩大深度学习架构而无需增加计算成本。尽管在实际应用中它非常受欢迎，但是其理论理解仍然是一个开放的问题。主要挑战在于 top-K 稀疏软max权重混合函数的结构，该函数将输入空间分成多个区域，每个区域具有不同的行为。通过关注 Gaussian mixture of experts，我们建立了关于 top-K 稀疏软max权重混合函数对输入空间的影响的理论结果。我们的结论基于定义新的损失函数来捕捉不同区域的输入行为。当真实的专家数量 $k_{\ast}$ 知道时，我们证明随样本大小的散度和参数估计的渐近率都是参数的。然而，当 $k_{\ast}$ 不知道，真实的模型被过度规定为 Gaussian mixture of $k$ 专家，其中 $k > k_{\ast}$，我们发现，为保证散度估计的渐近，从 top-K 稀疏软max权重混合函数中选择的专家数量必须大于真实参数的总 cardinality。此外，虽然散度估计率保持参数的，但参数估计率因软max权重和专家函数之间的内在互动而变得非常慢。
On the Effectiveness of Adversarial Samples against Ensemble Learning-based Windows PE Malware Detectors
paper_authors: Trong-Nghia To, Danh Le Kim, Do Thi Thu Hien, Nghi Hoang Khoa, Hien Do Hoang, Phan The Duy, Van-Hau Pham for:这个研究的目的是提出一个组合GAN和RL模型的变异系统，以抵消基于集成学习的检测器。methods:研究使用了GAN和RL模型，包括MalGAN和Deep Q-network anti-malware Engines Attacking Framework (DQEAF)。results:实验结果显示，100%的选择的异常样品保留了可执行档案的格式，而在执行可能性和黑客性方面也有一定的成功。Abstract
Recently, there has been a growing focus and interest in applying machine learning (ML) to the field of cybersecurity, particularly in malware detection and prevention. Several research works on malware analysis have been proposed, offering promising results for both academic and practical applications. In these works, the use of Generative Adversarial Networks (GANs) or Reinforcement Learning (RL) can aid malware creators in crafting metamorphic malware that evades antivirus software. In this study, we propose a mutation system to counteract ensemble learning-based detectors by combining GANs and an RL model, overcoming the limitations of the MalGAN model. Our proposed FeaGAN model is built based on MalGAN by incorporating an RL model called the Deep Q-network anti-malware Engines Attacking Framework (DQEAF). The RL model addresses three key challenges in performing adversarial attacks on Windows Portable Executable malware, including format preservation, executability preservation, and maliciousness preservation. In the FeaGAN model, ensemble learning is utilized to enhance the malware detector's evasion ability, with the generated adversarial patterns. The experimental results demonstrate that 100\% of the selected mutant samples preserve the format of executable files, while certain successes in both executability preservation and maliciousness preservation are achieved, reaching a stable success rate.
摘要
近些年来，机器学习（ML）在计算机安全领域的应用得到了越来越多的关注和兴趣，特别是在恶意软件检测和防范方面。一些关于恶意软件分析的研究工作已经提出，其中使用生成对抗网络（GANs）或强化学习（RL）可以帮助恶意软件创作者制作形态变化的恶意软件，从而躲避反恶意软件检测。在本研究中，我们提出了一种突变系统，用于对集成学习基于检测器的攻击者进行对抗。我们的提出的FeaGAN模型基于MalGAN模型，通过加入一个RL模型called Deep Q-network anti-malware Engines Attacking Framework（DQEAF），解决了对Windows Portable Executable恶意软件的三大挑战，包括格式保留、可执行性保留和害意保留。在FeaGAN模型中， ensemble learning被使用来增强恶意软件检测器的逃脱能力，通过生成的对抗模式。实验结果表明，100%的选择的突变样本保留了可执行文件的格式，而在可执行性和害意方面也有一定的成功，达到了稳定的成功率。
Penalized Principal Component Analysis using Nesterov Smoothing
results: 使用1000个基因计划数据集，我们实验ally示出了使用我们提议的精炼PEP可以提高数值稳定性并获得有意义的特征向量。我们还 investigate了对传统PCA的约束最小化方法的比较。Abstract
Principal components computed via PCA (principal component analysis) are traditionally used to reduce dimensionality in genomic data or to correct for population stratification. In this paper, we explore the penalized eigenvalue problem (PEP) which reformulates the computation of the first eigenvector as an optimization problem and adds an L1 penalty constraint. The contribution of our article is threefold. First, we extend PEP by applying Nesterov smoothing to the original LASSO-type L1 penalty. This allows one to compute analytical gradients which enable faster and more efficient minimization of the objective function associated with the optimization problem. Second, we demonstrate how higher order eigenvectors can be calculated with PEP using established results from singular value decomposition (SVD). Third, using data from the 1000 Genome Project dataset, we empirically demonstrate that our proposed smoothed PEP allows one to increase numerical stability and obtain meaningful eigenvectors. We further investigate the utility of the penalized eigenvector approach over traditional PCA.
摘要
<>使用主成分分析（PCA）计算主成分是传统地用于降维度的 genomic 数据或是对人口分布进行修正。在这篇论文中，我们探讨了增加 penalty 的 eigenvalue 问题（PEP），它将计算第一个 eigenvector 转换为优化问题，并添加 L1 罚项限制。我们的贡献有三个方面：首先，我们扩展了 PEP 方法，通过应用 Nesterov 缓和法来计算 analytical 导数，从而更快地和更有效地解决优化问题中关联的目标函数。第二，我们使用已有的 singular value decomposition（SVD）结果来计算高级别的 eigenvectors。第三，使用 1000 Genome Project 数据集，我们实际示出了我们提议的平滑 PEP 可以增加数值稳定性并获得有意义的 eigenvectors。我们进一步调查了使用增加 penalty 的 eigenvector 方法与传统 PCA 方法的优劣。
Backorder Prediction in Inventory Management: Classification Techniques and Cost Considerations
results: 结果表明，结合模型方法，包括集成技术和VAE，可以有效地处理不均衡数据集，提高预测精度，减少假阳性和假阴性，并增加可 interpretability。Abstract
This article introduces an advanced analytical approach for predicting backorders in inventory management. Backorder refers to an order that cannot be immediately fulfilled due to stock depletion. Multiple classification techniques, including Balanced Bagging Classifiers, Fuzzy Logic, Variational Autoencoder - Generative Adversarial Networks, and Multi-layer Perceptron classifiers, are assessed in this work using performance evaluation metrics such as ROC-AUC and PR-AUC. Moreover, this work incorporates a profit function and misclassification costs, considering the financial implications and costs associated with inventory management and backorder handling. The study suggests that a combination of modeling approaches, including ensemble techniques and VAE, can effectively address imbalanced datasets in inventory management, emphasizing interpretability and reducing false positives and false negatives. This research contributes to the advancement of predictive analytics and offers valuable insights for future investigations in backorder forecasting and inventory control optimization for decision-making.
摘要
Translated into Simplified Chinese:这篇文章介绍了一种高级分析方法，用于预测库存欠货（backorder）管理中的库存异常情况。这种方法包括多种分类技术，如均衡搅拌分类器、杂化逻辑、变量自动编码-生成敌对网络、多层感知器等，并使用表现评估指标，如ROC-AUC和PR-AUC来评估其表现。此外，这种研究还考虑了财务因素和误分类成本，包括库存管理和欠货处理中的财务影响和成本。研究表明，结合不同的模型方法，包括ensemble技术和VAE，可以有效地处理库存管理中的偏度数据，提高预测精度和减少false阳和false降。这项研究对预测分析领域的发展做出了贡献，并为未来的库存预测和库存控制优化做出了有价值的着想。
results: 论文的评估结果显示，NSOTree 能够在实际数据上实现高性能和可解性，并且在存生领域中提供了一个可靠的方法。Abstract
Survival analysis is a statistical method employed to scrutinize the duration until a specific event of interest transpires, known as time-to-event information characterized by censorship. Recently, deep learning-based methods have dominated this field due to their representational capacity and state-of-the-art performance. However, the black-box nature of the deep neural network hinders its interpretability, which is desired in real-world survival applications but has been largely neglected by previous works. In contrast, conventional tree-based methods are advantageous with respect to interpretability, while consistently grappling with an inability to approximate the global optima due to greedy expansion. In this paper, we leverage the strengths of both neural networks and tree-based methods, capitalizing on their ability to approximate intricate functions while maintaining interpretability. To this end, we propose a Neural Survival Oblique Tree (NSOTree) for survival analysis. Specifically, the NSOTree was derived from the ReLU network and can be easily incorporated into existing survival models in a plug-and-play fashion. Evaluations on both simulated and real survival datasets demonstrated the effectiveness of the proposed method in terms of performance and interpretability.
摘要
生存分析是一种统计方法，用于检查一个特定事件的发生时间，称为时间至事件信息，受到限制。最近，深度学习基于方法在这一领域占据主导地位，因为它们具有表达能力和现代性。然而，深度神经网络的黑盒特性阻碍了其解释性，这在实际生存应用中是极其重要的，但之前的工作却忽略了这一点。相比之下，传统的树状方法具有解释性的优势，但它们难以近似全局最优解。在这篇论文中，我们利用神经网络和树状方法的优点，同时维持解释性。为此，我们提出了神经生存斜树（NSOTree）方法。具体来说，NSOTree是基于ReLU网络的，可以轻松地与现有的生存模型集成。我们对实际和 simulations 数据进行了评估，并证明了我们提出的方法在性能和解释性两个方面具有效果。
Forecasting large collections of time series: feature-based methods
results: 论文详细介绍了现场 откры源软件实现的状态 искусственного预测方法，包括基于时间序列特征的模型选择和模型组合。Abstract
In economics and many other forecasting domains, the real world problems are too complex for a single model that assumes a specific data generation process. The forecasting performance of different methods changes depending on the nature of the time series. When forecasting large collections of time series, two lines of approaches have been developed using time series features, namely feature-based model selection and feature-based model combination. This chapter discusses the state-of-the-art feature-based methods, with reference to open-source software implementations.
摘要
在经济和许多其他预测领域中，现实世界问题太复杂，不可以单独采用一个模型，假设特定的数据生成过程。预测不同时序系列的表现，不同方法的预测性能会有所不同。当预测大量时序系列时，有两条方向的方法得到发展，一是基于时序特征的模型选择，二是基于时序特征的模型组合。本章介绍了当前最佳实践的特征基于方法，参考开源软件实现。
Projected Randomized Smoothing for Certified Adversarial Robustness
results: 对 CIFAR-10 和 SVHN 进行实验，表明我们的方法可以提供 tractable 的下界，并且在证明区域内捕捉到普通扰动的perturbationsAbstract
Randomized smoothing is the current state-of-the-art method for producing provably robust classifiers. While randomized smoothing typically yields robust $\ell_2$-ball certificates, recent research has generalized provable robustness to different norm balls as well as anisotropic regions. This work considers a classifier architecture that first projects onto a low-dimensional approximation of the data manifold and then applies a standard classifier. By performing randomized smoothing in the low-dimensional projected space, we characterize the certified region of our smoothed composite classifier back in the high-dimensional input space and prove a tractable lower bound on its volume. We show experimentally on CIFAR-10 and SVHN that classifiers without the initial projection are vulnerable to perturbations that are normal to the data manifold and yet are captured by the certified regions of our method. We compare the volume of our certified regions against various baselines and show that our method improves on the state-of-the-art by many orders of magnitude.
摘要
随机缓和是当前状态艺术方法，生成可证明抗干扰的分类器。通常情况下，随机缓和会生成 $\ell_2$-球证明，但最近的研究已经推广了不同 norm 球证明以及不规则区域。这个工作考虑一种分类器架构，首先将数据投影到低维度的数据投影空间，然后应用标准分类器。通过在低维度投影空间中进行随机缓和，我们Characterize了我们熔化 composite 分类器的证明区域，并证明了可读取的下界。我们在 CIFAR-10 和 SVHN 上进行实验，并证明了不包含初始投影的分类器容易受到数据投影方向的攻击，但是我们的方法可以捕捉这些攻击。我们比较了我们的证明区域的体积与各种基准值，并证明了我们的方法在状态艺术中提高了多个阶段的质量。
ReMasker: Imputing Tabular Data with Masked Autoencoding
results: 与优秀的方法进行比较，我们在多种缺失设定下进行了广泛的评估，并显示了 ReMasker 在缺失率不同的情况下的性能都在或超过了现有方法，而且其性能优势通常随缺失数据的比例增长。此外，我们还进行了理论准确性的探讨，并证明 ReMasker 通常学习缺失数据不变的表示。Abstract
We present ReMasker, a new method of imputing missing values in tabular data by extending the masked autoencoding framework. Compared with prior work, ReMasker is both simple -- besides the missing values (i.e., naturally masked), we randomly ``re-mask'' another set of values, optimize the autoencoder by reconstructing this re-masked set, and apply the trained model to predict the missing values; and effective -- with extensive evaluation on benchmark datasets, we show that ReMasker performs on par with or outperforms state-of-the-art methods in terms of both imputation fidelity and utility under various missingness settings, while its performance advantage often increases with the ratio of missing data. We further explore theoretical justification for its effectiveness, showing that ReMasker tends to learn missingness-invariant representations of tabular data. Our findings indicate that masked modeling represents a promising direction for further research on tabular data imputation. The code is publicly available.
摘要
我们提出了一种新的方法 called ReMasker，用于填充缺失数据的表格数据中的缺失值。与之前的工作相比，ReMasker 更简单，只有缺失值（即自然缺失）之外，我们随机“重新覆盖”另一个集合的值，然后优化自动编码器，重建这个重新覆盖集合，并使用训练模型预测缺失值。与此同时，我们还进行了广泛的评估，发现 ReMasker 在不同的缺失设定下，与或超过现有方法的稳定性和实用性。此外，我们还进行了理论 justify 其效果，发现 ReMasker 倾向于学习缺失性 invariable 的表格数据表示。我们的发现表明，masked modeling 是一个有前途的研究方向。代码公开 available。
Distribution-Free Statistical Dispersion Control for Societal Applications
results: 该论文通过实验表明，该方法可以在恶意评论检测、医疗影像和电影推荐等领域中提供精确的统计保证。Abstract
Explicit finite-sample statistical guarantees on model performance are an important ingredient in responsible machine learning. Previous work has focused mainly on bounding either the expected loss of a predictor or the probability that an individual prediction will incur a loss value in a specified range. However, for many high-stakes applications, it is crucial to understand and control the dispersion of a loss distribution, or the extent to which different members of a population experience unequal effects of algorithmic decisions. We initiate the study of distribution-free control of statistical dispersion measures with societal implications and propose a simple yet flexible framework that allows us to handle a much richer class of statistical functionals beyond previous work. Our methods are verified through experiments in toxic comment detection, medical imaging, and film recommendation.
摘要
具有具体 finite-sample 统计保证的机器学习是责任感知的重要组成部分。先前的工作主要关注在预测器的预期损失下的约束或者预测结果会在指定范围内带来损失值的概率上。然而，许多高度投资应用中，控制统计分布的偏差是关键，也就是不同人群受到机器决策的不同影响程度。我们开始研究不含统计分布的控制方法，并提出了简单 yet flexible 的框架，可以处理更加复杂的统计函数。我们的方法通过对毒评排除、医疗成像和电影推荐进行实验验证。
Multi-Task Learning For Reduced Popularity Bias In Multi-Territory Video Recommendations
results: 通过实验，我们 demonstarte 了我们的框架在多区域比基eline 表现出较好的效果，PR-AUC 指标中的增幅可达 $65.27%$。Abstract
Various data imbalances that naturally arise in a multi-territory personalized recommender system can lead to a significant item bias for globally prevalent items. A locally popular item can be overshadowed by a globally prevalent item. Moreover, users' viewership patterns/statistics can drastically change from one geographic location to another which may suggest to learn specific user embeddings. In this paper, we propose a multi-task learning (MTL) technique, along with an adaptive upsampling method to reduce popularity bias in multi-territory recommendations. Our proposed framework is designed to enrich training examples with active users representation through upsampling, and capable of learning geographic-based user embeddings by leveraging MTL. Through experiments, we demonstrate the effectiveness of our framework in multiple territories compared to a baseline not incorporating our proposed techniques.~Noticeably, we show improved relative gain of up to $65.27\%$ in PR-AUC metric. A case study is presented to demonstrate the advantages of our methods in attenuating the popularity bias of global items.
摘要
不同地区的用户偏好会自然出现在多地区个性化推荐系统中，导致全球热销商品的 item bias。一个地区热销商品可能被全球热销商品所掩蔽。此外，用户的视频浏览习惯可能在不同的地理位置发生重大变化，这可能建议学习特定用户嵌入。在这篇论文中，我们提出了一种多任务学习（MTL）技术，以及适应填充方法，以减少多地区推荐中的流行度偏好。我们的提议框架通过填充活跃用户表示来增强训练示例，并能够通过 MTL 学习地域基于用户嵌入。通过实验，我们证明了我们的框架在多地区比基eline不 incorporating 我们的提议技术时表现更高的效果。特别是，我们显示了改进的相对增长率达到 $65.27\%$ 的 PR-AUC 指标。一个案例研究表明了我们的方法在减少全球商品的流行度偏好中的优势。