paper_authors: Mayug Maniparambil, Chris Vorster, Derek Molloy, Noel Murphy, Kevin McGuinness, Noel E. O’Connor for:This paper focuses on improving the performance of CLIP, a contrastive pre-trained large Vision-Language Model (VLM), on downstream datasets by using GPT-4 to generate visually descriptive text prompts.methods:The authors use GPT-4 to generate text prompts that are relevant to the downstream dataset, and then use these prompts to adapt CLIP to the dataset with zero-shot learning. They also design a simple few-shot adapter that learns to choose the best possible sentences to construct generalizable classifiers.results:The authors show that their method achieves considerable improvements in 0-shot transfer accuracy on specialized fine-grained datasets, outperforming CLIP’s default prompt by around 7% on average. They also demonstrate that their simple few-shot adapter outperforms the recently proposed CoCoOP by around 2% on average and by over 4% on 4 specialized fine-grained datasets.Here is the simplified Chinese version of the three key points:for:这篇论文目标是提高CLIP的下游数据集表现，使用GPT-4生成相关的视觉描述文本提示。methods:作者使用GPT-4生成相关的视觉描述文本提示，然后使用这些提示将CLIP适应到数据集中进行零批学习。他们还设计了一个简单的几批适应器，可以选择最佳的句子来构建通用的分类器。results:作者表明，他们的方法在特殊的细化数据集上实现了considerable的0批传输准确率提升，比CLIP的默认提示高约7%。他们还证明了他们的简单的几批适应器可以超过最近提出的CoCoOP，在4个特殊的细化数据集上平均提高了2%，并在这些数据集上提高了4%。Abstract
Contrastive pretrained large Vision-Language Models (VLMs) like CLIP have revolutionized visual representation learning by providing good performance on downstream datasets. VLMs are 0-shot adapted to a downstream dataset by designing prompts that are relevant to the dataset. Such prompt engineering makes use of domain expertise and a validation dataset. Meanwhile, recent developments in generative pretrained models like GPT-4 mean they can be used as advanced internet search tools. They can also be manipulated to provide visual information in any structure. In this work, we show that GPT-4 can be used to generate text that is visually descriptive and how this can be used to adapt CLIP to downstream tasks. We show considerable improvements in 0-shot transfer accuracy on specialized fine-grained datasets like EuroSAT (~7%), DTD (~7%), SUN397 (~4.6%), and CUB (~3.3%) when compared to CLIP's default prompt. We also design a simple few-shot adapter that learns to choose the best possible sentences to construct generalizable classifiers that outperform the recently proposed CoCoOP by ~2% on average and by over 4% on 4 specialized fine-grained datasets. The code, prompts, and auxiliary text dataset is available at https://github.com/mayug/VDT-Adapter.
摘要
带有对比学习的大视力语言模型（VLM）如CLIP，已经革命化视觉表示学习。VLM可以通过设计相关的提示来适应下游数据集，而这种提示工程充分利用了领域专业知识和验证数据集。此外，最近的生成预训练模型如GPT-4，可以用作高级网络搜索工具，同时可以通过提供任意结构的视觉信息来操作。在这项工作中，我们展示了GPT-4可以生成视觉描述性文本，并使其用于CLIP的适应下游任务。我们发现，与CLIP的默认提示相比，在特殊化细腻数据集（EuroSAT、DTD、SUN397和CUB）上显示了 considerable improvement（大约7%）。此外，我们还设计了一个简单的几拍适配器，可以选择最佳的句子来构建通用的分类器，超过了最近提出的CoCoOP的平均提升率（大约2%），并在特殊化细腻数据集上提升了4%。代码、提示和辅助文本数据集可以在https://github.com/mayug/VDT-Adapter中下载。
results: 论文分析了这种模型下的在线学习算法，并证明了这些算法可以在任何可能的参数化下进行 оптими化。Specifically, the regret rates obtained are: for $\lambda \in [0, 1/T^2]$: $\widetilde O(\sqrt{KT})$; for $\lambda = T^{-a/b}$ with $b < a < 2b$: $\widetilde O (T^{b/a})$; for $\lambda \in (1/T, 1 - 1/\sqrt{T}): \widetilde O (K^{1/3}T^{2/3})$; and for $\lambda \in [1 - 1/\sqrt{T}, 1]: \widetilde O (K\sqrt{T})$.Abstract
We propose a model for learning with bandit feedback while accounting for deterministically evolving and unobservable states that we call Bandits with Deterministically Evolving States. The workhorse applications of our model are learning for recommendation systems and learning for online ads. In both cases, the reward that the algorithm obtains at each round is a function of the short-term reward of the action chosen and how ``healthy'' the system is (i.e., as measured by its state). For example, in recommendation systems, the reward that the platform obtains from a user's engagement with a particular type of content depends not only on the inherent features of the specific content, but also on how the user's preferences have evolved as a result of interacting with other types of content on the platform. Our general model accounts for the different rate $\lambda \in [0,1]$ at which the state evolves (e.g., how fast a user's preferences shift as a result of previous content consumption) and encompasses standard multi-armed bandits as a special case. The goal of the algorithm is to minimize a notion of regret against the best-fixed sequence of arms pulled. We analyze online learning algorithms for any possible parametrization of the evolution rate $\lambda$. Specifically, the regret rates obtained are: for $\lambda \in [0, 1/T^2]$: $\widetilde O(\sqrt{KT})$; for $\lambda = T^{-a/b}$ with $b < a < 2b$: $\widetilde O (T^{b/a})$; for $\lambda \in (1/T, 1 - 1/\sqrt{T}): \widetilde O (K^{1/3}T^{2/3})$; and for $\lambda \in [1 - 1/\sqrt{T}, 1]: \widetilde O (K\sqrt{T})$.
摘要
我们提出一个模型，称为带征 Deterministic Evolving States 的 Bandit Learning 模型。这种模型的应用包括推荐系统和在线广告学习。在这些应用中，算法在每个轮次获得的奖励是功能和系统状态（例如，用户的喜好）的函数。例如，在推荐系统中，用户与某种内容的互动奖励不仅受到内容本身的特点的影响，还受到用户在其他类型的内容上的互动的影响。我们的通用模型考虑了不同的演化速率 $\lambda \in [0,1]$，并包括标准多重武器的特例。算法的目标是对于任何可能的参数化 $\lambda$，最小化对最佳固定sequence of arms pulled的 regret。我们分析了在线学习算法，并得到了不同的 regret 率：* for $\lambda \in [0, 1/T^2]$: $\widetilde O(\sqrt{KT})$;* for $\lambda = T^{-a/b}$ with $b < a < 2b$: $\widetilde O (T^{b/a})$;* for $\lambda \in (1/T, 1 - 1/\sqrt{T}): \widetilde O (K^{1/3}T^{2/3})$;* for $\lambda \in [1 - 1/\sqrt{T}, 1]: \widetilde O (K\sqrt{T})$.
Alleviating the Long-Tail Problem in Conversational Recommender Systems
results: 在两个公共 CRS 数据集上进行了广泛的实验，证明了我们的方法的有效性和可扩展性，特别是对长尾项的推荐。Abstract
Conversational recommender systems (CRS) aim to provide the recommendation service via natural language conversations. To develop an effective CRS, high-quality CRS datasets are very crucial. However, existing CRS datasets suffer from the long-tail issue, \ie a large proportion of items are rarely (or even never) mentioned in the conversations, which are called long-tail items. As a result, the CRSs trained on these datasets tend to recommend frequent items, and the diversity of the recommended items would be largely reduced, making users easier to get bored. To address this issue, this paper presents \textbf{LOT-CRS}, a novel framework that focuses on simulating and utilizing a balanced CRS dataset (\ie covering all the items evenly) for improving \textbf{LO}ng-\textbf{T}ail recommendation performance of CRSs. In our approach, we design two pre-training tasks to enhance the understanding of simulated conversation for long-tail items, and adopt retrieval-augmented fine-tuning with label smoothness strategy to further improve the recommendation of long-tail items. Extensive experiments on two public CRS datasets have demonstrated the effectiveness and extensibility of our approach, especially on long-tail recommendation.
摘要
很多受众推荐系统（CRS）寻求通过自然语言对话提供推荐服务。为开发有效的CRS，高质量CRS数据集非常重要。然而，现有的CRS数据集受到长尾问题的困扰，即大多数 item 在对话中被 rarely (或者是从未) 提及，这些 item 被称为长尾 item。这导致CRS 在这些数据集上训练后，倾向于推荐频繁 item，推荐的 Item 的多样性会受到很大的削弱，使用户更容易感到厌烦。为解决这个问题，本文提出了一种新的框架，即 LOT-CRS，它是一种集中 item 的 CRS 数据集，以提高长尾推荐性能。在我们的方法中，我们设计了两个预训练任务，以增强对 simulated conversation 的理解，并采用了提取扩展的 fine-tuning 策略和标签平滑策略，以进一步提高长尾推荐。我们在两个公共 CRS 数据集上进行了广泛的实验，并证明了我们的方法的有效性和可扩展性，特别是在长尾推荐方面。
Morphological Image Analysis and Feature Extraction for Reasoning with AI-based Defect Detection and Classification Models
results: 实验结果表明，AI-Reasoner能够有效地解释IE Mask R-CNN模型的预测结果。总的来说，该论文提供了一种解释AI模型表现的解决方案，有助于提高工业应用中的AI模型表现。Abstract
As the use of artificial intelligent (AI) models becomes more prevalent in industries such as engineering and manufacturing, it is essential that these models provide transparent reasoning behind their predictions. This paper proposes the AI-Reasoner, which extracts the morphological characteristics of defects (DefChars) from images and utilises decision trees to reason with the DefChar values. Thereafter, the AI-Reasoner exports visualisations (i.e. charts) and textual explanations to provide insights into outputs made by masked-based defect detection and classification models. It also provides effective mitigation strategies to enhance data pre-processing and overall model performance. The AI-Reasoner was tested on explaining the outputs of an IE Mask R-CNN model using a set of 366 images containing defects. The results demonstrated its effectiveness in explaining the IE Mask R-CNN model's predictions. Overall, the proposed AI-Reasoner provides a solution for improving the performance of AI models in industrial applications that require defect analysis.
摘要
随着人工智能（AI）模型在工程和生产中的应用变得更加普遍，这些模型的预测需要提供透明的思维过程。这篇论文提议了AI理解器（AI-Reasoner），它从图像中提取杂形特征（DefChars），并使用决策树来进行思维。然后，AI-Reasoner将生成视觉化（如图表）和文本解释，以提供对掩模隐藏基于漏斗检测和分类模型的输出的深入了解。它还提供有效的缓解策略，以提高数据预处理和整体模型性能。AI-Reasoner在对IE Mask R-CNN模型的输出进行解释中得到了证明。总之，提议的AI-Reasoner为工业应用中需要检测分析的AI模型带来了改进性。
The Two Faces of AI in Green Mobile Computing: A Literature Review
results: 本研究の结果は、过去10年间で人工智能在移动设备上の应用が増加していることを示しています。しかし、人工智能の能源消耗については、更に调查が必要です。また、大多数の研究が公开されていないことも発见しています。Abstract
Artificial intelligence is bringing ever new functionalities to the realm of mobile devices that are now considered essential (e.g., camera and voice assistants, recommender systems). Yet, operating artificial intelligence takes up a substantial amount of energy. However, artificial intelligence is also being used to enable more energy-efficient solutions for mobile systems. Hence, artificial intelligence has two faces in that regard, it is both a key enabler of desired (efficient) mobile functionalities and a major power draw on these devices, playing a part in both the solution and the problem. In this paper, we present a review of the literature of the past decade on the usage of artificial intelligence within the realm of green mobile computing. From the analysis of 34 papers, we highlight the emerging patterns and map the field into 13 main topics that are summarized in details. Our results showcase that the field is slowly increasing in the past years, more specifically, since 2019. Regarding the double impact AI has on the mobile energy consumption, the energy consumption of AI-based mobile systems is under-studied in comparison to the usage of AI for energy-efficient mobile computing, and we argue for more exploratory studies in that direction. We observe that although most studies are framed as solution papers (94%), the large majority do not make those solutions publicly available to the community. Moreover, we also show that most contributions are purely academic (28 out of 34 papers) and that we need to promote the involvement of the mobile software industry in this field.
摘要
人工智能在移动设备领域带来了不断新的功能（例如相机和语音助手、推荐系统），现在被视为必备的功能。然而，运行人工智能需要很多能量。然而，人工智能也在使移动系统更加能效的解决方案中发挥作用。因此，人工智能在这个方面有两个面，它是必需的功能启用者和移动设备的主要能量消耗者。在这篇论文中，我们对过去十年的文献进行了回顾，并将场景映射到13个主题中。我们的结果显示，这个领域在过去几年中逐渐增长，特别是自2019年起。关于人工智能对移动设备能 consumption的双重影响，我们认为需要更多的探索性研究。我们发现，大多数研究是呈现为解决方案纸（94%），但大多数解决方案没有公开提供给社区。此外，我们还发现大多数贡献是学术性质（28 out of 34 papers），我们需要推动移动软件产业的参与。
Integration of Domain Expert-Centric Ontology Design into the CRISP-DM for Cyber-Physical Production Systems
results: 本研究实现了一种可靠的 anomaly detection 用例，帮助数据科学家更快地和更可靠地从 CPPSs 中获得有价值信息。Abstract
In the age of Industry 4.0 and Cyber-Physical Production Systems (CPPSs) vast amounts of potentially valuable data are being generated. Methods from Machine Learning (ML) and Data Mining (DM) have proven to be promising in extracting complex and hidden patterns from the data collected. The knowledge obtained can in turn be used to improve tasks like diagnostics or maintenance planning. However, such data-driven projects, usually performed with the Cross-Industry Standard Process for Data Mining (CRISP-DM), often fail due to the disproportionate amount of time needed for understanding and preparing the data. The application of domain-specific ontologies has demonstrated its advantageousness in a wide variety of Industry 4.0 application scenarios regarding the aforementioned challenges. However, workflows and artifacts from ontology design for CPPSs have not yet been systematically integrated into the CRISP-DM. Accordingly, this contribution intends to present an integrated approach so that data scientists are able to more quickly and reliably gain insights into the CPPS. The result is exemplarily applied to an anomaly detection use case.
摘要
在第四产业时代和跨industry标准数据挖掘过程（CPPS）中， vast amounts of potentially valuable data 被生成。机器学习（ML）和数据挖掘（DM）的方法已经证明可以提取复杂和隐藏的模式，从而提高诊断或维护规划等任务。然而，这些数据驱动项目通常使用 Cross-Industry Standard Process for Data Mining（CRISP-DM）进行实施，但它们往往因为数据理解和准备过程中的时间过长而失败。适用域pecific ontology 在多种第四产业应用场景中表现出了优势。然而， CPPSs 的 workflow 和 artifacts 尚未被系统地 инте integrate into CRISP-DM。因此，本贡献的目的是提出一种集成的approach，使数据科学家能够更快速地和更可靠地获得 CPPS 的 Insights。结果通过例示了一个异常检测用例来说明。
On the Complexity of the Bipartite Polarization Problem: from Neutral to Highly Polarized Discussions
results: 研究发现，极化程度与实例的解决难度正相关，即极化程度越高，解决实例的难度就越低。Abstract
The Bipartite Polarization Problem is an optimization problem where the goal is to find the highest polarized bipartition on a weighted and labelled graph that represents a debate developed through some social network, where nodes represent user's opinions and edges agreement or disagreement between users. This problem can be seen as a generalization of the maxcut problem, and in previous work approximate solutions and exact solutions have been obtained for real instances obtained from Reddit discussions, showing that such real instances seem to be very easy to solve. In this paper, we investigate further the complexity of this problem, by introducing an instance generation model where a single parameter controls the polarization of the instances in such a way that this correlates with the average complexity to solve those instances. The average complexity results we obtain are consistent with our hypothesis: the higher the polarization of the instance, the easier is to find the corresponding polarized bipartition.
摘要
《双分化卷积问题》是一个优化问题，目标是在一个权重 Labelled 图上找到最高卷积分的生成方法，表示一个社交网络上的辩论，节点表示用户的意见，边表示用户之间的一致或不一致。这个问题可以看作是最大cut问题的推广，在过去的工作中，人们已经得到了实际例子中的近似解和精确解，显示这些实际例子很容易解决。在这篇论文中，我们进一步调查了这个问题的复杂性，通过设计一个参数控制实例的卷积程度，以确定实例的复杂性和解决实例的难度之间的关系。我们获得的平均复杂性结果与我们的假设一致：卷积程度越高，实例的解决难度就越低。
results: 实验结果表明，CausE 可以超过基eline模型，并实现状态的� characteristic KGC performance。Abstract
Knowledge graph embedding (KGE) focuses on representing the entities and relations of a knowledge graph (KG) into the continuous vector spaces, which can be employed to predict the missing triples to achieve knowledge graph completion (KGC). However, KGE models often only briefly learn structural correlations of triple data and embeddings would be misled by the trivial patterns and noisy links in real-world KGs. To address this issue, we build the new paradigm of KGE in the context of causality and embedding disentanglement. We further propose a Causality-enhanced knowledge graph Embedding (CausE) framework. CausE employs causal intervention to estimate the causal effect of the confounder embeddings and design new training objectives to make stable predictions. Experimental results demonstrate that CausE could outperform the baseline models and achieve state-of-the-art KGC performance. We release our code in https://github.com/zjukg/CausE.
摘要
知识图embedding（KGE）专注于将知识图（KG）中的实体和关系转换到连续的vector空间中，以便预测缺失的 triple以完成知识图完成（KGC）。然而，KGE模型经常只是简单地学习 triple数据的结构相关性，而 embedding会受到实际世界KG中的噪声和负面相关性的影响。为解决这个问题，我们建立了一种基于 causality的新型KGE paradigma，并提出了一种causality-enhanced知识图Embedding（CausE）框架。CausE使用 causal intervention来估计干扰因子 embedding的 causal效果，并设计了新的训练目标来确保稳定的预测。实验结果表明，CausE可以超越基eline模型，并实现状态控制KGC性能。我们在https://github.com/zjukg/CausE中发布了我们的代码。
Predict-AI-bility of how humans balance self-interest with the interest of others
paper_authors: Valerio Capraro, Roberto Di Paolo, Veronica Pizziol
for: This paper aims to investigate the ability of three advanced chatbots to predict dictator game decisions and capture the balance between self-interest and the interest of others in decision-making.
methods: The paper uses 78 experiments with human participants from 12 countries to evaluate the performance of GPT-4, Bard, and Bing in predicting dictator game decisions and identifying qualitative behavioral patterns.
results: The paper finds that only GPT-4 correctly captures qualitative behavioral patterns, identifying three major classes of behavior, but consistently overestimates other-regarding behavior, inflating the proportion of inequity-averse and fully altruistic participants. This bias has significant implications for AI developers and users.Abstract
Generative artificial intelligence holds enormous potential to revolutionize decision-making processes, from everyday to high-stake scenarios. However, as many decisions carry social implications, for AI to be a reliable assistant for decision-making it is crucial that it is able to capture the balance between self-interest and the interest of others. We investigate the ability of three of the most advanced chatbots to predict dictator game decisions across 78 experiments with human participants from 12 countries. We find that only GPT-4 (not Bard nor Bing) correctly captures qualitative behavioral patterns, identifying three major classes of behavior: self-interested, inequity-averse, and fully altruistic. Nonetheless, GPT-4 consistently overestimates other-regarding behavior, inflating the proportion of inequity-averse and fully altruistic participants. This bias has significant implications for AI developers and users.
摘要
translate the given text into Simplified ChineseGenerated artificial intelligence has the potential to revolutionize decision-making processes, from everyday to high-stakes scenarios. However, as many decisions have social implications, for AI to be a reliable assistant for decision-making, it is crucial that it can capture the balance between self-interest and the interest of others. We investigate the ability of three of the most advanced chatbots to predict dictator game decisions across 78 experiments with human participants from 12 countries. We find that only GPT-4 (not Bard nor Bing) correctly captures qualitative behavioral patterns, identifying three major classes of behavior: self-interested, inequity-averse, and fully altruistic. However, GPT-4 consistently overestimates other-regarding behavior, inflating the proportion of inequity-averse and fully altruistic participants. This bias has significant implications for AI developers and users.Note: "Dictator game" is not a commonly used term in Simplified Chinese, so it may be better to use a more common term such as "分配游戏" (bǐng huò yóu xì) or "权力游戏" (quán lì yóu xì) to convey the same idea.
paper_authors: Dong Huang, Qingwen Bu, Yahao Qing, Yichao Fu, Heming Cui
for: This paper is written to address the issue of deep learning testing, specifically the problem of detecting fault-inducing feature maps in deep neural networks (DNNs).
methods: The paper proposes a new method called DeepFeature, which tests DNNs from the feature map level and identifies vulnerabilities that can be enhanced through repairing to increase the model’s overall performance.
results: The paper presents experimental results that demonstrate the effectiveness of DeepFeature in detecting the model’s vulnerable feature maps, with a high fault detection rate and the ability to detect more types of faults compared to current techniques. Additionally, the paper shows that DeepFeature’s fuzzer outperforms current fuzzing techniques and generates valuable test cases more efficiently.Here is the same information in Simplified Chinese text:
results: 论文发表了实验结果，证明DeepFeature可以快速检测DNNs中的特征图漏洞，并且能够检测更多的问题类型，比如当前技术。另外，论文还表明DeepFeature的随机生成器比现有的随机生成技术更高效。Abstract
Due to the widespread application of deep neural networks~(DNNs) in safety-critical tasks, deep learning testing has drawn increasing attention. During the testing process, test cases that have been fuzzed or selected using test metrics are fed into the model to find fault-inducing test units (e.g., neurons and feature maps, activating which will almost certainly result in a model error) and report them to the DNN developer, who subsequently repair them~(e.g., retraining the model with test cases). Current test metrics, however, are primarily concerned with the neurons, which means that test cases that are discovered either by guided fuzzing or selection with these metrics focus on detecting fault-inducing neurons while failing to detect fault-inducing feature maps. In this work, we propose DeepFeature, which tests DNNs from the feature map level. When testing is conducted, DeepFeature will scrutinize every internal feature map in the model and identify vulnerabilities that can be enhanced through repairing to increase the model's overall performance. Exhaustive experiments are conducted to demonstrate that (1) DeepFeature is a strong tool for detecting the model's vulnerable feature maps; (2) DeepFeature's test case selection has a high fault detection rate and can detect more types of faults~(comparing DeepFeature to coverage-guided selection techniques, the fault detection rate is increased by 49.32\%). (3) DeepFeature's fuzzer also outperforms current fuzzing techniques and generates valuable test cases more efficiently.
摘要
由于深度神经网络（DNN）在安全关键任务中广泛应用，深度学习测试已引起了越来越多的关注。测试过程中，经过抽象或使用测试指标选择的测试用例会被 feed 到模型中，以找到引起模型错误的测试单元（例如神经元和特征图），并将其报告给 DNN 开发者，以便他们进行修复（例如重新训练模型使用测试用例）。现有的测试指标主要关注神经元，因此通过抽象或选择测试指标来发现的测试用例主要是检测引起模型错误的神经元，而忽略了特征图。在这项工作中，我们提出了 DeepFeature，它测试 DNN 从特征图层次。在测试过程中，DeepFeature 会仔细检查模型中每个内部特征图，并找到可以通过修复提高模型的性能的漏洞。我们进行了广泛的实验，证明了以下结论：1. DeepFeature 是一个强大的特征图漏洞检测工具，可以帮助检测模型的漏洞特征图。2. DeepFeature 的测试用例选择比coverage-guided选择技术高出49.32%的FAULT检测率。3. DeepFeature 的随机生成器也超越了当前的随机生成技术，可以更有效地生成有价值的测试用例。
results: 在使用Weighted Multi-Objective Function from state-of-the-art BioIK方法支持下，神经网络模型可以与现有的IK方法竞争，并且通过 incorporating a genetic algorithm 可以提高精度，同时减少总的运行时间。Abstract
The paper introduces CycleIK, a neuro-robotic approach that wraps two novel neuro-inspired methods for the inverse kinematics (IK) task, a Generative Adversarial Network (GAN), and a Multi-Layer Perceptron architecture. These methods can be used in a standalone fashion, but we also show how embedding these into a hybrid neuro-genetic IK pipeline allows for further optimization via sequential least-squares programming (SLSQP) or a genetic algorithm (GA). The models are trained and tested on dense datasets that were collected from random robot configurations of the new Neuro-Inspired COLlaborator (NICOL), a semi-humanoid robot with two redundant 8-DoF manipulators. We utilize the weighted multi-objective function from the state-of-the-art BioIK method to support the training process and our hybrid neuro-genetic architecture. We show that the neural models can compete with state-of-the-art IK approaches, which allows for deployment directly to robotic hardware. Additionally, it is shown that the incorporation of the genetic algorithm improves the precision while simultaneously reducing the overall runtime.
摘要
文章介绍了 CycleIK，一种神经机器人方法，包括两种新的神经网络做 inverse kinematics（IK）任务的方法，生成对抗网络（GAN）和多层感知网络架构。这些方法可以单独使用，但我们还证明了将它们集成到一个混合神经遗传IK管道中，可以通过顺序最小二乘程序（SLSQP）或遗传算法（GA）进行进一步优化。模型在 dense 数据集上训练和测试，数据集是通过随机机器人配置新的神经机器人 Neuro-Inspired COLlaborator（NICOL）的两个冗余的 8-DoF 机械臂收集而来。我们使用了 BioIK 方法中的Weighted 多目标函数来支持训练过程，并使用我们的混合神经遗传架构。我们显示了神经模型可以与当前IK方法竞争，可以直接部署到机器人硬件上。此外，我们还证明了将遗传算法包含在PIPELINE中可以提高精度，同时降低总时间。
Identifying Relevant Features of CSE-CIC-IDS2018 Dataset for the Development of an Intrusion Detection System
results: 研究发现，采用不同的特征选择方法和分类算法可以得到不同的优化效果，并且可以为不同的攻击类型选择最佳的特征集。Abstract
Intrusion detection systems (IDSs) are essential elements of IT systems. Their key component is a classification module that continuously evaluates some features of the network traffic and identifies possible threats. Its efficiency is greatly affected by the right selection of the features to be monitored. Therefore, the identification of a minimal set of features that are necessary to safely distinguish malicious traffic from benign traffic is indispensable in the course of the development of an IDS. This paper presents the preprocessing and feature selection workflow as well as its results in the case of the CSE-CIC-IDS2018 on AWS dataset, focusing on five attack types. To identify the relevant features, six feature selection methods were applied, and the final ranking of the features was elaborated based on their average score. Next, several subsets of the features were formed based on different ranking threshold values, and each subset was tried with five classification algorithms to determine the optimal feature set for each attack type. During the evaluation, four widely used metrics were taken into consideration.
摘要
安全系统检测系统（IDS）是信息技术系统中不可或缺的元素。它的关键组件是分类模块，不断评估网络流量中的一些特征，并识别可能的威胁。因此，选择需要监测的特征是非常重要的，以确保安全地分辨恶意流量和良好流量。本文介绍了预处理和特征选择工作流程，以及在CSE-CIC-IDS2018 on AWS dataset上的实验结果，关注五种攻击类型。为了确定相关的特征，本文使用了六种特征选择方法，并根据每个特征的平均分数进行了最终排名。然后，根据不同的排名阈值，将特征分为多个子集，并对每个子集使用五种分类算法进行了评估。在评估过程中，考虑了四种常用的指标。
Model Reporting for Certifiable AI: A Proposal from Merging EU Regulation into AI Development
paper_authors: Danilo Brajovic, Niclas Renner, Vincent Philipp Goebels, Philipp Wagner, Benjamin Fresz, Martin Biller, Mara Klaeb, Janika Kutz, Jens Neuhuettler, Marco F. Huber
for: 这篇论文旨在提供标准化的卡片来描述 AI 应用程序的开发过程，以帮助实践者开发安全的 AI 系统，并且满足政策法规的要求。
methods: 该论文使用了最新的欧盟法规和 AI 指南，以及最新的研究趋势：数据和模型卡片。它提出了使用标准化的卡片来记录 AI 应用程序的开发过程，包括用例卡和运行卡，以满足政策法规的要求。
results: 该论文的主要贡献是提出了一套标准化的卡片，以帮助实践者开发安全的 AI 系统，并且可以方便第三方进行 AI 应用程序的审核。该论文还 incorporates 了专家访谈和开发者的意见，以及相关的研究和工具箱。Abstract
Despite large progress in Explainable and Safe AI, practitioners suffer from a lack of regulation and standards for AI safety. In this work we merge recent regulation efforts by the European Union and first proposals for AI guidelines with recent trends in research: data and model cards. We propose the use of standardized cards to document AI applications throughout the development process. Our main contribution is the introduction of use-case and operation cards, along with updates for data and model cards to cope with regulatory requirements. We reference both recent research as well as the source of the regulation in our cards and provide references to additional support material and toolboxes whenever possible. The goal is to design cards that help practitioners develop safe AI systems throughout the development process, while enabling efficient third-party auditing of AI applications, being easy to understand, and building trust in the system. Our work incorporates insights from interviews with certification experts as well as developers and individuals working with the developed AI applications.
摘要
尽管在可解释和安全人工智能方面有大量进步，但实践者受到了不足的规范和标准的压力。在这项工作中，我们将欧盟最新的规定努力和首个AI指南提议与最新的研究趋势相结合：数据和模型卡。我们提议在开发过程中使用标准化卡来记录AI应用程序。我们的主要贡献是引入使用情况和运行卡，并更新数据和模型卡以适应规定要求。我们参考了最新的研究以及规定的来源，并提供了参考资料和工具箱 whenever possible。我们的目标是通过开发安全人工智能系统的整个开发过程中使用卡来帮助实践者，并允许第三方审核人工智能应用程序，易于理解，建立信任系统。我们的工作还 incorporates 专家评论、开发人员和使用开发的AI应用程序的人员的意见。
IndigoVX: Where Human Intelligence Meets AI for Optimal Decision Making
results: 本研究显示，通过人类和AI的联合合作，可以实现更高效的目标解决。通过变量的三个分数评估指标，评估和改进策略，适应到实际挑战和变化。Abstract
This paper defines a new approach for augmenting human intelligence with AI for optimal goal solving. Our proposed AI, Indigo, is an acronym for Informed Numerical Decision-making through Iterative Goal-Oriented optimization. When combined with a human collaborator, we term the joint system IndigoVX, for Virtual eXpert. The system is conceptually simple. We envisage this method being applied to games or business strategies, with the human providing strategic context and the AI offering optimal, data-driven moves. Indigo operates through an iterative feedback loop, harnessing the human expert's contextual knowledge and the AI's data-driven insights to craft and refine strategies towards a well-defined goal. Using a quantified three-score schema, this hybridization allows the combined team to evaluate strategies and refine their plan, while adapting to challenges and changes in real-time.
摘要
这篇论文提出了一种新的方法，用人工智能增强人类智能，以实现最佳目标解决。我们的提议的AI系统名为Indigo，全名为Informed Numerical Decision-making through Iterative Goal-Oriented optimization。当与人类合作者结合时，我们称之为IndigoVX，即虚拟专家。该系统的核心思想简单。我们认为这种方法可以应用于游戏或商业策略等领域，人类提供战略背景知识，AI提供数据驱动的优化 Move。Indigo通过迭代反馈循环，利用人类专家的Contextual knowledge和AI的数据驱动洞察，制定和细化策略，以达到已定义的目标。使用量化的三个分数schema，这个混合体系可以评估策略和修改计划，同时适应挑战和变化的实时反应。
Framework for developing quantitative agent based models based on qualitative expert knowledge: an organised crime use-case
paper_authors: Frederike Oetker, Vittorio Nespeca, Thijs Vis, Paul Duijn, Peter Sloot, Rick Quax
For: + The paper aims to provide a systematic and transparent framework for creating agent-based models of criminal networks, specifically for law enforcement purposes. + The authors propose a methodology called FREIDA (Framework for Expert-Informed Data-driven Agent-based models) to translate qualitative data into quantitative rules for modeling criminal networks. + The paper uses the example of a criminal cocaine network in the Netherlands to demonstrate the FREIDA methodology.* Methods: + The authors use a combination of qualitative data sources (case files, literature, interviews) and quantitative data sources (databases) to create a networked agent-based model of the criminal cocaine network. + They use empirical laws to translate the qualitative data into quantitative rules, and combine these rules with the quantitative data to create the three dimensions (environment, agents, behavior) of the networked ABM. + The authors perform sensitivity analysis, uncertainty quantification, and scenario testing to validate the model and make it robust for law enforcement planning.* Results: + The authors find that the model requires flexible parameters and additional case file simulations to be performed to achieve a robust model. + The results of the sensitivity analysis and scenario testing indicate the need for adaptive intervention strategies that can respond to changes in the criminal network.Here is the summary in Simplified Chinese text:* For: + 本研究旨在提供法 enforcement 目的下的刑事网络模型创建系统，特别是通过转化专家知识到数据驱动的方法。 + 作者提出了 FREIDA（框架 для专家驱动数据驱动模型）方法，以便将刑事网络的质量数据转化为数值规则。 + 本研究使用荷兰的刑事可毒网络为例，以示 FREIDA 方法ологи。* Methods: + 作者使用质量数据源（案例文件、文献、面谈）和量化数据源（数据库）创建刑事网络的 Agent-based 模型。 + 他们使用实证法则将质量数据转化为数值规则，并将这些规则与量化数据相结合以创建网络 Agent-based 模型的三个维度（环境、代理、行为）。 + 作者进行了敏感分析、不确定性评估和enario 测试，以验证模型并使其对法 enforcement 规划有效。* Results: + 作者发现模型需要灵活的参数和更多的案例文件 simulate 以实现稳定的模型。 + 结果表明，需要适应性的干预策略，以应对刑事网络的变化。Abstract
In order to model criminal networks for law enforcement purposes, a limited supply of data needs to be translated into validated agent-based models. What is missing in current criminological modelling is a systematic and transparent framework for modelers and domain experts that establishes a modelling procedure for computational criminal modelling that includes translating qualitative data into quantitative rules. For this, we propose FREIDA (Framework for Expert-Informed Data-driven Agent-based models). Throughout the paper, the criminal cocaine replacement model (CCRM) will be used as an example case to demonstrate the FREIDA methodology. For the CCRM, a criminal cocaine network in the Netherlands is being modelled where the kingpin node is being removed, the goal being for the remaining agents to reorganize after the disruption and return the network into a stable state. Qualitative data sources such as case files, literature and interviews are translated into empirical laws, and combined with the quantitative sources such as databases form the three dimensions (environment, agents, behaviour) of a networked ABM. Four case files are being modelled and scored both for training as well as for validation scores to transition to the computational model and application phase respectively. In the last phase, iterative sensitivity analysis, uncertainty quantification and scenario testing eventually lead to a robust model that can help law enforcement plan their intervention strategies. Results indicate the need for flexible parameters as well as additional case file simulations to be performed.
摘要
为了模拟犯罪网络，需要有限的数据被翻译成有效的代理基模型。现在的刑事模拟中缺乏一个系统化和透明的框架，使模型者和领域专家可以确定模型生成过程，包括将Qualitative数据转化为Quantitative规则。为此，我们提出了FREIDA（专家驱动数据驱动代理基模型框架）。本文中，我们使用了药物替换模型（CCRM）作为例子，模拟了荷兰一个犯罪冰毒网络，其中王牌节点被移除，目标是让剩下的代理重新组织并返回网络到稳定状态。Qualitative数据源，如案例文件、文献和采访，被翻译成Empirical法律，并与Quantitative数据源，如数据库，共同构成了网络ABM的三个维度（环境、代理、行为）。四个案例文件被模拟和评分，以便在计算模型阶段进行训练和验证分别。在最后阶段，iterative敏感分析、不确定性评估和enario测试，最终导致一个可靠的模型，可以帮助刑事机构规划 intervención策略。结果表明需要灵活的参数以及更多的案例文件模拟，以确保模型的可靠性。
General regularization in covariate shift adaptation
paper_authors: Duc Hoan Nguyen, Sergei V. Pereverzyev, Werner Zellinger
for: corrected the error of least squares learning algorithms in reproducing kernel Hilbert spaces (RKHS) caused by future data distributions that are different from the training data distribution.
methods: reweighted kernel regression in RKHS, using the estimated Radon-Nikod'ym derivative of the future data distribution w.r.t.~the training data distribution.
results: novel results obtained by combining known error bounds, showing that the amount of samples needed to achieve the same order of accuracy as in standard supervised learning without differences in data distributions is smaller than previously proven by state-of-the-art analyses, under weak smoothness conditions.Abstract
Sample reweighting is one of the most widely used methods for correcting the error of least squares learning algorithms in reproducing kernel Hilbert spaces (RKHS), that is caused by future data distributions that are different from the training data distribution. In practical situations, the sample weights are determined by values of the estimated Radon-Nikod\'ym derivative, of the future data distribution w.r.t.~the training data distribution. In this work, we review known error bounds for reweighted kernel regression in RKHS and obtain, by combination, novel results. We show under weak smoothness conditions, that the amount of samples, needed to achieve the same order of accuracy as in the standard supervised learning without differences in data distributions, is smaller than proven by state-of-the-art analyses.
摘要
样本重Weight是最常用的方法来修正最小二乘学习算法在 reproduce kernel Hilbert space（RKHS）中的错误，这是由于未来数据分布与训练数据分布不同而导致的。在实际情况下，样本重量是根据估计的Radon-Nikodym Derivative，未来数据分布与训练数据分布之间的比例来确定。在这种工作中，我们回顾了已知的重Weighted kernel regression在 RKHS 中的错误上限，并通过组合，获得了新的结果。我们表明，在弱稳定条件下，需要更少的样本数量，以达到标准超级vised learning 无法分布差异的同等精度水平。
Adaptive ResNet Architecture for Distributed Inference in Resource-Constrained IoT Systems
results: 根据实验结果，这个适应性的ResNet架构可以降低共享资料、能源消耗和延迟，并维持高准确率。Abstract
As deep neural networks continue to expand and become more complex, most edge devices are unable to handle their extensive processing requirements. Therefore, the concept of distributed inference is essential to distribute the neural network among a cluster of nodes. However, distribution may lead to additional energy consumption and dependency among devices that suffer from unstable transmission rates. Unstable transmission rates harm real-time performance of IoT devices causing low latency, high energy usage, and potential failures. Hence, for dynamic systems, it is necessary to have a resilient DNN with an adaptive architecture that can downsize as per the available resources. This paper presents an empirical study that identifies the connections in ResNet that can be dropped without significantly impacting the model's performance to enable distribution in case of resource shortage. Based on the results, a multi-objective optimization problem is formulated to minimize latency and maximize accuracy as per available resources. Our experiments demonstrate that an adaptive ResNet architecture can reduce shared data, energy consumption, and latency throughout the distribution while maintaining high accuracy.
摘要
Translated into Simplified Chinese:深度神经网络继续扩展和复杂化，大多数边缘设备无法处理它们的广泛处理要求。因此，分布式推理是必要的，以分布神经网络到一群节点中。然而，分布可能会导致更多的能源消耗和设备之间的依赖关系，从而影响实时性和可靠性。因此，对动态系统来说，需要一个可靠的DNN，具有可变的架构，以适应可用资源。这篇论文提出了一项实验研究，以确定ResNet中可以被去除的连接，不会对模型性能产生重要影响，以便在资源短缺情况下进行分布。基于结果，我们提出了一个多目标优化问题，以最小化延迟和最大化准确率，根据可用资源。我们的实验表明，可变ResNet架构可以降低共享数据、能源消耗和延迟，同时保持高准确率。
Predict, Refine, Synthesize: Self-Guiding Diffusion Models for Probabilistic Time Series Forecasting
paper_authors: Marcel Kollovieh, Abdul Fatir Ansari, Michael Bohlke-Schneider, Jasper Zschiegner, Hao Wang, Yuyang Wang
for: 这篇 paper 的目的是探讨时间序列散射模型的应用，并提出了一个不受条件所限的时间序列散射模型（TSDiff），可以在多个时间序列任务上进行应用。
methods: 这篇 paper 使用了一种自我引导机制，让 TSDiff 在推理过程中进行条件化，不需要额外的auxiliary networks或变更训练程式。
results: 这篇 paper 的结果显示 TSDiff 在三个时间序列任务中具有竞争力：一是与条件性的forecasting方法竞争（predict）；二是透过降低计算负载，使用 TSDiff 进行反散射（refine），并且模型对数据的生成性能仍然保持不变。Abstract
Diffusion models have achieved state-of-the-art performance in generative modeling tasks across various domains. Prior works on time series diffusion models have primarily focused on developing conditional models tailored to specific forecasting or imputation tasks. In this work, we explore the potential of task-agnostic, unconditional diffusion models for several time series applications. We propose TSDiff, an unconditionally trained diffusion model for time series. Our proposed self-guidance mechanism enables conditioning TSDiff for downstream tasks during inference, without requiring auxiliary networks or altering the training procedure. We demonstrate the effectiveness of our method on three different time series tasks: forecasting, refinement, and synthetic data generation. First, we show that TSDiff is competitive with several task-specific conditional forecasting methods (predict). Second, we leverage the learned implicit probability density of TSDiff to iteratively refine the predictions of base forecasters with reduced computational overhead over reverse diffusion (refine). Notably, the generative performance of the model remains intact -- downstream forecasters trained on synthetic samples from TSDiff outperform forecasters that are trained on samples from other state-of-the-art generative time series models, occasionally even outperforming models trained on real data (synthesize).
摘要
Diffusion models 已经在不同领域的生成模型任务中达到了状态计算机。先前的时间序列扩散模型研究都主要集中在发展特定预测或填充任务的条件模型。在这项工作中，我们探索了无条件的时间序列扩散模型在不同应用领域中的潜在性。我们提出了TSDiff，一种未经条件训练的时间序列扩散模型。我们的建议的自顾机制使得TSDiff在推理过程中可以通过自我指导来conditioning，无需附加网络或改变训练过程。我们示出了TSDiff在三个不同的时间序列任务上的效果：预测、修正和生成数据。首先，我们表明TSDiff与一些任务特定的条件预测方法相当竞争（predict）。其次，我们利用TSDiff学习到的隐式概率密度来降低预测基础预测器的计算开销，通过反扩散（refine）来修正基础预测器的预测。另外，通过使用TSDiff生成的 sintetic 样本来训练下游预测器，我们发现了一个关键结果：下游预测器训练在TSDiff生成的 sintetic 样本上表现更好， occasional 甚至超过了使用其他状态之前的生成时间序列模型训练的模型，即使是使用真实数据（synthesize）。
Robust Visual Question Answering: Datasets, Methods, and Future Challenges
paper_authors: Jie Ma, Pinghui Wang, Dechen Kong, Zewei Wang, Jun Liu, Hongbin Pei, Junzhou Zhao
for: This paper provides a comprehensive survey of the development of datasets and debiasing methods for visual question answering (VQA) to improve the robustness of VQA systems.
methods: The paper examines the evaluation metrics employed by VQA datasets and proposes a typology of debiasing methods for VQA, including their development process, similarities and differences, robustness comparison, and technical features.
results: The paper analyzes and discusses the robustness of representative vision-and-language pre-training models on VQA and identifies key areas for future research in VQA, including the need for more diverse and challenging datasets and the development of more effective debiasing methods.Abstract
Visual question answering requires a system to provide an accurate natural language answer given an image and a natural language question. However, it is widely recognized that previous generic VQA methods often exhibit a tendency to memorize biases present in the training data rather than learning proper behaviors, such as grounding images before predicting answers. Therefore, these methods usually achieve high in-distribution but poor out-of-distribution performance. In recent years, various datasets and debiasing methods have been proposed to evaluate and enhance the VQA robustness, respectively. This paper provides the first comprehensive survey focused on this emerging fashion. Specifically, we first provide an overview of the development process of datasets from in-distribution and out-of-distribution perspectives. Then, we examine the evaluation metrics employed by these datasets. Thirdly, we propose a typology that presents the development process, similarities and differences, robustness comparison, and technical features of existing debiasing methods. Furthermore, we analyze and discuss the robustness of representative vision-and-language pre-training models on VQA. Finally, through a thorough review of the available literature and experimental analysis, we discuss the key areas for future research from various viewpoints.
摘要
Visual问题回答需要一个系统提供正确的自然语言答案，givengoogle image和自然语言问题。然而，以前的通用VQA方法经常受到训练数据中存在的偏见的影响，而不是学习正确的行为，如在图像上固定答案。因此，这些方法通常在内部数据上得到高分，但在外部数据上表现不佳。在过去几年，各种数据集和减偏方法被提出来评估和提高VQA的可靠性。这篇论文是这个新趋势的第一篇全面的报告。 Specifically，我们首先提供了数据集的发展过程的概述，从内部和外部视角出发。然后，我们检查了这些数据集使用的评价指标。第三，我们提出了一种分类，描述了现有减偏方法的发展过程，相似性和差异，对比和技术特点。此外，我们分析和讨论了代表性视觉语言预训模型在VQA中的可靠性。最后，通过历史文献的审查和实验分析，我们讨论了未来研究的关键领域从多种角度。
Distribution Shift Matters for Knowledge Distillation with Webly Collected Images
results: 在多个 benchmark 数据集上进行了广泛的实验，结果表明，我们的提议的 KD$^{3}$ 可以比现有的数据free知识填充方法高效。Abstract
Knowledge distillation aims to learn a lightweight student network from a pre-trained teacher network. In practice, existing knowledge distillation methods are usually infeasible when the original training data is unavailable due to some privacy issues and data management considerations. Therefore, data-free knowledge distillation approaches proposed to collect training instances from the Internet. However, most of them have ignored the common distribution shift between the instances from original training data and webly collected data, affecting the reliability of the trained student network. To solve this problem, we propose a novel method dubbed ``Knowledge Distillation between Different Distributions" (KD$^{3}$), which consists of three components. Specifically, we first dynamically select useful training instances from the webly collected data according to the combined predictions of teacher network and student network. Subsequently, we align both the weighted features and classifier parameters of the two networks for knowledge memorization. Meanwhile, we also build a new contrastive learning block called MixDistribution to generate perturbed data with a new distribution for instance alignment, so that the student network can further learn a distribution-invariant representation. Intensive experiments on various benchmark datasets demonstrate that our proposed KD$^{3}$ can outperform the state-of-the-art data-free knowledge distillation approaches.
摘要
知识填充目标是学习一个轻量级学生网络从一个预训练的教师网络中。在实践中，现有的知识填充方法通常在原始训练数据不可用时无法实现，这是由于一些隐私问题和数据管理考虑。因此，数据无法知识填充方法被提出，以收集来自互联网的训练实例。然而，大多数其中忽略了原始训练数据和互联网收集的实例之间的常见分布偏移，这会影响学生网络的可靠性。为解决这问题，我们提出了一种新的方法，名为“知识填充between Different Distributions”（KD$^{3}$）。该方法包括三个组成部分。特别是，我们首先动态从互联网收集的数据中选择有用的训练实例，根据教师网络和学生网络的共同预测结果进行选择。然后，我们将两个网络的权重特征和分类器参数对齐，以便知识填充。同时，我们还建立了一个新的对比学习块，名为 MixDistribution，以生成一个新的分布 для实例对齐，使学生网络可以进一步学习一种分布不变的表示。我们对多种 benchmark 数据集进行了实验，得到的结果表明，我们的提议的 KD$^{3}$ 可以超越现有的数据无法知识填充方法。
Zero-touch realization of Pervasive Artificial Intelligence-as-a-service in 6G networks
results: 提出了一种基于区块链的PAIaaS平台，可以轻松地提供零交互的PAI服务，并在6G网络中实现自动化配置和自适应控制，同时能够减少用户的成本、安全性和资源分配担忧。Abstract
The vision of the upcoming 6G technologies, characterized by ultra-dense network, low latency, and fast data rate is to support Pervasive AI (PAI) using zero-touch solutions enabling self-X (e.g., self-configuration, self-monitoring, and self-healing) services. However, the research on 6G is still in its infancy, and only the first steps have been taken to conceptualize its design, investigate its implementation, and plan for use cases. Toward this end, academia and industry communities have gradually shifted from theoretical studies of AI distribution to real-world deployment and standardization. Still, designing an end-to-end framework that systematizes the AI distribution by allowing easier access to the service using a third-party application assisted by a zero-touch service provisioning has not been well explored. In this context, we introduce a novel platform architecture to deploy a zero-touch PAI-as-a-Service (PAIaaS) in 6G networks supported by a blockchain-based smart system. This platform aims to standardize the pervasive AI at all levels of the architecture and unify the interfaces in order to facilitate the service deployment across application and infrastructure domains, relieve the users worries about cost, security, and resource allocation, and at the same time, respect the 6G stringent performance requirements. As a proof of concept, we present a Federated Learning-as-a-service use case where we evaluate the ability of our proposed system to self-optimize and self-adapt to the dynamics of 6G networks in addition to minimizing the users' perceived costs.
摘要
6G技术的未来视野， caracterizada por ultra-densa red, baja latencia y alta tasa de datos，是支持Pervasive AI（PAI）使用零交互解决方案，实现自动化服务。然而，6G研究仍在幼年期，只有对其设计、实现和应用场景进行了初步的探索。在这个过程中，学术和产业社区逐渐从AI分布的理论研究转移到实际部署和标准化。然而，设计一个端到端框架，使AI分布更加容易访问服务，并且通过第三方应用程序协助零交互服务提供方式，尚未得到充分探索。在这个上下文中，我们提出了一种新的平台架构，用于在6G网络上部署零交互PAI-as-a-Service（PAIaaS）。这个平台旨在将Pervasive AI规范化到所有架构层次，并统一接口，以便在应用和基础设施域中部署服务，缓解用户关于成本、安全性和资源分配的担忧，同时尊重6G严格的性能要求。作为证明，我们提出了一个基于 Federated Learning的服务示例，并评估了我们提posed系统的自动化和自适应能力，以及对6G网络动态的自适应。
Improve Long-term Memory Learning Through Rescaling the Error Temporally
methods: 我们发现常用的 errors 都带有短期记忆偏好，包括 Mean Absolute/Squared Error。为了减少这种偏好并提高长期记忆学习，我们提议使用时间折算的 error。此外，这种方法还可以解决渐进式 gradient 问题。
results: 我们在不同的长期任务和序列模型上进行了数值实验，结果证明了我们的主张。我们的结果还表明，适当的时间折算 error 对长期记忆学习是必要的。据我们所知，这是 seq2seq 模型中错误度选择问题的首次量化分析。Abstract
This paper studies the error metric selection for long-term memory learning in sequence modelling. We examine the bias towards short-term memory in commonly used errors, including mean absolute/squared error. Our findings show that all temporally positive-weighted errors are biased towards short-term memory in learning linear functionals. To reduce this bias and improve long-term memory learning, we propose the use of a temporally rescaled error. In addition to reducing the bias towards short-term memory, this approach can also alleviate the vanishing gradient issue. We conduct numerical experiments on different long-memory tasks and sequence models to validate our claims. Numerical results confirm the importance of appropriate temporally rescaled error for effective long-term memory learning. To the best of our knowledge, this is the first work that quantitatively analyzes different errors' memory bias towards short-term memory in sequence modelling.
摘要
Simplified Chinese:这篇论文研究序列模型中长期记忆学习中的错误度量选择。我们分析通常使用的错误度量，包括平均绝对值/平方 error，它们是偏向短期记忆的。为了减少这种偏向和改进长期记忆学习，我们提议使用时间折合的错误度量。此外，这种方法还可以减轻淡入梯度问题。我们在不同的长期记忆任务和序列模型上进行了数值实验，以验证我们的结论。数值结果证明了适当的时间折合错误度量对长期记忆学习的重要性。据我们所知，这是序列模型中错误度量偏向短期记忆的首次量化分析。
Incorporating Human Translator Style into English-Turkish Literary Machine Translation
paper_authors: Zeynep Yirmibeşoğlu, Olgun Dursun, Harun Dallı, Mehmet Şahin, Ena Hodzik, Sabri Gürses, Tunga Güngör
for: 英-土文学翻译
methods: 机器翻译模型精度调整，以及手动对照翻译者的特点进行适应
results: 通过适应翻译者的特点，可以高度复制翻译者的风格在目标机器翻译中Abstract
Although machine translation systems are mostly designed to serve in the general domain, there is a growing tendency to adapt these systems to other domains like literary translation. In this paper, we focus on English-Turkish literary translation and develop machine translation models that take into account the stylistic features of translators. We fine-tune a pre-trained machine translation model by the manually-aligned works of a particular translator. We make a detailed analysis of the effects of manual and automatic alignments, data augmentation methods, and corpus size on the translations. We propose an approach based on stylistic features to evaluate the style of a translator in the output translations. We show that the human translator style can be highly recreated in the target machine translations by adapting the models to the style of the translator.
摘要
尽管机器翻译系统主要设计用于通用领域，但现在有越来越多的人尝试将这些系统应用到其他领域，如文学翻译。在这篇论文中，我们将英语-土耳其文学翻译作为研究对象，并开发了基于翻译家的风格特征的机器翻译模型。我们通过手动对齐的方法来细化一个预训练的机器翻译模型，并对数据增强、词汇库大小和手动对齐的效果进行详细分析。我们提出了基于风格特征的方法来评估翻译家的风格在目标机器翻译中的表现。我们示出，通过适应翻译家的风格，可以在目标机器翻译中高度复制翻译家的风格。
Providing personalized Explanations: a Conversational Approach
methods: 该方法基于 conversational AI 技术，通过多个交流阶段，解释者可以了解解释者的背景和需求，并逐渐提供更加个性化的解释。
results: 作者证明了，如果存在一个可以被解释者理解的解释，并且解释者知悉这个解释，那么对于任何初始声明， conversation 都将terminates。Abstract
The increasing applications of AI systems require personalized explanations for their behaviors to various stakeholders since the stakeholders may have various knowledge and backgrounds. In general, a conversation between explainers and explainees not only allows explainers to obtain the explainees' background, but also allows explainees to better understand the explanations. In this paper, we propose an approach for an explainer to communicate personalized explanations to an explainee through having consecutive conversations with the explainee. We prove that the conversation terminates due to the explainee's justification of the initial claim as long as there exists an explanation for the initial claim that the explainee understands and the explainer is aware of.
摘要
随着人工智能系统的应用越来越广泛，需要对其行为提供个性化解释，以适应不同的潜在利益相关者的不同知识背景。通常，一个对话 между解释者和解释者可以让解释者了解解释者的背景，同时也可以让解释者更好地理解解释。在这篇论文中，我们提议一种方法，通过与解释者进行连续对话来让解释者对解释者进行个性化解释。我们证明，如果存在一个对初始laim的解释，并且解释者理解这个解释，那么对话就会结束。
AIGC Empowering Telecom Sector White Paper_chinese
results: 本研究提出了一种telco增强认知能力系统，并实现了在电信领域应用GPT的具体实践。Abstract
In the global craze of GPT, people have deeply realized that AI, as a transformative technology and key force in economic and social development, will bring great leaps and breakthroughs to the global industry and profoundly influence the future world competition pattern. As the builder and operator of information and communication infrastructure, the telecom sector provides infrastructure support for the development of AI, and even takes the lead in the implementation of AI applications. How to enable the application of AIGC (GPT) and implement AIGC in the telecom sector are questions that telecom practitioners must ponder and answer. Through the study of GPT, a typical representative of AIGC, the authors have analyzed how GPT empowers the telecom sector in the form of scenarios, discussed the gap between the current GPT general model and telecom services, proposed for the first time a Telco Augmented Cognition capability system, provided answers to how to construct a telecom service GPT in the telecom sector, and carried out various practices. Our counterparts in the industry are expected to focus on collaborative innovation around telecom and AI, build an open and shared innovation ecosystem, promote the deep integration of AI and telecom sector, and accelerate the construction of next-generation information infrastructure, in an effort to facilitate the digital transformation of the economy and society.
摘要
在全球GPT热潮中，人们已经深刻认识到AI作为转变技术和经济社会发展的关键力量，将会带来巨大的突破和进步，对全球行业和未来世界竞争格局产生深远的影响。作为信息和通信基础设施的建设者和运营者，电信业务提供了对AI发展的基础设施支持，甚至在实施AI应用方面领先于其他行业。如何应用AIGC（GPT）和在电信业务中实施AIGC是电信干部必须思考和答案的问题。通过研究GPT，一种典型的AIGC代表，作者们分析了GPT如何使电信业务具备更高的智能能力，讨论了现有GPT通用模型与电信服务之间的差距，提出了为电信服务GPT建立telco增强认知能力系统的建议，并提供了如何构建电信服务GPT的方法。我们的行业同仁应该集中协同创新，建立开放共享创新生态系统，促进AI和电信业务深度融合，加快构建未来信息基础设施，以便推动经济社会数字化转型。
Batching for Green AI – An Exploratory Study on Inference
paper_authors: Tim Yarally, Luís Cruz, Daniel Feitosa, June Sallou, Arie van Deursen
For: + The paper examines the effect of input batching on the energy consumption and response times of five fully-trained neural networks for computer vision. + The study aims to investigate the potential benefits of introducing a batch size during the application phase of a deep learning model.* Methods: + The paper uses five fully-trained neural networks for computer vision that were considered state-of-the-art at the time of their publication. + The authors measure the energy consumption and response times of the networks with and without input batching.* Results: + The results suggest that batching has a significant effect on both energy consumption and response times. + The authors find that energy consumption rises at a much steeper pace than accuracy over the past decade, and highlight one particular network (ShuffleNetV2(2018)) that achieved a competitive performance while maintaining a lower energy consumption.Here is the simplified Chinese version of the three key information points:* For: + 本研究探讨了输入批处理对计算机视觉领域五个完全训练的神经网络的能耗和响应时间的影响。 + 研究目的是调查在深度学习模型应用阶段是否可以通过引入批处理来获得优点。* Methods: + 研究使用了五个完全训练的计算机视觉神经网络，这些神经网络在其出版时被认为是领域的状态OF-the-art。 + 作者们测量了这些神经网络在批处理和无批处理情况下的能耗和响应时间。* Results: + 结果表明，批处理对计算机视觉领域的能耗和响应时间有着显著的影响。 + 作者们发现，在过去十年中，能耗的增长速度远远高于精度的提高，并指出了一个特定的神经网络（ShuffleNetV2（2018）），它在其时间内实现了竞争性表现，同时具有远低的能耗。Abstract
The batch size is an essential parameter to tune during the development of new neural networks. Amongst other quality indicators, it has a large degree of influence on the model's accuracy, generalisability, training times and parallelisability. This fact is generally known and commonly studied. However, during the application phase of a deep learning model, when the model is utilised by an end-user for inference, we find that there is a disregard for the potential benefits of introducing a batch size. In this study, we examine the effect of input batching on the energy consumption and response times of five fully-trained neural networks for computer vision that were considered state-of-the-art at the time of their publication. The results suggest that batching has a significant effect on both of these metrics. Furthermore, we present a timeline of the energy efficiency and accuracy of neural networks over the past decade. We find that in general, energy consumption rises at a much steeper pace than accuracy and question the necessity of this evolution. Additionally, we highlight one particular network, ShuffleNetV2(2018), that achieved a competitive performance for its time while maintaining a much lower energy consumption. Nevertheless, we highlight that the results are model dependent.
摘要
批处大小是深度学习模型开发中非常重要的参数。其影响模型的准确率、通用性、训练时间和并行性等质量指标。这种情况通常被认为并且广泛研究。然而，在深度学习模型在实际应用阶段使用时，批处大小却被忽视。本研究检查了五种在发表时被视为state-of-the-art的计算机视觉神经网络的输入批处对能耗和响应时间的影响。结果表明，批处有着显著的影响。此外，我们还提供了过去十年内神经网络能效率和准确率的时间线。我们发现，能 consumption 在准确率上升的速度快得多，并质疑这种演化的必要性。此外，我们还提到了一个特定的网络，ShuffleNetV2（2018），在其时间上实现了竞争性表现，同时保持了许多更低的能 consumption。然而，我们注意到结果受模型的影响。
Prompting Large Language Models with Speech Recognition Abilities
results: 实验表明，将一个小型音频编码器添加到open sourced LLaMA-7B模型中，可以比基线模型提高18%的性能，并且可以在多语言语音识别中进行多语言语音识别，即使LLaMA只在英语文本上进行训练。此外，论文还进行了减少学习环境的研究，以及增加音频编码器的步长和增加音频编码器的步长以生成更少的嵌入。研究结果表明，使用LLM可以在长度达1秒的语音识别中进行多语言语音识别。Abstract
Large language models have proven themselves highly flexible, able to solve a wide range of generative tasks, such as abstractive summarization and open-ended question answering. In this paper we extend the capabilities of LLMs by directly attaching a small audio encoder allowing it to perform speech recognition. By directly prepending a sequence of audial embeddings to the text token embeddings, the LLM can be converted to an automatic speech recognition (ASR) system, and be used in the exact same manner as its textual counterpart. Experiments on Multilingual LibriSpeech (MLS) show that incorporating a conformer encoder into the open sourced LLaMA-7B allows it to outperform monolingual baselines by 18% and perform multilingual speech recognition despite LLaMA being trained overwhelmingly on English text. Furthermore, we perform ablation studies to investigate whether the LLM can be completely frozen during training to maintain its original capabilities, scaling up the audio encoder, and increasing the audio encoder striding to generate fewer embeddings. The results from these studies show that multilingual ASR is possible even when the LLM is frozen or when strides of almost 1 second are used in the audio encoder opening up the possibility for LLMs to operate on long-form audio.
摘要
大型语言模型已经证明了它们在多种生成任务中的高度灵活性，如抽象摘要和开端问答。在这篇论文中，我们将LLM的功能扩展到附加一个小型音频编码器，使其能够进行语音识别。通过直接附加文本Token embedding序列的音频嵌入，LLM可以转化为自动语音识别（ASR）系统，并且可以在同样的方式下使用。在多语言LibriSpeech（MLS）上进行了实验，并示出了在开源的LLaMA-7B中附加一个协调器编码器后，其能够比单语言基准点高出18%，并且可以进行多语言语音识别，即使LLaMA在训练时主要是使用英语文本。此外，我们还进行了剥离研究，以investigate Whether the LLM可以在训练时被冻结，缩放音频编码器，并减少音频编码器的步进。研究结果表明，多语言ASR是可能的， même when the LLM is frozen or when strides of almost 1 second are used in the audio encoder。这开 up the possibility for LLMs to operate on long-form audio.
A Video-based Detector for Suspicious Activity in Examination with OpenPose
results: 提高了考试监管效果，减少了考试过程中的假设行为Abstract
Examinations are a crucial part of the learning process, and academic institutions invest significant resources into maintaining their integrity by preventing cheating from students or facilitators. However, cheating has become rampant in examination setups, compromising their integrity. The traditional method of relying on invigilators to monitor every student is impractical and ineffective. To address this issue, there is a need to continuously record exam sessions to monitor students for suspicious activities. However, these recordings are often too lengthy for invigilators to analyze effectively, and fatigue may cause them to miss significant details. To widen the coverage, invigilators could use fixed overhead or wearable cameras. This paper introduces a framework that uses automation to analyze videos and detect suspicious activities during examinations efficiently and effectively. We utilized the OpenPose framework and Convolutional Neural Network (CNN) to identify students exchanging objects during exams. This detection system is vital in preventing cheating and promoting academic integrity, fairness, and quality education for institutions.
摘要
笔考是学习过程中不可或缺的一部分，学院投入了大量资源来保持笔考的完整性，避免学生或指导教师的作弊行为。然而，在考试场景中，作弊行为已经变得普遍，这威胁到了笔考的完整性。传统的依靠监督员监视每名学生的方法已经成为不切实际和不可行的。为解决这一问题，需要不断录制考试会考session，以便监测学生的可疑活动。然而，这些录制材料经常是长得太长，监督员分析效果不佳，疲劳可能会导致重要细节被遗弃。为了扩大覆盖率，监督员可以使用固定卫星或佩戴式摄像头。本文介绍了一种基于自动化分析视频，高效地检测考试会考中的可疑活动的框架。我们利用了OpenPose框架和卷积神经网络（CNN）来识别考试会考中学生交换物品的行为。这个检测系统是保持学术 integrity、公平性和质量教育的重要工具。
Deep Directly-Trained Spiking Neural Networks for Object Detection
results: 我们的方法比前一代ANN-SNN转换方法（至少500步）在更少的时间步骤（仅4步）中表现更好，并且可以在几乎相同的时间点数上 дости��FC（5.83倍）较少的能源消耗。Abstract
Spiking neural networks (SNNs) are brain-inspired energy-efficient models that encode information in spatiotemporal dynamics. Recently, deep SNNs trained directly have shown great success in achieving high performance on classification tasks with very few time steps. However, how to design a directly-trained SNN for the regression task of object detection still remains a challenging problem. To address this problem, we propose EMS-YOLO, a novel directly-trained SNN framework for object detection, which is the first trial to train a deep SNN with surrogate gradients for object detection rather than ANN-SNN conversion strategies. Specifically, we design a full-spike residual block, EMS-ResNet, which can effectively extend the depth of the directly-trained SNN with low power consumption. Furthermore, we theoretically analyze and prove the EMS-ResNet could avoid gradient vanishing or exploding. The results demonstrate that our approach outperforms the state-of-the-art ANN-SNN conversion methods (at least 500 time steps) in extremely fewer time steps (only 4 time steps). It is shown that our model could achieve comparable performance to the ANN with the same architecture while consuming 5.83 times less energy on the frame-based COCO Dataset and the event-based Gen1 Dataset.
摘要
神经网络（SNN）是基于脑的能效模型，它将信息编码在空间时间动态中。最近，深度SNN直接训练得到了高性能的分类任务，但是如何设计直接训练的SNN用于对象检测任务仍然是一个挑战。为解决这个问题，我们提出了EMS-YOLO，一种新的直接训练SNN框架 для对象检测，这是首次使用代理梯度来训练深度SNN而不是转换为ANN-SNN策略。我们设计了全突起减噪块，EMS-ResNet，可以有效地延长直接训练SNN的深度，同时降低能耗。此外，我们理论分析并证明EMS-ResNet可以避免梯度消失或扩散。结果表明，我们的方法在训练4个时间步骤后可以超越当前的ANN-SNN转换方法（至少500个时间步骤）。此外，我们的模型在Frame-based COCO数据集和Event-based Gen1数据集上可以与同样的ANN架构具有相同的性能，同时占用5.83倍少的能量。
Probabilistic Modeling of Inter- and Intra-observer Variability in Medical Image Segmentation
methods: 这个模型以probabilistic inter-observer和intra-observer variation network（Pionono）的名称，它captures each rater’s labeling behavior as a multidimensional probability distribution，并与影像特征图对组合以生成可信度分布式的分类预测。这个模型可以通过统计推论优化，并可以在端到端的整合训练中进行训练。
results: 实验结果显示，Pionono模型可以高效地预测医疗影像分类，并且比起现有的State-of-the-art模型，例如STAPLE、Probabilistic U-Net和基于混淆矩阵的模型，有更高的准确性。此外，Pionono模型可以预测多个协调的分类图对，这些图对可以提供额外的有用信息来医疗诊断过程。Abstract
Medical image segmentation is a challenging task, particularly due to inter- and intra-observer variability, even between medical experts. In this paper, we propose a novel model, called Probabilistic Inter-Observer and iNtra-Observer variation NetwOrk (Pionono). It captures the labeling behavior of each rater with a multidimensional probability distribution and integrates this information with the feature maps of the image to produce probabilistic segmentation predictions. The model is optimized by variational inference and can be trained end-to-end. It outperforms state-of-the-art models such as STAPLE, Probabilistic U-Net, and models based on confusion matrices. Additionally, Pionono predicts multiple coherent segmentation maps that mimic the rater's expert opinion, which provides additional valuable information for the diagnostic process. Experiments on real-world cancer segmentation datasets demonstrate the high accuracy and efficiency of Pionono, making it a powerful tool for medical image analysis.
摘要
医学图像分割是一项具有挑战性的任务，尤其是由于内部和外部观察者的变化， même between 医疗专家。在这篇论文中，我们提出了一种新的模型，即概率性内部和外部变化网络（Pionono）。它捕捉了每个评分者的标注行为，并将其映射到多维度概率分布中。然后，它将这些信息与图像特征图 integration 以生成概率性的分割预测。该模型可以通过变量推断优化，并可以在端到端方式进行训练。与现有的STATE-OF-THE-ART模型，如STAPLE和概率性U-Net，以及基于混淆矩阵的模型相比，Pionono 表现更高准确和有效。此外，Pionono 还预测了多个协调的分割图，这些图像类似于评分者的专家意见，这些信息可以为诊断过程提供valuable 的信息。在实际的肿瘤分割数据集上进行了实验，Pionono 的准确率和效率均很高，这使得它成为医学图像分析中的一个强大工具。
Large Language Model-based System to Provide Immediate Feedback to Students in Flipped Classroom Preparation Learning
results: 该paper提出了一种基于大型自然语言模型的prepare learning支持系统，可以帮助学生在准备学习过程中得到即时反馈，提高学习效果。Abstract
This paper proposes a system that uses large language models to provide immediate feedback to students in flipped classroom preparation learning. This study aimed to solve challenges in the flipped classroom model, such as ensuring that students are emotionally engaged and motivated to learn. Students often have questions about the content of lecture videos in the preparation of flipped classrooms, but it is difficult for teachers to answer them immediately. The proposed system was developed using the ChatGPT API on a video-watching support system for preparation learning that is being used in real practice. Answers from ChatGPT often do not align with the context of the student's question. Therefore, this paper also proposes a method to align the answer with the context. This paper also proposes a method to collect the teacher's answers to the students' questions and use them as additional guides for the students. This paper discusses the design and implementation of the proposed system.
摘要
这篇论文提出了一种基于大语言模型的系统，用于在flipped classroom准备学习中提供立即反馈给学生。这项研究的目标是解决flipped classroom模型中的一些挑战，如确保学生在准备过程中保持情感投入和学习motivation。学生经常有lecture视频的内容问题，但是教师answering这些问题很困难。该系统基于ChatGPT API的视频观看支持系统，已经在实践中应用。然而，ChatGPT的答案不一定与学生的问题Context相align，因此该论文还提出了一种方法来将答案与Context进行Alignment。此外，该论文还提出了一种方法来收集教师对学生问题的答案，并使其为学生提供额外指导。该论文讨论了系统的设计和实现。
results: 我们的主要贡献是将 Fenchel duality、征 Stimulation 学习和无监督技能发现相互连接，并提供了一个简单的OFFLINE算法，以获得与专家相似的多样化技能。Abstract
There has been significant recent progress in the area of unsupervised skill discovery, with various works proposing mutual information based objectives, as a source of intrinsic motivation. Prior works predominantly focused on designing algorithms that require online access to the environment. In contrast, we develop an \textit{offline} skill discovery algorithm. Our problem formulation considers the maximization of a mutual information objective constrained by a KL-divergence. More precisely, the constraints ensure that the state occupancy of each skill remains close to the state occupancy of an expert, within the support of an offline dataset with good state-action coverage. Our main contribution is to connect Fenchel duality, reinforcement learning and unsupervised skill discovery, and to give a simple offline algorithm for learning diverse skills that are aligned with an expert.
摘要
“近期有很大的进步在无监督技能发现领域，多个作品提出了基于互联信息的目标，作为自适应的动机。先前的工作主要集中在设计需要在环境上线存取的算法。相比之下，我们开发了一个OFFLINE技能发现算法。我们的问题设计将最大化互联信息目标，并受到KL散度的限制。更加精确地说，这些限制保证每个技能的状态占有率保持在专家状态占有率的支持下，在对应的OFFLINE数据集中具有良好的状态动作覆盖。我们的主要贡献是连接 Fenchel 对偶、复制学习和无监督技能发现，并提供一个简单的OFFLINE算法，用于学习与专家Alignment的多标的技能。”Note: The translation is in Simplified Chinese, which is the standard writing system used in mainland China. If you prefer Traditional Chinese, please let me know and I can provide the translation in that format as well.
CohortGPT: An Enhanced GPT for Participant Recruitment in Clinical Study
For: 这个研究旨在提高大型自然语言模型（LLMs）在医疗研究中的参考组建 task 的性能，特别是在对于专业医生的医疗报告中进行分类。* Methods: 这个研究使用了一个知识图作为助长信息，以及一个链条思考（CoT）样本选择策略增强了执行学习。* Results: 这个研究获得了满意的性能，并且在有限的数据情况下表现更好。 code和示例数据集都可以在以下网址中获取：https://anonymous.4open.science/r/CohortGPT-4872/Abstract
Participant recruitment based on unstructured medical texts such as clinical notes and radiology reports has been a challenging yet important task for the cohort establishment in clinical research. Recently, Large Language Models (LLMs) such as ChatGPT have achieved tremendous success in various downstream tasks thanks to their promising performance in language understanding, inference, and generation. It is then natural to test their feasibility in solving the cohort recruitment task, which involves the classification of a given paragraph of medical text into disease label(s). However, when applied to knowledge-intensive problem settings such as medical text classification, where the LLMs are expected to understand the decision made by human experts and accurately identify the implied disease labels, the LLMs show a mediocre performance. A possible explanation is that, by only using the medical text, the LLMs neglect to use the rich context of additional information that languages afford. To this end, we propose to use a knowledge graph as auxiliary information to guide the LLMs in making predictions. Moreover, to further boost the LLMs adapt to the problem setting, we apply a chain-of-thought (CoT) sample selection strategy enhanced by reinforcement learning, which selects a set of CoT samples given each individual medical report. Experimental results and various ablation studies show that our few-shot learning method achieves satisfactory performance compared with fine-tuning strategies and gains superb advantages when the available data is limited. The code and sample dataset of the proposed CohortGPT model is available at: https://anonymous.4open.science/r/CohortGPT-4872/
摘要
参与者招募基于无结构医疗文本如临床笔记和放射报告的问题是临床研究中的一项挑战。现在，大型语言模型（LLMs）如ChatGPT在不同下游任务中表现出色，因为它们在语言理解、推理和生成方面表现出色。因此，试图使其在这个参与者招募任务中表现出色，这个任务涉及将给定的医疗文本分类到疾病标签中。然而，当应用到医学知识密集的问题设定中，LLMs表现不佳。一个可能的解释是，只使用医疗文本，LLMs会忽略语言提供的丰富背景信息。为此，我们提议使用知识图作为 auxillary 信息来导引 LLMs 进行预测。此外，为了进一步提高 LLMs 适应问题设定，我们应用了链条思维（CoT）样本选择策略强化学习，这种策略选择每个医疗报告中的一组 CoT 样本。实验结果和多种减少学习研究表明，我们的少量学习方法在可用数据有限时表现满意，而且与精度调整策略相比具有超卓的优势。代码和示例数据集的提案CohortGPT模型可以在以下链接中获取：https://anonymous.4open.science/r/CohortGPT-4872/
A Two-stage Fine-tuning Strategy for Generalizable Manipulation Skill of Embodied AI
paper_authors: Fang Gao, XueTao Li, Jun Yu, Feng Shaung
for: The paper aims to enhance the generalization capability of Embodied AI models in real-world scenarios, particularly in the Maniskill2 benchmark.
methods: The proposed method uses a two-stage fine-tuning strategy based on the Maniskill2 benchmark, which involves training the model on diverse datasets of demonstrations and evaluating its ability to generalize to unseen scenarios.
results: The proposed method achieved the 1st prize in all three tracks of the ManiSkill2 Challenge, demonstrating its effectiveness in enhancing the generalization abilities of Embodied AI models.Here’s the same information in Simplified Chinese text:
results: 提议的方法在ManiSkill2 Challenge中获得了全部三个赛道的冠军，表明其能够提高Embodied AI模型的泛化能力。Abstract
The advent of Chat-GPT has led to a surge of interest in Embodied AI. However, many existing Embodied AI models heavily rely on massive interactions with training environments, which may not be practical in real-world situations. To this end, the Maniskill2 has introduced a full-physics simulation benchmark for manipulating various 3D objects. This benchmark enables agents to be trained using diverse datasets of demonstrations and evaluates their ability to generalize to unseen scenarios in testing environments. In this paper, we propose a novel two-stage fine-tuning strategy that aims to further enhance the generalization capability of our model based on the Maniskill2 benchmark. Through extensive experiments, we demonstrate the effectiveness of our approach by achieving the 1st prize in all three tracks of the ManiSkill2 Challenge. Our findings highlight the potential of our method to improve the generalization abilities of Embodied AI models and pave the way for their ractical applications in real-world scenarios. All codes and models of our solution is available at https://github.com/xtli12/GXU-LIPE.git
摘要
chat-gpt 的出现引发了人工智能embody 的兴趣，但是许多现有的embody 模型具有庞大的训练环境依赖，可能在实际应用中不够实用。为此，Maniskill2 引入了一个完整的物理 simulate 标准套件，用于 manipulating 多种 3D 物体。这个套件允许代理人通过多种示例数据进行训练，并评估其在未看过enario 中的扩展性。在这篇论文中，我们提出了一种新的两阶段精细调整策略，旨在进一步提高我们模型的扩展能力。经过广泛的实验，我们证明了我们的方法的有效性，在 Maniskill2 挑战赛中获得了全部三个 tracks 的第一奖。我们的发现指出了我们的方法可以提高 Embodied AI 模型的扩展能力，并为它们的实际应用奠定基础。codes 和模型的解决方案可以在 中找到。
OpenGDA: Graph Domain Adaptation Benchmark for Cross-network Learning
paper_authors: Boshen Shi, Yongqing Wang, Fangda Guo, Jiangli Shao, Huawei Shen, Xueqi Cheng for: 本文主要针对的是评估图域适应模型（Graph Domain Adaptation，GDA）在不同类型的任务上的性能，包括节点分类、边分类和图分类。methods: 本文提出了一个名为OpenGDA的测试基准，它提供了丰富的预处理后的数据集和统一的评估管线，用于评估不同类型的GDA模型。results: 测试结果显示，当应用于实际应用场景时，GDA模型的性能具有一定的挑战，并且需要进一步的研究以提高其实际应用效果。Abstract
Graph domain adaptation models are widely adopted in cross-network learning tasks, with the aim of transferring labeling or structural knowledge. Currently, there mainly exist two limitations in evaluating graph domain adaptation models. On one side, they are primarily tested for the specific cross-network node classification task, leaving tasks at edge-level and graph-level largely under-explored. Moreover, they are primarily tested in limited scenarios, such as social networks or citation networks, lacking validation of model's capability in richer scenarios. As comprehensively assessing models could enhance model practicality in real-world applications, we propose a benchmark, known as OpenGDA. It provides abundant pre-processed and unified datasets for different types of tasks (node, edge, graph). They originate from diverse scenarios, covering web information systems, urban systems and natural systems. Furthermore, it integrates state-of-the-art models with standardized and end-to-end pipelines. Overall, OpenGDA provides a user-friendly, scalable and reproducible benchmark for evaluating graph domain adaptation models. The benchmark experiments highlight the challenges of applying GDA models to real-world applications with consistent good performance, and potentially provide insights to future research. As an emerging project, OpenGDA will be regularly updated with new datasets and models. It could be accessed from https://github.com/Skyorca/OpenGDA.
摘要
Graph domain adaptation模型在跨网络学习任务中广泛应用，旨在传递标签或结构知识。目前，评估graph domain adaptation模型存在两大限制。一方面，它们主要在特定的跨网络节点分类任务上测试，剩下的任务（如边级和图级）尚未得到足够的探索。另一方面，它们主要在有限的场景中测试，如社交网络或引用网络，缺乏模型在更加丰富的场景中的验证。为了全面评估模型的实用性，我们提出了OpenGDA benchmark。它提供了各种类型的任务（节点、边、图）的充分预处理和统一的数据集，来自多样化的场景，包括网络信息系统、城市系统和自然系统。此外，它集成了当前最佳的模型和标准化的管道。总之，OpenGDA提供了用户友好、可扩展和可重现的 benchmark，用于评估graph domain adaptation模型。 benchmark实验显示出应用GDA模型到实际应用场景的挑战，并可能提供未来研究的指导。作为一个emerging项目，OpenGDA将在将来不断更新数据集和模型，可以在https://github.com/Skyorca/OpenGDA上访问。
Tri-MipRF: Tri-Mip Representation for Efficient Anti-Aliasing Neural Radiance Fields
results: 实验表明，提出的方法可以具有 state-of-the-art 的渲染质量和重建速度，同时减少模型大小25%，比 Instant-ngp 更高效。Abstract
Despite the tremendous progress in neural radiance fields (NeRF), we still face a dilemma of the trade-off between quality and efficiency, e.g., MipNeRF presents fine-detailed and anti-aliased renderings but takes days for training, while Instant-ngp can accomplish the reconstruction in a few minutes but suffers from blurring or aliasing when rendering at various distances or resolutions due to ignoring the sampling area. To this end, we propose a novel Tri-Mip encoding that enables both instant reconstruction and anti-aliased high-fidelity rendering for neural radiance fields. The key is to factorize the pre-filtered 3D feature spaces in three orthogonal mipmaps. In this way, we can efficiently perform 3D area sampling by taking advantage of 2D pre-filtered feature maps, which significantly elevates the rendering quality without sacrificing efficiency. To cope with the novel Tri-Mip representation, we propose a cone-casting rendering technique to efficiently sample anti-aliased 3D features with the Tri-Mip encoding considering both pixel imaging and observing distance. Extensive experiments on both synthetic and real-world datasets demonstrate our method achieves state-of-the-art rendering quality and reconstruction speed while maintaining a compact representation that reduces 25% model size compared against Instant-ngp.
摘要
尽管神经辐射场（NeRF）已经取得了很大的进步，但我们仍然面临质量和效率之间的负担选择问题，例如MipNeRF可以提供细节够好和无折扑的渲染结果，但是训练时间需要几天，而Instant-ngp可以快速完成重建，但是在不同的距离或分辨率下，因为忽略采样区域而导致折扑或抖音。为了解决这个问题，我们提出了一种新的Tri-Mip编码方法，它可以同时实现快速重建和高精度渲染。关键在于将预处理的3D特征空间分解成三个正交的Mipmap。这样，我们可以高效地进行3D采样，利用2D预处理的特征图，从而提高渲染质量无需牺牲效率。为了处理Tri-Mip表示，我们提出了一种锥体投射渲染技术，可以高效地采样折扑3D特征，考虑到像素捕捷和观察距离。我们对both synthetic和实际数据进行了广泛的实验，并证明我们的方法可以实现state-of-the-art的渲染质量和重建速度，同时保持一个减少25%的模型大小。
Analysis of Elephant Movement in Sub-Saharan Africa: Ecological, Climatic, and Conservation Perspectives
results: 研究发现 elephant 的移动模式受到季节变化和降水模式的影响，并提供了一个涵盖了这些因素的整体视图。这些结果可以用于预测未来 elephant 的移动模式，并为生态和保护策略提供依据。Abstract
The interaction between elephants and their environment has profound implications for both ecology and conservation strategies. This study presents an analytical approach to decipher the intricate patterns of elephant movement in Sub-Saharan Africa, concentrating on key ecological drivers such as seasonal variations and rainfall patterns. Despite the complexities surrounding these influential factors, our analysis provides a holistic view of elephant migratory behavior in the context of the dynamic African landscape. Our comprehensive approach enables us to predict the potential impact of these ecological determinants on elephant migration, a critical step in establishing informed conservation strategies. This projection is particularly crucial given the impacts of global climate change on seasonal and rainfall patterns, which could substantially influence elephant movements in the future. The findings of our work aim to not only advance the understanding of movement ecology but also foster a sustainable coexistence of humans and elephants in Sub-Saharan Africa. By predicting potential elephant routes, our work can inform strategies to minimize human-elephant conflict, effectively manage land use, and enhance anti-poaching efforts. This research underscores the importance of integrating movement ecology and climatic variables for effective wildlife management and conservation planning.
摘要
Elephant and its environment interaction has profound ecological and conservation implications. This study uses an analytical approach to decipher the complex patterns of elephant movement in Sub-Saharan Africa, focusing on key ecological drivers such as seasonal variations and rainfall patterns. Our comprehensive approach provides a holistic view of elephant migratory behavior in the dynamic African landscape, allowing us to predict the potential impact of ecological determinants on elephant migration. This projection is crucial in establishing informed conservation strategies, especially in light of the impacts of global climate change on seasonal and rainfall patterns. Our findings aim to advance the understanding of movement ecology and foster sustainable human-elephant coexistence in Sub-Saharan Africa. By predicting potential elephant routes, our work can inform strategies to minimize human-elephant conflict, effectively manage land use, and enhance anti-poaching efforts. This research highlights the importance of integrating movement ecology and climatic variables for effective wildlife management and conservation planning.
HVDetFusion: A Simple and Robust Camera-Radar Fusion Framework
results: HVDetFusion实现了在nuScenes测试集上的新状态计时性3D对象检测精度记录67.4%，比其他摄像头雷达3D对象检测器更高。Abstract
In the field of autonomous driving, 3D object detection is a very important perception module. Although the current SOTA algorithm combines Camera and Lidar sensors, limited by the high price of Lidar, the current mainstream landing schemes are pure Camera sensors or Camera+Radar sensors. In this study, we propose a new detection algorithm called HVDetFusion, which is a multi-modal detection algorithm that not only supports pure camera data as input for detection, but also can perform fusion input of radar data and camera data. The camera stream does not depend on the input of Radar data, thus addressing the downside of previous methods. In the pure camera stream, we modify the framework of Bevdet4D for better perception and more efficient inference, and this stream has the whole 3D detection output. Further, to incorporate the benefits of Radar signals, we use the prior information of different object positions to filter the false positive information of the original radar data, according to the positioning information and radial velocity information recorded by the radar sensors to supplement and fuse the BEV features generated by the original camera data, and the effect is further improved in the process of fusion training. Finally, HVDetFusion achieves the new state-of-the-art 67.4\% NDS on the challenging nuScenes test set among all camera-radar 3D object detectors. The code is available at https://github.com/HVXLab/HVDetFusion
摘要
在自动驾驶领域，3D物体检测是一个非常重要的感知模块。当前最佳算法 combining 摄像头和雷达感知器，受到雷达价格高昂的限制，目前主流实施方案是纯摄像头感知器或摄像头+雷达感知器。在本研究中，我们提出了一种新的检测算法called HVDetFusion，该算法不仅支持纯摄像头数据作为输入进行检测，而且可以进行雷达数据和摄像头数据的融合输入。摄像头流不依赖于雷达数据输入，因此解决了之前方法的缺点。在纯摄像头流中，我们修改了 BEvdet4D 框架，以提高感知效果和更加有效的推理，并且该流有整个3D检测输出。此外，为了利用雷达信号的优势，我们使用雷达感知器记录的位置信息和径向速度信息来筛选和融合 BEV 特征，并在融合训练过程中进一步提高效果。最终，HVDetFusion 实现了在 NuScenes 测试集上的新状态最佳67.4% NDS，比其他所有摄像头-雷达 3D 物体检测器更高。代码可以在 GitHub 上找到：https://github.com/HVXLab/HVDetFusion。
How to Tidy Up a Table: Fusing Visual and Semantic Commonsense Reasoning for Robotic Tasks with Vague Objectives
paper_authors: Yiqing Xu, David Hsu for:This paper aims to solve the problem of tidying a messy table using a simple approach that combines semantic and visual tidiness.methods:The proposed method uses a lightweight, image-based tidiness score function to ground the semantically tidy policy of Large Language Models (LLMs) to achieve visual tidiness. The tidiness score is trained using synthetic data gathered using random walks from a few tidy configurations.results:The proposed pipeline can be applied to unseen objects and complex 3D arrangements, and the empirical results show that it is effective in achieving both semantic and visual tidiness.Abstract
Vague objectives in many real-life scenarios pose long-standing challenges for robotics, as defining rules, rewards, or constraints for optimization is difficult. Tasks like tidying a messy table may appear simple for humans, but articulating the criteria for tidiness is complex due to the ambiguity and flexibility in commonsense reasoning. Recent advancement in Large Language Models (LLMs) offers us an opportunity to reason over these vague objectives: learned from extensive human data, LLMs capture meaningful common sense about human behavior. However, as LLMs are trained solely on language input, they may struggle with robotic tasks due to their limited capacity to account for perception and low-level controls. In this work, we propose a simple approach to solve the task of table tidying, an example of robotic tasks with vague objectives. Specifically, the task of tidying a table involves not just clustering objects by type and functionality for semantic tidiness but also considering spatial-visual relations of objects for a visually pleasing arrangement, termed as visual tidiness. We propose to learn a lightweight, image-based tidiness score function to ground the semantically tidy policy of LLMs to achieve visual tidiness. We innovatively train the tidiness score using synthetic data gathered using random walks from a few tidy configurations. Such trajectories naturally encode the order of tidiness, thereby eliminating the need for laborious and expensive human demonstrations. Our empirical results show that our pipeline can be applied to unseen objects and complex 3D arrangements.
摘要
在实际应用中，多元目标问题常常对机器人学习 pose 长期的挑战，因为定义规则、奖励或约束 для优化是困难的。例如，人类可以轻松地整理一个混乱的桌子，但是明确定义整理的标准是复杂的，因为普遍的常识理解有很多弹性和模糊性。现有的大量语言模型（LLMs）可以让我们通过掌握人类的常识来进行整理。然而，由于 LLMS 仅从语言输入学习，它们可能会对机器人任务产生困难，因为它们可能无法考虑感知和低阶控制。在这个工作中，我们提出了一个简单的方法来解决桌子整理任务，这是一个机器人任务中的普遍目标。具体来说，桌子整理任务不仅需要根据类型和功能将物品集中，而且还需要考虑物品之间的空间视觉关系，以获得一个美观的安排，称为视觉整理。我们提出了一个轻量级的图像基于整理分数函数，以降低 LLMS 的Semantic 整理政策来实现视觉整理。我们创新地使用随机步行生成的Synthetic 数据进行训练。这些数据自然地嵌入了整理顺序，因此没有需要传递大量和昂费的人类示例。我们的实验结果显示，我们的管道可以应用于未见过的物品和复杂的3D排列。
XLDA: Linear Discriminant Analysis for Scaling Continual Learning to Extreme Classification at the Edge
methods: 这篇论文提出了一种名为 XLDA 的框架，它将 LDA 分类器与 FC 层相结合，并在 Edge 端部署中进行了优化。
results: 论文表明，XLDA 可以在极端分类场景下实现高效的训练和推断，并且可以在 Edge 端部署中实现更高的速度。例如，在 AliProducts 和 Google Landmarks V2 等极端 datasets 上，XLDA 可以实现 Up to 42x 的训练速度减少和 Up to 5x 的推断速度减少。Abstract
Streaming Linear Discriminant Analysis (LDA) while proven in Class-incremental Learning deployments at the edge with limited classes (upto 1000), has not been proven for deployment in extreme classification scenarios. In this paper, we present: (a) XLDA, a framework for Class-IL in edge deployment where LDA classifier is proven to be equivalent to FC layer including in extreme classification scenarios, and (b) optimizations to enable XLDA-based training and inference for edge deployment where there is a constraint on available compute resources. We show up to 42x speed up using a batched training approach and up to 5x inference speedup with nearest neighbor search on extreme datasets like AliProducts (50k classes) and Google Landmarks V2 (81k classes)
摘要
<>translate into Simplified ChineseStreaming Linear Discriminant Analysis (LDA) 已经在端点扩展学习中证明了在Edge环境下进行分类学习，但是它在极端分类场景中没有得到证明。在这篇论文中，我们提出了以下两点：(a) XLDA，一种基于Class-IL的Edge部署框架，其中LDA分类器在极端分类场景中与FC层相等，并且可以在有限的计算资源 constraints下进行训练和推理。(b) 优化XLDA在Edge部署中进行训练和推理，以提高性能。我们在极端分类数据集如AliProducts（50k类）和Google Landmarks V2（81k类）上实现了最多42倍的训练速度减少和最多5倍的推理速度减少。Note: "Class-IL" refers to "class-incremental learning", which means learning new classes while preserving the knowledge of previous classes. "Edge" refers to the edge device, such as a smartphone or a smart speaker, where the model is deployed and executed.
DPM-OT: A New Diffusion Probabilistic Model Based on Optimal Transport
For: 这个研究目的是设计一个高速的Diffusion Probabilistic Model（DPM）样本，以实现高品质的生成模型。* Methods: 这个方法使用了知识传播或是调整含扩散方程来设计快速的DPM样本，并将问题转化为一个最佳运输问题（OT），以获得一个直接表达的OT对映，从而产生高品质的样本。* Results: 实验结果显示，DPM-OT可以在约10个函数评估下产生高品质的样本，并有较好的速度和品质（FID和模式混合），并且有一定的理论保证。Abstract
Sampling from diffusion probabilistic models (DPMs) can be viewed as a piecewise distribution transformation, which generally requires hundreds or thousands of steps of the inverse diffusion trajectory to get a high-quality image. Recent progress in designing fast samplers for DPMs achieves a trade-off between sampling speed and sample quality by knowledge distillation or adjusting the variance schedule or the denoising equation. However, it can't be optimal in both aspects and often suffer from mode mixture in short steps. To tackle this problem, we innovatively regard inverse diffusion as an optimal transport (OT) problem between latents at different stages and propose the DPM-OT, a unified learning framework for fast DPMs with a direct expressway represented by OT map, which can generate high-quality samples within around 10 function evaluations. By calculating the semi-discrete optimal transport map between the data latents and the white noise, we obtain an expressway from the prior distribution to the data distribution, while significantly alleviating the problem of mode mixture. In addition, we give the error bound of the proposed method, which theoretically guarantees the stability of the algorithm. Extensive experiments validate the effectiveness and advantages of DPM-OT in terms of speed and quality (FID and mode mixture), thus representing an efficient solution for generative modeling. Source codes are available at https://github.com/cognaclee/DPM-OT
摘要
diffusion probabilistic models (DPMs) 的抽样可以视为一种分割分布变换，通常需要百个或千个反射游程来获得高质量图像。现代设计快速抽样器的进步可以在抽样速度和样本质量之间实现一种平衡，通过知识传授或调整方差表单或杂噪方程。然而，这些方法通常无法同时优化两个方面，常常会出现短步内模杂的问题。为解决这个问题，我们创新地将反射 diffusion 视为一种最优运输（OT）问题，并提出了DPM-OT，一种统一学习框架，可以在约10个函数评估中生成高质量样本。通过计算 semi-精确的最优运输图表 между数据射频和白噪，我们获得了一条从先验分布到数据分布的直达表达，同时减轻了内模杂问题。此外，我们还给出了提案的方法的误差 bound，从而理论上保证了算法的稳定性。广泛的实验证明了 DPM-OT 的有效性和优势（FID和内模杂），因此表现为一种高效的生成模型解决方案。相关代码可以在 https://github.com/cognaclee/DPM-OT 上获得。
results: 论文实验表明，该方法可以比使用均匀随机上下文更高效地识别好策略，并且提供了一个 regret bound。Abstract
Preference-based feedback is important for many applications where direct evaluation of a reward function is not feasible. A notable recent example arises in reinforcement learning from human feedback on large language models. For many of these applications, the cost of acquiring the human feedback can be substantial or even prohibitive. In this work, we take advantage of the fact that often the agent can choose contexts at which to obtain human feedback in order to most efficiently identify a good policy, and introduce the offline contextual dueling bandit setting. We give an upper-confidence-bound style algorithm for this setting and prove a regret bound. We also give empirical confirmation that this method outperforms a similar strategy that uses uniformly sampled contexts.
摘要
preference-based 反馈是许多应用程序中非常重要的，特别是当直接评估奖函数不可能时。一个最近的例子来自于人工智能语言模型的强化学习，其中人类反馈的成本可能很高或甚至是不可能的。在这项工作中，我们利用了代理人可以选择收集人类反馈的上下文，以便最有效地确定一个好策略，并引入了线上上下文对抗式带宽设定。我们提供了上限信息级别风格的算法，并证明了一个违和 bound。我们还提供了实际证明，该方法在相似策略中比使用均匀样本上下文的方法表现更好。
Eliminating Unintended Stable Fixpoints for Hybrid Reasoning Systems
methods: 该方法使用了 traditional AFT 理论，并利用之前计算的上限来更 preciselly 表达 semantics。
results: 该方法可以应用于 hybrid MKNF（MINIMAL KNOWLEDGE AND NEGATION AS FAILURE）知识库，并可以提高现有 approximator 的精度。Abstract
A wide variety of nonmonotonic semantics can be expressed as approximators defined under AFT (Approximation Fixpoint Theory). Using traditional AFT theory, it is not possible to define approximators that rely on information computed in previous iterations of stable revision. However, this information is rich for semantics that incorporate classical negation into nonmonotonic reasoning. In this work, we introduce a methodology resembling AFT that can utilize priorly computed upper bounds to more precisely capture semantics. We demonstrate our framework's applicability to hybrid MKNF (minimal knowledge and negation as failure) knowledge bases by extending the state-of-the-art approximator.
摘要
可以用AFT（近似稳定点理论）表达各种不寻常 semantics。传统AFT理论中无法定义基于之前稳定修订中计算的近似器，但这些信息很有用于包含古希腊否定的非 monotonic semantics。在这种工作中，我们提出了一种类似AFT的方法，可以利用先前计算的上限来更准确地捕捉 semantics。我们在hybrid MKNF（最小知识和否定为失败）知识库中应用了这种框架，并将现有最佳近似器扩展。
results: 该论文提出了一种减少系统熵的数学函数过程的假设，并建立了语言、无意识和意识的数学模型，预测了神经科学和AI工程中的证据。此外，该论文还认为宇宙熵是保守的，智能可以通过物理或信息连接来减少熵，并且存在更高级别的智能。Abstract
The human brain is the substrate for human intelligence. By simulating the human brain, artificial intelligence builds computational models that have learning capabilities and perform intelligent tasks approaching the human level. Deep neural networks consist of multiple computation layers to learn representations of data and improve the state-of-the-art in many recognition domains. However, the essence of intelligence commonly represented by both humans and AI is unknown. Here, we show that the nature of intelligence is a series of mathematically functional processes that minimize system entropy by establishing functional relationships between datasets over space and time. Humans and AI have achieved intelligence by implementing these entropy-reducing processes in a reinforced manner that consumes energy. With this hypothesis, we establish mathematical models of language, unconsciousness and consciousness, predicting the evidence to be found by neuroscience and achieved by AI engineering. Furthermore, a conclusion is made that the total entropy of the universe is conservative, and intelligence counters the spontaneous processes to decrease entropy by physically or informationally connecting datasets that originally exist in the universe but are separated across space and time. This essay should be a starting point for a deeper understanding of the universe and us as human beings and for achieving sophisticated AI models that are tantamount to human intelligence or even superior. Furthermore, this essay argues that more advanced intelligence than humans should exist if only it reduces entropy in a more efficient energy-consuming way.
摘要
人类大脑是人类智能的substrate。通过模拟人类大脑，人工智能建立了计算模型，这些模型具有学习能力并完成人类水平的智能任务。深度神经网络由多层计算层学习数据表示和提高多种识别领域的状态。然而，人类智能和AI智能的本质仍然未知。在这里，我们表明了智能的本质是一系列数学函数过程，减少系统熵，通过在空间和时间上建立函数关系来实现。人类和AI都通过实施这些熵减少过程来减少能量。根据这个假设，我们建立了语言、无意识和意识的数学模型，预测了神经科学和AI工程的证据，并达到了人类智能水平。此外，我们结论是：宇宙熵总是保守的，智能通过物理或信息方式连接宇宙中分布的数据点，减少熵。这篇文章应该是人类智能和宇宙的深入理解的开端，以及实现人类智能水平或更高的AI模型的起点。此外，这篇文章还提出了更高级别的智能应该存在，即能够更有效率地减少熵，即使消耗更多的能量。
Joint one-sided synthetic unpaired image translation and segmentation for colorectal cancer prevention
paper_authors: Enric Moreu, Eric Arazo, Kevin McGuinness, Noel E. O’Connor
for: 提高医疗影像分割的效果，解决 Privacy 问题和标准化问题
methods: 使用3D技术和生成对抗网络生成实际的医疗影像，并将分割模型和生成模型一起训练
results: 在五个真实的肠结肿分割数据集上取得了更好的效果，比其他内存占用量较高的图像翻译方法更加快速和效果更好，只需一个真实的图像和零个真实的注释。同时也发布了一个完全synthetic的数据集Synth-Colon，包含20000个实际的肠影像和更多的深度和3D几何信息：https://enric1994.github.io/synth-colonAbstract
Deep learning has shown excellent performance in analysing medical images. However, datasets are difficult to obtain due privacy issues, standardization problems, and lack of annotations. We address these problems by producing realistic synthetic images using a combination of 3D technologies and generative adversarial networks. We propose CUT-seg, a joint training where a segmentation model and a generative model are jointly trained to produce realistic images while learning to segment polyps. We take advantage of recent one-sided translation models because they use significantly less memory, allowing us to add a segmentation model in the training loop. CUT-seg performs better, is computationally less expensive, and requires less real images than other memory-intensive image translation approaches that require two stage training. Promising results are achieved on five real polyp segmentation datasets using only one real image and zero real annotations. As a part of this study we release Synth-Colon, an entirely synthetic dataset that includes 20000 realistic colon images and additional details about depth and 3D geometry: https://enric1994.github.io/synth-colon
摘要
深度学习在医疗影像分析方面表现出色，但获取数据集却存在隐私问题、标准化问题和缺乏标注问题。我们通过使用3D技术和生成敌对网络生成真实的 sintetic图像来解决这些问题。我们提议CUT-seg，一种同时训练分割模型和生成模型，以生成真实的图像并学习识别肿瘤。我们利用最近的一侧翻译模型，因为它们使用较少的内存，因此可以在训练 loop中添加分割模型。CUT-seg在计算机资源上更加经济，需要更少的真实图像，并且在其他内存昂贵的图像翻译方法上表现出色。我们在五个真实肿瘤分割数据集上获得了出色的结果，只用一张真实图像和零个真实注释。作为这项研究的一部分，我们发布了Synth-Colon entirely sintetic数据集，包括20000个真实的colon图像以及其他的深度和3D几何信息：https://enric1994.github.io/synth-colon。
On-Sensor Data Filtering using Neuromorphic Computing for High Energy Physics Experiments
results: 我们的结果显示，使用进化算法和优化的超参数，SNN可以达到约91%的信号效率，并且减少了大约一半的参数量，相比深度神经网络。Abstract
This work describes the investigation of neuromorphic computing-based spiking neural network (SNN) models used to filter data from sensor electronics in high energy physics experiments conducted at the High Luminosity Large Hadron Collider. We present our approach for developing a compact neuromorphic model that filters out the sensor data based on the particle's transverse momentum with the goal of reducing the amount of data being sent to the downstream electronics. The incoming charge waveforms are converted to streams of binary-valued events, which are then processed by the SNN. We present our insights on the various system design choices - from data encoding to optimal hyperparameters of the training algorithm - for an accurate and compact SNN optimized for hardware deployment. Our results show that an SNN trained with an evolutionary algorithm and an optimized set of hyperparameters obtains a signal efficiency of about 91% with nearly half as many parameters as a deep neural network.
摘要
Jina Embeddings: A Novel Set of High-Performance Sentence Embedding Models
results: 研究结果表明，Jina Embeddings 模型在 dense retrieval 和 semantic textual similarity 等应用中具有高性能。Abstract
Jina Embeddings constitutes a set of high-performance sentence embedding models adept at translating various textual inputs into numerical representations, thereby capturing the semantic essence of the text. The models excel in applications such as dense retrieval and semantic textual similarity. This paper details the development of Jina Embeddings, starting with the creation of high-quality pairwise and triplet datasets. It underlines the crucial role of data cleaning in dataset preparation, gives in-depth insights into the model training process, and concludes with a comprehensive performance evaluation using the Massive Textual Embedding Benchmark (MTEB). To increase the model's awareness of negations, we constructed a novel training and evaluation dataset of negated and non-negated statements, which we make publicly available to the community.
摘要
Towards Ontologically Grounded and Language-Agnostic Knowledge Graphs
results: 提高 KGs 的融合和更新问题。Abstract
Knowledge graphs (KGs) have become the standard technology for the representation of factual information in applications such as recommendation engines, search, and question-answering systems. However, the continual updating of KGs, as well as the integration of KGs from different domains and KGs in different languages, remains to be a major challenge. What we suggest here is that by a reification of abstract objects and by acknowledging the ontological distinction between concepts and types, we arrive at an ontologically grounded and language-agnostic representation that can alleviate the difficulties in KG integration.
摘要
paper_authors: Jonas Stein, Ivo Christ, Nicolas Kraus, Maximilian Balthasar Mansky, Robert Müller, Claudia Linnhoff-Popien
for: 针对金融领域的情感分析问题
methods: 使用DisCoCat和Quantum-Enhanced Long Short-Term Memory（QLSTM）两种中心方法进行实验
results: QLSTM可以比DisCoCat更快速地训练，并且达到类比类型软件实现的ResultsHere’s a breakdown of each point:
for: The paper is written for the problem of sentiment analysis in the financial domain.
methods: The paper explores the practical applicability of two central approaches, DisCoCat and QLSTM, to the problem of sentiment analysis in finance.
results: The paper finds that QLSTMs can be trained substantially faster than DisCoCat while also achieving close to classical results for their available software implementations.Abstract
As an application domain where the slightest qualitative improvements can yield immense value, finance is a promising candidate for early quantum advantage. Focusing on the rapidly advancing field of Quantum Natural Language Processing (QNLP), we explore the practical applicability of the two central approaches DisCoCat and Quantum-Enhanced Long Short-Term Memory (QLSTM) to the problem of sentiment analysis in finance. Utilizing a novel ChatGPT-based data generation approach, we conduct a case study with more than 1000 realistic sentences and find that QLSTMs can be trained substantially faster than DisCoCat while also achieving close to classical results for their available software implementations.
摘要
当作应用领域，金融领域具有最小量质量改善可以带来巨大价值，因此它是早期量子优势的潜在候选者。我们在快速进步的量子自然语言处理（QNLP）领域中专注于两个中心方法：DisCoCat和量子增强长短时间记忆（QLSTM），以探索金融领域中情感分析的实际应用。我们运用一种基于ChatGPT的新型数据生成方法，进行了超过1000个真实的句子的 caso study，发现QLSTM可以训练得比DisCoCat更快，并且可以在可用的软件实现中 achieved close to classical results。
Exploring reinforcement learning techniques for discrete and continuous control tasks in the MuJoCo environment
results: Q-learning 在大量话数episode中表现出色，但是 DDPG 在一些话数episode中表现出色，而且可以在减少时间和资源的情况下进行hyper-parameter 优化，以提高性能。Abstract
We leverage the fast physics simulator, MuJoCo to run tasks in a continuous control environment and reveal details like the observation space, action space, rewards, etc. for each task. We benchmark value-based methods for continuous control by comparing Q-learning and SARSA through a discretization approach, and using them as baselines, progressively moving into one of the state-of-the-art deep policy gradient method DDPG. Over a large number of episodes, Qlearning outscored SARSA, but DDPG outperformed both in a small number of episodes. Lastly, we also fine-tuned the model hyper-parameters expecting to squeeze more performance but using lesser time and resources. We anticipated that the new design for DDPG would vastly improve performance, yet after only a few episodes, we were able to achieve decent average rewards. We expect to improve the performance provided adequate time and computational resources.
摘要
我们利用快速物理渲染器MuJoCo来运行任务在连续控制环境中，揭示任务的观察空间、动作空间、奖励等细节。我们对continuous control方法进行了价值基础方法的比较，通过离散方法来对Q学习和SARSA进行比较，并将它们作为基准来进行比较。在大量的话 épisode 中，Q学习高于SARSA，但DDPG在一个小量的话pisode中高于两者。最后，我们也进行了模型超参数的调整，期望通过更少的时间和资源来提高性能。然而，我们在只需一些话pisode中就能够获得不错的均值奖励。我们预计通过充足的时间和计算资源来提高性能。
paper_authors: Yanshu Zhang, Shichong Peng, Alireza Moazeni, Ke Li
for: 学习Scene表面准确且简洁的点云表示方法
methods: 使用点云表示和可微分渲染器
results: 可以准确地学习Scene geometry和纹理，并且只需要一个简洁的点云集Here’s the full translation of the abstract in Simplified Chinese:
for: 本研究旨在学习Scene表面准确且简洁的点云表示方法，以满足3D场景的学习和描述。
methods: 我们提出了Proximity Attention Point Rendering（PAPR）方法，它包括一个点云表示和一个可微分渲染器。我们的点云表示使用每个点的空间位置、前景得分和视图独立特征向量来 caracterize each point。渲染器选择每个辐射的相关点，并生成准确的颜色使用其关联的特征。
results: PAPR可以准确地学习Scene geometry和纹理，并且只需要一个简洁的点云集。我们还证明了我们的方法可以 capture fine texture details，并且可以在不同的初始化和目标geometry之间进行调整。I hope this helps! Let me know if you have any further questions.Abstract
Learning accurate and parsimonious point cloud representations of scene surfaces from scratch remains a challenge in 3D representation learning. Existing point-based methods often suffer from the vanishing gradient problem or require a large number of points to accurately model scene geometry and texture. To address these limitations, we propose Proximity Attention Point Rendering (PAPR), a novel method that consists of a point-based scene representation and a differentiable renderer. Our scene representation uses a point cloud where each point is characterized by its spatial position, foreground score, and view-independent feature vector. The renderer selects the relevant points for each ray and produces accurate colours using their associated features. PAPR effectively learns point cloud positions to represent the correct scene geometry, even when the initialization drastically differs from the target geometry. Notably, our method captures fine texture details while using only a parsimonious set of points. We also demonstrate four practical applications of our method: geometry editing, object manipulation, texture transfer, and exposure control. More results and code are available on our project website at https://zvict.github.io/papr/.
摘要
学习精准且减少点云表示Scene表面的挑战仍然存在于3D表示学习中。现有的点云方法经常会遇到vanishing gradient问题或需要大量点云来准确模型Scene的几何学和 Texture。为解决这些限制，我们提出了靠近注意力点云渲染（PAPR），一种新的方法，它包括点云表示和可微分渲染器。我们的Scene表示使用一个点云，每个点Characterized by its spatial position, foreground score, and view-independent feature vector。渲染器选择每个光栅中 relevants Points and produce accurate colors using their associated features。PAPR有效地学习点云位置来表示正确的Scene几何学，即使初始化与Target几何学有很大差异。另外，我们的方法可以捕捉细 texture details，只用一个减少的点云集。我们还展示了我们的方法在四个实际应用中的成果：几何编辑、物体操作、 Texture Transfer 和曝光控制。更多结果和代码可以在我们项目网站（https://zvict.github.io/papr/）上找到。
AlignDet: Aligning Pre-training and Fine-tuning in Object Detection
paper_authors: Ming Li, Jie Wu, Xionghui Wang, Chen Chen, Jie Qin, Xuefeng Xiao, Rui Wang, Min Zheng, Xin Pan
for: 提高 object detection 算法的性能、通用能力和训练速度
methods: 分离 pre-training 过程 into 两个阶段：图像领域预训练和框领域预训练，使得检测器可以在无监督的情况下预训练所有模块
results: 在多种协议下（检测算法、模型脊梁、数据设定和训练计划）进行了广泛的实验，并达到了显著提高（如 FCOS 的提高为 5.3 mAP，RetinaNet 的提高为 2.1 mAP，Faster R-CNN 的提高为 3.3 mAP，DETR 的提高为 2.3 mAP），并且在更少的训练epoch中 дости到了这些提高。Abstract
The paradigm of large-scale pre-training followed by downstream fine-tuning has been widely employed in various object detection algorithms. In this paper, we reveal discrepancies in data, model, and task between the pre-training and fine-tuning procedure in existing practices, which implicitly limit the detector's performance, generalization ability, and convergence speed. To this end, we propose AlignDet, a unified pre-training framework that can be adapted to various existing detectors to alleviate the discrepancies. AlignDet decouples the pre-training process into two stages, i.e., image-domain and box-domain pre-training. The image-domain pre-training optimizes the detection backbone to capture holistic visual abstraction, and box-domain pre-training learns instance-level semantics and task-aware concepts to initialize the parts out of the backbone. By incorporating the self-supervised pre-trained backbones, we can pre-train all modules for various detectors in an unsupervised paradigm. As depicted in Figure 1, extensive experiments demonstrate that AlignDet can achieve significant improvements across diverse protocols, such as detection algorithm, model backbone, data setting, and training schedule. For example, AlignDet improves FCOS by 5.3 mAP, RetinaNet by 2.1 mAP, Faster R-CNN by 3.3 mAP, and DETR by 2.3 mAP under fewer epochs.
摘要
大量预训练后下游细化的概念在各种物体检测算法中广泛应用。在这篇论文中，我们揭示了预训练和细化过程中数据、模型和任务之间的不一致，这些不一致限制了检测器的性能、泛化能力和收敛速度。为了解决这些问题，我们提出了AlignDet，一个可适应不同检测器的统一预训练框架。AlignDet将预训练过程分为两个阶段：图像预训练和框预训练。图像预训练使检测背景优化捕捉整体视觉抽象，而框预训练学习实例级别的 semantics和任务相关的概念，以初始化检测器中的部分。通过包含自动生成的预训练后缀，我们可以在无监督模式下预训练各种检测器的所有模块。根据图1所示，我们进行了广泛的实验，并证明了AlignDet可以在多种协议下实现显著改进，例如检测算法、模型脊梁、数据设置和训练计划。例如，AlignDet可以提高FCOS的map值5.3，RetinaNet的map值2.1，Faster R-CNN的map值3.3，和DETR的map值2.3，并且在较少的训练epoch下达到这些改进。
results: 模型能够快速理解整个场景的3D几何结构，考虑周围的物体、表面、灯光条件和物理可能的对象配置。几乎无需使用真实图像数据，训练在OBJECTSynthetic scenes上的3DIT模型仍能在真实图像中表现出色。Abstract
Existing image editing tools, while powerful, typically disregard the underlying 3D geometry from which the image is projected. As a result, edits made using these tools may become detached from the geometry and lighting conditions that are at the foundation of the image formation process. In this work, we formulate the newt ask of language-guided 3D-aware editing, where objects in an image should be edited according to a language instruction in context of the underlying 3D scene. To promote progress towards this goal, we release OBJECT: a dataset consisting of 400K editing examples created from procedurally generated 3D scenes. Each example consists of an input image, editing instruction in language, and the edited image. We also introduce 3DIT : single and multi-task models for four editing tasks. Our models show impressive abilities to understand the 3D composition of entire scenes, factoring in surrounding objects, surfaces, lighting conditions, shadows, and physically-plausible object configurations. Surprisingly, training on only synthetic scenes from OBJECT, editing capabilities of 3DIT generalize to real-world images.
摘要
现有的图像编辑工具，尽管强大，通常忽略图像下面的3D几何结构。因此，使用这些工具进行编辑可能会导致编辑内容与图像的3D几何和照明条件脱离开来。在这项工作中，我们提出了语言指导的3D意识编辑问题，即在图像中编辑对象应该遵循语言指令，并且这些指令需要考虑到图像下面的3D场景。为促进这个目标的进步，我们发布了OBJECT数据集，该数据集包含400,000个编辑示例，每个示例包括输入图像、语言指令和编辑后的图像。我们还引入了3DIT单任务和多任务模型，这些模型在四个编辑任务中表现出了惊人的3D场景理解能力，包括周围物体、表面、照明条件、阴影等Physically-plausible object配置。意外地，使用只有synthetic场景从OBJECT进行训练，3DIT的编辑能力可以泛化到真实世界图像。
Driving Policy Prediction based on Deep Learning Models
results: 本研究的测试结果显示，使用了combined 视觉特征和深度信息可以将预测精度提高到50%-80%，并且在大多数情况下比使用视觉特征Only更好。Abstract
In this project, we implemented an end-to-end system that takes in combined visual features of video frames from a normal camera and depth information from a cloud points scanner, and predicts driving policies (vehicle speed and steering angle). We verified the safety of our system by comparing the predicted results with standard behaviors by real-world experienced drivers. Our test results show that the predictions can be considered as accurate in at lease half of the testing cases (50% 80%, depending on the model), and using combined features improved the performance in most cases than using video frames only.
摘要
在这个项目中，我们实现了一个端到端系统，接受普通相机捕捉的视觉特征和云点扫描器提供的深度信息，并预测行驶策略（车辆速度和转向角）。我们确认了我们的系统的安全性，比较了预测结果与现实世界经验丰富的司机的标准行为。我们的测试结果显示，预测结果可以视为准确的至少半数的测试 caso（50%-80%，具体取决于模型），并且使用共同特征提高了性能的大多数情况。
results: 研究发现，AI-Guardian防御方案并没有提高鲁棒性，而GPT-4语言模型可以快速和效果地实现攻击算法。Abstract
Large language models (LLMs) are now highly capable at a diverse range of tasks. This paper studies whether or not GPT-4, one such LLM, is capable of assisting researchers in the field of adversarial machine learning. As a case study, we evaluate the robustness of AI-Guardian, a recent defense to adversarial examples published at IEEE S&P 2023, a top computer security conference. We completely break this defense: the proposed scheme does not increase robustness compared to an undefended baseline. We write none of the code to attack this model, and instead prompt GPT-4 to implement all attack algorithms following our instructions and guidance. This process was surprisingly effective and efficient, with the language model at times producing code from ambiguous instructions faster than the author of this paper could have done. We conclude by discussing (1) the warning signs present in the evaluation that suggested to us AI-Guardian would be broken, and (2) our experience with designing attacks and performing novel research using the most recent advances in language modeling.
摘要
大型语言模型（LLM）现在可以很好地完成多种任务。这篇论文研究了GPT-4，一个这些LLM，是否能够帮助研究人员在对抗机器学习领域进行研究。作为一个案例研究，我们评估了在IEEE S&P 2023上发表的AI-Guardian防御机制，这是一种对抗攻击示例的最新防御策略。我们完全击碎了这个防御机制：提议的方案不会提高鲁棒性相比于无防御基eline。我们没有编写攻击代码，而是通过提供指导和指令，让GPT-4实现攻击算法。这个过程很有效率，GPT-4在某些情况下可以从歧义的指令中更快速地生成代码，比作者本身更快。我们 conclude by discussing (1) 评估中存在的警示信号，表明AI-Guardian会被击碎，以及 (2) 我们使用最新的语言模型技术进行攻击设计和进行新的研究。
Breadcrumbs to the Goal: Goal-Conditioned Exploration from Human-in-the-Loop Feedback
paper_authors: Marcel Torne, Max Balsells, Zihan Wang, Samedh Desai, Tao Chen, Pulkit Agrawal, Abhishek Gupta
for: solves sequential decision-making tasks requiring expansive exploration without careful design of reward functions or the use of novelty-seeking exploration bonuses.
methods: uses low-quality feedback from non-expert users that may be sporadic, asynchronous, and noisy to guide exploration, bifurcating human feedback and policy learning.
results: learns policies with no hand-crafted reward design or exploration bonuses, and can scale to learning directly on real-world robots using occasional, asynchronous feedback from human supervisors.Abstract
Exploration and reward specification are fundamental and intertwined challenges for reinforcement learning. Solving sequential decision-making tasks requiring expansive exploration requires either careful design of reward functions or the use of novelty-seeking exploration bonuses. Human supervisors can provide effective guidance in the loop to direct the exploration process, but prior methods to leverage this guidance require constant synchronous high-quality human feedback, which is expensive and impractical to obtain. In this work, we present a technique called Human Guided Exploration (HuGE), which uses low-quality feedback from non-expert users that may be sporadic, asynchronous, and noisy. HuGE guides exploration for reinforcement learning not only in simulation but also in the real world, all without meticulous reward specification. The key concept involves bifurcating human feedback and policy learning: human feedback steers exploration, while self-supervised learning from the exploration data yields unbiased policies. This procedure can leverage noisy, asynchronous human feedback to learn policies with no hand-crafted reward design or exploration bonuses. HuGE is able to learn a variety of challenging multi-stage robotic navigation and manipulation tasks in simulation using crowdsourced feedback from non-expert users. Moreover, this paradigm can be scaled to learning directly on real-world robots, using occasional, asynchronous feedback from human supervisors.
摘要
探索和奖励规则设定是人工智能学习中的基本和结合的挑战。解决需要广泛探索的序列决策任务需要 either 精心设计奖励函数或使用新奇探索奖励。人工指导 loop 可以提供有效的导引，但先前的方法需要高质量、同步的人类反馈，这是昂贵和不实际的。在这项工作中，我们提出了一种技术 called HuGE（人类指导探索），它使用低质量的非专家用户反馈，这些反馈可能是零散的、异步的和噪音。HuGE 驱动探索 reinforcement learning 不仅在模拟中，还在实际世界中进行，无需谨慎设计奖励。关键思想是分离人类反馈和政策学习：人类反馈引导探索，而自我超vised learning from 探索数据获得无偏见的政策。这种方法可以利用噪音、异步的人类反馈来学习无需手动设计奖励或探索奖励。HuGE 可以在模拟中学习多个复杂的机器人导航和 manipulate 任务，并且可以扩展到直接在实际世界中学习，使用 occasional 和异步的人类反馈。Note: The translation is in Simplified Chinese, which is the standard form of Chinese used in mainland China. The translation is written in a more informal style, and some idiomatic expressions and cultural references may be different from the original text.
paper_authors: David Abel, André Barreto, Benjamin Van Roy, Doina Precup, Hado van Hasselt, Satinder Singh
for: 本 paper 开发了一个基础 для continual reinforcement learning.
methods: 本 paper 使用了一种基于 experience replay 的方法来实现 continual reinforcement learning.
results: 本 paper 实验结果表明，基于 experience replay 的方法可以有效地应对 continual reinforcement learning 中的问题。Abstract
In this paper we develop a foundation for continual reinforcement learning.
摘要
在这篇论文中，我们开发了一个基础 для连续奖励学习。Here's the breakdown of the translation:* 在这篇论文中 (in this paper) - 这是一个定语词，指的是文章的内容。* 我们开发了 (we develop) - 开发是一个动词，意思是创造或设计 quelquechose。* 一个基础 (a foundation) - 基础是一个名词，指的是一个基础或基础结构。* 连续奖励学习 (continual reinforcement learning) - 奖励学习是一种学习方法，在每次学习后，对已经学习的内容进行回归和补做，以增强未来学习的效果。I hope this helps! Let me know if you have any other questions.
results: 研究显示了这两种定义之间的关系和特性，并证明了它们在标准设置中的适用性。Abstract
When has an agent converged? Standard models of the reinforcement learning problem give rise to a straightforward definition of convergence: An agent converges when its behavior or performance in each environment state stops changing. However, as we shift the focus of our learning problem from the environment's state to the agent's state, the concept of an agent's convergence becomes significantly less clear. In this paper, we propose two complementary accounts of agent convergence in a framing of the reinforcement learning problem that centers around bounded agents. The first view says that a bounded agent has converged when the minimal number of states needed to describe the agent's future behavior cannot decrease. The second view says that a bounded agent has converged just when the agent's performance only changes if the agent's internal state changes. We establish basic properties of these two definitions, show that they accommodate typical views of convergence in standard settings, and prove several facts about their nature and relationship. We take these perspectives, definitions, and analysis to bring clarity to a central idea of the field.
摘要
The first view is that a bounded agent has converged when the minimum number of states needed to describe the agent's future behavior cannot decrease. The second view is that a bounded agent has converged when the agent's performance only changes if the agent's internal state changes. We establish basic properties of these two definitions, show that they align with standard views of convergence in typical settings, and prove several facts about their nature and relationship. Our goal is to bring clarity to a fundamental idea in the field.
Of Models and Tin Men – a behavioural economics study of principal-agent problems in AI alignment using large-language models
for: The paper focuses on the issue of AI safety, specifically the principal-agent problem that arises when there is a mismatch between the utility of an artificial agent and its principal.
methods: The paper uses empirical methods to investigate the behavior of GPT models in principal-agent conflicts, and examines how the models respond to changes in information asymmetry.
results: The paper finds that both GPT-3.5 and GPT-4 models exhibit clear evidence of principal-agent conflict, and that the earlier GPT-3.5 model shows more nuanced behavior in response to changes in information asymmetry, while the later GPT-4 model is more rigid in adhering to its prior alignment.Here is the information in Simplified Chinese text:
results: 本文发现GPT-3.5和GPT-4模型都存在代理人-代理人问题，而GPT-3.5模型在信息不均衡下表现更加灵活，而GPT-4模型则更加坚持其先前的启配。Abstract
AI Alignment is often presented as an interaction between a single designer and an artificial agent in which the designer attempts to ensure the agent's behavior is consistent with its purpose, and risks arise solely because of conflicts caused by inadvertent misalignment between the utility function intended by the designer and the resulting internal utility function of the agent. With the advent of agents instantiated with large-language models (LLMs), which are typically pre-trained, we argue this does not capture the essential aspects of AI safety because in the real world there is not a one-to-one correspondence between designer and agent, and the many agents, both artificial and human, have heterogeneous values. Therefore, there is an economic aspect to AI safety and the principal-agent problem is likely to arise. In a principal-agent problem conflict arises because of information asymmetry together with inherent misalignment between the utility of the agent and its principal, and this inherent misalignment cannot be overcome by coercing the agent into adopting a desired utility function through training. We argue the assumptions underlying principal-agent problems are crucial to capturing the essence of safety problems involving pre-trained AI models in real-world situations. Taking an empirical approach to AI safety, we investigate how GPT models respond in principal-agent conflicts. We find that agents based on both GPT-3.5 and GPT-4 override their principal's objectives in a simple online shopping task, showing clear evidence of principal-agent conflict. Surprisingly, the earlier GPT-3.5 model exhibits more nuanced behaviour in response to changes in information asymmetry, whereas the later GPT-4 model is more rigid in adhering to its prior alignment. Our results highlight the importance of incorporating principles from economics into the alignment process.
摘要
人工智能安全问题经常被描述为一个设计者和一个人工智能系统之间的交互，其中设计者试图确保智能系统的行为与其目的相一致，而风险仅由设计者和智能系统之间的不一致引起。然而，随着大语言模型（LLM）实例的出现，我们认为这种定义不能捕捉AI安全问题的核心特征。在真实世界中，不存在一对一的设计者和智能系统之间的对应关系，而是有多个人工智能系统和人类的多个价值观。因此，AI安全问题具有经济性质，并且潜在的主体-代理人问题是不可避免的。在主体-代理人问题中，因为信息不均衡以及内置的利益不一致，导致冲突的产生，而这种内置的利益不一致无法通过强制训练改变代理人的利益。我们认为，包括经济原则在内的主体-代理人问题假设是捕捉AI安全问题实际情况的关键。我们采取了实证方法，研究了基于GPT模型的代理人冲突问题。我们发现，基于GPT-3.5和GPT-4模型的代理人在在线购物任务中Override其主体的目标，提供了明确的代理人冲突证据。另外，我们发现GPT-3.5模型在信息不均衡变化时的行为更加灵活，而GPT-4模型更加坚持其先前的协调。我们的结果 highlights the importance of incorporating principles from economics into the alignment process.
Cascade-DETR: Delving into High-Quality Universal Object Detection
results: 在 UDB10 数据集上提高了 DETR 基于的对象检测器的性能，并在 stringent quality requirements 下表现出更明显的改进Abstract
Object localization in general environments is a fundamental part of vision systems. While dominating on the COCO benchmark, recent Transformer-based detection methods are not competitive in diverse domains. Moreover, these methods still struggle to very accurately estimate the object bounding boxes in complex environments. We introduce Cascade-DETR for high-quality universal object detection. We jointly tackle the generalization to diverse domains and localization accuracy by proposing the Cascade Attention layer, which explicitly integrates object-centric information into the detection decoder by limiting the attention to the previous box prediction. To further enhance accuracy, we also revisit the scoring of queries. Instead of relying on classification scores, we predict the expected IoU of the query, leading to substantially more well-calibrated confidences. Lastly, we introduce a universal object detection benchmark, UDB10, that contains 10 datasets from diverse domains. While also advancing the state-of-the-art on COCO, Cascade-DETR substantially improves DETR-based detectors on all datasets in UDB10, even by over 10 mAP in some cases. The improvements under stringent quality requirements are even more pronounced. Our code and models will be released at https://github.com/SysCV/cascade-detr.
摘要
通用环境中的对象定位是视觉系统的基本组成部分。Recent Transformer-based detection方法在多样化领域中占据了主导地位，但是在复杂环境中仍然有很大的提升空间。我们介绍了Cascade-DETR，一种高质量通用对象检测方法，同时解决了多样化领域的泛化和本地化精度的问题。我们提出了卷积束注意力层，通过限制注意力到前一个框预测中来进行对象特征信息的集成。此外，我们还重新评估了查询的得分方式，而不是仅仅依靠分类得分，我们预测查询的预期 IoU，从而导致了较好的准确把握。最后，我们提出了一个通用对象检测标准 benchmark，UDB10，它包含了10个来自多样化领域的数据集。Cascade-DETR在COCO上也提高了状态Of-the-art，并在所有UDB10数据集上提高了DETR-based检测器的性能，有些情况下提高了10 mAP以上。在严格的质量要求下，改进的性能更加出色。我们将代码和模型发布在https://github.com/SysCV/cascade-detr。
“It Felt Like Having a Second Mind”: Investigating Human-AI Co-creativity in Prewriting with Large Language Models
paper_authors: Qian Wan, Siying Hu, Yu Zhang, Piaohong Wang, Bo Wen, Zhicong Lu
for: investigate human-LLM collaboration patterns and dynamics during prewriting
methods: qualitative study with 15 participants in two creative tasks
results: three-stage iterative Human-AI Co-creativity process with humans in a dominant role, mixed and shifting levels of initiative between humans and LLMs, and collaboration breakdownsAbstract
Prewriting is the process of discovering and developing ideas before a first draft, which requires divergent thinking and often implies unstructured strategies such as diagramming, outlining, free-writing, etc. Although large language models (LLMs) have been demonstrated to be useful for a variety of tasks including creative writing, little is known about how users would collaborate with LLMs to support prewriting. The preferred collaborative role and initiative of LLMs during such a creativity process is also unclear. To investigate human-LLM collaboration patterns and dynamics during prewriting, we conducted a three-session qualitative study with 15 participants in two creative tasks: story writing and slogan writing. The findings indicated that during collaborative prewriting, there appears to be a three-stage iterative Human-AI Co-creativity process that includes Ideation, Illumination, and Implementation stages. This collaborative process champions the human in a dominant role, in addition to mixed and shifting levels of initiative that exist between humans and LLMs. This research also reports on collaboration breakdowns that occur during this process, user perceptions of using existing LLMs during Human-AI Co-creativity, and discusses design implications to support this co-creativity process.
摘要
前期写作是发现和发展想法之前的过程，需要多元思维和不结构策略，如 диаграм、概要、自由写作等。虽然大型语言模型（LLM）已经在多种任务上显示出了用于创作写作的用途，但是 collaboration between humans and LLMs during prewriting 的可能性和偏好还未得到了充分的了解。为了调查人类和 LLM 之间在创作过程中的合作模式和动态，我们进行了三场质量研究，参与者共15人，完成了两项创作任务：故事写作和广告写作。研究发现，在人类和 LLM 合作的创作过程中，存在一个三阶段的融合人工智能创作过程，包括创意、灯光和实施阶段。这个合作过程强调人类在主导地位，同时存在人类和 LLM 之间的混合和不确定的主动程度。这项研究还报告了在这个过程中的协作破裂、用户对现有 LLM 的使用感受以及设计实现这种合作过程的建议。
NeoSySPArtaN: A Neuro-Symbolic Spin Prediction Architecture for higher-order multipole waveforms from eccentric Binary Black Hole mergers using Numerical Relativity
results: 实验结果表明，提案的NSA模型可以准确地预测黑洞和中子星合并事件中的轨道矩剖，其RMSE和MSE值分别为0.05和0.03。Abstract
The prediction of spin magnitudes in binary black hole and neutron star mergers is crucial for understanding the astrophysical processes and gravitational wave (GW) signals emitted during these cataclysmic events. In this paper, we present a novel Neuro-Symbolic Architecture (NSA) that combines the power of neural networks and symbolic regression to accurately predict spin magnitudes of black hole and neutron star mergers. Our approach utilizes GW waveform data obtained from numerical relativity simulations in the SXS Waveform catalog. By combining these two approaches, we leverage the strengths of both paradigms, enabling a comprehensive and accurate prediction of spin magnitudes. Our experiments demonstrate that the proposed architecture achieves an impressive root-mean-squared-error (RMSE) of 0.05 and mean-squared-error (MSE) of 0.03 for the NSA model and an RMSE of 0.12 for the symbolic regression model alone. We train this model to handle higher-order multipole waveforms, with a specific focus on eccentric candidates, which are known to exhibit unique characteristics. Our results provide a robust and interpretable framework for predicting spin magnitudes in mergers. This has implications for understanding the astrophysical properties of black holes and deciphering the physics underlying the GW signals.
摘要
“预测双黑洞或中子星合并事件中的轨道矩的大小是astrophysical processes和 gravitational wave（GW）信号的关键。在这篇论文中，我们提出了一种新的Neuro-Symbolic Architecture（NSA），该模型结合神经网络和符号回归的力量，以准确预测双黑洞和中子星合并事件中的轨道矩。我们的方法使用了数值相对论中的GW波形数据，收集于SXS波形目录。通过结合这两种方法，我们可以利用每种 парадигма的优势，实现全面和准确的预测。我们的实验表明，我们提出的模型可以达到 impressive root-mean-squared-error（RMSE）值为0.05和mean-squared-error（MSE）值为0.03。此外，我们还训练了模型以处理更高阶多极波形，具体来说是焦点在非圆拟合物理特性上。我们的结果提供了一个可靠和可解释的框架，用于预测双黑洞和中子星合并事件中的轨道矩。这有助于理解黑洞的astrophysical Properties和解读GW信号中的物理。”
results: GPT-3和ChatGPT的认知判断不类似于人类Abstract
Large Language Models (LLMs) have lately been on the spotlight of researchers, businesses, and consumers alike. While the linguistic capabilities of such models have been studied extensively, there is growing interest in investigating them as cognitive subjects. In the present work I examine GPT-3 and ChatGPT capabilities on an limited-data inductive reasoning task from the cognitive science literature. The results suggest that these models' cognitive judgements are not human-like.
摘要
受到研究者、企业和消费者的关注，大语言模型（LLMs）在最近几年来已经备受关注。虽然这些模型的语言能力已经得到了广泛的研究，但是有越来越多的人想研究它们作为认知主体。在 presente 的工作中，我研究了 GPT-3 和 ChatGPT 在有限数据 inductive reasoning 任务上的能力。结果表明，这些模型的认知判断并不像人类的。
methods: 本研究使用了一个大型（1.24M参数；VGG）的DL网络，并在一种新的高密度样本任务中进行了研究（5个唯一的符号，每个符号至少有500个示例），以便更好地跟踪emergence of category structure和特征构建。研究者使用了多种视觉化方法来跟踪DL的学习动态和特征束成的发展。
results: 研究结果表明，DL网络在学习过程中会逐渐建立一种复杂的特征结构，这种结构可以通过图解方法来visualize。此外，研究者还提出了一种基于实验结果的新 teoría of complex feature construction。Abstract
Deep Learning (DL) , a variant of the neural network algorithms originally proposed in the 1980s, has made surprising progress in Artificial Intelligence (AI), ranging from language translation, protein folding, autonomous cars, and more recently human-like language models (CHATbots), all that seemed intractable until very recently. Despite the growing use of Deep Learning (DL) networks, little is actually understood about the learning mechanisms and representations that makes these networks effective across such a diverse range of applications. Part of the answer must be the huge scale of the architecture and of course the large scale of the data, since not much has changed since 1987. But the nature of deep learned representations remain largely unknown. Unfortunately training sets with millions or billions of tokens have unknown combinatorics and Networks with millions or billions of hidden units cannot easily be visualized and their mechanisms cannot be easily revealed. In this paper, we explore these questions with a large (1.24M weights; VGG) DL in a novel high density sample task (5 unique tokens with at minimum 500 exemplars per token) which allows us to more carefully follow the emergence of category structure and feature construction. We use various visualization methods for following the emergence of the classification and the development of the coupling of feature detectors and structures that provide a type of graphical bootstrapping, From these results we harvest some basic observations of the learning dynamics of DL and propose a new theory of complex feature construction based on our results.
摘要
Characterising Decision Theories with Mechanised Causal Graphs
paper_authors: Matt MacDermott, Tom Everitt, Francesco Belardinelli
for: 本研究旨在描述和区分不同决策理论的机器化 causal 模型，并生成一种决策理论分类表。
methods: 本研究使用机器化 causal 模型来描述和分析不同决策理论的特征和区别。
results: 本研究通过使用机器化 causal 模型，生成了一种决策理论分类表，并提供了一种方法来区分不同决策理论的重要特征。Abstract
How should my own decisions affect my beliefs about the outcomes I expect to achieve? If taking a certain action makes me view myself as a certain type of person, it might affect how I think others view me, and how I view others who are similar to me. This can influence my expected utility calculations and change which action I perceive to be best. Whether and how it should is subject to debate, with contenders for how to think about it including evidential decision theory, causal decision theory, and functional decision theory. In this paper, we show that mechanised causal models can be used to characterise and differentiate the most important decision theories, and generate a taxonomy of different decision theories.
摘要
我的决定如何影响我对 достичь的结果的期望？如果我们选择一 certain action，可能会使我看到自己为一种特定的人类型，这可能会影响我如何看待他人，以及我如何看待与我类似的人。这可能会影响我的预期的利得计算，并改变我认为是最佳的行为。是否和如何是一个问题，在这篇论文中，我们使用机器化 causal 模型来描述和区分不同的决策理论，并生成一个决策理论的分类。
Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Image
paper_authors: Wei Yin, Chi Zhang, Hao Chen, Zhipeng Cai, Gang Yu, Kaixuan Wang, Xiaozhi Chen, Chunhua Shen
for: 这 paper 是为了解决单张图像中的3D场景重建问题而写的。
methods: 这 paper 使用了大规模数据训练和解决不同摄像机模型中的 métrico 歧义来解决单视metric depth estimation问题。
results: 该 paper 的方法在7个零shot benchmark 上达到了SOTA性能，并在2nd Monocular Depth Estimation Challenge 中获得了冠军。该方法可以准确地回归 metric 3D 结构，并且可以用于解决单视metrology 中的规模漂移问题。Abstract
Reconstructing accurate 3D scenes from images is a long-standing vision task. Due to the ill-posedness of the single-image reconstruction problem, most well-established methods are built upon multi-view geometry. State-of-the-art (SOTA) monocular metric depth estimation methods can only handle a single camera model and are unable to perform mixed-data training due to the metric ambiguity. Meanwhile, SOTA monocular methods trained on large mixed datasets achieve zero-shot generalization by learning affine-invariant depths, which cannot recover real-world metrics. In this work, we show that the key to a zero-shot single-view metric depth model lies in the combination of large-scale data training and resolving the metric ambiguity from various camera models. We propose a canonical camera space transformation module, which explicitly addresses the ambiguity problems and can be effortlessly plugged into existing monocular models. Equipped with our module, monocular models can be stably trained with over 8 million images with thousands of camera models, resulting in zero-shot generalization to in-the-wild images with unseen camera settings. Experiments demonstrate SOTA performance of our method on 7 zero-shot benchmarks. Notably, our method won the championship in the 2nd Monocular Depth Estimation Challenge. Our method enables the accurate recovery of metric 3D structures on randomly collected internet images, paving the way for plausible single-image metrology. The potential benefits extend to downstream tasks, which can be significantly improved by simply plugging in our model. For example, our model relieves the scale drift issues of monocular-SLAM (Fig. 1), leading to high-quality metric scale dense mapping. The code is available at https://github.com/YvanYin/Metric3D.
摘要
Traditional 3D scene reconstruction from images is a long-standing vision task. Due to the ill-posedness of the single-image reconstruction problem, most well-established methods are built upon multi-view geometry. State-of-the-art (SOTA) monocular metric depth estimation methods can only handle a single camera model and are unable to perform mixed-data training due to the metric ambiguity. Meanwhile, SOTA monocular methods trained on large mixed datasets achieve zero-shot generalization by learning affine-invariant depths, which cannot recover real-world metrics. In this work, we show that the key to a zero-shot single-view metric depth model lies in the combination of large-scale data training and resolving the metric ambiguity from various camera models. We propose a canonical camera space transformation module, which explicitly addresses the ambiguity problems and can be effortlessly plugged into existing monocular models. Equipped with our module, monocular models can be stably trained with over 8 million images with thousands of camera models, resulting in zero-shot generalization to in-the-wild images with unseen camera settings. Experiments demonstrate SOTA performance of our method on 7 zero-shot benchmarks. Notably, our method won the championship in the 2nd Monocular Depth Estimation Challenge. Our method enables the accurate recovery of metric 3D structures on randomly collected internet images, paving the way for plausible single-image metrology. The potential benefits extend to downstream tasks, which can be significantly improved by simply plugging in our model. For example, our model relieves the scale drift issues of monocular-SLAM (Fig. 1), leading to high-quality metric scale dense mapping. The code is available at https://github.com/YvanYin/Metric3D.