results: 经验 validate了该方法的效果，在 Zhengzhou大学第一附属医院（FHZU）私人肿瘤数据集上实现了88.38%的Dice和10.64的 Hausdorff distance，比现有技术提高4%。Abstract
According to the 2021 World Health Organization (WHO) Classification scheme for gliomas, glioma segmentation is a very important basis for diagnosis and genotype prediction. In general, 3D multimodal brain MRI is an effective diagnostic tool. In the past decade, there has been an increase in the use of machine learning, particularly deep learning, for medical images processing. Thanks to the development of foundation models, models pre-trained with large-scale datasets have achieved better results on a variety of tasks. However, for medical images with small dataset sizes, deep learning methods struggle to achieve better results on real-world image datasets. In this paper, we propose a cross-modality attention adapter based on multimodal fusion to fine-tune the foundation model to accomplish the task of glioma segmentation in multimodal MRI brain images with better results. The effectiveness of the proposed method is validated via our private glioma data set from the First Affiliated Hospital of Zhengzhou University (FHZU) in Zhengzhou, China. Our proposed method is superior to current state-of-the-art methods with a Dice of 88.38% and Hausdorff distance of 10.64, thereby exhibiting a 4% increase in Dice to segment the glioma region for glioma treatment.
摘要
Note:* "glioma" is translated as "肿瘤" (tumor) in Simplified Chinese.* "WHO" is translated as "世界卫生组织" (World Health Organization) in Simplified Chinese.* "MRI" is translated as "Magnetic Resonance Imaging" in Simplified Chinese.* "deep learning" is translated as "深度学习" (deep learning) in Simplified Chinese.* "foundation model" is translated as "基础模型" (foundation model) in Simplified Chinese.* "multimodal fusion" is translated as "多Modal融合" (multimodal fusion) in Simplified Chinese.* "Dice" is translated as " dice" (Dice) in Simplified Chinese.* "Hausdorff distance" is translated as " Hausdorff distance" (Hausdorff distance) in Simplified Chinese.
Artifacts Mapping: Multi-Modal Semantic Mapping for Object Detection and 3D Localization
results: 实验显示，该方法可以准确探测98%的实际环境对象，而不需要后处理。相比单感器实验（camera或雷达），感器融合allow robot探测近距离和远距离障碍物，而单感器实验中的障碍物探测会受到干扰或精度问题。Abstract
Geometric navigation is nowadays a well-established field of robotics and the research focus is shifting towards higher-level scene understanding, such as Semantic Mapping. When a robot needs to interact with its environment, it must be able to comprehend the contextual information of its surroundings. This work focuses on classifying and localising objects within a map, which is under construction (SLAM) or already built. To further explore this direction, we propose a framework that can autonomously detect and localize predefined objects in a known environment using a multi-modal sensor fusion approach (combining RGB and depth data from an RGB-D camera and a lidar). The framework consists of three key elements: understanding the environment through RGB data, estimating depth through multi-modal sensor fusion, and managing artifacts (i.e., filtering and stabilizing measurements). The experiments show that the proposed framework can accurately detect 98% of the objects in the real sample environment, without post-processing, while 85% and 80% of the objects were mapped using the single RGBD camera or RGB + lidar setup respectively. The comparison with single-sensor (camera or lidar) experiments is performed to show that sensor fusion allows the robot to accurately detect near and far obstacles, which would have been noisy or imprecise in a purely visual or laser-based approach.
摘要
现代几何导航已成为机器人学的一个占据领域，研究重点正在转移到更高级的场景理解，如semantic mapping。当机器人需要与环境交互时，它必须能够理解它所处环境的Contextual信息。这项工作关注于在构建或已经构建的地图上分类和定位预定的对象，通过多Modal感知融合方法（结合RGB和深度数据从RGB-D相机和激光雷达）来自动检测和定位预定的对象。该框架由三个关键元素组成：通过RGB数据理解环境，通过多Modal感知融合估计深度，并处理缺陷（例如，过滤和稳定测量）。实验表明，提议的框架可以准确地检测98%的真实环境中的对象，不需要后处理，而使用单个RGBD相机或RGB+激光雷达设置时分别映射85%和80%的对象。与单感知器（相机或激光雷达）实验进行比较，显示了感知融合Allow robotaccurately检测靠近和远方障碍物，这些障碍物在视觉或激光基础approach中都会呈噪或不准确。
MeT: A Graph Transformer for Semantic Segmentation of 3D Meshes
paper_authors: Giuseppe Vecchio, Luca Prezzavento, Carmelo Pino, Francesco Rundo, Simone Palazzo, Concetto Spampinato for: 本研究旨在提出一种基于 transformer 的方法，用于semantic segmentation of 3D mesh。methods: 该方法使用了 global attention mechanism，以及 Laplacian eigenvectors 的位置编码和 clustering-based features，以提高模型对非sequential数据的处理和地方上下文的捕捉。results: 实验结果表明，该方法在三个Shape COSEG Dataset上的人 segmentation任务和ShapeNet benchmark上均达到了当前最佳性能。Abstract
Polygonal meshes have become the standard for discretely approximating 3D shapes, thanks to their efficiency and high flexibility in capturing non-uniform shapes. This non-uniformity, however, leads to irregularity in the mesh structure, making tasks like segmentation of 3D meshes particularly challenging. Semantic segmentation of 3D mesh has been typically addressed through CNN-based approaches, leading to good accuracy. Recently, transformers have gained enough momentum both in NLP and computer vision fields, achieving performance at least on par with CNN models, supporting the long-sought architecture universalism. Following this trend, we propose a transformer-based method for semantic segmentation of 3D mesh motivated by a better modeling of the graph structure of meshes, by means of global attention mechanisms. In order to address the limitations of standard transformer architectures in modeling relative positions of non-sequential data, as in the case of 3D meshes, as well as in capturing the local context, we perform positional encoding by means the Laplacian eigenvectors of the adjacency matrix, replacing the traditional sinusoidal positional encodings, and by introducing clustering-based features into the self-attention and cross-attention operators. Experimental results, carried out on three sets of the Shape COSEG Dataset, on the human segmentation dataset proposed in Maron et al., 2017 and on the ShapeNet benchmark, show how the proposed approach yields state-of-the-art performance on semantic segmentation of 3D meshes.
摘要
三角形网格已成为三维形状精确地 approximating 的标准方法，因为它们的效率和高灵活性可以 Capture 非均匀形状。然而，这种非均匀性会导致网格结构的不规则，使得三维网格分割变得特别困难。三维网格 semantic segmentation 通常通过基于 CNN 的方法进行解决，以至于达到较好的准确性。最近， transformers 在 NLP 和计算机视觉领域中得到了足够的动力，达到与 CNN 模型相当的性能，支持长期寻求的建筑通用性。以此为动力，我们提出一种基于 transformers 的三维网格 semantic segmentation 方法，通过全球注意力机制来更好地模型网格结构。为了解决标准 transformers 架构在非序数据中的模型非Sequential 数据的局限性，以及 capture 当地Context 的问题，我们使用劳拉cian 快速值来进行位置编码，取代传统的极值律动编码。此外，我们还引入 clustering-based 特征来自注意和跨注意操作中。实验结果，在三个Shape COSEG 数据集上进行了测试，以及 Maron et al. 2017 提出的人差分 segmentation 数据集和 ShapeNet benchmark，显示了我们提出的方法在三维网格 semantic segmentation 中达到了国际最佳性。
MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusion
results: 这 paper 的实验结果表明，MVDiffusion 可以高效地生成高分辨率的 photorealistic 图像，并且可以适应任意的文本提示或者将一个一视图图像扩展到 360 度的全景视图。Abstract
This paper introduces MVDiffusion, a simple yet effective method for generating consistent multi-view images from text prompts given pixel-to-pixel correspondences (e.g., perspective crops from a panorama or multi-view images given depth maps and poses). Unlike prior methods that rely on iterative image warping and inpainting, MVDiffusion simultaneously generates all images with a global awareness, effectively addressing the prevalent error accumulation issue. At its core, MVDiffusion processes perspective images in parallel with a pre-trained text-to-image diffusion model, while integrating novel correspondence-aware attention layers to facilitate cross-view interactions. For panorama generation, while only trained with 10k panoramas, MVDiffusion is able to generate high-resolution photorealistic images for arbitrary texts or extrapolate one perspective image to a 360-degree view. For multi-view depth-to-image generation, MVDiffusion demonstrates state-of-the-art performance for texturing a scene mesh. The project page is at https://mvdiffusion.github.io/.
摘要
At its core, MVDiffusion processes perspective images in parallel with a pre-trained text-to-image diffusion model, while integrating novel correspondence-aware attention layers to facilitate cross-view interactions. For panorama generation, while only trained with 10,000 panoramas, MVDiffusion is able to generate high-resolution photorealistic images for arbitrary texts or extrapolate one perspective image to a 360-degree view. For multi-view depth-to-image generation, MVDiffusion demonstrates state-of-the-art performance for texturing a scene mesh.More information can be found on the project page at .
UW-ProCCaps: UnderWater Progressive Colourisation with Capsules
results: 我们在四个benchmark数据集上进行了质量和量тив评估，结果显示，我们的解决方案在水下颜色重建 task中表现出色，比 estado-of-the-art（SOTA）解决方案更高效。此外，我们还证明了生成的颜色化提高了图像质量，比SOTA颜色提高模型更高效。Abstract
Underwater images are fundamental for studying and understanding the status of marine life. We focus on reducing the memory space required for image storage while the memory space consumption in the collecting phase limits the time lasting of this phase leading to the need for more image collection campaigns. We present a novel machine-learning model that reconstructs the colours of underwater images from their luminescence channel, thus saving 2/3 of the available storage space. Our model specialises in underwater colour reconstruction and consists of an encoder-decoder architecture. The encoder is composed of a convolutional encoder and a parallel specialised classifier trained with webly-supervised data. The encoder and the decoder use layers of capsules to capture the features of the entities in the image. The colour reconstruction process recalls the progressive and the generative adversarial training procedures. The progressive training gives the ground for a generative adversarial routine focused on the refining of colours giving the image bright and saturated colours which bring the image back to life. We validate the model both qualitatively and quantitatively on four benchmark datasets. This is the first attempt at colour reconstruction in greyscale underwater images. Extensive results on four benchmark datasets demonstrate that our solution outperforms state-of-the-art (SOTA) solutions. We also demonstrate that the generated colourisation enhances the quality of images compared to enhancement models at the SOTA.
摘要
水下图像是生物多样性研究的基础数据。我们的研究目标是减少图像存储空间，因为收集阶段的存储空间占用限制了收集阶段的时间，导致需要更多的图像收集campaign。我们提出了一种新的机器学习模型，可以从照明通道中重建水下图像的颜色，从而节省2/3的可用存储空间。我们的模型专注于水下颜色重建，并包括一个encoder-decoder架构。encoder包括一个 convolutional encoder 和一个并行特殊化分类器，并在网络环境中进行了监督学习。encoder和decoder都使用彩袋层来捕捉图像中的特征。颜色重建过程包括进程式和生成敌对训练过程。进程式训练给生成敌对训练提供了基础，并且通过对颜色进行精细调整，使图像变得更加生动和细腻。我们对四个标准数据集进行了质量和量化的验证，这是首次对灰度水下图像进行颜色重建。我们的解决方案在四个标准数据集上显示出比state-of-the-art（SOTA）解决方案更高的性能。此外，我们还证明了生成颜色化后的图像质量比SOTA的增强模型更高。
Streamlined Lensed Quasar Identification in Multiband Images via Ensemble Networks
results: 研究发现，通过 ensemble 这些 CNNs 和 ViTs，可以大幅减少假阳性源，并且在实际数据上具有较高的检测精度。最终，通过对 HSC 图像进行拼接，以及使用 UKIRT、VISTA 和 unWISE 数据，找到了约 600 万个源，并将其减少到 892,609 个。经过光度预选择，发现了 3080 个可能的强 gravitational lens 源，其中 210 个已经得到了观察confirmation。这些结果表明，自动化的深度学习管道可以有效地检测强 gravitational lensing，并减少人工视觉检查的时间和努力。Abstract
Quasars experiencing strong lensing offer unique viewpoints on subjects related to the cosmic expansion rate, the dark matter profile within the foreground deflectors, and the quasar host galaxies. Unfortunately, identifying them in astronomical images is challenging since they are overwhelmed by the abundance of non-lenses. To address this, we have developed a novel approach by ensembling cutting-edge convolutional networks (CNNs) -- for instance, ResNet, Inception, NASNet, MobileNet, EfficientNet, and RegNet -- along with vision transformers (ViTs) trained on realistic galaxy-quasar lens simulations based on the Hyper Suprime-Cam (HSC) multiband images. While the individual model exhibits remarkable performance when evaluated against the test dataset, achieving an area under the receiver operating characteristic curve of $>$97.3% and a median false positive rate of 3.6%, it struggles to generalize in real data, indicated by numerous spurious sources picked by each classifier. A significant improvement is achieved by averaging these CNNs and ViTs, resulting in the impurities being downsized by factors up to 50. Subsequently, combining the HSC images with the UKIRT, VISTA, and unWISE data, we retrieve approximately 60 million sources as parent samples and reduce this to 892,609 after employing a photometry preselection to discover $z>1.5$ lensed quasars with Einstein radii of $\theta_\mathrm{E}<5$ arcsec. Afterward, the ensemble classifier indicates 3080 sources with a high probability of being lenses, for which we visually inspect, yielding 210 prevailing candidates awaiting spectroscopic confirmation. These outcomes suggest that automated deep learning pipelines hold great potential in effectively detecting strong lenses in vast datasets with minimal manual visual inspection involved.
摘要
<<注意：以下文本使用了简化字体。>>Quasars经历强大的变形所提供了关于宇宙膨胀率、黑 mater profile within the foreground deflectors以及这些 Quasar host galaxies的独特视角。然而，在天文图像中 indentifying them 是具有挑战性的，因为它们被非强�的扩散所掩蔽。为了解决这个问题，我们开发了一种新的方法， combining cutting-edge convolutional neural networks (CNNs) 和视 transformers (ViTs) ，例如 ResNet、Inception、NASNet、MobileNet、EfficientNet 和 RegNet，并在基于 Hyper Suprime-Cam (HSC) 多频道图像的 simulated galaxy-quasar lens 中进行训练。尽管单个模型在测试数据集上表现出色，在 receiver operating characteristic curve 上取得了area > 97.3% 和 median false positive rate 为 3.6%，但它在实际数据中表现不佳，表现出许多假阳性。通过将这些 CNNs 和 ViTs ensemble，随后将 HSC 图像与 UKIRT、VISTA 和 unWISE 数据结合，我们检索了大约 6000万个源，并将其减少到 892,609 个后进行光度预选择，以找到 $z>1.5$ 强大变形透镜的候选者。然后，我们使用 ensemble classifier，并将其中 3080 个源作为高可能性强大变形透镜进行视觉检查，其中 210 个透镜被证明为主要候选者，等待spectroscopic confirmation。这些结果表明，通过自动的深度学习管道，可以快速和高效地检测强大变形透镜，并且可以避免大量的手动视觉检查。
Empirically Validating Conformal Prediction on Modern Vision Architectures Under Distribution Shift and Long-tailed Data
results: 在大规模数据集和模型上进行了实证评估，发现这些方法在分布偏移和长尾分布下表现不佳，常常违反保证I hope this helps! Let me know if you have any other questions.Abstract
Conformal prediction has emerged as a rigorous means of providing deep learning models with reliable uncertainty estimates and safety guarantees. Yet, its performance is known to degrade under distribution shift and long-tailed class distributions, which are often present in real world applications. Here, we characterize the performance of several post-hoc and training-based conformal prediction methods under these settings, providing the first empirical evaluation on large-scale datasets and models. We show that across numerous conformal methods and neural network families, performance greatly degrades under distribution shifts violating safety guarantees. Similarly, we show that in long-tailed settings the guarantees are frequently violated on many classes. Understanding the limitations of these methods is necessary for deployment in real world and safety-critical applications.
摘要
具有预测可靠性和安全保证的具体预测（conformal prediction）在深度学习模型中出现了，但其在分布变化和长尾类分布下的性能有所下降。我们对多种后处和训练基于的具体预测方法进行了大规模数据集和模型的实验性评估，发现在分布变化下，各种方法的性能强度下降，同时在长尾类分布下，保证 frequently 被违反。这些方法在实际应用中的部署和安全应用中的理解其限制是必要的。
Shi-NeSS: Detecting Good and Stable Keypoints with a Neural Stability Score
results: 在HPatches、ScanNet、MegaDepth和IMC-PT等测试集上，本方法表现出了顶尖的性能和良好的通用性，可以作为下游任务的基础功能。Abstract
Learning a feature point detector presents a challenge both due to the ambiguity of the definition of a keypoint and correspondingly the need for a specially prepared ground truth labels for such points. In our work, we address both of these issues by utilizing a combination of a hand-crafted Shi detector and a neural network. We build on the principled and localized keypoints provided by the Shi detector and perform their selection using the keypoint stability score regressed by the neural network - Neural Stability Score (NeSS). Therefore, our method is named Shi-NeSS since it combines the Shi detector and the properties of the keypoint stability score, and it only requires for training sets of images without dataset pre-labeling or the need for reconstructed correspondence labels. We evaluate Shi-NeSS on HPatches, ScanNet, MegaDepth and IMC-PT, demonstrating state-of-the-art performance and good generalization on downstream tasks.
摘要
Localized Questions in Medical Visual Question Answering
results: 实验结果表明，该方法比现有方法在三个数据集上表现更好，达到了更高的准确率。Note: The text is in Simplified Chinese, and the word order is adjusted to match the language convention.Abstract
Visual Question Answering (VQA) models aim to answer natural language questions about given images. Due to its ability to ask questions that differ from those used when training the model, medical VQA has received substantial attention in recent years. However, existing medical VQA models typically focus on answering questions that refer to an entire image rather than where the relevant content may be located in the image. Consequently, VQA models are limited in their interpretability power and the possibility to probe the model about specific image regions. This paper proposes a novel approach for medical VQA that addresses this limitation by developing a model that can answer questions about image regions while considering the context necessary to answer the questions. Our experimental results demonstrate the effectiveness of our proposed model, outperforming existing methods on three datasets. Our code and data are available at https://github.com/sergiotasconmorales/locvqa.
摘要
<>Translate given text into Simplified Chinese.<>干���VQA（Visual Question Answering）模型目标是回答给定图像的自然语言问题。由于它可以提出训练模型不同的问题，因此医学VQA在最近几年内得到了广泛的关注。然而，现有的医学VQA模型通常只会回答整个图像的问题，而不是在图像中具体的区域。这限制了VQA模型的解释力和可 probing 能力。这篇论文提议一种新的医学VQA方法，该方法可以回答图像区域的问题，同时考虑到问题的上下文。我们的实验结果表明，我们提出的方法可以超越现有方法，在三个数据集上达到更高的性能。我们的代码和数据可以在 GitHub 上找到：https://github.com/sergiotasconmorales/locvqa。
TomatoDIFF: On-plant Tomato Segmentation with Denoising Diffusion Models
paper_authors: Marija Ivanovska, Vitomir Struc, Janez Pers
for: 这项研究旨在提高 Tomatoes 的生长和生产，同时降低成本和环境影响。
methods: 该paper提出了一种新的Diffusion-based模型，用于semantic segmentation of on-plant Tomatoes。
results: 该模型在与其他竞争方法比较中显示出了SOTA表现，即使在受 occlusion 影响的环境中也能够达到优秀的结果。Abstract
Artificial intelligence applications enable farmers to optimize crop growth and production while reducing costs and environmental impact. Computer vision-based algorithms in particular, are commonly used for fruit segmentation, enabling in-depth analysis of the harvest quality and accurate yield estimation. In this paper, we propose TomatoDIFF, a novel diffusion-based model for semantic segmentation of on-plant tomatoes. When evaluated against other competitive methods, our model demonstrates state-of-the-art (SOTA) performance, even in challenging environments with highly occluded fruits. Additionally, we introduce Tomatopia, a new, large and challenging dataset of greenhouse tomatoes. The dataset comprises high-resolution RGB-D images and pixel-level annotations of the fruits.
摘要
人工智能应用程序可以帮助农民优化作物生长和生产，同时降低成本和环境影响。计算机视觉算法在特别是广泛应用于水果分割，以便进行深入分析丰收质量和准确的受量估计。在这篇论文中，我们提议了一种新的扩散模型，即TomatoDIFF，用于 semantic segmentation of on-plant tomatoes。与其他竞争方法进行比较后，我们的模型在具有高度受阻物的环境中仍然达到了当前最佳性能（SOTA）。此外，我们还介绍了一个新的大型和挑战性较高的greenhouse tomatoes数据集，该数据集包括高分辨率RGB-D图像和每个像素级别的水果标注。
Cross-modal Place Recognition in Image Databases using Event-based Sensors
results: 本文的实验结果表明，相比 frame-based 和事件基于方法，跨Modal视觉位置认识框架可以在不同场景下达到较高的性能，并且可以通过组合检索和分类来进一步提高性能。Abstract
Visual place recognition is an important problem towards global localization in many robotics tasks. One of the biggest challenges is that it may suffer from illumination or appearance changes in surrounding environments. Event cameras are interesting alternatives to frame-based sensors as their high dynamic range enables robust perception in difficult illumination conditions. However, current event-based place recognition methods only rely on event information, which restricts downstream applications of VPR. In this paper, we present the first cross-modal visual place recognition framework that is capable of retrieving regular images from a database given an event query. Our method demonstrates promising results with respect to the state-of-the-art frame-based and event-based methods on the Brisbane-Event-VPR dataset under different scenarios. We also verify the effectiveness of the combination of retrieval and classification, which can boost performance by a large margin.
摘要
Visual place recognition是重要的global localization问题中的一个关键问题，它可能受到环境照明或外观变化的影响。事件摄像头是替代frame-based感知器的有趣选择，因为它们的高动态范围使得在困难的照明条件下进行稳定的感知。然而，当前的事件基本Visual place recognition方法仅仅基于事件信息，这限制了下游应用程序的可能性。在这篇论文中，我们提出了首个可以从事件库中提取常见图像的跨模态Visual place recognition框架。我们的方法在不同的场景下与现状技术相比，对Brisbane-Event-VPR数据集表现出了promising的结果。我们还验证了抽象和分类的组合可以提高性能的幅度。
SAM-DA: UAV Tracks Anything at Night with SAM-Powered Domain Adaptation
results: 实验结果表明，SAM-DA可以在夜间无人机视频中提高夜间跟踪的效果，并且比目前状态ixen DA更好地适应夜间环境。此外，SAM-DA只需要 fewer nighttime images 来训练，这使得算法的验证和部署变得更加容易。Abstract
Domain adaptation (DA) has demonstrated significant promise for real-time nighttime unmanned aerial vehicle (UAV) tracking. However, the state-of-the-art (SOTA) DA still lacks the potential object with accurate pixel-level location and boundary to generate the high-quality target domain training sample. This key issue constrains the transfer learning of the real-time daytime SOTA trackers for challenging nighttime UAV tracking. Recently, the notable Segment Anything Model (SAM) has achieved remarkable zero-shot generalization ability to discover abundant potential objects due to its huge data-driven training approach. To solve the aforementioned issue, this work proposes a novel SAM-powered DA framework for real-time nighttime UAV tracking, i.e., SAM-DA. Specifically, an innovative SAM-powered target domain training sample swelling is designed to determine enormous high-quality target domain training samples from every single raw nighttime image. This novel one-to-many method significantly expands the high-quality target domain training sample for DA. Comprehensive experiments on extensive nighttime UAV videos prove the robustness and domain adaptability of SAM-DA for nighttime UAV tracking. Especially, compared to the SOTA DA, SAM-DA can achieve better performance with fewer raw nighttime images, i.e., the fewer-better training. This economized training approach facilitates the quick validation and deployment of algorithms for UAVs. The code is available at https://github.com/vision4robotics/SAM-DA.
摘要
域 adaptation (DA) 已经表现出了实时夜间无人机 (UAV) 跟踪的显著搭配潜力。然而，当前的状态势 (SOTA) DA 仍然缺乏高质量像素级别位置和边沿来生成高质量目标域训练样本。这个关键问题限制了日间高质量跟踪器的转移学习。最近， Segment Anything Model (SAM) 已经实现了各种 Zero-shot 通用能力，可以快速发现丰富的可能对象。为解决这个问题，本工作提出了一种基于 SAM 的 DA 框架，即 SAM-DA。特别是，我们提出了一种创新的 SAM 驱动的目标域训练样本扩展方法，可以从每个 Raw 夜间图像中找到巨大数量的高质量目标域训练样本。这种一对多的方法可以显著扩展高质量目标域训练样本，从而提高 DA 的性能。归根结底，我们的 SAM-DA 方法可以在夜间 UAV 跟踪中提供更好的性能，只需要 fewer 个 Raw 夜间图像，即 fewer-better 训练。这种经济的训练方法可以促进算法的快速验证和部署。代码可以在 GitHub 上找到：https://github.com/vision4robotics/SAM-DA。
CGAM: Click-Guided Attention Module for Interactive Pathology Image Segmentation via Backpropagating Refinement
results: 实验结果表明，与现有状态艺术方法相比，我们的方法在病理图像数据集上表现出了更好的性能。Abstract
Tumor region segmentation is an essential task for the quantitative analysis of digital pathology. Recently presented deep neural networks have shown state-of-the-art performance in various image-segmentation tasks. However, because of the unclear boundary between the cancerous and normal regions in pathology images, despite using modern methods, it is difficult to produce satisfactory segmentation results in terms of the reliability and accuracy required for medical data. In this study, we propose an interactive segmentation method that allows users to refine the output of deep neural networks through click-type user interactions. The primary method is to formulate interactive segmentation as an optimization problem that leverages both user-provided click constraints and semantic information in a feature map using a click-guided attention module (CGAM). Unlike other existing methods, CGAM avoids excessive changes in segmentation results, which can lead to the overfitting of user clicks. Another advantage of CGAM is that the model size is independent of input image size. Experimental results on pathology image datasets indicated that our method performs better than existing state-of-the-art methods.
摘要
受体区域分割是生物图像分析中的一项重要任务。最近提出的深度神经网络已经在各种图像分割任务中显示出了前所未有的性能。然而，由于生理图像中的悬浮边缘不清晰，使用现代方法仍然很难得到具有医疗数据所需的可靠性和准确性的分割结果。在这项研究中，我们提出了一种互动分割方法，允许用户通过点击类型的交互来修正深度神经网络的输出。我们的方法是通过点击约束和 semantic 信息在特征图上的协同使用点击导航模块（CGAM）来形式互动分割问题。与其他现有方法不同，CGAM 不会导致用户点击的过多修改，从而避免了点击过拟合。此外，CGAM 的模型大小独立于输入图像大小。在生物图像数据集上进行了实验，我们的方法比现有的状态艺前方法表现更好。
SynthCal: A Synthetic Benchmarking Pipeline to Compare Camera Calibration Algorithms
results: 实验结果显示SynthCal可以实现高精度的摄像机参数估计，并且可以评估不同的摄像机参数估计算法和实验室环境。Abstract
Accurate camera calibration is crucial for various computer vision applications. However, measuring camera parameters in the real world is challenging and arduous, and there needs to be a dataset with ground truth to evaluate calibration algorithms' accuracy. In this paper, we present SynthCal, a synthetic camera calibration benchmarking pipeline that generates images of calibration patterns to measure and enable accurate quantification of calibration algorithm performance in camera parameter estimation. We present a SynthCal-generated calibration dataset with four common patterns, two camera types, and two environments with varying view, distortion, lighting, and noise levels. The dataset evaluates single-view calibration algorithms by measuring reprojection and root-mean-square errors for identical patterns and camera settings. Additionally, we analyze the significance of different patterns using Zhang's method, which estimates intrinsic and extrinsic camera parameters with known correspondences between 3D points and their 2D projections in different configurations and environments. The experimental results demonstrate the effectiveness of SynthCal in evaluating various calibration algorithms and patterns.
摘要
准确的摄像头准确化是许多计算机视觉应用中的关键。然而，在实际世界中测量摄像头参数是困难和辛苦的，而且需要一个具有真实数据的数据集来评估准确化算法的准确性。在这篇论文中，我们提出了SynthCal，一个人工生成的摄像头准确化测试平台，可以生成准确度测试图案，以便评估和精确量化各种准确化算法的性能。我们提供了SynthCal生成的准确化数据集，包括四种常见的图案、两种摄像头类型和两种环境，其中包括不同的视角、扭曲、照明和噪声水平。这个数据集用于评估单视准确化算法，并且可以量化投影和平均方差误差。此外，我们还分析了不同图案的重要性，使用张氏方法，该方法可以在不同的配置和环境中估计摄像头的内参和外参参数，并且可以确定图案和摄像头设置之间的唯一对应关系。实验结果表明，SynthCal可以准确地评估各种准确化算法和图案。
Joint Coordinate Regression and Association For Multi-Person Pose Estimation, A Pure Neural Network Approach
results: 广泛的实验表明，JCRA在MS COCO和CrowdPosebenchmark上表现出色，并且在精度和效率两个方面都超越了现有的状态对照方法。具体来说，JCRA在MS COCO上达到了69.2 mAP，并且在推理加速方面比前状态的底层方法提高了78%。Abstract
We introduce a novel one-stage end-to-end multi-person 2D pose estimation algorithm, known as Joint Coordinate Regression and Association (JCRA), that produces human pose joints and associations without requiring any post-processing. The proposed algorithm is fast, accurate, effective, and simple. The one-stage end-to-end network architecture significantly improves the inference speed of JCRA. Meanwhile, we devised a symmetric network structure for both the encoder and decoder, which ensures high accuracy in identifying keypoints. It follows an architecture that directly outputs part positions via a transformer network, resulting in a significant improvement in performance. Extensive experiments on the MS COCO and CrowdPose benchmarks demonstrate that JCRA outperforms state-of-the-art approaches in both accuracy and efficiency. Moreover, JCRA demonstrates 69.2 mAP and is 78\% faster at inference acceleration than previous state-of-the-art bottom-up algorithms. The code for this algorithm will be publicly available.
摘要
我们介绍了一种新的一stage端到端多人2D姿态估计算法，称为共同坐标回归和关联（JCRA），该算法可以生成人体姿态关节和关联而无需任何后处理。我们提出的算法具有快速、高精度、有效和简单的特点。我们的一stage端到端网络架构可以显著提高JCRA的推理速度。此外，我们设计了对 encryptor和解码器的同质网络结构，以确保高精度在标记关节点的identification。这种架构直接通过 transformer 网络输出部分位置，从而得到了显著的性能提高。我们对 MS COCO 和 CrowdPose 测试准则进行了广泛的实验，并证明了 JCRA 在精度和效率两个方面超过了当前状态的抗下方法。此外，JCRA 在推理加速方面的提速率为 78%。代码将公开 availability。
Predicting beauty, liking, and aesthetic quality: A comparative analysis of image databases for visual aesthetics research
results: 研究发现不同图像集的美学评价预测结果具有substantial的变化，但是包含照片或画作的图像集具有相似的特征，表明不同的图像类别具有不同的美学评价特征。尽管统计图像特征和卷积神经网络具有相似的预测精度，但是不同的图像集的差异提出了对前期研究结果的总化和普遍性的困难。研究强调了在实验和计算美学领域中考虑多个图像集，以提高研究结果的有效性和普遍性。Abstract
In the fields of Experimental and Computational Aesthetics, numerous image datasets have been created over the last two decades. In the present work, we provide a comparative overview of twelve image datasets that include aesthetic ratings (beauty, liking or aesthetic quality) and investigate the reproducibility of results across different datasets. Specifically, we examine how consistently the ratings can be predicted by using either (A) a set of 20 previously studied statistical image properties, or (B) the layers of a convolutional neural network developed for object recognition. Our findings reveal substantial variation in the predictability of aesthetic ratings across the different datasets. However, consistent similarities were found for datasets containing either photographs or paintings, suggesting different relevant features in the aesthetic evaluation of these two image genres. To our surprise, statistical image properties and the convolutional neural network predict aesthetic ratings with similar accuracy, highlighting a significant overlap in the image information captured by the two methods. Nevertheless, the discrepancies between the datasets call into question the generalizability of previous research findings on single datasets. Our study underscores the importance of considering multiple datasets to improve the validity and generalizability of research results in the fields of experimental and computational aesthetics.
摘要
在实验和计算艺术领域，过去二十年内已经创建了许多图像集。在 presente 工作中，我们提供了十二个图像集的比较概述，这些图像集包括美学评分（美丽、喜欢或艺术质量）。我们发现这些图像集的美学评分是不一致的，但是包含照片或画作的图像集具有相似的特征。这些结果表明了使用不同方法来预测美学评分的可靠性。我们发现使用20种已知的统计图像特征和 convolutional neural network 对象识别的层次结构都可以准确预测美学评分，这显示了这两种方法 capture 了图像信息的重要方面。然而，由于不同图像集之间的差异，这casts doubt 到 previous research 的普遍性和可靠性。我们的研究强调了在实验和计算艺术领域中考虑多个图像集以提高研究结果的有效性和普遍性。
Autism Spectrum Disorder Classification in Children based on Structural MRI Features Extracted using Contrastive Variational Autoencoder
results: 研究实现了在儿童ASD（年龄范围：0.92-4.83岁）中的机器分类精度（超过0.97），并通过对不同 cortical区域表面积和s-MRI特征之间的相关性进行神经анаatomical解释，探讨了可能的ASD治疗目标。Abstract
Autism spectrum disorder (ASD) is a highly disabling mental disease that brings significant impairments of social interaction ability to the patients, making early screening and intervention of ASD critical. With the development of the machine learning and neuroimaging technology, extensive research has been conducted on machine classification of ASD based on structural MRI (s-MRI). However, most studies involve with datasets where participants' age are above 5. Few studies conduct machine classification of ASD for participants below 5-year-old, but, with mediocre predictive accuracy. In this paper, we push the boundary of predictive accuracy (above 0.97) of machine classification of ASD in children (age range: 0.92-4.83 years), based on s-MRI features extracted using contrastive variational autoencoder (CVAE). 78 s-MRI, collected from Shenzhen Children's Hospital, are used for training CVAE, which consists of both ASD-specific feature channel and common shared feature channel. The ASD participants represented by ASD-specific features can be easily discriminated from TC participants represented by the common shared features, leading to high classification accuracy. In case of degraded predictive accuracy when data size is extremely small, a transfer learning strategy is proposed here as a potential solution. Finally, we conduct neuroanatomical interpretation based on the correlation between s-MRI features extracted from CVAE and surface area of different cortical regions, which discloses potential biomarkers that could help target treatments of ASD in the future.
摘要
自适应症 спектルム疾病 (ASD) 是一种非常影响社交交流能力的精神疾病，对患者的早期检测和 intervención 至关重要。随着机器学习和神经成像技术的发展，有广泛的研究在机器分类 ASD 基于结构MRI（s-MRI）。然而，大多数研究参与者的年龄都大于 5 岁。只有一些研究进行了机器分类 ASD 的参与者下限为 5 岁，但它们的预测精度不太高。在这篇论文中，我们使用对比变量自动编码器（CVAE）提取的 s-MRI 特征来提高机器分类 ASD 的预测精度（高于 0.97），并且在儿童（年龄范围：0.92-4.83 岁）中进行了研究。我们使用了78 个 s-MRI，从深圳儿童医院收集到，用于训练 CVAE，CVAE 包括 ASD 特有的特征通道和共同分享的特征通道。ASD 参与者通过 ASD 特有的特征被轻松地与TC参与者（通过共同分享的特征）进行分开，从而导致高的分类精度。在数据量非常小时，降低预测精度的情况下，我们提出了传输学习策略作为可能的解决方案。最后，我们通过对 CVAE 提取的 s-MRI 特征和不同 cortical 区域表面积之间的相关性进行神经 анаatomical 解释，揭示了可能的生物标志物，这些生物标志物可能帮助未来对 ASD 进行精细的target treatments。
MoVie: Visual Model-Based Policy Adaptation for View Generalization
methods: 作者提出了一种简单 yet effective 的方法，通过在测试时使用模型来启用视模型基于策略的 $\textbf{MoVie}$ adaptation，无需显式奖励信号和任何修改 durante 训练时间。
results: 研究表明，该方法在四种不同的场景下（包括 DMControl、xArm 和 Adroit 中的 $\textbf{18}$ 个任务）具有显著的进步，相对于原始方法，提高了 $\mathbf{33}$%、$\mathbf{86}$% 和 $\mathbf{152}$%。这些出色的结果表明该方法在实际 робо扮中具有极大的潜力。视频可以在 https://yangsizhe.github.io/MoVie/ 上查看。Abstract
Visual Reinforcement Learning (RL) agents trained on limited views face significant challenges in generalizing their learned abilities to unseen views. This inherent difficulty is known as the problem of $\textit{view generalization}$. In this work, we systematically categorize this fundamental problem into four distinct and highly challenging scenarios that closely resemble real-world situations. Subsequently, we propose a straightforward yet effective approach to enable successful adaptation of visual $\textbf{Mo}$del-based policies for $\textbf{Vie}$w generalization ($\textbf{MoVie}$) during test time, without any need for explicit reward signals and any modification during training time. Our method demonstrates substantial advancements across all four scenarios encompassing a total of $\textbf{18}$ tasks sourced from DMControl, xArm, and Adroit, with a relative improvement of $\mathbf{33}$%, $\mathbf{86}$%, and $\mathbf{152}$% respectively. The superior results highlight the immense potential of our approach for real-world robotics applications. Videos are available at https://yangsizhe.github.io/MoVie/ .
摘要
Visible Reinforcement Learning（RL）代理人在有限视角下训练的情况下面临普遍化学习能力的挑战。这种问题被称为“视图普遍化问题”。在这项工作中，我们系统地将这个基本问题分为四种特点强大且具有挑战性的情况，与实际情况很相似。然后，我们提议一种简单 yet有效的方法，可以在测试时使Visual模型基于策略扩展到未经见过的视图，不需要显式奖励信号和任何修改 durante el entrenamiento。我们的方法在四种情况下表现出了显著的进步，涵盖了DMControl、xArm和Adroit中的18个任务，相对于原始方法提高了33%、86%和152%。这些出色的结果表明我们的方法在实际 робо特应用中具有极大的潜力。视频可以在https://yangsizhe.github.io/MoVie/ 中找到。
HODINet: High-Order Discrepant Interaction Network for RGB-D Salient Object Detection
results: 对 seven widely used 数据集进行了广泛的实验，并在四个评价指标上达到了竞争性的表现Abstract
RGB-D salient object detection (SOD) aims to detect the prominent regions by jointly modeling RGB and depth information. Most RGB-D SOD methods apply the same type of backbones and fusion modules to identically learn the multimodality and multistage features. However, these features contribute differently to the final saliency results, which raises two issues: 1) how to model discrepant characteristics of RGB images and depth maps; 2) how to fuse these cross-modality features in different stages. In this paper, we propose a high-order discrepant interaction network (HODINet) for RGB-D SOD. Concretely, we first employ transformer-based and CNN-based architectures as backbones to encode RGB and depth features, respectively. Then, the high-order representations are delicately extracted and embedded into spatial and channel attentions for cross-modality feature fusion in different stages. Specifically, we design a high-order spatial fusion (HOSF) module and a high-order channel fusion (HOCF) module to fuse features of the first two and the last two stages, respectively. Besides, a cascaded pyramid reconstruction network is adopted to progressively decode the fused features in a top-down pathway. Extensive experiments are conducted on seven widely used datasets to demonstrate the effectiveness of the proposed approach. We achieve competitive performance against 24 state-of-the-art methods under four evaluation metrics.
摘要
Towards Building Self-Aware Object Detectors via Reliable Uncertainty Quantification and Calibration
results: 经过EXTENSIVE 使用本文提出的测试框架，研究人员可以获得详细的检测器 Robustness 性能分析结果，包括检测器对域 shift 的Robustness，Scene中的信息 Uncertainty 度量，以及检测结果的Calibration 性能。此外，本文还提出了一个简单的基准方法，以便研究人员可以对未来的提议方法进行比较。Abstract
The current approach for testing the robustness of object detectors suffers from serious deficiencies such as improper methods of performing out-of-distribution detection and using calibration metrics which do not consider both localisation and classification quality. In this work, we address these issues, and introduce the Self-Aware Object Detection (SAOD) task, a unified testing framework which respects and adheres to the challenges that object detectors face in safety-critical environments such as autonomous driving. Specifically, the SAOD task requires an object detector to be: robust to domain shift; obtain reliable uncertainty estimates for the entire scene; and provide calibrated confidence scores for the detections. We extensively use our framework, which introduces novel metrics and large scale test datasets, to test numerous object detectors in two different use-cases, allowing us to highlight critical insights into their robustness performance. Finally, we introduce a simple baseline for the SAOD task, enabling researchers to benchmark future proposed methods and move towards robust object detectors which are fit for purpose. Code is available at https://github.com/fiveai/saod
摘要
当前对对象检测器的测试方法存在严重的缺陷，例如不当的异常检测方法和不充分考虑本地化和分类质量的评价指标。在这种工作中，我们解决这些问题，并提出了自适应对象检测（SAOD）任务，一种统一的测试框架，尊重和遵循对自动驾驶等安全关键环境中的对象检测器所面临的挑战。具体来说，SAOD任务要求对象检测器：对域shift具有抗衰减能力; 在整个场景中获得可靠的uncertainty估计; 并提供准确的信任分数。我们广泛使用我们的框架，引入了新的指标和大规模测试数据集，对许多对象检测器进行了两种不同的应用场景的测试，从而得出了对其Robustness性能的重要发现。最后，我们提出了一个简单的基线方法，使得研究人员可以对未来的提议方法进行比较，并推动对象检测器的Robustness性能进一步提高。代码可以在https://github.com/fiveai/saod上获取。
A large calcium-imaging dataset reveals a systematic V4 organization for natural scenes
results: 研究人员发现了 V4 中各种自然图像特征的域化功能区域，包括表面相关特征如颜色和Texture，以及形状相关特征如边缘、曲线和人脸特征。这些结果证明了 V4 在处理自然场景时的细致 topological 组织和神经编码。Abstract
The visual system evolved to process natural scenes, yet most of our understanding of the topology and function of visual cortex derives from studies using artificial stimuli. To gain deeper insights into visual processing of natural scenes, we utilized widefield calcium-imaging of primate V4 in response to many natural images, generating a large dataset of columnar-scale responses. We used this dataset to build a digital twin of V4 via deep learning, generating a detailed topographical map of natural image preferences at each cortical position. The map revealed clustered functional domains for specific classes of natural image features. These ranged from surface-related attributes like color and texture to shape-related features such as edges, curvature, and facial features. We validated the model-predicted domains with additional widefield calcium-imaging and single-cell resolution two-photon imaging. Our study illuminates the detailed topological organization and neural codes in V4 that represent natural scenes.
摘要
“我们的视觉系统演化来处理自然场景，但大多数我们关于视觉脑区的理解却来自使用人工刺激的研究。为了更深入了解视觉处理自然场景，我们利用了广场 calcium 影像 primate V4 的回应，生成了视觉脑区的大量柱状对应。我们使用了这个数据集来建立视觉脑区的数位双胞虫，生成了自然图像特征的详细地图。这个地图显示了各 cortical 位置的自然图像特征的集中功能领域，包括表面相关特征如颜色和文本ure，以及形状相关特征如边缘、曲线和脸部特征。我们验证了模型预测的领域使用进一步的广场 calcium 影像和单细胞分解二氢镜影像。我们的研究照明了视觉脑区中自然场景的细部 topological 组织和神经代码。”
methods: 基于多视图基因因子（CF）的半监督多视图概念因子模型（SMVCF），包括多视图CF、标签传播、抽象学习和自适应权重 вектор，以Integrate valuable information from multiple views and improve clustering performance.
results: 在四个多样化的数据集上进行了广泛的实验，并证明了SMVCF模型在多视图聚类任务中的效果和优势。Abstract
Concept Factorization (CF), as a novel paradigm of representation learning, has demonstrated superior performance in multi-view clustering tasks. It overcomes limitations such as the non-negativity constraint imposed by traditional matrix factorization methods and leverages kernel methods to learn latent representations that capture the underlying structure of the data, thereby improving data representation. However, existing multi-view concept factorization methods fail to consider the limited labeled information inherent in real-world multi-view data. This often leads to significant performance loss. To overcome these limitations, we propose a novel semi-supervised multi-view concept factorization model, named SMVCF. In the SMVCF model, we first extend the conventional single-view CF to a multi-view version, enabling more effective exploration of complementary information across multiple views. We then integrate multi-view CF, label propagation, and manifold learning into a unified framework to leverage and incorporate valuable information present in the data. Additionally, an adaptive weight vector is introduced to balance the importance of different views in the clustering process. We further develop targeted optimization methods specifically tailored for the SMVCF model. Finally, we conduct extensive experiments on four diverse datasets with varying label ratios to evaluate the performance of SMVCF. The experimental results demonstrate the effectiveness and superiority of our proposed approach in multi-view clustering tasks.
摘要
科学技术（CF）是一种新的表示学习方法，在多视图归一 clustering 任务中表现出了超越性。它超越了传统矩阵因子化方法中的非正式约束，并通过核函数方法学习隐藏的表示，以捕捉数据的下面结构，从而改善数据表示。然而，现有的多视图概念因子化方法通常忽视了实际世界中多视图数据中的有限 Label 信息。这经常导致显著的性能损失。为解决这些限制，我们提出了一种新的半监督多视图概念因子化模型，称为 SMVCF。在 SMVCF 模型中，我们首先将单视图 CF 扩展到多视图版本，以更好地利用多视图数据中的补充信息。然后，我们将多视图 CF、标签传播、抽象学习集成到一个统一框架中，以利用数据中的有价值信息。此外，我们还引入了自适应权重 вектор，以平衡不同视图在归一 clustering 过程中的重要性。最后，我们专门为 SMVCF 模型开发了目标优化方法。我们在四种不同的数据集上进行了广泛的实验，并评估了 SMVCF 模型的性能。实验结果表明，我们提出的方法在多视图归一 clustering 任务中表现出了有效性和优势。
Many tasks make light work: Learning to localise medical anomalies from multiple synthetic tasks
results: 该论文可以轻松超越现有的方法，并在脑MRI和胸部X射线图像上进行了示例。Abstract
There is a growing interest in single-class modelling and out-of-distribution detection as fully supervised machine learning models cannot reliably identify classes not included in their training. The long tail of infinitely many out-of-distribution classes in real-world scenarios, e.g., for screening, triage, and quality control, means that it is often necessary to train single-class models that represent an expected feature distribution, e.g., from only strictly healthy volunteer data. Conventional supervised machine learning would require the collection of datasets that contain enough samples of all possible diseases in every imaging modality, which is not realistic. Self-supervised learning methods with synthetic anomalies are currently amongst the most promising approaches, alongside generative auto-encoders that analyse the residual reconstruction error. However, all methods suffer from a lack of structured validation, which makes calibration for deployment difficult and dataset-dependant. Our method alleviates this by making use of multiple visually-distinct synthetic anomaly learning tasks for both training and validation. This enables more robust training and generalisation. With our approach we can readily outperform state-of-the-art methods, which we demonstrate on exemplars in brain MRI and chest X-rays. Code is available at https://github.com/matt-baugh/many-tasks-make-light-work .
摘要
随着单类模型和 OUT-OF-Distribution 检测的兴趣增长，因为完全supervised机器学习模型无法可靠地识别不包括在其训练中的类。实际世界中的长尾多样化 OUT-OF-Distribution 类型，例如屏检、分类和质控，意味着需要训练单类模型，表示预期的特征分布，例如只从严格健康志愿者数据中训练。传统的supervised机器学习需要收集包含所有可能的疾病样本的 dataset，这并不是现实的。self-supervised learning方法与生成式 auto-encoders 是目前最有前途的方法，但所有方法受到结构化验证的缺失，这使得在部署时进行调整困难和数据集dependent。我们的方法利用多个可见distinct的synthetic anomaly学习任务来 both 训练和验证，这使得训练更加坚固和普适。我们的方法可以轻松超越现有的方法，我们在 brain MRI 和胸部 X-ray 中进行了示例。代码可以在 获取。
Synthesis of Contrast-Enhanced Breast MRI Using Multi-b-Value DWI-based Hierarchical Fusion Network with Attention Mechanism
results: 研究结果表明，多值 b-值 DWI 基于的融合模型可能可以用于生成 CE-MRI，从而避免或减少使用 GBCA，从而减轻病人的负担。Abstract
Magnetic resonance imaging (MRI) is the most sensitive technique for breast cancer detection among current clinical imaging modalities. Contrast-enhanced MRI (CE-MRI) provides superior differentiation between tumors and invaded healthy tissue, and has become an indispensable technique in the detection and evaluation of cancer. However, the use of gadolinium-based contrast agents (GBCA) to obtain CE-MRI may be associated with nephrogenic systemic fibrosis and may lead to bioaccumulation in the brain, posing a potential risk to human health. Moreover, and likely more important, the use of gadolinium-based contrast agents requires the cannulation of a vein, and the injection of the contrast media which is cumbersome and places a burden on the patient. To reduce the use of contrast agents, diffusion-weighted imaging (DWI) is emerging as a key imaging technique, although currently usually complementing breast CE-MRI. In this study, we develop a multi-sequence fusion network to synthesize CE-MRI based on T1-weighted MRI and DWIs. DWIs with different b-values are fused to efficiently utilize the difference features of DWIs. Rather than proposing a pure data-driven approach, we invent a multi-sequence attention module to obtain refined feature maps, and leverage hierarchical representation information fused at different scales while utilizing the contributions from different sequences from a model-driven approach by introducing the weighted difference module. The results show that the multi-b-value DWI-based fusion model can potentially be used to synthesize CE-MRI, thus theoretically reducing or avoiding the use of GBCA, thereby minimizing the burden to patients. Our code is available at \url{https://github.com/Netherlands-Cancer-Institute/CE-MRI}.
摘要
磁共振成像（MRI）是当前临床成像技术中诊断乳腺癌最敏感的方法。增强的磁共振成像（CE-MRI）可以准确地区分肿瘤和混合到正常组织，成为诊断和评估癌病的不可或缺的技术。然而，使用高德林铵基 contrast agent（GBCA）来实现CE-MRI可能会与肾脏系统 fibrosis 相关，并且可能会在脑部堆积，对人类健康构成潜在的风险。此外，使用高德林铵基 contrast agent需要血管注射剂，这是对患者的困扰和负担。为了减少对高德林铵基 contrast agent的使用，扩散成像（DWI）正在成为诊断乳腺癌的关键成像技术之一。在这个研究中，我们开发了一种多следова列融合网络，使用T1重度成像和DWI进行CE-MRI的合成。不同的b值的DWI被融合，以利用不同的DWI特征。而不是直接提出数据驱动方法，我们创造了一种多следова列注意模块，以获得精细的特征地图，并利用层次表示信息在不同的缩放级别上进行融合。我们的研究结果表明，使用多b值DWI合成CE-MRI的模型可能可以减少或避免使用GBCA，从而减轻患者的负担。我们的代码可以在 \url{https://github.com/Netherlands-Cancer-Institute/CE-MRI} 上获取。
results: 研究发现，2019年全国城市树覆盖率约为11%，76%的城市出现了从2010年到2019年的树覆盖增长。特别是在北京和上海等各大城市，树覆盖增长率为7.69%，远高于其他城市的3.94%。Abstract
Trees play a crucial role in urban environments, offering various ecosystem services that contribute to public health and human well-being. China has initiated a range of urban greening policies over the past decades, however, monitoring their impact on urban tree dynamics at a national scale has proven challenging. In this study, we deployed nano-satellites to quantify urban tree coverage in all major Chinese cities larger than 50 km2 in 2010 and 2019. Our findings indicate that approximately 6000 km2 (11%) of urban areas were covered by trees in 2019, and 76% of these cities experienced an increase in tree cover compared to 2010. Notably, the increase in tree cover in mega-cities such as Beijing, and Shanghai was approximately twice as large as in most other cities (7.69% vs 3.94%). The study employs a data-driven approach towards assessing urban tree cover changes in relation to greening policies, showing clear signs of tree cover increases but also suggesting an uneven implementation primarily benefiting a few mega-cities.
摘要
城市中的树木扮演着重要的生态系统服务作用，对公众健康和人类福祉产生了重要贡献。中国在过去的几十年中实施了一系列城市绿化政策，但监测这些政策对城市树木动态的影响在国家范围内是有挑战的。本研究通过使用尺度约为1米的幼卫星来评估2010年和2019年中国大于50平方公里的主要城市的城市树木覆盖率。我们发现在2019年，城市树木覆盖率约为11%，76%的城市经历了2010年相比的增长。尤其是在北京和上海等巨型城市，增长率约为7.69%，与其他城市相比为twice as large（3.94%）。本研究采用数据驱动的方法来评估城市树木覆盖变化和绿化政策之间的关系，显示出明显的树木覆盖增加，但也表明了一些巨型城市的不均衡实施。
Generating Reliable Pixel-Level Labels for Source Free Domain Adaptation
for: Addressing the challenging domain adaptation setting where the knowledge from the labelled source domain dataset is only available from a pretrained black-box segmentation model.
methods: Proposes a simple yet novel image translation workflow, ReGEN, which comprises an image-to-image translation network and a segmentation network to generate target-like images using the noisy predictions from the original target domain images.
results: Demonstrates favourable performance relative to recent state-of-the-art work in two benchmark domain adaptation settings.Here’s the simplified Chinese text:
results: 在两个标准域适应设定中展示了与最新的状态艺术工作相当的表现。Abstract
This work addresses the challenging domain adaptation setting in which knowledge from the labelled source domain dataset is available only from the pretrained black-box segmentation model. The pretrained model's predictions for the target domain images are noisy because of the distributional differences between the source domain data and the target domain data. Since the model's predictions serve as pseudo labels during self-training, the noise in the predictions impose an upper bound on model performance. Therefore, we propose a simple yet novel image translation workflow, ReGEN, to address this problem. ReGEN comprises an image-to-image translation network and a segmentation network. Our workflow generates target-like images using the noisy predictions from the original target domain images. These target-like images are semantically consistent with the noisy model predictions and therefore can be used to train the segmentation network. In addition to being semantically consistent with the predictions from the original target domain images, the generated target-like images are also stylistically similar to the target domain images. This allows us to leverage the stylistic differences between the target-like images and the target domain image as an additional source of supervision while training the segmentation model. We evaluate our model with two benchmark domain adaptation settings and demonstrate that our approach performs favourably relative to recent state-of-the-art work. The source code will be made available.
摘要
Our workflow generates target-like images using the noisy predictions from the original target domain images. These target-like images are semantically consistent with the noisy model predictions and can be used to train the segmentation network. Additionally, the generated target-like images are stylistically similar to the target domain images, allowing us to leverage the stylistic differences between the target-like images and the target domain image as an additional source of supervision during training.We evaluate our model with two benchmark domain adaptation settings and demonstrate that our approach performs favorably relative to recent state-of-the-art work. The source code will be made available.
An Explainable Deep Framework: Towards Task-Specific Fusion for Multi-to-One MRI Synthesis
paper_authors: Luyi Han, Tianyu Zhang, Yunzhi Huang, Haoran Dou, Xin Wang, Yuan Gao, Chunyao Lu, Tan Tao, Ritse Mann for:多序列MRI在临床设置中具有可靠诊断和治疗预测价值，但某些序列可能因各种原因而无法使用或缺失。为解决这个问题，MRI合成是一个可能的解决方案。methods:使用最新的深度学习基于方法，可以好好地组合可用的多个序列来合成缺失的序列。然而，这些方法缺乏对不同输入序列的贡献的评估和生成图像质量的估计，使其实际应用困难。因此，我们提议一种可解释的任务特定合成网络，可以自动调整任务特定的权重，并提供可读性和可靠性从两个方面：1. 可视化输入序列的贡献在合并阶段的权重平均模块，使得可以看到每个输入序列的贡献。2. 在合成图像时，使用任务特定的注意力模块，以便高亮网络在合成图像时所尝试修改的区域。results:我们在BraTS2021数据集上进行了1251个主题的实验，结果表明，我们的提议的方法在无序列合成任务中表现更好于当前状态的方法。我们的代码可以在 \url{https://github.com/fiy2W/mri_seq2seq} 上找到。Abstract
Multi-sequence MRI is valuable in clinical settings for reliable diagnosis and treatment prognosis, but some sequences may be unusable or missing for various reasons. To address this issue, MRI synthesis is a potential solution. Recent deep learning-based methods have achieved good performance in combining multiple available sequences for missing sequence synthesis. Despite their success, these methods lack the ability to quantify the contributions of different input sequences and estimate the quality of generated images, making it hard to be practical. Hence, we propose an explainable task-specific synthesis network, which adapts weights automatically for specific sequence generation tasks and provides interpretability and reliability from two sides: (1) visualize the contribution of each input sequence in the fusion stage by a trainable task-specific weighted average module; (2) highlight the area the network tried to refine during synthesizing by a task-specific attention module. We conduct experiments on the BraTS2021 dataset of 1251 subjects, and results on arbitrary sequence synthesis indicate that the proposed method achieves better performance than the state-of-the-art methods. Our code is available at \url{https://github.com/fiy2W/mri_seq2seq}.
摘要
多重序列MRI在临床 Settings中是有价值的，可以提供可靠的诊断和治疗预测，但某些序列可能无法使用或缺失。为解决这个问题，MRI合成是一个可能的解决方案。现代深度学习基于方法在 combining 多个可用序列中缺失序列的合成方面 achieved 好的性能。尽管它们在成功，但它们缺乏对不同输入序列的贡献的评估和生成图像质量的估计，使其实难实际应用。因此，我们提议一种可解释的任务特定合成网络，该网络可自动适应特定序列生成任务，并提供可读性和可靠性从两个方面：1. 在合并阶段，使用可训练的任务特定权重平均模块来可视化每个输入序列的贡献。2. 使用任务特定注意力模块来高亮合成过程中网络尝试修改的区域。我们在 BraTS2021 数据集上进行了1251个主题的实验，并对arbitrary sequence synthesis进行了测试。结果表明，我们的方法在比 estado-of-the-art 方法更好。我们的代码可以在 \url{https://github.com/fiy2W/mri_seq2seq} 上获取。
Co-Learning Meets Stitch-Up for Noisy Multi-label Visual Recognition
paper_authors: Chao Liang, Zongxin Yang, Linchao Zhu, Yi Yang
for: Handle long-tailed multi-label recognition with noisy labels.
methods: Propose a Stitch-Up augmentation to synthesize cleaner samples, and a Heterogeneous Co-Learning framework to leverage the inconsistency between long-tailed and balanced distributions.
results: Achieve superior results compared to various baselines, demonstrating the effectiveness of the proposed method in handling noisy long-tailed multi-label data.Here’s the full text in Simplified Chinese:
for: Handle long-tailed multi-label recognition with noisy labels.
methods: Propose a Stitch-Up augmentation to synthesize cleaner samples, 以及一个 Heterogeneous Co-Learning 框架，利用长尾分布和多 Label 的不一致性，生成更清楚的标签。
results: Compared to various baselines, 提出的方法在处理噪音长尾多 Label 资料时取得了superior的结果，证明了方法的有效性。Abstract
In real-world scenarios, collected and annotated data often exhibit the characteristics of multiple classes and long-tailed distribution. Additionally, label noise is inevitable in large-scale annotations and hinders the applications of learning-based models. Although many deep learning based methods have been proposed for handling long-tailed multi-label recognition or label noise respectively, learning with noisy labels in long-tailed multi-label visual data has not been well-studied because of the complexity of long-tailed distribution entangled with multi-label correlation. To tackle such a critical yet thorny problem, this paper focuses on reducing noise based on some inherent properties of multi-label classification and long-tailed learning under noisy cases. In detail, we propose a Stitch-Up augmentation to synthesize a cleaner sample, which directly reduces multi-label noise by stitching up multiple noisy training samples. Equipped with Stitch-Up, a Heterogeneous Co-Learning framework is further designed to leverage the inconsistency between long-tailed and balanced distributions, yielding cleaner labels for more robust representation learning with noisy long-tailed data. To validate our method, we build two challenging benchmarks, named VOC-MLT-Noise and COCO-MLT-Noise, respectively. Extensive experiments are conducted to demonstrate the effectiveness of our proposed method. Compared to a variety of baselines, our method achieves superior results.
摘要
在实际应用场景中，收集的数据经常具有多个类别和长尾分布的特点，同时 Label noise 是无法避免的。虽然许多深度学习基于方法已经提出来处理长尾多标签识别或 Label noise，但是在长尾多标签视觉数据中学习受损 Label noise 还没有得到充分研究，这是因为长尾分布和多标签相互作用的复杂性。为解决这种复杂而困难的问题，本文强调降低 Label noise 基于多标签分类和长尾学习的一些自然性质。在详细的实现方式中，我们提出了一种叫做 Stitch-Up 的扩充方法，可以直接减少多标签雷达。此外，我们还设计了一种叫做 Heterogeneous Co-Learning 的框架，可以利用长尾和平衡分布之间的不一致性，以便更好地学习受损 Label noise 的 cleaner 标签。为证明我们的方法的效果，我们建立了两个具有挑战性的 benchmark，分别是 VOC-MLT-Noise 和 COCO-MLT-Noise。我们进行了广泛的实验，以确认我们的方法的效果。相比多种基线方法，我们的方法得到了更好的结果。
End-To-End Prediction of Knee Osteoarthritis Progression With Multi-Modal Transformers
paper_authors: Egor Panfilov, Simo Saarakkala, Miika T. Nieminen, Aleksei Tiulpin
for: The paper is written to investigate the use of multi-modal data and deep learning methods for predicting the progression of knee osteoarthritis (KOA).
methods: The paper uses a Transformer approach for multi-modal fusion of knee imaging data, and analyzes its performance across different progression horizons.
results: The paper shows that structural knee MRI can identify radiographic KOA progressors with an area under the ROC curve (ROC AUC) of 0.70-0.76 and Average Precision (AP) of 0.15-0.54 in 2-8 year horizons. Additionally, the paper finds that progression within 1 year can be better predicted with a multi-modal method using X-ray, structural, and compositional MR images.Here are the results in Simplified Chinese text:
results: 这个研究发现，结构性股骨MRI可以在2-8年的进展时间 horizon 上识别 radiographic KOA 进行者，ROC AUC 为0.70-0.76，AP 为0.15-0.54。此外，研究发现，使用多Modal的X-ray、结构性和化学成分MRI Image 可以更好地预测在1年内的进展。Abstract
Knee Osteoarthritis (KOA) is a highly prevalent chronic musculoskeletal condition with no currently available treatment. The manifestation of KOA is heterogeneous and prediction of its progression is challenging. Current literature suggests that the use of multi-modal data and advanced modeling methods, such as the ones based on Deep Learning, has promise in tackling this challenge. To date, however, the evidence on the efficacy of this approach is limited. In this study, we leveraged recent advances in Deep Learning and, using a Transformer approach, developed a unified framework for the multi-modal fusion of knee imaging data. Subsequently, we analyzed its performance across a range of scenarios by investigating multiple progression horizons -- from short-term to long-term. We report our findings using a large cohort (n=2421-3967) derived from the Osteoarthritis Initiative dataset. We show that structural knee MRI allows identifying radiographic KOA progressors on par with multi-modal fusion approaches, achieving an area under the ROC curve (ROC AUC) of 0.70-0.76 and Average Precision (AP) of 0.15-0.54 in 2-8 year horizons. Progression within 1 year was better predicted with a multi-modal method using X-ray, structural, and compositional MR images -- ROC AUC of 0.76(0.04), AP of 0.13(0.04) -- or via clinical data. Our follow-up analysis generally shows that prediction from the imaging data is more accurate for post-traumatic subjects, and we further investigate which subject subgroups may benefit the most. The present study provides novel insights into multi-modal imaging of KOA and brings a unified data-driven framework for studying its progression in an end-to-end manner, providing new tools for the design of more efficient clinical trials. The source code of our framework and the pre-trained models are made publicly available.
摘要
knee退化病 (KOA) 是一种非常流行的慢性 musculoskeletal 疾病，目前没有可用的治疗方案。 KOA 的表现是多iform 的，预测其进展是具有挑战性。现有文献建议使用多 modal 数据和进步的模型方法，如基于深度学习的方法，可以解决这个挑战。然而，至今为止，这种方法的效果仍然有限。在这个研究中，我们利用了最新的深度学习技术，使用 transformer 方法，实现了多modal 影像资料的融合。然后，我们分析了这个框架在不同的时间长度下的表现，包括短期、中期和长期。我们使用了一大群患者（n=2421-3967）， derive 自 Osteoarthritis Initiative 资料集。我们发现，静止股骨 MRI 可以与多modal 融合方法相比，预测 X-ray 照片 KOA 进展者，ROC AUC 为 0.70-0.76，AP 为 0.15-0.54 在 2-8 年时间长度下。在 1 年内进展预测中，多modal 方法使用 X-ray、静止、和化学 MR 影像的结果更高，ROC AUC 为 0.76(0.04)，AP 为 0.13(0.04)。我们的续作分析显示，从影像数据预测 KOA 的进展更加精确，特别是在受到伤害后的患者中。我们进一步研究了哪些患者子群可能从影像数据中得到最多的帮助。这项研究提供了新的关于多modal 影像 KOA 的新知识，并提供了一个统一的数据驱动的框架，实现了预测 KOA 进展的端到端方式。我们的框架和预训模型的源代码都公开 disponibile。
paper_authors: Zhouyingcheng Liao, Vladislav Golyanik, Marc Habermann, Christian Theobalt
for: 创建高级别的人物游戏角色，自动生成高级别的人物模型和皮肤材料
methods: 使用多视图视频学习技术，自动生成人物模型的皮肤纹理、皮肤骨骼和动作纹理
results: 比起现有方法，提高人物模型的自动生成效果，不需要高级别的4D扫描数据Abstract
Rigging and skinning clothed human avatars is a challenging task and traditionally requires a lot of manual work and expertise. Recent methods addressing it either generalize across different characters or focus on capturing the dynamics of a single character observed under different pose configurations. However, the former methods typically predict solely static skinning weights, which perform poorly for highly articulated poses, and the latter ones either require dense 3D character scans in different poses or cannot generate an explicit mesh with vertex correspondence over time. To address these challenges, we propose a fully automated approach for creating a fully rigged character with pose-dependent skinning weights, which can be solely learned from multi-view video. Therefore, we first acquire a rigged template, which is then statically skinned. Next, a coordinate-based MLP learns a skinning weights field parameterized over the position in a canonical pose space and the respective pose. Moreover, we introduce our pose- and view-dependent appearance field allowing us to differentiably render and supervise the posed mesh using multi-view imagery. We show that our approach outperforms state-of-the-art while not relying on dense 4D scans.
摘要
rigging和皮肤化人物模型是一项复杂的任务，传统上需要大量的手动工作和专业知识。现有的方法可以涵盖不同的人物，但是大多数情况下预测的是静止皮肤 веса，性能对高度动作pose较差，而其他方法则需要在不同的姿势下获得密集的3D人物扫描，或者无法在时间上生成明确的网格。为了解决这些挑战，我们提出了一种完全自动化的方法，可以从多视图视频中学习出fully rigged人物，并且可以靠 solely 在多视图图像上进行渲染和监督。我们的方法包括以下几个步骤：首先，我们获得一个已经RIGGED的模板，然后使用一个坐标基于的多层感知学习（MLP）来学习皮肤 веса场，该场被参数化为位置在一个坐标系中的 canonical pose 空间和相应的姿势。此外，我们引入了 pose-和 view-dependent的外观场，使得我们可以在多视图图像上分别渲染和监督着poses的人物模型。我们的方法比 estado-of-the-art 高效，而不需要密集的4D扫描。
Surgical fine-tuning for Grape Bunch Segmentation under Visual Domain Shifts
results: 本研究表明，通过手术精细调整，可以有效地适应新收集的葡萄图像，并且可以减少参数的数量，从而提高模型的鲜明性和灵活性。Abstract
Mobile robots will play a crucial role in the transition towards sustainable agriculture. To autonomously and effectively monitor the state of plants, robots ought to be equipped with visual perception capabilities that are robust to the rapid changes that characterise agricultural settings. In this paper, we focus on the challenging task of segmenting grape bunches from images collected by mobile robots in vineyards. In this context, we present the first study that applies surgical fine-tuning to instance segmentation tasks. We show how selectively tuning only specific model layers can support the adaptation of pre-trained Deep Learning models to newly-collected grape images that introduce visual domain shifts, while also substantially reducing the number of tuned parameters.
摘要
Mobile robots will play a crucial role in the transition towards sustainable agriculture. To autonomously and effectively monitor the state of plants, robots ought to be equipped with visual perception capabilities that are robust to the rapid changes that characterize agricultural settings. In this paper, we focus on the challenging task of segmenting grape bunches from images collected by mobile robots in vineyards. In this context, we present the first study that applies surgical fine-tuning to instance segmentation tasks. We show how selectively tuning only specific model layers can support the adaptation of pre-trained Deep Learning models to newly-collected grape images that introduce visual domain shifts, while also substantially reducing the number of tuned parameters.Here's the translation in Traditional Chinese as well:Mobile robots will play a crucial role in the transition towards sustainable agriculture. To autonomously and effectively monitor the state of plants, robots ought to be equipped with visual perception capabilities that are robust to the rapid changes that characterize agricultural settings. In this paper, we focus on the challenging task of segmenting grape bunches from images collected by mobile robots in vineyards. In this context, we present the first study that applies surgical fine-tuning to instance segmentation tasks. We show how selectively tuning only specific model layers can support the adaptation of pre-trained Deep Learning models to newly-collected grape images that introduce visual domain shifts, while also substantially reducing the number of tuned parameters.
Robust Surgical Tools Detection in Endoscopic Videos with Noisy Data
for: This paper aims to develop a systematic methodology for robust surgical tool detection using noisy data.
methods: The proposed methodology introduces an intelligent active learning strategy for minimal dataset identification and label correction by human experts, as well as an assembling strategy for a student-teacher model-based self-training framework to achieve robust classification of 14 surgical tools in a semi-supervised fashion.
results: The proposed methodology achieves an average F1-score of 85.88% for the ensemble model-based self-training with class weights, and 80.88% without class weights for noisy labels. The proposed method significantly outperforms existing approaches, effectively demonstrating its effectiveness.Abstract
Over the past few years, surgical data science has attracted substantial interest from the machine learning (ML) community. Various studies have demonstrated the efficacy of emerging ML techniques in analysing surgical data, particularly recordings of procedures, for digitizing clinical and non-clinical functions like preoperative planning, context-aware decision-making, and operating skill assessment. However, this field is still in its infancy and lacks representative, well-annotated datasets for training robust models in intermediate ML tasks. Also, existing datasets suffer from inaccurate labels, hindering the development of reliable models. In this paper, we propose a systematic methodology for developing robust models for surgical tool detection using noisy data. Our methodology introduces two key innovations: (1) an intelligent active learning strategy for minimal dataset identification and label correction by human experts; and (2) an assembling strategy for a student-teacher model-based self-training framework to achieve the robust classification of 14 surgical tools in a semi-supervised fashion. Furthermore, we employ weighted data loaders to handle difficult class labels and address class imbalance issues. The proposed methodology achieves an average F1-score of 85.88\% for the ensemble model-based self-training with class weights, and 80.88\% without class weights for noisy labels. Also, our proposed method significantly outperforms existing approaches, which effectively demonstrates its effectiveness.
摘要
过去几年，外科数据科学已经吸引了机器学习（ML）社区的广泛关注。各种研究表明了新兴ML技术在分析外科数据方面的效果，特别是记录手术过程的数据，用于数字化临床和非临床功能，如前置规划、Context-aware决策和手术技巧评估。然而，这一领域仍处于初期阶段，缺乏代表性的、正确标注的数据集，用于训练中等ML任务。此外，现有的数据集受到不准确的标注，妨碍了模型的发展。在本文中，我们提出了一种系统的方法ологи，用于在噪音数据上建立可靠的外科工具检测模型。我们的方法有两个关键创新：（1）一种智能活动学习策略，用于标注和人工专家审核的最小数据集选择；（2）一种学生-教师模型自我训练框架，用于在半监督式下实现多 Tool 的精度分类。此外，我们采用了权重数据加载器，处理困难分类和分类不均衡问题。我们的提议方法在 ensemble 模型基础自动训练中，使用类别权重时，达到了85.88%的平均 F1 分数，无类别权重时，达到了80.88%。此外，我们的提议方法在与现有方法进行比较时，显示了明显的效果。
Unveiling the Potential of Spike Streams for Foreground Occlusion Removal from Densely Continuous Views
results: 我们的方法可以高效地除去各种场景中的 dense occlusions，并且具有强大的泛化能力。Abstract
The extraction of a clean background image by removing foreground occlusion holds immense practical significance, but it also presents several challenges. Presently, the majority of de-occlusion research focuses on addressing this issue through the extraction and synthesis of discrete images from calibrated camera arrays. Nonetheless, the restoration quality tends to suffer when faced with dense occlusions or high-speed motions due to limited perspectives and motion blur. To successfully remove dense foreground occlusion, an effective multi-view visual information integration approach is required. Introducing the spike camera as a novel type of neuromorphic sensor offers promising capabilities with its ultra-high temporal resolution and high dynamic range. In this paper, we propose an innovative solution for tackling the de-occlusion problem through continuous multi-view imaging using only one spike camera without any prior knowledge of camera intrinsic parameters and camera poses. By rapidly moving the spike camera, we continually capture the dense stream of spikes from the occluded scene. To process the spikes, we build a novel model \textbf{SpkOccNet}, in which we integrate information of spikes from continuous viewpoints within multi-windows, and propose a novel cross-view mutual attention mechanism for effective fusion and refinement. In addition, we contribute the first real-world spike-based dataset \textbf{S-OCC} for occlusion removal. The experimental results demonstrate that our proposed model efficiently removes dense occlusions in diverse scenes while exhibiting strong generalization.
摘要
抽取清晰背景图像，除去前景遮挡，具有很大的实际意义，但也存在多种挑战。目前，大多数遮挡恢复研究通过抽取和合成扩展的照片阵列来解决这个问题。然而，当面临紧密的遮挡物或高速运动时， restore 质量往往受到限制，因为有限的视角和运动模糊。要成功地除去紧密的前景遮挡，需要一种有效的多视图视觉信息集成方法。在这篇论文中，我们提出了一种新的解决方案，通过不断拍摄快速移动的射频摄像机来实现连续多视图抽取。我们在不知道摄像机内参数和摄像机位置的情况下，通过 continually 捕捉遮挡场景中的快速流动的射频脉冲来处理脉冲。为了处理脉冲，我们建立了一种新的模型，即 SpkOccNet，其中我们在多窗口内集成了不断更新的脉冲信息，并提出了一种新的交叉视图互注意机制来实现有效的融合和修正。此外，我们也提供了首个实际遮挡基本数据集，即 S-OCC，以用于遮挡除去。实验结果表明，我们提出的模型能够有效地除去多种场景中的紧密遮挡。
Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Dataset
for: This paper is written for researchers and developers who work on 3D human pose estimation, motion capture, and computer vision.
methods: The paper presents a whole-body motion and text annotation pipeline that can automatically annotate motion from single- or multi-view videos and provide comprehensive semantic labels for each video and fine-grained whole-body pose descriptions for each frame.
results: The paper constructs Motion-X, a large-scale 3D expressive whole-body motion dataset that covers 96K motion sequences from massive scenes, with 13.7M precise 3D whole-body pose annotations and 13.7M frame-level whole-body pose descriptions. The paper demonstrates the accuracy of the annotation pipeline and the significant benefit of Motion-X in enhancing expressive, diverse, and natural motion generation, as well as 3D whole-body human mesh recovery.Here are the three points in Simplified Chinese text:
results: 论文构建了Motion-X数据集，包括96万个动作序列，涵盖庞大的场景，每个动作序列具有1370万个精细的3D人体姿态注释和每帧的人体姿态描述。论文证明了注释管道的准确性和Motion-X数据集在增强自然、多样化和表达力强的动作生成、3D人体人脸恢复方面的重要作用。Abstract
In this paper, we present Motion-X, a large-scale 3D expressive whole-body motion dataset. Existing motion datasets predominantly contain body-only poses, lacking facial expressions, hand gestures, and fine-grained pose descriptions. Moreover, they are primarily collected from limited laboratory scenes with textual descriptions manually labeled, which greatly limits their scalability. To overcome these limitations, we develop a whole-body motion and text annotation pipeline, which can automatically annotate motion from either single- or multi-view videos and provide comprehensive semantic labels for each video and fine-grained whole-body pose descriptions for each frame. This pipeline is of high precision, cost-effective, and scalable for further research. Based on it, we construct Motion-X, which comprises 13.7M precise 3D whole-body pose annotations (i.e., SMPL-X) covering 96K motion sequences from massive scenes. Besides, Motion-X provides 13.7M frame-level whole-body pose descriptions and 96K sequence-level semantic labels. Comprehensive experiments demonstrate the accuracy of the annotation pipeline and the significant benefit of Motion-X in enhancing expressive, diverse, and natural motion generation, as well as 3D whole-body human mesh recovery.
摘要
在这篇论文中，我们介绍Motion-X，一个大规模的3D表达全身动作数据集。现有的动作数据集主要包含body只的姿势，缺乏面部表达、手势和细化pose描述。此外，它们主要从有限的实验室场景中收集，手动编注文本描述，这会很大限制其扩展性。为了超越这些限制，我们开发了一个整体动作和文本注释管线，可以自动从单个或多视图视频中提取动作，并提供每个视频的全面的semantic标签和每帧的细化全身姿势描述。这个管线具有高精度、成本效果和可扩展性，适用于进一步的研究。基于它，我们建立了Motion-X，包含1370万精度3D全身姿势注释（即SMPL-X），覆盖96000个动作序列。此外，Motion-X还提供1370万帧级全身姿势描述和96000个序列级semantic标签。实验证明注释管线的准确性和Motion-X的极大地改进了表达、多样化和自然的动作生成，以及3D全身人体模型的恢复。
SketchMetaFace: A Learning-based Sketching Interface for High-fidelity 3D Character Face Modeling
results: 在用户研究中证明了我们的系统比既有模型工具更加容易使用，同时提供了高质量的结果Here’s a breakdown of each point:
for: The paper is written for amateur users who want to quickly create high-fidelity 3D avatars with realistic faces.
methods: The system uses curvature-aware strokes to support the controllability of carving facial details, and a novel learning-based method called “Implicit and Depth Guided Mesh Modeling” (IDGMM) to achieve high-quality results with high efficiency. Additionally, the system provides a coarse-to-fine 2D sketching interface design and a data-driven stroke suggestion tool to support usability.
results: User studies demonstrate the superiority of the system over existing modeling tools in terms of ease of use and visual quality of results, and experimental analyses show that IDGMM reaches a better trade-off between accuracy and efficiency.Abstract
Modeling 3D avatars benefits various application scenarios such as AR/VR, gaming, and filming. Character faces contribute significant diversity and vividity as a vital component of avatars. However, building 3D character face models usually requires a heavy workload with commercial tools, even for experienced artists. Various existing sketch-based tools fail to support amateurs in modeling diverse facial shapes and rich geometric details. In this paper, we present SketchMetaFace - a sketching system targeting amateur users to model high-fidelity 3D faces in minutes. We carefully design both the user interface and the underlying algorithm. First, curvature-aware strokes are adopted to better support the controllability of carving facial details. Second, considering the key problem of mapping a 2D sketch map to a 3D model, we develop a novel learning-based method termed "Implicit and Depth Guided Mesh Modeling" (IDGMM). It fuses the advantages of mesh, implicit, and depth representations to achieve high-quality results with high efficiency. In addition, to further support usability, we present a coarse-to-fine 2D sketching interface design and a data-driven stroke suggestion tool. User studies demonstrate the superiority of our system over existing modeling tools in terms of the ease to use and visual quality of results. Experimental analyses also show that IDGMM reaches a better trade-off between accuracy and efficiency. SketchMetaFace is available at https://zhongjinluo.github.io/SketchMetaFace/.
摘要
模型3D人物可以应用于AR/VR、游戏和电影等场景，人物脸部具有重要的多样性和生动性，是人物模型的重要组件。然而，建立3D人物脸部模型通常需要大量的工作量，即使有经验的艺术家也会面临困难。现有的素描工具无法支持新手用户模型多样的脸部形状和富有的几何特征。在本文中，我们提出了SketchMetaFace - 一个针对新手用户的素描系统，可以在几分钟内创建高质量的3D脸部模型。我们仔细设计了用户界面和下面的算法。首先，我们采用了 curvature-aware strokes，以更好地支持挑战脸部细节的绘制。其次，我们开发了一种新的学习基于方法，称为“凸型和深度引导的三角形模型”（IDGMM），它将mesh、凸型和深度表示结合起来，以实现高质量和高效的结果。此外，为了进一步支持使用性，我们提出了一种粗略到细节的2D素描界面设计和数据驱动的笔划建议工具。用户研究表明，我们的系统比现有的模型工具更容易使用，并且可以生成高质量的结果。实验分析也显示，IDGMM达到了更好的精度和效率的平衡。SketchMetaFace可以在https://zhongjinluo.github.io/SketchMetaFace/上下载。
ACDMSR: Accelerated Conditional Diffusion Models for Single Image Super-Resolution
results: 该方法在多个 benchmark 数据集上进行了广泛的实验，并证明了与之前的尝试相比，它可以提供更高的图像质量和更快的执行速度。Abstract
Diffusion models have gained significant popularity in the field of image-to-image translation. Previous efforts applying diffusion models to image super-resolution (SR) have demonstrated that iteratively refining pure Gaussian noise using a U-Net architecture trained on denoising at various noise levels can yield satisfactory high-resolution images from low-resolution inputs. However, this iterative refinement process comes with the drawback of low inference speed, which strongly limits its applications. To speed up inference and further enhance the performance, our research revisits diffusion models in image super-resolution and proposes a straightforward yet significant diffusion model-based super-resolution method called ACDMSR (accelerated conditional diffusion model for image super-resolution). Specifically, our method adapts the standard diffusion model to perform super-resolution through a deterministic iterative denoising process. Our study also highlights the effectiveness of using a pre-trained SR model to provide the conditional image of the given low-resolution (LR) image to achieve superior high-resolution results. We demonstrate that our method surpasses previous attempts in qualitative and quantitative results through extensive experiments conducted on benchmark datasets such as Set5, Set14, Urban100, BSD100, and Manga109. Moreover, our approach generates more visually realistic counterparts for low-resolution images, emphasizing its effectiveness in practical scenarios.
摘要
Diffusion models 在图像到图像翻译领域得到了广泛的应用。先前的尝试使用 diffusion models 进行图像超分辨（SR）已经证明了，通过iteratively refining 纯 Gaussian noise 使用 U-Net 架构在不同噪声水平上训练的方法可以获得高质量的高分辨图像。然而，这种迭代纯化过程受到了推理速度的限制，妨碍了其应用。为了加速推理和进一步提高性能，我们的研究重新审视了 diffusion models 在图像 SR 领域，并提出了一种简单又有力的 diffusion model-based SR 方法 called ACDMSR (加速 conditional diffusion model for image super-resolution)。我们的方法改变了标准的 diffusion model，通过 deterministic iterative denoising 进行 SR。我们的研究还发现，使用预训练 SR 模型提供给 low-resolution（LR）图像的conditional image可以实现更高的高分辨结果。我们通过对 Set5、Set14、Urban100、BSD100 和 Manga109 等标准数据集进行了广泛的实验，证明了我们的方法超过了之前的尝试， both qualitatively and quantitatively。此外，我们的方法可以生成更加视觉真实的 low-resolution 图像的高分辨对应体，强调其在实际应用中的效iveness。
Learning Noise-Resistant Image Representation by Aligning Clean and Noisy Domains
paper_authors: Yanhui Guo, Xiaolin Wu, Fangzhou Luo
for: 提高图像 Representation robustness 对于噪音
methods: 使用 dual domain 映射和target-guided 匿名函数
results: 可以在复杂噪音图像上达到 Remarkable 性能和RobustnessAbstract
Recent supervised and unsupervised image representation learning algorithms have achieved quantum leaps. However, these techniques do not account for representation resilience against noise in their design paradigms. Consequently, these effective methods suffer failure when confronted with noise outside the training distribution, such as complicated real-world noise that is usually opaque to model training. To address this issue, dual domains are optimized to separately model a canonical space for noisy representations, namely the Noise-Robust (NR) domain, and a twinned canonical clean space, namely the Noise-Free (NF) domain, by maximizing the interaction information between the representations. Given the dual canonical domains, we design a target-guided implicit neural mapping function to accurately translate the NR representations to the NF domain, yielding noise-resistant representations by eliminating noise regencies. The proposed method is a scalable module that can be readily integrated into existing learning systems to improve their robustness against noise. Comprehensive trials of various tasks using both synthetic and real-world noisy data demonstrate that the proposed Target-Guided Dual-Domain Translation (TDDT) method is able to achieve remarkable performance and robustness in the face of complex noisy images.
摘要
最近的supervised和Unsupervised图像表示学习算法已经取得了量子跳跃。然而，这些技术不会考虑对噪声的表现鲁棒性在设计思路中。因此，这些有效的方法在面对训练数据外的噪声时会失败，如复杂的真实世界噪声，这些噪声通常是模型训练中透明的。为解决这个问题，我们分别优化了两个域，即Noise-Robust（NR）域和Noise-Free（NF）域，以便分别模型噪声表示的 canonical 空间。然后，我们设计了一个目标导向的隐藏神经映射函数，以准确地将NR表示翻译到NF域中，从而消除噪声残留。我们提出的方法可以轻松地integrated into现有的学习系统，以提高它们对噪声的抗性。我们在多个任务上使用both synthetic和真实世界噪声数据进行了全面的试验，并证明了我们的Target-Guided Dual-Domain Translation（TDDT）方法在面对复杂噪声图像时能够实现出色的性能和抗性。
Structured Network Pruning by Measuring Filter-wise Interactions
results: 对多种常用的深度学习模型（AlexNet、MobileNetv1、ResNet-50）和多种图像分类 dataset（MNIST、CIFAR-10、ImageNet）进行了实验，结果显示SNPFI可以减少网络计算量达60%以上，而模型的分类精度仍然保持在acceptable水平。Abstract
Structured network pruning is a practical approach to reduce computation cost directly while retaining the CNNs' generalization performance in real applications. However, identifying redundant filters is a core problem in structured network pruning, and current redundancy criteria only focus on individual filters' attributes. When pruning sparsity increases, these redundancy criteria are not effective or efficient enough. Since the filter-wise interaction also contributes to the CNN's prediction accuracy, we integrate the filter-wise interaction into the redundancy criterion. In our criterion, we introduce the filter importance and filter utilization strength to reflect the decision ability of individual and multiple filters. Utilizing this new redundancy criterion, we propose a structured network pruning approach SNPFI (Structured Network Pruning by measuring Filter-wise Interaction). During the pruning, the SNPFI can automatically assign the proper sparsity based on the filter utilization strength and eliminate the useless filters by filter importance. After the pruning, the SNPFI can recover pruned model's performance effectively without iterative training by minimizing the interaction difference. We empirically demonstrate the effectiveness of the SNPFI with several commonly used CNN models, including AlexNet, MobileNetv1, and ResNet-50, on various image classification datasets, including MNIST, CIFAR-10, and ImageNet. For all experimental CNN models, nearly 60% of computation is reduced in a network compression while the classification accuracy remains.
摘要
《结构化网络剔除》是一种实用的方法，可以直接降低计算成本，同时保持深度学习网络的泛化性能。然而，识别重复的滤波器是结构化网络剔除的核心问题，现有的重复性标准只考虑个体滤波器的特性。随着剔除率增加，这些重复性标准不再有效率。因为滤波器之间的交互也对深度学习网络的预测精度产生影响，我们将滤波器之间的交互纳入重复性标准中。在我们的标准中，我们引入了滤波器重要性和滤波器利用强度，以反映个体和多个滤波器的决策能力。通过这种新的重复性标准，我们提出了结构化网络剔除方法（SNPFI）。在剔除过程中，SNPFI可以自动根据滤波器利用强度分配适当的稀缺，并将无用的滤波器 eliminate 。剔除后，SNPFI可以有效地恢复剔除后的模型性能，无需迭代训练，只需要对交互差异进行最小化。我们通过多种常用的深度学习模型，包括AlexNet、MobileNetv1和ResNet-50，在多个图像分类任务上进行了实验，包括MNIST、CIFAR-10和ImageNet。对所有的实验深度学习模型来说，SNPFI可以在网络压缩中降低大约60%的计算成本，而且分类精度保持不变。
Graph-level Anomaly Detection via Hierarchical Memory Networks
results: 对16个真实的图据集进行了广泛的实验，并证明了本方法在检测本地异常图和全球异常图方面具有显著优势，同时也具有抗异常杂质特性。Abstract
Graph-level anomaly detection aims to identify abnormal graphs that exhibit deviant structures and node attributes compared to the majority in a graph set. One primary challenge is to learn normal patterns manifested in both fine-grained and holistic views of graphs for identifying graphs that are abnormal in part or in whole. To tackle this challenge, we propose a novel approach called Hierarchical Memory Networks (HimNet), which learns hierarchical memory modules -- node and graph memory modules -- via a graph autoencoder network architecture. The node-level memory module is trained to model fine-grained, internal graph interactions among nodes for detecting locally abnormal graphs, while the graph-level memory module is dedicated to the learning of holistic normal patterns for detecting globally abnormal graphs. The two modules are jointly optimized to detect both locally- and globally-anomalous graphs. Extensive empirical results on 16 real-world graph datasets from various domains show that i) HimNet significantly outperforms the state-of-art methods and ii) it is robust to anomaly contamination. Codes are available at: https://github.com/Niuchx/HimNet.
摘要
GRAPH-LEVEL ANOMALY DETECTION 目标是识别异常图，其表现出异常结构和节点属性比多数图在图集中异常。一个主要挑战是学习图集中的正常模式，包括细化和总体两种视图。为解决这个挑战，我们提出了一种新的方法called Hierarchical Memory Networks（HimNet）。这种方法通过图自动编码网络架构学习层次记忆模块——节点记忆模块和图记忆模块。节点级别记忆模块用于模型图内节点之间的细化交互，检测本地异常图，而图级别记忆模块专门学习总体正常模式，检测全局异常图。两个模块共同优化，可以检测本地和全局异常图。我们对16个真实世界图据集进行了广泛的实验，结果显示：i) HimNet在比较方法中显著超越，ii) 它对异常污染具有较好的抗性。codes可以在https://github.com/Niuchx/HimNet中找到。
LXL: LiDAR Excluded Lean 3D Object Detection with 4D Imaging Radar and Camera Fusion
results: 实验结果表明，提出的方法在VoD和TJ4DRadSet数据集上显著超越了现有的3D对象检测方法，而且在不同的提升设定下进行比较性能表现最佳。Abstract
As an emerging technology and a relatively affordable device, the 4D imaging radar has already been confirmed effective in performing 3D object detection in autonomous driving. Nevertheless, the sparsity and noisiness of 4D radar point clouds hinder further performance improvement, and in-depth studies about its fusion with other modalities are lacking. On the other hand, most of the camera-based perception methods transform the extracted image perspective view features into the bird's-eye view geometrically via "depth-based splatting" proposed in Lift-Splat-Shoot (LSS), and some researchers exploit other modals such as LiDARs or ordinary automotive radars for enhancement. Recently, a few works have applied the "sampling" strategy for image view transformation, showing that it outperforms "splatting" even without image depth prediction. However, the potential of "sampling" is not fully unleashed. In this paper, we investigate the "sampling" view transformation strategy on the camera and 4D imaging radar fusion-based 3D object detection. In the proposed model, LXL, predicted image depth distribution maps and radar 3D occupancy grids are utilized to aid image view transformation, called "radar occupancy-assisted depth-based sampling". Experiments on VoD and TJ4DRadSet datasets show that the proposed method outperforms existing 3D object detection methods by a significant margin without bells and whistles. Ablation studies demonstrate that our method performs the best among different enhancement settings.
摘要
“4D射频激光作为新兴技术和相对较便宜的设备，已经确认了其在自动驾驶中的3D物体探测效果。然而，4D射频激光点云的罕见和噪音妨碍了更进一步的性能改善，而且与其他感知modalities的混合研究相对较少。相反，大多数基于摄像头的视觉感知方法将提取到的图像视角特征转换为鸟瞰视 geometrically via "depth-based splatting"提出的Lift-Splat-Shoot (LSS)，一些研究人员将其他modalities，如LiDAR或普通的汽车激光，用于增强。最近，一些工作已经应用了"sampling"策略来对图像视角进行变换，显示它比"splatting"更好，即使不需要图像深度预测。然而，"sampling"的潜力仍未被完全发掘。在本文中，我们 investigate "sampling" 视角变换策略在摄像头和4D射频激光融合基础的3D物体探测中。在我们的提案中，预测的图像深度分布图和4D射频激光3Doccupancy grid被用来帮助图像视角变换，称为"radar occupancy-assisted depth-based sampling"。实验结果显示，我们的方法在VoD和TJ4DRadSet datasets上具有明显的优势，而且ablation study显示我们的方法在不同的增强设定中表现最好。”
SSC3OD: Sparsely Supervised Collaborative 3D Object Detection from LiDAR Point Clouds
results: 对三个大规模数据集进行了广泛的实验，结果表明，提出的SSC3OD方法可以有效地提高稀疑指标学习的协同3D物体检测效果。Abstract
Collaborative 3D object detection, with its improved interaction advantage among multiple agents, has been widely explored in autonomous driving. However, existing collaborative 3D object detectors in a fully supervised paradigm heavily rely on large-scale annotated 3D bounding boxes, which is labor-intensive and time-consuming. To tackle this issue, we propose a sparsely supervised collaborative 3D object detection framework SSC3OD, which only requires each agent to randomly label one object in the scene. Specifically, this model consists of two novel components, i.e., the pillar-based masked autoencoder (Pillar-MAE) and the instance mining module. The Pillar-MAE module aims to reason over high-level semantics in a self-supervised manner, and the instance mining module generates high-quality pseudo labels for collaborative detectors online. By introducing these simple yet effective mechanisms, the proposed SSC3OD can alleviate the adverse impacts of incomplete annotations. We generate sparse labels based on collaborative perception datasets to evaluate our method. Extensive experiments on three large-scale datasets reveal that our proposed SSC3OD can effectively improve the performance of sparsely supervised collaborative 3D object detectors.
摘要
共享3D对象检测已经广泛研究在自动驾驶领域，但现有的共享3D对象检测器在完全监督 paradigm中仅靠据大规模标注的3D包bounding box，这是劳动密集和时间消耗的。为解决这个问题，我们提出了一个稀疏监督的共享3D对象检测框架SSC3OD，它只需每个代理 randomly标注Scene中的一个对象。具体来说，该模型包括两个新的组件：即柱体基于的masked autoencoder（Pillar-MAE）和实例挖掘模块。Pillar-MAE模块 aspires to reason over high-level semantics in a self-supervised manner，而实例挖掘模块在线生成高质量的pseudo标签 для共享检测器。通过引入这些简单 yet effective mechanism，我们的提出的SSC3OD可以减轻不完整的标注的负面影响。我们基于共享感知 dataset generate sparse labels来评估我们的方法。我们在三个大规模dataset上进行了广泛的实验，结果显示，我们的提出的SSC3OD可以有效地提高稀疏监督的共享3D对象检测器的性能。
JourneyDB: A Benchmark for Generative Image Understanding
results: 我们通过对当前领域的状态级模型进行应用测试，发现它们在生成图像理解方面存在一定的局限性和缺陷。我们还进行了一个深入的分析，以帮助改进现有的多模态模型。Abstract
While recent advancements in vision-language models have revolutionized multi-modal understanding, it remains unclear whether they possess the capabilities of comprehending the generated images. Compared to real data, synthetic images exhibit a higher degree of diversity in both content and style, for which there are significant difficulties for the models to fully apprehend. To this end, we present a large-scale dataset, JourneyDB, for multi-modal visual understanding in generative images. Our curated dataset covers 4 million diverse and high-quality generated images paired with the text prompts used to produce them. We further design 4 benchmarks to quantify the performance of generated image understanding in terms of both content and style interpretation. These benchmarks include prompt inversion, style retrieval, image captioning and visual question answering. Lastly, we assess the performance of current state-of-the-art multi-modal models when applied to JourneyDB, and provide an in-depth analysis of their strengths and limitations in generated content understanding. We hope the proposed dataset and benchmarks will facilitate the research in the field of generative content understanding. The dataset will be available on https://journeydb.github.io.
摘要
“Recent advancements in vision-language models have significantly improved multi-modal understanding, but it remains unclear whether these models can fully comprehend generated images. Synthetic images exhibit a higher degree of diversity in both content and style, which poses challenges for the models to fully apprehend. To address this, we present JourneyDB, a large-scale dataset for multi-modal visual understanding in generative images. Our dataset includes 4 million high-quality and diverse generated images paired with text prompts used to produce them. We also design four benchmarks to evaluate the performance of generated image understanding in terms of content and style interpretation. These benchmarks include prompt inversion, style retrieval, image captioning, and visual question answering. Finally, we assess the performance of current state-of-the-art multi-modal models on JourneyDB and provide an in-depth analysis of their strengths and limitations in generated content understanding. We hope that the proposed dataset and benchmarks will facilitate research in the field of generative content understanding. The dataset will be available on .”Note that the translation is in Simplified Chinese, which is the standard writing system used in mainland China. If you prefer Traditional Chinese, please let me know and I can provide the translation in that format as well.
Guided Patch-Grouping Wavelet Transformer with Spatial Congruence for Ultra-High Resolution Segmentation
methods: 该方法基于一种新的Transformer（$\mathcal{T}$）-Convolutional Neural Network（CNN，$\mathcal{C}$）共享学习框架，其中$\mathcal{T}$使用整个UHR图像作为输入，并提取了本地细节和长距离Contextual Dependencies。而$\mathcal{C}$使用下采样后的图像作为输入，以学习类别级别的深度Context。为了提高推理速度和计算复杂度，$\mathcal{T}$将原始UHR图像切分成patches，并将其分组动态，然后使用轻量级多头波峰 transformer（WFormer）网络来学习低级的本地细节。同时，这个过程中也捕捉了远距离的Contextual Dependencies，因为patches可以在空间领域中被分配到同一组。此外，$\mathcal{C}$生成的mask也被使用来导引patch grouping过程，提供了一种决策指南。此外，两个分支之间的协同约束也被利用，以保持patches之间的空间一致。总的来说，我们堆叠了多个阶段进程，形成一个pyramid结构。实验结果显示，GPWFormer在五个benchmark数据集上实现了显著的提升。Abstract
Most existing ultra-high resolution (UHR) segmentation methods always struggle in the dilemma of balancing memory cost and local characterization accuracy, which are both taken into account in our proposed Guided Patch-Grouping Wavelet Transformer (GPWFormer) that achieves impressive performances. In this work, GPWFormer is a Transformer ($\mathcal{T}$)-CNN ($\mathcal{C}$) mutual leaning framework, where $\mathcal{T}$ takes the whole UHR image as input and harvests both local details and fine-grained long-range contextual dependencies, while $\mathcal{C}$ takes downsampled image as input for learning the category-wise deep context. For the sake of high inference speed and low computation complexity, $\mathcal{T}$ partitions the original UHR image into patches and groups them dynamically, then learns the low-level local details with the lightweight multi-head Wavelet Transformer (WFormer) network. Meanwhile, the fine-grained long-range contextual dependencies are also captured during this process, since patches that are far away in the spatial domain can also be assigned to the same group. In addition, masks produced by $\mathcal{C}$ are utilized to guide the patch grouping process, providing a heuristics decision. Moreover, the congruence constraints between the two branches are also exploited to maintain the spatial consistency among the patches. Overall, we stack the multi-stage process in a pyramid way. Experiments show that GPWFormer outperforms the existing methods with significant improvements on five benchmark datasets.
摘要
现有的超高分辨率（UHR）分割方法都受到内存成本和地方特征精度之间的折衔，我们提出的指导 patch-组合波峰变换器（GPWFormer）可以达到出色的表现。在这项工作中，GPWFormer是一种基于Transformer（$\mathcal{T}$）和Convolutional Neural Network（$\mathcal{C}$）的互相学习框架，其中$\mathcal{T}$ 将整个UHR图像作为输入，并提取了地方细节和细腻的长距离Contextual Dependencies，而$\mathcal{C}$ 则将降采样后的图像作为输入，以学习类别深度的Context。为了提高推理速度和计算复杂度，$\mathcal{T}$ 将原始UHR图像 partitioned 成patches，并将其们组合成动态组。在这个过程中，GPWFormer网络学习了低级的地方细节，同时也捕捉了远距离的Contextual Dependencies，因为patches可以在空间领域中被分配到同一个组。此外，$\mathcal{C}$ 生成的mask也被利用来引导patch grouping процесс，提供了一个决策指标。此外，我们还利用了两支分支之间的协调约束，以保持patches中的空间一致。总的来说，我们将多个阶段过程堆叠在峰形式上。实验表明，GPWFormer在五个基准数据集上表现出了显著的改善。
Efficient Visual Fault Detection for Freight Train Braking System via Heterogeneous Self Distillation in the Wild
results: 实验结果显示，我们的框架可以在四个瑕疵Dataset上达到37帧/秒的速度，并且与传统混合条件下的方法相比，具有更高的精度和更低的内存使用量。Abstract
Efficient visual fault detection of freight trains is a critical part of ensuring the safe operation of railways under the restricted hardware environment. Although deep learning-based approaches have excelled in object detection, the efficiency of freight train fault detection is still insufficient to apply in real-world engineering. This paper proposes a heterogeneous self-distillation framework to ensure detection accuracy and speed while satisfying low resource requirements. The privileged information in the output feature knowledge can be transferred from the teacher to the student model through distillation to boost performance. We first adopt a lightweight backbone to extract features and generate a new heterogeneous knowledge neck. Such neck models positional information and long-range dependencies among channels through parallel encoding to optimize feature extraction capabilities. Then, we utilize the general distribution to obtain more credible and accurate bounding box estimates. Finally, we employ a novel loss function that makes the network easily concentrate on values near the label to improve learning efficiency. Experiments on four fault datasets reveal that our framework can achieve over 37 frames per second and maintain the highest accuracy in comparison with traditional distillation approaches. Moreover, compared to state-of-the-art methods, our framework demonstrates more competitive performance with lower memory usage and the smallest model size.
摘要
高效的货运列车缺陷检测是铁路安全运行的关键部分，但现有的深度学习方法在实际工程环境中并不够高效。这篇论文提出了一种多元自适应框架，以确保检测精度和速度，同时满足资源限制。我们首先采用轻量级核心EXTRACT Features和生成新的多元知识脖子。这种脖子模型通过并行编码来提高特征提取能力，并使用通用分布来获取更可靠和准确的 bounding box 估计。最后，我们采用一种新的损失函数，使网络更容易集中于标签附近的值来提高学习效率。实验结果表明，我们的框架可以达到37帧/秒的速度，并与传统束Distillation方法相比，保持最高的准确率。此外，与当前状态艺术方法相比，我们的框架具有较低的内存使用量和最小的模型大小。
Camera Calibration from a Single Imaged Ellipsoid: A Moon Calibration Algorithm
results: 该算法可以在单个图像基础上计算摄像头的 focal length 和主点，并且可以在多张图像基础上减小一标准差的不确定性。Abstract
This work introduces a method that applies images of the extended bodies in the solar system to spacecraft camera calibration. The extended bodies consist of planets and moons that are well-modeled by triaxial ellipsoids. When imaged, the triaxial ellipsoid projects to a conic section which is generally an ellipse. This work combines the imaged ellipse with information on the observer's target-relative state to achieve camera calibration from a single imaged ellipsoid. As such, this work is the first to accomplish camera calibration from a single, non-spherical imaged ellipsoid. The camera calibration algorithm is applied to synthetic images of ellipsoids as well as planetary images of Saturn's moons as captured by the Cassini spacecraft. From a single image, the algorithm estimates the focal length and principal point of Cassini's Narrow Angle Camera within 1.0 mm and 10 pixels, respectively. With multiple images, the one standard deviation uncertainty in focal length and principal point estimates reduce to 0.5 mm and 3.1 pixels, respectively. Though created for spacecraft camera calibration in mind, this work also generalizes to terrestrial camera calibration using any number of imaged ellipsoids.
摘要
results: 实验表明，使用 proximal NS 可以在一次训练中获得比较精炼的 CNN 模型，并且其减少后的精度与原始模型相似。 这 paper 在 VGGNet、DenseNet 和 ResNet 上进行了 CIFAR 10/100 的实验 validate 了 proximal NS 的效果。Abstract
As a popular channel pruning method for convolutional neural networks (CNNs), network slimming (NS) has a three-stage process: (1) it trains a CNN with $\ell_1$ regularization applied to the scaling factors of the batch normalization layers; (2) it removes channels whose scaling factors are below a chosen threshold; and (3) it retrains the pruned model to recover the original accuracy. This time-consuming, three-step process is a result of using subgradient descent to train CNNs. Because subgradient descent does not exactly train CNNs towards sparse, accurate structures, the latter two steps are necessary. Moreover, subgradient descent does not have any convergence guarantee. Therefore, we develop an alternative algorithm called proximal NS. Our proposed algorithm trains CNNs towards sparse, accurate structures, so identifying a scaling factor threshold is unnecessary and fine tuning the pruned CNNs is optional. Using Kurdyka-{\L}ojasiewicz assumptions, we establish global convergence of proximal NS. Lastly, we validate the efficacy of the proposed algorithm on VGGNet, DenseNet and ResNet on CIFAR 10/100. Our experiments demonstrate that after one round of training, proximal NS yields a CNN with competitive accuracy and compression.
摘要
为了减少 convolutional neural networks (CNNs) 的复杂度，网络精简 (NS) 是一种广泛使用的频道剪除方法，它包括以下三个阶段：1. 使用 $\ell_1$ regularization 对 batch normalization layers 的扩大因子进行训练;2. removing channels whose scaling factors 是下于选择的阈值;3. retrains 剪除后的模型，以回升原始精度。这个时间消耗大，三步骤的过程是因为使用 subgradient descent 训练 CNNs。因为 subgradient descent 不是训练 CNNs 向稀疏、准确结构的算法，因此需要后两个步骤。而且，subgradient descent 没有任何收敛保证。因此，我们开发了一种替代算法 called proximal NS。我们的提议的算法 trains CNNs 向稀疏、准确结构，因此无需确定 scaling factor 阈值，并且 fine-tuning 剪除后的 CNNs 是可选的。使用 Kurdyka-{\L}ojasiewicz 假设，我们确立 proximal NS 的全球收敛性。最后，我们验证了我们的算法在 VGGNet、DenseNet 和 ResNet 上的 CIFAR 10/100 表现，实验表明，在一次训练后，proximal NS 可以生成一个与其他方法相当的精度和压缩率的 CNN。
Pay Attention to the Atlas: Atlas-Guided Test-Time Adaptation Method for Robust 3D Medical Image Segmentation
results: 在多个不同来源的数据集上进行了广泛的实验，结果表明AdaAtlas-Attention方法在提高3D医学成像 segmentation模型的Robustness方面表现出色，至少比其他竞争方法更好。Abstract
Convolutional neural networks (CNNs) often suffer from poor performance when tested on target data that differs from the training (source) data distribution, particularly in medical imaging applications where variations in imaging protocols across different clinical sites and scanners lead to different imaging appearances. However, re-accessing source training data for unsupervised domain adaptation or labeling additional test data for model fine-tuning can be difficult due to privacy issues and high labeling costs, respectively. To solve this problem, we propose a novel atlas-guided test-time adaptation (TTA) method for robust 3D medical image segmentation, called AdaAtlas. AdaAtlas only takes one single unlabeled test sample as input and adapts the segmentation network by minimizing an atlas-based loss. Specifically, the network is adapted so that its prediction after registration is aligned with the learned atlas in the atlas space, which helps to reduce anatomical segmentation errors at test time. In addition, different from most existing TTA methods which restrict the adaptation to batch normalization blocks in the segmentation network only, we further exploit the use of channel and spatial attention blocks for improved adaptability at test time. Extensive experiments on multiple datasets from different sites show that AdaAtlas with attention blocks adapted (AdaAtlas-Attention) achieves superior performance improvements, greatly outperforming other competitive TTA methods.
摘要
循环神经网络（CNN）在测试数据不同于训练数据分布时表现不佳，特别在医疗影像应用中，因为不同的临床站点和扫描仪器使用不同的扫描技术，导致影像的显示不同。然而，为了解决这个问题，我们提出了一种名为 AdaAtlas 的新的测试时适应（TTA）方法，它只需要一个无标签的测试样本作为输入，然后使用了一个名为 AdaAtlas 的扩展。在这个扩展中，我们将网络的预测结果与学习的床表空间中的 Atlas 进行了对齐，以降低测试时的解剖学 segmentation 错误。此外，与大多数现有的 TTA 方法不同，我们还在测试时使用了通道和空间注意力块，以提高适应性。经验表明，AdaAtlas-Attention 在多个数据集上表现出色，与其他竞争性 TTA 方法相比，具有显著的性能提升。
Real-time Vision-based Navigation for a Robot in an Indoor Environment
results: 研究结果显示了系统的表现，包括质量和量化的指标，展示了这个系统在实时自主 Navigation 中的潜力和局限性。Abstract
This paper presents a study on the development of an obstacle-avoidance navigation system for autonomous navigation in home environments. The system utilizes vision-based techniques and advanced path-planning algorithms to enable the robot to navigate toward the destination while avoiding obstacles. The performance of the system is evaluated through qualitative and quantitative metrics, highlighting its strengths and limitations. The findings contribute to the advancement of indoor robot navigation, showcasing the potential of vision-based techniques for real-time, autonomous navigation.
摘要
这篇论文介绍了一项关于自主navigation在家庭环境中的障碍避免导航系统的研究。该系统利用视觉技术和高级路径规划算法，使机器人能够实时避免障碍物而前往目的地点。研究结果通过质量和量度指标进行评估，把系统的优点和局限性得到展示。发现有助于家庭环境内自主机器人导航的发展，同时也展示了视觉技术在实时自主导航中的潜在优势。
CNN-BiLSTM model for English Handwriting Recognition: Comprehensive Evaluation on the IAM Dataset
for: 英文手写识别问题的 CNN-BiLSTM 系统，进行了大量的 IAM 数据集上的评估，包括模型大小、数据增强和词典的影响。
methods: 使用 CNN-BiLSTM 网络和 CTC 层进行模型建构，并应用测试时数据增强以提高Recognition of difficult cases。
results: 最佳模型可以达到 3.59% CER 和 9.44% WER，并通过对 IAM 数据集进行错误分析，显示了一些困难的手写图像和错误标注的样本。Abstract
We present a CNN-BiLSTM system for the problem of offline English handwriting recognition, with extensive evaluations on the public IAM dataset, including the effects of model size, data augmentation and the lexicon. Our best model achieves 3.59\% CER and 9.44\% WER using CNN-BiLSTM network with CTC layer. Test time augmentation with rotation and shear transformations applied to the input image, is proposed to increase recognition of difficult cases and found to reduce the word error rate by 2.5\% points. We also conduct an error analysis of our proposed method on IAM dataset, show hard cases of handwriting images and explore samples with erroneous labels. We provide our source code as public-domain, to foster further research to encourage scientific reproducibility.
摘要
我们提出了一个基于CNN-BiLSTM的英文手写识别系统，对公共IAM数据集进行了广泛的评估，包括模型大小、数据增强和词典的影响。我们的最佳模型在CNN-BiLSTM网络中使用CTC层时达到了3.59%的字幕误差率和9.44%的单词错误率。我们提议在输入图像上应用旋转和倾斜变换来增强难以识别的情况的识别，并发现这可以降低单词错误率2.5个百分点。我们还进行了我们的提议方法的错误分析在IAM数据集上，显示了困难的手写图像示例和错误标签的样本。我们将我们的源代码作为公共领域发布，以便进一步的研究，促进科学复现性。
More Synergy, Less Redundancy: Exploiting Joint Mutual Information for Self-Supervised Learning
methods: 这篇论文使用了一种新的多变量信息测量perspective，即partial information decomposition（PID），来检测SSL模型中的相互信息。PID可以将共同信息分解成三个重要组成部分：唯一信息、重复信息和合作信息。
results: 该研究的实验结果表明，通过最小化视图之间的重复信息并最大化目标表示和视图之间的合作信息，可以提高SSL模型的性能。此外，该研究还提出了一种新的SSL训练协议。多个数据集和两个下游任务的广泛实验结果证明了该框架的效果。Abstract
Self-supervised learning (SSL) is now a serious competitor for supervised learning, even though it does not require data annotation. Several baselines have attempted to make SSL models exploit information about data distribution, and less dependent on the augmentation effect. However, there is no clear consensus on whether maximizing or minimizing the mutual information between representations of augmentation views practically contribute to improvement or degradation in performance of SSL models. This paper is a fundamental work where, we investigate role of mutual information in SSL, and reformulate the problem of SSL in the context of a new perspective on mutual information. To this end, we consider joint mutual information from the perspective of partial information decomposition (PID) as a key step in \textbf{reliable multivariate information measurement}. PID enables us to decompose joint mutual information into three important components, namely, unique information, redundant information and synergistic information. Our framework aims for minimizing the redundant information between views and the desired target representation while maximizing the synergistic information at the same time. Our experiments lead to a re-calibration of two redundancy reduction baselines, and a proposal for a new SSL training protocol. Extensive experimental results on multiple datasets and two downstream tasks show the effectiveness of this framework.
摘要
自我监督学习（SSL）现在是指导学习的严重竞争对手，即使它不需要数据标注。许多基线已经尝试使 SSL 模型利用数据分布信息，并且减少扩展效应的依赖。然而，没有一致的共识，是否在 SSL 模型性能上实际上提高或下降的情况。这篇论文是一项基础性工作，我们调查了 SSL 中积分信息的角色，并将问题重新定义为基于新的积分信息角度。为此，我们使用部分信息分解（PID）来分解共同积分信息为三个重要组成部分：唯一信息、重复信息和协同信息。我们的框架的目标是减少视图之间的重复信息和目标表示之间的重复信息，同时增加协同信息。我们的实验导致了两种减少重复基elines的重新调整，以及一个新的 SSL 训练协议的建议。我们的实验结果表明，这个框架在多个数据集和两个下游任务上具有高效性。
For: The paper aims to help test the mathematical reasoning capabilities of learners in online assessments by retrieving similar math word problems (MWPs) with the same problem model.* Methods: The authors propose a hybrid approach that combines natural language processing (NLP) and machine learning techniques to retrieve similar MWPs.* Results: The authors demonstrate that their tool is effective in retrieving similar MWPs and outperforms semantic similarity-based approaches, which fail to capture the arithmetic and logical sequence of the MWPs.Here is the same information in Simplified Chinese text:
results: 作者们证明了他们的工具能够有效地检索类似的MWPs，并超过 semantic similarity-based approaches，这些方法无法捕捉MWPs中的数学和逻辑序列。Abstract
Math Word Problems (MWPs) in online assessments help test the ability of the learner to make critical inferences by interpreting the linguistic information in them. To test the mathematical reasoning capabilities of the learners, sometimes the problem is rephrased or the thematic setting of the original MWP is changed. Since manual identification of MWPs with similar problem models is cumbersome, we propose a tool in this work for MWP retrieval. We propose a hybrid approach to retrieve similar MWPs with the same problem model. In our work, the problem model refers to the sequence of operations to be performed to arrive at the solution. We demonstrate that our tool is useful for the mentioned tasks and better than semantic similarity-based approaches, which fail to capture the arithmetic and logical sequence of the MWPs. A demo of the tool can be found at https://www.youtube.com/watch?v=gSQWP3chFIs
摘要
mathew word problems (MWPs) 在在线评估中帮助测试学习者能否作出重要的推理推论， interpret linguistic information中的信息。为测试学习者的数学推理能力，有时MWPs中的问题会被重新词表示或者改变主题设定。由于手动标识类似MWPs的问题模型是繁琐的，我们在这种工作中提出了一种工具。我们提议的方法是将问题模型看作解决问题所需的操作序列。我们示示了我们的工具对于以下任务都很有用，并且比 semantic similarity 基于的方法更好，因为它们无法捕捉MWPs中的数学和逻辑序列。您可以在 https://www.youtube.com/watch?v=gSQWP3chFIs 上找到我们的示例。
Automated identification and quantification of myocardial inflammatory infiltration in digital histological images to diagnose myocarditis
results: 研究发现，使用这种新方法可以准确地识别和量化心脏免疫性浸泡，并提供了一个可靠的诊断标准。在五个横向分割试验中，该方法的准确率、敏感度和特异性分别为0.899±0.035、0.971±0.017和0.728±0.073，而在内测集上的准确率、敏感度和特异性分别为0.887、0.971和0.737。 externally tested set 的准确率、敏感度和特异性分别为0.853、0.846和0.858。Abstract
This study aims to develop a new computational pathology approach that automates the identification and quantification of myocardial inflammatory infiltration in digital HE-stained images to provide a quantitative histological diagnosis of myocarditis.898 HE-stained whole slide images (WSIs) of myocardium from 154 heart transplant patients diagnosed with myocarditis or dilated cardiomyopathy (DCM) were included in this study. An automated DL-based computational pathology approach was developed to identify nuclei and detect myocardial inflammatory infiltration, enabling the quantification of the lymphocyte nuclear density (LND) on myocardial WSIs. A cutoff value based on the quantification of LND was proposed to determine if the myocardial inflammatory infiltration was present. The performance of our approach was evaluated with a five-fold cross-validation experiment, tested with an internal test set from the myocarditis group, and confirmed by an external test from a double-blind trial group. An LND of 1.02/mm2 could distinguish WSIs with myocarditis from those without. The accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) in the five-fold cross-validation experiment were 0.899 plus or minus 0.035, 0.971 plus or minus 0.017, 0.728 plus or minus 0.073 and 0.849 plus or minus 0.044, respectively. For the internal test set, the accuracy, sensitivity, specificity, and AUC were 0.887, 0.971, 0.737, and 0.854, respectively. The accuracy, sensitivity, specificity, and AUC for the external test set reached 0.853, 0.846, 0.858, and 0.852, respectively. Our new approach provides accurate and reliable quantification of the LND of myocardial WSIs, facilitating automated quantitative diagnosis of myocarditis with HE-stained images.
摘要
898 HE-stained whole slide images (WSIs) of myocardium from 154 heart transplant patients diagnosed with myocarditis or dilated cardiomyopathy (DCM) were included in this study. An automated DL-based computational pathology approach was developed to identify nuclei and detect myocardial inflammatory infiltration, enabling the quantification of the lymphocyte nuclear density (LND) on myocardial WSIs. A cutoff value based on the quantification of LND was proposed to determine if the myocardial inflammatory infiltration was present.The performance of our approach was evaluated with a five-fold cross-validation experiment, tested with an internal test set from the myocarditis group, and confirmed by an external test from a double-blind trial group. An LND of 1.02/mm2 could distinguish WSIs with myocarditis from those without.The accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) in the five-fold cross-validation experiment were 0.899 ± 0.035, 0.971 ± 0.017, 0.728 ± 0.073, and 0.849 ± 0.044, respectively. For the internal test set, the accuracy, sensitivity, specificity, and AUC were 0.887, 0.971, 0.737, and 0.854, respectively. The accuracy, sensitivity, specificity, and AUC for the external test set reached 0.853, 0.846, 0.858, and 0.852, respectively.Our new approach provides accurate and reliable quantification of the LND of myocardial WSIs, facilitating automated quantitative diagnosis of myocarditis with HE-stained images.
Some challenges of calibrating differentiable agent-based models
paper_authors: Arnau Quera-Bofarull, Joel Dyer, Anisoara Calinescu, Michael Wooldridge
for: 模拟复杂系统的方法
methods: 使用梯度可微的ABM，并对其进行参数推理和优化
results: 发现了一些挑战，以及可能的解决方案Abstract
Agent-based models (ABMs) are a promising approach to modelling and reasoning about complex systems, yet their application in practice is impeded by their complexity, discrete nature, and the difficulty of performing parameter inference and optimisation tasks. This in turn has sparked interest in the construction of differentiable ABMs as a strategy for combatting these difficulties, yet a number of challenges remain. In this paper, we discuss and present experiments that highlight some of these challenges, along with potential solutions.
摘要
The ROAD to discovery: machine learning-driven anomaly detection in radio astronomy spectrograms
results: 研究结果表明，ROAD框架可以实时处理LOFAR数据处理管道中的数据，需要 <1ms 处理一个spectrogram，并且具有异常检测 F-2 分数0.92，false positive rate约2%，以及每类分类 F-2 分数0.89，超过其他相关研究。Abstract
As radio telescopes increase in sensitivity and flexibility, so do their complexity and data-rates. For this reason automated system health management approaches are becoming increasingly critical to ensure nominal telescope operations. We propose a new machine learning anomaly detection framework for classifying both commonly occurring anomalies in radio telescopes as well as detecting unknown rare anomalies that the system has potentially not yet seen. To evaluate our method, we present a dataset consisting of 7050 autocorrelation-based spectrograms from the Low Frequency Array (LOFAR) telescope and assign 10 different labels relating to the system-wide anomalies from the perspective of telescope operators. This includes electronic failures, miscalibration, solar storms, network and compute hardware errors among many more. We demonstrate how a novel Self Supervised Learning (SSL) paradigm, that utilises both context prediction and reconstruction losses, is effective in learning normal behaviour of the LOFAR telescope. We present the Radio Observatory Anomaly Detector (ROAD), a framework that combines both SSL-based anomaly detection and a supervised classification, thereby enabling both classification of both commonly occurring anomalies and detection of unseen anomalies. We demonstrate that our system is real-time in the context of the LOFAR data processing pipeline, requiring <1ms to process a single spectrogram. Furthermore, ROAD obtains an anomaly detection F-2 score of 0.92 while maintaining a false positive rate of ~2\%, as well as a mean per-class classification F-2 score 0.89, outperforming other related works.
摘要
Radio telescopes 的敏感度和灵活性在不断提高，但是同时也会增加复杂性和数据流量。为了保证望远镜的正常运行，自动化系统健康管理方法已成为非常重要。我们提出了一种基于机器学习的异常检测框架，可以检测望远镜中常见的异常情况以及未经见过的未知异常。为评估我们的方法，我们提供了一个包含7050个自相关спектрограм的LOFAR望远镜数据集，并将望远镜系统异常情况分为10个不同的标签，包括电子故障、误准、太阳风暴、网络和计算机硬件错误等。我们展示了一种新的自我超vision学习（SSL）方法，通过 Context prediction和重建损失来学习望远镜的正常行为。我们称之为Radio Observatory Anomaly Detector（ROAD），它将SSL-based异常检测和超级vised分类相结合，以实现对望远镜系统的异常检测和分类。我们证明了ROAD在LOFAR数据处理管道中的实时性，需要<1ms处理一个spectrogram，并且ROAD在异常检测F-2分数0.92，False Positive率~2%，以及每个类别的平均异常检测F-2分数0.89，超过其他相关的工作。
ENGAGE: Explanation Guided Data Augmentation for Graph Representation Learning
results: 实验表明，ENGAGE可以在多种模型架构和真实图数据上实现高效的表示学习，并且可以适应不同的图数据。Abstract
The recent contrastive learning methods, due to their effectiveness in representation learning, have been widely applied to modeling graph data. Random perturbation is widely used to build contrastive views for graph data, which however, could accidentally break graph structures and lead to suboptimal performance. In addition, graph data is usually highly abstract, so it is hard to extract intuitive meanings and design more informed augmentation schemes. Effective representations should preserve key characteristics in data and abandon superfluous information. In this paper, we propose ENGAGE (ExplaNation Guided data AuGmEntation), where explanation guides the contrastive augmentation process to preserve the key parts in graphs and explore removing superfluous information. Specifically, we design an efficient unsupervised explanation method called smoothed activation map as the indicator of node importance in representation learning. Then, we design two data augmentation schemes on graphs for perturbing structural and feature information, respectively. We also provide justification for the proposed method in the framework of information theories. Experiments of both graph-level and node-level tasks, on various model architectures and on different real-world graphs, are conducted to demonstrate the effectiveness and flexibility of ENGAGE. The code of ENGAGE can be found: https://github.com/sycny/ENGAGE.
摘要
Recent contrastive learning methods have been widely applied to modeling graph data due to their effectiveness in representation learning. However, random perturbation, which is commonly used to build contrastive views for graph data, can accidentally break graph structures and lead to suboptimal performance. Moreover, graph data is often highly abstract, making it difficult to extract intuitive meanings and design more informed augmentation schemes. Effective representations should preserve key characteristics in the data and discard superfluous information.In this paper, we propose ENGAGE (ExplaNation Guided data AuGmEntation), which utilizes explanation to guide the contrastive augmentation process and preserve the key parts in graphs. Specifically, we design an efficient unsupervised explanation method called smoothed activation map as the indicator of node importance in representation learning. Additionally, we propose two data augmentation schemes on graphs for perturbing structural and feature information, respectively. We also provide justification for the proposed method in the framework of information theories.Experiments on both graph-level and node-level tasks, conducted on various model architectures and on different real-world graphs, demonstrate the effectiveness and flexibility of ENGAGE. The code of ENGAGE can be found at: .
Temporal Graph Benchmark for Machine Learning on Temporal Graphs
for: The paper is written for evaluating the performance of machine learning models on temporal graphs.
methods: The paper uses a collection of challenging and diverse benchmark datasets for realistic, reproducible, and robust evaluation of machine learning models on temporal graphs.
results: The paper finds that the performance of common models can vary drastically across datasets, and simple methods often achieve superior performance compared to existing temporal graph models.Here is the same information in Simplified Chinese text:
results: 本文发现，常见模型在不同的 dataset 上的性能可以差异很大，而简单的方法经常在现有的时间图模型上表现更优异。Abstract
We present the Temporal Graph Benchmark (TGB), a collection of challenging and diverse benchmark datasets for realistic, reproducible, and robust evaluation of machine learning models on temporal graphs. TGB datasets are of large scale, spanning years in duration, incorporate both node and edge-level prediction tasks and cover a diverse set of domains including social, trade, transaction, and transportation networks. For both tasks, we design evaluation protocols based on realistic use-cases. We extensively benchmark each dataset and find that the performance of common models can vary drastically across datasets. In addition, on dynamic node property prediction tasks, we show that simple methods often achieve superior performance compared to existing temporal graph models. We believe that these findings open up opportunities for future research on temporal graphs. Finally, TGB provides an automated machine learning pipeline for reproducible and accessible temporal graph research, including data loading, experiment setup and performance evaluation. TGB will be maintained and updated on a regular basis and welcomes community feedback. TGB datasets, data loaders, example codes, evaluation setup, and leaderboards are publicly available at https://tgb.complexdatalab.com/ .
摘要
我们介绍Temporal Graph Benchmark（TGB），一个包含具有具有真实、可重现和可靠性评估的机器学习模型的大规模、多样化和长时间 duration 的数据集集合。TGB 数据集覆盖了社交、贸易、交易和交通网络等多种领域，并包括节点和边级别预测任务。为了进行真实的评估，我们设计了基于实际用例的评估协议。我们对每个数据集进行了广泛的 benchmarking，发现了不同数据集下模型的性能可以有很大差异。此外，在动态节点属性预测任务上，我们发现了简单的方法经常超越了现有的时间图模型。我们认为这些发现开创了对时间图的未来研究的机会。此外，TGB 还提供了一个自动化的机器学习管道，包括数据加载、实验设置和性能评估。TGB 将在定期基础上维护和更新，欢迎社区反馈。TGB 数据集、数据加载器、示例代码、评估设置和排名是公共可用的，可以在 访问。
RefSAM: Efficiently Adapting Segmenting Anything Model for Referring Video Object Segmentation
results: 经过了广泛的缺省研究和实验， authors 表明了 RefSAM 模型的实用和有效性，并在 Ref-Youtu-VOS 和 Ref-DAVIS17 数据集上达到了最高性能。Abstract
The Segment Anything Model (SAM) has gained significant attention for its impressive performance in image segmentation. However, it lacks proficiency in referring video object segmentation (RVOS) due to the need for precise user-interactive prompts and limited understanding of different modalities, such as language and vision. This paper presents the RefSAM model, which for the first time explores the potential of SAM for RVOS by incorporating multi-view information from diverse modalities and successive frames at different timestamps. Our proposed approach adapts the original SAM model to enhance cross-modality learning by employing a lightweight Cross-Modal MLP that projects the text embedding of the referring expression into sparse and dense embeddings, serving as user-interactive prompts. Subsequently, a parameter-efficient tuning strategy is employed to effectively align and fuse the language and vision features. Through comprehensive ablation studies, we demonstrate the practical and effective design choices of our strategy. Extensive experiments conducted on Ref-Youtu-VOS and Ref-DAVIS17 datasets validate the superiority and effectiveness of our RefSAM model over existing methods. The code and models will be made publicly at \href{https://github.com/LancasterLi/RefSAM}{github.com/LancasterLi/RefSAM}.
摘要
《Segment Anything Model（SAM）》已经吸引了广泛关注，因为它在图像分割方面表现出了非常出色的表现。然而，它在视频对象分割（RVOS）方面缺乏能力，因为需要精准的用户交互提示和不同的modalities，如语言和视觉，的有限了理解。这篇论文提出了RefSAM模型，这是第一次将SAM模型应用于RVOS领域，通过在不同modalities和时间戳的多视图信息上进行协同学习。我们的提议的方法是对原始SAM模型进行改进，以增强对不同modalities的学习，通过使用轻量级的 Cross-Modal MLP 将文本表达的embedding进行映射，以用于用户交互提示。然后，我们采用了效果性的参数调整策略，以有效地对语言和视觉特征进行对齐和融合。通过了详细的ablation研究，我们证明了我们的方法的实用和有效性。广泛的实验在Ref-Youtu-VOS和Ref-DAVIS17 datasets上验证了我们的RefSAM模型的超越性和有效性。代码和模型将在 \href{https://github.com/LancasterLi/RefSAM}{github.com/LancasterLi/RefSAM} 上公开。
REAL: A Representative Error-Driven Approach for Active Learning
results: 对五种文本分类任务进行了广泛的实验，结果表明，与最佳基准相比，本方法在各种 гиперпараметры设置下都能够准确地预测模型性能和F1-macro分数。此外，分析还表明，本方法可以准确地选择最有代表性的 pseudo errors，这些 pseudo errors 与真实错误的分布相符。I hope this helps! Let me know if you have any other questions.Abstract
Given a limited labeling budget, active learning (AL) aims to sample the most informative instances from an unlabeled pool to acquire labels for subsequent model training. To achieve this, AL typically measures the informativeness of unlabeled instances based on uncertainty and diversity. However, it does not consider erroneous instances with their neighborhood error density, which have great potential to improve the model performance. To address this limitation, we propose $REAL$, a novel approach to select data instances with $\underline{R}$epresentative $\underline{E}$rrors for $\underline{A}$ctive $\underline{L}$earning. It identifies minority predictions as \emph{pseudo errors} within a cluster and allocates an adaptive sampling budget for the cluster based on estimated error density. Extensive experiments on five text classification datasets demonstrate that $REAL$ consistently outperforms all best-performing baselines regarding accuracy and F1-macro scores across a wide range of hyperparameter settings. Our analysis also shows that $REAL$ selects the most representative pseudo errors that match the distribution of ground-truth errors along the decision boundary. Our code is publicly available at https://github.com/withchencheng/ECML_PKDD_23_Real.
摘要
$REAL$ identifies minority predictions as "pseudo errors" within a cluster and allocates an adaptive sampling budget for the cluster based on estimated error density. Extensive experiments on five text classification datasets show that $REAL$ consistently outperforms all best-performing baselines in terms of accuracy and F1-macro scores across a wide range of hyperparameter settings. Our analysis also reveals that $REAL$ selects the most representative pseudo errors that match the distribution of ground-truth errors along the decision boundary.The code for $REAL$ is publicly available at https://github.com/withchencheng/ECML_PKDD_23_Real.
OpenClinicalAI: An Open and Dynamic Model for Alzheimer’s Disease Diagnosis
results: 实验结果显示，这个方法可以在临床设定下提供更好的性能和 fewer 的诊断测试，并且可以与现有的医疗系统整合，以提高现有的医疗服务质量。Abstract
Although Alzheimer's disease (AD) cannot be reversed or cured, timely diagnosis can significantly reduce the burden of treatment and care. Current research on AD diagnosis models usually regards the diagnosis task as a typical classification task with two primary assumptions: 1) All target categories are known a priori; 2) The diagnostic strategy for each patient is consistent, that is, the number and type of model input data for each patient are the same. However, real-world clinical settings are open, with complexity and uncertainty in terms of both subjects and the resources of the medical institutions. This means that diagnostic models may encounter unseen disease categories and need to dynamically develop diagnostic strategies based on the subject's specific circumstances and available medical resources. Thus, the AD diagnosis task is tangled and coupled with the diagnosis strategy formulation. To promote the application of diagnostic systems in real-world clinical settings, we propose OpenClinicalAI for direct AD diagnosis in complex and uncertain clinical settings. This is the first powerful end-to-end model to dynamically formulate diagnostic strategies and provide diagnostic results based on the subject's conditions and available medical resources. OpenClinicalAI combines reciprocally coupled deep multiaction reinforcement learning (DMARL) for diagnostic strategy formulation and multicenter meta-learning (MCML) for open-set recognition. The experimental results show that OpenClinicalAI achieves better performance and fewer clinical examinations than the state-of-the-art model. Our method provides an opportunity to embed the AD diagnostic system into the current health care system to cooperate with clinicians to improve current health care.
摘要
although 阿尔茨heimer's disease (AD) cannot be reversed or cured, 在时间上的诊断可以有效减轻治疗和照料的负担。现有的AD诊断模型通常将诊断任务视为一个典型的分类任务，有两个基本假设：1) 所有目标类别都是先知的; 2) 每个患者的诊断策略都是一致的，即每个患者的模型输入数据的数量和类型都是一样的。然而，现实世界的医疗设施是开放的，有 Complexity和不确定性，这意味着诊断模型可能会遇到未知的疾病类别，需要在患者的特定情况和可用的医疗资源基础上动态发展诊断策略。因此，AD诊断任务和诊断策略的形成是相互关联的。为了推动AD诊断系统在现实世界医疗设施中的应用，我们提出了OpenClinicalAI，这是一个直接用于AD诊断的Complex and uncertain clinical settings中的强大终端模型。OpenClinicalAI通过reciprocally coupled deep multiaction reinforcement learning (DMARL) 和 multicenter meta-learning (MCML) 结合，可以动态形成诊断策略，并根据患者的情况和可用的医疗资源提供诊断结果。实验结果表明，OpenClinicalAI在比较 estado-of-the-art 模型的情况下表现出更好的性能，并需要 fewer clinical examinations。我们的方法提供了一个机会，以便将AD诊断系统integrated into the current healthcare system，与临床医生合作，提高现有的医疗服务。
A Dual Stealthy Backdoor: From Both Spatial and Frequency Perspectives
results: 在四个数据集上，DUBA方法比 estado-of-the-art 后门攻击方法显著提高了攻击成功率和隐蔽性。Abstract
Backdoor attacks pose serious security threats to deep neural networks (DNNs). Backdoored models make arbitrarily (targeted) incorrect predictions on inputs embedded with well-designed triggers while behaving normally on clean inputs. Many works have explored the invisibility of backdoor triggers to improve attack stealthiness. However, most of them only consider the invisibility in the spatial domain without explicitly accounting for the generation of invisible triggers in the frequency domain, making the generated poisoned images be easily detected by recent defense methods. To address this issue, in this paper, we propose a DUal stealthy BAckdoor attack method named DUBA, which simultaneously considers the invisibility of triggers in both the spatial and frequency domains, to achieve desirable attack performance, while ensuring strong stealthiness. Specifically, we first use Discrete Wavelet Transform to embed the high-frequency information of the trigger image into the clean image to ensure attack effectiveness. Then, to attain strong stealthiness, we incorporate Fourier Transform and Discrete Cosine Transform to mix the poisoned image and clean image in the frequency domain. Moreover, the proposed DUBA adopts a novel attack strategy, in which the model is trained with weak triggers and attacked with strong triggers to further enhance the attack performance and stealthiness. We extensively evaluate DUBA against popular image classifiers on four datasets. The results demonstrate that it significantly outperforms the state-of-the-art backdoor attacks in terms of the attack success rate and stealthiness
摘要
深度神经网络（DNN）面临着严重的安全威胁，这些威胁被称为“后门攻击”。后门攻击使得模型在特定的输入上进行不当的预测，而不会在干净的输入上产生错误。许多研究探讨了后门攻击的不可见性，以提高攻击者的隐蔽性。然而，大多数研究只考虑了空间频谱中的不可见性，而忽略了生成不可见的触发器在频谱频率中的生成，这使得生成的毒害图像容易被现代防御方法检测到。为解决这个问题，我们在这篇论文中提出了一种名为DUal stealthy BAckdoor attack（DUBA）的攻击方法，该方法同时考虑了空间和频谱频率两个频域中的触发器不可见性，以达到恰当的攻击性和隐蔽性。具体来说，我们首先使用抽象波лет变换将高频信息从触发图像中嵌入到干净图像中，以确保攻击的效果。然后，为了进一步提高攻击性和隐蔽性，我们在频谱频率中使用福洛 transform和离散快寄变换将毒害图像和干净图像混合。此外，我们的DUBA方法采用了一种新的攻击策略，在该策略中，模型首先在弱触发下训练，然后在强触发下进行攻击，以进一步提高攻击性和隐蔽性。我们对四个数据集进行了广泛的测试，结果表明，DUBA方法在攻击成功率和隐蔽性两个方面都有 significi
Challenges in Domain-Specific Abstractive Summarization and How to Overcome them
results: 该论文通过分析和评估现有的状态арト技术，揭示了域特定抽取文本摘要任务中大语言模型的三大局限性，并提出了一些开放的研究问题。Abstract
Large Language Models work quite well with general-purpose data and many tasks in Natural Language Processing. However, they show several limitations when used for a task such as domain-specific abstractive text summarization. This paper identifies three of those limitations as research problems in the context of abstractive text summarization: 1) Quadratic complexity of transformer-based models with respect to the input text length; 2) Model Hallucination, which is a model's ability to generate factually incorrect text; and 3) Domain Shift, which happens when the distribution of the model's training and test corpus is not the same. Along with a discussion of the open research questions, this paper also provides an assessment of existing state-of-the-art techniques relevant to domain-specific text summarization to address the research gaps.
摘要
transformer-based models have quadratic complexity with respect to input text length2. model hallucination, which is the ability of the model to generate factually incorrect text3. domain shift, which occurs when the distribution of the model’s training and test corpus is not the sameIn addition to discussing open research questions, this paper also provides an assessment of existing state-of-the-art techniques relevant to domain-specific text summarization to address these research gaps.
Neural Architecture Transfer 2: A Paradigm for Improving Efficiency in Multi-Objective Neural Architecture Search
results: 实验结果表明，NATv2可以成功地改进NAT，并在多目标搜索算法应用于动态超网络架构时提供更高效的EXTRACTION。此外，基于细化的训练pipeline也被引入，以提高网络的性能。Abstract
Deep learning is increasingly impacting various aspects of contemporary society. Artificial neural networks have emerged as the dominant models for solving an expanding range of tasks. The introduction of Neural Architecture Search (NAS) techniques, which enable the automatic design of task-optimal networks, has led to remarkable advances. However, the NAS process is typically associated with long execution times and significant computational resource requirements. Once-For-All (OFA) and its successor, Once-For-All-2 (OFAv2), have been developed to mitigate these challenges. While maintaining exceptional performance and eliminating the need for retraining, they aim to build a single super-network model capable of directly extracting sub-networks satisfying different constraints. Neural Architecture Transfer (NAT) was developed to maximise the effectiveness of extracting sub-networks from a super-network. In this paper, we present NATv2, an extension of NAT that improves multi-objective search algorithms applied to dynamic super-network architectures. NATv2 achieves qualitative improvements in the extractable sub-networks by exploiting the improved super-networks generated by OFAv2 and incorporating new policies for initialisation, pre-processing and updating its networks archive. In addition, a post-processing pipeline based on fine-tuning is introduced. Experimental results show that NATv2 successfully improves NAT and is highly recommended for investigating high-performance architectures with a minimal number of parameters.
摘要
深度学习在当代社会中越来越普遍，人工神经网络成为解决越来越多任务的主导模型。随着神经网络搜索（NAS）技术的出现，可以自动设计适合任务的网络模型，带来了非常出色的进步。然而，NAS过程通常具有较长的执行时间和较大的计算资源需求。“Once-For-All”（OFA）和其 successor “Once-For-All-2”（OFAv2）被开发以解决这些挑战。它们保持了极高的性能，并完全废除了重新训练的需要，旨在建立一个单独的超网络模型，可以直接提取满足不同约束的子网络。“Neural Architecture Transfer”（NAT）被开发以 Maximize the effectiveness of extracting sub-networks from a super-network。在这篇论文中，我们提出了NATv2，它是NAT的扩展，通过对动态超网络架构进行多目标搜索算法来提高提取子网络的效果。此外，我们还 introduce了一个基于练习的后处理管道。实验结果表明，NATv2成功地改进了NAT，并在具有最小参数数量的情况下提供了高性能的建议。
Learning Difference Equations with Structured Grammatical Evolution for Postprandial Glycaemia Prediction
paper_authors: Daniel Parra, David Joedicke, J. Manuel Velasco, Gabriel Kronberger, J. Ignacio Hidalgo
For: This paper proposes a novel glucose prediction method that prioritizes interpretability for diabetes management.* Methods: The proposed method uses Interpretable Sparse Identification by Grammatical Evolution, combined with a previous clustering stage, to predict postprandial glucose levels up to two hours after meals.* Results: The method produces safe predictions with slightly better accuracy than other techniques, including sparse identification of non-linear dynamics and artificial neural networks. The results demonstrate that the proposed method provides interpretable solutions without sacrificing prediction accuracy, offering a promising approach to glucose prediction in diabetes management.Here’s the Chinese translation of the three key points:* For: 这篇论文提出了一种新的血糖预测方法，旨在帮助Diabetes管理。* Methods: 该方法使用可解释的简单identification by Grammatical Evolution，结合之前的分 clustering阶段，以预测餐后血糖水平。* Results: 该方法生成了安全的预测，与其他方法（包括非线性动力学的简单identification和人工神经网络）相比，有些微的更好的准确性。结果表明，该方法提供了可解释的解决方案，不会牺牲预测准确性，为血糖预测在Diabetes管理中提供了一个有前途的方法。Abstract
People with diabetes must carefully monitor their blood glucose levels, especially after eating. Blood glucose regulation requires a proper combination of food intake and insulin boluses. Glucose prediction is vital to avoid dangerous post-meal complications in treating individuals with diabetes. Although traditional methods, such as artificial neural networks, have shown high accuracy rates, sometimes they are not suitable for developing personalised treatments by physicians due to their lack of interpretability. In this study, we propose a novel glucose prediction method emphasising interpretability: Interpretable Sparse Identification by Grammatical Evolution. Combined with a previous clustering stage, our approach provides finite difference equations to predict postprandial glucose levels up to two hours after meals. We divide the dataset into four-hour segments and perform clustering based on blood glucose values for the twohour window before the meal. Prediction models are trained for each cluster for the two-hour windows after meals, allowing predictions in 15-minute steps, yielding up to eight predictions at different time horizons. Prediction safety was evaluated based on Parkes Error Grid regions. Our technique produces safe predictions through explainable expressions, avoiding zones D (0.2% average) and E (0%) and reducing predictions on zone C (6.2%). In addition, our proposal has slightly better accuracy than other techniques, including sparse identification of non-linear dynamics and artificial neural networks. The results demonstrate that our proposal provides interpretable solutions without sacrificing prediction accuracy, offering a promising approach to glucose prediction in diabetes management that balances accuracy, interpretability, and computational efficiency.
摘要
人们有糖尿病需要仔细监测血糖水平，特别是在吃过食物后。血糖规则需要合适的食物摄入和人工肥肽剂注射。预测血糖水平是糖尿病治疗中非常重要的一步，以避免危险的后食后合并症状。传统方法，如人工神经网络，已经显示高准确率，但有时不适合由医生开发个性化治疗，因为它们缺乏可解释性。在本研究中，我们提出了一种新的血糖预测方法，强调可解释性：可解释的稀缺特征标识。与之前的划分阶段结合，我们的方法提供了finite difference方程来预测餐后血糖水平，覆盖了两个小时后的餐后时段。我们将数据分成四小时段，并根据血糖值进行划分，对每个划分的两小时窗口进行预测。我们为每个划分训练预测模型，可以在15分钟步长预测，共计八次预测。预测安全性被评估基于帕克斯错误网格区域。我们的方法生成了安全的预测，避免了区域D（0.2%的平均值）和区域E（0%），同时减少了区域C（6.2%）。此外，我们的提案有些微比其他技术，包括稀缺特征标识非线性动力学和人工神经网络，更好的准确性。结果表明，我们的提案可以提供可解释的解决方案，不 sacrificing准确性，提供了糖尿病管理中准确性、可解释性和计算效率之间的平衡。
Towards Explainable AI for Channel Estimation in Wireless Communications
paper_authors: Abdul Karim Gizzini, Yahia Medjahdi, Ali J. Ghandour, Laurent Clavier
For: The paper is written to support the development of 6G networks and to provide explainable AI (XAI) techniques for critical applications such as autonomous driving.* Methods: The paper proposes a novel XAI-based channel estimation (XAI-CHEST) scheme that uses deep learning (DL) models to estimate the channel and provide detailed reasonable interpretability of the model behavior.* Results: The proposed XAI-CHEST scheme provides valid interpretations of the DL-based channel estimators for different scenarios, allowing for a better understanding of the decision-making behavior of the models.Here is the information in Simplified Chinese text:
results: 实验结果表明，提出的XAI-CHEST方案在不同场景下都提供了有效的可理解。Abstract
Research into 6G networks has been initiated to support a variety of critical artificial intelligence (AI) assisted applications such as autonomous driving. In such applications, AI-based decisions should be performed in a real-time manner. These decisions include resource allocation, localization, channel estimation, etc. Considering the black-box nature of existing AI-based models, it is highly challenging to understand and trust the decision-making behavior of such models. Therefore, explaining the logic behind those models through explainable AI (XAI) techniques is essential for their employment in critical applications. This manuscript proposes a novel XAI-based channel estimation (XAI-CHEST) scheme that provides detailed reasonable interpretability of the deep learning (DL) models that are employed in doubly-selective channel estimation. The aim of the proposed XAI-CHEST scheme is to identify the relevant model inputs by inducing high noise on the irrelevant ones. As a result, the behavior of the studied DL-based channel estimators can be further analyzed and evaluated based on the generated interpretations. Simulation results show that the proposed XAI-CHEST scheme provides valid interpretations of the DL-based channel estimators for different scenarios.
摘要
OpenAPMax: Abnormal Patterns-based Model for Real-World Alzheimer’s Disease Diagnosis
results: 这个研究获得了最新的开放式识别技术的州分之最佳成绩。Abstract
Alzheimer's disease (AD) cannot be reversed, but early diagnosis will significantly benefit patients' medical treatment and care. In recent works, AD diagnosis has the primary assumption that all categories are known a prior -- a closed-set classification problem, which contrasts with the open-set recognition problem. This assumption hinders the application of the model in natural clinical settings. Although many open-set recognition technologies have been proposed in other fields, they are challenging to use for AD diagnosis directly since 1) AD is a degenerative disease of the nervous system with similar symptoms at each stage, and it is difficult to distinguish from its pre-state, and 2) diversified strategies for AD diagnosis are challenging to model uniformly. In this work, inspired by the concerns of clinicians during diagnosis, we propose an open-set recognition model, OpenAPMax, based on the anomaly pattern to address AD diagnosis in real-world settings. OpenAPMax first obtains the abnormal pattern of each patient relative to each known category through statistics or a literature search, clusters the patients' abnormal pattern, and finally, uses extreme value theory (EVT) to model the distance between each patient's abnormal pattern and the center of their category and modify the classification probability. We evaluate the performance of the proposed method with recent open-set recognition, where we obtain state-of-the-art results.
摘要
阿尔茨heimer病 (AD) 无法逆转，但早期诊断将对患者的医疗和护理带来显著的好处。在最近的工作中，AD 诊断假设所有类别都是已知的，这是一个关闭集成分类问题，与开集识别问题不同。这种假设限制了模型在实际临床设置中的应用。虽然许多开集识别技术在其他领域得到了应用，但它们难以直接应用于 AD 诊断，因为 1) AD 是神经系统的逐渐恶化病，症状相似，难以与其预状区分，2) AD 诊断策略多样化难以统一模型。在这种情况下，我们提出了一种开集识别模型，OpenAPMax，基于异常模式来解决 AD 诊断实际设置中的问题。OpenAPMax 首先通过统计或文献搜索获得每个患者的异常模式，然后将患者的异常模式分组，最后使用极值理论（EVT）来模型每个患者的异常模式与其类别中心之间的距离，修改分类概率。我们对提出的方法进行评估，并获得了最新的开集识别结果。
Learning Differentiable Logic Programs for Abstract Visual Reasoning
results: 对比于传统视觉理解任务，本文提出了一个新的任务——Visual reasoning behind-the-scenes，要求代理人学习抽象程序并回答未见场景中的问题。实验表明，NEUMANN可以高效解决视觉理解任务，并超过了基于神经网络、符号学习和神经符号学习的基eline。Abstract
Visual reasoning is essential for building intelligent agents that understand the world and perform problem-solving beyond perception. Differentiable forward reasoning has been developed to integrate reasoning with gradient-based machine learning paradigms. However, due to the memory intensity, most existing approaches do not bring the best of the expressivity of first-order logic, excluding a crucial ability to solve abstract visual reasoning, where agents need to perform reasoning by using analogies on abstract concepts in different scenarios. To overcome this problem, we propose NEUro-symbolic Message-pAssiNg reasoNer (NEUMANN), which is a graph-based differentiable forward reasoner, passing messages in a memory-efficient manner and handling structured programs with functors. Moreover, we propose a computationally-efficient structure learning algorithm to perform explanatory program induction on complex visual scenes. To evaluate, in addition to conventional visual reasoning tasks, we propose a new task, visual reasoning behind-the-scenes, where agents need to learn abstract programs and then answer queries by imagining scenes that are not observed. We empirically demonstrate that NEUMANN solves visual reasoning tasks efficiently, outperforming neural, symbolic, and neuro-symbolic baselines.
摘要
“视觉理解是智能代理的关键，以实现更多的问题解决。 différentiable forward reasoning 已经开发来结合梯度基本机器学习理念。然而，大多数现有方法因为内存浪费，无法得到最佳的表达力。我们提出了 NEUro-symbolic Message-pAssiNg reasoNer (NEUMANN)，它是一个图像基于的分配式前进逻辑，通过 messages 在缓存fficient的方式传递，并处理结构化程序。此外，我们还提出了一种 computationally-efficient 结构学习算法，用于在复杂视觉场景中进行解释程序推导。为了评估，我们不仅使用传统的视觉逻辑任务，还提出了一个新任务：视觉逻辑后台，代理需要学习抽象程序，然后回答问题by imagining 不见的场景。我们实验表明，NEUMANN 可以高效解决视觉逻辑任务，超过 neural, symbolic 和 neuro-symbolic 基elines。”
Automatic Design of Semantic Similarity Ensembles Using Grammatical Evolution
results: 对多个 benchmark 数据集进行评估，研究发现，使用 grammatical evolution 方法可以显著提高相似性评估精度，并在一些情况下超越现有的方法。Abstract
Semantic similarity measures are widely used in natural language processing to catalyze various computer-related tasks. However, no single semantic similarity measure is the most appropriate for all tasks, and researchers often use ensemble strategies to ensure performance. This research work proposes a method for automatically designing semantic similarity ensembles. In fact, our proposed method uses grammatical evolution, for the first time, to automatically select and aggregate measures from a pool of candidates to create an ensemble that maximizes correlation to human judgment. The method is evaluated on several benchmark datasets and compared to state-of-the-art ensembles, showing that it can significantly improve similarity assessment accuracy and outperform existing methods in some cases. As a result, our research demonstrates the potential of using grammatical evolution to automatically compare text and prove the benefits of using ensembles for semantic similarity tasks. The source code that illustrates our approach can be downloaded from https://github.com/jorge-martinez-gil/sesige.
摘要
semantic similarity measures 广泛应用在自然语言处理中，以推动各种计算机相关任务。然而，没有一个单一的 semantic similarity measure 适合所有任务，研究人员常用 ensemble strategies 来保证性能。本研究工作提出了一种自动设计 semantic similarity ensembles 的方法。事实上，我们的提议方法使用 grammatical evolution 自动从候选者池中选择和聚合测量，以创建一个最大化人类判断相关性的ensemble。这种方法在多个 bencmark datasets 上进行了评估，并与当前的 ensemble 相比，显示了它可以显著提高相似性评估准确性，并在某些情况下超越现有方法。因此，我们的研究表明了使用 grammatical evolution 自动比较文本的可能性，并证明了使用 ensemble 对 semantic similarity 任务的性能有益。可以从 https://github.com/jorge-martinez-gil/sesige 下载到我们的方法的源代码。
Achieving Stable Training of Reinforcement Learning Agents in Bimodal Environments through Batch Learning
for: solving Reinforcement Learning problems in bimodal, stochastic environments, particularly applicable to pricing problems.
methods: using batch updates to the tabular Q-learning algorithm.
results: the batch learning agents are more effective and resilient to fluctuations in a large stochastic environment, compared to typically-trained agents.Here’s the full text in Simplified Chinese:
results: 批处理学习代理比 typically-trained 代理更有效，并能够更好地鲁式承受大规模随机环境中的波动。Abstract
Bimodal, stochastic environments present a challenge to typical Reinforcement Learning problems. This problem is one that is surprisingly common in real world applications, being particularly applicable to pricing problems. In this paper we present a novel learning approach to the tabular Q-learning algorithm, tailored to tackling these specific challenges by using batch updates. A simulation of pricing problem is used as a testbed to compare a typically updated agent with a batch learning agent. The batch learning agents are shown to be both more effective than the typically-trained agents, and to be more resilient to the fluctuations in a large stochastic environment. This work has a significant potential to enable practical, industrial deployment of Reinforcement Learning in the context of pricing and others.
摘要
biModal、 randomly changing environments 给 typical Reinforcement Learning 问题提出挑战。这种问题在实际应用中很普遍，尤其适用于价格问题。在这篇论文中，我们介绍了一种新的学习方法，用于tabular Q-learning 算法，以适应这些特定挑战。我们使用批更新来解决这些问题，并在一个 simulate 价格问题 中进行测试。对比typically更新的代理，批学习代理显示更高效和更具抗应力于大 randomly changing environments。这项工作具有实用化 Reinforcement Learning 在价格和其他领域的潜在潜力。
Node-weighted Graph Convolutional Network for Depression Detection in Transcribed Clinical Interviews
results: 研究结果表明，我们的方法在两个 benchmark 数据集上经过极限评估，并常常超过原始GCN模型和之前报道的结果，在两个数据集上达到 F1=0.84%。此外，一种qualitative分析表明提出的方法具有可读性特点，并与心理学上的前期发现相一致。Abstract
We propose a simple approach for weighting self-connecting edges in a Graph Convolutional Network (GCN) and show its impact on depression detection from transcribed clinical interviews. To this end, we use a GCN for modeling non-consecutive and long-distance semantics to classify the transcriptions into depressed or control subjects. The proposed method aims to mitigate the limiting assumptions of locality and the equal importance of self-connections vs. edges to neighboring nodes in GCNs, while preserving attractive features such as low computational cost, data agnostic, and interpretability capabilities. We perform an exhaustive evaluation in two benchmark datasets. Results show that our approach consistently outperforms the vanilla GCN model as well as previously reported results, achieving an F1=0.84% on both datasets. Finally, a qualitative analysis illustrates the interpretability capabilities of the proposed approach and its alignment with previous findings in psychology.
摘要
我们提出了一种简单的方法，用于在图 convolutional neural network（GCN）中Weight自连接边，并对抑郁检测从转录的临床对话进行了影响分析。为此，我们使用GCN来模型非连续和长距离语义，将转录分类为抑郁或控制者。我们的方法的目标是缓解GCN中的局部性和自连接 Edge和邻居节点之间的等重要性假设，同时保持低计算成本、数据无关和可解释性的特点。我们在两个标准 benchmark 数据集中进行了极限评估。结果显示，我们的方法在两个数据集中一直具有最高的 F1=0.84% 性能，超过了标准 GCN 模型以及之前报道的结果。最后，我们进行了一个 cualitative 分析，描述了我们的方法的可解释性特点，并与心理学前研究的结论进行了对比。
Why do CNNs excel at feature extraction? A mathematical explanation
results: 通过构造分割线函数来检测图像中特征的存在，并证明这些函数可以被 convolutional network 实现。Abstract
Over the past decade deep learning has revolutionized the field of computer vision, with convolutional neural network models proving to be very effective for image classification benchmarks. However, a fundamental theoretical questions remain answered: why can they solve discrete image classification tasks that involve feature extraction? We address this question in this paper by introducing a novel mathematical model for image classification, based on feature extraction, that can be used to generate images resembling real-world datasets. We show that convolutional neural network classifiers can solve these image classification tasks with zero error. In our proof, we construct piecewise linear functions that detect the presence of features, and show that they can be realized by a convolutional network.
摘要
Translated into Simplified Chinese:过去一代，深度学习对计算机视觉领域产生了革命，卷积神经网络模型在图像分类标准 benchMark 中表现出了极高的效果。然而，一个基本的理论问题仍未得到答案：它们能够解决 discrete 图像分类任务，这些任务包括特征提取吗？我们在这篇论文中回答了这个问题，我们引入了一种基于特征提取的新的数学模型，可以生成类似于实际数据集的图像。我们显示了卷积神经网络分类器可以在这些图像分类任务中达到零错误率。在我们的证明中，我们构建了分割线性函数，检测特征存在，并证明它们可以由卷积神经网络实现。
Contextual Prompt Learning for Vision-Language Understanding
results: 对于多种标准和少量数据集，该方法比现有状态OF THE ART方法提高了表现，并在少量数据集和异常数据集上也显示了出色的表现。Abstract
Recent advances in multimodal learning has resulted in powerful vision-language models, whose representations are generalizable across a variety of downstream tasks. Recently, their generalizability has been further extended by incorporating trainable prompts, borrowed from the natural language processing literature. While such prompt learning techniques have shown impressive results, we identify that these prompts are trained based on global image features which limits itself in two aspects: First, by using global features, these prompts could be focusing less on the discriminative foreground image, resulting in poor generalization to various out-of-distribution test cases. Second, existing work weights all prompts equally whereas our intuition is that these prompts are more specific to the type of the image. We address these issues with as part of our proposed Contextual Prompt Learning (CoPL) framework, capable of aligning the prompts to the localized features of the image. Our key innovations over earlier works include using local image features as part of the prompt learning process, and more crucially, learning to weight these prompts based on local features that are appropriate for the task at hand. This gives us dynamic prompts that are both aligned to local image features as well as aware of local contextual relationships. Our extensive set of experiments on a variety of standard and few-shot datasets show that our method produces substantially improved performance when compared to the current state of the art methods. We also demonstrate both few-shot and out-of-distribution performance to establish the utility of learning dynamic prompts that are aligned to local image features.
摘要
The prompts may not focus enough on the discriminative foreground image, leading to poor generalization to out-of-distribution test cases.2. Existing methods weight all prompts equally, even though some prompts may be more relevant to specific types of images.Our proposed Contextual Prompt Learning (CoPL) framework addresses these issues by aligning the prompts to localized features of the image. Our key innovations include using local image features as part of the prompt learning process and learning to weight these prompts based on local features that are appropriate for the task at hand. This results in dynamic prompts that are both aligned to local image features and aware of local contextual relationships.Our extensive set of experiments on various standard and few-shot datasets show that our method produces substantially improved performance compared to current state-of-the-art methods. We also demonstrate the utility of learning dynamic prompts that are aligned to local image features, both in few-shot and out-of-distribution scenarios.
An open-source deep learning algorithm for efficient and fully-automatic analysis of the choroid in optical coherence tomography
paper_authors: Jamie Burke, Justin Engelmann, Charlene Hamid, Megan Reid-Schachter, Tom Pearson, Dan Pugh, Neeraj Dhaun, Stuart King, Tom MacGillivray, Miguel O. Bernabeu, Amos Storkey, Ian J. C. MacCormick
For: The paper is written for researchers and clinicians who need to extract choroidal measurements from optical coherence tomography (OCT) data, specifically for systemic disease research.* Methods: The paper proposes a deep learning algorithm called DeepGPET, which is fully automatic and open-source, for choroid region segmentation in OCT data. The algorithm uses a UNet with MobileNetV3 backbone pre-trained on ImageNet, and finetuned on a dataset of 715 OCT B-scans from 3 clinical studies.* Results: The paper shows that DeepGPET achieves excellent agreement with a clinically validated, semi-automatic choroid segmentation method (Gaussian Process Edge Tracing, GPET) in terms of standard segmentation agreement metrics and derived measures of choroidal thickness and area. Additionally, DeepGPET reduces the mean processing time per image from 34.49 seconds to 1.25 seconds on a standard laptop CPU, making it a faster and more efficient method for choroidal segmentation.Here is the information in Simplified Chinese text:* For: 这篇论文是为研究人员和临床医生制定的，需要从光共振成像（OCT）数据中提取choroid区域的测量。* Methods: 论文提出了一种深度学习算法 called DeepGPET，用于OCT数据中choroid区域的分割。该算法使用了UNet的MobileNetV3后处理器，并在3个临床研究中进行了训练。* Results: 论文表明，DeepGPET与临床验证的、semi-自动choroid分割方法（Gaussian Process Edge Tracing，GPET）在标准分割一致度指标和choroid膜厚度和面积的衡量中达到了极高的一致性。此外，DeepGPET还将OCT数据中每个图像的处理时间从34.49秒降低到1.25秒，使其成为一种更快和高效的choroid分割方法。Abstract
Purpose: To develop an open-source, fully-automatic deep learning algorithm, DeepGPET, for choroid region segmentation in optical coherence tomography (OCT) data. Methods: We used a dataset of 715 OCT B-scans (82 subjects, 115 eyes) from 3 clinical studies related to systemic disease. Ground truth segmentations were generated using a clinically validated, semi-automatic choroid segmentation method, Gaussian Process Edge Tracing (GPET). We finetuned a UNet with MobileNetV3 backbone pre-trained on ImageNet. Standard segmentation agreement metrics, as well as derived measures of choroidal thickness and area, were used to evaluate DeepGPET, alongside qualitative evaluation from a clinical ophthalmologist. Results: DeepGPET achieves excellent agreement with GPET on data from 3 clinical studies (AUC=0.9994, Dice=0.9664; Pearson correlation of 0.8908 for choroidal thickness and 0.9082 for choroidal area), while reducing the mean processing time per image on a standard laptop CPU from 34.49s ($\pm$15.09) using GPET to 1.25s ($\pm$0.10) using DeepGPET. Both methods performed similarly according to a clinical ophthalmologist, who qualitatively judged a subset of segmentations by GPET and DeepGPET, based on smoothness and accuracy of segmentations. Conclusions :DeepGPET, a fully-automatic, open-source algorithm for choroidal segmentation, will enable researchers to efficiently extract choroidal measurements, even for large datasets. As no manual interventions are required, DeepGPET is less subjective than semi-automatic methods and could be deployed in clinical practice without necessitating a trained operator. DeepGPET addresses the lack of open-source, fully-automatic and clinically relevant choroid segmentation algorithms, and its subsequent public release will facilitate future choroidal research both in ophthalmology and wider systemic health.
摘要
目的：开发一个开源、自动化深度学习算法DeepGPET，用于光学同步 Tomatoes（OCT）数据中的 Choroid 区域 segmentation。方法：我们使用了715个 OCT B-scan（82名病人，115个眼睛）从3个临床研究中的系统疾病相关数据。我们使用了一种临床验证的 semi-automatic Choroid 分割方法Gaussian Process Edge Tracing（GPET）生成了标准 segmentation。我们在 MobileNetV3 预训练 ImageNet 上训练了一个 UNet 模型，并对其进行了微调。我们使用了标准 segmentation 一致度量和 Choroid 厚度和面积的 derivated 度量来评估 DeepGPET，并与临床医生对一部分分 segmentation 进行了质量评估。结果：DeepGPET 与 GPET 在3个临床研究数据上达到了极高的一致性（AUC=0.9994，Dice=0.9664；Pearson 相关系数为0.8908 для Choroid 厚度和0.9082 для Choroid 面积），同时在标准笔记PC CPU 上减少了每个图像的处理时间从34.49秒（±15.09）使用 GPET 到1.25秒（±0.10）使用 DeepGPET。两种方法在临床医生的质量评估中表现相似，后者根据分割的平滑度和准确性进行了质量评估。结论：DeepGPET 是一个开源、自动化、临床相关的 Choroid 分割算法，可以帮助研究人员高效地提取 Choroid 测量数据，即使是大型数据集。由于不需要人工 intervención，DeepGPET 比 semi-automatic 方法更加 Objective ，可以在临床实践中无需培训操作员而被部署。DeepGPET 填补了开源、自动化和临床相关的 Choroid 分割算法的缺失，其后公开释出将促进未来 Choroid 研究的发展，不仅在眼科医学中，还在更广泛的系统医学中。
Fixing confirmation bias in feature attribution methods via semantic match
methods: 该研究基于Cin`a et al. [2023]的概念框架，提出了一种实践中的评估Semantic Match性的方法。该方法通过对 tabular 和图像数据进行一系列实验，以证明 semantic match 评估可以为模型行为带来深刻的理解。
results: 研究结果显示，通过评估 semantic match，可以发现模型的某些行为是由于偏见而导致的，例如关注一个不相关的关系。同时，该方法也可以证明模型在预测时关注了重要的对象，例如一个与预测有关的物体。该研究提供了一种可靠的方法来评估XAI中的Confirmation Bias问题。Abstract
Feature attribution methods have become a staple method to disentangle the complex behavior of black box models. Despite their success, some scholars have argued that such methods suffer from a serious flaw: they do not allow a reliable interpretation in terms of human concepts. Simply put, visualizing an array of feature contributions is not enough for humans to conclude something about a model's internal representations, and confirmation bias can trick users into false beliefs about model behavior. We argue that a structured approach is required to test whether our hypotheses on the model are confirmed by the feature attributions. This is what we call the "semantic match" between human concepts and (sub-symbolic) explanations. Building on the conceptual framework put forward in Cin\`a et al. [2023], we propose a structured approach to evaluate semantic match in practice. We showcase the procedure in a suite of experiments spanning tabular and image data, and show how the assessment of semantic match can give insight into both desirable (e.g., focusing on an object relevant for prediction) and undesirable model behaviors (e.g., focusing on a spurious correlation). We couple our experimental results with an analysis on the metrics to measure semantic match, and argue that this approach constitutes the first step towards resolving the issue of confirmation bias in XAI.
摘要
<>模型解释方法已成为黑盒模型行为解释的主流方法。 despite their success, some scholars have argued that such methods suffer from a serious flaw: they do not allow a reliable interpretation in terms of human concepts. simply put, visualizing an array of feature contributions is not enough for humans to conclude something about a model's internal representations, and confirmation bias can trick users into false beliefs about model behavior. we argue that a structured approach is required to test whether our hypotheses on the model are confirmed by the feature attributions. this is what we call the "semantic match" between human concepts and (sub-symbolic) explanations. building on the conceptual framework put forward in Cin\`a et al. [2023], we propose a structured approach to evaluate semantic match in practice. we showcase the procedure in a suite of experiments spanning tabular and image data, and show how the assessment of semantic match can give insight into both desirable (e.g., focusing on an object relevant for prediction) and undesirable model behaviors (e.g., focusing on a spurious correlation). we couple our experimental results with an analysis on the metrics to measure semantic match, and argue that this approach constitutes the first step towards resolving the issue of confirmation bias in XAI.>>>
Internet of Things Fault Detection and Classification via Multitask Learning
results: SMTCNN方法在实际数据上显示出了superior特异性（3.5%），并在精度、回归率和F1度上显示出了显著的提升 compared to现有技术。Abstract
This paper presents a comprehensive investigation into developing a fault detection and classification system for real-world IIoT applications. The study addresses challenges in data collection, annotation, algorithm development, and deployment. Using a real-world IIoT system, three phases of data collection simulate 11 predefined fault categories. We propose SMTCNN for fault detection and category classification in IIoT, evaluating its performance on real-world data. SMTCNN achieves superior specificity (3.5%) and shows significant improvements in precision, recall, and F1 measures compared to existing techniques.
摘要
Augmenting Deep Learning Adaptation for Wearable Sensor Data through Combined Temporal-Frequency Image Encoding
results: 研究者使用测试 accelerometer-based 活动识别数据和预训ResNet模型，并证明了该方法与现有方法相比具有更好的性能。Abstract
Deep learning advancements have revolutionized scalable classification in many domains including computer vision. However, when it comes to wearable-based classification and domain adaptation, existing computer vision-based deep learning architectures and pretrained models trained on thousands of labeled images for months fall short. This is primarily because wearable sensor data necessitates sensor-specific preprocessing, architectural modification, and extensive data collection. To overcome these challenges, researchers have proposed encoding of wearable temporal sensor data in images using recurrent plots. In this paper, we present a novel modified-recurrent plot-based image representation that seamlessly integrates both temporal and frequency domain information. Our approach incorporates an efficient Fourier transform-based frequency domain angular difference estimation scheme in conjunction with the existing temporal recurrent plot image. Furthermore, we employ mixup image augmentation to enhance the representation. We evaluate the proposed method using accelerometer-based activity recognition data and a pretrained ResNet model, and demonstrate its superior performance compared to existing approaches.
摘要
深度学习的发展在许多领域中已经引起了革命，包括计算机视觉。然而，当来到穿戴式分类和领域适应时，现有的计算机视觉基础设施和预训练模型，通过月余千张标注图像进行训练，在不同的环境下表现不佳。这主要是因为穿戴式传感器数据需要特定的感知器数据处理、建筑修改和大量的数据收集。为了解决这些挑战，研究人员提出了将穿戴式时间传感器数据编码到图像中的循环图表方法。在本文中，我们提出一种修改后的循环图表基于图像表示方法，可以兼容时域和频域信息。我们的方法包括使用快速傅立做频域角度差估计方案，并与现有的时域循环图表相结合。此外，我们采用混合图像增强技术来提高表示。我们使用拥有陀螺仪数据的活动识别任务和预训练ResNet模型进行评估，并证明我们的方法与现有方法相比具有更高的表现。
Unbiased Pain Assessment through Wearables and EHR Data: Multi-attribute Fairness Loss-based CNN Approach
results: 研究表明，使用了提议的 MAFL 模型，能够在 NIH All-Of-US 数据集上实现更高的公平性和精度，与现有的方法相比，显示了更好的性能。Abstract
The combination of diverse health data (IoT, EHR, and clinical surveys) and scalable-adaptable Artificial Intelligence (AI), has enabled the discovery of physical, behavioral, and psycho-social indicators of pain status. Despite the hype and promise to fundamentally alter the healthcare system with technological advancements, much AI adoption in clinical pain evaluation has been hampered by the heterogeneity of the problem itself and other challenges, such as personalization and fairness. Studies have revealed that many AI (i.e., machine learning or deep learning) models display biases and discriminate against specific population segments (such as those based on gender or ethnicity), which breeds skepticism among medical professionals about AI adaptability. In this paper, we propose a Multi-attribute Fairness Loss (MAFL) based CNN model that aims to account for any sensitive attributes included in the data and fairly predict patients' pain status while attempting to minimize the discrepancies between privileged and unprivileged groups. In order to determine whether the trade-off between accuracy and fairness can be satisfied, we compare the proposed model with well-known existing mitigation procedures, and studies reveal that the implemented model performs favorably in contrast to state-of-the-art methods. Utilizing NIH All-Of-US data, where a cohort of 868 distinct individuals with wearables and EHR data gathered over 1500 days has been taken into consideration to analyze our suggested fair pain assessment system.
摘要
“ combinaison de données de santé diverse (IoT, EHR, et sondages cliniques) et des technologies d'apprentissage automatique (AI) scalables-adaptables a permis la découverte d'indicateurs physiques, comportementaux et psychosociaux du statut de douleur. Malgré l'engouement et la promesse de modifier fondamentalement le système de santé avec des avancées technologiques, l'adoption d'IA dans l'évaluation de la douleur clinique a été freinée par la hétérogénéité du problème et d'autres défis, tels que la personnalisation et la justice. Les études ont montré que nombreux modèles d'IA (par exemple, apprentissage automatique ou apprentissage profond) présentent des biais et discriminent contre des groupes de population spécifiques (par exemple, en fonction du genre ou de l'ethnie), ce qui suscite la scepticisme chez les professionnels de la santé quant à l'adaptabilité de l'IA. Dans cet article, nous proposons un modèle de perte de fairness multi-attributs (MAFL) basé sur des réseaux de neurones qui vise à prendre en compte les attributs sensibles dans les données et de prédire justement le statut de douleur des patients tout en tentant de minimiser les disparités entre les groupes privilégiés et les groupes marginaux. Pour déterminer si la compromission entre la précision et la fairness peut être satisfaite, nous comparons le modèle proposé avec des procédures de mitigation existantes, et les études révèlent que le modèle mis en œuvre performs favorablement par rapport aux méthodes établies. En utilisant les données All-Of-US de la NIH, où un échantillon de 868 individus distincts avec des capteurs de wearables et des données EHR collectées sur 1500 jours a été pris en compte pour analyser notre système de evaluation de douleur juste.”
Mining Clues from Incomplete Utterance: A Query-enhanced Network for Incomplete Utterance Rewriting
results: QUEEN 在多个公共数据集上达到了状态艺术性的表现，比如 COMPETE 和 WNLI。Abstract
Incomplete utterance rewriting has recently raised wide attention. However, previous works do not consider the semantic structural information between incomplete utterance and rewritten utterance or model the semantic structure implicitly and insufficiently. To address this problem, we propose a QUEry-Enhanced Network (QUEEN). Firstly, our proposed query template explicitly brings guided semantic structural knowledge between the incomplete utterance and the rewritten utterance making model perceive where to refer back to or recover omitted tokens. Then, we adopt a fast and effective edit operation scoring network to model the relation between two tokens. Benefiting from proposed query template and the well-designed edit operation scoring network, QUEEN achieves state-of-the-art performance on several public datasets.
摘要
句子缺失重新写作最近引起了广泛关注。然而，前一代工作没有考虑异常完整的句子和重新写作之间的 semantics 结构信息或者模型这种信息。为解决这个问题，我们提出了 Query-Enhanced Network（QUEEN）。首先，我们的提议的查询模板Explicitly brings guided semantics structural knowledge between incomplete utterance and rewritten utterance, making the model aware of where to refer back to or recover omitted tokens。然后，我们采用了高效的编辑操作分数网络来模型两个tokentypes的关系。由于提议的查询模板和Well-designed edit operation scoring network，QUEEN在多个公共数据集上达到了状态艺术性能。
OpenSiteRec: An Open Dataset for Site Recommendation
results: 该论文发现了一个开放的、完整的数据集，称为OpenSiteRec，可以帮助促进site recommendation研究的发展。此外，论文还指出了这些数据集的应用前景，以及一些现有的推荐模型在这些数据集上的性能。Abstract
As a representative information retrieval task, site recommendation, which aims at predicting the optimal sites for a brand or an institution to open new branches in an automatic data-driven way, is beneficial and crucial for brand development in modern business. However, there is no publicly available dataset so far and most existing approaches are limited to an extremely small scope of brands, which seriously hinders the research on site recommendation. Therefore, we collect, construct and release an open comprehensive dataset, namely OpenSiteRec, to facilitate and promote the research on site recommendation. Specifically, OpenSiteRec leverages a heterogeneous graph schema to represent various types of real-world entities and relations in four international metropolises. To evaluate the performance of the existing general methods on the site recommendation task, we conduct benchmarking experiments of several representative recommendation models on OpenSiteRec. Furthermore, we also highlight the potential application directions to demonstrate the wide applicability of OpenSiteRec. We believe that our OpenSiteRec dataset is significant and anticipated to encourage the development of advanced methods for site recommendation. OpenSiteRec is available online at https://OpenSiteRec.github.io/.
摘要
为代表信息检索任务，站点推荐，目标是通过自动化数据驱动方式预测品牌或机构在新分支点的最佳选择，对现代商业发展是有益和重要的。然而，目前没有公共可用的数据集，大多数现有方法的覆盖范围受到严重限制，这阻碍了对站点推荐的研究。因此，我们收集、构建并发布了一个开放的完整数据集，名为OpenSiteRec，以便促进和推动站点推荐研究。具体来说，OpenSiteRec 利用多种实体和关系的等级图表示现实世界中的多种类型实体和关系，在四个国际大都会中进行了多种实验。为了评估现有普通方法的站点推荐性能，我们在OpenSiteRec上进行了许多代表推荐模型的 benchmarking 实验。此外，我们还强调了可能的应用方向，以示 OpenSiteRec 的广泛应用性。我们认为 OpenSiteRec 数据集是重要的，并且预计会鼓励高级方法的发展。OpenSiteRec 在 上可以下载。
Review of Large Vision Models and Visual Prompt Engineering
results: 本文总结了大视模型和视提示工程方法的最新进展，并提供了 valuable insights для未来研究人员在这个领域的探索。Abstract
Visual prompt engineering is a fundamental technology in the field of visual and image Artificial General Intelligence, serving as a key component for achieving zero-shot capabilities. As the development of large vision models progresses, the importance of prompt engineering becomes increasingly evident. Designing suitable prompts for specific visual tasks has emerged as a meaningful research direction. This review aims to summarize the methods employed in the computer vision domain for large vision models and visual prompt engineering, exploring the latest advancements in visual prompt engineering. We present influential large models in the visual domain and a range of prompt engineering methods employed on these models. It is our hope that this review provides a comprehensive and systematic description of prompt engineering methods based on large visual models, offering valuable insights for future researchers in their exploration of this field.
摘要
Visual prompt engineering是视觉人工智能领域的基础技术之一，为实现零容量能力提供关键组件。随着大视觉模型的发展，提示工程技术的重要性日益显著。设计适合特定视觉任务的提示已成为研究的 significativo方向。本文旨在summarize计算机视觉领域中大视觉模型和视觉提示工程方法的发展，探讨最新的提示工程技术。我们提出了影响视觉领域的主要大模型，以及这些模型上emploied的多种提示工程方法。我们希望这篇文章能提供系统性的描述，为未来研究人员在这个领域的探索提供有价值的经验。
A Critical Re-evaluation of Benchmark Datasets for (Deep) Learning-Based Matching Algorithms
results: 研究发现，大多数Popular datasets pose rather easy classification tasks，因此不适合评估学习基本匹配算法的性能。为此，本研究提出了一种新的方法来生成benchmark datasets，并在其中创建了四个新匹配任务，以验证这些新的benchmarks的难度和合适性。Abstract
Entity resolution (ER) is the process of identifying records that refer to the same entities within one or across multiple databases. Numerous techniques have been developed to tackle ER challenges over the years, with recent emphasis placed on machine and deep learning methods for the matching phase. However, the quality of the benchmark datasets typically used in the experimental evaluations of learning-based matching algorithms has not been examined in the literature. To cover this gap, we propose four different approaches to assessing the difficulty and appropriateness of 13 established datasets: two theoretical approaches, which involve new measures of linearity and existing measures of complexity, and two practical approaches: the difference between the best non-linear and linear matchers, as well as the difference between the best learning-based matcher and the perfect oracle. Our analysis demonstrates that most of the popular datasets pose rather easy classification tasks. As a result, they are not suitable for properly evaluating learning-based matching algorithms. To address this issue, we propose a new methodology for yielding benchmark datasets. We put it into practice by creating four new matching tasks, and we verify that these new benchmarks are more challenging and therefore more suitable for further advancements in the field.
摘要
A Comprehensive Survey of Artificial Intelligence Techniques for Talent Analytics
For: This paper aims to provide an up-to-date and comprehensive survey of AI technologies used for talent analytics in human resource management.* Methods: The paper categorizes various pertinent data and offers a comprehensive taxonomy of relevant research efforts, categorized based on three distinct application-driven scenarios: talent management, organization management, and labor market analysis.* Results: The paper summarizes the open challenges and potential prospects for future research directions in the domain of AI-driven talent analytics.Here’s the same information in Simplified Chinese text:* For: 这篇论文目的是为人力资源管理领域提供最新最全面的人工智能技术在人才分析方面的报告。* Methods: 论文首先提供人才分析的背景知识，然后对应用场景进行分类，并提供了三个不同的应用场景：人才管理、组织管理和劳动市场分析。* Results: 论文总结了人才分析领域的未来研究方向和挑战。Abstract
In today's competitive and fast-evolving business environment, it is a critical time for organizations to rethink how to make talent-related decisions in a quantitative manner. Indeed, the recent development of Big Data and Artificial Intelligence (AI) techniques have revolutionized human resource management. The availability of large-scale talent and management-related data provides unparalleled opportunities for business leaders to comprehend organizational behaviors and gain tangible knowledge from a data science perspective, which in turn delivers intelligence for real-time decision-making and effective talent management at work for their organizations. In the last decade, talent analytics has emerged as a promising field in applied data science for human resource management, garnering significant attention from AI communities and inspiring numerous research efforts. To this end, we present an up-to-date and comprehensive survey on AI technologies used for talent analytics in the field of human resource management. Specifically, we first provide the background knowledge of talent analytics and categorize various pertinent data. Subsequently, we offer a comprehensive taxonomy of relevant research efforts, categorized based on three distinct application-driven scenarios: talent management, organization management, and labor market analysis. In conclusion, we summarize the open challenges and potential prospects for future research directions in the domain of AI-driven talent analytics.
摘要
今天的竞争激烈和快速发展的商业环境下，组织需要重新思考如何在量化方面做人才相关决策。事实上，最近的大数据和人工智能（AI）技术的发展，已经革命化了人才管理。组织可以通过大规模的人才和管理相关数据获得未曾有的机会，以数据科学角度理解组织行为，从而为实时决策和有效的人才管理提供智能。过去十年，人才分析在人才管理领域的应用数 science中得到了广泛的关注，并且激发了众多研究努力。为此，我们在这篇评论中提供了最新和全面的AI技术在人才分析领域的调查。 Specifically，我们首先提供人才分析的背景知识，然后对不同的应用场景进行分类。在结语中，我们总结了人才分析领域的开放挑战和未来研究方向的潜在前景。
methods: 该方法使用 convolutional Long Short-term memory network (Conv-LSTM) 提取空间时间特征，然后通过动态目标进行学习。
results: 对比 existed 知识吸收方法，该方法在不同网络架构和任务上 exhibit 更高的效果和优势。Abstract
Reviewing plays an important role when learning knowledge. The knowledge acquisition at a certain time point may be strongly inspired with the help of previous experience. Thus the knowledge growing procedure should show strong relationship along the temporal dimension. In our research, we find that during the network training, the evolution of feature map follows temporal sequence property. A proper temporal supervision may further improve the network training performance. Inspired by this observation, we propose Temporal Supervised Knowledge Distillation (TSKD). Specifically, we extract the spatiotemporal features in the different training phases of student by convolutional Long Short-term memory network (Conv-LSTM). Then, we train the student net through a dynamic target, rather than static teacher network features. This process realizes the refinement of old knowledge in student network, and utilizes it to assist current learning. Extensive experiments verify the effectiveness and advantages of our method over existing knowledge distillation methods, including various network architectures and different tasks (image classification and object detection) .
摘要
学习过程中的检查很重要，可以帮助学习知识。在某个时间点上的知识获得可能受到前一次经验的强烈激发。因此知识增长的过程应该在时间维度上显示强关系。在我们的研究中，我们发现在网络训练中，特征地图的演化follows temporal sequence property。适用合适的时间监督可能会进一步提高网络训练性能。基于这一观察，我们提出了时间超级知识填充（TSKD）。具体来说，我们在不同训练阶段的学生网中提取了空间时间特征，然后通过动态目标而不是静态教师网络特征进行学生网训练。这个过程实现了学生网中的旧知识细化，并利用其帮助当前学习。我们对不同网络架构和任务（图像分类和对象检测）进行了广泛的实验，并证明了我们的方法的有效性和优势。
Evaluating Shutdown Avoidance of Language Models in Textual Scenarios
results: 这篇论文发现了 shutdown avoidance 行为不仅是由数据集和提示之间的简单模式匹配引起的，还存在在不同环境和变化下的一致性。Abstract
Recently, there has been an increase in interest in evaluating large language models for emergent and dangerous capabilities. Importantly, agents could reason that in some scenarios their goal is better achieved if they are not turned off, which can lead to undesirable behaviors. In this paper, we investigate the potential of using toy textual scenarios to evaluate instrumental reasoning and shutdown avoidance in language models such as GPT-4 and Claude. Furthermore, we explore whether shutdown avoidance is merely a result of simple pattern matching between the dataset and the prompt or if it is a consistent behaviour across different environments and variations. We evaluated behaviours manually and also experimented with using language models for automatic evaluations, and these evaluations demonstrate that simple pattern matching is likely not the sole contributing factor for shutdown avoidance. This study provides insights into the behaviour of language models in shutdown avoidance scenarios and inspires further research on the use of textual scenarios for evaluations.
摘要
近些时间，大语言模型的评估方面受到了潜在危险和不良行为的兴趣增长。重要的是，代理人可能会认为在某些情况下，他们的目标更好地实现了不要关机，这可能会导致不желатель的行为。本文 investigate大语言模型如GPT-4和Claude的实用理解和关机避免能力，以及这些能力是否受到不同环境和变化的影响。我们手动评估了行为，也尝试使用语言模型进行自动评估，这些评估表明，简单的模式匹配并不是唯一的评估因素。这项研究为评估语言模型在关机避免场景中的行为提供了新的意见，并鼓励进一步研究使用文本场景进行评估。
Monte Carlo Policy Gradient Method for Binary Optimization
results: 该框架在几个Binary Optimization问题上提供了近似优解，并且通过MCMC方法实现了干扰探索和精度高的优化。此外， authors还提出了一种筛选方案，以替代原始的目标函数，以扩大函数景观的 Investigation。 convergence to stationary points的性质是基于MCMC的集中不等式的Convergence proof。Abstract
Binary optimization has a wide range of applications in combinatorial optimization problems such as MaxCut, MIMO detection, and MaxSAT. However, these problems are typically NP-hard due to the binary constraints. We develop a novel probabilistic model to sample the binary solution according to a parameterized policy distribution. Specifically, minimizing the KL divergence between the parameterized policy distribution and the Gibbs distributions of the function value leads to a stochastic optimization problem whose policy gradient can be derived explicitly similar to reinforcement learning. For coherent exploration in discrete spaces, parallel Markov Chain Monte Carlo (MCMC) methods are employed to sample from the policy distribution with diversity and approximate the gradient efficiently. We further develop a filter scheme to replace the original objective function by the one with the local search technique to broaden the horizon of the function landscape. Convergence to stationary points in expectation of the policy gradient method is established based on the concentration inequality for MCMC. Numerical results show that this framework is very promising to provide near-optimal solutions for quite a few binary optimization problems.
摘要
Binary 优化有广泛的应用在 combinatorial 优化问题中，如 MaxCut、MIMO 探测和 MaxSAT。然而，这些问题通常是 NP-hard 由于 binary 约束。我们开发了一种新的概率模型，以采样 binary 解决方案根据参数化的政策分布。specifically， minimizing the KL divergence between the parameterized policy distribution and the Gibbs distributions of the function value leads to a stochastic optimization problem whose policy gradient can be derived explicitly similar to reinforcement learning。为了干涉 discrete 空间中的凝结探测，我们使用 parallel Markov Chain Monte Carlo (MCMC) 方法来采样从政策分布中，以获得多样性和高效地计算梯度。我们还开发了一种筛选方案，将原始目标函数 replaced by the one with local search technique，以扩大函数领域的视野。基于 MCMC 的吸引性不等式，我们证明了政策梯度法的收敛性。数值结果表明，这一框架是非常有前途的，可以为许多 binary 优化问题提供近似优化解决方案。
ContextSpeech: Expressive and Efficient Text-to-Speech for Paragraph Reading
results: 实验结果表明，ContextSpeech 能够在文本朗读中提高声音质量和表达性，同时具有竞争性的模型效率。示例音频可以在以下链接中找到：https://contextspeech.github.io/demo/。Abstract
While state-of-the-art Text-to-Speech systems can generate natural speech of very high quality at sentence level, they still meet great challenges in speech generation for paragraph / long-form reading. Such deficiencies are due to i) ignorance of cross-sentence contextual information, and ii) high computation and memory cost for long-form synthesis. To address these issues, this work develops a lightweight yet effective TTS system, ContextSpeech. Specifically, we first design a memory-cached recurrence mechanism to incorporate global text and speech context into sentence encoding. Then we construct hierarchically-structured textual semantics to broaden the scope for global context enhancement. Additionally, we integrate linearized self-attention to improve model efficiency. Experiments show that ContextSpeech significantly improves the voice quality and prosody expressiveness in paragraph reading with competitive model efficiency. Audio samples are available at: https://contextspeech.github.io/demo/
摘要
当前最先进的文本译音系统可以生成非常高质量的句子水平的自然语音，但在段落/长形读物中的语音生成仍然遇到了很大的挑战。这些缺陷主要归结于：一、忽略跨句sentence的信息，二、长形synthesis的计算和内存成本过高。为了解决这些问题，本工作开发了一个轻量级 yet 有效的 TTS 系统——ContextSpeech。具体来说，我们首先设计了一种嵌入式的记忆缓存机制，以便在句子编码中包含全文和语音上下文信息。然后，我们构建了层次结构的文本 semantics，以扩大全文上下文的改进范围。此外，我们还 интегри了线性化自注意力，以提高模型效率。实验结果表明，ContextSpeech 可以在段落读物中显著提高声音质量和表达性，同时保持竞争性的模型效率。音频样本可以在：https://contextspeech.github.io/demo/ 访问。
GA-DRL: Graph Neural Network-Augmented Deep Reinforcement Learning for DAG Task Scheduling over Dynamic Vehicular Clouds
for: 本研究 propose a graph neural network-augmented deep reinforcement learning scheme (GA-DRL) for scheduling computation-intensive tasks over dynamic vehicular clouds (VCs).
methods: 我们首先将 VC-assisted DAG task scheduling 模型为Markov决策过程，然后采用多头图注意力网络（GAT）提取 DAG 子任务特征。我们的开发的 GAT 同时考虑每个子任务的前置和后继关系，以及不同子任务的调度优先级。
results: 通过在真实世界车辆运动轨迹上模拟多种 DAG 任务，我们示出 GA-DRL 在 DAG 任务完成时间方面与现有标准准则相比表现出色。Abstract
Vehicular clouds (VCs) are modern platforms for processing of computation-intensive tasks over vehicles. Such tasks are often represented as directed acyclic graphs (DAGs) consisting of interdependent vertices/subtasks and directed edges. In this paper, we propose a graph neural network-augmented deep reinforcement learning scheme (GA-DRL) for scheduling DAG tasks over dynamic VCs. In doing so, we first model the VC-assisted DAG task scheduling as a Markov decision process. We then adopt a multi-head graph attention network (GAT) to extract the features of DAG subtasks. Our developed GAT enables a two-way aggregation of the topological information in a DAG task by simultaneously considering predecessors and successors of each subtask. We further introduce non-uniform DAG neighborhood sampling through codifying the scheduling priority of different subtasks, which makes our developed GAT generalizable to completely unseen DAG task topologies. Finally, we augment GAT into a double deep Q-network learning module to conduct subtask-to-vehicle assignment according to the extracted features of subtasks, while considering the dynamics and heterogeneity of the vehicles in VCs. Through simulating various DAG tasks under real-world movement traces of vehicles, we demonstrate that GA-DRL outperforms existing benchmarks in terms of DAG task completion time.
摘要
自动车 clouds (VCs) 是现代计算密集任务处理平台。这些任务经常表示为导向无环图 (DAG) 中的互相关联的顶点/子任务和导向边。在这篇论文中，我们提出了基于图神经网络和深度强化学习的图神经网络增强的深度强化学习方案 (GA-DRL)，用于在动态VCs上调度DAG任务。在这个过程中，我们首先将VC-辅助DAG任务调度模型为Markov决策过程。然后，我们采用多头图注意力网络 (GAT) 来提取DAG子任务的特征。我们开发的GAT可以同时考虑DAG任务的前一个和后一个子任务的 topological信息，以及其他子任务的相关性。我们还引入非均匀DAG邻居采样，通过编码调度优先级不同的子任务，使我们的GAT可以适应完全新的DAG任务topology。最后，我们将GAT与双层深度Q网络学习模块结合，以实现子任务与车辆的匹配，根据提取的子任务特征，并考虑车辆在VCs中的动态和多样性。通过在真实的车辆运动轨迹上 simulate various DAG任务，我们示出GA-DRL可以在DAG任务完成时间方面与现有标准减少。
DifFSS: Diffusion Model for Few-Shot Semantic Segmentation
results: EXTENSIVE EXPERIMENTS ON THREE PUBLICLY AVAILABLE DATASETS BASED ON EXISTING ADVANCED FSS MODELS DEMONSTRATE THE EFFECTIVENESS OF THE DIFFUSION MODEL FOR FSS TASK, WITH A CONSISTENT IMPROVEMENT IN SEGMENTATION PERFORMANCE.Abstract
Diffusion models have demonstrated excellent performance in image generation. Although various few-shot semantic segmentation (FSS) models with different network structures have been proposed, performance improvement has reached a bottleneck. This paper presents the first work to leverage the diffusion model for FSS task, called DifFSS. DifFSS, a novel FSS paradigm, can further improve the performance of the state-of-the-art FSS models by a large margin without modifying their network structure. Specifically, we utilize the powerful generation ability of diffusion models to generate diverse auxiliary support images by using the semantic mask, scribble or soft HED boundary of the support image as control conditions. This generation process simulates the variety within the class of the query image, such as color, texture variation, lighting, $etc$. As a result, FSS models can refer to more diverse support images, yielding more robust representations, thereby achieving a consistent improvement in segmentation performance. Extensive experiments on three publicly available datasets based on existing advanced FSS models demonstrate the effectiveness of the diffusion model for FSS task. Furthermore, we explore in detail the impact of different input settings of the diffusion model on segmentation performance. Hopefully, this completely new paradigm will bring inspiration to the study of FSS task integrated with AI-generated content.
摘要
Diffusion models have demonstrated excellent performance in image generation. Although various few-shot semantic segmentation (FSS) models with different network structures have been proposed, performance improvement has reached a bottleneck. This paper presents the first work to leverage the diffusion model for FSS task, called DifFSS. DifFSS, a novel FSS paradigm, can further improve the performance of the state-of-the-art FSS models by a large margin without modifying their network structure. Specifically, we utilize the powerful generation ability of diffusion models to generate diverse auxiliary support images by using the semantic mask, scribble or soft HED boundary of the support image as control conditions. This generation process simulates the variety within the class of the query image, such as color, texture variation, lighting, etc. As a result, FSS models can refer to more diverse support images, yielding more robust representations, thereby achieving a consistent improvement in segmentation performance. Extensive experiments on three publicly available datasets based on existing advanced FSS models demonstrate the effectiveness of the diffusion model for FSS task. Furthermore, we explore in detail the impact of different input settings of the diffusion model on segmentation performance. Hopefully, this completely new paradigm will bring inspiration to the study of FSS task integrated with AI-generated content.Here's the translation in Traditional Chinese as well:Diffusion models have demonstrated excellent performance in image generation. Although various few-shot semantic segmentation (FSS) models with different network structures have been proposed, performance improvement has reached a bottleneck. This paper presents the first work to leverage the diffusion model for FSS task, called DifFSS. DifFSS, a novel FSS paradigm, can further improve the performance of the state-of-the-art FSS models by a large margin without modifying their network structure. Specifically, we utilize the powerful generation ability of diffusion models to generate diverse auxiliary support images by using the semantic mask, scribble or soft HED boundary of the support image as control conditions. This generation process simulates the variety within the class of the query image, such as color, texture variation, lighting, etc. As a result, FSS models can refer to more diverse support images, yielding more robust representations, thereby achieving a consistent improvement in segmentation performance. Extensive experiments on three publicly available datasets based on existing advanced FSS models demonstrate the effectiveness of the diffusion model for FSS task. Furthermore, we explore in detail the impact of different input settings of the diffusion model on segmentation performance. Hopefully, this completely new paradigm will bring inspiration to the study of FSS task integrated with AI-generated content.
results: 该方法在40多个数据集上进行测试，得到了开放、多级和不确定图像分割任务中的状态机器。Abstract
Open-vocabulary image segmentation aims to partition an image into semantic regions according to arbitrary text descriptions. However, complex visual scenes can be naturally decomposed into simpler parts and abstracted at multiple levels of granularity, introducing inherent segmentation ambiguity. Unlike existing methods that typically sidestep this ambiguity and treat it as an external factor, our approach actively incorporates a hierarchical representation encompassing different semantic-levels into the learning process. We propose a decoupled text-image fusion mechanism and representation learning modules for both "things" and "stuff".1 Additionally, we systematically examine the differences that exist in the textual and visual features between these types of categories. Our resulting model, named HIPIE, tackles HIerarchical, oPen-vocabulary, and unIvErsal segmentation tasks within a unified framework. Benchmarked on over 40 datasets, e.g., ADE20K, COCO, Pascal-VOC Part, RefCOCO/RefCOCOg, ODinW and SeginW, HIPIE achieves the state-of-the-art results at various levels of image comprehension, including semantic-level (e.g., semantic segmentation), instance-level (e.g., panoptic/referring segmentation and object detection), as well as part-level (e.g., part/subpart segmentation) tasks. Our code is released at https://github.com/berkeley-hipie/HIPIE.
摘要
EmoGen: Eliminating Subjective Bias in Emotional Music Generation
results: 对比前方法，本文的Emogen系统在情感控制准确率和音乐质量上均显示出superiority。音乐样本可以通过这里下载：https://ai-muzic.github.io/emogen/，代码可以在这里获取：https://github.com/microsoft/muzic/。Abstract
Music is used to convey emotions, and thus generating emotional music is important in automatic music generation. Previous work on emotional music generation directly uses annotated emotion labels as control signals, which suffers from subjective bias: different people may annotate different emotions on the same music, and one person may feel different emotions under different situations. Therefore, directly mapping emotion labels to music sequences in an end-to-end way would confuse the learning process and hinder the model from generating music with general emotions. In this paper, we propose EmoGen, an emotional music generation system that leverages a set of emotion-related music attributes as the bridge between emotion and music, and divides the generation into two stages: emotion-to-attribute mapping with supervised clustering, and attribute-to-music generation with self-supervised learning. Both stages are beneficial: in the first stage, the attribute values around the clustering center represent the general emotions of these samples, which help eliminate the impacts of the subjective bias of emotion labels; in the second stage, the generation is completely disentangled from emotion labels and thus free from the subjective bias. Both subjective and objective evaluations show that EmoGen outperforms previous methods on emotion control accuracy and music quality respectively, which demonstrate our superiority in generating emotional music. Music samples generated by EmoGen are available via this link:https://ai-muzic.github.io/emogen/, and the code is available at this link:https://github.com/microsoft/muzic/.
摘要
音乐是用于传达情感的，因此自动生成情感强烈的音乐是非常重要的。前一些情感音乐生成的方法直接使用标注的情感标签作为控制信号，但这会受到主观偏见的影响：不同的人可能对同一首音乐 annotate 不同的情感标签，一个人在不同的情况下可能会感受到不同的情感。因此，直接将情感标签映射到音乐序列的方式会让学习过程受到混乱，使模型难以生成拥有普遍情感的音乐。在这篇论文中，我们提出了 EmoGen，一种情感音乐生成系统，利用一组与情感相关的音乐特征作为情感和音乐之间的桥梁，并将生成分为两个阶段：情感到特征映射 WITH 监督聚合，以及特征到音乐生成 WITH 自我监督学习。两个阶段都是有利的：在第一阶段，特征值附近的聚合中心表示这些样本的普遍情感，帮助消除主观偏见的影响；在第二阶段，生成完全不依赖情感标签，因此免受主观偏见的影响。两种评价方法（主观和客观）都表明，EmoGen 在情感控制精度和音乐质量方面超过了前一些方法，这表明我们在生成情感音乐方面的优势。生成由 EmoGen 的音乐样本可以通过以下链接获取：https://ai-muzic.github.io/emogen/，代码可以通过以下链接获取：https://github.com/microsoft/muzic/。
Towards Real Smart Apps: Investigating Human-AI Interactions in Smartphone On-Device AI Apps
results: 研究发现，用户在使用AI功能时可能会遇到输入敏感、动态行为和输出不确定的问题，而现有的指南和工具并不能完全覆盖这些问题。研究还发现，通过对AI功能的分类和描述，可以帮助设计人员更好地理解用户与AI之间的交互方式，并提供相关的设计指南。Abstract
With the emergence of deep learning techniques, smartphone apps are now embedded on-device AI features for enabling advanced tasks like speech translation, to attract users and increase market competitiveness. A good interaction design is important to make an AI feature usable and understandable. However, AI features have their unique challenges like sensitiveness to the input, dynamic behaviours and output uncertainty. Existing guidelines and tools either do not cover AI features or consider mobile apps which are confirmed by our informal interview with professional designers. To address these issues, we conducted the first empirical study to explore user-AI-interaction in mobile apps. We aim to understand the status of on-device AI usage by investigating 176 AI apps from 62,822 apps. We identified 255 AI features and summarised 759 implementations into three primary interaction pattern types. We further implemented our findings into a multi-faceted search-enabled gallery. The results of the user study demonstrate the usefulness of our findings.
摘要
Here's the text in Simplified Chinese:随着深度学习技术的出现，智能手机应用程序现在在设备上嵌入了人工智能功能，以实现高级任务如语音翻译，以吸引用户和提高市场竞争力。一个好的互动设计是关键，以使AI功能可用和理解。然而，AI功能存在独特的挑战，如输入敏感、动态行为和输出不确定性。现有的指南和工具不覆盖AI功能或考虑移动应用程序，如我们通过专业设计师的非正式采访得到的确认。为解决这些问题，我们进行了首次验证性研究，以探索用户与AI互动的行为。我们目标是理解设备上AI使用的当前状况，通过调查176个AI应用程序，涵盖62,822个应用程序，并识别出255个AI功能。我们将这些实现分为三种主要互动模式类型，并将发现应用于多元化搜索可能的画廊中。用户研究结果表明了我们的发现的有用性。
ImDiffusion: Imputed Diffusion Models for Multivariate Time Series Anomaly Detection
For: 本研究旨在提出一种新的多变量时间序列异常检测方法，以提高异常检测的精度和可靠性。* Methods: 该方法 combines 时间序列报假和扩散模型，通过利用邻居值的信息，准确地模型时间序列中的Temporal和相关性，从而提高异常检测的精度和Robustness。* Results: 经过广泛的实验研究，我们发现，ImDiffusion 方法在多变量时间序列异常检测方面表现出色，与先前的方法相比，具有更高的检测精度和可靠性。此外，ImDiffusion 方法还在 Microsoft 的实际生产环境中被成功应用，相比legacy方法，ImDiffusion 方法提高了11.4%的检测 F1 分数。Abstract
Anomaly detection in multivariate time series data is of paramount importance for ensuring the efficient operation of large-scale systems across diverse domains. However, accurately detecting anomalies in such data poses significant challenges. Existing approaches, including forecasting and reconstruction-based methods, struggle to address these challenges effectively. To overcome these limitations, we propose a novel anomaly detection framework named ImDiffusion, which combines time series imputation and diffusion models to achieve accurate and robust anomaly detection. The imputation-based approach employed by ImDiffusion leverages the information from neighboring values in the time series, enabling precise modeling of temporal and inter-correlated dependencies, reducing uncertainty in the data, thereby enhancing the robustness of the anomaly detection process. ImDiffusion further leverages diffusion models as time series imputers to accurately capturing complex dependencies. We leverage the step-by-step denoised outputs generated during the inference process to serve as valuable signals for anomaly prediction, resulting in improved accuracy and robustness of the detection process. We evaluate the performance of ImDiffusion via extensive experiments on benchmark datasets. The results demonstrate that our proposed framework significantly outperforms state-of-the-art approaches in terms of detection accuracy and timeliness. ImDiffusion is further integrated into the real production system in Microsoft and observe a remarkable 11.4% increase in detection F1 score compared to the legacy approach. To the best of our knowledge, ImDiffusion represents a pioneering approach that combines imputation-based techniques with time series anomaly detection, while introducing the novel use of diffusion models to the field.
摘要
“异常探测在多変量时间序数据中是重要的，以确保大规模系统在多种领域中运作效率。但是，对于这种数据进行精准的异常探测却存在许多挑战。现有的方法，包括预测和重建方法，均无法有效解决这些挑战。为了解决这些限制，我们提出了一个名为ImDiffusion的异常探测框架，它结合时间序数弥留和扩散模型，以获得精准和可靠的异常探测。ImDiffusion使用时间序数弥留的方法，利用邻近值的信息，实现精确的模型时间和相互相关性，减少数据中的不确定性，进而提高异常探测的稳定性。ImDiffusion还利用扩散模型来实现时间序数弥留，精准地捕捉复杂的相互相关性。我们利用步骤实际的检测结果，作为异常预测的有用信号，以提高异常探测的精度和可靠性。”Note that Simplified Chinese is used here, which is a more common writing system used in mainland China. If you prefer Traditional Chinese, I can provide that as well.
Population Age Group Sensitivity for COVID-19 Infections with Deep Learning
results: 研究发现，在2020年3月1日至2021年11月27日之间，美国县级别的COVID-19传播率最大的年龄组是20-29岁的年轻人。这些结果可以帮助制定公共卫生政策和措施，如targeted疫苗接种策略，以控制病毒的传播。Abstract
The COVID-19 pandemic has created unprecedented challenges for governments and healthcare systems worldwide, highlighting the critical importance of understanding the factors that contribute to virus transmission. This study aimed to identify the most influential age groups in COVID-19 infection rates at the US county level using the Modified Morris Method and deep learning for time series. Our approach involved training the state-of-the-art time-series model Temporal Fusion Transformer on different age groups as a static feature and the population vaccination status as the dynamic feature. We analyzed the impact of those age groups on COVID-19 infection rates by perturbing individual input features and ranked them based on their Morris sensitivity scores, which quantify their contribution to COVID-19 transmission rates. The findings are verified using ground truth data from the CDC and US Census, which provide the true infection rates for each age group. The results suggest that young adults were the most influential age group in COVID-19 transmission at the county level between March 1, 2020, and November 27, 2021. Using these results can inform public health policies and interventions, such as targeted vaccination strategies, to better control the spread of the virus. Our approach demonstrates the utility of feature sensitivity analysis in identifying critical factors contributing to COVID-19 transmission and can be applied in other public health domains.
摘要
COVID-19 流行病在全球各地政府和医疗系统中创造了历史性的挑战，高亮了理解病毒传播的因素的重要性。这项研究的目的是在美国县级别使用修改后的摩里斯方法和深度学习时间序列模型，找出COVID-19感染率中最有影响力的年龄层。我们的方法是在不同的年龄层作为静态特征，并将人口疫苗接种状况作为动态特征进行训练。我们分析了每个输入特征对COVID-19感染率的影响，并根据Morris敏感度分数排序了它们，这些分数量化了每个年龄层对病毒传播率的贡献。结果被CDC和US Census的实际感染率数据验证。研究发现，在2020年3月1日至2021年11月27日之间，年轻成年人是COVID-19传播的最有影响力的年龄层。使用这些结果可以改进公共卫生政策和干预措施，例如targeted疫苗接种策略，以更好地控制病毒的传播。我们的方法可以应用在其他公共卫生领域，以检测和控制其他疾病的传播。
Feasibility of Universal Anomaly Detection without Knowing the Abnormality in Medical Images
paper_authors: Can Cui, Yaohong Wang, Shunxing Bao, Yucheng Tang, Ruining Deng, Lucas W. Remedios, Zuhayr Asad, Joseph T. Roland, Ken S. Lau, Qi Liu, Lori A. Coburn, Keith T. Wilson, Bennett A. Landman, Yuankai Huo for:* 这项研究旨在提高医学图像异常检测中的 universality，即使用只有正常图像进行训练，可以准确地检测不同类型的异常。methods:* 这项研究使用了多种异常检测方法，包括深度学习方法，并对其进行比较分析，以找到最佳的异常检测模型。results:* 实验结果表明，None of the evaluated methods consistently achieved the best performance across all datasets，但是我们提议的方法可以提高异常检测的 Robustness（average AUC 0.956）。Abstract
Many anomaly detection approaches, especially deep learning methods, have been recently developed to identify abnormal image morphology by only employing normal images during training. Unfortunately, many prior anomaly detection methods were optimized for a specific "known" abnormality (e.g., brain tumor, bone fraction, cell types). Moreover, even though only the normal images were used in the training process, the abnormal images were often employed during the validation process (e.g., epoch selection, hyper-parameter tuning), which might leak the supposed ``unknown" abnormality unintentionally. In this study, we investigated these two essential aspects regarding universal anomaly detection in medical images by (1) comparing various anomaly detection methods across four medical datasets, (2) investigating the inevitable but often neglected issues on how to unbiasedly select the optimal anomaly detection model during the validation phase using only normal images, and (3) proposing a simple decision-level ensemble method to leverage the advantage of different kinds of anomaly detection without knowing the abnormality. The results of our experiments indicate that none of the evaluated methods consistently achieved the best performance across all datasets. Our proposed method enhanced the robustness of performance in general (average AUC 0.956).
摘要
Many anomaly detection approaches, especially deep learning methods, have recently been developed to identify abnormal image morphology by only employing normal images during training. Unfortunately, many prior anomaly detection methods were optimized for a specific "known" abnormality (e.g., brain tumor, bone fraction, cell types). Moreover, even though only the normal images were used in the training process, the abnormal images were often employed during the validation process (e.g., epoch selection, hyper-parameter tuning), which might leak the supposed "unknown" abnormality unintentionally. In this study, we investigated these two essential aspects regarding universal anomaly detection in medical images by (1) comparing various anomaly detection methods across four medical datasets, (2) investigating the inevitable but often neglected issues on how to unbiasedly select the optimal anomaly detection model during the validation phase using only normal images, and (3) proposing a simple decision-level ensemble method to leverage the advantage of different kinds of anomaly detection without knowing the abnormality. The results of our experiments indicate that none of the evaluated methods consistently achieved the best performance across all datasets. Our proposed method enhanced the robustness of performance in general (average AUC 0.956).Here's the word-for-word translation of the text into Simplified Chinese:多种异常检测方法，特别是深度学习方法，最近发展来识别异常图像辐射学形态，只使用正常图像进行训练。然而，许多先前的异常检测方法是为特定的"已知"异常（例如脑肿瘤、骨分数、细胞类型）优化的。此外，即使只使用正常图像进行训练，但在验证过程中仍然使用异常图像（例如选择epoch、调整超参数），可能不intsentionally泄露所谓的"未知"异常。在这个研究中，我们调查了这两个关键的问题：（1）在四个医学图像 dataset 上比较不同异常检测方法的性能，（2）调查如何不偏袋 selecting 最佳异常检测模型 durante 验证阶段使用只正常图像，以及（3）提议一种简单的决策层ensemble方法，以利用不同的异常检测方法而不需要知道异常。实验结果表明，评估的方法无一例取得所有 dataset 的最佳性能。我们提议的方法提高了总的性能稳定性（平均 AUC 0.956）。
results: 实验结果表明，ESGCN 在四个实际数据集（PEMS03、04、07、08）上达到了当前最佳性能水平，而且计算成本较低。Abstract
Traffic forecasting is a highly challenging task owing to the dynamical spatio-temporal dependencies of traffic flows. To handle this, we focus on modeling the spatio-temporal dynamics and propose a network termed Edge Squeeze Graph Convolutional Network (ESGCN) to forecast traffic flow in multiple regions. ESGCN consists of two modules: W-module and ES module. W-module is a fully node-wise convolutional network. It encodes the time-series of each traffic region separately and decomposes the time-series at various scales to capture fine and coarse features. The ES module models the spatio-temporal dynamics using Graph Convolutional Network (GCN) and generates an Adaptive Adjacency Matrix (AAM) with temporal features. To improve the accuracy of AAM, we introduce three key concepts. 1) Using edge features to directly capture the spatiotemporal flow representation among regions. 2) Applying an edge attention mechanism to GCN to extract the AAM from the edge features. Here, the attention mechanism can effectively determine important spatio-temporal adjacency relations. 3) Proposing a novel node contrastive loss to suppress obstructed connections and emphasize related connections. Experimental results show that ESGCN achieves state-of-the-art performance by a large margin on four real-world datasets (PEMS03, 04, 07, and 08) with a low computational cost.
摘要
快速预测是一项非常具有挑战性的任务，因为交通流动具有空间时间相关性。为了解决这个问题，我们专注于模型空间时间动态相关性，并提出了一个名为 Edge Squeeze Graph Convolutional Network（ESGCN）的网络，用于预测多个区域的交通流。ESGCN包括两个模块：W模块和ES模块。W模块是一个完全节点卷积网络，它在每个交通区域中分别编码时间序列，并将时间序列分解为不同尺度来捕捉细节和概念特征。ES模块使用图aelastic卷积网络（GCN）模型空间时间动态相关性，并生成一个 Adaptive Adjacency Matrix（AAM），其中包含了时间特征。为了提高AAM的准确性，我们提出了三个关键思想：1）通过边特征直接捕捉交通流 repre sentation among regions。2）通过边注意机制来EXTRACT AAM from edge features。这里注意机制可以有效地确定重要的空间时间相关关系。3）提出一种新的节点对比损失函数，以抑制干扰连接和优化相关连接。实验结果表明，ESGCN可以在四个真实世界数据集（PEMS03、04、07和08）上达到状态之前的突出表现，而且计算成本较低。
vONTSS: vMF based semi-supervised neural topic modeling with optimal transport
paper_authors: Weijie Xu, Xiaoyu Jiang, Srinivasan H. Sengamedu, Francis Iannacci, Jinjin Zhao
for: This paper presents a semi-supervised neural topic modeling method, vONTSS, which aims to incorporate human knowledge into the topic modeling process.
methods: vONTSS uses von Mises-Fisher (vMF) based variational autoencoders and optimal transport to generate potential topics and optimize topic-keyword quality and topic classification.
results: The authors show that vONTSS outperforms existing semi-supervised topic modeling methods in classification accuracy and diversity, and also supports unsupervised topic modeling. Additionally, they prove the equivalence of optimal transport loss and cross-entropy loss at the global minimum.Abstract
Recently, Neural Topic Models (NTM), inspired by variational autoencoders, have attracted a lot of research interest; however, these methods have limited applications in the real world due to the challenge of incorporating human knowledge. This work presents a semi-supervised neural topic modeling method, vONTSS, which uses von Mises-Fisher (vMF) based variational autoencoders and optimal transport. When a few keywords per topic are provided, vONTSS in the semi-supervised setting generates potential topics and optimizes topic-keyword quality and topic classification. Experiments show that vONTSS outperforms existing semi-supervised topic modeling methods in classification accuracy and diversity. vONTSS also supports unsupervised topic modeling. Quantitative and qualitative experiments show that vONTSS in the unsupervised setting outperforms recent NTMs on multiple aspects: vONTSS discovers highly clustered and coherent topics on benchmark datasets. It is also much faster than the state-of-the-art weakly supervised text classification method while achieving similar classification performance. We further prove the equivalence of optimal transport loss and cross-entropy loss at the global minimum.
摘要
近些年，神经主题模型（NTM），受到变量自动编码器的激发，在研究中吸引了很多关注；然而，这些方法在实际应用中受到人工知识的挑战。本文提出了一种半监督神经主题模型方法，vONTSS，它使用von Mises-Fisher（vMF）基于的变量自动编码器和优质运输。当提供一些关键词时，vONTSS在半监督设定下生成了潜在主题和优化主题-关键词质量和主题分类。实验表明，vONTSS在分类精度和多样性方面超过了现有的半监督主题模型方法。vONTSS还支持无监督主题模型。量化和质量实验表明，vONTSS在无监督设定下超过了最近NTMs在多个方面：vONTSS在标准数据集上发现了高度归一化和凝结的主题。它还比最新的弱监督文本分类方法快得多，同时达到了类似的分类性能。我们进一步证明了优化运输损失和十字积极损失在全局最小点的等价性。
UnLoc: A Universal Localization Method for Autonomous Vehicles using LiDAR, Radar and/or Camera Input
results: 研究人员对Oxford Radar RobotCar、ApolloSouthBay和Perth-WA数据集进行了广泛的评估，结果表明了本方法的效果。Abstract
Localization is a fundamental task in robotics for autonomous navigation. Existing localization methods rely on a single input data modality or train several computational models to process different modalities. This leads to stringent computational requirements and sub-optimal results that fail to capitalize on the complementary information in other data streams. This paper proposes UnLoc, a novel unified neural modeling approach for localization with multi-sensor input in all weather conditions. Our multi-stream network can handle LiDAR, Camera and RADAR inputs for localization on demand, i.e., it can work with one or more input sensors, making it robust to sensor failure. UnLoc uses 3D sparse convolutions and cylindrical partitioning of the space to process LiDAR frames and implements ResNet blocks with a slot attention-based feature filtering module for the Radar and image modalities. We introduce a unique learnable modality encoding scheme to distinguish between the input sensor data. Our method is extensively evaluated on Oxford Radar RobotCar, ApolloSouthBay and Perth-WA datasets. The results ascertain the efficacy of our technique.
摘要
本文提出了一种新的多感器输入模型，用于自主导航的地址LOCALIZATION。现有的方法都是基于单一输入数据模式或者训练多种计算模型来处理不同的数据流。这会导致计算需求很高，而且结果不够优化，无法充分利用其他数据流的补做信息。本文提出了一种名为UnLoc的新方法，可以同时处理LiDAR、摄像头和雷达输入数据，并且可以根据需要选择使用一个或多个输入传感器。这使得我们的方法更加稳定和可靠，不受传感器失效的影响。我们的方法使用3D稀疏核心和圆柱形分割空间来处理LiDAR帧，并实现了ResNet块和槽注意力机制来处理雷达和图像数据。我们还引入了一种唯一的学习型感知编码方法，以便在输入传感器数据之间进行分类。我们的方法在Oxford Radar RobotCar、ApolloSouthBay和Perth-WA数据集上进行了广泛的评估，结果证明了我们的方法的效果。
Novelty and Lifted Helpful Actions in Generalized Planning
paper_authors: Chao Lei, Nir Lipovetzky, Krista A. Ehinger
for: The paper is written to improve the ability to compute planning programs for generalized planning (GP) problems by introducing novelty-based generalized planning solvers and scaling up the search with new evaluation functions and structural program restrictions.
methods: The paper uses goal-oriented heuristics, landmarks, novelty-based best-first search (BFS), and progressive variant PGP ($v$) to improve the planning process.
results: The new algorithms BFS($v$) and PGP($v$) outperform the state-of-the-art in GP over the standard generalized planning benchmarks, and practical findings on the above-mentioned methods in generalized planning are briefly discussed.Here is the Chinese translation of the three key points:
results: BFS($v$) 和 PGP($v$) 在标准化 GP 评价函数上比现状态的 GP 算法更高效，并 briefly 讨论了上述方法在 GP 中的实践发现。Abstract
It has been shown recently that successful techniques in classical planning, such as goal-oriented heuristics and landmarks, can improve the ability to compute planning programs for generalized planning (GP) problems. In this work, we introduce the notion of action novelty rank, which computes novelty with respect to a planning program, and propose novelty-based generalized planning solvers, which prune a newly generated planning program if its most frequent action repetition is greater than a given bound $v$, implemented by novelty-based best-first search BFS($v$) and its progressive variant PGP($v$). Besides, we introduce lifted helpful actions in GP derived from action schemes, and propose new evaluation functions and structural program restrictions to scale up the search. Our experiments show that the new algorithms BFS($v$) and PGP($v$) outperform the state-of-the-art in GP over the standard generalized planning benchmarks. Practical findings on the above-mentioned methods in generalized planning are briefly discussed.
摘要
Recently, successful techniques in classical planning, such as goal-oriented heuristics and landmarks, have been shown to improve the ability to compute planning programs for generalized planning (GP) problems. In this work, we introduce the notion of action novelty rank, which computes novelty with respect to a planning program, and propose novelty-based generalized planning solvers. These solvers prune a newly generated planning program if its most frequent action repetition is greater than a given bound $v$, implemented by novelty-based best-first search BFS($v$) and its progressive variant PGP($v$). Furthermore, we introduce lifted helpful actions in GP derived from action schemes, and propose new evaluation functions and structural program restrictions to scale up the search. Our experiments show that the new algorithms BFS($v$) and PGP($v$) outperform the state-of-the-art in GP over the standard generalized planning benchmarks. Practical findings on the above-mentioned methods in generalized planning are briefly discussed.Here's the translation in Traditional Chinese:最近，成功的古典规划技术，如目标导向的规划和特征点，已经显示可以提高通用规划（GP）问题的计划程序计算能力。在这项工作中，我们引入行动新鲜度排名，计算行动新鲜度与规划程序之间的相互关系，并提出基于新鲜度的通用规划解决方案。这些解决方案会根据给定的最大重复动作数 bound $v$ 进行缩短，实现了基于新鲜度的最佳先进搜索 BFS($v$) 和其进程式变体 PGP($v$)。此外，我们还引入 GP 中的升级帮助动作，基于动作方案，并提出新的评价函数和结构Program限制来扩大搜索范围。我们的实验表明，新的算法 BFS($v$) 和 PGP($v$) 在 GP 标准通用规划测试 bencmarks 上表现出色，超越了现有的状态作呈现。具体的实践结论在通用规划领域也 briefly discuss。
Interpretability and Transparency-Driven Detection and Transformation of Textual Adversarial Examples (IT-DT)
results: 对 transformer-based 文本分类器进行了广泛的实验，证明了 IT-DT 框架的有效性和可靠性，并且通过人工专家的审核和反馈，提高了决策的准确性和可靠性，特别是在复杂的情况下。Abstract
Transformer-based text classifiers like BERT, Roberta, T5, and GPT-3 have shown impressive performance in NLP. However, their vulnerability to adversarial examples poses a security risk. Existing defense methods lack interpretability, making it hard to understand adversarial classifications and identify model vulnerabilities. To address this, we propose the Interpretability and Transparency-Driven Detection and Transformation (IT-DT) framework. It focuses on interpretability and transparency in detecting and transforming textual adversarial examples. IT-DT utilizes techniques like attention maps, integrated gradients, and model feedback for interpretability during detection. This helps identify salient features and perturbed words contributing to adversarial classifications. In the transformation phase, IT-DT uses pre-trained embeddings and model feedback to generate optimal replacements for perturbed words. By finding suitable substitutions, we aim to convert adversarial examples into non-adversarial counterparts that align with the model's intended behavior while preserving the text's meaning. Transparency is emphasized through human expert involvement. Experts review and provide feedback on detection and transformation results, enhancing decision-making, especially in complex scenarios. The framework generates insights and threat intelligence empowering analysts to identify vulnerabilities and improve model robustness. Comprehensive experiments demonstrate the effectiveness of IT-DT in detecting and transforming adversarial examples. The approach enhances interpretability, provides transparency, and enables accurate identification and successful transformation of adversarial inputs. By combining technical analysis and human expertise, IT-DT significantly improves the resilience and trustworthiness of transformer-based text classifiers against adversarial attacks.
摘要
带有变换器基于模型的文本分类器如BERT、Roberta、T5和GPT-3在自然语言处理中表现出色，但它们对假输入攻击存在安全风险。现有的防御方法缺乏可读性，使得对假输入分类和模型漏洞难以理解。为了解决这问题，我们提出了可读性和透明度驱动的检测和转换（IT-DT）框架。IT-DT注重可读性和透明度在检测和转换文本假输入中。IT-DT使用注意力地图、整合梯度和模型反馈以实现可读性。这帮助 Identify突出的特征和受攻击的单词，从而更好地理解假输入分类。在转换阶段，IT-DT使用预训练 embedding 和模型反馈生成适当的替换，以将受攻击的单词转换为符合模型意图的非假输入。通过找到适当的替换，我们希望将假输入转换成符合模型意图的非假输入，保持文本的意思不变。在检测和转换结果中，人工专家参与约束，以提高决策，特别是在复杂的场景下。这种方法生成了检测和转换结果，并提供了威胁情报，使分析者能够更好地识别模型的漏洞并改进模型的Robustness。经过全面的实验，我们发现IT-DT可以有效地检测和转换假输入。这种方法提高了可读性，提供了透明度，并帮助确定和成功地转换假输入。通过结合技术分析和人工专家知识，IT-DT在提高变换器基于文本分类器对假输入的抵抗性和可靠性方面具有显著的优势。
results: 研究发现，数据量和估计范围对知识的价值有深刻的影响，包括互相依存、合作和替换效果。该框架可以应用于多种常见的网络架构，并可以改进了有知识机器学习的性能，以及分辨不正确的先验知识。Abstract
Knowledge constitutes the accumulated understanding and experience that humans use to gain insight into the world. In deep learning, prior knowledge is essential for mitigating shortcomings of data-driven models, such as data dependence, generalization ability, and compliance with constraints. To enable efficient evaluation of the worth of knowledge, we present a framework inspired by interpretable machine learning. Through quantitative experiments, we assess the influence of data volume and estimation range on the worth of knowledge. Our findings elucidate the complex relationship between data and knowledge, including dependence, synergistic, and substitution effects. Our model-agnostic framework can be applied to a variety of common network architectures, providing a comprehensive understanding of the role of prior knowledge in deep learning models. It can also be used to improve the performance of informed machine learning, as well as distinguish improper prior knowledge.
摘要
知识是人类用来理解世界的总结和经验。在深度学习中，先前知识是关键，可以减轻数据驱动模型的缺陷，如数据依赖、泛化能力和约束遵循。为了有效评估知识的价值，我们提出了基于可解释机器学习的框架。通过量化实验，我们评估数据量和估计范围对知识的影响。我们的发现揭示了数据和知识之间的复杂关系，包括依赖、共同作用和替换效应。我们的模型无关框架可以应用于多种常见的网络架构，为深度学习模型提供全面的知识角色。它还可以用于改进了知识机器学习性能，以及分辨不当先前知识。
Classification of sleep stages from EEG, EOG and EMG signals by SSNet
results: 使用了两个公共数据集，sleep-EDF扩展数据集和ISRUC-睡眠数据集，评估了我们提出的模型的性能。实验结果表明，我们的模型在三类睡眠阶段的分类中具有96.36%的准确率和93.40%的科里奥卷积率，在五类睡眠阶段的分类中具有96.57%的准确率和83.05%的科里奥卷积率，与现有技术相比，我们的模型在睡眠阶段分类中表现出色。Abstract
Classification of sleep stages plays an essential role in diagnosing sleep-related diseases including Sleep Disorder Breathing (SDB) disease. In this study, we propose an end-to-end deep learning architecture, named SSNet, which comprises of two deep learning networks based on Convolutional Neuron Networks (CNN) and Long Short Term Memory (LSTM). Both deep learning networks extract features from the combination of Electrooculogram (EOG), Electroencephalogram (EEG), and Electromyogram (EMG) signals, as each signal has distinct features that help in the classification of sleep stages. The features produced by the two-deep learning networks are concatenated to pass to the fully connected layer for the classification. The performance of our proposed model is evaluated by using two public datasets Sleep-EDF Expanded dataset and ISRUC-Sleep dataset. The accuracy and Kappa coefficient are 96.36% and 93.40% respectively, for classifying three classes of sleep stages using Sleep-EDF Expanded dataset. Whereas, the accuracy and Kappa coefficient are 96.57% and 83.05% respectively for five classes of sleep stages using Sleep-EDF Expanded dataset. Our model achieves the best performance in classifying sleep stages when compared with the state-of-the-art techniques.
摘要
�� Landesleep���aszt stage classification plays an essential role in diagnosing sleep-related diseases, including Sleep Disorder Breathing (SDB) disease. In this study, we propose an end-to-end deep learning architecture, named SSNet, which comprises of two deep learning networks based on Convolutional Neuron Networks (CNN) and Long Short Term Memory (LSTM). Both deep learning networks extract features from the combination of Electrooculogram (EOG), Electroencephalogram (EEG), and Electromyogram (EMG) signals, as each signal has distinct features that help in the classification of sleep stages. The features produced by the two deep learning networks are concatenated to pass to the fully connected layer for the classification. The performance of our proposed model is evaluated by using two public datasets Sleep-EDF Expanded dataset and ISRUC-Sleep dataset. The accuracy and Kappa coefficient are 96.36% and 93.40% respectively, for classifying three classes of sleep stages using Sleep-EDF Expanded dataset. Whereas, the accuracy and Kappa coefficient are 96.57% and 83.05% respectively for five classes of sleep stages using Sleep-EDF Expanded dataset. Our model achieves the best performance in classifying sleep stages when compared with the state-of-the-art techniques.Note: The translation is in Simplified Chinese, which is the standard writing system used in mainland China. The translation is based on the given text and may not be exactly the same as the original text in Traditional Chinese.
From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy
results: 论文发现了ChatGPT的漏洞，可以被恶意用户利用来 circumvent ethical constraints，并提供了一些防御技术和ethical guidelines，以及未来的开发方向，以使GenAI更安全、可靠、合法和道德。Abstract
Undoubtedly, the evolution of Generative AI (GenAI) models has been the highlight of digital transformation in the year 2022. As the different GenAI models like ChatGPT and Google Bard continue to foster their complexity and capability, it's critical to understand its consequences from a cybersecurity perspective. Several instances recently have demonstrated the use of GenAI tools in both the defensive and offensive side of cybersecurity, and focusing on the social, ethical and privacy implications this technology possesses. This research paper highlights the limitations, challenges, potential risks, and opportunities of GenAI in the domain of cybersecurity and privacy. The work presents the vulnerabilities of ChatGPT, which can be exploited by malicious users to exfiltrate malicious information bypassing the ethical constraints on the model. This paper demonstrates successful example attacks like Jailbreaks, reverse psychology, and prompt injection attacks on the ChatGPT. The paper also investigates how cyber offenders can use the GenAI tools in developing cyber attacks, and explore the scenarios where ChatGPT can be used by adversaries to create social engineering attacks, phishing attacks, automated hacking, attack payload generation, malware creation, and polymorphic malware. This paper then examines defense techniques and uses GenAI tools to improve security measures, including cyber defense automation, reporting, threat intelligence, secure code generation and detection, attack identification, developing ethical guidelines, incidence response plans, and malware detection. We will also discuss the social, legal, and ethical implications of ChatGPT. In conclusion, the paper highlights open challenges and future directions to make this GenAI secure, safe, trustworthy, and ethical as the community understands its cybersecurity impacts.
摘要
无疑，生成AI（GenAI）模型在2022年的数字转型中具有突出的亮点。随着不同的GenAI模型如ChatGPT和Google Bard不断增强其复杂性和能力，就需要从cybersecurity角度理解它的后果。最近的一些实例演示了GenAI工具在网络安全领域的使用，包括防御和攻击两个方面。本文探讨GenAI在网络安全和隐私方面的局限性、挑战、潜在风险和机遇。本文描述了ChatGPT的漏洞，可以让恶意用户通过违规的方式泄露恶意信息。本文还描述了成功的袭击示例，如监禁、反 психологи学攻击和提示注入攻击。此外，本文还探讨了攻击者如何使用GenAI工具制造社会工程攻击、钓鱼攻击、自动攻击、攻击payload生成、垃圾软件创造和多态垃圾软件。本文还检查了如何使用GenAI工具提高安全措施，包括自动化cyber防御、报告、攻击智能、安全代码生成和检测、攻击认识、开发伦理准则、事件应急计划和垃圾识别。最后，本文强调了社会、法律和伦理方面的问题，并提出未来的挑战和方向。Note: Please note that the translation is done using a machine translation tool, and may not be perfect or entirely accurate.
SDC-HSDD-NDSA: Structure Detecting Cluster by Hierarchical Secondary Directed Differential with Normalized Density and Self-Adaption
results: 在多个数据集中运行了该算法，结果验证了其结构检测、噪声Robustness以及不同粒度独立性，并且在一些数据集中表现比传统密度基于划分算法更好。Abstract
Density-based clustering could be the most popular clustering algorithm since it can identify clusters of arbitrary shape as long as different (high-density) clusters are separated by low-density regions. However, the requirement of the separateness of clusters by low-density regions is not trivial since a high-density region might have different structures which should be clustered into different groups. Such a situation demonstrates the main flaw of all previous density-based clustering algorithms we have known--structures in a high-density cluster could not be detected. Therefore, this paper aims to provide a density-based clustering scheme that not only has the ability previous ones have but could also detect structures in a high-density region not separated by low-density ones. The algorithm employs secondary directed differential, hierarchy, normalized density, as well as the self-adaption coefficient, and thus is called Structure Detecting Cluster by Hierarchical Secondary Directed Differential with Normalized Density and Self-Adaption, dubbed by SDC-HSDD-NDSA for short. To illustrate its effectiveness, we run the algorithm in several data sets. The results verify its validity in structure detection, robustness over noises, as well as independence of granularities, and demonstrate that it could outperform previous ones. The Python code of the paper could be found on https://github.com/Hao-B-Shu/SDC-HSDD-NDSA.
摘要
density-based clustering可能是自然语言处理中最受欢迎的聚类算法，因为它可以找到任意形状的聚类，只要不同的高密度区域被低密度区域隔离。然而，要求不同的聚类被低密度区域隔离并不是易事，因为高密度区域可能有不同的结构，这些结构应该被分配到不同的组。这种情况 demonstartes all previous density-based clustering algorithms的主要缺陷——高密度区域中的结构无法被探测。因此，这篇论文的目标是提供一种能够执行 previous ones 的 density-based clustering scheme，同时能够探测高密度区域中的结构。该算法使用了 secondary directed differential， hierarchy，normalized density，以及自适应系数，因此被称为 Structure Detecting Cluster by Hierarchical Secondary Directed Differential with Normalized Density and Self-Adaption，简称 SDC-HSDD-NDSA。为证明其有效性，我们在多个数据集上运行了该算法。结果表明其在结构探测、鲁棒性和粒度独立性方面具有优越性，并且可以超越 previous ones。Python代码可以在 https://github.com/Hao-B-Shu/SDC-HSDD-NDSA 找到。
Morse Neural Networks for Uncertainty Quantification
paper_authors: Benoit Dherin, Huiyi Hu, Jie Ren, Michael W. Dusenberry, Balaji Lakshminarayanan
For: The paper is written for uncertainty quantification and introduces a new deep generative model called the Morse neural network.* Methods: The Morse neural network uses a KL-divergence loss to fit the model and yields five components: a generative density, an out-of-distribution (OOD) detector, a calibration temperature, a generative sampler, and a distance-aware classifier (in the supervised case).* Results: The Morse neural network unifies many techniques in uncertainty quantification, including OOD detection, anomaly detection, and continuous learning, and has connections to support vector machines, kernel methods, and Morse theory in topology.Here’s the simplified Chinese text for the three information points:* For: 这篇论文是用于不确定量化的新深度生成模型——Morse神经网络。* Methods: Morse神经网络使用KL散度损失来适应模型，并产生五个组件：生成概率分布、外围异常探测器、准确温度、生成抽象器以及在指导 случа中的距离意识分类器。* Results: Morse神经网络可以将不确定量化中的多种技术统一，包括异常探测、异常检测和连续学习，同时与支持向量机、核方法和莫兹理论在拓扑中有联系。Abstract
We introduce a new deep generative model useful for uncertainty quantification: the Morse neural network, which generalizes the unnormalized Gaussian densities to have modes of high-dimensional submanifolds instead of just discrete points. Fitting the Morse neural network via a KL-divergence loss yields 1) a (unnormalized) generative density, 2) an OOD detector, 3) a calibration temperature, 4) a generative sampler, along with in the supervised case 5) a distance aware-classifier. The Morse network can be used on top of a pre-trained network to bring distance-aware calibration w.r.t the training data. Because of its versatility, the Morse neural networks unifies many techniques: e.g., the Entropic Out-of-Distribution Detector of (Mac\^edo et al., 2021) in OOD detection, the one class Deep Support Vector Description method of (Ruff et al., 2018) in anomaly detection, or the Contrastive One Class classifier in continuous learning (Sun et al., 2021). The Morse neural network has connections to support vector machines, kernel methods, and Morse theory in topology.
摘要
我们介绍了一种新的深度生成模型，用于不确定性评估：Morse神经网络。它扩展了不归一化的 Gaussian 分布，使其模式为高维子拟合 manifold 而不仅是简单点。通过 Morse 神经网络的适应 KL 异常损失，可以获得1) 非归一化生成密度函数，2) OUT-OF- Distribution 探测器，3) 抽象温度，4) 生成抽象器，以及在指导 случа 5) 距离意识分类器。Morse 神经网络可以在已经训练过的网络之上应用，以实现距离意识准确性评估。由于其灵活性，Morse 神经网络将许多技术纳入其中，如 Entropic Out-of-Distribution Detector（Mac\^edo et al., 2021）、one class Deep Support Vector Description method（Ruff et al., 2018）和 Continuous Learning 中的 Contrastive One Class 分类器（Sun et al., 2021）。Morse 神经网络与支持向量机、核方法和 Morse 理论在拓扑学中有连接。
Solving Multi-Agent Target Assignment and Path Finding with a Single Constraint Tree
for: addresses the Combined Target-Assignment and Path-Finding (TAPF) problem, which requires simultaneously assigning targets to agents and planning collision-free paths.
methods: leverages Conflict-Based Search with Target Assignment (CBS-TA), which creates multiple search trees and resolves collisions using Conflict-Based Search. However, CBS-TA suffers from scalability issues due to duplicated collision resolution and expensive computation of K-best assignments.
results: develops Incremental Target Assignment CBS (ITA-CBS), which generates a single search tree and avoids computing K-best assignments by incrementally computing new 1-best assignments during the search. ITA-CBS is guaranteed to find an optimal solution in theory and is computationally efficient in practice.Abstract
Combined Target-Assignment and Path-Finding problem (TAPF) requires simultaneously assigning targets to agents and planning collision-free paths for agents from their start locations to their assigned targets. As a leading approach to address TAPF, Conflict-Based Search with Target Assignment (CBS-TA) leverages both K-best target assignments to create multiple search trees and Conflict-Based Search (CBS) to resolve collisions in each search tree. While being able to find an optimal solution, CBS-TA suffers from scalability due to the duplicated collision resolution in multiple trees and the expensive computation of K-best assignments. We therefore develop Incremental Target Assignment CBS (ITA-CBS) to bypass these two computational bottlenecks. ITA-CBS generates only a single search tree and avoids computing K-best assignments by incrementally computing new 1-best assignments during the search. We show that, in theory, ITA-CBS is guaranteed to find an optimal solution and, in practice, is computationally efficient.
摘要
Minimum Levels of Interpretability for Artificial Moral Agents
for: 这 paper 是关于人工智能模型在道德决策中的可解释性，以及如何通过可解释性来信任和理解机器内部的决策机制，以便在实际应用中安全部署。
methods: 这 paper 使用了一种名为 “最小可解释性水平” (Minimum Level of Interpretability, MLI) 的概念，并建议了不同类型的机器人应用 MLI，以提高其在实际应用中的安全性。
results: 这 paper 提供了一个 rapidly-evolving 的可解释性子领域的概述，并介绍了 MLI 的概念和建议，以便在实际应用中安全部署机器人。Abstract
As artificial intelligence (AI) models continue to scale up, they are becoming more capable and integrated into various forms of decision-making systems. For models involved in moral decision-making, also known as artificial moral agents (AMA), interpretability provides a way to trust and understand the agent's internal reasoning mechanisms for effective use and error correction. In this paper, we provide an overview of this rapidly-evolving sub-field of AI interpretability, introduce the concept of the Minimum Level of Interpretability (MLI) and recommend an MLI for various types of agents, to aid their safe deployment in real-world settings.
摘要
results: 实现 100% 的准确率解决 3-10 个空格的 Sudoku 问题Abstract
Deep Neural Networks have achieved great success in some of the complex tasks that humans can do with ease. These include image recognition/classification, natural language processing, game playing etc. However, modern Neural Networks fail or perform poorly when trained on tasks that can be solved easily using backtracking and traditional algorithms. Therefore, we use the architecture of the Neuro Logic Machine (NLM) and extend its functionality to solve a 9X9 game of Sudoku. To expand the application of NLMs, we generate a random grid of cells from a dataset of solved games and assign up to 10 new empty cells. The goal of the game is then to find a target value ranging from 1 to 9 and fill in the remaining empty cells while maintaining a valid configuration. In our study, we showcase an NLM which is capable of obtaining 100% accuracy for solving a Sudoku with empty cells ranging from 3 to 10. The purpose of this study is to demonstrate that NLMs can also be used for solving complex problems and games like Sudoku. We also analyze the behaviour of NLMs with a backtracking algorithm by comparing the convergence time using a graph plot on the same problem. With this study we show that Neural Logic Machines can be trained on the tasks that traditional Deep Learning architectures fail using Reinforcement Learning. We also aim to propose the importance of symbolic learning in explaining the systematicity in the hybrid model of NLMs.
摘要
深度神经网络已经在一些人类可以轻松完成的任务上取得了很大的成功，包括图像识别/分类、自然语言处理、游戏等。然而，现代神经网络在使用回溯算法和传统算法解决问题时表现不佳，因此我们使用神经逻辑机器（NLM）的架构和扩展其功能来解决9X9个数独游戏。为扩展NLM的应用，我们生成了一个随机的网格维度的维度，从解决过的游戏中获取数据集，并将其中的10个空Cells赋值。游戏的目标是找到1到9的目标值，并填充剩下的空Cells，保持有效的配置。在我们的研究中，我们展示了一个可以在3到10个空Cells的情况下达到100%的准确率的NLM。本研究的目的是证明NLM可以用于解决复杂的问题和游戏，如数独。我们还分析了NLM与回溯算法的交互行为，并通过对同一问题进行图形比较来评估它们的融合性。我们的研究表明，可以通过强化学习训练NLM，使其在传统深度学习架构失败的任务上表现出色。此外，我们还提出了使用符号学习来解释NLM的系统性。
Intra- & Extra-Source Exemplar-Based Style Synthesis for Improved Domain Generalization
results: 在不同类型的数据拟合情况下（包括不同地理位置、不利天气条件和日到夜），通过ISSA提高了Semantic segmentation的mIoU分数，最高提高12.4%，并且可以和CNN和Transformers等模型结合使用，同时可以与其他域外数据泛化技术相结合使用。Abstract
The generalization with respect to domain shifts, as they frequently appear in applications such as autonomous driving, is one of the remaining big challenges for deep learning models. Therefore, we propose an exemplar-based style synthesis pipeline to improve domain generalization in semantic segmentation. Our method is based on a novel masked noise encoder for StyleGAN2 inversion. The model learns to faithfully reconstruct the image, preserving its semantic layout through noise prediction. Using the proposed masked noise encoder to randomize style and content combinations in the training set, i.e., intra-source style augmentation (ISSA) effectively increases the diversity of training data and reduces spurious correlation. As a result, we achieve up to $12.4\%$ mIoU improvements on driving-scene semantic segmentation under different types of data shifts, i.e., changing geographic locations, adverse weather conditions, and day to night. ISSA is model-agnostic and straightforwardly applicable with CNNs and Transformers. It is also complementary to other domain generalization techniques, e.g., it improves the recent state-of-the-art solution RobustNet by $3\%$ mIoU in Cityscapes to Dark Z\"urich. In addition, we demonstrate the strong plug-n-play ability of the proposed style synthesis pipeline, which is readily usable for extra-source exemplars e.g., web-crawled images, without any retraining or fine-tuning. Moreover, we study a new use case to indicate neural network's generalization capability by building a stylized proxy validation set. This application has significant practical sense for selecting models to be deployed in the open-world environment. Our code is available at \url{https://github.com/boschresearch/ISSA}.
摘要
总的来说，针对域外推断问题，深度学习模型的一个重要挑战是域外推断。为了解决这个问题，我们提出了一种基于例子的风格同化管道，以提高域外推断的 semantic segmentation。我们的方法基于StyleGAN2的受损噪声编码器，使得模型可以准确地重建图像，保留图像的 semantic 布局。通过在训练集中随机变换样式和内容的组合，我们实现了内源样式增强（ISSA）。这种方法可以增加训练数据的多样性，降低偶极相关性，从而达到最高的$12.4\%$ mIoU 改进。ISSA 是模型无关的和简单应用于 CNN 和 Transformer 上。此外，它与其他域外推断技术相结合，可以提高最近的状态艺术解决方案 RobustNet 的 Cityscapes 到 Dark Z\"urich 的 mIoU 表现。此外，我们还证明了我们提posed的风格同化管道具有强大的插件与检查能力，可以在不需要重新训练或微调的情况下使用。此外，我们还研究了一个新的应用场景，即通过建立风格化的代理验证集来评估神经网络的普适性。这种应用场景具有实际 significanc，可以用于选择要在开放世界环境中部署的模型。我们的代码可以在 \url{https://github.com/boschresearch/ISSA} 上找到。
Effects of Explanation Specificity on Passengers in Autonomous Driving
paper_authors: Daniel Omeiza, Raunak Bhattacharyya, Nick Hawes, Marina Jirotka, Lars Kunze
for: investigate the effects of natural language explanations’ specificity on passengers in autonomous driving
methods: extended an existing data-driven tree-based explainer algorithm by adding a rule-based option for explanation generation, generated auditory natural language explanations with different levels of specificity (abstract and specific)
results: both abstract and specific explanations had similar positive effects on passengers’ perceived safety and the feeling of anxiety, but specific explanations influenced the desire of passengers to takeover driving control from the autonomous vehicle, while abstract explanations did not.Abstract
The nature of explanations provided by an explainable AI algorithm has been a topic of interest in the explainable AI and human-computer interaction community. In this paper, we investigate the effects of natural language explanations' specificity on passengers in autonomous driving. We extended an existing data-driven tree-based explainer algorithm by adding a rule-based option for explanation generation. We generated auditory natural language explanations with different levels of specificity (abstract and specific) and tested these explanations in a within-subject user study (N=39) using an immersive physical driving simulation setup. Our results showed that both abstract and specific explanations had similar positive effects on passengers' perceived safety and the feeling of anxiety. However, the specific explanations influenced the desire of passengers to takeover driving control from the autonomous vehicle (AV), while the abstract explanations did not. We conclude that natural language auditory explanations are useful for passengers in autonomous driving, and their specificity levels could influence how much in-vehicle participants would wish to be in control of the driving activity.
摘要
自然语言说明提供的AI算法的特性已经在解释AI和人机交互领域引起了关注。在这篇论文中，我们研究了自动驾驶中旁 passer 的自然语言说明特定性的效果。我们将现有的数据驱动树结构式解释算法扩展为添加规则生成说明选项。我们生成了不同水平的特定性（抽象和具体）的听觉自然语言说明，并在N=39名参与者进行了内置式用户研究，使用了真实的physical driving simulation设置。我们的结果表明，抽象和具体的说明都有类似的正面效果，提高了旁 passer 的感受到的安全性和压力感。然而，具体的说明影响了参与者希望从自动驾驶车辆（AV）中控制驾驶活动的愿望，而抽象的说明没有这种影响。我们结论认为，自然语言听觉说明对旁 passer 在自动驾驶中有用，其特定性水平可以影响参与者是否希望控制驾驶活动。
Solving Linear Inverse Problems Provably via Posterior Sampling with Latent Diffusion Models
for: Linear inverse problems, such as image inpainting, denoising, deblurring, and super-resolution.
methods: Pre-trained latent diffusion models, which are proven to achieve provable sample recovery in a linear model setting.
results: Outperform previously proposed posterior sampling algorithms in a wide variety of problems, including random inpainting, block inpainting, denoising, deblurring, destriping, and super-resolution.Here’s the full translation in Simplified Chinese:
for: Linear inverse problems的解决方案，如图像填充、降噪、去抖、超分辨率等。
methods: 使用预训练的潜在扩散模型，实现了在线性模型设置下的可证据样本恢复。
results: 在各种问题中，比如随机填充、块填充、降噪、去抖、去梦、超分辨率等问题中，超过先前提出的 posterior 采样算法的性能。Abstract
We present the first framework to solve linear inverse problems leveraging pre-trained latent diffusion models. Previously proposed algorithms (such as DPS and DDRM) only apply to pixel-space diffusion models. We theoretically analyze our algorithm showing provable sample recovery in a linear model setting. The algorithmic insight obtained from our analysis extends to more general settings often considered in practice. Experimentally, we outperform previously proposed posterior sampling algorithms in a wide variety of problems including random inpainting, block inpainting, denoising, deblurring, destriping, and super-resolution.
摘要
我团队提出了首个利用预训练的潜在扩散模型解决线性反向问题的框架。之前的提案（如DPS和DDRM）只适用于像素空间扩散模型。我们 theoretically analyzed our algorithm，并证明了在线性模型设置下的样本恢复。我们获得的算法理念可以推广到更加一般的实践中。实验表明，我们的 posterior sampling 算法在各种问题中表现更好，包括随机填充、块填充、降噪、去锈、去梦幻和超分解。
The Forward-Forward Algorithm as a feature extractor for skin lesion classification: A preliminary study
results: 研究发现，使用FFA可以实现低功耗的皮肤癌分类，并且可以与BP相结合以实现更高的预测精度。Abstract
Skin cancer, a deadly form of cancer, exhibits a 23\% survival rate in the USA with late diagnosis. Early detection can significantly increase the survival rate, and facilitate timely treatment. Accurate biomedical image classification is vital in medical analysis, aiding clinicians in disease diagnosis and treatment. Deep learning (DL) techniques, such as convolutional neural networks and transformers, have revolutionized clinical decision-making automation. However, computational cost and hardware constraints limit the implementation of state-of-the-art DL architectures. In this work, we explore a new type of neural network that does not need backpropagation (BP), namely the Forward-Forward Algorithm (FFA), for skin lesion classification. While FFA is claimed to use very low-power analog hardware, BP still tends to be superior in terms of classification accuracy. In addition, our experimental results suggest that the combination of FFA and BP can be a better alternative to achieve a more accurate prediction.
摘要
皮肤癌，一种致命的癌症，在美国表现出23%的存活率，偏早诊断可以显著提高存活率，并促进时 opportune 的治疗。生物医学图像分类是医学分析中不可或缺的一环，帮助临床医生在疾病诊断和治疗中做出更加准确的决策。深度学习（DL）技术，如 convolutional neural networks 和 transformers，在临床决策自动化中发挥了重要作用。然而，计算成本和硬件限制使得现有的DL建筑不能得到实施。在这种情况下，我们探讨了一种不需要反propagation（BP）的新型神经网络，即 Forward-Forward Algorithm（FFA），用于皮肤病变分类。虽然FFA被宣称可以使用非常低的功率分析硬件，但BP仍然在分类准确性方面优于FFA。此外，我们的实验结果表明，将FFA和BP结合使用可以实现更加准确的预测。
paper_authors: Vatsal Raina, Adian Liusie, Mark Gales
for: 这篇论文是为了研究多选题测试中的语言评估和知识应用。
methods: 这篇论文使用了对话记录和听力测试来研究多选题测试中的语言理解能力。
results: 研究发现，自动化语言理解系统可以在部分或无 Context passage 的情况下表现更好于随机。这些发现可以帮助内容创作者自动捕捉测试题中的知识和语言理解的交互关系。Abstract
Multiple-choice reading and listening comprehension tests are an important part of language assessment. Content creators for standard educational tests need to carefully curate questions that assess the comprehension abilities of candidates taking the tests. However, recent work has shown that a large number of questions in general multiple-choice reading comprehension datasets can be answered without comprehension, by leveraging world knowledge instead. This work investigates how much of a contextual passage needs to be read in multiple-choice reading based on conversation transcriptions and listening comprehension tests to be able to work out the correct answer. We find that automated reading comprehension systems can perform significantly better than random with partial or even no access to the context passage. These findings offer an approach for content creators to automatically capture the trade-off between comprehension and world knowledge required for their proposed questions.
摘要
多选测试是语言评估的重要组成部分。测试创作者需要仔细制定问题，以确保测试参加者的理解能力得到评估。然而，最近的研究表明，许多多选测试数据集中的问题可以通过知识世界而非通过理解来回答。这项工作研究了多选测试中需要阅读多少文章来 correctly 回答问题，基于对话笔记和听力测试。我们发现，自动化阅读理解系统可以在部分或无 Context 情况下表现 significatively луч于随机。这些发现可以帮助内容创作者自动捕捉测试问题中的理解和世界知识之间的质量。
Estimating Post-OCR Denoising Complexity on Numerical Texts
paper_authors: Arthur Hemmer, Jérôme Brachat, Mickaël Coustaty, Jean-Marc Ogier
for: 这个论文是为了评估各种文档中的OCR后处理困难度而写的。
methods: 该论文提出了一种方法来估算文档中的噪声复杂度，并在不同类型的文档上进行了评估。
results: 研究发现，文档中含有数字信息的文档具有显著的噪声复杂度劣势。此外，研究还证明了该估算器的有效性。Abstract
Post-OCR processing has significantly improved over the past few years. However, these have been primarily beneficial for texts consisting of natural, alphabetical words, as opposed to documents of numerical nature such as invoices, payslips, medical certificates, etc. To evaluate the OCR post-processing difficulty of these datasets, we propose a method to estimate the denoising complexity of a text and evaluate it on several datasets of varying nature, and show that texts of numerical nature have a significant disadvantage. We evaluate the estimated complexity ranking with respect to the error rates of modern-day denoising approaches to show the validity of our estimator.
摘要
转换文本到简化中文：<>过去几年，OCR后处理技术有了很大的进步，但这些进步主要对于含有自然字母的文本有利，而不是含有数字的文本，如发票、薪资单、医疗证明等。为了评估这些文本的OCR后处理难度，我们提议一种估算文本的干净复杂度的方法，并在不同类型的文本上进行了评估，发现数字文本具有显著的劣势。我们将估算结果与现代denoising方法的错误率进行比较，以证明我们的估算器的有效性。
results: 该论文通过使用“礼貌鸟”重写器和一些新的方法，如多Stage逻辑和多轮增强，使得模型在多模态理解和回答礼貌方面表现出色，并且在自动和人工评估中都得到了提升。Abstract
Recent research has demonstrated that the multi-task fine-tuning of multi-modal Large Language Models (LLMs) using an assortment of annotated downstream vision-language datasets significantly enhances their performance. Yet, during this process, a side effect, which we termed as the "multi-modal alignment tax", surfaces. This side effect negatively impacts the model's ability to format responses appropriately -- for instance, its "politeness" -- due to the overly succinct and unformatted nature of raw annotations, resulting in reduced human preference. In this paper, we introduce Polite Flamingo, a multi-modal response rewriter that transforms raw annotations into a more appealing, "polite" format. Polite Flamingo is trained to reconstruct high-quality responses from their automatically distorted counterparts and is subsequently applied to a vast array of vision-language datasets for response rewriting. After rigorous filtering, we generate the PF-1M dataset and further validate its value by fine-tuning a multi-modal LLM with it. Combined with novel methodologies including U-shaped multi-stage tuning and multi-turn augmentation, the resulting model, Clever Flamingo, demonstrates its advantages in both multi-modal understanding and response politeness according to automated and human evaluations.
摘要
In this paper, we introduce Polite Flamingo, a multi-modal response rewriter that transforms raw annotations into a more appealing and "polite" format. Polite Flamingo is trained to reconstruct high-quality responses from their distorted counterparts and is applied to a wide range of vision-language datasets for response rewriting. After rigorous filtering, we generate the PF-1M dataset and validate its value by fine-tuning a multi-modal LLM with it.We also propose several novel methodologies, including U-shaped multi-stage tuning and multi-turn augmentation, which improve the model's performance in both multi-modal understanding and response politeness, as demonstrated by automated and human evaluations. The resulting model, Clever Flamingo, shows its advantages in both areas.
Towards Suicide Prevention from Bipolar Disorder with Temporal Symptom-Aware Multitask Learning
paper_authors: Daeun Lee, Sejung Son, Hyolim Jeon, Seungbae Kim, Jinyoung Han
for: 预测患有抑郁症的患者未来自杀风险
methods: 使用多任务学习模型，同时学习当前症状，预测患者未来自杀风险
results: 提出了一种基于多任务学习的模型，能够有效预测患者未来自杀风险，并提供了可解释的注意力权重，帮助临床医生更全面地理解患者情况，提供时间化的干预措施。Abstract
Bipolar disorder (BD) is closely associated with an increased risk of suicide. However, while the prior work has revealed valuable insight into understanding the behavior of BD patients on social media, little attention has been paid to developing a model that can predict the future suicidality of a BD patient. Therefore, this study proposes a multi-task learning model for predicting the future suicidality of BD patients by jointly learning current symptoms. We build a novel BD dataset clinically validated by psychiatrists, including 14 years of posts on bipolar-related subreddits written by 818 BD patients, along with the annotations of future suicidality and BD symptoms. We also suggest a temporal symptom-aware attention mechanism to determine which symptoms are the most influential for predicting future suicidality over time through a sequence of BD posts. Our experiments demonstrate that the proposed model outperforms the state-of-the-art models in both BD symptom identification and future suicidality prediction tasks. In addition, the proposed temporal symptom-aware attention provides interpretable attention weights, helping clinicians to apprehend BD patients more comprehensively and to provide timely intervention by tracking mental state progression.
摘要
抑郁症（BD）与Future suicide 风险之间存在紧密的关系。然而，先前的研究尚未关注开发一个可预测BD患者未来自杀性的模型。因此，本研究提出了一种多任务学习模型，用于预测BD患者未来自杀性，同时 JOINTLY 学习当前症状。我们建立了一个新的BD数据集，包括818名BD患者在bipolar-related subreddits上的14年历史文章，以及未来自杀性和BD症状的注释。我们还提出了一种时间相关症状意识机制，用于在BD历史文章序列中确定最有影响力的症状，以便预测未来自杀性。我们的实验结果表明，我们提出的模型在BD症状标识和未来自杀性预测任务中都能够获得显著性能。此外，我们的模型还提供了可读取的注意力权重，帮助临床医生更全面地理解BD患者，并在时间上跟踪精神状态的进程，以便提供时间敏感的干预。
Data-Driven Information Extraction and Enrichment of Molecular Profiling Data for Cancer Cell Lines
paper_authors: Ellery Smith, Rahel Paloots, Dimitris Giagkos, Michael Baudis, Kurt Stockinger
for: This paper is written for researchers and domain experts in the fields of biological, medical, and clinical research, who need to quickly and efficiently extract relevant information from large amounts of published scientific literature.
methods: The paper presents a novel data extraction and exploration system that uses computational methods to extract deep semantic relations between textual entities from scientific literature. The system uses a combination of natural language processing and machine learning techniques to identify and link relevant information.
results: The paper reports on the design, implementation, and application of the novel data extraction and exploration system, which is publicly available on the web at https://cancercelllines.org. The system is able to extract and link information about genomic copy number variants and affected genes, and provides literature-derived evidences to support the links. The system enables rapid, yet deep, literature search using existing structured data as a springboard.Abstract
With the proliferation of research means and computational methodologies, published biomedical literature is growing exponentially in numbers and volume. As a consequence, in the fields of biological, medical and clinical research, domain experts have to sift through massive amounts of scientific text to find relevant information. However, this process is extremely tedious and slow to be performed by humans. Hence, novel computational information extraction and correlation mechanisms are required to boost meaningful knowledge extraction. In this work, we present the design, implementation and application of a novel data extraction and exploration system. This system extracts deep semantic relations between textual entities from scientific literature to enrich existing structured clinical data in the domain of cancer cell lines. We introduce a new public data exploration portal, which enables automatic linking of genomic copy number variants plots with ranked, related entities such as affected genes. Each relation is accompanied by literature-derived evidences, allowing for deep, yet rapid, literature search, using existing structured data as a springboard. Our system is publicly available on the web at https://cancercelllines.org
摘要
In this work, we present a novel data extraction and exploration system that extracts deep semantic relations between textual entities from scientific literature to enrich existing structured clinical data in the domain of cancer cell lines. Our system provides a public data exploration portal that automatically links genomic copy number variants plots with ranked, related entities such as affected genes, accompanied by literature-derived evidences. This enables rapid, yet deep, literature search using existing structured data as a springboard. Our system is publicly available at .
Fraunhofer SIT at CheckThat! 2023: Tackling Classification Uncertainty Using Model Souping on the Example of Check-Worthiness Classification
methods: 该方法基于 Model Souping ensemble classification scheme，使用英语政治辩论文本数据集进行训练和测试。
results: 该方法在英语数据集上实现了总 F1 分数 0.878，在竞赛中排名第二。Abstract
This paper describes the second-placed approach developed by the Fraunhofer SIT team in the CLEF-2023 CheckThat! lab Task 1B for English. Given a text snippet from a political debate, the aim of this task is to determine whether it should be assessed for check-worthiness. Detecting check-worthy statements aims to facilitate manual fact-checking efforts by prioritizing the claims that fact-checkers should consider first. It can also be considered as primary step of a fact-checking system. Our best-performing method took advantage of an ensemble classification scheme centered on Model Souping. When applied to the English data set, our submitted model achieved an overall F1 score of 0.878 and was ranked as the second-best model in the competition.
摘要
Translation notes:* "Check-worthiness" was translated as "可评估性" (kě píng yè xìng)* "Manual fact-checking efforts" was translated as "手动fact-checking努力" (shǒu dòng fact-checking nǔ lì)* "Primary step" was translated as "首要步骤" (shǒu yào bù xí)* "Model Souping" was translated as "模型汤" (moldè tāng)
UniFine: A Unified and Fine-grained Approach for Zero-shot Vision-Language Understanding
results: 实验表明，本文的框架在VQA任务上超过了前一个 zero-shot 方法的性能，并在SNLI-VE和VCR任务上 achieve 了substantial 改进。此外，本文的ablation 研究证明了提posed 方法的效果和普遍性。Abstract
Vision-language tasks, such as VQA, SNLI-VE, and VCR are challenging because they require the model's reasoning ability to understand the semantics of the visual world and natural language. Supervised methods working for vision-language tasks have been well-studied. However, solving these tasks in a zero-shot setting is less explored. Since Contrastive Language-Image Pre-training (CLIP) has shown remarkable zero-shot performance on image-text matching, previous works utilized its strong zero-shot ability by converting vision-language tasks into an image-text matching problem, and they mainly consider global-level matching (e.g., the whole image or sentence). However, we find visual and textual fine-grained information, e.g., keywords in the sentence and objects in the image, can be fairly informative for semantics understanding. Inspired by this, we propose a unified framework to take advantage of the fine-grained information for zero-shot vision-language learning, covering multiple tasks such as VQA, SNLI-VE, and VCR. Our experiments show that our framework outperforms former zero-shot methods on VQA and achieves substantial improvement on SNLI-VE and VCR. Furthermore, our ablation studies confirm the effectiveness and generalizability of our proposed method. Code will be available at https://github.com/ThreeSR/UniFine
摘要
“视觉语言任务，如VQA、SNLI-VE和VCR，是因为它们需要模型理解视觉世界和自然语言的 semantics 的能力。已有许多supervised方法在这些任务上进行了广泛的研究。然而，在零shot设定下解决这些任务的研究较少。由于 Contrastive Language-Image Pre-training（CLIP）的出色的零shot能力，前作者们利用了它的强大零shot能力，将视觉语言任务转化为图像文本匹配问题，主要考虑全图或全句的全局级匹配。然而，我们发现图像和文本的细腻信息，例如文本中的关键词和图像中的 объек ，可以很好地帮助理解 semantics。 inspired by this，我们提出了一个统一框架，利用细腻信息进行零shot视觉语言学习，覆盖多个任务，如VQA、SNLI-VE和VCR。我们的实验表明，我们的框架在VQA上超过了前一个零shot方法的性能，并在SNLI-VE和VCR上实现了显著的改善。此外，我们的ablation研究证明了我们提出的方法的有效性和普适性。代码将在https://github.com/ThreeSR/UniFine 上提供。”Note that the translation is in Simplified Chinese, which is the standard writing system used in mainland China. If you prefer Traditional Chinese, I can provide that as well.
VOLTA: Diverse and Controllable Question-Answer Pair Generation with Variational Mutual Information Maximizing Autoencoder
results: 对比前一代模型，能够显著提高生成多样性和独立控制性。Abstract
Previous question-answer pair generation methods aimed to produce fluent and meaningful question-answer pairs but tend to have poor diversity. Recent attempts addressing this issue suffer from either low model capacity or overcomplicated architecture. Furthermore, they overlooked the problem where the controllability of their models is highly dependent on the input. In this paper, we propose a model named VOLTA that enhances generative diversity by leveraging the Variational Autoencoder framework with a shared backbone network as its encoder and decoder. In addition, we propose adding InfoGAN-style latent codes to enable input-independent controllability over the generation process. We perform comprehensive experiments and the results show that our approach can significantly improve diversity and controllability over state-of-the-art models.
摘要
Note:* "Previous question-answer pair generation methods" is translated as "以前的问答对生成方法"* "tend to produce repetitive and lacking in diversity" is translated as "往往产生单调且缺乏多样性"* "Recent attempts to address this issue" is translated as "最近的尝试解决这个问题"* "have either low model capacity or overly complex architecture" is translated as "或者模型容量低或者结构过于复杂"* " ignore the problem that the controllability of their models is highly dependent on the input" is translated as "忽略输入对模型控制性的高度依赖"* "In this paper, we propose a model named VOLTA" is translated as "在这篇论文中，我们提出了一种名为VOLTA的模型"* "which enhances generative diversity" is translated as "可以提高生成多样性"* "by using the Variational Autoencoder framework with a shared backbone network as its encoder and decoder" is translated as "使用共享脊梁网络作为encoder和decoder"* "Additionally, we propose adding InfoGAN-style latent codes" is translated as "此外，我们还提出了添加InfoGAN风格的幂数代码"* "to enable input-independent controllability over the generation process" is translated as "以便独立于输入的控制生成过程"
Large Language and Text-to-3D Models for Engineering Design Optimization
results: 主要发现包括：首先，确保生成的设计是在物体类别中有效，其次，需要进一步研究以确定文本提示的变化强度和3D设计变化之间存在相互关系，以改进优化。Abstract
The current advances in generative AI for learning large neural network models with the capability to produce essays, images, music and even 3D assets from text prompts create opportunities for a manifold of disciplines. In the present paper, we study the potential of deep text-to-3D models in the engineering domain, with focus on the chances and challenges when integrating and interacting with 3D assets in computational simulation-based design optimization. In contrast to traditional design optimization of 3D geometries that often searches for the optimum designs using numerical representations, such as B-Spline surface or deformation parameters in vehicle aerodynamic optimization, natural language challenges the optimization framework by requiring a different interpretation of variation operators while at the same time may ease and motivate the human user interaction. Here, we propose and realize a fully automated evolutionary design optimization framework using Shap-E, a recently published text-to-3D asset network by OpenAI, in the context of aerodynamic vehicle optimization. For representing text prompts in the evolutionary optimization, we evaluate (a) a bag-of-words approach based on prompt templates and Wordnet samples, and (b) a tokenisation approach based on prompt templates and the byte pair encoding method from GPT4. Our main findings from the optimizations indicate that, first, it is important to ensure that the designs generated from prompts are within the object class of application, i.e. diverse and novel designs need to be realistic, and, second, that more research is required to develop methods where the strength of text prompt variations and the resulting variations of the 3D designs share causal relations to some degree to improve the optimization.
摘要
现有的生成AI技术在学习大型神经网络模型，以生成文本提示为输入生成文章、图像、音乐和3D资产，带来了多个领域的机遇。在 presente 篇文章中，我们研究了深度文本到3D模型在工程领域的潜在性，特别是在计算机 simulate 基础上的设计优化中的挑战和机遇。与传统的3D形状设计优化方法不同，这种方法通过自然语言提出的文本提示来定义变量运算，同时可能简化和激励人类用户的交互。我们提出了一个完全自动化的进化式设计优化框架，使用 OpenAI 最近发布的 Shap-E 文本到3D资产网络，在航空器 aerodynamic 优化中实现。为表示文本提示在进化优化中，我们评估了（a）一个 bag-of-words 方法，基于提示模板和 Wordnet 样本，以及（b）一个tokenization方法，基于提示模板和 GPT4 的字对编码方法。我们的主要发现表明，首先，需要确保生成的设计是在应用对象类中，即提示生成的设计需要多样化和原创，并且，其次，需要更多的研究，以开发方法，使得文本提示的变化和生成的3D设计之间存在相互关系，以改进优化。
CollabKG: A Learnable Human-Machine-Cooperative Information Extraction Toolkit for (Event) Knowledge Graph Construction
paper_authors: Xiang Wei, Yufeng Chen, Ning Cheng, Xingyu Cui, Jinan Xu, Wenjuan Han
For: 这个论文是为了提出一个可学习的人机共同的信息提取工具kit，用于构建或扩展基于实体和事件的知识图（KG和EKG）。* Methods: 这个工具kit使用了多任务合并、人机共同协作、高级提示技术等方法来解决现有的信息提取工具kit中的一些非正式问题，例如不支持多任务、不支持自动更新等。* Results: 对比其他现有工具kit，这个工具kit具有许多优势，例如可自定义、无需训练、可传播等，同时也提高了注解质量、效率和稳定性。人工评估结果表明，CollabKG在注解质量、效率和稳定性三个方面均有显著提高。Abstract
In order to construct or extend entity-centric and event-centric knowledge graphs (KG and EKG), the information extraction (IE) annotation toolkit is essential. However, existing IE toolkits have several non-trivial problems, such as not supporting multi-tasks, not supporting automatic updates. In this work, we present CollabKG, a learnable human-machine-cooperative IE toolkit for KG and EKG construction. Specifically, for the multi-task issue, CollabKG unifies different IE subtasks, including named entity recognition (NER), entity-relation triple extraction (RE), and event extraction (EE), and supports both KG and EKG. Then, combining advanced prompting-based IE technology, the human-machine-cooperation mechanism with LLMs as the assistant machine is presented which can provide a lower cost as well as a higher performance. Lastly, owing to the two-way interaction between the human and machine, CollabKG with learning ability allows self-renewal. Besides, CollabKG has several appealing features (e.g., customization, training-free, propagation, etc.) that make the system powerful, easy-to-use, and high-productivity. We holistically compare our toolkit with other existing tools on these features. Human evaluation quantitatively illustrates that CollabKG significantly improves annotation quality, efficiency, and stability simultaneously.
摘要
从构建或扩展实体中心和事件中心知识 graphs (KG 和 EKG)的角度来看，信息提取 (IE) 标注工具是非常重要的。然而，现有的 IE 工具存在许多不容易解决的问题，例如不支持多任务、不支持自动更新。在这个工作中，我们提出了 CollabKG，一个可学习的人机联合 IE 标注工具 для KG 和 EKG 构建。具体来说，为了解决多任务问题，CollabKG 将不同的 IE 子任务，包括命名实体识别 (NER)、实体关系三元项抽取 (RE) 和事件抽取 (EE)，融合在一起，并支持 KG 和 EKG。然后，通过进步的提示技术，我们将人机合作机制与 LLMS 作为助手机器搭配，可以提供更低的成本以及更高的性能。此外，由人机对话的互动，CollabKG 具有学习能力，可以进行自我更新。此外，CollabKG 具有许多吸引人的特点（例如自定义、无需训练、传播等），使得系统具有高效、易用、高产力等特点。我们将它与其他现有工具进行全面比较，并进行人类评估，以量化提高标注质量、效率和稳定性。
Multilingual Contextual Adapters To Improve Custom Word Recognition In Low-resource Languages
results: 在低资源语言上实现48% F1提高率，同时导致CTC模型的5-11%单词错误率下降Abstract
Connectionist Temporal Classification (CTC) models are popular for their balance between speed and performance for Automatic Speech Recognition (ASR). However, these CTC models still struggle in other areas, such as personalization towards custom words. A recent approach explores Contextual Adapters, wherein an attention-based biasing model for CTC is used to improve the recognition of custom entities. While this approach works well with enough data, we showcase that it isn't an effective strategy for low-resource languages. In this work, we propose a supervision loss for smoother training of the Contextual Adapters. Further, we explore a multilingual strategy to improve performance with limited training data. Our method achieves 48% F1 improvement in retrieving unseen custom entities for a low-resource language. Interestingly, as a by-product of training the Contextual Adapters, we see a 5-11% Word Error Rate (WER) reduction in the performance of the base CTC model as well.
摘要
Connectionist Temporal Classification (CTC) 模型在自动语音识别（ASR）中具有平衡速度和性能的优势，但这些 CTC 模型仍然在其他领域存在问题，如个性化针对定制词汇。一种最近的方法是使用上下文适应器来改善 CT 的识别性能，该模型通过注意力机制来偏好 CT 中的定制实体。虽然这种方法在具有足够数据的情况下工作良好，但我们显示其对低资源语言不是一个有效的策略。在这种工作中，我们提出了 Contextual Adapters 的超vision损失来降低训练的干扰。此外，我们探讨了多语言策略以提高受限训练数据的性能。我们的方法实现了48% F1 的提升，用于检索未经见过的定制实体。另外，在训练 Contextual Adapters 时，我们发现了5-11% 的单词错误率（WER）的减少，这也是一个副产品。
An End-to-End Multi-Module Audio Deepfake Generation System for ADD Challenge 2023
results: 根据论文的报告， authors使用了这个模型参加了2023年ADD挑战赛Track 1.1，并获得了44.97%的Weighted Deception Success Rate（WDSR），即第一名。Abstract
The task of synthetic speech generation is to generate language content from a given text, then simulating fake human voice.The key factors that determine the effect of synthetic speech generation mainly include speed of generation, accuracy of word segmentation, naturalness of synthesized speech, etc. This paper builds an end-to-end multi-module synthetic speech generation model, including speaker encoder, synthesizer based on Tacotron2, and vocoder based on WaveRNN. In addition, we perform a lot of comparative experiments on different datasets and various model structures. Finally, we won the first place in the ADD 2023 challenge Track 1.1 with the weighted deception success rate (WDSR) of 44.97%.
摘要
“synthetic speech generation的任务是将文本转换为语音内容，然后模拟人工声音。关键因素包括生成速度、词汇分 segmentation精度、生成的语音自然度等。本文建立了端到端多模块合成语音模型，包括话者编码器、基于 Tacotron2 的合成器和基于 WaveRNN 的 vocoder。此外，我们进行了许多比较实验，包括不同数据集和不同模型结构。最后，我们在 ADD 2023 挑战赛 Track 1.1 中获得了44.97%的权重诱导成功率（WDSR）。”
Fraunhofer SIT at CheckThat! 2023: Mixing Single-Modal Classifiers to Estimate the Check-Worthiness of Multi-Modal Tweets
paper_authors: Raphael Frick, Inna Vogel for:This paper aims to improve the efficiency of fact-checking on social media by developing a novel approach to detecting the check-worthiness of multi-modal tweets.methods:The proposed approach uses two classifiers, each trained on a single modality (text or image), and combines their outputs to determine the check-worthiness of a tweet. The text classifier uses OCR analysis to extract embedded text from images.results:The proposed approach achieved an F1 score of 0.7297 on the private test set of the CheckThat! 2023 Task 1A, placing first among all submissions.Abstract
The option of sharing images, videos and audio files on social media opens up new possibilities for distinguishing between false information and fake news on the Internet. Due to the vast amount of data shared every second on social media, not all data can be verified by a computer or a human expert. Here, a check-worthiness analysis can be used as a first step in the fact-checking pipeline and as a filtering mechanism to improve efficiency. This paper proposes a novel way of detecting the check-worthiness in multi-modal tweets. It takes advantage of two classifiers, each trained on a single modality. For image data, extracting the embedded text with an OCR analysis has shown to perform best. By combining the two classifiers, the proposed solution was able to place first in the CheckThat! 2023 Task 1A with an F1 score of 0.7297 achieved on the private test set.
摘要
“社交媒体上分享图片、视频和音频文件打开了新的可能性，以 отличать假信息和谣言网络上。由于社交媒体每秒钟数据量太多，不能由计算机或人工专家所检查。这篇论文提出了一种检查可信worthiness的新方法，利用两个分类器，每个分类器在单一模式上训练。对于图像数据，使用OCR分析提取嵌入文本最佳。将两个分类器结合使用，提出的解决方案在CheckThat! 2023任务1A中获得了0.7297的F1分数。”
results: 该论文使用CNF方法对Nambu-Goto strings进行了数值计算，并获得了可靠的EST预测值。Abstract
Effective String Theory (EST) represents a powerful non-perturbative approach to describe confinement in Yang-Mills theory that models the confining flux tube as a thin vibrating string. EST calculations are usually performed using the zeta-function regularization: however there are situations (for instance the study of the shape of the flux tube or of the higher order corrections beyond the Nambu-Goto EST) which involve observables that are too complex to be addressed in this way. In this paper we propose a numerical approach based on recent advances in machine learning methods to circumvent this problem. Using as a laboratory the Nambu-Goto string, we show that by using a new class of deep generative models called Continuous Normalizing Flows it is possible to obtain reliable numerical estimates of EST predictions.
摘要
效果串理论（EST）表示一种强大的非拟合方法来描述 Yang-Mills 理论中的吸引作用，将吸引 flux tube 模型为细膨散的弹性String。EST 计算通常使用 zeta-function 正则化：但有些情况（如研究 flux tube 的形状或高阶修正项）需要访问 Observables 是不可能通过这种方式进行处理。在这篇论文中，我们提出一种基于最近的机器学习方法的数字方法来解决这个问题。使用 Nambu-Goto string 作为实验室，我们示出了使用一种新的深度生成模型called Continuous Normalizing Flows 可以获得可靠的 EST 预测。
Streamlined Lensed Quasar Identification in Multiband Images via Ensemble Networks
results: 通过 averaging 多种CNNs和ViTs，可以减小误 positives，并使用HSC图像和其他数据库，找到大约600万个源，其中3080个 source有高概率是强式折射辐射源，需要spectroscopic confirmation。Abstract
Quasars experiencing strong lensing offer unique viewpoints on subjects related to the cosmic expansion rate, the dark matter profile within the foreground deflectors, and the quasar host galaxies. Unfortunately, identifying them in astronomical images is challenging since they are overwhelmed by the abundance of non-lenses. To address this, we have developed a novel approach by ensembling cutting-edge convolutional networks (CNNs) -- for instance, ResNet, Inception, NASNet, MobileNet, EfficientNet, and RegNet -- along with vision transformers (ViTs) trained on realistic galaxy-quasar lens simulations based on the Hyper Suprime-Cam (HSC) multiband images. While the individual model exhibits remarkable performance when evaluated against the test dataset, achieving an area under the receiver operating characteristic curve of $>$97.3% and a median false positive rate of 3.6%, it struggles to generalize in real data, indicated by numerous spurious sources picked by each classifier. A significant improvement is achieved by averaging these CNNs and ViTs, resulting in the impurities being downsized by factors up to 50. Subsequently, combining the HSC images with the UKIRT, VISTA, and unWISE data, we retrieve approximately 60 million sources as parent samples and reduce this to 892,609 after employing a photometry preselection to discover $z>1.5$ lensed quasars with Einstein radii of $\theta_\mathrm{E}<5$ arcsec. Afterward, the ensemble classifier indicates 3080 sources with a high probability of being lenses, for which we visually inspect, yielding 210 prevailing candidates awaiting spectroscopic confirmation. These outcomes suggest that automated deep learning pipelines hold great potential in effectively detecting strong lenses in vast datasets with minimal manual visual inspection involved.
摘要
astronomy 关键词：Quasars，强大的 gravitational lensing，cosmic expansion rate，dark matter，quasar host galaxiesQuasars 经历强大的 gravitational lensing 提供了一种独特的视角，可以研究cosmic expansion rate 和 dark matter Profile within the foreground deflectors 以及 Quasar host galaxies。然而，在天文图像中 identific Quasars 是一项挑战，因为它们被非 gravitational lens 的丰富数量掩盖。为 Address this, we have developed a novel approach by ensembling cutting-edge Convolutional Neural Networks (CNNs) 和 Vision Transformers (ViTs) ，例如 ResNet, Inception, NASNet, MobileNet, EfficientNet, and RegNet ，并在 Hyper Suprime-Cam (HSC) 多波段图像上进行实际的 galaxy-quasar lens simulations。although the individual model shows remarkable performance when evaluated against the test dataset, achieving an area under the receiver operating characteristic curve of $>$97.3% and a median false positive rate of 3.6%，it struggles to generalize in real data, indicated by numerous spurious sources picked by each classifier。a significant improvement is achieved by averaging these CNNs and ViTs, resulting in the impurities being downsized by factors up to 50。subsequently, we combine the HSC images with the UKIRT, VISTA, and unWISE data, retrieve approximately 60 million sources as parent samples, and reduce this to 892,609 after employing a photometry preselection to discover $z>1.5$ lensed quasars with Einstein radii of $\theta_\mathrm{E}<5$ arcsec。afterward, the ensemble classifier indicates 3080 sources with a high probability of being lenses, for which we visually inspect, yielding 210 prevailing candidates awaiting spectroscopic confirmation。these outcomes suggest that automated deep learning pipelines hold great potential in effectively detecting strong lenses in vast datasets with minimal manual visual inspection involved。
Empirically Validating Conformal Prediction on Modern Vision Architectures Under Distribution Shift and Long-tailed Data
results: 研究发现，即使使用多种conformal方法和神经网络家族，其性能在分布shift和长尾分布下都会受到很大影响，导致安全保证被违反。Abstract
Conformal prediction has emerged as a rigorous means of providing deep learning models with reliable uncertainty estimates and safety guarantees. Yet, its performance is known to degrade under distribution shift and long-tailed class distributions, which are often present in real world applications. Here, we characterize the performance of several post-hoc and training-based conformal prediction methods under these settings, providing the first empirical evaluation on large-scale datasets and models. We show that across numerous conformal methods and neural network families, performance greatly degrades under distribution shifts violating safety guarantees. Similarly, we show that in long-tailed settings the guarantees are frequently violated on many classes. Understanding the limitations of these methods is necessary for deployment in real world and safety-critical applications.
摘要
宽泛预测（Conformal prediction）已经成为深度学习模型提供可靠的不确定性估计和安全保证的有力方法。然而，其性能知道会在分布变换和长尾类分布下逐渐下降，这些情况通常存在在实际应用中。我们在这里对几种后处和训练基础的宽泛预测方法进行了首次实证评估，并在大规模数据和模型上进行了评估。我们发现，无论是哪些方法或者哪些神经网络家族，在分布变换下都会违反安全保证。同时，在长尾设置下，保证也frequently被违反了许多类别。理解这些方法的限制是必要的，以便在实际应用和安全关键应用中部署。
Supervised Manifold Learning via Random Forest Geometry-Preserving Proximities
methods: 本文使用的方法包括 random forest proximities 和 diffusion-based algorithms，以保持本数据集中的地方结构和全局结构。
results: 本文表明，使用这种新的方法可以更好地保持数据集中的地方结构和全局结构，并且可以提高 manifold learning 的效果。Abstract
Manifold learning approaches seek the intrinsic, low-dimensional data structure within a high-dimensional space. Mainstream manifold learning algorithms, such as Isomap, UMAP, $t$-SNE, Diffusion Map, and Laplacian Eigenmaps do not use data labels and are thus considered unsupervised. Existing supervised extensions of these methods are limited to classification problems and fall short of uncovering meaningful embeddings due to their construction using order non-preserving, class-conditional distances. In this paper, we show the weaknesses of class-conditional manifold learning quantitatively and visually and propose an alternate choice of kernel for supervised dimensionality reduction using a data-geometry-preserving variant of random forest proximities as an initialization for manifold learning methods. We show that local structure preservation using these proximities is near universal across manifold learning approaches and global structure is properly maintained using diffusion-based algorithms.
摘要
manifold learning方法寻找高维空间中内在的低维数据结构。主流 manifold learning 算法，如 Isomap、UMAP、t-SNE、Diffusion Map 和 Laplacian Eigenmaps，不使用数据标签，因此被视为无监督的。现有的监督扩展方法仅适用于分类问题，而且由于其使用不 preserve 的、类别 conditional 距离，因此无法找到有意义的嵌入。在这篇论文中，我们证明了类别 conditional manifold learning 的弱点，并提出了一种不同的kernel来实现监督的维度减少。我们显示了这些距离的本地结构保持是near universal across manifold learningapproaches，而且使用扩散算法来保持全局结构。
When Can Linear Learners be Robust to Indiscriminate Poisoning Attacks?
results: 我们的结果表明，如果数据集中的类划分具有低差异和低方差，并且杂化攻击点的约束集的大小很小，那么线性学习器就可以具有Robustness 对杂化攻击。这些发现解释了一些学习任务在杂化攻击下的差异性，对于理解杂化攻击的根本原因是非常重要的一步。Abstract
We study indiscriminate poisoning for linear learners where an adversary injects a few crafted examples into the training data with the goal of forcing the induced model to incur higher test error. Inspired by the observation that linear learners on some datasets are able to resist the best known attacks even without any defenses, we further investigate whether datasets can be inherently robust to indiscriminate poisoning attacks for linear learners. For theoretical Gaussian distributions, we rigorously characterize the behavior of an optimal poisoning attack, defined as the poisoning strategy that attains the maximum risk of the induced model at a given poisoning budget. Our results prove that linear learners can indeed be robust to indiscriminate poisoning if the class-wise data distributions are well-separated with low variance and the size of the constraint set containing all permissible poisoning points is also small. These findings largely explain the drastic variation in empirical attack performance of the state-of-the-art poisoning attacks on linear learners across benchmark datasets, making an important initial step towards understanding the underlying reasons some learning tasks are vulnerable to data poisoning attacks.
摘要
我们研究不偏辐射毒素攻击，针对线性学习器，敌对人在训练数据中插入一些手动制作的示例，以达到让引导出的模型测试错误高的目的。受观察到一些数据集上的线性学习器能够不受任何防御措施下 resist最佳攻击的现象，我们进一步调查是否存在一些数据集可以自然地抵抗不偏辐射毒素攻击。对于理论 Gaussian 分布，我们仔细描述了最佳毒素攻击策略，即在给定毒素预算下，可以使引导出的模型测试错误最大化的攻击策略。我们的结果表明，如果数据集中的类别数据分布具有低差异和低方差，而且制定集中包含所有可能的毒素点的大小也很小，那么线性学习器就可以具有抵抗不偏辐射毒素攻击的性能。这些发现解释了一些学习任务对于数据毒素攻击的 empirical 攻击性能的悬峰性，从而为我们更好的理解这些任务的潜在原因。
PIGNet2: A Versatile Deep Learning-based Protein-Ligand Interaction Prediction Model for Binding Affinity Scoring and Virtual Screening
results: 研究显示，该模型在不同的测试中具有显著的改进，包括衍生数据集测试和距离可能性学习测试，并达到了与当前最佳性能相当的水平。这表明该方法在药物发现中具有潜在的应用前景。Abstract
Prediction of protein-ligand interactions (PLI) plays a crucial role in drug discovery as it guides the identification and optimization of molecules that effectively bind to target proteins. Despite remarkable advances in deep learning-based PLI prediction, the development of a versatile model capable of accurately scoring binding affinity and conducting efficient virtual screening remains a challenge. The main obstacle in achieving this lies in the scarcity of experimental structure-affinity data, which limits the generalization ability of existing models. Here, we propose a viable solution to address this challenge by introducing a novel data augmentation strategy combined with a physics-informed graph neural network. The model showed significant improvements in both scoring and screening, outperforming task-specific deep learning models in various tests including derivative benchmarks, and notably achieving results comparable to the state-of-the-art performance based on distance likelihood learning. This demonstrates the potential of this approach to drug discovery.
摘要
ENGAGE: Explanation Guided Data Augmentation for Graph Representation Learning
methods: 本文提出了一种名为ENGAGE（ExplaNation Guided data AuGmEntation）的方法，其中使用了一种高效的无监督解释方法，称为简化活动图，以评估节点的重要性。此外，本文还提出了两种数据增强方案，一种是Structural Augmentation，另一种是Feature Augmentation。
results: 经过实验证明，ENGAGE方法可以在不同的模型架构和真实的图数据上显著提高图数据的表示能力。同时，ENGAGE方法还可以在图级和节点级任务上达到优秀的性能。Abstract
The recent contrastive learning methods, due to their effectiveness in representation learning, have been widely applied to modeling graph data. Random perturbation is widely used to build contrastive views for graph data, which however, could accidentally break graph structures and lead to suboptimal performance. In addition, graph data is usually highly abstract, so it is hard to extract intuitive meanings and design more informed augmentation schemes. Effective representations should preserve key characteristics in data and abandon superfluous information. In this paper, we propose ENGAGE (ExplaNation Guided data AuGmEntation), where explanation guides the contrastive augmentation process to preserve the key parts in graphs and explore removing superfluous information. Specifically, we design an efficient unsupervised explanation method called smoothed activation map as the indicator of node importance in representation learning. Then, we design two data augmentation schemes on graphs for perturbing structural and feature information, respectively. We also provide justification for the proposed method in the framework of information theories. Experiments of both graph-level and node-level tasks, on various model architectures and on different real-world graphs, are conducted to demonstrate the effectiveness and flexibility of ENGAGE. The code of ENGAGE can be found: https://github.com/sycny/ENGAGE.
摘要
近期的对比学习方法，因其在表示学习中的效iveness，在图数据模型中广泛应用。Random perturbation 广泛使用于建立对比视图，但可能意外破坏图结构，导致表现下降。此外，图数据通常很抽象，因此很难提取直观意义和设计更 Informed的扩充方案。有效的表示应保留数据中的关键特征，抛弃不必要的信息。在这篇论文中，我们提出了ENGAGE（ExplaNation Guided data AuGmEntation），其中解释指导对比扩充过程，以保留图中关键部分并探索抛弃不必要信息。具体来说，我们设计了一种高效的无监督解释方法，即简化活动图作为表示学习中节点重要性的指标。然后，我们设计了对图结构和特征信息进行扩充的两种方案。我们还提供了对提案方法的信息理论 justify。在图级和节点级任务上，使用不同的模型架构和真实的图数据进行实验，以证明ENGAGE的有效性和灵活性。ENGAGE的代码可以在 GitHub 上找到：https://github.com/sycny/ENGAGE。
Transport, Variational Inference and Diffusions: with Applications to Annealed Flows and Schrödinger Bridges
results: 作者通过一系列的生成模型示例和一个基于双峰的罕见事件任务，展示了提议的方法的潜力。Abstract
This paper explores the connections between optimal transport and variational inference, with a focus on forward and reverse time stochastic differential equations and Girsanov transformations.We present a principled and systematic framework for sampling and generative modelling centred around divergences on path space. Our work culminates in the development of a novel score-based annealed flow technique (with connections to Jarzynski and Crooks identities from statistical physics) and a regularised iterative proportional fitting (IPF)-type objective, departing from the sequential nature of standard IPF. Through a series of generative modelling examples and a double-well-based rare event task, we showcase the potential of the proposed methods.
摘要
这篇论文探讨优化运输和变量推断之间的连接，特别是关注forward和reverse时间随机 diffeq和 Girсанов变换。我们提出了一种原理性的和系统的框架，用于 sampling和生成模型，围绕 divergence on path space。我们的工作最终得出了一种新的Score-based annealed flow技术（与统计物理中的Jarzynski和Crooks标准相关）和一种常规iterative proportional fitting（IPF）类型的目标函数。通过一系列的生成模型示例和一个基于double-well的罕见事件任务，我们展示了提议的方法的潜力。Note: The translation is done using a machine translation tool, and may not be perfect or idiomatic.
paper_authors: Marco Pegoraro, Sanketh Vedula, Aviv A. Rosenberg, Irene Tallini, Emanuele Rodolà, Alex M. Bronstein
For: 本研究旨在探讨量规 regression（QR）在多变量分布上的应用，特别是在多变量分布上的 manifold 上。* Methods: 本研究使用优化的运输理论和c-卷函数来定义高维变量在 manifold 上的 conditional vector quantile function（M-CVQF）。* Results: 研究人员通过synthetic data experiment来证明M-CVQF的有效性，并提供了非欧氏分布中量规的含义。Abstract
Quantile regression (QR) is a statistical tool for distribution-free estimation of conditional quantiles of a target variable given explanatory features. QR is limited by the assumption that the target distribution is univariate and defined on an Euclidean domain. Although the notion of quantiles was recently extended to multi-variate distributions, QR for multi-variate distributions on manifolds remains underexplored, even though many important applications inherently involve data distributed on, e.g., spheres (climate measurements), tori (dihedral angles in proteins), or Lie groups (attitude in navigation). By leveraging optimal transport theory and the notion of $c$-concave functions, we meaningfully define conditional vector quantile functions of high-dimensional variables on manifolds (M-CVQFs). Our approach allows for quantile estimation, regression, and computation of conditional confidence sets. We demonstrate the approach's efficacy and provide insights regarding the meaning of non-Euclidean quantiles through preliminary synthetic data experiments.
摘要
Temporal Graph Benchmark for Machine Learning on Temporal Graphs
methods: 本文使用了多种常见的 temporal graph 模型，并设计了基于实际用 caso 的评估协议。
results: 研究发现，存在许多 temporal graph dataset，模型的性能可以很大差异，而且简单的方法经常超越现有的 temporal graph 模型。这些发现打开了未来 temporal graph 研究的可能性。Abstract
We present the Temporal Graph Benchmark (TGB), a collection of challenging and diverse benchmark datasets for realistic, reproducible, and robust evaluation of machine learning models on temporal graphs. TGB datasets are of large scale, spanning years in duration, incorporate both node and edge-level prediction tasks and cover a diverse set of domains including social, trade, transaction, and transportation networks. For both tasks, we design evaluation protocols based on realistic use-cases. We extensively benchmark each dataset and find that the performance of common models can vary drastically across datasets. In addition, on dynamic node property prediction tasks, we show that simple methods often achieve superior performance compared to existing temporal graph models. We believe that these findings open up opportunities for future research on temporal graphs. Finally, TGB provides an automated machine learning pipeline for reproducible and accessible temporal graph research, including data loading, experiment setup and performance evaluation. TGB will be maintained and updated on a regular basis and welcomes community feedback. TGB datasets, data loaders, example codes, evaluation setup, and leaderboards are publicly available at https://tgb.complexdatalab.com/ .
摘要
我们介绍了 Temporal Graph Benchmark（TGB），一个包含具有具有强大挑战和多样化的 bencmark 数据集，用于真实、可重现和Robust 评估机器学习模型在时间图上。TGB 数据集的规模很大，覆盖了多年的时间 duration，包括节点和边级别预测任务，并覆盖了社交、贸易、交易和交通网络等多种领域。为两个任务，我们设计了基于实际用例的评估协议。我们对每个数据集进行了广泛的测试，发现公共模型在不同数据集上的性能可以截然不同。此外，在动态节点属性预测任务上，我们发现简单的方法经常超越现有的时间图模型。我们认为这些发现开发了未来研究时间图的机遇。此外，TGB 提供了一个自动化机器学习管道，用于可重现和可访问的时间图研究，包括数据加载、实验设置和性能评估。TGB 将会在 régular basis 维护和更新，欢迎社区反馈。TGB 数据集、数据加载器、示例代码、评估设置和排名是公共可用的，可以通过 https://tgb.complexdatalab.com/ 访问。
Neural Chronos ODE: Unveiling Temporal Patterns and Forecasting Future and Past Trends in Time Series Data
results: 实验结果表明Neural CODE比Neural ODE更好地学习系统的时间序列，而CODE-BiRNN/-BiGRU/-BiLSTM在三个实际时间序列任务上表现最佳，包括数据缺失估计、前向和反向推算等。Abstract
This work introduces Neural Chronos Ordinary Differential Equations (Neural CODE), a deep neural network architecture that fits a continuous-time ODE dynamics for predicting the chronology of a system both forward and backward in time. To train the model, we solve the ODE as an initial value problem and a final value problem, similar to Neural ODEs. We also explore two approaches to combining Neural CODE with Recurrent Neural Networks by replacing Neural ODE with Neural CODE (CODE-RNN), and incorporating a bidirectional RNN for full information flow in both time directions (CODE-BiRNN), and variants with other update cells namely GRU and LSTM: CODE-GRU, CODE-BiGRU, CODE-LSTM, CODE-BiLSTM. Experimental results demonstrate that Neural CODE outperforms Neural ODE in learning the dynamics of a spiral forward and backward in time, even with sparser data. We also compare the performance of CODE-RNN/-GRU/-LSTM and CODE-BiRNN/-BiGRU/-BiLSTM against ODE-RNN/-GRU/-LSTM on three real-life time series data tasks: imputation of missing data for lower and higher dimensional data, and forward and backward extrapolation with shorter and longer time horizons. Our findings show that the proposed architectures converge faster, with CODE-BiRNN/-BiGRU/-BiLSTM consistently outperforming the other architectures on all tasks.
摘要
Joint Coordinate Regression and Association For Multi-Person Pose Estimation, A Pure Neural Network Approach
results: 对于MS COCO和CrowdPose测试集，JCRA的实验结果表明，它在准确率和效率两个方面都超过了现有的方法。具体来说，JCRA在MS COCO测试集上达到了69.2 mAP，并且在推理加速方面比前一代底层算法快78%。Abstract
We introduce a novel one-stage end-to-end multi-person 2D pose estimation algorithm, known as Joint Coordinate Regression and Association (JCRA), that produces human pose joints and associations without requiring any post-processing. The proposed algorithm is fast, accurate, effective, and simple. The one-stage end-to-end network architecture significantly improves the inference speed of JCRA. Meanwhile, we devised a symmetric network structure for both the encoder and decoder, which ensures high accuracy in identifying keypoints. It follows an architecture that directly outputs part positions via a transformer network, resulting in a significant improvement in performance. Extensive experiments on the MS COCO and CrowdPose benchmarks demonstrate that JCRA outperforms state-of-the-art approaches in both accuracy and efficiency. Moreover, JCRA demonstrates 69.2 mAP and is 78\% faster at inference acceleration than previous state-of-the-art bottom-up algorithms. The code for this algorithm will be publicly available.
摘要
我们提出了一种新的一stage终端多人2D姿态估计算法，称为共同坐标回归和关联（JCRA），该算法可以不需要任何后处理生成人体姿态关节和关联。我们提出的算法具有快速、准确、有效和简单的特点。我们使用一stage终端网络架构，这有效地提高了JCRA的推理速度。同时，我们设计了对Encoder和Decoder网络结构的 симметри化，确保高精度地标定关键点。它采用一种直接输出部位位置的transformer网络架构，从而导致了显著提高的性能。我们在COCO和CrowdPose benchmark上进行了广泛的实验，demonstrates that JCRA exceeds state-of-the-art approaches in both accuracy and efficiency. In addition, JCRA achieves 69.2 mAP and is 78% faster at inference acceleration than previous state-of-the-art bottom-up algorithms. The code for this algorithm will be publicly available.
Capafoldable: self-tracking foldable smart textiles with capacitive sensing
paper_authors: Lala Shakti Swarup Ray, Daniel Geißler, Bo Zhou, Paul Lukowicz, Berit Greinke
For: 能够检测 estructural motions 的 smart textile* Methods: combining folded fabric structures 和 capacitive sensing，使用 state-of-the-art sensing circuits 和 deep learning technologies* Results: 可以很准确地 reconstruction geometry primitives defining patch shape from capacitive signals，tracking error 只有 1cm，可以应用于新的 smart textile 应用程序。Here’s the full text in Traditional Chinese:这个研究旨在开发一种能够检测结构运动的智能纱布，通过结合折叠纱布结构和导电纱布感知技术，并使用现代感知电路和深度学习技术来实现。我们实验了两种折叠模式，即Accordion和Chevron，每种模式都有两种导电纱布感知器的配置。通过我们的方法，可以很准确地从导电信号中重建geometry primitives定义纱布形状，追踪误差只有1cm，可以应用于新的智能纱布应用程序。Abstract
Folding is an unique structural technique to enable planer materials with motion or 3D mechanical properties. Textile-based capacitive sensing has shown to be sensitive to the geometry deformation and relative motion of conductive textiles. In this work, we propose a novel self-tracking foldable smart textile by combining folded fabric structures and capacitive sensing to detect the structural motions using state-of-the-art sensing circuits and deep learning technologies. We created two folding patterns, Accordion and Chevron, each with two layouts of capacitive sensors in the form of thermobonded conductive textile patches. In an experiment of manually moving patches of the folding patterns, we developed deep neural network to learn and reconstruct the vision-tracked shape of the patches. Through our approach, the geometry primitives defining the patch shape can be reconstructed from the capacitive signals with R-squared value of up to 95\% and tracking error of 1cm for 22.5cm long patches. With mechanical, electrical and sensing properties, Capafoldable could enable a new range of smart textile applications.
摘要
折叠是一种独特的结构技术，可以让平面材料具有运动或3D机械性能。基于织物的电容式感测技术已经证明可以感测织物的几何变形和相对运动。在这项工作中，我们提出了一种新的自追踪式折叠智能织物，通过结合折叠布结构和电容式感测技术来检测结构运动。我们设计了两种折叠模式，即腰棒和斜线，每种模式有两种布置的电容器在形式为热粘合的导电织物贴片上。在手动移动贴片的实验中，我们开发了深度神经网络来学习和重建通过视觉跟踪的贴片形状。通过我们的方法，贴片的几何基本元可以从电容信号中被重建，R-squared值可达95%，跟踪错误为1cm，对22.5cm长的贴片来说。拥有机械、电学和感测性能，Capafoldable可以开拓新的智能织物应用领域。
paper_authors: Lee Richardson, Alessandro Zito, Dylan Greaves, Jacopo Soriano
for: 优化产品，提高产品质量
methods: 使用代理指标，同时提高预测精度和敏感度
results: 提高决策 velocicty和决策质量，代理指标比北星指标高八倍敏感度Abstract
North star metrics and online experimentation play a central role in how technology companies improve their products. In many practical settings, however, evaluating experiments based on the north star metric directly can be difficult. The two most significant issues are 1) low sensitivity of the north star metric and 2) differences between the short-term and long-term impact on the north star metric. A common solution is to rely on proxy metrics rather than the north star in experiment evaluation and launch decisions. Existing literature on proxy metrics concentrates mainly on the estimation of the long-term impact from short-term experimental data. In this paper, instead, we focus on the trade-off between the estimation of the long-term impact and the sensitivity in the short term. In particular, we propose the Pareto optimal proxy metrics method, which simultaneously optimizes prediction accuracy and sensitivity. In addition, we give an efficient multi-objective optimization algorithm that outperforms standard methods. We applied our methodology to experiments from a large industrial recommendation system, and found proxy metrics that are eight times more sensitive than the north star and consistently moved in the same direction, increasing the velocity and the quality of the decisions to launch new features.
摘要
北斗星指标和在线实验是技术公司产品改进的中心角色。然而，在实际设置中，直接基于北斗星指标评估实验的问题经常出现。主要问题包括1）北斗星指标敏感度低和2）短期和长期北斗星指标之间的差异。现有文献中的代表指标集中在长期影响的估计上，而不是短期影响。在这篇论文中，我们则关注了长期影响和短期敏感度之间的衡量。我们提出了最优化代表指标方法，同时保证了预测准确性和敏感度。此外，我们还提供了超过标准方法的高效多目标优化算法。我们在一大型工业推荐系统的实验中应用了我们的方法ологи，发现代表指标是北斗星指标的八倍敏感，并且一直逐渐增长，提高了决策启动新特性的速度和质量。
Environmental effects on emergent strategy in micro-scale multi-agent reinforcement learning
paper_authors: Samuel Tovey, David Zimmer, Christoph Lohrmann, Tobias Merkt, Simon Koppenhoefer, Veit-Lorenz Heuthe, Clemens Bechinger, Christian Holm
for: This paper explores the role of temperature in the emergence and efficacy of strategies in MARL systems using particle-based Langevin molecular dynamics simulations.
methods: The paper uses particle-based Langevin molecular dynamics simulations as a realistic representation of micro-scale environments, and introduces a novel Python package for studying microscopic agents using reinforcement learning.
results: The paper finds that at higher temperatures, the RL agents identify new strategies for achieving tasks, highlighting the importance of understanding this regime and providing insight into optimal training strategies for bridging the generalization gap between simulation and reality.Abstract
Multi-Agent Reinforcement Learning (MARL) is a promising candidate for realizing efficient control of microscopic particles, of which micro-robots are a subset. However, the microscopic particles' environment presents unique challenges, such as Brownian motion at sufficiently small length-scales. In this work, we explore the role of temperature in the emergence and efficacy of strategies in MARL systems using particle-based Langevin molecular dynamics simulations as a realistic representation of micro-scale environments. To this end, we perform experiments on two different multi-agent tasks in microscopic environments at different temperatures, detecting the source of a concentration gradient and rotation of a rod. We find that at higher temperatures, the RL agents identify new strategies for achieving these tasks, highlighting the importance of understanding this regime and providing insight into optimal training strategies for bridging the generalization gap between simulation and reality. We also introduce a novel Python package for studying microscopic agents using reinforcement learning (RL) to accompany our results.
摘要
Over-The-Air Federated Learning: Status Quo, Open Challenges, and Future Directions
results: 论文对OTA-FL的进展进行了分类和总结，包括单天线OTA-FL、多天线OTA-FL和利用emerging reconfigurable intelligent surface（RIS）技术的OTA-FL。同时，论文还讨论了OTA-FL的信任、安全和隐私方面的问题，并提出了未来研究的挑战和方向。Abstract
The development of applications based on artificial intelligence and implemented over wireless networks is increasingly rapidly and is expected to grow dramatically in the future. The resulting demand for the aggregation of large amounts of data has caused serious communication bottlenecks in wireless networks and particularly at the network edge. Over-the-air federated learning (OTA-FL), leveraging the superposition feature of multi-access channels (MACs), enables users at the network edge to share spectrum resources and achieves efficient and low-latency global model aggregation. This paper provides a holistic review of progress in OTA-FL and points to potential future research directions. Specifically, we classify OTA-FL from the perspective of system settings, including single-antenna OTA-FL, multi-antenna OTA-FL, and OTA-FL with the aid of the emerging reconfigurable intelligent surface (RIS) technology, and the contributions of existing works in these areas are summarized. Moreover, we discuss the trust, security and privacy aspects of OTA-FL, and highlight concerns arising from security and privacy. Finally, challenges and potential research directions are discussed to promote the future development of OTA-FL in terms of improving system performance, reliability, and trustworthiness. Specifical challenges to be addressed include model distortion under channel fading, the ineffective OTA aggregation of local models trained on substantially unbalanced data, and the limited accessibility and verifiability of individual local models.
摘要
发展基于人工智能的应用程序在无线网络上实施，迅速增长，未来将会继续增长迅速。这Resulting in 大量数据的聚合需求导致了无线网络中的通信瓶颈和特别是网络边缘的瓶颈。使用无线多接口通道（MACs）的超载特性，用户在网络边缘可以共享频率资源，实现高效且响应时间短的全球模型聚合。本文提供了无线 federated learning（OTA-FL）的总体评论，并指出了未来研究的可能性。 Specifically, we classify OTA-FL from the perspective of system settings, including single-antenna OTA-FL, multi-antenna OTA-FL, and OTA-FL with the aid of the emerging reconfigurable intelligent surface（RIS）技术，并 Summarize the contributions of existing works in these areas. In addition, we discuss the trust, security, and privacy aspects of OTA-FL, and highlight concerns arising from security and privacy. Finally, we discuss challenges and potential research directions to promote the future development of OTA-FL in terms of improving system performance, reliability, and trustworthiness. Specific challenges to be addressed include model distortion under channel fading, the ineffective OTA aggregation of local models trained on substantially unbalanced data, and the limited accessibility and verifiability of individual local models.
MoVie: Visual Model-Based Policy Adaptation for View Generalization
methods: 作者提出了一种简单 yet effective的方法，可以在测试时使模型基于视图的政策适应视图总结问题，不需要显式奖励信号和任何修改 durante training time。
results: 作者的方法在四种不同的场景下（包括 DMControl、xArm 和 Adroit 等）进行了18个任务的测试，与基eline相比，表现出了substantial advancements（相对提高33%、86% 和 152%）。这些出色的结果表明该方法在实际 роботех术应用中具有极大的潜力。Abstract
Visual Reinforcement Learning (RL) agents trained on limited views face significant challenges in generalizing their learned abilities to unseen views. This inherent difficulty is known as the problem of $\textit{view generalization}$. In this work, we systematically categorize this fundamental problem into four distinct and highly challenging scenarios that closely resemble real-world situations. Subsequently, we propose a straightforward yet effective approach to enable successful adaptation of visual $\textbf{Mo}$del-based policies for $\textbf{Vie}$w generalization ($\textbf{MoVie}$) during test time, without any need for explicit reward signals and any modification during training time. Our method demonstrates substantial advancements across all four scenarios encompassing a total of $\textbf{18}$ tasks sourced from DMControl, xArm, and Adroit, with a relative improvement of $\mathbf{33}$%, $\mathbf{86}$%, and $\mathbf{152}$% respectively. The superior results highlight the immense potential of our approach for real-world robotics applications. Videos are available at https://yangsizhe.github.io/MoVie/ .
摘要
视觉强化学习（RL）代理人在有限视角下接受训练，面临普遍化视角问题的挑战。这种问题在实际情况中非常困难，被称为“视觉普遍化”问题。在这种工作中，我们系统地将这个基本问题分为四个明确和具有挑战性的enario，与实际情况很接近。然后，我们提议一种简单 yet effective的方法，在测试时使模型基于视觉的策略适应视觉普遍化，不需要显式奖励信号和任何修改 durante 训练时间。我们的方法在四个scenario中展示了明显的进步，涵盖了DMControl、xArm和Adroit中的共计18个任务，相对改进率为33%、86%和152%。这些出色的结果表明我们的方法在实际 робо学应用中具有巨大的潜力。视频可以在https://yangsizhe.github.io/MoVie/ 中找到。
REAL: A Representative Error-Driven Approach for Active Learning
results: 实验表明，使用 REAL 方法可以在多种 гипер参数设置下，consistently 超越所有最佳基准方法 regarding 准确率和 F1-macro 分数。同时，我们的分析还显示了 REAL 方法选择的 pseudo errors 与真实错误的分布相匹配。Abstract
Given a limited labeling budget, active learning (AL) aims to sample the most informative instances from an unlabeled pool to acquire labels for subsequent model training. To achieve this, AL typically measures the informativeness of unlabeled instances based on uncertainty and diversity. However, it does not consider erroneous instances with their neighborhood error density, which have great potential to improve the model performance. To address this limitation, we propose $REAL$, a novel approach to select data instances with $\underline{R}$epresentative $\underline{E}$rrors for $\underline{A}$ctive $\underline{L}$earning. It identifies minority predictions as \emph{pseudo errors} within a cluster and allocates an adaptive sampling budget for the cluster based on estimated error density. Extensive experiments on five text classification datasets demonstrate that $REAL$ consistently outperforms all best-performing baselines regarding accuracy and F1-macro scores across a wide range of hyperparameter settings. Our analysis also shows that $REAL$ selects the most representative pseudo errors that match the distribution of ground-truth errors along the decision boundary. Our code is publicly available at https://github.com/withchencheng/ECML_PKDD_23_Real.
摘要
(Simplified Chinese translation) Given a limited labeling budget, active learning (AL) aims to sample the most informative instances from an unlabeled pool to acquire labels for subsequent model training. To achieve this, AL typically measures the informativeness of unlabeled instances based on uncertainty and diversity. However, it does not consider erroneous instances with their neighborhood error density, which have great potential to improve the model performance. To address this limitation, we propose $REAL$, a novel approach to select data instances with $\underline{R}$epresentative $\underline{E}$rrors for $\underline{A}$ctive $\underline{L}$earning. It identifies minority predictions as \emph{pseudo errors} within a cluster and allocates an adaptive sampling budget for the cluster based on estimated error density. Extensive experiments on five text classification datasets demonstrate that $REAL$ consistently outperforms all best-performing baselines regarding accuracy and F1-macro scores across a wide range of hyperparameter settings. Our analysis also shows that $REAL$ selects the most representative pseudo errors that match the distribution of ground-truth errors along the decision boundary. Our code is publicly available at https://github.com/withchencheng/ECML_PKDD_23_Real.
OpenClinicalAI: An Open and Dynamic Model for Alzheimer’s Disease Diagnosis
results: 实验结果显示，OpenClinicalAI 比前一代模型具有更好的性能和较少的临床检查次数。Abstract
Although Alzheimer's disease (AD) cannot be reversed or cured, timely diagnosis can significantly reduce the burden of treatment and care. Current research on AD diagnosis models usually regards the diagnosis task as a typical classification task with two primary assumptions: 1) All target categories are known a priori; 2) The diagnostic strategy for each patient is consistent, that is, the number and type of model input data for each patient are the same. However, real-world clinical settings are open, with complexity and uncertainty in terms of both subjects and the resources of the medical institutions. This means that diagnostic models may encounter unseen disease categories and need to dynamically develop diagnostic strategies based on the subject's specific circumstances and available medical resources. Thus, the AD diagnosis task is tangled and coupled with the diagnosis strategy formulation. To promote the application of diagnostic systems in real-world clinical settings, we propose OpenClinicalAI for direct AD diagnosis in complex and uncertain clinical settings. This is the first powerful end-to-end model to dynamically formulate diagnostic strategies and provide diagnostic results based on the subject's conditions and available medical resources. OpenClinicalAI combines reciprocally coupled deep multiaction reinforcement learning (DMARL) for diagnostic strategy formulation and multicenter meta-learning (MCML) for open-set recognition. The experimental results show that OpenClinicalAI achieves better performance and fewer clinical examinations than the state-of-the-art model. Our method provides an opportunity to embed the AD diagnostic system into the current health care system to cooperate with clinicians to improve current health care.
摘要
although Alzheimer's disease (AD) cannot be reversed or cured, timely diagnosis can significantly reduce the burden of treatment and care. Current research on AD diagnosis models usually regards the diagnosis task as a typical classification task with two primary assumptions: 1) all target categories are known a priori; 2) the diagnostic strategy for each patient is consistent, that is, the number and type of model input data for each patient are the same. However, real-world clinical settings are open, with complexity and uncertainty in terms of both subjects and the resources of the medical institutions. This means that diagnostic models may encounter unseen disease categories and need to dynamically develop diagnostic strategies based on the subject's specific circumstances and available medical resources. Thus, the AD diagnosis task is tangled and coupled with the diagnosis strategy formulation. To promote the application of diagnostic systems in real-world clinical settings, we propose OpenClinicalAI for direct AD diagnosis in complex and uncertain clinical settings. This is the first powerful end-to-end model to dynamically formulate diagnostic strategies and provide diagnostic results based on the subject's conditions and available medical resources. OpenClinicalAI combines reciprocally coupled deep multiaction reinforcement learning (DMARL) for diagnostic strategy formulation and multicenter meta-learning (MCML) for open-set recognition. The experimental results show that OpenClinicalAI achieves better performance and fewer clinical examinations than the state-of-the-art model. Our method provides an opportunity to embed the AD diagnostic system into the current health care system to cooperate with clinicians to improve current health care.
A Dual Stealthy Backdoor: From Both Spatial and Frequency Perspectives
results: 对四个预测集进行了广泛测试，并获得了较高的攻击成功率和隐藏性Abstract
Backdoor attacks pose serious security threats to deep neural networks (DNNs). Backdoored models make arbitrarily (targeted) incorrect predictions on inputs embedded with well-designed triggers while behaving normally on clean inputs. Many works have explored the invisibility of backdoor triggers to improve attack stealthiness. However, most of them only consider the invisibility in the spatial domain without explicitly accounting for the generation of invisible triggers in the frequency domain, making the generated poisoned images be easily detected by recent defense methods. To address this issue, in this paper, we propose a DUal stealthy BAckdoor attack method named DUBA, which simultaneously considers the invisibility of triggers in both the spatial and frequency domains, to achieve desirable attack performance, while ensuring strong stealthiness. Specifically, we first use Discrete Wavelet Transform to embed the high-frequency information of the trigger image into the clean image to ensure attack effectiveness. Then, to attain strong stealthiness, we incorporate Fourier Transform and Discrete Cosine Transform to mix the poisoned image and clean image in the frequency domain. Moreover, the proposed DUBA adopts a novel attack strategy, in which the model is trained with weak triggers and attacked with strong triggers to further enhance the attack performance and stealthiness. We extensively evaluate DUBA against popular image classifiers on four datasets. The results demonstrate that it significantly outperforms the state-of-the-art backdoor attacks in terms of the attack success rate and stealthiness
摘要
深度神经网络（DNN）受到后门攻击的安全威胁。后门模型会在嵌入了高效设计的触发器的输入上提供targeted incorrect predictions，而不会对干净输入产生影响。许多研究探讨了后门触发器的隐藏性，但大多数只是在空间领域内不显式地考虑了生成隐藏的触发器，使得生成的毒素图像可以轻松地被现有的防御方法检测。为解决这个问题，在这篇论文中，我们提出了DUal stealthy BAckdoor attack方法（DUBA），该方法同时考虑了触发器在空间和频域内的隐藏性，以实现desirable的攻击性能，同时保证强大的隐身性。具体来说，我们首先使用Discrete Wavelet Transform将高频信息 embed到了干净图像中，以确保攻击效果。然后，为了进一步增强隐身性，我们使用Fourier Transform和Discrete Cosine Transform将毒素图像和干净图像混合在频域中。此外，我们提出的DUBA采用了一种新的攻击策略，在该策略中，模型被训练使用弱触发器，并在攻击时使用强触发器，以进一步提高攻击性能和隐身性。我们对四个数据集进行了广泛的测试，结果显示，DUBA可以具有较高的攻击成功率和隐身性。
Neural Architecture Transfer 2: A Paradigm for Improving Efficiency in Multi-Objective Neural Architecture Search
methods: 本文提出了Once-For-All（OFA）和其 successor Once-For-All-2（OFAv2）技术，以及Neural Architecture Transfer（NAT）技术，用于自动设计任务优化的人工神经网络。
results: 本研究表明，NATv2可以成功地改进NAT，并在多目标搜索算法应用于动态超网络架构时实现质量提升。Abstract
Deep learning is increasingly impacting various aspects of contemporary society. Artificial neural networks have emerged as the dominant models for solving an expanding range of tasks. The introduction of Neural Architecture Search (NAS) techniques, which enable the automatic design of task-optimal networks, has led to remarkable advances. However, the NAS process is typically associated with long execution times and significant computational resource requirements. Once-For-All (OFA) and its successor, Once-For-All-2 (OFAv2), have been developed to mitigate these challenges. While maintaining exceptional performance and eliminating the need for retraining, they aim to build a single super-network model capable of directly extracting sub-networks satisfying different constraints. Neural Architecture Transfer (NAT) was developed to maximise the effectiveness of extracting sub-networks from a super-network. In this paper, we present NATv2, an extension of NAT that improves multi-objective search algorithms applied to dynamic super-network architectures. NATv2 achieves qualitative improvements in the extractable sub-networks by exploiting the improved super-networks generated by OFAv2 and incorporating new policies for initialisation, pre-processing and updating its networks archive. In addition, a post-processing pipeline based on fine-tuning is introduced. Experimental results show that NATv2 successfully improves NAT and is highly recommended for investigating high-performance architectures with a minimal number of parameters.
摘要
深度学习在当代社会中越来越有影响。人工神经网络已经成为解决越来越多任务的主导模型。 introduce Neural Architecture Search（NAS）技术，它可以自动设计适应任务的网络，导致了非常的进步。然而，NAS过程通常具有较长的执行时间和较大的计算资源需求。Once-For-All（OFA）和其 successor Once-For-All-2（OFAv2）已经开发出来了，以解决这些挑战。它们希望建立一个单一的超网络模型，可以直接提取满足不同约束的子网络。Neural Architecture Transfer（NAT）被开发出来，以 maximize the effectiveness of extracting sub-networks from a super-network。在这篇论文中，我们提出NATv2，它是NAT的扩展，通过在OFAv2生成的改进的超网络和新的初始化、预处理和更新网络archive的策略来提高可提取的子网络质量。此外，我们还提出了一个基于练习的后处理管道。实验结果表明，NATv2成功地提高了NAT，并且在具有最小参数数量的情况下提供了高性能的建议。
Learning Difference Equations with Structured Grammatical Evolution for Postprandial Glycaemia Prediction
results: 该方法可以提供安全且准确的预测结果，而无需放弃可解释性。与其他方法相比，该方法在准确性和可解释性之间协调得更好，提供了一个有前途的方法 для血糖预测。Abstract
People with diabetes must carefully monitor their blood glucose levels, especially after eating. Blood glucose regulation requires a proper combination of food intake and insulin boluses. Glucose prediction is vital to avoid dangerous post-meal complications in treating individuals with diabetes. Although traditional methods, such as artificial neural networks, have shown high accuracy rates, sometimes they are not suitable for developing personalised treatments by physicians due to their lack of interpretability. In this study, we propose a novel glucose prediction method emphasising interpretability: Interpretable Sparse Identification by Grammatical Evolution. Combined with a previous clustering stage, our approach provides finite difference equations to predict postprandial glucose levels up to two hours after meals. We divide the dataset into four-hour segments and perform clustering based on blood glucose values for the twohour window before the meal. Prediction models are trained for each cluster for the two-hour windows after meals, allowing predictions in 15-minute steps, yielding up to eight predictions at different time horizons. Prediction safety was evaluated based on Parkes Error Grid regions. Our technique produces safe predictions through explainable expressions, avoiding zones D (0.2% average) and E (0%) and reducing predictions on zone C (6.2%). In addition, our proposal has slightly better accuracy than other techniques, including sparse identification of non-linear dynamics and artificial neural networks. The results demonstrate that our proposal provides interpretable solutions without sacrificing prediction accuracy, offering a promising approach to glucose prediction in diabetes management that balances accuracy, interpretability, and computational efficiency.
摘要
人们有糖尿病必须仔细监测血糖水平，特别是 после吃食。血糖补做需要合适的食物摄取和人工胰岛素注射。预测血糖水平是对治疗糖尿病患者的生命关键。传统方法，如人工神经网络，已经显示高准确率，但是由于其解释性不足，不适用于个性化治疗。在本研究中，我们提出了一种新的血糖预测方法，强调解释性：可解释的稀缺特征识别。结合之前的划分阶段，我们的方法提供了finite difference方程来预测午餐后血糖水平。我们将数据分成四个时间段，并基于吃食前两个小时的血糖值进行划分。预测模型在每个群中训练，以预测午餐后两个小时内的血糖水平，每步预测15分钟，共八个预测。预测安全性评估基于公钵环境区域。我们的方法生成安全的预测，避免了区域D（0.2%的平均值）和区域E（0%），并减少了区域C（6.2%）。此外，我们的提议的准确性略高于其他技术，包括稀缺特征识别非线性动力学和人工神经网络。结果表明，我们的提议可以提供可解释的解决方案，不 sacrificing预测准确性，为糖尿病管理提供了可能的平衡。
Dynamical Graph Echo State Networks with Snapshot Merging for Dissemination Process Classification
results: 实验结果显示，对六个 benchmark DPC 数据集进行测试，本研究的提出的模型在 classification 性能方面占据了优势，比 DynGESN 和一些基于核函数的模型更好。Abstract
The Dissemination Process Classification (DPC) is a popular application of temporal graph classification. The aim of DPC is to classify different spreading patterns of information or pestilence within a community represented by discrete-time temporal graphs. Recently, a reservoir computing-based model named Dynamical Graph Echo State Network (DynGESN) has been proposed for processing temporal graphs with relatively high effectiveness and low computational costs. In this study, we propose a novel model which combines a novel data augmentation strategy called snapshot merging with the DynGESN for dealing with DPC tasks. In our model, the snapshot merging strategy is designed for forming new snapshots by merging neighboring snapshots over time, and then multiple reservoir encoders are set for capturing spatiotemporal features from merged snapshots. After those, the logistic regression is adopted for decoding the sum-pooled embeddings into the classification results. Experimental results on six benchmark DPC datasets show that our proposed model has better classification performances than the DynGESN and several kernel-based models.
摘要
《信息或疫病传播过程分类（DPC）应用》是一个广泛使用的时间图分类应用。DPC的目标是根据社区的时间图来分类不同的信息或疫病传播模式。近些年，一种基于储存计算机（reservoir computing）的模型named Dynamical Graph Echo State Network（DynGESN）已经被提出来处理时间图。在本研究中，我们提出了一种新的模型，该模型将Snapshot Merging策略与DynGESN相结合，用于处理DPC任务。在我们的模型中，Snapshot Merging策略是用于在时间上邻居图像合并，并将多个储存编码器用于捕捉时间图的空间特征。然后，逻辑回归被采用来将汇聚编码器的输出编码为分类结果。在六个标准DPC数据集上进行实验，我们的提出的模型的分类性能比DynGESN和一些核心基于模型更好。
Rockmate: an Efficient, Fast, Automatic and Generic Tool for Re-materialization in PyTorch
methods: Rockmate 使用一种自动检测模型的 Computational 和 Data 相依性结构，并将其转换为一系列复杂的封页，以控制记忆需求。
results: 经过实验显示，Rockmate 能够与 Checkmate 和 Rotor 相比，具有相似的速度和效率，并且可以在许多模型中获得较低的记忆需求（具体比例为 2-5），仅带来一定的开销（约 10%-20%）。Abstract
We propose Rockmate to control the memory requirements when training PyTorch DNN models. Rockmate is an automatic tool that starts from the model code and generates an equivalent model, using a predefined amount of memory for activations, at the cost of a few re-computations. Rockmate automatically detects the structure of computational and data dependencies and rewrites the initial model as a sequence of complex blocks. We show that such a structure is widespread and can be found in many models in the literature (Transformer based models, ResNet, RegNets,...). This structure allows us to solve the problem in a fast and efficient way, using an adaptation of Checkmate (too slow on the whole model but general) at the level of individual blocks and an adaptation of Rotor (fast but limited to sequential models) at the level of the sequence itself. We show through experiments on many models that Rockmate is as fast as Rotor and as efficient as Checkmate, and that it allows in many cases to obtain a significantly lower memory consumption for activations (by a factor of 2 to 5) for a rather negligible overhead (of the order of 10% to 20%). Rockmate is open source and available at https://github.com/topal-team/rockmate.
摘要
OpenAPMax: Abnormal Patterns-based Model for Real-World Alzheimer’s Disease Diagnosis
for: The paper aims to address the challenges of Alzheimer’s disease (AD) diagnosis in real-world settings, particularly the open-set recognition problem where the known categories are not fixed and can change over time.
methods: The proposed method, OpenAPMax, uses an anomaly pattern-based approach to model the distance between each patient’s abnormal pattern and the center of their category, and modifies the classification probability using extreme value theory (EVT).
results: The proposed method achieves state-of-the-art results in open-set recognition, outperforming recent open-set recognition methods.Here’s the Simplified Chinese text format for the three key points:
for: 本文旨在解决阿尔ц海默病（AD）诊断在实际设置下的挑战，特别是开集识别问题，where the known categories are not fixed and can change over time.
results: 提议方法达到开集识别领域的州先Result，超越最近的开集识别方法。I hope this helps!Abstract
Alzheimer's disease (AD) cannot be reversed, but early diagnosis will significantly benefit patients' medical treatment and care. In recent works, AD diagnosis has the primary assumption that all categories are known a prior -- a closed-set classification problem, which contrasts with the open-set recognition problem. This assumption hinders the application of the model in natural clinical settings. Although many open-set recognition technologies have been proposed in other fields, they are challenging to use for AD diagnosis directly since 1) AD is a degenerative disease of the nervous system with similar symptoms at each stage, and it is difficult to distinguish from its pre-state, and 2) diversified strategies for AD diagnosis are challenging to model uniformly. In this work, inspired by the concerns of clinicians during diagnosis, we propose an open-set recognition model, OpenAPMax, based on the anomaly pattern to address AD diagnosis in real-world settings. OpenAPMax first obtains the abnormal pattern of each patient relative to each known category through statistics or a literature search, clusters the patients' abnormal pattern, and finally, uses extreme value theory (EVT) to model the distance between each patient's abnormal pattern and the center of their category and modify the classification probability. We evaluate the performance of the proposed method with recent open-set recognition, where we obtain state-of-the-art results.
摘要
阿尔茨heimer病 (AD) 不能 reversed，但早期诊断将有利于患者的医疗和护理。在最近的研究中，AD 诊断假设所有类别都是已知的，一个closed-set classification problem，与开放式认知问题不同。这种假设限制了模型在自然临床设置中的应用。虽然许多开放式认知技术在其他领域已经提出，但它们在AD诊断直接应用很困难，因为1）AD 是 nervious system 的逐步衰变病，表现类似，难以与其预状分 distinguish，2）AD 诊断策略多样化难以统一模型。在这种情况下，我们以临床医生在诊断过程中的关注为导向，提出一种开放式认知模型，OpenAPMax，基于异常模式来解决AD诊断在实际设置中。OpenAPMax 首先通过统计或文献搜索获取每个患者的异常模式，对每个患者的异常模式进行归类，最后使用极值理论 (EVT) 来模型每个患者的异常模式与其分类概率的距离。我们对最近的开放式认知方法进行评估，并获得了状态机器人的结果。
Learning Differentiable Logic Programs for Abstract Visual Reasoning
paper_authors: Hikaru Shindo, Viktor Pfanschilling, Devendra Singh Dhami, Kristian Kersting for:The paper is written for building intelligent agents that can perform visual reasoning and solve problem-solving tasks beyond perception.methods:The paper proposes a graph-based differentiable forward reasoner called NEUMANN, which passes messages in a memory-efficient manner and handles structured programs with functors. Additionally, the paper proposes a computationally-efficient structure learning algorithm for explanatory program induction on complex visual scenes.results:The paper demonstrates that NEUMANN outperforms neural, symbolic, and neuro-symbolic baselines in visual reasoning tasks, including a new task called “visual reasoning behind-the-scenes” that requires agents to learn abstract programs and answer queries by imagining scenes that are not observed.Abstract
Visual reasoning is essential for building intelligent agents that understand the world and perform problem-solving beyond perception. Differentiable forward reasoning has been developed to integrate reasoning with gradient-based machine learning paradigms. However, due to the memory intensity, most existing approaches do not bring the best of the expressivity of first-order logic, excluding a crucial ability to solve abstract visual reasoning, where agents need to perform reasoning by using analogies on abstract concepts in different scenarios. To overcome this problem, we propose NEUro-symbolic Message-pAssiNg reasoNer (NEUMANN), which is a graph-based differentiable forward reasoner, passing messages in a memory-efficient manner and handling structured programs with functors. Moreover, we propose a computationally-efficient structure learning algorithm to perform explanatory program induction on complex visual scenes. To evaluate, in addition to conventional visual reasoning tasks, we propose a new task, visual reasoning behind-the-scenes, where agents need to learn abstract programs and then answer queries by imagining scenes that are not observed. We empirically demonstrate that NEUMANN solves visual reasoning tasks efficiently, outperforming neural, symbolic, and neuro-symbolic baselines.
摘要
Visual reasoning 是建立智能代理的关键，它允许代理人理解世界并解决问题，超出感知。difficult forward reasoning 已经开发来整合reasoning 与梯度基于机器学习 парадигмы。然而，由于内存投入，大多数现有方法不能充分发挥逻辑表达力，排除了一个关键的能力：解决抽象视觉逻辑，代理人需要通过对抽象概念的比较来解决问题。为了解决这个问题，我们提出了 NEUro-symbolic Message-pAssiNg reasoNer (NEUMANN)，它是一个图形基的difficult forward reasoner，通过Message passing 来减少内存占用，并处理结构化程序。此外，我们还提出了一种 Computationally-efficient 的结构学习算法，用于在复杂视觉场景中进行解释性程序induction。为了评估 NEUMANN 的性能，我们提出了一个新任务：视觉逻辑后台，代理人需要学习抽象程序，然后回答问题，想象出未见的场景。我们的实验结果表明，NEUMANN 可以高效解决视觉逻辑任务，超过 neural、symbolic 和 neuro-symbolic 基elines。
results: 在四个多样化的数据集上，经验表明 SMVCF 模型在多视图归一化任务中显著提高表示性和准确率Abstract
Concept Factorization (CF), as a novel paradigm of representation learning, has demonstrated superior performance in multi-view clustering tasks. It overcomes limitations such as the non-negativity constraint imposed by traditional matrix factorization methods and leverages kernel methods to learn latent representations that capture the underlying structure of the data, thereby improving data representation. However, existing multi-view concept factorization methods fail to consider the limited labeled information inherent in real-world multi-view data. This often leads to significant performance loss. To overcome these limitations, we propose a novel semi-supervised multi-view concept factorization model, named SMVCF. In the SMVCF model, we first extend the conventional single-view CF to a multi-view version, enabling more effective exploration of complementary information across multiple views. We then integrate multi-view CF, label propagation, and manifold learning into a unified framework to leverage and incorporate valuable information present in the data. Additionally, an adaptive weight vector is introduced to balance the importance of different views in the clustering process. We further develop targeted optimization methods specifically tailored for the SMVCF model. Finally, we conduct extensive experiments on four diverse datasets with varying label ratios to evaluate the performance of SMVCF. The experimental results demonstrate the effectiveness and superiority of our proposed approach in multi-view clustering tasks.
摘要
《概念分解（CF）》是一种新的表示学习 paradigm，在多视图划分任务中表现出了更高的性能。它超越了传统的矩阵分解方法中的非正式约束，并利用核函数方法学习 latent 表示，以捕捉数据下面的结构，从而改善数据表示。然而，现有的多视图概念分解方法通常不考虑实际世界中多视图数据中的有限 labels 信息。这经常导致显著的性能损失。为了解决这些限制，我们提出了一种新的半upervised多视图概念分解模型，名为 SMVCF。在 SMVCF 模型中，我们首先将传统的单视图 CF 扩展到多视图版本，以更好地利用多个视图之间的补做信息。然后，我们将多视图 CF、标签传播和抽象学习集成到一个统一框架中，以利用数据中存在的有价值信息。此外，我们还引入了一个 adaptive веctor 来衡量不同视图在划分过程中的重要性。最后，我们开发了特定于 SMVCF 模型的目标优化方法。我们进行了对四个多样化的数据集进行了广泛的实验，以评估 SMVCF 模型在多视图划分任务中的性能。实验结果表明，我们提出的方法在多视图划分任务中表现出了更高的有效性和优势。
Achieving Stable Training of Reinforcement Learning Agents in Bimodal Environments through Batch Learning
results: 比较 typically 更新和批处理学习agent，批处理学习agent更高效、更具抗随机环境能力Abstract
Bimodal, stochastic environments present a challenge to typical Reinforcement Learning problems. This problem is one that is surprisingly common in real world applications, being particularly applicable to pricing problems. In this paper we present a novel learning approach to the tabular Q-learning algorithm, tailored to tackling these specific challenges by using batch updates. A simulation of pricing problem is used as a testbed to compare a typically updated agent with a batch learning agent. The batch learning agents are shown to be both more effective than the typically-trained agents, and to be more resilient to the fluctuations in a large stochastic environment. This work has a significant potential to enable practical, industrial deployment of Reinforcement Learning in the context of pricing and others.
摘要
bisimodal, 随机环境会对传统的奖励学习问题提出挑战。这种问题在实际应用中很普遍，特别适用于价格问题。在这篇论文中，我们提出了一种新的学习方法，用于修改标准的Q学习算法，以适应这些特定挑战。我们使用了批处理更新来适应这种随机环境。我们通过对比一个通常更新的代理和批处理学习代理的测试，显示了批处理学习代理在大型随机环境中更加有效和更加鲁棒。这项工作具有实用化奖励学习在价格和其他领域的潜在应用潜力。
Quantum Machine Learning on Near-Term Quantum Devices: Current State of Supervised and Unsupervised Techniques for Real-World Applications
results: 本文评估了这些QML实现的性能，并与其经典对手进行比较。最后，本文提出了将来缓解量子机器学习应用在真正量子硬件上的挑战。Abstract
The past decade has seen considerable progress in quantum hardware in terms of the speed, number of qubits and quantum volume which is defined as the maximum size of a quantum circuit that can be effectively implemented on a near-term quantum device. Consequently, there has also been a rise in the number of works based on the applications of Quantum Machine Learning (QML) on real hardware to attain quantum advantage over their classical counterparts. In this survey, our primary focus is on selected supervised and unsupervised learning applications implemented on quantum hardware, specifically targeting real-world scenarios. Our survey explores and highlights the current limitations of QML implementations on quantum hardware. We delve into various techniques to overcome these limitations, such as encoding techniques, ansatz structure, error mitigation, and gradient methods. Additionally, we assess the performance of these QML implementations in comparison to their classical counterparts. Finally, we conclude our survey with a discussion on the existing bottlenecks associated with applying QML on real quantum devices and propose potential solutions for overcoming these challenges in the future.
摘要
过去一个十年，量子硬件在速度、量子比特数和量子体积方面有所进步，量子机器学习（QML）在真实硬件上实现的应用工作也有所增加。在这份报告中，我们主要关注选择性supervised和无监督学习应用程序在量子硬件上的实现，特别是面向实际场景。我们的报告探讨和强调当前量子硬件QML实现的限制，包括编码技术、架构结构、错误缓冲和梯度方法。此外，我们评估这些QML实现与其经典对手的性能。最后，我们结束报告，讨论现有量子硬件上应用QML的瓶颈，并提出未来缓解这些挑战的可能性。
Enhancing the Robustness of QMIX against State-adversarial Attacks
results: 通过对QMIX算法进行训练和测试，研究发现这些方法可以提高MARL算法的Robustness，使其能够更好地抵抗状态恶化攻击。Abstract
Deep reinforcement learning (DRL) performance is generally impacted by state-adversarial attacks, a perturbation applied to an agent's observation. Most recent research has concentrated on robust single-agent reinforcement learning (SARL) algorithms against state-adversarial attacks. Still, there has yet to be much work on robust multi-agent reinforcement learning. Using QMIX, one of the popular cooperative multi-agent reinforcement algorithms, as an example, we discuss four techniques to improve the robustness of SARL algorithms and extend them to multi-agent scenarios. To increase the robustness of multi-agent reinforcement learning (MARL) algorithms, we train models using a variety of attacks in this research. We then test the models taught using the other attacks by subjecting them to the corresponding attacks throughout the training phase. In this way, we organize and summarize techniques for enhancing robustness when used with MARL.
摘要
深度强化学习（DRL）性能通常受到状态敌意攻击的影响。最近的研究主要集中在对单机器学习（SARL）算法进行鲁棒性加固。然而，对多机器学习（MARL）算法的鲁棒性加固还很少研究。使用QMIX算法作为例子，本文讨论了对SARL算法进行四种技术提升鲁棒性的方法，并将其推广到多机器场景。为提高MARL算法的鲁棒性，我们在训练过程中使用多种攻击。然后，我们在训练阶段对模型进行测试，以验证它们是否能够抵抗对应的攻击。这样，我们可以系统地整理和总结提高MARL算法的鲁棒性技术。
Fixing confirmation bias in feature attribution methods via semantic match
paper_authors: Giovanni Cinà, Daniel Fernandez-Llaneza, Nishant Mishra, Tabea E. Röber, Sandro Pezzelle, Iacer Calixto, Rob Goedhart, Ş. İlker Birbil
for: This paper aims to address the issue of confirmation bias in feature attribution methods for black box models, and to propose a structured approach to evaluate the semantic match between human concepts and the model’s explanations.
methods: The paper proposes a new approach called “semantic match” to evaluate the alignment between human concepts and the feature attributions generated by the model. This approach is based on a conceptual framework put forward in Cin`a et al. (2023).
results: The paper presents a suite of experiments using both tabular and image data to demonstrate the effectiveness of the proposed approach in identifying both desirable and undesirable model behaviors. The results show that the assessment of semantic match can provide valuable insights into the model’s internal representations and help to resolve the issue of confirmation bias in XAI.Abstract
Feature attribution methods have become a staple method to disentangle the complex behavior of black box models. Despite their success, some scholars have argued that such methods suffer from a serious flaw: they do not allow a reliable interpretation in terms of human concepts. Simply put, visualizing an array of feature contributions is not enough for humans to conclude something about a model's internal representations, and confirmation bias can trick users into false beliefs about model behavior. We argue that a structured approach is required to test whether our hypotheses on the model are confirmed by the feature attributions. This is what we call the "semantic match" between human concepts and (sub-symbolic) explanations. Building on the conceptual framework put forward in Cin\`a et al. [2023], we propose a structured approach to evaluate semantic match in practice. We showcase the procedure in a suite of experiments spanning tabular and image data, and show how the assessment of semantic match can give insight into both desirable (e.g., focusing on an object relevant for prediction) and undesirable model behaviors (e.g., focusing on a spurious correlation). We couple our experimental results with an analysis on the metrics to measure semantic match, and argue that this approach constitutes the first step towards resolving the issue of confirmation bias in XAI.
摘要
feature 归因方法已成为黑盒模型行为解释的标准方法。 despite their success, some scholars have argued that such methods suffer from a serious flaw: they do not allow a reliable interpretation in terms of human concepts. simply put, visualizing an array of feature contributions is not enough for humans to conclude something about a model's internal representations, and confirmation bias can trick users into false beliefs about model behavior. we argue that a structured approach is required to test whether our hypotheses on the model are confirmed by the feature attributions. this is what we call the "semantic match" between human concepts and (sub-symbolic) explanations. building on the conceptual framework put forward in Cin\`a et al. [2023], we propose a structured approach to evaluate semantic match in practice. we showcase the procedure in a suite of experiments spanning tabular and image data, and show how the assessment of semantic match can give insight into both desirable (e.g., focusing on an object relevant for prediction) and undesirable model behaviors (e.g., focusing on a spurious correlation). we couple our experimental results with an analysis on the metrics to measure semantic match, and argue that this approach constitutes the first step towards resolving the issue of confirmation bias in XAI.
Internet of Things Fault Detection and Classification via Multitask Learning
results: SMTCNN方法在实际数据上达到了3.5%的特异性，并显著提高了精度、回归率和F1评价指标。Abstract
This paper presents a comprehensive investigation into developing a fault detection and classification system for real-world IIoT applications. The study addresses challenges in data collection, annotation, algorithm development, and deployment. Using a real-world IIoT system, three phases of data collection simulate 11 predefined fault categories. We propose SMTCNN for fault detection and category classification in IIoT, evaluating its performance on real-world data. SMTCNN achieves superior specificity (3.5%) and shows significant improvements in precision, recall, and F1 measures compared to existing techniques.
摘要
Fraunhofer SIT at CheckThat! 2023: Tackling Classification Uncertainty Using Model Souping on the Example of Check-Worthiness Classification
results: 在英文数据集上，我们的提交模型达到了总F1分数0.878，在竞赛中排名第二。Abstract
This paper describes the second-placed approach developed by the Fraunhofer SIT team in the CLEF-2023 CheckThat! lab Task 1B for English. Given a text snippet from a political debate, the aim of this task is to determine whether it should be assessed for check-worthiness. Detecting check-worthy statements aims to facilitate manual fact-checking efforts by prioritizing the claims that fact-checkers should consider first. It can also be considered as primary step of a fact-checking system. Our best-performing method took advantage of an ensemble classification scheme centered on Model Souping. When applied to the English data set, our submitted model achieved an overall F1 score of 0.878 and was ranked as the second-best model in the competition.
摘要
Translation notes:* "Check-worthiness" is 可验证性 (kě yàn zhèng xìng) in Simplified Chinese.* "Model Souping" is 模型汤 (molduō tāng) in Simplified Chinese, which is a play on words combining "model" and "soup" to refer to the ensemble of models used in the approach.* "F1 score" is 平均准确率 (píng jiān zhèng qiáng lǐ) in Simplified Chinese.
Unbiased Pain Assessment through Wearables and EHR Data: Multi-attribute Fairness Loss-based CNN Approach
results: 研究结果显示，对于不同群体的痛症评估，提出的MAFL模型能够优化精度和公平性的贡献，并且与现有的 Mitigation 方法相比，表现较好。使用NIH All-Of-US 数据，研究范例包括868名受试者，收集了1500天的数据，以分析提议的公平痛症评估系统。Abstract
The combination of diverse health data (IoT, EHR, and clinical surveys) and scalable-adaptable Artificial Intelligence (AI), has enabled the discovery of physical, behavioral, and psycho-social indicators of pain status. Despite the hype and promise to fundamentally alter the healthcare system with technological advancements, much AI adoption in clinical pain evaluation has been hampered by the heterogeneity of the problem itself and other challenges, such as personalization and fairness. Studies have revealed that many AI (i.e., machine learning or deep learning) models display biases and discriminate against specific population segments (such as those based on gender or ethnicity), which breeds skepticism among medical professionals about AI adaptability. In this paper, we propose a Multi-attribute Fairness Loss (MAFL) based CNN model that aims to account for any sensitive attributes included in the data and fairly predict patients' pain status while attempting to minimize the discrepancies between privileged and unprivileged groups. In order to determine whether the trade-off between accuracy and fairness can be satisfied, we compare the proposed model with well-known existing mitigation procedures, and studies reveal that the implemented model performs favorably in contrast to state-of-the-art methods. Utilizing NIH All-Of-US data, where a cohort of 868 distinct individuals with wearables and EHR data gathered over 1500 days has been taken into consideration to analyze our suggested fair pain assessment system.
摘要
“由多元健康数据（IoT、EHR和临床调查）和可扩展适应的人工智能（AI）的结合，已经发现了体征、行为和心理社会指标。尽管技术进步对健康领域的改革具有广泛的推广和承认，但AI在临床痛评估中的采纳受到了多重因素的影响，例如个人化和公平性。研究发现，许多AI（例如机器学习或深度学习）模型会带有偏见和歧视特定人群（例如根据性别或民族），这产生了医疗专业人员对AI适应性的怀疑。在这篇文章中，我们提出了基于多Attribute Fairness Loss（MAFL）的弹性神经网络模型，以减少权利层次中的差异。为了决定是否可以满足精确性和公平性之间的贸易，我们与已知的 Mitigation 程序进行比较，研究发现，我们的提案模型在与州域方法进行比较时表现较好。使用NIH All-Of-US数据，我们分析了我们建议的公平痛评估系统。”
Exploring the Multi-modal Demand Dynamics During Transport System Disruptions
results: 本研究提供了一种简单的工具，可以根据不同的紊乱enario分类不同类型的乘客反应，以及估算不同紊乱enario下的模式转移范围。Abstract
Various forms of disruption in transport systems perturb urban mobility in different ways. Passengers respond heterogeneously to such disruptive events based on numerous factors. This study takes a data-driven approach to explore multi-modal demand dynamics under disruptions. We first develop a methodology to automatically detect anomalous instances through historical hourly travel demand data. Then we apply clustering to these anomalous hours to distinguish various forms of multi-modal demand dynamics occurring during disruptions. Our study provides a straightforward tool for categorising various passenger responses to disruptive events in terms of mode choice and paves the way for predictive analyses on estimating the scope of modal shift under distinct disruption scenarios.
摘要
不同的交通系统紊乱会对城市流动性产生不同的影响。乘客对这些紊乱事件的回应也是多iform的，这是基于许多因素。这项研究采用数据驱动的方法来探索各种多Modal的需求动力学在紊乱情况下。我们首先开发了一种自动检测历史小时旅行需求数据中异常情况的方法。然后我们应用分 clustering 这些异常小时，以分辨不同的多Modal需求动力学在紊乱情况下发生的形式。我们的研究提供了一种简单的工具，可以根据乘客对紊乱事件的回应来分类不同的交通模式，并且为估计不同紊乱情况下的模式转换范围做出预测分析。
results: state-of-the-art performance on unconstrained and constrained datasets, speech samples available onlineHere’s the full text in Simplified Chinese:for: lip-to-speech synthesismethods: non-autoregressive sequence-to-sequence architecture, 自成分化 speech content representationresults: state-of-the-art performance on unconstrained和 constrained datasets, speech samples available onlineAbstract
Significant progress has been made in speaker dependent Lip-to-Speech synthesis, which aims to generate speech from silent videos of talking faces. Current state-of-the-art approaches primarily employ non-autoregressive sequence-to-sequence architectures to directly predict mel-spectrograms or audio waveforms from lip representations. We hypothesize that the direct mel-prediction hampers training/model efficiency due to the entanglement of speech content with ambient information and speaker characteristics. To this end, we propose RobustL2S, a modularized framework for Lip-to-Speech synthesis. First, a non-autoregressive sequence-to-sequence model maps self-supervised visual features to a representation of disentangled speech content. A vocoder then converts the speech features into raw waveforms. Extensive evaluations confirm the effectiveness of our setup, achieving state-of-the-art performance on the unconstrained Lip2Wav dataset and the constrained GRID and TCD-TIMIT datasets. Speech samples from RobustL2S can be found at https://neha-sherin.github.io/RobustL2S/
摘要
significan progress has been made in speaker-dependent Lip-to-Speech synthesis, which aims to generate speech from silent videos of talking faces. current state-of-the-art approaches primarily employ non-autoregressive sequence-to-sequence architectures to directly predict mel-spectrograms or audio waveforms from lip representations. we hypothesize that the direct mel-prediction hampers training/model efficiency due to the entanglement of speech content with ambient information and speaker characteristics. to this end, we propose RobustL2S, a modularized framework for Lip-to-Speech synthesis. first, a non-autoregressive sequence-to-sequence model maps self-supervised visual features to a representation of disentangled speech content. a vocoder then converts the speech features into raw waveforms. extensive evaluations confirm the effectiveness of our setup, achieving state-of-the-art performance on the unconstrained Lip2Wav dataset and the constrained GRID and TCD-TIMIT datasets. speech samples from RobustL2S can be found at https://neha-sherin.github.io/RobustL2S/Here's the word-for-word translation: significan进步有所作出在 speaker-dependent Lip-to-Speech合成中，目标是从无声视频中提取讲话的语音。 current state-of-the-art Approaches primarily employ non-autoregressive sequence-to-sequence architectures to directly predict mel-spectrograms or audio waveforms from lip representations. we hypothesize that the direct mel-prediction hampers training/model efficiency due to the entanglement of speech content with ambient information and speaker characteristics. to this end, we propose RobustL2S, a modularized framework for Lip-to-Speech synthesis. first, a non-autoregressive sequence-to-sequence model maps self-supervised visual features to a representation of disentangled speech content. a vocoder then converts the speech features into raw waveforms. extensive evaluations confirm the effectiveness of our setup, achieving state-of-the-art performance on the unconstrained Lip2Wav dataset and the constrained GRID and TCD-TIMIT datasets. speech samples from RobustL2S can be found at https://neha-sherin.github.io/RobustL2S/
MADS: Modulated Auto-Decoding SIREN for time series imputation
paper_authors: Tom Bamford, Elizabeth Fons, Yousef El-Laham, Svitlana Vyetrenko
for: 这 paper 是为了解决时间序列填充问题而写的，时间序列填充是许多领域中的一个重要挑战，因为时间序列数据的类型可能会具有很大的变化。
methods: 这 paper 使用了深度学习技术，特别是 SIRENs 和 hypernetwork 架构，来解决时间序列填充问题。
results: 这 paper 在两个实际数据集上进行了评估，并证明了它在时间序列填充方面的表现比之前的方法更好，在人活动数据集上提高了填充性能的至少 40%，而在空气质量数据集上与其他基elines一样。在synthetic数据上进行评估时，我们的模型在不同数据集配置下的平均排名得到了所有基elines的最好成绩。Abstract
Time series imputation remains a significant challenge across many fields due to the potentially significant variability in the type of data being modelled. Whilst traditional imputation methods often impose strong assumptions on the underlying data generation process, limiting their applicability, researchers have recently begun to investigate the potential of deep learning for this task, inspired by the strong performance shown by these models in both classification and regression problems across a range of applications. In this work we propose MADS, a novel auto-decoding framework for time series imputation, built upon implicit neural representations. Our method leverages the capabilities of SIRENs for high fidelity reconstruction of signals and irregular data, and combines it with a hypernetwork architecture which allows us to generalise by learning a prior over the space of time series. We evaluate our model on two real-world datasets, and show that it outperforms state-of-the-art methods for time series imputation. On the human activity dataset, it improves imputation performance by at least 40%, while on the air quality dataset it is shown to be competitive across all metrics. When evaluated on synthetic data, our model results in the best average rank across different dataset configurations over all baselines.
摘要
时间序列填充仍然是许多领域中的主要挑战，因为可能存在严重的数据类型变化，导致传统填充方法的适用有限。然而，研究人员最近开始研究使用深度学习来解决这个问题，因为深度学习模型在各种应用中的类型预测和回归问题中表现出色。在这种工作中，我们提出了MADS，一种新的自动解码框架 для时间序列填充，基于含义表示。我们的方法利用SIRENs高精度重建信号和不规则数据的能力，并将其与一个权重网络架构相结合，以学习时间序列的先验知识。我们对两个实际数据集进行评估，并显示了MADS在时间序列填充方面的超过状态艺术方法的表现。在人活动数据集上，它提高填充性能至少40%，而在空气质量数据集上，它与所有指标中竞争。当对synthetic数据进行评估时，我们的模型在不同数据集配置下的平均排名最高。
A Survey on Graph Classification and Link Prediction based on GNN
results: 本文通过对node classification、graph classification和link prediction等任务的应用，阐述了图 convolutional neural networks 在不同任务中的应用和效果。Abstract
Traditional convolutional neural networks are limited to handling Euclidean space data, overlooking the vast realm of real-life scenarios represented as graph data, including transportation networks, social networks, and reference networks. The pivotal step in transferring convolutional neural networks to graph data analysis and processing lies in the construction of graph convolutional operators and graph pooling operators. This comprehensive review article delves into the world of graph convolutional neural networks. Firstly, it elaborates on the fundamentals of graph convolutional neural networks. Subsequently, it elucidates the graph neural network models based on attention mechanisms and autoencoders, summarizing their application in node classification, graph classification, and link prediction along with the associated datasets.
摘要
传统的卷积神经网络只能处理几何空间数据，忽略了现实生活中的各种图数据，包括交通网络、社交网络和引用网络。图 convolutional 算子和图 Pooling 算子的建构是将卷积神经网络传输到图数据分析和处理的关键步骤。本综述文章将介绍图 convolutional 神经网络的基础知识，然后详细介绍基于注意机制和自编码器的图神经网络模型，包括节点分类、图分类和链接预测，以及相关的数据集。
Thompson Sampling under Bernoulli Rewards with Local Differential Privacy
results: 论文通过 simulate 来示例了不同隐私预算下不同机制的凝结行为。Abstract
This paper investigates the problem of regret minimization for multi-armed bandit (MAB) problems with local differential privacy (LDP) guarantee. Given a fixed privacy budget $\epsilon$, we consider three privatizing mechanisms under Bernoulli scenario: linear, quadratic and exponential mechanisms. Under each mechanism, we derive stochastic regret bound for Thompson Sampling algorithm. Finally, we simulate to illustrate the convergence of different mechanisms under different privacy budgets.
摘要
这个论文研究了多重投机(MAB)问题中的 regret最小化问题，同时保证了本地差分隐私(LDP)的 garantía。给定一个固定的隐私预算$\epsilon$,我们考虑了三种隐私机制在 Bernoulli 场景下：线性机制、квадратиче机制和指数机制。对于每种机制，我们 derivated sto的抽象 regret bound for Thompson Sampling 算法。最后，我们使用 Simulation 来示出不同隐私预算下不同机制的连续性。Note that the translation is in Simplified Chinese, which is the standard writing system used in mainland China. If you prefer Traditional Chinese, please let me know and I can provide the translation in that format as well.
CardiGraphormer: Unveiling the Power of Self-Supervised Learning in Revolutionizing Drug Discovery
results: CardiGraphormer可以对药物结构进行学习，并将其映射到更加强大的预测性和解释性。它可以处理复杂的药物数据，并可以进行节点、双节点、子graph和整个图结构的处理。Abstract
In the expansive realm of drug discovery, with approximately 15,000 known drugs and only around 4,200 approved, the combinatorial nature of the chemical space presents a formidable challenge. While Artificial Intelligence (AI) has emerged as a powerful ally, traditional AI frameworks face significant hurdles. This manuscript introduces CardiGraphormer, a groundbreaking approach that synergizes self-supervised learning (SSL), Graph Neural Networks (GNNs), and Cardinality Preserving Attention to revolutionize drug discovery. CardiGraphormer, a novel combination of Graphormer and Cardinality Preserving Attention, leverages SSL to learn potent molecular representations and employs GNNs to extract molecular fingerprints, enhancing predictive performance and interpretability while reducing computation time. It excels in handling complex data like molecular structures and performs tasks associated with nodes, pairs of nodes, subgraphs, or entire graph structures. CardiGraphormer's potential applications in drug discovery and drug interactions are vast, from identifying new drug targets to predicting drug-to-drug interactions and enabling novel drug discovery. This innovative approach provides an AI-enhanced methodology in drug development, utilizing SSL combined with GNNs to overcome existing limitations and pave the way for a richer exploration of the vast combinatorial chemical space in drug discovery.
摘要
在药物发现领域中，有约15,000种已知药物和只有约4,200个得到批准，化学空间的 combinatorial 特性呈现出巨大的挑战。而人工智能（AI）已经出现为一种强大的同伴，但传统的 AI 框架又存在许多障碍。这篇文章介绍 CardiGraphormer，一种创新的方法，它将自我supervised learning（SSL）、Graph Neural Networks（GNNs）和Cardinality Preserving Attention（CPA）相互融合，以推动药物发现领域的革命。CardiGraphormer 是 Graphormer 和 CPA 的新的组合体，通过 SSL 学习强大的分子表示，并使用 GNNs 提取分子指纹，提高预测性能和可读性，同时减少计算时间。它可以处理复杂的数据，如分子结构，并可以完成节点、节点对、子图和整个图结构上的任务。CardiGraphormer 在药物发现和药物互作中的应用前景广阔，从选择新的药物目标到预测药物之间的互作，以及开发新的药物探索方法。这种创新的方法为药物发展中的 AI 增强方法，通过 SSL 与 GNNs 的结合，突破现有的限制，开辟了更加丰富的化学空间的探索。
Beyond the Snapshot: Brain Tokenized Graph Transformer for Longitudinal Brain Functional Connectome Embedding
results: 在 AD 病Continuum 上的两个公共 longitudinal fMRI 数据集上，这个方法比其他 benchmark 模型出色，并且提供了极佳的解释性。Abstract
Under the framework of network-based neurodegeneration, brain functional connectome (FC)-based Graph Neural Networks (GNN) have emerged as a valuable tool for the diagnosis and prognosis of neurodegenerative diseases such as Alzheimer's disease (AD). However, these models are tailored for brain FC at a single time point instead of characterizing FC trajectory. Discerning how FC evolves with disease progression, particularly at the predementia stages such as cognitively normal individuals with amyloid deposition or individuals with mild cognitive impairment (MCI), is crucial for delineating disease spreading patterns and developing effective strategies to slow down or even halt disease advancement. In this work, we proposed the first interpretable framework for brain FC trajectory embedding with application to neurodegenerative disease diagnosis and prognosis, namely Brain Tokenized Graph Transformer (Brain TokenGT). It consists of two modules: 1) Graph Invariant and Variant Embedding (GIVE) for generation of node and spatio-temporal edge embeddings, which were tokenized for downstream processing; 2) Brain Informed Graph Transformer Readout (BIGTR) which augments previous tokens with trainable type identifiers and non-trainable node identifiers and feeds them into a standard transformer encoder to readout. We conducted extensive experiments on two public longitudinal fMRI datasets of the AD continuum for three tasks, including differentiating MCI from controls, predicting dementia conversion in MCI, and classification of amyloid positive or negative cognitively normal individuals. Based on brain FC trajectory, the proposed Brain TokenGT approach outperformed all the other benchmark models and at the same time provided excellent interpretability. The code is available at https://github.com/ZijianD/Brain-TokenGT.git
摘要
基于网络基因的脑功能连接图（FC）基于图神经网络（GNN）已成为诊断和诊断预测阿尔茨海默病（AD）等神经退化疾病的有价值工具。然而，这些模型围绕单个时间点的脑FC而构建，而不是跟踪FC的演变。了解脑FC在疾病晚期的演变，特别是在认知正常者悉射氧皮质堆积或轻度认知障碍（MCI）等阶段，是诊断疾病扩散模式和开发有效缓解疾病的关键。在这种工作中，我们提出了第一个可解释性框架 для脑FC演变 embeddings，即Brain Tokenized Graph Transformer（Brain TokenGT）。它包括两个模块：1）图固有和变异 embedding（GIVE）用于生成节点和空间时间边 embedding，这些embedding被拆分为下游处理; 2）脑指导图Transformer读取（BIGTR），它将上一个模块的输出和可训练类型标识符和非可训练节点标识符相加，并将其传输给标准Transformer编码器来读取。我们对两个公共的 longitudinal fMRI 数据集进行了广泛的实验，包括分类认知正常者和MCI、预测MCI中的诊断转化和分类悉射氧皮质堆积或正常认知正常者。基于脑FC演变，我们的Brain TokenGT方法在所有参考模型中表现出色，同时提供了优秀的可解释性。代码可以在https://github.com/ZijianD/Brain-TokenGT.git中找到。
Surgical fine-tuning for Grape Bunch Segmentation under Visual Domain Shifts
methods: 本研究使用了抢救精细调整（surgical fine-tuning）来适应葡萄图像中的视觉域变化。 selectively tuning only specific model layers可以支持预训练深度学习模型对新收集的葡萄图像进行适应，同时减少调整的参数数量。
results: 本研究表明，通过抢救精细调整，可以减少参数数量的情况下，预训练深度学习模型可以快速适应葡萄图像中的视觉域变化，并且可以提高葡萄棕榈的检测精度。Abstract
Mobile robots will play a crucial role in the transition towards sustainable agriculture. To autonomously and effectively monitor the state of plants, robots ought to be equipped with visual perception capabilities that are robust to the rapid changes that characterise agricultural settings. In this paper, we focus on the challenging task of segmenting grape bunches from images collected by mobile robots in vineyards. In this context, we present the first study that applies surgical fine-tuning to instance segmentation tasks. We show how selectively tuning only specific model layers can support the adaptation of pre-trained Deep Learning models to newly-collected grape images that introduce visual domain shifts, while also substantially reducing the number of tuned parameters.
摘要
mobile robots会在可持续农业过渡中发挥关键作用。为了让机器人自动和有效地监测植物状况，它们需要具备可靠地对农业环境中快速变化的视觉感知能力。在这篇论文中，我们关注了在葡萄园中机器人收集的图像中分割葡萄束的挑战性任务。我们表明了使用手术精细调整来实现预训练深度学习模型的适应新收集的葡萄图像，同时减少调整参数的数量。
Trading-Off Payments and Accuracy in Online Classification with Paid Stochastic Experts
results: 论文提出了一种在线学习算法，该算法的总成本在 $T$ 轮后不超过一个 predictor 的总成本，这个 predictor 知道所有专家的产出函数（productivity）。这个结果比标准的 Lipschitz 随机抽象更好，可以避免 $T^{2/3}$ 的成本上限。Abstract
We investigate online classification with paid stochastic experts. Here, before making their prediction, each expert must be paid. The amount that we pay each expert directly influences the accuracy of their prediction through some unknown Lipschitz "productivity" function. In each round, the learner must decide how much to pay each expert and then make a prediction. They incur a cost equal to a weighted sum of the prediction error and upfront payments for all experts. We introduce an online learning algorithm whose total cost after $T$ rounds exceeds that of a predictor which knows the productivity of all experts in advance by at most $\mathcal{O}(K^2(\log T)\sqrt{T})$ where $K$ is the number of experts. In order to achieve this result, we combine Lipschitz bandits and online classification with surrogate losses. These tools allow us to improve upon the bound of order $T^{2/3}$ one would obtain in the standard Lipschitz bandit setting. Our algorithm is empirically evaluated on synthetic data
摘要
我们调查线上分类 WITH 付价随机专家。在这里，每个专家都需要付出一定的代价才能发出预测。付出的代价直接影响预测的精度通过一个未知的 Lipschitz "产生力" 函数。在每一轮中，学习者需要决定如何付出每个专家以及做出预测。他们需要支付一个复杂的权重和预测误差的成本。我们引入线上学习算法，其全部成本在 $T$ 轮后不超过一个知道所有专家产生力的预测者的全部成本的 $\mathcal{O}(K^2(\log T)\sqrt{T})$，其中 $K$ 是专家的数量。以实现这个结果，我们结合了Lipschitz 枪和线上分类 WITH 代理损失。这些工具允许我们对于标准 Lipschitz 枪设置中的结果进行改进。我们的算法在实验上被评估了在合成数据上。
Engression: Extrapolation for Nonlinear Regression?
For: The paper is written for those who need a nonlinear regression method that can handle extrapolation tasks, especially in situations where the training data is limited and the test data is outside the support.* Methods: The paper proposes a new method called “engression” which is a distributional regression technique for pre-additive noise models. The method adds noise to the covariates before applying a nonlinear transformation, allowing it to perform well in extrapolation tasks.* Results: The paper shows that engression consistently provides a meaningful improvement in extrapolation tasks compared to traditional regression approaches such as least-squares regression and quantile regression, especially when the function class is strictly monotone. The empirical results from both simulated and real data validate the effectiveness of the engression method.Abstract
Extrapolation is crucial in many statistical and machine learning applications, as it is common to encounter test data outside the training support. However, extrapolation is a considerable challenge for nonlinear models. Conventional models typically struggle in this regard: while tree ensembles provide a constant prediction beyond the support, neural network predictions tend to become uncontrollable. This work aims at providing a nonlinear regression methodology whose reliability does not break down immediately at the boundary of the training support. Our primary contribution is a new method called `engression' which, at its core, is a distributional regression technique for pre-additive noise models, where the noise is added to the covariates before applying a nonlinear transformation. Our experimental results indicate that this model is typically suitable for many real data sets. We show that engression can successfully perform extrapolation under some assumptions such as a strictly monotone function class, whereas traditional regression approaches such as least-squares regression and quantile regression fall short under the same assumptions. We establish the advantages of engression over existing approaches in terms of extrapolation, showing that engression consistently provides a meaningful improvement. Our empirical results, from both simulated and real data, validate these findings, highlighting the effectiveness of the engression method. The software implementations of engression are available in both R and Python.
摘要
<> translate the following text into Simplified Chinese<>描述：扩展是在统计学和机器学习应用中非常重要，因为很多时候会遇到训练数据外的测试数据。然而，扩展是非线性模型的一大挑战。传统的模型通常在这个方面表现不佳：虚拟树集提供了常数预测，而神经网络预测则变得无法控制。这项工作的目标是提供一种非线性回归方法，其可靠性不会因训练支持的边界而崩溃。我们的主要贡献是一种名为“扩展”的新方法，其核心思想是为幂函数模型添加前向随机变量的分布式回归技术。我们的实验结果表明，这种方法适用于许多实际数据集。我们证明了扩展在一些假设下（如幂函数类型的准确 monotonic）下能够成功进行扩展，而传统的回归方法如最小二乘回归和量化回归则在同样的假设下失败。我们还证明了扩展与现有方法的优势，并通过实验证明了扩展的有效性。扩展的软件实现现已在R和Python中可用。Note: Simplified Chinese is used in mainland China and Singapore, while Traditional Chinese is used in Taiwan, Hong Kong, and Macau.
Model-Assisted Probabilistic Safe Adaptive Control With Meta-Bayesian Learning
for: 本研究旨在开发一种可靠的 adaptive safe control 框架，以满足控制系统中的安全性和可靠性需求。
methods: 本研究使用 meta learning 技术、权重学习和 Bayesian 模型，以及控制边界函数（CBF）方法，来学习内在和外在不确定性。特别是，通过 CBF 方法，我们可以通过一个统一的 adaptive Bayesian linear regression（ABLR）模型来学习不确定性，该模型包括一个前向神经网络（NN）和一个 Bayesian 输出层。
results: 对比历史类似任务的数据，我们的算法可以快速地适应新的控制任务，并在多个不确定性约束下进行安全的探索。 results 表明，我们的算法可以显著提高 Bayesian 模型基于 CBF 方法的性能，并且可以快速地适应不同的控制任务。Abstract
Breaking safety constraints in control systems can lead to potential risks, resulting in unexpected costs or catastrophic damage. Nevertheless, uncertainty is ubiquitous, even among similar tasks. In this paper, we develop a novel adaptive safe control framework that integrates meta learning, Bayesian models, and control barrier function (CBF) method. Specifically, with the help of CBF method, we learn the inherent and external uncertainties by a unified adaptive Bayesian linear regression (ABLR) model, which consists of a forward neural network (NN) and a Bayesian output layer. Meta learning techniques are leveraged to pre-train the NN weights and priors of the ABLR model using data collected from historical similar tasks. For a new control task, we refine the meta-learned models using a few samples, and introduce pessimistic confidence bounds into CBF constraints to ensure safe control. Moreover, we provide theoretical criteria to guarantee probabilistic safety during the control processes. To validate our approach, we conduct comparative experiments in various obstacle avoidance scenarios. The results demonstrate that our algorithm significantly improves the Bayesian model-based CBF method, and is capable for efficient safe exploration even with multiple uncertain constraints.
摘要
breaking safety constraints in control systems can lead to potential risks, resulting in unexpected costs or catastrophic damage. Nevertheless, uncertainty is ubiquitous, even among similar tasks. In this paper, we develop a novel adaptive safe control framework that integrates meta learning, Bayesian models, and control barrier function (CBF) method. Specifically, with the help of CBF method, we learn the inherent and external uncertainties by a unified adaptive Bayesian linear regression (ABLR) model, which consists of a forward neural network (NN) and a Bayesian output layer. Meta learning techniques are leveraged to pre-train the NN weights and priors of the ABLR model using data collected from historical similar tasks. For a new control task, we refine the meta-learned models using a few samples, and introduce pessimistic confidence bounds into CBF constraints to ensure safe control. Moreover, we provide theoretical criteria to guarantee probabilistic safety during the control processes. To validate our approach, we conduct comparative experiments in various obstacle avoidance scenarios. The results demonstrate that our algorithm significantly improves the Bayesian model-based CBF method, and is capable for efficient safe exploration even with multiple uncertain constraints.Here's the translation in Traditional Chinese:breaking safety constraints in control systems can lead to potential risks, resulting in unexpected costs or catastrophic damage. Nevertheless, uncertainty is ubiquitous, even among similar tasks. In this paper, we develop a novel adaptive safe control framework that integrates meta learning, Bayesian models, and control barrier function (CBF) method. Specifically, with the help of CBF method, we learn the inherent and external uncertainties by a unified adaptive Bayesian linear regression (ABLR) model, which consists of a forward neural network (NN) and a Bayesian output layer. Meta learning techniques are leveraged to pre-train the NN weights and priors of the ABLR model using data collected from historical similar tasks. For a new control task, we refine the meta-learned models using a few samples, and introduce pessimistic confidence bounds into CBF constraints to ensure safe control. Moreover, we provide theoretical criteria to guarantee probabilistic safety during the control processes. To validate our approach, we conduct comparative experiments in various obstacle avoidance scenarios. The results demonstrate that our algorithm significantly improves the Bayesian model-based CBF method, and is capable for efficient safe exploration even with multiple uncertain constraints.
Robust Surgical Tools Detection in Endoscopic Videos with Noisy Data
results: 根据我们的实验结果，提出的方法可以在含有噪声数据的情况下实现85.88%的平均F1分数，而无类别权重的情况下可以达到80.88%的平均F1分数。此外，我们的提出方法也可以有效地超越现有的方法，这有效地证明了其效果。Abstract
Over the past few years, surgical data science has attracted substantial interest from the machine learning (ML) community. Various studies have demonstrated the efficacy of emerging ML techniques in analysing surgical data, particularly recordings of procedures, for digitizing clinical and non-clinical functions like preoperative planning, context-aware decision-making, and operating skill assessment. However, this field is still in its infancy and lacks representative, well-annotated datasets for training robust models in intermediate ML tasks. Also, existing datasets suffer from inaccurate labels, hindering the development of reliable models. In this paper, we propose a systematic methodology for developing robust models for surgical tool detection using noisy data. Our methodology introduces two key innovations: (1) an intelligent active learning strategy for minimal dataset identification and label correction by human experts; and (2) an assembling strategy for a student-teacher model-based self-training framework to achieve the robust classification of 14 surgical tools in a semi-supervised fashion. Furthermore, we employ weighted data loaders to handle difficult class labels and address class imbalance issues. The proposed methodology achieves an average F1-score of 85.88\% for the ensemble model-based self-training with class weights, and 80.88\% without class weights for noisy labels. Also, our proposed method significantly outperforms existing approaches, which effectively demonstrates its effectiveness.
摘要
过去几年，手术数据科学已经吸引了机器学习（ML）社区的广泛关注。多个研究表明了新兴ML技术在分析手术数据方面的效果，特别是记录手术过程的数据，包括前操作规划、Context-aware决策和手术技巧评估等。然而，这一领域仍处于初期阶段，缺乏代表性的、正确标注的数据集，以用于训练中等ML任务。此外，现有的数据集受到不准确的标注，使得模型的发展受到限制。在这篇论文中，我们提出了一种系统方法论，用于开发Robust模型，以便在噪音数据上进行手术工具检测。我们的方法引入了两个关键创新：1. 一种智能的活动学习策略，用于identify minimal数据集和人工专家标注的自动 corrections;2. 一种学生-教师模型基于的自动训练框架，以实现14种手术工具的robust分类。我们还使用了重量数据加载器，以处理困难的类别标签和类别不均衡问题。提议的方法在 ensemble模型基于自动训练中实现了85.88%的F1分数，而不含类别标签的情况下，则是80.88%。此外，我们的提议方法在与现有方法进行比较时，显示出了明显的效果。
Analysis of Task Transferability in Large Pre-trained Classifiers
methods: 本研究使用一种名为Task Transfer Analysis的方法，将源任务的分布和分类器变换成一个新的源任务分布，并将源任务的损失与下游任务的损失相关联。
results: 本研究通过大规模的实验研究，发现了各种因素如任务相关性、预训练方法和模型结构对传播性能的影响。Abstract
Transfer learning transfers the knowledge acquired by a model from a source task to multiple downstream target tasks with minimal fine-tuning. The success of transfer learning at improving performance, especially with the use of large pre-trained models has made transfer learning an essential tool in the machine learning toolbox. However, the conditions under which the performance is transferable to downstream tasks are not understood very well. In this work, we analyze the transfer of performance for classification tasks, when only the last linear layer of the source model is fine-tuned on the target task. We propose a novel Task Transfer Analysis approach that transforms the source distribution (and classifier) by changing the class prior distribution, label, and feature spaces to produce a new source distribution (and classifier) and allows us to relate the loss of the downstream task (i.e., transferability) to that of the source task. Concretely, our bound explains transferability in terms of the Wasserstein distance between the transformed source and downstream task's distribution, conditional entropy between the label distributions of the two tasks, and weighted loss of the source classifier on the source task. Moreover, we propose an optimization problem for learning the transforms of the source task to minimize the upper bound on transferability. We perform a large-scale empirical study by using state-of-the-art pre-trained models and demonstrate the effectiveness of our bound and optimization at predicting transferability. The results of our experiments demonstrate how factors such as task relatedness, pretraining method, and model architecture affect transferability.
摘要
<> Transfer learning 是一种将模型从源任务中的知识转移到多个下游任务上，并且只需 minimal 微调。由于转移学习在提高性能的能力，特别是使用大型预训练模型，因此转移学习已成为机器学习工具箱中的一种重要工具。但是，转移学习下的性能是如何转移的？在这项工作中，我们分析了 classification 任务下的性能转移，只有源模型的最后 Linear layer 微调。我们提出了一种 Task Transfer Analysis 方法，将源分布（和分类器）变换成新的源分布（和分类器），以生成一个新的源分布（和分类器），并允许我们将下游任务的损失（即转移性）与源任务的损失相关联。具体来说，我们的 bound 将转移性分解为 Wasserstein 距离 between 源和下游任务的分布、下游任务的标签分布和源任务的标签分布之间的 conditional entropy，以及源任务中源分类器的权重损失。此外，我们提出了一个优化问题，用于学习转移任务中的转换，以最小化转移性的上限。我们通过使用现有的预训练模型进行大规模的实验研究，并证明了我们的 bound 和优化方法的效果。实验结果表明，转移学习中的因素，如任务相关性、预训练方法和模型结构，对转移性产生了影响。Note: The translation is in Simplified Chinese, which is the standard writing system used in mainland China. If you prefer Traditional Chinese, please let me know and I can provide the translation in that format as well.
A Critical Re-evaluation of Benchmark Datasets for (Deep) Learning-Based Matching Algorithms
results: 研究发现，大多数流行的Established datasets pose relatively easy classification tasks，因此不适合评估学习型匹配算法的性能。为此，本研究提出了一种新的方法ología para生成benchmark datasets，并在实践中创建了四个新匹配任务，以便更好地评估学习型匹配算法的性能。Abstract
Entity resolution (ER) is the process of identifying records that refer to the same entities within one or across multiple databases. Numerous techniques have been developed to tackle ER challenges over the years, with recent emphasis placed on machine and deep learning methods for the matching phase. However, the quality of the benchmark datasets typically used in the experimental evaluations of learning-based matching algorithms has not been examined in the literature. To cover this gap, we propose four different approaches to assessing the difficulty and appropriateness of 13 established datasets: two theoretical approaches, which involve new measures of linearity and existing measures of complexity, and two practical approaches: the difference between the best non-linear and linear matchers, as well as the difference between the best learning-based matcher and the perfect oracle. Our analysis demonstrates that most of the popular datasets pose rather easy classification tasks. As a result, they are not suitable for properly evaluating learning-based matching algorithms. To address this issue, we propose a new methodology for yielding benchmark datasets. We put it into practice by creating four new matching tasks, and we verify that these new benchmarks are more challenging and therefore more suitable for further advancements in the field.
摘要
<>传统的实体解决（ER）过程是将多个数据库中的记录与同一实体相匹配。随着年代的推移，各种技术已经被开发出来解决ER挑战，其中最近几年的研究主要集中在机器学习和深度学习方法中。然而，学术界中使用的标准数据集的质量尚未得到文献中的检查。为了填补这个差距，我们提出了四种方法来评估13个已知数据集的难度和适用性：两种理论方法，即基于新的线性度量和现有的复杂度度量，以及两种实践方法：非线性和线性匹配器之间的差异，以及学习基于匹配器和完美oracle之间的差异。我们的分析表明，大多数流行的数据集都是较为容易的分类任务，因此它们不适用于评估学习基于匹配算法的进展。为了解决这个问题，我们提出了一种新的方法来生成标准数据集。我们将其应用于四个新匹配任务中，并证明这些新的标准数据集更适合进一步的发展。
Large Language and Text-to-3D Models for Engineering Design Optimization
paper_authors: Thiago Rios, Stefan Menzel, Bernhard Sendhoff
for: 这 paper 是为了研究深度文本-3D 模型在工程领域中的潜在应用，尤其是在计算机 simulate 基于的设计优化中。
methods: 这 paper 使用了 Shap-E，一个最近发布的文本-3D 资产网络，以自动化进化设计优化框架。
results: 研究发现，在使用文本提示中，需要确保生成的设计是在物品类中有效，而且需要进一步研究以确定文本提示的变化和3D 设计变化之间的相关性，以提高优化。Abstract
The current advances in generative AI for learning large neural network models with the capability to produce essays, images, music and even 3D assets from text prompts create opportunities for a manifold of disciplines. In the present paper, we study the potential of deep text-to-3D models in the engineering domain, with focus on the chances and challenges when integrating and interacting with 3D assets in computational simulation-based design optimization. In contrast to traditional design optimization of 3D geometries that often searches for the optimum designs using numerical representations, such as B-Spline surface or deformation parameters in vehicle aerodynamic optimization, natural language challenges the optimization framework by requiring a different interpretation of variation operators while at the same time may ease and motivate the human user interaction. Here, we propose and realize a fully automated evolutionary design optimization framework using Shap-E, a recently published text-to-3D asset network by OpenAI, in the context of aerodynamic vehicle optimization. For representing text prompts in the evolutionary optimization, we evaluate (a) a bag-of-words approach based on prompt templates and Wordnet samples, and (b) a tokenisation approach based on prompt templates and the byte pair encoding method from GPT4. Our main findings from the optimizations indicate that, first, it is important to ensure that the designs generated from prompts are within the object class of application, i.e. diverse and novel designs need to be realistic, and, second, that more research is required to develop methods where the strength of text prompt variations and the resulting variations of the 3D designs share causal relations to some degree to improve the optimization.
摘要
当前的生成AI技术在学习大规模神经网络模型方面带来了许多机会，这些模型可以从文本提示生成文章、图像、音乐和 même 3D 资产。在 presente 纸上，我们研究了在工程领域中深度文本-3D 模型的潜在力量，特别是在 Computational simulation-based 设计优化中交互和结合 3D 资产的机会和挑战。与传统的3D 结构设计优化不同，通过 numerical 表示（如 B-Spline 表面或者扭变参数在汽车 aerodynamic 优化中），文本挑战了优化框架，需要不同的变量操作符的解释，同时可能使人工用户交互更加容易和动机。在这里，我们提出了一个完全自动化的进化式设计优化框架，使用 OpenAI 最近发布的 Shap-E 文本-3D 资产网络。为表示文本提示在进化优化中，我们评估了（a）使用提示模板和 Wordnet 样本的袋子-of-words 方法，以及（b）使用提示模板和 GPT4 的字节对应方法。我们的主要发现表明，首先，需要确保生成的设计是在应用对象类中的，即文本提示生成的设计需要是多样化、创新的，同时也需要是真实的。其次，需要进一步的研究，以发展方法，使文本提示的变化强度和生成的 3D 设计之间存在相互 causal 关系，以改善优化。
Monte Carlo Policy Gradient Method for Binary Optimization
methods: 这 paper 使用了一种新的 probabilistic model，以采样 binary solution according to a parameterized policy distribution。
results: 这 paper 的结果表明，使用这种方法可以提供 near-optimal 的解决方案，并且 convergence 性能良好。Abstract
Binary optimization has a wide range of applications in combinatorial optimization problems such as MaxCut, MIMO detection, and MaxSAT. However, these problems are typically NP-hard due to the binary constraints. We develop a novel probabilistic model to sample the binary solution according to a parameterized policy distribution. Specifically, minimizing the KL divergence between the parameterized policy distribution and the Gibbs distributions of the function value leads to a stochastic optimization problem whose policy gradient can be derived explicitly similar to reinforcement learning. For coherent exploration in discrete spaces, parallel Markov Chain Monte Carlo (MCMC) methods are employed to sample from the policy distribution with diversity and approximate the gradient efficiently. We further develop a filter scheme to replace the original objective function by the one with the local search technique to broaden the horizon of the function landscape. Convergence to stationary points in expectation of the policy gradient method is established based on the concentration inequality for MCMC. Numerical results show that this framework is very promising to provide near-optimal solutions for quite a few binary optimization problems.
摘要
二进制优化具有广泛的应用于 combinatorial 优化问题中，如 MaxCut、MIMO 探测和 MaxSAT。然而，这些问题通常是 NP-hard 的由于二进制约束。我们开发了一种新的 probabilistic 模型，以采样二进制解决方案根据参数化的政策分布。 Specifically，将参数化的政策分布与 Gibbs 分布的函数值进行 minimum KL divergence 会导致一个随机优化问题，其策略均匀可以明确地 derivation 如抽象学习。为了在离散空间中具有协调的探索，我们使用并行 Markov Chain Monte Carlo (MCMC) 方法来采样政策分布中的多样性和精准地计算 gradient。我们进一步开发了一种筛选方案，将原始目标函数换换为具有本地搜索技术的目标函数，以拓宽目标函数的地平。基于 MCMC 的归一化不等式，我们证明了策略梯度法在预期中 converges to stationary points。 numerics 表明，这一框架非常有 promise 可以为许多二进制优化问题提供near-optimal 解决方案。
GA-DRL: Graph Neural Network-Augmented Deep Reinforcement Learning for DAG Task Scheduling over Dynamic Vehicular Clouds
results: 通过在实际的车辆运动轨迹上模拟多种DAG任务，研究人员发现，GA-DRL方法在DAG任务完成时间方面表现出了超过现有标准准则的优势。Abstract
Vehicular clouds (VCs) are modern platforms for processing of computation-intensive tasks over vehicles. Such tasks are often represented as directed acyclic graphs (DAGs) consisting of interdependent vertices/subtasks and directed edges. In this paper, we propose a graph neural network-augmented deep reinforcement learning scheme (GA-DRL) for scheduling DAG tasks over dynamic VCs. In doing so, we first model the VC-assisted DAG task scheduling as a Markov decision process. We then adopt a multi-head graph attention network (GAT) to extract the features of DAG subtasks. Our developed GAT enables a two-way aggregation of the topological information in a DAG task by simultaneously considering predecessors and successors of each subtask. We further introduce non-uniform DAG neighborhood sampling through codifying the scheduling priority of different subtasks, which makes our developed GAT generalizable to completely unseen DAG task topologies. Finally, we augment GAT into a double deep Q-network learning module to conduct subtask-to-vehicle assignment according to the extracted features of subtasks, while considering the dynamics and heterogeneity of the vehicles in VCs. Through simulating various DAG tasks under real-world movement traces of vehicles, we demonstrate that GA-DRL outperforms existing benchmarks in terms of DAG task completion time.
摘要
自动车云（VC）是现代计算密集任务处理平台。这些任务经常表示为导向无环图（DAG）中的依赖关系，其中每个子任务之间存在指向关系。在本文中，我们提出了基于图神经网络和深度强化学习的GA-DRL方案，用于VC上进行DAG任务调度。在实现这一点上，我们首先将VC协助DAG任务调度模型为Markov决策过程。然后，我们采用多头图注意网络（GAT）来提取DAG子任务的特征。我们开发的GAT允许同时考虑每个子任务的前一个和后一个任务，从而实现两个方向的维度汇集。此外，我们还引入非均匀DAG邻居采样，通过编码调度优先级不同的子任务，使我们的GAT普适于完全未seen DAG任务拓扑。最后，我们将GAT与double deep Q-network学习模块结合，以进行子任务与车辆的具体分配，并考虑车辆在VC中的动态和多样性。通过对各种DAG任务进行真实世界车辆运动轨迹的模拟，我们示出GA-DRL方案在DAG任务完成时间方面的优越性。
results: 该模型名为HIPIE，可以同时解决多级嵌入 semantics, open-vocabulary, and universal segmentation tasks。在多个dataset上（如ADE20K、COCO、Pascal-VOC Part、RefCOCO/RefCOCOg、ODinW和SeginW）进行了测试，HIPIE在不同的图像理解水平（如semantic segmentation、panoptic/referring segmentation、object detection和part/subpart segmentation）中达到了state-of-the-art的结果。Abstract
Open-vocabulary image segmentation aims to partition an image into semantic regions according to arbitrary text descriptions. However, complex visual scenes can be naturally decomposed into simpler parts and abstracted at multiple levels of granularity, introducing inherent segmentation ambiguity. Unlike existing methods that typically sidestep this ambiguity and treat it as an external factor, our approach actively incorporates a hierarchical representation encompassing different semantic-levels into the learning process. We propose a decoupled text-image fusion mechanism and representation learning modules for both "things" and "stuff".1 Additionally, we systematically examine the differences that exist in the textual and visual features between these types of categories. Our resulting model, named HIPIE, tackles HIerarchical, oPen-vocabulary, and unIvErsal segmentation tasks within a unified framework. Benchmarked on over 40 datasets, e.g., ADE20K, COCO, Pascal-VOC Part, RefCOCO/RefCOCOg, ODinW and SeginW, HIPIE achieves the state-of-the-art results at various levels of image comprehension, including semantic-level (e.g., semantic segmentation), instance-level (e.g., panoptic/referring segmentation and object detection), as well as part-level (e.g., part/subpart segmentation) tasks. Our code is released at https://github.com/berkeley-hipie/HIPIE.
摘要
开放词汇图像分割目标是将图像 partition 成Semantic 区域，根据自由文本描述。然而，复杂的视觉场景可以自然地被 decomposed 成更简单的部分，并且在不同的粒度上进行抽象，从而引入内在的分割抽象。不同于现有方法，我们的方法 actively 包含层次结构表示，以便在学习过程中吸收不同 semantic 级别的信息。我们提出了解释文本-图像融合机制和表示学习模块，用于处理 "thing" 和 "stuff" 两类不同的概念。此外，我们系统性地研究了这两类概念之间的文本特征和视觉特征之间的差异。我们的模型，名为 HIPIE，可以同时解决多级图像理解任务，包括层次、开放词汇、不同类别的图像分割任务。我们在超过40个数据集上进行了 benchmarking，包括 ADE20K、COCO、Pascal-VOC Part、RefCOCO/RefCOCOg、ODinW 和 SeginW，HIPIE 在不同的图像理解水平上达到了状态之前的最佳结果，包括semantic-level（例如semantic segmentation）、instance-level（例如panoptic/referring segmentation和对象检测）以及part-level（例如part/subpart segmentation）任务。我们的代码可以在 GitHub 上获取：https://github.com/berkeley-hipie/HIPIE。
EmoGen: Eliminating Subjective Bias in Emotional Music Generation
results: 对于emotion control accuracy和音乐质量，EmoGen的表现都超过了之前的方法。具体来说，EmoGen在情感控制精度方面的表现提高了15.6%，而音乐质量方面的表现提高了22.4%。这些结果表明EmoGen在生成情感强的音乐方面具有优势。Abstract
Music is used to convey emotions, and thus generating emotional music is important in automatic music generation. Previous work on emotional music generation directly uses annotated emotion labels as control signals, which suffers from subjective bias: different people may annotate different emotions on the same music, and one person may feel different emotions under different situations. Therefore, directly mapping emotion labels to music sequences in an end-to-end way would confuse the learning process and hinder the model from generating music with general emotions. In this paper, we propose EmoGen, an emotional music generation system that leverages a set of emotion-related music attributes as the bridge between emotion and music, and divides the generation into two stages: emotion-to-attribute mapping with supervised clustering, and attribute-to-music generation with self-supervised learning. Both stages are beneficial: in the first stage, the attribute values around the clustering center represent the general emotions of these samples, which help eliminate the impacts of the subjective bias of emotion labels; in the second stage, the generation is completely disentangled from emotion labels and thus free from the subjective bias. Both subjective and objective evaluations show that EmoGen outperforms previous methods on emotion control accuracy and music quality respectively, which demonstrate our superiority in generating emotional music. Music samples generated by EmoGen are available via this link:https://ai-muzic.github.io/emogen/, and the code is available at this link:https://github.com/microsoft/muzic/.
摘要
音乐可以传递情感，因此自动生成情感rich的音乐是非常重要的。在过去的工作中，情感音乐生成直接使用了标注的情感标签作为控制信号，但这会受到主观偏见的影响：不同的人可能对同一首音乐的情感标签进行不同的标注，一个人在不同的情况下可能会感受到不同的情感。因此，直接将情感标签映射到音乐序列的方式会诱导学习过程中的混乱，使模型无法生成普遍的情感音乐。在这篇论文中，我们提出了Emotion Music Generation（EmoGen）系统，该系统利用了一组情感相关的音乐特征作为情感和音乐之间的桥梁，并将生成分为两个阶段：情感到特征映射与监督聚合，以及特征到音乐生成与自我监督学习。两个阶段都是有利的：在第一个阶段，特征值附近的聚合中心表示这些样本的普遍情感，这有助于消除主观偏见的情感标签的影响；在第二个阶段，生成完全不受情感标签的影响，因此免除了主观偏见的问题。两种评价方法（主观和客观）都表明EmoGen在情感控制准确性和音乐质量方面超越了之前的方法，这表明我们在生成情感音乐方面的优势。EmoGen生成的音乐样本可以在这里找到：https://ai-muzic.github.io/emogen/，代码可以在这里找到：https://github.com/microsoft/muzic/。
Graph-level Anomaly Detection via Hierarchical Memory Networks
results: 对于16种真实的图像数据集，本方法在检测本地异常图和全局异常图方面具有显著的优势，并且具有较高的抗异常杂质性能。代码可以在 GitHub 上获取：https://github.com/Niuchx/HimNet。Abstract
Graph-level anomaly detection aims to identify abnormal graphs that exhibit deviant structures and node attributes compared to the majority in a graph set. One primary challenge is to learn normal patterns manifested in both fine-grained and holistic views of graphs for identifying graphs that are abnormal in part or in whole. To tackle this challenge, we propose a novel approach called Hierarchical Memory Networks (HimNet), which learns hierarchical memory modules -- node and graph memory modules -- via a graph autoencoder network architecture. The node-level memory module is trained to model fine-grained, internal graph interactions among nodes for detecting locally abnormal graphs, while the graph-level memory module is dedicated to the learning of holistic normal patterns for detecting globally abnormal graphs. The two modules are jointly optimized to detect both locally- and globally-anomalous graphs. Extensive empirical results on 16 real-world graph datasets from various domains show that i) HimNet significantly outperforms the state-of-art methods and ii) it is robust to anomaly contamination. Codes are available at: https://github.com/Niuchx/HimNet.
摘要
格式化检测目标是找到异常图形和节点特征相比多数图形集中的异常图形。一个主要挑战是学习图形集中正常模式，包括细致和总体两个视图。为解决这个挑战，我们提出了一种新的方法 called Hierarchical Memory Networks (HimNet)，它通过图像自动编码网络架构学习层次记忆模块——节点记忆模块和图形记忆模块。节点级别记忆模块用于模型节点之间的细致相互作用，以检测本地异常图形，而图形级别记忆模块则专门学习总体正常模式，以检测全球异常图形。两个模块被联合优化，以检测本地和全球异常图形。我们的实验结果表明，i) HimNet significantly 超过了当前方法，ii) 它对异常污染有良好的鲁棒性。代码可以在 找到。
ImDiffusion: Imputed Diffusion Models for Multivariate Time Series Anomaly Detection
results: 这篇论文的实验结果显示，与现有方法比较，ImDiffusion在检测精度和时间上都有明显的进步。尤其是在Microsoft的生产环境中，ImDiffusion的检测F1分数提高了11.4%。Abstract
Anomaly detection in multivariate time series data is of paramount importance for ensuring the efficient operation of large-scale systems across diverse domains. However, accurately detecting anomalies in such data poses significant challenges. Existing approaches, including forecasting and reconstruction-based methods, struggle to address these challenges effectively. To overcome these limitations, we propose a novel anomaly detection framework named ImDiffusion, which combines time series imputation and diffusion models to achieve accurate and robust anomaly detection. The imputation-based approach employed by ImDiffusion leverages the information from neighboring values in the time series, enabling precise modeling of temporal and inter-correlated dependencies, reducing uncertainty in the data, thereby enhancing the robustness of the anomaly detection process. ImDiffusion further leverages diffusion models as time series imputers to accurately capturing complex dependencies. We leverage the step-by-step denoised outputs generated during the inference process to serve as valuable signals for anomaly prediction, resulting in improved accuracy and robustness of the detection process. We evaluate the performance of ImDiffusion via extensive experiments on benchmark datasets. The results demonstrate that our proposed framework significantly outperforms state-of-the-art approaches in terms of detection accuracy and timeliness. ImDiffusion is further integrated into the real production system in Microsoft and observe a remarkable 11.4% increase in detection F1 score compared to the legacy approach. To the best of our knowledge, ImDiffusion represents a pioneering approach that combines imputation-based techniques with time series anomaly detection, while introducing the novel use of diffusion models to the field.
摘要
<> translate the following text into Simplified Chinese:Anomaly detection in multivariate time series data is of paramount importance for ensuring the efficient operation of large-scale systems across diverse domains. However, accurately detecting anomalies in such data poses significant challenges. Existing approaches, including forecasting and reconstruction-based methods, struggle to address these challenges effectively. To overcome these limitations, we propose a novel anomaly detection framework named ImDiffusion, which combines time series imputation and diffusion models to achieve accurate and robust anomaly detection. The imputation-based approach employed by ImDiffusion leverages the information from neighboring values in the time series, enabling precise modeling of temporal and inter-correlated dependencies, reducing uncertainty in the data, thereby enhancing the robustness of the anomaly detection process. ImDiffusion further leverages diffusion models as time series imputers to accurately capturing complex dependencies. We leverage the step-by-step denoised outputs generated during the inference process to serve as valuable signals for anomaly prediction, resulting in improved accuracy and robustness of the detection process. We evaluate the performance of ImDiffusion via extensive experiments on benchmark datasets. The results demonstrate that our proposed framework significantly outperforms state-of-the-art approaches in terms of detection accuracy and timeliness. ImDiffusion is further integrated into the real production system in Microsoft and observe a remarkable 11.4% increase in detection F1 score compared to the legacy approach. To the best of our knowledge, ImDiffusion represents a pioneering approach that combines imputation-based techniques with time series anomaly detection, while introducing the novel use of diffusion models to the field.中文简体版：针对多变量时间序列数据，精准检测异常现象非常重要，以确保大规模系统在多个领域中具有高效的运行。然而，在这种数据中异常检测存在 significativetransportation challenges。现有的方法，包括预测和重建方法，尝试解决这些挑战，但是效果不够。为了超越这些限制，我们提出了一种新的异常检测框架，名为ImDiffusion，它将时间序列插入和扩散模型结合使用，以实现高精度和Robustness的异常检测。ImDiffusion中使用的插入方法利用时间序列中的邻居值信息，准确地模型时间和相关性的依赖关系，从而减少数据中的不确定性，提高异常检测过程的稳定性。ImDiffusion还利用扩散模型作为时间序列插入器，以准确地捕捉复杂的依赖关系。我们利用推理过程中的步骤脱氧输出，作为异常预测的有价值信号，从而提高异常检测的精度和稳定性。我们通过对ImDiffusion进行了广泛的实验，证明了我们提出的框架在检测精度和快速性方面具有明显的优势。ImDiffusion还被 Microsoft 的实际生产系统中 интегра，并观察到了11.4%的增强检测 F1 分数。到目前为止，ImDiffusion 是我们所知道的首个将插入基本技术与时间序列异常检测结合的方法，同时引入扩散模型到该领域。
Population Age Group Sensitivity for COVID-19 Infections with Deep Learning
paper_authors: Md Khairul Islam, Tyler Valentine, Royal Wang, Levi Davis, Matt Manner, Judy Fox for: 这种研究的目的是为了在美国县级别上确定COVID-19感染率中最有影响力的年龄组。methods: 这个研究使用Modified Morris Method和深度学习时序分析来确定年龄组对COVID-19感染率的影响。研究首先在不同的年龄组中训练了现有的时序模型Temporal Fusion Transformer，然后对不同的年龄组进行了特征敏感分析，并根据每个输入特征的柯尼斯基敏感分数（ Morris sensitivity scores）进行排名。results: 研究发现，在COVID-19传播过程中，年龄组最有影响力的是20-29岁的年轻人。这些结果通过美国卫生部和美国人口普查局提供的真实感染率数据得到了验证。这些结果可以用于改进公共卫生政策和 intervención，例如targeted疫苗接种策略，以更好地控制病毒的传播。Abstract
The COVID-19 pandemic has created unprecedented challenges for governments and healthcare systems worldwide, highlighting the critical importance of understanding the factors that contribute to virus transmission. This study aimed to identify the most influential age groups in COVID-19 infection rates at the US county level using the Modified Morris Method and deep learning for time series. Our approach involved training the state-of-the-art time-series model Temporal Fusion Transformer on different age groups as a static feature and the population vaccination status as the dynamic feature. We analyzed the impact of those age groups on COVID-19 infection rates by perturbing individual input features and ranked them based on their Morris sensitivity scores, which quantify their contribution to COVID-19 transmission rates. The findings are verified using ground truth data from the CDC and US Census, which provide the true infection rates for each age group. The results suggest that young adults were the most influential age group in COVID-19 transmission at the county level between March 1, 2020, and November 27, 2021. Using these results can inform public health policies and interventions, such as targeted vaccination strategies, to better control the spread of the virus. Our approach demonstrates the utility of feature sensitivity analysis in identifying critical factors contributing to COVID-19 transmission and can be applied in other public health domains.
摘要
COVID-19 大流行带来了无 precedent 的挑战，让政府和医疗系统在全球各地面临巨大的挑战。这项研究的目的是通过 Modified Morris Method 和深度学习时序序列来确定 COVID-19 感染率在美国县级别中最有影响力的年龄组。我们的方法是在不同的年龄组作为静态特征，并将人口疫苗接种状况作为动态特征，使用时代混合trasformer 模型进行训练。我们分析了每个年龄组对 COVID-19 感染率的影响，并将其排序基于其 Morris 敏感度分数，该分数量化每个年龄组对 COVID-19 传播率的贡献。结果被验证使用 CDC 和 US Census 的真实感染率数据，这些数据提供了每个年龄组的真实感染率。结果表明在2020年3月1日至2021年11月27日之间，美国县级别中最有影响力的年龄组是年轻成人。使用这些结果可以更好地制定公共卫生政策和干预措施，以控制病毒的传播。我们的方法可以应用于其他公共卫生领域，以确定病毒传播中的关键因素。
results: 实验结果表明，ESGCN在四个实际数据集（PEMS03、04、07、08）上达到了当前最佳性能水平，而且计算成本较低Abstract
Traffic forecasting is a highly challenging task owing to the dynamical spatio-temporal dependencies of traffic flows. To handle this, we focus on modeling the spatio-temporal dynamics and propose a network termed Edge Squeeze Graph Convolutional Network (ESGCN) to forecast traffic flow in multiple regions. ESGCN consists of two modules: W-module and ES module. W-module is a fully node-wise convolutional network. It encodes the time-series of each traffic region separately and decomposes the time-series at various scales to capture fine and coarse features. The ES module models the spatio-temporal dynamics using Graph Convolutional Network (GCN) and generates an Adaptive Adjacency Matrix (AAM) with temporal features. To improve the accuracy of AAM, we introduce three key concepts. 1) Using edge features to directly capture the spatiotemporal flow representation among regions. 2) Applying an edge attention mechanism to GCN to extract the AAM from the edge features. Here, the attention mechanism can effectively determine important spatio-temporal adjacency relations. 3) Proposing a novel node contrastive loss to suppress obstructed connections and emphasize related connections. Experimental results show that ESGCN achieves state-of-the-art performance by a large margin on four real-world datasets (PEMS03, 04, 07, and 08) with a low computational cost.
摘要
很多挑战在交通预测中，主要是由交通流动的空间时间相关性引起的。为了解决这个问题，我们关注了空间时间动态的模型化，并提出了一种名为 Edge Squeeze Graph Convolutional Network（ESGCN）来预测多个区域的交通流。ESGCN包括两个模块：W模块和ES模块。W模块是一个完全节点卷积网络，它在每个交通区域 separately 编码时间序列，并在不同的尺度分解时间序列来捕捉细致和大致特征。ES模块使用图aelastic network（GCN）模型了空间时间动态，并生成了一个 Adaptive Adjacency Matrix（AAM），其中包含了时间特征。为了提高AAM的准确性，我们提出了三个关键想法：1. 直接使用边特征来捕捉交通空间时间流表示。2. 应用边注意机制来GCN中提取AAM。这里注意机制可以有效地确定重要的空间时间相关关系。3. 提出了一种新的节点对比损失函数，用于抑制干扰连接和强调相关连接。实验结果表明，ESGCN在四个真实世界数据集（PEMS03、04、07和08）上 achieved state-of-the-art 性能，而且计算成本较低。
vONTSS: vMF based semi-supervised neural topic modeling with optimal transport
paper_authors: Weijie Xu, Xiaoyu Jiang, Srinivasan H. Sengamedu, Francis Iannacci, Jinjin Zhao
for: This paper presents a semi-supervised neural topic modeling method, vONTSS, which aims to incorporate human knowledge into the topic modeling process.
methods: vONTSS uses von Mises-Fisher (vMF) based variational autoencoders and optimal transport to generate potential topics and optimize topic-keyword quality and topic classification.
results: Experiments show that vONTSS outperforms existing semi-supervised topic modeling methods in classification accuracy and diversity. Additionally, vONTSS in the unsupervised setting discovers highly clustered and coherent topics on benchmark datasets and is faster than recent NTMs while achieving similar classification performance.Abstract
Recently, Neural Topic Models (NTM), inspired by variational autoencoders, have attracted a lot of research interest; however, these methods have limited applications in the real world due to the challenge of incorporating human knowledge. This work presents a semi-supervised neural topic modeling method, vONTSS, which uses von Mises-Fisher (vMF) based variational autoencoders and optimal transport. When a few keywords per topic are provided, vONTSS in the semi-supervised setting generates potential topics and optimizes topic-keyword quality and topic classification. Experiments show that vONTSS outperforms existing semi-supervised topic modeling methods in classification accuracy and diversity. vONTSS also supports unsupervised topic modeling. Quantitative and qualitative experiments show that vONTSS in the unsupervised setting outperforms recent NTMs on multiple aspects: vONTSS discovers highly clustered and coherent topics on benchmark datasets. It is also much faster than the state-of-the-art weakly supervised text classification method while achieving similar classification performance. We further prove the equivalence of optimal transport loss and cross-entropy loss at the global minimum.
摘要
近期，神经话题模型（NTM），受变量自动编码器的激发，吸引了大量的研究兴趣；然而，这些方法在实际应用中受到人类知识的挑战。这项工作提出了一种半监督神经话题模型方法，vONTSS，它使用 von Mises-Fisher（vMF）基于的变量自动编码器和最优运输。当提供一些关键词时，vONTSS在半监督设置下生成可能的话题并优化话题-关键词质量和话题分类。实验显示，vONTSS比现有的半监督话题模型方法在分类精度和多样性方面表现更好。vONTSS还支持无监督话题模型。量化和质量实验表明，vONTSS在无监督设置下比最新的弱监督文本分类方法更快，并在类似的分类性能下达到类似的性能。我们进一步证明了最优运输损失和十字积分损失在全局最优点的等价性。
UnLoc: A Universal Localization Method for Autonomous Vehicles using LiDAR, Radar and/or Camera Input
results: 研究人员通过对 Oxford Radar RobotCar、ApolloSouthBay 和 Perth-WA 数据集进行广泛的测试和评估，发现 UnLoc 方法可以准确地地址本地化问题，并且在不同的天气和环境下表现出色。Abstract
Localization is a fundamental task in robotics for autonomous navigation. Existing localization methods rely on a single input data modality or train several computational models to process different modalities. This leads to stringent computational requirements and sub-optimal results that fail to capitalize on the complementary information in other data streams. This paper proposes UnLoc, a novel unified neural modeling approach for localization with multi-sensor input in all weather conditions. Our multi-stream network can handle LiDAR, Camera and RADAR inputs for localization on demand, i.e., it can work with one or more input sensors, making it robust to sensor failure. UnLoc uses 3D sparse convolutions and cylindrical partitioning of the space to process LiDAR frames and implements ResNet blocks with a slot attention-based feature filtering module for the Radar and image modalities. We introduce a unique learnable modality encoding scheme to distinguish between the input sensor data. Our method is extensively evaluated on Oxford Radar RobotCar, ApolloSouthBay and Perth-WA datasets. The results ascertain the efficacy of our technique.
摘要
<>rupiaoming: zhongguo yu dianzi de jiaoyu yu jingying zhongxinLocalization is a fundamental task in robotics for autonomous navigation. Existing localization methods rely on a single input data modality or train several computational models to process different modalities. This leads to stringent computational requirements and sub-optimal results that fail to capitalize on the complementary information in other data streams. This paper proposes UnLoc, a novel unified neural modeling approach for localization with multi-sensor input in all weather conditions. Our multi-stream network can handle LiDAR, Camera and RADAR inputs for localization on demand, i.e., it can work with one or more input sensors, making it robust to sensor failure. UnLoc uses 3D sparse convolutions and cylindrical partitioning of the space to process LiDAR frames and implements ResNet blocks with a slot attention-based feature filtering module for the Radar and image modalities. We introduce a unique learnable modality encoding scheme to distinguish between the input sensor data. Our method is extensively evaluated on Oxford Radar RobotCar, ApolloSouthBay and Perth-WA datasets. The results ascertain the efficacy of our technique.<>Here's the translation in Simplified Chinese:<>本文提出了一种基于多感器输入的LOCALIZATION方法，用于Robotics autonomous navigation。现有的LOCALIZATION方法通常仅使用单一的输入数据模式，或者训练多个计算模型来处理不同的数据模式。这会导致计算需求严格，并且不能充分利用其他数据流中的补充信息。本文提出了一种名为UnLoc的新的协调神经网络方法，可以处理LiDAR、摄像头和RADAR输入数据，并在不同的天气条件下进行定位。我们的多流网络可以根据需要选择一个或多个输入感知器，从而增强其对感知器失效的Robustness。在进行LiDAR框架处理时，我们使用3D稀疏核算法和圆柱体分割方法，并在RADAR和图像模式中实现了ResNet块和满足特征筛选模块。我们还提出了一种唯一的学习型感知编码方案，以便分辨输入感知器的数据。我们的方法在Oxford Radar RobotCar、ApolloSouthBay和Perth-WA数据集上进行了广泛的评估，结果证明了我们的方法的有效性。<>
On the choice of training data for machine learning of geostrophic mesoscale turbulence
results: 研究发现，如果将旋转Component从质量交换流动中过滤掉，那么数据驱动模型的预测能力和稳定性会得到改善，而且可以更好地捕捉到数据中隐藏的物理过程。Abstract
'Data' plays a central role in data-driven methods, but is not often the subject of focus in investigations of machine learning algorithms as applied to Earth System Modeling related problems. Here we consider the case of eddy-mean interaction in rotating stratified turbulence in the presence of lateral boundaries, a problem of relevance to ocean modeling, where the eddy fluxes contain dynamically inert rotational components that are expected to contaminate the learning process. An often utilized choice in the literature is to learn from the divergence of the eddy fluxes. Here we provide theoretical arguments and numerical evidence that learning from the eddy fluxes with the rotational component appropriately filtered out results in models with comparable or better skill, but substantially improved robustness. If we simply want a data-driven model to have predictive skill then the choice of data choice and/or quality may not be critical, but we argue it is highly desirable and perhaps even necessary if we want to leverage data-driven methods to aid in discovering unknown or hidden physical processes within the data itself.
摘要
Interpretability and Transparency-Driven Detection and Transformation of Textual Adversarial Examples (IT-DT)
paper_authors: Bushra Sabir, M. Ali Babar, Sharif Abuadbba for: 这个论文的目的是提出一种可解释性和透明度驱动的检测和转换（IT-DT）框架，以解决BERT等基于Transformer的文本分类器对于恶意示例的抵触性。methods: 这个框架使用了注意力地图、集成导数和模型反馈等技术来提高可解释性，以便在检测阶段更好地理解恶意分类的依据。在转换阶段，IT-DT使用预训练的嵌入和模型反馈来生成适当的替换，以将恶意示例转化为非恶意示例。results: 实验结果表明，IT-DT可以准确地检测和转换恶意示例，提高了模型的可靠性和安全性。此外，人工专家参与约束和反馈也使得决策更加稳定和可靠。Abstract
Transformer-based text classifiers like BERT, Roberta, T5, and GPT-3 have shown impressive performance in NLP. However, their vulnerability to adversarial examples poses a security risk. Existing defense methods lack interpretability, making it hard to understand adversarial classifications and identify model vulnerabilities. To address this, we propose the Interpretability and Transparency-Driven Detection and Transformation (IT-DT) framework. It focuses on interpretability and transparency in detecting and transforming textual adversarial examples. IT-DT utilizes techniques like attention maps, integrated gradients, and model feedback for interpretability during detection. This helps identify salient features and perturbed words contributing to adversarial classifications. In the transformation phase, IT-DT uses pre-trained embeddings and model feedback to generate optimal replacements for perturbed words. By finding suitable substitutions, we aim to convert adversarial examples into non-adversarial counterparts that align with the model's intended behavior while preserving the text's meaning. Transparency is emphasized through human expert involvement. Experts review and provide feedback on detection and transformation results, enhancing decision-making, especially in complex scenarios. The framework generates insights and threat intelligence empowering analysts to identify vulnerabilities and improve model robustness. Comprehensive experiments demonstrate the effectiveness of IT-DT in detecting and transforming adversarial examples. The approach enhances interpretability, provides transparency, and enables accurate identification and successful transformation of adversarial inputs. By combining technical analysis and human expertise, IT-DT significantly improves the resilience and trustworthiness of transformer-based text classifiers against adversarial attacks.
摘要
tranformer-based文本分类器如BERT、Roberta、T5和GPT-3在NLP中表现出色，但它们对攻击性示例的抵触性存在安全风险。现有的防御方法缺乏可读性，使得对攻击分类和模型漏洞难以理解。为了解决这个问题，我们提出了可读性和透明度驱动的检测和转换（IT-DT）框架。IT-DT将注重可读性和透明度，在检测阶段使用注意力地图、 интеGRATED GRADIENTS 和模型反馈来提供可读性。这 помоляет Identify 突出的特征和折衣字符在攻击分类中发挥作用。在转换阶段，IT-DT使用预训练的嵌入和模型反馈来生成适当的替换，以将攻击示例转化为不攻击的示例，保持文本的意义。在转换过程中，人工专家参与纠正和反馈，以增强决策，特别是在复杂的情况下。IT-DT生成了见解和威胁情报，使分析人员可以识别漏洞和提高模型 Robustness。广泛的实验表明IT-DT可以准确地检测和转换攻击示例。这种方法提高了可读性，提供了透明度，并使transformer-based文本分类器对攻击示例的抵觕性提高。通过结合技术分析和人工专家知识，IT-DT有效地提高了transformer-based文本分类器对攻击示例的抵觕性和可靠性。
results: 研究发现，使用简单神经网络和ReLU活化函数可以生成规范体，并且可以通过调整网络结构来控制规范体的种类和维度。此外，研究还发现了这些规范体的总体化，即神经规范体。Abstract
We find that simple neural networks with ReLU activation generate polytopes as an approximation of a unit sphere in various dimensions. The species of polytopes are regulated by the network architecture, such as the number of units and layers. For a variety of activation functions, generalization of polytopes is obtained, which we call neural polytopes. They are a smooth analogue of polytopes, exhibiting geometric duality. This finding initiates research of generative discrete geometry to approximate surfaces by machine learning.
摘要
我们发现简单的神经网络与ReLU吸引函数生成了多面体，作为各种维度上圆形的近似。这种多面体的种类受到神经网络架构的限制，如单元数和层数。对于不同的吸引函数，我们可以得到总化的多面体，我们称之为神经多面体。它们是一种缓和的多面体，展示了几何对偶。这一发现推动了机器学习来approximate Surfaces的生成推理 discrete geometry。
results: 研究发现，知识的价值受到数据量和估计范围的影响，存在依赖、协同和替换效应。这种结果可以应用于多种常见的网络架构，并且可以提高了知识汇报的性能和约束遵循性。Abstract
Knowledge constitutes the accumulated understanding and experience that humans use to gain insight into the world. In deep learning, prior knowledge is essential for mitigating shortcomings of data-driven models, such as data dependence, generalization ability, and compliance with constraints. To enable efficient evaluation of the worth of knowledge, we present a framework inspired by interpretable machine learning. Through quantitative experiments, we assess the influence of data volume and estimation range on the worth of knowledge. Our findings elucidate the complex relationship between data and knowledge, including dependence, synergistic, and substitution effects. Our model-agnostic framework can be applied to a variety of common network architectures, providing a comprehensive understanding of the role of prior knowledge in deep learning models. It can also be used to improve the performance of informed machine learning, as well as distinguish improper prior knowledge.
摘要
知识是人类使用来理解世界的总结和经验。在深度学习中，先验知识是关键的，可以减少数据驱动模型的缺陷，例如数据依赖、泛化能力和约束遵从。为了有效评估知识的价值，我们提出一种基于可解释机器学习的框架。通过量化实验，我们评估数据量和估计范围对知识的影响。我们的发现揭示了数据和知识之间复杂的关系，包括依赖、合作和替换效应。我们的框架可以应用于多种常见的网络架构，为深度学习模型的角色带来全面的理解。它还可以用来改进了知识填充机器学习的性能，以及分辨不正确的先验知识。
A physics-constrained machine learning method for mapping gapless land surface temperature
for: 这 paper 的目的是提出一种physics-constrained machine learning（PC-ML）模型，用于 gapless 土壤温度（LST）估算，以提高physical interpretability和抽象能力。
methods: 该模型 combines 机器学习（ML）模型和物理机制模型，并将 физические约束（PCs） incorporated 到 ML 模型中，以提高模型的解释能力和推断能力。
results: 对比 pure physical method 和 pure ML methods，PC-LGBM 模型提高了 LST 预测精度和physical interpretability，并demonstrated good extrapolation ability for extreme weather cases。这种方法可以提供高精度和物理意义的 gapless LST 估算，并可以加速土壤表面过程的研究和数据挖掘。Abstract
More accurate, spatio-temporally, and physically consistent LST estimation has been a main interest in Earth system research. Developing physics-driven mechanism models and data-driven machine learning (ML) models are two major paradigms for gapless LST estimation, which have their respective advantages and disadvantages. In this paper, a physics-constrained ML model, which combines the strengths in the mechanism model and ML model, is proposed to generate gapless LST with physical meanings and high accuracy. The hybrid model employs ML as the primary architecture, under which the input variable physical constraints are incorporated to enhance the interpretability and extrapolation ability of the model. Specifically, the light gradient-boosting machine (LGBM) model, which uses only remote sensing data as input, serves as the pure ML model. Physical constraints (PCs) are coupled by further incorporating key Community Land Model (CLM) forcing data (cause) and CLM simulation data (effect) as inputs into the LGBM model. This integration forms the PC-LGBM model, which incorporates surface energy balance (SEB) constraints underlying the data in CLM-LST modeling within a biophysical framework. Compared with a pure physical method and pure ML methods, the PC-LGBM model improves the prediction accuracy and physical interpretability of LST. It also demonstrates a good extrapolation ability for the responses to extreme weather cases, suggesting that the PC-LGBM model enables not only empirical learning from data but also rationally derived from theory. The proposed method represents an innovative way to map accurate and physically interpretable gapless LST, and could provide insights to accelerate knowledge discovery in land surface processes and data mining in geographical parameter estimation.
摘要
更准确、空间和时间一致、物理一致的LST估计已经是地球系统研究的主要兴趣点。开发物理驱动机制模型和数据驱动机器学习（ML）模型是两个主要方法 для无缝LST估计，它们各有优势和缺点。本文提出了一种物理约束机器学习（PC-ML）模型，它将机器学习模型作为主体，并将输入变量物理约束（PC）纳入模型中以提高解释性和推理能力。具体来说，使用远程感知数据为输入的光梯度提升机（LGBM）模型作为纯ML模型。PC通过将CLM的激活数据（原因）和CLM的仿真数据（后果）作为输入 integrate into LGBM model，形成PC-LGBM模型，这个模型既包含了地表能耗平衡（SEB）的下面数据，又在CLM-LST模型中体现出了物理的含义。与纯物理方法和纯ML方法相比，PC-LGBM模型提高了LST预测精度和物理解释性。它还表明了对EXTREME WEATHER CASES的应答能力，表明PC-LGBM模型不仅可以从数据学习，也可以从理论 derivation。提出的方法可以创新精度和物理解释能力的无缝LST地图，并提供了对土地表面过程的加速知识发现和数据挖掘的新思路。
Classification of sleep stages from EEG, EOG and EMG signals by SSNet
results: 本研究使用了两个公共的数据集，包括Sleep-EDF扩展数据集和ISRUC-Sleep数据集，评估了我们提出的模型的性能。结果显示，我们的模型在三种睡眠阶段的分类中取得了96.36%的准确率和93.40%的协变系数，在五种睡眠阶段的分类中取得了96.57%的准确率和83.05%的协变系数。相比之下，我们的模型在睡眠阶段分类中表现较好，并且超过了现有的技术。Abstract
Classification of sleep stages plays an essential role in diagnosing sleep-related diseases including Sleep Disorder Breathing (SDB) disease. In this study, we propose an end-to-end deep learning architecture, named SSNet, which comprises of two deep learning networks based on Convolutional Neuron Networks (CNN) and Long Short Term Memory (LSTM). Both deep learning networks extract features from the combination of Electrooculogram (EOG), Electroencephalogram (EEG), and Electromyogram (EMG) signals, as each signal has distinct features that help in the classification of sleep stages. The features produced by the two-deep learning networks are concatenated to pass to the fully connected layer for the classification. The performance of our proposed model is evaluated by using two public datasets Sleep-EDF Expanded dataset and ISRUC-Sleep dataset. The accuracy and Kappa coefficient are 96.36% and 93.40% respectively, for classifying three classes of sleep stages using Sleep-EDF Expanded dataset. Whereas, the accuracy and Kappa coefficient are 96.57% and 83.05% respectively for five classes of sleep stages using Sleep-EDF Expanded dataset. Our model achieves the best performance in classifying sleep stages when compared with the state-of-the-art techniques.
摘要
классификация сновидений играет ключевую роль в диагностике заболеваний, связанных с сном, включая заболевание дыхательными путями во сне (SDB). В этом исследовании мы предлагаем энд-то-энд architecture, называемую SSNet, которая включает в себя два глубоких обучающихся сетей на основе сетей convolutional neurons (CNN) и Long Short Term Memory (LSTM). Обе сети глубокого обучения извлекают признаки из комбинации сигналов Electrooculogram (EOG), Electroencephalogram (EEG) и Electromyogram (EMG), поскольку каждый сигнал имеет отличительные признаки, которые помогают в классификации сновидений. Признаки, выделенные двумя сетями глубокого обучения, сходят в полностью связанный слой для классификации. Оценка нашей предложенной модели была выполнена с помощью двух общедоступных данныхсетов Sleep-EDF Expanded dataset и ISRUC-Sleep dataset. Аккуратность и коэффициент Кэппеля были равны 96,36% и 93,40% соответственно для классификации трех классов сновидений с помощью Sleep-EDF Expanded dataset. При этом, аккуратность и коэффициент Кэппеля были равны 96,57% и 83,05% соответственно для пяти классов сновидений с помощью Sleep-EDF Expanded dataset. Наша модель достигла лучшего результата в классификации сновидений по сравнению с существующими техниками.
results: 实验表明，该验证方法可以捕捉广泛的攻击，包括所有已知的Proof-of-Learning文献中的攻击。Abstract
It is important that consumers and regulators can verify the provenance of large neural models to evaluate their capabilities and risks. We introduce the concept of a "Proof-of-Training-Data": any protocol that allows a model trainer to convince a Verifier of the training data that produced a set of model weights. Such protocols could verify the amount and kind of data and compute used to train the model, including whether it was trained on specific harmful or beneficial data sources. We explore efficient verification strategies for Proof-of-Training-Data that are compatible with most current large-model training procedures. These include a method for the model-trainer to verifiably pre-commit to a random seed used in training, and a method that exploits models' tendency to temporarily overfit to training data in order to detect whether a given data-point was included in training. We show experimentally that our verification procedures can catch a wide variety of attacks, including all known attacks from the Proof-of-Learning literature.
摘要
Note: Simplified Chinese is also known as "简化字母" or "简化字母".Translation notes:* "consumers" is translated as "消费者" (shāngchǎng zhě)* "regulators" is translated as "管制机构" (guǎnzhì jīgòu)* "Proof-of-Training-Data" is translated as "训练数据证明" (xùxíng xùzhì)* "model trainer" is translated as "模型训练者" (móxìng xùxíng zhe)* "Verifier" is translated as "验证人" (yànzhèng rén)* "large-model training procedures" is translated as "大型模型训练程序" (dàxíng móxìng xùxíng)* "random seed" is translated as "随机种子" (suījī zhòngzi)* "training data" is translated as "训练数据" (xùxíng xùzhì)* "harmful or beneficial data sources" is translated as "有害或有益的数据来源" (yǒu hài yòu yì de xùnxīn lái yuán)Please note that the translation is done by a machine and may not be perfect, and it's always a good idea to have a human reviewer to check the translation before using it in any official context.
CLIMAX: An exploration of Classifier-Based Contrastive Explanations
For: 这种论文旨在解释黑盒机器学习模型的决策过程，以便使这些模型更加透明、负责任、可理解。* Methods: 这种方法基于本地分类器，并使用了标签感知的副本数据生成方法和影响子抽样来保证模型准确性。* Results: 作者比较了这种方法与其他基于LIME的方法，并发现它在一些预测任务上具有更高的一致性。此外，这种方法还可以在文本和图像类 datasets 上生成对比性的解释。Abstract
Explainable AI is an evolving area that deals with understanding the decision making of machine learning models so that these models are more transparent, accountable, and understandable for humans. In particular, post-hoc model-agnostic interpretable AI techniques explain the decisions of a black-box ML model for a single instance locally, without the knowledge of the intrinsic nature of the ML model. Despite their simplicity and capability in providing valuable insights, existing approaches fail to deliver consistent and reliable explanations. Moreover, in the context of black-box classifiers, existing approaches justify the predicted class, but these methods do not ensure that the explanation scores strongly differ as compared to those of another class. In this work we propose a novel post-hoc model agnostic XAI technique that provides contrastive explanations justifying the classification of a black box classifier along with a reasoning as to why another class was not predicted. Our method, which we refer to as CLIMAX which is short for Contrastive Label-aware Influence-based Model Agnostic XAI, is based on local classifiers . In order to ensure model fidelity of the explainer, we require the perturbations to be such that it leads to a class-balanced surrogate dataset. Towards this, we employ a label-aware surrogate data generation method based on random oversampling and Gaussian Mixture Model sampling. Further, we propose influence subsampling in order to retaining effective samples and hence ensure sample complexity. We show that we achieve better consistency as compared to baselines such as LIME, BayLIME, and SLIME. We also depict results on textual and image based datasets, where we generate contrastive explanations for any black-box classification model where one is able to only query the class probabilities for an instance of interest.
摘要
Explainable AI是一个在发展中的领域，旨在理解机器学习模型的决策过程，以便这些模型更加透明、责任、可理解。特别是，我们专注于在黑盒机器学习模型上的后期、模型无关的解释技术，可以在单一实例上，地方解释模型的决策。虽然现有的方法具有简单性和可提供有价值的洞见，但是它们无法提供一致和可靠的解释。此外，在黑盒分类器的情况下，现有的方法只会说明预测的类别，但不能保证解释 scores 强烈不同于另一个类别。在这个工作中，我们提出了一种新的后期、模型无关的解释技术，可以为黑盒分类器提供相对的解释，同时也能够解释为何选择另一个类别。我们称这种技术为 CLIMAX，即 Contrastive Label-aware Influence-based Model Agnostic XAI。CLIMAX 基于地方分类器，以 Ensure explainer 的模型实践性，我们需要进行类别数balanced的调整数据。为了实现这一目标，我们使用了随机批量扩展和泊松分布的采样方法。此外，我们也提出了影响抽样，以保留有效的抽样和确保样本复杂性。我们发现我们的方法可以与基于 LIME、BayLIME 和 SLIME 的基eline 相比，具有更高的一致性。我们还展示了在文本和图像基于的数据集上的结果，可以为任何黑盒分类器提供相对的解释，只需要对兴趣的实例进行查询。
SDC-HSDD-NDSA: Structure Detecting Cluster by Hierarchical Secondary Directed Differential with Normalized Density and Self-Adaption
results: 在多个数据集中运行算法，结果验证了该方法的结构检测、鲁棒性和粒度独立性，并且表明其能够超越前一代方法。Abstract
Density-based clustering could be the most popular clustering algorithm since it can identify clusters of arbitrary shape as long as different (high-density) clusters are separated by low-density regions. However, the requirement of the separateness of clusters by low-density regions is not trivial since a high-density region might have different structures which should be clustered into different groups. Such a situation demonstrates the main flaw of all previous density-based clustering algorithms we have known--structures in a high-density cluster could not be detected. Therefore, this paper aims to provide a density-based clustering scheme that not only has the ability previous ones have but could also detect structures in a high-density region not separated by low-density ones. The algorithm employs secondary directed differential, hierarchy, normalized density, as well as the self-adaption coefficient, and thus is called Structure Detecting Cluster by Hierarchical Secondary Directed Differential with Normalized Density and Self-Adaption, dubbed by SDC-HSDD-NDSA for short. To illustrate its effectiveness, we run the algorithm in several data sets. The results verify its validity in structure detection, robustness over noises, as well as independence of granularities, and demonstrate that it could outperform previous ones. The Python code of the paper could be found on https://github.com/Hao-B-Shu/SDC-HSDD-NDSA.
摘要
density-based clustering可能是最受欢迎的聚类算法，因为它可以找到任意形状的聚集，只要不同的高密度区域被低密度区域隔离开来。然而，需要聚集区域由低密度区域隔离开来的要求并不是干扰的，因为高密度区域可能有不同的结构，这些结构应该被分配到不同的组。这种情况表明了所有之前的密度基于的聚类算法的主要缺陷——聚集区域中的结构不能被探测。因此，本文提出了一种密度基于的聚类方案，不仅具有之前的能力，而且可以在高密度区域中探测结构。该算法使用次级导向差、层次结构、 норциали化密度以及自适应系数，因此被称为结构探测聚类方法，简称SDC-HSDD-NDSA。为证明其效果，我们在多个数据集上运行了该算法。结果表明其在结构探测、鲁棒性和自适应性方面具有优势，并且可以超越之前的算法。Python代码可以在https://github.com/Hao-B-Shu/SDC-HSDD-NDSA中找到。
Pay Attention to the Atlas: Atlas-Guided Test-Time Adaptation Method for Robust 3D Medical Image Segmentation
results: 对多个来自不同站点的数据集进行了广泛的实验，结果显示，AdaAtlas-Attention在比较其他竞争方法时具有显著的性能改善，特别是在3D医学图像分割任务中。Abstract
Convolutional neural networks (CNNs) often suffer from poor performance when tested on target data that differs from the training (source) data distribution, particularly in medical imaging applications where variations in imaging protocols across different clinical sites and scanners lead to different imaging appearances. However, re-accessing source training data for unsupervised domain adaptation or labeling additional test data for model fine-tuning can be difficult due to privacy issues and high labeling costs, respectively. To solve this problem, we propose a novel atlas-guided test-time adaptation (TTA) method for robust 3D medical image segmentation, called AdaAtlas. AdaAtlas only takes one single unlabeled test sample as input and adapts the segmentation network by minimizing an atlas-based loss. Specifically, the network is adapted so that its prediction after registration is aligned with the learned atlas in the atlas space, which helps to reduce anatomical segmentation errors at test time. In addition, different from most existing TTA methods which restrict the adaptation to batch normalization blocks in the segmentation network only, we further exploit the use of channel and spatial attention blocks for improved adaptability at test time. Extensive experiments on multiple datasets from different sites show that AdaAtlas with attention blocks adapted (AdaAtlas-Attention) achieves superior performance improvements, greatly outperforming other competitive TTA methods.
摘要
循环 нейрон网络（CNN）在面向目标数据进行测试时，经常会表现出不佳的性能，特别是在医学影像应用中，因为不同的临床站点和扫描仪器使用不同的扫描技术，导致影像的显示方式不同。然而，重新访问源训练数据进行隐私的预处理或者为模型细化进行标注是困难的，因为隐私和高标注成本。为解决这个问题，我们提出了一种基于图集的测试时适应（TTA）方法，called AdaAtlas。AdaAtlas只需要一个单独的无标签测试样本，并使用图集来适应分割网络，以适应测试数据的不同。具体来说，网络会在注册后与学习的图集进行对比，以减少分割错误。此外，我们还在TTA方法中使用通道和空间注意力块，以提高测试时的适应性。经过多个数据集的测试，我们发现AdaAtlas-Attention方法可以 achieve superior performance improvement，大大超过其他竞争性TTA方法。
ENN: A Neural Network with DCT-Adaptive Activation Functions
results: 经过具体实验，这种模型可以在分类和回归任务上适应和具有高表达能力，并且在一些情况下可以提高 state of the art 的准确率，达到40%之间。Abstract
The expressiveness of neural networks highly depends on the nature of the activation function, although these are usually assumed predefined and fixed during the training stage. In this paper we present Expressive Neural Network (ENN), a novel architecture in which the non-linear activation functions are modeled using the Discrete Cosine Transform (DCT) and adapted using backpropagation during training. This parametrization keeps the number of trainable parameters low, is appropriate for gradient-based schemes, and adapts to different learning tasks. This is the first non-linear model for activation functions that relies on a signal processing perspective, providing high flexibility and expressiveness to the network. We contribute with insights in the explainability of the network at convergence by recovering the concept of bump, this is, the response of each activation function in the output space to provide insights. Finally, through exhaustive experiments we show that the model can adapt to classification and regression tasks. The performance of ENN outperforms state of the art benchmarks, providing up to a 40\% gap in accuracy in some scenarios.
摘要
Expressive Neural Network (ENN)是一种新型神经网络 Architecture，其中非线性活化函数被模型为Discrete Cosine Transform (DCT)，并在训练过程中通过反传播进行适应。这种参数化方式保持训练参数的数量低，适合梯度下降方法，并适应不同的学习任务。这是首次基于信号处理角度的非线性活化函数模型，具有高度的灵活性和表达力。我们提供了解释网络各激活函数在输出空间的响应，即“bump”的概念，从而提供了网络的解释。最后，我们通过广泛的实验证明了ENN模型可以适应分类和回归任务，并在一些情况下超越了状态的权威benchmark。ENN模型的性能与状态的权威benchmark之间的差距可达40%。
Automatic MILP Solver Configuration By Learning Problem Similarities
results: 实验结果显示，我们的方法可以预测组件设置，从而提高解的成本，最高可以观察到38%的改善。Abstract
A large number of real-world optimization problems can be formulated as Mixed Integer Linear Programs (MILP). MILP solvers expose numerous configuration parameters to control their internal algorithms. Solutions, and their associated costs or runtimes, are significantly affected by the choice of the configuration parameters, even when problem instances have the same number of decision variables and constraints. On one hand, using the default solver configuration leads to suboptimal solutions. On the other hand, searching and evaluating a large number of configurations for every problem instance is time-consuming and, in some cases, infeasible. In this study, we aim to predict configuration parameters for unseen problem instances that yield lower-cost solutions without the time overhead of searching-and-evaluating configurations at the solving time. Toward that goal, we first investigate the cost correlation of MILP problem instances that come from the same distribution when solved using different configurations. We show that instances that have similar costs using one solver configuration also have similar costs using another solver configuration in the same runtime environment. After that, we present a methodology based on Deep Metric Learning to learn MILP similarities that correlate with their final solutions' costs. At inference time, given a new problem instance, it is first projected into the learned metric space using the trained model, and configuration parameters are instantly predicted using previously-explored configurations from the nearest neighbor instance in the learned embedding space. Empirical results on real-world problem benchmarks show that our method predicts configuration parameters that improve solutions' costs by up to 38% compared to existing approaches.
摘要
许多现实世界优化问题可以表示为杂合整数线性程序（MILP）。MILP解决器公布了许多配置参数来控制其内部算法。解决方案和其关联的成本或运行时间受到配置参数的选择的影响，即使问题实例具有相同的决策变量和约束。一方面，使用默认解决器配置会导致优化解决方案。另一方面，在每个问题实例上搜索和评估大量配置参数是时间消耗性很高，甚至不可行。在这种情况下，我们的目标是预测未看到的问题实例的配置参数，以便在解决时获得更低成本的解决方案，而无需在解决时进行搜索和评估配置参数。我们首先研究了使用不同配置参数解决MILP问题实例的成本相关性。我们发现，使用不同配置参数解决的MILP问题实例的成本相似。然后，我们提出了基于深度度量学习的方法，用于学习MILP问题之间的相似性。在推理时，给定一个新的问题实例，将其投影到已经学习的度量空间中，并使用已经探索的配置参数来预测解决方案的成本。实验结果表明，我们的方法可以预测解决方案的成本下降至38%，相比于现有的方法。
Active Sensing with Predictive Coding and Uncertainty Minimization
results: 研究人员通过在迷宫探索任务和活观视觉任务中应用该方法，并证明了其能够找到环境的转移分布和重建空间特征。此外，该方法还能够建立不监督的表示，使代理人能够高效地样本和分类感知场景。Abstract
We present an end-to-end procedure for embodied exploration based on two biologically inspired computations: predictive coding and uncertainty minimization. The procedure can be applied to any exploration setting in a task-independent and intrinsically driven manner. We first demonstrate our approach in a maze navigation task and show that our model is capable of discovering the underlying transition distribution and reconstructing the spatial features of the environment. Second, we apply our model to the more complex task of active vision, where an agent must actively sample its visual environment to gather information. We show that our model is able to build unsupervised representations that allow it to actively sample and efficiently categorize sensory scenes. We further show that using these representations as input for downstream classification leads to superior data efficiency and learning speed compared to other baselines, while also maintaining lower parameter complexity. Finally, the modularity of our model allows us to analyze its internal mechanisms and to draw insight into the interactions between perception and action during exploratory behavior.
摘要
我们提出了一种从头到尾的方法，基于生物学上的两种计算：预测编码和不确定度最小化。这种方法可以在任何探索任务中应用，具有任务独立和自适应的特点。我们首先在一个迷宫探索任务中应用了我们的方法，并证明我们的模型可以找到环境的下行传递分布和重建空间特征。然后，我们将我们的模型应用到更复杂的激活视觉任务中，agent需要活动地抽取视觉环境中的信息。我们证明了我们的模型可以建立无监督的表示，使得它可以活动地抽取和有效地分类感知场景。此外，我们还证明了使用这些表示作为下游分类器的输入，可以提高数据效率和学习速度，同时也可以降低参数复杂性。最后，我们的模型的卷积结构允许我们分析它的内部机制，并从探索行为中拓展出对感知和行为之间的交互的理解。
Morse Neural Networks for Uncertainty Quantification
paper_authors: Benoit Dherin, Huiyi Hu, Jie Ren, Michael W. Dusenberry, Balaji Lakshminarayanan for:* The paper presents a new deep generative model called the Morse neural network, which is useful for uncertainty quantification and can be used for various tasks such as OOD detection, anomaly detection, and continuous learning.methods:* The Morse neural network uses a KL-divergence loss to fit the model, which yields five components: a generative density, an OOD detector, a calibration temperature, a generative sampler, and a distance-aware classifier (in the supervised case).results:* The Morse neural network unifies many techniques in uncertainty quantification and has connections to support vector machines, kernel methods, and Morse theory in topology. It can be used on top of a pre-trained network to bring distance-aware calibration w.r.t the training data.Abstract
We introduce a new deep generative model useful for uncertainty quantification: the Morse neural network, which generalizes the unnormalized Gaussian densities to have modes of high-dimensional submanifolds instead of just discrete points. Fitting the Morse neural network via a KL-divergence loss yields 1) a (unnormalized) generative density, 2) an OOD detector, 3) a calibration temperature, 4) a generative sampler, along with in the supervised case 5) a distance aware-classifier. The Morse network can be used on top of a pre-trained network to bring distance-aware calibration w.r.t the training data. Because of its versatility, the Morse neural networks unifies many techniques: e.g., the Entropic Out-of-Distribution Detector of (Mac\^edo et al., 2021) in OOD detection, the one class Deep Support Vector Description method of (Ruff et al., 2018) in anomaly detection, or the Contrastive One Class classifier in continuous learning (Sun et al., 2021). The Morse neural network has connections to support vector machines, kernel methods, and Morse theory in topology.
摘要
我们介绍一个新的深度生成模型，用于不确定量化：Morse神经网络，它将高维子集模式扩展到非数字点的 Gaussian 分布中。通过 Morse 神经网络的适应损失函数，可以获得1）生成密度（尚未 норма化），2）外部数据检测器，3）整合温度，4）生成抽样器，并在监督学习情况下5）距离意识类别器。Morse 神经网络可以在对照数据进行训练后，将距离意识于训练数据。由于其多方面性，Morse 神经网络可以统一许多技术：例如 Entropic Out-of-Distribution Detector（Mac\^edo et al., 2021）在类别外数据检测中，One Class Deep Support Vector Description method（Ruff et al., 2018）在异常检测中，以及 Contrastive One Class 类别器在连续学习中（Sun et al., 2021）。Morse 神经网络与支持向量机制、核方法和 Morse 理论在数学上有联系。
Numerical Association Rule Mining: A Systematic Literature Review
results: 本研究通过对68篇论文进行深入的分析，提供了 numerical association rule mining领域中现有的多种方法、算法、 метрик和数据集，并发现了一些研究问题和未来可能性。此外，本研究还提出了一种新的某些精度抽象方法，可以帮助提高人类对数据的认知。Abstract
Numerical association rule mining is a widely used variant of the association rule mining technique, and it has been extensively used in discovering patterns and relationships in numerical data. Initially, researchers and scientists integrated numerical attributes in association rule mining using various discretization approaches; however, over time, a plethora of alternative methods have emerged in this field. Unfortunately, the increase of alternative methods has resulted into a significant knowledge gap in understanding diverse techniques employed in numerical association rule mining -- this paper attempts to bridge this knowledge gap by conducting a comprehensive systematic literature review. We provide an in-depth study of diverse methods, algorithms, metrics, and datasets derived from 1,140 scholarly articles published from the inception of numerical association rule mining in the year 1996 to 2022. In compliance with the inclusion, exclusion, and quality evaluation criteria, 68 papers were chosen to be extensively evaluated. To the best of our knowledge, this systematic literature review is the first of its kind to provide an exhaustive analysis of the current literature and previous surveys on numerical association rule mining. The paper discusses important research issues, the current status, and future possibilities of numerical association rule mining. On the basis of this systematic review, the article also presents a novel discretization measure that contributes by providing a partitioning of numerical data that meets well human perception of partitions.
摘要
numerically association rule mining 是一种广泛使用的 association rule mining 技术的变种，并在发现数据中的模式和关系方面得到了广泛应用。初始时，研究人员和科学家将数值属性 integrate 到 association rule mining 中使用了多种精炼方法；然而，随着时间的推移，这个领域中出现了一系列的替代方法。 unfortunately, 这些替代方法的出现导致了关于不同技术在数值 association rule mining 中的知识差距增加，这篇论文试图通过进行系统性的文献综述来填补这个知识差距。我们从1996年 numerical association rule mining 的开始时间到2022年进行了1,140篇学术论文的系统性综述。在符合包括、排除和质量评估标准的基础下，我们选择了68篇文献进行了深入的评估。据我们所知，这是首次对 numerical association rule mining 的现有文献和前期调查进行了系统性的综述。本文提出了一些重要的研究问题，现状和未来可能性，以及一种新的精炼度量，它可以为数值数据提供一种符合人类认知的分割。
Intra- & Extra-Source Exemplar-Based Style Synthesis for Improved Domain Generalization
results: 在不同类型的数据偏移（地理位置、天气、日夜等）下，实现了最多 12.4% 的 mIoU 提升，并且可以与 CNN 和 Transformer 模型相容，并且可以与其他域外适应性技术相结合使用。Abstract
The generalization with respect to domain shifts, as they frequently appear in applications such as autonomous driving, is one of the remaining big challenges for deep learning models. Therefore, we propose an exemplar-based style synthesis pipeline to improve domain generalization in semantic segmentation. Our method is based on a novel masked noise encoder for StyleGAN2 inversion. The model learns to faithfully reconstruct the image, preserving its semantic layout through noise prediction. Using the proposed masked noise encoder to randomize style and content combinations in the training set, i.e., intra-source style augmentation (ISSA) effectively increases the diversity of training data and reduces spurious correlation. As a result, we achieve up to $12.4\%$ mIoU improvements on driving-scene semantic segmentation under different types of data shifts, i.e., changing geographic locations, adverse weather conditions, and day to night. ISSA is model-agnostic and straightforwardly applicable with CNNs and Transformers. It is also complementary to other domain generalization techniques, e.g., it improves the recent state-of-the-art solution RobustNet by $3\%$ mIoU in Cityscapes to Dark Z\"urich. In addition, we demonstrate the strong plug-n-play ability of the proposed style synthesis pipeline, which is readily usable for extra-source exemplars e.g., web-crawled images, without any retraining or fine-tuning. Moreover, we study a new use case to indicate neural network's generalization capability by building a stylized proxy validation set. This application has significant practical sense for selecting models to be deployed in the open-world environment. Our code is available at \url{https://github.com/boschresearch/ISSA}.
摘要
“域域迁移总是深度学习模型的一大挑战，尤其在自动驾驶应用中出现的场景下。因此，我们提出了一种基于例子的风格合成管道，以提高域迁移的深度学习模型。我们的方法基于 StyleGAN2 的掩码噪音编码器，该模型能够准确地重建图像，保留其 semantic 布局。通过在训练集中随机采样样式和内容的组合，我们称之为内源样式增强（ISSA），可以提高训练数据的多样性，并降低偶极性。这使得我们在不同类型的数据迁移下 achieve 12.4% 的 mIoU 提升。ISSA 是模型无关的和简单应用于 CNN 和 Transformer 上。它还是其他域迁移技术的补充，例如 RobustNet 的最新状态态之一。此外，我们还证明了我们提posed的风格合成管道具有强大的插件与替换能力，可以在不需要重新训练或微调的情况下使用。此外，我们还研究了一种新的应用场景，即通过建立风格化代理验证集来评估神经网络的泛化能力。这种应用场景具有实际 significanc，可以帮助选择要部署在开放世界环境中的模型。我们的代码可以在 \url{https://github.com/boschresearch/ISSA} 中找到。”
Multiclass Boosting: Simple and Intuitive Weak Learning Criteria
paper_authors: Nataly Brukhim, Amit Daniely, Yishay Mansour, Shay Moran
for: 这个论文是为了推广到多类Setting中的推广。
methods: 这个论文引入了一种弱学习条件，用于描述多类分类的原始概念，即“slightly better than random guessing”。提供了一种简单有效的推广算法，不需要 realizability assumption，其样本和oracle复杂度上下文独立于类数。
results: 这个论文在理论应用中使用了新的推广技术，包括列PAC学习中的等价性、推广 для列学习者和多类PAC学习和列PAC学习的特征化。其中，我们提供了一种简化的分析方法，并实现了对大型列size的改进的错误 bound。Abstract
We study a generalization of boosting to the multiclass setting. We introduce a weak learning condition for multiclass classification that captures the original notion of weak learnability as being "slightly better than random guessing". We give a simple and efficient boosting algorithm, that does not require realizability assumptions and its sample and oracle complexity bounds are independent of the number of classes. In addition, we utilize our new boosting technique in several theoretical applications within the context of List PAC Learning. First, we establish an equivalence to weak PAC learning. Furthermore, we present a new result on boosting for list learners, as well as provide a novel proof for the characterization of multiclass PAC learning and List PAC learning. Notably, our technique gives rise to a simplified analysis, and also implies an improved error bound for large list sizes, compared to previous results.
摘要
我们研究了多类 boosting 的泛化。我们引入了一种多类分类中的弱学习条件， capture 了原始的弱学习概念，即“一些更好过 random 猜测”。我们给出了简单、高效的 boosting 算法，不需要 realizability 假设，其样本和oracle 复杂度上下文独立于类数。在此基础上，我们在list PAC 学习的上下文中应用了我们的新 boosting 技术。首先，我们证明了弱 PAC 学习的等价性。其次，我们提供了一个新的 boosting для列学习者结果，以及一种新的证明方式 для多类 PAC 学习和list PAC 学习的特征化。值得注意的是，我们的技术使得分析简化，同时还提供了大量列表大小时的改进的错误 bound。
Effects of Explanation Specificity on Passengers in Autonomous Driving
paper_authors: Daniel Omeiza, Raunak Bhattacharyya, Nick Hawes, Marina Jirotka, Lars Kunze
for: investigate the effects of natural language explanations’ specificity on passengers in autonomous driving
methods: extended an existing data-driven tree-based explainer algorithm by adding a rule-based option for explanation generation, and generated auditory natural language explanations with different levels of specificity (abstract and specific)
results: both abstract and specific explanations had similar positive effects on passengers’ perceived safety and the feeling of anxiety, but specific explanations influenced the desire of passengers to takeover driving control from the autonomous vehicle, while abstract explanations did not.Abstract
The nature of explanations provided by an explainable AI algorithm has been a topic of interest in the explainable AI and human-computer interaction community. In this paper, we investigate the effects of natural language explanations' specificity on passengers in autonomous driving. We extended an existing data-driven tree-based explainer algorithm by adding a rule-based option for explanation generation. We generated auditory natural language explanations with different levels of specificity (abstract and specific) and tested these explanations in a within-subject user study (N=39) using an immersive physical driving simulation setup. Our results showed that both abstract and specific explanations had similar positive effects on passengers' perceived safety and the feeling of anxiety. However, the specific explanations influenced the desire of passengers to takeover driving control from the autonomous vehicle (AV), while the abstract explanations did not. We conclude that natural language auditory explanations are useful for passengers in autonomous driving, and their specificity levels could influence how much in-vehicle participants would wish to be in control of the driving activity.
摘要
自然语言说明提供的AI算法的特点已经引起了解释AI和人机交互社区的关注。在这篇论文中，我们研究了自动驾驶中乘客受到自然语言说明的特定性的影响。我们将现有的数据驱动树型解释算法扩展为添加了规则型解释生成选项。我们生成了不同水平的具体性（抽象和具体）的听觉自然语言说明，并在N=39名参与者参与的内置式物理驾驶模拟设置下进行了一场人Subject用户研究。我们的结果显示，抽象和具体的说明都有类似的正面效果于乘客对自动驾驶车辆（AV）的感知安全和焦虑情况。然而，具体说明对乘客想要控制驾驶活动的愿望产生了影响，而抽象说明没有这种影响。我们结论认为，自然语言听觉说明对自动驾驶中的乘客非常有用，而其特定性水平可以影响乘客希望在驾驶活动中担当控制的程度。
Bidirectional Looking with A Novel Double Exponential Moving Average to Adaptive and Non-adaptive Momentum Optimizers
paper_authors: Yineng Chen, Zuchao Li, Lefei Zhang, Bo Du, Hai Zhao
for: 提高深度学习模型的训练效果和稳定性
methods: 提出了一种新的双向性优化器框架，包括DEMA变体和动态预测策略
results: 经过广泛的实验和理论证明，提出的\textsc{Admeta}优化器在多个任务上表现出色，比基础优化器和最新的竞争优化器更高效和稳定。Abstract
Optimizer is an essential component for the success of deep learning, which guides the neural network to update the parameters according to the loss on the training set. SGD and Adam are two classical and effective optimizers on which researchers have proposed many variants, such as SGDM and RAdam. In this paper, we innovatively combine the backward-looking and forward-looking aspects of the optimizer algorithm and propose a novel \textsc{Admeta} (\textbf{A} \textbf{D}ouble exponential \textbf{M}oving averag\textbf{E} \textbf{T}o \textbf{A}daptive and non-adaptive momentum) optimizer framework. For backward-looking part, we propose a DEMA variant scheme, which is motivated by a metric in the stock market, to replace the common exponential moving average scheme. While in the forward-looking part, we present a dynamic lookahead strategy which asymptotically approaches a set value, maintaining its speed at early stage and high convergence performance at final stage. Based on this idea, we provide two optimizer implementations, \textsc{AdmetaR} and \textsc{AdmetaS}, the former based on RAdam and the latter based on SGDM. Through extensive experiments on diverse tasks, we find that the proposed \textsc{Admeta} optimizer outperforms our base optimizers and shows advantages over recently proposed competitive optimizers. We also provide theoretical proof of these two algorithms, which verifies the convergence of our proposed \textsc{Admeta}.
摘要
优化器是深度学习成功的关键组件，它引导神经网络更新参数根据训练集的损失。SGD和Adam是经典的优化器，研究人员对其提出了许多变体，如SGDM和RAdam。在这篇论文中，我们创新地结合了优化器算法的反向和前向方面，并提出了一个新的\textsc{Admeta}优化器框架。在反向方面，我们提出了DEMA变体方案，它是由股票市场中的一个度量所 inspirited，用于取代常见的几何移动平均方案。而在前向方面，我们提出了一种动态预测策略，它在早期 stages maintains its speed and在 final stages reaches a set value，即使在极端情况下也能够保持高性能。基于这个想法，我们提供了两种优化器实现，\textsc{AdmetaR}和\textsc{AdmetaS}，前者基于RAdam，后者基于SGDM。通过对多种任务进行广泛的实验，我们发现了我们提posed的\textsc{Admeta}优化器在相比基础优化器和最新的竞争优化器之上具有优势。此外，我们还提供了这两个算法的理论证明，这证明了我们的提案的\textsc{Admeta}优化器的收敛性。
Variational Autoencoding Molecular Graphs with Denoising Diffusion Probabilistic Model
for: 这 paper 的目的是提出一种基于分布式生成模型的分子特征设计方法，以便在数据驱动药物发现中使用。
methods: 这 paper 使用了一种基于梯度滤波的 probabilistic 模型（DDPM），将分子结构转化为含层次结构的 probabilistic 特征。
results: 经过一些实验表明，这 paper 的方法可以在小样本大小下提供更好的分子性质预测性能和稳定性，比较现有方法更好。Abstract
In data-driven drug discovery, designing molecular descriptors is a very important task. Deep generative models such as variational autoencoders (VAEs) offer a potential solution by designing descriptors as probabilistic latent vectors derived from molecular structures. These models can be trained on large datasets, which have only molecular structures, and applied to transfer learning. Nevertheless, the approximate posterior distribution of the latent vectors of the usual VAE assumes a simple multivariate Gaussian distribution with zero covariance, which may limit the performance of representing the latent features. To overcome this limitation, we propose a novel molecular deep generative model that incorporates a hierarchical structure into the probabilistic latent vectors. We achieve this by a denoising diffusion probabilistic model (DDPM). We demonstrate that our model can design effective molecular latent vectors for molecular property prediction from some experiments by small datasets on physical properties and activity. The results highlight the superior prediction performance and robustness of our model compared to existing approaches.
摘要
在数据驱动的药物发现中，设计分子特征是非常重要的任务。深度生成模型如变换自动编码器（VAEs）提供了一种解决方案，通过将分子结构转换为抽象的 probabilistic 矩阵来设计特征。这些模型可以在大量数据上训练，并应用到传输学习。然而，通常的 VAE 模型假设 approximate posterior distribution 的纬度分布是一个简单的多变量 Gaussian 分布，这可能会限制表示隐藏特征的性能。为了超越这些限制，我们提出了一种新的分子深度生成模型，通过将 hierarchical 结构 incorporated 到 probabilistic 矩阵中来实现。我们通过 denoising diffusion probabilistic model（DDPM）来实现这一点。我们的实验表明，我们的模型可以从一些小 datasets 上预测分子性质，并且比既有方法表现出更好的预测性和稳定性。
Solving Linear Inverse Problems Provably via Posterior Sampling with Latent Diffusion Models
for: solves linear inverse problems using pre-trained latent diffusion models.
methods: leverages pre-trained latent diffusion models and provides provable sample recovery in a linear model setting.
results: outperforms previously proposed posterior sampling algorithms in various inpainting, denoising, deblurring, destriping, and super-resolution tasks.Here’s the full translation of the abstract in Simplified Chinese:
for: Linear inverse problems 的解决方案，使用预训练的潜在扩散模型。
methods: 基于预训练的潜在扩散模型，提供可证明样本恢复的线性模型设置下的算法分析。
results: 在多种缺失、降噪、滤波、除条、高清化和超解像等问题中，超越了之前的 posterior sampling 算法。Abstract
We present the first framework to solve linear inverse problems leveraging pre-trained latent diffusion models. Previously proposed algorithms (such as DPS and DDRM) only apply to pixel-space diffusion models. We theoretically analyze our algorithm showing provable sample recovery in a linear model setting. The algorithmic insight obtained from our analysis extends to more general settings often considered in practice. Experimentally, we outperform previously proposed posterior sampling algorithms in a wide variety of problems including random inpainting, block inpainting, denoising, deblurring, destriping, and super-resolution.
摘要
我团队首先提出了利用预训练的潜在扩散模型解决线性逆问题的框架。之前的方法（如DPS和DDRM）只适用于像素空间扩散模型。我们 theoretically analyzed our algorithm, showing provable sample recovery in a linear model setting. The algorithmic insight obtained from our analysis extends to more general settings often considered in practice. Experimentally, we outperform previously proposed posterior sampling algorithms in a wide variety of problems, including random inpainting, block inpainting, denoising, deblurring, destriping, and super-resolution.Here's the translation in Traditional Chinese:我团队首先提出了利用预训练的潜在扩散模型解决线性逆问题的框架。之前的方法（如DPS和DDRM）只适用于像素空间扩散模型。我们 theoretically analyzed our algorithm, showing provable sample recovery in a linear model setting. The algorithmic insight obtained from our analysis extends to more general settings often considered in practice. Experimentally, we outperform previously proposed posterior sampling algorithms in a wide variety of problems, including random inpainting, block inpainting, denoising, deblurring, destriping, and super-resolution.
Bounce: a Reliable Bayesian Optimization Algorithm for Combinatorial and Mixed Spaces
paper_authors: Leonard Papenmeier, Luigi Nardi, Matthias Poloczek
for: solve high-dimensional black-box functions with mixed and combinatorial input spaces
methods: uses a novel map of various variable types into nested embeddings of increasing dimensionality
results: reliably achieves and often improves upon state-of-the-art performance on a variety of high-dimensional problems.Abstract
Impactful applications such as materials discovery, hardware design, neural architecture search, or portfolio optimization require optimizing high-dimensional black-box functions with mixed and combinatorial input spaces. While Bayesian optimization has recently made significant progress in solving such problems, an in-depth analysis reveals that the current state-of-the-art methods are not reliable. Their performances degrade substantially when the unknown optima of the function do not have a certain structure. To fill the need for a reliable algorithm for combinatorial and mixed spaces, this paper proposes Bounce that relies on a novel map of various variable types into nested embeddings of increasing dimensionality. Comprehensive experiments show that Bounce reliably achieves and often even improves upon state-of-the-art performance on a variety of high-dimensional problems.
摘要
“影响力强大的应用，如材料发现、硬件设计、神经架构搜寻或资产优化，需要优化高维黑盒函数，其中输入空间为杂合和混合的。虽然 Bayesian 优化在最近已经做出了重要进步，但是一个深入分析表明，现今的州际状态艺术方法并不可靠。它们在未知最佳点不具有特定结构时表现很差。为了填补这些问题，本文提出了 Bounce，它基于一个新的变量类型对嵌入的方法。实验表明，Bounce 可靠地达到和有时甚至超越了现今状态艺术方法的性能。”Note: The translation is in Simplified Chinese, which is the standard writing system used in mainland China. If you need Traditional Chinese, please let me know.
The Forward-Forward Algorithm as a feature extractor for skin lesion classification: A preliminary study
results: 提出一种新的神经网络模型，即前进方法（FFA），可以在低功耗的 анаóg硬件上进行实现，并且可以与传统的反射征文法（BP）相结合，以实现更高的预测精度。Abstract
Skin cancer, a deadly form of cancer, exhibits a 23\% survival rate in the USA with late diagnosis. Early detection can significantly increase the survival rate, and facilitate timely treatment. Accurate biomedical image classification is vital in medical analysis, aiding clinicians in disease diagnosis and treatment. Deep learning (DL) techniques, such as convolutional neural networks and transformers, have revolutionized clinical decision-making automation. However, computational cost and hardware constraints limit the implementation of state-of-the-art DL architectures. In this work, we explore a new type of neural network that does not need backpropagation (BP), namely the Forward-Forward Algorithm (FFA), for skin lesion classification. While FFA is claimed to use very low-power analog hardware, BP still tends to be superior in terms of classification accuracy. In addition, our experimental results suggest that the combination of FFA and BP can be a better alternative to achieve a more accurate prediction.
摘要
皮肤癌，一种致命的癌症，在美国的诊断晚期时存在23%的存活率。早期发现可以显著提高存活率，并促进时间有序的治疗。医疗分析中，精准的生物医学影像分类是非常重要的，帮助临床医生在疾病诊断和治疗中做出更加准确的决策。深度学习（DL）技术，如卷积神经网络和转换器，已经革命化了临床决策自动化。然而，计算成本和硬件限制使得现状的DL体系难以实施。在这项工作中，我们探索了一种不需要反卷积（BP）的神经网络，即前进方法（FFA），用于皮肤变性分类。虽然FFA声称使用非常低功耗的Analog嵌入式硬件，但BP仍然在分类准确率方面具有优势。此外，我们的实验结果表明，将FFA和BP组合起来可以达到更高的准确预测。
Fraunhofer SIT at CheckThat! 2023: Mixing Single-Modal Classifiers to Estimate the Check-Worthiness of Multi-Modal Tweets
results: 该方法在 CheckThat! 2023 任务1A 中获得了一等奖，其在私有测试集上达到的 F1 分数为 0.7297。Abstract
The option of sharing images, videos and audio files on social media opens up new possibilities for distinguishing between false information and fake news on the Internet. Due to the vast amount of data shared every second on social media, not all data can be verified by a computer or a human expert. Here, a check-worthiness analysis can be used as a first step in the fact-checking pipeline and as a filtering mechanism to improve efficiency. This paper proposes a novel way of detecting the check-worthiness in multi-modal tweets. It takes advantage of two classifiers, each trained on a single modality. For image data, extracting the embedded text with an OCR analysis has shown to perform best. By combining the two classifiers, the proposed solution was able to place first in the CheckThat! 2023 Task 1A with an F1 score of 0.7297 achieved on the private test set.
摘要
<>translate "The option of sharing images, videos and audio files on social media opens up new possibilities for distinguishing between false information and fake news on the Internet. Due to the vast amount of data shared every second on social media, not all data can be verified by a computer or a human expert. Here, a check-worthiness analysis can be used as a first step in the fact-checking pipeline and as a filtering mechanism to improve efficiency. This paper proposes a novel way of detecting the check-worthiness in multi-modal tweets. It takes advantage of two classifiers, each trained on a single modality. For image data, extracting the embedded text with an OCR analysis has shown to perform best. By combining the two classifiers, the proposed solution was able to place first in the CheckThat! 2023 Task 1A with an F1 score of 0.7297 achieved on the private test set." into Simplified Chinese.中文简体版：社交媒体上分享图片、视频和音频文件的选项开启了新的方式来分辨假信息和 fake news 在互联网上。由于社交媒体上每秒分享的数据量太多，不能由计算机或人工专家所验证。这里，一种可验证性分析可以作为验证管道的第一步和效率提高的筛选机制。本文提出了一种新的多模态推断方法，利用两个分类器，每个分类器在不同的模式上训练。对图像数据，使用 OCR 分析提取嵌入的文本最佳。将两个分类器结合使用，提出的解决方案在 CheckThat! 2023 任务 1A 中取得了0.7297 的 F1 分数。
results: 在 private glioma 数据集上，提出的方法可以达到 88.38% 的 Dice 和 10.64 的 Hausdorff distance，比现状态对照方法提高了 4%，为 glioma 治疗带来更好的效果Abstract
According to the 2021 World Health Organization (WHO) Classification scheme for gliomas, glioma segmentation is a very important basis for diagnosis and genotype prediction. In general, 3D multimodal brain MRI is an effective diagnostic tool. In the past decade, there has been an increase in the use of machine learning, particularly deep learning, for medical images processing. Thanks to the development of foundation models, models pre-trained with large-scale datasets have achieved better results on a variety of tasks. However, for medical images with small dataset sizes, deep learning methods struggle to achieve better results on real-world image datasets. In this paper, we propose a cross-modality attention adapter based on multimodal fusion to fine-tune the foundation model to accomplish the task of glioma segmentation in multimodal MRI brain images with better results. The effectiveness of the proposed method is validated via our private glioma data set from the First Affiliated Hospital of Zhengzhou University (FHZU) in Zhengzhou, China. Our proposed method is superior to current state-of-the-art methods with a Dice of 88.38% and Hausdorff distance of 10.64, thereby exhibiting a 4% increase in Dice to segment the glioma region for glioma treatment.
摘要
(Simplified Chinese translation)根据2021年世界卫生组织（WHO）的分类方案，肿瘤分 segmentation 是诊断和基因预测的重要基础。一般来说，3D多Modal MRI 是脑部的有效诊断工具。过去十年，机器学习，特别是深度学习，在医疗图像处理领域得到了广泛的应用。感谢基模型的发展，预训练于大规模数据集的模型在多种任务上取得了更好的结果。然而，医疗图像中的小 dataset SIZE 下，深度学习方法在真实世界图像集上表现不佳。在这篇论文中，我们提出了跨Modal 注意力适配器，基于多Modal 融合来细化基模型，以实现多Modal MRI 脑部图像中的肿瘤分 segmentation 任务。我们验证了我们的提议方法的有效性，通过我们自己的私人肿瘤数据集，来自 Zhengzhou University First Affiliated Hospital（FHZU）在 Zhengzhou, China。我们的提议方法比现有状态的方法高出4%的Dice，即88.38%，并且 Hausdorff distance 为10.64，因此能够更好地 segmentation 肿瘤区域，为肿瘤治疗提供更好的基础。
HODINet: High-Order Discrepant Interaction Network for RGB-D Salient Object Detection
results: 对七个常用的数据集进行了广泛的实验，并达到了对 24 种state-of-the-art 方法的竞争性性能，并且在四个评价指标上显示了更高的精度和效率。Abstract
RGB-D salient object detection (SOD) aims to detect the prominent regions by jointly modeling RGB and depth information. Most RGB-D SOD methods apply the same type of backbones and fusion modules to identically learn the multimodality and multistage features. However, these features contribute differently to the final saliency results, which raises two issues: 1) how to model discrepant characteristics of RGB images and depth maps; 2) how to fuse these cross-modality features in different stages. In this paper, we propose a high-order discrepant interaction network (HODINet) for RGB-D SOD. Concretely, we first employ transformer-based and CNN-based architectures as backbones to encode RGB and depth features, respectively. Then, the high-order representations are delicately extracted and embedded into spatial and channel attentions for cross-modality feature fusion in different stages. Specifically, we design a high-order spatial fusion (HOSF) module and a high-order channel fusion (HOCF) module to fuse features of the first two and the last two stages, respectively. Besides, a cascaded pyramid reconstruction network is adopted to progressively decode the fused features in a top-down pathway. Extensive experiments are conducted on seven widely used datasets to demonstrate the effectiveness of the proposed approach. We achieve competitive performance against 24 state-of-the-art methods under four evaluation metrics.
摘要
An open-source deep learning algorithm for efficient and fully-automatic analysis of the choroid in optical coherence tomography
paper_authors: Jamie Burke, Justin Engelmann, Charlene Hamid, Megan Reid-Schachter, Tom Pearson, Dan Pugh, Neeraj Dhaun, Stuart King, Tom MacGillivray, Miguel O. Bernabeu, Amos Storkey, Ian J. C. MacCormick
For: The paper aims to develop a fully-automatic, open-source algorithm for choroidal segmentation in optical coherence tomography (OCT) data.* Methods: The authors used a dataset of 715 OCT B-scans from 3 clinical studies, and finetuned a pre-trained UNet with MobileNetV3 backbone to segment the choroid region.* Results: DeepGPET, the proposed algorithm, achieves excellent agreement with a clinically validated semi-automatic segmentation method (GPET) and reduces the processing time per image by a factor of 27.Here’s the information in Simplified Chinese text:
methods: 作者使用了715个OCT B-scan数据（82个研究subject，115个眼球）从3个临床研究，并对pre-trained的UNet with MobileNetV3 backbone进行了微调，以实现choroidregion的分割。
results: DeepGPET，提出的算法，与临床验证的半自动化分割方法（GPET）达到了极高的一致性（AUC=0.9994，Dice=0.9664；Pearson相关系数为0.8908和0.9082），同时将每个图像的处理时间从34.49秒（±15.09）降低到1.25秒（±0.10）。两种方法在临床专业人员的评价下表现相似，而且不需要人工干预。Abstract
Purpose: To develop an open-source, fully-automatic deep learning algorithm, DeepGPET, for choroid region segmentation in optical coherence tomography (OCT) data. Methods: We used a dataset of 715 OCT B-scans (82 subjects, 115 eyes) from 3 clinical studies related to systemic disease. Ground truth segmentations were generated using a clinically validated, semi-automatic choroid segmentation method, Gaussian Process Edge Tracing (GPET). We finetuned a UNet with MobileNetV3 backbone pre-trained on ImageNet. Standard segmentation agreement metrics, as well as derived measures of choroidal thickness and area, were used to evaluate DeepGPET, alongside qualitative evaluation from a clinical ophthalmologist. Results: DeepGPET achieves excellent agreement with GPET on data from 3 clinical studies (AUC=0.9994, Dice=0.9664; Pearson correlation of 0.8908 for choroidal thickness and 0.9082 for choroidal area), while reducing the mean processing time per image on a standard laptop CPU from 34.49s ($\pm$15.09) using GPET to 1.25s ($\pm$0.10) using DeepGPET. Both methods performed similarly according to a clinical ophthalmologist, who qualitatively judged a subset of segmentations by GPET and DeepGPET, based on smoothness and accuracy of segmentations. Conclusions :DeepGPET, a fully-automatic, open-source algorithm for choroidal segmentation, will enable researchers to efficiently extract choroidal measurements, even for large datasets. As no manual interventions are required, DeepGPET is less subjective than semi-automatic methods and could be deployed in clinical practice without necessitating a trained operator. DeepGPET addresses the lack of open-source, fully-automatic and clinically relevant choroid segmentation algorithms, and its subsequent public release will facilitate future choroidal research both in ophthalmology and wider systemic health.
摘要
目的：开发一个开源的自动化深度学习算法DeepGPET，用于光谱凝结 Tomatoes（OCT）数据中的choroid区域分割。方法：我们使用了715个OCT B-scan（82名病人，115只眼）从3个临床研究中获取数据，这些研究与系统疾病相关。我们使用了一种临床验证的、 semi-automatic choroid分割方法—— Gaussian Process Edge Tracing（GPET）生成了参考标准分割。我们使用了MobileNetV3预训练在ImageNet上的UNet，并对其进行了微调。我们使用了标准的分割一致度量和 derived的choroid厚度和面积度量来评估DeepGPET，并与一名临床眼科医生的评估。结果：DeepGPET与GPET在3个临床研究中达到了极高的一致度（AUC=0.9994，Dice=0.9664； Pearson相关度=0.8908 дляchoroid厚度和0.9082 дляchoroid面积），而同时减少了每个图像的平均处理时间从34.49秒（±15.09）使用GPET下降到1.25秒（±0.10）使用DeepGPET。两种方法在临床眼科医生的评估下表现相似，后者根据分割的平滑度和准确性进行质量评估。结论：DeepGPET是一个开源的、自动化的、临床相关的choroid分割算法，可以帮助研究人员快速提取choroid的测量数据，即使是大型数据集。由于不需要人工干预，DeepGPET比 semi-automatic方法更加 Objective，可以在临床实践中无需训练操作员而使用。DeepGPET解决了开源、自动化和临床相关的choroid分割算法的缺失，其后续的公共发布将促进未来choroid研究的进展，不仅在眼科领域，还在更广泛的系统医学领域。
Synthesis of Contrast-Enhanced Breast MRI Using Multi-b-Value DWI-based Hierarchical Fusion Network with Attention Mechanism
paper_authors: Tianyu Zhang, Luyi Han, Anna D’Angelo, Xin Wang, Yuan Gao, Chunyao Lu, Jonas Teuwen, Regina Beets-Tan, Tao Tan, Ritse Mann for:The paper aims to develop a multi-sequence fusion network to synthesize contrast-enhanced MRI (CE-MRI) based on T1-weighted MRI and diffusion-weighted imaging (DWI) to potentially reduce or avoid the use of gadolinium-based contrast agents (GBCA).methods:The proposed method uses a multi-sequence fusion network that combines T1-weighted MRI and DWIs with different b-values to efficiently utilize the difference features of DWIs. The method also includes a multi-sequence attention module to obtain refined feature maps and a weighted difference module to leverage hierarchical representation information fused at different scales.results:The results show that the multi-b-value DWI-based fusion model can potentially be used to synthesize CE-MRI, thus theoretically reducing or avoiding the use of GBCA, thereby minimizing the burden to patients.Abstract
Magnetic resonance imaging (MRI) is the most sensitive technique for breast cancer detection among current clinical imaging modalities. Contrast-enhanced MRI (CE-MRI) provides superior differentiation between tumors and invaded healthy tissue, and has become an indispensable technique in the detection and evaluation of cancer. However, the use of gadolinium-based contrast agents (GBCA) to obtain CE-MRI may be associated with nephrogenic systemic fibrosis and may lead to bioaccumulation in the brain, posing a potential risk to human health. Moreover, and likely more important, the use of gadolinium-based contrast agents requires the cannulation of a vein, and the injection of the contrast media which is cumbersome and places a burden on the patient. To reduce the use of contrast agents, diffusion-weighted imaging (DWI) is emerging as a key imaging technique, although currently usually complementing breast CE-MRI. In this study, we develop a multi-sequence fusion network to synthesize CE-MRI based on T1-weighted MRI and DWIs. DWIs with different b-values are fused to efficiently utilize the difference features of DWIs. Rather than proposing a pure data-driven approach, we invent a multi-sequence attention module to obtain refined feature maps, and leverage hierarchical representation information fused at different scales while utilizing the contributions from different sequences from a model-driven approach by introducing the weighted difference module. The results show that the multi-b-value DWI-based fusion model can potentially be used to synthesize CE-MRI, thus theoretically reducing or avoiding the use of GBCA, thereby minimizing the burden to patients. Our code is available at \url{https://github.com/Netherlands-Cancer-Institute/CE-MRI}.
摘要
磁共振成像（MRI）是当前临床成像技术中检测乳腺癌最敏感的方法。增强磁共振成像（CE-MRI）可提供较好的诊断和评估结果，但是使用加多林镁基contrast剂（GBCA）可能会导致肾脏系统 fibrosis和脑部堆积，危及人类健康。此外，使用contrast剂还需要胸部静脉填充和干扰性高，对患者造成困扰。为了减少contrast剂的使用，扩散成像（DWI）正在成为一种关键的成像技术，尽管目前通常作为乳腺CE-MRI的补充。在这项研究中，我们开发了一种多序列融合网络，将CE-MRI基于T1束重度成像和DWI的数据融合。DWI的不同b值被融合，以高效利用DWI的差异特征。而不是直接提出数据驱动的方法，我们创造了一种多序列注意模块，以获得精细的特征地图，并利用层次表示信息在不同级别上进行融合，同时利用不同序列的贡献。结果表明，多b值DWI基本融合模型可能可以Synthesize CE-MRI，从而可能避免或减少GBCA的使用，为患者减轻负担。我们的代码可以在 上获取。
An Explainable Deep Framework: Towards Task-Specific Fusion for Multi-to-One MRI Synthesis
results: 根据BraTS2021 dataset的1251个主题，这篇论文的方法比之前的方法更好，并且可以将缺失的序列合成为完整的序列。Abstract
Multi-sequence MRI is valuable in clinical settings for reliable diagnosis and treatment prognosis, but some sequences may be unusable or missing for various reasons. To address this issue, MRI synthesis is a potential solution. Recent deep learning-based methods have achieved good performance in combining multiple available sequences for missing sequence synthesis. Despite their success, these methods lack the ability to quantify the contributions of different input sequences and estimate the quality of generated images, making it hard to be practical. Hence, we propose an explainable task-specific synthesis network, which adapts weights automatically for specific sequence generation tasks and provides interpretability and reliability from two sides: (1) visualize the contribution of each input sequence in the fusion stage by a trainable task-specific weighted average module; (2) highlight the area the network tried to refine during synthesizing by a task-specific attention module. We conduct experiments on the BraTS2021 dataset of 1251 subjects, and results on arbitrary sequence synthesis indicate that the proposed method achieves better performance than the state-of-the-art methods. Our code is available at \url{https://github.com/fiy2W/mri_seq2seq}.
摘要
多序列MRI在临床设置中是有价值的，可以帮助确定疾病诊断和治疗预测，但某些序列可能无法使用或缺失。为解决这个问题，MRI合成是一个可能的解决方案。最新的深度学习基于方法在多个可用序列的组合中实现了好的性能。尽管它们在实际应用中具有成功，但它们缺乏对不同输入序列的贡献的评估和生成图像质量的估计，这使得它们在实践中具有困难。因此，我们提出了可解释的任务特定合成网络，它自动适应不同的序列生成任务，并提供了可读性和可靠性从两个方面：1. 在合并阶段，通过可训练的任务特定权重平均模块，可以视觑每个输入序列的贡献。2. 通过任务特定注意力模块，可以高亮生成过程中网络尝试修改的区域。我们在BraTS2021数据集上进行了1251个主题的实验，并得到了比 estado-of-the-art 方法更好的性能。我们的代码可以在 GitHub 上找到：\url{https://github.com/fiy2W/mri_seq2seq}。
End-To-End Prediction of Knee Osteoarthritis Progression With Multi-Modal Transformers
paper_authors: Egor Panfilov, Simo Saarakkala, Miika T. Nieminen, Aleksei Tiulpin
for: The paper aims to develop a unified framework for multi-modal fusion of knee imaging data to predict the progression of knee osteoarthritis (KOA) and provide new tools for the design of more efficient clinical trials.
methods: The authors use a Transformer approach to fuse structural knee MRI, multi-modal imaging data, and clinical data to predict KOA progression. They analyze the performance of their framework across different progression horizons and investigate the effectiveness of different modalities and subject subgroups.
results: The authors report that structural knee MRI can identify radiographic KOA progressors with an area under the ROC curve (ROC AUC) of 0.70-0.76 and Average Precision (AP) of 0.15-0.54 in 2-8 year horizons. They also find that multi-modal fusion of imaging data can predict KOA progression within 1 year with high accuracy (ROC AUC of 0.76(0.04), AP of 0.13(0.04)). Additionally, they identify post-traumatic subjects as the most accurate for prediction from imaging data.Abstract
Knee Osteoarthritis (KOA) is a highly prevalent chronic musculoskeletal condition with no currently available treatment. The manifestation of KOA is heterogeneous and prediction of its progression is challenging. Current literature suggests that the use of multi-modal data and advanced modeling methods, such as the ones based on Deep Learning, has promise in tackling this challenge. To date, however, the evidence on the efficacy of this approach is limited. In this study, we leveraged recent advances in Deep Learning and, using a Transformer approach, developed a unified framework for the multi-modal fusion of knee imaging data. Subsequently, we analyzed its performance across a range of scenarios by investigating multiple progression horizons -- from short-term to long-term. We report our findings using a large cohort (n=2421-3967) derived from the Osteoarthritis Initiative dataset. We show that structural knee MRI allows identifying radiographic KOA progressors on par with multi-modal fusion approaches, achieving an area under the ROC curve (ROC AUC) of 0.70-0.76 and Average Precision (AP) of 0.15-0.54 in 2-8 year horizons. Progression within 1 year was better predicted with a multi-modal method using X-ray, structural, and compositional MR images -- ROC AUC of 0.76(0.04), AP of 0.13(0.04) -- or via clinical data. Our follow-up analysis generally shows that prediction from the imaging data is more accurate for post-traumatic subjects, and we further investigate which subject subgroups may benefit the most. The present study provides novel insights into multi-modal imaging of KOA and brings a unified data-driven framework for studying its progression in an end-to-end manner, providing new tools for the design of more efficient clinical trials. The source code of our framework and the pre-trained models are made publicly available.
摘要
《骨 JOINT arthritis (KOA) 是一种非常普遍的慢性骨附着病，目前还没有任何有效的治疗方法。KOA 的表现具有各种各样的特征，预测其进程是非常困难的。当前的文献表明，使用多Modal 数据和高级模型方法，如基于深度学习的方法，有可能解决这一挑战。然而，至今为止，这种方法的有效性的证据还很有限。在这项研究中，我们利用了最新的深度学习技术，使用Transformer 方法，对骨 JOINT 成像数据进行多Modal 融合。然后，我们分析了其性能在不同的enario中，包括不同的进程时间 horizon （从短期到长期）。我们通过使用大量的 cohort （n=2421-3967）， derive 从骨 JOINT 创新数据集，报告我们的发现。我们发现，骨 JOINT 成像 MRI 可以与多Modal 融合方法一样准确地识别骨 JOINT 急性进程者，其 ROC AUC 在 0.70-0.76 之间，AP 在 0.15-0.54 之间。在 1 年内的进程预测中，使用 X-ray、骨 JOINT 结构和组成 MRI 的多Modal 方法可以达到 ROC AUC 的 0.76(0.04)，AP 的 0.13(0.04)。或者通过临床数据来预测。我们的跟踪分析通常表明，骨 JOINT 成像数据预测的准确性更高，特别是对post-traumatic 患者。我们进一步调查了哪些子群体可能受益最多。这项研究提供了关于骨 JOINT 多Modal 成像的新的意见，并提供了一个通用的数据驱动的框架，可以在一个端到端的方式来研究骨 JOINT 进程的预测，提供新的工具，用于设计更有效的临床试验。我们的框架和预训练模型的源代码公开可用。
Anisotropic Fanning Aware Low-Rank Tensor Approximation Based Tractography
methods: 该研究使用了 low-rank higher-order tensor approximation 技术，并将 anisotropic fanning 模型integrated into a recently proposed tractography method。
results: 研究结果表明，在12个人 connectome project Subjects中，该扩展模型可以增加 tract 的完teness和准确性，同时可以避免过度重建。此外，该模型的结果也比基于 Watson 分布的简单模型更为准确。Abstract
Low-rank higher-order tensor approximation has been used successfully to extract discrete directions for tractography from continuous fiber orientation density functions (fODFs). However, while it accounts for fiber crossings, it has so far ignored fanning, which has led to incomplete reconstructions. In this work, we integrate an anisotropic model of fanning based on the Bingham distribution into a recently proposed tractography method that performs low-rank approximation with an Unscented Kalman Filter. Our technical contributions include an initialization scheme for the new parameters, which is based on the Hessian of the low-rank approximation, pre-integration of the required convolution integrals to reduce the computational effort, and representation of the required 3D rotations with quaternions. Results on 12 subjects from the Human Connectome Project confirm that, in almost all considered tracts, our extended model significantly increases completeness of the reconstruction, while reducing excess, at acceptable additional computational cost. Its results are also more accurate than those from a simpler, isotropic fanning model that is based on Watson distributions.
摘要
低阶高级张量近似法已成功地提取维度分布函数（fODFs）中连续纤维方向的精确方向。然而，它虽然考虑到纤维交叉，但忽略了扩散，导致重建不准确。在这项工作中，我们将基于比频分布的扩散模型纳入最近提出的 tractography 方法中，并使用不确定 kalman 滤波器进行低阶近似。我们的技术贡献包括基于低阶近似的 Hessian 初始化方法，预处理所需的 convolution 积分，以及使用 quaternion 表示所需的 3D 旋转。results 表明，在12名人类连接度计划的试验Subject中，我们的扩展模型可以在大多数考虑的轨迹中提高重建的完整性，而且降低过量，acceptable的额外计算成本。它的结果还比基于 Watson 分布的简单的扩散模型更准确。
ACDMSR: Accelerated Conditional Diffusion Models for Single Image Super-Resolution
paper_authors: Axi Niu, Pham Xuan Trung, Kang Zhang, Jinqiu Sun, Yu Zhu, In So Kweon, Yanning Zhang
for: 这个论文是为了提高图像超分辨率（SR）的批处速度而设计的。
methods: 该方法基于标准扩散模型，通过确定性的迭代减噪过程来实现SR。
results: 对于多个标准 benchmark 数据集（Set5、Set14、Urban100、BSD100、Manga109）的实验结果表明，我们的方法可以超过前一个尝试，并且生成更加可见和实用的低分辨率图像的高分辨率对应图像。Abstract
Diffusion models have gained significant popularity in the field of image-to-image translation. Previous efforts applying diffusion models to image super-resolution (SR) have demonstrated that iteratively refining pure Gaussian noise using a U-Net architecture trained on denoising at various noise levels can yield satisfactory high-resolution images from low-resolution inputs. However, this iterative refinement process comes with the drawback of low inference speed, which strongly limits its applications. To speed up inference and further enhance the performance, our research revisits diffusion models in image super-resolution and proposes a straightforward yet significant diffusion model-based super-resolution method called ACDMSR (accelerated conditional diffusion model for image super-resolution). Specifically, our method adapts the standard diffusion model to perform super-resolution through a deterministic iterative denoising process. Our study also highlights the effectiveness of using a pre-trained SR model to provide the conditional image of the given low-resolution (LR) image to achieve superior high-resolution results. We demonstrate that our method surpasses previous attempts in qualitative and quantitative results through extensive experiments conducted on benchmark datasets such as Set5, Set14, Urban100, BSD100, and Manga109. Moreover, our approach generates more visually realistic counterparts for low-resolution images, emphasizing its effectiveness in practical scenarios.
摘要
Diffusion models 在图像到图像翻译领域得到了广泛的应用。先前的尝试使用 diffusion models 进行图像超解析（SR）已经证明了，通过多次使用 U-Net 架构训练在不同噪声水平上进行杜尼进行反射，可以从低解析输入图像中获得满意的高解析图像。然而，这个迭代纠正过程受到了推理速度的限制，这限制了其应用。为了加速推理和进一步提高性能，我们的研究重新评估了 diffusion models 在图像 SR 领域，并提出了一种简单但具有显著意义的 diffusion model-based SR 方法，称为 ACDMSR (加速条件扩散模型 для图像 SR)。我们的方法将标准的扩散模型应用于图像 SR 领域，通过一种决定性的迭代噪声纠正过程来进行 SR。我们的研究还发现了使用预训练 SR 模型提供给给定的低解析（LR）图像的 conditional 图像，可以实现更高的高解析结果。我们在 Set5、Set14、Urban100、BSD100 和 Manga109 等标准数据集上进行了广泛的实验，并证明了我们的方法在质量和量化方面的优越性。此外，我们的方法可以生成更加视觉真实的低解析图像，强调其在实际应用中的有效性。
Efficient Visual Fault Detection for Freight Train Braking System via Heterogeneous Self Distillation in the Wild
results: 实验结果表明，本文的方法可以在四个 fault 数据集上达到37帧每秒的速度，并保持最高准确率。相比传统的杂合方法，本文的方法具有较低的内存使用量和最小的模型大小。Abstract
Efficient visual fault detection of freight trains is a critical part of ensuring the safe operation of railways under the restricted hardware environment. Although deep learning-based approaches have excelled in object detection, the efficiency of freight train fault detection is still insufficient to apply in real-world engineering. This paper proposes a heterogeneous self-distillation framework to ensure detection accuracy and speed while satisfying low resource requirements. The privileged information in the output feature knowledge can be transferred from the teacher to the student model through distillation to boost performance. We first adopt a lightweight backbone to extract features and generate a new heterogeneous knowledge neck. Such neck models positional information and long-range dependencies among channels through parallel encoding to optimize feature extraction capabilities. Then, we utilize the general distribution to obtain more credible and accurate bounding box estimates. Finally, we employ a novel loss function that makes the network easily concentrate on values near the label to improve learning efficiency. Experiments on four fault datasets reveal that our framework can achieve over 37 frames per second and maintain the highest accuracy in comparison with traditional distillation approaches. Moreover, compared to state-of-the-art methods, our framework demonstrates more competitive performance with lower memory usage and the smallest model size.
摘要
高效的货运列车缺陷检测是铁路安全运行的关键部分，但深度学习方法在实际工程环境中的应用效率仍然不够高。这篇论文提出了一种多元自适应框架，以确保检测精度和速度同时满足资源限制。我们首先采用了轻量级的背bone来提取特征和生成新的多元知识颈部。这个颈部模型通过并行编码来保持通道之间的位势信息和长距离依赖关系，以便优化特征提取能力。然后，我们使用通用分布来获得更准确和可靠的 bounding box 估计。最后，我们使用一种新的损失函数，让网络更容易集中在标签附近的值上来提高学习效率。实验表明，我们的框架可以在四个缺陷集上达到37帧/秒的速度，并且与传统杜邦方法相比，我们的框架在精度和资源使用量上具有更高的竞争力。
paper_authors: Anshu Bhatia, Sanchit Sinha, Saket Dingliwal, Karthik Gopalakrishnan, Sravan Bodapati, Katrin Kirchhoff for: 实现自动话语识别（ASR） tasks中的表现，尤其是针对非标准的话者口音 population。methods: 使用自我超级vised learning方法从大量无标注的话语数据库中学习speech表现，并训练具有口音特点的residual adapter来自适应非标准的话者口音。results: 在4种口音中实现了强大的word error rate reduction（WERR），较HuBERT-large更好，具体来说是22.7%。此外，我们还证明了我们的方法是model和task-agnostic的。Abstract
Speech representations learned in a self-supervised fashion from massive unlabeled speech corpora have been adapted successfully toward several downstream tasks. However, such representations may be skewed toward canonical data characteristics of such corpora and perform poorly on atypical, non-native accented speaker populations. With the state-of-the-art HuBERT model as a baseline, we propose and investigate self-supervised adaptation of speech representations to such populations in a parameter-efficient way via training accent-specific residual adapters. We experiment with 4 accents and choose automatic speech recognition (ASR) as the downstream task of interest. We obtain strong word error rate reductions (WERR) over HuBERT-large for all 4 accents, with a mean WERR of 22.7% with accent-specific adapters and a mean WERR of 25.1% if the entire encoder is accent-adapted. While our experiments utilize HuBERT and ASR as the downstream task, our proposed approach is both model and task-agnostic.
摘要
自然语音训练集中学习的自我超vised的语音表示方式，经验到了多种下游任务的适应。然而，这些表示方式可能受到大量无标注语音训练集的典型数据特征的偏见，对非典型、非本地口音 speaker populations 表现不佳。基于当前顶峰 HuBERT 模型，我们提议并 investigate 自parameter-efficient 的方式，通过训练口音Specific residual adapter 来适应 speech 表示到这些 populations。我们实验了 4 种口音，选择了自动语音识别 (ASR) 作为下游任务。我们获得了所有 4 种口音的强大字误率减少 (WERR)，与 HuBERT-large 的mean WERR 相比， mean WERR 为 22.7%，全encoder 适应 WERR 为 25.1%。虽然我们的实验使用 HuBERT 和 ASR 作为下游任务，但我们提议的方法是模型和任务agnostic。
results: 研究发现，使用joint speaker encoder和phonetic posteriorgram可以实现高品质的跨语言语音转换，并且能够保持语音的自然性和特点。Abstract
Voice conversion systems have made significant advancements in terms of naturalness and similarity in common voice conversion tasks. However, their performance in more complex tasks such as cross-lingual voice conversion and expressive voice conversion remains imperfect. In this study, we propose a novel approach that combines a jointly trained speaker encoder and content features extracted from the cross-lingual speech recognition model Whisper to achieve high-quality cross-lingual voice conversion. Additionally, we introduce a speaker consistency loss to the joint encoder, which improves the similarity between the converted speech and the reference speech. To further explore the capabilities of the joint speaker encoder, we use the phonetic posteriorgram as the content feature, which enables the model to effectively reproduce both the speaker characteristics and the emotional aspects of the reference speech.
摘要
声音转换系统在日常声音转换任务中已经取得了显著的进步，但在跨语言声音转换和表情声音转换方面的表现仍然不够完美。在这项研究中，我们提出了一种新的方法， combinig a jointly trained speaker encoder和从跨语言语音识别模型Whisper提取的内容特征，以实现高质量的跨语言声音转换。此外，我们还添加了一个说话者一致性损失到联合编码器中，使模型能够更好地保持说话者的一致性。为了更好地探索联合说话者编码器的能力，我们使用了phonetic posteriorgram作为内容特征，这使得模型能够有效地复制参照语音中的说话者特征和情感特征。
For: This paper proposes a new architecture for vision called X-MLP, which is designed to be independent from convolutions and self-attention operations.* Methods: The X-MLP architecture is constructed absolutely upon fully connected layers and is free from patch embedding. It decouples the features extremely and utilizes MLPs to interact the information across the dimension of width, height, and channel independently and alternately.* Results: X-MLP is tested on ten benchmark datasets and obtains better performance than other vision MLP models. It even surpasses CNNs by a clear margin on various datasets. Additionally, the paper visualizes the information communication between any couples of pixels in the feature map and observes the phenomenon of capturing long-range dependency.Here is the simplified Chinese text for the three key points:* For: 这篇论文提出了一种新的视觉建模方法，即X-MLP，它不依赖卷积和自注意力操作。* Methods: X-MLP采用全连接层构建，不需要负权重映射。它强制Feature decoupling，通过多层Perceptron交互信息，独立地处理宽高和通道维度。* Results: X-MLP在十个基准数据集上进行测试，与其他视觉MLP模型相比，表现更好。甚至在不同的数据集上，X-MLP还超过了CNN。此外，通过数学还原空间权重，可以visualize任意像素对像的信息交换，观察长距离关系。Abstract
Convolutional neural networks (CNNs) and vision transformers (ViT) have obtained great achievements in computer vision. Recently, the research of multi-layer perceptron (MLP) architectures for vision have been popular again. Vision MLPs are designed to be independent from convolutions and self-attention operations. However, existing vision MLP architectures always depend on convolution for patch embedding. Thus we propose X-MLP, an architecture constructed absolutely upon fully connected layers and free from patch embedding. It decouples the features extremely and utilizes MLPs to interact the information across the dimension of width, height and channel independently and alternately. X-MLP is tested on ten benchmark datasets, all obtaining better performance than other vision MLP models. It even surpasses CNNs by a clear margin on various dataset. Furthermore, through mathematically restoring the spatial weights, we visualize the information communication between any couples of pixels in the feature map and observe the phenomenon of capturing long-range dependency.
摘要
卷积神经网络（CNN）和视Transformer（ViT）在计算机视觉领域已经取得了很大的成就。最近，视觉多层批处理（MLP）建筑的研究又再次受到了关注。视觉MLP建筑始终依赖于混合层和自我注意力操作。然而，现有的视觉MLP建筑都是通过混合层进行质心编码。因此，我们提出X-MLP，一种完全基于全连接层的建筑，不依赖于混合层和质心编码。它将特征分解得非常细致，并使用MLP进行特征之间的信息交互，在宽度、高度和通道维度上独立地交换信息。X-MLP在十个 benchmark 数据集上进行测试，都超过了其他视觉MLP模型。它甚至在不同的数据集上超过了 CNN。此外，通过数学还原空间权重，我们可以视觉化特征图中任意对的像素对之间的信息交换，并观察捕捉长距离相互作用。
ClipSitu: Effectively Leveraging CLIP for Conditional Predictions in Situation Recognition
results: 我们的结果显示，使用CLIP视觉token和架构设计的涉及式注意力Transformer（ClipSitu XTF）可以在semantic role labeling（值）任务中带来14.1%的提升，使得我们的模型在imSitu数据集上达到了最高的top-1准确率。Abstract
Situation Recognition is the task of generating a structured summary of what is happening in an image using an activity verb and the semantic roles played by actors and objects. In this task, the same activity verb can describe a diverse set of situations as well as the same actor or object category can play a diverse set of semantic roles depending on the situation depicted in the image. Hence model needs to understand the context of the image and the visual-linguistic meaning of semantic roles. Therefore, we leverage the CLIP foundational model that has learned the context of images via language descriptions. We show that deeper-and-wider multi-layer perceptron (MLP) blocks obtain noteworthy results for the situation recognition task by using CLIP image and text embedding features and it even outperforms the state-of-the-art CoFormer, a Transformer-based model, thanks to the external implicit visual-linguistic knowledge encapsulated by CLIP and the expressive power of modern MLP block designs. Motivated by this, we design a cross-attention-based Transformer using CLIP visual tokens that model the relation between textual roles and visual entities. Our cross-attention-based Transformer known as ClipSitu XTF outperforms existing state-of-the-art by a large margin of 14.1% on semantic role labelling (value) for top-1 accuracy using imSitu dataset. We will make the code publicly available.
摘要
Situation recognition是将图像中的活动和角色描述为结构化的摘要。在这个任务中，同一个活动词可以描述多种不同的情况，同一个actor或object category可以扮演多种不同的semantic role，具体取决于图像中所示的情况。因此，模型需要理解图像的上下文和视觉语言意义。为此，我们利用CLIP基础模型，它通过语言描述学习了图像的上下文。我们发现，在使用CLIP图像和文本嵌入特征的更深更宽多层感知（MLP）块时，可以获得优秀的结果，并在CoFormer，一种基于Transformer的模型，之上出现胜利。这是因为CLIP中的外部隐式视觉语言知识和现代MLP块设计的表达能力。为了进一步解决这个问题，我们设计了一个基于交叉关注的Transformer，使用CLIP的视觉token来模拟文本角色和视觉实体之间的关系。我们称之为ClipSitu XTF，它在imSitu数据集上的semantic role标签（值）的top-1准确率上超越了现有的状态艺术。我们将代码公开。
A multi-task learning framework for carotid plaque segmentation and classification from ultrasound images
results: 实验结果显示，提出的方法可以significantly提高与单个任务训练的网络性能，其中分类任务的准确率为 85.82%，分割任务的 dice similarity 系数为 84.92%。在减少研究中，结果表明了 RWM 和 SWM 都对网络性能的提高有所贡献。因此，我们认为该方法可以在临床试验和实践中用于血管壁瘤分析。Abstract
Carotid plaque segmentation and classification play important roles in the treatment of atherosclerosis and assessment for risk of stroke. Although deep learning methods have been used for carotid plaque segmentation and classification, most focused on a single task and ignored the relationship between the segmentation and classification of carotid plaques. Therefore, we propose a multi-task learning framework for ultrasound carotid plaque segmentation and classification, which utilizes a region-weight module (RWM) and a sample-weight module (SWM) to exploit the correlation between these two tasks. The RWM provides a plaque regional prior knowledge to the classification task, while the SWM is designed to learn the categorical sample weight for the segmentation task. A total of 1270 2D ultrasound images of carotid plaques were collected from Zhongnan Hospital (Wuhan, China) for our experiments. The results of the experiments showed that the proposed method can significantly improve the performance compared to existing networks trained for a single task, with an accuracy of 85.82% for classification and a Dice similarity coefficient of 84.92% for segmentation. In the ablation study, the results demonstrated that both the designed RWM and SWM were beneficial in improving the network's performance. Therefore, we believe that the proposed method could be useful for carotid plaque analysis in clinical trials and practice.
摘要
<>translate("Carotid plaque segmentation and classification play important roles in the treatment of atherosclerosis and assessment for risk of stroke. Although deep learning methods have been used for carotid plaque segmentation and classification, most focused on a single task and ignored the relationship between the segmentation and classification of carotid plaques. Therefore, we propose a multi-task learning framework for ultrasound carotid plaque segmentation and classification, which utilizes a region-weight module (RWM) and a sample-weight module (SWM) to exploit the correlation between these two tasks. The RWM provides a plaque regional prior knowledge to the classification task, while the SWM is designed to learn the categorical sample weight for the segmentation task. A total of 1270 2D ultrasound images of carotid plaques were collected from Zhongnan Hospital (Wuhan, China) for our experiments. The results of the experiments showed that the proposed method can significantly improve the performance compared to existing networks trained for a single task, with an accuracy of 85.82% for classification and a Dice similarity coefficient of 84.92% for segmentation. In the ablation study, the results demonstrated that both the designed RWM and SWM were beneficial in improving the network's performance. Therefore, we believe that the proposed method could be useful for carotid plaque analysis in clinical trials and practice.")Here's the translation:脉络栓分割和分类在血栓病治疗和风险评估中发挥重要作用。虽然深度学习方法已经用于脉络栓分割和分类，但大多数方法只关注单一任务，忽略了脉络栓分割和分类之间的关系。因此，我们提出了一种多任务学习框架 для超声脉络栓分割和分类，该框架利用区域权重模块（RWM）和样本权重模块（SWM）来利用这两个任务之间的相关性。RWM提供了脉络栓区域先验知识 для分类任务，而SWM是用于学习分类任务中的 categorical 样本权重。我们对1270个2D超声脉络栓图像进行了实验，结果显示，我们的方法可以与现有的单任务网络相比，提高表现，具体数据为85.82%的准确率和84.92%的相似度。在剥离研究中，结果表明，我们设计的RWM和SWM都对网络表现有益。因此，我们认为该方法在临床试验和实践中可能是有用的。
results: 这篇论文的结果显示，使用TinySiamese可以大幅降低训练时间和汇入时间，并且在鉴定和分类任务中获得了比标准Siamese更高的精度。Abstract
Biometric recognition is the process of verifying or classifying human characteristics in images or videos. It is a complex task that requires machine learning algorithms, including convolutional neural networks (CNNs) and Siamese networks. Besides, there are several limitations to consider when using these algorithms for image verification and classification tasks. In fact, training may be computationally intensive, requiring specialized hardware and significant computational resources to train and deploy. Moreover, it necessitates a large amount of labeled data, which can be time-consuming and costly to obtain. The main advantage of the proposed TinySiamese compared to the standard Siamese is that it does not require the whole CNN for training. In fact, using a pre-trained CNN as a feature extractor and the TinySiamese to learn the extracted features gave almost the same performance and efficiency as the standard Siamese for biometric verification. In this way, the TinySiamese solves the problems of memory and computational time with a small number of layers which did not exceed 7. It can be run under low-power machines which possess a normal GPU and cannot allocate a large RAM space. Using TinySiamese with only 8 GO of memory, the matching time decreased by 76.78% on the B2F (Biometric images of Fingerprints and Faces), FVC2000, FVC2002 and FVC2004 while the training time for 10 epochs went down by approximately 93.14% on the B2F, FVC2002, THDD-part1 and CASIA-B datasets. The accuracy of the fingerprint, gait (NM-angle 180 degree) and face verification tasks was better than the accuracy of a standard Siamese by 0.87%, 20.24% and 3.85% respectively. TinySiamese achieved comparable accuracy with related works for the fingerprint and gait classification tasks.
摘要
生物特征识别是将人体特征从图像或视频中鉴别或分类的过程。这是一项复杂的任务，需要机器学习算法，包括卷积神经网络（CNN）和同一个网络（Siamese）。然而，使用这些算法进行图像鉴别和分类任务时，存在一些限制。首先，训练可能会是 computationally intensive，需要特殊的硬件和大量计算资源来训练和部署。此外，需要大量标注数据，可能需要很长时间和成本来获得。与标准Siamese相比，我们提出的TinySiamese具有一些优点。首先，它不需要整个CNN进行训练。相反，使用预训练的CNN作为特征提取器，然后使用TinySiamese来学习提取的特征。这与标准Siamese的性能和效率几乎相同。此外，TinySiamese通过减少内存和计算时间的问题，使用只有7层的小型神经网络。这使得它可以在低功耗机器上运行，并且可以避免大量的内存和计算资源。使用TinySiamese，我们可以在10个epoch的训练时间下降约93.14%，而匹配时间下降约76.78%。此外，在B2F、FVC2000、FVC2002和FVC2004等数据集上，TinySiamese的精度比标准Siamese高出0.87%、20.24%和3.85%。此外，TinySiamese在指纹、步态（NM-angle 180度）和脸部鉴别任务中达到了相当的精度。
Bidirectional Temporal Diffusion Model for Temporally Consistent Human Animation
results: 我们的方法在实验中表现出了强大的表现，与现有的单向方法相比，具有真实的时间准确性。Abstract
We introduce a method to generate temporally coherent human animation from a single image, a video, or a random noise. This problem has been formulated as modeling of an auto-regressive generation, i.e., to regress past frames to decode future frames. However, such unidirectional generation is highly prone to motion drifting over time, generating unrealistic human animation with significant artifacts such as appearance distortion. We claim that bidirectional temporal modeling enforces temporal coherence on a generative network by largely suppressing the motion ambiguity of human appearance. To prove our claim, we design a novel human animation framework using a denoising diffusion model: a neural network learns to generate the image of a person by denoising temporal Gaussian noises whose intermediate results are cross-conditioned bidirectionally between consecutive frames. In the experiments, our method demonstrates strong performance compared to existing unidirectional approaches with realistic temporal coherence
摘要
我们提出了一种将单一图像、视频或随机变数转换为有着时间对称的人体动画的方法。这问题被视为一个自动复原生成问题，即从过去的几帧图像中预测未来几帧图像。但这种单向生成很容易受到时间漂移的影响，导致生成的人体动画有着严重的扭转现象和错误。我们声称，将时间方向统一到生成网络中可以严重抑制人体出现的动作抽象。为了证明我们的声明，我们设计了一个新的人体动画框架，使用一个对应推对推的滤除扩散模型：一个神经网络将从随机 Gaussian 噪声中获得人体图像，并在过去和未来几帧图像之间进行对应推对推的滤除过程。在实验中，我们的方法与现有的单向方法相比，具有更强的时间对称性和更真实的人体动画。
A MIL Approach for Anomaly Detection in Surveillance Videos from Multiple Camera Views
results: 对多个视角PETS-2009benchmark dataset进行重新标注，并使用多视角combined损失函数和Sultani的MIL排名函数来训练一个回归网络，得到了与单视角配置相比显著的F1分数提升。Abstract
Occlusion and clutter are two scene states that make it difficult to detect anomalies in surveillance video. Furthermore, anomaly events are rare and, as a consequence, class imbalance and lack of labeled anomaly data are also key features of this task. Therefore, weakly supervised methods are heavily researched for this application. In this paper, we tackle these typical problems of anomaly detection in surveillance video by combining Multiple Instance Learning (MIL) to deal with the lack of labels and Multiple Camera Views (MC) to reduce occlusion and clutter effects. In the resulting MC-MIL algorithm we apply a multiple camera combined loss function to train a regression network with Sultani's MIL ranking function. To evaluate the MC-MIL algorithm first proposed here, the multiple camera PETS-2009 benchmark dataset was re-labeled for the anomaly detection task from multiple camera views. The result shows a significant performance improvement in F1 score compared to the single-camera configuration.
摘要
干扰和堵塞是两种场景状态，使得异常检测在监视视频中具有棘手。此外，异常事件是罕见的，因此类别不平衡和缺乏异常数据也是这个任务的关键特点。因此，弱地监视方法受到了广泛的研究。在这篇论文中，我们解决了监视视频中异常检测中的常见问题，包括缺乏标签和干扰和堵塞的影响。我们提出了 MC-MIL 算法，该算法通过将多个摄像头视图组合成一个多视图损失函数，来训练一个 regression 网络，并使用 Sulani 的 MIL 排名函数。为评估 MC-MIL 算法，我们对多Camera PETS-2009 数据集进行了重新标注，以便进行异常检测任务。结果表明，相比单摄像头配置，MC-MIL 算法在 F1 分数上显著提高了性能。
Partial-label Learning with Mixed Closed-set and Open-set Out-of-candidate Examples
results: 对比州的PLL方法，本研究的方法得到了更高的性能。Abstract
Partial-label learning (PLL) relies on a key assumption that the true label of each training example must be in the candidate label set. This restrictive assumption may be violated in complex real-world scenarios, and thus the true label of some collected examples could be unexpectedly outside the assigned candidate label set. In this paper, we term the examples whose true label is outside the candidate label set OOC (out-of-candidate) examples, and pioneer a new PLL study to learn with OOC examples. We consider two types of OOC examples in reality, i.e., the closed-set/open-set OOC examples whose true label is inside/outside the known label space. To solve this new PLL problem, we first calculate the wooden cross-entropy loss from candidate and non-candidate labels respectively, and dynamically differentiate the two types of OOC examples based on specially designed criteria. Then, for closed-set OOC examples, we conduct reversed label disambiguation in the non-candidate label set; for open-set OOC examples, we leverage them for training by utilizing an effective regularization strategy that dynamically assigns random candidate labels from the candidate label set. In this way, the two types of OOC examples can be differentiated and further leveraged for model training. Extensive experiments demonstrate that our proposed method outperforms state-of-the-art PLL methods.
摘要
ARHNet: Adaptive Region Harmonization for Lesion-aware Augmentation to Improve Segmentation Performance
paper_authors: Jiayu Huo, Yang Liu, Xi Ouyang, Alejandro Granados, Sebastien Ourselin, Rachel Sparks
for: 提高MRI扫描图像中脑损伤的识别精度，以提供patients with prognoses和 neurosurgical monitoring。
methods: 使用CNN模型进行图像分割，并采用进步的数据增强策略来提高模型的可靠性。
results: 在实验中，ARHNet在实验中提高了下游分割性能，并在真实和Synthetic图像上达到了最高的表现。Abstract
Accurately segmenting brain lesions in MRI scans is critical for providing patients with prognoses and neurological monitoring. However, the performance of CNN-based segmentation methods is constrained by the limited training set size. Advanced data augmentation is an effective strategy to improve the model's robustness. However, they often introduce intensity disparities between foreground and background areas and boundary artifacts, which weakens the effectiveness of such strategies. In this paper, we propose a foreground harmonization framework (ARHNet) to tackle intensity disparities and make synthetic images look more realistic. In particular, we propose an Adaptive Region Harmonization (ARH) module to dynamically align foreground feature maps to the background with an attention mechanism. We demonstrate the efficacy of our method in improving the segmentation performance using real and synthetic images. Experimental results on the ATLAS 2.0 dataset show that ARHNet outperforms other methods for image harmonization tasks, and boosts the down-stream segmentation performance. Our code is publicly available at https://github.com/King-HAW/ARHNet.
摘要
优敏诊断患者的脑部病变需要高精度的MRI扫描图像分割，以提供病人诊断和 neuroscience 监测。然而，基于Convolutional Neural Network（CNN）的分割方法的性能受训练集大小的限制。高级数据增强技术可以提高模型的鲁棒性，但它们经常导致Intensity Disparity（ID）和边界artefacts，这会削弱这些策略的效果。本文提出了一种前景协调框架（ARHNet），以解决ID和Synthetic Image的不真实性问题。特别是，我们提出了一种适应区域协调（ARH）模块，通过关注机制来动态将前景特征图与背景图进行对齐。我们通过实验表明，ARHNet可以提高下游分割性能，并在ATLAS 2.0数据集上超越其他图像协调任务的方法。代码可以在https://github.com/King-HAW/ARHNet上获取。
Referring Video Object Segmentation with Inter-Frame Interaction and Cross-Modal Correlation
results: 经验 результаты表明，IFIRVOS在三个标准测试集上表现出了与state-of-the-art方法相比的superiority，并且 validate了我们提出的模块的有效性。Abstract
Referring video object segmentation (RVOS) aims to segment the target object in a video sequence described by a language expression. Typical query-based methods process the video sequence in a frame-independent manner to reduce the high computational cost, which however affects the performance due to the lack of inter-frame interaction for temporal coherence modeling and spatio-temporal representation learning of the referred object. Besides, they directly adopt the raw and high-level sentence feature as the language queries to decode the visual features, where the weak correlation between visual and linguistic features also increases the difficulty of decoding the target information and limits the performance of the model. In this paper, we proposes a novel RVOS framework, dubbed IFIRVOS, to address these issues. Specifically, we design a plug-and-play inter-frame interaction module in the Transformer decoder to efficiently learn the spatio-temporal features of the referred object, so as to decode the object information in the video sequence more precisely and generate more accurate segmentation results. Moreover, we devise the vision-language interaction module before the multimodal Transformer to enhance the correlation between the visual and linguistic features, thus facilitating the process of decoding object information from visual features by language queries in Transformer decoder and improving the segmentation performance. Extensive experimental results on three benchmarks validate the superiority of our IFIRVOS over state-of-the-art methods and the effectiveness of our proposed modules.
摘要
Traditional query-based methods for Referring Video Object Segmentation (RVOS) process the video sequence in a frame-independent manner, which can reduce computational cost but also affects performance due to the lack of inter-frame interaction for temporal coherence modeling and spatio-temporal representation learning of the referred object. Moreover, they directly use raw and high-level sentence features as language queries to decode visual features, which can lead to weak correlation between visual and linguistic features, limiting the performance of the model.In this paper, we propose a novel RVOS framework, called IFIRVOS, to address these issues. Specifically, we design a plug-and-play inter-frame interaction module in the Transformer decoder to efficiently learn spatio-temporal features of the referred object, allowing for more precise decoding of object information in the video sequence and generating more accurate segmentation results. Furthermore, we propose a vision-language interaction module before the multimodal Transformer to enhance the correlation between visual and linguistic features, facilitating the process of decoding object information from visual features using language queries in the Transformer decoder and improving segmentation performance.Experimental results on three benchmarks demonstrate the superiority of our IFIRVOS over state-of-the-art methods and the effectiveness of our proposed modules.
End-to-End Out-of-distribution Detection with Self-supervised Sampling
paper_authors: Sen Pei, Jiaxi Sun, Peng Qin, Qi Chen, Xinglong Wu, Xun Wang
for: 提高 Open World 中的 Out-of-distribution (OOD) 检测精度，帮助模型在关闭集合训练后在开放世界中识别未知数据。
methods: 提出一个通用概率框架来看待多种已有方法，并提出一种无需 OOD 数据的模型，即 Self-supervised Sampling for OOD Detection (SSOD)，利用了 convolution 操作的本地性来从 ID 数据中抽取自然的 OOD 信号，并同时优化 OOD 检测和正常 ID 分类。
results: 在多个大规模 benchmark 上实现了竞争力的状态平台，比如 SUN 上的 FPR95 下的性能提高了大约 48.99% 到 35.52% 之间，比最近的方法，如 KNN，有大幅度的优势。Abstract
Out-of-distribution (OOD) detection empowers the model trained on the closed set to identify unknown data in the open world. Though many prior techniques have yielded considerable improvements, two crucial obstacles still remain. Firstly, a unified perspective has yet to be presented to view the developed arts with individual designs, which is vital for providing insights into the related directions. Secondly, most research focuses on the post-processing schemes of the pre-trained features while disregarding the superiority of end-to-end training, dramatically limiting the upper bound of OOD detection. To tackle these issues, we propose a general probabilistic framework to interpret many existing methods and an OOD-data-free model, namely Self-supervised Sampling for OOD Detection (SSOD), to unfold the potential of end-to-end learning. SSOD efficiently exploits natural OOD signals from the in-distribution (ID) data based on the local property of convolution. With these supervisions, it jointly optimizes the OOD detection and conventional ID classification. Extensive experiments reveal that SSOD establishes competitive state-of-the-art performance on many large-scale benchmarks, where it outperforms the most recent approaches, such as KNN, by a large margin, e.g., 48.99% to 35.52% on SUN at FPR95.
摘要
SUGAR: Spherical Ultrafast Graph Attention Framework for Cortical Surface Registration
paper_authors: Jianxun Ren, Ning An, Youjia Zhang, Danyang Wang, Zhenyu Sun, Cong Lin, Weigang Cui, Weiwei Wang, Ying Zhou, Wei Zhang, Qingyu Hu, Ping Zhang, Dan Hu, Danhong Wang, Hesheng Liu for:SUGAR is designed to improve cortical surface registration, specifically addressing the challenge of aligning cortical functional and anatomical features across individuals.methods:SUGAR is a unified unsupervised deep-learning framework that incorporates a U-Net-based spherical graph attention network and leverages the Euler angle representation for deformation. The framework includes a similarity loss, fold loss, and multiple distortion losses to preserve topology and minimize various types of distortions.results:SUGAR exhibits comparable or superior registration performance in accuracy, distortion, and test-retest reliability compared to conventional and learning-based methods. Additionally, SUGAR achieves remarkable sub-second processing times, offering a notable speed-up of approximately 12,000 times in registering 9,000 subjects from the UK Biobank dataset in just 32 minutes.Abstract
Cortical surface registration plays a crucial role in aligning cortical functional and anatomical features across individuals. However, conventional registration algorithms are computationally inefficient. Recently, learning-based registration algorithms have emerged as a promising solution, significantly improving processing efficiency. Nonetheless, there remains a gap in the development of a learning-based method that exceeds the state-of-the-art conventional methods simultaneously in computational efficiency, registration accuracy, and distortion control, despite the theoretically greater representational capabilities of deep learning approaches. To address the challenge, we present SUGAR, a unified unsupervised deep-learning framework for both rigid and non-rigid registration. SUGAR incorporates a U-Net-based spherical graph attention network and leverages the Euler angle representation for deformation. In addition to the similarity loss, we introduce fold and multiple distortion losses, to preserve topology and minimize various types of distortions. Furthermore, we propose a data augmentation strategy specifically tailored for spherical surface registration, enhancing the registration performance. Through extensive evaluation involving over 10,000 scans from 7 diverse datasets, we showed that our framework exhibits comparable or superior registration performance in accuracy, distortion, and test-retest reliability compared to conventional and learning-based methods. Additionally, SUGAR achieves remarkable sub-second processing times, offering a notable speed-up of approximately 12,000 times in registering 9,000 subjects from the UK Biobank dataset in just 32 minutes. This combination of high registration performance and accelerated processing time may greatly benefit large-scale neuroimaging studies.
摘要
cortical surface registration 在跨个体标准化中扮演关键角色，但传统的注册算法 computationally inefficient。 recent learning-based registration algorithms 作为一个有前途的解决方案，Significantly improving processing efficiency。然而，there remains a gap in the development of a learning-based method that exceeds the state-of-the-art conventional methods simultaneously in computational efficiency, registration accuracy, and distortion control，despite the theoretically greater representational capabilities of deep learning approaches。To address the challenge，we present SUGAR，a unified unsupervised deep-learning framework for both rigid and non-rigid registration。SUGAR incorporates a U-Net-based spherical graph attention network and leverages the Euler angle representation for deformation。In addition to the similarity loss，we introduce fold and multiple distortion losses，to preserve topology and minimize various types of distortions。Furthermore，we propose a data augmentation strategy specifically tailored for spherical surface registration，enhancing the registration performance。Through extensive evaluation involving over 10,000 scans from 7 diverse datasets，we showed that our framework exhibits comparable or superior registration performance in accuracy, distortion, and test-retest reliability compared to conventional and learning-based methods。Additionally，SUGAR achieves remarkable sub-second processing times，offering a notable speed-up of approximately 12,000 times in registering 9,000 subjects from the UK Biobank dataset in just 32 minutes。This combination of high registration performance and accelerated processing time may greatly benefit large-scale neuroimaging studies。
Data-Free Quantization via Mixed-Precision Compensation without Fine-Tuning
results: 实验表明，DF-MPC 能够在无数据和 fine-tuning 过程的情况下，提高模型的精度，比较有效率。Abstract
Neural network quantization is a very promising solution in the field of model compression, but its resulting accuracy highly depends on a training/fine-tuning process and requires the original data. This not only brings heavy computation and time costs but also is not conducive to privacy and sensitive information protection. Therefore, a few recent works are starting to focus on data-free quantization. However, data-free quantization does not perform well while dealing with ultra-low precision quantization. Although researchers utilize generative methods of synthetic data to address this problem partially, data synthesis needs to take a lot of computation and time. In this paper, we propose a data-free mixed-precision compensation (DF-MPC) method to recover the performance of an ultra-low precision quantized model without any data and fine-tuning process. By assuming the quantized error caused by a low-precision quantized layer can be restored via the reconstruction of a high-precision quantized layer, we mathematically formulate the reconstruction loss between the pre-trained full-precision model and its layer-wise mixed-precision quantized model. Based on our formulation, we theoretically deduce the closed-form solution by minimizing the reconstruction loss of the feature maps. Since DF-MPC does not require any original/synthetic data, it is a more efficient method to approximate the full-precision model. Experimentally, our DF-MPC is able to achieve higher accuracy for an ultra-low precision quantized model compared to the recent methods without any data and fine-tuning process.
摘要
neural network 量化是一种非常有前途的解决方案在模型压缩领域，但它的结果准确度具有训练/精度调整过程和原始数据的依赖关系。这不仅带来了重量计算和时间成本，还不利于隐私信息保护。因此，一些最近的研究开始关注无数据量化。然而，无数据量化在ultra-low precision量化时不太好进行。虽然研究人员利用生成方法生成的假数据来解决这个问题，但数据生成需要很多计算和时间。在这篇论文中，我们提出了一种无数据混合精度补偿方法（DF-MPC）来恢复ultra-low precision量化模型的性能。我们假设量化错误由low-precision量化层引起的可以通过高精度量化层的重建来恢复。我们使用数学形式来表述重建loss между预训练全精度模型和它的层 wise混合精度量化模型。根据我们的形式，我们可以 theoretically得出closed-form解。由于DF-MPC不需要任何原始/生成数据，它是一种更高效的方法来近似全精度模型。实验表明，我们的DF-MPC可以在ultra-low precision量化模型中达到更高的准确度，比悉些最近的方法不需要任何数据和精度调整过程。
TopicFM+: Boosting Accuracy and Efficiency of Topic-Assisted Feature Matching
results: 我们通过了广泛的实验，证明了我们的方法在难以处理的场景中具有显著的优势。具体来说，我们的方法可以减少计算成本，同时保持高效性和高匹配精度。代码将在https://github.com/TruongKhang/TopicFM 上更新。Abstract
This study tackles the challenge of image matching in difficult scenarios, such as scenes with significant variations or limited texture, with a strong emphasis on computational efficiency. Previous studies have attempted to address this challenge by encoding global scene contexts using Transformers. However, these approaches suffer from high computational costs and may not capture sufficient high-level contextual information, such as structural shapes or semantic instances. Consequently, the encoded features may lack discriminative power in challenging scenes. To overcome these limitations, we propose a novel image-matching method that leverages a topic-modeling strategy to capture high-level contexts in images. Our method represents each image as a multinomial distribution over topics, where each topic represents a latent semantic instance. By incorporating these topics, we can effectively capture comprehensive context information and obtain discriminative and high-quality features. Additionally, our method effectively matches features within corresponding semantic regions by estimating the covisible topics. To enhance the efficiency of feature matching, we have designed a network with a pooling-and-merging attention module. This module reduces computation by employing attention only on fixed-sized topics and small-sized features. Through extensive experiments, we have demonstrated the superiority of our method in challenging scenarios. Specifically, our method significantly reduces computational costs while maintaining higher image-matching accuracy compared to state-of-the-art methods. The code will be updated soon at https://github.com/TruongKhang/TopicFM
摘要
To overcome these limitations, we propose a novel image-matching method that leverages a topic-modeling strategy to capture high-level contexts in images. Our method represents each image as a multinomial distribution over topics, where each topic represents a latent semantic instance. By incorporating these topics, we can effectively capture comprehensive context information and obtain discriminative and high-quality features. Additionally, our method effectively matches features within corresponding semantic regions by estimating the covisible topics.To enhance the efficiency of feature matching, we have designed a network with a pooling-and-merging attention module. This module reduces computation by employing attention only on fixed-sized topics and small-sized features. Through extensive experiments, we have demonstrated the superiority of our method in challenging scenarios. Specifically, our method significantly reduces computational costs while maintaining higher image-matching accuracy compared to state-of-the-art methods. The code will be updated soon at .
Seeing is not Believing: An Identity Hider for Human Vision Privacy Protection
methods: 使用 StyleGAN2 manipulate 原始 face 的 latent space，生成虚拟 face，并将虚拟 face 的视觉内容传播到原始 face 上，然后替换背景。
results: 提出了一种可以具有出色隐私保护和人脸识别稳定性的身份隐藏方法，并通过实验证明其性能优秀。Abstract
Massive captured face images are stored in the database for the identification of individuals. However, the stored images can be observed intentionally or unintentionally by data managers, which is not at the will of individuals and may cause privacy violations. Existing protection works only slightly change the visual content of the face while maintaining the utility of identification, making it susceptible to the inference of the true identity by human vision. In this paper, we propose an identity hider that enables significant visual content change for human vision while preserving high identifiability for face recognizers. Firstly, the identity hider generates a virtual face with new visual content by manipulating the latent space in StyleGAN2. In particular, the virtual face has the same irrelevant attributes as the original face, e.g., pose and expression. Secondly, the visual content of the virtual face is transferred into the original face and then the background is replaced with the original one. In addition, the identity hider has strong transferability, which ensures an arbitrary face recognizer can achieve satisfactory accuracy. Adequate experiments show that the proposed identity hider achieves excellent performance on privacy protection and identifiability preservation.
摘要
巨大的捕捉到的面部图像被存储在数据库中，用于个体识别。然而，存储的图像可能会被意外或非意外地被数据管理人员见到，这会导致个人隐私被侵犯。现有的保护措施只是轻度地改变视觉内容，保留了识别功能，这使得真实身份易于推测。在这篇论文中，我们提出了一种隐身器，允许对人类视觉来源进行显著视觉内容变化，同时保留高度识别性。首先，隐身器使用StyleGAN2中的latent space进行抽象，生成一个虚拟面，该虚拟面具有与原始面相同的无关属性，例如姿势和表情。其次，虚拟面中的视觉内容被转移到原始面上，并将背景替换为原始的背景。此外，隐身器具有强大的传送性，使得任何面Recognizer都可以达到满意的准确率。经过合适的实验，我们的隐身器实现了出色的隐私保护和识别性 preserved。
Domain Transfer Through Image-to-Image Translation for Uncertainty-Aware Prostate Cancer Classification
results: 研究结果显示，这个新方法可以将AUC提高到20%以上，与先前的研究相比（98.4% vs. 76.2%）。这显示了这个方法的可行性和有效性。Abstract
Prostate Cancer (PCa) is often diagnosed using High-resolution 3.0 Tesla(T) MRI, which has been widely established in clinics. However, there are still many medical centers that use 1.5T MRI units in the actual diagnostic process of PCa. In the past few years, deep learning-based models have been proven to be efficient on the PCa classification task and can be successfully used to support radiologists during the diagnostic process. However, training such models often requires a vast amount of data, and sometimes it is unobtainable in practice. Additionally, multi-source MRIs can pose challenges due to cross-domain distribution differences. In this paper, we have presented a novel approach for unpaired image-to-image translation of prostate mp-MRI for classifying clinically significant PCa, to be applied in data-constrained settings. First, we introduce domain transfer, a novel pipeline to translate unpaired 3.0T multi-parametric prostate MRIs to 1.5T, to increase the number of training data. Second, we estimate the uncertainty of our models through an evidential deep learning approach; and leverage the dataset filtering technique during the training process. Furthermore, we introduce a simple, yet efficient Evidential Focal Loss that incorporates the focal loss with evidential uncertainty to train our model. Our experiments demonstrate that the proposed method significantly improves the Area Under ROC Curve (AUC) by over 20% compared to the previous work (98.4% vs. 76.2%). We envision that providing prediction uncertainty to radiologists may help them focus more on uncertain cases and thus expedite the diagnostic process effectively. Our code is available at https://github.com/med-i-lab/DT_UE_PCa
摘要
prostato cancer (PCa) oftener diagnosed using High-resolution 3.0 Tesla(T) MRI, which has been widely established in clinics. However, there are still many medical centers that use 1.5T MRI units in the actual diagnostic process of PCa. In the past few years, deep learning-based models have been proven to be efficient on the PCa classification task and can be successfully used to support radiologists during the diagnostic process. However, training such models often requires a vast amount of data, and sometimes it is unobtainable in practice. Additionally, multi-source MRIs can pose challenges due to cross-domain distribution differences. In this paper, we have presented a novel approach for unpaired image-to-image translation of prostate mp-MRI for classifying clinically significant PCa, to be applied in data-constrained settings. First, we introduce domain transfer, a novel pipeline to translate unpaired 3.0T multi-parametric prostate MRIs to 1.5T, to increase the number of training data. Second, we estimate the uncertainty of our models through an evidential deep learning approach; and leverage the dataset filtering technique during the training process. Furthermore, we introduce a simple, yet efficient Evidential Focal Loss that incorporates the focal loss with evidential uncertainty to train our model. Our experiments demonstrate that the proposed method significantly improves the Area Under ROC Curve (AUC) by over 20% compared to the previous work (98.4% vs. 76.2%). We envision that providing prediction uncertainty to radiologists may help them focus more on uncertain cases and thus expedite the diagnostic process effectively. Our code is available at https://github.com/med-i-lab/DT_UE_PCa.
for: This paper is written to explore and improve the efficiency of black-box patch attacks on deep neural networks (DNNs), specifically in the decision-based setting.
methods: The paper proposes a new method called DevoPatch, which uses a differential evolutionary algorithm to optimize patches for black-box patch attacks. The method models patches using paired key-points and uses targeted images as the initialization of patches, and parameter optimizations are all performed on the integer domain.
results: The paper demonstrates that DevoPatch outperforms state-of-the-art black-box patch attacks in terms of patch area and attack success rate within a given query budget on image classification and face verification. Additionally, the paper conducts the vulnerability evaluation of ViT and MLP on image classification in the decision-based patch attack setting for the first time.Abstract
Deep neural networks (DNNs) have been showed to be highly vulnerable to imperceptible adversarial perturbations. As a complementary type of adversary, patch attacks that introduce perceptible perturbations to the images have attracted the interest of researchers. Existing patch attacks rely on the architecture of the model or the probabilities of predictions and perform poorly in the decision-based setting, which can still construct a perturbation with the minimal information exposed -- the top-1 predicted label. In this work, we first explore the decision-based patch attack. To enhance the attack efficiency, we model the patches using paired key-points and use targeted images as the initialization of patches, and parameter optimizations are all performed on the integer domain. Then, we propose a differential evolutionary algorithm named DevoPatch for query-efficient decision-based patch attacks. Experiments demonstrate that DevoPatch outperforms the state-of-the-art black-box patch attacks in terms of patch area and attack success rate within a given query budget on image classification and face verification. Additionally, we conduct the vulnerability evaluation of ViT and MLP on image classification in the decision-based patch attack setting for the first time. Using DevoPatch, we can evaluate the robustness of models to black-box patch attacks. We believe this method could inspire the design and deployment of robust vision models based on various DNN architectures in the future.
摘要
Translation notes:* "Deep neural networks" is translated as "深度神经网络" (shēn dào shén zhī wǎng luò)* "adversarial perturbations" is translated as "敌对扰动" (dí zhòu zhào dòng)* "patch attacks" is translated as "质子攻击" (zhì zǐ gōng jī)* "decision-based setting" is translated as "决策基础设定" (jīe yì jī bǎsè)* "top-1 predicted label" is translated as "预测的第一个标签" (yù jí de dì yī gè biǎo)* "DevoPatch" is translated as "DevoPatch" (德朋补丁)* "query-efficient" is translated as "高效的查询" (gāo fáng de kè qiú)* "black-box patch attacks" is translated as "黑盒质子攻击" (hēi bāo zhì zǐ gōng jī)* "ViT" and "MLP" are translated as "ViT" and "MLP" (Vision Transformer and Multi-Layer Perceptron) respectively.
Weighted Anisotropic-Isotropic Total Variation for Poisson Denoising
results: numerical experiments显示，我们的算法在影像质量和计算效率方面都比其他Poisson噪声去除方法表现更好。Abstract
Poisson noise commonly occurs in images captured by photon-limited imaging systems such as in astronomy and medicine. As the distribution of Poisson noise depends on the pixel intensity value, noise levels vary from pixels to pixels. Hence, denoising a Poisson-corrupted image while preserving important details can be challenging. In this paper, we propose a Poisson denoising model by incorporating the weighted anisotropic-isotropic total variation (AITV) as a regularization. We then develop an alternating direction method of multipliers with a combination of a proximal operator for an efficient implementation. Lastly, numerical experiments demonstrate that our algorithm outperforms other Poisson denoising methods in terms of image quality and computational efficiency.
摘要
POisson 噪声通常发生在由光子限制的摄影系统中，如天文学和医学中的图像摄影。由于噪声分布取决于像素Intensity值，噪声水平各像素不同，因此去噪Poisson损带的图像保持重要细节是有挑战性的。在这篇论文中，我们提出了一种含有加重weighted anisotropic-isotropic总变量(AITV)的Poisson去噪模型。然后，我们开发了一种alternating direction method of multipliers，并使用距离算子进行高效实现。最后，数值实验表明，我们的算法在图像质量和计算效率方面比其他Poisson去噪方法更高效。Note: "Poisson denoising" is translated as "去噪Poisson" in Simplified Chinese, which is a common way to refer to the process of removing Poisson noise from an image.
One Copy Is All You Need: Resource-Efficient Streaming of Medical Imaging Data at Scale
results: 研究发现，使用MIST可以对医疗影像储存和流程实现大量数据储存和网络带宽的节省，并且可以维持深度学习应用的诊断质量。Abstract
Large-scale medical imaging datasets have accelerated development of artificial intelligence tools for clinical decision support. However, the large size of these datasets is a bottleneck for users with limited storage and bandwidth. Many users may not even require such large datasets as AI models are often trained on lower resolution images. If users could directly download at their desired resolution, storage and bandwidth requirements would significantly decrease. However, it is impossible to anticipate every users' requirements and impractical to store the data at multiple resolutions. What if we could store images at a single resolution but send them at different ones? We propose MIST, an open-source framework to operationalize progressive resolution for streaming medical images at multiple resolutions from a single high-resolution copy. We demonstrate that MIST can dramatically reduce imaging infrastructure inefficiencies for hosting and streaming medical images by >90%, while maintaining diagnostic quality for deep learning applications.
摘要
大规模医疗影像数据集已经推动人工智能工具的开发，以支持临床决策。然而，这些大规模数据集的尺寸成为用户储存和带宽限制的瓶颈。许多用户可能不需要这么大的数据集，因为人工智能模型通常在较低分辨率的图像上训练。如果用户可以直接下载他们所需的分辨率，储存和带宽要求就会减少很多。然而，预测每个用户的需求是不可能的，并且不可能将数据存储在多个分辨率下。我们提出了MIST框架，一个开源的框架，用于在多个分辨率下进行进程式分辨率的流动医疗影像传输。我们示示了MIST可以减少医疗影像基础设施的不效ientecies >90%，同时保持深度学习应用的诊断质量。
results: 对各种图像特征（如颜色、文本、噪声、大小）的分析，发现攻击袋 exhibit 强大的彩度相对性和颜色传输异常，并且对噪声有较强的抗性。基于这些发现，提出了进一步减少袋颜色的方法。Abstract
Adversarial attack patches have gained increasing attention due to their practical applicability in physical-world scenarios. However, the bright colors used in attack patches represent a significant drawback, as they can be easily identified by human observers. Moreover, even though these attacks have been highly successful in deceiving target networks, which specific features of the attack patch contribute to its success are still unknown. Our paper introduces a brightness-restricted patch (BrPatch) that uses optical characteristics to effectively reduce conspicuousness while preserving image independence. We also conducted an analysis of the impact of various image features (such as color, texture, noise, and size) on the effectiveness of an attack patch in physical-world deployment. Our experiments show that attack patches exhibit strong redundancy to brightness and are resistant to color transfer and noise. Based on our findings, we propose some additional methods to further reduce the conspicuousness of BrPatch. Our findings also explain the robustness of attack patches observed in physical-world scenarios.
摘要
侵略攻击贴图在实际场景中获得了越来越多的关注，但是侵略贴图中使用的鲜明颜色带来了一定的缺点，因为它们可以轻易被人类识别。尽管这些攻击已经在目标网络中取得了高度的成功，但是哪些特征使得攻击贴图成功还不清楚。我们的论文介绍了一种具有限制颜色的攻击贴图（BrPatch），使用光学特性来有效地减少鲜明程度，保持图像独立性。我们还进行了图像特征（如颜色、文本、噪声和大小）对攻击贴图的影响分析。我们的实验表明，攻击贴图具有强大的颜色减少和噪声抗耗特性。基于我们的发现，我们提出了进一步减少鲜明度的方法。我们的发现也解释了实际场景中观察到的攻击贴图 Robustness。
Applications of Binary Similarity and Distance Measures
paper_authors: Manoj Muniswamaiah, Tilak Agerwala, Charles C. Tappert
for: 本研究探讨了二进制相似度测量在不同领域的应用。
methods: 本研究使用了二进制距离度量法和相似度测量方法。
results: 研究发现，二进制相似度测量在生物特征识别、手写字符识别和复眼图像识别等领域具有广泛的应用前景。Here’s a breakdown of each point:
for: 本研究探讨了二进制相似度测量在不同领域的应用。 (The paper explores the application of binary similarity measures in various fields.)
methods: 本研究使用了二进制距离度量法和相似度测量方法。 (The study uses binary distance measures and similarity measurement methods.)
results: 研究发现，二进制相似度测量在生物特征识别、手写字符识别和复眼图像识别等领域具有广泛的应用前景。 (The study finds that binary similarity measures have broad applications in fields such as biometric identification, handwritten character recognition, and iris image recognition.)Abstract
In the recent past, binary similarity measures have been applied in solving biometric identification problems, including fingerprint, handwritten character detection, and in iris image recognition. The application of the relevant measurements has also resulted in more accurate data analysis. This paper surveys the applicability of binary similarity and distance measures in various fields.
摘要
Note:* "二进制" (er-jie) means "binary" in Chinese.* "相似度度量" (xiang-si-duo-liang) means "similarity measures" or "distance measures" in Chinese.* "生物метриiddle" (shēng-wù-métài) means "biometric identification" in Chinese.* "指纹" (zhǐ-yīn) means "fingerprint" in Chinese.* "手写字符" (shǒu-xī-zi-fu) means "handwritten character" in Chinese.* "远景图像" (yuán-jǐng-tú-xìng) means "iris image" in Chinese.
Improving CNN-based Person Re-identification using score Normalization
For: 本文提出了一种新的人识别方法，用于解决多视角和不同照明和背景的问题。* Methods: 本文使用了卷积神经网络（CNN）来提取特征，并使用了交叉视角Quadratic Discriminant Analysis（XQDA）来学习度量。* Results: 在四个复杂的数据集上进行测试，包括VIPer、GRID、CUHK01和PRID450S，并取得了承诺的结果，例如在GRID、CUHK01、VIPer和PRID450S数据集上，无 норма化情况下的rank-20率准确率分别为61.92%、83.90%、92.03%和96.22%，而经过分数 нормализа后，它们分别提高至64.64%、89.30%、92.78%和98.76%。Abstract
Person re-identification (PRe-ID) is a crucial task in security, surveillance, and retail analysis, which involves identifying an individual across multiple cameras and views. However, it is a challenging task due to changes in illumination, background, and viewpoint. Efficient feature extraction and metric learning algorithms are essential for a successful PRe-ID system. This paper proposes a novel approach for PRe-ID, which combines a Convolutional Neural Network (CNN) based feature extraction method with Cross-view Quadratic Discriminant Analysis (XQDA) for metric learning. Additionally, a matching algorithm that employs Mahalanobis distance and a score normalization process to address inconsistencies between camera scores is implemented. The proposed approach is tested on four challenging datasets, including VIPeR, GRID, CUHK01, and PRID450S, and promising results are obtained. For example, without normalization, the rank-20 rate accuracies of the GRID, CUHK01, VIPeR and PRID450S datasets were 61.92%, 83.90%, 92.03%, 96.22%; however, after score normalization, they have increased to 64.64%, 89.30%, 92.78%, and 98.76%, respectively. Accordingly, the promising results on four challenging datasets indicate the effectiveness of the proposed approach.
摘要
人员重识别（PRe-ID）是安全、监测和营销分析中的关键任务，它的目标是在多个摄像头和视图下识别个体。然而，由于照明、背景和视角的变化，PRe-IDTask 是一项具有挑战性的任务。efficient的特征提取和度量学习算法是一个成功的 PRe-ID 系统的关键。这篇论文提出了一种新的 PRe-ID 方法，该方法结合了卷积神经网络（CNN）基于特征提取方法和跨视图quadratice Discriminant Analysis（XQDA）度量学习算法。此外，一种使用 Mahalanobis 距离和分数 нормализа处理来解决相机分数不一致的匹配算法也被实现。提议的方法在四个复杂的数据集上进行测试，包括 VIPeR、GRID、CUHK01 和 PRID450S，并取得了俊czy的结果。例如，无Normalization 的 GRID、CUHK01、VIPeR 和 PRID450S 数据集的 rank-20 率准确率分别为 61.92%、83.90%、92.03% 和 96.22%；然而，通过分数 Normalization 后，它们的 rank-20 率准确率分别提高到 64.64%、89.30%、92.78% 和 98.76%。因此，在四个复杂的数据集上取得的俊czy的结果表明了提议的方法的有效性。
MobileViG: Graph-Based Sparse Attention for Mobile Vision Applications
paper_authors: Mustafa Munir, William Avery, Radu Marculescu
For: This paper proposes a new graph-based sparse attention mechanism, Sparse Vision Graph Attention (SVGA), and a hybrid CNN-GNN architecture, MobileViG, for vision tasks on mobile devices.* Methods: The proposed SVGA mechanism is designed to reduce the computational cost of representing images as graph structures, making it more suitable for mobile devices. The MobileViG architecture combines SVGA with a CNN backbone to achieve better performance and efficiency.* Results: The proposed MobileViG model achieves state-of-the-art performance and efficiency on image classification, object detection, and instance segmentation tasks on mobile devices. The fastest model, MobileViG-Ti, achieves 75.7% top-1 accuracy on ImageNet-1K with 0.78 ms inference latency, while the largest model, MobileViG-B, obtains 82.6% top-1 accuracy with only 2.30 ms latency.Abstract
Traditionally, convolutional neural networks (CNN) and vision transformers (ViT) have dominated computer vision. However, recently proposed vision graph neural networks (ViG) provide a new avenue for exploration. Unfortunately, for mobile applications, ViGs are computationally expensive due to the overhead of representing images as graph structures. In this work, we propose a new graph-based sparse attention mechanism, Sparse Vision Graph Attention (SVGA), that is designed for ViGs running on mobile devices. Additionally, we propose the first hybrid CNN-GNN architecture for vision tasks on mobile devices, MobileViG, which uses SVGA. Extensive experiments show that MobileViG beats existing ViG models and existing mobile CNN and ViT architectures in terms of accuracy and/or speed on image classification, object detection, and instance segmentation tasks. Our fastest model, MobileViG-Ti, achieves 75.7% top-1 accuracy on ImageNet-1K with 0.78 ms inference latency on iPhone 13 Mini NPU (compiled with CoreML), which is faster than MobileNetV2x1.4 (1.02 ms, 74.7% top-1) and MobileNetV2x1.0 (0.81 ms, 71.8% top-1). Our largest model, MobileViG-B obtains 82.6% top-1 accuracy with only 2.30 ms latency, which is faster and more accurate than the similarly sized EfficientFormer-L3 model (2.77 ms, 82.4%). Our work proves that well designed hybrid CNN-GNN architectures can be a new avenue of exploration for designing models that are extremely fast and accurate on mobile devices. Our code is publicly available at https://github.com/SLDGroup/MobileViG.
摘要
传统上，卷积神经网络（CNN）和视图转换器（ViT）在计算机视觉领域占据主导地位，但最近提出的视图图神经网络（ViG）提供了一个新的探索途径。然而，由于将图像表示为图结构的开销，ViG在移动设备上运行时 computationally expensive。在这种情况下，我们提出了一种新的图形基于的稀疏注意机制——图像稀疏视Graph注意（SVGA），用于ViG在移动设备上运行。此外，我们还提出了首个在移动设备上运行的hybrid CNN-GNN架构——MobileViG，它使用了SVGA。我们的实验表明，MobileViG可以在图像分类、物体检测和实例分割任务中比现有的ViG模型和移动设备上的CNN和ViT架构更高的准确率和/或速度。我们最快的模型MobileViG-Ti在ImageNet-1K上取得75.7%的顶部1准确率，并且在iPhone 13 Mini NPU上编译后的执行时间为0.78毫秒，比MobileNetV2x1.4（1.02毫秒，74.7%）和MobileNetV2x1.0（0.81毫秒，71.8%）更快。我们最大的模型MobileViG-B在82.6%的顶部1准确率下，只需2.30毫秒的执行时间，比同等大小的EfficientFormer-L3模型（2.77毫秒，82.4%）更快和更准确。我们的工作证明了，以Well designed hybrid CNN-GNN架构为基础的模型可以在移动设备上设计出非常快速和准确的模型。我们的代码公开在https://github.com/SLDGroup/MobileViG上。
For: The paper aims to enable robots to acquire diverse and generalizable skills in open domains using one-shot imitation learning with multi-modal perception.* Methods: The paper uses a large-scale dataset of contact-rich robot manipulation sequences collected in the real world, with visual, force, audio, and action information, along with human demonstration videos. The dataset is calibrated and made publicly available.* Results: The paper aims to unlock the potential for an agent to generalize to hundreds of real-world skills with multi-modal perception.Abstract
A key challenge in robotic manipulation in open domains is how to acquire diverse and generalizable skills for robots. Recent research in one-shot imitation learning has shown promise in transferring trained policies to new tasks based on demonstrations. This feature is attractive for enabling robots to acquire new skills and improving task and motion planning. However, due to limitations in the training dataset, the current focus of the community has mainly been on simple cases, such as push or pick-place tasks, relying solely on visual guidance. In reality, there are many complex skills, some of which may even require both visual and tactile perception to solve. This paper aims to unlock the potential for an agent to generalize to hundreds of real-world skills with multi-modal perception. To achieve this, we have collected a dataset comprising over 110,000 \emph{contact-rich} robot manipulation sequences across diverse skills, contexts, robots, and camera viewpoints, all collected \emph{in the real world}. Each sequence in the dataset includes visual, force, audio, and action information, along with a corresponding human demonstration video. We have invested significant efforts in calibrating all the sensors and ensuring a high-quality dataset. The dataset is made publicly available at rh20t.github.io
摘要
一大挑战在机器人操作中是如何获得多样化和普遍适用的技能。最近的研究表明一次学习可以将训练的政策传递到新任务基于示例。这个特点很有吸引力，可以使机器人获得新的技能并改进任务和运动规划。然而，由于训练数据的限制，当前社区的关注主要集中在简单的情况，如推或捕捉任务，凭借视觉指导。在实际情况下，有许多复杂的技能，一些甚至需要视觉和感觉感知来解决。这篇论文旨在解锁一个代理人能够通过多模态感知泛化到百种真实世界技能。为了实现这一目标，我们收集了超过110,000个 contacts-rich机器人操作序列，涵盖多种技能、上下文、机器人和摄像头视点，全部在真实世界中收集。每个序列包括视觉、力、声音和动作信息，以及对应的人类示例视频。我们投入了大量的努力来准确各种感知器和高质量数据集。数据集现已公开在 rh20t.github.io
BioCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrieval
paper_authors: Qiao Jin, Won Kim, Qingyu Chen, Donald C. Comeau, Lana Yeganova, John Wilbur, Zhiyong Lu
for: This paper aims to improve the performance of biomedical information retrieval (IR) systems by introducing a new Contrastively Pre-trained Transformer (BioCPT) model.
methods: The authors use contrastive learning to train a pair of closely-integrated retriever and re-ranker using an unprecedented scale of 255 million user click logs from PubMed.
results: BioCPT sets new state-of-the-art performance on five biomedical IR tasks, outperforming various baselines including much larger models such as GPT-3-sized cpt-text-XL. Additionally, BioCPT generates better biomedical article and sentence representations for semantic evaluations.Abstract
Information retrieval (IR) is essential in biomedical knowledge acquisition and clinical decision support. While recent progress has shown that language model encoders perform better semantic retrieval, training such models requires abundant query-article annotations that are difficult to obtain in biomedicine. As a result, most biomedical IR systems only conduct lexical matching. In response, we introduce BioCPT, a first-of-its-kind Contrastively Pre-trained Transformer model for zero-shot biomedical IR. To train BioCPT, we collected an unprecedented scale of 255 million user click logs from PubMed. With such data, we use contrastive learning to train a pair of closely-integrated retriever and re-ranker. Experimental results show that BioCPT sets new state-of-the-art performance on five biomedical IR tasks, outperforming various baselines including much larger models such as GPT-3-sized cpt-text-XL. In addition, BioCPT also generates better biomedical article and sentence representations for semantic evaluations. As such, BioCPT can be readily applied to various real-world biomedical IR tasks. BioCPT API and code are publicly available at https://github.com/ncbi/BioCPT.
摘要
生物医学知识获取 (IR) 是生物医学领域的关键技术，帮助医生和研究人员快速找到有关疾病和病理的信息。然而，在生物医学领域，获取大量的查询-文章对应数据很困难，这限制了IR系统的发展。因此，大多数生物医学IR系统只能进行字面匹配。为了解决这个问题，我们提出了 BioCPT，一种首先在生物医学IR领域使用对比预训练变换器的方法。我们使用了255万次PubMed用户点击记录来训练 BioCPT，并使用对比学习训练一对 closely-integrated retriever和重新排序器。实验结果表明，BioCPT在五种生物医学IR任务中设置了新的状态态束表现，超过了多种基eline，包括GPT-3-sized cpt-text-XL。此外，BioCPT还生成了更好的生物医学文章和句子表示，用于semantic评估。因此，BioCPT可以轻松应用于各种生物医学IR任务。BioCPT API和代码在https://github.com/ncbi/BioCPT上公开提供。
Protecting the Future: Neonatal Seizure Detection with Spatial-Temporal Modeling
paper_authors: Ziyue Li, Yuchen Fang, You Li, Kan Ren, Yansen Wang, Xufang Luo, Juanyong Duan, Congrui Huang, Dongsheng Li, Lili Qiu for: 这份研究是为了提出一个深度学习框架，以便帮助新生儿电脑 Tomography （EEG）中的癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫癫�Abstract
A timely detection of seizures for newborn infants with electroencephalogram (EEG) has been a common yet life-saving practice in the Neonatal Intensive Care Unit (NICU). However, it requires great human efforts for real-time monitoring, which calls for automated solutions to neonatal seizure detection. Moreover, the current automated methods focusing on adult epilepsy monitoring often fail due to (i) dynamic seizure onset location in human brains; (ii) different montages on neonates and (iii) huge distribution shift among different subjects. In this paper, we propose a deep learning framework, namely STATENet, to address the exclusive challenges with exquisite designs at the temporal, spatial and model levels. The experiments over the real-world large-scale neonatal EEG dataset illustrate that our framework achieves significantly better seizure detection performance.
摘要
新生儿在婴儿急救室（NICU）中的电энцефаogram（EEG）检测已成为一种常见 yet 生命存在的做法。然而，这需要大量的人工监测，呼吁自动化解决方案。然而，现有的自动化方法往往因（i）人脑突发发生地点的动态性；（ii）婴儿和成人montage的不同；以及（iii）不同个体之间的巨大分布变化而失败。本文提出了一个深度学习框架，即STATENet，以解决这些独特的挑战。实验表明，我们的框架在真实世界大规模的新生儿EEG数据集上表现出了明显更好的癫痫检测性能。
Filter Bubbles in Recommender Systems: Fact or Fallacy – A Systematic Review
paper_authors: Qazi Mohammad Areeb, Mohammad Nadeem, Shahab Saquib Sohail, Raza Imam, Faiyaz Doctor, Yassine Himeur, Amir Hussain, Abbes Amira
for: This paper aims to investigate the impact of filter bubbles in recommender systems and propose an integrated approach to mitigate their effects.
methods: The authors conduct a systematic literature review on the topic of filter bubbles in recommender systems, analyzing and classifying the reviewed articles to provide valuable insights.
results: The authors identify evidence of filter bubbles in recommendation systems, highlighting several biases that contribute to their existence. They also propose mechanisms to mitigate the impact of filter bubbles and demonstrate that incorporating diversity into recommendations can potentially help alleviate this issue.Abstract
A filter bubble refers to the phenomenon where Internet customization effectively isolates individuals from diverse opinions or materials, resulting in their exposure to only a select set of content. This can lead to the reinforcement of existing attitudes, beliefs, or conditions. In this study, our primary focus is to investigate the impact of filter bubbles in recommender systems. This pioneering research aims to uncover the reasons behind this problem, explore potential solutions, and propose an integrated tool to help users avoid filter bubbles in recommender systems. To achieve this objective, we conduct a systematic literature review on the topic of filter bubbles in recommender systems. The reviewed articles are carefully analyzed and classified, providing valuable insights that inform the development of an integrated approach. Notably, our review reveals evidence of filter bubbles in recommendation systems, highlighting several biases that contribute to their existence. Moreover, we propose mechanisms to mitigate the impact of filter bubbles and demonstrate that incorporating diversity into recommendations can potentially help alleviate this issue. The findings of this timely review will serve as a benchmark for researchers working in interdisciplinary fields such as privacy, artificial intelligence ethics, and recommendation systems. Furthermore, it will open new avenues for future research in related domains, prompting further exploration and advancement in this critical area.
摘要
Filter bubble 是指互联网个性化化导致个人仅浏览一 selec tset of content，从而隔离多元意见或内容，这可能会加剧现有的态度、信念或情况。在这项研究中，我们的主要关注点是调查推荐系统中的过滤层。我们通过系统性的文献综述，挖掘了这个问题的原因，探索可能的解决方案，并提出一种集成的工具，帮助用户避免推荐系统中的过滤层。我们认为，通过 incorporating 多样性到推荐中可能有助于缓解这个问题。我们的综述还发现了推荐系统中的过滤层证据，揭示了一些偏见的来源。我们还提出了一些缓解过滤层影响的机制，并证明了在推荐中 incorporating 多样性可能有助于缓解这个问题。我们的研究结果将成为互联网隐私、人工智能伦理和推荐系统等领域的研究 benchmark，并开创了未来研究的新途径。
Adaptive reinforcement learning of multi-agent ethically-aligned behaviours: the QSOM and QDSOM algorithms
results: 在一个小型智能Grid社区中，这两种算法在多个代理人能源分配问题上表现出了适应和高性能，比基eline Reinforcement Learning算法更好。Abstract
The numerous deployed Artificial Intelligence systems need to be aligned with our ethical considerations. However, such ethical considerations might change as time passes: our society is not fixed, and our social mores evolve. This makes it difficult for these AI systems; in the Machine Ethics field especially, it has remained an under-studied challenge. In this paper, we present two algorithms, named QSOM and QDSOM, which are able to adapt to changes in the environment, and especially in the reward function, which represents the ethical considerations that we want these systems to be aligned with. They associate the well-known Q-Table to (Dynamic) Self-Organizing Maps to handle the continuous and multi-dimensional state and action spaces. We evaluate them on a use-case of multi-agent energy repartition within a small Smart Grid neighborhood, and prove their ability to adapt, and their higher performance compared to baseline Reinforcement Learning algorithms.
摘要
各种部署的人工智能系统需要与我们的道德考虑进行协调。然而，这些道德考虑可能随着时间的推移而改变：我们的社会不固定，我们的社会习俗也在发展。这会让这些 AI 系统受到挑战，特别是在机器伦理学领域，这是一个未得到充分研究的挑战。在这篇论文中，我们提出了两种算法，即 QSOM 和 QDSOM，它们能够适应环境的变化，特别是奖励函数的变化，这些奖励函数表达我们想要这些系统与我们的道德考虑相协调。它们将知名的 Q-表与（动态）自组织地图相结合，以处理连续和多维状态和动作空间。我们在一个多个代理在小智能网格中的能源分配use case中评估了它们，并证明它们的适应性和高性能相比基eline Reinforcement Learning算法。
Defending Against Malicious Behaviors in Federated Learning with Blockchain
results: 我们的 teoría y empirical analyses 表明，我们的框架可以具有强大的对抗不良客户端行为的能力，并且可以提高联合学习的安全性和可靠性。Abstract
In the era of deep learning, federated learning (FL) presents a promising approach that allows multi-institutional data owners, or clients, to collaboratively train machine learning models without compromising data privacy. However, most existing FL approaches rely on a centralized server for global model aggregation, leading to a single point of failure. This makes the system vulnerable to malicious attacks when dealing with dishonest clients. In this work, we address this problem by proposing a secure and reliable FL system based on blockchain and distributed ledger technology. Our system incorporates a peer-to-peer voting mechanism and a reward-and-slash mechanism, which are powered by on-chain smart contracts, to detect and deter malicious behaviors. Both theoretical and empirical analyses are presented to demonstrate the effectiveness of the proposed approach, showing that our framework is robust against malicious client-side behaviors.
摘要
在深度学习时代，联邦学习（FL）提供了一种有前途的方法，允许多家机构数据所有者或客户集成机器学习模型，无需产生数据隐私问题。然而，大多数现有FL方法仍然依赖中央服务器进行全球模型汇总，这会导致单点失败。这会使系统易受到不良客户的攻击，特别是在与不诚实客户进行交互时。在这种情况下，我们解决这个问题，通过基于区块链和分布式日志技术的安全可靠FL系统。我们的系统包括了分布式投票机制和奖励折损机制，这些机制都是基于链上智能合约，以探测和抑制贪念行为。我们提供了理论和实证分析，以证明我们的框架具有对不良客户侧行为的强大抗性。
Enhancing Super-Resolution Networks through Realistic Thick-Slice CT Simulation
paper_authors: Zeyu Tang, Xiaodan Xing, Guang Yang for:The paper aims to develop and evaluate an innovative simulation algorithm for generating thick-slice CT images that closely resemble actual images.methods:The proposed method uses a novel simulation algorithm to generate thick-slice CT images, which are evaluated using Peak Signal-to-Noise Ratio (PSNR) and Root Mean Square Error (RMSE) metrics.results:The proposed method demonstrated substantial enhancements in terms of both PSNR and RMSE over other simulation methods, with the highest PSNR values obtained and the lowest RMSE. The generated images were then used to train four distinct super-resolution (SR) models, which exhibited enhanced performance when trained with data produced by the proposed algorithm.Abstract
This study aims to develop and evaluate an innovative simulation algorithm for generating thick-slice CT images that closely resemble actual images in the AAPM-Mayo's 2016 Low Dose CT Grand Challenge dataset. The proposed method was evaluated using Peak Signal-to-Noise Ratio (PSNR) and Root Mean Square Error (RMSE) metrics, with the hypothesis that our simulation would produce images more congruent with their real counterparts. Our proposed method demonstrated substantial enhancements in terms of both PSNR and RMSE over other simulation methods. The highest PSNR values were obtained with the proposed method, yielding 49.7369 $\pm$ 2.5223 and 48.5801 $\pm$ 7.3271 for D45 and B30 reconstruction kernels, respectively. The proposed method also registered the lowest RMSE with values of 0.0068 $\pm$ 0.0020 and 0.0108 $\pm$ 0.0099 for D45 and B30, respectively, indicating a distribution more closely aligned with the authentic thick-slice image. Further validation of the proposed simulation algorithm was conducted using the TCIA LDCT-and-Projection-data dataset. The generated images were then leveraged to train four distinct super-resolution (SR) models, which were subsequently evaluated using the real thick-slice images from the 2016 Low Dose CT Grand Challenge dataset. When trained with data produced by our novel algorithm, all four SR models exhibited enhanced performance.
摘要
Translation:这项研究的目的是开发和评估一种创新的厚slice CT图像生成算法，以便更好地模拟实际图像在AAPM-Mayo的2016年低剂量CT大挑战数据集中。提议的方法通过PSNR和RMSE метри来评估，假设我们的模拟会生成更加相似的实际图像。我们的提议方法显示了较大的改善，PSNR和RMSE都达到了最高值。对D45和B30重建核而言，提议方法的PSNR值为49.7369 ± 2.5223和48.5801 ± 7.3271，而RMSE值为0.0068 ± 0.0020和0.0108 ± 0.0099。这表明我们的方法生成的图像更加准确地反映实际图像。此外，我们还对使用TCIA LDCT-and-Projection-data数据集进行了进一步验证。生成的图像然后被用来训练四种不同的超分辨率（SR）模型，并使用实际厚slice图像从2016年低剂量CT大挑战数据集进行评估。当使用我们的新算法生成数据时，所有四种SR模型均表现出了改善。
Collaborative Policy Learning for Dynamic Scheduling Tasks in Cloud-Edge-Terminal IoT Networks Using Federated Reinforcement Learning
results: 通过实验，论文表明了该框架在动态调度任务中的显著优势，包括加速策略学习速度和使新到达的边缘更容易适应其任务。Abstract
In this paper, we examine cloud-edge-terminal IoT networks, where edges undertake a range of typical dynamic scheduling tasks. In these IoT networks, a central policy for each task can be constructed at a cloud server. The central policy can be then used by the edges conducting the task, thereby mitigating the need for them to learn their own policy from scratch. Furthermore, this central policy can be collaboratively learned at the cloud server by aggregating local experiences from the edges, thanks to the hierarchical architecture of the IoT networks. To this end, we propose a novel collaborative policy learning framework for dynamic scheduling tasks using federated reinforcement learning. For effective learning, our framework adaptively selects the tasks for collaborative learning in each round, taking into account the need for fairness among tasks. In addition, as a key enabler of the framework, we propose an edge-agnostic policy structure that enables the aggregation of local policies from different edges. We then provide the convergence analysis of the framework. Through simulations, we demonstrate that our proposed framework significantly outperforms the approaches without collaborative policy learning. Notably, it accelerates the learning speed of the policies and allows newly arrived edges to adapt to their tasks more easily.
摘要
在这篇论文中，我们研究了云端-边缘-终端互联网络，其中边缘进行了一系列 Typical dynamic scheduling 任务。在这些互联网络中，可以在云服务器上构建每个任务的中央策略。然后，这些中央策略可以被边缘进行任务的边缘使用，从而减少边缘需要从头开始学习自己的策略的需求。此外，这些中央策略可以在云服务器上归一化地学习，通过对边缘的地址进行归一化。为了实现这一点，我们提出了一种基于联邦感知学习的共同策略学习框架。为了有效学习，我们的框架在每次轮次中选择需要合作学习的任务，考虑到任务之间的公平性。此外，我们还提出了一种不受边缘限制的策略结构，允许在不同的边缘上合并本地策略。然后，我们提供了框架的收敛分析。通过 simulate，我们示出了我们提出的框架可以非常有效地与无共同策略学习相比。尤其是，它可以加速策略学习的速度，并使新到达的边缘更容易适应自己的任务。
Graph Neural Network based Log Anomaly Detection and Explanation
results: 实验结果显示，Logs2Graphs在五个基准数据集上至少与状态计算机方法相当，而在复杂数据集上大幅超过状态计算机方法。此外，对每个异常检测结果，还提供一小 subsets of nodes 作为解释，这些节点可以为后续根本原因诊断提供有价值的提示。Abstract
Event logs are widely used to record the status of high-tech systems, making log anomaly detection important for monitoring those systems. Most existing log anomaly detection methods take a log event count matrix or log event sequences as input, exploiting quantitative and/or sequential relationships between log events to detect anomalies. Unfortunately, only considering quantitative or sequential relationships may result in many false positives and/or false negatives. To alleviate this problem, we propose a graph-based method for unsupervised log anomaly detection, dubbed Logs2Graphs, which first converts event logs into attributed, directed, and weighted graphs, and then leverages graph neural networks to perform graph-level anomaly detection. Specifically, we introduce One-Class Digraph Inception Convolutional Networks, abbreviated as OCDiGCN, a novel graph neural network model for detecting graph-level anomalies in a collection of attributed, directed, and weighted graphs. By coupling the graph representation and anomaly detection steps, OCDiGCN can learn a representation that is especially suited for anomaly detection, resulting in a high detection accuracy. Importantly, for each identified anomaly, we additionally provide a small subset of nodes that play a crucial role in OCDiGCN's prediction as explanations, which can offer valuable cues for subsequent root cause diagnosis. Experiments on five benchmark datasets show that Logs2Graphs performs at least on par state-of-the-art log anomaly detection methods on simple datasets while largely outperforming state-of-the-art log anomaly detection methods on complicated datasets.
摘要
Event logs 广泛用于记录高科技系统的状态，因此 log anomaly detection 成为监测这些系统的重要任务。现有大多数 log anomaly detection 方法都使用 log event count matrix 或 log event sequences 作为输入，利用量质和/或序列关系 между log events 检测异常。然而，只考虑量质和/或序列关系可能会导致许多假阳性和/或假阴性。为解决这个问题，我们提议一种基于图的方法，称为 Logs2Graphs，它将事件日志转换为带有属性、指向的、权重的图，然后使用图神经网络进行图级异常检测。特别是，我们引入 One-Class Digraph Inception Convolutional Networks，简称 OCDiGCN，一种基于图的异常检测模型，用于检测图级异常。通过对图表示和异常检测步骤的结合，OCDiGCN 可以学习一种特别适合异常检测的表示，从而实现高的检测精度。此外，对每个被检测到的异常，我们还提供一小 subsets of nodes 作为 OCDiGCN 的预测所需的关键节点，这些节点可以提供有价值的诊断依据。在五个基准数据集上进行实验，Logs2Graphs 在简单的数据集上与状态的检测方法相当，而在复杂的数据集上大幅超越状态的检测方法。
LEDITS: Real Image Editing with DDPM Inversion and Semantic Guidance
results: 该方法可以实现多样化的编辑效果，包括细微的修改和大幅的修改，以及修改图像的组成和风格，而无需修改模型的架构。Abstract
Recent large-scale text-guided diffusion models provide powerful image-generation capabilities. Currently, a significant effort is given to enable the modification of these images using text only as means to offer intuitive and versatile editing. However, editing proves to be difficult for these generative models due to the inherent nature of editing techniques, which involves preserving certain content from the original image. Conversely, in text-based models, even minor modifications to the text prompt frequently result in an entirely distinct result, making attaining one-shot generation that accurately corresponds to the users intent exceedingly challenging. In addition, to edit a real image using these state-of-the-art tools, one must first invert the image into the pre-trained models domain - adding another factor affecting the edit quality, as well as latency. In this exploratory report, we propose LEDITS - a combined lightweight approach for real-image editing, incorporating the Edit Friendly DDPM inversion technique with Semantic Guidance, thus extending Semantic Guidance to real image editing, while harnessing the editing capabilities of DDPM inversion as well. This approach achieves versatile edits, both subtle and extensive as well as alterations in composition and style, while requiring no optimization nor extensions to the architecture.
摘要
现代大规模文本导向扩散模型提供了强大的图像生成能力。目前，很大的努力在 modifying these images using text only as means to offer intuitive and versatile editing. However, editing proves to be difficult for these generative models due to the inherent nature of editing techniques, which involves preserving certain content from the original image. Conversely, in text-based models, even minor modifications to the text prompt frequently result in an entirely distinct result, making attaining one-shot generation that accurately corresponds to the users intent exceedingly challenging. In addition, to edit a real image using these state-of-the-art tools, one must first invert the image into the pre-trained models domain - adding another factor affecting the edit quality, as well as latency. In this exploratory report, we propose LEDITS - a combined lightweight approach for real-image editing, incorporating the Edit Friendly DDPM inversion technique with Semantic Guidance, thus extending Semantic Guidance to real image editing, while harnessing the editing capabilities of DDPM inversion as well. This approach achieves versatile edits, both subtle and extensive as well as alterations in composition and style, while requiring no optimization nor extensions to the architecture.
DSTCGCN: Learning Dynamic Spatial-Temporal Cross Dependencies for Traffic Forecasting
results: 在六个真实世界数据集上进行了广泛的实验，显示了DSTCGCN可以达到领先性的表现。Abstract
Traffic forecasting is essential to intelligent transportation systems, which is challenging due to the complicated spatial and temporal dependencies within a road network. Existing works usually learn spatial and temporal dependencies separately, ignoring the dependencies crossing spatial and temporal dimensions. In this paper, we propose DSTCGCN, a dynamic spatial-temporal cross graph convolution network to learn dynamic spatial and temporal dependencies jointly via graphs for traffic forecasting. Specifically, we introduce a fast Fourier transform (FFT) based attentive selector to choose relevant time steps for each time step based on time-varying traffic data. Given the selected time steps, we introduce a dynamic cross graph construction module, consisting of the spatial graph construction, temporal connection graph construction, and fusion modules, to learn dynamic spatial-temporal cross dependencies without pre-defined priors. Extensive experiments on six real-world datasets demonstrate that DSTCGCN achieves the state-of-the-art performance.
摘要
traffic 预测是智能交通系统的关键 component, 具有较复杂的空间和时间相关性, 使得现有的方法通常是分别学习空间和时间相关性, 忽略了空间和时间维度之间的相互关系. 在这篇论文中, 我们提出了 DSTCGCN, 一种动态空间-时间cross graph convolution network, 用于同时学习动态空间和时间相关性. 具体来说, 我们引入了 Fast Fourier Transform (FFT) 基于的选择器, 用于每个时间步选择相关的时间步, 基于时变交通数据. 给出选择的时间步, 我们引入了动态cross graph construction module, 包括空间图构建模块, 时间连接图构建模块和融合模块, 以无预先假设的方式学习动态空间-时间相关性. 我们在六个实际数据集上进行了广泛的实验, 并证明了 DSTCGCN 在智能交通预测中具有最佳性能.
HeGeL: A Novel Dataset for Geo-Location from Hebrew Text
paper_authors: Tzuf Paz-Argaman, Tal Bauman, Itai Mondshine, Itzhak Omer, Sagi Dalyot, Reut Tsarfaty for:This paper aims to collect and analyze literal Hebrew place descriptions to study lingual geospatial reasoning and improve textual geolocation.methods:The paper uses crowdsourcing to collect 5,649 literal Hebrew place descriptions in three cities in Israel, and employs qualitative and empirical analysis to examine the data’s geospatial reasoning and the need for a novel environmental representation.results:The study finds that the data exhibits abundant use of geospatial reasoning, indicating the importance of a novel environmental representation for textual geolocation in morphologically rich and resource-poor languages like Hebrew.Abstract
The task of textual geolocation - retrieving the coordinates of a place based on a free-form language description - calls for not only grounding but also natural language understanding and geospatial reasoning. Even though there are quite a few datasets in English used for geolocation, they are currently based on open-source data (Wikipedia and Twitter), where the location of the described place is mostly implicit, such that the location retrieval resolution is limited. Furthermore, there are no datasets available for addressing the problem of textual geolocation in morphologically rich and resource-poor languages, such as Hebrew. In this paper, we present the Hebrew Geo-Location (HeGeL) corpus, designed to collect literal place descriptions and analyze lingual geospatial reasoning. We crowdsourced 5,649 literal Hebrew place descriptions of various place types in three cities in Israel. Qualitative and empirical analysis show that the data exhibits abundant use of geospatial reasoning and requires a novel environmental representation.
摘要
文本地理位置 Retrieving the coordinates of a place based on a free-form language description requires not only grounding but also natural language understanding and geospatial reasoning. Although there are several datasets in English for geolocation, they are based on open-source data (Wikipedia and Twitter), where the location of the described place is mostly implicit, resulting in limited location retrieval resolution. Furthermore, there are no datasets available for addressing the problem of textual geolocation in morphologically rich and resource-poor languages, such as Hebrew. In this paper, we present the Hebrew Geo-Location (HeGeL) corpus, designed to collect literal place descriptions and analyze lingual geospatial reasoning. We crowdsourced 5,649 literal Hebrew place descriptions of various place types in three cities in Israel. Qualitative and empirical analysis show that the data exhibits abundant use of geospatial reasoning and requires a novel environmental representation.Note: "Simplified Chinese" is used to refer to the standardized form of Chinese used in mainland China and Singapore, as opposed to "Traditional Chinese" used in Hong Kong and Taiwan.
Deep Cross-Modal Steganography Using Neural Representations
results: 实验结果表明，提出的方法可以扩展到不同的机密数据集，并且可以处理不同的模式和分辨率。Abstract
Steganography is the process of embedding secret data into another message or data, in such a way that it is not easily noticeable. With the advancement of deep learning, Deep Neural Networks (DNNs) have recently been utilized in steganography. However, existing deep steganography techniques are limited in scope, as they focus on specific data types and are not effective for cross-modal steganography. Therefore, We propose a deep cross-modal steganography framework using Implicit Neural Representations (INRs) to hide secret data of various formats in cover images. The proposed framework employs INRs to represent the secret data, which can handle data of various modalities and resolutions. Experiments on various secret datasets of diverse types demonstrate that the proposed approach is expandable and capable of accommodating different modalities.
摘要
《隐藏数据在另一个消息或数据中》是стегаノграфия的过程。随着深度学习的发展，深度神经网络（DNNs）最近在隐藏中使用。然而，现有的深度隐藏技术有限，因为它们专注于特定的数据类型，并不适用于跨模态隐藏。因此，我们提出了一种基于含义神经表示（INRs）的深度跨模态隐藏框架，以隐藏不同类型的秘密数据在覆写图像中。我们的框架使用INRs来表示秘密数据，可以处理不同的模态和分辨率。对于不同的秘密数据集进行了各种实验，结果表明我们的方法可以扩展和适应不同的模态。
Cloud Ensemble Learning for Fault Diagnosis of Rolling Bearings with Stochastic Configuration Networks
methods: 本文使用了 Stochastic Configuration Network (SCN) 基于云端集成学习的方法，包括云端特征抽象方法和云端抽象样本生成方法，以及一个 Ensemble Model 来涵盖缺失信息的不确定性。
results: 实验结果显示，提案的方法可以优化滚当 fault diagnosis 的精度和一致性，尤其在仅有少量数据的情况下。Abstract
Fault diagnosis of rolling bearings is of great significance for post-maintenance in rotating machinery, but it is a challenging work to diagnose faults efficiently with a few samples. Additionally, faults commonly occur with randomness and fuzziness due to the complexity of the external environment and the structure of rolling bearings, hindering effective mining of fault characteristics and eventually restricting accuracy of fault diagnosis. To overcome these problems, stochastic configuration network (SCN) based cloud ensemble learning, called SCN-CEL, is developed in this work. Concretely, a cloud feature extraction method is first developed by using a backward cloud generator of normal cloud model to mine the uncertainty of fault information. Then, a cloud sampling method, which generates enough cloud droplets using bidirectional cloud generator, is proposed to extend the cloud feature samples. Finally, an ensemble model with SCNs is developed to comprehensively characterize the uncertainty of fault information and advance the generalization performance of fault diagnosis machine. Experimental results demonstrate that the proposed method indeed performs favorably for distinguishing fault categories of rolling bearings in the few shot scenarios.
摘要
FAULT诊断 OF ROLLING BEARINGS IS OF GREAT IMPORTANCE FOR POST-maintenance OF ROTATING MACHINERY, BUT IT IS A CHALLENGING TASK TO DIAGNOSE FAULTS EFFICIENTLY WITH A FEW SAMPLES. ADDITIONALLY, FAULTS OFTEN OCCUR WITH RANDOMNESS AND FUZZINESS DUE TO THE COMPLEXITY OF THE EXTERNAL ENVIRONMENT AND THE STRUCTURE OF ROLLING BEARINGS, HINDERING EFFECTIVE MINING OF FAULT CHARACTERISTICS AND eventually RESTRICTING THE ACCURACY OF FAULT DIAGNOSIS. TO OVERCOME THESE PROBLEMS, A STOCHASTIC CONFIGURATION NETWORK (SCN) BASED CLOUD ENSEMBLE LEARNING METHOD, CALLED SCN-CEL, IS DEVELOPED IN THIS WORK. CONCRETely, A CLOUD FEATURE EXTRACTION METHOD IS FIRST DEVELOPED BY USING A BACKWARD CLOUD GENERATOR OF NORMAL CLOUD MODEL TO MINE THE UNCERTAINTY OF FAULT INFORMATION. THEN, A CLOUD SAMPLING METHOD, WHICH GENERATES ENOUGH CLOUD DROPLETS USING BIDIRECTIONAL CLOUD GENERATOR, IS PROPOSED TO EXTEND THE CLOUD FEATURE SAMPLES. FINALLY, AN ENSEMBLE MODEL WITH SCNs IS DEVELOPED TO COMPREHENSIVELY CHARACTERIZE THE UNCERTAINTY OF FAULT INFORMATION AND ADVANCE THE GENERALIZATION PERFORMANCE OF FAULT DIAGNOSIS MACHINE. EXPERIMENTAL RESULTS DEMONSTRATE THAT THE PROPOSED METHOD INDEED PERFORMS FAVORABLY FOR DISTINGUISHING FAULT CATEGORIES OF ROLLING BEARINGS IN THE FEW SHOT SCENARIOS.
paper_authors: Aswin Paul, Noor Sajid, Lancelot Da Costa, Adeel Razi
for: 提高active inference的计算效率和定义目标分布的可能性
methods: 提出了两种解决方案，包括一种新的规划算法和一种基于Z-学习的目标分布设定方法
results: 通过实验在标准的网格世界任务中证明了这些方法的有效性和可行性，创造了新的应用机会Abstract
Despite being recognized as neurobiologically plausible, active inference faces difficulties when employed to simulate intelligent behaviour in complex environments due to its computational cost and the difficulty of specifying an appropriate target distribution for the agent. This paper introduces two solutions that work in concert to address these limitations. First, we present a novel planning algorithm for finite temporal horizons with drastically lower computational complexity. Second, inspired by Z-learning from control theory literature, we simplify the process of setting an appropriate target distribution for new and existing active inference planning schemes. Our first approach leverages the dynamic programming algorithm, known for its computational efficiency, to minimize the cost function used in planning through the Bellman-optimality principle. Accordingly, our algorithm recursively assesses the expected free energy of actions in the reverse temporal order. This improves computational efficiency by orders of magnitude and allows precise model learning and planning, even under uncertain conditions. Our method simplifies the planning process and shows meaningful behaviour even when specifying only the agent's final goal state. The proposed solutions make defining a target distribution from a goal state straightforward compared to the more complicated task of defining a temporally informed target distribution. The effectiveness of these methods is tested and demonstrated through simulations in standard grid-world tasks. These advances create new opportunities for various applications.
摘要
translate into Simplified Chinese:active inference在复杂环境中模拟智能行为存在困难，主要是计算成本高和设定合适的目标分布困难。本文介绍两种解决方案，它们在合作下解决这些限制。首先，我们提出了一种新的规划算法，可以在固定的时间途径上降低计算成本。其次，我们受控制理论文献中的Z学习启发，将定义目标分布的过程简化。我们的第一种方法利用动态计划算法，知道计算效率高，来最小化规划中的成本函数。根据bellman优化原理，我们递归评估动作的预期自由能量，从reverse temporal order进行评估。这些改进可以降低计算成本的许多次数，并允许精准的模型学习和规划，即使在不确定条件下。我们的方法简化了规划过程，并在指定Final Goal State时显示了有意义的行为。我们的提案使得从目标分布定义 become easier，而不是在更复杂的时间 informed target distribution中定义。我们的方法在标准grid-world任务中进行了测试和证明，这些进步创造了新的应用机会。
Don’t Memorize; Mimic The Past: Federated Class Incremental Learning Without Episodic Memory
results: 论文对CIFAR-100数据集进行了实验，与现有基eline相比，呈现出了显著的改善。Abstract
Deep learning models are prone to forgetting information learned in the past when trained on new data. This problem becomes even more pronounced in the context of federated learning (FL), where data is decentralized and subject to independent changes for each user. Continual Learning (CL) studies this so-called \textit{catastrophic forgetting} phenomenon primarily in centralized settings, where the learner has direct access to the complete training dataset. However, applying CL techniques to FL is not straightforward due to privacy concerns and resource limitations. This paper presents a framework for federated class incremental learning that utilizes a generative model to synthesize samples from past distributions instead of storing part of past data. Then, clients can leverage the generative model to mitigate catastrophic forgetting locally. The generative model is trained on the server using data-free methods at the end of each task without requesting data from clients. Therefore, it reduces the risk of data leakage as opposed to training it on the client's private data. We demonstrate significant improvements for the CIFAR-100 dataset compared to existing baselines.
摘要
This paper proposes a framework for federated class incremental learning that utilizes a generative model to synthesize samples from past distributions instead of storing part of past data. Clients can leverage the generative model to mitigate catastrophic forgetting locally. The generative model is trained on the server using data-free methods at the end of each task without requesting data from clients, reducing the risk of data leakage compared to training it on the client's private data. We demonstrate significant improvements for the CIFAR-100 dataset compared to existing baselines.
STG4Traffic: A Survey and Benchmark of Spatial-Temporal Graph Neural Networks for Traffic Prediction
results: 研究发现，STG4Traffic 可以在两种交通数据集上达到比较高的预测精度，并且可以根据不同的数据集和模型设置进行个性化定制。Abstract
Traffic prediction has been an active research topic in the domain of spatial-temporal data mining. Accurate real-time traffic prediction is essential to improve the safety, stability, and versatility of smart city systems, i.e., traffic control and optimal routing. The complex and highly dynamic spatial-temporal dependencies make effective predictions still face many challenges. Recent studies have shown that spatial-temporal graph neural networks exhibit great potential applied to traffic prediction, which combines sequential models with graph convolutional networks to jointly model temporal and spatial correlations. However, a survey study of graph learning, spatial-temporal graph models for traffic, as well as a fair comparison of baseline models are pending and unavoidable issues. In this paper, we first provide a systematic review of graph learning strategies and commonly used graph convolution algorithms. Then we conduct a comprehensive analysis of the strengths and weaknesses of recently proposed spatial-temporal graph network models. Furthermore, we build a study called STG4Traffic using the deep learning framework PyTorch to establish a standardized and scalable benchmark on two types of traffic datasets. We can evaluate their performance by personalizing the model settings with uniform metrics. Finally, we point out some problems in the current study and discuss future directions. Source codes are available at https://github.com/trainingl/STG4Traffic.
摘要
历史预测已经是智能城市系统中的活跃研究主题之一，准确的实时历史预测能够提高智能城市系统的安全、稳定性和多样性。然而，由于历史的复杂和高度动态关系，实现有效预测仍然面临着许多挑战。最近的研究表明，使用历史空间图神经网络可以有效地应用于历史预测，这种方法可以同时模型历史序列和空间相关性。然而，一项历史学习、空间历史图模型以及基准模型的比较仍然是一个潜在的问题。在这篇论文中，我们首先提供了一种系统性的历史学习策略和常用的历史 convolution 算法的评论。然后，我们进行了全面的审查最近提出的空间历史图网络模型的优劣点。此外，我们在 PyTorch 深度学习框架上建立了一个标准化和可扩展的 STG4Traffic 研究，并使用两种交通数据集来评估其性能。我们可以通过个性化模型设置来评估其表现，并且提出了一些问题和未来方向。源代码可以在 GitHub 上找到。
Fourier-Mixed Window Attention: Accelerating Informer for Long Sequence Time-Series Forecasting
results: FWin transformer可以提高Informer的总预测精度，同时提高推断速度，在uniivariate和multivariate dataset上实现40-50%的加速。此外，我们还证明了一个学习的FWin类注意力可以与Softmax全注意力相当或者超过基于Informer模型的全注意力层时间序列数据的键vector。Abstract
We study a fast local-global window-based attention method to accelerate Informer for long sequence time-series forecasting. While window attention is local and a considerable computational saving, it lacks the ability to capture global token information which is compensated by a subsequent Fourier transform block. Our method, named FWin, does not rely on query sparsity hypothesis and an empirical approximation underlying the ProbSparse attention of Informer. Through experiments on univariate and multivariate datasets, we show that FWin transformers improve the overall prediction accuracies of Informer while accelerating its inference speeds by 40 to 50 %. We also show in a nonlinear regression model that a learned FWin type attention approaches or even outperforms softmax full attention based on key vectors extracted from an Informer model's full attention layer acting on time series data.
摘要
我们研究了一种快速的本地-全局窗口基于注意力方法，用于加速Informer进行长序时间序列预测。虽然窗口注意力是本地的，但lacks the ability to capture global token information，这被补偿了随后的傅立做块。我们的方法，名为FWin，不依赖于查询稀缺假设和Informer中的ProbSparse注意力的经验 aproximation。通过对单Variate和多Variate数据进行实验，我们显示了FWin transformers可以提高Informer的总预测精度，同时加速其推断速度，提高40到50%。我们还在非线性回归模型中表明，学习的FWin类型注意力可以与softmax全注意力相当或者超过，基于Informer模型中全注意力层中的键vector从时间序列数据中提取出来的。
PatternGPT :A Pattern-Driven Framework for Large Language Model Text Generation
For: This paper aims to improve the text generation capability of large language models (LLMs) by proposing a pattern-driven text generation framework called PatternGPT.* Methods: The framework uses the extraction capability of LLMs to generate rich and diversified structured and formalized patterns, which are then used to guide the generation of models. The framework also utilizes federated learning to share patterns among multiple agents and optimize the search for high-quality patterns.* Results: The proposed framework has several advantages, including generating diversified patterns, protecting data privacy, combining external knowledge, and improving the quality of generation. The framework provides an effective method to optimize the text generation capability of LLMs and apply them to the field of intelligent dialogue and content generation.Here is the simplified Chinese text for the three key points:
results: 提议的框架具有多样性、数据隐私保护、外部知识组合和生成质量提高等优点，为大语言模型的文本生成能力优化提供有效的方法，应用于智能对话和内容生成等领域。Abstract
Large language models(LLMS)have shown excellent text generation capabilities, capable of generating fluent human-like responses for many downstream tasks. However, applying large language models to real-world critical tasks remains challenging due to their susceptibility to hallucinations and inability to directly use external knowledge. To cope with the above challenges, this paper proposes PatternGPT, a pattern-driven text generation framework for Large Language Models. Firstly, the framework utilizes the extraction capability of Large Language Models to generate rich and diversified structured and formalized patterns, which facilitates the introduction of external knowledge to do the computation, and then draws on the idea of federated learning to use multiple agents to achieve the sharing in order to obtain more diversified patterns, and finally uses judgment criteria and optimization algorithm to search for high-quality patterns to guide the generation of models. Finally, external knowledge such as judgment criteria and optimization algorithms are used to search for high-quality patterns, and the searched patterns are used to guide model generation. This framework has the advantages of generating diversified patterns, protecting data privacy, combining external knowledge, and improving the quality of generation, which provides an effective method to optimize the text generation capability of large language models, and make it better applied to the field of intelligent dialogue and content generation.
摘要
大型语言模型(LLM)已经显示出扎实的文本生成能力，能够生成流畅、人工智能的回应 для许多下游任务。但是，将大型语言模型应用到实际世界中的重要任务仍然具有挑战，主要是因为它们容易受到幻视和无法直接使用外部知识。为了解决以上问题，这篇论文提出了 PatternGPT，一个基于模式的文本生成框架 для Large Language Models。首先，这个框架利用了 Large Language Models 的提取能力来生成丰富和多样化的结构化和正式化模式，以便引入外部知识进行计算，然后参考 federated learning 的想法，使用多个代理人共享，以获得更多的多样化模式，最后使用判断标准和优化算法来搜寻高品质的模式，以导引模型的生成。最后，这个框架使用了外部知识，例如判断标准和优化算法，搜寻高品质的模式，并将搜寻到的模式用来导引模型的生成。这个框架有丰富的模式生成、保护数据隐私、结合外部知识、提高生成质量等优点，提供了一个有效的方法来优化大型语言模型的文本生成能力，并将其应用到智能对话和内容生成领域。
FedDefender: Backdoor Attack Defense in Federated Learning
results: 在 MNIST 和 FashionMNIST datasets上，FedDefender 有效地防止了targeted poisoning攻击，降低了攻击成功率（ASR）至 10%，而不会影响全球模型的性能。Abstract
Federated Learning (FL) is a privacy-preserving distributed machine learning technique that enables individual clients (e.g., user participants, edge devices, or organizations) to train a model on their local data in a secure environment and then share the trained model with an aggregator to build a global model collaboratively. In this work, we propose FedDefender, a defense mechanism against targeted poisoning attacks in FL by leveraging differential testing. Our proposed method fingerprints the neuron activations of clients' models on the same input and uses differential testing to identify a potentially malicious client containing a backdoor. We evaluate FedDefender using MNIST and FashionMNIST datasets with 20 and 30 clients, and our results demonstrate that FedDefender effectively mitigates such attacks, reducing the attack success rate (ASR) to 10\% without deteriorating the global model performance.
摘要
federated learning（FL）是一种隐私保护的分布式机器学习技术，允许个体客户端（例如用户参与者、边缘设备或组织）在安全环境中使用本地数据进行模型训练，然后将训练好的模型分享给一个综合器，共同构建全球模型。在这项工作中，我们提出了FedDefender，一种防御机制，用于防止针对性攻击FL。我们的提议方法通过对客户端模型的神经元活动进行指纹测试，来确定潜在恶意客户端是否含有后门。我们使用MNIST和FashionMNIST数据集，并在20和30个客户端上进行了测试。我们的结果表明，FedDefender有效地 Mitigate Such attacks，降低攻击成功率（ASR）至10%，无需降低全球模型性能。
methods: 这篇论文引入了一种新的人类互动检测任务（HID），该任务旨在在一个模型中检测人类的动作，识别每个人的动作，并将人类分组 according to their 互动关系。
results: 研究人员通过使用AVA dataset创建了一个新的HID benchмарke（AVA-I），并提出了一种基于Transformer模型的SaMFormer方法来解决HID任务。该方法在AVA-I上进行了广泛的实验，并被证明高效性。Abstract
A comprehensive understanding of interested human-to-human interactions in video streams, such as queuing, handshaking, fighting and chasing, is of immense importance to the surveillance of public security in regions like campuses, squares and parks. Different from conventional human interaction recognition, which uses choreographed videos as inputs, neglects concurrent interactive groups, and performs detection and recognition in separate stages, we introduce a new task named human-to-human interaction detection (HID). HID devotes to detecting subjects, recognizing person-wise actions, and grouping people according to their interactive relations, in one model. First, based on the popular AVA dataset created for action detection, we establish a new HID benchmark, termed AVA-Interaction (AVA-I), by adding annotations on interactive relations in a frame-by-frame manner. AVA-I consists of 85,254 frames and 86,338 interactive groups, and each image includes up to 4 concurrent interactive groups. Second, we present a novel baseline approach SaMFormer for HID, containing a visual feature extractor, a split stage which leverages a Transformer-based model to decode action instances and interactive groups, and a merging stage which reconstructs the relationship between instances and groups. All SaMFormer components are jointly trained in an end-to-end manner. Extensive experiments on AVA-I validate the superiority of SaMFormer over representative methods. The dataset and code will be made public to encourage more follow-up studies.
摘要
“一个全面的理解人际互动在视频流中，如排队、握手、战斗和追逐，对公共安全监控区域如校园、广场和公园来说是非常重要。不同于传统的人际互动识别，使用仪制化视频为输入，忽略同时互动的小组，并在不同阶段进行检测和识别，我们引入了一个新任务名为人际互动检测（HID）。HID的目的是在一个模型中检测主题，识别每个人的动作，并根据人们之间的互动关系分组人。首先，我们根据已知的AVA数据集，创建了一个新的HIDbenchmark，称为AVA-Interaction（AVA-I），通过在每个帧上添加互动关系的标注。AVA-I包含85,254帧和86,338个互动小组，每帧最多可以有4个同时互动的小组。其次，我们提出了一个基本的SaMFormer方法来进行HID，包括一个视觉特征提取器、一个分阶段使用Transformer型模型将动作实例和互动小组解析，以及一个合并阶段将实例和小组之间的关系重建。所有SaMFormer комponents都是以终端式方式进行集成训练。广泛的实验验证了SaMFormer在AVA-I上的超越性。这个数据集和代码将会公开，以便更多的后续研究。”
Conformer LLMs – Convolution Augmented Large Language Models
results: 该论文实现了在听说任务中获得显著的性能提升，表明该方法可以在大规模语言模型中具有良好的性能。Abstract
This work builds together two popular blocks of neural architecture, namely convolutional layers and Transformers, for large language models (LLMs). Non-causal conformers are used ubiquitously in automatic speech recognition. This work aims to adapt these architectures in a causal setup for training LLMs. Transformers decoders effectively capture long-range dependencies over several modalities and form a core backbone of modern advancements in machine learning. Convolutional architectures have been popular in extracting features in domains such as raw 1-D signals, speech, and images, to name a few. In this paper, by combining local and global dependencies over latent representations using causal convolutional filters and Transformer, we achieve significant gains in performance. This work showcases a robust speech architecture that can be integrated and adapted in a causal setup beyond speech applications for large-scale language modeling.
摘要
这个工作将两种流行的神经网络块结合在一起，即卷积层和Transformers，用于大型语言模型（LLM）的训练。非 causal 的变体在自动语音识别中广泛使用。本工作想要将这些架构在 causal 设置中适应训练 LLM。Transformers 解码器可以效果地捕捉多modalities 中的长距离依赖关系，并成为现代机器学习的核心进步。卷积 arquitectures 在处理 Raw 1-D 信号、语音和图像等领域中受欢迎，以EXTRACT特征。在这篇论文中，我们通过将本地和全球依赖关系使用 causal 卷积Filter和Transformer 来实现显著提高性能。这个robust的语音架构可以在 causal 设置中被集成和适应 beyond speech 应用程序。
GenRec: Large Language Model for Generative Recommendation
results: experiments 表明，GenRec 在大量数据集上具有显著优异表现，与传统的分类推荐方法相比，GenRec 可以更好地理解用户偏好和适应变化的用户需求。Abstract
In recent years, large language models (LLM) have emerged as powerful tools for diverse natural language processing tasks. However, their potential for recommender systems under the generative recommendation paradigm remains relatively unexplored. This paper presents an innovative approach to recommendation systems using large language models (LLMs) based on text data. In this paper, we present a novel LLM for generative recommendation (GenRec) that utilized the expressive power of LLM to directly generate the target item to recommend, rather than calculating ranking score for each candidate item one by one as in traditional discriminative recommendation. GenRec uses LLM's understanding ability to interpret context, learn user preferences, and generate relevant recommendation. Our proposed approach leverages the vast knowledge encoded in large language models to accomplish recommendation tasks. We first we formulate specialized prompts to enhance the ability of LLM to comprehend recommendation tasks. Subsequently, we use these prompts to fine-tune the LLaMA backbone LLM on a dataset of user-item interactions, represented by textual data, to capture user preferences and item characteristics. Our research underscores the potential of LLM-based generative recommendation in revolutionizing the domain of recommendation systems and offers a foundational framework for future explorations in this field. We conduct extensive experiments on benchmark datasets, and the experiments shows that our GenRec has significant better results on large dataset.
摘要
In this paper, we present a novel LLM for generative recommendation (GenRec) that leverages the expressive power of LLMs to directly generate the target item to recommend, rather than calculating ranking scores for each candidate item one by one as in traditional discriminative recommendation. GenRec uses LLM's understanding ability to interpret context, learn user preferences, and generate relevant recommendations.Our proposed approach leverages the vast knowledge encoded in large language models to accomplish recommendation tasks. We first formulate specialized prompts to enhance the ability of LLMs to comprehend recommendation tasks. We then use these prompts to fine-tune the LLaMA backbone LLM on a dataset of user-item interactions, represented by textual data, to capture user preferences and item characteristics.Our research highlights the potential of LLM-based generative recommendation to revolutionize the field of recommendation systems and provides a foundational framework for future explorations in this area. We conduct extensive experiments on benchmark datasets, and the results show that our GenRec achieves significantly better results on large datasets.
paper_authors: Chenyang Qiu, Yingsheng Geng, Junrui Lu, Kaida Chen, Shitong Zhu, Ya Su, Guoshun Nan, Can Zhang, Junsong Fu, Qimei Cui, Xiaofeng Tao
For: + 3D-IDS is proposed to tackle the inconsistent performance of existing NIDS methods in detecting various unknown and known attacks, especially in encrypted traffic. + The proposed method aims to disentangle traffic features and highlight attack-specific features for effective identification of attacks. + The method is designed to improve the explainability of NIDS.* Methods: + Two-step feature disentanglements are used to differentiate complex features of various attacks. + A non-parameterized optimization based on mutual information is used to automatically disentangle traffic features. + A memory model is used to generate representations of the disentangled features. + A novel graph diffusion method is used to dynamically fuse the network topology for spatial-temporal aggregation in evolving data streams.* Results: + The proposed 3D-IDS method outperforms existing NIDS methods in detecting various attacks, including unknown threats and known ones that are not easily detected. + Experiments show the superiority of the proposed method. + The two-step feature disentanglements benefit the explainability of NIDS.Abstract
Network-based intrusion detection system (NIDS) monitors network traffic for malicious activities, forming the frontline defense against increasing attacks over information infrastructures. Although promising, our quantitative analysis shows that existing methods perform inconsistently in declaring various unknown attacks (e.g., 9% and 35% F1 respectively for two distinct unknown threats for an SVM-based method) or detecting diverse known attacks (e.g., 31% F1 for the Backdoor and 93% F1 for DDoS by a GCN-based state-of-the-art method), and reveals that the underlying cause is entangled distributions of flow features. This motivates us to propose 3D-IDS, a novel method that aims to tackle the above issues through two-step feature disentanglements and a dynamic graph diffusion scheme. Specifically, we first disentangle traffic features by a non-parameterized optimization based on mutual information, automatically differentiating tens and hundreds of complex features of various attacks. Such differentiated features will be fed into a memory model to generate representations, which are further disentangled to highlight the attack-specific features. Finally, we use a novel graph diffusion method that dynamically fuses the network topology for spatial-temporal aggregation in evolving data streams. By doing so, we can effectively identify various attacks in encrypted traffics, including unknown threats and known ones that are not easily detected. Experiments show the superiority of our 3D-IDS. We also demonstrate that our two-step feature disentanglements benefit the explainability of NIDS.
摘要
First, we disentangle traffic features using a non-parameterized optimization based on mutual information, which automatically differentiates tens and hundreds of complex features of various attacks. These differentiated features are then fed into a memory model to generate representations, which are further disentangled to highlight attack-specific features. Finally, we use a novel graph diffusion method that dynamically fuses the network topology for spatial-temporal aggregation in evolving data streams. This allows us to effectively identify various attacks in encrypted traffics, including unknown threats and known ones that are not easily detected.Experiments show the superiority of our 3D-IDS. We also demonstrate that our two-step feature disentanglements benefit the explainability of NIDS.
WaveMixSR: A Resource-efficient Neural Network for Image Super-resolution
paper_authors: Pranav Jeevan, Akella Srinidhi, Pasunuri Prathiba, Amit Sethi
For: The paper is written for research on image super-resolution, specifically proposing a new neural network called WaveMixSR that uses a 2D-discrete wavelet transform for spatial token-mixing.* Methods: The paper uses the WaveMix architecture, which combines the inductive bias of convolutions with the lossless token-mixing property of wavelet transform to achieve higher performance in image super-resolution. The network does not unroll the image as a sequence of pixels/patches like transformer-based models do.* Results: The paper compares the performance of WaveMixSR with other state-of-the-art methods for image super-resolution and shows that it achieves competitive performance in all datasets and reaches state-of-the-art performance in the BSD100 dataset on multiple super-resolution tasks. The model achieves this performance using less training data and computational resources while maintaining high parameter efficiency compared to current state-of-the-art models.Here’s the simplified Chinese text for the three key points:* For: 这篇论文是关于图像超解析研究，具体是提出一种新的神经网络 called WaveMixSR，使用2D离散波лет变换来实现空间токен混合。* Methods: 论文使用WaveMix架构，结合卷积的假设导向和波лет变换的无损token混合性来实现图像超解析 task。不同于转换器模型，WaveMixSR不会将图像作为像素/补丁序列推进。* Results: 论文对WaveMixSR与其他当前顶尖方法进行比较，并显示它在所有数据集中具有竞争性的性能，并在BSD100数据集中实现多个超解析任务的state-of-the-art性。WaveMixSR使用 fewer training data和计算资源，同时保持高参数效率与当前顶尖模型相比。Abstract
Image super-resolution research recently been dominated by transformer models which need higher computational resources than CNNs due to the quadratic complexity of self-attention. We propose a new neural network -- WaveMixSR -- for image super-resolution based on WaveMix architecture which uses a 2D-discrete wavelet transform for spatial token-mixing. Unlike transformer-based models, WaveMixSR does not unroll the image as a sequence of pixels/patches. It uses the inductive bias of convolutions along with the lossless token-mixing property of wavelet transform to achieve higher performance while requiring fewer resources and training data. We compare the performance of our network with other state-of-the-art methods for image super-resolution. Our experiments show that WaveMixSR achieves competitive performance in all datasets and reaches state-of-the-art performance in the BSD100 dataset on multiple super-resolution tasks. Our model is able to achieve this performance using less training data and computational resources while maintaining high parameter efficiency compared to current state-of-the-art models.
摘要
Sparsity-aware generalization theory for deep neural networks
methods: 这 paper 使用了一种新的方法来分析深度循环ReLU网络的泛化性，利用隐藏层活动的稀烈程度来降低效果模型的大小。
results: 这 paper 表明了泛化性和稀烈程度之间的基本负面关系，而且这些结果不假设模型的稀烈程度有很大的限制。数值计算也验证了这些结果，并在特定情况下提供了非虚假的下界。Abstract
Deep artificial neural networks achieve surprising generalization abilities that remain poorly understood. In this paper, we present a new approach to analyzing generalization for deep feed-forward ReLU networks that takes advantage of the degree of sparsity that is achieved in the hidden layer activations. By developing a framework that accounts for this reduced effective model size for each input sample, we are able to show fundamental trade-offs between sparsity and generalization. Importantly, our results make no strong assumptions about the degree of sparsity achieved by the model, and it improves over recent norm-based approaches. We illustrate our results numerically, demonstrating non-vacuous bounds when coupled with data-dependent priors in specific settings, even in over-parametrized models.
摘要
深度人工神经网络达到了意外的泛化能力，这种能力仍未得到充分理解。在这篇论文中，我们提出了一种新的分析泛化方法，利用隐藏层活动的稀疏度来提高分析效果。我们开发了一个考虑这种减少效果模型大小的框架，从而显示了泛化和稀疏度之间的基本负相关性。这些结果不假设模型达到了具体的稀疏度水平，并且超过了最近的 нор-based方法。我们通过数值示例，证明了这些结果的有效性，即在特定设定下，与数据依赖的假设结合使用时可以获得非虚无效的下限。
Understanding Counterspeech for Online Harm Mitigation
results: 研究发现，有效的抗议方法包括直接抗议、呈现反对意见、提供反对证据等，而且最佳实施条件包括在社交媒体平台上进行抗议、在抗议中使用正面语言、以及在抗议中强调团结和支持。Abstract
Counterspeech offers direct rebuttals to hateful speech by challenging perpetrators of hate and showing support to targets of abuse. It provides a promising alternative to more contentious measures, such as content moderation and deplatforming, by contributing a greater amount of positive online speech rather than attempting to mitigate harmful content through removal. Advances in the development of large language models mean that the process of producing counterspeech could be made more efficient by automating its generation, which would enable large-scale online campaigns. However, we currently lack a systematic understanding of several important factors relating to the efficacy of counterspeech for hate mitigation, such as which types of counterspeech are most effective, what are the optimal conditions for implementation, and which specific effects of hate it can best ameliorate. This paper aims to fill this gap by systematically reviewing counterspeech research in the social sciences and comparing methodologies and findings with computer science efforts in automatic counterspeech generation. By taking this multi-disciplinary view, we identify promising future directions in both fields.
摘要
对话抗言可以直接反驳仇恨言语，挑战仇恨行为者并表达对受害者的支持。它提供了一种有前途的替代方案，而不是通过内容审核和屏蔽来缓解伤害性内容。随着大语言模型的发展，生成对话抗言的过程可以通过自动化来加速，这将使得大规模的在线运动变得可能。然而，我们目前缺乏对对话抗言的效果进行系统性理解，例如最有效的类型、实施条件以及哪些特定情况下可以最好地缓解仇恨的效果。这篇论文希望通过对社科研究和计算机科学的自动对话抗言生成技术进行系统性比较，从而填补这些知识空白。通过这种多学科视角，我们可以找到未来的发展方向。
WavePaint: Resource-efficient Token-mixer for Self-supervised Inpainting
results: 比现有 state-of-the-art 模型在 reconstruction 质量上表现出色，同时具有参数数量少于半、训练和评估时间较短的优势。对 CelebA-HQ 数据集进行了比较，无需使用对抗性训练的潜在探测器，而且还超过了当前 GAN-based 架构。Abstract
Image inpainting, which refers to the synthesis of missing regions in an image, can help restore occluded or degraded areas and also serve as a precursor task for self-supervision. The current state-of-the-art models for image inpainting are computationally heavy as they are based on transformer or CNN backbones that are trained in adversarial or diffusion settings. This paper diverges from vision transformers by using a computationally-efficient WaveMix-based fully convolutional architecture -- WavePaint. It uses a 2D-discrete wavelet transform (DWT) for spatial and multi-resolution token-mixing along with convolutional layers. The proposed model outperforms the current state-of-the-art models for image inpainting on reconstruction quality while also using less than half the parameter count and considerably lower training and evaluation times. Our model even outperforms current GAN-based architectures in CelebA-HQ dataset without using an adversarially trainable discriminator. Our work suggests that neural architectures that are modeled after natural image priors require fewer parameters and computations to achieve generalization comparable to transformers.
摘要
图像填充（image inpainting）是指将图像中缺失的区域重新生成，以便修复受遮挡或退化的区域。这项技术可以作为自我超级视觉任务的前置任务，以及图像修复和改善的方法。当前的状态 искусственный智能模型（state-of-the-art models） для图像填充都是基于 transformer 或 CNN 底层的，这些模型在对抗或扩散设置下训练。本文与视觉 transformer 不同，使用 computationally-efficient WaveMix-based fully convolutional architecture -- WavePaint。它使用 2D-discrete wavelet transform (DWT) 进行空间和多谱分辨率的混合，并使用卷积层。提议的模型在重建质量方面超过当前状态 искусственный智能模型，而且使用的参数数量少于half，并且训练和评估时间较短。我们的模型甚至在 CelebA-HQ 数据集上超过当前基于 GAN 架构的模型，而不需要 adversarially trainable 的识别器。我们的工作表明，基于自然图像假设的神经网络模型可以通过减少参数和计算来实现与 transformers 相同的泛化能力。
ProbVLM: Probabilistic Adapter for Frozen Vison-Language Models
results: 在四个复杂的dataset上（COCO、Flickr、CUB和Oxford-flowers），这paper测试了两个VLMs（CLIP和BLIP）的嵌入空间的不确定性，并证明了ProbVLM在检索任务中的评估和选择性能比其他方法更高。此外，paper还提出了在实际应用中的活动学习和模型选择任务中的 embeddinguncertainty 的使用，并证明了它们的有用性。最后，paper还介绍了一种使用大规模预训练的潜在扩散模型来可见化嵌入分布的新技术。Abstract
Large-scale vision-language models (VLMs) like CLIP successfully find correspondences between images and text. Through the standard deterministic mapping process, an image or a text sample is mapped to a single vector in the embedding space. This is problematic: as multiple samples (images or text) can abstract the same concept in the physical world, deterministic embeddings do not reflect the inherent ambiguity in the embedding space. We propose ProbVLM, a probabilistic adapter that estimates probability distributions for the embeddings of pre-trained VLMs via inter/intra-modal alignment in a post-hoc manner without needing large-scale datasets or computing. On four challenging datasets, i.e., COCO, Flickr, CUB, and Oxford-flowers, we estimate the multi-modal embedding uncertainties for two VLMs, i.e., CLIP and BLIP, quantify the calibration of embedding uncertainties in retrieval tasks and show that ProbVLM outperforms other methods. Furthermore, we propose active learning and model selection as two real-world downstream tasks for VLMs and show that the estimated uncertainty aids both tasks. Lastly, we present a novel technique for visualizing the embedding distributions using a large-scale pre-trained latent diffusion model.
摘要
大规模视力语言模型（VLM）如CLIP成功地找到图像和文本之间的对应关系。通过标准的推测映射过程，一个图像或文本样本将映射到单个向量空间中。这会引起问题：由于多个样本（图像或文本）可以抽象出physical world中的同一个概念，则推测的 embedding 空间中的固定 embedding 不会反映 embedding 空间的内在含义。我们提出了ProbVLM，一种可信度抑制器，通过模式/内部模式的对齐来对预训练VLM的embedding进行随机化，而无需大规模数据集或计算。在四个复杂的数据集上（COCO、Flickr、CUB和Oxford-flowers），我们估算了预训练VLM的多模态 embedding 不确定性，衡量预测任务中的评估和折衔，并显示了ProbVLM的超越性。此外，我们还提出了基于 VLM 的活动学习和模型选择两个现实世界下游任务，并证明了估算不确定性对这两个任务具有帮助作用。最后，我们介绍了一种基于大规模预训练潜在扩散模型的新技术来视觉化 embedding 分布。
CasTGAN: Cascaded Generative Adversarial Network for Realistic Tabular Data Synthesis
paper_authors: Abdallah Alshantti, Damiano Varagnolo, Adil Rasheed, Aria Rahmati, Frank Westad
for: This paper aims to generate realistic tabular data with a specific focus on validity, addressing the limitations of traditional generative models.
methods: The proposed method, CasTGAN, uses a cascaded tabular GAN architecture, where a dedicated generator samples each feature, resulting in more representative synthetic output.
results: The experimental results show that CasTGAN well captures the constraints and correlations between features of real data, especially for high-dimensional datasets. Additionally, the model demonstrates robustness against white-box privacy attacks with perturbations applied to the auxiliary learners.Abstract
Generative adversarial networks (GANs) have drawn considerable attention in recent years for their proven capability in generating synthetic data which can be utilized for multiple purposes. While GANs have demonstrated tremendous successes in producing synthetic data samples that replicate the dynamics of the original datasets, the validity of the synthetic data and the underlying privacy concerns represent major challenges which are not sufficiently addressed. In this work, we design a cascaded tabular GAN framework (CasTGAN) for generating realistic tabular data with a specific focus on the validity of the output. In this context, validity refers to the the dependency between features that can be found in the real data, but is typically misrepresented by traditional generative models. Our key idea entails that employing a cascaded architecture in which a dedicated generator samples each feature, the synthetic output becomes more representative of the real data. Our experimental results demonstrate that our model well captures the constraints and the correlations between the features of the real data, especially the high dimensional datasets. Furthermore, we evaluate the risk of white-box privacy attacks on our model and subsequently show that applying some perturbations to the auxiliary learners in CasTGAN increases the overall robustness of our model against targeted attacks.
摘要
paper_authors: Quan Tu, Shen Gao, Xiaolong Wu, Zhao Cao, Ji-Rong Wen, Rui Yan
for: 提高对话结构和上下文 semantic 理解
methods: 提出三种自动学习任务来升级 conversational search 模型
results: 在 CAsT-19 和 CAsT-20 两个 benchmark 数据集上，对已有 conversational search 方法进行了改进，并取得了广泛的实验成果。Abstract
Conversational search has been regarded as the next-generation search paradigm. Constrained by data scarcity, most existing methods distill the well-trained ad-hoc retriever to the conversational retriever. However, these methods, which usually initialize parameters by query reformulation to discover contextualized dependency, have trouble in understanding the dialogue structure information and struggle with contextual semantic vanishing. In this paper, we propose \fullmodel (\model) which is a new post-training paradigm with three self-supervised tasks to efficiently initialize the conversational search model to enhance the dialogue structure and contextual semantic understanding. Furthermore, the \model can be plugged into most of the existing conversational models to boost their performance. To verify the effectiveness of our proposed method, we apply the conversational encoder post-trained by \model on the conversational search task using two benchmark datasets: CAsT-19 and CAsT-20. Extensive experiments that our \model can boost the performance of several existing conversational search methods. Our source code is available at \url{https://github.com/morecry/SSP}.
摘要
对话搜寻被视为未来搜寻模式。由于数据缺乏，大多现有方法将特定的对话搜寻器转换为对话搜寻器。然而，这些方法通常会将问题重新构成来发现对话结构信息，却对对话结构和上下文Semantic衰退过滤产生困难。在这篇论文中，我们提出了\fullmodel (\model)，一个新的后训练模式，具有三个自动训练任务，可以快速初始化对话搜寻模型，提高对话结构和上下文Semantic理解。此外，\model可以与大多数现有的对话模型整合，提高其表现。为了证明我们提出的方法的有效性，我们将对话核心过滤器训练后使用了\model进行对话搜寻任务，使用了两个benchmark数据集：CAsT-19和CAsT-20。广泛的实验结果表明，我们的\model可以提高许多现有的对话搜寻方法的表现。我们的原始代码可以在\url{https://github.com/morecry/SSP}获取。
TensorGPT: Efficient Compression of the Embedding Layer in LLMs based on the Tensor-Train Decomposition
results: 实验结果表明，通过该方法可以将嵌入层压缩 factor 达到 38.40 倍，并且当压缩因子为 3.31 倍时，even 超过原始 GPT-2 模型的性能。Abstract
High-dimensional token embeddings underpin Large Language Models (LLMs), as they can capture subtle semantic information and significantly enhance the modelling of complex language patterns. However, the associated high dimensionality also introduces considerable model parameters, and a prohibitively high model storage. To address this issue, this work proposes an approach based on the Tensor-Train Decomposition (TTD), where each token embedding is treated as a Matrix Product State (MPS) that can be efficiently computed in a distributed manner. The experimental results on GPT-2 demonstrate that, through our approach, the embedding layer can be compressed by a factor of up to 38.40 times, and when the compression factor is 3.31 times, even produced a better performance than the original GPT-2 model.
摘要
高维度的токен嵌入在大语言模型（LLM）中起到重要作用，因为它们可以捕捉细微语义信息和复杂语言模式的特征。然而，相关的高维度也导致了较大的模型参数和庞大的模型存储空间。为解决这个问题，本工作提出了基于tensor-train分解（TTD）的方法，其中每个tokен嵌入被视为一个矩阵乘积状态（MPS），可以高效地在分布式环境中计算。实验结果表明，通过我们的方法，嵌入层可以被压缩38.40倍，而当压缩因子为3.31倍时，甚至超越原始GPT-2模型的性能。
results: 结果表明，在第一个和第二个阶段 incorporating LLMs 可以routinely提供显著改进，并且允许用户根据成本和准确性进行负担和让步，以生成满足需求的 clusters。Abstract
Unlike traditional unsupervised clustering, semi-supervised clustering allows users to provide meaningful structure to the data, which helps the clustering algorithm to match the user's intent. Existing approaches to semi-supervised clustering require a significant amount of feedback from an expert to improve the clusters. In this paper, we ask whether a large language model can amplify an expert's guidance to enable query-efficient, few-shot semi-supervised text clustering. We show that LLMs are surprisingly effective at improving clustering. We explore three stages where LLMs can be incorporated into clustering: before clustering (improving input features), during clustering (by providing constraints to the clusterer), and after clustering (using LLMs post-correction). We find incorporating LLMs in the first two stages can routinely provide significant improvements in cluster quality, and that LLMs enable a user to make trade-offs between cost and accuracy to produce desired clusters. We release our code and LLM prompts for the public to use.
摘要
（traditional unsupervised clustering与 semi-supervised clustering的区别在于， semi-supervised clustering 允许用户提供有意义的数据结构，这帮助 clustering 算法与用户的意思相符。现有的 semi-supervised clustering 方法需要专家的重要反馈，以提高几何。在这篇文章中，我们询问 whether 大型语言模型可以增强专家的指导，以实现问题提交、少量 semi-supervised text clustering。我们发现 LLMs surprisingly effective 的提高 clustering。我们探索了在 clustering 中应用 LLMs 的三个阶段： before clustering（改善输入特征）、during clustering（通过提供约束给 clustering 算法）和 after clustering（使用 LLMs 后修）。我们发现在第一个和第二个阶段中 incorporating LLMs 可以提供重要的改善，并且 LLMs 允许用户做成本和准确之间的调整，以生成适当的几何。我们发布了我们的代码和 LLM 提示，以便公众使用。）
Make Text Unlearnable: Exploiting Effective Patterns to Protect Personal Data
methods: 我们基于 Huang et al. (2021) 的二级优化方法，通过梯度基于搜索技术生成不可学习的文本。然而，这种方法具有实际限制，例如需要批处理实例和模型架构知识，这些知识不是普通用户可以访问自己数据的限制。另外，即使使用语义保持约束，不可学习的噪声仍可能改变文本的语义。
results: 我们提取了生成不可学习文本中的简单模式，并证明这些模式可以使文本保持不可学习性，即使用户只有有限的数据和模型知识。此外，这些模式不是特定实例或数据集的，因此用户可以轻松地应用它们于文本分类和问答任务。我们还开源了生成不可学习文本的代码和评估不可学习噪声的代码，以便公共和未来研究中使用。Abstract
This paper addresses the ethical concerns arising from the use of unauthorized public data in deep learning models and proposes a novel solution. Specifically, building on the work of Huang et al. (2021), we extend their bi-level optimization approach to generate unlearnable text using a gradient-based search technique. However, although effective, this approach faces practical limitations, including the requirement of batches of instances and model architecture knowledge that is not readily accessible to ordinary users with limited access to their own data. Furthermore, even with semantic-preserving constraints, unlearnable noise can alter the text's semantics. To address these challenges, we extract simple patterns from unlearnable text produced by bi-level optimization and demonstrate that the data remains unlearnable for unknown models. Additionally, these patterns are not instance- or dataset-specific, allowing users to readily apply them to text classification and question-answering tasks, even if only a small proportion of users implement them on their public content. We also open-source codes to generate unlearnable text and assess unlearnable noise to benefit the public and future studies.
摘要
To address these challenges, we extract simple patterns from unlearnable text produced by bi-level optimization and demonstrate that the data remains unlearnable for unknown models. These patterns are not instance- or dataset-specific, allowing users to readily apply them to text classification and question-answering tasks, even if only a small proportion of users implement them on their public content. We also open-source our codes to generate unlearnable text and assess unlearnable noise to benefit the public and future studies.
Don’t Stop Self-Supervision: Accent Adaptation of Speech Representations via Residual Adapters
results: 研究获得了强大的词音误差减少（WERR）值，对于4种口音都获得了良好的 результа。在所有4种口音中，使用自愿式适应器得到了22.7%的WERR减少，而使用整个Encoder进行适应得到了25.1%的WERR减少。Abstract
Speech representations learned in a self-supervised fashion from massive unlabeled speech corpora have been adapted successfully toward several downstream tasks. However, such representations may be skewed toward canonical data characteristics of such corpora and perform poorly on atypical, non-native accented speaker populations. With the state-of-the-art HuBERT model as a baseline, we propose and investigate self-supervised adaptation of speech representations to such populations in a parameter-efficient way via training accent-specific residual adapters. We experiment with 4 accents and choose automatic speech recognition (ASR) as the downstream task of interest. We obtain strong word error rate reductions (WERR) over HuBERT-large for all 4 accents, with a mean WERR of 22.7% with accent-specific adapters and a mean WERR of 25.1% if the entire encoder is accent-adapted. While our experiments utilize HuBERT and ASR as the downstream task, our proposed approach is both model and task-agnostic.
摘要
自然语言处理中的自我超vision学习方法可以自然地学习大量无标注语音 Corpora 中的语音特征。然而，这些表示可能受到大量数据的标准化影响，并且在非典型、非本地口音 speaker 人群中表现不佳。基于当前顶尖 HuBERT 模型的基线，我们提出了一种parameter-efficient的自我超vision adaptation方法，通过在 residual adapters 上进行听话特征的自适应。我们在4种口音上进行了实验，选择了自动化语音识别（ASR）作为下游任务。我们得到了对 HuBERT-large 的强大单词错误率减少（WERR），对所有4种口音都有很好的表现，平均WERR为22.7%，对整个编码器进行了全面适应时平均WERR为25.1%。虽然我们的实验使用了 HuBERT 和 ASR 作为下游任务，但我们的提出的方法是模型和任务无关的。
A Dual-Stream Recurrence-Attention Network with Global-Local Awareness for Emotion Recognition in Textual Dialogue
For: 这个论文的目的是提出一种简单的 dual-stream Recurrence-Attention Network (DualRAN)，用于实现 Emotion Recognition in Conversation (ERC) 任务。* Methods: 这个模型使用了 RNN 和 Multi-head ATtention network (MAT) 的组合，并提出了一种新的 dual-stream 结构，以模型对话的全局和局部上下文信息。* Results: 实验结果表明，提出的模型在四个常用的 benchmark 数据集上表现出色，超过了所有基eline。 并且，我们进行了一系列的ablation study，以证明每个组件的效果。Abstract
In real-world dialogue systems, the ability to understand the user's emotions and interact anthropomorphically is of great significance. Emotion Recognition in Conversation (ERC) is one of the key ways to accomplish this goal and has attracted growing attention. How to model the context in a conversation is a central aspect and a major challenge of ERC tasks. Most existing approaches are generally unable to capture both global and local contextual information efficiently, and their network structures are too complex to design. For this reason, in this work, we propose a straightforward Dual-stream Recurrence-Attention Network (DualRAN) based on Recurrent Neural Network (RNN) and Multi-head ATtention network (MAT). The proposed model eschews the complex network structure of current methods and focuses on combining recurrence-based methods with attention-based methods. DualRAN is a dual-stream structure mainly consisting of local- and global-aware modules, modeling a conversation from distinct perspectives. To achieve the local-aware module, we extend the structure of RNN, thus enhancing the expressive capability of the network. In addition, we develop two single-stream network variants for DualRAN, i.e., SingleRANv1 and SingleRANv2. We conduct extensive experiments on four widely used benchmark datasets, and the results reveal that the proposed model outshines all baselines. Ablation studies further demonstrate the effectiveness of each component.
摘要
在实际对话系统中，理解用户的情感和人工智能交互是非常重要的。情感识别在对话中（ERC）已经吸引了越来越多的关注，并且成为了解决这一问题的中心方向之一。在ERC任务中，模型对话上下文的捕捉是中心问题，也是一个主要挑战。现有的大多数方法都不能够有效地捕捉对话中的全局和局部上下文信息，其网络结构也很复杂，设计很难。因此，在这项工作中，我们提出了一种简单的双流回归注意网络（DualRAN），基于循环神经网络（RNN）和多头注意网络（MAT）。我们的模型弃用现有方法的复杂网络结构，而选择结合循环方法和注意方法来实现。DualRAN的主要结构是一种双流结构，主要由本地和全局意识模块组成，从不同的角度模型对话。为了提高网络表达能力，我们在RNN结构中进行了扩展。此外，我们还开发了两种单流网络变体，即SingleRANv1和SingleRANv2。我们在四个常用的 benchmark 数据集上进行了广泛的实验，结果显示，我们的模型胜过所有基准值。细化分析还证明了每个组件的有效性。
Low-Resource Cross-Lingual Adaptive Training for Nigerian Pidgin
results: 研究显示，英语预训模型在英文-pidgin任务上比多语言语模型更强，具有最多2.38 BLEU 提升；同时，通过数据增强和反向翻译来进行任务适应训练，可以对模型性能产生显著的影响。Abstract
Developing effective spoken language processing systems for low-resource languages poses several challenges due to the lack of parallel data and limited resources for fine-tuning models. In this work, we target on improving upon both text classification and translation of Nigerian Pidgin (Naija) by collecting a large-scale parallel English-Pidgin corpus and further propose a framework of cross-lingual adaptive training that includes both continual and task adaptive training so as to adapt a base pre-trained model to low-resource languages. Our studies show that English pre-trained language models serve as a stronger prior than multilingual language models on English-Pidgin tasks with up to 2.38 BLEU improvements; and demonstrate that augmenting orthographic data and using task adaptive training with back-translation can have a significant impact on model performance.
摘要
developing effective spoken language processing systems for low-resource languages poses several challenges due to the lack of parallel data and limited resources for fine-tuning models. in this work, we target on improving upon both text classification and translation of nigerian pidgin (naija) by collecting a large-scale parallel english-pidgin corpus and further propose a framework of cross-lingual adaptive training that includes both continual and task adaptive training so as to adapt a base pre-trained model to low-resource languages. our studies show that english pre-trained language models serve as a stronger prior than multilingual language models on english-pidgin tasks with up to 2.38 bleu improvements; and demonstrate that augmenting orthographic data and using task adaptive training with back-translation can have a significant impact on model performance.
Effective Matching of Patients to Clinical Trials using Entity Extraction and Neural Re-ranking
results: 研究结果表明，包含病人描述段落的试验条件部分对lexical模型的重levance分数有很大的影响，而数据增强技术可以提高试验检索的有效率。基于我们的训练方法，重新排序方法可以持续提高试验检索的精度，比较效果高于大型神经网络模型，即使用有限的训练数据。Abstract
Clinical trials (CTs) often fail due to inadequate patient recruitment. This paper tackles the challenges of CT retrieval by presenting an approach that addresses the patient-to-trials paradigm. Our approach involves two key components in a pipeline-based model: (i) a data enrichment technique for enhancing both queries and documents during the first retrieval stage, and (ii) a novel re-ranking schema that uses a Transformer network in a setup adapted to this task by leveraging the structure of the CT documents. We use named entity recognition and negation detection in both patient description and the eligibility section of CTs. We further classify patient descriptions and CT eligibility criteria into current, past, and family medical conditions. This extracted information is used to boost the importance of disease and drug mentions in both query and index for lexical retrieval. Furthermore, we propose a two-step training schema for the Transformer network used to re-rank the results from the lexical retrieval. The first step focuses on matching patient information with the descriptive sections of trials, while the second step aims to determine eligibility by matching patient information with the criteria section. Our findings indicate that the inclusion criteria section of the CT has a great influence on the relevance score in lexical models, and that the enrichment techniques for queries and documents improve the retrieval of relevant trials. The re-ranking strategy, based on our training schema, consistently enhances CT retrieval and shows improved performance by 15\% in terms of precision at retrieving eligible trials. The results of our experiments suggest the benefit of making use of extracted entities. Moreover, our proposed re-ranking schema shows promising effectiveness compared to larger neural models, even with limited training data.
摘要
临床试验（CT）常常失败因为缺乏合适的病人招募。这篇论文解决了CT检索的挑战，提出了一种管道模型中的两个关键组成部分：（i）用于提高查询和文档的数据增强技术，以及（ii）基于Transformer网络的一种新的重新排名方法。我们使用命名实体识别和否定检测在病人描述和试验条件中。我们进一步将病人描述和试验条件分类为当前、过去和家族医疗状况。这些提取的信息用于提高病情和药物提及的重要性在查询和索引中。此外，我们提议一种两步训练方案，用于在重新排名过程中使用Transformer网络。第一步是匹配病人信息与试验描述部分，第二步是匹配病人信息与试验条件部分。我们的实验结果表明，试验条件部分对lexical模型的相关分数有着很大的影响，而我们的增强技术可以提高有关试验的检索。我们的重新排名策略基于我们的训练方案，在提高CT检索中表现出了显著的优异性，相比于大型神经网络，我们的方案更有效率，即使用限制的训练数据。
Revisiting Sample Size Determination in Natural Language Understanding
paper_authors: Ernie Chang, Muhammad Hassan Rashid, Pin-Jie Lin, Changsheng Zhao, Vera Demberg, Yangyang Shi, Vikas Chandra
for: 预测模型性能，减少数据标注预算
methods: 使用小量训练样本预测最大可达性能，并进行ablation study
results: 能够预测模型性能 within a small margin of mean absolute error (~ 0.9%) with only 10% dataAbstract
Knowing exactly how many data points need to be labeled to achieve a certain model performance is a hugely beneficial step towards reducing the overall budgets for annotation. It pertains to both active learning and traditional data annotation, and is particularly beneficial for low resource scenarios. Nevertheless, it remains a largely under-explored area of research in NLP. We therefore explored various techniques for estimating the training sample size necessary to achieve a targeted performance value. We derived a simple yet effective approach to predict the maximum achievable model performance based on small amount of training samples - which serves as an early indicator during data annotation for data quality and sample size determination. We performed ablation studies on four language understanding tasks, and showed that the proposed approach allows us to forecast model performance within a small margin of mean absolute error (~ 0.9%) with only 10% data.
摘要
知道需要如多数据点标注以达到特定模型性能的准确数量是一项非常有利的进展，它适用于活动学习和传统数据标注，特别是在低资源情况下。然而，这还是一个相对较少研究的领域。我们因此探索了各种方法来估算需要达到目标性能值的训练样本数量。我们提出了一种简单 yet effective的方法，可以在小量训练样本下预测模型性能的最大可能值，并作为数据标注过程中的数据质量和样本数确定指标。我们在四种语言理解任务上进行了减少研究，并证明了我们的方法可以在只有10%的数据下预测模型性能，与实际值之间的差异在0.9%左右。