paper_authors: Andrew H. Song, Mane Williams, Drew F. K. Williamson, Guillaume Jaume, Andrew Zhang, Bowen Chen, Robert Serafin, Jonathan T. C. Liu, Alex Baras, Anil V. Parwani, Faisal Mahmood for:* 这个论文是为了提出一种基于深度学习的3D生物组织图像分析平台，用于评估癌症患者的风险水平和疾病进程。methods:* 该论文使用了多样化的3D生物组织图像技术，包括open-top light-sheet microscopy和微计算tomography，以获取病理样本的3D图像数据。然后，使用深度学习算法对这些数据进行分析和预测。results:* 该论文的实验结果表明，使用3D块分析方法可以提高预测癌症患者的5年生物化学回暂念 outcome的准确率，比传统的2D单片预测方法更高。此外，该论文还发现，capturing更大的生物组织体积可以提高预测性能，并减少采样偏见导致的预测变化。Abstract
Human tissue and its constituent cells form a microenvironment that is fundamentally three-dimensional (3D). However, the standard-of-care in pathologic diagnosis involves selecting a few two-dimensional (2D) sections for microscopic evaluation, risking sampling bias and misdiagnosis. Diverse methods for capturing 3D tissue morphologies have been developed, but they have yet had little translation to clinical practice; manual and computational evaluations of such large 3D data have so far been impractical and/or unable to provide patient-level clinical insights. Here we present Modality-Agnostic Multiple instance learning for volumetric Block Analysis (MAMBA), a deep-learning-based platform for processing 3D tissue images from diverse imaging modalities and predicting patient outcomes. Archived prostate cancer specimens were imaged with open-top light-sheet microscopy or microcomputed tomography and the resulting 3D datasets were used to train risk-stratification networks based on 5-year biochemical recurrence outcomes via MAMBA. With the 3D block-based approach, MAMBA achieves an area under the receiver operating characteristic curve (AUC) of 0.86 and 0.74, superior to 2D traditional single-slice-based prognostication (AUC of 0.79 and 0.57), suggesting superior prognostication with 3D morphological features. Further analyses reveal that the incorporation of greater tissue volume improves prognostic performance and mitigates risk prediction variability from sampling bias, suggesting the value of capturing larger extents of heterogeneous 3D morphology. With the rapid growth and adoption of 3D spatial biology and pathology techniques by researchers and clinicians, MAMBA provides a general and efficient framework for 3D weakly supervised learning for clinical decision support and can help to reveal novel 3D morphological biomarkers for prognosis and therapeutic response.
摘要
人体组织和其中的细胞形成一个基本三维（3D）的微环境。然而，现行的诊断标准仅选择一些二维（2D）的 slice 进行微scopic 诊断，可能会导致采样偏见和误诊断。各种用于捕捉 3D 组织结构的方法已经开发出来，但它们尚未得到了临床应用；手动和计算机的评估这些大量 3D 数据的实际和/或无法提供病人级别的临床信息。我们现在提出了模式不敏感多例学习 дляvolumetric Block分析（MAMBA），一种基于深度学习的平台，可以处理不同成像模式的3D组织图像并预测病人结果。已有的肠癌 Specimen 通过开放式光sheet Microscopy或微计Tomography imaging，并将结果用于通过MAMBA训练基于5年生物化学回报的风险 stratification 网络。使用3D block 的方法，MAMBA在 receiver 操作特征曲线（AUC）中 achievement 0.86和0.74，高于2D传统单片 slice 基于的预测（AUC 0.79和0.57），表明3D形态特征的超越预测。进一步分析表明，包括更大的组织量可以提高预测性能，并减少采样偏见导致的预测变化，表明捕捉更大的多样性3D形态的价值。随着3D空间生物学和病理学技术的快速发展和广泛应用，MAMBA提供了一个通用和有效的3Dweakly supervised learning框架，可以帮助抽取 novel 3D形态生物标志物并提供临床决策支持。
A full-resolution training framework for Sentinel-2 image fusion
results: 在我们的初步实验中，提出的方案已经显示了与supervised方法相当的承诺性。此外，由构建的损失函数，得到的训练网络可以被归类为多resolution分析方法。Abstract
This work presents a new unsupervised framework for training deep learning models for super-resolution of Sentinel-2 images by fusion of its 10-m and 20-m bands. The proposed scheme avoids the resolution downgrade process needed to generate training data in the supervised case. On the other hand, a proper loss that accounts for cycle-consistency between the network prediction and the input components to be fused is proposed. Despite its unsupervised nature, in our preliminary experiments the proposed scheme has shown promising results in comparison to the supervised approach. Besides, by construction of the proposed loss, the resulting trained network can be ascribed to the class of multi-resolution analysis methods.
摘要
这个工作提出了一种新的无监督框架，用于训练深度学习模型以提高遥感二号卫星（Sentinel-2）图像的超分辨率。该方案不需要生成训练数据的分辨率降低过程，而是提出了一种适当的损失函数，以考虑网络预测和输入组件之间的循环一致性。尽管是无监督的，但在我们的初步实验中，该方案已经展示了与批处学习方法相比的扎实表现。此外，由于构建的损失函数，得到的训练网络可以被归类为多分辨率分析方法。
Seeing through the Brain: Image Reconstruction of Visual Perception from Human Brain Signals
methods: 该 paper 提出了一个全面的拓展管道，名为 NeuroImagen，以重构观察到的图像。该管道包括一种新的多级感知信息解码器，以提取多级输出信息。然后，一种幂等扩散模型将利用提取的信息来重构高分辨率的图像。
results: 实验结果表明，该方法可以有效地重构图像，并且对比 traditional methods 表现出优异的数值性能。Abstract
Seeing is believing, however, the underlying mechanism of how human visual perceptions are intertwined with our cognitions is still a mystery. Thanks to the recent advances in both neuroscience and artificial intelligence, we have been able to record the visually evoked brain activities and mimic the visual perception ability through computational approaches. In this paper, we pay attention to visual stimuli reconstruction by reconstructing the observed images based on portably accessible brain signals, i.e., electroencephalography (EEG) data. Since EEG signals are dynamic in the time-series format and are notorious to be noisy, processing and extracting useful information requires more dedicated efforts; In this paper, we propose a comprehensive pipeline, named NeuroImagen, for reconstructing visual stimuli images from EEG signals. Specifically, we incorporate a novel multi-level perceptual information decoding to draw multi-grained outputs from the given EEG data. A latent diffusion model will then leverage the extracted information to reconstruct the high-resolution visual stimuli images. The experimental results have illustrated the effectiveness of image reconstruction and superior quantitative performance of our proposed method.
摘要
⟨SYS⟩视觉是信任的起点，但是人类视觉与认知之间的底层机制仍然是一个谜。随着 neuroscience 和人工智能的Recent advances，我们可以记录人类视觉活动和模拟视觉能力通过计算方法。在这篇论文中，我们关注于基于电encephalography（EEG）数据的视觉刺激重建。由于EEG信号是时间序列格式的动态信号，容易受到噪声的影响，因此处理和提取有用信息需要更多的努力。在这篇论文中，我们提出了一个完整的管道，名为NeuroImagen，用于从EEG信号中重建视觉刺激图像。 Specifically，我们采用了一种新的多级感知信息解码器，以从给定的EEG数据中提取多级输出。然后，一种扩散模型会利用提取的信息来重建高分辨率的视觉刺激图像。实验结果表明了图像重建的效果和我们提出的方法的数量性表现优于其他方法。Note: Simplified Chinese is used in this translation, which is a standardized form of Chinese used in mainland China and Singapore. Traditional Chinese is used in Taiwan and Hong Kong.
Test Time Adaptation for Blind Image Quality Assessment
results: 实验显示，即使仅使用小批量的测试数据，也可以通过更新源模型的批量常数汇总来 дости� Significant improvement in performance。Abstract
While the design of blind image quality assessment (IQA) algorithms has improved significantly, the distribution shift between the training and testing scenarios often leads to a poor performance of these methods at inference time. This motivates the study of test time adaptation (TTA) techniques to improve their performance at inference time. Existing auxiliary tasks and loss functions used for TTA may not be relevant for quality-aware adaptation of the pre-trained model. In this work, we introduce two novel quality-relevant auxiliary tasks at the batch and sample levels to enable TTA for blind IQA. In particular, we introduce a group contrastive loss at the batch level and a relative rank loss at the sample level to make the model quality aware and adapt to the target data. Our experiments reveal that even using a small batch of images from the test distribution helps achieve significant improvement in performance by updating the batch normalization statistics of the source model.
摘要
“对照设计盲目图像质量评估（IQA）算法的改进，但是在测试场景下的分布迁移常会导致这些方法在推导时表现不佳。这为测试时间适应（TTA）技术的研究带来动机。现有的辅助任务和损失函数用于TTA可能不适用于质量感知的改进。在这个工作中，我们介绍两个新的质量相关的辅助任务，一为批量级的集成对比损失函数，另一为样本级的相对排名损失函数，以使模型变得质量感知，适应目标数据。我们的实验表明，甚至仅使用测试分布中的小批量图像，仍可以在更新源模型的批量 нормализаation 统计时取得显著的改进。”
A Multimodal Supervised Machine Learning Approach for Satellite-based Wildfire Identification in Europe
results: 实验结果表明，该方法在热点识别任务中表现了效果。Abstract
The increasing frequency of catastrophic natural events, such as wildfires, calls for the development of rapid and automated wildfire detection systems. In this paper, we propose a wildfire identification solution to improve the accuracy of automated satellite-based hotspot detection systems by leveraging multiple information sources. We cross-reference the thermal anomalies detected by the Moderate-resolution Imaging Spectroradiometer (MODIS) and the Visible Infrared Imaging Radiometer Suite (VIIRS) hotspot services with the European Forest Fire Information System (EFFIS) database to construct a large-scale hotspot dataset for wildfire-related studies in Europe. Then, we propose a novel multimodal supervised machine learning approach to disambiguate hotspot detections, distinguishing between wildfires and other events. Our methodology includes the use of multimodal data sources, such as the ERSI annual Land Use Land Cover (LULC) and the Copernicus Sentinel-3 data. Experimental results demonstrate the effectiveness of our approach in the task of wildfire identification.
摘要
随着自然灾害的频发增长，如野火，需要开发高速自动化野火检测系统。在本文中，我们提出了一种野火识别解决方案，以提高自动化卫星基于热点检测系统的准确性。我们将模拟迪拜摄像机（MODIS）和可见谱偏振辐射仪（VIIRS）热点服务与欧洲森林火灾信息系统（EFFIS）数据库进行交叉引用，构建欧洲大规模热点数据集，用于野火相关研究。然后，我们提出了一种新的多模式超vised机器学习方法，用于分解热点检测结果，将野火与其他事件分开。我们的方法包括使用多模式数据源，如地理信息系统（ERSI）年度土地用途土地覆盖（LULC）数据和科学实验室三号卫星数据。实验结果表明我们的方法在野火识别任务中的有效性。
LLDiffusion: Learning Degradation Representations in Diffusion Models for Low-Light Image Enhancement
results: 在多个 Synthetic 和实际无对数据集上进行了广泛的实验，并证明了与状态艺术方法相比，提出的 LLDiffusion 方法可以更好地修升低光照图像，同时保持图像的颜色准确性。Abstract
Current deep learning methods for low-light image enhancement (LLIE) typically rely on pixel-wise mapping learned from paired data. However, these methods often overlook the importance of considering degradation representations, which can lead to sub-optimal outcomes. In this paper, we address this limitation by proposing a degradation-aware learning scheme for LLIE using diffusion models, which effectively integrates degradation and image priors into the diffusion process, resulting in improved image enhancement. Our proposed degradation-aware learning scheme is based on the understanding that degradation representations play a crucial role in accurately modeling and capturing the specific degradation patterns present in low-light images. To this end, First, a joint learning framework for both image generation and image enhancement is presented to learn the degradation representations. Second, to leverage the learned degradation representations, we develop a Low-Light Diffusion model (LLDiffusion) with a well-designed dynamic diffusion module. This module takes into account both the color map and the latent degradation representations to guide the diffusion process. By incorporating these conditioning factors, the proposed LLDiffusion can effectively enhance low-light images, considering both the inherent degradation patterns and the desired color fidelity. Finally, we evaluate our proposed method on several well-known benchmark datasets, including synthetic and real-world unpaired datasets. Extensive experiments on public benchmarks demonstrate that our LLDiffusion outperforms state-of-the-art LLIE methods both quantitatively and qualitatively. The source code and pre-trained models are available at https://github.com/TaoWangzj/LLDiffusion.
摘要
当前的深度学习方法 для低光照图像提高（LLIE）通常基于像素级映射，学习从对应的数据集中获得的映射。然而，这些方法经常忽略了考虑降低表示，这可能会导致优化结果不佳。在这篇论文中，我们解决这个限制，提出一种基于降低表示的学习方案，使得图像提高更加有效。我们的提议的降低表示学习方案基于降低表示对准特定的降低特征，以便准确地模型低光照图像中的特有降低模式。为此，我们首先提出了一种联合学习框架，用于同时学习图像生成和图像提高。其次，我们开发了一种低光照扩散模型（LLDiffusion），该模型具有一个合理的动态扩散模块。这个模块考虑了图像的颜色地图和隐藏的降低表示，以指导扩散过程。通过这种方式，我们的LLDiffusion可以更好地提高低光照图像，考虑到降低特征和颜色准确性。最后，我们对多个公共 benchmark 上进行了广泛的实验，并证明了我们的LLDiffusion在量化和质量上都超过了现有的LLIE方法。我们的源代码和预训练模型可以在https://github.com/TaoWangzj/LLDiffusion 上获取。
A Weakly Supervised Segmentation Network Embedding Cross-scale Attention Guidance and Noise-sensitive Constraint for Detecting Tertiary Lymphoid Structures of Pancreatic Tumors
For: 这个研究的目的是对腺苷癌病例中的次次性リンパ球结构（TLSs）进行检测，以便诊断和治疗腺苷癌病人。* Methods: 我们提出了一种几何学习（few-shot learning）的弱化监督分类网络，并使用了一个预训练的核� Species segmentation模型和域对抗网络来获得核细胞数据。我们还实现了跨Scale的注意力导引机制，并且将粗细度特征学习自原始血液病理图像，以及我们的设计的核细胞数据注意力。在训练过程中，我们引入了一个体积敏感的嵌入距离函数损失，以减少微小预测错误。* Results: 我们的提案方法在两个收集的数据集上进行了实验，结果显示，该方法可以对TLSs检测精度有所提高，较以传统的监督学习方法为佳。此外，我们还应用了我们的方法来研究TLSs数据中与周围血管浸润的相互关系，并获得了一些临床 statistically significant 的结果。Abstract
The presence of tertiary lymphoid structures (TLSs) on pancreatic pathological images is an important prognostic indicator of pancreatic tumors. Therefore, TLSs detection on pancreatic pathological images plays a crucial role in diagnosis and treatment for patients with pancreatic tumors. However, fully supervised detection algorithms based on deep learning usually require a large number of manual annotations, which is time-consuming and labor-intensive. In this paper, we aim to detect the TLSs in a manner of few-shot learning by proposing a weakly supervised segmentation network. We firstly obtain the lymphocyte density maps by combining a pretrained model for nuclei segmentation and a domain adversarial network for lymphocyte nuclei recognition. Then, we establish a cross-scale attention guidance mechanism by jointly learning the coarse-scale features from the original histopathology images and fine-scale features from our designed lymphocyte density attention. A noise-sensitive constraint is introduced by an embedding signed distance function loss in the training procedure to reduce tiny prediction errors. Experimental results on two collected datasets demonstrate that our proposed method significantly outperforms the state-of-the-art segmentation-based algorithms in terms of TLSs detection accuracy. Additionally, we apply our method to study the congruent relationship between the density of TLSs and peripancreatic vascular invasion and obtain some clinically statistical results.
摘要
在胰腺癌图像中存在辐合细胞结构（TLS）是诊断和治疗胰腺癌的重要预测指标。因此，TLS的检测在胰腺癌诊断和治疗中扮演着关键的角色。然而，通常需要大量的手动标注，这是时间consuming和劳动密集的。在本文中，我们提出了一种几何学学习的弱类划分网络，以实现几何学学习的TLSVessel检测。我们首先通过将核体分割模型和血液识别模型进行结合，获得了血液细胞密度图。然后，我们实现了跨度级别的注意力引导机制，通过同时学习来自原始 histopathology 图像的粗细度特征和我们设计的血液细胞注意力的细胞特征。在训练过程中，我们引入了一种噪声敏感的约束，通过嵌入签名距离函数损失来减少微型预测错误。实验结果表明，我们提出的方法在两个收集的数据集上明显超过了状态部署的 segmentation-based 算法的TLSVessel检测精度。此外，我们通过应用我们的方法来研究胰腺癌中TLS的浓度和周围血管浸润的关系，并获得了一些临床 statistically 有用的结果。
FocalErrorNet: Uncertainty-aware focal modulation network for inter-modal registration error estimation in ultrasound-guided neurosurgery
results: 该论文提出了一种基于3D焦点模ulation的深度学习技术，用于精准评估MRI-iUS注射注射错误，以避免手术中的不良后果。Abstract
In brain tumor resection, accurate removal of cancerous tissues while preserving eloquent regions is crucial to the safety and outcomes of the treatment. However, intra-operative tissue deformation (called brain shift) can move the surgical target and render the pre-surgical plan invalid. Intra-operative ultrasound (iUS) has been adopted to provide real-time images to track brain shift, and inter-modal (i.e., MRI-iUS) registration is often required to update the pre-surgical plan. Quality control for the registration results during surgery is important to avoid adverse outcomes, but manual verification faces great challenges due to difficult 3D visualization and the low contrast of iUS. Automatic algorithms are urgently needed to address this issue, but the problem was rarely attempted. Therefore, we propose a novel deep learning technique based on 3D focal modulation in conjunction with uncertainty estimation to accurately assess MRI-iUS registration errors for brain tumor surgery. Developed and validated with the public RESECT clinical database, the resulting algorithm can achieve an estimation error of 0.59+-0.57 mm.
摘要
在脑肿减除手术中，准确地除去肿瘤组织，保留语言功能区的安全和结果是手术治疗的关键。然而，在手术中的脑组织塑形（脑移动）可以使手术目标移动，使原来的预手术计划无效。实时ultrasound（iUS）已被采用，以提供实时图像，跟踪脑组织塑形。然而，在手术中的质量控制是重要的，因为Difficult 3D 可视化和低对ultrasound的冲击力，导致困难的手动验证。因此，我们提出了一种新的深度学习技术，基于3D 焦点变化，并与不确定性估计，准确地评估MRI-iUS 注册错误，用于脑肿手术。我们在公共的RESECT临床数据库中开发和验证了该算法，其误差估计为0.59±0.57毫米。
Phenotype-preserving metric design for high-content image reconstruction by generative inpainting
paper_authors: Vaibhav Sharma, Artur Yakimovich for:This paper focuses on the problem of image restoration in high-content microscopy datasets, specifically the issue of imaging and sample preparation artefacts in fluorescence microscopy images of cultured cells with labelled nuclei.methods:The authors evaluate state-of-the-art inpainting methods for image restoration, including DeepFill V2 and Edge Connect, and fine-tune these models with relatively little data to faithfully restore microscopy images.results:The authors demonstrate that the area of the region to be restored is more important than shape, and propose a novel phenotype-preserving metric design strategy that quantifies the size and count of restored biological phenotypes like cell nuclei to penalize undesirable manipulation.Abstract
In the past decades, automated high-content microscopy demonstrated its ability to deliver large quantities of image-based data powering the versatility of phenotypic drug screening and systems biology applications. However, as the sizes of image-based datasets grew, it became infeasible for humans to control, avoid and overcome the presence of imaging and sample preparation artefacts in the images. While novel techniques like machine learning and deep learning may address these shortcomings through generative image inpainting, when applied to sensitive research data this may come at the cost of undesired image manipulation. Undesired manipulation may be caused by phenomena such as neural hallucinations, to which some artificial neural networks are prone. To address this, here we evaluate the state-of-the-art inpainting methods for image restoration in a high-content fluorescence microscopy dataset of cultured cells with labelled nuclei. We show that architectures like DeepFill V2 and Edge Connect can faithfully restore microscopy images upon fine-tuning with relatively little data. Our results demonstrate that the area of the region to be restored is of higher importance than shape. Furthermore, to control for the quality of restoration, we propose a novel phenotype-preserving metric design strategy. In this strategy, the size and count of the restored biological phenotypes like cell nuclei are quantified to penalise undesirable manipulation. We argue that the design principles of our approach may also generalise to other applications.
摘要
Optimization of Image Acquisition for Earth Observation Satellites via Quantum Computing
results: 实验结果显示，选择适当的形式和缓存处理方法是解决这个问题的关键。此外，本研究还提供了现有量子计算机可行解决的问题实际大小上限。Abstract
Satellite image acquisition scheduling is a problem that is omnipresent in the earth observation field; its goal is to find the optimal subset of images to be taken during a given orbit pass under a set of constraints. This problem, which can be modeled via combinatorial optimization, has been dealt with many times by the artificial intelligence and operations research communities. However, despite its inherent interest, it has been scarcely studied through the quantum computing paradigm. Taking this situation as motivation, we present in this paper two QUBO formulations for the problem, using different approaches to handle the non-trivial constraints. We compare the formulations experimentally over 20 problem instances using three quantum annealers currently available from D-Wave, as well as one of its hybrid solvers. Fourteen of the tested instances have been obtained from the well-known SPOT5 benchmark, while the remaining six have been generated ad-hoc for this study. Our results show that the formulation and the ancilla handling technique is crucial to solve the problem successfully. Finally, we also provide practical guidelines on the size limits of problem instances that can be realistically solved on current quantum computers.
摘要
卫星图像获取计划是地球观测领域中一个普遍存在的问题，其目标是在给定的轨道过行下找到最佳的图像子集。这个问题可以通过 combinatorial optimization 模型来表示，在人工智能和运筹学社区中已经得到了广泛研究。然而，尽管它具有潜在的 интерес，它在量子计算 Paradigma 中几乎没有被研究。在这种情况下，我们在这篇文章中提出了两种 QUBO 表述方法，使用不同的方法处理非质量的约束。我们通过实验对三个 D-Wave 提供的量子泵浦机和其中一个混合解决方案进行比较，测试了 20 个问题实例。其中 14 个问题来自 SPOT5 标准套件，剩下的 6 个问题被 специаль地设计为这项研究。我们的结果表明，表述和 ancilla 处理技术是解决问题的关键。最后，我们还提供了现有量子计算机上可行解决问题的实际大小限制。