results: 实验结果表明，该方法可以在压缩率和图像质量之间取得平衡，并且与手工编码和其他神经网络编码方法相比，有较高的压缩率和较好的图像质量。Abstract
The design of a neural image compression network is governed by how well the entropy model matches the true distribution of the latent code. Apart from the model capacity, this ability is indirectly under the effect of how close the relaxed quantization is to the actual hard quantization. Optimizing the parameters of a rate-distortion variational autoencoder (R-D VAE) is ruled by this approximated quantization scheme. In this paper, we propose a feature-level frequency disentanglement to help the relaxed scalar quantization achieve lower bit rates by guiding the high entropy latent features to include most of the low-frequency texture of the image. In addition, to strengthen the de-correlating power of the transformer-based analysis/synthesis transform, an augmented self-attention score calculation based on the Hadamard product is utilized during both encoding and decoding. Channel-wise autoregressive entropy modeling takes advantage of the proposed frequency separation as it inherently directs high-informational low-frequency channels to the first chunks and conditions the future chunks on it. The proposed network not only outperforms hand-engineered codecs, but also neural network-based codecs built on computation-heavy spatially autoregressive entropy models.
摘要
neural image compression network 的设计受到 latent code 的真实分布如何匹配 entropy model 的影响。除了模型容量之外，这种能力受到较量化 quantization 的距离实际hard quantization的影响。在这篇文章中，我们提出了一种基于 frequency separation 的特征级解耦，以帮助 relaxed scalar quantization 实现更低的比特率，使高 entropy 的 latent features 包含大多数低频Texture of the image。此外，为强化 transformer 基于分析/synthesis transform 的分割力，我们在编码和解码过程中使用了增强的自注意力分数计算方法，基于 Hadamard 乘法。通道级自适应 entropy modeling 利用了我们提出的频谱分离，因为它自然地将高信息价值的低频通道分配给首 chunk，并将未来 chunk conditional 于它。提出的网络不仅超越了手动设计的编码器，还超越了基于 computation-heavy 空间自相关 entropy model 的神经网络编码器。
Brain MRI Segmentation using Template-Based Training and Visual Perception Augmentation
results: 使用这种方法训练了 mouse、rat、兔子、猩猩和人类大脑MRI 3D U-Net 模型，并实现了分割任务，这种工具可以有效地解决深度学习应用图像分析中的数据有限问题，为研究人员提供一个统一的解决方案，只需要一个图像样本就能训练深度神经网络。Abstract
Deep learning models usually require sufficient training data to achieve high accuracy, but obtaining labeled data can be time-consuming and labor-intensive. Here we introduce a template-based training method to train a 3D U-Net model from scratch using only one population-averaged brain MRI template and its associated segmentation label. The process incorporated visual perception augmentation to enhance the model's robustness in handling diverse image inputs and mitigating overfitting. Leveraging this approach, we trained 3D U-Net models for mouse, rat, marmoset, rhesus, and human brain MRI to achieve segmentation tasks such as skull-stripping, brain segmentation, and tissue probability mapping. This tool effectively addresses the limited availability of training data and holds significant potential for expanding deep learning applications in image analysis, providing researchers with a unified solution to train deep neural networks with only one image sample.
摘要
深度学习模型通常需要充足的训练数据以达到高精度，但获取标注数据可以是时间consuming和劳动密集的。我们介绍了一个模板基本训练方法，用于从零开始训练3D U-Net模型，只使用一个人类大脑MRI模板和其关联的分割标注。该过程包括视觉感知增强，以提高模型对多种图像输入的可以性和避免过拟合。通过这种方法，我们训练了 mouse、rat、marmoset、rhesus和人类大脑MRI 3D U-Net模型，用于实现分解任务 such as skull-stripping、brain segmentation和组织概率地图。这个工具有效地解决了训练数据的有限性问题，并有潜在的扩展深度学习应用于图像分析领域，为研究人员提供了一个统一的解决方案，只需一个图像样本就能训练深度神经网络。
T-UNet: Triplet UNet for Change Detection in High-Resolution Remote Sensing Images
For: 这个研究旨在提出一个新的网络模型，以便更精确地检测遥感图像之间的变化。* Methods: 这个模型使用了一个三条分支Encoder，并且将 triplet Encoder 用于同时提取物件特征和时间间隔图像之间的变化特征。另外，这个模型还具有多条分支空间特征交互模组 (MBSSCA)，以便有效地交互和融合各分支中的特征。* Results: 这个模型可以更精确地检测遥感图像之间的变化，并且可以实时地提取变化的细节信息。Abstract
Remote sensing image change detection aims to identify the differences between images acquired at different times in the same area. It is widely used in land management, environmental monitoring, disaster assessment and other fields. Currently, most change detection methods are based on Siamese network structure or early fusion structure. Siamese structure focuses on extracting object features at different times but lacks attention to change information, which leads to false alarms and missed detections. Early fusion (EF) structure focuses on extracting features after the fusion of images of different phases but ignores the significance of object features at different times for detecting change details, making it difficult to accurately discern the edges of changed objects. To address these issues and obtain more accurate results, we propose a novel network, Triplet UNet(T-UNet), based on a three-branch encoder, which is capable to simultaneously extract the object features and the change features between the pre- and post-time-phase images through triplet encoder. To effectively interact and fuse the features extracted from the three branches of triplet encoder, we propose a multi-branch spatial-spectral cross-attention module (MBSSCA). In the decoder stage, we introduce the channel attention mechanism (CAM) and spatial attention mechanism (SAM) to fully mine and integrate detailed textures information at the shallow layer and semantic localization information at the deep layer.
摘要
distant 感知图像变化检测 aimsto identify 不同时间在同一个区域中的图像差异。 它广泛应用于土地管理、环境监测、灾害评估等领域。 现在，大多数变化检测方法基于 Siamese 网络结构或早期融合结构。 Siamese 结构专注于在不同时间抽取对象特征，但缺乏关注变化信息，这会导致假报警和错过检测。 Early Fusion 结构专注于在不同阶段图像融合后抽取特征，但忽视对象特征在不同时间的变化细节检测，这使得准确地识别变化对象的边缘很难。 为了解决这些问题并获得更加准确的结果，我们提出了一种新的网络， Triplet UNet（T-UNet），基于三个分支编码器。 T-UNet 可以同时提取对象特征和不同时间图像之间的变化特征，通过 triplet 编码器。 为了有效地交互和融合 triplet 编码器中的特征，我们提出了多个分支空间特征交叉注意力模块（MBSSCA）。 在解码阶段，我们引入通道注意力机制（CAM）和空间注意力机制（SAM），以全面挖掘和融合图像的细节信息和semantic 本地化信息。
Generative Image Priors for MRI Reconstruction Trained from Magnitude-Only Images
paper_authors: Guanxiong Luo, Xiaoqing Wang, Mortiz Blumenthal, Martin Schilling, Erik Hans Ulrich Rauf, Raviteja Kotikalapudi, Niels Focke, Martin Uecker
results: 实验结果表明，基于复杂图像的先验比只基于磁场图像的先验更高效。此外，一个训练于更大数据集的先验表现了更高的可靠性。最后，我们发现使用生成型先验比L1-wavelet常量化抑制可以在高抽样率下实现更好的压缩成像。I hope that helps! Let me know if you have any other questions.Abstract
Purpose: In this work, we present a workflow to construct generic and robust generative image priors from magnitude-only images. The priors can then be used for regularization in reconstruction to improve image quality. Methods: The workflow begins with the preparation of training datasets from magnitude-only MR images. This dataset is then augmented with phase information and used to train generative priors of complex images. Finally, trained priors are evaluated using both linear and nonlinear reconstruction for compressed sensing parallel imaging with various undersampling schemes. Results: The results of our experiments demonstrate that priors trained on complex images outperform priors trained only on magnitude images. Additionally, a prior trained on a larger dataset exhibits higher robustness. Finally, we show that the generative priors are superior to L1 -wavelet regularization for compressed sensing parallel imaging with high undersampling. Conclusion: These findings stress the importance of incorporating phase information and leveraging large datasets to raise the performance and reliability of the generative priors for MRI reconstruction. Phase augmentation makes it possible to use existing image databases for training.
摘要
目的：在这项工作中，我们提出了一种工作流程，用于从偏差图像中构建通用和Robust的生成图像先验。这些先验然后可以用于图像重建中的规范化，以提高图像质量。方法：工作流程开始于准备各种训练集，其中包括偏差图像。这些训练集然后被扩展以包括相位信息，并用于训练生成图像先验。最后，我们使用不同的抽样方案进行重建，以评估训练过的先验。结果：我们的实验结果表明，基于复杂图像的先验在重建中表现较好，而且一个基于更大的数据集的先验具有更高的稳定性。此外，我们还证明了这些生成先验在高度抽样下的扩散成像中超过L1-wavelet规范化。结论：这些结果强调了在MRI重建中包含相位信息和利用大型数据集来提高生成先验的性能和可靠性。相位扩展使得可以使用现有的图像库进行训练。
CT Reconstruction from Few Planar X-rays with Application towards Low-resource Radiotherapy
For: 提供了一种方法，使用几（<5）个平面X射影像生成CT量子，并在临床应用中进行了首次评估：放疗规划。* Methods: 使用深度生成模型，基于神经隐式表示法生成三维CT剖图从平面X射影像中。通过在训练过程中使用解剖指导，使模型专注于临床相关特征。* Results: 对于 Thoracic CT 的放疗规划，通过模型生成的剖图，让是ocoenter radiation dose与临床获取的 CT 图像中的 radiation dose差异<1%。此外，模型也比最近的稀疙CT重建基eline在 LIDC 肺CT 数据集上的标准像素和结构级指标（PSNR、SSIM、Dice 分数）上表现更好。Abstract
CT scans are the standard-of-care for many clinical ailments, and are needed for treatments like external beam radiotherapy. Unfortunately, CT scanners are rare in low and mid-resource settings due to their costs. Planar X-ray radiography units, in comparison, are far more prevalent, but can only provide limited 2D observations of the 3D anatomy. In this work, we propose a method to generate CT volumes from few (<5) planar X-ray observations using a prior data distribution, and perform the first evaluation of such a reconstruction algorithm for a clinical application: radiotherapy planning. We propose a deep generative model, building on advances in neural implicit representations to synthesize volumetric CT scans from few input planar X-ray images at different angles. To focus the generation task on clinically-relevant features, our model can also leverage anatomical guidance during training (via segmentation masks). We generated 2-field opposed, palliative radiotherapy plans on thoracic CTs reconstructed by our method, and found that isocenter radiation dose on reconstructed scans have <1% error with respect to the dose calculated on clinically acquired CTs using <=4 X-ray views. In addition, our method is better than recent sparse CT reconstruction baselines in terms of standard pixel and structure-level metrics (PSNR, SSIM, Dice score) on the public LIDC lung CT dataset. Code is available at: https://github.com/wanderinrain/Xray2CT.
摘要
干扰CT扫描是许多临床病情的标准治疗方式，并用于外部束辐射治疗。然而，CT扫描仪在LOW和中等资源设置中罕见，主要因为它们的成本高。相比之下，平面X射线投影机更加普遍，但它们只能提供2D结构的有限观察。在这种情况下，我们提出了一种方法，使用先前数据分布来生成CT卷积体从几个平面X射线图像中。我们还提出了一种深度生成模型，基于神经隐式表示来生成3D CT扫描图像从几个平面X射线图像的不同角度。为了聚焦生成任务在临床相关特征上，我们的模型还可以在训练过程中使用解剖指导（通过分剖排序）。我们使用这种方法生成了2个场 opposed、肺部Palliative radiotherapy计划，并发现在我们重建的CT扫描图像上的辐射剂量与临床获得的CT扫描图像使用4个X射线视图计算的辐射剂量之间的差异小于1%。此外，我们的方法也比最近的稀疏CT重建基准值更高于标准像素级和结构级度指标（PSNR、SSIM、Dice分数）在公共的LIDC肺CT数据集上。代码可以在：https://github.com/wanderinrain/Xray2CT中找到。
paper_authors: Syed M. Arshad, Lee C. Potter, Chong Chen, Yingmin Liu, Preethi Chandrasekaran, Christopher Crabtree, Yuchi Han, Rizwan Ahmad
For: The paper is written to present and validate an outlier rejection method for free-running cardiovascular MRI (CMR) to make it more motion robust.* Methods: The proposed method, called compressive recovery with outlier rejection (CORe), models outliers as an auxiliary variable and enforces MR physics-guided group-sparsity on it, which is jointly estimated with the image using an iterative algorithm.* Results: The simulation studies show that CORe outperforms traditional compressed sensing (CS), robust regression (RR), and another outlier rejection method in terms of normalized mean squared error (NMSE) and structural similarity index (SSIM) across 50 different realizations. The expert reader evaluation of 3D cine images demonstrates that CORe is more effective in suppressing artifacts while maintaining or improving image sharpness. The flow consistency evaluation in 4D flow images shows that CORe yields more consistent flow measurements, especially under exercise stress.Here is the information in Simplified Chinese text:* For: 本研究是为提出并验证一种用于自由运行心血管MRI（CMR）的异常拒绝方法，以提高其震动稳定性。* 方法: 提议的方法是压缩恢复与异常拒绝（CORe），它将异常作为辅助变量，并在这个变量上遵循MR物理指导的群集稀缺，通过迭代算法来同时估计异常和图像。* 结果: 实验研究表明，CORe在50个不同实现中的normalized mean squared error（NMSE）和结构相似指数（SSIM）比CS、RR和另一种异常拒绝方法更高。专家读者评估3D cinema图像表明，CORe更有效地抑制artefacts，保持或改善图像锐度。4D流动图像中的流量一致性评估表明，CORe在运动压力下得到了更一致的流量测量。Abstract
PURPOSE: To present and validate an outlier rejection method that makes free-running cardiovascular MRI (CMR) more motion robust. METHODS: The proposed method, called compressive recovery with outlier rejection (CORe), models outliers as an auxiliary variable that is added to the measured data. We enforce MR physics-guided group-sparsity on the auxiliary variable and jointly estimate it along with the image using an iterative algorithm. For validation, CORe is first compared to traditional compressed sensing (CS), robust regression (RR), and another outlier rejection method using two simulation studies. Then, CORe is compared to CS using five 3D cine and ten rest and stress 4D flow imaging datasets. RESULTS: Our simulation studies show that CORe outperforms CS, RR, and the outlier rejection method in terms of normalized mean squared error (NMSE) and structural similarity index (SSIM) across 50 different realizations. The expert reader evaluation of 3D cine images demonstrates that CORe is more effective in suppressing artifacts while maintaining or improving image sharpness. The flow consistency evaluation in 4D flow images show that CORe yields more consistent flow measurements, especially under exercise stress. CONCLUSION: An outlier rejection method is presented and validated using simulated and measured data. This method can help suppress motion artifacts in a wide range of free-running CMR applications. CODE: MATLAB implementation code is available on GitHub at https://github.com/syedmurtazaarshad/motion-robust-CMR
摘要
目的：提出和验证一种可以使自由运行征Cardiovascular MRI（CMR）更加鲁棒于运动 artifacts的方法。方法：提出的方法称为压缩恢复与外围异常值拒绝（CORe），将异常值视为一个辅助变量，并将其添加到测量数据中。我们遵循MR物理指导的群集稀缺性来限制这个辅助变量，并使用迭代算法来同时估计它和图像。验证：首先，CORe与传统的压缩感知（CS）、Robust Regression（RR）和另一种异常值拒绝方法进行了两个 simulations studies。然后，CORe与CS进行了五个3D缓冲和十个Rest和奋斗4D流图像数据集的比较。结果：我们的 simulations studies表明，CORe在NMSE和SSIM方面与CS、RR和另一种异常值拒绝方法都有更好的性能，并且在50个不同的实现中保持稳定。专业读者对3D缓冲图像进行了评估，表明CORe更有效地抑制artefacts，同时保持或改善图像的锐度。在4D流图像中，CORe得到了更一致的流量测量结果，特别是在运动压力下。结论：在自由运行CMR应用中，我们提出了一种可以抑制运动artefacts的异常值拒绝方法，并通过 simulations 和实验验证了其效果。代码：MATLAB实现代码可以在 GitHub 上找到，https://github.com/syedmurtazaarshad/motion-robust-CMR。
Diffusion Models for Counterfactual Generation and Anomaly Detection in Brain Images
results: 这个方法可以将健康样本转换为疾病影像，并且可以提高这些影像的重建精度。在实验中，这个方法比于其他弱型指导方法，对于stroke病变和脑癌分类而言，可以提高DICE score。Abstract
Segmentation masks of pathological areas are useful in many medical applications, such as brain tumour and stroke management. Moreover, healthy counterfactuals of diseased images can be used to enhance radiologists' training files and to improve the interpretability of segmentation models. In this work, we present a weakly supervised method to generate a healthy version of a diseased image and then use it to obtain a pixel-wise anomaly map. To do so, we start by considering a saliency map that approximately covers the pathological areas, obtained with ACAT. Then, we propose a technique that allows to perform targeted modifications to these regions, while preserving the rest of the image. In particular, we employ a diffusion model trained on healthy samples and combine Denoising Diffusion Probabilistic Model (DDPM) and Denoising Diffusion Implicit Model (DDIM) at each step of the sampling process. DDPM is used to modify the areas affected by a lesion within the saliency map, while DDIM guarantees reconstruction of the normal anatomy outside of it. The two parts are also fused at each timestep, to guarantee the generation of a sample with a coherent appearance and a seamless transition between edited and unedited parts. We verify that when our method is applied to healthy samples, the input images are reconstructed without significant modifications. We compare our approach with alternative weakly supervised methods on IST-3 for stroke lesion segmentation and on BraTS2021 for brain tumour segmentation, where we improve the DICE score of the best competing method from $0.6534$ to $0.7056$.
摘要
Segmentation masks of pathological areas are useful in many medical applications, such as brain tumour and stroke management. Moreover, healthy counterfactuals of diseased images can be used to enhance radiologists' training files and to improve the interpretability of segmentation models. In this work, we present a weakly supervised method to generate a healthy version of a diseased image and then use it to obtain a pixel-wise anomaly map. To do so, we start by considering a saliency map that approximately covers the pathological areas, obtained with ACAT. Then, we propose a technique that allows to perform targeted modifications to these regions, while preserving the rest of the image. In particular, we employ a diffusion model trained on healthy samples and combine Denoising Diffusion Probabilistic Model (DDPM) and Denoising Diffusion Implicit Model (DDIM) at each step of the sampling process. DDPM is used to modify the areas affected by a lesion within the saliency map, while DDIM guarantees reconstruction of the normal anatomy outside of it. The two parts are also fused at each timestep, to guarantee the generation of a sample with a coherent appearance and a seamless transition between edited and unedited parts. We verify that when our method is applied to healthy samples, the input images are reconstructed without significant modifications. We compare our approach with alternative weakly supervised methods on IST-3 for stroke lesion segmentation and on BraTS2021 for brain tumour segmentation, where we improve the DICE score of the best competing method from $0.6534$ to $0.7056$.
Predicting Ki67, ER, PR, and HER2 Statuses from H&E-stained Breast Cancer Images
paper_authors: Amir Akbarnejad, Nilanjan Ray, Penny J. Barnes, Gilbert Bigras
for: The paper aims to investigate whether machine learning methods can accurately predict molecular information from histomorphology.
methods: The authors built a large-scale dataset of 185538 images with reliable measurements for Ki67, ER, PR, and HER2 statuses, using mirrored images of H&E and IHC assays. They used a standard ViT-based pipeline to train the classifiers and achieved prediction performances around 90% in terms of AUC.
results: The authors showed that the trained classifiers can localize relevant regions, which encourages future work to improve the localizations. They also made their dataset publicly available for further research.Abstract
Despite the advances in machine learning and digital pathology, it is not yet clear if machine learning methods can accurately predict molecular information merely from histomorphology. In a quest to answer this question, we built a large-scale dataset (185538 images) with reliable measurements for Ki67, ER, PR, and HER2 statuses. The dataset is composed of mirrored images of H\&E and corresponding images of immunohistochemistry (IHC) assays (Ki67, ER, PR, and HER2. These images are mirrored through registration. To increase reliability, individual pairs were inspected and discarded if artifacts were present (tissue folding, bubbles, etc). Measurements for Ki67, ER and PR were determined by calculating H-Score from image analysis. HER2 measurement is based on binary classification: 0 and 1+ (IHC scores representing a negative subset) vs 3+ (IHC score positive subset). Cases with IHC equivocal score (2+) were excluded. We show that a standard ViT-based pipeline can achieve prediction performances around 90% in terms of Area Under the Curve (AUC) when trained with a proper labeling protocol. Finally, we shed light on the ability of the trained classifiers to localize relevant regions, which encourages future work to improve the localizations. Our proposed dataset is publicly available: https://ihc4bc.github.io/
摘要
尽管机器学习和数字 PATHOLOGY 技术已经得到了进步，但是目前还没有确切地知道机器学习方法是否可以从 histomorphology 中精确预测分子信息。为了回答这个问题，我们创建了一个大规模数据集 (185538 张图像)，其中包含可靠的测量值 для Ki67、ER、PR 和 HER2 状况。这个数据集包括 H\&E 和相关的免疫染色技术 (IHC) 图像的相互镜像 (Ki67、ER、PR 和 HER2)。这些图像通过注册进行镜像。为了增加可靠性，我们 manually 检查并排除了ifacts 存在的个体对 (肿瘤卷绕、气泡等)。我们使用 H-Score 来计算 Ki67、ER 和 PR 的测量值，而 HER2 测量值则基于二分类：0 和 1+ (IHC 分数表示负 subsets) vs 3+ (IHC 分数表示正 subsets)。我们排除了 IHC equivocal 分数 (2+) 的情况。我们显示，使用标准 ViT-based 管道可以在训练时达到约 90% 的区域 beneath the curve (AUC) 性能。最后，我们探讨了训练的分类器是否可以准确地呈现相关区域，这有助于未来的工作。我们提供的数据集可以在以下链接中下载：https://ihc4bc.github.io/