paper_authors: Di Xu, Qihui Lyu, Dan Ruan, Ke Sheng
For: breast tissue differentiation* Methods: dual-energy computed tomography (DECT), deep learning (DL) methods, non-recursive setup, raw projection data* Results: high-fidelity decomposition of adipose, calcification, and fibroglandular materials with low RMSE, MAE, negative PSNR, and SSIM compared to ground truth, fast inference time (<1s) on a 4xRTX A6000 GPU cluster.Abstract
Dual-energy computed tomography (DECT) utilizes separate X-ray energy spectra to improve multi-material decomposition (MMD) for various diagnostic applications. However accurate decomposing more than two types of material remains challenging using conventional methods. Deep learning (DL) methods have shown promise to improve the MMD performance, but typical approaches of conducing DL-MMD in the image domain fail to fully utilize projection information or under iterative setup are computationally inefficient in both training and prediction. In this work, we present a clinical-applicable MMD (>2) framework rFast-MMDNet, operating with raw projection data in non-recursive setup, for breast tissue differentiation. rFast-MMDNet is a two-stage algorithm, including stage-one SinoNet to perform dual energy projection decomposition on tissue sinograms and stage-two FBP-DenoiseNet to perform domain adaptation and image post-processing. rFast-MMDNet was tested on a 2022 DL-Spectral-Challenge breast phantom dataset. The two stages of rFast-MMDNet were evaluated separately and then compared with four noniterative reference methods including a direct inversion method (AA-MMD), an image domain DL method (ID-UNet), AA-MMD/ID-UNet + DenoiseNet and a sinogram domain DL method (Triple-CBCT). Our results show that models trained from information stored in DE transmission domain can yield high-fidelity decomposition of the adipose, calcification, and fibroglandular materials with averaged RMSE, MAE, negative PSNR, and SSIM of 0.004+/-~0, 0.001+/-~0, -45.027+/-~0.542, and 0.002+/-~0 benchmarking to the ground truth, respectively. Training of entire rFast-MMDNet on a 4xRTX A6000 GPU cluster took a day with inference time <1s. All DL methods generally led to more accurate MMD than AA-MMD. rFast-MMDNet outperformed Triple-CBCT, but both are superior to the image-domain based methods.
摘要
dual-energy computed tomography (DECT) 利用不同的X射线能谱spectrum来提高多种材料分解(MMD)的精度，以满足各种诊断应用。然而，使用传统方法对多于两种材料的分解仍然是一个挑战。深度学习(DL)方法已经展示了改进MMD性能的投入，但通常在图像域中进行DL-MMD会不全利用投影信息，或者在迭代设置下 computationally inefficient。在这项工作中，我们提出了一种临床可用的MMD(>2)框架，称为rFast-MMDNet，可以在原始投影数据上进行非递归设置。rFast-MMDNet包括两个阶段：第一阶段SinoNet用于在组织射ogram上进行双能量投影分解，第二阶段FBP-DenoiseNet用于适应频谱和图像后处理。我们在2022年DL-Spectral-Challenge乳腺phantom数据集上测试了rFast-MMDNet。我们将两个阶段的rFast-MMDNet分别评估，并与四种非迭代参照方法进行比较，包括直接投影方法(AA-MMD)、图像域DL方法(ID-UNet)、AA-MMD/ID-UNet+DenoiseNet和投影域DL方法(Triple-CBCT)。我们的结果显示，基于DE传输域信息的模型可以很准确地分解脂肪、 calcification和 fibroglandular材料的均值RMSE、MAE、负PSNR和SSIM分别为0.004±~0、0.001±~0、-45.027±~0.542和0.002±~0。这与真实值相对较好。在一个4xRTX A6000 GPU集群上训练整个rFast-MMDNet只需一天时间，并且在预测中时间<1s。所有DL方法都比AA-MMD更精准地进行MMD，而rFast-MMDNet超过了Triple-CBCT，但两者都比图像域基本方法更好。
Fast, multicolour optical sectioning over extended fields of view by combining interferometric SIM with machine learning
paper_authors: Edward N. Ward, Rebecca M. McClelland, Jacob R. Lamb, Roger Rubio-Sánchez, Charles N. Christensen, Bismoy Mazumder, Sofia Kapsiani, Luca Mascheroni, Lorenzo Di Michele, Gabriele S. Kaminski Schierle, Clemens F. Kaminski
results: 在silico validate和实验中，能够实现高速、高对比度的图像重建，并且可以在不同的样品上进行图像捕捉，包括活和Fixed生物细胞以及人工生物系统。Abstract
Structured illumination can reject out-of-focus signal from a sample, enabling high-speed and high-contrast imaging over large areas with widefield detection optics. Currently, this optical-sectioning technique is limited by image reconstruction artefacts and the need for sequential imaging of multiple colour channels. We combine multicolour interferometric pattern generation with machine-learning processing, permitting high-contrast, real-time reconstruction of image data. The method is insensitive to background noise and unevenly phase-stepped illumination patterns. We validate the method in silico and demonstrate its application on diverse specimens, ranging from fixed and live biological cells to synthetic biosystems, imaging at up to 37 Hz across a 44 x 44 $\mu m^2$ field of view.
摘要
《结构化照明可以排除样本上的不锐化信号，允许高速和高对比度的成像，覆盖大面积的广角探测仪。目前，这种光学分 slice 技术受到图像重建瑕疵和多色通道的顺序扫描限制。我们将多色干扰Pattern生成与机器学习处理相结合，允许高对比度、实时重建图像数据。该方法具有背景噪声和不均分阶段照明模式的敏感性。我们通过计算机模拟和实验 validate 该方法，并在不同的样本上应用，包括固定和活体细胞、人造生物系统，成像频率达到 37 Hz，扫描领域为 44 x 44 $\mu m^2$。
UAV Immersive Video Streaming: A Comprehensive Survey, Benchmarking, and Open Challenges
results: 研究发现，使用hardware实现的HEVC编码器可以实现最佳的编码效率和复杂度平衡，而使用软件实现的AV1编码器在编码效率方面表现出色。此外，研究还展示了一个实际的360度视频流传输测试环境，使用5G无线网络控制无人机。Abstract
Over the past decade, the utilization of UAVs has witnessed significant growth, owing to their agility, rapid deployment, and maneuverability. In particular, the use of UAV-mounted 360-degree cameras to capture omnidirectional videos has enabled truly immersive viewing experiences with up to 6DoF. However, achieving this immersive experience necessitates encoding omnidirectional videos in high resolution, leading to increased bitrates. Consequently, new challenges arise in terms of latency, throughput, perceived quality, and energy consumption for real-time streaming of such content. This paper presents a comprehensive survey of research efforts in UAV-based immersive video streaming, benchmarks popular video encoding schemes, and identifies open research challenges. Initially, we review the literature on 360-degree video coding, packaging, and streaming, with a particular focus on standardization efforts to ensure interoperability of immersive video streaming devices and services. Subsequently, we provide a comprehensive review of research efforts focused on optimizing video streaming for timevarying UAV wireless channels. Additionally, we introduce a high resolution 360-degree video dataset captured from UAVs under different flying conditions. This dataset facilitates the evaluation of complexity and coding efficiency of software and hardware video encoders based on popular video coding standards and formats, including AVC/H.264, HEVC/H.265, VVC/H.266, VP9, and AV1. Our results demonstrate that HEVC achieves the best trade-off between coding efficiency and complexity through its hardware implementation, while AV1 format excels in coding efficiency through its software implementation, specifically using the libsvt-av1 encoder. Furthermore, we present a real testbed showcasing 360-degree video streaming over a UAV, enabling remote control of the drone via a 5G cellular network.
摘要
过去一个十年，无人机（UAV）的应用已经经历了显著的增长，归功于它们的灵活、快速部署和掌控能力。尤其是通过无人机搭载的360度摄像头捕捉全天候视频，使得观看体验更加 immerse 和真实，达到6度自由度（6DoF）。然而，实现这种 immerse 体验需要编码全天候视频的高分辨率，导致带宽和延迟问题的增加。这篇论文提出了关于无人机基于全天候视频流式的研究努力，比较各种视频编码方案，并标识了未来研究中的挑战。首先，我们回顾了360度视频编码、封装和流式的文献，尤其是标准化努力，以确保全天候视频流式设备和服务的互operability。然后，我们提供了全面的研究努力，旨在优化视频流式在时变UAV无线通道中的性能。此外，我们还提供了高分辨率360度视频 dataset， captured from UAVs under different flying conditions。这个 dataset 可以用于评估不同视频编码标准和格式的软硬件编码器 Complexity and coding efficiency。我们的结果显示，HEVC 在硬件实现中实现了最佳的平衡 между编码效率和复杂性，而 AV1 格式在软件实现中在编码效率方面表现出色。此外，我们还提供了一个实际的测试环境，让读者通过5G 无线网络 remote control UAV。
Harmonization-enriched domain adaptation with light fine-tuning for multiple sclerosis lesion segmentation
results: 实验表明，将一shot 适应数据与融合培处数据混合使用可以超越使用各自数据源时的性能。此外，只需要微调2-5个轮次，就可以达到最佳性能。Abstract
Deep learning algorithms utilizing magnetic resonance (MR) images have demonstrated cutting-edge proficiency in autonomously segmenting multiple sclerosis (MS) lesions. Despite their achievements, these algorithms may struggle to extend their performance across various sites or scanners, leading to domain generalization errors. While few-shot or one-shot domain adaptation emerges as a potential solution to mitigate generalization errors, its efficacy might be hindered by the scarcity of labeled data in the target domain. This paper seeks to tackle this challenge by integrating one-shot adaptation data with harmonized training data that incorporates labels. Our approach involves synthesizing new training data with a contrast akin to that of the test domain, a process we refer to as "contrast harmonization" in MRI. Our experiments illustrate that the amalgamation of one-shot adaptation data with harmonized training data surpasses the performance of utilizing either data source in isolation. Notably, domain adaptation using exclusively harmonized training data achieved comparable or even superior performance compared to one-shot adaptation. Moreover, all adaptations required only minimal fine-tuning, ranging from 2 to 5 epochs for convergence.
摘要
深度学习算法利用核磁共振（MR）图像已经达到了自动分割多发性中风（MS）损伤的高水平。然而，这些算法可能会在不同的场景或扫描仪上表现不佳，导致领域泛化错误。一些几 shot或一 shot领域适应技术可能会解决这个问题，但它们可能受到目标领域的标注数据的罕见性的限制。这篇论文的目的是解决这个挑战，通过结合一 shot适应数据与融合标注数据进行协调。我们称之为“对比融合”（contrast harmonization）。我们的实验表明，将一 shot适应数据与融合标注数据混合使用，不仅超过了使用各自的数据源来达到的性能，而且还可以在不需要大量的精度调整下实现。特别是，使用尚未标注的融合训练数据可以达到与一 shot适应数据相同或者甚至更高的性能。此外，所有的适应都需要只需要2-5个熔断 epoch 来达到 converges。
C-Silicon-based metasurfaces for aperture-robust spectrometer/imaging with angle integration
results: 实验表明，提出的方法可以在400nm至800nm的工作带宽内保持谱论器的 spectral consistency，并且可以准确重建入射谱论信号，其准确率高于99%。此外，该研究还实现了一个400x400像素的spectral imaging系统。Abstract
Compared with conventional grating-based spectrometers, reconstructive spectrometers based on spectrally engineered filtering have the advantage of miniaturization because of the less demand for dispersive optics and free propagation space. However, available reconstructive spectrometers fail to balance the performance on operational bandwidth, spectral diversity and angular stability. In this work, we proposed a compact silicon metasurfaces based spectrometer/camera. After angle integration, the spectral response of the system is robust to angle/aperture within a wide working bandwidth from 400nm to 800nm. It is experimentally demonstrated that the proposed method could maintain the spectral consistency from F/1.8 to F/4 (The corresponding angle of incident light ranges from 7{\deg} to 16{\deg}) and the incident hyperspectral signal could be accurately reconstructed with a fidelity exceeding 99%. Additionally, a spectral imaging system with 400x400 pixels is also established in this work. The accurate reconstructed hyperspectral image indicates that the proposed aperture-robust spectrometer has the potential to be extended as a high-resolution broadband hyperspectral camera.
摘要