by HPC_ZY

期刊论文《A Benchmarking Tool to Evaluate Computer Tomography Perfusion Infarct Core Predictions Against a DWI Standard》阅读笔记

Introduction

Several studies have identified CTP parameters that could serve as surrogates for DWI imaging. These studies have, however, reported different results in terms of the optimal perfusion parameter (e.g. CBF vs CBV) and its optimal threshold to identify the ischemic core. This variability is attributed to differences in (1) CTP processing algorithms, (2) definitions of the gold standard for ischemic core, and (3) implementations of ROC analysis (definition of true negative region, ROC analysis per-patient or all voxels pooled)1–9 (Table 1).
To address the heterogeneity of prior studies, we developed a benchmarking tool that can be used to evaluate CTP post-processing software algorithms in a standardized way. We used this tool to evaluate the performance of in-house developed CTP post-processing software algorithms.
有研究证明CTP可以替代DWI成像。
但在最佳灌注参数（例如CBF与CBV）及其识别缺血核心的最佳阈值，他们给出的结果不同。原因在于：
（1）CTP处理算法不同
（2）缺血核心金标准的定义不同
（3）ROC分析的实施不同（真阴性区域的定义，每个患者或所有体素的ROC分析）1-9（表1）。
为了解决以往研究的异质性，RAPID开发了一个基准测试工具，可以用来以标准化的方式评估CTP后处理软件算法。我们使用这个工具来评估内部开发的CTP后处理软件算法的性能。

Materials and methods

下图显示了为评估CTP后处理软件算法而开发的基准测试工具的示意图。

To use the tool, investigators need to (1) generate CTP ischemic core masks (in DICOM format) by processing the included CTP source data (in DICOM format) with their own CTP post-processing software; (2) place their CTP ischemic core masks in a predefined folder structure; and (3) run the benchmarking tool’s analysis executable program with the mask folder as input.
使用流程：
（1）通过使用自己的CTP后处理软件，处理包含的CTP源数据（DICOM格式），生成CTP缺血性核心mask（DICOM格式）；
（2）将CTP缺血性核心mask结果放入事前准备的的文件夹中；
（3）将mask所在文件夹作为输入，运行benchmarking tool软件。

The tool will then generate a performance report of the user’s CTP post-processing software algorithm based on the correspondence between the CTP masks and the tool’s included gold standard DWI lesion masks using multiple metrics.
然后，该工具将根据CTP mask和自带的金标准DWI损伤掩模之间的对应关系，使用多个指标生成用户CTP后处理软件算法的性能报告

Since the purpose of the tool is to provide an objective quantitative evaluation of the performance of CTP post-processing algorithms, all steps except for the perfusion algorithm itself, are standardized (Figure 1).
由于该工具的目的是提供CTP后处理算法性能的客观定量评估，因此除了灌注算法本身外，所有步骤都是标准化的（图1）。

In order to ensure the credibility and integrity of these steps, the tool features fully transparent and commented source code (Matlab v. R2013b, MathWorks Inc., Nattick, MA, USA) and a set of images for each case to verify the appropriateness of co-registration and DWI lesion outlines (Figure 2).
为了确保这些步骤的可靠性和完整性，该工具提供完全透明和注释性的源代码（Matlab v.R2013b）和测试病例（每个病例的一组图像，用于验证和DWI病变轮廓的适当性（如下图2）。也就是说，验证你自己写的CTP算法时，要用该软件包自带的数据

The benchmarking tool has only two technical requirements of the CTP software that is evaluated, which ensures compatibility with all open-source and most commercial CTP software packages:
(1) the CTP software should output infarct mask data in the same pixel dimensions as the provided CTP input data (256 256 matrix);
(2) the CTP software should not perform motion correction or spatial down-sampling because the CTP input data has already been motion corrected to ensure spatial correspondence with the coregistered DWI lesion outlines.
基准测试工具对所评估的CTP软件只有两个技术要求，确保与所有开源和大多数商业CTP软件包兼容：
（1） CTP软件需输出与提供的CTP输入数据相同的像素尺寸的梗死掩模数据（256*256）；
（2） CTP软件不应执行运动校正或空间下采样，因为该数据集CTP输入数据已经进行了运动校正，以确保与相应的DWI病变轮廓在空间上一致。

A large multicenter imaging dataset from acute stroke patients who underwent back-to-back CTP and DWI imaging within 3 h of each other.
一个大型的多中心影像数据集，来自于在3小时内连续接受CTP和DWI成像的急性脑卒中患者。

评价参数及定义如下图

Our CTP software algorithm was first run with its default rCBF threshold (rCBF <30%) to generate ischemic core segmentation masks for each case. Next, the CTP software was set up to produce 27 segmentation masks of the ischemic core per case. This was based on 27 rCBF thresholds ranging from 0 to 1 with the finest resolution (0.02) between 0.2 and 0.5 as prior studies and previous experience with our perfusion algorithm indicated this range to be the most relevant for segmentation of the ischemic core.4 Three optimal rCBF thresholds were determined:
(1) a volume-optimized rCBF threshold defined as the threshold at which the mean difference between predicted core volumes and observed DWI volumes was minimized
(2) a volume-optimized rCBF threshold defined as the threshold at which the median absolute difference between predicted core volumes and observed DWI volumes was minimized
(3) a voxel-optimized rCBF threshold defined as the threshold at which the Youden’s index based on ROC analysis for predicting DWI positive voxels was maximized.
CTP软件算法首先使用其默认的rCBF阈值（rCBF<30%）来为每个病例生成缺血核心分割msak。然后，以分辨率为（0.02）在0.2到0.5之间的27个rCBF阈值，为每个病例生成27个缺血核心的分割mask（因为前期的研究和我们的灌注算法的经验表明这个范围与缺血核心分割最相关）。
确定了三个最佳的rCBF阈值：
（1）体积优化的rCBF阈值阈值定义为预测的核心体积和观察到的DWI体积之间的平均差最小化的阈值，
（2）体积优化的rCBF阈值定义为预测的核心体积和观测的DWI体积之间的中值绝对差最小化的阈值，
（3）体素优化的rCBF阈值定义为基于ROC分析的预测DWI阳性体素的Youden指数最大化的阈值。

there has been no clear movement towards a consensus among scientists of what constitutes an adequate quality for CTP post-processing algorithms, and there is considerable variability in the type of algorithms that are being used. By making our imaging data and evaluation methods available to others, we aim to create a global research environment that is conducive to continuous improvements of CTP post-processing software algorithms.

Our CTP post-processing algorithm showed the smallest difference between CTP and DWI ischemic core estimates at an rCBF threshold <38%.
科学家们对什么是CTP后处理算法的适当质量尚未达成共识，而且所使用的算法类型存在很大的差异。通过将我们的成像数据和评估方法提供给其他人，我们的目标是创造一个有利于不断改进CTP后处理软件算法的全球研究环境。
我们的CTP后处理算法显示，在rCBF阈值<38%时，CTP和DWI缺血核心估计值之间的差异最小。

While we took extreme care to optimize co-registration in the imaging dataset, minor errors are unavoidable because of the many challenges of registering between CTP and DWI modalities, including non-isotropic data, different slice angulations, and inherent distortions in DWI images.
虽然我们非常小心地优化了成像数据集中的共配准，但是由于在CTP和DWI模式之间的注册存在许多挑战，包括非各向同性数据、不同的切片角度和DWI图像中固有的失真，所以小误差是不可避免的。

From a clinical standpoint, a threshold that is more restrictive and thus more specific than the threshold at which the mean volumetric difference is optimized may be desirable, because a more restrictive threshold would err on the side of underestimating the ischemic core.
从临床角度来看，可能需要一个比优化平均体积差的阈值更具限制性和更具体的阈值，因为一个更严格的阈值会导致低估缺血核心。