使用git 中cocoapi的pycocoEvalDemo.ipynb更改
需要：

1 自己验证集的标注

此处使用labelme制作。
格式如下：

{
    "images": [
        {
            "height": 2304,
            "width": 3456,
            "id": 1,
            "file_name": "s1.jpg"
        },
......
    ],
    "categories": [
        {
            "supercategory": "q",
            "id": 1,
            "name": "q"
        },
        {
            "supercategory": "z",
            "id": 2,
            "name": "z"
        },
        {
            "supercategory": "b",
            "id": 3,
            "name": "b"
        },
        {
            "supercategory": "x",
            "id": 4,
            "name": "x"
        }
    ],
    "annotations": [
        {
            "segmentation": [
                [
                    1393.2380952380952,
                    323.2380952380952,
                    1427.2380952380952,
                    383.2380952380952,
                    1322.2380952380952,
                    504.2380952380952,
                    867.2380952380952,
                ]
            ],
            "iscrowd": 0,
            "image_id": 1,
            "bbox": [
                522.0,
                173.0,
                919.0,
                457.0
            ],
            "category_id": 1,
            "id": 1
        }，
       ......

2 对测试集利用模型生成结果json

这里测试了24张图片并保存了bbox的结果
https://blog.****.net/qq_35608277/article/details/88920728
结果为list,其中一个结果格式包含以下几个key：
{“image_id”: 24, “category_id”: 1, “bbox”: [992.66943359375, 570.6240844726562, 1033.0626220703125, 527.0519409179688], “score”: 0.9999897480010986}
bbox此处的格式与labelme中的格式保持一致：矩形框左上角坐标+宽高

3 评估代码

import matplotlib.pyplot as plt
from pycocotools.coco import COCO
from pycocotools.cocoeval import COCOeval
import numpy as np
import skimage.io as io
import pylab
pylab.rcParams['figure.figsize'] = (10.0, 8.0)


annType = ['segm','bbox','keypoints']
annType = annType[1]      #specify type here
prefix = 'person_keypoints' if annType=='keypoints' else 'instances'
print ('Running demo for *%s* results.'%(annType))

# use the valadation labelme file
annFile = 'xxx/instances_val2017.json'
cocoGt=COCO(annFile)
#initialize COCO detections api
# use the generated results
resFile = '/hxxx/test_data.json'
cocoDt=cocoGt.loadRes(resFile)

imgIds=sorted(cocoGt.getImgIds())
imgIds=imgIds[0:24]
imgId = imgIds[np.random.randint(24)]

# running box evaluation
cocoEval = COCOeval(cocoGt,cocoDt,annType)
cocoEval.params.imgIds  = imgIds
cocoEval.evaluate()
cocoEval.accumulate()
cocoEval.summarize()

out

因为为了方便测试结果，直接用了训练集中的标注进行验证，结果虚高。

此处maxDets参数还不知道什么作用，有人能否解答？

Running demo for *bbox* results.
loading annotations into memory...
Done (t=0.00s)
creating index...
index created!
Loading and preparing results...
DONE (t=0.00s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=0.02s).
Accumulating evaluation results...
DONE (t=0.02s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 1.000
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 1.000
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 1.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 1.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 1.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.853
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 1.000

文档结构

为了直接在同一个文件夹下引用方便，放的比较散，可以优化

P-R图绘制