上城区建设局网站,网页设计的基本原则,哈尔滨小程序建设,济南网站建设首选传承网络文章目录摘要模型详解C2F模块Losshead部分模型实战训练COCO数据集下载数据集COCO转yolo格式数据集#xff08;适用V4#xff0c;V5#xff0c;V6#xff0c;V7#xff0c;V8#xff09;配置yolov8环境训练测试训练自定义数据集Labelme数据集摘要
YOLOv8 是 ultralytics …
文章目录摘要模型详解C2F模块Losshead部分模型实战训练COCO数据集下载数据集COCO转yolo格式数据集适用V4V5V6V7V8配置yolov8环境训练测试训练自定义数据集Labelme数据集摘要
YOLOv8 是 ultralytics 公司在 2023 年 1月 10 号开源的 YOLOv5 的下一个重大更新版本目前支持图像分类、物体检测和实例分割任务鉴于Yolov5的良好表现Yolov8在还没有开源时就收到了用户的广泛关注。yolov8的整体架构如下
Yolov8的改进之处有以下几个地方
Backbone使用的依旧是CSP的思想将YOLOv5中的C3模块被替换成了C2f模块实现了进一步的轻量化同时YOLOv8依旧使用了YOLOv5等架构中使用的SPPF模块PAN-FPNYOLOv8依旧使用了PAN的思想不同的是YOLOv8将YOLOv5中PAN-FPN上采样阶段中的卷积结构删除了同时也将C3模块替换为了C2f模块Decoupled-Head这一点源自YOLOX分类和回归两个任务的head不再共享参数YoloV8也借鉴了这样的head设计。Anchor-FreeYOLOv8抛弃了以往的Anchor-Base使用了Anchor-Free的思想损失函数YOLOv8使用VFL Loss作为分类损失使用DFL LossCIOU Loss作为分类损失样本匹配YOLOv8抛弃了以往的IOU匹配或者单边比例的分配方式而是使用了Task-Aligned Assigner匹配方式。
yolov8是个模型簇从小到大包括yolov8n、yolov8s、yolov8m、yolov8l、yolov8x等。模型参数、运行速度、参数量等详见下表 对比yolov5如下表 mAP和参数量都上升了不少具体的感受还是要亲自实践一番。
这篇文章首先对YoloV8做详细的讲解然后实现对COCO数据集的训练和测试最后实现自定义数据集的训练和测试。 希望能帮助到朋友们
分割的结果 分类的结果 模型详解
C2F模块
yolov8将yolov5中的C3模块换成了C2F模型我们先了解一下C3模块如图 C3模块其主要是借助CSPNet提取分流的思想同时结合残差结构的思想设计了所谓的C3 Block这里的CSP主分支梯度模块为BottleNeck模块堆叠的个数由参数n来进行控制不同的模型n的个数也不相同。C3的pytorch代码如下
class C3(nn.Module):# CSP Bottleneck with 3 convolutionsdef __init__(self, c1, c2, n1, shortcutTrue, g1, e0.5): # ch_in, ch_out, number, shortcut, groups, expansionsuper().__init__()c_ int(c2 * e) # hidden channelsself.cv1 Conv(c1, c_, 1, 1)self.cv2 Conv(c1, c_, 1, 1)self.cv3 Conv(2 * c_, c2, 1) # optional actFReLU(c2)self.m nn.Sequential(*(Bottleneck(c_, c_, shortcut, g, k((1, 1), (3, 3)), e1.0) for _ in range(n)))def forward(self, x):return self.cv3(torch.cat((self.m(self.cv1(x)), self.cv2(x)), 1))接下来我们一起学习C2F模块先经过一个Conv然后使用chunk函数将out平均拆分成两个向量然后保存到list中将后半部分输入到Bottleneck Block里面Bottleneck Block里面有n个Bottleneck将每个Bottleneck的输出都追加list中有n个所以拼接之后的out等于0.5✖(n2)。然后经过一个Conv输出所以输出为h×w×c_out。如下图 如果还是比较难懂我将具体的数据代入图中得出下图
Loss
对于YOLOv8其分类损失为VFL Loss其回归损失为CIOU LossDFL的形式这里Reg_max默认为16。
VFL主要改进是提出了非对称的加权操作FL和QFL都是对称的。而非对称加权的思想来源于论文PISA该论文指出首先正负样本有不平衡问题即使在正样本中也存在不等权问题因为mAP的计算是主正样本。 q是label正样本时候q为bbox和gt的IoU负样本时候q0当为正样本时候其实没有采用FL而是普通的BCE只不过多了一个自适应IoU加权用于突出主样本。而为负样本时候就是标准的FL了。可以明显发现VFL比QFL更加简单主要特点是正负样本非对称加权、突出正样本为主样本。
针对这里的DFLDistribution Focal Loss其主要是将框的位置建模成一个 general distribution让网络快速的聚焦于和目标位置距离近的位置的分布。
DFL 能够让网络更快地聚焦于目标 y 附近的值增大它们的概率
DFL的含义是以交叉熵的形式去优化与标签y最接近的一左一右2个位置的概率从而让网络更快的聚焦到目标位置的邻近区域的分布也就是说学出来的分布理论上是在真实浮点坐标的附近并且以线性插值的模式得到距离左右整数坐标的权重。
head部分
相对于YOLOv5YOLOv8将Head里面C3模块替换为了C2f将上采样之前的1×1卷积去除了将Backbone不同阶段输出的特征直接送入了上采样操作。通过下图对比可以看出差别 YOLOv8则是使用了Decoupled-Head同时由于使用了DFL 的思想因此回归头的通道数也变成了4*reg_max的形式
模型实战
训练COCO数据集
本次使用2017版本的COCO数据集作为例子演示如何使用YoloV8训练和预测。
下载数据集
Images:
2017 Train images [118K/18GB] http://images.cocodataset.org/zips/train2017.zip2017 Val images [5K/1GB]http://images.cocodataset.org/zips/val2017.zip2017 Test images [41K/6GB]http://images.cocodataset.org/zips/unlabeled2017.zip
Annotations:
2017 annotations_trainval2017 [241MB]http://images.cocodataset.org/annotations/annotations_trainval2017.zip
COCO转yolo格式数据集适用V4V5V6V7V8
最初的研究论文中COCO中有91个对象类别。然而在2014年的第一次发布中仅发布了80个标记和分割图像的对象类别。2014年发布之后2017年发布了后续版本。详细的类别如下
IDOBJECT (PAPER)OBJECT (2014 REL.)OBJECT (2017 REL.)SUPER CATEGORY1personpersonpersonperson2bicyclebicyclebicyclevehicle3carcarcarvehicle4motorcyclemotorcyclemotorcyclevehicle5airplaneairplaneairplanevehicle6busbusbusvehicle7traintraintrainvehicle8trucktrucktruckvehicle9boatboatboatvehicle10trafficlighttraffic lighttraffic lightoutdoor11fire hydrantfire hydrantfire hydrantoutdoor12streetsign--13stop signstop signstop signoutdoor14parking meterparking meterparking meteroutdoor15benchbenchbenchoutdoor16birdbirdbirdanimal17catcatcatanimal18dogdogdoganimal19horsehorsehorseanimal20sheepsheepsheepanimal21cowcowcowanimal22elephantelephantelephantanimal23bearbearbearanimal24zebrazebrazebraanimal25giraffegiraffegiraffeanimal26hat--accessory27backpackbackpackbackpackaccessory28umbrellaumbrellaumbrellaaccessory29shoe--accessory30eye glasses--accessory31handbaghandbaghandbagaccessory32tietietieaccessory33suitcasesuitcasesuitcaseaccessory34frisbeefrisbeefrisbeesports35skisskisskissports36snowboardsnowboardsnowboardsports37sports ballsports ballsports ballsports38kitekitekitesports39baseball batbaseball batbaseball batsports40baseball glovebaseball glovebaseball glovesports41skateboardskateboardskateboardsports42surfboardsurfboardsurfboardsports43tennis rackettennis rackettennis racketsports44bottlebottlebottlekitchen45plate--kitchen46wine glasswine glasswine glasskitchen47cupcupcupkitchen48forkforkforkkitchen49knifeknifeknifekitchen50spoonspoonspoonkitchen51bowlbowlbowlkitchen52bananabananabananafood53appleappleapplefood54sandwichsandwichsandwichfood55orangeorangeorangefood56broccolibroccolibroccolifood57carrotcarrotcarrotfood58hot doghot doghot dogfood59pizzapizzapizzafood60donutdonutdonutfood61cakecakecakefood62chairchairchairfurniture63couchcouchcouchfurniture64potted plantpotted plantpotted plantfurniture65bedbedbedfurniture66mirror--furniture67dining tabledining tabledining tablefurniture68window--furniture69desk--furniture70toilettoilettoiletfurniture71door--furniture72tvtvtvelectronic73laptoplaptoplaptopelectronic74mousemousemouseelectronic75remoteremoteremoteelectronic76keyboardkeyboardkeyboardelectronic77cell phonecell phonecell phoneelectronic78microwavemicrowavemicrowaveappliance79ovenovenovenappliance80toastertoastertoasterappliance81sinksinksinkappliance82refrigeratorrefrigeratorrefrigeratorappliance83blender--appliance84bookbookbookindoor85clockclockclockindoor86vasevasevaseindoor87scissorsscissorsscissorsindoor88teddy bearteddy bearteddy bearindoor89hair drierhair drierhair drierindoor90toothbrushtoothbrushtoothbrushindoor91hair brush--indoor
可以看到2014年和2017年发布的对象列表是相同的它们是论文中最初91个对象类别中的80个对象。所以在转换的时候要重新对类别做映射映射函数如下
def coco91_to_coco80_class(): # converts 80-index (val2014) to 91-index (paper)# https://tech.amikelive.com/node-718/what-object-categories-labels-are-in-coco-dataset/# a np.loadtxt(data/coco.names, dtypestr, delimiter\n)# b np.loadtxt(data/coco_paper.names, dtypestr, delimiter\n)# x1 [list(a[i] b).index(True) 1 for i in range(80)] # darknet to coco# x2 [list(b[i] a).index(True) if any(b[i] a) else None for i in range(91)] # coco to darknetx [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, None, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, None, 24, 25, None,None, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, None, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50,51, 52, 53, 54, 55, 56, 57, 58, 59, None, 60, None, None, 61, None, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72,None, 73, 74, 75, 76, 77, 78, 79, None]return x
接下来开始格式转换工程的目录如下
coco存放解压后的数据集。 -out保存输出结果。 -coco2yolo.py转换脚本。
转换代码如下
import json
import glob
import os
import shutil
from pathlib import Path
import numpy as np
from tqdm import tqdmdef make_folders(path../out/):# Create foldersif os.path.exists(path):shutil.rmtree(path) # delete output folderos.makedirs(path) # make new output folderos.makedirs(path os.sep labels) # make new labels folderos.makedirs(path os.sep images) # make new labels folderreturn pathdef convert_coco_json(json_dir./coco/annotations_trainval2017/annotations/):jsons glob.glob(json_dir *.json)coco80 coco91_to_coco80_class()# Import jsonfor json_file in sorted(jsons):fn out/labels/%s/ % Path(json_file).stem.replace(instances_, ) # folder namefn_images out/images/%s/ % Path(json_file).stem.replace(instances_, ) # folder nameos.makedirs(fn,exist_okTrue)os.makedirs(fn_images,exist_okTrue)with open(json_file) as f:data json.load(f)print(fn)# Create image dictimages {%g % x[id]: x for x in data[images]}# Write labels filefor x in tqdm(data[annotations], descAnnotations %s % json_file):if x[iscrowd]:continueimg images[%g % x[image_id]]h, w, f img[height], img[width], img[file_name]file_pathcoco/fn.split(/)[-2]/f# The Labelbox bounding box format is [top left x, top left y, width, height]box np.array(x[bbox], dtypenp.float64)box[:2] box[2:] / 2 # xy top-left corner to centerbox[[0, 2]] / w # normalize xbox[[1, 3]] / h # normalize yif (box[2] 0.) and (box[3] 0.): # if w 0 and h 0with open(fn Path(f).stem .txt, a) as file:file.write(%g %.6f %.6f %.6f %.6f\n % (coco80[x[category_id] - 1], *box))file_path_tfn_imagesfprint(file_path,file_path_t)shutil.copy(file_path,file_path_t)def coco91_to_coco80_class(): # converts 80-index (val2014) to 91-index (paper)# https://tech.amikelive.com/node-718/what-object-categories-labels-are-in-coco-dataset/# a np.loadtxt(data/coco.names, dtypestr, delimiter\n)# b np.loadtxt(data/coco_paper.names, dtypestr, delimiter\n)# x1 [list(a[i] b).index(True) 1 for i in range(80)] # darknet to coco# x2 [list(b[i] a).index(True) if any(b[i] a) else None for i in range(91)] # coco to darknetx [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, None, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, None, 24, 25, None,None, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, None, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50,51, 52, 53, 54, 55, 56, 57, 58, 59, None, 60, None, None, 61, None, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72,None, 73, 74, 75, 76, 77, 78, 79, None]return xconvert_coco_json()开始运行
转换完成后验证转换的结果
import cv2
import osdef draw_box_in_single_image(image_path, txt_path):# 读取图像image cv2.imread(image_path)# 读取txt文件信息def read_list(txt_path):pos []with open(txt_path, r) as file_to_read:while True:lines file_to_read.readline() # 整行读取数据if not lines:break# 将整行数据分割处理如果分割符是空格括号里就不用传入参数如果是逗号 则传入‘字符。p_tmp [float(i) for i in lines.split( )]pos.append(p_tmp) # 添加新读取的数据# Efield.append(E_tmp)passreturn pos# txt转换为boxdef convert(size, box):xmin (box[1]-box[3]/2.)*size[1]xmax (box[1]box[3]/2.)*size[1]ymin (box[2]-box[4]/2.)*size[0]ymax (box[2]box[4]/2.)*size[0]box (int(xmin), int(ymin), int(xmax), int(ymax))return boxpos read_list(txt_path)print(pos)tl int((image.shape[0]image.shape[1])/2)lf max(tl-1,1)for i in range(len(pos)):label str(int(pos[i][0]))print(label is label)box convert(image.shape, pos[i])image cv2.rectangle(image,(box[0], box[1]),(box[2],box[3]),(0,0,255),2)cv2.putText(image,label,(box[0],box[1]-2), 0, 1, [0,0,255], thickness2, lineTypecv2.LINE_AA)passif pos:cv2.imwrite(./Data/see_images/{}.png.format(image_path.split(\\)[-1][:-4]), image)else:print(None)img_folder ./out/images/val2017
img_list os.listdir(img_folder)
img_list.sort()label_folder ./out/labels/val2017
label_list os.listdir(label_folder)
label_list.sort()
if not os.path.exists(./Data/see_images):os.makedirs(./Data/see_images)
for i in range(len(img_list)):image_path img_folder \\ img_list[i]txt_path label_folder \\ label_list[i]draw_box_in_single_image(image_path, txt_path)结果展示
配置yolov8环境
可以直接安装requirements.txt里面所有的库文件执行安装命令
pip install -r requirements.txt如果不想安装这么多库文件在运行的时候查看缺少哪个库就安装哪个库比如我的环境
pip install thop我的本地只缺少了这个库文件。
训练
下载代码https://github.com/ultralytics/ultralytics通过下载的方式可以下载到源码这样方便修改。 也可以使用命令
pip install ultralytics如果仅仅是为了使用yolov8可以使用这种方式安装。
yolov8还支持使用命令的方式例如
yolo predict modelyolov8n.pt sourcehttps://ultralytics.com/images/bus.jpg接下来创建训练脚本可以使用yaml文件创建例如
from ultralytics import YOLO# Load a model
model YOLO(yolov8n.yaml) # build a new model from scratch模型文件在ultralytics/models/v8下面如图 也可以使用预训练模型创建。例如
model YOLO(yolov8n.pt) # load a pretrained model (recommended for training)然后开启训练。
# Use the model
model.train(datacoco128.yaml, epochs3) # train the model数据集的配置文件在ultralytics/datasets/下面如图 是不是很简单
接下来我们配置自己的环境。 第一步 找到ultralytics/datasets/coco.yaml文件。 然后将其复制到根目录 将里面的路径修改为
# Ultralytics YOLO , GPL-3.0 license
# COCO 2017 dataset http://cocodataset.org by Microsoft
# Example usage: yolo train datacoco.yaml
# parent
# ├── ultralytics
# └── datasets
# └── coco ← downloads here (20.1 GB)# Train/val/test sets as 1) dir: path/to/imgs, 2) file: path/to/imgs.txt, or 3) list: [path/to/imgs1, path/to/imgs2, ..]train: ./coco/images/train2017 # train images (relative to path) 118287 images
val: ./coco/images/val2017 # val images (relative to path) 5000 images
test: test-dev2017.txt # 20288 of 40670 images, submit to https://competitions.codalab.org/competitions/20794关于数据集的路径大家可以自行尝试我经过多次尝试发现YoloV8会自行添加datasets这个文件所以设置./coco/images/train2017,则实际路径是datasets/coco/images/train2017。 第二步 新建train.py脚本。
from ultralytics import YOLO# 加载模型
model YOLO(ultralytics/models/v8/yolov8n.yaml) # 从头开始构建新模型# Use the model
results model.train(datacoco.yaml, epochs3,device3) # 训练模型然后点击train.py可以运行了。 如果设置多卡可以在device中设置例如我使用四张卡可以设置为
results model.train(datacoco.yaml, epochs3,device0,1,2,3) # 训练模型第三步 修改参数在ultralytics/yolo/cfg/default.yaml文件中查看。例如
# Train settings -------------------------------------------------------------------------------------------------------
model: # path to model file, i.e. yolov8n.pt, yolov8n.yaml
data: # path to data file, i.e. coco128.yaml
epochs: 100 # number of epochs to train for
patience: 50 # epochs to wait for no observable improvement for early stopping of training
batch: 16 # number of images per batch (-1 for AutoBatch)
imgsz: 640 # size of input images as integer or w,h
save: True # save train checkpoints and predict results
save_period: -1 # Save checkpoint every x epochs (disabled if 1)
cache: False # True/ram, disk or False. Use cache for data loading
device: # device to run on, i.e. cuda device0 or device0,1,2,3 or devicecpu
workers: 8 # number of worker threads for data loading (per RANK if DDP)
project: # project name
name: # experiment name, results saved to project/name directory
exist_ok: False # whether to overwrite existing experiment
pretrained: False # whether to use a pretrained model
optimizer: SGD # optimizer to use, choices[SGD, Adam, AdamW, RMSProp]
verbose: True # whether to print verbose output
seed: 0 # random seed for reproducibility
deterministic: True # whether to enable deterministic mode
single_cls: False # train multi-class data as single-class
image_weights: False # use weighted image selection for training
rect: False # support rectangular training if modetrain, support rectangular evaluation if modeval
cos_lr: False # use cosine learning rate scheduler
close_mosaic: 10 # disable mosaic augmentation for final 10 epochs
resume: False # resume training from last checkpoint上面是训练过程中常用的参数我们调用yolo函数可以自行修改。 等待测试完成后就可以看到结果如下图 测试
新建测试脚本test.py.
from ultralytics import YOLO# Load a model
model YOLO(runs/detect/train11/weights/best.pt) # load a pretrained model (recommended for training)results model.predict(sourceultralytics/assets,device3) # predict on an image
print(results)这个results保存了所有的结果。如下图 predict的参数也可以在ultralytics/yolo/cfg/default.yaml文件中查看。例如
# Prediction settings --------------------------------------------------------------------------------------------------
source: # source directory for images or videos
show: False # show results if possible
save_txt: False # save results as .txt file
save_conf: False # save results with confidence scores
save_crop: False # save cropped images with results
hide_labels: False # hide labels
hide_conf: False # hide confidence scores
vid_stride: 1 # video frame-rate stride
line_thickness: 3 # bounding box thickness (pixels)
visualize: False # visualize model features
augment: False # apply image augmentation to prediction sources
agnostic_nms: False # class-agnostic NMS
classes: # filter results by class, i.e. class0, or class[0,2,3]
retina_masks: False # use high-resolution segmentation masks
boxes: True # Show boxes in segmentation predictions训练自定义数据集
Labelme数据集
