What is this

TopDown Pose Estimation on iOS and PureSwift.

• BBox: Yolov7-tiny
• Pose Estimation: ViTPose

Install

$ git clone https://github.com/otmb/TopDownPoseEstimation.git
$ cd TopDownPoseEstimation/TopDownPoseEstimation
$ curl -OL https://github.com/mbotsu/KeypointDecoder/releases/download/0.0.1/vitpose-b256x192_fp16.mlmodel
$ curl -OL https://github.com/mbotsu/KeypointDecoder/releases/download/0.0.1/yolov7-tiny_fp16.mlmodel

Example

COCO MS val2017

Models	AP
yolov7-tiny_fp16 + vitpose-b256x192_fp16.mlmodel	0.589
yolov7-tiny_fp16 + vitpose_s256x192_wholebody_fp16.mlmodel	0.579
yolov7-tiny_fp16 + vitpose_b256x192_wholebody_fp16.mlmodel	0.600

Detail

Sample Models

Models	Size	Keypoint
vitpose-b256x192_fp16.mlmodel	172MB	17
vitpose_s256x192_wholebody_fp16.mlmodel	46.5MB	133
vitpose_b256x192_wholebody_fp16.mlmodel	172MB	133
yolov7-tiny_fp16.mlmodel	12.1MB	-

COCO-Wholebody 133 When using Keypoint, change the following after introducing the model to the project.

Edit: PoseEstimation.swift

• keypointsNumber
• modelName

Create Model References

• ViTPose to CoreML
  • mbotsu/20221128_convert.ipynb
• Yolov7 to CoreML
  • john-rocky/CoreML-Models
    • Yolov7 Google Colab

References

• microsoft/human-pose-estimation.pytorch
• PaddlePaddle/PaddleDetection
• ViTAE-Transformer/ViTPose
• ViTPose to CoreML
  • mbotsu/20221128_convert.ipynb
• WongKinYiu/yolov7
• Yolov7 to CoreML
  • john-rocky/CoreML-Models
• AffineTransform
  • Perspective transform from quadrilateral to quadrilateral in Swift
• Drawing processing
  • Detecting human body poses in an image