BodyPix: Real-time Person Segmentation

Question

BodyPix is an open-source machine learning model which allows for person and body-part segmentation in the browser with TensorFlow.js. I will like to convert the model to a .pb frozen graph in order to use it on Python.

How can I do it?

I try to find the solution on different places, but not working.

Answer 1

• Download the model.json file

Eg: https://storage.googleapis.com/tfjs-models/savedmodel/bodypix/resnet50/float/model-stride16.json

• Download Corresponding weights

https://storage.googleapis.com/tfjs-models/savedmodel/bodypix/resnet50/float/group1-shard1of23.bin

...

https://storage.googleapis.com/tfjs-models/savedmodel/bodypix/resnet50/float/group1-shard23of23.bin

• Install tfjs_graph_converter

from https://github.com/patlevin/tfjs-to-tf

• Convert model to .pb file

tfjs_graph_converter path/to/js/model path/to/frozen/model.pb

Answer 2

Segmentation using bodypix in Python. But got better results only when person is in front of wall rather than other objects.

from tf_bodypix.api import download_model, load_model, BodyPixModelPaths
import cv2
bodypix_model = load_model(download_model(BodyPixModelPaths.MOBILENET_FLOAT_50_STRIDE_16))

cap = cv2.VideoCapture(0) 
while cap.isOpened(): 
    ret, frame = cap.read()
    # BodyPix Segmentation
    result = bodypix_model.predict_single(frame)
    mask = result.get_mask(threshold=0.5).numpy().astype(np.uint8)
    seg = result.get_colored_part_mask(mask)

    tf.keras.preprocessing.image.save_img(
    pwd+"\\output-colored-mask.jpg",
    seg
)