I was curious of the new "turn on/off" background blur functionality of Google Meet (currently in test). I have investigated a bit and it seems it is using Tensorflow Lite models:
segm_heavy.tflite
segm_lite.tflite
via WASM
mediapipe_wasm_simd.wasm
while the model graph should be
background_blur_graph.binarypb
The model seems works at the level of the HTMLCanvasElement
as far as I can see. Anyone aware of a similar model?
[UPDATE]
Thanks to Jason Mayes and Physical Ed I was able to reproduce a very close background blur effect in the Google's BodyPix demo
The settings of the application are showed in the Controls box. There is a backgroundBlurAmount
option that let you customize the blur percentage to apply as well.
The result is almost close to the official Google Meet application.
A requirement to use blurred backgrounds and other background effects in Google Meet is to run the 64-bit version of Google Chrome. If you are not running this version of Chrome, you won't be able to activate any background effects.
Below are the steps to change the background in the Google Meet Android app. Open the App, select a meeting. Before joining, on the bottom of your self-view, tap on change background. Choose between slightly blur or the available presets.
Go to Google Meet https://meet.google.com/ and select meeting. At the bottom right of the self-view, click 'Change Background'. Now to completely blur your background, click the 'Blur your Background' option. If you want to slightly blur your background, click the 'Slightly blur your background' option.
majority of segmentation models give alpha channel as a result (some give more, but alpha is most useful) - what is masked and what is not
so if you want to blur the background, its a multi-step process:
ctx.globalCompositeOperation = 'darken'
ctx.filter =
blur(8px)`;so if you want to blur the background, simply apply apply blur simply copy canvas from 4, apply blur on it and draw if back before going to step 5
regarding models, google meet is not bad but i had better results with google selfie model
bodypix is older model, great configurability but not that great results
example code: https://github.com/vladmandic/human/blob/main/src/segmentation/segmentation.ts
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With