Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Google Meet background Blur

I was curious of the new "turn on/off" background blur functionality of Google Meet (currently in test). I have investigated a bit and it seems it is using Tensorflow Lite models:

segm_heavy.tflite
segm_lite.tflite

via WASM

mediapipe_wasm_simd.wasm

while the model graph should be

background_blur_graph.binarypb

The model seems works at the level of the HTMLCanvasElement as far as I can see. Anyone aware of a similar model?

[UPDATE]

Thanks to Jason Mayes and Physical Ed I was able to reproduce a very close background blur effect in the Google's BodyPix demo

The settings of the application are showed in the Controls box. There is a backgroundBlurAmount option that let you customize the blur percentage to apply as well.

enter image description here

The result is almost close to the official Google Meet application.

enter image description here

like image 986
loretoparisi Avatar asked Sep 29 '20 14:09

loretoparisi


People also ask

Why am I not getting option to blur background in Google Meet?

A requirement to use blurred backgrounds and other background effects in Google Meet is to run the 64-bit version of Google Chrome. If you are not running this version of Chrome, you won't be able to activate any background effects.

Can we blur background in Google Meet in phone?

Below are the steps to change the background in the Google Meet Android app. Open the App, select a meeting. Before joining, on the bottom of your self-view, tap on change background. Choose between slightly blur or the available presets.

Can you change background in Google Meet?

Go to Google Meet https://meet.google.com/ and select meeting. At the bottom right of the self-view, click 'Change Background'. Now to completely blur your background, click the 'Blur your Background' option. If you want to slightly blur your background, click the 'Slightly blur your background' option.


Video Answer


1 Answers

majority of segmentation models give alpha channel as a result (some give more, but alpha is most useful) - what is masked and what is not

so if you want to blur the background, its a multi-step process:

  1. resize input to model expected size
  2. run model to get alpha channel
  3. resize output back to original size
  4. draw original image on canvas
  5. draw alpha channel over it so only foreground stays visible
    for example using ctx.globalCompositeOperation = 'darken'
  6. optionally blur it a bit since model output is never perfect
    for example using ctx.filter = blur(8px)`;

so if you want to blur the background, simply apply apply blur simply copy canvas from 4, apply blur on it and draw if back before going to step 5

regarding models, google meet is not bad but i had better results with google selfie model
bodypix is older model, great configurability but not that great results

example code: https://github.com/vladmandic/human/blob/main/src/segmentation/segmentation.ts

like image 158
Vladimir Mandic Avatar answered Sep 20 '22 07:09

Vladimir Mandic