Menu
I would like to get the raw data (TypedArray or something) from video element and manipulate them with JavaScript.
Currently I create a new canvas, draw the video into canvas and then get the image data.
ctx.drawImage(myVideo);
var data = ctx.getImageData(0, 0, w, h).data;
It works fine, but it drains a CPU (putting the video to canvas and copying back from canvas) and it creates a lot of garbage (about 50 MB each second). Is there any other simpler solution? It would be great if I could pass my own buffer to getImageData(...).
BTW. drawing video with WebGL and loading it back from GPU is not any faster :( http://jsperf.com/getting-raw-data-from-video
279
At this point you should see two identical video-clips - one on a video tag, one on a canvas element. Now that our video is on a canvas, we can access its raw pixel-data using ctx.getImageData(), manipulate it and draw it onto the second canvas.
Now that our video is on a canvas, we can access its raw pixel-data using ctx.getImageData(), manipulate it and draw it onto the second canvas. We’ll do this in drawToCanvas()on every frame to apply a simple greyscale filter:
To get the pixel of any specific portion from HTML canvas you can use the HTML canvas getImageData () Method. The getImageData () method usually returns an ImageData object that contains the pixel information of the specified object on the HTML canvas. Example 1: This example will display each pixel value of the square box.
Playing the stream on an HTML5 video element Painting each frame of this video onto an HTML5 Canvas Element Reading the pixels from the first canvas on every frame, manipulating them and drawing them onto a second canvas element Using the second canvas’ .captureStream()method to create a video stream and transmit it via our peer connection
Please read
https://www.scirra.com/blog/76/how-to-write-low-garbage-real-time-javascript
Because you don't actually show any code, mention the browser or give out information about the video (resolution, FPS, encoding) it is impossible to tell why your code is slow or why it creating show much garbage. Real-time video effects with Javasrcipt are possible with some resolution constrain.
Here is an Mozilla example of real-time video filtering using .
https://developer.mozilla.org/samples/video/chroma-key/index.xhtml https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API/Manipulating_video_using_canvas
You won't get any raw access to video data without blitting a video frame on <canvas> first. However, I believe this operation should be HW accelerated as it happens all in GPU memory. Downloading pixels down from GPU manipulating them in Javascript and then uploading back to the display memory might be the slowest step for high resolution.
Optimization tips
Consider skipping frames (process effects in lower FPS)
Server-side HTML5 video rendering Rendering HTML5 animation server-side?
192
clearRect() is what I think you are looking for.
To the best of my knowledge, uploading images on canvases takes space, but clearRect() deletes all memory on the canvas.
What I did on JS code:
On the Code down below is I set a stream on a video tag. See https://www.kirupa.com/html5/accessing_your_webcam_in_html5.htm for more WebCam info.
I set an interval of one second(calls function every second) to Clear the canvas and then draw what the stream was on that frame.
Now It your turn. Minimizing the video or canvas should have no effect on what to do. now you can grab the pixle colors on the webcam or video whenever.
Here is the Working Code:
Note: I found that this Example does not work with the built in web viewer. This example needs it's own web page.
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta content="stuff, to, help, search, engines, not" name="keywords">
<meta content="What this page is about." name="description">
<meta content="Display Webcam Stream" name="title">
<title>Display Webcam Stream</title>
<style>
#container {
margin: 0px auto;
width: 500px;
height: 375px;
border: 10px #333 solid;
}
#videoElement {
width: 500px;
height: 375px;
background-color: #666;
}
#canvas {
background-color: #666;
}
</style>
</head>
<body>
<video autoplay="true" id="videoElement"></video>
<canvas id="canvas" onclick= canvasOnclick()></canvas>
<script>
var video = document.querySelector("#videoElement");
var canvas = document.getElementById('canvas');
var ctx = canvas.getContext('2d');
if (navigator.mediaDevices.getUserMedia) {
navigator.mediaDevices.getUserMedia({video: true})
.then(function(stream) {
video.srcObject = stream;
})
.catch(function(err0r) {
console.log("Something went wrong!");
});
}
var myVar = setInterval(intervalTimer, 1000);
function intervalTimer() {
ctx.clearRect(0,0,400,400);
ctx.drawImage(video,0,0,400,400);
}
//var data = ctx.getImageData(0, 0, w, h).data;
</script>
</body>
</html>If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
