Detect Audio with Selenium Webdriver and Python

Question

Maybe not easily done, but I want to be able to load a web page in Selenium and test if flash or html5 automatically played audio.

In theory when a user visits my sites they can see the video play but no audio ( correct way) Sometimes the audio will play automatically with the video, this is what I don't want.

I'm wondering if there is a way to detect if ANY audio is playing via python and firefox? My internal tests will load the page and do quite a few selenium checks to make sure everything is good. What'd I like to do is add a test that also checks with python or selenium if audio is playing and flag it.

Any ideas?

Answer 1

I have similar issues. I deal with a lot of audio over the web. I have to automate checks for audio degradation, audio plays, etc. I couldn't really find a library to help me in Groovy or Ruby (I didn't check Python), so I made a call to an installed program (sox) and it would check real time if audio was heard or not.

This was my groovy call/script:

 def audioCheck = "sox -t coreaudio Soundflower /Users/me/project/record.wav silence 1 0.1 1% 1 .1 1%".execute()
             audioCheck.waitFor()
            println "EXIT VALUE FOR SOX IS: ${audioCheck.exitValue()}"

            if(audioCheck.exitValue() == 0){
                 // some stuff would happen now, if exit code is 0
            }

About Core Audio and "Soundflower"

The sox call specifies the audio board with the -t coreaudio (On linux you would probably use alsa instead of 'coreaudio' - coreauduio is the OSX audio interface.) The "Soundflower" parameter is the channel I created in Soundflower. This was important to me to record sound in isolation (as I was doing audio quality checks.) For example, if you used the default channel, like alsa default or coreaudio default, it will pick up your mic... so if the guy next to you sneezes... it corrupts the test.

By using a virtual audio channel (like with Soundflower on OSX), you can set all your audio "out" to go out "Soundflower" then have the sox command run on that channel... and you'll only be listening to the system audio coming out the virtual channel.

You can instead use your default channel (but it will end when it hears other noises in the room.)

I ran that code async using groovy tasks... so that code was wrapped in something like:

def listener = task { ... my script above ... }

Since it was listening asynchronously, it wouldn't block the rest of the test.

The Sox command

The actual sox real time command I used, came from this guy's video (check it out as he uses some different parameters):

https://www.youtube.com/watch?v=Q5ntlKE0ze4

Extending It

I pushed it a bit further by being able to automate the recording of the audio played through the browser and then use the PESQ algorithm to determine how closely the recorded audio was to the original audio being played. If you're interested in that, feel free to check out my post on it:

http://sdet.us/webrtc-audio-quality-automation-with-pesq/

Python PCAP Scraping

Maybe unrelated to what you're doing, but I've also played around with using the Pyshark library to rebuild audio from packet captures... it's more complex and perhaps more fragile. But if is of interest, here's my implementation:

http://sdet.us/python-pcap-parsing-audio-from-sip-call/