How to use bluetoothctl like hcitool lescan to report repeated proximity beacons

Question

I can use hcitool lescan with the --duplicates flag to capture periodic LE Advertising Reports (proximity beacons) from two nearby BLE devices:

$ sudo hcitool lescan --duplicates
LE Scan ...
C8:0F:10:29:4D:98 MI1S
C8:0F:10:29:4E:75 MI1S
C8:0F:10:29:4E:75 MI1S
C8:0F:10:29:4D:98 MI1S
C8:0F:10:29:4E:75 MI1S
C8:0F:10:29:4D:98 MI1S
<snip>

Here is that scan again, with timestamps added to show the periodicity:

$ sudo stdbuf -i0 -o0 -e0 hcitool lescan --duplicates | perl -nle 'print scalar(localtime), " ", $_'
Wed Apr 13 13:46:45 2016 LE Scan ...
Wed Apr 13 13:46:46 2016 C8:0F:10:29:4E:75 MI1S
Wed Apr 13 13:46:47 2016 C8:0F:10:29:4D:98 MI1S
Wed Apr 13 13:46:48 2016 C8:0F:10:29:4E:75 MI1S
Wed Apr 13 13:46:48 2016 C8:0F:10:29:4D:98 MI1S
Wed Apr 13 13:46:50 2016 C8:0F:10:29:4D:98 MI1S
Wed Apr 13 13:46:52 2016 C8:0F:10:29:4E:75 MI1S
<snip>

At the Embedded Linux Conference and IoT Summit last week (April 2016), a presenter who works on the BlueZ stack said to no longer use hcitool, and to use bluetoothctl instead.

I tried that today, but it only shows the first LE Advertising Report for a device (like hcitool does without the --duplicates option):

$ sudo bluetoothctl
[NEW] Controller 5C:F3:70:62:68:28 BlueZ 5.38 [default]

[bluetooth]# power on
Changing power on succeeded
[CHG] Controller 5C:F3:70:62:68:28 Powered: yes

[bluetooth]# scan on
Discovery started
[CHG] Controller 5C:F3:70:62:68:28 Discovering: yes
[CHG] Device C8:0F:10:29:4E:75 RSSI: -72
[CHG] Device C8:0F:10:29:4D:98 RSSI: -65

[bluetooth]# devices
Device C8:0F:10:29:4D:98 MI1S
Device C8:0F:10:29:4E:75 MI1S

How do you use bluetoothctl to capture repeated LE Advertising Reports from the same device, like hcitool does with the --duplicates option?

Answer 1

Just found the following to work for me (Ubuntu 18.04.1, bluez 5.48):

$ bluetoothctl
[bluetooth]# scan on
[bluetooth]# menu scan
[bluetooth]# clear
SetDiscoveryFilter success
[NEW] Device de:ad:be:ef:ca:fe SampleDev
[CHG] Device de:ad:be:ef:ca:fe RSSI: -73
[CHG] Device de:ad:be:ef:ca:fe RSSI: -73
[CHG] Device de:ad:be:ef:ca:fe RSSI: -74

Looks like there is a default scan filter active that blocks most advertisements.

Answer 2

Thanks Florian for your suggestion.

Since my original post, I have migrated to an Ubuntu 16.04.4 embedded Linux distribution which includes bluez 5.42. Unfortunately, bluetoothctl with this version does not recognize 'menu scan' or 'clear':

[bluetooth]# menu scan
Invalid command

[bluetooth]# clear
Invalid command

However, encouraged by your mention of 'default scan filter active that blocks most advertisements', I experimented with the commands available in my version of bluetoothctl as seen in the --help output, and got something working:

root@iot:~# bluetoothctl
[NEW] Controller 00:1A:7D:DA:71:13 iot #1 [default]
[NEW] Controller 70:2C:1F:31:F4:AF iot 

[bluetooth]# set-scan-filter-clear
SetDiscoveryFilter success

[bluetooth]# set-scan-filter-transport le
SetDiscoveryFilter success

[bluetooth]# scan on
Discovery started

[CHG] Controller 00:1A:7D:DA:71:13 Discovering: yes
[NEW] Device 0F:64:64:EE:E7:C4 0F-64-64-EE-E7-C4
[NEW] Device 0D:6F:45:77:87:F3 0D-6F-45-77-87-F3
[NEW] Device 40:CB:C0:F2:96:27 40-CB-C0-F2-96-27
[CHG] Device 0D:6F:45:77:87:F3 RSSI: -71
[CHG] Device FC:F1:36:73:77:B3 RSSI: -57
[CHG] Device 0F:64:64:EE:E7:C4 RSSI: -49

It takes a bit of typing in bluetoothctl to get it configured the way I want, and that typing is obscured by bluetoothctl logging activity rapidly in our beacon rich environment. So I put together a bash script that uses a heredoc and expect to feed the commands to bluetoothctl and sed/grep/perl massages the output:

$ cat beacon-scan.sh 
#!/bin/bash

# beacon-scan.sh
# Displays beacons including duplicates in real time.
# Uses expect to automate interaction with bluetoothctl.
# Uses sed to remove bluetoothctl colorization escape characters.
# Uses grep to filter out beacon manufacturer data logging.
# Uses perl to prefix each beacon with a timestamp.

if [ "$(id -u)" != "0" ]; then
    echo "ERROR: must run as root"
    exit 1
fi

(cat <<'END' | /usr/bin/expect

    set prompt "#"
    set timeout -1

    spawn bluetoothctl

    expect -re $prompt
    send "scan off\r"

    expect -re $prompt
    send "remove *\r"

    expect -re $prompt
    send "set-scan-filter-clear\r"

    expect -re $prompt
    send "set-scan-filter-transport le\r"

    expect -re $prompt
    send "scan on\r"

    trap {
        expect -re $prompt
        send "scan off\r"

        expect -re $prompt
        send "remove *\r"

        expect -re $prompt
        send "quit\r"
    } SIGINT

    expect eof

END
) | sed --unbuffered --quiet --expression 's/^.*Device //p' \
  | grep --line-buffered -v ManufacturerData \
  | perl -nle 'print scalar(localtime), " ", $_'

It works:

$ sudo ./beacon-scan.sh 
Wed Aug 22 19:34:07 2018 0F:64:64:EE:E7:C4 RSSI: -59
Wed Aug 22 19:34:07 2018 03:46:00:1D:E9:91 03-46-00-1D-E9-91
Wed Aug 22 19:34:07 2018 4E:20:6B:C7:68:D0 RSSI: -55
Wed Aug 22 19:34:07 2018 76:F1:1A:B9:ED:28 RSSI: -57
Wed Aug 22 19:34:07 2018 32:5D:8C:6A:72:C2 32-5D-8C-6A-72-C2
^C

Beacons that repeat are now reported by bluetoothctl, similar to when hcitool lescan is run with the duplicates flag.

I will say that bluetoothctl would be easier to use if it could be configured from the command line without needing to configure it interactively or having to resort to more complex scripting.

Thanks Florian for your help.