- initial import

This commit is contained in:
Dmytro Bogovych 2023-08-09 19:53:31 +03:00
commit b701793923
73 changed files with 5835 additions and 0 deletions

77
README.md Normal file
View File

@ -0,0 +1,77 @@
Requirements:
- Raspberri Pi 3 B+ (RPi 4 is ok too; however RPi 3 are cheaper).
- https://downloads.raspberrypi.org/raspios_oldstable_armhf/images/raspios_oldstable_armhf-2023-02-22/2023-02-21-raspios-buster-armhf.img.xz
or
https://downloads.raspberrypi.org/raspios_armhf/images/raspios_armhf-2023-02-22/2023-02-21-raspios-bullseye-armhf.img.xz
- external USB bluetooth adapter https://www.amazon.com/TP-Link-Bluetooth-Receiver-Controllers-UB400/dp/B07V1SZCY6?th=1
Probably another external USB adapters are ok (it should be). However we tested with this one only - it is cheap enough.
sudo pip3 install rabbitpy sox pydub pyyaml
BT MAC addresses for test phones:
- D0:C5:F3:E0:4E:2D - iPhone BT MAC
- 50:8E:49:EC:B3:A4 - Redmi Note 9T
- A8:96:75:01:AA:57 - Moto G5
- 20:F4:78:63:0E:E5 - Xiaomi 11 5G Lite
Tricks:
- to fix BT error 'Waiting for phone index...' load bluetooth module(s)
pacmd load-module module-bluetooth-discover
This happens on RPi 4 sometimes.
- to avoid ALSA error "device busy" on RPi:
sudo apt-get install alsa-base
sudo alsa force-reload
or (for raspbian)
sudo /etc/init.d/alsa-utils stop
sudo alsa force-reload
(or sudo alsactl kill rescan)
sudo /etc/init.d/alsa-utils start
- to fix ofono error:
sudo systemctl restart ofono
- to check if mobile helper app is installed:
adb shell pm path biz.sevana.qualtestgsm
- to start mobile helper app:
adb shell am start -n biz.sevana.qualtestgsm/.MainActivity
1. Connect bluetooth to your phone.
- bluetoothctl
- power on
- agent on
- default-agent
Scan the nearby devices
- scan on
Once you see your phone device:
- scan off
Pair to your phone, remember type "yes" to confirm passkey. And in your phone also.
- pair 00:D2:79:79:01:89
- trust 00:D2:79:79:01:89
- connect 00:D2:79:79:01:89
- exit
try
pulseaudio -k
pulseaudio --start
exit from bluetoothctl and start again if BT connection doesn't work

3
audio/README.txt Normal file
View File

@ -0,0 +1,3 @@
jane_8k - file with public domain license
jane2_8k - the same file concatenated with itself
ref_woman_voice_16k.wav - 16K reference (with MOS 5.0)

BIN
audio/jane2_8k.wav Normal file

Binary file not shown.

BIN
audio/jane_8k.wav Normal file

Binary file not shown.

BIN
audio/ref_woman_man_16k.wav Normal file

Binary file not shown.

Binary file not shown.

BIN
bin/linux/aqua-wb Executable file

Binary file not shown.

BIN
bin/linux/pjsua Executable file

Binary file not shown.

BIN
bin/linux/pvqa Executable file

Binary file not shown.

BIN
bin/linux/silence_eraser Executable file

Binary file not shown.

BIN
bin/linux/speech_detector Executable file

Binary file not shown.

1
bin/macos/README.txt Normal file
View File

@ -0,0 +1 @@
This pjsua requires 10.14 at least!

BIN
bin/macos/aqua-wb Executable file

Binary file not shown.

BIN
bin/macos/pjsua Executable file

Binary file not shown.

BIN
bin/macos/pvqa Executable file

Binary file not shown.

226
bin/pvqa.cfg Normal file
View File

@ -0,0 +1,226 @@
BOF Common
IntervalLength = 0.68
IsUseUncertain = false
IsUseMixMode = true
IsUseDistance = false
AllWeight = 1.0
SilWeight = 1
VoiWeight = 1
AllCoefficient = 1.0
SilCoefficient = 1.0
VoiCoefficient = 1.0
SilThreshold = -37.50
IsOnePointSil = false
IsNormResult = true
IsMapScore = true
EOF Common
BOF Detector
Name = SNR
DetectorType = SNR
IntThresh = 0.10
FrameThresh = 14
DetThresh = 0.10
PVQA-Flag = true
PVQA-Weight = 1.0
DetMode = Both
EOF Detector
BOF Detector
Name = DeadAir-00
DetectorType = DeadAir
IntThresh = 0.60
DetThresh = 0.60
PVQA-Flag = true
PVQA-Weight = 1.0
DetMode = Both
EOF Detector
BOF Detector
Name = DeadAir-01
DetectorType = DeadAir
IntThresh = 0.5
DetThresh = 0.5
PVQA-Flag = true
PVQA-Weight = 1.0
DetMode = Both
EOF Detector
BOF Detector
Name = Click
DetectorType = Clicking
IntThresh = 0.10
DetThresh = 0.10
PVQA-Flag = true
PVQA-Weight = 1.0
DetMode = Both
EOF Detector
BOF Detector
Name = VAD-Clipping
DetectorType = VADClipping
IntThresh = 0.0
FrameThresh = 0.0
DetThresh = 0.0
PVQA-Flag = true
PVQA-Weight = 1.0
DetMode = Both
EOF Detector
BOF Detector
Name = Amplitude-Clipping
DetectorType = AmpClipping
IntThresh = 0.00
FrameThresh = 1.00
DetThresh = 0.00
PVQA-Flag = true
PVQA-Weight = 1.00
DetMode = Both
EOF Detector
BOF Detector
Name = Dynamic-Clipping
DetectorType = AmpClipping
IntThresh = 0.05
FrameThresh = 1.50
DetThresh = 0
PVQA-Flag = true
PVQA-Weight = 0.0
DetMode = Voice
EOF Detector
BOF Base EchoMono
SamplesType = UnKnownCodec
StepLengthSec = 0.5
MinDelayMs = 50
MaxLengthMs = 2800
WindowFunckID = 0
SpanLengthMs = 50
EOF Base EchoMono
BOF Detector
Name = ECHO
DetectorType = EchoMono
IntThresh = 0.00
FrameThresh = -40.0
DetThresh = 0.00
PVQA-Flag = true
PVQA-Weight = 1.0
DetMode = Voice
STAT-Flag = true
SpanLengthMs = 50
EOF Detector
BOF Detector
Name = Silent-Call-Detection
DetectorType = DeadAir
IntThresh = 0.99
DetThresh = 0.99
PVQA-Flag = false
PVQA-Weight = 1.0
EOF Detector
BOF Base SNR
MinPowerThresh = 1.0000
LogEnergyCoefficient = 10.0000
MinSignalLevel = 40.0000
MinSNRDelta = 0.0001
MinEnergyDisp = 3.0000
MinEnergyDelta = 1.0000
SamplesType = UnKnownCodec
EOF Base SNR
BOF Base AmpClipping
FlyAddingCoefficient = 0.1000
IsUseDynamicClipping = false
SamplesType = UnKnownCodec
EOF Base AmpClipping
BOF Base Clicking
SamplesType = UnKnownCodec
EOF Base Clicking
BOF Base DeadAir
StuckDeltaThreshold = 6
MinNonStuckTime = 80
MinStuckTime = 80
MinStartNonStuckTime = 1920
MinLevelThreshold = 256
SamplesType = UnKnownCodec
EOF Base DeadAir
BOF Base VADClipping
SamplesType = UnKnownCodec
EOF Base VADClipping
BOF DeadAir-01
MinLevelThreshold = 0
EOF DeadAir-01
BOF Silent-Call-Detection
MinLevelThreshold = 0
IsUseRMSPower = true
MinRMSThreshold = -70
EOF Silent-Call-Detection
BOF Dynamic-Clipping
FlyAddingCoefficient = 0.1000
SamplesType = UnKnownCodec
IsUseDynamicClipping = true
EOF Dynamic-Clipping
BOF Correction
IntStart = 5.0
IntEnd = 4.2
Mult = 1.0
#Shift = -1.7
Shift = 0
EOF Correction
BOF Correction
IntStart = 4.2
IntEnd = 3.5
Mult = 1.0
#Shift = -0.85
Shift = 0
EOF Correction
BOF SR Correction
SampleRate = 11000.0
Shift = 0.05
EOF SR Correction
BOF SR Correction
SampleRate = 16000.0
Shift = 0.1
EOF SR Correction
BOF SR Correction
SampleRate = 22000.0
Shift = 0.2
EOF SR Correction
BOF SR Correction
SampleRate = 32000.0
Shift = 0.3
EOF SR Correction
BOF SR Correction
SampleRate = 48000.0
Shift = 0.45
EOF SR Correction
BOF SR Correction
SampleRate = 96000.0
Shift = 0.5
EOF SR Correction
BOF SR Correction
SampleRate = 192000.0
Shift = 0.6
EOF SR Correction
BOF Scores Map
ScoresLine = 4;3.027000;2.935000;2.905000;2.818000;2.590000;2.432000;2.310000;1.665000;1.000000;
EOF Scores Map

BIN
bin/rpi/aqua-wb Executable file

Binary file not shown.

Binary file not shown.

BIN
bin/rpi/dist/ofono_1.21-1_armhf.deb vendored Normal file

Binary file not shown.

BIN
bin/rpi/dist/rtkit_0.11-6_armhf.deb vendored Normal file

Binary file not shown.

BIN
bin/rpi/pvqa Executable file

Binary file not shown.

BIN
bin/rpi/silence_eraser Executable file

Binary file not shown.

BIN
bin/rpi/speech_detector Executable file

Binary file not shown.

5
config/README.md Normal file
View File

@ -0,0 +1,5 @@
Various config files.
- subdirectory mc/ - helper config file for MidnightCommander. Provides dark mode. I just like it.
- systemd/ - config file for systemd to allow autostart on boot
- agent.in.yaml - main config file for QualTest agent

49
config/agent.in.yaml Normal file
View File

@ -0,0 +1,49 @@
# Backend URL
backend: http://89.184.187.247:8080
# Phone / instance name
name:
# Task name used to answer calls. Required for answerers only.
task:
# Exit from script after number of calls. Usually systemd restarts script after.
task_limit: 1
# Should we force first task in the list ? It will run immediately
force_task: no
# Use lite speech_detector instead of silence_eraser
speech_detector: yes
audio:
# Audio device used to play audio
play_device: "hw:2,0"
# Audio device used to record audio
record_device: "hw:2,0"
# Use native audio utilities from alsa-utils package instead of PyAudio based implementation
ALSA: yes
# Use samplerate
samplerate: 48000
bluetooth: no
bluetooth_mac: ""
log:
# Log file path (otherwise log will be sent to syslog)
path:
# Verbose logging
verbose: yes
# Log ADB output
adb: yes
# Upload full audio recordings
audio: yes
# Where to keep audio
audio_dir: /dev/shm

64
config/agent.yaml Normal file
View File

@ -0,0 +1,64 @@
# Backend URL
backend: $BACKEND
# Phone / instance name
name: $PHONE_NAME
# Task name used to answer calls. Required for answerers only.
task: $TASK_NAME
# Exit from script after number of calls. Usually systemd restarts script after.
task_limit: 1
# Should we force first task in the list ? It will run immediately
force_task: no
# Use lite speech_detector instead of silence_eraser
speech_detector: yes
# Reboot the phone on start
reboot_on_start: no
# adb watchdog check interval
phone_watchdog_interval: 180
# RabbitMQ related settings
rabbitmq:
url:
exchange:
queue:
audio:
# Audio device used to play audio
play_device: "auto"
# Audio device used to record audio
record_device: "auto"
# Use native audio utilities from alsa-utils package instead of PyAudio based implementation
ALSA: yes
# Use samplerate
samplerate: 48000
# Silence prefix & suffix lengths (in seconds)
silence_prefix: 30
silence_suffix: 30
# Should we start play with call established notification ? This is actual for caller nodes only
wait_for_notify: no
log:
# Log file path (otherwise log will be sent to syslog)
path:
# Verbose logging
verbose: yes
# Log ADB output
adb: yes
# Upload full audio recordings
audio: yes

143
config/mc/ini Normal file
View File

@ -0,0 +1,143 @@
[Midnight-Commander]
verbose=true
shell_patterns=true
auto_save_setup=true
preallocate_space=false
auto_menu=false
use_internal_view=true
use_internal_edit=false
clear_before_exec=true
confirm_delete=true
confirm_overwrite=true
confirm_execute=false
confirm_history_cleanup=true
confirm_exit=false
confirm_directory_hotlist_delete=false
confirm_view_dir=false
safe_delete=false
safe_overwrite=false
use_8th_bit_as_meta=false
mouse_move_pages_viewer=true
mouse_close_dialog=false
fast_refresh=false
drop_menus=false
wrap_mode=true
old_esc_mode=true
cd_symlinks=true
show_all_if_ambiguous=false
use_file_to_guess_type=true
alternate_plus_minus=false
only_leading_plus_minus=true
show_output_starts_shell=false
xtree_mode=false
file_op_compute_totals=true
classic_progressbar=true
use_netrc=true
ftpfs_always_use_proxy=false
ftpfs_use_passive_connections=true
ftpfs_use_passive_connections_over_proxy=false
ftpfs_use_unix_list_options=true
ftpfs_first_cd_then_ls=true
ignore_ftp_chattr_errors=true
editor_fill_tabs_with_spaces=false
editor_return_does_auto_indent=false
editor_backspace_through_tabs=false
editor_fake_half_tabs=true
editor_option_save_position=true
editor_option_auto_para_formatting=false
editor_option_typewriter_wrap=false
editor_edit_confirm_save=true
editor_syntax_highlighting=true
editor_persistent_selections=true
editor_drop_selection_on_copy=true
editor_cursor_beyond_eol=false
editor_cursor_after_inserted_block=false
editor_visible_tabs=true
editor_visible_spaces=true
editor_line_state=false
editor_simple_statusbar=false
editor_check_new_line=false
editor_show_right_margin=false
editor_group_undo=true
editor_state_full_filename=true
editor_ask_filename_before_edit=false
nice_rotating_dash=true
mcview_remember_file_position=false
auto_fill_mkdir_name=true
copymove_persistent_attr=true
pause_after_run=1
mouse_repeat_rate=100
double_click_speed=250
old_esc_mode_timeout=1000000
max_dirt_limit=10
num_history_items_recorded=60
vfs_timeout=60
ftpfs_directory_timeout=900
ftpfs_retry_seconds=30
fish_directory_timeout=900
editor_tab_spacing=8
editor_word_wrap_line_length=72
editor_option_save_mode=0
editor_backup_extension=~
editor_filesize_threshold=64M
editor_stop_format_chars=-+*\\,.;:&>
mcview_eof=
skin=default
[Layout]
message_visible=1
keybar_visible=1
xterm_title=1
output_lines=0
command_prompt=1
menubar_visible=1
free_space=1
horizontal_split=0
vertical_equal=1
left_panel_size=94
horizontal_equal=1
top_panel_size=1
[Misc]
timeformat_recent=%b %e %H:%M
timeformat_old=%b %e %Y
ftp_proxy_host=gate
ftpfs_password=anonymous@
display_codepage=UTF-8
source_codepage=Other_8_bit
autodetect_codeset=
spell_language=en
clipboard_store=
clipboard_paste=
[Colors]
base_color=linux:normal=gray,black:marked=yellow,black:input=,green:menu=black:menusel=gray:menuhot=red,:menuhotsel=black,red:dfocus=gray,black:dhotnormal=gray,black:dhotfocus=gray,black:executable=,black:directory=gray,black:link=gray,black:device=gray,black:special=gray,black:core=,black:stalelink=red,black:editnormal=gray,black
xterm-256color=
color_terminals=
[Panels]
show_mini_info=true
kilobyte_si=false
mix_all_files=false
show_backups=true
show_dot_files=true
fast_reload=false
fast_reload_msg_shown=false
mark_moves_down=true
reverse_files_only=true
auto_save_setup_panels=false
navigate_with_arrows=false
panel_scroll_pages=true
panel_scroll_center=false
mouse_move_pages=true
filetype_mode=true
permission_mode=false
torben_fj_mode=false
quick_search_mode=2
select_flags=6
[Panelize]
Find *.orig after patching=find . -name \\*.orig -print
Find SUID and SGID programs=find . \\( \\( -perm -04000 -a -perm /011 \\) -o \\( -perm -02000 -a -perm /01 \\) \\) -print
Find rejects after patching=find . -name \\*.rej -print
Modified git files=git ls-files --modified

View File

@ -0,0 +1,31 @@
[Unit]
Description=QualTest GSM node
ConditionPathExists=ABSOLUTE_INSTALL_DIR/run_node.sh
After=network.target
[Service]
Type=simple
User=pi
Group=pi
LimitNOFILE=1024
Restart=always
RestartSec=30
# startLimitIntervalSec=60
WorkingDirectory=ABSOLUTE_INSTALL_DIR
ExecStart=/bin/bash ABSOLUTE_INSTALL_DIR/run_node.sh --check-pid-file
KillSignal=SIGQUIT
# make sure log directory exists and owned by syslog
PermissionsStartOnly=true
ExecStartPre=/usr/bin/rm -f ABSOLUTE_INSTALL_DIR/qualtest.pid
#ExecStartPre=/bin/chown syslog:adm /var/log/sleepservice
#ExecStartPre=/bin/chmod 755 /var/log/sleepservice
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=agent_gsm
[Install]
WantedBy=multi-user.target

View File

@ -0,0 +1,13 @@
[Unit]
Description=Tunnel to jumpbox
Wants=network-online.target
After=network.target network-online.target
[Service]
User=pi
Restart=always
RestartSec=60
ExecStart=/bin/bash /home/pi/create_tunnel.sh
[Install]
WantedBy=multi-user.target

View File

@ -0,0 +1,11 @@
[Unit]
Description=VPN to backend
Wants=network-online.target
After=network.target network-online.target
[Service]
Type=oneshot
ExecStart=/home/pi/create_vpn.sh
[Install]
WantedBy=multi-user.target

22
disable_onboard_bluetooth.sh Executable file
View File

@ -0,0 +1,22 @@
#!/bin/bash
################################################################################
###
############### function to disable Raspberry Pi onboard bluetooth #############
###
################################################################################
function disable-onboard-bluetooth()
{
info "Disabling onboard bluetooth"
isInFile=$(cat /etc/modprobe.d/raspi-blacklist.conf | grep -c "blacklist btbcm")
if [ $isInFile -eq 0 ]; then
sudo bash -c 'echo "blacklist btbcm" >> /etc/modprobe.d/raspi-blacklist.conf'
fi
isInFile=$(cat /etc/modprobe.d/raspi-blacklist.conf | grep -c "blacklist hci_uart")
if [ $isInFile -eq 0 ]; then
sudo bash -c 'echo "blacklist hci_uart" >> /etc/modprobe.d/raspi-blacklist.conf'
fi
}
disable_onboard_bluetooth

1
docs/gsm_agent.drawio Normal file
View File

@ -0,0 +1 @@
<mxfile host="Electron" modified="2021-07-15T06:07:40.759Z" agent="5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) draw.io/14.1.8 Chrome/87.0.4280.88 Electron/11.1.1 Safari/537.36" etag="d51T50QXu8YFy8trsaFb" version="14.1.8" type="device"><diagram id="sR8Lh_MfxnFICZR0rMTi" name="Page-1">zVptc5s4EP41mbn7UA8Yv+Wjk7bX3qRzTpO7Xj9lZJBBE4GoJOzQX38r3pHsQny2yZcELUJI++zus7vmyrkNX/7gKA6+MA/Tq7HlvVw576/GY9u2xvBPSdJcsrCnucDnxCsm1YIH8hMXQquQJsTDojVRMkYlidtCl0URdmVLhjhnu/a0DaPtt8bIx4bgwUXUlH4jngyKU0ytWv4JEz+Q1YGLOyEqJxcCESCP7Roi58OVc8sZk/lV+HKLqVJeqZf8uY8H7lYb4ziSfR54+uhPv/35ZfVtM/n+5N19/bH9fP+uWGWLaFIcGA4UySdfhKM4vRrPKKx9s+Zw5asrU/IVS07wFjACZAKs/iLxrEYbzkL4d58g+oiFhMs1cp9x5F2Nb5U+4Kp4KMxQhT+JJFQ8hQDmqFCaTEskOEsiD6vD2PDaXUAkfoiRq+7uwPZAFsiQFreLY2Eu8ctBfdkVCmC+mIVwEjiyVTwwvi6AKyy3MsldbQf2opAFDRsoZagwPb9aukYHLgqAXgHW2ACr1q0OS6VqTYtghbG6dFNKQJ3c6dblOlf83boSwNp+BsdfiYRlcCEXufeCh58EgEVb/xNT/bNLat8xtG8oFxS+VDFH6ZciIYirtCIRl6a4oWDQB0//hYE1mpbD781775VGrGqUNkcrzAkcEPNS+EJkYy0YfW/cqVdSg3Kh/CDYM0KhBhUcliXcxd0BBc7sY9llyyb0DXCne8AtZRxTJMm2vd19iBdvWDESydqy7PEB1y6XyI9ZPNWMqfpC8/ZC9kxbKNeDsRCYA0ob02I1QRzesKP5gj2xfrmvyll6zrftX86Hi3zHtfdUmB3vUBPDoRrhvx/5fEEkUobKmYvBryIfBpSxGNjDuiNC4igjIsaVeW9xpuOSl0K2JlTxVRwwCGEZJ5GISIJkRmbq9ICQ6+JY1mNwtJBExZSc7yBZoEK98XPjaZR4hGWW6jLu5TuDXaNQBdf8r3o1RSncG1V3orWIK4cckPw0e3Nme7jveo+DVp518vA7NazlscgzcmA4jhnPfailOdCBbKtHSM6e8S2jTMXMSIHv3GwIpZoIUeJHKmaDzlR4vVEaJYD2srgREs9Tr9mLR42YdRpI7Knmo/vykX2YnA2S2dtjxGHJb9KT/A4AfRny07nBcY4kPyNB1lOvE5GfPZ+2N6wXOh1kqc0/D5nND5CZwFsUoZ509gnTGPOiDgIuwTV7rf65X1axbnmfqAEsTNOfmIvhiyVN5bPh+WJxAJAfUDIBRcuekNyUdZS1XH3ONq8gGpkTP2LpBnlWkPOSyieSmDKUVbkVPWWlL8z0EprNLrOHgQGcHkqJuxA8W711fQDALLPqid4qXRZ5GAljikPF5N4JsjP9PY9ZpMuWzvoYMdthvkmyVhgWz5LFCt0UEtLwDWCtOeuiL9S2XiadDOtyYRNsKvYHz+VrEDTw3xE4KLj03cOyGW1VEq9D+XVFlBqYkG8PueveYfZ8yJlNqZMlgWUy10zlqsTu7MncfJAkrWx2TY9MyvSmmXN9lpxMq0XmHSmZNn06f9X08hBnTeDsS/T3mrVMo7TpqmaGcYDCcjqrmckgjlKGUN1UjvSbWXsd50y1jFZ7LWbTX+5Kmz7raONp0yeXKHxss413cr95Z40sa9p2nom1OKY93t8/3rbdz69bUNuWXkj1NXwt1jpn6mDrDWmrq4jXCMZ+1fSJdQnGMFuSWdLat3dNhAtvQRFmiagqS3VFIpcmXtnN9v28BrH+FllWus7Unm0vZFkVOXxe6mjV477EdGzt8YYzJqZmdxK/IFUFPKFIQHmWwVQ0VMo7qhhvyLvKvnpBRFOBewMfkRBRIiQEvOrdZb0hAgiIqrej9ljWNWqPam9aLQMB5C38Uj6xDsSQTvTP1v2xzX7cyVnpyGyuLG0unM317k3PhqQ1R8/ndDrqS2uOltBN9EBzgNdORg5m//FkBniEIR1rtCc0wL5p1WJI+5voP+jPj02rNEY0Qt257c9sn2ZNzyfVEmtSXy7N22g9CexDzlj1jxN6Y3ZgPpppfDRbmHxkX/TboRL8BhoPQf7chuSa/G0kgt/3/K7wgPmWuLgxM0sTMBI4SxUl9nmWB1SIbgne1ZljneOEGCwuctMMHBbH+c3y94c39gGCjuHCGhxD81tJl0Ub4o9SBCc3FBawcA2J/SWUZXzPtEdZl/1S8dimwJlozpl3Zmf/r1nQyX69uwrDfhugpV/GtyO9vw2YdfDo0fQHw/qb6Xx6/eW58+E/</diagram></mxfile>

BIN
docs/gsm_agent.pdf Normal file

Binary file not shown.

View File

@ -0,0 +1 @@
<mxfile host="Electron" modified="2021-11-26T13:15:54.983Z" agent="5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) draw.io/14.6.13 Chrome/89.0.4389.128 Electron/12.0.7 Safari/537.36" etag="TQXjrr8-nKDB6EBuQmlO" version="14.6.13" type="device"><diagram id="sR8Lh_MfxnFICZR0rMTi" name="Page-1">zVtbc5s4FP41nmkf6gEDvjwmadN2J5kmTbfZPmVkULA2gCjITtxfv0dI3CRsYxZf8pAgIQQ639H5Ph0pA+sqfPucoHhxSz0cDEaG9zawPg5GI9M0RvCH16xFzdR0RIWfEE82KiseyB8sKw1ZuyQeTmsNGaUBI3G90qVRhF1Wq0NJQl/rzZ5pUH9rjHysVTy4KNBrH4nHFnIUjlHWf8HEX7BiwPJOiPLGsiJdII++VqqsTwPrKqGUiavw7QoH3Hi5XcRz1xvuFh+W4Ii1eeDp2nce/7q9e3y2fz15N99/r77ef5C9rFCwlAOGAUXsyU/DYbwejMYB9H05T+DK51d6zXfMEoJXgBEgs8D8N0pfeOk5oSH8uV+i4AdOGVzOkfuCI28wuuL2gCv5UJihCr+WjATpUwhgDqXR2DpHIqHLyMN8MCa89nVBGH6IkcvvvoLvQd2ChYG8rRsnHylOGH6rVEljfcY0hJHAkA15dzSTwEnPLVzytfQDcyrrFhUfyOuQdD2/6LpEBy4kQHuANdLAKm2rwlKYWrEieGHML911QMCcibXblnNh+Jt5UQF9+xkc35YMusGyPhWzF2Z4LwBM6/a3dfOPj2l9S7O+Zlww+AWPOdy+AUpT4nKrMJQwvbpiYDBRsv4HCsbQyYu/qvc+cosYRWldLd3hhMAAcZJXvhFW6QtKvyp3yp54Ie9oI1gpXSYu3h0/YIg+ZrtdF3u1aKtDXwHXaQA3r0twgBhZ1WN0E+LyDXeURKz0LHO0YWrnXYhxy6eqMVXtaFLvyBwrHQnDaB2BO6B1pVnMG6SbP9hS5oJpG1u/q5gsLdub5tb2cCG+uJw9BWbdJ5StTahK+G9HPreIRNxzE+pimFeRD4WA0hjYw7ghKcNRRkQ04b63wpmNc14K6ZwEnK/iBYUQlnESiQgjiGVkxkcPCLkujllZhokWkkg2EXwHYiFI+Ru/Vp5GS4/QzFNdmnjiy+CrUciDq/jNXx2gNdwbFneieRoXE/KE5Kf4mzVu4L5ZwwQtZlbv4dfRvOWH1BkCmATHNBFzqGY5GDCrmydlCX3BVzSgPGZGHHzr8pkEgVKFAuJHPGaDGXl4veTmI4D2hbwREs/jr2nEo0Rsa4RtD4npKHO0SY80YXIwSMbnx4hHJT+7JfmZZ0V+KjdYVkfy0wSyKr16Ij9z4tQ/WF3o7CBLpf1hyGyygcxSvEIRaklnX3AQ40Sug4BLcMledz/vL4pYd3G/5AXoOFj/wUl6+sWSYnJn1JYvjEMFp+kGQH7DkgkomrWE5DJfRxkXd1+zj+cQDfWG15i5C6EKBC9xPbGMA4qyVW5BT9nSF1p6yyBrnauHEwPobJLEuxA82HprpgE4Z0+ZVmsLHTzIKIWvhzeFcYBDTuReqfh40gi0QBAIefa8jFxGaCYZ33FYShEYpa/ZvBTFBYr8ZcwjOnOH708OnWXUoZu1Fmvqsqc37PKO6+Dl5ob50w7BT9EyxAmQ3Qb5LJN+4naWQKpi7uEVgWXB6efW9Pzw0VNJvUm3XIJVBVghx/qWYJPzkFZ5isrpKKXUVJc1O4iSUlYQzmS7kHKav+qgOsrUswK9eWa+qPhgDA2jtq4whrYx7ZJt2+i5OxcF9nl4roR6MqtBPbE7OrLiMtah8mFG4+duXBEozc29mtvGMfxez29kerVtIoykLrwFRZgu00Km8isSucHSy1Njvi/STcbfaaaE5pnVs88LaSZJT0+Xzmw3X46M4/KlnurAb4hryiehDTOYpDzM73AJWalXM3wqhGWHsKpL24rcWxKREAUkZRDuinfncihdQDjM9Wuuoq7lqkOUC2WMuPY9Pfq2sSGE7ET/YHkuU1/c985JRp2OnF1kJFVWrrgOq7JaJ7rGZ0VqlrKrYats1JbVCh/MO1IDzQZa640c9GRGbw7YwZG6Om13B2wrqqZn5X+2ujs46aqqlOSMFuoO7X96Libb33ri+1dV6hO1Ys+r7RpfMFaZ6RRpuqynrIsT89FY4aPxVOcj86gHEXLwK2g8LMRzz0RY8t0wXbxvSFI+4ITnRCotM5mAUYozqciwn2Q6oEB0RfBrqRxLjRNi8LjIXWfg0DgWN/Nk5pntZqoYTo2TY6gfvHJp9Ez84RrByDWDLWg4B2F/DGNphyMajHXcY09dUwIHojlrslOd7ZUq2Ml+rXMKZ7bRqMgvbSO69UbjeAeP9kd/tmO64fd/v91Q8oh+/mSp//tHwyFJnsyGULd/RvsLWJmHU7zKzslW09VyxyGE9Z+fLcgG9S2JgLzgstn2jYnqol+eMclae4P6iRTBsn3EZy2+NLh62wT5tO3eUy8L/kbIR02Qp8SHtfm+e7k5hgXZvrEEuRxdJk7Fim4HxcEk7ejQ2QHUqIGOC9AxTmPWw/6o7Zp8/50PJWDvZIltUapd4vlUnDBV8sydN0xmQ8OelT/1EyK2OR3OQPQVP3Yrvtg3C60IpenY2U5jShba2nGGU8mDzfo9xdLoQwfcfOlwMqvrKbBuk2rrRnZ1Vm2efeeS6Jqpp2q6Jrqm6vw8sNLSN0H69r9uC4C6/xkH8b//FdRnJ3W/cU/uZ5vd3G/v7UOFL5TIvW9shWL5b1yiefnPcNan/wA=</diagram></mxfile>

Binary file not shown.

View File

@ -0,0 +1 @@
<mxfile host="Electron" modified="2021-11-26T13:35:09.600Z" agent="5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) draw.io/14.6.13 Chrome/89.0.4389.128 Electron/12.0.7 Safari/537.36" etag="1SBZ1Ykjyk53EAZALH_-" version="14.6.13" type="device"><diagram id="sR8Lh_MfxnFICZR0rMTi" name="Page-1">3VnLbts6EP0aA+kihh6WkyyjpO1dtEhQN2i7pCVaIkqJCkn5ka+/Q4rU047t1m6KbhJxNOLjzJzDIT3y77L1R46K9DOLMR15Trwe+fcjz3Ndx4N/yrKpLNduUBkSTmLj1Bhm5AUbo2OsJYmx6DhKxqgkRdcYsTzHkezYEOds1XVbMNodtUAJHhhmEaJD6zcSy9SsInAa+3+YJKmsF2zeZMg6G4NIUcxWLZP/fuTfccZk9ZSt7zBV4Flcqu8+7HhbT4zjXB7ywfO37OkhfAp/vLzEn8s7kl1dPV2aXpaIlmbBX5Ao5phzAuZH9ccPR15oliA3FhfOyjzGqmt35IerlEg8K1Ck3q4gE8CWyoya12YQzCVe75y9W2MCyYRZhiXfgIv9wMJo88g0V01Qape0HZArY0QmEZK66wYreDBwHQGdN4DuaQYrdUJoQ4rCrN4asxqk3Zhdb4Fsci7E/AFiA4hwHt8q1kIrokgIEgEUQiIuh+YWWAAI33yHhmMbP1RjHNjm/br98n5jWtX4OB5oQA9iEB3EEyz3pcMwFC2sgy1YWxvHFEmy7E5jWwDMCI+MwASbSE+8TqRrFbJdCFbyCJuv2iLR68jvpYzX76jCYdCRzod62b+eIpNBitzmMWfwjecwrr5+mKkApSzHGl1FMwfnaE6x8hl5U5QpJlV/dSwJ16/+Hl4GB2jZdEuy+N65mHlzFDNzBb4fxkikNWothJT9EUmJea4tsIVoEnP2s95DPUXFNZHfLYHhuUVZaDWMVQ1L2IboY8eZdsk+dW/20F23HjEngBrmR2tARaL9W8JerZi8pVZMnG72uf2sOlQrJkG3o/5Ou0MqIIvQpuVWKAdxxHwd5/Vp9fy9q+vX/fuS1/WHh2rGJ9U5W962GBd+bcpYwnJoXCqUypiwrcIGNSrVX2QZymMxJCxk9Cc0h3q8w05ESaJ4GUF2Kw6ESsoIdHZrXmQkjlUfIceCvChhNUQxgYLOg3AU3L+mhWYZ5uNRXQO3SbVbiHYKJ5A+cPxOtPzf44R1YYuFwGfZ0dxhiX2yqscIqDO+ufE6Inrteq/LKDT6OtiS1qAtrO4eUT2hfh5aa72pftZb86anN8fq57QnVJN+4X0iAe0fAzz3dUHs+9t5HSq4+/x3rfu8gjs8fFTHtZbi+reqnSrUFbx5VVB+sKUnyCyc9jNsotEmMOif7HKzKnfuGGW8qZgWhNKe6XA53laXNpWr85ocH16a+v3YuMPadCu5zlaausMjwcn0s1E83ztK87YWkrUcH1PPbhHiUwpqcKCi7siKP6SowU0n6YLpLyqqe9PN3sGB6USK2p/w1OB8XgULBkQIUfQTK5nq8UGkqFCP0YYS0Afu7z/Wzisl+TSvDdB3ovXloZTQDbbCVl3RusGJ7qh6glPHvpV6287Crns2wZluEZwp1foOa5km6unj7LOiFRa6iEz0nuCUguRJ96ah3l6WRG7Gth+YV9OVNcZk2Yni9LlUV8PhguXyskJd7U/utFg3L20nX1Os14WR+ifU3NSpQZCsBGri3qRSjGJd7toLlC9IzNWlrwpM+9LXufiimxPlyvhPUQEODQDyHazHmZGsoKr/WXinAs8ZbHJ6+6RUXb6biZCcSFJNpNpRUS5WmNfNFOVJWZhTjfpqSdRK9JmIqJ1xoRJXw7kgSVnd6OhDUjXBOUM8HteHpHwuiha0MIZG94SA207mvG85w2C3OmtLoVcN3JI6DlwDArEs6moFLRGh+tSlCYTpRscIL6F0gYeHsgrRUv+mAiAzolF9LqEMkVoBc0Q3L5jXgQPvCAuhRT5iPNZTsIfSalCOC8alfhAllfWnc6tOzoVMdU7qP6xQxRai72xdpZ0hsIJRPP5XQmYImUMVQtXmqHO1AVMjwcs8rxQDyFgWY6GoKCJOCqmJEeoyAIijfWzq284Mzy7IooO1YktSO1Xj4OcSlArH7/5ZhlRwq4Wa216OFxj2IJ3eBimNhU3djOXq33iF1KgXK9wwQysL7ivdALc/iOPfM7Rz6OgnOyAJKFkUAfz7oGl9ZTD2/aXn7ChsGNQkC6or/xTOTzg/x1lp0q1crrZULlenusaHZvNLcVVRNr+3++//Bw==</diagram></mxfile>

Binary file not shown.

149
pi_bootstrap_bt.sh Executable file
View File

@ -0,0 +1,149 @@
#!/bin/bash
# fail on error , debug all lines
# set -eu -o pipefail
cecho () {
declare -A colors;
colors=(\
['black']='\E[0;47m'\
['red']='\E[0;31m'\
['green']='\E[0;32m'\
['yellow']='\E[0;33m'\
['blue']='\E[0;34m'\
['magenta']='\E[0;35m'\
['cyan']='\E[0;36m'\
['white']='\E[0;37m'\
);
local defaultMSG="No message passed.";
local defaultColor="black";
local defaultNewLine=true;
while [[ $# -gt 1 ]];
do
key="$1";
case $key in
-c|--color)
color="$2";
shift;
;;
-n|--noline)
newLine=false;
;;
*)
# unknown option
;;
esac
shift;
done
message=${1:-$defaultMSG}; # Defaults to default message.
color=${color:-$defaultColor}; # Defaults to default color, if not specified.
newLine=${newLine:-$defaultNewLine};
echo -en "${colors[$color]}";
echo -en "$message";
if [ "$newLine" = true ] ; then
echo;
fi
tput sgr0; # Reset text attributes to normal without clearing screen.
return;
}
warn () {
cecho -c 'yellow' "Warn: $@";
}
error () {
cecho -c 'red' "Erro: $@";
}
info () {
cecho -c 'green' "Info: $@";
}
################################################################################
###
############### function to disable Raspberry Pi onboard bluetooth #############
###
################################################################################
function disable-onboard-bluetooth()
{
info "Disabling onboard bluetooth"
isInFile=$(cat /etc/modprobe.d/raspi-blacklist.conf | grep -c "blacklist btbcm")
if [ $isInFile -eq 0 ]; then
sudo bash -c 'echo "blacklist btbcm" >> /etc/modprobe.d/raspi-blacklist.conf'
fi
isInFile=$(cat /etc/modprobe.d/raspi-blacklist.conf | grep -c "blacklist hci_uart")
if [ $isInFile -eq 0 ]; then
sudo bash -c 'echo "blacklist hci_uart" >> /etc/modprobe.d/raspi-blacklist.conf'
fi
}
function install-prerequisites()
{
info 'installing the must-have pre-requisites'
sudo apt-get install -y ofono
if [ $? != 0 ]; then
sudo apt-get install ./bin/ofono_1.21-1_armhf.deb -y
sudo apt-get install ./bin/libasound2-plugins_1.1.8-1_armhf.deb -y
sudo apt-get install ./bin/rtkit_0.11-6_armhf.deb -y
fi
while read -r p ; do sudo apt-get install -y $p ; done < <(cat << "EOF"
pulseaudio
pulseaudio-module-bluetooth
EOF
)
}
function remove-pkg()
{
info 'rempving bluealsa'
sudo apt-get purge bluealsa -y
}
function enable-headset-ofono()
{
info 'enable headset ofono'
sudo sed -i '/^load-module module-bluetooth-discover/ s/$/ headset=ofono/' /etc/pulse/default.pa
}
function install_python_pkg()
{
info 'installing python libraries'
pip install pexpect
pip3 install pexpect rabbitmq sox soundfile pyyaml
}
# main
sudo -n true
test $? -eq 0 || exit 1 "you should have sudo priveledge to run this script"
#
disable-onboard-bluetooth
install-prerequisites
remove-pkg
enable-headset-ofono
install_python_pkg
# Disable sleep on wifi connection
sudo iw wlan0 set power_save off
echo installing the nice-to-have pre-requisites
echo you have 5 seconds to reboot
echo or
echo hit Ctrl+C to quit
echo -e "\n"
sleep 5
sudo reboot

21
run_node.sh Executable file
View File

@ -0,0 +1,21 @@
#!/bin/bash
# Oneliner to find script's directory. Please note - last path component should NOT be symlink.
SCRIPT_DIR="$( cd -- "$( dirname -- "${BASH_SOURCE[0]:-$0}"; )" &> /dev/null && pwd 2> /dev/null; )";
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
# DBUS_SESSION_BUS_PID=`cat /run/dbus/pid`
export DBUS_SESSION_BUS_ADDRESS
# export DBUS_SESSION_BUS_PID
# To avoid problems with pulseaudio
pkill pulseaudio
# Ensure BT stack is here
python3 -u $SCRIPT_DIR/src/bt_preconnect.py $SCRIPT_DIR/config/agent.yaml
# Run main script
python3 -u $SCRIPT_DIR/src/agent_gsm.py --config $SCRIPT_DIR/config/agent.yaml

54
setup/setup_agent_gsm.sh Executable file
View File

@ -0,0 +1,54 @@
#!/bin/bash
# Installation directory
INSTALL_DIR=agent_gsm
# Re
GIT_SOURCE=https://deploy:deploy@git.sevana.biz/agent_gsm_redist
# Install prerequisites
sudo apt install git mc python3 sox vim
if [ -f "$INSTALL_DIR" ]; then
rm -rf "$INSTALL_DIR"
fi
# Anonymous cloning
git clone $GIT_SOURCE $INSTALL_DIR
# Go to cloned directory
cd $INSTALL_DIR
# Run bootstrap app
./pi_bootstrap_bt.sh
# Update config file
BACKEND_URL=""
PHONE_NAME=""
TASK_NAME=""
read -p "Please specify backend URL (ex: https://q.sevana.biz ): " BACKEND_URL
read -p "Please specify phone name (ex: moto_1): " PHONE_NAME
read -p "Please specify expected task name (if this is answerer phone):" TASK_NAME
# Get a copy of config file from redist
cp config/agent.in.yaml config/config.yaml
# Replace the values
if [[ $BACKEND_URL != "" ]]; then
sed -i "s/BACKEND_URL/$BACKEND_URL/" config/config.yaml
fi
if [[ $PHONE_NAME != "" ]]; then
sed -i "s/PHONE_NAME/$PHONE_NAME/" config/config.yaml
fi
sed -i "s/TASK_NAME/$TASK_NAME/" config/config.yaml
# Update systemD unit file
cp config/systemd/agent_gsm.in.service config/systemd/agent_gsm.service
ABSOLUTE_INSTALL_DIR=`realpath $INSTALL_DIR`
sed -i "s/ABSOLUTE_INSTALL_DIR/$ABSOLUTE_INSTALL_DIR/" config/systemd/agent_gsm.service
echo Please run $INSTALL_DIR/run_node.sh to see if everything works ok.
echo After you can use $INSTALL_DIR/config/systemd/agent_gsm.service to enable this work with systemD.

52
src/.vscode/launch.json vendored Normal file
View File

@ -0,0 +1,52 @@
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "Example: answerer",
"type": "python",
"request": "launch",
"program": "example_answer.py",
"console": "integratedTerminal",
"args": [""]
},
{
"name": "rabbitmq: utils_mcon",
"type": "python",
"request": "launch",
"program": "utils_mcon.py",
"console": "integratedTerminal",
"args": ["--verbose",
"--alsa-audio",
"--play-device", "auto",
"--record-device", "auto",
"--play-file", "../audio/jane_8k.wav",
"--record-file", "../audio/audio_recorded.wav",
"--call-timelimit", "90",
"--rabbitmq-connection", "amqp://qualtest:ablerluschar@amqp.sevana.biz:5672/qualtest",
"--rabbitmq-queue", "test_phone",
"--rabbitmq-exchange", "qualtest_exchange",
"--exec", "/usr/bin/python3 example_analyze.py --reference=../audio/jane_8k.wav --test=\\$RECORDED"]
},
{
"name": "Call by call",
"type": "python",
"request": "launch",
"program": "bt_loop_caller.py",
"console": "integratedTerminal",
"args": [""]
},
{
"name": "Run agent",
"type": "python",
"request": "launch",
"program": "agent_gsm.py",
"console": "integratedTerminal",
"cwd": "..",
"args": ["--config", "config/agent.yaml"]
}
]
}

547
src/agent_gsm.py Normal file
View File

@ -0,0 +1,547 @@
#!/usr/bin/python3
import os
import platform
import json
import subprocess
import time
import argparse
import sys
import shlex
import select
import uuid
import utils_qualtest
import utils_sevana
import utils_mcon
import utils_logcat
import utils
from bt_controller import Bluetoothctl
import bt_call_controller
import bt_signal
from bt_signal import SignalBoundaries
import multiprocessing
import shutil
import signal
import yaml
import pathlib
from pathlib import Path
from datetime import datetime
# Name of intermediary file with audio recorded from the GSM phone
RECORD_FILE = "/dev/shm/qualtest_recorded.wav"
# Backend instance
BackendServer : utils_qualtest.QualtestBackend = None
# Reference audio to play
REFERENCE_AUDIO = "/dev/shm/reference.wav"
# Loaded refernce audio (from backend)
LOADED_AUDIO = "/dev/shm/loaded_audio.wav"
# Script to exec after mobile call answering
EXEC_SCRIPT = None
# Current task name.
CURRENT_TASK = None
# Current task list
TASK_LIST: utils_qualtest.TaskList = utils_qualtest.TaskList()
# Number of finished calls
CALL_COUNTER = multiprocessing.Value('i', 0)
# Maximum number of calls to to. Zero means unlimited number of calls.
CALL_LIMIT = 0
# Find script's directory
DIR_THIS = Path(__file__).resolve().parent
# PID file name
QUALTEST_PID = DIR_THIS / "qualtest.pid"
# Keep the recorded audio in the directory
LOG_AUDIO = False
# Recorded audio directory
LOG_AUDIO_DIR = DIR_THIS.parent / 'log_audio'
# Should the first task run immediately ?
FORCE_RUN = False
# Exit codes
EXIT_OK = 0
EXIT_ERROR = 1
# Use silence eraser or not (speech detector is used in this case)
USE_SILENCE_ERASER = True
def remove_oldest_log_audio():
list_of_files = os.listdir(LOG_AUDIO_DIR)
if len(list_of_files) > 20:
full_path = [(LOG_AUDIO_DIR + "/{0}".format(x)) for x in list_of_files]
oldest_file = min(full_path, key=os.path.getctime)
# os.remove(oldest_file)
def detect_degraded_signal(file_test: Path, file_reference: Path) -> SignalBoundaries:
global USE_SILENCE_ERASER, LOG_AUDIO, LOG_AUDIO_DIR
if utils.get_wav_length(file_test) < utils.get_wav_length(file_reference):
# Seems some problem with recording, return zero boundaries
return SignalBoundaries()
r = bt_signal.find_reference_signal(file_test)
if r.offset_finish == 0.0:
r.offset_finish = 20.0 # Remove possible ring tones in the end of call on my test system
return r
def detect_reference_signal(file_reference: Path) -> SignalBoundaries:
global USE_SILENCE_ERASER, LOG_AUDIO, LOG_AUDIO_DIR
# Run silence eraser on reference file as well
result = bt_signal.find_reference_signal(file_reference)
return result
def run_analyze(file_test: str, file_reference: str, number: str) -> bool:
global CALL_COUNTER
result = False
if file_test:
# Wait 5 seconds to give a chance to flush recorded file
time.sleep(5.0)
# Check how long audio file is
audio_length = utils.get_wav_length(file_test)
# Check if audio length is strange - skip such calls. Usually this is missed call.
if ('caller' in BackendServer.phone.role and audio_length >= utils_mcon.TIME_LIMIT_CALL) or ('answer' in BackendServer.phone.role and audio_length >= utils_mcon.TIME_LIMIT_CALL * 1.2):
utils.log_error(f'Recorded audio call duration: {audio_length}s, skipping analysis')
return False
try:
bounds_signal : SignalBoundaries = detect_degraded_signal(Path(file_test), Path(file_reference))
bounds_signal.offset_start = 0
bounds_signal.offset_finish = 0
print(f'Found signal bounds: {bounds_signal}')
# Check if there is a time to remove oldest files
if LOG_AUDIO:
remove_oldest_log_audio()
remove_oldest_log_audio()
# PVQA report
pvqa_mos, pvqa_report, pvqa_rfactor = utils_sevana.find_pvqa_mos(file_test, bounds_signal.offset_start, bounds_signal.offset_finish)
utils.log(f'PVQA MOS: {pvqa_mos}, PVQA R-factor: {pvqa_rfactor}')
# AQuA report
bounds_reference : SignalBoundaries = detect_reference_signal(Path(file_reference))
bounds_reference.offset_start = 0
bounds_reference.offset_finish = 0
print(f'Found reference signal bounds: {bounds_reference}')
aqua_mos, aqua_percents, aqua_report = utils_sevana.find_aqua_mos(file_reference, file_test,
bounds_signal.offset_start, bounds_signal.offset_finish,
bounds_reference.offset_start, bounds_reference.offset_finish)
utils.log(f'AQuA MOS: {aqua_mos}, AQuA percents: {aqua_percents}')
# Build report for qualtest
r = None
if pvqa_mos == 0.0:
r = utils_qualtest.build_error_report(int(time.time()), 'PVQA analyzer error.')
else:
r = dict()
r['id'] = uuid.uuid1().urn[9:]
r['duration'] = round(utils.get_wav_length(file_test), 3)
# print(r['duration']) # This must be a float
r['endtime'] = int(time.time())
r['mos_pvqa'] = pvqa_mos
r['mos_aqua'] = aqua_mos
r['mos_network'] = 0.0
r['report_pvqa'] = pvqa_report
r['report_aqua'] = aqua_report
r['r_factor'] = pvqa_rfactor
r["percents_aqua"] = aqua_percents
r['error'] = ''
r['target'] = number
r['audio_id'] = 0
r['phone_id'] = BackendServer.phone.identifier
r['phone_name'] = ''
r['task_id'] = 0
r['task_name'] = CURRENT_TASK
# Upload report
upload_id = BackendServer.upload_report(r, [])
if upload_id != None:
utils.log('Report is uploaded ok.')
# Upload recorded audio
upload_result = BackendServer.upload_audio(r['id'], file_test)
if upload_result:
utils.log('Recorded audio is uploaded ok.')
result = True
else:
utils.log_error('Recorded audio is not uploaded.')
else:
utils.log_error('Failed to upload report.')
except Exception as e:
utils.log_error(e)
else:
utils.log_error('Seems the file is not recorded. Usually it happens because adb logcat is not stable sometimes. Return signal to restart')
# Increase finished calls counter
CALL_COUNTER.value = CALL_COUNTER.value + 1
return result
def run_error(error_message: str):
utils.log_error(error_message)
CALL_COUNTER.value = CALL_COUNTER.value + 1
def make_call(target: str):
global REFERENCE_AUDIO
# Remove old recorded file
record_file = '/dev/shm/bt_record.wav'
# if Path(record_file).exists():
# os.remove(record_file)
# Add prefix and suffix silence for reference to give a chance to record all the file
reference_filename = '/dev/shm/prepared_reference.wav'
utils.prepare_reference_file(fname=REFERENCE_AUDIO, silence_prefix_length=5.0, silence_suffix_length=5.0, output_fname=reference_filename)
# Find duration of prepared reference file
reference_length = int(utils.get_wav_length(reference_filename))
# Compose a command
# target = '+380995002747'
cmd = f'/usr/bin/python3 {DIR_THIS}/bt_call_controller.py --play-file {reference_filename} --record-file {record_file} --timelimit {reference_length} --target {target}'
retcode = os.system(cmd)
if retcode != 0:
utils.log_error(f'BT caller script exited with non-zero code {retcode}, skipping analysis.')
else:
run_analyze(record_file, REFERENCE_AUDIO, target)
def perform_answerer():
global CALL_LIMIT
# Get reference audio duration in seconds
reference_length = utils.get_wav_length(REFERENCE_AUDIO)
# Setup analyzer script
# Run answering script
while True:
# Remove old recording
record_file = f'/dev/shm/bt_record.wav'
# if Path(record_file).exists():
# os.remove(record_file)
cmd = f'/usr/bin/python3 {DIR_THIS}/bt_call_controller.py --play-file {REFERENCE_AUDIO} --record-file {record_file} --timelimit {int(reference_length)}'
retcode = os.system(cmd)
if retcode != 0:
utils.log(f'Got non-zero exit code {retcode} from BT call controller, exiting.')
break
# Call analyzer script
run_analyze(record_file, REFERENCE_AUDIO, '')
def run_caller_task(t):
global CURRENT_TASK, LOADED_AUDIO, REFERENCE_AUDIO
utils.log("Running task:" + str(t))
# Ensure we have international number format - add '+' if missed
target_addr = t['target'].strip()
if not target_addr.startswith('+'):
target_addr = '+' + target_addr
task_name = t['name'].strip()
# Load reference audio
if not BackendServer.load_audio(t["audio_id"], LOADED_AUDIO):
utils.log_error('No audio is available, exiting.')
sys.exit(EXIT_ERROR)
# Use loaded audio as reference
REFERENCE_AUDIO = LOADED_AUDIO
CURRENT_TASK = task_name
# Check attributes for precall scenaris
attrs: dict = utils_qualtest.ParseAttributes(t['attributes'])
retcode = 0
if 'precall' in attrs:
# Run precall scenario
utils.log('Running precall commands...')
retcode = os.system(attrs['precall'])
# If all requirements are ok - run the test
if retcode != 0:
utils.log_error(f'Precall script returned non-zero exit code {retcode}, skipping the actual test.')
return
# Start call. It will analyse audio as well and upload results
make_call(target_addr)
# Runs caller probe - load task list and perform calls
def run_probe():
global TASK_LIST, REFERENCE_AUDIO, LOADED_AUDIO, CURRENT_TASK
while True:
# Get task list update
tasks = BackendServer.load_tasks()
# Did we fetch anything ?
if tasks:
# Merge with existing ones. Some tasks can be removed, some can be add.
changed = TASK_LIST.merge_with(tasks)
else:
utils.log_verbose(f"No task list assigned, exiting.")
sys.exit(EXIT_ERROR)
# Sort tasks by triggering time
TASK_LIST.schedule()
if TASK_LIST.tasks is not None:
utils.log_verbose(f"Resulting task list: {TASK_LIST.tasks}")
if FORCE_RUN and len(TASK_LIST.tasks) > 0:
run_caller_task(TASK_LIST.tasks[0])
break
# Process tasks and measure spent time
start_time = time.monotonic()
for t in TASK_LIST.tasks:
if t["scheduled_time"] <= time.monotonic():
if t["command"] == "call":
try:
# Remove sheduled time
del t['scheduled_time']
# Run task
run_caller_task(t)
utils.log_verbose(f'Call #{CALL_COUNTER.value} finished')
if CALL_COUNTER.value >= CALL_LIMIT and CALL_LIMIT > 0:
# Time to exit from the script
utils.log(f'Call limit {CALL_LIMIT} hit, exiting.')
return
except Exception as err:
utils.log_error(message="Unexpected error.", err=err)
spent_time = time.monotonic() - start_time
# Wait 1 minute
if spent_time < 60:
time.sleep(60 - spent_time)
# In case of empty task list wait 1 minute before refresh
if len(TASK_LIST.tasks) == 0:
time.sleep(60)
def receive_signal(signal_number, frame):
# Delete PID file
if os.path.exists(QUALTEST_PID):
os.remove(QUALTEST_PID)
# Debugging info
print(f'Got signal {signal_number} from {frame}')
# Stop GSM call
utils_mcon.gsm_stop_call()
# Exit
raise SystemExit('Exiting')
return
# Check if Python version is ok
assert sys.version_info >= (3, 6)
# Use later configuration files
# https://stackoverflow.com/questions/3609852/which-is-the-best-way-to-allow-configuration-options-be-overridden-at-the-comman
parser = argparse.ArgumentParser()
parser.add_argument("--config", help="Path to config file, see config.in.yaml.")
parser.add_argument("--check-pid-file", action="store_true", help="Check if .pid file exists and exit if yes. Useful for using with .service files")
parser.add_argument("--test", action="store_true", help="Run the first task immediately. Useful for testing.")
# Parse arguments
args = parser.parse_args()
# Show help and exit if required
if len(sys.argv) < 2:
parser.print_help()
sys.exit(EXIT_OK)
if Path(QUALTEST_PID).exists() and args.check_pid_file:
print(f'File {QUALTEST_PID} exists, seems another instance of script is running. Please delete {QUALTEST_PID} to allow the start.')
sys.exit(EXIT_OK)
# Check if config file exists
config = None
config_path = 'config.yaml'
if args.config:
config_path = args.config
with open(config_path, 'r') as stream:
config = yaml.safe_load(stream)
# register the signals to be caught
signal.signal(signal.SIGINT, receive_signal)
signal.signal(signal.SIGQUIT, receive_signal)
# signal.signal(signal.SIGTERM, receive_signal)
# SIGTERM is sent from utils_mcon as well (multiprocessing?)
# Override default audio samplerate if needed
if 'samplerate' in config['audio']:
if config['audio']['samplerate']:
utils_mcon.SAMPLERATE = int(config['audio']['samplerate'])
if config['force_task']:
FORCE_RUN = True
if 'speech_detector' in config:
if config['speech_detector']:
USE_SILENCE_ERASER = False
if 'bluetooth_mac' in config['audio']:
bt_mac = config['audio']['bluetooth_mac']
if len(bt_mac) > 0:
# Connect to phone before
bt_ctl = Bluetoothctl()
bt_ctl.connect(bt_mac)
# Logging settings
utils.verbose_logging = config['log']['verbose']
if config['log']['path']:
utils.open_log_file(config['log']['path'], 'wt')
# Use native ALSA utilities on RPi
if utils.is_raspberrypi():
utils.log('RPi detected, using alsa-utils player & recorded')
utils_mcon.USE_ALSA_AUDIO = True
if 'ALSA' in config['audio']:
if config['audio']['ALSA']:
utils_mcon.USE_ALSA_AUDIO = True
if config['log']['adb']:
utils_mcon.VERBOSE_ADB = True
utils.log('Enabled adb logcat output')
# Audio directories
if 'audio_dir' in config['log']:
if config['log']['audio_dir']:
LOG_AUDIO_DIR = config['log']['audio_dir']
# Ensure subdirectory log_audio exists
if not os.path.exists(LOG_AUDIO_DIR):
utils.log(f'Creating {LOG_AUDIO_DIR}')
os.mkdir(LOG_AUDIO_DIR)
if 'audio' in config['log']:
if config['log']['audio']:
LOG_AUDIO = True
# Update path to pvqa/aqua-wb
dir_script = os.path.dirname(os.path.realpath(__file__))
utils_sevana.find_binaries(os.path.join(dir_script, "../bin"))
utils.log('Analyzer binaries are found')
# Load latest licenses & configs - this requires utils_sevana.find_binaries() to be called before
utils_sevana.load_config_and_licenses(config['backend'])
# Audio devices
if 'record_device' in config['audio'] and 'play_device' in config['audio']:
utils_mcon.AUDIO_DEV_RECORD = config['audio']['record_device']
utils_mcon.AUDIO_DEV_PLAY = config['audio']['play_device']
# Limit number of calls
if config['task_limit']:
CALL_LIMIT = config['task_limit']
utils.log(f'Limiting number of calls to {CALL_LIMIT}')
# Reset task list
utils_qualtest.TASK_LIST = []
# Init backend server
BackendServer = utils_qualtest.QualtestBackend()
BackendServer.instance = config['name']
BackendServer.address = config['backend']
# Write pid file to current working directory
with open(QUALTEST_PID, "w") as f:
f.write(str(os.getpid()))
f.close()
try:
# Load information about phone
utils.log(f'Loading information about the node {BackendServer.instance} from {BackendServer.address}')
BackendServer.preload()
if 'answerer' in BackendServer.phone.role:
# Check if task name is specified
if not config['task']:
utils.log_error('Please specify task value in config file.')
if os.path.exists(QUALTEST_PID):
os.remove(QUALTEST_PID)
sys.exit(utils_mcon.EXIT_ERROR)
# Save current task name
CURRENT_TASK = config['task']
# Load reference audio
utils.log('Loading reference audio...')
if not BackendServer.load_audio(BackendServer.phone.audio_id, REFERENCE_AUDIO):
utils.log_error('Audio is not available, exiting.')
sys.exit(EXIT_ERROR)
# Preparing reference audio
utils.log('Running answering loop...')
perform_answerer()
elif 'caller' in BackendServer.phone.role:
utils.log('Running caller...')
run_probe()
except Exception as e:
utils.log_error('Error', e)
# Close log file
utils.close_log_file()
# Exit with success code
if os.path.exists(QUALTEST_PID):
os.remove(QUALTEST_PID)
sys.exit(EXIT_OK)

36
src/audio_play.py Normal file
View File

@ -0,0 +1,36 @@
import sys
import argparse
import os
import time
import utils
import utils_audio
import typing
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument("--input", help=".wav file to play")
parser.add_argument("--device", help="audio device index or name to use")
parser.add_argument("--show-devices", help="list available output audio devices", action="store_true")
parser.add_argument("--silence-prefix", help="silence prefix length in seconds")
parser.add_argument("--silence-suffix", help="silence suffix length in seconds")
args = parser.parse_args()
if args.show_devices:
utils_audio.show_output_devices()
if args.input is not None:
# Check if file exists
if not os.path.exists(args.input):
print(f'File {args.input} does not exists, exiting.')
sys.exit(1)
# Look for device index
devices = utils_audio.get_output_devices()
device_index = 0
if args.device is not None:
silence_prefix = int(args.silence_prefix) if args.silence_prefix else 0
silence_suffix = int(args.silence_suffix) if args.silence_suffix else 0
utils_audio.play(args.device, args.input, silence_prefix, silence_suffix)
sys.exit(0)

37
src/audio_record.py Normal file
View File

@ -0,0 +1,37 @@
import argparse
import os
import sys
import time
import utils
import typing
import utils_audio
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument("--device", help="Index or name of capture audio device.")
parser.add_argument("--show-devices", help="List available capture audio devices.", action="store_true")
parser.add_argument("--output", help="File to write audio.")
parser.add_argument("--samplerate", help="Recording samplerate, default is 48000.")
parser.add_argument("--limit", help="Limit recording in seconds, default is 300.")
args = parser.parse_args()
# Time limitation from parameters
if args.limit:
TIME_LIMIT = float(args.limit)
# Bring up pyaudio
devices = utils_audio.get_input_devices()
if args.show_devices:
utils_audio.show_input_devices()
if args.samplerate is not None:
RATE = int(args.samplerate)
if args.output is not None:
utils_audio.capture(args.device, RATE, TIME_LIMIT, args.output)
# Exit
sys.exit(0)

334
src/bt_call_controller.py Normal file
View File

@ -0,0 +1,334 @@
#!/usr/bin/python3
import signal
import subprocess
import os
import time
import dbus
import tempfile
import argparse
import threading
import multiprocessing
import soundfile
import utils
import utils_bt_audio
import bt_phone
from bt_controller import Bluetoothctl
# Current call path
CALL_PATH = ''
CALL_ADDED = multiprocessing.Value('b', False)
CALL_REMOVED = multiprocessing.Value('b', False)
CALL_LOCK = threading.Lock()
# Call state change event
class CallState(bt_phone.Observer):
def update(self, call_object, event_type):
global CALL_PATH, CALL_LOCK, CALL_ADDED, CALL_REMOVED
utils.log(f'Call path: {call_object}, event: {event_type}. PID: {os.getpid()}, TID: {threading.get_ident()}')
if event_type == bt_phone.EVENT_CALL_REMOVE:
CALL_PATH = None
CALL_REMOVED.value = True
utils.log('Set CALL_REMOVED = True')
elif event_type == bt_phone.EVENT_CALL_ADD:
CALL_PATH = str(call_object)
CALL_REMOVED.value = False
CALL_ADDED.value = True
# Listen to call changes
CALL_STATE_EVENT = CallState()
PHONE = bt_phone.Phone()
PHONE.addObserver(CALL_STATE_EVENT)
# virtualmic module
PA_MODULE_IDX = -1
# Set volume 0..100%
def set_headset_spk_volume(vol: float):
cmd = f'pacmd set-sink-volume 0 0x {format(vol*100)}'
ret = os.popen(cmd).read()
return ret
def set_headset_mic_volume(vol: float):
cmd = f'pacmd set-source-volume 0 0x {format(vol*100)}'
ret = os.popen(cmd).read()
return ret
# Function to get the phone stream index to capture the downlink.
def get_headset_spk_idx():
utils.log('Waiting for phone stream index (please ensure all PA Bluetooth modules are loaded before)... ')
phoneIdx = ''
while phoneIdx == '':
time.sleep(1)
# grep 1-4 digit
phoneIdx = os.popen('pacmd list-sink-inputs | grep -B5 alsa_output | grep index | grep -oP "[0-9]{1,4}"').read()
return phoneIdx
# Start a call
def dial_number(number: str, play_file: str):
global CALL_PATH, CALL_LOCK, CALL_ADDED, CALL_REMOVED
if CALL_PATH is not None and len(CALL_PATH) > 0:
utils.log('Call exists already')
return
# Start audio inject
utils.log(f'Inject to uplink {play_file}')
inject_to_uplink(play_file)
# Initiate a call
utils.log(f'Initiate call to {number}')
PHONE.call_number(number)
# Answer the call
def answer_call(play_file: str):
global CALL_PATH, CALL_LOCK, CALL_ADDED
utils.log('Waiting for incoming call...')
# Wait for incoming call
while not CALL_ADDED.value:
time.sleep(0.1)
utils.log(f'Found incoming call {CALL_PATH}')
# CALL_LOCK.release()
# Start audio inject
inject_to_uplink(play_file)
# Answer the call
utils.log(f'Accepting the call {CALL_PATH}')
# Accept the call
PHONE.answer_call(CALL_PATH)
# Record downlink.
def capture_phone_alsaoutput(output_path: str):
default_output = get_headset_spk_idx().rstrip('\n')
cmd = f'parec --monitor-stream={default_output} --file-format=wav {output_path}'
utils.log(cmd)
# Example: parec --monitor-stream=34 --file-format=wav sample1.wav
parec_process = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE)
utils.log('Start recording downlink.')
return parec_process
# Cleanup
def cleanup():
global PA_MODULE_IDX, CALL_PATH, CALL_LOCK
utils.log(f'Cleaning call {CALL_PATH}...')
if PA_MODULE_IDX != -1:
cmd = f'pactl unload-module {PA_MODULE_IDX}'
utils.log(f'Unloading PulseAudio module... {cmd}')
p = subprocess.Popen(cmd, shell=True,stdout=subprocess.PIPE)
# Wait process to terminate to prevent hang the ssh session
(err, out) = p.communicate()
utils.log(f'PulseAudio module is unloaded.')
PA_MODULE_IDX = -1
# Stop the call itself
stop_call()
PHONE.quit_dbus_loop()
utils.log(f'Cleanup is finished: PID: {os.getpid()}')
# Function to inject to the uplink.
# Note: This function must run prior to the dial_number.
def inject_to_uplink(input_filename: str, verbose: bool = True):
global PA_MODULE_IDX
source_name = 'virtualmic'
default = '1'
format = 's16le'
rate = '44100'
channels = '1'
# Generate name for pipe
pipe_filename = tempfile.NamedTemporaryFile().name
cmd = f'pactl load-module module-pipe-source source_name={source_name} file={pipe_filename} format={format} rate={rate} channels={channels}'
utils.log(cmd)
# Create source
try:
p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE)
outdata = p.stdout.read()
PA_MODULE_IDX = int( outdata.decode('utf8').rstrip("\n") )
if verbose:
utils.log(f'PulseAudio module index: {PA_MODULE_IDX}')
if default != '':
cmd = f'pactl set-default-source {source_name}'
utils.log(cmd)
p = subprocess.Popen(cmd, shell=True,stdout=subprocess.PIPE)
outdata = p.stdout.read()
# print(outdata)
except Exception as e:
print('Failed to inject audio to uplink')
pass
# Send file to pipe - use ffmpeg
cmd = f'ffmpeg -hide_banner -loglevel error -re -i {input_filename} -f {format} -ar {rate} -ac {channels} - > {pipe_filename}'
utils.log(cmd)
p = subprocess.Popen(cmd, shell=True,stdout=subprocess.PIPE)
# (err, out) = p.communicate()
utils.log('Audio is injecting to uplink')
# Connect Rpi to phone as headset.
def connect_to_phone():
utils.log("Init bluetooth...")
bl = Bluetoothctl()
utils.log('BT control ready.')
devices = bl.get_paired_devices()
utils.log(f'List BT devices: {devices}')
if devices != None:
# dev = bl.get_device_info( devices[0].get('mac_address') )
utils.log(devices)
# disconnect before connect
bl.disconnect( devices[0].get('mac_address') )
ret = bl.connect(devices[0].get('mac_address'))
if ret == False:
utils.log( 'Connect to %s:%s failed' % ( devices[0].get('name'),devices[0].get('mac_address') ) )
return False
else:
utils.log( 'Connect to %s:%s success' % ( devices[0].get('name'), devices[0].get('mac_address') ) )
return True
else:
utils.log("no bluetooth device")
return False
# Function to stop the call once timing is expired.
def stop_call():
utils.log('Stopping all calls...')
PHONE.hangup_call()
# Returns pid of specified process
def get_pid(name):
return int(subprocess(["pidof","-s",name]))
def main(args: dict):
global CALL_PATH, CALL_LOCK, CALL_ADDED, CALL_REMOVED
# Ensure Ctrl-C handler is default
# signal.signal(signal.SIGINT, signal.SIG_DFL)
# Check if input file exists
if not os.path.exists(args['play_file']):
utils.log(f'Problem: file to play ({args["play_file"]}) doesn\'t exists.')
exit(os.EX_DATAERR)
# Duration in seconds
watchdog_timeout = int(args['timelimit'])
if watchdog_timeout == 0:
# Use duration of played file
audio_file = soundfile.SoundFile(args['play_file'])
watchdog_timeout = int(audio_file.frames / audio_file.samplerate + 0.5)
utils.log(f'Play timeout is set to {watchdog_timeout} seconds')
# Empty call path means 'no call started'
# CALL_LOCK.acquire()
CALL_PATH = ''
CALL_ADDED.value = False
CALL_REMOVED.value = False
# CALL_LOCK.release()
# This is done in preconnect script
# Ensure PulseAudio is running
# if not utils_bt_audio.start_PA():
# utils.log('Exiting.')
# exit(1)
# Attach to DBus (detach will happen in cleanup() function)
PHONE.setup_dbus_loop()
# Start call
if 'target' in args:
target_number = args['target']
if target_number is not None and len(target_number) > 0:
# Make a call
dial_number(target_number, args['play_file'])
else:
answer_call(args['play_file'])
else:
answer_call(args['play_file'])
# Don't make volume 100% - that's too much
audio_volume = 50
utils.log(f'Adjust speaker and microphone volume to {audio_volume}%')
set_headset_spk_volume(audio_volume)
set_headset_mic_volume(audio_volume)
# Start recording
utils.log(f'Start recording with ALSA to {args["record_file"]}')
process_recording = capture_phone_alsaoutput(args['record_file'])
utils.log(f'Main loop PID: {os.getpid()}, TID: {threading.get_ident()}')
# Wait until call is finished
time_start = time.time()
while not CALL_REMOVED.value and time_start + watchdog_timeout > time.time():
time.sleep(0.5)
utils.log(f'Call {CALL_PATH} finished.')
process_recording.kill()
cleanup()
retcode = os.system('pkill parec')
if retcode != 0:
print(f'Failed to terminate parec, exit code {retcode}')
utils.log('Exit')
if __name__ == "__main__":
parser = argparse.ArgumentParser(description='Raspberry Pi headset.')
parser.add_argument('--play-file', help='File to play.', required=True)
parser.add_argument('--record-file', help='File to record.', default='bt_recorded.wav', required=True)
parser.add_argument('--timelimit', help='Call duration.', default=0, type=int, required=True)
parser.add_argument('--target', help='Phone number to dial. If missed - try to answer the call.', type=str)
args = vars(parser.parse_args())
retcode = 0
try:
main(args)
except KeyboardInterrupt as e:
print('Ctrl-C pressed, exiting')
cleanup()
retcode = 130 # From http://tldp.org/LDP/abs/html/exitcodes.html
except Exception as e:
print(e)
print('Finalizing...')
cleanup()
retcode = 1
print(f'Call controller exits with return code {retcode}')
exit(retcode)

163
src/bt_controller.py Normal file
View File

@ -0,0 +1,163 @@
import time
import pexpect
import subprocess
import sys
class BluetoothctlError(Exception):
"""This exception is raised, when bluetoothctl fails to start."""
pass
class Bluetoothctl:
"""A wrapper for bluetoothctl utility."""
def __init__(self):
out = subprocess.check_output("rfkill unblock bluetooth", shell = True)
# print("Bluetoothctl")
self.child = pexpect.spawn("bluetoothctl", echo = False)
def get_output(self, command, pause = 0):
"""Run a command in bluetoothctl prompt, return output as a list of lines."""
self.child.send(command + "\n")
time.sleep(pause)
start_failed = self.child.expect(["[.]*", pexpect.TIMEOUT, pexpect.EOF])
if start_failed:
raise BluetoothctlError("Bluetoothctl failed after running " + command)
t = self.child.before
return t.decode('utf-8').split("\r\n")
def start_scan(self):
"""Start bluetooth scanning process."""
try:
out = self.get_output("scan on")
except BluetoothctlError as e:
print(e)
return None
def make_discoverable(self):
"""Make device discoverable."""
try:
out = self.get_output("discoverable on")
except BluetoothctlError as e:
print(e)
return None
def parse_device_info(self, info_string):
"""Parse a string corresponding to a device."""
device = {}
block_list = ["[\x1b[0;", "removed"]
string_valid = not any(keyword in info_string for keyword in block_list)
if string_valid:
try:
device_position = info_string.index("Device")
except ValueError:
pass
else:
if device_position > -1:
attribute_list = info_string[device_position:].split(" ", 2)
device = {
"mac_address": attribute_list[1],
"name": attribute_list[2]
}
return device
def get_available_devices(self):
"""Return a list of tuples of paired and discoverable devices."""
try:
out = self.get_output("devices")
except BluetoothctlError as e:
print(e)
return None
else:
available_devices = []
for line in out:
device = self.parse_device_info(line)
if device:
available_devices.append(device)
return available_devices
def get_paired_devices(self):
"""Return a list of tuples of paired devices."""
try:
out = self.get_output("paired-devices")
except BluetoothctlError as e:
print(e)
return None
else:
paired_devices = []
for line in out:
device = self.parse_device_info(line)
if device:
paired_devices.append(device)
return paired_devices
def get_discoverable_devices(self):
"""Filter paired devices out of available."""
available = self.get_available_devices()
paired = self.get_paired_devices()
return [d for d in available if d not in paired]
def get_device_info(self, mac_address):
"""Get device info by mac address."""
try:
out = self.get_output("info " + mac_address)
except BluetoothctlError as e:
print(e)
return None
else:
return out
def pair(self, mac_address):
"""Try to pair with a device by mac address."""
try:
out = self.get_output("pair " + mac_address, 4)
except BluetoothctlError as e:
print(e)
return None
else:
res = self.child.expect(["Failed to pair", "Pairing successful", pexpect.EOF])
success = True if res == 1 else False
return success
def remove(self, mac_address):
"""Remove paired device by mac address, return success of the operation."""
try:
out = self.get_output("remove " + mac_address, 3)
except BluetoothctlError as e:
print(e)
return None
else:
res = self.child.expect(["not available", "Device has been removed", pexpect.EOF])
success = True if res == 1 else False
return success
def connect(self, mac_address):
"""Try to connect to a device by mac address."""
try:
out = self.get_output("connect " + mac_address, 2)
except BluetoothctlError as e:
print(e)
return None
else:
res = self.child.expect(["Failed to connect", "Connection successful", pexpect.EOF])
success = True if res == 1 else False
return success
def disconnect(self, mac_address):
"""Try to disconnect to a device by mac address."""
try:
out = self.get_output("disconnect " + mac_address, 2)
except BluetoothctlError as e:
print(e)
return None
else:
res = self.child.expect(["Failed to disconnect", "Successful disconnected", pexpect.EOF])
success = True if res == 1 else False
return success

59
src/bt_loop_answerer.py Normal file
View File

@ -0,0 +1,59 @@
#!/usr/bin/python3
# This file runs the call script N times.
# The idea is to make long test and collect some statistics about reliability.
import os
import argparse
import typing
import time
import soundfile
# Used audio files to testing
PLAY_FILE = 'audio/reference_answerer.wav'
DIR_RECORD = '/dev/shm'
def run_test():
# Find duration of play audio
sf = soundfile.SoundFile(PLAY_FILE)
duration = int(sf.frames / sf.samplerate + 0.5)
# Remove old recordings
os.system(f'rm -f {DIR_RECORD}/bt_record*.wav')
# Tests
test_idx = 0
while True:
try:
# Recording file name
record_file = f'{DIR_RECORD}/bt_record_{test_idx:05d}.wav'
# Answer the call
cmd = f'/usr/bin/python3 call_controller.py --play-file {PLAY_FILE} --record-file {record_file} --timelimit {duration}'
retcode = os.system(cmd)
if retcode == 2:
print('Call finished in strange way, probably Ctrl-C. Exiting.')
exit(retcode)
else:
print(f'Call finished with return code {retcode}. Preparing to next one...')
test_idx += 1
except Exception as e:
print(e)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Test answerer.')
args = vars(parser.parse_args())
# Check if input audio file exists
if not os.path.exists('audio/example_1.wav'):
print(f'Problem: file to play ({args["play_file"]}) doesn\'t exists.')
exit(os.EX_DATAERR)
run_test()

62
src/bt_loop_caller.py Normal file
View File

@ -0,0 +1,62 @@
#!/usr/bin/python3
# This file runs the call script N times.
# The idea is to make long test and collect some statistics about reliability.
import os
import argparse
import typing
import time
import soundfile
import subprocess
from pathlib import Path
# Used audio files to testing
PLAY_FILE = '../audio/ref_woman_voice_16k.wav'
DIR_RECORD = '/dev/shm'
def run_test(nr_of_tests: int, delay: int, target_number: str):
# Find duration of play audio
sf = soundfile.SoundFile(PLAY_FILE)
# Use the reference audio with increased silence prefix length; as this is place for ringing.
duration = int(sf.frames / sf.samplerate + 0.5)
# Remove old recordings
os.system(f'rm -f {DIR_RECORD}/bt_record*.wav')
for i in range(nr_of_tests):
try:
# Recording file name
record_file = f'{DIR_RECORD}/bt_record_{i:05d}.wav'
cmd = f'/usr/bin/python3 bt_call_controller.py --play-file {PLAY_FILE} --record-file {record_file} --timelimit {duration} --target {target_number}'
os.system(cmd)
print('Call finished.')
print(f'Wait {delay}s for next scheduled call...')
time.sleep(delay)
except Exception as e:
print(e)
if __name__ == '__main__':
DEFAULT_FILE = '../audio/ref_woman_voice_16k.wav'
parser = argparse.ArgumentParser(description='Test caller.')
parser.add_argument('--tests', help='Number of tests', default=100, required=True)
parser.add_argument('--delay', help='Delay between calls', default = 30, required=True)
parser.add_argument('--target', help='Target number to call', required=True)
args = vars(parser.parse_args())
# Check if input audio file exists
if not os.path.exists(PLAY_FILE):
print(f'Problem: file to play ({PLAY_FILE}) doesn\'t exists.')
exit(os.EX_DATAERR)
run_test(int(args['tests']), int(args['delay']), args['target'])

182
src/bt_phone.py Normal file
View File

@ -0,0 +1,182 @@
import time
import utils
import dbus
import dbus.mainloop.glib
from gi.repository import GLib
from threading import Thread
from threading import Event
import abc
EVENT_CALL_ADD = 'call_add'
EVENT_CALL_REMOVE = 'call_remove'
ABC = abc.ABCMeta('ABC', (object,), {'__slots__': ()})
class Observer(ABC):
@abc.abstractmethod
def update(self, observable, event_type):
pass
class Observable(object):
def __init__(self):
self.__observers = []
def addObserver(self, observer):
self.__observers.append(observer)
def removeObserver(self, observer):
self.__observers.remove(observer)
def notifyObservers(self, call_object, event_type):
for o in self.__observers:
o.update(call_object, event_type)
class Phone(Observable):
def get_manager(self):
self.manager = dbus.Interface(self.bus.get_object('org.ofono', '/'), 'org.ofono.Manager')
def get_VCM(self):
return dbus.Interface(self.bus.get_object('org.ofono', self.modem), 'org.ofono.VoiceCallManager')
def get_online_modem(self):
# Refresh access to manager and modems list
# Get access to ofono manager via DBus
self.manager = dbus.Interface(self.bus.get_object('org.ofono', '/'), 'org.ofono.Manager')
# Get available modems
self.modems = self.manager.GetModems()
# Looking for modem which is online
for path, properties in self.modems:
if 'Online' in properties and 'Name' in properties and 'Serial' in properties:
modem_name = properties['Name']
model_serial = properties['Serial']
modem_online = properties['Online']
print(f'Found modem: {path} name: {modem_name} serial: {model_serial} online: {modem_online}')
if modem_online == 1:
return path
return None
# Wait for online modem and return this
def wait_for_online_modem(self):
while True:
modem = self.get_online_modem()
if modem != None:
return modem
# Sleep another 10 seconds and check again
time.sleep(10.0)
def get_incoming_call(self):
calls = self.vcm.GetCalls()
for path, properties in calls:
if properties['State'] == "incoming":
return path
return None
def answer_call(self, path):
call = dbus.Interface(self.bus.get_object('org.ofono', path), 'org.ofono.VoiceCall')
call.Answer()
def __init__(self):
super(Phone,self).__init__()
# Attach to DBus
dbus.mainloop.glib.DBusGMainLoop(set_as_default=True)
utils.log('Phone set up')
self.bus = dbus.SystemBus()
# Get ofono manager
self.manager = dbus.Interface(self.bus.get_object('org.ofono', '/'), 'org.ofono.Manager')
# Get access to modems
self.modems = self.manager.GetModems()
# Wait for online modem
utils.log('Waiting for BT modem (phone must be paired and connected before)...')
self.modem = self.wait_for_online_modem()
# Log about found modem
utils.log(f'BT modem found. Modem: {self.modem}')
# Get access to ofono API
self.org_ofono_obj = self.bus.get_object('org.ofono', self.modem)
self.vcm = dbus.Interface(self.org_ofono_obj, 'org.ofono.VoiceCallManager')
self.call_in_progress = False
# self._setup_dbus_loop()
utils.log('Initialized Dbus')
def quit_dbus_loop(self):
self.loop.quit()
def setup_dbus_loop(self):
utils.log('Connecting to D-Bus...')
dbus.mainloop.glib.DBusGMainLoop(set_as_default=True)
self.loop = GLib.MainLoop()
# self.loop = gobject.MainLoop() run
# gobject.threads_init()
try:
self._thread = Thread(target=self.loop.run)
self._thread.start()
except KeyboardInterrupt:
self.loop.quit()
utils.log('Connect to CallAdded & CallRemoved signals...')
self.org_ofono_obj.connect_to_signal("CallAdded", self.set_call_add, dbus_interface='org.ofono.VoiceCallManager')
self.org_ofono_obj.connect_to_signal("CallRemoved", self.set_call_ended, dbus_interface='org.ofono.VoiceCallManager')
def set_call_add(self, object, properties):
# print('Call add')
self.notifyObservers(object, EVENT_CALL_ADD)
self.call_in_progress = True
def set_call_ended(self, object):
# print('Call removed')
self.notifyObservers(object, EVENT_CALL_REMOVE)
self.call_in_progress = False
def hangup_call(self):
self.vcm.HangupAll()
def call_number(self, number: str, hide_callerid = 'default'):
utils.log(f'Calling number {number}')
try:
self.vcm.Dial(str(number), hide_callerid)
except dbus.exceptions.DBusException as e:
name = e.get_dbus_name()
msg = None
if name == 'org.freedesktop.DBus.Error.UnknownMethod':
msg = 'Most probably ofono not running'
elif name == 'org.ofono.Error.InvalidFormat':
msg = 'Invalid dialed number format'
# Print error info with explanation
utils.log(str(e))
if msg is not None:
utils.log(msg)
def close(self):
self.loop.quit()

37
src/bt_preconnect.py Executable file
View File

@ -0,0 +1,37 @@
#!/usr/bin/python3
import os
import sys
import yaml
import subprocess
import utils_bt_audio
from bt_controller import Bluetoothctl
if __name__ == '__main__':
if len(sys.argv) < 2:
print(f'Usage: bt_preconnect.py <path to config file>')
exit(0)
with open(sys.argv[1], 'r') as config_stream:
config = yaml.safe_load(config_stream)
if 'bluetooth_mac' in config['audio'] and 'bluetooth' in config['audio']:
use_bt = config['audio']['bluetooth']
bt_mac = config['audio']['bluetooth_mac']
if use_bt and len(bt_mac) > 0:
if not utils_bt_audio.start_PA():
print('Exiting')
exit(1)
# Connect to phone
print(f'Connecting to {bt_mac} ...')
bt_ctl = Bluetoothctl()
status = bt_ctl.connect(bt_mac)
if status:
print(f'Connected ok.')
else:
print(f'Not connected, sorry.')
else:
print('BT config not found.')
exit(0)

146
src/bt_setup.sh Executable file
View File

@ -0,0 +1,146 @@
#!/bin/bash
# fail on error , debug all lines
# set -eu -o pipefail
cecho () {
declare -A colors;
colors=(\
['black']='\E[0;47m'\
['red']='\E[0;31m'\
['green']='\E[0;32m'\
['yellow']='\E[0;33m'\
['blue']='\E[0;34m'\
['magenta']='\E[0;35m'\
['cyan']='\E[0;36m'\
['white']='\E[0;37m'\
);
local defaultMSG="No message passed.";
local defaultColor="black";
local defaultNewLine=true;
while [[ $# -gt 1 ]];
do
key="$1";
case $key in
-c|--color)
color="$2";
shift;
;;
-n|--noline)
newLine=false;
;;
*)
# unknown option
;;
esac
shift;
done
message=${1:-$defaultMSG}; # Defaults to default message.
color=${color:-$defaultColor}; # Defaults to default color, if not specified.
newLine=${newLine:-$defaultNewLine};
echo -en "${colors[$color]}";
echo -en "$message";
if [ "$newLine" = true ] ; then
echo;
fi
tput sgr0; # Reset text attributes to normal without clearing screen.
return;
}
warn () {
cecho -c 'yellow' "Warn: $@";
}
error () {
cecho -c 'red' "Erro: $@";
}
info () {
cecho -c 'green' "Info: $@";
}
################################################################################
###
############### function to disable Raspberry Pi onboard bluetooth #############
###
################################################################################
function disable-onboard-bluetooth()
{
info "Disabling onboard bluetooth"
isInFile=$(cat /etc/modprobe.d/raspi-blacklist.conf | grep -c "blacklist btbcm")
if [ $isInFile -eq 0 ]; then
sudo bash -c 'echo "blacklist btbcm" >> /etc/modprobe.d/raspi-blacklist.conf'
fi
isInFile=$(cat /etc/modprobe.d/raspi-blacklist.conf | grep -c "blacklist hci_uart")
if [ $isInFile -eq 0 ]; then
sudo bash -c 'echo "blacklist hci_uart" >> /etc/modprobe.d/raspi-blacklist.conf'
fi
}
function install-prerequisites()
{
info 'installing the must-have pre-requisites'
sudo apt-get install -y ofono
if [ $? != 0 ]; then
sudo apt-get install ./bin/ofono_1.21-1_armhf.deb -y
sudo apt-get install ./bin/libasound2-plugins_1.1.8-1_armhf.deb -y
sudo apt-get install ./bin/rtkit_0.11-6_armhf.deb -y
fi
while read -r p ; do sudo apt-get install -y $p ; done < <(cat << "EOF"
pulseaudio
pulseaudio-module-bluetooth
EOF
)
}
function remove-pkg()
{
info 'rempving bluealsa'
sudo apt-get purge bluealsa -y
}
function enable-headset-ofono()
{
info 'enable headset ofono'
sudo sed -i '/^load-module module-bluetooth-discover/ s/$/ headset=ofono/' /etc/pulse/default.pa
}
function install_python_pkg()
{
info 'installing python libraries'
pip install pexpect
pip3 install pexpect
}
# main
sudo -n true
test $? -eq 0 || exit 1 "you should have sudo priveledge to run this script"
#
disable-onboard-bluetooth
install-prerequisites
remove-pkg
enable-headset-ofono
install_python_pkg
echo installing the nice-to-have pre-requisites
echo you have 5 seconds to reboot
echo or
echo hit Ctrl+C to quit
echo -e "\n"
sleep 5
sudo reboot

62
src/bt_signal.py Normal file
View File

@ -0,0 +1,62 @@
#!/usr/bin/python3
import sys
import os
import pathlib
from pydub import silence, AudioSegment
class SignalBoundaries:
# Offset from start (in seconds)
offset_start: float
# Offset from finish (in seconds)
offset_finish: float
def __init__(self, offset_start = 0.0, offset_finish = 0.0) -> None:
self.offset_start = offset_start
self.offset_finish = offset_finish
def __repr__(self) -> str:
return f'[offset_start: {round(self.offset_start, 3)}, offset_finish : {round(self.offset_finish, 3)}]'
def find_reference_signal(input_file: pathlib.Path, output_file: pathlib.Path = None, use_end_offset: bool = True) -> SignalBoundaries:
myaudio = AudioSegment.from_wav(str(input_file))
dBFS = myaudio.dBFS
# Find silence intervals
intervals = silence.detect_nonsilent(myaudio, min_silence_len=1000, silence_thresh=dBFS-17, seek_step=50)
# Translate to seconds
intervals = [((start/1000),(stop/1000)) for start,stop in intervals] #in sec
# print(intervals)
# Example of intervals: [(5.4, 6.4), (18.7, 37.05)]
for p in intervals:
if p[1] - p[0] > 17:
bounds = SignalBoundaries(offset_start=p[0], offset_finish=p[1])
if output_file is not None:
signal = myaudio[bounds.offset_start * 1000 : bounds.offset_finish * 1000]
signal.export(str(output_file), format='wav', parameters=['-ar', '44100', '-sample_fmt', 's16'])
if use_end_offset:
bounds.offset_finish = myaudio.duration_seconds - bounds.offset_finish
return bounds
return SignalBoundaries()
if __name__ == '__main__':
if len(sys.argv) < 2:
print(f'Please specify input filename.')
exit(os.EX_NOINPUT)
# Output file
output_file = pathlib.Path(sys.argv[2]) if len(sys.argv) > 2 else None
# Input file
input_file = sys.argv[1]
bounds: SignalBoundaries = find_reference_signal(pathlib.Path(input_file), output_file)
print (bounds)

4
src/crontab/__init__.py Normal file
View File

@ -0,0 +1,4 @@
from ._crontab import CronTab
__all__ = ['CronTab']

465
src/crontab/_crontab.py Normal file
View File

@ -0,0 +1,465 @@
'''
crontab.py
Written July 15, 2011 by Josiah Carlson
Copyright 2011-2018 Josiah Carlson
Released under the GNU LGPL v2.1 and v3
available:
http://www.gnu.org/licenses/old-licenses/lgpl-2.1.html
http://www.gnu.org/licenses/lgpl.html
Other licenses may be available upon request.
'''
from collections import namedtuple
from datetime import datetime, timedelta
import sys
import warnings
_ranges = [
(0, 59),
(0, 59),
(0, 23),
(1, 31),
(1, 12),
(0, 6),
(1970, 2099),
]
ENTRIES = len(_ranges)
SECOND_OFFSET, MINUTE_OFFSET, HOUR_OFFSET, DAY_OFFSET, MONTH_OFFSET, WEEK_OFFSET, YEAR_OFFSET = range(ENTRIES)
_attribute = [
'second',
'minute',
'hour',
'day',
'month',
'isoweekday',
'year'
]
_alternate = {
MONTH_OFFSET: {'jan': 1, 'feb': 2, 'mar': 3, 'apr': 4, 'may': 5, 'jun': 6,
'jul': 7, 'aug': 8, 'sep': 9, 'oct': 10, 'nov':11, 'dec':12},
WEEK_OFFSET: {'sun': 0, 'mon': 1, 'tue': 2, 'wed': 3, 'thu': 4, 'fri': 5,
'sat': 6},
}
_aliases = {
'@yearly': '0 0 1 1 *',
'@annually': '0 0 1 1 *',
'@monthly': '0 0 1 * *',
'@weekly': '0 0 * * 0',
'@daily': '0 0 * * *',
'@hourly': '0 * * * *',
}
WARNING_CHANGE_MESSAGE = '''\
Version 0.22.0+ of crontab will use datetime.utcnow() and
datetime.utcfromtimestamp() instead of datetime.now() and
datetime.fromtimestamp() as was previous. This had been a bug, which will be
remedied. If you would like to keep the *old* behavior:
`ct.next(..., default_utc=False)` . If you want to use the new behavior *now*:
`ct.next(..., default_utc=True)`. If you pass a datetime object with a tzinfo
attribute that is not None, timezones will *just work* to the best of their
ability. There are tests...'''
if sys.version_info >= (3, 0):
_number_types = (int, float)
xrange = range
else:
_number_types = (int, long, float)
SECOND = timedelta(seconds=1)
MINUTE = timedelta(minutes=1)
HOUR = timedelta(hours=1)
DAY = timedelta(days=1)
WEEK = timedelta(days=7)
MONTH = timedelta(days=28)
YEAR = timedelta(days=365)
WARN_CHANGE = object()
# find the next scheduled time
def _end_of_month(dt):
ndt = dt + DAY
while dt.month == ndt.month:
dt += DAY
return ndt.replace(day=1) - DAY
def _month_incr(dt, m):
odt = dt
dt += MONTH
while dt.month == odt.month:
dt += DAY
# get to the first of next month, let the backtracking handle it
dt = dt.replace(day=1)
return dt - odt
def _year_incr(dt, m):
# simple leapyear stuff works for 1970-2099 :)
mod = dt.year % 4
if mod == 0 and (dt.month, dt.day) < (2, 29):
return YEAR + DAY
if mod == 3 and (dt.month, dt.day) > (2, 29):
return YEAR + DAY
return YEAR
_increments = [
lambda *a: SECOND,
lambda *a: MINUTE,
lambda *a: HOUR,
lambda *a: DAY,
_month_incr,
lambda *a: DAY,
_year_incr,
lambda dt,x: dt.replace(second=0),
lambda dt,x: dt.replace(minute=0),
lambda dt,x: dt.replace(hour=0),
lambda dt,x: dt.replace(day=1) if x > DAY else dt,
lambda dt,x: dt.replace(month=1) if x > DAY else dt,
lambda dt,x: dt,
]
# find the previously scheduled time
def _day_decr(dt, m):
if m.day.input != 'l':
return -DAY
odt = dt
ndt = dt = dt - DAY
while dt.month == ndt.month:
dt -= DAY
return dt - odt
def _month_decr(dt, m):
odt = dt
# get to the last day of last month, let the backtracking handle it
dt = dt.replace(day=1) - DAY
return dt - odt
def _year_decr(dt, m):
# simple leapyear stuff works for 1970-2099 :)
mod = dt.year % 4
if mod == 0 and (dt.month, dt.day) > (2, 29):
return -(YEAR + DAY)
if mod == 1 and (dt.month, dt.day) < (2, 29):
return -(YEAR + DAY)
return -YEAR
def _day_decr_reset(dt, x):
if x >= -DAY:
return dt
cur = dt.month
while dt.month == cur:
dt += DAY
return dt - DAY
_decrements = [
lambda *a: -SECOND,
lambda *a: -MINUTE,
lambda *a: -HOUR,
_day_decr,
_month_decr,
lambda *a: -DAY,
_year_decr,
lambda dt,x: dt.replace(second=59),
lambda dt,x: dt.replace(minute=59),
lambda dt,x: dt.replace(hour=23),
_day_decr_reset,
lambda dt,x: dt.replace(month=12) if x < -DAY else dt,
lambda dt,x: dt,
_year_decr,
]
Matcher = namedtuple('Matcher', 'second, minute, hour, day, month, weekday, year')
def _assert(condition, message, *args):
if not condition:
raise ValueError(message%args)
class _Matcher(object):
__slots__ = 'allowed', 'end', 'any', 'input', 'which', 'split'
def __init__(self, which, entry):
_assert(0 <= which <= YEAR_OFFSET,
"improper number of cron entries specified")
self.input = entry.lower()
self.split = self.input.split(',')
self.which = which
self.allowed = set()
self.end = None
self.any = '*' in self.split or '?' in self.split
for it in self.split:
al, en = self._parse_crontab(which, it)
if al is not None:
self.allowed.update(al)
self.end = en
_assert(self.end is not None,
"improper item specification: %r", entry.lower()
)
self.allowed = frozenset(self.allowed)
def __call__(self, v, dt):
for i, x in enumerate(self.split):
if x == 'l':
if v == _end_of_month(dt).day:
return True
elif x.startswith('l'):
# We have to do this in here, otherwise we can end up, for
# example, accepting *any* Friday instead of the *last* Friday.
if dt.month == (dt + WEEK).month:
continue
x = x[1:]
if x.isdigit():
x = int(x) if x != '7' else 0
if v == x:
return True
continue
start, end = map(int, x.partition('-')[::2])
allowed = set(range(start, end+1))
if 7 in allowed:
allowed.add(0)
if v in allowed:
return True
return self.any or v in self.allowed
def __lt__(self, other):
if self.any:
return self.end < other
return all(item < other for item in self.allowed)
def __gt__(self, other):
if self.any:
return _ranges[self.which][0] > other
return all(item > other for item in self.allowed)
def __eq__(self, other):
if self.any:
return other.any
return self.allowed == other.allowed
def __hash__(self):
return hash((self.any, self.allowed))
def _parse_crontab(self, which, entry):
'''
This parses a single crontab field and returns the data necessary for
this matcher to accept the proper values.
See the README for information about what is accepted.
'''
# this handles day of week/month abbreviations
def _fix(it):
if which in _alternate and not it.isdigit():
if it in _alternate[which]:
return _alternate[which][it]
_assert(it.isdigit(),
"invalid range specifier: %r (%r)", it, entry)
it = int(it, 10)
_assert(_start <= it <= _end_limit,
"item value %r out of range [%r, %r]",
it, _start, _end_limit)
return it
# this handles individual items/ranges
def _parse_piece(it):
if '-' in it:
start, end = map(_fix, it.split('-'))
# Allow "sat-sun"
if which in (DAY_OFFSET, WEEK_OFFSET) and end == 0:
end = 7
elif it == '*':
start = _start
end = _end
else:
start = _fix(it)
end = _end
if increment is None:
return set([start])
_assert(_start <= start <= _end_limit,
"%s range start value %r out of range [%r, %r]",
_attribute[which], start, _start, _end_limit)
_assert(_start <= end <= _end_limit,
"%s range end value %r out of range [%r, %r]",
_attribute[which], end, _start, _end_limit)
_assert(start <= end,
"%s range start value %r > end value %r",
_attribute[which], start, end)
return set(range(start, end+1, increment or 1))
_start, _end = _ranges[which]
_end_limit = _end
# wildcards
if entry in ('*', '?'):
if entry == '?':
_assert(which in (DAY_OFFSET, WEEK_OFFSET),
"cannot use '?' in the %r field", _attribute[which])
return None, _end
# last day of the month
if entry == 'l':
_assert(which == DAY_OFFSET,
"you can only specify a bare 'L' in the 'day' field")
return None, _end
# for the last 'friday' of the month, for example
elif entry.startswith('l'):
_assert(which == WEEK_OFFSET,
"you can only specify a leading 'L' in the 'weekday' field")
es, _, ee = entry[1:].partition('-')
_assert((entry[1:].isdigit() and 0 <= int(es) <= 7) or
(_ and es.isdigit() and ee.isdigit() and 0 <= int(es) <= 7 and 0 <= int(ee) <= 7),
"last <day> specifier must include a day number or range in the 'weekday' field, you entered %r", entry)
return None, _end
increment = None
# increments
if '/' in entry:
entry, increment = entry.split('/')
increment = int(increment, 10)
_assert(increment > 0,
"you can only use positive increment values, you provided %r",
increment)
# allow Sunday to be specified as weekday 7
if which == WEEK_OFFSET:
_end_limit = 7
# handle singles and ranges
good = _parse_piece(entry)
# change Sunday to weekday 0
if which == WEEK_OFFSET and 7 in good:
good.discard(7)
good.add(0)
return good, _end
class CronTab(object):
__slots__ = 'matchers',
def __init__(self, crontab):
self.matchers = self._make_matchers(crontab)
def _make_matchers(self, crontab):
'''
This constructs the full matcher struct.
'''
crontab = _aliases.get(crontab, crontab)
ct = crontab.split()
if len(ct) == 5:
ct.insert(0, '0')
ct.append('*')
elif len(ct) == 6:
ct.insert(0, '0')
_assert(len(ct) == 7,
"improper number of cron entries specified; got %i need 5 to 7"%(len(ct,)))
matchers = [_Matcher(which, entry) for which, entry in enumerate(ct)]
return Matcher(*matchers)
def _test_match(self, index, dt):
'''
This tests the given field for whether it matches with the current
datetime object passed.
'''
at = _attribute[index]
attr = getattr(dt, at)
if index == WEEK_OFFSET:
attr = attr() % 7
return self.matchers[index](attr, dt)
def next(self, now=None, increments=_increments, delta=True, default_utc=WARN_CHANGE):
'''
How long to wait in seconds before this crontab entry can next be
executed.
'''
if default_utc is WARN_CHANGE and (isinstance(now, _number_types) or (now and not now.tzinfo) or now is None):
warnings.warn(WARNING_CHANGE_MESSAGE, FutureWarning, 2)
default_utc = False
now = now or (datetime.utcnow() if default_utc and default_utc is not WARN_CHANGE else datetime.now())
if isinstance(now, _number_types):
now = datetime.utcfromtimestamp(now) if default_utc else datetime.fromtimestamp(now)
# handle timezones if the datetime object has a timezone and get a
# reasonable future/past start time
onow, now = now, now.replace(tzinfo=None)
tz = onow.tzinfo
future = now.replace(microsecond=0) + increments[0]()
if future < now:
# we are going backwards...
_test = lambda: future.year < self.matchers.year
if now.microsecond:
future = now.replace(microsecond=0)
else:
# we are going forwards
_test = lambda: self.matchers.year < future.year
# Start from the year and work our way down. Any time we increment a
# higher-magnitude value, we reset all lower-magnitude values. This
# gets us performance without sacrificing correctness. Still more
# complicated than a brute-force approach, but also orders of
# magnitude faster in basically all cases.
to_test = ENTRIES - 1
while to_test >= 0:
if not self._test_match(to_test, future):
inc = increments[to_test](future, self.matchers)
future += inc
for i in xrange(0, to_test):
future = increments[ENTRIES+i](future, inc)
try:
if _test():
return None
except:
print(future, type(future), type(inc))
raise
to_test = ENTRIES-1
continue
to_test -= 1
# verify the match
match = [self._test_match(i, future) for i in xrange(ENTRIES)]
_assert(all(match),
"\nYou have discovered a bug with crontab, please notify the\n" \
"author with the following information:\n" \
"crontab: %r\n" \
"now: %r", ' '.join(m.input for m in self.matchers), now)
if not delta:
onow = now = datetime(1970, 1, 1)
delay = future - now
if tz:
delay += _fix_none(onow.utcoffset())
if hasattr(tz, 'localize'):
delay -= _fix_none(tz.localize(future).utcoffset())
else:
delay -= _fix_none(future.replace(tzinfo=tz).utcoffset())
return delay.days * 86400 + delay.seconds + delay.microseconds / 1000000.
def previous(self, now=None, delta=True, default_utc=WARN_CHANGE):
return self.next(now, _decrements, delta, default_utc)
def test(self, entry):
if isinstance(entry, _number_types):
entry = datetime.utcfromtimestamp(entry)
for index in xrange(ENTRIES):
if not self._test_match(index, entry):
return False
return True
def _fix_none(d, _=timedelta(0)):
if d is None:
return _
return d

226
src/pvqa.cfg Normal file
View File

@ -0,0 +1,226 @@
BOF Common
IntervalLength = 0.68
IsUseUncertain = false
IsUseMixMode = true
IsUseDistance = false
AllWeight = 1.0
SilWeight = 1
VoiWeight = 1
AllCoefficient = 1.0
SilCoefficient = 1.0
VoiCoefficient = 1.0
SilThreshold = -37.50
IsOnePointSil = false
IsNormResult = true
IsMapScore = true
EOF Common
BOF Detector
Name = SNR
DetectorType = SNR
IntThresh = 0.10
FrameThresh = 14
DetThresh = 0.10
PVQA-Flag = true
PVQA-Weight = 1.0
DetMode = Both
EOF Detector
BOF Detector
Name = DeadAir-00
DetectorType = DeadAir
IntThresh = 0.60
DetThresh = 0.60
PVQA-Flag = true
PVQA-Weight = 1.0
DetMode = Both
EOF Detector
BOF Detector
Name = DeadAir-01
DetectorType = DeadAir
IntThresh = 0.7
DetThresh = 0.7
PVQA-Flag = true
PVQA-Weight = 1.0
DetMode = Both
EOF Detector
BOF Detector
Name = Click
DetectorType = Clicking
IntThresh = 0.10
DetThresh = 0.10
PVQA-Flag = true
PVQA-Weight = 1.0
DetMode = Both
EOF Detector
BOF Detector
Name = VAD-Clipping
DetectorType = VADClipping
IntThresh = 0.0
FrameThresh = 0.0
DetThresh = 0.0
PVQA-Flag = true
PVQA-Weight = 1.0
DetMode = Both
EOF Detector
BOF Detector
Name = Amplitude-Clipping
DetectorType = AmpClipping
IntThresh = 0.00
FrameThresh = 1.00
DetThresh = 0.00
PVQA-Flag = true
PVQA-Weight = 1.00
DetMode = Both
EOF Detector
BOF Detector
Name = Dynamic-Clipping
DetectorType = AmpClipping
IntThresh = 0.05
FrameThresh = 1.50
DetThresh = 0
PVQA-Flag = true
PVQA-Weight = 0.0
DetMode = Voice
EOF Detector
BOF Base EchoMono
SamplesType = UnKnownCodec
StepLengthSec = 0.5
MinDelayMs = 50
MaxLengthMs = 2800
WindowFunckID = 0
SpanLengthMs = 50
EOF Base EchoMono
BOF Detector
Name = ECHO
DetectorType = EchoMono
IntThresh = 0.00
FrameThresh = -40.0
DetThresh = 0.00
PVQA-Flag = true
PVQA-Weight = 1.0
DetMode = Voice
STAT-Flag = true
SpanLengthMs = 50
EOF Detector
#BOF Detector
# Name = Silent-Call-Detection
# DetectorType = DeadAir
# IntThresh = 0.99
# DetThresh = 0.99
# PVQA-Flag = false
# PVQA-Weight = 1.0
#EOF Detector
BOF Base SNR
MinPowerThresh = 1.0000
LogEnergyCoefficient = 10.0000
MinSignalLevel = 40.0000
MinSNRDelta = 0.0001
MinEnergyDisp = 3.0000
MinEnergyDelta = 1.0000
SamplesType = UnKnownCodec
EOF Base SNR
BOF Base AmpClipping
FlyAddingCoefficient = 0.1000
IsUseDynamicClipping = false
SamplesType = UnKnownCodec
EOF Base AmpClipping
BOF Base Clicking
SamplesType = UnKnownCodec
EOF Base Clicking
BOF Base DeadAir
StuckDeltaThreshold = 6
MinNonStuckTime = 80
MinStuckTime = 80
MinStartNonStuckTime = 1920
MinLevelThreshold = 256
SamplesType = UnKnownCodec
EOF Base DeadAir
BOF Base VADClipping
SamplesType = UnKnownCodec
EOF Base VADClipping
BOF DeadAir-01
MinLevelThreshold = 0
EOF DeadAir-01
#BOF Silent-Call-Detection
# MinLevelThreshold = 0
# IsUseRMSPower = true
# MinRMSThreshold = -70
#EOF Silent-Call-Detection
BOF Dynamic-Clipping
FlyAddingCoefficient = 0.1000
SamplesType = UnKnownCodec
IsUseDynamicClipping = true
EOF Dynamic-Clipping
BOF Correction
IntStart = 5.0
IntEnd = 4.2
Mult = 1.0
#Shift = -1.7
Shift = 0
EOF Correction
BOF Correction
IntStart = 4.2
IntEnd = 3.5
Mult = 1.0
#Shift = -0.85
Shift = 0
EOF Correction
BOF SR Correction
SampleRate = 11000.0
Shift = 0.05
EOF SR Correction
BOF SR Correction
SampleRate = 16000.0
Shift = 0.1
EOF SR Correction
BOF SR Correction
SampleRate = 22000.0
Shift = 0.2
EOF SR Correction
BOF SR Correction
SampleRate = 32000.0
Shift = 0.3
EOF SR Correction
BOF SR Correction
SampleRate = 48000.0
Shift = 0.45
EOF SR Correction
BOF SR Correction
SampleRate = 96000.0
Shift = 0.5
EOF SR Correction
BOF SR Correction
SampleRate = 192000.0
Shift = 0.6
EOF SR Correction
BOF Scores Map
ScoresLine = 4;3.027000;2.935000;2.905000;2.818000;2.590000;2.432000;2.310000;1.665000;1.000000;
EOF Scores Map

253
src/utils.py Normal file
View File

@ -0,0 +1,253 @@
#!/usr/bin/python
import typing
import datetime
import traceback
import wave
import contextlib
import os
import sys
import smtplib
import socket
import sox
import io
from email.mime.multipart import MIMEMultipart
from email.mime.application import MIMEApplication
from email.mime.text import MIMEText
from email.utils import COMMASPACE, formatdate
# mute logging
silent_logging: bool = False
# verbose logging flag
verbose_logging: bool = False
# Log file
the_log = None
# 1 minute network timeout
NETWORK_TIMEOUT = 60
def open_log_file(path: str, mode: str):
global the_log
try:
the_log = open(path, mode)
except Exception as e:
log_error("Failed to open log file.", err=e)
def close_log_file():
global the_log
if the_log:
the_log.close()
def get_current_time_str():
return str(datetime.datetime.now())
def get_log_line(message: str) -> str:
current_time = get_current_time_str()
pid = os.getpid()
line = f'{current_time} : {pid} : {message}'
return line
def log(message: str):
global silent_logging, the_log
if not silent_logging:
line = get_log_line(message)
print(line)
if the_log:
if not the_log.closed:
the_log.write(f'{line}\n')
the_log.flush()
def log_error(message: str, err: Exception = None):
global the_log
err_string = message
if isinstance(err, Exception):
message = message + "".join(traceback.format_exception(err.__class__, err, err.__traceback__))
elif err:
message = message + str(err)
line = get_log_line(message)
print(line)
if the_log:
if not the_log.closed:
the_log.write(f'{line}\n')
the_log.flush()
def log_verbose(message):
global verbose_logging, silent_logging, the_log
if verbose_logging and len(message) > 0 and not silent_logging:
line = get_log_line(message)
print(line)
if the_log:
if not the_log.closed:
the_log.write(f'{line}\n')
the_log.flush()
def merge_two_dicts(x, y):
z = x.copy() # start with x's keys and values
z.update(y) # modifies z with y's keys and values & returns None
return z
def fix_sip_address(sip_target):
if not sip_target:
return None
if sip_target.startswith("sip:"):
return sip_target
if sip_target.startswith("sips:"):
return sip_target
return "sip:" + sip_target
# Finds length of audio file in seconds
def find_file_length(path):
with contextlib.closing(wave.open(path, 'r')) as f:
frames = f.getnframes()
rate = f.getframerate()
duration = frames / float(rate)
return duration
def get_script_path():
return os.path.dirname(os.path.realpath(sys.argv[0]))
def send_mail_report(email_config: dict, title: str, report: dict, files):
try:
log_verbose("Sending report via email...")
msg = MIMEMultipart()
# Prepare text contents
title = "PVQA MOS: " + str(report["mos_pvqa"]) + ", AQuA MOS: " + str(report["mos_aqua"])
text = title
# Setup email headers
msg["Subject"] = title
msg["From"] = email_config['email_from']
msg["To"] = email_config['email_to']
msg["Date"] = formatdate(localtime=True)
# Add text
msg.attach(MIMEText(text))
# Add files
for f in files or []:
with open(f, "rb") as fil:
part = MIMEApplication(
fil.read(),
Name=os.path.basename(f)
)
# After the file is closed
part['Content-Disposition'] = 'attachment; filename="%s"' % os.path.basename(f)
msg.attach(part)
# Login & send
smtp = smtplib.SMTP(email_config['email_server'])
log_verbose("Login to SMTP server...")
smtp.login(email_config['email_user'], email_config['email_password'])
log_verbose("Sending files...")
smtp.sendmail(email_config['email_from'], email_config['email_to'], msg.as_string())
smtp.close()
log_verbose("Email sent.")
except Exception as err:
print("Exception when sending email: {0}".format(err))
def get_wav_length(path) -> float:
try:
with wave.open(str(path)) as f:
return f.getnframes() / f.getframerate()
except Exception as e:
log_error(f'Failed to get .wav file {path} length. Error: {e}')
return 0.0
def is_port_busy(port: int) -> bool:
try:
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind(('0.0.0.0', port))
s.close()
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as s:
s.bind(('0.0.0.0', port))
s.close()
with socket.socket(socket.AF_INET6, socket.SOCK_STREAM) as s:
s.bind(('::1', port))
s.close()
with socket.socket(socket.AF_INET6, socket.SOCK_DGRAM) as s:
s.bind(('::1', port))
s.close()
return False
except:
log_error(f"Failed to check if port {port} is busy.", err=sys.exc_info()[0])
return True
def resample_to(path: str, rate: int):
with wave.open(path, 'rb') as wf:
if rate == wf.getframerate():
return # Resampling is not needed
else:
log(f'Resampling {path} from {wf.getframerate()} to {rate}.')
TEMP_RESAMPLED = '/dev/shm/temp_resampled.wav'
retcode = os.system(f'sox {path} -c 1 -r {rate} {TEMP_RESAMPLED}')
if retcode != 0:
raise RuntimeError(f'Failed to convert {path} to samplerate {rate}')
os.remove(path)
os.rename(TEMP_RESAMPLED, path)
def join_host_and_path(hostname: str, path):
if not hostname.startswith("http://") and not hostname.startswith("https://"):
hostname = "http://" + hostname
if not hostname.endswith("/"):
hostname = hostname + "/"
if path.startswith("/"):
path = path[1:]
return hostname + path
# Prepare audio reference for playing. Generates silence prefix & suffix, merges them with audio itself.
# Resamples everything to 48K and stereo (currently it is required )
def prepare_reference_file(fname: str, silence_prefix_length: float, silence_suffix_length: float, output_fname: str):
tfm = sox.Transformer()
tfm.rate(44100)
tfm.channels(2)
tfm.pad(start_duration=silence_prefix_length, end_duration=silence_suffix_length)
tfm.build_file(input_filepath=fname, output_filepath=output_fname)
def is_raspberrypi():
try:
with io.open('/sys/firmware/devicetree/base/model', 'r') as m:
if 'raspberry pi' in m.read().lower(): return True
except Exception: pass
return False

126
src/utils_alsa.py Normal file
View File

@ -0,0 +1,126 @@
import wave
import argparse
import os
import sys
import signal
import time
import utils
import typing
import subprocess
import sox
import re
# To mono audio
CHANNELS = 1
# Target rate is 16K
RATE = 48000
CHUNK = 1024
# Time limitation 300 seconds
TIME_LIMIT = 300
# Restart PyAudio
def restart_audio():
return
class AlsaRecorder:
def __init__(self, device_name: str, channels: int = 1, rate: int = RATE, fname: str = None):
self.channels = channels
self.rate = rate
self.device_name = device_name
self.fname = fname
def __exit__(self, exception, value, traceback):
self.stop_recording()
def close(self):
self.stop_recording()
def start_recording(self):
utils.log(f'Start recording with device name {self.device_name}, channels {self.channels}, samplerate {self.rate} to {self.fname}')
# /usr/bin/nice -n -5
cmd = f'/usr/bin/arecord -D {self.device_name} --format S16_LE --rate {self.rate} -c {self.channels} --buffer-size 262144 {self.fname}'
utils.log_verbose(cmd)
self.process_handle = subprocess.Popen(cmd.split())
return self
def stop_recording(self):
if self.process_handle:
try:
self.process_handle.send_signal(signal.SIGINT)
self.process_handle.wait(timeout=5.0)
except:
utils.log_error(f'/usr/bin/arecord timeout on exit')
self.process_handle = None
utils.log(f'ALSA recording stopped.')
return self
@classmethod
def find_default(cls) -> str:
return find_alsa_usb_device('arecord')
class AlsaPlayer:
def __init__(self, device_name: str, channels: int = 1, rate: int = RATE, fname: str = None):
self.channels = channels
self.rate = rate
self.device_name = device_name
self.fname = fname
def __exit__(self, exception, value, traceback):
self.stop_playing()
def close(self):
self.stop_playing()
def start_playing(self):
utils.log(f'Start playing with device name {self.device_name}, channels {self.channels}, samplerate {self.rate} from {self.fname}')
# /usr/bin/nice -n -5
cmd = f'/usr/bin/aplay -D {self.device_name} --format S16_LE --rate {self.rate} -c {self.channels} --buffer-size 128000 {self.fname}'
utils.log_verbose(cmd)
self.process_handle = subprocess.Popen(cmd.split())
return self
def stop_playing(self):
if self.process_handle:
try:
self.process_handle.send_signal(signal.SIGINT)
self.process_handle.wait(timeout=5.0)
except:
utils.log_error(f'/usr/bin/aplay timeout on exit')
self.process_handle = None
utils.log(f'ALSA playing stopped.')
return self
@classmethod
def find_default(cls) -> str:
return find_alsa_usb_device('aplay')
# utility should aplay or arecord
def find_alsa_usb_device(utility: str) -> str:
retcode, aplay_output = subprocess.getstatusoutput(f'/usr/bin/{utility} -l')
if retcode != 0:
return None
# Parse data line by line
pattern = r'card\s(?P<card_id>\d+):(?P<card_name>.+)device\s(?P<device_id>\d+):(?P<device_name>.+)'
lines = aplay_output.splitlines()
for l in lines:
found = re.match(pattern, l)
if found:
if 'card_id' in found.groupdict() and 'card_name' in found.groupdict() and 'device_id' in found.groupdict() and 'device_name' in found.groupdict():
card_id = found.group('card_id')
card_name = found.group('card_name')
device_id = found.group('device_id')
device_name = found.group('device_name')
if 'usb' in card_name.lower() and 'usb' in device_name.lower():
return f'hw:{card_id},{device_id}'
return None

457
src/utils_audio.py Normal file
View File

@ -0,0 +1,457 @@
import pyaudio
import wave
import argparse
import os
import sys
import signal
import time
import utils
import typing
import subprocess
import sox
import re
from typing import Tuple
# Record with bitrate width 16 bits
FORMAT = pyaudio.paInt16
# To mono audio
CHANNELS = 1
# Target rate is 16K
RATE = 48000
CHUNK = 1024
# Time limitation 300 seconds
TIME_LIMIT = 300
# Open PyAudio instance
PY_AUDIO = pyaudio.PyAudio()
# Restart PyAudio
def restart_audio():
global PY_AUDIO
if PY_AUDIO:
PY_AUDIO.terminate()
PY_AUDIO = None
PY_AUDIO = pyaudio.PyAudio()
# Get list of input files
def get_input_devices():
# Dump info about available audio devices
info = PY_AUDIO.get_host_api_info_by_index(0)
numdevices = info.get('deviceCount')
result = []
for i in range(0, numdevices):
device_info = PY_AUDIO.get_device_info_by_host_api_device_index(0, i)
num_channels = device_info.get('maxInputChannels')
if num_channels > 0:
result.append({'name': device_info.get('name'), 'index': i, 'num_channels': num_channels, 'default_samplerate': device_info['defaultSampleRate']})
return result
class Recorder(object):
'''A recorder class for recording audio to a WAV file.
Records in mono by default.
'''
def __init__(self, device_index=0, channels=1, rate=RATE, frames_per_buffer=1024):
self.channels = channels
self.rate = rate
self.frames_per_buffer = frames_per_buffer
self.device_index = device_index
def open(self, fname, mode='wb'):
return RecordingFile(fname, mode, self.device_index, self.channels, self.rate,
self.frames_per_buffer)
class RecordingFile(object):
def __init__(self, fname, mode, device_index, channels,
rate, frames_per_buffer):
self.fname = fname
self.mode = mode
self.channels = channels
self.rate = rate
self.frames_per_buffer = frames_per_buffer
self.wavefile = self._prepare_file(self.fname, self.mode)
self._stream = None
self.device_index = device_index
def __enter__(self):
return self
def __exit__(self, exception, value, traceback):
self.close()
def start_recording(self):
utils.log(f'Start recording with device index {self.device_index}, channels {self.channels}, samplerate {self.rate} to {self.fname}')
# Use a stream with a callback in non-blocking mode
self._stream = PY_AUDIO.open(format=pyaudio.paInt16,
channels=self.channels,
rate=int(self.rate),
input=True,
input_device_index=self.device_index,
frames_per_buffer=self.frames_per_buffer,
stream_callback=self.get_callback())
self._stream.start_stream()
return self
def stop_recording(self):
self._stream.stop_stream()
utils.log(f'Recording stopped.')
return self
def get_callback(self):
def callback(in_data, frame_count, time_info, status):
self.wavefile.writeframes(in_data)
return in_data, pyaudio.paContinue
return callback
def close(self):
if self._stream:
self._stream.close()
self._stream = None
if self.wavefile:
self.wavefile.close()
self.wavefile = None
utils.log('Recorder device & file are closed.')
def _prepare_file(self, fname, mode='wb'):
wavefile = wave.open(fname, mode)
wavefile.setnchannels(self.channels)
wavefile.setsampwidth(PY_AUDIO.get_sample_size(pyaudio.paInt16))
wavefile.setframerate(self.rate)
return wavefile
# Show available input devices
def show_input_devices():
# Get list of devices
devices = get_input_devices()
for d in devices:
print(f'Idx: {d["index"]} name: {d["name"]} channels: {d["num_channels"]} default samplerate: {d["default_samplerate"]}')
# Returns tuple with device index and device rate
def get_input_device_index(device_name: str) -> Tuple[int, int]:
# Get list of devices to find device index
devices = get_input_devices()
# Find device index
device_index = 0
rate = 0
if device_name.isnumeric():
device_index = int(device_name)
found_devices = list(filter(lambda item: int(item['index']) == device_index, devices))
if found_devices is None or len(found_devices) == 0:
utils.log_error(f'Failed to find record audio device with index {device_index}, exiting')
return -1, 0
rate = found_devices[0]['default_samplerate']
else:
found_devices = list(filter(lambda item: device_name.lower() in item['name'].lower(), devices))
if found_devices is None or len(found_devices) == 0:
utils.log_error(f'Failed to find record audio device {device_name}, exiting')
return -1
device_index = found_devices[0]['index']
rate = found_devices[0]['default_samplerate']
return device_index, rate
# Capture on device with name device_name (or it can be index in string representation)
def capture(device_name: str, samplerate: int, limit: int, output_path: str) -> bool:
if os.path.exists(output_path):
utils.log("Warning - output file exists, it will be rewritten.")
device_index, rate = get_input_device_index(device_name)
if device_index == -1:
return False
utils.log_verbose('Starting record with device {device_name}, samplerate {samplerate}, output file {output_path}')
rec = Recorder(device_index=device_index, channels=CHANNELS, rate=rate)
with rec.open(output_path) as recfile:
recfile.start_recording()
time.sleep(limit)
recfile.stop_recording()
# Playing support
def get_output_devices():
# Dump info about available audio devices
info = PY_AUDIO.get_host_api_info_by_index(0)
numdevices = info.get('deviceCount')
result = []
for i in range(0, numdevices):
device_info = PY_AUDIO.get_device_info_by_host_api_device_index(0, i)
num_channels = device_info.get('maxOutputChannels')
if num_channels > 0:
result.append({'name': device_info.get('name'), 'index': i, 'num_channels': num_channels, 'default_samplerate': device_info['defaultSampleRate']})
return result
def get_output_device_index(device_name: str) -> Tuple[int, int]:
# Look for device index
devices = get_output_devices()
device_index = -1
rate = 0
if device_name.isnumeric():
# Get device by index
device_index = int(device_name)
# Check if this index belongs to playing devices
found_devices = list(filter(lambda item: int(item['index']) == device_index, devices))
if found_devices is None or len(found_devices) == 0:
utils.log_error(f'Failed to find play audio device with index {device_index}, exiting')
return -1, 0
rate = found_devices[0]['default_samplerate']
else:
found_devices = list(filter(lambda item: device_name.lower() in item['name'].lower(), devices))
if found_devices is None or len(found_devices) == 0:
utils.log_error(f'Failed to find play audio device {device_name}, exiting')
return -1, 0
device_index = found_devices[0]['index']
rate = found_devices[0]['default_samplerate']
return device_index, rate
class Player(object):
'''A player class for playing audio from a WAV file.
'''
def __init__(self, device_index=0, frames_per_buffer=1024):
self.device_index = device_index
self.frames_per_buffer = frames_per_buffer
def open(self, fname, mode='rb', silence_prefix: int = 0, silence_suffix: int = 0):
return PlayingFile(fname, mode, self.device_index,
self.frames_per_buffer, silence_prefix, silence_suffix)
class PlayingFile(object):
def __init__(self, fname, mode, device_index, frames_per_buffer, silence_prefix: int = 0, silence_suffix: int = 0):
self.fname = fname
self.mode = mode
self.frames_per_buffer = frames_per_buffer
self.wavefile = self._prepare_file(self.fname, self.mode)
self._stream = None
self.device_index = device_index
self.frames_counter = 0
# Normalize silence lengths
if silence_prefix is None:
silence_prefix = 0
if silence_suffix is None:
silence_suffix = 0
self.silence_prefix_total_frames: int = int(silence_prefix) * self.wavefile.getframerate()
self.silence_suffix_total_frames: int = int(silence_suffix) * self.wavefile.getframerate()
self.silence_prefix_frame_counter: int = 0
self.silence_suffix_frame_counter: int = 0
self.silence_prefix_finished: bool = False
self.silence_suffix_finished: bool = False
# Read all samples from wave file before playing to minimize possible delays
self.wavefile.rewind()
self.wavefile_frames = self.wavefile.readframes(self.wavefile.getnframes())
self.wavefile_read = 0 # Current offset
self.wavefile_length = self.wavefile.getnframes() # Total number of available frames
self.wavefile_finished = False
utils.log(f'Available {self.wavefile_length} frames in wave file {self.fname}')
def __enter__(self):
return self
def __exit__(self, exception, value, traceback):
self.close()
def start_playing(self):
rate = self.wavefile.getframerate()
channels = self.wavefile.getnchannels()
total_frames = self.wavefile.getnframes()
utils.log(f'Start playing with device #{self.device_index}, samplerate {rate}, channels {channels}, total frames {total_frames}')
utils.log(f'Silence prefix length: {self.silence_prefix_total_frames} frames, silence suffix length: {self.silence_suffix_total_frames} frames')
# Use a stream with a callback in non-blocking mode
self._stream = PY_AUDIO.open(format=pyaudio.paInt16,
channels=channels,
rate=rate,
output=True,
output_device_index=self.device_index,
frames_per_buffer=self.frames_per_buffer,
stream_callback=self.get_callback())
self._stream.start_stream()
return self
def stop_playing(self):
self._stream.stop_stream()
utils.log(f'Playing stopped.')
return self
def get_callback(self):
def callback(in_data, frame_count, time_info, status):
# print(f'Enter audio callback')
# Initialize with empty bytes
data = bytes(0)
# Save initial frame counter value
original_frame_count = frame_count
# Fill by 'prefix' silence if configured
if self.silence_prefix_total_frames and not self.silence_prefix_finished:
if self.silence_prefix_frame_counter < self.silence_prefix_total_frames:
# utils.log('Playing silence prefix')
# Check how much silence frames has to be sent
silence_frames_available = min(self.silence_prefix_total_frames - self.silence_prefix_frame_counter, frame_count)
# utils.log(f'Playing prefix silence {silence_frames_available} frames')
# Replace byte object
if silence_frames_available > 0:
data = bytes(silence_frames_available * 2)
self.silence_prefix_frame_counter += silence_frames_available
frame_count -= silence_frames_available
self.silence_prefix_finished = self.silence_prefix_frame_counter == self.silence_prefix_total_frames
if self.silence_prefix_finished:
utils.log(f'Silence prefix is played.')
# Fill by audio from file
if frame_count > 0 and not self.wavefile_finished:
# utils.log('Playing wave file')
# Read the audio
wavefile_available = min(self.wavefile_length - self.wavefile_read, frame_count)
# Frames are 16 bits - but this is a byte array
frames = self.wavefile_frames[self.wavefile_read * 2: (self.wavefile_read + wavefile_available) * 2]
# print(type(frames), type(self.wavefile_frames), len(frames))
# Increase counter of read frames
self.wavefile_read = self.wavefile_read + wavefile_available
# utils.log(f'Played {wavefile_available} frames, requested {frame_count}')
# utils.log(f'Playing wave file audio {len(frames)/2} frames')
if len(frames) > 0:
frame_count -= len(frames) / 2
data = data + frames
self.wavefile_finished = self.wavefile_read >= self.wavefile_length
if self.wavefile_finished:
utils.log(f'Wave file content is played.')
#else:
# utils.log('Wave file content is not played yet')
# Do we need silence_suffix ?
if self.silence_prefix_finished and self.wavefile_finished and frame_count > 0 and not self.silence_suffix_finished:
# utils.log('Playing silence suffix')
# File reading is over, switch to 'suffix' silence
silence_frames_available = int(min(self.silence_suffix_total_frames - self.silence_suffix_frame_counter, frame_count))
# utils.log(f'Playing suffix silence {silence_frames_available} frames')
if silence_frames_available > 0:
data = data + bytes(silence_frames_available * 2)
frame_count -= silence_frames_available
self.silence_suffix_finished = self.silence_suffix_frame_counter == self.silence_suffix_total_frames
# Increase counter of total read frames
self.frames_counter += original_frame_count - frame_count
if frame_count > 0:
# print(f'Have to read {frame_count} frames, available {len(data)}. Total read frames: {self.frames_counter}. Playing finished.')
code = pyaudio.paComplete
else:
code = pyaudio.paContinue
return (data, code)
return callback
def close(self):
if self._stream:
self._stream.close()
self._stream = None
if self.wavefile:
self.wavefile.close()
self.wavefile = None
utils.log('Player device & file are closed.')
def _prepare_file(self, fname, mode='rb') -> wave.Wave_read:
wavefile = wave.open(fname, mode)
return wavefile
def show_output_devices():
devices = get_output_devices()
for d in devices:
print(f'Idx: {d["index"]} name: {d["name"]} channels: {d["num_channels"]} default samplerate: {d["default_samplerate"]}')
def play(device_name: str, input_path: str, silence_prefix: int, silence_suffix: int) -> bool:
# Audio device will be opened with samplerate from input audio file
device_index, _ = get_output_device_index(device_name)
player = Player(device_index=device_index)
with player.open(input_path, 'rb', silence_prefix, silence_suffix) as pf:
pf.start_playing()
total_frames = pf.wavefile.getnframes() + (silence_prefix + silence_suffix) * pf.wavefile.getframerate()
while pf.frames_counter < total_frames:
time.sleep(0.1)
pf.stop_playing()
return True
def start_PA() -> bool:
# Ensure pulseaudio is available
retcode = os.system('pulseaudio --start')
if retcode != 0:
utils.log(f'pulseaudio failed to start, exit code: {retcode}')
return False
# Check if module-bluetooth-discover is available
retcode, output = subprocess.getoutput('/bin/bash pacmd list modules | grep module-bluetooth-discover')
if retcode == 0 and 'module-bluetooth-discover' in output:
utils.log('PA module-bluetooth-discover is loaded already.')
return True
utils.log('Attempt to load module-bluetooth-discover...')
retcode = os.system('pacmd load-module module-bluetooth-discover')
if retcode != 0:
utils.log(f'Failed to load module-bluetooth-discover, exit code: {retcode}')
return False
else:
print('...success.')
return True

52
src/utils_bt_audio.py Executable file
View File

@ -0,0 +1,52 @@
#!/usr/bin/python3
import os
import time
import utils
import typing
import subprocess
def start_PA() -> bool:
# Ensure pulseaudio is available
retcode = os.system('pulseaudio --check')
if retcode == 0:
utils.log('Stopping pulse audio...')
retcode = os.system('pulseaudio --kill')
if retcode != 0:
utils.log(f'pulseaudio failed to stop, exit code: {retcode}')
# return False
# Wait 5 second
utils.log('Waiting 5s for pulseaudio stop...')
time.sleep(5.0)
utils.log('Starting pulseaudio...')
retcode = os.system('pulseaudio --start')
if retcode != 0:
utils.log(f'pulseaudio failed to start, exit code: {retcode}')
return False
# Check if module-bluetooth-discover is available
retcode, output = subprocess.getstatusoutput('/usr/bin/pacmd list modules | /usr/bin/grep module-bluetooth-discover')
if retcode == 0:
if 'module-bluetooth-discover' in output:
utils.log('PA module-bluetooth-discover is loaded already.')
return True
else:
utils.log('PA module-bluetooth-discover is not loaded yet.')
utils.log('Attempt to load module-bluetooth-discover...')
retcode = os.system('pacmd load-module module-bluetooth-discover')
if retcode != 0:
utils.log(f'Failed to load module-bluetooth-discover, exit code: {retcode}')
return False
else:
print('...success.')
return True
if __name__ == '__main__':
start_PA()

109
src/utils_dtmf.py Normal file
View File

@ -0,0 +1,109 @@
#!/usr/bin/python3
import argparse
import os
import sys
import typing
import time
import subprocess
import select
import multiprocessing
import signal
import utils
import uiautomator2 as u2
import utils_mcon
# Exit codes
EXIT_SUCCESS = 0
EXIT_ERROR = 1
AUTOMATOR = None
def gsm_attach_automator():
# Run stock dialer as way to preload automator stack
utils.log("Connecting to device...")
d = u2.connect()
# Preload GSM helper app
utils.log("Preloading GSM helper app")
d.app_start("biz.sevana.qualtestgsm")
# Wait timeout for UI element is 60.0s
d.implicitly_wait(60.0)
# Preload stock dialer
# utils.log("Preloading stock dialer")
# d.app_start("com.skype.raider")
return d
def gsm_switch_to_dtmf_panel(d):
# As stub for now - use Skype Contact click
# d(resourceId="com.skype.raider:id/vm_name", text=contact_name).click()
return None
# Send DTMF string
def send_dtmf(dtmf: str):
global AUTOMATOR
gsm_switch_to_dtmf_panel(AUTOMATOR)
for c in dtmf:
utils_mcon.gsm_send_digit(c)
# Number of finished calls
CALL_COUNTER = multiprocessing.Value('i', 0)
def on_call_finished(file):
# Increase finished calls counter
CALL_COUNTER.value = CALL_COUNTER.value + 1
def make_call(target: str, dtmf: str):
global CALL_COUNTER
# Start subprocess to monitor events from Qualtest GSM
start_handler = lambda file_record, file_play, number: send_dtmf(dtmf)
finish_handler = lambda file_record, file_play, number: on_call_finished()
PROCESS_MONITOR = multiprocessing.Process(target=utils_mcon.gsm_monitor,
args=(None, None, start_handler, finish_handler, None))
PROCESS_MONITOR.start()
# Initiate GSM phone call via adb
utils_mcon.gsm_make_call(target)
# Wait for call finish with some timeout. Kill monitoring process on finish.
while CALL_COUNTER.value == 0:
time.sleep(1)
PROCESS_MONITOR.terminate()
if __name__ == '__main__':
# Default exit code
retcode = EXIT_SUCCESS
# Command line parameters
parser = argparse.ArgumentParser()
parser.add_argument("--target", help="target number")
parser.add_argument("--dtmf", help="DTMF string to send after call established")
args = parser.parse_args()
# Check if we have to make a call
try:
if args.target:
# Preload automator framework
AUTOMATOR = gsm_attach_automator()
# Start call
make_call(args.target, args.dtmf)
except Exception as e:
utils.log_error(e)
retcode = EXIT_ERROR
# Exit code 0 (success)
sys.exit(retcode)

114
src/utils_event.py Normal file
View File

@ -0,0 +1,114 @@
import json
import utils
# Constants from Qualtest GSM
EVENT_PREFIX = "[EVENT]"
EVENT_CALL_INCOMING = "INCOMING"
EVENT_CALL_FINISHED = "FINISHED"
EVENT_CALL_ESTABLISHED = "STARTED"
EVENT_IDLE = "IDLE"
# Call event - idle / incoming / established / stop from phone,
class CallEvent:
name: str = ''
number: str = ''
device_id: str = ''
session_id: str = ''
version: str = ''
permissions: str = ''
def __init__(self):
return
def __repr__(self):
return f'{self.device_id} / {self.session_id} / {self.name} / {self.number}'
@staticmethod
def parse_unified(line: str):
result: CallEvent = CallEvent()
# Strip line from logcat
if EVENT_PREFIX in line:
line = line[line.find(EVENT_PREFIX):].strip()
tokens = line.split(' ')
for token in tokens:
if '=' in token:
token_name = token[:token.find('=')].strip()
token_value = token[token.find('=') + 1:].strip().strip('"')
if token_name == 'event':
result.name = token_value
elif token_name == 'permissions':
result.permissions = token_value
elif token_name == 'network':
result.network = token_value
elif token_name == 'number':
result.number = token_value
elif token_name == 'version':
result.version = token_value
elif token_name == 's_id':
result.session_id = token_value
elif token_name == 'd_id':
result.device_id = token_value
if len(result.name) > 0:
return result
else:
return None
@staticmethod
def parse_logcat(line: str):
result: CallEvent = None
if not EVENT_PREFIX in line:
return None
line = line[line.find(EVENT_PREFIX):].strip()
# Split the components
parts = line.split(sep=' ')
# First is prefix, second is name
if len(parts) >= 3:
event_prefix = parts[0]
event_name = parts[1]
if event_prefix == EVENT_PREFIX:
result = CallEvent()
result.name = event_name
if event_name == EVENT_IDLE:
result.version = parts[2]
result.permissions = parts[3]
elif event_name in [EVENT_CALL_ESTABLISHED, EVENT_CALL_FINISHED, EVENT_CALL_INCOMING]:
result.number = parts[2]
return result
@staticmethod
def parse_json(v: str):
# Parse json
result: CallEvent = None
# Example of incoming JSON:
# '{"permissions":"yes","sessionid":"c7f5ff7a-2046-11ec-a3b4-c56aa472a250","deviceid":"test_phone","event":"IDLE","version":"1.1.0"}
try:
d = json.loads(v)
result = CallEvent()
result.device_id = d['deviceid']
result.session_id = d['sessionid']
if 'permissions' in d:
result.permissions = d['permissions']
result.name = d['event']
if 'number' in d:
result.number = d['number']
if 'version' in d:
result.version = d['version']
except Exception as e:
utils.log_error(f'Problem when building call event from AMQP source: {e}')
return result

107
src/utils_logcat.py Normal file
View File

@ -0,0 +1,107 @@
import os
import utils
import multiprocessing
import subprocess
import select
import time
import utils
import utils_event
# adb utility location
ADB = '/usr/bin/adb'
class LogcatEventSource(multiprocessing.Process):
terminate_flag: multiprocessing.Value = multiprocessing.Value('b')
# Monitoring time limit (in seconds)
timelimit: float = 300.0
# Please set this value before opening the logcat
queue: multiprocessing.Queue
def __init__(self):
super().__init__()
return
# def __repr__(self) -> str:
# return ''
def run(self):
# Clear from old logcat output
os.system(f'{ADB} logcat -c')
# Open adb logcat - show only messages from QualTestGSM
cmdline = f'{ADB} logcat QualTestGSM:D *.S'
utils.log_verbose(f'ADB command line: {cmdline}')
process_logcat = subprocess.Popen(cmdline, stdout=subprocess.PIPE, shell=True)
process_poll = select.poll()
process_poll.register(process_logcat.stdout, select.POLLIN)
# Monitoring start time
current_timestamp = time.monotonic()
# Read logcat output line by line
while self.terminate_flag.value == 0:
# Check if time limit is hit
if time.monotonic() - current_timestamp > self.timelimit:
break
current_timestamp = time.monotonic()
# Look for available data on stdout
try:
if not process_poll.poll(1):
continue
except:
break
current_line = None
try:
current_line = process_logcat.stdout.readline().decode()
except:
continue
if not current_line:
break
# Log read line
if 'QualTestGSM' in current_line:
utils.log_verbose(current_line.strip())
# Reset event name
event = utils_event.CallEvent.parse_unified(current_line)
if event is None:
# This line is not event description
continue
if self.queue is not None:
utils.log_verbose(f'Logcat event: {event}')
self.queue.put_nowait(event)
return
def open(self):
if self.is_alive():
return
# Reset terminate flag
self.terminate_flag.value = 0
# Start worker process
self.start()
# self.worker_process = multiprocessing.Process(target=self.worker, args=(self))
return
def close(self):
if not self.is_alive():
return
self.terminate_flag.value = 1
self.join()
return

648
src/utils_mcon.py Normal file
View File

@ -0,0 +1,648 @@
#!/usr/bin/python3
# coding: utf-8
import argparse
from multiprocessing.synchronize import Event
import os
import sys
import traceback
import time
import subprocess
import multiprocessing
import signal
import enum
import utils
import utils_logcat
import utils_rabbitmq
import utils_event
from enum import Enum
from multiprocessing import Value
import utils_alsa
# if not utils.is_raspberrypi():
# import utils_audio
# import uiautomator2 as u2
# This script is a bridge between android phone & audio recording & mobile helper app (Qualtest GSM)
ADB = utils_logcat.ADB
# This script version number
MCON_VERSION = "1.2.7"
# Audio devices to play & record
AUDIO_DEV_PLAY = None
AUDIO_DEV_RECORD = None
# Files to play & record
FILE_PLAY = None
FILE_RECORD = None
# Exit codes
EXIT_SUCCESS = 0
EXIT_ERROR = 1
# Time limitation for monitoring function
TIME_LIMIT_MONITORING = 86400*10000
# Subprocesses
PROCESS_MONITOR : multiprocessing.Process = None
# PROCESS_RECORD : multiprocessing.Process = None
# PROCESS_PLAY : multiprocessing.Process = None
# Log ADB messages in verbose mode ?
VERBOSE_ADB = False
# Call time limit (in seconds)
TIME_LIMIT_CALL = 120
# Silence suffix length (in seconds)
SILENCE_SUFFIX_LENGTH = 30
# Silence prefix length (in seconds)
SILENCE_PREFIX_LENGTH = 15
# Override samplerate if needed
SAMPLERATE: int = 48000
# Processing script
PROCESSING_SCRIPT = None
# Nr of processed calls
PROCESSED_CALLS: Value = Value('i', 0)
# Number of calls todo
LIMIT_CALLS: Value = Value('i', 0)
# Use aplay / arecord from alsa-utils to play&capture an audio
USE_ALSA_AUDIO: bool = False
# Stop notification. Put it to non-zero when script has to be stopped.
STOP_FLAG = multiprocessing.Value('i', 0)
RABBITMQ_CONNECTION = None
RABBITMQ_EXCHANGE = None
RABBITMQ_QUEUE = None
RABBITMQ_SESSIONID = None
# Can be 'caller' or 'answerer'
class Role(Enum):
Caller = 1
Answerer = 2
ROLE = None
def signal_handler(signum, frame):
print(f'Signal handler with code {signum}')
if PROCESS_MONITOR:
if PROCESS_MONITOR.is_alive:
print('Finishing the monitoring process...')
try:
if PROCESS_MONITOR._popen is not None:
PROCESS_MONITOR.terminate()
except Exception:
traceback.print_exc()
print('Signal handler exit.')
exit(0)
def start_gsm_app():
cmdline = f'{ADB} shell am start -n biz.sevana.qualtestgsm/.MainActivity'
retcode = os.system(cmdline)
if retcode != 0:
raise IOError()
# Initiates file playing and wait for finish (optionally)
def play_file(path: str, wait: bool, device: str, samplerate: int = None):
path_to_player = os.path.join(os.path.dirname(os.path.realpath(__file__)), 'audio_play.py')
cmdline = f'python3 {path_to_player} --device "{device}" --input "{path}"'
if samplerate:
cmdline = cmdline + f' --samplerate {samplerate}'
utils.log_verbose(cmdline)
if wait:
os.system(cmdline)
else:
p = subprocess.Popen(cmdline, stdout=subprocess.PIPE, shell=True)
return p
# Initiates file playing and wait for finish (optionally)
def record_file(path: str, wait: bool, device: str, time_limit: int = 10, samplerate: int = None):
path_to_recorder = os.path.join(os.path.dirname(os.path.realpath(__file__)), 'audio_record.py')
# Please be aware - macOS prohibits recording from microphone by default. When debugging under VSCode please ensure it has permission to record audio.
cmdline = f'python3 {path_to_recorder} --device "{device}" --output "{path}" --limit {time_limit}'
if samplerate:
cmdline = cmdline + f' --samplerate {samplerate}'
utils.log_verbose(cmdline)
if wait:
os.system(cmdline)
else:
p = subprocess.Popen(cmdline, stdout=subprocess.PIPE, shell=True)
return p
# Accept incoming GSM call
def gsm_accept_incoming():
os.system(f"{ADB} shell input keyevent 5")
# Reject incoming GSM call
def gsm_reject_incoming():
os.system(f"{ADB} shell input keyevent 6")
# Initiate new GSM call
def gsm_make_call(target: str):
os.system(f"{ADB} shell am start -a android.intent.action.CALL -d tel:{target}")
# End current GSM call
def gsm_stop_call():
os.system(f"{ADB} shell input keyevent 6")
utils.log_verbose('GSM call stop keyevent is sent.')
def gsm_send_digit(digit: str):
os.system(f"{ADB} shell input KEYCODE_{digit}")
#def gsm_attach_automator():
# # Run stock dialer as way to preload automator stack
# utils.log("Connecting to device...")
# d = u2.connect()
# # Preload GSM helper app
# utils.log("Preloading GSM helper app")
# d.app_start("biz.sevana.qualtestgsm")
# # Wait timeout for UI element is 60.0s
# d.implicitly_wait(60.0)
# # Preload stock dialer
# # utils.log("Preloading stock dialer")
# # d.app_start("com.skype.raider")
# return d
def gsm_switch_to_dtmf_panel(d):
# As stub for now - use Skype Contact click
# d(resourceId="com.skype.raider:id/vm_name", text=contact_name).click()
return None
def run_shell_script(file_recorded: str, file_played: str, number: str):
global PROCESSED_CALLS
# Log about passed parameters
utils.log_verbose(f'Running shell script with variables: recorded - {file_recorded}, played - {file_played}, number - {number}')
utils.log_verbose(f'Template: {PROCESSING_SCRIPT}')
# Prepare command line
cmdline = PROCESSING_SCRIPT.replace('$RECORDED', file_recorded).replace('$PLAYED', file_played).replace('$NUMBER', number)
utils.log_verbose(cmdline)
# Run script
retcode = os.system(cmdline)
if retcode != 0:
utils.log_error(f'Processing script call \'{cmdline}\' returned exit code {retcode}')
PROCESSED_CALLS.value = PROCESSED_CALLS.value + 1
return True
def run_error_handler(error_message):
global PROCESSED_CALLS
utils.log_error(f'Processing script call ended with problem: {error_message}')
# Increase counter of processed calls to allow script to exit
PROCESSED_CALLS.value = PROCESSED_CALLS.value + 1
class CallState(enum.Enum):
IDLE = 0
INCOMING = 1
ESTABLISHED = 2
# Monitor logcat output and tell about events
# on_start is lambda with 3 parameters (file_test, file_reference, phone_number)
# on_finish is lambda with 3 parameters (file_test, file_reference, phone_number)
PREPARED_REFERENCE_AUDIO = '/dev/shm/reference_prepared.wav'
def gsm_monitor(file_to_play: str, file_to_record: str, on_start, on_finish, on_error):
global PREPARED_REFERENCE_AUDIO, STOP_FLAG, USE_ALSA_AUDIO, AUDIO_DEV_RECORD, AUDIO_DEV_PLAY
utils.log_verbose(f'File to play: {file_to_play}, file to record: {file_to_record}')
utils.log_verbose(f'on_start: {on_start}, on_finish: {on_finish}, on_error: {on_error}')
# Reset stop flag
STOP_FLAG.value = 0
# Prepare reference audio for RPi
utils.prepare_reference_file(fname=file_to_play,
silence_prefix_length=SILENCE_PREFIX_LENGTH,
silence_suffix_length=SILENCE_SUFFIX_LENGTH,
output_fname=PREPARED_REFERENCE_AUDIO)
# Create event queue
event_queue = multiprocessing.Queue()
# Logcat event source
logcat = utils_logcat.LogcatEventSource()
logcat.queue = event_queue
logcat.open()
# RabbitMQ event source
rabbitmq = utils_rabbitmq.RabbitMQServer()
rabbitmq.event_queue = event_queue
rabbitmq.queue_name = RABBITMQ_QUEUE
rabbitmq.exchange_name = RABBITMQ_EXCHANGE
rabbitmq.url = RABBITMQ_CONNECTION
rabbitmq.open()
# Audio related processes and poll objects
audio_player = None
audio_recorder = None
# Ensure audio devices are recognized
if USE_ALSA_AUDIO:
if AUDIO_DEV_RECORD == 'auto':
AUDIO_DEV_RECORD = utils_alsa.AlsaRecorder.find_default()
utils.log(f'Recording device resolved to {AUDIO_DEV_RECORD}')
if AUDIO_DEV_PLAY == 'auto':
AUDIO_DEV_PLAY = utils_alsa.AlsaPlayer.find_default()
utils.log(f'Playing device resolved to {AUDIO_DEV_PLAY}')
# Monitoring start time
timestamp_start = time.monotonic()
# Call start time
timestamp_call = None
if ROLE == Role.Caller:
timestamp_call = time.monotonic()
# Should call to be stopped ?
force_call_stop = False
call_state : CallState = CallState.IDLE
# Read logcat output line by line
while True:
# Check if time limit is hit
if time.monotonic() - timestamp_start > TIME_LIMIT_MONITORING:
break
# Check if limit of calls hit
if LIMIT_CALLS.value != 0 and PROCESSED_CALLS.value >= LIMIT_CALLS.value:
break
# Check if call hit maximum length - smth goes weird, exit from the script
if timestamp_call:
if time.monotonic() - timestamp_call > TIME_LIMIT_CALL:
utils.log_verbose(f'Call time limit ({TIME_LIMIT_CALL}s). Stop the call.')
timestamp_call = None
# Try to end mobile call twice. Sometimes first attempt fails (observed on Galaxy M11).
gsm_stop_call()
gsm_stop_call()
if ROLE == Role.Caller:
# Treat call as stopped
# Exit from loop
utils.log_verbose(f'Exit from the processing loop as call time limit hit; smth goes wrong, exit from the script.')
# Signal to caller to stop processing outer script
STOP_FLAG.value = 1
# Exit
exit(1)
# break
# Next event ?
event: utils_event.CallEvent = None
try:
event = event_queue.get(timeout = 1.0)
except:
# No event available
continue
if event is None:
continue
if len(event.session_id) > 0 and event.session_id != RABBITMQ_SESSIONID:
utils.log_verbose(f'Skip event from old session')
continue
# Process events
if event.name == utils_event.EVENT_IDLE:
idle_detected = True
elif event.name == utils_event.EVENT_CALL_INCOMING:
if call_state != CallState.IDLE:
utils.log(f'Duplicate event {event}, ignoring.')
continue
call_state = CallState.INCOMING
# Accept incoming call
utils.log_verbose(f'Detected Incoming call notification (number {event.number}) from mobile helper app.')
# Double accept - sometimes phones ignore the first attempts
gsm_accept_incoming()
gsm_accept_incoming()
utils.log_verbose(f'Incoming call accepted.')
elif event.name == utils_event.EVENT_CALL_FINISHED:
if call_state != CallState.ESTABLISHED:
utils.log(f'Duplicate event {event}, ignoring.')
call_state = CallState.IDLE
utils.log_verbose(f'Detected call stop notification from the mobile helper app')
# Reset counter of call length
timestamp_call = None
# Stop playing & capturing
utils.log_verbose(f'Call from {event.number} finished.')
if audio_recorder:
audio_recorder.stop_recording()
audio_recorder.close()
audio_recorder = None
if audio_player:
audio_player.stop_playing()
audio_player.close()
audio_player = None
# Restart audio - lot of debugging output from ALSA libraries can be here. It is a known problem of ALSA libraries.
if USE_ALSA_AUDIO:
utils_alsa.restart_audio()
else:
utils_audio.restart_audio()
# Here recording finished, call script to process
if on_finish:
if os.path.exists(file_to_record):
utils.log(f'Recorded file: {file_to_record}')
# Call handler
if on_finish(file_to_record, file_to_play, event.permissions) in [False, None] :
utils.log_error(f'Analyzer routine returned negative result, exiting.')
# Signal to caller to stop processing outer script
STOP_FLAG.value = 1
sys.exit(EXIT_ERROR)
# Remove processed file before writing the next one
# if os.path.exists(file_to_record):
# os.remove(file_to_record)
else:
utils.log_error(f'Smth wrong - no recorded file {file_to_record}')
if not on_finish(None, file_to_play, None):
# Signal to caller to stop processing outer script
STOP_FLAG.value = 1
sys.exit(EXIT_ERROR)
elif event.name == utils_event.EVENT_CALL_ESTABLISHED:
if call_state == CallState.ESTABLISHED:
utils.log(f'Duplicate event {event}, ignoring.')
continue
call_state = CallState.ESTABLISHED
utils.log_verbose(f'Detected call start notification from the mobile helper app, trying to start audio.')
# Save call start time
timestamp_call = time.monotonic()
# Is audio failed
audio_failed = False
# Start playing
utils.log_verbose(f'Call with {event.number} is established.')
if file_to_play:
if not USE_ALSA_AUDIO:
device_index, device_rate = utils_audio.get_output_device_index(AUDIO_DEV_PLAY)
if SAMPLERATE:
device_rate = SAMPLERATE
utils.resample_to(file_to_play, int(device_rate))
utils.log_verbose(f'Playing file: {file_to_play}')
try:
if USE_ALSA_AUDIO:
audio_player = utils_alsa.AlsaPlayer(device_name=AUDIO_DEV_PLAY, channels=2, rate=48000, fname=PREPARED_REFERENCE_AUDIO)
else:
audio_player = utils_audio.Player(device_index=device_index).open(fname=file_to_play,
silence_prefix=SILENCE_PREFIX_LENGTH, silence_suffix=SILENCE_SUFFIX_LENGTH)
audio_player.start_playing()
except Exception as e:
utils.log_error(e)
audio_failed = True
# Start capturing
if file_to_record and not audio_failed:
utils.log_verbose(f'Recording file: {file_to_record}')
# Remove old file if needed
if os.path.exists(file_to_record):
os.remove(file_to_record)
if not USE_ALSA_AUDIO:
device_index, device_rate = utils_audio.get_input_device_index(AUDIO_DEV_RECORD)
if SAMPLERATE:
device_rate = SAMPLERATE
try:
if USE_ALSA_AUDIO:
audio_recorder = utils_alsa.AlsaRecorder(device_name=AUDIO_DEV_RECORD, rate=int(device_rate), fname=file_to_record)
else:
audio_recorder = utils_audio.Recorder(device_index=device_index, rate=int(device_rate)).open(fname=file_to_record)
audio_recorder.start_recording()
except Exception as e:
utils.log_error(e)
audio_failed = True
if audio_failed:
gsm_stop_call()
gsm_stop_call()
if on_error:
on_error('Audio failed.')
elif on_start:
on_start(file_to_record, file_to_play, event.number)
def make_call(target: str):
global ROLE, PROCESS_MONITOR, STOP_FLAG, PROCESSED_CALLS
ROLE = Role.Caller
# Start subprocess to monitor events from Qualtest GSM
finish_handler = lambda file_record, file_play, number: run_shell_script(file_record, file_play, number)
error_handler = lambda error_message: run_error_handler(error_message)
PROCESS_MONITOR = multiprocessing.Process(target=gsm_monitor, args=(FILE_PLAY, FILE_RECORD, None, finish_handler, error_handler))
PROCESS_MONITOR.start()
# Initiate GSM phone call via adb
gsm_make_call(target)
# Log
utils.log_verbose('Call is initiated, processing...')
# Wait for call finish with some timeout. Kill monitoring process on finish.
while True and STOP_FLAG.value != 1 and PROCESSED_CALLS.value == 0:
time.sleep(0.5)
# Kill the monitoring process - this will send SIGTERM signal. It is a cause why agent_gsm doesn't handle SIGTERM
PROCESS_MONITOR.terminate()
return None
def answer_calls():
global ROLE, PROCESS_MONITOR, STOP_FLAG, PROCESSED_CALLS
ROLE = Role.Answerer
# Start subprocess to monitor events from Qualtest GSM.
finish_handler = lambda file_record, file_play, number: run_shell_script(file_record, file_play, number)
error_handler = lambda error_message: run_error_handler(error_message)
PROCESS_MONITOR = multiprocessing.Process(target=gsm_monitor, args=(FILE_PLAY, FILE_RECORD, None, finish_handler, error_handler))
PROCESS_MONITOR.start()
# Indefinite loop. Exit is in signal handler
while True and STOP_FLAG.value != 1 and PROCESSED_CALLS.value == 0:
time.sleep(0.5)
# Kill the monitoring process - this will send SIGTERM signal. It is a cause why agent_gsm doesn't handle SIGTERM
PROCESS_MONITOR.terminate()
return None
if __name__ == '__main__':
# Default exit code
retcode = EXIT_SUCCESS
# Handle signals
signal.signal(signal.SIGINT, signal_handler)
signal.signal(signal.SIGTERM, signal_handler)
# Command line parameters
parser = argparse.ArgumentParser()
parser.add_argument("--play-device", help="Output audio device name. Used to play reference audio to mobile call. Example (for ALSA): hw:2,0")
parser.add_argument("--record-device", help="Input device name. Used to record audio received from the mobile call.")
# parser.add_argument("--show-devices", help="list available output audio devices.", action="store_true")
parser.add_argument("--make-call", help="Target number as is. Usuall smth like +XYZ. Initiate a call to target number invoking the call on mobile phone and playing/recording audio to/from the call. Otherwise script will run expecting for incoming call.")
parser.add_argument("--play-file", help="Path to played (reference) audio. On RPi platform this should be 48KHz stereo audio.")
parser.add_argument("--record-file", help="Path to recorded audio (received from mobile call). On RPi platform it will be 48KHz mono audio.")
parser.add_argument("--exec", help="Path to postprocessing script. Postprocessing script will be run after the call finish with path to recorded audio as parameter. This should be a string like /home/user/postprocessing.sh $RECORDED. Substring $RECORDED will be replaced with actual path to recorded audio.")
# parser.add_argument("--adb-path", help="Path to adb utility. This must be set to work with mobile phone!")
parser.add_argument("--call-timelimit", help="Number of seconds. Call will be ended after specified timeout. Default value is 0 - no timeout.")
parser.add_argument("--test-play", help="Play test audio file. Useful when testing configuration. However this will not work on RPi.", action="store_true")
parser.add_argument("--test-record", help="Record test audio file for 10 seconds. Useful when testing configuration. However this will not work on RPi.", action="store_true")
parser.add_argument("--silence-prefix", help="Number of seconds. Adds silence before played audio. Default value is 10 (seconds)")
parser.add_argument("--silence-suffix", help="Number of seconds. Adds silence after played audio. Default value is 10 (seconds)")
parser.add_argument("--verbose", help="Run in verbose mode. It doesn't generate too much data, recommended to set.", action="store_true")
parser.add_argument("--verbose-adb", help="Log ADB messages when running in verbose mode. This can generate a lot of data, please be aware.", action="store_true")
parser.add_argument("--log-file", help="Path to log file. By default log is sent to console.")
parser.add_argument("--version", help="Show version number & exit", action="store_true")
parser.add_argument("--alsa-audio", help="Use ALSA audio instead of PyAudio (portaudio)", action="store_true")
parser.add_argument("--rabbitmq-connection")
parser.add_argument("--rabbitmq-exchange")
parser.add_argument("--rabbitmq-queue")
parser.add_argument("--rabbitmq-sessionid")
# parser.add_argument("--dtmf", help="Send DTMF string after call establishing and finish a call. Helper tool for some cases.")
# parser.add_argument("--samplerate", help="<audio samplerate>. Overrides default audio samplerate.")
args = parser.parse_args()
if args.version:
print(f"Version: {MCON_VERSION}")
sys.exit(0)
RABBITMQ_CONNECTION = args.rabbitmq_connection
RABBITMQ_EXCHANGE = args.rabbitmq_exchange
RABBITMQ_QUEUE = args.rabbitmq_queue
RABBITMQ_SESSIONID = args.rabbitmq_sessionid
# ALSA audio ? Required on RPi
USE_ALSA_AUDIO = args.alsa_audio
# Open log file if needed
VERBOSE_ADB = args.verbose_adb
utils.verbose_logging = args.verbose
if args.log_file:
utils.open_log_file(args.log_file, "at")
utils.log(f"mcon version: {MCON_VERSION}")
if args.call_timelimit:
TIME_LIMIT_CALL = int(args.call_timelimit)
elif args.play_file:
TIME_LIMIT_CALL = utils.get_wav_length(args.play_file)
utils.log(f'Limiting call time to {TIME_LIMIT_CALL}')
# Save audio devices
if args.play_device:
AUDIO_DEV_PLAY = args.play_device
if args.record_device:
AUDIO_DEV_RECORD = args.record_device
# Save files to play & record
if args.play_file:
FILE_PLAY = args.play_file
if args.record_file:
FILE_RECORD = args.record_file
# Processing script
if args.exec:
PROCESSING_SCRIPT = args.exec
# Should we make test here ?
if args.test_play:
if FILE_PLAY:
utils.log(f"Start test playing {FILE_PLAY} to {AUDIO_DEV_PLAY}")
play_file(FILE_PLAY, device=AUDIO_DEV_PLAY, wait=True)
else:
utils.log_error("File to play is not specified, exiting.")
retcode = EXIT_ERROR
sys.exit(retcode)
if args.test_record:
if FILE_RECORD:
utils.log(f"Start test recording from {AUDIO_DEV_RECORD} to {FILE_RECORD}")
record_file(FILE_RECORD, device=AUDIO_DEV_RECORD, wait=True)
else:
utils.log_error("File to record is not specified, exiting")
retcode = EXIT_ERROR
sys.exit(retcode)
# Check if we have to make a call
try:
if args.make_call:
make_call(args.make_call)
else:
answer_calls()
except Exception as e:
utils.log_error(e)
# Close log file
utils.close_log_file()
# Exit code 0 (success)
sys.exit(retcode)

349
src/utils_qualtest.py Normal file
View File

@ -0,0 +1,349 @@
#!/usr/bin/python
import utils
import re
import subprocess
import typing
import csv
import platform
import json
import os
import sys
import urllib.request
import urllib
import uuid
import time
import requests
from socket import timeout
from crontab import CronTab
start_system_time = time.time()
start_monotonic_time = time.monotonic()
# Error report produced by this function has to be updated with 'task_name' & 'phone_name' keys
def build_error_report(endtime: int, reason: str):
r = dict()
r["id"] = uuid.uuid1().urn[9:]
r["duration"] = 0
r["endtime"] = endtime
r["mos_pvqa"] = 0.0
r["mos_aqua"] = 0.0
r["mos_network"] = 0.0
r["r_factor"] = 0
r["percents_aqua"] = 0.0
r["error"] = reason
return r
class TaskList:
tasks: list = []
def __init__(self):
self.tasks = []
# Merges incoming task list to existing one
# It preserves existing schedules
# New items are NOT scheduled automatically
def merge_with(self, tasklist) -> bool:
changed = False
if tasklist.tasks is None:
return True
# Iterate all tasks, see if task with the same name exists already
# Copy all keys, but keep existing ones
for new_task in tasklist.tasks:
# Find if this task exists already
existing_task = self.find_task_by_name(new_task["name"])
# If task is found - copy all items to it.
# It is required as task can hold schedule items already
# Bad idea to copy tasks itself.
if existing_task is not None:
# Check if scheduled time point has to be removed (if cron string changed)
if new_task["schedule"] != existing_task["schedule"] and "scheduled_time" in existing_task:
del existing_task["scheduled_time"]
# Finally copy new values
for key, value in new_task.items():
if existing_task[key] != value:
existing_task[key] = value
changed = True
else:
# Copy new task to list
self.tasks.extend([new_task])
changed = True
# Check if old tasks are here... And delete them
for existing_task in self.tasks:
new_task = self.find_task_by_name(existing_task["name"])
if new_task is None:
self.tasks.remove(existing_task)
changed = True
return changed
def schedule(self):
# Remove items without schedule before
self.tasks = [t for t in self.tasks if len(t['schedule']) > 0]
# https://crontab.guru is good for crontab strings generation
# Use monotonic time source!
current_time = time.monotonic()
for task in self.tasks:
if 'scheduled_time' not in task and 'schedule' in task:
# No schedule flag, so time to schedule
try:
cron_string = task['schedule'].strip()
if cron_string == '* * * * *':
task['scheduled_time'] = time.monotonic() - 0.001 # To ensure further comparison will not be affected by precision errors
else:
cron = CronTab(task['schedule'])
task['scheduled_time'] = current_time + cron.next(default_utc=True)
# Just to help in further log reading & debugging - show the scheduled time in readable form
task['scheduled_time_str'] = time.ctime(task['scheduled_time'] - start_monotonic_time + start_system_time)
except:
utils.log_error("Error", sys.exc_info()[0])
# Remove non scheduled items
self.tasks = [t for t in self.tasks if 'scheduled_time' in t]
# Sort everything
self.tasks = sorted(self.tasks, key=lambda t: t["scheduled_time"])
# Returns None if failed
def find_task_by_name(self, name):
for t in self.tasks:
if t["name"] == name:
return t
return None
def ParseAttributes(t: str) -> dict:
result: dict = dict()
for l in t.split('\n'):
tokens = l.strip().split('=')
if len(tokens) == 2:
result[tokens[0].strip()] = tokens[1].strip()
return result
class Phone:
identifier: int = 0
name: str = ""
role: str = ""
attributes: dict = ""
audio_id: int = 0
def __init__(self):
self.identifier = 0
self.name = ""
self.role = ""
self.attributes = dict()
self.audio_id = 0
class QualtestBackend:
address: str
instance: str
def __init__(self):
self.address = ""
self.instance = ""
self.__phone = None
@property
def phone(self) -> Phone:
return self.__phone
def preload(self):
self.__phone = self.load_phone()
def upload_report(self, report, files) -> str:
# UUID string as result
result = None
# Log about upload attempt
utils.log_verbose(f"Uploading to {self.address} files {files} and report: {json.dumps(report, indent=4)}")
# POST will be sent to args.qualtest_server with args.qualtest_instance ID
json_content = json.dumps(report, indent=4).encode('utf8')
# Find URL for uploading
url = utils.join_host_and_path(self.address, "/probes/")
try:
# Step 1 - upload result record
req = urllib.request.Request(url,
data=json_content,
headers={'content-type': 'application/json'})
response = urllib.request.urlopen(req, timeout=utils.NETWORK_TIMEOUT)
result = response.read().decode('utf8')
utils.log_verbose(f"Response (probe ID): {result}")
utils.log_verbose(f"Upload to {self.address} finished.")
except Exception as e:
utils.log_error(f"Upload to {self.address} finished with error.", err=e)
return result
def upload_audio(self, probe_id, path_recorded):
result = False
# Log about upload attempt
utils.log_verbose(f"Uploading to {self.address} audio {path_recorded}")
# Find URL for uploading
url = utils.join_host_and_path(self.address, "/upload_audio/")
try:
files = {'file': (os.path.basename(path_recorded), open(path_recorded, 'rb')),
'probe_id': (None, probe_id),
'audio_kind': (None, '1'),
'audio_name': (None, os.path.basename(path_recorded))}
# values = {'probe_id': probe_id}
response = requests.post(url, files=files, timeout=utils.NETWORK_TIMEOUT)
if response.status_code != 200:
utils.log_error(f"Upload audio to {self.address} finished with error {response.status_code}", None)
else:
utils.log_verbose(f"Response (audio ID): {response.text}")
utils.log_verbose(f"Upload audio to {self.address} finished.")
result = True
except Exception as e:
utils.log_error(f"Upload audio to {self.address} finished with error.", err=e)
return result
def load_tasks(self) -> TaskList:
try:
# Build query for both V1 & V2 API
instance = urllib.parse.urlencode({"phone_id": self.instance, "phone_name": self.instance})
# Find URL
url = utils.join_host_and_path(self.address, "/tasks/?") + instance
# Get response from server
response = urllib.request.urlopen(url, timeout=utils.NETWORK_TIMEOUT)
if response.getcode() != 200:
utils.log_error("Failed to get task list. Error code: %s" % response.getcode())
return None
result = TaskList()
response_content = response.read().decode()
result.tasks = json.loads(response_content)
return result
except Exception as err:
utils.log_error("Exception when fetching task list: {0}".format(err))
return None
def load_phone(self) -> dict:
try:
# Build query for both V1 & V2 API
instance = urllib.parse.urlencode({"phone_id": self.instance, "phone_name": self.instance})
# Find URL
url = utils.join_host_and_path(self.address, "/phones/?") + instance
# Get response from server
response = urllib.request.urlopen(url, timeout=utils.NETWORK_TIMEOUT)
if response.getcode() != 200:
utils.log_error("Failed to get task list. Error code: %s" % response.getcode())
return None
result: Phone = Phone()
phones = json.loads(response.read().decode())
if len(phones) == 0:
return result
phone = phones[0]
attr_dict = dict()
attributes_string = phone['attributes']
attributes_lines = attributes_string.split('\n')
for l in attributes_lines:
parts = l.split('=')
if len(parts) == 2:
p0: str = parts[0]
p1: str = parts[1]
attr_dict[p0.strip()] = p1.strip()
# Fix received attributes
if 'stun_server' in attr_dict:
attr_dict['sip_stunserver'] = attr_dict.pop('stun_server')
if 'transport' in attr_dict:
attr_dict['sip_transport'] = attr_dict.pop('transport')
if 'sip_secure' not in attr_dict:
attr_dict['sip_secure'] = False
if 'sip_useproxy' not in attr_dict:
attr_dict['sip_useproxy'] = True
result.attributes = attr_dict
result.identifier = phone['id']
result.name = phone['instance']
result.role = phone['type']
result.audio_id = phone['audio_id']
return result
except Exception as err:
utils.log_error("Exception when fetching task list: {0}".format(err))
return dict()
def load_audio(self, audio_id: int, output_path: str):
utils.log(f'Loading audio with ID: {audio_id}')
try:
# Build query for both V1 & V2 API
params = urllib.parse.urlencode({"audio_id": audio_id})
# Find URL
url = utils.join_host_and_path(self.address, "/play_audio/?") + params
# Get response from server
response = urllib.request.urlopen(url, timeout=utils.NETWORK_TIMEOUT)
if response.getcode() != 200:
utils.log_error("Failed to get audio. Error code: %s" % response.getcode())
return False
audio_content = response.read()
with open (output_path, 'wb') as f:
f.write(audio_content)
return True
except Exception as err:
utils.log_error("Exception when fetching list: {0}".format(err))
return False
def load_task(self, task_name: str) -> dict:
try:
params = urllib.parse.urlencode({'task_name': task_name})
url = utils.join_host_and_path(self.address, "/tasks/?" + params)
response = urllib.request.urlopen(url, timeout=utils.NETWORK_TIMEOUT)
if response.getcode() != 200:
utils.log_error(f'Failed to get task info. Error code: {response.getcode()}')
return None
task_list = json.loads(response.read().decode())
if len(task_list) > 0:
return task_list[0]
else:
return None
except Exception as err:
utils.log_error(f'Exception when fetching task info: {err}')
return None

53
src/utils_rabbitmq.py Normal file
View File

@ -0,0 +1,53 @@
#!/usr/bin/python3
import rabbitpy
import multiprocessing
import json
import utils
import utils_event
class RabbitMQServer(multiprocessing.Process):
channel = None
connection = None
url: str = None
queue_name: str = None
exchange_name: str = None
event_queue : multiprocessing.Queue = None
def __init__(self):
super().__init__()
return
def __repr__(self) -> str:
return f'URL: {self.url} , queue: {self.queue_name}, exchange: {self.exchange_name}'
def open(self):
if self.url is None or self.queue_name is None or self.exchange_name is None:
raise Exception('RabbitMQ server object parameters are not set.')
try:
self.start()
except Exception as e:
utils.log_error(e)
def close(self):
try:
self.join()
except Exception as e:
utils.log_error(e)
def run(self):
for message in rabbitpy.consume(uri=self.url, queue_name=self.queue_name):
# utils.log_verbose(message.body.decode('utf8'))
message.ack()
try:
event = utils_event.CallEvent.parse_unified(message.body.decode('utf8'))
if self.event_queue is not None and event is not None:
utils.log_verbose(f'AMQP event: {event}')
self.event_queue.put(event)
except Exception as e:
utils.log_error(e)

271
src/utils_sevana.py Normal file
View File

@ -0,0 +1,271 @@
#!/usr/bin/python
import utils
import re
import subprocess
import typing
import csv
import platform
import json
import os
import sys
import time
import urllib
from pathlib import Path
from colorama import Fore, Style
PVQA_CMD = "{pvqa} --license {pvqa_lic} --config {pvqa_cfg} --mode analysis --channel 0 " \
"--report {output} --input {input} --cut-begin {cut_begin} --cut-end {cut_end}"
PVQA_CMD_LIC_SERVER = "{pvqa} --license-server {pvqa_lic} --config {pvqa_cfg} --mode analysis --channel 0 " \
"--report {output} --input {input}"
AQUA_CMD = ("{aqua} {aqua_lic} -mode files -src file \"{reference}\" -tstf \"{input}\" -avlp off -smtnrm on "
"-decor off -mprio off -acr auto -npnt auto -voip on -enorm rms -g711 off "
"-spfrcor on -grad off -tmc on -hist-pitch on on -hist-levels on on on -miter 1 -specp 32 {spectrum} "
"-ratem %%m -fau {faults} -output json -trim r 15 -cut-tst {cut_begin} {cut_end} -cut-src {cut_begin_src} {cut_end_src}")
PVQA_PATH = ""
PVQA_LIC_PATH = "pvqa.lic"
PVQA_CFG_PATH = "pvqa.cfg"
AQUA_PATH = ""
AQUA_LIC_PATH = "aqua-wb.lic"
SILER_PATH = ""
if platform.system() == 'Windows':
PVQA_OUTPUT = 'pvqa_output.txt'
AQUA_FAULTS = 'aqua_faults.txt'
AQUA_SPECTRUM = 'aqua_spectrum.csv'
else:
PVQA_OUTPUT = '/dev/shm/pvqa_output.txt'
AQUA_FAULTS = '/dev/shm/aqua_faults.txt'
AQUA_SPECTRUM = '/dev/shm/aqua_spectrum.csv'
if platform.system() == 'Windows':
PVQA_PATH = 'pvqa.exe'
AQUA_PATH = 'aqua-wb.exe'
SILER_PATH = 'silence_eraser.exe'
SPEECH_DETECTOR_PATH = 'speech_detector.exe'
else:
PVQA_PATH = 'pvqa'
AQUA_PATH = 'aqua-wb'
SILER_PATH = 'silence_eraser'
SPEECH_DETECTOR_PATH = 'speech_detector'
def load_file(url: str, output_path: str):
try:
response = urllib.request.urlopen(url, timeout=utils.NETWORK_TIMEOUT)
if response.getcode() != 200:
utils.log_error(f'Fetch file {output_path} from URL {url} failed with code {response.getcode()}')
return
except urllib.error.HTTPError as e:
utils.log_error(f'Fetch file {output_path} from URL {url} failed with code {e.code}')
return
# Write downloaded content to file
response_content = response.read()
open(output_path, 'wb').write(response_content)
def load_config_and_licenses(server: str):
load_file(utils.join_host_and_path(server, '/deploy/pvqa.cfg'), PVQA_CFG_PATH)
load_file(utils.join_host_and_path(server, '/deploy/pvqa.lic'), PVQA_LIC_PATH)
load_file(utils.join_host_and_path(server, '/deploy/aqua-wb.lic'), AQUA_LIC_PATH)
def find_binaries(directory: str, license_server: str = None):
# Update path to pvqa/aqua-wb
global PVQA_CFG_PATH, PVQA_LIC_PATH, AQUA_LIC_PATH, PVQA_PATH, AQUA_PATH, PVQA_CMD, AQUA_CMD, SILER_PATH, SPEECH_DETECTOR_PATH
# Find platform prefix
platform_prefix = platform.system().lower()
if utils.is_raspberrypi():
platform_prefix = 'rpi'
bin_directory = Path(directory)
PVQA_PATH = bin_directory / platform_prefix / PVQA_PATH
PVQA_LIC_PATH = bin_directory / PVQA_LIC_PATH
PVQA_CFG_PATH = bin_directory / PVQA_CFG_PATH
AQUA_PATH = bin_directory / platform_prefix / AQUA_PATH
AQUA_LIC_PATH = bin_directory / AQUA_LIC_PATH
SILER_PATH = bin_directory / platform_prefix / SILER_PATH
SPEECH_DETECTOR_PATH = bin_directory / platform_prefix / SPEECH_DETECTOR_PATH
print(f'Looking for binaries/licenses/configs at {directory}...', end=' ')
# Check if binaries exist
if not PVQA_PATH.exists():
print(f'Failed to find pvqa binary at {PVQA_PATH}. Exiting.')
sys.exit(1)
if not PVQA_CFG_PATH.exists():
PVQA_CFG_PATH = Path(utils.get_script_path()) / 'pvqa.cfg'
if not PVQA_CFG_PATH.exists():
print(f'Failed to find pvqa config. Exiting.')
sys.exit(1)
if not AQUA_PATH.exists():
print(f'Failed to find aqua-wb binary. Exiting.')
sys.exit(1)
if not SILER_PATH.exists():
print(f'Failed to find silence_eraser binary. Exiting.')
sys.exit(1)
if license_server is not None:
AQUA_LIC_PATH = '"license://' + license_server + '"'
PVQA_LIC_PATH = license_server
PVQA_CMD = PVQA_CMD_LIC_SERVER
else:
if not PVQA_LIC_PATH.exists():
PVQA_LIC_PATH = Path(utils.get_script_path()) / 'pvqa.lic'
if not PVQA_LIC_PATH.exists():
print(f'Failed to find pvqa license. Exiting.')
sys.exit(1)
if not AQUA_LIC_PATH.exists():
AQUA_LIC_PATH = Path(utils.get_script_path()) / 'aqua-wb.lic'
if not AQUA_LIC_PATH.exists():
print(f'Failed to find AQuA license. Exiting.')
sys.exit(1)
print(f'Found all analyzers.')
def speech_detector(test_path: str):
cmd = f'{SPEECH_DETECTOR_PATH} --input "{test_path}"'
utils.log_verbose(cmd)
retcode, output = subprocess.getstatusoutput(cmd)
if retcode != 0:
return retcode
utils.log_verbose(output)
r = json.loads(output)
utils.log_verbose(f'Parsed: {r}')
if 'error' in r:
return r['error']
if 'offset_start' in r and 'offset_end' in r:
return r
return None
# Erases silence on the begin & end of audio file
def silence_eraser(test_path: str, file_offset_begin: float = 0.0, file_offset_end: float = 0.0) -> int:
TEMP_FILE = 'silence_removed.wav'
if os.path.exists(TEMP_FILE):
os.remove(TEMP_FILE)
# Find total duration of audio
duration = utils.get_wav_length(test_path)
# Find correct end file offset
if file_offset_end is None:
cmd = f'{SILER_PATH} {test_path} {TEMP_FILE} --process-body off --starttime {file_offset_begin}'
else:
file_offset_end = duration - file_offset_end
cmd = f'{SILER_PATH} {test_path} {TEMP_FILE} --process-body off --starttime {file_offset_begin} --endtime {file_offset_end}'
utils.log(f'Silence eraser command: {cmd}')
retcode = os.system(cmd)
if retcode == 0 and os.path.exists(TEMP_FILE):
os.remove(test_path)
os.rename(TEMP_FILE, test_path)
utils.log(f'Prefix/suffix silence is removed on: {test_path}')
return 0
else:
return retcode
def find_pvqa_mos(test_path: str, file_offset_begin: float = 0.0, file_offset_end: float = 0.0):
cmd = PVQA_CMD.format(pvqa=PVQA_PATH, pvqa_lic=PVQA_LIC_PATH, pvqa_cfg=PVQA_CFG_PATH,
output=PVQA_OUTPUT, input=test_path, cut_begin=file_offset_begin, cut_end=file_offset_end)
utils.log_verbose(cmd)
# print(cmd)
exit_code, out_data = subprocess.getstatusoutput(cmd)
# Check if failed
if exit_code != 0:
utils.log_error(f'PVQA returned exit code {exit_code} and message {out_data}')
return 0.0, '', 0
# Verbose logging
utils.log_verbose(out_data)
# print(out_data)
p_mos = re.compile(r".*= ([\d\.]+)", re.MULTILINE)
m = p_mos.search(out_data)
if m:
with open(PVQA_OUTPUT, 'r') as report_file:
content = report_file.read()
# Find R-factor from content
count_intervals = 0
count_bad = 0
csv_parser = csv.reader(open(PVQA_OUTPUT, newline=''), delimiter=';')
for row in csv_parser:
# Check status
status = row[-1].strip()
# log_verbose("New CSV row is read. Last two items: %s and %s" % (status_0, status))
if status in ['Poor', 'Ok', 'Uncertain']:
count_intervals += 1
if status == 'Poor':
count_bad += 1
utils.log_verbose(f'Nr of intervals {count_intervals}, nr of bad intervals {count_bad}')
if count_intervals > 0:
r_factor = float(count_intervals - count_bad) / float(count_intervals)
else:
r_factor = 0.0
return round(float(m.group(1)), 3), content, int(r_factor * 100)
return 0.0, out_data, 0
# Runs AQuA utility on reference and test files. file_offset_begin / file_offset_end are offsets in seconds
def find_aqua_mos(good_path, test_path, test_file_offset_begin: float = 0.0, test_file_offset_end: float = 0.0,
good_file_offset_begin: float = 0.0, good_file_offset_end: float = 0.0):
try:
out_data = ""
cmd = AQUA_CMD.format(aqua=AQUA_PATH, aqua_lic=AQUA_LIC_PATH,
reference=good_path, input=test_path, spectrum=AQUA_SPECTRUM,
faults=AQUA_FAULTS,
cut_begin=int(test_file_offset_begin * 1000), cut_end=int(test_file_offset_end * 1000),
cut_begin_src=int(good_file_offset_begin * 1000), cut_end_src=int(good_file_offset_end * 1000))
utils.log_verbose(cmd)
# print(cmd)
exit_code, out_data = subprocess.getstatusoutput(cmd)
# Return
if exit_code != 0:
utils.log_error(f'AQuA returned error code {exit_code} with message {out_data}')
return 0.0, 0, '{}'
# Log for debugging purposes
utils.log_verbose(out_data)
with open(AQUA_FAULTS, 'r') as f:
report = f.read()
json_data = json.loads(report)
# print (out_data)
if 'AQuAReport' in json_data:
aqua_report = json_data['AQuAReport']
if 'QualityResults' in aqua_report:
qr = aqua_report['QualityResults']
return round(qr['MOS'], 3), round(qr['Percent'], 3), report
except Exception as err:
utils.log_error(message='Unexpected error.', err=err)
return 0.0, 0.0, out_data
return 0.0, 0.0, out_data

2
start.sh Executable file
View File

@ -0,0 +1,2 @@
#!/bin/bash
sudo systemctl start qualtest

1
stop.sh Executable file
View File

@ -0,0 +1 @@
sudo systemctl stop qualtest

3
stop_gsm_call.sh Executable file
View File

@ -0,0 +1,3 @@
#!/bin/bash
adb shell input keyevent 6

3
view_log.sh Executable file
View File

@ -0,0 +1,3 @@
#!/bin/bash
journalctl -u qualtest -f

3
view_mobile_app_info.sh Executable file
View File

@ -0,0 +1,3 @@
#!/bin/bash
adb shell dumpsys package biz.sevana.qualtestgsm