Bump version

* Rebuild app OTP authentication

* Adding OTP readme

* Fix error message // Remove duplicated scopes

* Add dynamic header building including latest app version

* Fix missing startupcheck if using token from file

* Change version

.github/workflows/build_image.yml

@@ -27,20 +27,20 @@ jobs:
 
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v3  
+        uses: actions/checkout@v4
       - name: Set up QEMU
-        uses: docker/setup-qemu-action@v2
+        uses: docker/setup-qemu-action@v3
       - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v2
+        uses: docker/setup-buildx-action@v3
       - name: Log in to the Container registry
-        uses: docker/login-action@65b78e6e13532edd9afa3aa52ac7964289d1a9c1
+        uses: docker/login-action@v3
         with:
           registry: ${{ env.REGISTRY }}
           username: ${{ github.actor }}
           password: ${{ secrets.GITHUB_TOKEN }}
       - name: Extract metadata (tags, labels) for Docker
         id: meta
-        uses: docker/metadata-action@9ec57ed1fcdbf14dcef7dfbe97b2010124a938b7
+        uses: docker/metadata-action@v5
         with:
           images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
           tags: |
@@ -52,7 +52,7 @@ jobs:
             type=semver,pattern={{major}}
             type=sha
       - name: Build and push
-        uses: docker/build-push-action@v2
+        uses: docker/build-push-action@v5
         with:
           context: .
           platforms: linux/amd64,linux/arm/v7,linux/arm/v6,linux/arm/v8

README.md

@@ -15,6 +15,8 @@ Maybe this component works also with other Volvo cars. Please try out the native
 
 Home Assistant thread can be found [here](https://community.home-assistant.io/t/volvo2mqtt-connect-your-aaos-volvo/585699).
 
+<b>Important note: The Volvo api currently ONLY works in these [countries](https://developer.volvocars.com/terms-and-conditions/apis-supported-locations/)</b>
+
 If you like my work:<br>
 [![ko-fi](https://ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/U7U8MFXCF)
 
@@ -26,13 +28,18 @@ If you like my work:<br>
 - V60 T8 PHEV (2023)
 - C40 BEV (2023)
 - C40 BEV (2022)
+- C40 BEV (2024)
 - XC90 T8 PHEV (2023)
 - XC60 PHEV (2024)
 - XC60 PHEV (2023)
 - XC60 PHEV (2022)
+- XC60 B5 Mildhybrid (2023)
 - XC90 PHEV T8 (2023)
 - XC90 PHEV T8 (2024)
+- XC90 B5 Mildhybrid (2024)
 - V90 PHEV T8 (2019)*
+- V90 PHEV T6 (2024)
+- V90 B5 Mildhybrid (2023)
 
 *only partly working
 
@@ -43,6 +50,7 @@ Please let me know if your car works with this addon so I can expand the list!<b
 - Lock/unlock car
 - Start/stop climate
 - Sensor "Battery Charge Level"
+- Sensor "Battery Capacity"
 - Sensor "Electric Range"
 - Sensor "Charging System Status"
 - Sensor "Charging Connection Status" (partly broken since 2.10 car update)
@@ -66,6 +74,50 @@ Please let me know if your car works with this addon so I can expand the list!<b
 
 NOTE: Energy status currently available only for cars in the Europe / Middle East / Africa regions. [source](https://developer.volvocars.com/apis/energy/v1/overview/#availability)
 
+## OTP Authentication
+
+As of version <b>v1.9.0</b>, this addon uses the same OTP authentication as the Volvo app. 
+The following steps are required for authentication in exactly this order:
+
+1. Setup volvo2Mqtt, either via Docker, or via HA addon (take a look at the "Setup" section below)
+2. Fill in your settings and start volvo2Mqtt
+3. Your log will show the following lines <br> 
+```Waiting for otp code... Please check your mailbox and post your otp code to the following mqtt topic "volvoAAOS2mqtt/otp_code". Retry 0/15```<br>
+```Waiting for otp code... Please check your mailbox and post your otp code to the following mqtt topic "volvoAAOS2mqtt/otp_code". Retry 1/15```<br>
+```etc ...```
+4. Now, open your mailbox and copy your OTP Code
+5. Open HomeAssistant and search for the entity ID ```text.volvo_otp```
+6. Paste your OTP into the text entity and **<ins>press Enter</ins>**<br>
+   6a. (optional) If your OTP cannot be safed, try to paste your OTP code via MQTT to the following topic: `volvoAAOS2mqtt/otp_code` <p>
+       <details>
+             <summary>OTP Post Script</summary>
+             <div>
+                <p>
+                1. Safe the following script in your script section <br>
+                2. Go to https://yourHAUrl/developer-tools/service<br>
+                3. Search for "Post Volvo OTP"<br>
+                4. Tick "OTP" and fill in your OTP<br>
+                5. Run this service<br>
+             </div>
+       
+            alias: Post Volvo OTP
+            sequence:
+              - service: mqtt.publish
+                metadata: {}
+                data:
+                  topic: volvoAAOS2mqtt/otp_code
+                  payload: "{{ otp }}"
+            mode: single
+            fields:
+              otp:
+                selector:
+                  text: null
+                name: OTP
+                description: Volvo OTP
+   
+   </details>
+8. If everything has worked, your addon is now authenticated. In the future, OTP authentication only needs to be done when updating, not when restarting the container.
+
 ## Setup
 <b>Docker:</b>
 
@@ -99,7 +151,6 @@ Here is what every option means:
 | `TZ`                      | `string`  |                                       | **required** | Container timezone eg "Europe/Berlin" from [here](https://docs.diladele.com/docker/timezones.html)|
 
 
-
 [aarch64-shield]: https://img.shields.io/badge/aarch64-yes-green.svg
 [amd64-shield]: https://img.shields.io/badge/amd64-yes-green.svg
 [armhf-shield]: https://img.shields.io/badge/armhf-yes-green.svg
@@ -107,6 +158,6 @@ Here is what every option means:
 [i386-shield]: https://img.shields.io/badge/i386-yes-green.svg
 [releases]: https://github.com/Dielee/volvo2mqtt/releases
 [releases-shield]: https://img.shields.io/github/release/Dielee/volvo2mqtt.svg
-[maintenance-shield]: https://img.shields.io/maintenance/yes/2023.svg
+[maintenance-shield]: https://img.shields.io/maintenance/yes/2024.svg
 [commits-shield]: https://img.shields.io/github/commit-activity/y/Dielee/volvo2mqtt.svg
 [commits]: https://github.com/Dielee/volvo2mqtt/commits/main

requirements.txt

@@ -1,5 +1,6 @@
-requests~=2.29.0
+requests~=2.32.0
 dynaconf~=3.1.12
 paho-mqtt~=1.6.1
 Babel~=2.12.1
-pytz~=2023.3
+pytz~=2023.3
+google_play_scraper~=1.2.6

src/CHANGELOG.md

@@ -1,3 +1,204 @@
+## v1.9.6
+### 🐛 Bug Fixes:
+
+- Fixed interior and exterior Images #219
+- Take care of broken token files #220
+
+## v1.9.5
+### 🚀 Features:
+
+- Added "retain" states to survive HA reboot
+
+## v1.9.4
+### 🐛 Bug Fixes:
+
+- Fix backend state API url
+
+## v1.9.3
+### 🐛 Bug Fixes:
+
+- Ignore retained OTP messages #193
+
+## v1.9.2
+### 🐛 Bug Fixes:
+
+- Try to fix OTP Input
+- Fix `SyntaxWarning: invalid escape sequence '\d'` #196 
+
+## v1.9.1
+### 🚀 Features:
+
+- The token file now survives an HA addon update/restart
+
+### 🐛 Bug Fixes:
+
+- Extend OTP retry time to 120 seconds
+
+## v1.9.0
+### ⚠️ Breaking change 
+### 🚀 Features:
+
+- Add "Battery Capacity" sensor. Thanks to @gurtjun!
+- OTP Auth
+        This addon uses the same OTP authentication as the Volvo app, now. 
+        The following steps are required for authentication in exactly this order:
+        
+        1. Setup volvo2Mqtt, either via Docker, or via HA addon (take a look at the "Setup" section)
+        2. Fill in your settings and start volvo2Mqtt
+        3. Your log will show the following lines
+        Waiting for otp code... Please check your mailbox and post your otp code to the following mqtt topic "volvoAAOS2mqtt/otp_code". Retry 0/15
+        Waiting for otp code... Please check your mailbox and post your otp code to the following mqtt topic "volvoAAOS2mqtt/otp_code". Retry 1/15
+        etc ...
+        4. Now, open your mailbox and copy your OTP Code
+        5. Open HomeAssistant and search for the entity ID text.volvo_otp
+        6. Paste your OTP into the text entity and press Enter
+        7. If everything has worked, your addon is now authenticated. In the future, OTP authentication only needs to be done when updating, not when restarting the container.
+
+
+## v1.8.27
+### 🚀 Features:
+
+- Add state caching for "diagnostics" endpoint
+
+## v1.8.26
+### 🚀 Features:
+
+- Add option to disable updates #160 (Set the update interval to -1)
+
+## v1.8.25
+### 🚀 Features:
+
+- Fix INTERNAL_SERVER_ERROR #165
+
+## v1.8.24
+### 🚀 Features:
+
+- Add option for dynamic interval settings #160
+- Add warnings endpoint #151
+
+## v1.8.23
+### 🚀 Features:
+
+- Add new charging system states #152
+
+## v1.8.22
+### 🚀 Features:
+
+- Change units to HA default #148
+
+## v1.8.21
+### 🐛 Bug Fixes:
+
+- Try to fix addon build #146
+
+## v1.8.20
+### 🐛 Bug Fixes:
+
+- Fix Addon crash if icon state is `UNKOWN`
+- Add `UNSPECIFIED` state for windows
+
+
+## v1.8.19
+### 🐛 Bug Fixes:
+
+- Fix Estimated Charging Time not changing to zero #133 
+
+## v1.8.18
+### 🐛 Bug Fixes:
+
+- Fix Backend State Sensor returning `null`
+
+## v1.8.17
+### 🚀 Features:
+
+- Move vehicle and vehicle details to v2 endpoint
+
+### 🐛 Bug Fixes:
+
+- Fix Battery Charge Level icon update
+
+## v1.8.16
+### 🚀 Features:
+
+- Add `Average energy consumption`, `Distance to empty battery` and `Washer fluid level` sensors
+- Support force update for `Backend state` Sensor
+- Use cached requests for fuel endpoint
+
+## v1.8.15
+### 🚀 Features:
+
+- Move `batteryChargeLevel` to the new v2 fuel endpoint
+- Add api backend status sensor
+
+## v1.8.14
+### 🐛 Bug Fixes:
+
+- Fix distance to empty sensor #127
+- Remove unused service warning trigger sensor
+
+## v1.8.13
+### 🐛 Bug Fixes:
+
+- Add "Time to Service" sensor #125<p>
+  <b>Breaking change: The "Months to Service" sensor will be automatically deleted.<br>
+  "Time to Service" contains this information now.
+
+## v1.8.12
+### 🐛 Bug Fixes:
+
+- Fix `AJAR` state for windows
+- Fix deprecation from `currentThread` to `current_thread`
+
+## v1.8.11
+### 🐛 Bug Fixes:
+
+- Remove old code from checks for running engine sensor
+- Move battery charge level sensor from v2 to v1, as v2 returns 404
+
+## v1.8.10
+### 🐛 Bug Fixes:
+
+- Add `AJAR` state for sunroof #123
+
+## v1.8.9
+### 🐛 Bug Fixes:
+
+- Fix api changes from volvo #121
+
+## v1.8.8
+### 🐛 Bug Fixes:
+
+- Fix json decode error, if volvo API returns a simple string #104
+
+## v1.8.7
+### 🚀 Features:
+
+- Add option to disable log completely #96 
+
+### 🐛 Bug Fixes:
+
+- Fix regex error for some mailaddresses #97
+
+## v1.8.6
+### 🚀 Features:
+
+- Add option to use multiple docker containers (with different logins) #93
+
+## v1.8.5
+### 🚀 Features:
+
+- Allow phone number as username #91
+
+## v1.8.4
+### 🚀 Features:
+
+- Optimize Addon configuration #90
+
+## v1.8.3
+### 🚀 Features:
+
+- Add `device_class: battery` for battery state sensors from BEV and PHEV 
+
 ## v1.8.2
 ### 🚀 Features:
 
@@ -14,7 +215,7 @@
 ### 🚀 Features:
 
 - <b>Breaking change: The `vccapikey` setting is now a list. It is possible to
-  add multiple VCCAPIKEYs. This will be helpful if you are extending your 10.000 call limit some times.
+  add multiple VCCAPIKEYs. This will be helpful if you are extending your 10.000 call limit sometimes.
   Change your vccapikey config like this:
   
   Old:

src/Dockerfile

@@ -11,7 +11,7 @@ WORKDIR /volvoAAOS2mqtt
 RUN wget --no-cache https://raw.githubusercontent.com/Dielee/volvo2mqtt/main/requirements.txt
 
 # install dependencies
-RUN pip install -r requirements.txt
+RUN pip install -r requirements.txt --break-system-packages
 
 # copy the content of the local src directory to the working directory
 COPY / .

src/apparmor.txt

@@ -4,8 +4,10 @@ profile volvo2mqtt flags=(attach_disconnected,mediate_deleted) {
   #include <abstractions/base>
 
   # Capabilities
+  capability,
   file,
   signal (send) set=(kill,term,int,hup,cont),
+  network,
 
   # S6-Overlay
   /init ix,

src/config.yaml

@@ -1,6 +1,6 @@
 name: "Volvo2Mqtt"
 description: "Volvo AAOS MQTT bridge"
-version: "1.8.2"
+version: "1.9.6"
 slug: "volvo2mqtt"
 init: false
 url: "https://github.com/Dielee/volvo2mqtt"
@@ -15,6 +15,7 @@ options:
   babelLocale: null
   TZ: null
   debug: false
+  disable_logging: false
   mqtt:
     broker: "auto_broker"
     port: "auto_port"
@@ -24,26 +25,28 @@ options:
     username: null
     password: null
     vin: ""
-    vccapikey: null
+    vccapikey:
+        - null
     odometerMultiplier: 1
     averageSpeedDivider: 1
     averageFuelConsumptionMultiplier: 1
 schema:
   updateInterval: int(60,)
   babelLocale: str
-  TZ: str
+  TZ: match(^.+/.+$)
   debug: bool
+  disable_logging: bool
   mqtt:
     broker: str
     port: str
     username: str?
     password: str?
   volvoData:
-    username: str
+    username: match(^([a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+)|(\+\d{5,20})$)
     password: str
     vin: str?
     vccapikey:
-        - str
+        - match(^\b\w{32}\b$)
     odometerMultiplier: int(1,)
     averageSpeedDivider: int(1,)
     averageFuelConsumptionMultiplier: int(1,)

src/const.py

@@ -1,10 +1,11 @@
 from config import settings
 
-VERSION = "v1.8.2"
+VERSION = "v1.9.6"
 
-OAUTH_URL = "https://volvoid.eu.volvocars.com/as/token.oauth2"
+OAUTH_TOKEN_URL = "https://volvoid.eu.volvocars.com/as/token.oauth2"
-VEHICLES_URL = "https://api.volvocars.com/connected-vehicle/v1/vehicles"
+OAUTH_AUTH_URL = "https://volvoid.eu.volvocars.com/as/authorization.oauth2"
-VEHICLE_DETAILS_URL = "https://api.volvocars.com/connected-vehicle/v1/vehicles/{0}"
+VEHICLES_URL = "https://api.volvocars.com/connected-vehicle/v2/vehicles"
+VEHICLE_DETAILS_URL = "https://api.volvocars.com/connected-vehicle/v2/vehicles/{0}"
 WINDOWS_STATE_URL = "https://api.volvocars.com/connected-vehicle/v2/vehicles/{0}/windows"
 CLIMATE_START_URL = "https://api.volvocars.com/connected-vehicle/v2/vehicles/{0}/commands/climatization-start"
 CLIMATE_STOP_URL = "https://api.volvocars.com/connected-vehicle/v2/vehicles/{0}/commands/climatization-stop"
@@ -16,42 +17,54 @@ ODOMETER_STATE_URL = "https://api.volvocars.com/connected-vehicle/v2/vehicles/{0
 LOCATION_STATE_URL = "https://api.volvocars.com/location/v1/vehicles/{0}/location"
 TYRE_STATE_URL = "https://api.volvocars.com/connected-vehicle/v2/vehicles/{0}/tyres"
 ENGINE_STATE_URL = "https://api.volvocars.com/connected-vehicle/v2/vehicles/{0}/engine-status"
-BATTERY_CHARGE_STATE_URL = "https://api.volvocars.com/connected-vehicle/v2/vehicles/{0}/battery-charge-level"
+FUEL_BATTERY_STATE_URL = "https://api.volvocars.com/connected-vehicle/v2/vehicles/{0}/fuel"
-FUEL_STATE_URL = "https://api.volvocars.com/connected-vehicle/v2/vehicles/{0}/fuel"
 STATISTICS_URL = "https://api.volvocars.com/connected-vehicle/v2/vehicles/{0}/statistics"
 ENGINE_DIAGNOSTICS_URL = "https://api.volvocars.com/connected-vehicle/v2/vehicles/{0}/engine"
 VEHICLE_DIAGNOSTICS_URL = "https://api.volvocars.com/connected-vehicle/v2/vehicles/{0}/diagnostics"
+WARNINGS_URL = "https://api.volvocars.com/connected-vehicle/v2/vehicles/{0}/warnings"
+API_BACKEND_STATUS = "https://public-developer-portal-bff.weu-prod.ecpaz.volvocars.biz/api/v1/backend-status"
+
+LENGTH_KILOMETERS = "km"
+SPEED_KILOMETERS_PER_HOUR = "km/h"
+TIME_MINUTES = "min"
+LENGTH_MILES = "mi"
+SPEED_MILES_PER_HOUR = "mph"
+VOLUME_LITERS = "L"
+ENERGY_KILO_WATT_HOUR = "kWh"
+TIME_HOURS = "h"
+TIME_MONTHS = "m"
 
 units = {
             "en_GB": {
                 "divider": 1.60934,
-                "electric_range": {"unit": "mi"},
+                "electric_range": {"unit": LENGTH_MILES},
-                "odometer": {"unit": "mi"},
+                "odometer": {"unit": LENGTH_MILES},
-                "average_speed": {"unit": "mph"},
+                "average_speed": {"unit": SPEED_MILES_PER_HOUR},
-                "distance_to_empty": {"unit": "mi"},
+                "distance_to_empty": {"unit": LENGTH_MILES},
-                "distance_to_service": {"unit": "mi"}
+                "distance_to_service": {"unit": LENGTH_MILES}
             },
             "en_US": {
                 "divider": 1.60934,
-                "electric_range": {"unit": "mi"},
+                "electric_range": {"unit": LENGTH_MILES},
-                "odometer": {"unit": "mi"},
+                "odometer": {"unit": LENGTH_MILES},
-                "average_speed": {"unit": "mph"},
+                "average_speed": {"unit": SPEED_MILES_PER_HOUR},
-                "distance_to_empty": {"unit": "mi"},
+                "distance_to_empty": {"unit": LENGTH_MILES},
-                "distance_to_service": {"unit": "mi"}
+                "distance_to_service": {"unit": LENGTH_MILES}
             }
         }
 
 availability_topic = "volvoAAOS2mqtt/availability"
 
 charging_system_states = {"CHARGING_SYSTEM_CHARGING": "Charging", "CHARGING_SYSTEM_IDLE": "Idle",
-                          "CHARGING_SYSTEM_FAULT": "Fault", "CHARGING_SYSTEM_UNSPECIFIED": "Unspecified"}
+                          "CHARGING_SYSTEM_FAULT": "Fault", "CHARGING_SYSTEM_UNSPECIFIED": "Unspecified",
+                          "CHARGING_SYSTEM_DONE": "Done", "CHARGING_SYSTEM_SCHEDULED": "Scheduled"}
 
 charging_connection_states = {"CONNECTION_STATUS_DISCONNECTED": "Disconnected", "CONNECTION_STATUS_UNSPECIFIED": "Unspecified",
                               "CONNECTION_STATUS_CONNECTED_DC": "Connected DC", "CONNECTION_STATUS_CONNECTED_AC": "Connected AC",
                               "CONNECTION_STATUS_FAULT": "Fault"}
 
-window_states = {"CLOSED": "OFF", "OPEN": "ON"}
+window_states = {"CLOSED": "OFF", "OPEN": "ON", "UNSPECIFIED": "UNKNOWN", "AJAR": "ON"}
-door_states = {"CLOSED": "OFF", "OPEN": "ON"}
+door_states = {"CLOSED": "OFF", "OPEN": "ON", "UNSPECIFIED": "UNKNOWN", "AJAR": "ON"}
 engine_states = {"RUNNING": "ON", "STOPPED": "OFF", "true": "ON", "false": "OFF"}
 
 icon_states = {
@@ -77,14 +90,15 @@ icon_states = {
 }
 
 supported_entities = [
-                        {"name": "Battery Charge Level", "domain": "sensor", "id": "battery_charge_level", "unit": "%", "icon": "car-battery", "url": RECHARGE_STATE_URL, "state_class": "measurement"},
+                        {"name": "Battery Charge Level", "domain": "sensor", "device_class": "battery", "id": "battery_charge_level", "unit": "%", "icon": "car-battery", "url": RECHARGE_STATE_URL, "state_class": "measurement"},
-                        {"name": "Battery Charge Level", "domain": "sensor", "id": "battery_charge_level", "unit": "%", "icon": "car-battery", "url": BATTERY_CHARGE_STATE_URL, "state_class": "measurement"},
+                        {"name": "Battery Charge Level", "domain": "sensor", "device_class": "battery", "id": "battery_charge_level", "unit": "%", "icon": "car-battery", "url": FUEL_BATTERY_STATE_URL, "state_class": "measurement"},
-                        {"name": "Electric Range", "domain": "sensor", "id": "electric_range", "unit": "km" if not units.get(settings["babelLocale"]) else units[settings["babelLocale"]]["electric_range"]["unit"], "icon": "map-marker-distance", "url": RECHARGE_STATE_URL, "state_class": "measurement"},
+                        {"name": "Battery Capacity", "domain": "sensor", "device_class": "energy_storage", "id": "battery_capacity", "unit": ENERGY_KILO_WATT_HOUR, "icon": "car-battery", "url": VEHICLE_DETAILS_URL, "state_class": "measurement"},
-                        {"name": "Estimated Charging Time", "domain": "sensor", "id": "estimated_charging_time", "unit": "minutes", "icon": "timer-sync-outline", "url": RECHARGE_STATE_URL, "state_class": "measurement"},
+                        {"name": "Electric Range", "domain": "sensor", "id": "electric_range", "unit": LENGTH_KILOMETERS if not units.get(settings["babelLocale"]) else units[settings["babelLocale"]]["electric_range"]["unit"], "icon": "map-marker-distance", "url": RECHARGE_STATE_URL, "state_class": "measurement"},
+                        {"name": "Estimated Charging Time", "domain": "sensor", "id": "estimated_charging_time", "unit": TIME_MINUTES, "icon": "timer-sync-outline", "url": RECHARGE_STATE_URL, "state_class": "measurement"},
                         {"name": "Charging System Status", "domain": "sensor", "id": "charging_system_status", "icon": "ev-station", "url": RECHARGE_STATE_URL},
                         {"name": "Charging Connection Status", "domain": "sensor", "id": "charging_connection_status", "icon": "ev-plug-ccs2", "url": RECHARGE_STATE_URL},
                         {"name": "Estimated Charging Finish Time", "domain": "sensor", "id": "estimated_charging_finish_time", "icon": "timer-sync-outline", "url": RECHARGE_STATE_URL},
-                        {"name": "Odometer", "domain": "sensor", "id": "odometer", "unit": "km" if not units.get(settings["babelLocale"]) else units[settings["babelLocale"]]["odometer"]["unit"], "icon": "counter", "url": ODOMETER_STATE_URL, "state_class":"total_increasing"},
+                        {"name": "Odometer", "domain": "sensor", "id": "odometer", "unit": LENGTH_KILOMETERS if not units.get(settings["babelLocale"]) else units[settings["babelLocale"]]["odometer"]["unit"], "icon": "counter", "url": ODOMETER_STATE_URL, "state_class":"total_increasing"},
                         {"name": "Last Data Update", "domain": "sensor", "id": "last_data_update", "icon": "timer", "url": ""},
                         {"name": "Active schedules", "domain": "sensor", "id": "active_schedules", "icon": "timer", "url": ""},
                         {"name": "Window Front Left", "domain": "binary_sensor", "device_class": "window", "id": "window_front_left", "icon": "car-door-lock", "url": WINDOWS_STATE_URL},
@@ -108,14 +122,22 @@ supported_entities = [
                         {"name": "Tire Rear Left", "domain": "sensor", "id": "tyre_rear_left", "icon": "car-tire-alert", "url": TYRE_STATE_URL},
                         {"name": "Tire Rear Right", "domain": "sensor", "id": "tyre_rear_right", "icon": "car-tire-alert", "url": TYRE_STATE_URL},
                         {"name": "Engine State", "domain": "binary_sensor", "device_class": "running", "id": "engine_state", "icon": "engine", "url": ENGINE_STATE_URL},
-                        {"name": "Engine State", "domain": "binary_sensor", "device_class": "running", "id": "engine_state", "icon": "engine", "url": ENGINE_DIAGNOSTICS_URL},
+                        {"name": "Fuel Level", "domain": "sensor", "id": "fuel_level", "unit": VOLUME_LITERS, "icon": "fuel", "url": FUEL_BATTERY_STATE_URL, "state_class": "measurement"},
-                        {"name": "Fuel Level", "domain": "sensor", "id": "fuel_level", "unit": "liters", "icon": "fuel", "url": FUEL_STATE_URL, "state_class": "measurement"},
+                        {"name": "Average Fuel Consumption", "domain": "sensor", "id": "average_fuel_consumption", "unit": VOLUME_LITERS, "icon": "fuel", "url": STATISTICS_URL},
-                        {"name": "Average Fuel Consumption", "domain": "sensor", "id": "average_fuel_consumption", "unit": "liters", "icon": "fuel", "url": STATISTICS_URL},
+                        {"name": "Average Energy Consumption", "domain": "sensor", "id": "average_energy_consumption", "unit": ENERGY_KILO_WATT_HOUR, "icon": "car-electric", "url": STATISTICS_URL},
-                        {"name": "Distance to Empty", "domain": "sensor", "id": "distance_to_empty", "unit": "km" if not units.get(settings["babelLocale"]) else units[settings["babelLocale"]]["distance_to_empty"]["unit"], "icon": "map-marker-distance", "url": STATISTICS_URL, "state_class": "measurement"},
+                        {"name": "Distance to Empty Tank", "domain": "sensor", "id": "distance_to_empty_tank", "unit": LENGTH_KILOMETERS if not units.get(settings["babelLocale"]) else units[settings["babelLocale"]]["distance_to_empty"]["unit"], "icon": "map-marker-distance", "url": STATISTICS_URL, "state_class": "measurement"},
-                        {"name": "Average Speed", "domain": "sensor", "id": "average_speed", "unit": "km/h" if not units.get(settings["babelLocale"]) else units[settings["babelLocale"]]["average_speed"]["unit"], "icon": "speedometer", "url": STATISTICS_URL, "state_class": "measurement"},
+                        {"name": "Distance to Empty Battery", "domain": "sensor", "id": "distance_to_empty_battery", "unit": LENGTH_KILOMETERS if not units.get(settings["babelLocale"]) else units[settings["babelLocale"]]["distance_to_empty"]["unit"], "icon": "map-marker-distance", "url": STATISTICS_URL, "state_class": "measurement"},
-                        {"name": "Hours to Service", "domain": "sensor", "id": "hours_to_service", "unit": "hour", "icon": "wrench-clock", "url": VEHICLE_DIAGNOSTICS_URL},
+                        {"name": "Average Speed", "domain": "sensor", "id": "average_speed", "unit": SPEED_KILOMETERS_PER_HOUR if not units.get(settings["babelLocale"]) else units[settings["babelLocale"]]["average_speed"]["unit"], "icon": "speedometer", "url": STATISTICS_URL, "state_class": "measurement"},
-                        {"name": "Distance to Service", "domain": "sensor", "id": "km_to_service", "unit": "km" if not units.get(settings["babelLocale"]) else units[settings["babelLocale"]]["distance_to_service"]["unit"], "icon": "wrench-clock", "url": VEHICLE_DIAGNOSTICS_URL},
+                        {"name": "Hours to Service", "domain": "sensor", "id": "hours_to_service", "unit": TIME_HOURS, "icon": "wrench-clock", "url": VEHICLE_DIAGNOSTICS_URL},
-                        {"name": "Months to Service", "domain": "sensor", "id": "months_to_service", "unit": "month", "icon": "wrench-clock", "url": VEHICLE_DIAGNOSTICS_URL},
+                        {"name": "Distance to Service", "domain": "sensor", "id": "km_to_service", "unit": LENGTH_KILOMETERS if not units.get(settings["babelLocale"]) else units[settings["babelLocale"]]["distance_to_service"]["unit"], "icon": "wrench-clock", "url": VEHICLE_DIAGNOSTICS_URL},
+                        {"name": "Time to Service", "domain": "sensor", "id": "time_to_service", "icon": "wrench-clock", "url": VEHICLE_DIAGNOSTICS_URL},
                         {"name": "Service warning status", "domain": "sensor", "id": "service_warning_status", "icon": "alert-outline", "url": VEHICLE_DIAGNOSTICS_URL},
-                        {"name": "Service warning trigger", "domain": "sensor", "id": "service_warning_trigger", "icon": "alert-outline", "url": VEHICLE_DIAGNOSTICS_URL}
+                        {"name": "Washer Fluid Level warning", "domain": "sensor", "id": "washer_fluid_warning", "icon": "alert-outline", "url": VEHICLE_DIAGNOSTICS_URL},
+                        {"name": "API Backend status", "domain": "sensor", "id": "api_backend_status", "icon": "alert"},
+                        {"name": "Update Interval", "domain": "number", "id": "update_interval", "unit": "seconds", "icon": "timer", "min": -1, "max": 600, "mode": "box"},
+                        {"name": "Warnings", "domain": "sensor", "id": "warnings", "icon": "alert", "url": WARNINGS_URL}
 ]
+
+old_entity_ids = ["months_to_service", "service_warning_trigger", "distance_to_empty"]
+otp_max_loops = 24
+otp_mqtt_topic = "volvoAAOS2mqtt/otp_code"

src/mqtt.py

@@ -4,13 +4,14 @@ import paho.mqtt.client as mqtt
 import json
 import volvo
 import util
+import os
+import requests
 from threading import Thread, Timer
 from datetime import datetime
 from babel.dates import format_datetime
 from config import settings
 from const import CLIMATE_START_URL, CLIMATE_STOP_URL, CAR_LOCK_URL, \
-            CAR_UNLOCK_URL, availability_topic, icon_states
+    CAR_UNLOCK_URL, availability_topic, icon_states, old_entity_ids, otp_mqtt_topic
-
 
 mqtt_client: mqtt.Client
 subscribed_topics = []
@@ -20,10 +21,12 @@ climate_timer = {}
 engine_status = {}
 devices = {}
 active_schedules = {}
-
+otp_code = None
 
 def connect():
-    client = mqtt.Client("volvoAAOS2mqtt")
+    client = mqtt.Client("volvoAAOS2mqtt") if os.environ.get("IS_HA_ADDON") \
+        else mqtt.Client("volvoAAOS2mqtt_" + settings.volvoData["username"].replace("+", ""))
+
     client.will_set(availability_topic, "offline", 0, False)
     if settings["mqtt"]["username"] and settings["mqtt"]["password"]:
         client.username_pw_set(settings["mqtt"]["username"], settings["mqtt"]["password"])
@@ -35,6 +38,7 @@ def connect():
                 port = settings["mqtt"]["port"]
     client.connect(settings["mqtt"]["broker"], port)
     client.loop_start()
+    client.subscribe("volvoAAOS2mqtt/otp_code")
     client.on_message = on_message
     client.on_disconnect = on_disconnect
     client.on_connect = on_connect
@@ -43,17 +47,52 @@ def connect():
     mqtt_client = client
 
 
+def create_otp_input():
+    state_topic = otp_mqtt_topic + "/state"
+    config = {
+        "name": "Volvo OTP",
+        "object_id": f"volvo_otp",
+        "schema": "state",
+        "command_topic": otp_mqtt_topic,
+        "state_topic": state_topic,
+        "unique_id": "volvoAAOS2mqtt_otp",
+        "pattern": r"\d{6}",
+        "icon": "mdi:two-factor-authentication",
+        "mode": "text"
+    }
+
+    mqtt_client.publish(
+        "homeassistant/text/volvoAAOS2mqtt/volvo_otp/config",
+        json.dumps(config),
+        retain=True
+    )
+
+    mqtt_client.publish(
+        state_topic,
+        "000000",
+        retain=True
+    )
+
+def set_otp_state():
+    mqtt_client.publish(
+        otp_mqtt_topic + "/state",
+        otp_code,
+        retain=True
+    )
+
+
 def send_car_images(vin, data, device):
     if util.keys_exists(data, "images"):
         for entity in [{"name": "Exterior Image", "id": "exterior_image"},
                        {"name": "Interior Image", "id": "interior_image"}]:
-            url_topic = f"homeassistant/image/{vin}_{entity['id']}/image_url"
+            image_topic = f"homeassistant/image/{vin}_{entity['id']}/image_topic"
             config = {
                 "name": entity["name"],
                 "object_id": f"volvo_{vin}_{entity['id']}",
                 "schema": "state",
                 "icon": "mdi:image-area",
-                "url_topic": url_topic,
+                "content_type": "image/png",
+                "image_topic": image_topic,
                 "device": device,
                 "unique_id": f"volvoAAOS2mqtt_{vin}_{entity['id']}",
                 "availability_topic": availability_topic
@@ -65,19 +104,41 @@ def send_car_images(vin, data, device):
                 retain=True
             )
 
+            headers = {"User-Agent": "(Windows NT 10.0; Win64; x64)",
+                       "sec-ch-ua-platform": "\"Windows\"", "sec-fetch-dest": "document", "Accept-Encoding": "gzip, deflate, br, zstd",
+                       "sec-fetch-mode": "navigate", "sec-fetch-site": "none", "sec-fetch-user": "?1", "upgrade-insecure-requests": "1"}
+
+            # Post exterior image
             if entity["id"] == "exterior_image":
-                mqtt_client.publish(
+                if not os.path.exists("exterior_image.png"):
-                    url_topic,
+                    response = requests.get(data["images"]["exteriorImageUrl"], headers=headers)
-                    data["images"]["exteriorDefaultUrl"],
+                    if response.status_code == 200:
-                    retain=True
+                        with open("exterior_image.png", 'wb') as image:
-                )
+                            image.write(response.content)
+
+                if os.path.exists("exterior_image.png"):
+                    ext_image = open("exterior_image.png", mode="rb").read()
+                    mqtt_client.publish(
+                        image_topic,
+                        ext_image,
+                        retain=True
+                    )
 
             if entity["id"] == "interior_image":
-                mqtt_client.publish(
+                # Post interior Image
-                    url_topic,
+                if not os.path.exists("interior_image.png"):
-                    data["images"]["interiorDefaultUrl"],
+                    response = requests.get(data["images"]["internalImageUrl"], headers=headers)
-                    retain=True
+                    if response.status_code == 200:
-                )
+                        with open("interior_image.png", 'wb') as image:
+                            image.write(response.content)
+
+                if os.path.exists("interior_image.png"):
+                    int_image = open("interior_image.png", mode="rb").read()
+                    mqtt_client.publish(
+                        image_topic,
+                        int_image,
+                        retain=True
+                    )
 
 
 def on_connect(client, userdata, flags, rc):
@@ -87,18 +148,27 @@ def on_connect(client, userdata, flags, rc):
             mqtt_client.subscribe(topic)
 
 
-def on_disconnect(client, userdata,  rc):
+def on_disconnect(client, userdata, rc):
     logging.warning("MQTT disconnected, reconnecting automatically")
 
 
 def on_message(client, userdata, msg):
-    try:
-        vin = msg.topic.split('/')[2].split('_')[0]
-    except IndexError:
-        logging.error("Error - Cannot get vin from MQTT topic!")
-        return None
-
     payload = msg.payload.decode("UTF-8")
+    if msg.topic == otp_mqtt_topic:
+        if msg.retain == 0:
+            global otp_code
+            otp_code = payload
+            set_otp_state()
+        else:
+            logging.warning("Found retained OTP, this can't work! Please clean retained messages!")
+        return None
+    else:
+        try:
+            vin = msg.topic.split('/')[2].split('_')[0]
+        except IndexError:
+            logging.error("Error - Cannot get vin from MQTT topic!")
+            return None
+
     if "climate_status" in msg.topic:
         if payload == "ON":
             start_climate(vin)
@@ -112,6 +182,13 @@ def on_message(client, userdata, msg):
     elif "update_data" in msg.topic:
         if payload == "PRESS":
             update_car_data(True)
+    elif "update_interval" in msg.topic:
+        update_interval = int(payload)
+        if update_interval >= 60 or update_interval == -1:
+            settings.update({"updateInterval": int(update_interval)})
+        else:
+            logging.warning("Interval " + str(update_interval) + " seconds is to low. Doing nothing!")
+        update_car_data()
     elif "schedule" in msg.topic:
         try:
             d = json.loads(payload)
@@ -138,7 +215,7 @@ def start_climate_timer(d, vin):
         return None
 
     if timer_seconds > 0:
-        Timer(timer_seconds, activate_climate_timer, (vin, start_datetime.isoformat(), )).start()
+        Timer(timer_seconds, activate_climate_timer, (vin, start_datetime.isoformat(),)).start()
         active_schedules[vin]["timers"].append(start_datetime.isoformat())
         logging.debug("Climate timer set to " + str(start_datetime))
         update_car_data()
@@ -212,11 +289,15 @@ def start_climate(vin):
 def update_loop():
     create_ha_devices()
     while True:
-        logging.info("Sending mqtt update...")
+        if settings["updateInterval"] > 0:
-        send_heartbeat()
+            logging.info("Sending mqtt update...")
-        update_car_data()
+            send_heartbeat()
-        logging.info("Mqtt update done. Next run in " + str(settings["updateInterval"]) + " seconds.")
+            update_car_data()
-        time.sleep(settings["updateInterval"])
+            logging.info("Mqtt update done. Next run in " + str(settings["updateInterval"]) + " seconds.")
+            time.sleep(settings["updateInterval"])
+        else:
+            logging.info("Data update is disabled, doing nothing for 30 seconds")
+            time.sleep(30)
 
 
 def update_car_data(force_update=False, overwrite={}):
@@ -224,7 +305,7 @@ def update_car_data(force_update=False, overwrite={}):
     last_data_update = format_datetime(datetime.now(util.TZ), format="medium", locale=settings["babelLocale"])
     for vin in volvo.vins:
         for entity in volvo.supported_endpoints[vin]:
-            if entity["domain"] == "button":
+            if entity["domain"] in ["button"]:
                 continue
 
             ov_entity_id = ""
@@ -241,6 +322,13 @@ def update_car_data(force_update=False, overwrite={}):
                 state = last_data_update
             elif entity["id"] == "active_schedules":
                 state = active_schedules[vin]
+            elif entity["id"] == "update_interval":
+                state = settings["updateInterval"]
+            elif entity["id"] == "api_backend_status":
+                if force_update:
+                    state = volvo.get_backend_status()
+                else:
+                    state = volvo.backend_status
             else:
                 if entity["id"] == ov_entity_id and vin == ov_vin:
                     state = ov_state
@@ -249,25 +337,45 @@ def update_car_data(force_update=False, overwrite={}):
 
             if entity["domain"] == "device_tracker" or entity["id"] == "active_schedules":
                 topic = f"homeassistant/{entity['domain']}/{vin}_{entity['id']}/attributes"
+            elif entity["id"] == "warnings":
+                mqtt_client.publish(
+                    f"homeassistant/{entity['domain']}/{vin}_{entity['id']}/attributes",
+                    json.dumps(state),
+                    retain=True
+                )
+                if state:
+                    state = sum(value == "FAILURE" for value in state.values())
+                else:
+                    state = 0
+
+                topic = f"homeassistant/{entity['domain']}/{vin}_{entity['id']}/state"
             else:
                 topic = f"homeassistant/{entity['domain']}/{vin}_{entity['id']}/state"
 
-            mqtt_client.publish(
+            if state or state == 0:
-                topic,
+                mqtt_client.publish(
-                json.dumps(state) if isinstance(state, dict) or isinstance(state, list) else state
+                    topic,
-            )
+                    json.dumps(state) if isinstance(state, dict) or isinstance(state, list) else state,
-            update_ha_device(entity, vin, state)
+                    retain=True
+                )
+                update_ha_device(entity, vin, state)
 
 
 def update_ha_device(entity, vin, state):
     icon_config = icon_states.get(entity["id"])
-    if icon_config and state:
+    try:
-        if state.replace(".", "").isnumeric():
+        if icon_config and state:
-            state = float(state)
+            logging.debug("Entity " + entity["id"] + " state " + str(state))
-            icon = util.get_icon_between(icon_config, state)
+            if isinstance(state, float):
+                icon = util.get_icon_between(icon_config, state)
+            elif state.replace(".", "").isnumeric():
+                state = float(state)
+                icon = util.get_icon_between(icon_config, state)
+            else:
+                icon = icon_config[state]
         else:
-            icon = icon_config[state]
+            return None
-    else:
+    except:
         return None
 
     logging.debug("Updating icon to " + icon + " for " + entity["id"])
@@ -289,7 +397,7 @@ def update_ha_device(entity, vin, state):
 
     if entity.get("state_class"):
         config["state_class"] = entity["state_class"]
-        
+
     if entity.get("domain") == "device_tracker":
         config["json_attributes_topic"] = f"homeassistant/{entity['domain']}/{vin}_{entity['id']}/attributes"
 
@@ -313,15 +421,16 @@ def create_ha_devices():
         devices[vin] = device
         for entity in volvo.supported_endpoints[vin]:
             config = {
-                        "name": entity['name'],
+                "name": entity['name'],
-                        "object_id": f"volvo_{vin}_{entity['id']}",
+                "object_id": f"volvo_{vin}_{entity['id']}",
-                        "schema": "state",
+                "schema": "state",
-                        "icon": f"mdi:{entity['icon']}",
+                "icon": f"mdi:{entity['icon']}",
-                        "state_topic": f"homeassistant/{entity['domain']}/{vin}_{entity['id']}/state",
+                "state_topic": f"homeassistant/{entity['domain']}/{vin}_{entity['id']}/state",
-                        "device": device,
+                "device": device,
-                        "unique_id": f"volvoAAOS2mqtt_{vin}_{entity['id']}",
+                "unique_id": f"volvoAAOS2mqtt_{vin}_{entity['id']}",
-                        "availability_topic": availability_topic
+                "availability_topic": availability_topic
-                    }
+            }
+
             if entity.get("device_class"):
                 config["device_class"] = entity["device_class"]
 
@@ -331,13 +440,20 @@ def create_ha_devices():
             if entity.get("state_class"):
                 config["state_class"] = entity["state_class"]
 
-            if entity.get("domain") == "device_tracker" or entity.get("id") == "active_schedules":
+            if (entity.get("domain") == "device_tracker" or entity.get("id") == "active_schedules"
+                    or entity.get("id") == "warnings"):
                 config["json_attributes_topic"] = f"homeassistant/{entity['domain']}/{vin}_{entity['id']}/attributes"
-            elif entity.get("domain") in ["switch", "lock", "button"]:
+
+            if entity.get("domain") in ["switch", "lock", "button", "number"]:
                 command_topic = f"homeassistant/{entity['domain']}/{vin}_{entity['id']}/command"
                 config["command_topic"] = command_topic
                 subscribed_topics.append(command_topic)
                 mqtt_client.subscribe(command_topic)
+
+                if entity["domain"] == "number":
+                    config["min"] = entity["min"]
+                    config["max"] = entity["max"]
+                    config["mode"] = entity["mode"]
             elif entity.get("domain") == "image":
                 config["url_topic"] = f"homeassistant/{entity['domain']}/{vin}_{entity['id']}/image_url"
 
@@ -351,9 +467,15 @@ def create_ha_devices():
 
 
 def send_heartbeat():
-    mqtt_client.publish(availability_topic, "online")
+    mqtt_client.publish(availability_topic, "online", retain=True)
 
 
 def send_offline():
-    mqtt_client.publish(availability_topic, "offline")
+    mqtt_client.publish(availability_topic, "offline", retain=True)
 
+
+def delete_old_entities():
+    for vin in volvo.vins:
+        for entity in old_entity_ids:
+            topic = f"homeassistant/sensor/volvoAAOS2mqtt/{vin}_{entity}/config"
+            mqtt_client.publish(topic, payload="", retain=True)

src/settings.json

@@ -2,6 +2,8 @@
   "updateInterval": 300,
   "babelLocale": "de",
   "TZ": "Europe/Berlin",
+  "debug": false,
+  "disable_logging": false,
   "mqtt": {
     "broker": "",
     "port": 1883,

src/translations/en.yaml

@@ -11,6 +11,9 @@ configuration:
   debug:
     name: API debug mode
     description: Enable Volvo API debug, normaly this can stay off. If enabled, the complete log file can be found under \\<Your HA Host IP>\addons\volvo2mqtt\log\volvo2mqtt.log.
+  disable_logging:
+    name: Disable logging
+    description: Disable logging completely to reduce IO and SD card access.
   mqtt:
     name: MQTT Broker settings
     description: Leave the settings as they are if you are using the MQTT Mosquitto Addon. If not, take a look at the readme from volvo2mqtt.

src/util.py

@@ -4,6 +4,8 @@ import os
 import re
 import sys
 import config
+import json
+from google_play_scraper import app
 from logging import handlers
 from datetime import datetime
 from const import units
@@ -28,6 +30,29 @@ class SensitiveDataFilter(logging.Filter):
             record.msg = re.sub(pattern, "<REDACTED>", record.msg)
         return True
 
+def get_volvo_app_version():
+    result = app(
+        'se.volvo.vcc',
+        lang='en',
+        country='us'
+    )
+
+    if "version" in result:
+        return result["version"]
+    else:
+        return "5.37.0"
+
+def get_token_path():
+    token_path = ".token"
+    if os.environ.get("IS_HA_ADDON"):
+        check_existing_folder("/addons/volvo2mqtt/token/")
+        token_path = "/addons/volvo2mqtt/token/.token"
+
+    return token_path
+
+def save_to_json(data, token_path):
+    with open(token_path, 'w', encoding='utf-8') as f:
+        json.dump(data, f, ensure_ascii=False, indent=4)
 
 def get_icon_between(icon_list, state):
     icon = None
@@ -40,7 +65,7 @@ def get_icon_between(icon_list, state):
 def setup_logging():
     log_location = "volvo2mqtt.log"
     if os.environ.get("IS_HA_ADDON"):
-        check_existing_folder()
+        check_existing_folder("/addons/volvo2mqtt/log/")
         log_location = "/addons/volvo2mqtt/log/volvo2mqtt.log"
 
     logging.Formatter.converter = lambda *args: datetime.now(tz=TZ).timetuple()
@@ -68,9 +93,13 @@ def setup_logging():
         if settings["debug"]:
             logger.setLevel(logging.DEBUG)
 
+    if "disable_logging" in settings:
+        if settings["disable_logging"]:
+            logger.setLevel(logging.ERROR)
 
-def check_existing_folder():
+
-    Path("/addons/volvo2mqtt/log/").mkdir(parents=True, exist_ok=True)
+def check_existing_folder(path):
+    Path(path).mkdir(parents=True, exist_ok=True)
 
 
 def keys_exists(element, *keys):

src/volvo.py

@@ -1,17 +1,21 @@
+import json
 import logging
 import requests
 import mqtt
 import util
 import time
 import re
-from threading import currentThread
+import os.path
+from threading import current_thread, Thread
 from datetime import datetime, timedelta
 from config import settings
 from babel.dates import format_datetime
+from json import JSONDecodeError
 from const import charging_system_states, charging_connection_states, door_states, window_states, \
-    OAUTH_URL, VEHICLES_URL, VEHICLE_DETAILS_URL, RECHARGE_STATE_URL, CLIMATE_START_URL, \
+    OAUTH_AUTH_URL, OAUTH_TOKEN_URL, VEHICLES_URL, VEHICLE_DETAILS_URL, RECHARGE_STATE_URL, CLIMATE_START_URL, \
-    WINDOWS_STATE_URL, LOCK_STATE_URL, TYRE_STATE_URL, supported_entities, BATTERY_CHARGE_STATE_URL, \
+    WINDOWS_STATE_URL, LOCK_STATE_URL, TYRE_STATE_URL, supported_entities, FUEL_BATTERY_STATE_URL, \
-    STATISTICS_URL, ENGINE_DIAGNOSTICS_URL, engine_states
+    STATISTICS_URL, ENGINE_DIAGNOSTICS_URL, VEHICLE_DIAGNOSTICS_URL, API_BACKEND_STATUS, WARNINGS_URL, engine_states, \
+    otp_max_loops
 
 session = requests.Session()
 session.headers = {
@@ -26,38 +30,144 @@ vins = []
 supported_endpoints = {}
 cached_requests = {}
 vcc_api_keys = []
+backend_status = ""
 
 
-def authorize():
+def authorize(renew_tokenfile=False):
-    headers = {
+    global refresh_token
-        "authorization": "Basic aDRZZjBiOlU4WWtTYlZsNnh3c2c1WVFxWmZyZ1ZtSWFEcGhPc3kxUENhVXNpY1F0bzNUUjVrd2FKc2U0QVpkZ2ZJZmNMeXc=",
+
-        "content-type": "application/x-www-form-urlencoded",
+    token_path = util.get_token_path()
-        "accept": "application/json"
+    if os.path.isfile(token_path) and not renew_tokenfile:
-    }
+        logging.info("Using login from token file")
+        f = open(token_path)
+        try:
+            data = json.load(f)
+            refresh_token = data["refresh_token"]
+            refresh_auth()
+        except ValueError:
+            logging.warning("Detected corrupted token file, restarting auth process")
+            authorize(True)
+    else:
+        logging.info("Starting login with OTP")
+        auth_session = requests.session()
+        auth_session.headers = {
+            "authorization": "Basic aDRZZjBiOlU4WWtTYlZsNnh3c2c1WVFxWmZyZ1ZtSWFEcGhPc3kxUENhVXNpY1F0bzNUUjVrd2FKc2U0QVpkZ2ZJZmNMeXc=",
+            "User-Agent": "vca-android/" + util.get_volvo_app_version(),
+            "Accept-Encoding": "gzip",
+            "Content-Type": "application/json; charset=utf-8"
+        }
+
+        url_params = ("?client_id=h4Yf0b"
+                      "&response_type=code"
+                      "&acr_values=urn:volvoid:aal:bronze:2sv"
+                      "&response_mode=pi.flow"
+                      "&scope=openid email profile care_by_volvo:financial_information:invoice:read care_by_volvo:financial_information:payment_method care_by_volvo:subscription:read customer:attributes customer:attributes:write order:attributes vehicle:attributes tsp_customer_api:all conve:brake_status conve:climatization_start_stop conve:command_accessibility conve:commands conve:diagnostics_engine_status conve:diagnostics_workshop conve:doors_status conve:engine_status conve:environment conve:fuel_status conve:honk_flash conve:lock conve:lock_status conve:navigation conve:odometer_status conve:trip_statistics conve:tyre_status conve:unlock conve:vehicle_relation conve:warnings conve:windows_status energy:battery_charge_level energy:charging_connection_status energy:charging_system_status energy:electric_range energy:estimated_charging_time energy:recharge_status vehicle:attributes")
+
+        auth = auth_session.get(OAUTH_AUTH_URL + url_params)
+        if auth.status_code == 200:
+            response = auth.json()
+            auth_state = response["status"]
+
+            if auth_state == "USERNAME_PASSWORD_REQUIRED":
+                auth_session.headers.update({"x-xsrf-header": "PingFederate"})
+                response = check_username_password(auth_session, response)
+                auth_state = response["status"]
+
+                if auth_state == "OTP_REQUIRED":
+                    response = send_otp(auth_session, response)
+                    response = continue_auth(auth_session, response)
+                    token_data = get_token(auth_session, response)
+                else:
+                    raise Exception("Unkown auth state " + auth_state)
+            elif auth_state == "OTP_REQUIRED":
+                response = send_otp(auth_session, response)
+                response = continue_auth(auth_session, response)
+                token_data = get_token(auth_session, response)
+            elif auth_state == "OTP_VERIFIED":
+                response = continue_auth(auth_session, response)
+                token_data = get_token(auth_session, response)
+            elif auth_state == "COMPLETED":
+                token_data = get_token(auth_session, response)
+            else:
+                raise Exception("Unkown auth state " + auth_state)
+
+            session.headers.update({"authorization": "Bearer " + token_data["access_token"]})
+
+            global token_expires_at
+            token_expires_at = datetime.now(util.TZ) + timedelta(seconds=(token_data["expires_in"] - 30))
+            refresh_token = token_data["refresh_token"]
+
+            util.save_to_json(token_data, token_path)
+        else:
+            message = auth.json()
+            raise Exception(message["details"][0]["userMessage"])
+
+    get_vcc_api_keys()
+    get_vehicles()
+    check_supported_endpoints()
+    Thread(target=backend_status_loop).start()
+
+def continue_auth(auth_session, data):
+    next_url = data["_links"]["continueAuthentication"]["href"] + "?action=continueAuthentication"
+    auth = auth_session.get(next_url)
 
-    body = {
-        "username": settings.volvoData["username"],
-        "password": settings.volvoData["password"],
-        "grant_type": "password",
-        "scope": "openid email profile care_by_volvo:financial_information:invoice:read care_by_volvo:financial_information:payment_method care_by_volvo:subscription:read customer:attributes customer:attributes:write order:attributes vehicle:attributes tsp_customer_api:all conve:brake_status conve:climatization_start_stop conve:command_accessibility conve:commands conve:diagnostics_engine_status conve:diagnostics_workshop conve:doors_status conve:engine_status conve:environment conve:fuel_status conve:honk_flash conve:lock conve:lock_status conve:navigation conve:odometer_status conve:trip_statistics conve:tyre_status conve:unlock conve:vehicle_relation conve:warnings conve:windows_status energy:battery_charge_level energy:charging_connection_status energy:charging_system_status energy:electric_range energy:estimated_charging_time energy:recharge_status vehicle:attributes"
-    }
-    auth = requests.post(OAUTH_URL, data=body, headers=headers)
     if auth.status_code == 200:
-        data = auth.json()
+        return auth.json()
-        session.headers.update({"authorization": "Bearer " + data["access_token"]})
-
-        global token_expires_at, refresh_token
-        token_expires_at = datetime.now(util.TZ) + timedelta(seconds=(data["expires_in"] - 30))
-        refresh_token = data["refresh_token"]
-
-        get_vcc_api_keys()
-        get_vehicles()
-        check_supported_endpoints()
     else:
         message = auth.json()
+        raise Exception(message["details"][0]["userMessage"])
+
+
+def get_token(auth_session, data):
+    auth_session.headers.update({"content-type": "application/x-www-form-urlencoded"})
+    body = {"code": data["authorizeResponse"]["code"], "grant_type": "authorization_code"}
+    token_auth = auth_session.post(OAUTH_TOKEN_URL, data=body)
+
+    if token_auth.status_code == 200:
+        return token_auth.json()
+    else:
+        message = token_auth.json()
         raise Exception(message["error_description"])
 
 
+def check_username_password(auth_session, data):
+    next_url = data["_links"]["checkUsernamePassword"]["href"] + "?action=checkUsernamePassword"
+    body = {"username": settings.volvoData["username"],
+             "password": settings.volvoData["password"]}
+    auth = auth_session.post(next_url, data=json.dumps(body))
+
+    if auth.status_code == 200:
+        return auth.json()
+    else:
+        message = auth.json()
+        raise Exception(message["details"][0]["userMessage"])
+
+
+def send_otp(auth_session, data):
+    mqtt.create_otp_input()
+    next_url = data["_links"]["checkOtp"]["href"] + "?action=checkOtp"
+    body = {"otp": ""}
+
+    for i in range(otp_max_loops):
+        if mqtt.otp_code:
+            body["otp"] = mqtt.otp_code
+            break
+
+        logging.info("Waiting for otp code... Please check your mailbox and post your otp code to the following "
+                     "mqtt topic \"volvoAAOS2mqtt/otp_code\". Retry " + str(i) + "/" + str(otp_max_loops))
+        time.sleep(5)
+
+    if not mqtt.otp_code:
+        raise Exception ("No OTP found, exting...")
+
+    auth = auth_session.post(next_url, data=json.dumps(body))
+    if auth.status_code == 200:
+        return auth.json()
+    else:
+        message = auth.json()
+        raise Exception(message["details"][0]["userMessage"])
+
+
 def refresh_auth():
     logging.info("Refreshing credentials")
     global refresh_token
@@ -73,18 +183,22 @@ def refresh_auth():
     }
 
     try:
-        auth = requests.post(OAUTH_URL, data=body, headers=headers)
+        auth = requests.post(OAUTH_TOKEN_URL, data=body, headers=headers)
     except requests.exceptions.RequestException as e:
         logging.error("Error refreshing credentials data: " + str(e))
         return None
 
     if auth.status_code == 200:
+        token_path = util.get_token_path()
         data = auth.json()
+        util.save_to_json(data, token_path)
         session.headers.update({"authorization": "Bearer " + data["access_token"]})
 
         global token_expires_at
         token_expires_at = datetime.now(util.TZ) + timedelta(seconds=(data["expires_in"] - 30))
         refresh_token = data["refresh_token"]
+    else:
+        authorize(renew_tokenfile=True)
 
 
 def get_vehicles():
@@ -115,6 +229,7 @@ def get_vehicles():
     else:
         initialize_climate(vins)
         initialize_scheduler(vins)
+        mqtt.delete_old_entities()
         logging.info("Vin: " + str(vins) + " found!")
 
 
@@ -239,16 +354,11 @@ def check_supported_endpoints():
     for vin in vins:
         supported_endpoints[vin] = []
         for entity in supported_entities:
-            if entity["id"] == "battery_charge_level" and entity["url"] == BATTERY_CHARGE_STATE_URL \
+            if entity["id"] == "battery_charge_level" and entity["url"] == FUEL_BATTERY_STATE_URL \
                     and any("battery_charge_level" in d["id"] for d in supported_endpoints[vin]):
                 # If battery charge level could be found in recharge-api, skip the second battery charge sensor
                 continue
 
-            if entity["id"] == "engine_state" and entity["url"] == ENGINE_DIAGNOSTICS_URL \
-                    and any("engine_state" in d["id"] for d in supported_endpoints[vin]):
-                # If engine state could be found in engine state endpoint, skip the second engine running sensor
-                continue
-
             if entity.get('url'):
                 state = api_call(entity["url"], "GET", vin, entity["id"])
             else:
@@ -307,7 +417,7 @@ def check_engine_status(vin):
         # Exit thread as engine state is unsupported by car
         return None
 
-    t = currentThread()
+    t = current_thread()
     while getattr(t, "do_run", True):
         engine_state = api_call(endpoint_url, "GET", vin, "engine_state", True)
         if engine_state == "ON":
@@ -317,12 +427,37 @@ def check_engine_status(vin):
         time.sleep(5)
 
 
+def backend_status_loop():
+    while True:
+        get_backend_status()
+        # Update every hour
+        time.sleep(3600)
+
+
+def get_backend_status():
+    global backend_status
+    response = session.get(API_BACKEND_STATUS, timeout=15)
+    try:
+        data = response.json()
+        if util.keys_exists(data, "message"):
+            if data["message"]:
+                backend_status = data["message"]
+            else:
+                backend_status = "NO_WARNING"
+        else:
+            backend_status = "NO_WARNING"
+
+    except JSONDecodeError as e:
+        backend_status = "NO_WARNING"
+    return backend_status
+
+
 def api_call(url, method, vin, sensor_id=None, force_update=False, key_change=False):
     if datetime.now(util.TZ) >= token_expires_at:
         refresh_auth()
 
     if url in [RECHARGE_STATE_URL, WINDOWS_STATE_URL, LOCK_STATE_URL, TYRE_STATE_URL,
-               STATISTICS_URL, ENGINE_DIAGNOSTICS_URL]:
+               STATISTICS_URL, ENGINE_DIAGNOSTICS_URL, VEHICLE_DIAGNOSTICS_URL, FUEL_BATTERY_STATE_URL, WARNINGS_URL]:
         # Minimize API calls for endpoints with multiple values
         response = cached_request(url, method, vin, force_update, key_change)
         if response is None:
@@ -347,7 +482,11 @@ def api_call(url, method, vin, sensor_id=None, force_update=False, key_change=Fa
         return None
 
     logging.debug("Response status code: " + str(response.status_code))
-    data = response.json()
+    try:
+        data = response.json()
+    except JSONDecodeError as e:
+        logging.error("Fetched json decode error, Volvo API seems to return garbage. Skipping update. Error: " + str(e))
+        return None
 
     if response.status_code == 200:
         logging.debug(response.text)
@@ -364,7 +503,7 @@ def api_call(url, method, vin, sensor_id=None, force_update=False, key_change=Fa
             return None
         elif response.status_code == 403 and "message" in data:
             if "Out of call volume quota" in data["message"]:
-                logging.warn("Quota extended. Try to change VCCAPIKEY!")
+                logging.warning("Quota extended. Try to change VCCAPIKEY!")
                 change_vcc_api_key()
                 api_call(url, method, vin, sensor_id, force_update, True)
             else:
@@ -410,9 +549,13 @@ def cached_request(url, method, vin, force_update=False, key_change=False):
 
 
 def parse_api_data(data, sensor_id=None):
-    data = data["data"]
+    if sensor_id != "api_backend_status":
+        data = data["data"]
+
     if sensor_id == "battery_charge_level":
         return data["batteryChargeLevel"]["value"] if util.keys_exists(data, "batteryChargeLevel") else None
+    elif sensor_id == "battery_capacity":
+        return data["batteryCapacityKWH"] if util.keys_exists(data, "batteryCapacityKWH") else None
     elif sensor_id == "electric_range":
         return util.convert_metric_values(data["electricRange"]["value"]) \
             if util.keys_exists(data, "electricRange") else None
@@ -443,7 +586,7 @@ def parse_api_data(data, sensor_id=None):
                 return ""
         return None
     elif sensor_id == "lock_status":
-        return data["carLocked"]["value"] if util.keys_exists(data, "carLocked") else None
+        return data["centralLock"]["value"] if util.keys_exists(data, "centralLock") else None
     elif sensor_id == "odometer":
         multiplier = 1
         if util.keys_exists(settings["volvoData"], "odometerMultiplier"):
@@ -455,44 +598,44 @@ def parse_api_data(data, sensor_id=None):
         return util.convert_metric_values(int(data["odometer"]["value"]) * multiplier) \
             if util.keys_exists(data, "odometer") else None
     elif sensor_id == "window_front_left":
-        return window_states[data["frontLeftWindowOpen"]["value"]] if util.keys_exists(data, "frontLeftWindowOpen") \
+        return window_states[data["frontLeftWindow"]["value"]] if util.keys_exists(data, "frontLeftWindow") \
             else None
     elif sensor_id == "window_front_right":
-        return window_states[data["frontRightWindowOpen"]["value"]] if util.keys_exists(data, "frontRightWindowOpen") \
+        return window_states[data["frontRightWindow"]["value"]] if util.keys_exists(data, "frontRightWindow") \
             else None
     elif sensor_id == "window_rear_left":
-        return window_states[data["rearLeftWindowOpen"]["value"]] if util.keys_exists(data, "rearLeftWindowOpen") \
+        return window_states[data["rearLeftWindow"]["value"]] if util.keys_exists(data, "rearLeftWindow") \
             else None
     elif sensor_id == "window_rear_right":
-        return window_states[data["rearRightWindowOpen"]["value"]] if util.keys_exists(data, "rearRightWindowOpen") \
+        return window_states[data["rearRightWindow"]["value"]] if util.keys_exists(data, "rearRightWindow") \
             else None
     elif sensor_id == "door_front_left":
-        return door_states[data["frontLeftDoorOpen"]["value"]] if util.keys_exists(data, "frontLeftDoorOpen") else None
+        return door_states[data["frontLeftDoor"]["value"]] if util.keys_exists(data, "frontLeftDoor") else None
     elif sensor_id == "door_front_right":
-        return door_states[data["frontRightDoorOpen"]["value"]] \
+        return door_states[data["frontRightDoor"]["value"]] \
-            if util.keys_exists(data, "frontRightDoorOpen") else None
+            if util.keys_exists(data, "frontRightDoor") else None
     elif sensor_id == "door_rear_left":
-        return door_states[data["rearLeftDoorOpen"]["value"]] if util.keys_exists(data, "rearLeftDoorOpen") else None
+        return door_states[data["rearLeftDoor"]["value"]] if util.keys_exists(data, "rearLeftDoor") else None
     elif sensor_id == "door_rear_right":
-        return door_states[data["rearRightDoorOpen"]["value"]] if util.keys_exists(data, "rearRightDoorOpen") else None
+        return door_states[data["rearRightDoor"]["value"]] if util.keys_exists(data, "rearRightDoor") else None
     elif sensor_id == "tailgate":
-        return door_states[data["tailGateOpen"]["value"]] if util.keys_exists(data, "tailGateOpen") else None
+        return door_states[data["tailgate"]["value"]] if util.keys_exists(data, "tailgate") else None
     elif sensor_id == "sunroof":
-        return door_states[data["sunRoofOpen"]["value"]] if util.keys_exists(data, "sunRoofOpen") else None
+        return door_states[data["sunroof"]["value"]] if util.keys_exists(data, "sunroof") else None
     elif sensor_id == "engine_hood":
-        return door_states[data["hoodOpen"]["value"]] if util.keys_exists(data, "hoodOpen") else None
+        return door_states[data["hood"]["value"]] if util.keys_exists(data, "hood") else None
     elif sensor_id == "tank_lid":
-        return door_states[data["tankLidOpen"]["value"]] if util.keys_exists(data, "tankLidOpen") else None
+        return door_states[data["tankLid"]["value"]] if util.keys_exists(data, "tankLid") else None
     elif sensor_id == "tyre_front_left":
-        return data["frontLeftTyrePressure"]["value"] if util.keys_exists(data, "frontLeftTyrePressure") else None
+        return data["frontLeft"]["value"] if util.keys_exists(data, "frontLeft") else None
     elif sensor_id == "tyre_front_right":
-        return data["frontRightTyrePressure"]["value"] if util.keys_exists(data, "frontRightTyrePressure") else None
+        return data["frontRight"]["value"] if util.keys_exists(data, "frontRight") else None
     elif sensor_id == "tyre_rear_left":
-        return data["rearLeftTyrePressure"]["value"] if util.keys_exists(data, "rearLeftTyrePressure") else None
+        return data["rearLeft"]["value"] if util.keys_exists(data, "rearLeft") else None
     elif sensor_id == "tyre_rear_right":
-        return data["rearRightTyrePressure"]["value"] if util.keys_exists(data, "rearRightTyrePressure") else None
+        return data["rearRight"]["value"] if util.keys_exists(data, "rearRight") else None
     elif sensor_id == "engine_state":
-        return engine_states[data["engineRunning"]["value"]] if util.keys_exists(data, "engineRunning") else None
+        return engine_states[data["engineStatus"]["value"]] if util.keys_exists(data, "engineStatus") else None
     elif sensor_id == "fuel_level":
         if util.keys_exists(data, "fuelAmount"):
             fuel_amount = float(data["fuelAmount"]["value"])
@@ -534,25 +677,48 @@ def parse_api_data(data, sensor_id=None):
                                "latitude": raw_data["coordinates"][1],
                                "gps_accuracy": 1}
         return coordinates
-    elif sensor_id == "distance_to_empty":
+    elif sensor_id == "distance_to_empty_tank":
-        if util.keys_exists(data, "distanceToEmpty"):
+        if util.keys_exists(data, "distanceToEmptyTank"):
-            distance_to_empty = int(data["distanceToEmpty"]["value"])
+            distance_to_empty = int(data["distanceToEmptyTank"]["value"])
             if distance_to_empty > 0:
-                return util.convert_metric_values(data["distanceToEmpty"]["value"])
+                return util.convert_metric_values(data["distanceToEmptyTank"]["value"])
+        return None
+    elif sensor_id == "distance_to_empty_battery":
+        if util.keys_exists(data, "distanceToEmptyBattery"):
+            distance_to_empty = int(data["distanceToEmptyBattery"]["value"])
+            if distance_to_empty > 0:
+                return util.convert_metric_values(data["distanceToEmptyBattery"]["value"])
         return None
     elif sensor_id == "hours_to_service":
         return data["engineHoursToService"]["value"] if util.keys_exists(data, "engineHoursToService") else None
     elif sensor_id == "km_to_service":
-        if util.keys_exists(data, "kmToService"):
+        if util.keys_exists(data, "distanceToService"):
-            km_to_service = int(data["kmToService"]["value"])
+            km_to_service = int(data["distanceToService"]["value"])
             if km_to_service > 0:
-                return util.convert_metric_values(data["kmToService"]["value"])
+                return util.convert_metric_values(data["distanceToService"]["value"])
         return None
-    elif sensor_id == "months_to_service":
+    elif sensor_id == "time_to_service":
-        return data["monthsToService"]["value"] if util.keys_exists(data, "monthsToService") else None
+        if util.keys_exists(data, "timeToService"):
+            return str(data["timeToService"]["value"]) + " " + data["timeToService"]["unit"]
+        else:
+            return None
     elif sensor_id == "service_warning_status":
-        return data["serviceWarningStatus"]["value"] if util.keys_exists(data, "serviceWarningStatus") else None
+        return data["serviceWarning"]["value"] if util.keys_exists(data, "serviceWarning") else None
-    elif sensor_id == "service_warning_trigger":
+    elif sensor_id == "average_energy_consumption":
-        return data["serviceWarningTrigger"]["value"] if util.keys_exists(data, "serviceWarningTrigger") else None
+        return data["averageEnergyConsumption"]["value"] if util.keys_exists(data, "averageEnergyConsumption") else None
+    elif sensor_id == "washer_fluid_warning":
+        return data["washerFluidLevelWarning"]["value"] if util.keys_exists(data, "washerFluidLevelWarning") else None
+    elif sensor_id == "warnings":
+        warnings = 0
+        cleaned_data = {}
+        for key, dicts in data.items():
+            contains_data = sum(value == "NO_WARNING" or value == "FAILURE" for value in dicts.values())
+            if contains_data:
+                warnings = warnings + 1
+                cleaned_data[key] = dicts["value"]
+
+        return cleaned_data if warnings > 0 else None
     else:
         return None
+
+