chore(all): Bump flutter_secure_storage

- Bump moxxmpp to allow queueing stanzas that are sent offline
- Should fix #75.

.github/FUNDING.yml

@@ -0,0 +1 @@
+ko_fi: papatutuwawa

.gitignore

@@ -60,3 +60,6 @@ lib/i18n/*.dart
 
 # Android artifacts
 .android
+
+# Build scripts
+release-*/

.gitlint

@@ -7,7 +7,7 @@ line-length=72
 [title-trailing-punctuation]
 [title-hard-tab]
 [title-match-regex]
-regex=^(feat|fix|chore|refactor|docs|release|test)\((xmpp|service|ui|shared|meta|tests|i18n)+(,(xmpp|service|ui|shared|meta|tests|i18n))*\): .*$
+regex=^((feat|fix|chore|refactor)\((service|ui|shared|all|tests|i18n|docs|flake)+(,(service|ui|shared|all|tests|i18n|docs|flake))*\)|release): [A-Z0-9].*$
 
 
 [body-trailing-whitespace]

CONTRIBUTING.md

@@ -0,0 +1,81 @@
+# Contribution Guide
+
+Thanks for your interest in the Moxxy XMPP client! This document contains guidelines and guides for working
+on the Moxxy codebase.
+
+## Prerequisites
+
+Before building or working on Moxxy, please make sure that your development environment is correctly set up.
+Moxxy requires Flutter 3.7.3, since we use a fork of the Flutter library, and the JDK 17. Building Moxxy
+is currently only supported for Android.
+
+### Android Studio
+
+If you use Android Studio, make sure that you use version "Flamingo Canary 3", as that one comes bundled with
+JDK 17, instead of JDK 11 ([See here](https://codeberg.org/moxxy/moxxy/issues/252)). If that is
+not an option, you can manually add a JDK 17 installation in Android Studio and tell the Flutter addon
+to use that installation instead.
+
+### NixOS
+
+If you use NixOS or Nix, you can use the dev shell provided by the Flake in the repository's root. It contains
+the correct JDK and Flutter version. However, make sure that other environment variables, like
+`ANDROID_HOME` and `ANDROID_AVD_HOME`, are correctly set.
+
+## Building
+
+Currently, Moxxy contains a git submodule. While it is not utilised at the moment, it contains
+the list of suggested XMPP providers to use for auto-registration. To properly clone the
+repository, use `git clone --recursive https://codeberg.org/moxxy/moxxy.git`
+
+In order to build Moxxy, you first have to run the code generator. To do that, first install all dependencies with
+`flutter pub get`. Next, run the code generator using `flutter pub run build_runner build`. This builds required
+data classes and the i18n support.
+
+Finally, you can build Moxxy using `flutter run`, if you want to test a change, or `flutter build apk --release` to build
+an optimized release build. The release builds found in the repository's releases are build using `flutter build apk --release --split-per-abi`.
+
+## Contributing
+
+If you want to fix a small issue, you can just fork, create a new branch, and start working right away. However, if you want to work
+on a bigger feature, please first create an issue (if an issue does not already exist) or join the [development chat](xmpp:moxxy@muc.moxxy.org?join) (xmpp:moxxy@muc.moxxy.org?join)
+to discuss the feature first.
+
+Before creating a pull request, please make sure you checked every item on the following checklist:
+
+- [ ] I formatted the code with the dart formatter (`dart format`) before running the linter
+- [ ] I ran the linter (`flutter analyze`) and introduced no new linter warnings
+- [ ] I ran the tests (`flutter test`) and introduced no new failing tests
+- [ ] I used [gitlint](https://github.com/jorisroovers/gitlint) to ensure propper formatting of my commig messages
+
+If you think that your code is ready for a pull request, but you are not sure if it is ready, prefix the PR's title with "WIP: ", so that discussion
+can happen there. If you think your PR is ready for review, remove the "WIP: " prefix.
+
+### Code Guidelines
+#### Commit messages
+
+Commit messages should be uniformly formatted. `gitlint` is a linter for commit messages that enforces those guidelines. They are defined in the `.gitlint` file
+at the root of the repository. `gitlint` can be installed as a pre-commit hook using
+`gitlint install-hook`. That way, `gitlint` runs on every commit and warns you if the
+commit message violates any of the defined rules.
+
+Commit messages always follow the following format:
+
+```
+<type>(<areas>): <summary>
+
+<full message>
+```
+
+`<type>` is the type of action that was performed in the commit and is one of the following: `feat` (Addition of a feature), `fix` (Fix a bug or other issue), `chore` (Bump dependency versions, fix formatter issues), `refactor` (A bigger "moving around" or rewriting of code), `docs` (Commits that just touch the documentation, be it code or, for example, the README).
+
+`<areas>` are the areas inside the code that are touched by the change. They are a comma-separated list of one or more of the following: `service` (Everything inside `lib/service`), `ui` (Everything inside `lib/ui`), `shared` (Everything inside `lib/shared`), `all` (A bit of everything is involved), `tests` (Everyting inside `test` or `integration_test`), `i18n` (The translation files have been modified), `docs` (Documentation of any kind), `flake` (The NixOS flake has been modified).
+
+`<summary>` is the summary of the entire commit in a few words. Make that that the entire
+first line is not longer than 72 characters. `<summary>` also must start with an uppercase
+letter or a number.
+
+The `<full message>` is optional. In case your commit requires more explanation, write it
+there. Make sure that there is an empty line between the full message and the summary line.
+
+The exception to these rules is a commit message of the format `release: Release version x.y.z`, as it touches everything and is thus implicitly using `(all)` as an area code.

README.md

@@ -2,35 +2,20 @@
 
 An experimental XMPP client that tries to be as easy, modern and beautiful as possible.
 
-The code is also available on [codeberg](https://codeberg.org/moxxy/moxxyv2).
+The code is also available on [codeberg](https://codeberg.org/moxxy/moxxy).
 
-[<img src="https://gitlab.com/IzzyOnDroid/repo/-/raw/master/assets/IzzyOnDroid.png" alt="Get it on IzzyOnDroid" height="80" />](https://apt.izzysoft.de/fdroid/index/apk/org.moxxy.moxxyv2)
+[<img src="https://gitlab.com/IzzyOnDroid/repo/-/raw/master/assets/IzzyOnDroid.png" alt="Get it on IzzyOnDroid" height="80">](https://apt.izzysoft.de/fdroid/index/apk/org.moxxy.moxxyv2)
+
+Or [get the latest APK from Codeberg](https://codeberg.org/moxxy/moxxy/releases/latest).
 
 ## Screenshots
 
 [<img src="https://codeberg.org/moxxy/moxxyv2/raw/branch/master/fastlane/metadata/android/en-US/images/phoneScreenshots/1.png" width="20%"></img>](./fastlane/metadata/android/en-US/images/phoneScreenshots/1.png)
 [<img src="https://codeberg.org/moxxy/moxxyv2/raw/branch/master/fastlane/metadata/android/en-US/images/phoneScreenshots/2.png" width="20%"></img>](./fastlane/metadata/android/en-US/images/phoneScreenshots/2.png)
 
-## Developing and Building
+## Building and Contributing
 
-Clone using `git clone --recursive https://github.com/Polynomdivision/moxxyv2.git`.
-
-In order to build Moxxy, you need to have [Flutter](https://docs.flutter.dev/get-started/install) set
-up. If you are running NixOS or using Nix, you can also use the Flake at the root of the repository
-by running `nix develop` to get a development shell including everything that is needed. Note
-that if you decide to use the Flake, `ANDROID_HOME` and `ANDROID_AVD_HOME` must be set to the respective directories.
-
-Before building Moxxy, you need to generate all needed data classes. To do this, run
-`flutter pub get` to install all dependencies. Then run `flutter pub run build_runner build` to generate
-state classes, data classes and the database schemata. After that is done, you can either
-build the app with `flutter build apk --debug` to create a debug build,
-`flutter build apk --release` to create a relase build or just run the app in development
-mode with `flutter run`.
-
-After implementing a change or a feature, please ensure that nothing is broken by the change
-by running `flutter test` afterwards. Also make sure that the code passes the linter by
-running `flutter analyze`. This project also uses [gitlint](https://github.com/jorisroovers/gitlint)
-to ensure uniform formatting of commit messages.
+For build and contribution guidelines, please refer to [`CONTRIBUTING.md`](./CONTRIBUTING.md)
 
 Also, feel free to join the development chat at `moxxy@muc.moxxy.org`.
 
@@ -46,3 +31,9 @@ See `./LICENSE`.
 ## Special Thanks
 
 - New logo designed by [Synoh](https://twitter.com/synoh_manda)
+
+## Support
+
+If you like what I do and you want to support me, feel free to donate to me on Ko-Fi.
+
+[<img src="https://codeberg.org/moxxy/moxxyv2/raw/branch/master/assets/repo/kofi.png" height="36" style="height: 36px; border: 0px;"></img>](https://ko-fi.com/papatutuwawa)

analysis_options.yaml

@@ -6,13 +6,11 @@ linter:
     use_setters_to_change_properties: false
     avoid_positional_boolean_parameters: false
     avoid_bool_literals_in_conditional_expressions: false
+    file_names: false
 
 analyzer:
   exclude:
     - "**/*.g.dart"
     - "**/*.freezed.dart"
     - "**/*.moxxy.dart"
-    - "test/"
-    - "integration_test/"
-    - "lib/service/database/migrations/*.dart"
     - "lib/i18n/*.dart"

android/app/src/main/AndroidManifest.xml

@@ -48,6 +48,7 @@
    <uses-permission android:name="android.permission.INTERNET" />
    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
    <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
+   <uses-permission android:name="android.permission.READ_CONTACTS" />
    <queries>
      <intent>
        <action android:name="android.intent.action.VIEW" />

android/build.gradle

@@ -1,5 +1,5 @@
 buildscript {
-    ext.kotlin_version = '1.6.10'
+    ext.kotlin_version = '1.8.21'
     repositories {
         google()
         mavenCentral()

assets/i18n/strings.i18n.json

@@ -25,15 +25,45 @@
 	    "messagesChannelDescription": "The notification channel for received messages",
 	    "warningChannelName": "Warnings",
 	    "warningChannelDescription": "Warnings related to Moxxy"
+	},
+	"titles": {
+	    "error": "Error"
 	}
     },
+    "dateTime": {
+	"justNow": "Just now",
+	"nMinutesAgo": "${min}min ago",
+	"mondayAbbrev": "Mon",
+	"tuesdayAbbrev": "Tue",
+	"wednessdayAbbrev": "Wed",
+	"thursdayAbbrev": "Thu",
+	"fridayAbbrev": "Fri",
+	"saturdayAbbrev": "Sat",
+	"sundayAbbrev": "Sun",
+	"january": "January",
+	"february": "February",
+	"march": "March",
+	"april": "April",
+	"may": "May",
+	"june": "June",
+	"july": "July",
+	"august": "August",
+	"september": "September",
+	"october": "October",
+	"november": "November",
+	"december": "December",
+	"today": "Today",
+	"yesterday": "Yesterday"
+    },
     "messages": {
 	"image": "Image",
 	"video": "Video",
 	"audio": "Audio",
 	"file": "File",
+	"sticker": "Sticker",
 	"retracted": "The message has been retracted",
-	"retractedFallback": "A previous message has been retracted but your client does not support it"
+	"retractedFallback": "A previous message has been retracted but your client does not support it",
+	"you": "You"
     },
     "errors": {
 	"omemo": {
@@ -41,10 +71,19 @@
 	    "notEncryptedForDevice": "This message was not encrypted for this device",
 	    "invalidHmac": "Could not decrypt message",
 	    "noDecryptionKey": "No decryption key available",
-	    "messageInvalidAfixElement": "Invalid encrypted message"
+	    "messageInvalidAfixElement": "Invalid encrypted message",
+
+	    "verificationInvalidOmemoUrl": "Invalid OMEMO:2 fingerprint",
+	    "verificationWrongJid": "Wrong XMPP-address",
+	    "verificationWrongDevice": "Wrong OMEMO:2 device",
+	    "verificationNotInList": "Wrong OMEMO:2 device",
+	    "verificationWrongFingerprint": "Wrong OMEMO:2 fingerprint"
 	},
 	"connection": {
-	    "connectionTimeout": "Could not connect to server"
+	    "connectionTimeout": "Could not connect to server",
+	    "saslAccountDisabled": "Your account is disabled",
+	    "saslInvalidCredentials": "Your account credentials are invalid",
+	    "unrecoverable": "Connection lost due to unrecoverable error"
 	},
 	"login": {
 	    "saslFailed": "Invalid login credentials",
@@ -66,7 +105,10 @@
 	    "fileNotEncrypted": "The chat is encrypted but the file is not encrypted"
 	},
 	"conversation": {
-	    "audioRecordingError": "Failed to finalize audio recording"
+	    "audioRecordingError": "Failed to finalize audio recording",
+	    "openFileNoAppError": "No app found to open this file",
+	    "openFileGenericError": "Failed to open file",
+	    "messageErrorDialogTitle": "Error"
 	}
     },
     "warnings": {
@@ -93,6 +135,7 @@
 	"conversations": {
 	    "speeddialNewChat": "New chat",
 	    "speeddialJoinGroupchat": "Join groupchat",
+		"speeddialAddNoteToSelf": "Note to self",
 	    "overlaySettings": "Settings",
 	    "noOpenChats": "You have no open chats",
 	    "startChat": "Start a chat",
@@ -115,11 +158,20 @@
 	    "edit": "Edit",
 	    "quote": "Quote",
 	    "copy": "Copy content",
+	    "addReaction": "Add reaction",
 	    "showError": "Show error",
 	    "showWarning": "Show warning",
 	    "addToContacts": "Add to contacts",
 	    "addToContactsTitle": "Add ${jid} to contacts",
-	    "addToContactsBody": "Are you sure you want to add ${jid} to your contacts?"
+	    "addToContactsBody": "Are you sure you want to add ${jid} to your contacts?",
+	    "stickerPickerNoStickersLine1": "You have no sticker packs installed.",
+	    "stickerPickerNoStickersLine2": "They can be installed in the sticker settings.",
+	    "stickerSettings": "Sticker settings",
+	    "newDeviceMessage": "${title} added a new encryption device",
+	    "messageHint": "Send a message...",
+	    "sendImages": "Send images",
+	    "sendFiles": "Send files",
+	    "takePhotos": "Take photos"
 	},
 	"addcontact": {
 	    "title": "Add new contact",
@@ -141,15 +193,16 @@
 	    "confirmBody": "One or more chats are unencrypted. This means that the file will be leaked to the server. Do you still want to continue?"
 	},
 	"profile": {
-	    "self": {
-		"devices": "Devices"
+	    "general": {
+		"omemo": "Security",
+		"profile": "Profile",
+		"media": "Media"
 	    },
 	    "conversation": {
-		"muteChatTooltip": "Mute chat",
-		"unmuteChatTooltip": "Unmute chat",
-		"muteChat": "Mute",
-		"unmuteChat": "Unmute",
-		"devices": "Devices"
+		"notifications": "Notifications",
+		"notificationsMuted": "Muted",
+		"notificationsEnabled": "Enabled",
+		"sharedMedia": "Media"
 	    },
 	    "owndevices": {
 		"title": "Own Devices",
@@ -165,10 +218,11 @@
 		"recreateOwnDeviceConfirmBody": "This will recreate this device's cryptographic identity. It will take some time. If contacts verified your device, they will have to do it again. Continue?"
 	    },
 	    "devices": {
-		"title": "Devices",
+		"title": "Security",
 		"recreateSessions": "Rebuild sessions",
 		"recreateSessionsConfirmTitle": "Rebuild sessions?",
-		"recreateSessionsConfirmBody": "This will recreate the cryptographic sessions with your own devices. Use only if your own devices throw decryption errors."
+		"recreateSessionsConfirmBody": "This will recreate the cryptographic sessions with your own devices. Use only if your own devices throw decryption errors.",
+		"noSessions": "There are no cryptographic sessions that are used for end-to-end encryption."
 	    }
 	},
 	"blocklist": {
@@ -184,6 +238,14 @@
 	    "blur": "Blur background",
 	    "setAsBackground": "Set as background image"
 	},
+	"stickerPack": {
+	    "removeConfirmTitle": "Remove sticker pack",
+	    "removeConfirmBody": "Are you sure you want to remove this sticker pack?",
+	    "installConfirmTitle": "Install sticker pack",
+	    "installConfirmBody": "Are you sure you want to install this sticker pack?",
+	    "restricted": "This sticker pack is restricted. That means that the stickers will be displayed but cannot be sent.",
+	    "fetchingFailure": "Could not find the sticker pack"
+	},
 	"settings": {
 	    "settings": {
 		"title": "Settings",
@@ -193,13 +255,17 @@
 		"signOutConfirmTitle": "Sign Out",
 		"signOutConfirmBody": "You are about to sign out. Proceed?",
 		"miscellaneousSection": "Miscellaneous",
-		"debuggingSection": "Debugging"
+		"debuggingSection": "Debugging",
+		"general": "General"
 	    },
 	    "about": {
 		"title": "About",
 		"licensed": "Licensed under GPL3",
 		"version": "Version ${version}",
-		"viewSourceCode": "View source code"
+		"viewSourceCode": "View source code",
+		"nMoreToGo": "${n} more to go...",
+		"debugMenuShown": "You are now a developer!",
+		"debugMenuAlreadyShown": "You are already a developer!"
 	    },
 	    "appearance": {
 		"title": "Appearance",
@@ -222,7 +288,10 @@
 		"removeBackgroundImageConfirmBody": "Are you sure you want to remove your conversation background image?",
 		"newChatsSection": "New Conversations",
 		"newChatsMuteByDefault": "Mute new chats by default",
-		"newChatsE2EE": "Enable end-to-end encryption by default. WARNING: Experimental"
+		"newChatsE2EE": "Enable end-to-end encryption by default. WARNING: Experimental",
+		"behaviourSection": "Behaviour",
+		"contactsIntegration": "Contacts integration",
+		"contactsIntegrationBody": "When enabled, data from the phonebook will be used to provide chat titles and profile pictures. No data will be sent to the server."
 	    },
 	    "debugging": {
 		"title": "Debugging options",
@@ -241,6 +310,7 @@
 		"automaticDownloadsText": "Moxxy will automatically download files on...",
 		"automaticDownloadsMaximumSize": "Maximum Download Size",
 		"automaticDownloadsMaximumSizeSubtext": "The maximum file size for a file to be automatically downloaded",
+		"automaticDownloadAlways": "Always",
 		"wifi": "Wifi",
 		"mobileData": "Mobile data"
 	    },
@@ -251,8 +321,6 @@
 		"showContactRequestsSubtext": "This will show people who added you to their contact list but sent no message yet",
 		"profilePictureVisibility": "Make profile picture public",
 		"profilePictureVisibilitSubtext": "If enabled, everyone can see your profile picture. If disabled, only users on your contact list can see your profile picture.",
-		"autoAcceptSubscriptionRequests": "Auto-accept subscription requests",
-		"autoAcceptSubscriptionRequestsSubtext": "If enabled, subscription requests will be automatically accepted if the user is in the contact list.",
 		"conversationsSection": "Conversation",
 		"sendChatMarkers": "Send chat markers",
 		"sendChatMarkersSubtext": "This will tell your conversation partner if you received or read a message",
@@ -266,7 +334,20 @@
 		"cannotEnableRedirectSubtext": "You must first set a proxy service to redirect to. To do so, tap the field next to the switch.",
 		"urlEmpty": "URL cannot be empty",
 		"urlInvalid": "Invalid URL",
-		"redirectDialogTitle": "$serviceName Redirect"
+		"redirectDialogTitle": "$serviceName Redirect",
+		"stickersPrivacy": "Keep sticker list public",
+		"stickersPrivacySubtext": "If enabled, everyone will be able to see your list of installed sticker packs."
+	    },
+	    "stickers": {
+		"title": "Stickers",
+		"stickerSection": "Sticker",
+		"displayStickers": "Display stickers in chat",
+		"autoDownload": "Automatically download stickers",
+		"autoDownloadBody": "If enabled, stickers are automatically downloaded when the sender is in your contact list.",
+		"stickerPacksSection": "Sticker packs",
+		"importStickerPack": "Import sticker pack",
+		"importSuccess": "Sticker pack successfully imported",
+		"importFailure": "Failed to import sticker pack"
 	    }
 	}
     }

assets/i18n/strings_de.i18n.json

@@ -25,15 +25,45 @@
 	    "messagesChannelDescription": "Empfangene Nachrichten",
 	    "warningChannelName": "Warnungen",
 	    "warningChannelDescription": "Warnungen im Bezug auf Moxxy"
+	},
+	"titles": {
+	    "error": "Fehler"
 	}
     },
+    "dateTime": {
+	"justNow": "Gerade",
+	"nMinutesAgo": "vor ${min}min",
+	"mondayAbbrev": "Mon",
+	"tuesdayAbbrev": "Die",
+	"wednessdayAbbrev": "Mit",
+	"thursdayAbbrev": "Don",
+	"fridayAbbrev": "Fre",
+	"saturdayAbbrev": "Sam",
+	"sundayAbbrev": "Son",
+	"january": "Januar",
+	"february": "Februar",
+	"march": "März",
+	"april": "April",
+	"may": "Mai",
+	"june": "Juni",
+	"july": "Juli",
+	"august": "August",
+	"september": "September",
+	"october": "Oktober",
+	"november": "November",
+	"december": "Dezember",
+	"today": "Heute",
+	"yesterday": "Gestern"
+    },
     "messages": {
 	"image": "Bild",
 	"video": "Video",
 	"audio": "Audio",
 	"file": "Datei",
+	"sticker": "Sticker",
 	"retracted": "Die Nachricht wurde zurückgezogen",
-	"retractedFallback": "Eine vorherige Nachricht wurde zurückgezogen. Dein Client unterstüzt dies jedoch nicht"
+	"retractedFallback": "Eine vorherige Nachricht wurde zurückgezogen. Dein Client unterstüzt dies jedoch nicht",
+	"you": "Du"
     },
     "errors": {
 	"omemo": {
@@ -41,10 +71,19 @@
 	    "notEncryptedForDevice": "Die Nachricht wurde nicht für dieses Gerät verschlüsselt",
 	    "invalidHmac": "Die Nachricht konnte nicht entschlüsselt werden",
 	    "noDecryptionKey": "Kein Schlüssel zum Entschlüsseln vorhanden",
-	    "messageInvalidAfixElement": "Ungültige verschlüsselte Nachricht"
+	    "messageInvalidAfixElement": "Ungültige verschlüsselte Nachricht",
+
+	    "verificationInvalidOmemoUrl": "Ungültiger OMEMO:2 Fingerabdruck",
+	    "verificationWrongJid": "Falsche XMPP-Addresse",
+	    "verificationWrongDevice": "Falsches OMEMO:2 Gerät",
+	    "verificationNotInList": "OMEMO:2 Gerät unbekannt",
+	    "verificationWrongFingerprint": "Falscher OMEMO:2 Fingerabdruck"
 	},
 	"connection": {
-	    "connectionTimeout": "Verbindung zum Server nicht möglich"
+	    "connectionTimeout": "Verbindung zum Server nicht möglich",
+	    "saslAccountDisabled": "Dein Account ist deaktiviert",
+	    "saslInvalidCredentials": "Deine Anmeldedaten sind ungültig",
+	    "unrecoverable": "Verbindung zum Server durch nicht behebbaren Fehler verloren"
 	},
 	"login": {
 	    "saslFailed": "Ungültige Logindaten",
@@ -66,7 +105,10 @@
 	    "fileNotEncrypted": "Der Chat ist verschlüsselt, aber die Datei wurde unverschlüsselt übertragen"
 	},
 	"conversation": {
-	    "audioRecordingError": "Fehler beim Fertigstellen der Audioaufnahme"
+	    "audioRecordingError": "Fehler beim Fertigstellen der Audioaufnahme",
+	    "openFileNoAppError": "Keine App vorhanden, um die Datei zu öffnen",
+	    "openFileGenericError": "Fehler beim Öffnen der Datei",
+	    "messageErrorDialogTitle": "Fehler"
 	}
     },
     "warnings": {
@@ -93,6 +135,7 @@
 	"conversations": {
 	    "speeddialNewChat": "Neuer chat",
 	    "speeddialJoinGroupchat": "Gruppenchat beitreten",
+		"speeddialAddNoteToSelf": "Notiz an mich",
 	    "overlaySettings": "Einstellungen",
 	    "noOpenChats": "Du hast keine offenen chats",
 	    "startChat": "Einen chat anfangen",
@@ -115,11 +158,20 @@
 	    "edit": "Bearbeiten",
 	    "quote": "Zitieren",
 	    "copy": "Inhalt kopieren",
+	    "addReaction": "Reaktion hinzufügen",
 	    "showError": "Fehler anzeigen",
 	    "showWarning": "Warnung anzeigen",
 	    "addToContacts": "Zu Kontaken hinzufügen",
 	    "addToContactsTitle": "${jid} zu Kontakten hinzufügen",
-	    "addToContactsBody": "Bist du dir sicher, dass du ${jid} zu deinen Kontakten hinzufügen möchtest?"
+	    "addToContactsBody": "Bist du dir sicher, dass du ${jid} zu deinen Kontakten hinzufügen möchtest?",
+	    "stickerPickerNoStickersLine1": "Du hast keine Stickerpacks installiert.",
+	    "stickerPickerNoStickersLine2": "Diese können in den Stickereinstellungen installiert werden.",
+	    "stickerSettings": "Stickereinstellungen",
+	    "newDeviceMessage": "${title} hat ein neues Verschlüsselungsgerät hinzugefügt",
+	    "messageHint": "Nachricht senden...",
+	    "sendImages": "Bilder senden",
+	    "sendFiles": "Dateien senden",
+	    "takePhotos": "Bilder aufnehmen"
 	},
 	"addcontact": {
 	    "title": "Neuen Kontakt hinzufügen",
@@ -141,15 +193,16 @@
 	    "confirmBody": "Einer oder mehr Chats sind unverschlüsselt. Das bedeutet, dass die Dateien dem Server unverschlüsselt vorliegen. Dateien trotzdem senden?"
 	},
 	"profile": {
-	    "self": {
-		"devices": "Geräte"
+	    "general": {
+		"omemo": "Sicherheit",
+		"profile": "Profil",
+		"media": "Medien"
 	    },
 	    "conversation": {
-		"muteChatTooltip": "Chat stummschalten",
-		"unmuteChatTooltip": "Chat lautstellen",
-		"muteChat": "Stummschalten",
-		"unmuteChat": "Lautstellen",
-		"devices": "Geräte"
+		"notifications": "Benachrichtigungen",
+		"notificationsMuted": "Stumm",
+		"notificationsEnabled": "Eingeschaltet",
+		"sharedMedia": "Medien"
 	    },
 	    "owndevices": {
 		"title": "Eigene Geräte",
@@ -165,10 +218,11 @@
 		"recreateOwnDeviceConfirmBody": "Das wird die kryptographische Identität dieses Geräts neu erstellen. Wenn Kontakte die kryptographische Indentität verifiziert haben, dann müssen diese es erneut tun. Fortfahren?"
 	    },
 	    "devices": {
-		"title": "Devices",
-		"recreateSessions": "Rebuild sessions",
-		"recreateSessionsConfirmTitle": "Rebuild sessions?",
-		"recreateSessionsConfirmBody": "This will recreate the cryptographic sessions with your own devices. Use only if your own devices throw decryption errors."
+		"title": "Sicherheit",
+		"recreateSessions": "Sessions zurücksetzen",
+		"recreateSessionsConfirmTitle": "Sessions zurücksetzen?",
+		"recreateSessionsConfirmBody": "Dies wird alle Sessions mit Deinen Geräten neu erstellen. Tue dies nur, wenn deine Geräte Fehler beim Entschlüsseln erzeugen.",
+		"noSessions": "Es sind keine kryptographischen Sessions vorhanden, die für Ende-zu-Ende-Verschlüsselung verwendet werden."
 	    }
 	},
 	"blocklist": {
@@ -184,6 +238,14 @@
 	    "blur": "Hintergrund weichzeichnen",
 	    "setAsBackground": "Als Hintergrundbild festlegen"
 	},
+	"stickerPack": {
+	    "removeConfirmTitle": "Stickerpack entfernen",
+	    "removeConfirmBody": "Bist Du Dir sicher, dass du das Stickerpack entfernen möchtest?",
+	    "installConfirmTitle": "Stickerpack installieren",
+	    "installConfirmBody": "Bist Du Dir sicher, dass Du das Stickerpack installieren möchtest?",
+	    "restricted": "Dieses Stickerpack ist eingeschränkt. Das bedeutet, dass es im Chat angezeigt wird, jedoch nicht versendet werden kann.",
+	    "fetchingFailure": "Konnte das Stickerpack nicht finden"
+	},
 	"settings": {
 	    "settings": {
 		"title": "Einstellungen",
@@ -193,13 +255,17 @@
 		"signOutConfirmTitle": "Abmelden",
 		"signOutConfirmBody": "Du bist dabei dich abzumelden. Fortfahren?",
 		"miscellaneousSection": "Unterschiedlich",
-		"debuggingSection": "Debugging"
+		"debuggingSection": "Debugging",
+		"general": "Generell"
 	    },
 	    "about": {
 		"title": "Über",
 		"licensed": "Lizensiert unter GPL3",
 		"version": "Version ${version}",
-		"viewSourceCode": "Quellcode anschauen"
+		"viewSourceCode": "Quellcode anschauen",
+		"nMoreToGo": "Noch ${n}...",
+		"debugMenuShown": "Du bist jetzt ein Entwickler!",
+		"debugMenuAlreadyShown": "Du bist bereits ein Entwickler!"
 	    },
 	    "appearance": {
 		"title": "Aussehen",
@@ -222,7 +288,10 @@
 		"removeBackgroundImageConfirmBody": "Bist Du dir sicher, dass Du das Hintergrundbild entfernen möchtest?",
 		"newChatsSection": "Neue Chats",
 		"newChatsMuteByDefault": "Neue Chats standardmäßig stummschalten",
-		"newChatsE2EE": "Ende-zu-Ende-Verschlüsselung standardmäßig aktivieren. WARNUNG: Experimentell"
+		"newChatsE2EE": "Ende-zu-Ende-Verschlüsselung standardmäßig aktivieren. WARNUNG: Experimentell",
+		"behaviourSection": "Verhalten",
+		"contactsIntegration": "Kontaktintegration",
+		"contactsIntegrationBody": "Wenn aktiviert, dann werden Kontakte aus dem Kontaktbuch verwendet, um Chatnamen und Profilbilder anzuzeigen. Dabei werden keine Daten an den Server gesendet."
 	    },
 	    "debugging": {
 		"title": "Debuggingoptionen",
@@ -241,6 +310,7 @@
 		"automaticDownloadsText": "Moxxy läd Dateien automatisch herunter, wenn verbunden mit...",
 		"automaticDownloadsMaximumSize": "Maximale Downloadgröße",
 		"automaticDownloadsMaximumSizeSubtext": "Die maximale Dateigröße, die automatisch heruntergeladen werden soll",
+		"automaticDownloadAlways": "Immer",
 		"wifi": "Wifi",
 		"mobileData": "Mobile Daten"
 	    },
@@ -251,8 +321,6 @@
 		"showContactRequestsSubtext": "Dies zeigt Personen in der Chatübersicht an, die Dich zu ihrer Kontaktliste hinzugefügt haben, aber noch keine Nachricht gesendet haben",
 		"profilePictureVisibility": "Öffentliches Profilbild",
 		"profilePictureVisibilitSubtext": "Wenn aktiviert, dann kann jeder Dein Profilbild sehen. Wenn deaktiviert, dann können nur Personen aus deiner Kontaktliste kein Profilbild sehen",
-		"autoAcceptSubscriptionRequests": "Subscriptionanfragen automatisch annehmen",
-		"autoAcceptSubscriptionRequestsSubtext": "Wenn aktiviert, dann werden Subscriptionanfragen automatisch angenommen, wenn die Person in deiner Kontaktliste ist",
 		"conversationsSection": "Unterhaltungen",
 		"sendChatMarkers": "Chatmarker senden",
 		"sendChatMarkersSubtext": "Dies teilt Deinem Gesprächspartner mit, ob du Nachrichten empfangen oder gelesen hast",
@@ -266,7 +334,20 @@
 		"cannotEnableRedirectSubtext": "Du must zuerst einen Proxydienst auswählen. Dazu berühre das Feld neben dem Schalter.",
 		"urlEmpty": "URL kann nicht leer sein",
 		"urlInvalid": "Ungültige URL",
-		"redirectDialogTitle": "${serviceName}weiterleitung"
+		"redirectDialogTitle": "${serviceName}weiterleitung",
+		"stickersPrivacy": "Stickerliste öffentlich halten",
+		"stickersPrivacySubtext": "Wenn eingeschaltet, dann kann jeder die Liste Deiner installierten Stickerpacks sehen."
+	    },
+	    "stickers": {
+		"title": "Stickers",
+		"stickerSection": "Sticker",
+		"displayStickers": "Sticker im Chat anzeigen",
+		"autoDownload": "Sticker automatisch herunterladen",
+		"autoDownloadBody": "Wenn aktiviert, dann werden Sticker automatisch heruntergeladen, wenn der Sender in der Kontaktliste ist.",
+		"stickerPacksSection": "Stickerpacks",
+		"importStickerPack": "Stickerpack importieren",
+		"importSuccess": "Stickerpack erfolgreich importiert",
+		"importFailure": "Beim Import des Stickerpacks ist ein Fehler aufgetreten"
 	    }
 	}
     }

docs/stickerpacks.md

@@ -0,0 +1,43 @@
+# Sticker Packs
+
+Moxxy supports sending and receiving sticker packs using XEP-0449 version 0.1.1. Sticker
+packs can also be imported using a Moxxy specific format.
+
+## File Format
+
+A Moxxy sticker pack is a flat tar archive that contains the following files:
+
+- `urn.xmpp.stickers.0.xml`
+- The sticker files
+
+### `urn.xmpp.stickers.0.xml`
+
+This file is the sticker pack's metadata file. It describes the sticker pack the same
+way as the examples in XEP-0449 do. There are, however, some differences:
+
+- Each `<file />` element must contain a `<name />` element that matches with a file in the tar archive
+- Each sticker MUST contain at least one HTTP(s) source
+- The `<hash />` of the `<pack />` element is ignored as Moxxy computes it itself, so it can be omitted
+
+An example for the metadata file is the following:
+
+```xml
+<pack xmlns='urn:xmpp:stickers:0'>
+    <name>Example</name>
+    <summary>Example sticker pack.</summary>
+    <item>
+        <file xmlns='urn:xmpp:file:metadata:0'>
+            <media-type>image/png</media-type>
+            <desc>:some-sticker:</desc>
+            <name>suprise.png</name>
+            <size>531910</size>
+            <dimensions>1030x1030</dimensions>
+            <hash xmlns='urn:xmpp:hashes:2' algo='sha-256'>1Ha4okUGNRAA04KibwWUmklqqBqdhg7+20dfsr/wLik=</hash>
+        </file>
+        <sources xmlns='urn:xmpp:sfs:0'>
+            <url-data xmlns='http://jabber.org/protocol/url-data' target='...' />
+        </sources>
+    </item>
+	<!-- ... -->
+</pack>
+```

fastlane/metadata/android/en-US/changelogs/9.txt

@@ -0,0 +1,7 @@
+* Expose the debug menu by tapping the Moxxy icon on the about page 10 times
+* Maybe fix a connection race condition
+* Allow sharing media with the app when it was closed
+* Make quotes prettier
+* Make the bottom part of the conversation page prettier
+* Fix roster fetching
+* Fix OMEMO key generation

fastlane/metadata/android/en-US/full_description.txt

@@ -10,12 +10,14 @@ Currently supported features include:
   <li>Typing indicators and message markers</li>
   <li>Chat backgrounds</li>
   <li>Runs in the background without Push Notifications</li>
+  <li>OMEMO (Currently not compatible with most apps)</li>
+  <li>Stickers</li>
 </ul>
 
 For the best experience, I recommend a server that:
 <ul>
   <li>Supports direct TLS/StartTLS on the same domain as in the Jid</li>
-  <li>Supports SCRAM-SHA-1 or SCRAM-SHA-256</li>
+  <li>Supports SCRAM-SHA-1, SCRAM-SHA-256 or SCRAM-SHA-512</li>
   <li>Supports HTTP File Upload</li>
   <li>Supports Stream Management</li>
   <li>Supports Client State Indication</li>

flake.lock

@@ -17,16 +17,16 @@
     },
     "nixpkgs": {
       "locked": {
-        "lastModified": 1669165918,
-        "narHash": "sha256-hIVruk2+0wmw/Kfzy11rG3q7ev3VTi/IKVODeHcVjFo=",
-        "owner": "NixOS",
+        "lastModified": 1676076353,
+        "narHash": "sha256-mdUtE8Tp40cZETwcq5tCwwLqkJVV1ULJQ5GKRtbshag=",
+        "owner": "AtaraxiaSjel",
         "repo": "nixpkgs",
-        "rev": "3b400a525d92e4085e46141ff48cbf89fd89739e",
+        "rev": "5deb99bdccbbb97e7562dee4ba8a3ee3021688e6",
         "type": "github"
       },
       "original": {
-        "owner": "NixOS",
-        "ref": "nixpkgs-unstable",
+        "owner": "AtaraxiaSjel",
+        "ref": "update/flutter",
         "repo": "nixpkgs",
         "type": "github"
       }

flake.nix

@@ -1,7 +1,7 @@
 {
   description = "Moxxy v2";
   inputs = {
-    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
+    nixpkgs.url = "github:AtaraxiaSjel/nixpkgs/update/flutter";
     flake-utils.url = "github:numtide/flake-utils";
   };
 
@@ -29,7 +29,7 @@
       useGoogleAPIs = false;
       useGoogleTVAddOns = false;
     };
-    pinnedJDK = pkgs.jdk;
+    pinnedJDK = pkgs.jdk17;
 
     pythonEnv = pkgs.python3.withPackages (ps: with ps; [
       requests pyyaml # For the build scripts

integration_test/backoff_test.dart

@@ -1,67 +0,0 @@
-import 'package:get_it/get_it.dart';
-import 'package:logging/logging.dart';
-import 'package:moxxmpp/moxxmpp.dart';
-import 'package:moxxmpp_socket_tcp/moxxmpp_socket_tcp.dart';
-import 'package:moxxyv2/service/connectivity.dart';
-import 'package:moxxyv2/service/moxxmpp/reconnect.dart';
-import 'package:connectivity_plus/connectivity_plus.dart';
-import 'package:test/test.dart';
-
-class StubConnectivityService extends ConnectivityService {
-  StubConnectivityService() : super();
-
-  @override
-  ConnectivityResult get currentState => ConnectivityResult.wifi;
-}
-
-void main() {
-  Logger.root.level = Level.ALL;
-  Logger.root.onRecord.listen((record) {
-    print('${record.level.name}: ${record.time}: ${record.message}');
-  });
-  final log = Logger('FailureReconnectionTest');
-  GetIt.I.registerSingleton<ConnectivityService>(StubConnectivityService());
-
-  test('Failing an awaited connection with MoxxyReconnectionPolicy', () async {
-    var errors = 0;
-    final connection = XmppConnection(
-      MoxxyReconnectionPolicy(maxBackoffTime: 1),
-      TCPSocketWrapper(false),
-    );
-    connection.registerFeatureNegotiators([
-      StartTlsNegotiator(),
-    ]);
-    connection.registerManagers([
-      DiscoManager(),
-      RosterManager(),
-      PingManager(),
-      MessageManager(),
-      PresenceManager('http://moxxmpp.example'),
-    ]);
-    connection.asBroadcastStream().listen((event) {
-      if (event is ConnectionStateChangedEvent) {
-        if (event.state == XmppConnectionState.error) {
-          errors++;
-        }
-      }
-    });
-
-    connection.setConnectionSettings(
-      ConnectionSettings(
-        jid: JID.fromString('testuser@no-sasl.badxmpp.eu'),
-        password: 'abc123',
-        useDirectTLS: true,
-        allowPlainAuth: true,
-      ),
-    );
-
-    final result = await connection.connectAwaitable();
-    log.info('Connection failed as expected');
-    expect(result.success, false);
-    expect(errors, 1);
-
-    log.info('Waiting 20 seconds for unexpected reconnections');
-    await Future.delayed(const Duration(seconds: 20));
-    expect(errors, 1);
-  }, timeout: Timeout.factor(2));
-}

lib/data_classes.yaml

@@ -36,14 +36,8 @@ files:
           roster:
             type: List<RosterItem>?
             deserialise: true
-      # Returned by [GetMessagesForJidCommand]
-      - name: MessagesResultEvent
-        extends: BackgroundEvent
-        implements:
-          - JsonImplementation
-        attributes:
-          messages:
-            type: List<Message>
+          stickers:
+            type: List<StickerPack>?
             deserialise: true
       # Triggered if a conversation has been added.
       # Also returned by [AddConversationCommand]
@@ -71,7 +65,7 @@ files:
         extends: BackgroundEvent
         implements:
           - JsonImplementation
-      # Send by the service if a message has been received or returned by # [SendMessageCommand].
+      # Send by the service if a message has been received or returned by [SendMessageCommand].
       - name: MessageAddedEvent
         extends: BackgroundEvent
         implements:
@@ -103,6 +97,13 @@ files:
         extends: BackgroundEvent
         implements:
           - JsonImplementation
+      # Triggered in response to a [GetBlocklistCommand]
+      - name: GetBlocklistResultEvent
+        extends: BackgroundEvent
+        implements:
+          - JsonImplementation
+        attributes:
+          entries: List<String>
       # Triggered by DownloadService or UploadService.
       - name: ProgressEvent
         extends: BackgroundEvent
@@ -208,6 +209,71 @@ files:
           conversationJid: String
           title: String
           avatarUrl: String
+      - name: StickerPackImportSuccessEvent
+        extends: BackgroundEvent
+        implements:
+          - JsonImplementation
+        attributes:
+          stickerPack:
+            type: StickerPack
+            deserialise: true
+      - name: StickerPackImportFailureEvent
+        extends: BackgroundEvent
+        implements:
+          - JsonImplementation
+        attributes:
+      - name: FetchStickerPackSuccessResult
+        extends: BackgroundEvent
+        implements:
+          - JsonImplementation
+        attributes:
+          stickerPack:
+            type: StickerPack
+            deserialise: true
+      - name: FetchStickerPackFailureResult
+        extends: BackgroundEvent
+        implements:
+          - JsonImplementation
+        attributes:
+      - name: StickerPackInstallSuccessEvent
+        extends: BackgroundEvent
+        implements:
+          - JsonImplementation
+        attributes:
+          stickerPack:
+            type: StickerPack
+            deserialise: true
+      - name: StickerPackInstallFailureEvent
+        extends: BackgroundEvent
+        implements:
+          - JsonImplementation
+        attributes:
+      - name: StickerPackAddedEvent
+        extends: BackgroundEvent
+        implements:
+          - JsonImplementation
+        attributes:
+          stickerPack:
+            type: StickerPack
+            deserialise: true
+      # Returned by [GetPagedMessagesCommand]
+      - name: PagedMessagesResultEvent
+        extends: BackgroundEvent
+        implements:
+          - JsonImplementation
+        attributes:
+          messages:
+            type: List<Message>
+            deserialise: true
+      # Returned by [GetReactionsForMessageCommand]
+      - name: ReactionsForMessageResult
+        extends: BackgroundEvent
+        implements:
+          - JsonImplementation
+        attributes:
+          reactions:
+            type: List<ReactionGroup>
+            deserialise: true
     generate_builder: true
     builder_name: "Event"
     builder_baseclass: "BackgroundEvent"
@@ -237,12 +303,7 @@ files:
           lastMessageBody: String
           avatarUrl: String
           jid: String
-      - name: GetMessagesForJidCommand
-        extends: BackgroundCommand
-        implements:
-          - JsonImplementation
-        attributes:
-          jid: String
+          conversationType: String
       - name: SetOpenConversationCommand
         extends: BackgroundCommand
         implements:
@@ -262,6 +323,7 @@ files:
             deserialise: true
           editSid: String?
           editId: int?
+          currentConversationJid: String?
       - name: SendFilesCommand
         extends: BackgroundCommand
         implements:
@@ -305,6 +367,12 @@ files:
           - JsonImplementation
         attributes:
           jid: String
+      - name: RemoveContactCommand
+        extends: BackgroundCommand
+        implements:
+          - JsonImplementation
+        attributes:
+          jid: String
       - name: RequestDownloadCommand
         extends: BackgroundCommand
         implements:
@@ -410,7 +478,7 @@ files:
         implements:
           - JsonImplementation
         attributes:
-          conversationId: int
+          conversationJid: String
       - name: MarkMessageAsReadCommand
         extends: BackgroundCommand
         implements:
@@ -419,6 +487,95 @@ files:
           conversationJid: String
           sid: String
           newUnreadCounter: int
+      - name: AddReactionToMessageCommand
+        extends: BackgroundCommand
+        implements:
+          - JsonImplementation
+        attributes:
+          messageId: int
+          conversationJid: String
+          emoji: String
+      - name: RemoveReactionFromMessageCommand
+        extends: BackgroundCommand
+        implements:
+          - JsonImplementation
+        attributes:
+          messageId: int
+          conversationJid: String
+          emoji: String
+      - name: MarkOmemoDeviceAsVerifiedCommand
+        extends: BackgroundCommand
+        implements:
+          - JsonImplementation
+        attributes:
+          deviceId: int
+          jid: String
+      - name: ImportStickerPackCommand
+        extends: BackgroundCommand
+        implements:
+          - JsonImplementation
+        attributes:
+          path: String
+      - name: RemoveStickerPackCommand
+        extends: BackgroundCommand
+        implements:
+          - JsonImplementation
+        attributes:
+          stickerPackId: String
+      - name: SendStickerCommand
+        extends: BackgroundCommand
+        implements:
+          - JsonImplementation
+        attributes:
+          sticker:
+            type: Sticker
+            deserialise: true
+          recipient: String
+          quotes:
+            type: Message?
+            deserialise: true
+      - name: FetchStickerPackCommand
+        extends: BackgroundCommand
+        implements:
+          - JsonImplementation
+        attributes:
+          stickerPackId: String
+          jid: String
+      - name: InstallStickerPackCommand
+        extends: BackgroundCommand
+        implements:
+          - JsonImplementation
+        attributes:
+          stickerPack:
+            type: StickerPack
+            deserialise: true
+      - name: GetBlocklistCommand
+        extends: BackgroundCommand
+        implements:
+          - JsonImplementation
+        attributes:
+      - name: GetPagedMessagesCommand
+        extends: BackgroundCommand
+        implements:
+          - JsonImplementation
+        attributes:
+          conversationJid: String
+          olderThan: bool
+          timestamp: int?
+      - name: GetPagedSharedMediaCommand
+        extends: BackgroundCommand
+        implements:
+          - JsonImplementation
+        attributes:
+          conversationJid: String
+          olderThan: bool
+          timestamp: int?
+      - name: GetReactionsForMessageCommand
+        extends: BackgroundCommand
+        implements:
+          - JsonImplementation
+        attributes:
+          messageId: int
     generate_builder: true
     # get${builder_Name}FromJson
     builder_name: "Command"

lib/main.dart

@@ -26,8 +26,10 @@ import 'package:moxxyv2/ui/bloc/profile_bloc.dart';
 import 'package:moxxyv2/ui/bloc/sendfiles_bloc.dart';
 import 'package:moxxyv2/ui/bloc/server_info_bloc.dart';
 import 'package:moxxyv2/ui/bloc/share_selection_bloc.dart';
-import 'package:moxxyv2/ui/bloc/sharedmedia_bloc.dart';
+import 'package:moxxyv2/ui/bloc/sticker_pack_bloc.dart';
+import 'package:moxxyv2/ui/bloc/stickers_bloc.dart';
 import 'package:moxxyv2/ui/constants.dart';
+import 'package:moxxyv2/ui/controller/conversation_controller.dart';
 import 'package:moxxyv2/ui/events.dart';
 /*
 import "package:moxxyv2/ui/pages/register/register.dart";
@@ -55,21 +57,25 @@ import 'package:moxxyv2/ui/pages/settings/licenses.dart';
 import 'package:moxxyv2/ui/pages/settings/network.dart';
 import 'package:moxxyv2/ui/pages/settings/privacy/privacy.dart';
 import 'package:moxxyv2/ui/pages/settings/settings.dart';
+import 'package:moxxyv2/ui/pages/settings/stickers.dart';
 import 'package:moxxyv2/ui/pages/share_selection.dart';
-import 'package:moxxyv2/ui/pages/sharedmedia.dart';
+//import 'package:moxxyv2/ui/pages/sharedmedia.dart';
 import 'package:moxxyv2/ui/pages/splashscreen/splashscreen.dart';
+import 'package:moxxyv2/ui/pages/sticker_pack.dart';
 import 'package:moxxyv2/ui/pages/util/qrcode.dart';
 import 'package:moxxyv2/ui/service/data.dart';
 import 'package:moxxyv2/ui/service/progress.dart';
+import 'package:moxxyv2/ui/service/sharing.dart';
 import 'package:moxxyv2/ui/theme.dart';
 import 'package:page_transition/page_transition.dart';
-import 'package:share_handler/share_handler.dart';
 
 void setupLogging() {
   Logger.root.level = kDebugMode ? Level.ALL : Level.INFO;
   Logger.root.onRecord.listen((record) {
     // ignore: avoid_print
-    print('[${record.level.name}] (${record.loggerName}) ${record.time}: ${record.message}');
+    print(
+      '[${record.level.name}] (${record.loggerName}) ${record.time}: ${record.message}',
+    );
   });
   GetIt.I.registerSingleton<Logger>(Logger('MoxxyMain'));
 }
@@ -77,17 +83,19 @@ void setupLogging() {
 Future<void> setupUIServices() async {
   GetIt.I.registerSingleton<UIProgressService>(UIProgressService());
   GetIt.I.registerSingleton<UIDataService>(UIDataService());
+  GetIt.I.registerSingleton<UISharingService>(UISharingService());
 }
 
 void setupBlocs(GlobalKey<NavigatorState> navKey) {
-  GetIt.I.registerSingleton<NavigationBloc>(NavigationBloc(navigationKey: navKey));
+  GetIt.I
+      .registerSingleton<NavigationBloc>(NavigationBloc(navigationKey: navKey));
   GetIt.I.registerSingleton<ConversationsBloc>(ConversationsBloc());
   GetIt.I.registerSingleton<NewConversationBloc>(NewConversationBloc());
   GetIt.I.registerSingleton<ConversationBloc>(ConversationBloc());
-  GetIt.I.registerSingleton<BlocklistBloc>(BlocklistBloc()); GetIt.I.registerSingleton<ProfileBloc>(ProfileBloc());
+  GetIt.I.registerSingleton<BlocklistBloc>(BlocklistBloc());
+  GetIt.I.registerSingleton<ProfileBloc>(ProfileBloc());
   GetIt.I.registerSingleton<PreferencesBloc>(PreferencesBloc());
   GetIt.I.registerSingleton<AddContactBloc>(AddContactBloc());
-  GetIt.I.registerSingleton<SharedMediaBloc>(SharedMediaBloc());
   GetIt.I.registerSingleton<CropBloc>(CropBloc());
   GetIt.I.registerSingleton<SendFilesBloc>(SendFilesBloc());
   GetIt.I.registerSingleton<CropBackgroundBloc>(CropBackgroundBloc());
@@ -95,11 +103,10 @@ void setupBlocs(GlobalKey<NavigatorState> navKey) {
   GetIt.I.registerSingleton<ServerInfoBloc>(ServerInfoBloc());
   GetIt.I.registerSingleton<DevicesBloc>(DevicesBloc());
   GetIt.I.registerSingleton<OwnDevicesBloc>(OwnDevicesBloc());
+  GetIt.I.registerSingleton<StickersBloc>(StickersBloc());
+  GetIt.I.registerSingleton<StickerPackBloc>(StickerPackBloc());
 }
 
-// TODO(Unknown): Replace all Column(children: [ Padding(), Padding, ...]) with a
-//                Padding(padding: ..., child: Column(children: [ ... ]))
-// TODO(Unknown): Theme the switches
 void main() async {
   setupLogging();
   await setupUIServices();
@@ -111,6 +118,8 @@ void main() async {
 
   await initializeServiceIfNeeded();
 
+  imageCache.maximumSizeBytes = 500 * 1024 * 1024;
+
   runApp(
     MultiBlocProvider(
       providers: [
@@ -141,9 +150,6 @@ void main() async {
         BlocProvider<AddContactBloc>(
           create: (_) => GetIt.I.get<AddContactBloc>(),
         ),
-        BlocProvider<SharedMediaBloc>(
-          create: (_) => GetIt.I.get<SharedMediaBloc>(),
-        ),
         BlocProvider<CropBloc>(
           create: (_) => GetIt.I.get<CropBloc>(),
         ),
@@ -165,6 +171,12 @@ void main() async {
         BlocProvider<OwnDevicesBloc>(
           create: (_) => GetIt.I.get<OwnDevicesBloc>(),
         ),
+        BlocProvider<StickersBloc>(
+          create: (_) => GetIt.I.get<StickersBloc>(),
+        ),
+        BlocProvider<StickerPackBloc>(
+          create: (_) => GetIt.I.get<StickerPackBloc>(),
+        ),
       ],
       child: TranslationProvider(
         child: MyApp(navKey),
@@ -174,7 +186,6 @@ void main() async {
 }
 
 class MyApp extends StatefulWidget {
-
   const MyApp(this.navigationKey, {super.key});
   final GlobalKey<NavigatorState> navigationKey;
 
@@ -188,46 +199,20 @@ class MyAppState extends State<MyApp> with WidgetsBindingObserver {
   @override
   void initState() {
     super.initState();
+    _initState();
+  }
+
+  /// Async "version" of initState()
+  Future<void> _initState() async {
     WidgetsBinding.instance.addObserver(this);
 
-    _setupSharingHandler();
+    // Set up receiving share intents
+    await GetIt.I.get<UISharingService>().initialize();
 
     // Lift the UI block
-    GetIt.I.get<SynchronizedQueue<Map<String, dynamic>?>>().removeQueueLock();
-  }
-
-  Future<void> _handleSharedMedia(SharedMedia media) async {
-    final attachments = media.attachments ?? [];
-    GetIt.I.get<ShareSelectionBloc>().add(
-      ShareSelectionRequestedEvent(
-        attachments.map((a) => a!.path).toList(),
-        media.content,
-        media.content != null ? ShareSelectionType.text : ShareSelectionType.media,
-      ),
-    );
-  }
-  
-  Future<void> _setupSharingHandler() async {
-    final handler = ShareHandlerPlatform.instance;
-    final media = await handler.getInitialSharedMedia();
-
-    // Shared while the app was closed
-    if (media != null) {
-      if (GetIt.I.get<UIDataService>().isLoggedIn) {
-        await _handleSharedMedia(media);
-      }
-
-      await handler.resetInitialSharedMedia();
-    }
-
-    // Shared while the app is stil running
-    handler.sharedMediaStream.listen((SharedMedia media) async {
-      if (GetIt.I.get<UIDataService>().isLoggedIn) {
-        await _handleSharedMedia(media);
-      }
-
-      await handler.resetInitialSharedMedia();
-    });
+    await GetIt.I
+        .get<SynchronizedQueue<Map<String, dynamic>?>>()
+        .removeQueueLock();
   }
 
   @override
@@ -246,13 +231,15 @@ class MyAppState extends State<MyApp> with WidgetsBindingObserver {
         sender.sendData(
           SetCSIStateCommand(active: false),
         );
-        GetIt.I.get<ConversationBloc>().add(AppStateChanged(false));
+        BidirectionalConversationController.currentController
+            ?.handleAppStateChange(false);
         break;
       case AppLifecycleState.resumed:
         sender.sendData(
           SetCSIStateCommand(active: true),
         );
-        GetIt.I.get<ConversationBloc>().add(AppStateChanged(true));
+        BidirectionalConversationController.currentController
+            ?.handleAppStateChange(true);
         break;
       case AppLifecycleState.detached:
       case AppLifecycleState.inactive:
@@ -272,37 +259,72 @@ class MyAppState extends State<MyApp> with WidgetsBindingObserver {
       navigatorKey: widget.navigationKey,
       onGenerateRoute: (settings) {
         switch (settings.name) {
-          case introRoute: return Intro.route;
-          case loginRoute: return Login.route;
-          case conversationsRoute: return ConversationsPage.route;
-          case newConversationRoute: return NewConversationPage.route;
-          case conversationRoute: return PageTransition<dynamic>(
+          case introRoute:
+            return Intro.route;
+          case loginRoute:
+            return Login.route;
+          case conversationsRoute:
+            return ConversationsPage.route;
+          case newConversationRoute:
+            return NewConversationPage.route;
+          case conversationRoute:
+            return PageTransition<dynamic>(
               type: PageTransitionType.rightToLeft,
               settings: settings,
-            child: const ConversationPage(),
+              child: ConversationPage(
+                conversationJid: settings.arguments! as String,
+              ),
             );
-          case sharedMediaRoute: return SharedMediaPage.route;
-          case blocklistRoute: return BlocklistPage.route;
-          case profileRoute: return ProfilePage.route;
-          case settingsRoute: return SettingsPage.route;
-          case aboutRoute: return SettingsAboutPage.route;
-          case licensesRoute: return SettingsLicensesPage.route;
-          case networkRoute: return NetworkPage.route;
-          case privacyRoute: return PrivacyPage.route;
-          case debuggingRoute: return DebuggingPage.route;
-          case addContactRoute: return AddContactPage.route;
-          case cropRoute: return CropPage.route;
-          case sendFilesRoute: return SendFilesPage.route;
-          case backgroundCroppingRoute: return CropBackgroundPage.route;
-          case shareSelectionRoute: return ShareSelectionPage.route;
-          case serverInfoRoute: return ServerInfoPage.route;
-          case conversationSettingsRoute: return ConversationSettingsPage.route;
-          case devicesRoute: return DevicesPage.route;
-          case ownDevicesRoute: return OwnDevicesPage.route;
-          case appearanceRoute: return AppearanceSettingsPage.route;
-          case qrCodeScannerRoute: return QrCodeScanningPage.getRoute(
+          // case sharedMediaRoute:
+          //   return SharedMediaPage.getRoute(
+          //     settings.arguments! as SharedMediaPageArguments,
+          //   );
+          case blocklistRoute:
+            return BlocklistPage.route;
+          case profileRoute:
+            return ProfilePage.getRoute(
+              settings.arguments! as ProfileArguments,
+            );
+          case settingsRoute:
+            return SettingsPage.route;
+          case aboutRoute:
+            return SettingsAboutPage.route;
+          case licensesRoute:
+            return SettingsLicensesPage.route;
+          case networkRoute:
+            return NetworkPage.route;
+          case privacyRoute:
+            return PrivacyPage.route;
+          case debuggingRoute:
+            return DebuggingPage.route;
+          case addContactRoute:
+            return AddContactPage.route;
+          case cropRoute:
+            return CropPage.route;
+          case sendFilesRoute:
+            return SendFilesPage.route;
+          case backgroundCroppingRoute:
+            return CropBackgroundPage.route;
+          case shareSelectionRoute:
+            return ShareSelectionPage.route;
+          case serverInfoRoute:
+            return ServerInfoPage.route;
+          case conversationSettingsRoute:
+            return ConversationSettingsPage.route;
+          case devicesRoute:
+            return DevicesPage.route;
+          case ownDevicesRoute:
+            return OwnDevicesPage.route;
+          case appearanceRoute:
+            return AppearanceSettingsPage.route;
+          case qrCodeScannerRoute:
+            return QrCodeScanningPage.getRoute(
               settings.arguments! as QrCodeScanningArguments,
             );
+          case stickersRoute:
+            return StickersSettingsPage.route;
+          case stickerPackRoute:
+            return StickerPackPage.route;
         }
 
         return null;

lib/service/avatars.dart

@@ -4,13 +4,12 @@ import 'package:cryptography/cryptography.dart';
 import 'package:get_it/get_it.dart';
 import 'package:hex/hex.dart';
 import 'package:logging/logging.dart';
-import 'package:moxlib/moxlib.dart';
 import 'package:moxxmpp/moxxmpp.dart';
 import 'package:moxxyv2/service/conversation.dart';
 import 'package:moxxyv2/service/preferences.dart';
 import 'package:moxxyv2/service/roster.dart';
 import 'package:moxxyv2/service/service.dart';
-import 'package:moxxyv2/service/xmpp.dart';
+import 'package:moxxyv2/service/xmpp_state.dart';
 import 'package:moxxyv2/shared/avatar.dart';
 import 'package:moxxyv2/shared/events.dart';
 import 'package:moxxyv2/shared/helpers.dart';
@@ -26,56 +25,60 @@ String _cleanBase64String(String original) {
   return ret;
 }
 
+class _AvatarData {
+  const _AvatarData(this.data, this.id);
+  final List<int> data;
+  final String id;
+}
+
 class AvatarService {
+  final Logger _log = Logger('AvatarService');
 
-  AvatarService() : _log = Logger('AvatarService');
-  final Logger _log;
+  Future<void> handleAvatarUpdate(AvatarUpdatedEvent event) async {
+    await updateAvatarForJid(
+      event.jid,
+      event.hash,
+      base64Decode(_cleanBase64String(event.base64)),
+    );
+  }
 
-  UserAvatarManager _getUserAvatarManager() => GetIt.I.get<XmppConnection>().getManagerById<UserAvatarManager>(userAvatarManager)!;
-
-  DiscoManager _getDiscoManager() => GetIt.I.get<XmppConnection>().getManagerById<DiscoManager>(discoManager)!;
-  
-  Future<void> updateAvatarForJid(String jid, String hash, String base64) async {
+  Future<void> updateAvatarForJid(
+    String jid,
+    String hash,
+    List<int> data,
+  ) async {
     final cs = GetIt.I.get<ConversationService>();
     final rs = GetIt.I.get<RosterService>();
     final originalConversation = await cs.getConversationByJid(jid);
-    var saved = false;
+    final originalRoster = await rs.getRosterItemByJid(jid);
+
+    if (originalConversation == null && originalRoster == null) return;
 
-    // Clean the raw data. Since this may arrive by chunks, those chunks may contain
-    // weird data pieces.
-    final base64Data = base64Decode(_cleanBase64String(base64));
-    if (originalConversation != null) {
     final avatarPath = await saveAvatarInCache(
-        base64Data,
+      data,
       hash,
       jid,
-        originalConversation.avatarUrl,
+      (originalConversation?.avatarUrl ?? originalRoster?.avatarUrl)!,
     );
-      saved = true;
-      final conv = await cs.updateConversation(
-        originalConversation.id,
+
+    if (originalConversation != null) {
+      final conversation = await cs.createOrUpdateConversation(
+        jid,
+        update: (c) async {
+          return cs.updateConversation(
+            jid,
             avatarUrl: avatarPath,
           );
-
-      sendEvent(ConversationUpdatedEvent(conversation: conv));
-    } else {
-      _log.warning('Failed to get conversation');
-    }
-
-    final originalRoster = await rs.getRosterItemByJid(jid);
-    if (originalRoster != null) {
-      var avatarPath = '';
-      if (saved) {
-        avatarPath = await getAvatarPath(jid, hash);
-      } else {
-        avatarPath = await saveAvatarInCache(
-          base64Data,
-          hash,
-          jid,
-          originalRoster.avatarUrl,
+        },
+      );
+      if (conversation != null) {
+        sendEvent(
+          ConversationUpdatedEvent(conversation: conversation),
         );
       }
+    }
 
+    if (originalRoster != null) {
       final roster = await rs.updateRosterItem(
         originalRoster.id,
         avatarUrl: avatarPath,
@@ -86,65 +89,78 @@ class AvatarService {
     }
   }
 
-  Future<void> fetchAndUpdateAvatarForJid(String jid, String oldHash) async {
-    final response = await _getDiscoManager().discoItemsQuery(jid);
-    final items = response.isType<DiscoError>() ?
-      <DiscoItem>[] :
-      response.get<List<DiscoItem>>();
-    final itemNodes = items.map((i) => i.node);
+  Future<_AvatarData?> _handleUserAvatar(String jid, String oldHash) async {
+    final am = GetIt.I
+        .get<XmppConnection>()
+        .getManagerById<UserAvatarManager>(userAvatarManager)!;
+    final idResult = await am.getAvatarId(JID.fromString(jid));
+    if (idResult.isType<AvatarError>()) {
+      _log.warning('Failed to get avatar id via XEP-0084 for $jid');
+      return null;
+    }
+    final id = idResult.get<String>();
+    if (id == oldHash) return null;
 
-    _log.finest('Disco items for $jid:');
-    for (final item in itemNodes) {
-      _log.finest('- $item');
+    final avatarResult = await am.getUserAvatar(jid);
+    if (avatarResult.isType<AvatarError>()) {
+      _log.warning('Failed to get avatar data via XEP-0084 for $jid');
+      return null;
+    }
+    final avatar = avatarResult.get<UserAvatar>();
+
+    return _AvatarData(
+      base64Decode(_cleanBase64String(avatar.base64)),
+      avatar.hash,
+    );
   }
 
-    var base64 = '';
-    var hash = '';
-    if (listContains<DiscoItem>(items, (item) => item.node == userAvatarDataXmlns)) {
-      final avatar = _getUserAvatarManager();
-      final pubsubHash = await avatar.getAvatarId(jid);
-
-      // Don't request if we already have the newest avatar
-      if (pubsubHash == oldHash) return;
-      
-      // Query via PubSub
-      final data = await avatar.getUserAvatar(jid);
-      if (data == null) return;
-      
-      base64 = data.base64;
-      hash = data.hash;
-    } else {
+  Future<_AvatarData?> _handleVcardAvatar(String jid, String oldHash) async {
     // Query the vCard
-      final vm = GetIt.I.get<XmppConnection>().getManagerById<VCardManager>(vcardManager)!;
-      final vcard = await vm.requestVCard(jid);
-      if (vcard != null) {
-        final binval = vcard.photo?.binval;
-        if (binval != null) {
-          // Clean the raw data. Since this may arrive by chunks, those chunks may contain
-          // weird data pieces.
-          base64 = _cleanBase64String(binval);
+    final vm = GetIt.I
+        .get<XmppConnection>()
+        .getManagerById<VCardManager>(vcardManager)!;
+    final vcardResult = await vm.requestVCard(jid);
+    if (vcardResult.isType<VCardError>()) return null;
 
-          final rawHash = await Sha1().hash(base64Decode(base64));
-          hash = HEX.encode(rawHash.bytes);
+    final binval = vcardResult.get<VCard>().photo?.binval;
+    if (binval == null) return null;
+
+    final data = base64Decode(_cleanBase64String(binval));
+    final rawHash = await Sha1().hash(data);
+    final hash = HEX.encode(rawHash.bytes);
 
     vm.setLastHash(jid, hash);
-        } else {
-          return;
-        }
-      } else {
-        return;
-      }
+
+    return _AvatarData(
+      data,
+      hash,
+    );
   }
 
-    await updateAvatarForJid(jid, hash, base64);
+  Future<void> fetchAndUpdateAvatarForJid(String jid, String oldHash) async {
+    _AvatarData? data;
+    data ??= await _handleUserAvatar(jid, oldHash);
+    data ??= await _handleVcardAvatar(jid, oldHash);
+
+    if (data != null) {
+      await updateAvatarForJid(jid, data.id, data.data);
+    }
   }
 
   Future<bool> subscribeJid(String jid) async {
-    return _getUserAvatarManager().subscribe(jid);
+    return (await GetIt.I
+            .get<XmppConnection>()
+            .getManagerById<UserAvatarManager>(userAvatarManager)!
+            .subscribe(jid))
+        .isType<bool>();
   }
 
   Future<bool> unsubscribeJid(String jid) async {
-    return _getUserAvatarManager().unsubscribe(jid);
+    return (await GetIt.I
+            .get<XmppConnection>()
+            .getManagerById<UserAvatarManager>(userAvatarManager)!
+            .unsubscribe(jid))
+        .isType<bool>();
   }
 
   /// Publishes the data at [path] as an avatar with PubSub ID
@@ -161,13 +177,23 @@ class AvatarService {
     final imageSize = (await getImageSizeFromData(bytes))!;
 
     // Publish data and metadata
-    final manager = _getUserAvatarManager();
-    await manager.publishUserAvatar(
+    final am = GetIt.I
+        .get<XmppConnection>()
+        .getManagerById<UserAvatarManager>(userAvatarManager)!;
+
+    _log.finest('Publishing avatar...');
+    final dataResult = await am.publishUserAvatar(
       base64,
       hash,
       public,
     );
-    await manager.publishUserAvatarMetadata(
+    if (dataResult.isType<AvatarError>()) {
+      _log.finest('Avatar data publishing failed');
+      return false;
+    }
+
+    // TODO(Unknown): Make sure that the image is not too large.
+    final metadataResult = await am.publishUserAvatarMetadata(
       UserAvatarMetadata(
         hash,
         bytes.length,
@@ -178,39 +204,56 @@ class AvatarService {
       ),
       public,
     );
+    if (metadataResult.isType<AvatarError>()) {
+      _log.finest('Avatar metadata publishing failed');
+      return false;
+    }
 
+    _log.finest('Avatar publishing done');
     return true;
   }
 
   Future<void> requestOwnAvatar() async {
-    final avatar = _getUserAvatarManager();
-    final xmpp = GetIt.I.get<XmppService>();
-    final state = await xmpp.getXmppState();
+    final am = GetIt.I
+        .get<XmppConnection>()
+        .getManagerById<UserAvatarManager>(userAvatarManager)!;
+    final xss = GetIt.I.get<XmppStateService>();
+    final state = await xss.getXmppState();
     final jid = state.jid!;
-    final id = await avatar.getAvatarId(jid);
+    final idResult = await am.getAvatarId(JID.fromString(jid));
+    if (idResult.isType<AvatarError>()) {
+      _log.info('Error while getting latest avatar id for own avatar');
+      return;
+    }
+    final id = idResult.get<String>();
 
     if (id == state.avatarHash) return;
 
-    _log.info('Mismatch between saved avatar data and server-side avatar data about ourself');
-    final data = await avatar.getUserAvatar(jid);
-    if (data == null) {
+    _log.info(
+      'Mismatch between saved avatar data and server-side avatar data about ourself',
+    );
+    final avatarDataResult = await am.getUserAvatar(jid);
+    if (avatarDataResult.isType<AvatarError>()) {
       _log.severe('Failed to fetch our avatar');
       return;
     }
+    final avatarData = avatarDataResult.get<UserAvatar>();
 
     _log.info('Received data for our own avatar');
 
     final avatarPath = await saveAvatarInCache(
-      base64Decode(_cleanBase64String(data.base64)),
-      data.hash,
+      base64Decode(_cleanBase64String(avatarData.base64)),
+      avatarData.hash,
       jid,
       state.avatarUrl,
     );
-    await xmpp.modifyXmppState((state) => state.copyWith(
+    await xss.modifyXmppState(
+      (state) => state.copyWith(
         avatarUrl: avatarPath,
-      avatarHash: data.hash,
-    ),);
+        avatarHash: avatarData.hash,
+      ),
+    );
 
-    sendEvent(SelfAvatarChangedEvent(path: avatarPath, hash: data.hash));
+    sendEvent(SelfAvatarChangedEvent(path: avatarPath, hash: avatarData.hash));
   }
 }

lib/service/blocking.dart

@@ -1,43 +1,125 @@
+import 'dart:async';
 import 'package:get_it/get_it.dart';
+import 'package:logging/logging.dart';
 import 'package:moxxmpp/moxxmpp.dart';
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:moxxyv2/service/database/database.dart';
 import 'package:moxxyv2/service/service.dart';
 import 'package:moxxyv2/shared/events.dart';
 
-enum BlockPushType {
-  block,
-  unblock
-}
+enum BlockPushType { block, unblock }
 
 class BlocklistService {
+  BlocklistService();
+  List<String>? _blocklist;
+  bool _requested = false;
+  bool? _supported;
+  final Logger _log = Logger('BlocklistService');
 
-  BlocklistService() :
-    _blocklistCache = List.empty(growable: true),
-    _requestedBlocklist = false;
-  final List<String> _blocklistCache;
-  bool _requestedBlocklist;
+  Future<void> _removeBlocklistEntry(String jid) async {
+    await GetIt.I.get<DatabaseService>().database.delete(
+      blocklistTable,
+      where: 'jid = ?',
+      whereArgs: [jid],
+    );
+  }
 
-  Future<List<String>> _requestBlocklist() async {
-    final manager = GetIt.I.get<XmppConnection>().getManagerById<BlockingManager>(blockingManager)!;
-    _blocklistCache
-      ..clear()
-      ..addAll(await manager.getBlocklist());
-    _requestedBlocklist = true;
-    return _blocklistCache;
+  Future<void> _addBlocklistEntry(String jid) async {
+    await GetIt.I.get<DatabaseService>().database.insert(
+      blocklistTable,
+      {
+        'jid': jid,
+      },
+    );
+  }
+
+  void onNewConnection() {
+    // Invalidate the caches
+    _blocklist = null;
+    _requested = false;
+    _supported = null;
+  }
+
+  Future<bool> _checkSupport() async {
+    return _supported ??= await GetIt.I
+        .get<XmppConnection>()
+        .getManagerById<BlockingManager>(blockingManager)!
+        .isSupported();
+  }
+
+  Future<void> _requestBlocklist() async {
+    assert(
+      _blocklist != null,
+      'The blocklist must be loaded from the database before requesting',
+    );
+
+    // Check if blocking is supported
+    if (!(await _checkSupport())) {
+      _log.warning('Blocklist requested but server does not support it.');
+      return;
+    }
+
+    final blocklist = await GetIt.I
+        .get<XmppConnection>()
+        .getManagerById<BlockingManager>(blockingManager)!
+        .getBlocklist();
+
+    // Diff the received blocklist with the cache
+    final newItems = List<String>.empty(growable: true);
+    final removedItems = List<String>.empty(growable: true);
+    for (final item in blocklist) {
+      if (!_blocklist!.contains(item)) {
+        await _addBlocklistEntry(item);
+        _blocklist!.add(item);
+        newItems.add(item);
+      }
+    }
+
+    // Diff the cache with the received blocklist
+    for (final item in _blocklist!) {
+      if (!blocklist.contains(item)) {
+        await _removeBlocklistEntry(item);
+        _blocklist!.remove(item);
+        removedItems.add(item);
+      }
+    }
+
+    _requested = true;
+
+    // Trigger an UI event if we have anything to tell the UI
+    if (newItems.isNotEmpty || removedItems.isNotEmpty) {
+      sendEvent(
+        BlocklistPushEvent(
+          added: newItems,
+          removed: removedItems,
+        ),
+      );
+    }
   }
 
   /// Returns the blocklist from the database
   Future<List<String>> getBlocklist() async {
-    if (!_requestedBlocklist) {
-      _blocklistCache
-        ..clear()
-        ..addAll(await _requestBlocklist());
+    if (_blocklist == null) {
+      final blocklistRaw =
+          await GetIt.I.get<DatabaseService>().database.query(blocklistTable);
+      _blocklist = blocklistRaw.map((m) => m['jid']! as String).toList();
+
+      if (!_requested) {
+        unawaited(_requestBlocklist());
       }
 
-    return _blocklistCache;
+      return _blocklist!;
+    }
+
+    if (!_requested) {
+      unawaited(_requestBlocklist());
+    }
+
+    return _blocklist!;
   }
 
   void onUnblockAllPush() {
-    _blocklistCache.clear();
+    _blocklist = List<String>.empty(growable: true);
     sendEvent(
       BlocklistUnblockAllEvent(),
     );
@@ -45,21 +127,27 @@ class BlocklistService {
 
   Future<void> onBlocklistPush(BlockPushType type, List<String> items) async {
     // We will fetch it later when getBlocklist is called
-    if (!_requestedBlocklist) return;
+    if (!_requested) return;
 
     final newBlocks = List<String>.empty(growable: true);
     final removedBlocks = List<String>.empty(growable: true);
     for (final item in items) {
       switch (type) {
-        case BlockPushType.block: {
-          if (_blocklistCache.contains(item)) continue;
-          _blocklistCache.add(item);
+        case BlockPushType.block:
+          {
+            if (_blocklist!.contains(item)) continue;
+            _blocklist!.add(item);
             newBlocks.add(item);
+
+            await _addBlocklistEntry(item);
           }
           break;
-        case BlockPushType.unblock: {
-          _blocklistCache.removeWhere((i) => i == item);
+        case BlockPushType.unblock:
+          {
+            _blocklist!.removeWhere((i) => i == item);
             removedBlocks.add(item);
+
+            await _removeBlocklistEntry(item);
           }
           break;
       }
@@ -74,17 +162,50 @@ class BlocklistService {
   }
 
   Future<bool> blockJid(String jid) async {
-    final manager = GetIt.I.get<XmppConnection>().getManagerById<BlockingManager>(blockingManager)!;
-    return manager.block([ jid ]);
+    // Check if blocking is supported
+    if (!(await _checkSupport())) {
+      _log.warning('Blocking $jid requested but server does not support it.');
+      return false;
+    }
+
+    _blocklist!.add(jid);
+    await _addBlocklistEntry(jid);
+    return GetIt.I
+        .get<XmppConnection>()
+        .getManagerById<BlockingManager>(blockingManager)!
+        .block([jid]);
   }
 
   Future<bool> unblockJid(String jid) async {
-    final manager = GetIt.I.get<XmppConnection>().getManagerById<BlockingManager>(blockingManager)!;
-    return manager.unblock([ jid ]);
+    // Check if blocking is supported
+    if (!(await _checkSupport())) {
+      _log.warning('Unblocking $jid requested but server does not support it.');
+      return false;
+    }
+
+    _blocklist!.remove(jid);
+    await _removeBlocklistEntry(jid);
+    return GetIt.I
+        .get<XmppConnection>()
+        .getManagerById<BlockingManager>(blockingManager)!
+        .unblock([jid]);
   }
 
   Future<bool> unblockAll() async {
-    final manager = GetIt.I.get<XmppConnection>().getManagerById<BlockingManager>(blockingManager)!;
-    return manager.unblockAll();
+    // Check if blocking is supported
+    if (!(await _checkSupport())) {
+      _log.warning(
+        'Unblocking all JIDs requested but server does not support it.',
+      );
+      return false;
+    }
+
+    _blocklist!.clear();
+    await GetIt.I.get<DatabaseService>().database.delete(blocklistTable);
+
+    return GetIt.I
+        .get<XmppConnection>()
+        .getManagerById<BlockingManager>(blockingManager)!
+        .unblockAll();
   }
 }

lib/service/connectivity.dart

@@ -1,22 +1,32 @@
-import 'dart:io' show Platform;
+import 'dart:async';
 import 'package:connectivity_plus/connectivity_plus.dart';
-import 'package:get_it/get_it.dart';
 import 'package:logging/logging.dart';
 import 'package:meta/meta.dart';
-import 'package:moxxmpp/moxxmpp.dart';
-import 'package:moxxyv2/service/httpfiletransfer/httpfiletransfer.dart';
-import 'package:moxxyv2/service/moxxmpp/reconnect.dart';
+
+class ConnectivityEvent {
+  const ConnectivityEvent(this.regained, this.lost);
+  final bool regained;
+  final bool lost;
+}
 
 class ConnectivityService {
-  ConnectivityService() : _log = Logger('ConnectivityService');
-  final Logger _log;
+  /// The internal stream controller
+  final StreamController<ConnectivityEvent> _controller =
+      StreamController<ConnectivityEvent>.broadcast();
+
+  /// The logger
+  final Logger _log = Logger('ConnectivityService');
 
   /// Caches the current connectivity state
   late ConnectivityResult _connectivity;
 
+  Stream<ConnectivityEvent> get stream => _controller.stream;
+
   @visibleForTesting
   void setConnectivity(ConnectivityResult result) {
-    _log.warning('Internal connectivity state changed by request originating from outside ConnectivityService');
+    _log.warning(
+      'Internal connectivity state changed by request originating from outside ConnectivityService',
+    );
     _connectivity = result;
   }
 
@@ -24,23 +34,24 @@ class ConnectivityService {
     final conn = Connectivity();
     _connectivity = await conn.checkConnectivity();
 
-    // TODO(Unknown): At least on Android, the stream fires directly after listening although the
-    //                network does not change. So just skip it.
-    // See https://github.com/fluttercommunity/plus_plugins/issues/567
-    final skipAmount = Platform.isAndroid ? 1 : 0;
-    conn.onConnectivityChanged.skip(skipAmount).listen((ConnectivityResult result) {
-      final regained = _connectivity == ConnectivityResult.none && result != ConnectivityResult.none;
+    conn.onConnectivityChanged.listen((ConnectivityResult result) {
+      final regained = _connectivity == ConnectivityResult.none &&
+          result != ConnectivityResult.none;
       final lost = result == ConnectivityResult.none;
       _connectivity = result;
 
-      // TODO(PapaTutuWawa): Should we use Streams?
-      // Notify other services
-      (GetIt.I.get<XmppConnection>().reconnectionPolicy as MoxxyReconnectionPolicy)
-        .onConnectivityChanged(regained, lost);
-
-      GetIt.I.get<HttpFileTransferService>().onConnectivityChanged(regained);
+      _controller.add(
+        ConnectivityEvent(
+          regained,
+          lost,
+        ),
+      );
     });
   }
 
   ConnectivityResult get currentState => _connectivity;
+
+  Future<bool> hasConnection() async {
+    return _connectivity != ConnectivityResult.none;
+  }
 }

lib/service/connectivity_watcher.dart

@@ -1,56 +1,75 @@
 import 'dart:async';
-import 'package:connectivity_plus/connectivity_plus.dart';
 import 'package:get_it/get_it.dart';
 import 'package:logging/logging.dart';
 import 'package:moxxmpp/moxxmpp.dart';
 import 'package:moxxyv2/i18n/strings.g.dart';
 import 'package:moxxyv2/service/connectivity.dart';
 import 'package:moxxyv2/service/notifications.dart';
+import 'package:synchronized/synchronized.dart';
 
 class ConnectivityWatcherService {
+  /// Logger.
+  final Logger _log = Logger('ConnectivityWatcherService');
 
-  ConnectivityWatcherService() : _log = Logger('ConnectivityWatcherService');
-  final Logger _log;
-
-  // Timer counting how much time has passed since we were last connected
+  /// Timer counting how much time has passed since we were last connected.
   Timer? _timer;
 
+  /// Lock for accessing _timer
+  final Lock _lock = Lock();
+
+  Future<void> initialize() async {
+    GetIt.I.get<ConnectivityService>().stream.listen(_onConnectivityEvent);
+  }
+
+  Future<void> _onConnectivityEvent(ConnectivityEvent event) async {
+    if (event.lost) {
+      _log.finest('Network connection lost. Stopping timer');
+      await _stopTimer();
+    }
+  }
+
   Future<void> _onTimerElapsed() async {
+    await _stopTimer();
     await GetIt.I.get<NotificationsService>().showWarningNotification(
           'Moxxy',
           t.errors.connection.connectionTimeout,
         );
-    _stopTimer();
   }
 
   /// Stops the currently running timer, if there is one.
-  void _stopTimer() {
-    if (_timer != null) {
-      _timer!.cancel();
+  Future<void> _stopTimer() async {
+    await _lock.synchronized(() {
+      _timer?.cancel();
       _timer = null;
-    }
+    });
   }
 
   /// Starts the timer. If it is already running, it stops the currently running one before
   /// starting the new one.
-  void _startTimer() {
-    _stopTimer();
+  Future<void> _startTimer() async {
+    await _stopTimer();
     _timer = Timer(const Duration(minutes: 30), _onTimerElapsed);
   }
 
   /// Called when the XMPP connection state changed
-  Future<void> onConnectionStateChanged(XmppConnectionState before, XmppConnectionState current) async {
-    if (before == XmppConnectionState.connected && current != XmppConnectionState.connected) {
+  Future<void> onConnectionStateChanged(
+    XmppConnectionState before,
+    XmppConnectionState current,
+  ) async {
+    if (before == XmppConnectionState.connected &&
+        current != XmppConnectionState.connected) {
       // We somehow lost connection
-      if (GetIt.I.get<ConnectivityService>().currentState != ConnectivityResult.none) {
+      if (await GetIt.I.get<ConnectivityService>().hasConnection()) {
         _log.finest('Lost connection to server. Starting warning timer...');
-        _startTimer();
+        await _startTimer();
       } else {
-        _log.finest('Lost connection to server but no network connectivity available. Stopping warning timer...');
-        _stopTimer();
+        _log.finest(
+          'Lost connection to server but no network connectivity available. Stopping warning timer...',
+        );
+        await _stopTimer();
       }
     } else if (current == XmppConnectionState.connected) {
-      _stopTimer();
+      await _stopTimer();
     }
   }
 }

lib/service/contacts.dart

@@ -0,0 +1,331 @@
+import 'dart:async';
+import 'dart:io';
+import 'dart:typed_data';
+import 'package:flutter_contacts/flutter_contacts.dart';
+import 'package:get_it/get_it.dart';
+import 'package:logging/logging.dart';
+import 'package:moxxyv2/service/conversation.dart';
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:moxxyv2/service/database/database.dart';
+import 'package:moxxyv2/service/preferences.dart';
+import 'package:moxxyv2/service/roster.dart';
+import 'package:moxxyv2/service/service.dart';
+import 'package:moxxyv2/shared/events.dart';
+import 'package:moxxyv2/shared/helpers.dart';
+import 'package:moxxyv2/shared/models/roster.dart';
+import 'package:permission_handler/permission_handler.dart';
+
+class ContactWrapper {
+  const ContactWrapper(this.id, this.jid, this.displayName, this.thumbnail);
+  final String id;
+  final String jid;
+  final String displayName;
+  final Uint8List? thumbnail;
+}
+
+class ContactsService {
+  ContactsService() {
+    // NOTE: Apparently, this means that if false, contacts that are in 0 groups
+    //       are not returned.
+    FlutterContacts.config.includeNonVisibleOnAndroid = true;
+  }
+
+  /// Logger.
+  final Logger _log = Logger('ContactsService');
+
+  /// JID -> Id.
+  Map<String, String>? _contactIds;
+
+  /// Contact ID -> Display name from the contact or null if we cached that there is
+  /// none
+  final Map<String, String?> _contactDisplayNames = {};
+
+  Future<void> initialize() async {
+    if (await _canUseContactIntegration()) {
+      enableDatabaseListener();
+    }
+  }
+
+  /// Enable listening to contact database events
+  void enableDatabaseListener() {
+    FlutterContacts.addListener(_onContactsDatabaseUpdate);
+  }
+
+  /// Disable listening to contact database events
+  void disableDatabaseListener() {
+    FlutterContacts.removeListener(_onContactsDatabaseUpdate);
+  }
+
+  Future<void> _onContactsDatabaseUpdate() async {
+    _log.finest('Got contacts database update');
+    await scanContacts();
+  }
+
+  /// Queries the contact list for contacts that include a XMPP URI.
+  Future<List<ContactWrapper>> _fetchContactsWithJabber() async {
+    final contacts = await FlutterContacts.getContacts(
+      withProperties: true,
+      withThumbnail: true,
+    );
+    _log.finest('Got ${contacts.length} contacts');
+
+    final jabberContacts = List<ContactWrapper>.empty(growable: true);
+    for (final c in contacts) {
+      final index =
+          c.socialMedias.indexWhere((s) => s.label == SocialMediaLabel.jabber);
+      if (index == -1) continue;
+
+      jabberContacts.add(
+        ContactWrapper(
+          c.id,
+          c.socialMedias[index].userName,
+          c.displayName,
+          c.thumbnail,
+        ),
+      );
+    }
+    _log.finest('${jabberContacts.length} contacts have an XMPP address');
+
+    return jabberContacts;
+  }
+
+  /// Checks whether the contact integration is enabled by the user in the preferences.
+  /// Returns true if that is the case. If not, returns false.
+  Future<bool> isContactIntegrationEnabled() async {
+    final prefs = await GetIt.I.get<PreferencesService>().getPreferences();
+    return prefs.enableContactIntegration;
+  }
+
+  /// Checks if we a) have the permission to access the contact list and b) if the
+  /// user wants to use this integration.
+  /// Returns true if we can proceed with accessing the contact list. False, if not.
+  Future<bool> _canUseContactIntegration() async {
+    if (!(await isContactIntegrationEnabled())) {
+      _log.finest(
+        '_canUseContactIntegration: Returning false since enableContactIntegration is false',
+      );
+      return false;
+    }
+
+    final permission = await Permission.contacts.status;
+    if (permission == PermissionStatus.denied) {
+      _log.finest(
+        "_canUseContactIntegration: Returning false since we don't have the contacts permission",
+      );
+      return false;
+    }
+
+    return true;
+  }
+
+  /// Queries the database for the mapping of JID -> Contact ID. The result is
+  /// cached after the first call.
+  Future<Map<String, String>> _getContactIds() async {
+    if (_contactIds != null) return _contactIds!;
+
+    // TODO(Unknown): Can we just .cast<String, String>() here?
+    _contactIds = Map<String, String>.fromEntries(
+      (await GetIt.I.get<DatabaseService>().database.query(contactsTable)).map(
+        (item) => MapEntry(
+          item['jid']! as String,
+          item['id']! as String,
+        ),
+      ),
+    );
+    return _contactIds!;
+  }
+
+  /// Queries the contact list, if enabled and allowed, and returns the contact's
+  /// display name.
+  ///
+  /// [id] is the id of the contact. A null value indicates that there is no
+  /// contact and null will be returned immediately.
+  Future<String?> getContactDisplayName(String? id) async {
+    if (id == null || !(await _canUseContactIntegration())) return null;
+    if (_contactDisplayNames.containsKey(id)) return _contactDisplayNames[id];
+
+    final result = await FlutterContacts.getContact(
+      id,
+      withThumbnail: false,
+    );
+    _contactDisplayNames[id] = result?.displayName;
+    return result?.displayName;
+  }
+
+  /// Returns the contact Id for the JID [jid]. If either the contact integration is
+  /// disabled, not possible (due to missing permissions) or there is no contact with
+  /// [jid] as their Jabber attribute, returns null.
+  Future<String?> getContactIdForJid(String jid) async {
+    if (!(await _canUseContactIntegration())) return null;
+
+    return (await _getContactIds())[jid];
+  }
+
+  /// Returns the path to the avatar file for the contact with JID [jid] as their
+  /// Jabber attribute. If either the contact integration is disabled, not possible
+  /// (due to missing permissions) or there is no contact with [jid] as their Jabber
+  /// attribute, returns null.
+  Future<String?> getProfilePicturePathForJid(String jid) async {
+    final id = await getContactIdForJid(jid);
+    if (id == null) return null;
+
+    final avatarPath = await getContactProfilePicturePath(id);
+    return File(avatarPath).existsSync() ? avatarPath : null;
+  }
+
+  Future<void> scanContacts() async {
+    final cs = GetIt.I.get<ConversationService>();
+    final rs = GetIt.I.get<RosterService>();
+    final contacts = await _fetchContactsWithJabber();
+    // JID -> Id
+    final knownContactIds = await _getContactIds();
+    // Id -> JID
+    final knownContactIdsReverse =
+        knownContactIds.map((key, value) => MapEntry(value, key));
+    final modifiedRosterItems = List<RosterItem>.empty(growable: true);
+    final addedRosterItems = List<RosterItem>.empty(growable: true);
+    final removedRosterItems = List<String>.empty(growable: true);
+
+    for (final id in List<String>.from(knownContactIds.values)) {
+      final index = contacts.indexWhere((c) => c.id == id);
+      if (index != -1) continue;
+
+      final jid = knownContactIdsReverse[id]!;
+      await GetIt.I.get<DatabaseService>().database.delete(
+        contactsTable,
+        where: 'id = ?',
+        whereArgs: [id],
+      );
+
+      _contactIds!.remove(knownContactIdsReverse[id]);
+
+      // Remove the avatar file, if it existed
+      final avatarPath = await getContactProfilePicturePath(id);
+      final avatarFile = File(avatarPath);
+      if (avatarFile.existsSync()) {
+        unawaited(avatarFile.delete());
+      }
+
+      // Remove the contact attributes from the conversation, if it existed
+      final conversation = await cs.createOrUpdateConversation(
+        jid,
+        update: (c) async {
+          return cs.updateConversation(
+            jid,
+            contactId: null,
+            contactAvatarPath: null,
+            contactDisplayName: null,
+          );
+        },
+      );
+      if (conversation != null) {
+        sendEvent(
+          ConversationUpdatedEvent(
+            conversation: conversation,
+          ),
+        );
+      }
+
+      // Remove the contact attributes from the roster item, if it existed
+      final r = await rs.getRosterItemByJid(jid);
+      if (r != null) {
+        if (r.pseudoRosterItem) {
+          _log.finest('Removing pseudo roster item $jid');
+          await rs.removeRosterItem(r.id);
+          removedRosterItems.add(jid);
+        } else {
+          final newRosterItem = await rs.updateRosterItem(
+            r.id,
+            contactId: null,
+            contactAvatarPath: null,
+            contactDisplayName: null,
+          );
+          modifiedRosterItems.add(newRosterItem);
+        }
+      }
+    }
+
+    for (final contact in contacts) {
+      // Add the ID to the cache and the database if it does not already exist
+      if (!knownContactIds.containsKey(contact.jid)) {
+        await GetIt.I.get<DatabaseService>().database.insert(
+          contactsTable,
+          <String, String>{
+            'id': contact.id,
+            'jid': contact.jid,
+          },
+        );
+        _contactIds![contact.jid] = contact.id;
+      }
+
+      // Store the avatar image
+      // NOTE: We do not check if the file already exists since this function may also
+      //       be triggered by the contact database listener. That listener fires when
+      //       a change happened, without telling us exactly what happened. So, we
+      //       just overwrite it.
+      final contactAvatarPath = await getContactProfilePicturePath(contact.id);
+      if (contact.thumbnail != null) {
+        final file = File(contactAvatarPath);
+        await file.writeAsBytes(contact.thumbnail!);
+      }
+
+      // Update a possibly existing conversation
+      final conversation = await cs.createOrUpdateConversation(
+        contact.jid,
+        update: (c) async {
+          return cs.updateConversation(
+            contact.jid,
+            contactId: contact.id,
+            contactAvatarPath: contactAvatarPath,
+            contactDisplayName: contact.displayName,
+          );
+        },
+      );
+      if (conversation != null) {
+        sendEvent(
+          ConversationUpdatedEvent(
+            conversation: conversation,
+          ),
+        );
+      }
+
+      // Update a possibly existing roster item
+      final r = await rs.getRosterItemByJid(contact.jid);
+      if (r != null) {
+        final newRosterItem = await rs.updateRosterItem(
+          r.id,
+          contactId: contact.id,
+          contactAvatarPath: contactAvatarPath,
+          contactDisplayName: contact.displayName,
+        );
+        modifiedRosterItems.add(newRosterItem);
+      } else {
+        final newRosterItem = await rs.addRosterItemFromData(
+          '',
+          '',
+          contact.jid,
+          contact.jid.split('@').first,
+          'none',
+          'none',
+          true,
+          contact.id,
+          contactAvatarPath,
+          contact.displayName,
+        );
+        addedRosterItems.add(newRosterItem);
+      }
+    }
+
+    if (addedRosterItems.isNotEmpty ||
+        modifiedRosterItems.isNotEmpty ||
+        removedRosterItems.isNotEmpty) {
+      sendEvent(
+        RosterDiffEvent(
+          added: addedRosterItems,
+          modified: modifiedRosterItems,
+          removed: removedRosterItems,
+        ),
+      );
+    }
+  }
+}

lib/service/conversation.dart

@@ -1,61 +1,137 @@
 import 'package:get_it/get_it.dart';
-import 'package:moxlib/moxlib.dart';
 import 'package:moxxmpp/moxxmpp.dart';
+import 'package:moxxyv2/service/database/constants.dart';
 import 'package:moxxyv2/service/database/database.dart';
+import 'package:moxxyv2/service/database/helpers.dart';
+import 'package:moxxyv2/service/message.dart';
+import 'package:moxxyv2/service/not_specified.dart';
 import 'package:moxxyv2/service/preferences.dart';
-import 'package:moxxyv2/shared/cache.dart';
+import 'package:moxxyv2/service/roster.dart';
 import 'package:moxxyv2/shared/models/conversation.dart';
 import 'package:moxxyv2/shared/models/message.dart';
+import 'package:synchronized/synchronized.dart';
+
+typedef CreateConversationCallback = Future<Conversation> Function();
+
+typedef UpdateConversationCallback = Future<Conversation> Function(
+  Conversation,
+);
+
+typedef PreRunConversationCallback = Future<void> Function(Conversation?);
 
 class ConversationService {
-  ConversationService()
-    : _conversationCache = LRUCache(100),
-      _loadedConversations = false;
+  /// The list of known conversations.
+  Map<String, Conversation>? _conversationCache;
 
-  final LRUCache<int, Conversation> _conversationCache;
-  bool _loadedConversations;
+  /// The lock for accessing _conversationCache
+  final Lock _lock = Lock();
 
-  /// Wrapper around DatabaseService's loadConversations that adds the loaded
-  /// to the cache.
-  Future<void> _loadConversations() async {
-    final conversations = await GetIt.I.get<DatabaseService>().loadConversations();
-    for (final c in conversations) {
-      _conversationCache.cache(c.id, c);
+  /// When called with a JID [jid], then first, if non-null, [preRun] is
+  /// executed.
+  /// Next, if a conversation with JID [jid] exists, [update] is called with
+  /// the conversation as its argument. If not, then [create] is executed.
+  /// Returns either the result of [create], [update] or null.
+  Future<Conversation?> createOrUpdateConversation(
+    String jid, {
+    CreateConversationCallback? create,
+    UpdateConversationCallback? update,
+    PreRunConversationCallback? preRun,
+  }) async {
+    return _lock.synchronized(() async {
+      final conversation = await _getConversationByJid(jid);
+
+      // Pre run
+      if (preRun != null) {
+        await preRun(conversation);
+      }
+
+      if (conversation != null) {
+        // Conversation exists
+        if (update != null) {
+          return update(conversation);
+        }
+      } else {
+        // Conversation does not exist
+        if (create != null) {
+          return create();
         }
       }
 
-  /// Returns the conversation with jid [jid] or null if not found.
-  Future<Conversation?> getConversationByJid(String jid) async {
-    if (!_loadedConversations) {
-      await _loadConversations();
-      _loadedConversations = true;
+      return null;
+    });
   }
 
-    return firstWhereOrNull(
-      // TODO(Unknown): Maybe have it accept an iterable
-      _conversationCache.getValues(),
-      (Conversation c) => c.jid == jid,
+  /// Loads all conversations from the database and adds them to the state and cache.
+  Future<List<Conversation>> loadConversations() async {
+    final db = GetIt.I.get<DatabaseService>().database;
+    final conversationsRaw = await db.query(
+      conversationsTable,
+      orderBy: 'lastChangeTimestamp DESC',
+    );
+
+    final tmp = List<Conversation>.empty(growable: true);
+    for (final c in conversationsRaw) {
+      final jid = c['jid']! as String;
+      final rosterItem =
+          await GetIt.I.get<RosterService>().getRosterItemByJid(jid);
+
+      Message? lastMessage;
+      if (c['lastMessageId'] != null) {
+        lastMessage = await GetIt.I.get<MessageService>().getMessageById(
+              c['lastMessageId']! as int,
+              jid,
+              queryReactionPreview: false,
             );
       }
 
-  /// Returns the conversation by its database id or null if it does not exist.
-  Future<Conversation?> _getConversationById(int id) async {
-    if (!_loadedConversations) {
-      await _loadConversations();
-      _loadedConversations = true;
+      tmp.add(
+        Conversation.fromDatabaseJson(
+          c,
+          rosterItem != null && !rosterItem.pseudoRosterItem,
+          rosterItem?.subscription ?? 'none',
+          lastMessage,
+        ),
+      );
     }
 
-    return _conversationCache.getValue(id);
+    return tmp;
+  }
+
+  /// Wrapper around DatabaseService's loadConversations that adds the loaded
+  /// to the cache.
+  Future<void> _loadConversationsIfNeeded() async {
+    if (_conversationCache != null) return;
+
+    final conversations = await loadConversations();
+    _conversationCache = Map<String, Conversation>.fromEntries(
+      conversations.map((c) => MapEntry(c.jid, c)),
+    );
+  }
+
+  /// Returns the conversation with jid [jid] or null if not found.
+  Future<Conversation?> _getConversationByJid(String jid) async {
+    await _loadConversationsIfNeeded();
+    return _conversationCache![jid];
+  }
+
+  /// Wrapper around [ConversationService._getConversationByJid] that aquires
+  /// the lock for the cache.
+  Future<Conversation?> getConversationByJid(String jid) async {
+    return _lock.synchronized(() async => _getConversationByJid(jid));
   }
 
   /// For modifying the cache without writing it to disk. Useful, for example, when
   /// changing the chat state.
   void setConversation(Conversation conversation) {
-    _conversationCache.cache(conversation.id, conversation);
+    _conversationCache![conversation.jid] = conversation;
   }
 
-  /// Wrapper around [DatabaseService]'s [updateConversation] that modifies the cache.
-  Future<Conversation> updateConversation(int id, {
+  /// Updates the conversation with JID [jid] inside the database.
+  ///
+  /// To prevent issues with the cache, only call from within
+  /// [ConversationService.createOrUpdateConversation].
+  Future<Conversation> updateConversation(
+    String jid, {
     int? lastChangeTimestamp,
     Message? lastMessage,
     bool? open,
@@ -64,33 +140,81 @@ class ConversationService {
     ChatState? chatState,
     bool? muted,
     bool? encrypted,
+    Object? contactId = notSpecified,
+    Object? contactAvatarPath = notSpecified,
+    Object? contactDisplayName = notSpecified,
   }) async {
-    final conversation = (await _getConversationById(id))!;
-    var newConversation = await GetIt.I.get<DatabaseService>().updateConversation(
-      id,
-      lastMessage: lastMessage,
-      lastChangeTimestamp: lastChangeTimestamp,
-      open: open,
-      unreadCounter: unreadCounter,
-      avatarUrl: avatarUrl,
-      chatState: conversation.chatState,
-      muted: muted,
-      encrypted: encrypted,
+    final conversation = (await _getConversationByJid(jid))!;
+
+    final c = <String, dynamic>{};
+
+    if (lastMessage != null) {
+      c['lastMessageId'] = lastMessage.id;
+    }
+    if (lastChangeTimestamp != null) {
+      c['lastChangeTimestamp'] = lastChangeTimestamp;
+    }
+    if (open != null) {
+      c['open'] = boolToInt(open);
+    }
+    if (unreadCounter != null) {
+      c['unreadCounter'] = unreadCounter;
+    }
+    if (avatarUrl != null) {
+      c['avatarUrl'] = avatarUrl;
+    }
+    if (muted != null) {
+      c['muted'] = boolToInt(muted);
+    }
+    if (encrypted != null) {
+      c['encrypted'] = boolToInt(encrypted);
+    }
+    if (contactId != notSpecified) {
+      c['contactId'] = contactId as String?;
+    }
+    if (contactAvatarPath != notSpecified) {
+      c['contactAvatarPath'] = contactAvatarPath as String?;
+    }
+    if (contactDisplayName != notSpecified) {
+      c['contactDisplayName'] = contactDisplayName as String?;
+    }
+
+    final result =
+        await GetIt.I.get<DatabaseService>().database.updateAndReturn(
+      conversationsTable,
+      c,
+      where: 'jid = ?',
+      whereArgs: [jid],
+    );
+
+    final rosterItem =
+        await GetIt.I.get<RosterService>().getRosterItemByJid(jid);
+    var newConversation = Conversation.fromDatabaseJson(
+      result,
+      rosterItem != null,
+      rosterItem?.subscription ?? 'none',
+      lastMessage,
     );
 
     // Copy over the old lastMessage if a new one was not set
     if (conversation.lastMessage != null && lastMessage == null) {
-      newConversation = newConversation.copyWith(lastMessage: conversation.lastMessage);
+      newConversation =
+          newConversation.copyWith(lastMessage: conversation.lastMessage);
     }
 
-    _conversationCache.cache(id, newConversation);
+    _conversationCache![jid] = newConversation;
     return newConversation;
   }
 
-  /// Wrapper around [DatabaseService]'s [addConversationFromData] that updates the cache.
+  /// Creates a [Conversation] inside the database given the data. This is so that the
+  /// [Conversation] object can carry its database id.
+  ///
+  /// To prevent issues with the cache, only call from within
+  /// [ConversationService.createOrUpdateConversation].
   Future<Conversation> addConversationFromData(
     String title,
     Message? lastMessage,
+    ConversationType type,
     String avatarUrl,
     String jid,
     int unreadCounter,
@@ -98,20 +222,39 @@ class ConversationService {
     bool open,
     bool muted,
     bool encrypted,
+    String? contactId,
+    String? contactAvatarPath,
+    String? contactDisplayName,
   ) async {
-    final newConversation = await GetIt.I.get<DatabaseService>().addConversationFromData(
+    final rosterItem =
+        await GetIt.I.get<RosterService>().getRosterItemByJid(jid);
+    final newConversation = Conversation(
       title,
       lastMessage,
       avatarUrl,
       jid,
       unreadCounter,
+      type,
       lastChangeTimestamp,
       open,
+      rosterItem != null && !rosterItem.pseudoRosterItem,
+      rosterItem?.subscription ?? 'none',
       muted,
       encrypted,
+      ChatState.gone,
+      contactId: contactId,
+      contactAvatarPath: contactAvatarPath,
+      contactDisplayName: contactDisplayName,
+    );
+    await GetIt.I.get<DatabaseService>().database.insert(
+          conversationsTable,
+          newConversation.toDatabaseJson(),
         );
 
-    _conversationCache.cache(newConversation.id, newConversation);
+    if (_conversationCache != null) {
+      _conversationCache![newConversation.jid] = newConversation;
+    }
+
     return newConversation;
   }
 

lib/service/cryptography/cryptography.dart

@@ -20,24 +20,30 @@ List<int> _randomBuffer(int length) {
 
 CipherAlgorithm _sfsToCipher(SFSEncryptionType type) {
   switch (type) {
-    case SFSEncryptionType.aes128GcmNoPadding: return CipherAlgorithm.aes128GcmNoPadding;
-    case SFSEncryptionType.aes256GcmNoPadding: return CipherAlgorithm.aes256GcmNoPadding;
-    case SFSEncryptionType.aes256CbcPkcs7: return CipherAlgorithm.aes256CbcPkcs7;
+    case SFSEncryptionType.aes128GcmNoPadding:
+      return CipherAlgorithm.aes128GcmNoPadding;
+    case SFSEncryptionType.aes256GcmNoPadding:
+      return CipherAlgorithm.aes256GcmNoPadding;
+    case SFSEncryptionType.aes256CbcPkcs7:
+      return CipherAlgorithm.aes256CbcPkcs7;
   }
 }
 
 class CryptographyService {
-
   CryptographyService() : _log = Logger('CryptographyService');
   final Logger _log;
 
   /// Encrypt the file at path [source] and write the encrypted data to [dest]. For the
   /// encryption, use the algorithm indicated by [encryption].
-  Future<EncryptionResult> encryptFile(String source, String dest, SFSEncryptionType encryption) async {
+  Future<EncryptionResult> encryptFile(
+    String source,
+    String dest,
+    SFSEncryptionType encryption,
+  ) async {
     _log.finest('Beginning encryption routine for $source');
-    final key = encryption == SFSEncryptionType.aes128GcmNoPadding ?
-      _randomBuffer(16) :
-      _randomBuffer(32);
+    final key = encryption == SFSEncryptionType.aes128GcmNoPadding
+        ? _randomBuffer(16)
+        : _randomBuffer(32);
     final iv = _randomBuffer(12);
     final result = (await MoxplatformPlugin.crypto.encryptFile(
       source,
@@ -52,11 +58,11 @@ class CryptographyService {
     return EncryptionResult(
       key,
       iv,
-      <String, String>{
-        hashSha256: base64Encode(result.plaintextHash),
+      <HashFunction, String>{
+        HashFunction.sha256: base64Encode(result.plaintextHash),
       },
-      <String, String>{
-        hashSha256: base64Encode(result.ciphertextHash),
+      <HashFunction, String>{
+        HashFunction.sha256: base64Encode(result.ciphertextHash),
       },
     );
   }
@@ -70,8 +76,8 @@ class CryptographyService {
     SFSEncryptionType encryption,
     List<int> key,
     List<int> iv,
-    Map<String, String> plaintextHashes,
-    Map<String, String> ciphertextHashes,
+    Map<HashFunction, String> plaintextHashes,
+    Map<HashFunction, String> ciphertextHashes,
   ) async {
     _log.finest('Beginning decryption for $source');
     final result = await MoxplatformPlugin.crypto.encryptFile(
@@ -88,7 +94,7 @@ class CryptographyService {
     var passedPlaintextIntegrityCheck = true;
     var passedCiphertextIntegrityCheck = true;
     for (final entry in plaintextHashes.entries) {
-      if (entry.key == hashSha256) {
+      if (entry.key == HashFunction.sha256) {
         if (base64Encode(result!.plaintextHash) != entry.value) {
           passedPlaintextIntegrityCheck = false;
         } else {
@@ -99,7 +105,7 @@ class CryptographyService {
       }
     }
     for (final entry in ciphertextHashes.entries) {
-      if (entry.key == hashSha256) {
+      if (entry.key == HashFunction.sha256) {
         if (base64Encode(result!.ciphertextHash) != entry.value) {
           passedCiphertextIntegrityCheck = false;
         } else {

lib/service/cryptography/implementations.dart

@@ -1,150 +0,0 @@
-import 'dart:convert';
-import 'dart:io';
-import 'package:cryptography/cryptography.dart';
-import 'package:moxxmpp/moxxmpp.dart';
-import 'package:moxxyv2/service/cryptography/types.dart';
-
-Future<List<int>> hashFileImpl(HashRequest request) async {
-  final data = await File(request.path).readAsBytes();
-
-  return CryptographicHashManager.hashFromData(data, request.hash);
-}
-
-Future<EncryptionResult> encryptFileImpl(EncryptionRequest request) async { 
-  Cipher algorithm;
-  switch (request.encryption) {
-    case SFSEncryptionType.aes128GcmNoPadding:
-      algorithm = AesGcm.with128bits();
-      break;
-    case SFSEncryptionType.aes256GcmNoPadding:
-      algorithm = AesGcm.with256bits();
-      break;
-    case SFSEncryptionType.aes256CbcPkcs7:
-      // TODO(Unknown): Implement
-      throw Exception();
-      // ignore: dead_code
-      break;
-  }
-
-  // Generate a key and an IV for the file
-  final key = await algorithm.newSecretKey();
-  final iv = algorithm.newNonce();
-  final plaintext = await File(request.source).readAsBytes();
-  final secretBox = await algorithm.encrypt(
-    plaintext,
-    secretKey: key,
-    nonce: iv,
-  );
-  final ciphertext = [
-    ...secretBox.cipherText,
-    ...secretBox.mac.bytes,
-  ];
-
-  // Write the file
-  await File(request.dest).writeAsBytes(ciphertext);
-
-  return EncryptionResult(
-    await key.extractBytes(),
-    iv,
-    {
-      hashSha256: base64Encode(
-        await CryptographicHashManager.hashFromData(plaintext, HashFunction.sha256),
-      ),
-    },
-    {
-      hashSha256: base64Encode(
-        await CryptographicHashManager.hashFromData(ciphertext, HashFunction.sha256),
-      ),
-    },
-  );
-}
-
-// TODO(PapaTutuWawa): Somehow fail when the ciphertext hash is not matching the provided data
-Future<DecryptionResult> decryptFileImpl(DecryptionRequest request) async {
-  Cipher algorithm;
-  switch (request.encryption) {
-    case SFSEncryptionType.aes128GcmNoPadding:
-      algorithm = AesGcm.with128bits();
-      break;
-    case SFSEncryptionType.aes256GcmNoPadding:
-      algorithm = AesGcm.with256bits();
-      break;
-    case SFSEncryptionType.aes256CbcPkcs7:
-      // TODO(Unknown): Implement
-      throw Exception();
-      // ignore: dead_code
-      break;
-  }
-  
-  final ciphertextRaw = await File(request.source).readAsBytes();
-  final mac = List<int>.empty(growable: true);
-  final ciphertext = List<int>.empty(growable: true);
-  // TODO(PapaTutuWawa): Somehow handle aes256CbcPkcs7
-  if (request.encryption == SFSEncryptionType.aes128GcmNoPadding ||
-      request.encryption == SFSEncryptionType.aes256GcmNoPadding) {
-    mac.addAll(ciphertextRaw.sublist(ciphertextRaw.length - 16));
-    ciphertext.addAll(ciphertextRaw.sublist(0, ciphertextRaw.length - 16));
-  }
-
-  var passedCiphertextIntegrityCheck = true;
-  var passedPlaintextIntegrityCheck = true;
-  // Try to find one hash we can verify
-  for (final entry in request.ciphertextHashes.entries) {
-    if ([hashSha256, hashSha512, hashBlake2b512].contains(entry.key)) {
-      final hash = await CryptographicHashManager.hashFromData(
-        ciphertext,
-        hashFunctionFromName(entry.key),
-      );
-
-      if (base64Encode(hash) == entry.value) {
-        passedCiphertextIntegrityCheck = true;
-      } else {
-        passedCiphertextIntegrityCheck = false;
-      }
-      break;
-    }
-  }
-  
-  final secretBox = SecretBox(
-    ciphertext,
-    nonce: request.iv,
-    mac: Mac(mac),
-  );
-
-  try {
-    final data = await algorithm.decrypt(
-      secretBox,
-      secretKey: SecretKey(request.key),
-    );
-
-    for (final entry in request.plaintextHashes.entries) {
-      if ([hashSha256, hashSha512, hashBlake2b512].contains(entry.key)) {
-        final hash = await CryptographicHashManager.hashFromData(
-          data,
-          hashFunctionFromName(entry.key),
-        );
-
-        if (base64Encode(hash) == entry.value) {
-          passedPlaintextIntegrityCheck = true;
-        } else {
-          passedPlaintextIntegrityCheck = false;
-        }
-        break;
-      }
-    }
-
-    await File(request.dest).writeAsBytes(data);
-  } catch (_) {
-    return DecryptionResult(
-      false,
-      passedPlaintextIntegrityCheck,
-      passedCiphertextIntegrityCheck,
-    );
-  }
-
-  return DecryptionResult(
-    true,
-    passedPlaintextIntegrityCheck,
-    passedCiphertextIntegrityCheck,
-  );
-}

lib/service/cryptography/types.dart

@@ -3,18 +3,21 @@ import 'package:moxxmpp/moxxmpp.dart';
 
 @immutable
 class EncryptionResult {
-
-  const EncryptionResult(this.key, this.iv, this.plaintextHashes, this.ciphertextHashes);
+  const EncryptionResult(
+    this.key,
+    this.iv,
+    this.plaintextHashes,
+    this.ciphertextHashes,
+  );
   final List<int> key;
   final List<int> iv;
 
-  final Map<String, String> plaintextHashes;
-  final Map<String, String> ciphertextHashes;
+  final Map<HashFunction, String> plaintextHashes;
+  final Map<HashFunction, String> ciphertextHashes;
 }
 
 @immutable
 class EncryptionRequest {
-
   const EncryptionRequest(this.source, this.dest, this.encryption);
   final String source;
   final String dest;
@@ -23,7 +26,6 @@ class EncryptionRequest {
 
 @immutable
 class DecryptionResult {
-
   const DecryptionResult(
     this.decryptionOkay,
     this.plaintextOkay,
@@ -36,7 +38,6 @@ class DecryptionResult {
 
 @immutable
 class DecryptionRequest {
-
   const DecryptionRequest(
     this.source,
     this.dest,
@@ -51,14 +52,6 @@ class DecryptionRequest {
   final SFSEncryptionType encryption;
   final List<int> key;
   final List<int> iv;
-  final Map<String, String> plaintextHashes;
-  final Map<String, String> ciphertextHashes;
-}
-
-@immutable
-class HashRequest {
-
-  const HashRequest(this.path, this.hash);
-  final String path;
-  final HashFunction hash;
+  final Map<HashFunction, String> plaintextHashes;
+  final Map<HashFunction, String> ciphertextHashes;
 }

lib/service/database/constants.dart

@@ -9,7 +9,16 @@ const omemoRatchetsTable = 'OmemoSessions';
 const omemoTrustCacheTable = 'OmemoTrustCacheList';
 const omemoTrustDeviceListTable = 'OmemoTrustDeviceList';
 const omemoTrustEnableListTable = 'OmemoTrustEnableList';
+const omemoFingerprintCache = 'OmemoFingerprintCache';
 const xmppStateTable = 'XmppState';
+const contactsTable = 'Contacts';
+const stickersTable = 'Stickers';
+const stickerPacksTable = 'StickerPacks';
+const blocklistTable = 'Blocklist';
+const subscriptionsTable = 'SubscriptionRequests';
+const fileMetadataTable = 'FileMetadata';
+const fileMetadataHashesTable = 'FileMetadataHashes';
+const reactionsTable = 'Reactions';
 
 const typeString = 0;
 const typeInt = 1;

lib/service/database/creation.dart

@@ -1,4 +1,5 @@
 import 'package:moxxyv2/service/database/constants.dart';
+import 'package:moxxyv2/service/database/helpers.dart';
 import 'package:moxxyv2/shared/models/preference.dart';
 import 'package:sqflite_sqlcipher/sqflite.dart';
 
@@ -17,8 +18,7 @@ Future<void> createDatabase(Database db, int version) async {
   );
 
   // Messages
-  await db.execute(
-    '''
+  await db.execute('''
     CREATE TABLE $messagesTable (
       id INTEGER PRIMARY KEY AUTOINCREMENT,
       sender TEXT NOT NULL,
@@ -26,69 +26,109 @@ Future<void> createDatabase(Database db, int version) async {
       timestamp INTEGER NOT NULL,
       sid TEXT NOT NULL,
       conversationJid TEXT NOT NULL,
-      isMedia INTEGER NOT NULL,
       isFileUploadNotification INTEGER NOT NULL,
       encrypted INTEGER NOT NULL,
       errorType INTEGER,
       warningType INTEGER,
-      mediaUrl TEXT,
-      mediaType TEXT,
-      thumbnailData TEXT,
-      mediaWidth INTEGER,
-      mediaHeight INTEGER,
-      srcUrl TEXT,
-      key TEXT,
-      iv TEXT,
-      encryptionScheme TEXT,
       received INTEGER,
       displayed INTEGER,
       acked INTEGER,
       originId TEXT,
       quote_id INTEGER,
-      filename TEXT,
-      plaintextHashes TEXT,
-      ciphertextHashes TEXT,
+      file_metadata_id TEXT,
       isDownloading INTEGER NOT NULL,
       isUploading INTEGER NOT NULL,
-      mediaSize INTEGER,
       isRetracted INTEGER,
       isEdited INTEGER NOT NULL,
+      containsNoStore INTEGER NOT NULL,
+      stickerPackId   TEXT,
+      pseudoMessageType INTEGER,
+      pseudoMessageData TEXT,
       CONSTRAINT fk_quote FOREIGN KEY (quote_id) REFERENCES $messagesTable (id)
-    )''',
+      CONSTRAINT fk_file_metadata FOREIGN KEY (file_metadata_id) REFERENCES $fileMetadataTable (id)
+    )''');
+  await db.execute(
+    'CREATE INDEX idx_messages_id ON $messagesTable (id, sid, originId)',
+  );
+
+  // Reactions
+  await db.execute('''
+    CREATE TABLE $reactionsTable (
+      senderJid  TEXT NOT NULL,
+      emoji      TEXT NOT NULL,
+      message_id INTEGER NOT NULL,
+      CONSTRAINT pk_sender PRIMARY KEY (senderJid, emoji, message_id),
+      CONSTRAINT fk_message FOREIGN KEY (message_id) REFERENCES $messagesTable (id)
+        ON DELETE CASCADE
+    )''');
+  await db.execute(
+    'CREATE INDEX idx_reactions_message_id ON $reactionsTable (message_id, senderJid)',
+  );
+
+  // File metadata
+  await db.execute('''
+    CREATE TABLE $fileMetadataTable (
+      id               TEXT NOT NULL PRIMARY KEY,
+      path             TEXT,
+      sourceUrls       TEXT,
+      mimeType         TEXT,
+      thumbnailType    TEXT,
+      thumbnailData    TEXT,
+      width            INTEGER,
+      height           INTEGER,
+      plaintextHashes  TEXT,
+      encryptionKey    TEXT,
+      encryptionIv     TEXT,
+      encryptionScheme TEXT,
+      cipherTextHashes TEXT,
+      filename         TEXT NOT NULL,
+      size             INTEGER
+    )''');
+  await db.execute('''
+    CREATE TABLE $fileMetadataHashesTable (
+      algorithm TEXT NOT NULL,
+      value     TEXT NOT NULL,
+      id        TEXT NOT NULL,
+      CONSTRAINT f_primarykey PRIMARY KEY (algorithm, value),
+      CONSTRAINT fk_id FOREIGN KEY (id) REFERENCES $fileMetadataTable (id)
+        ON DELETE CASCADE
+    )''');
+  await db.execute(
+    'CREATE INDEX idx_file_metadata_message_id ON $fileMetadataTable (id)',
   );
 
   // Conversations
   await db.execute(
     '''
     CREATE TABLE $conversationsTable (
-      id INTEGER PRIMARY KEY AUTOINCREMENT,
-      jid TEXT NOT NULL,
+      jid TEXT NOT NULL PRIMARY KEY,
       title TEXT NOT NULL,
       avatarUrl TEXT NOT NULL,
+      type TEXT NOT NULL,
       lastChangeTimestamp INTEGER NOT NULL,
       unreadCounter INTEGER NOT NULL,
       open INTEGER NOT NULL,
       muted INTEGER NOT NULL,
       encrypted INTEGER NOT NULL,
-      lastMessageId INTEGER NOT NULL,
-      CONSTRAINT fk_last_message FOREIGN KEY (lastMessageId) REFERENCES $messagesTable (id)
+      lastMessageId INTEGER,
+      contactId TEXT,
+      contactAvatarPath TEXT,
+      contactDisplayName TEXT,
+      CONSTRAINT fk_last_message FOREIGN KEY (lastMessageId) REFERENCES $messagesTable (id),
+      CONSTRAINT fk_contact_id FOREIGN KEY (contactId) REFERENCES $contactsTable (id)
+        ON DELETE SET NULL
     )''',
   );
+  await db.execute(
+    'CREATE INDEX idx_conversation_id ON $conversationsTable (jid)',
+  );
 
-  // Shared media
-  await db.execute(
-    '''
-    CREATE TABLE $mediaTable (
-      id INTEGER PRIMARY KEY AUTOINCREMENT,
-      path TEXT NOT NULL,
-      mime TEXT,
-      timestamp INTEGER NOT NULL,
-      conversation_id INTEGER NOT NULL,
-      message_id INTEGER,
-      FOREIGN KEY (conversation_id) REFERENCES $conversationsTable (id),
-      FOREIGN KEY (message_id) REFERENCES $messagesTable (id)
-    )''',
-  );
+  // Contacts
+  await db.execute('''
+    CREATE TABLE $contactsTable (
+      id TEXT PRIMARY KEY,
+      jid TEXT NOT NULL
+    )''');
 
   // Roster
   await db.execute(
@@ -100,10 +140,57 @@ Future<void> createDatabase(Database db, int version) async {
       avatarUrl TEXT NOT NULL,
       avatarHash TEXT NOT NULL,
       subscription TEXT NOT NULL,
-      ask TEXT NOT NULL
+      ask TEXT NOT NULL,
+      contactId TEXT,
+      contactAvatarPath TEXT,
+      contactDisplayName TEXT,
+      pseudoRosterItem INTEGER NOT NULL,
+      CONSTRAINT fk_contact_id FOREIGN KEY (contactId) REFERENCES $contactsTable (id)
+        ON DELETE SET NULL
     )''',
   );
 
+  // Stickers
+  await db.execute(
+    '''
+    CREATE TABLE $stickersTable (
+      id               TEXT PRIMARY KEY,
+      desc             TEXT NOT NULL,
+      suggests         TEXT NOT NULL,
+      file_metadata_id TEXT NOT NULL,
+      stickerPackId  TEXT NOT NULL,
+      CONSTRAINT fk_sticker_pack FOREIGN KEY (stickerPackId) REFERENCES $stickerPacksTable (id)
+        ON DELETE CASCADE,
+      CONSTRAINT fk_file_metadata FOREIGN KEY (file_metadata_id) REFERENCES $fileMetadataTable (id)
+    )''',
+  );
+  await db.execute(
+    '''
+    CREATE TABLE $stickerPacksTable (
+      id             TEXT PRIMARY KEY,
+      name           TEXT NOT NULL,
+      description    TEXT NOT NULL,
+      hashAlgorithm  TEXT NOT NULL,
+      hashValue      TEXT NOT NULL,
+      restricted     INTEGER NOT NULL
+    )''',
+  );
+
+  // Blocklist
+  await db.execute(
+    '''
+    CREATE TABLE $blocklistTable (
+      jid TEXT PRIMARY KEY
+    );
+    ''',
+  );
+
+  // Subscription requests
+  await db.execute('''
+    CREATE TABLE $subscriptionsTable(
+      jid TEXT PRIMARY KEY
+    )''');
+
   // OMEMO
   await db.execute(
     '''
@@ -166,6 +253,15 @@ Future<void> createDatabase(Database db, int version) async {
       PRIMARY KEY (jid, id)
     )''',
   );
+  await db.execute(
+    '''
+    CREATE TABLE $omemoFingerprintCache (
+      jid  TEXT NOT NULL,
+      id   INTEGER NOT NULL,
+      fingerprint TEXT NOT NULL,
+      PRIMARY KEY (jid, id)
+    )''',
+  );
 
   // Settings
   await db.execute(
@@ -240,14 +336,6 @@ Future<void> createDatabase(Database db, int version) async {
       'true',
     ).toDatabaseJson(),
   );
-  await db.insert(
-    preferenceTable,
-    Preference(
-      'autoAcceptSubscriptionRequests',
-      typeBool,
-      'false',
-    ).toDatabaseJson(),
-  );
   await db.insert(
     preferenceTable,
     Preference(
@@ -336,4 +424,28 @@ Future<void> createDatabase(Database db, int version) async {
       'default',
     ).toDatabaseJson(),
   );
+  await db.insert(
+    preferenceTable,
+    Preference(
+      'enableContactIntegration',
+      typeBool,
+      'false',
+    ).toDatabaseJson(),
+  );
+  await db.insert(
+    preferenceTable,
+    Preference(
+      'isStickersNodePublic',
+      typeBool,
+      'true',
+    ).toDatabaseJson(),
+  );
+  await db.insert(
+    preferenceTable,
+    Preference(
+      'showDebugMenu',
+      typeBool,
+      boolToString(false),
+    ).toDatabaseJson(),
+  );
 }

lib/service/database/helpers.dart

@@ -1,3 +1,5 @@
+import 'package:moxxyv2/shared/models/conversation.dart';
+
 /// Conversion helpers for bool <-> int as sqlite has no "real" booleans
 int boolToInt(bool b) => b ? 1 : 0;
 bool intToBool(int i) => i == 0 ? false : true;
@@ -7,3 +9,43 @@ bool stringToBool(String s) => s == 'true' ? true : false;
 
 String intToString(int i) => '$i';
 int stringToInt(String s) => int.parse(s);
+
+String conversationTypeToString(ConversationType type) {
+  switch (type) {
+    case ConversationType.chat:
+      {
+        return 'chat';
+      }
+    case ConversationType.note:
+      {
+        return 'note';
+      }
+  }
+}
+
+ConversationType stringToConversationType(String type) {
+  switch (type) {
+    case 'chat':
+      {
+        return ConversationType.chat;
+      }
+    default:
+      {
+        return ConversationType.note;
+      }
+  }
+}
+
+/// Given a map [map], extract all key-value pairs from [map] where the key starts with
+/// [prefix]. Combine those key-value pairs into a new map, where the leading [prefix]
+/// is removed from all key names.
+Map<String, T> getPrefixedSubMap<T>(Map<String, T> map, String prefix) {
+  return Map<String, T>.fromEntries(
+    map.entries.where((entry) => entry.key.startsWith(prefix)).map(
+          (entry) => MapEntry<String, T>(
+            entry.key.substring(prefix.length),
+            entry.value,
+          ),
+        ),
+  );
+}

lib/service/database/migration.dart

@@ -0,0 +1,44 @@
+import 'package:logging/logging.dart';
+
+/// A function to be called when a migration should be performed.
+typedef DatabaseMigrationCallback<T> = Future<void> Function(T);
+
+/// This class represents a single database migration.
+class DatabaseMigration<T> {
+  const DatabaseMigration(this.version, this.migration);
+
+  /// The version this migration upgrades the database to.
+  final int version;
+
+  /// The migration callback. Called the the database version is less than [version].
+  final DatabaseMigrationCallback<T> migration;
+}
+
+/// Given the database [db] with the current version [version], goes through the list of
+/// migrations [migrations] and applies all migrations with a version greater than
+/// [version]. [migrations] is sorted before usage.
+///
+/// NOTE: This entire setup is written as a generic to make testing easier. We cannot easily
+///       mock, or better "instantiate", a Database object. Thus, to avoid having nullable
+///       database argument, just pass in whatever (the tests use an integer).
+Future<void> runMigrations<T>(
+  Logger log,
+  T db,
+  List<DatabaseMigration<T>> migrations,
+  int version,
+) async {
+  final sortedMigrations = List<DatabaseMigration<T>>.from(migrations)
+    ..sort(
+      (a, b) => a.version.compareTo(b.version),
+    );
+  var currentVersion = version;
+  for (final migration in sortedMigrations) {
+    if (version < migration.version) {
+      log.info(
+        'Running database migration $currentVersion -> ${migration.version}',
+      );
+      await migration.migration(db);
+      currentVersion = migration.version;
+    }
+  }
+}

lib/service/database/migrations/0000_blocklist.dart

@@ -0,0 +1,12 @@
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV22ToV23(Database db) async {
+  await db.execute(
+    '''
+    CREATE TABLE $blocklistTable (
+      jid TEXT PRIMARY KEY
+    );
+    ''',
+  );
+}

lib/service/database/migrations/0000_contacts_integration.dart

@@ -0,0 +1,72 @@
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:moxxyv2/shared/models/preference.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV13ToV14(Database db) async {
+  // Create the new table
+  await db.execute('''
+    CREATE TABLE $contactsTable (
+      id TEXT PRIMARY KEY,
+      jid TEXT NOT NULL
+    )''');
+
+  // Migrate the conversations
+  await db.execute(
+    '''
+    CREATE TABLE ${conversationsTable}_new (
+      id INTEGER PRIMARY KEY AUTOINCREMENT,
+      jid TEXT NOT NULL,
+      title TEXT NOT NULL,
+      avatarUrl TEXT NOT NULL,
+      lastChangeTimestamp INTEGER NOT NULL,
+      unreadCounter INTEGER NOT NULL,
+      open INTEGER NOT NULL,
+      muted INTEGER NOT NULL,
+      encrypted INTEGER NOT NULL,
+      lastMessageId INTEGER,
+      contactId TEXT,
+      CONSTRAINT fk_last_message FOREIGN KEY (lastMessageId) REFERENCES $messagesTable (id),
+      CONSTRAINT fk_contact_id FOREIGN KEY (contactId) REFERENCES $contactsTable (id)
+        ON DELETE SET NULL
+    )''',
+  );
+  await db.execute(
+    'INSERT INTO ${conversationsTable}_new SELECT *, NULL from $conversationsTable',
+  );
+  await db.execute('DROP TABLE $conversationsTable;');
+  await db.execute(
+    'ALTER TABLE ${conversationsTable}_new RENAME TO $conversationsTable;',
+  );
+
+  // Migrate the roster items
+  await db.execute(
+    '''
+    CREATE TABLE ${rosterTable}_new (
+      id INTEGER PRIMARY KEY AUTOINCREMENT,
+      jid TEXT NOT NULL,
+      title TEXT NOT NULL,
+      avatarUrl TEXT NOT NULL,
+      avatarHash TEXT NOT NULL,
+      subscription TEXT NOT NULL,
+      ask TEXT NOT NULL,
+      contactId TEXT,
+      CONSTRAINT fk_contact_id FOREIGN KEY (contactId) REFERENCES $contactsTable (id)
+        ON DELETE SET NULL
+    )''',
+  );
+  await db.execute(
+    'INSERT INTO ${rosterTable}_new SELECT *, NULL from $rosterTable',
+  );
+  await db.execute('DROP TABLE $rosterTable;');
+  await db.execute('ALTER TABLE ${rosterTable}_new RENAME TO $rosterTable;');
+
+  // Introduce the new preference key
+  await db.insert(
+    preferenceTable,
+    Preference(
+      'enableContactIntegration',
+      typeBool,
+      'false',
+    ).toDatabaseJson(),
+  );
+}

lib/service/database/migrations/0000_contacts_integration_avatar.dart

@@ -0,0 +1,17 @@
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV14ToV15(Database db) async {
+  await db.execute(
+    'ALTER TABLE $conversationsTable ADD COLUMN contactAvatarPath TEXT DEFAULT NULL;',
+  );
+  await db.execute(
+    'ALTER TABLE $rosterTable ADD COLUMN contactAvatarPath TEXT DEFAULT NULL;',
+  );
+  await db.execute(
+    'ALTER TABLE $conversationsTable ADD COLUMN contactDisplayName TEXT DEFAULT NULL;',
+  );
+  await db.execute(
+    'ALTER TABLE $rosterTable ADD COLUMN contactDisplayName TEXT DEFAULT NULL;',
+  );
+}

lib/service/database/migrations/0000_contacts_integration_pseudo.dart

@@ -0,0 +1,9 @@
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:moxxyv2/service/database/helpers.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV15ToV16(Database db) async {
+  await db.execute(
+    'ALTER TABLE $rosterTable ADD COLUMN pseudoRosterItem INTEGER NOT NULL DEFAULT ${boolToInt(false)};',
+  );
+}

lib/service/database/migrations/0000_conversations.dart

@@ -1,13 +1,11 @@
 import 'package:moxxyv2/service/database/constants.dart';
-import 'package:moxxyv2/shared/models/conversation.dart';
 import 'package:sqflite_sqlcipher/sqflite.dart';
 
 Future<void> upgradeFromV6ToV7(Database db) async {
   await db.execute(
-    'ALTER TABLE $conversationsTable ADD COLUMN lastMessageState INTEGER NOT NULL DEFAULT 0;'
+    'ALTER TABLE $conversationsTable ADD COLUMN lastMessageState INTEGER NOT NULL DEFAULT 0;',
   );
   await db.execute(
-    "ALTER TABLE $conversationsTable ADD COLUMN lastMessageSender TEXT NOT NULL DEFAULT '';"
+    "ALTER TABLE $conversationsTable ADD COLUMN lastMessageSender TEXT NOT NULL DEFAULT '';",
   );
-
 }

lib/service/database/migrations/0000_conversations2.dart

@@ -3,15 +3,15 @@ import 'package:sqflite_sqlcipher/sqflite.dart';
 
 Future<void> upgradeFromV7ToV8(Database db) async {
   await db.execute(
-    'ALTER TABLE $conversationsTable DROP COLUMN lastMessageState;'
+    'ALTER TABLE $conversationsTable DROP COLUMN lastMessageState;',
   );
   await db.execute(
-    "ALTER TABLE $conversationsTable DROP COLUMN lastMessageSender;"
+    'ALTER TABLE $conversationsTable DROP COLUMN lastMessageSender;',
   );
   await db.execute(
-    "ALTER TABLE $conversationsTable DROP COLUMN lastMessageBody;"
+    'ALTER TABLE $conversationsTable DROP COLUMN lastMessageBody;',
   );
   await db.execute(
-    "ALTER TABLE $conversationsTable DROP COLUMN lastMessageRetracted;"
+    'ALTER TABLE $conversationsTable DROP COLUMN lastMessageRetracted;',
   );
 }

lib/service/database/migrations/0000_conversations3.dart

@@ -25,13 +25,17 @@ Future<void> upgradeFromV8ToV9(Database db) async {
   );
 
   // Step 5
-  await db.execute('INSERT INTO ${conversationsTable}_new SELECT * from $conversationsTable');
+  await db.execute(
+    'INSERT INTO ${conversationsTable}_new SELECT * from $conversationsTable',
+  );
 
   // Step 6
   await db.execute('DROP TABLE $conversationsTable;');
 
   // Step 7
-  await db.execute('ALTER TABLE ${conversationsTable}_new RENAME TO $conversationsTable;');
+  await db.execute(
+    'ALTER TABLE ${conversationsTable}_new RENAME TO $conversationsTable;',
+  );
 
   // Step 10
   //await db.execute('PRAGMA foreign_key_check;');

lib/service/database/migrations/0000_lmc.dart

@@ -1,6 +1,5 @@
 import 'package:moxxyv2/service/database/constants.dart';
 import 'package:moxxyv2/service/database/helpers.dart';
-import 'package:moxxyv2/shared/models/preference.dart';
 import 'package:sqflite_sqlcipher/sqflite.dart';
 
 Future<void> upgradeFromV9ToV10(Database db) async {

lib/service/database/migrations/0000_omemo_fingerprint_cache.dart

@@ -0,0 +1,14 @@
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV12ToV13(Database db) async {
+  await db.execute(
+    '''
+    CREATE TABLE $omemoFingerprintCache (
+      jid  TEXT NOT NULL,
+      id   INTEGER NOT NULL,
+      fingerprint TEXT NOT NULL,
+      PRIMARY KEY (jid, id)
+    )''',
+  );
+}

lib/service/database/migrations/0000_pseudo_messages.dart

@@ -0,0 +1,11 @@
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV23ToV24(Database db) async {
+  await db.execute(
+    'ALTER TABLE $messagesTable ADD COLUMN pseudoMessageType INTEGER;',
+  );
+  await db.execute(
+    'ALTER TABLE $messagesTable ADD COLUMN pseudoMessageData TEXT;',
+  );
+}

lib/service/database/migrations/0000_reactions.dart

@@ -0,0 +1,8 @@
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV10ToV11(Database db) async {
+  await db.execute(
+    "ALTER TABLE $messagesTable ADD COLUMN reactions TEXT NOT NULL DEFAULT '[]';",
+  );
+}

lib/service/database/migrations/0000_reactions_store_hint.dart

@@ -0,0 +1,9 @@
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:moxxyv2/service/database/helpers.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV11ToV12(Database db) async {
+  await db.execute(
+    'ALTER TABLE $messagesTable ADD COLUMN containsNoStore INTEGER NOT NULL DEFAULT ${boolToInt(false)};',
+  );
+}

lib/service/database/migrations/0000_retraction.dart

@@ -1,6 +1,5 @@
 import 'package:moxxyv2/service/database/constants.dart';
 import 'package:moxxyv2/service/database/helpers.dart';
-import 'package:moxxyv2/shared/models/preference.dart';
 import 'package:sqflite_sqlcipher/sqflite.dart';
 
 Future<void> upgradeFromV3ToV4(Database db) async {

lib/service/database/migrations/0000_retraction_conversation.dart

@@ -1,6 +1,5 @@
 import 'package:moxxyv2/service/database/constants.dart';
 import 'package:moxxyv2/service/database/helpers.dart';
-import 'package:moxxyv2/shared/models/preference.dart';
 import 'package:sqflite_sqlcipher/sqflite.dart';
 
 Future<void> upgradeFromV4ToV5(Database db) async {

lib/service/database/migrations/0000_shared_media.dart

@@ -1,5 +1,4 @@
 import 'package:moxxyv2/service/database/constants.dart';
-import 'package:moxxyv2/shared/models/preference.dart';
 import 'package:sqflite_sqlcipher/sqflite.dart';
 
 Future<void> upgradeFromV5ToV6(Database db) async {

lib/service/database/migrations/0000_stickers.dart

@@ -0,0 +1,59 @@
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:moxxyv2/shared/models/preference.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV16ToV17(Database db) async {
+  await db.execute(
+    '''
+    CREATE TABLE $stickersTable (
+      id            INTEGER PRIMARY KEY AUTOINCREMENT,
+      mediaType     TEXT NOT NULL,
+      desc          TEXT NOT NULL,
+      size          INTEGER NOT NULL,
+      width         INTEGER,
+      height        INTEGER,
+      hashes        TEXT NOT NULL,
+      urlSources    TEXT NOT NULL,
+      path          TEXT NOT NULL,
+      stickerPackId TEXT NOT NULL,
+      CONSTRAINT fk_sticker_pack FOREIGN KEY (stickerPackId) REFERENCES $stickerPacksTable (id)
+        ON DELETE CASCADE
+    )''',
+  );
+  await db.execute(
+    '''
+    CREATE TABLE $stickerPacksTable (
+      id            TEXT PRIMARY KEY,
+      name          TEXT NOT NULL,
+      description   TEXT NOT NULL,
+      hashAlgorithm TEXT NOT NULL,
+      hashValue     TEXT NOT NULL
+    )''',
+  );
+
+  // Add the sticker attributes to Messages
+  await db.execute(
+    'ALTER TABLE $messagesTable ADD COLUMN stickerPackId TEXT;',
+  );
+  await db.execute(
+    'ALTER TABLE $messagesTable ADD COLUMN stickerId INTEGER;',
+  );
+
+  // Add the new preferences
+  await db.insert(
+    preferenceTable,
+    Preference(
+      'enableStickers',
+      typeBool,
+      'true',
+    ).toDatabaseJson(),
+  );
+  await db.insert(
+    preferenceTable,
+    Preference(
+      'autoDownloadStickersFromContacts',
+      typeBool,
+      'true',
+    ).toDatabaseJson(),
+  );
+}

lib/service/database/migrations/0000_stickers_hash_key.dart

@@ -0,0 +1,45 @@
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV17ToV18(Database db) async {
+  // Update messages
+  await db.execute(
+    'ALTER TABLE $messagesTable DROP COLUMN stickerId;',
+  );
+  await db.execute(
+    'ALTER TABLE $messagesTable ADD COLUMN stickerHashKey TEXT;',
+  );
+
+  // Drop stickers
+  await db.execute('DROP TABLE $stickerPacksTable;');
+  await db.execute('DROP TABLE $stickersTable;');
+
+  await db.execute(
+    '''
+    CREATE TABLE $stickersTable (
+      hashKey       TEXT PRIMARY KEY,
+      mediaType     TEXT NOT NULL,
+      desc          TEXT NOT NULL,
+      size          INTEGER NOT NULL,
+      width         INTEGER,
+      height        INTEGER,
+      hashes        TEXT NOT NULL,
+      urlSources    TEXT NOT NULL,
+      path          TEXT NOT NULL,
+      stickerPackId TEXT NOT NULL,
+      CONSTRAINT fk_sticker_pack FOREIGN KEY (stickerPackId) REFERENCES $stickerPacksTable (id)
+        ON DELETE CASCADE
+    )''',
+  );
+  await db.execute(
+    '''
+    CREATE TABLE $stickerPacksTable (
+      id             TEXT PRIMARY KEY,
+      name           TEXT NOT NULL,
+      description    TEXT NOT NULL,
+      hashAlgorithm  TEXT NOT NULL,
+      hashValue      TEXT NOT NULL,
+      stickerHashKey TEXT NOT NULL
+    )''',
+  );
+}

lib/service/database/migrations/0000_stickers_hash_key2.dart

@@ -0,0 +1,8 @@
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV18ToV19(Database db) async {
+  await db.execute(
+    'ALTER TABLE $stickerPacksTable DROP COLUMN stickerHashKey;',
+  );
+}

lib/service/database/migrations/0000_stickers_missing_attributes.dart

@@ -0,0 +1,12 @@
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:moxxyv2/service/database/helpers.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV19ToV20(Database db) async {
+  await db.execute(
+    'ALTER TABLE $stickerPacksTable ADD COLUMN restricted DEFAULT ${boolToInt(false)};',
+  );
+  await db.execute(
+    'ALTER TABLE $stickersTable ADD COLUMN suggests DEFAULT "";',
+  );
+}

lib/service/database/migrations/0000_stickers_missing_attributes2.dart

@@ -0,0 +1,19 @@
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:moxxyv2/service/database/helpers.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV20ToV21(Database db) async {
+  await db.execute(
+    'ALTER TABLE $stickerPacksTable DROP COLUMN restricted;',
+  );
+  await db.execute(
+    'ALTER TABLE $stickersTable DROP COLUMN suggests;',
+  );
+
+  await db.execute(
+    'ALTER TABLE $stickerPacksTable ADD COLUMN restricted INTEGER NOT NULL DEFAULT ${boolToInt(false)};',
+  );
+  await db.execute(
+    'ALTER TABLE $stickersTable ADD COLUMN suggests TEXT NOT NULL DEFAULT "";',
+  );
+}

lib/service/database/migrations/0000_stickers_missing_attributes3.dart

@@ -0,0 +1,12 @@
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV21ToV22(Database db) async {
+  await db.execute(
+    'ALTER TABLE $stickersTable DROP COLUMN suggests;',
+  );
+
+  await db.execute(
+    'ALTER TABLE $stickersTable ADD COLUMN suggests TEXT NOT NULL DEFAULT "{}";',
+  );
+}

lib/service/database/migrations/0000_stickers_privacy.dart

@@ -0,0 +1,14 @@
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:moxxyv2/shared/models/preference.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV24ToV25(Database db) async {
+  await db.insert(
+    preferenceTable,
+    Preference(
+      'isStickersNodePublic',
+      typeBool,
+      'true',
+    ).toDatabaseJson(),
+  );
+}

lib/service/database/migrations/0001_conversation_media_amount.dart

@@ -0,0 +1,36 @@
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV29ToV30(Database db) async {
+  await db.execute(
+    'ALTER TABLE $conversationsTable ADD COLUMN sharedMediaAmount INTEGER NOT NULL DEFAULT 0;',
+  );
+
+  // Get all conversations
+  final conversations = await db.query(
+    conversationsTable,
+  );
+
+  for (final conversation in conversations) {
+    // Count the amount of shared media
+    final jid = conversation['jid']! as String;
+    final result = Sqflite.firstIntValue(
+          await db.rawQuery(
+            'SELECT COUNT(*) FROM $mediaTable WHERE conversation_jid = ?',
+            [jid],
+          ),
+        ) ??
+        0;
+
+    final c = Map<String, Object?>.from(conversation)..remove('id');
+    await db.update(
+      conversationsTable,
+      {
+        ...c,
+        'sharedMediaAmount': result,
+      },
+      where: 'jid = ?',
+      whereArgs: [jid],
+    );
+  }
+}

lib/service/database/migrations/0001_conversation_primary_key.dart

@@ -0,0 +1,77 @@
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV27ToV28(Database db) async {
+  // Collect conversations so that we have a mapping id -> jid
+  final idMap = <int, String>{};
+  final conversations = await db.query(conversationsTable);
+  for (final c in conversations) {
+    idMap[c['id']! as int] = c['jid']! as String;
+  }
+
+  // Migrate the conversations
+  await db.execute(
+    '''
+    CREATE TABLE ${conversationsTable}_new (
+      jid TEXT NOT NULL PRIMARY KEY,
+      title TEXT NOT NULL,
+      avatarUrl TEXT NOT NULL,
+      lastChangeTimestamp INTEGER NOT NULL,
+      unreadCounter INTEGER NOT NULL,
+      open INTEGER NOT NULL,
+      muted INTEGER NOT NULL,
+      encrypted INTEGER NOT NULL,
+      lastMessageId INTEGER,
+      contactId TEXT,
+      contactAvatarPath TEXT,
+      contactDisplayName TEXT,
+      CONSTRAINT fk_last_message FOREIGN KEY (lastMessageId) REFERENCES $messagesTable (id),
+      CONSTRAINT fk_contact_id FOREIGN KEY (contactId) REFERENCES $contactsTable (id)
+        ON DELETE SET NULL
+    )''',
+  );
+  await db.execute(
+    'INSERT INTO ${conversationsTable}_new SELECT jid, title, avatarUrl, lastChangeTimestamp, unreadCounter, open, muted, encrypted, lastMessageId, contactid, contactAvatarPath, contactDisplayName from $conversationsTable',
+  );
+  await db.execute('DROP TABLE $conversationsTable;');
+  await db.execute(
+    'ALTER TABLE ${conversationsTable}_new RENAME TO $conversationsTable;',
+  );
+
+  // Add the jid column to shared media
+  await db.execute(
+    "ALTER TABLE $mediaTable ADD COLUMN conversation_jid TEXT NOT NULL DEFAULT '';",
+  );
+
+  // Update all shared media items
+  for (final entry in idMap.entries) {
+    await db.update(
+      mediaTable,
+      {
+        'conversation_jid': entry.value,
+      },
+      where: 'conversation_id = ?',
+      whereArgs: [entry.key],
+    );
+  }
+
+  // Migrate shared media
+  await db.execute(
+    '''
+    CREATE TABLE ${mediaTable}_new (
+      id INTEGER PRIMARY KEY AUTOINCREMENT,
+      path TEXT NOT NULL,
+      mime TEXT,
+      timestamp INTEGER NOT NULL,
+      conversation_jid TEXT NOT NULL,
+      message_id INTEGER,
+      FOREIGN KEY (conversation_jid) REFERENCES $conversationsTable (jid),
+      FOREIGN KEY (message_id) REFERENCES $messagesTable (id)
+    )''',
+  );
+  await db.execute(
+    'INSERT INTO ${mediaTable}_new SELECT id, path, mime, timestamp, message_id, conversation_jid from $mediaTable',
+  );
+  await db.execute('DROP TABLE $mediaTable;');
+  await db.execute('ALTER TABLE ${mediaTable}_new RENAME TO $mediaTable;');
+}

lib/service/database/migrations/0001_conversations_type.dart

@@ -0,0 +1,8 @@
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV30ToV31(Database db) async {
+  await db.execute(
+    'ALTER TABLE $conversationsTable ADD COLUMN type TEXT NOT NULL DEFAULT "chat";',
+  );
+}

lib/service/database/migrations/0001_debug_menu.dart

@@ -0,0 +1,15 @@
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:moxxyv2/service/database/helpers.dart';
+import 'package:moxxyv2/shared/models/preference.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV25ToV26(Database db) async {
+  await db.insert(
+    preferenceTable,
+    Preference(
+      'showDebugMenu',
+      typeBool,
+      boolToString(false),
+    ).toDatabaseJson(),
+  );
+}

lib/service/database/migrations/0001_remove_auto_accept_subscriptions.dart

@@ -0,0 +1,9 @@
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV28ToV29(Database db) async {
+  await db.delete(
+    preferenceTable,
+    where: 'key = "autoAcceptSubscriptionRequests"',
+  );
+}

lib/service/database/migrations/0001_subscriptions.dart

@@ -0,0 +1,9 @@
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV26ToV27(Database db) async {
+  await db.execute('''
+    CREATE TABLE $subscriptionsTable(
+      jid TEXT PRIMARY KEY
+    )''');
+}

lib/service/database/migrations/0002_file_metadata_table.dart

@@ -0,0 +1,226 @@
+import 'dart:convert';
+import 'package:moxxmpp/moxxmpp.dart';
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:moxxyv2/service/database/helpers.dart';
+import 'package:moxxyv2/service/files.dart';
+import 'package:moxxyv2/shared/models/file_metadata.dart';
+import 'package:path/path.dart' as path;
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV31ToV32(Database db) async {
+  // Create the tracking table
+  await db.execute('''
+    CREATE TABLE $fileMetadataTable (
+      id               TEXT NOT NULL PRIMARY KEY,
+      path             TEXT,
+      sourceUrls       TEXT,
+      mimeType         TEXT,
+      thumbnailType    TEXT,
+      thumbnailData    TEXT,
+      width            INTEGER,
+      height           INTEGER,
+      plaintextHashes  TEXT,
+      encryptionKey    TEXT,
+      encryptionIv     TEXT,
+      encryptionScheme TEXT,
+      cipherTextHashes TEXT,
+      filename         TEXT NOT NULL,
+      size             INTEGER
+    )''');
+  await db.execute('''
+    CREATE TABLE $fileMetadataHashesTable (
+      algorithm TEXT NOT NULL,
+      value     TEXT NOT NULL,
+      id        TEXT NOT NULL,
+      CONSTRAINT f_primarykey PRIMARY KEY (algorithm, value),
+      CONSTRAINT fk_id FOREIGN KEY (id) REFERENCES $fileMetadataTable (id)
+        ON DELETE CASCADE
+    )''');
+
+  // Add the file_metadata_id column
+  await db.execute(
+    'ALTER TABLE $messagesTable ADD COLUMN file_metadata_id TEXT DEFAULT NULL;',
+  );
+
+  // Migrate the media messages' attributes to new table
+  final messages = await db.query(
+    messagesTable,
+    where: 'isMedia = ${boolToInt(true)}',
+  );
+  for (final message in messages) {
+    // Do we know of a hash?
+    String id;
+    if (message['plaintextHashes'] != null) {
+      // Plaintext hashes available (SFS)
+      final plaintextHashes = deserializeHashMap(
+        message['plaintextHashes']! as String,
+      );
+      final result = await db.query(
+        fileMetadataHashesTable,
+        where: 'algorithm = ? AND value = ?',
+        whereArgs: [
+          plaintextHashes.entries.first.key,
+          plaintextHashes.entries.first.value,
+        ],
+        limit: 1,
+      );
+
+      if (result.isEmpty) {
+        final metadata = FileMetadata(
+          getStrongestHashFromMap(plaintextHashes) ??
+              DateTime.now().millisecondsSinceEpoch.toString(),
+          message['mediaUrl'] as String?,
+          message['srcUrl'] != null ? [message['srcUrl']! as String] : null,
+          message['mediaType'] as String?,
+          message['mediaSize'] as int?,
+          message['thumbnailData'] != null ? 'blurhash' : null,
+          message['thumbnailData'] as String?,
+          message['mediaWidth'] as int?,
+          message['mediaHeight'] as int?,
+          plaintextHashes,
+          message['key'] as String?,
+          message['iv'] as String?,
+          message['encryptionScheme'] as String?,
+          message['plaintextHashes'] == null
+              ? null
+              : deserializeHashMap(message['ciphertextHashes']! as String),
+          message['filename']! as String,
+        );
+
+        // Create the metadata
+        await db.insert(
+          fileMetadataTable,
+          metadata.toDatabaseJson(),
+        );
+        id = metadata.id;
+      } else {
+        id = result[0]['id']! as String;
+      }
+    } else {
+      // No plaintext hashes are available (OOB data)
+      int? size;
+      int? height;
+      int? width;
+      Map<HashFunction, String>? hashes;
+      String? filePath;
+      String? urlSource;
+      String? mediaType;
+      String? filename;
+      if (message['filename'] == null) {
+        // We are dealing with a sticker
+        assert(
+          message['stickerPackId'] != null,
+          'The message must contain a sticker',
+        );
+        assert(
+          message['stickerHashKey'] != null,
+          'The message must contain a sticker',
+        );
+        final sticker = (await db.query(
+          stickersTable,
+          where: 'stickerPackId = ? AND hashKey = ?',
+          whereArgs: [message['stickerPackId'], message['stickerHashKey']],
+          limit: 1,
+        ))
+            .first;
+        size = sticker['size']! as int;
+        width = sticker['width'] as int?;
+        height = sticker['height'] as int?;
+        hashes = deserializeHashMap(sticker['hashes']! as String);
+        filePath = sticker['path']! as String;
+        urlSource =
+            ((jsonDecode(sticker['urlSources']! as String) as List<dynamic>)
+                    .cast<String>())
+                .first;
+        mediaType = sticker['mediaType']! as String;
+        filename = path.basename(sticker['path']! as String);
+      } else {
+        size = message['mediaSize'] as int?;
+        width = message['mediaWidth'] as int?;
+        height = message['mediaHeight'] as int?;
+        filePath = message['mediaUrl'] as String?;
+        urlSource = message['srcUrl'] as String?;
+        mediaType = message['mediaType'] as String?;
+        filename = message['filename'] as String?;
+      }
+
+      final metadata = FileMetadata(
+        DateTime.now().millisecondsSinceEpoch.toString(),
+        filePath,
+        urlSource != null ? [urlSource] : null,
+        mediaType,
+        size,
+        message['thumbnailData'] != null ? 'blurhash' : null,
+        message['thumbnailData'] as String?,
+        width,
+        height,
+        hashes,
+        message['key'] as String?,
+        message['iv'] as String?,
+        message['encryptionScheme'] as String?,
+        null,
+        filename!,
+      );
+
+      // Create the metadata
+      await db.insert(
+        fileMetadataTable,
+        metadata.toDatabaseJson(),
+      );
+      id = metadata.id;
+    }
+
+    // Update the message
+    await db.update(
+      messagesTable,
+      {
+        'file_metadata_id': id,
+      },
+      where: 'id = ?',
+      whereArgs: [message['id']],
+    );
+  }
+
+  // Remove columns and add foreign key
+  await db.execute(
+    '''
+    CREATE TABLE ${messagesTable}_new (
+      id INTEGER PRIMARY KEY AUTOINCREMENT,
+      sender TEXT NOT NULL,
+      body TEXT,
+      timestamp INTEGER NOT NULL,
+      sid TEXT NOT NULL,
+      conversationJid TEXT NOT NULL,
+      isFileUploadNotification INTEGER NOT NULL,
+      encrypted INTEGER NOT NULL,
+      errorType INTEGER,
+      warningType INTEGER,
+      received INTEGER,
+      displayed INTEGER,
+      acked INTEGER,
+      originId TEXT,
+      quote_id INTEGER,
+      file_metadata_id TEXT,
+      isDownloading INTEGER NOT NULL,
+      isUploading INTEGER NOT NULL,
+      isRetracted INTEGER,
+      isEdited INTEGER NOT NULL,
+      reactions TEXT NOT NULL,
+      containsNoStore INTEGER NOT NULL,
+      stickerPackId   TEXT,
+      stickerHashKey  TEXT,
+      pseudoMessageType INTEGER,
+      pseudoMessageData TEXT,
+      CONSTRAINT fk_quote FOREIGN KEY (quote_id) REFERENCES $messagesTable (id)
+      CONSTRAINT fk_file_metadata FOREIGN KEY (file_metadata_id) REFERENCES $fileMetadataTable (id)
+    )''',
+  );
+
+  await db.execute(
+    'INSERT INTO ${messagesTable}_new SELECT id, sender, body, timestamp, sid, conversationJid, isFileUploadNotification, encrypted, errorType, warningType, received, displayed, acked, originId, quote_id, file_metadata_id, isDownloading, isUploading, isRetracted, isEdited, reactions, containsNoStore, stickerPackId, stickerHashKey, pseudoMessageType, pseudoMessageData FROM $messagesTable',
+  );
+  await db.execute('DROP TABLE $messagesTable');
+  await db.execute(
+    'ALTER TABLE ${messagesTable}_new RENAME TO $messagesTable;',
+  );
+}

lib/service/database/migrations/0002_indices.dart

@@ -0,0 +1,24 @@
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV36ToV37(Database db) async {
+  // Queries against messages by id (and sid/originId happen regularly)
+  await db.execute(
+    'CREATE INDEX idx_messages_id ON $messagesTable (id, sid, originId)',
+  );
+
+  // Conversations are often queried by their jid
+  await db.execute(
+    'CREATE INDEX idx_conversation_id ON $conversationsTable (jid)',
+  );
+
+  // Reactions must be quickly queried
+  await db.execute(
+    'CREATE INDEX idx_reactions_message_id ON $reactionsTable (message_id, senderJid)',
+  );
+
+  // File metadata should also be quickly queriable by its id
+  await db.execute(
+    'CREATE INDEX idx_file_metadata_message_id ON $fileMetadataTable (id)',
+  );
+}

lib/service/database/migrations/0002_reactions.dart

@@ -0,0 +1,60 @@
+import 'dart:convert';
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:moxxyv2/service/database/helpers.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV34ToV35(Database db) async {
+  // Create the table
+  await db.execute('''
+    CREATE TABLE $reactionsTable (
+      senderJid  TEXT NOT NULL,
+      emoji      TEXT NOT NULL,
+      message_id INTEGER NOT NULL,
+      CONSTRAINT pk_sender PRIMARY KEY (senderJid, emoji, message_id),
+      CONSTRAINT fk_message FOREIGN KEY (message_id) REFERENCES $messagesTable (id)
+        ON DELETE CASCADE
+    )''');
+
+  // Figure out our JID
+  final rawJid = await db.query(
+    xmppStateTable,
+    where: "key = 'jid'",
+    limit: 1,
+  );
+  String? jid;
+  if (rawJid.isNotEmpty) {
+    jid = rawJid.first['value']! as String;
+  }
+
+  // Migrate messages
+  final messages = await db.query(
+    messagesTable,
+    where: "reactions IS NOT '[]'",
+  );
+  for (final message in messages) {
+    final reactions =
+        (jsonDecode(message['reactions']! as String) as List<dynamic>)
+            .cast<Map<String, Object?>>();
+
+    for (final reaction in reactions) {
+      final senders = [
+        ...reaction['senders']! as List<String>,
+        if (intToBool(reaction['reactedBySelf']! as int) && jid != null) jid,
+      ];
+
+      for (final sender in senders) {
+        await db.insert(
+          reactionsTable,
+          {
+            'senderJid': sender,
+            'emoji': reaction['emoji']! as String,
+            'message_id': message['id']! as int,
+          },
+        );
+      }
+    }
+  }
+
+  // Remove the column
+  await db.execute('ALTER TABLE $messagesTable DROP COLUMN reactions');
+}

lib/service/database/migrations/0002_reactions_2.dart

@@ -0,0 +1,15 @@
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV35ToV36(Database db) async {
+  await db.execute('DROP TABLE $reactionsTable');
+  await db.execute('''
+    CREATE TABLE $reactionsTable (
+      senderJid  TEXT NOT NULL,
+      emoji      TEXT NOT NULL,
+      message_id INTEGER NOT NULL,
+      CONSTRAINT pk_sender PRIMARY KEY (senderJid, emoji, message_id),
+      CONSTRAINT fk_message FOREIGN KEY (message_id) REFERENCES $messagesTable (id)
+        ON DELETE CASCADE
+    )''');
+}

lib/service/database/migrations/0002_shared_media.dart

@@ -0,0 +1,14 @@
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV33ToV34(Database db) async {
+  // Remove the shared media counter...
+  await db.execute(
+    'ALTER TABLE $conversationsTable DROP COLUMN sharedMediaAmount',
+  );
+
+  // ... and the entire table.
+  await db.execute(
+    'DROP TABLE $mediaTable',
+  );
+}

lib/service/database/migrations/0002_sticker_metadata.dart

@@ -0,0 +1,113 @@
+import 'dart:convert';
+import 'package:collection/collection.dart';
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:path/path.dart' as path;
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+Future<void> upgradeFromV32ToV33(Database db) async {
+  final stickers = await db.query(stickersTable);
+
+  await db.execute(
+    '''
+    CREATE TABLE ${stickersTable}_new (
+      id               TEXT PRIMARY KEY,
+      desc             TEXT NOT NULL,
+      suggests         TEXT NOT NULL,
+      file_metadata_id TEXT NOT NULL,
+      stickerPackId  TEXT NOT NULL,
+      CONSTRAINT fk_sticker_pack FOREIGN KEY (stickerPackId) REFERENCES $stickerPacksTable (id)
+        ON DELETE CASCADE,
+      CONSTRAINT fk_file_metadata FOREIGN KEY (file_metadata_id) REFERENCES $fileMetadataTable (id)
+    )''',
+  );
+
+  // Mapping stickerHashKey -> fileMetadataId
+  final stickerHashMap = <String, String>{};
+  for (final sticker in stickers) {
+    final hashes =
+        (jsonDecode(sticker['hashes']! as String) as Map<String, dynamic>)
+            .cast<String, String>();
+
+    final buffer = StringBuffer();
+    for (var i = 0; i < hashes.length; i++) {
+      buffer.write('(algorithm = ? AND value = ?) AND');
+    }
+    final query = buffer.toString();
+
+    final rawFm = await db.query(
+      fileMetadataHashesTable,
+      where: query.substring(0, query.length - 1 - 3),
+      whereArgs: hashes.entries
+          .map<List<String>>((entry) => [entry.key, entry.value])
+          .flattened
+          .toList(),
+      limit: 1,
+    );
+
+    String fileMetadataId;
+    if (rawFm.isEmpty) {
+      // Create the metadata
+      fileMetadataId = DateTime.now().toString();
+      await db.insert(
+        fileMetadataTable,
+        {
+          'id': fileMetadataId,
+          'path': sticker['path']! as String,
+          'size': sticker['size']! as int,
+          'width': sticker['width'] as int?,
+          'height': sticker['height'] as int?,
+          'plaintextHashes': sticker['hashes']! as String,
+          'mimeType': sticker['mediaType']! as String,
+          'sourceUrls': sticker['urlSources'],
+          'filename': path.basename(sticker['path']! as String),
+        },
+      );
+
+      // Create hash pointers
+      for (final hashEntry in hashes.entries) {
+        await db.insert(
+          fileMetadataHashesTable,
+          {
+            'algorithm': hashEntry.key,
+            'value': hashEntry.value,
+            'id': fileMetadataId,
+          },
+        );
+      }
+    } else {
+      fileMetadataId = rawFm.first['id']! as String;
+    }
+
+    final hashKey = sticker['hashKey']! as String;
+    stickerHashMap[hashKey] = fileMetadataId;
+    await db.insert(
+      '${stickersTable}_new',
+      {
+        'id': hashKey,
+        'desc': sticker['desc']! as String,
+        'suggests': sticker['suggests']! as String,
+        'file_metadata_id': fileMetadataId,
+        'stickerPackId': sticker['stickerPackId']! as String,
+      },
+    );
+  }
+
+  // Rename the table
+  await db.execute('DROP TABLE $stickersTable');
+  await db.execute('ALTER TABLE ${stickersTable}_new RENAME TO $stickersTable');
+
+  // Migrate messages
+  for (final stickerEntry in stickerHashMap.entries) {
+    await db.update(
+      messagesTable,
+      {
+        'file_metadata_id': stickerEntry.value,
+      },
+      where: 'stickerHashKey = ?',
+      whereArgs: [stickerEntry.key],
+    );
+  }
+
+  // Remove the hash key from messages
+  await db.execute('ALTER TABLE $messagesTable DROP COLUMN stickerHashKey');
+}

lib/service/files.dart

@@ -0,0 +1,340 @@
+import 'dart:convert';
+import 'dart:io';
+import 'dart:ui';
+import 'package:get_it/get_it.dart';
+import 'package:logging/logging.dart';
+import 'package:moxxmpp/moxxmpp.dart';
+import 'package:moxxyv2/service/cryptography/cryptography.dart';
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:moxxyv2/service/database/database.dart';
+import 'package:moxxyv2/service/httpfiletransfer/location.dart';
+import 'package:moxxyv2/service/not_specified.dart';
+import 'package:moxxyv2/shared/models/file_metadata.dart';
+import 'package:path/path.dart' as path;
+import 'package:path_provider/path_provider.dart';
+
+/// A class for returning whether a file metadata element was just created or retrieved.
+class FileMetadataWrapper {
+  FileMetadataWrapper(
+    this.fileMetadata,
+    this.retrieved,
+  );
+
+  /// The file metadata.
+  FileMetadata fileMetadata;
+
+  /// Indicates whether the file metadata already exists (true) or
+  /// if it has been created (false).
+  bool retrieved;
+}
+
+/// Returns the strongest hash from [map], if [map] is not null. If no known hash is found
+/// or [map] is null, returns null.
+String? getStrongestHashFromMap(Map<HashFunction, String>? map) {
+  if (map == null) {
+    return null;
+  }
+
+  return map[HashFunction.blake2b512] ??
+      map[HashFunction.blake2b256] ??
+      map[HashFunction.sha3_512] ??
+      map[HashFunction.sha3_256] ??
+      map[HashFunction.sha512] ??
+      map[HashFunction.sha256];
+}
+
+/// Calculates the path for a given file with filename [filename] and the optional
+/// plaintext hashes [hashes]. If the base directory for the file does not exist, then it
+/// will be created.
+Future<String> computeCachedPathForFile(
+  String filename,
+  Map<HashFunction, String>? hashes,
+) async {
+  final basePath = path.join(
+    (await getApplicationDocumentsDirectory()).path,
+    'media',
+  );
+  final baseDir = Directory(basePath);
+
+  if (!baseDir.existsSync()) {
+    await baseDir.create(recursive: true);
+  }
+
+  // Keep the extension of the file. Otherwise Android will be really confused
+  // as to what it should open the file with.
+  final ext = path.extension(filename);
+  final hash = getStrongestHashFromMap(hashes)?.replaceAll('/', '_');
+  return path.join(
+    basePath,
+    hash != null
+        ? '$hash.$ext'
+        : '$filename.${DateTime.now().millisecondsSinceEpoch}.$ext',
+  );
+}
+
+class FilesService {
+  // Logging.
+  final Logger _log = Logger('FilesService');
+
+  Future<void> createMetadataHashEntries(
+    Map<HashFunction, String> plaintextHashes,
+    String metadataId,
+  ) async {
+    final db = GetIt.I.get<DatabaseService>().database;
+    for (final hash in plaintextHashes.entries) {
+      await db.insert(
+        fileMetadataHashesTable,
+        {
+          'algorithm': hash.key.toName(),
+          'value': hash.value,
+          'id': metadataId,
+        },
+      );
+    }
+  }
+
+  Future<FileMetadata?> getFileMetadataFromFile(FileMetadata metadata) async {
+    final hash = metadata.plaintextHashes?[HashFunction.sha256] ??
+        await GetIt.I
+            .get<CryptographyService>()
+            .hashFile(metadata.path!, HashFunction.sha256);
+    final fm = await getFileMetadataFromHash({
+      HashFunction.sha256: hash,
+    });
+
+    if (fm != null) {
+      return fm;
+    }
+
+    final result = await addFileMetadataFromData(
+      metadata.copyWith(
+        plaintextHashes: {
+          ...metadata.plaintextHashes ?? {},
+          HashFunction.sha256: hash,
+        },
+      ),
+    );
+    await createMetadataHashEntries(result.plaintextHashes!, result.id);
+    return result;
+  }
+
+  Future<FileMetadata?> getFileMetadataFromHash(
+    Map<HashFunction, String>? plaintextHashes,
+  ) async {
+    if (plaintextHashes?.isEmpty ?? true) {
+      return null;
+    }
+
+    final db = GetIt.I.get<DatabaseService>().database;
+    final values = List<String>.empty(growable: true);
+    final query = plaintextHashes!.entries.map((entry) {
+      values
+        ..add(entry.key.toName())
+        ..add(entry.value);
+      return '(algorithm = ? AND value = ?)';
+    }).join(' OR ');
+    final hashes = await db.query(
+      fileMetadataHashesTable,
+      where: query,
+      whereArgs: values,
+      limit: 1,
+    );
+    if (hashes.isEmpty) {
+      return null;
+    }
+
+    final result = await db.query(
+      fileMetadataTable,
+      where: 'id = ?',
+      whereArgs: [hashes[0]['id']! as String],
+      limit: 1,
+    );
+    if (result.isEmpty) {
+      return null;
+    }
+
+    return FileMetadata.fromDatabaseJson(result[0]);
+  }
+
+  /// Create a FileMetadata entry if we do not know the plaintext hashes described in
+  /// [location].
+  /// If we know of at least one hash, return that FileMetadata element.
+  ///
+  /// If [createHashPointers] is true and we have to create a new FileMetadata element,
+  /// then also create the hash pointers, if plaintext hashes are specified. If no
+  /// plaintext hashes are specified or [createHashPointers] is false, no pointers will be
+  /// created.
+  Future<FileMetadataWrapper> createFileMetadataIfRequired(
+    MediaFileLocation location,
+    String? mimeType,
+    int? size,
+    Size? dimensions,
+    String? thubnailType,
+    String? thumbnailData, {
+    bool createHashPointers = true,
+    String? path,
+  }) async {
+    if (location.plaintextHashes?.isNotEmpty ?? false) {
+      final result = await getFileMetadataFromHash(location.plaintextHashes);
+      if (result != null) {
+        _log.finest('Not creating new metadata as we found the hash');
+        return FileMetadataWrapper(
+          result,
+          true,
+        );
+      }
+    }
+
+    final db = GetIt.I.get<DatabaseService>().database;
+    final fm = FileMetadata(
+      getStrongestHashFromMap(location.plaintextHashes) ??
+          DateTime.now().millisecondsSinceEpoch.toString(),
+      path,
+      location.urls,
+      mimeType,
+      size,
+      thubnailType,
+      thumbnailData,
+      dimensions?.width.toInt(),
+      dimensions?.height.toInt(),
+      location.plaintextHashes,
+      location.key != null ? base64Encode(location.key!) : null,
+      location.iv != null ? base64Encode(location.iv!) : null,
+      location.encryptionScheme,
+      location.ciphertextHashes,
+      location.filename,
+    );
+    await db.insert(fileMetadataTable, fm.toDatabaseJson());
+
+    if ((location.plaintextHashes?.isNotEmpty ?? false) && createHashPointers) {
+      await createMetadataHashEntries(
+        location.plaintextHashes!,
+        fm.id,
+      );
+    }
+
+    return FileMetadataWrapper(
+      fm,
+      false,
+    );
+  }
+
+  Future<void> removeFileMetadata(String id) async {
+    await GetIt.I.get<DatabaseService>().database.delete(
+      fileMetadataTable,
+      where: 'id = ?',
+      whereArgs: [id],
+    );
+  }
+
+  Future<FileMetadata> updateFileMetadata(
+    String id, {
+    Object? path = notSpecified,
+    int? size,
+    String? encryptionScheme,
+    String? encryptionKey,
+    String? encryptionIv,
+    List<String>? sourceUrls,
+    int? width,
+    int? height,
+    String? mimeType,
+    Map<String, String>? plaintextHashes,
+    Map<String, String>? ciphertextHashes,
+  }) async {
+    final db = GetIt.I.get<DatabaseService>().database;
+    final m = <String, dynamic>{};
+
+    if (path != notSpecified) {
+      m['path'] = path as String?;
+    }
+    if (encryptionScheme != null) {
+      m['encryptionScheme'] = encryptionScheme;
+    }
+    if (size != null) {
+      m['size'] = size;
+    }
+    if (encryptionKey != null) {
+      m['encryptionKey'] = encryptionKey;
+    }
+    if (encryptionIv != null) {
+      m['encryptionIv'] = encryptionIv;
+    }
+    if (sourceUrls != null) {
+      m['sourceUrl'] = jsonEncode(sourceUrls);
+    }
+    if (width != null) {
+      m['width'] = width;
+    }
+    if (height != null) {
+      m['height'] = height;
+    }
+    if (mimeType != null) {
+      m['mimeType'] = mimeType;
+    }
+    if (plaintextHashes != null) {
+      m['plaintextHashes'] = jsonEncode(plaintextHashes);
+    }
+    if (ciphertextHashes != null) {
+      m['cipherTextHashes'] = jsonEncode(ciphertextHashes);
+    }
+
+    final result = await db.updateAndReturn(
+      fileMetadataTable,
+      m,
+      where: 'id = ?',
+      whereArgs: [id],
+    );
+
+    return FileMetadata.fromDatabaseJson(result);
+  }
+
+  /// Removes the file metadata described by [metadata] if it is referenced by exactly 0
+  /// messages and no stickers use this file. If the file is referenced by > 1 messages
+  /// or a sticker, does nothing.
+  Future<void> removeFileIfNotReferenced(FileMetadata metadata) async {
+    final db = GetIt.I.get<DatabaseService>().database;
+    final messagesCount = await db.count(
+      messagesTable,
+      'file_metadata_id = ?',
+      [metadata.id],
+    );
+    final stickersCount = await db.count(
+      stickersTable,
+      'file_metadata_id = ?',
+      [metadata.id],
+    );
+
+    if (messagesCount == 0 && stickersCount == 0) {
+      _log.finest(
+        'Removing file metadata as no stickers and no messages reference it',
+      );
+      await removeFileMetadata(metadata.id);
+
+      // Only remove the file if we have a path
+      if (metadata.path != null) {
+        try {
+          await File(metadata.path!).delete();
+        } catch (ex) {
+          _log.warning('Failed to remove file ${metadata.path!}: $ex');
+        }
+      } else {
+        _log.info('Not removing file as there is no path associated with it');
+      }
+    } else {
+      _log.info(
+        'Not removing file as $messagesCount messages and $stickersCount stickers reference this file',
+      );
+    }
+  }
+
+  Future<FileMetadata> addFileMetadataFromData(
+    FileMetadata metadata,
+  ) async {
+    final result =
+        await GetIt.I.get<DatabaseService>().database.insertAndReturn(
+              fileMetadataTable,
+              metadata.toDatabaseJson(),
+            );
+    return FileMetadata.fromDatabaseJson(result);
+  }
+}

lib/service/helpers.dart

@@ -3,6 +3,8 @@ import 'dart:ui';
 import 'package:flutter/foundation.dart';
 import 'package:moxxmpp/moxxmpp.dart';
 import 'package:moxxyv2/i18n/strings.g.dart';
+import 'package:moxxyv2/shared/helpers.dart';
+import 'package:moxxyv2/shared/models/message.dart';
 import 'package:native_imaging/native_imaging.dart' as native;
 
 Future<String?> _generateBlurhashThumbnailImpl(String path) async {
@@ -65,11 +67,93 @@ Future<String?> generateBlurhashThumbnail(String path) async {
 String xmppErrorToTranslatableString(XmppError error) {
   if (error is StartTLSFailedError) {
     return t.errors.login.startTlsFailed;
-  } else if (error is SaslFailedError) {
+  } else if (error is SaslError) {
     return t.errors.login.saslFailed;
-  } else if (error is NoConnectionError) {
+  } else if (error is NoConnectionPossibleError) {
     return t.errors.login.noConnection;
   }
 
   return t.errors.login.unspecified;
 }
+
+HashFunction getStickerHashKeyType(Map<HashFunction, String> hashes) {
+  if (hashes.containsKey(HashFunction.blake2b512)) {
+    return HashFunction.blake2b512;
+  } else if (hashes.containsKey(HashFunction.blake2b256)) {
+    return HashFunction.blake2b256;
+  } else if (hashes.containsKey(HashFunction.sha3_512)) {
+    return HashFunction.sha3_512;
+  } else if (hashes.containsKey(HashFunction.sha3_256)) {
+    return HashFunction.sha3_256;
+  } else if (hashes.containsKey(HashFunction.sha512)) {
+    return HashFunction.sha512;
+  } else if (hashes.containsKey(HashFunction.sha256)) {
+    return HashFunction.sha256;
+  }
+
+  assert(false, 'No valid hash found');
+  return HashFunction.sha256;
+}
+
+// TODO(PapaTutuWawa): Replace with getStrongestHash
+String getStickerHashKey(Map<HashFunction, String> hashes) {
+  final key = getStickerHashKeyType(hashes);
+  return '$key:${hashes[key]}';
+}
+
+/// Return a human readable string describing an unrecoverable error event [event].
+String getUnrecoverableErrorString(NonRecoverableErrorEvent event) {
+  final error = event.error;
+  if (error is SaslAccountDisabledError) {
+    return t.errors.connection.saslAccountDisabled;
+  } else if (error is SaslCredentialsExpiredError ||
+      error is SaslNotAuthorizedError) {
+    return t.errors.connection.saslInvalidCredentials;
+  }
+
+  return t.errors.connection.unrecoverable;
+}
+
+/// Creates the fallback body for quoted messages.
+/// If the quoted message contains text, it simply quotes the text.
+/// If it contains a media file, the messageEmoji (usually an emoji
+/// representing the mime type) is shown together with the file size
+/// (from experience this information is sufficient, as most clients show
+/// the file size, and including time information might be confusing and a
+/// potential privacy issue).
+/// This information is complemented either the srcUrl or – if unavailable –
+/// by the body of the quoted message. For non-media messages, we always use
+/// the body as fallback.
+String? createFallbackBodyForQuotedMessage(Message? quotedMessage) {
+  if (quotedMessage == null) {
+    return null;
+  }
+
+  if (quotedMessage.isMedia) {
+    // Create formatted size string, if size is stored
+    String quoteMessageSize;
+    if (quotedMessage.fileMetadata!.size != null &&
+        quotedMessage.fileMetadata!.size! > 0) {
+      quoteMessageSize =
+          '(${fileSizeToString(quotedMessage.fileMetadata!.size!)}) ';
+    } else {
+      quoteMessageSize = '';
+    }
+
+    // Create media url string, or use body if no srcUrl is stored
+    String quotedMediaUrl;
+    if (quotedMessage.fileMetadata!.sourceUrls != null &&
+        quotedMessage.fileMetadata!.sourceUrls!.first.isNotEmpty) {
+      quotedMediaUrl = '• ${quotedMessage.fileMetadata!.sourceUrls!.first}';
+    } else if (quotedMessage.body.isNotEmpty) {
+      quotedMediaUrl = '• ${quotedMessage.body}';
+    } else {
+      quotedMediaUrl = '';
+    }
+
+    // Concatenate emoji, size string, and media url and return
+    return '${quotedMessage.messageEmoji} $quoteMessageSize$quotedMediaUrl';
+  } else {
+    return quotedMessage.body;
+  }
+}

lib/service/httpfiletransfer/client.dart

@@ -0,0 +1,146 @@
+import 'dart:async';
+import 'dart:io';
+import 'package:meta/meta.dart';
+import 'package:moxxyv2/service/httpfiletransfer/helpers.dart';
+
+typedef ProgressCallback = void Function(int total, int current);
+
+@immutable
+class HttpPeekResult {
+  const HttpPeekResult(this.contentType, this.contentLength);
+  final String? contentType;
+  final int? contentLength;
+}
+
+/// Download the file found at [uri] into the file [destination]. [onProgress] is
+/// called whenever new data has been downloaded.
+///
+/// Returns the status code if the server responded. If an error occurs, returns null.
+Future<int?> downloadFile(
+  Uri uri,
+  String destination,
+  ProgressCallback onProgress,
+) async {
+  // TODO(Unknown): How do we close fileSink? Do we have to?
+  IOSink? fileSink;
+  final client = HttpClient();
+  try {
+    final req = await client.getUrl(uri);
+    final resp = await req.close();
+
+    if (!isRequestOkay(resp.statusCode)) {
+      client.close(force: true);
+      return resp.statusCode;
+    }
+
+    // The size of the remote file
+    final length = resp.contentLength;
+
+    fileSink = File(destination).openWrite(mode: FileMode.append);
+    var bytes = 0;
+    final downloadCompleter = Completer<void>();
+    unawaited(
+      resp
+          .transform(
+            StreamTransformer<List<int>, List<int>>.fromHandlers(
+              handleData: (data, sink) {
+                bytes += data.length;
+                onProgress(length, bytes);
+
+                sink.add(data);
+              },
+              handleDone: (sink) {
+                downloadCompleter.complete();
+              },
+            ),
+          )
+          .pipe(fileSink),
+    );
+
+    // Wait for the download to complete
+    await downloadCompleter.future;
+    client.close(force: true);
+    //await fileSink.close();
+
+    return resp.statusCode;
+  } catch (ex) {
+    client.close(force: true);
+    //await fileSink?.close();
+    return null;
+  }
+}
+
+/// Upload the file found at [filePath] to [destination]. [headers] are HTTP headers
+/// that are added to the PUT request. [onProgress] is called whenever new data has
+/// been downloaded.
+///
+/// Returns the status code if the server responded. If an error occurs, returns null.
+Future<int?> uploadFile(
+  Uri destination,
+  Map<String, String> headers,
+  String filePath,
+  ProgressCallback onProgress,
+) async {
+  final client = HttpClient();
+  try {
+    final req = await client.putUrl(destination);
+    final file = File(filePath);
+    final length = await file.length();
+    req.contentLength = length;
+
+    // Set all known headers
+    headers.forEach((headerName, headerValue) {
+      req.headers.set(headerName, headerValue);
+    });
+
+    var bytes = 0;
+    final stream = file.openRead().transform(
+          StreamTransformer<List<int>, List<int>>.fromHandlers(
+            handleData: (data, sink) {
+              bytes += data.length;
+              onProgress(length, bytes);
+
+              sink.add(data);
+            },
+            handleDone: (sink) {
+              sink.close();
+            },
+          ),
+        );
+    await req.addStream(stream);
+    final resp = await req.close();
+
+    return resp.statusCode;
+  } catch (ex) {
+    client.close(force: true);
+    return null;
+  }
+}
+
+/// Sends a HEAD request to [uri].
+///
+/// Returns the content type and content length if the server responded. If an error
+/// occurs, returns null.
+Future<HttpPeekResult?> peekUrl(Uri uri) async {
+  final client = HttpClient();
+
+  try {
+    final req = await client.headUrl(uri);
+    final resp = await req.close();
+
+    if (!isRequestOkay(resp.statusCode)) {
+      client.close(force: true);
+      return null;
+    }
+
+    client.close(force: true);
+    final contentType = resp.headers['Content-Type'];
+    return HttpPeekResult(
+      contentType != null && contentType.isNotEmpty ? contentType.first : null,
+      resp.contentLength,
+    );
+  } catch (ex) {
+    client.close(force: true);
+    return null;
+  }
+}

lib/service/httpfiletransfer/helpers.dart

@@ -1,40 +1,4 @@
-import 'dart:io';
-import 'package:dio/dio.dart';
-import 'package:external_path/external_path.dart';
-import 'package:moxxyv2/shared/helpers.dart';
-import 'package:path/path.dart' as path;
-
-/// Calculates the path for a given file to be saved to and, if neccessary, create it.
-Future<String> getDownloadPath(String filename, String conversationJid, String? mime) async {
-  String type;
-  var prependMoxxy = true;
-  if (mime != null && ['image/', 'video/'].any((e) => mime.startsWith(e))) {
-    type = ExternalPath.DIRECTORY_PICTURES;
-  } else {
-    type = ExternalPath.DIRECTORY_DOWNLOADS;
-    prependMoxxy = false;
-  }
-  
-  final externalDir = await ExternalPath.getExternalStoragePublicDirectory(type);
-  final fileDirectory = prependMoxxy ? path.join(externalDir, 'Moxxy', conversationJid) : externalDir;
-  final dir = Directory(fileDirectory);
-  if (!dir.existsSync()) {
-    await dir.create(recursive: true);
-  }
-
-  var i = 0;
-  while (true) {
-    final filenameSuffix = i == 0 ? '' : '($i)';
-    final suffixedFilename = filenameWithSuffix(filename, filenameSuffix);
-
-    final filePath = path.join(fileDirectory, suffixedFilename);
-    if (!File(filePath).existsSync()) {
-      return filePath;
-    }
-
-    i++;
-  }
-}
+import 'package:moxxyv2/service/httpfiletransfer/client.dart';
 
 /// Returns true if the request was successful based on [statusCode].
 /// Based on https://developer.mozilla.org/en-US/docs/Web/HTTP/Status
@@ -42,9 +6,8 @@ bool isRequestOkay(int? statusCode) {
   return statusCode != null && statusCode >= 200 && statusCode <= 399;
 }
 
-class FileMetadata {
-
-  const FileMetadata({ this.mime, this.size });
+class FileUploadMetadata {
+  const FileUploadMetadata({this.mime, this.size});
   final String? mime;
   final int? size;
 }
@@ -52,16 +15,11 @@ class FileMetadata {
 /// Returns the size of the file at [url] in octets. If an error occurs or the server
 /// does not specify the Content-Length header, null is returned.
 /// See https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Length
-Future<FileMetadata> peekFile(String url) async {
-  final response = await Dio().headUri<dynamic>(Uri.parse(url));
+Future<FileUploadMetadata> peekFile(String url) async {
+  final result = await peekUrl(Uri.parse(url));
 
-  if (!isRequestOkay(response.statusCode)) return const FileMetadata();
-
-  final contentLengthHeaders = response.headers['Content-Length'];
-  final contentTypeHeaders = response.headers['Content-Type'];
-  
-  return FileMetadata(
-    mime: contentTypeHeaders?.first,
-    size: contentLengthHeaders != null && contentLengthHeaders.isNotEmpty ? int.parse(contentLengthHeaders.first) : null,
+  return FileUploadMetadata(
+    mime: result?.contentType,
+    size: result?.contentLength,
   );
 }

lib/service/httpfiletransfer/httpfiletransfer.dart

@@ -1,10 +1,8 @@
 import 'dart:async';
 import 'dart:collection';
-import 'dart:convert';
 import 'dart:io';
 import 'dart:math';
 import 'package:connectivity_plus/connectivity_plus.dart';
-import 'package:dio/dio.dart' as dio;
 import 'package:get_it/get_it.dart';
 import 'package:logging/logging.dart';
 import 'package:mime/mime.dart';
@@ -14,9 +12,11 @@ import 'package:moxxyv2/service/connectivity.dart';
 import 'package:moxxyv2/service/conversation.dart';
 import 'package:moxxyv2/service/cryptography/cryptography.dart';
 import 'package:moxxyv2/service/cryptography/types.dart';
-import 'package:moxxyv2/service/database/database.dart';
+import 'package:moxxyv2/service/files.dart';
+import 'package:moxxyv2/service/httpfiletransfer/client.dart' as client;
 import 'package:moxxyv2/service/httpfiletransfer/helpers.dart';
 import 'package:moxxyv2/service/httpfiletransfer/jobs.dart';
+import 'package:moxxyv2/service/httpfiletransfer/location.dart';
 import 'package:moxxyv2/service/message.dart';
 import 'package:moxxyv2/service/notifications.dart';
 import 'package:moxxyv2/service/service.dart';
@@ -32,37 +32,39 @@ import 'package:uuid/uuid.dart';
 
 /// This service is responsible for managing the up- and download of files using Http.
 class HttpFileTransferService {
-  HttpFileTransferService()
-    : _uploadQueue = Queue<FileUploadJob>(),
-      _downloadQueue = Queue<FileDownloadJob>(),
-      _uploadLock = Lock(),
-      _downloadLock = Lock(),
-      _log = Logger('HttpFileTransferService');
+  HttpFileTransferService() {
+    GetIt.I.get<ConnectivityService>().stream.listen(_onConnectivityChanged);
+  }
 
-  final Logger _log;
+  final Logger _log = Logger('HttpFileTransferService');
 
   /// Queues for tracking up- and download tasks
-  final Queue<FileDownloadJob> _downloadQueue;
-  final Queue<FileUploadJob> _uploadQueue;
+  final Queue<FileDownloadJob> _downloadQueue = Queue<FileDownloadJob>();
+  final Queue<FileUploadJob> _uploadQueue = Queue<FileUploadJob>();
+
   /// The currently running job and their lock
   FileUploadJob? _currentUploadJob;
   FileDownloadJob? _currentDownloadJob;
 
   /// Locks for upload and download state
-  final Lock _uploadLock;
-  final Lock _downloadLock;
+  final Lock _uploadLock = Lock();
+  final Lock _downloadLock = Lock();
 
   /// Called by the ConnectivityService if the connection got lost but then was regained.
-  Future<void> onConnectivityChanged(bool regained) async {
-    if (!regained) return;
+  Future<void> _onConnectivityChanged(ConnectivityEvent event) async {
+    if (!event.regained) return;
 
     await _uploadLock.synchronized(() async {
       if (_currentUploadJob != null) {
-        _log.finest('Connectivity regained and there is still an upload job. Restarting it.');
+        _log.finest(
+          'Connectivity regained and there is still an upload job. Restarting it.',
+        );
         unawaited(_performFileUpload(_currentUploadJob!));
       } else {
         if (_uploadQueue.isNotEmpty) {
-          _log.finest('Connectivity regained and the upload queue is not empty. Starting a new upload job.');
+          _log.finest(
+            'Connectivity regained and the upload queue is not empty. Starting a new upload job.',
+          );
           _currentUploadJob = _uploadQueue.removeFirst();
           unawaited(_performFileUpload(_currentUploadJob!));
         }
@@ -71,11 +73,15 @@ class HttpFileTransferService {
 
     await _downloadLock.synchronized(() async {
       if (_currentDownloadJob != null) {
-        _log.finest('Connectivity regained and there is still a download job. Restarting it.');
+        _log.finest(
+          'Connectivity regained and there is still a download job. Restarting it.',
+        );
         unawaited(_performFileDownload(_currentDownloadJob!));
       } else {
         if (_downloadQueue.isNotEmpty) {
-          _log.finest('Connectivity regained and the download queue is not empty. Starting a new download job.');
+          _log.finest(
+            'Connectivity regained and the download queue is not empty. Starting a new download job.',
+          );
           _currentDownloadJob = _downloadQueue.removeFirst();
           unawaited(_performFileDownload(_currentDownloadJob!));
         }
@@ -102,44 +108,38 @@ class HttpFileTransferService {
 
   /// Queue the download job [job] to be performed.
   Future<void> downloadFile(FileDownloadJob job) async {
-    var canDownload = false;
     await _uploadLock.synchronized(() async {
       if (_currentDownloadJob != null) {
+        _log.finest('Queuing up download task.');
         _downloadQueue.add(job);
       } else {
+        _log.finest('Executing download task.');
         _currentDownloadJob = job;
-        canDownload = true;
-      }
-    });
 
-    if (canDownload) {
         unawaited(_performFileDownload(job));
       }
+    });
   }
 
-  Future<void> _copyFile(FileUploadJob job) async {
-    for (final recipient in job.recipients) {
-      final newPath = await getDownloadPath(
-        pathlib.basename(job.path),
-        recipient,
-        job.mime,
-      );
-
-      await File(job.path).copy(newPath);
+  Future<void> _copyFile(
+    FileUploadJob job,
+    String to,
+  ) async {
+    if (!File(to).existsSync()) {
+      await File(job.path).copy(to);
 
       // Let the media scanner index the file
-      MoxplatformPlugin.media.scanFile(newPath);
-
-      // Update the message
-      await GetIt.I.get<MessageService>().updateMessage(
-        job.messageMap[recipient]!.id,
-        mediaUrl: newPath,
+      MoxplatformPlugin.media.scanFile(to);
+    } else {
+      _log.finest(
+        'Skipping file copy on upload as file is already at media location',
       );
     }
   }
 
   Future<void> _fileUploadFailed(FileUploadJob job, int error) async {
     final ms = GetIt.I.get<MessageService>();
+    final cs = GetIt.I.get<ConversationService>();
 
     // Notify UI of upload failure
     for (final recipient in job.recipients) {
@@ -149,6 +149,19 @@ class HttpFileTransferService {
         isUploading: false,
       );
       sendEvent(MessageUpdatedEvent(message: msg));
+
+      // Update the conversation list
+      final conversation = await cs.getConversationByJid(recipient);
+      if (conversation?.lastMessage?.id == msg.id) {
+        final newConversation = conversation!.copyWith(
+          lastMessage: msg,
+        );
+
+        // Update the cache
+        cs.setConversation(newConversation);
+
+        sendEvent(ConversationUpdatedEvent(conversation: newConversation));
+      }
     }
 
     await _pickNextUploadTask();
@@ -183,12 +196,12 @@ class HttpFileTransferService {
     }
 
     final file = File(path);
-    final data = await file.readAsBytes();
     final stat = file.statSync();
 
     // Request the upload slot
     final conn = GetIt.I.get<XmppConnection>();
-    final httpManager = conn.getManagerById<HttpFileUploadManager>(httpFileUploadManager)!;
+    final httpManager =
+        conn.getManagerById<HttpFileUploadManager>(httpFileUploadManager)!;
     final slotResult = await httpManager.requestUploadSlot(
       pathlib.basename(path),
       stat.size,
@@ -200,20 +213,16 @@ class HttpFileTransferService {
       return;
     }
     final slot = slotResult.get<HttpFileUploadSlot>();
-    try {
-      final response = await dio.Dio().putUri<dynamic>(
+
+    final uploadStatusCode = await client.uploadFile(
       Uri.parse(slot.putUrl),
-        options: dio.Options(
-          headers: slot.headers,
-          contentType: 'application/octet-stream',
-          requestEncoder: (_, __) => data,
-        ),
-        data: data,
-        onSendProgress: (count, total) {
+      slot.headers,
+      path,
+      (total, current) {
         // TODO(PapaTutuWawa): Make this smarter by also checking if one of those chats
         //                     is open.
         if (job.recipients.length == 1) {
-            final progress = count.toDouble() / total.toDouble();
+          final progress = current.toDouble() / total.toDouble();
           sendEvent(
             ProgressEvent(
               id: job.messageMap.values.first.id,
@@ -225,40 +234,17 @@ class HttpFileTransferService {
     );
 
     final ms = GetIt.I.get<MessageService>();
-      if (response.statusCode != 201) {
-        // TODO(PapaTutuWawa): Trigger event
-        _log.severe('Upload failed');
+    if (!isRequestOkay(uploadStatusCode)) {
+      _log.severe('Upload failed due to status code $uploadStatusCode');
       await _fileUploadFailed(job, fileUploadFailedError);
       return;
     } else {
       _log.fine('Upload was successful');
 
-        const uuid = Uuid();
-        for (final recipient in job.recipients) {
-          // Notify UI of upload completion
-          var msg = await ms.updateMessage(
-            job.messageMap[recipient]!.id,
-            mediaSize: stat.size,
-            errorType: noError,
-            encryptionScheme: encryption != null ?
-              SFSEncryptionType.aes256GcmNoPadding.toNamespace() :
-              null,
-            key: encryption != null ? base64Encode(encryption.key) : null,
-            iv: encryption != null ? base64Encode(encryption.iv) : null,
-            isUploading: false,
-            srcUrl: slot.getUrl,
-          );
-          // TODO(Unknown): Maybe batch those two together?
-          final oldSid = msg.sid;
-          msg = await ms.updateMessage(
-            msg.id,
-            sid: uuid.v4(),
-            originId: uuid.v4(),
-          );
-          sendEvent(MessageUpdatedEvent(message: msg));
-
+      // Get hashes
       StatelessFileSharingSource source;
-          final plaintextHashes = <String, String>{};
+      final plaintextHashes = <HashFunction, String>{};
+      Map<HashFunction, String>? ciphertextHashes;
       if (encryption != null) {
         source = StatelessFileSharingEncryptedSource(
           SFSEncryptionType.aes256GcmNoPadding,
@@ -269,16 +255,88 @@ class HttpFileTransferService {
         );
 
         plaintextHashes.addAll(encryption.plaintextHashes);
+        ciphertextHashes = encryption.ciphertextHashes;
       } else {
         source = StatelessFileSharingUrlSource(slot.getUrl);
         try {
-              plaintextHashes[hashSha256] = await GetIt.I.get<CryptographyService>()
+          plaintextHashes[HashFunction.sha256] = await GetIt.I
+              .get<CryptographyService>()
               .hashFile(job.path, HashFunction.sha256);
         } catch (ex) {
           _log.warning('Failed to hash file ${job.path} using SHA-256: $ex');
         }
       }
 
+      // Update the metadata
+      final filename = pathlib.basename(job.path);
+      final filePath = await computeCachedPathForFile(
+        filename,
+        plaintextHashes,
+      );
+      final metadataWrapper =
+          await GetIt.I.get<FilesService>().createFileMetadataIfRequired(
+                MediaFileLocation(
+                  [slot.getUrl],
+                  filename,
+                  encryption != null
+                      ? SFSEncryptionType.aes256GcmNoPadding.toNamespace()
+                      : null,
+                  encryption?.key,
+                  encryption?.iv,
+                  plaintextHashes,
+                  ciphertextHashes,
+                  stat.size,
+                ),
+                job.mime,
+                stat.size,
+                null,
+                // TODO(Unknown): job.thumbnails.first
+                null,
+                null,
+                path: filePath,
+              );
+      var metadata = metadataWrapper.fileMetadata;
+
+      // Remove the tempoary metadata if we already know the file
+      if (metadataWrapper.retrieved) {
+        // Only skip the copy if the existing file metadata has a path associated with it
+        if (metadataWrapper.fileMetadata.path != null) {
+          _log.fine(
+            'Uploaded file $filename is already tracked. Skipping copy.',
+          );
+        } else {
+          _log.fine(
+            'Uploaded file $filename is already tracked but has no path. Copying...',
+          );
+          await _copyFile(job, filePath);
+          metadata = await GetIt.I.get<FilesService>().updateFileMetadata(
+                metadata.id,
+                path: filePath,
+              );
+        }
+      } else {
+        _log.fine('Uploaded file $filename not tracked. Copying...');
+        await _copyFile(job, metadataWrapper.fileMetadata.path!);
+      }
+
+      const uuid = Uuid();
+      for (final recipient in job.recipients) {
+        // Notify UI of upload completion
+        var msg = await ms.updateMessage(
+          job.messageMap[recipient]!.id,
+          errorType: noError,
+          isUploading: false,
+          fileMetadata: metadata,
+        );
+        // TODO(Unknown): Maybe batch those two together?
+        final oldSid = msg.sid;
+        msg = await ms.updateMessage(
+          msg.id,
+          sid: uuid.v4(),
+          originId: uuid.v4(),
+        );
+        sendEvent(MessageUpdatedEvent(message: msg));
+
         // Send the message to the recipient
         conn.getManagerById<MessageManager>(messageManager)!.sendMessage(
               MessageDetails(
@@ -291,7 +349,7 @@ class HttpFileTransferService {
                   FileMetadataData(
                     mediaType: job.mime,
                     size: stat.size,
-                  name: pathlib.basename(job.path),
+                    name: filename,
                     thumbnails: job.thumbnails,
                     hashes: plaintextHashes,
                   ),
@@ -301,27 +359,27 @@ class HttpFileTransferService {
                 funReplacement: oldSid,
               ),
             );
-          _log.finest('Sent message with file upload for ${job.path} to $recipient');
+        _log.finest(
+          'Sent message with file upload for ${job.path} to $recipient',
+        );
+      }
 
-          final isMultiMedia = job.mime?.startsWith('image/') == true || job.mime?.startsWith('video/') == true;
-          if (isMultiMedia) {
-            _log.finest('File appears to be either an image or a video. Copying it to the correct directory...');
-            unawaited(_copyFile(job));
+      // Remove the old metadata only here because we would otherwise violate a foreign key
+      // constraint.
+      if (metadataWrapper.retrieved) {
+        await GetIt.I.get<FilesService>().removeFileMetadata(
+              job.metadataId,
+            );
       }
     }
-      }
-    } on dio.DioError {
-      _log.finest('Upload failed due to connection error');
-      await _fileUploadFailed(job, fileUploadFailedError);
-      return;
-    }
 
     await _pickNextUploadTask();
   }
 
   Future<void> _pickNextUploadTask() async {
     // Free the upload resources for the next one
-    if (GetIt.I.get<ConnectivityService>().currentState == ConnectivityResult.none) return;
+    if (GetIt.I.get<ConnectivityService>().currentState ==
+        ConnectivityResult.none) return;
     await _uploadLock.synchronized(() async {
       if (_uploadQueue.isNotEmpty) {
         _currentUploadJob = _uploadQueue.removeFirst();
@@ -349,8 +407,10 @@ class HttpFileTransferService {
   /// Actually attempt to download the file described by the job [job].
   Future<void> _performFileDownload(FileDownloadJob job) async {
     final filename = job.location.filename;
-    _log.finest('Downloading ${job.location.url} as $filename');
-    final downloadedPath = await getDownloadPath(filename, job.conversationJid, job.mimeGuess);
+    final downloadedPath = await computeCachedPathForFile(
+      job.location.filename,
+      job.location.plaintextHashes,
+    );
 
     var downloadPath = downloadedPath;
     if (job.location.key != null && job.location.iv != null) {
@@ -359,13 +419,21 @@ class HttpFileTransferService {
       downloadPath = pathlib.join(tempDir.path, filename);
     }
 
-    dio.Response<dynamic>? response;
+    // TODO(Unknown): Maybe try other URLs?
+    final downloadUrl = job.location.urls.first;
+    _log.finest(
+      'Downloading $downloadUrl as $filename (MIME guess ${job.mimeGuess}) to $downloadPath (-> $downloadedPath)',
+    );
+
+    int? downloadStatusCode;
+    var integrityCheckPassed = true;
     try {
-      response = await dio.Dio().downloadUri(
-        Uri.parse(job.location.url),
+      _log.finest('Beginning download...');
+      downloadStatusCode = await client.downloadFile(
+        Uri.parse(downloadUrl),
         downloadPath,
-        onReceiveProgress: (count, total) {
-          final progress = count.toDouble() / total.toDouble();
+        (total, current) {
+          final progress = current.toDouble() / total.toDouble();
           sendEvent(
             ProgressEvent(
               id: job.mId,
@@ -374,23 +442,22 @@ class HttpFileTransferService {
           );
         },
       );
-    } on dio.DioError catch(err) {
-      // TODO(PapaTutuWawa): React if we received an error that is not related to the
-      //                     connection.
+      _log.finest('Download done...');
+    } catch (err) {
       _log.finest('Failed to download: $err');
+    }
+
+    if (!isRequestOkay(downloadStatusCode)) {
+      _log.warning(
+        'HTTP GET of $downloadUrl returned $downloadStatusCode',
+      );
       await _fileDownloadFailed(job, fileDownloadFailedError);
       return;
     }
 
-    if (!isRequestOkay(response.statusCode)) {
-      _log.warning('HTTP GET of ${job.location.url} returned ${response.statusCode}');
-      await _fileDownloadFailed(job, fileDownloadFailedError);
-      return;
-    } else {
-      var integrityCheckPassed = true;
-      final conv = (await GetIt.I.get<ConversationService>()
-          .getConversationByJid(job.conversationJid))!;
-      final decryptionKeysAvailable = job.location.key != null && job.location.iv != null;
+    final decryptionKeysAvailable =
+        job.location.key != null && job.location.iv != null;
+    final crypto = GetIt.I.get<CryptographyService>();
     if (decryptionKeysAvailable) {
       // The file was downloaded and is now being decrypted
       sendEvent(
@@ -400,10 +467,10 @@ class HttpFileTransferService {
       );
 
       try {
-          final result = await GetIt.I.get<CryptographyService>().decryptFile(
+        final result = await crypto.decryptFile(
           downloadPath,
           downloadedPath,
-            encryptionTypeFromNamespace(job.location.encryptionScheme!),
+          SFSEncryptionType.fromNamespace(job.location.encryptionScheme!),
           job.location.key!,
           job.location.iv!,
           job.location.plaintextHashes ?? {},
@@ -418,12 +485,38 @@ class HttpFileTransferService {
 
         integrityCheckPassed = result.plaintextOkay && result.ciphertextOkay;
       } catch (ex) {
-          _log.warning('Decryption of $downloadPath ($downloadedPath) failed: $ex');
+        _log.warning(
+          'Decryption of $downloadPath ($downloadedPath) failed: $ex',
+        );
         await _fileDownloadFailed(job, messageFailedToDecryptFile);
         return;
       }
 
-        unawaited(Directory(pathlib.dirname(downloadPath)).delete(recursive: true));
+      unawaited(
+        Directory(pathlib.dirname(downloadPath)).delete(recursive: true),
+      );
+    } else if (job.location.plaintextHashes?.isNotEmpty ?? false) {
+      // Verify only the plaintext hash
+      // TODO(Unknown): Allow verification of other hash functions
+      if (job.location.plaintextHashes![HashFunction.sha256] != null) {
+        final hash = await crypto.hashFile(
+          downloadPath,
+          HashFunction.sha256,
+        );
+        integrityCheckPassed =
+            hash == job.location.plaintextHashes![HashFunction.sha256];
+      } else if (job.location.plaintextHashes![HashFunction.sha512] != null) {
+        final hash = await crypto.hashFile(
+          downloadPath,
+          HashFunction.sha512,
+        );
+        integrityCheckPassed =
+            hash == job.location.plaintextHashes![HashFunction.sha512];
+      } else {
+        _log.warning(
+          'Could not verify file integrity as no accelerated hash function is available (${job.location.plaintextHashes!.keys})',
+        );
+      }
     }
 
     // Check the MIME type
@@ -467,63 +560,74 @@ class HttpFileTransferService {
       }
     }
 
+    final fs = GetIt.I.get<FilesService>();
+    final metadata = await fs.updateFileMetadata(
+      job.metadataId,
+      path: downloadedPath,
+      size: File(downloadedPath).lengthSync(),
+      width: mediaWidth,
+      height: mediaHeight,
+      mimeType: mime,
+    );
+
+    // Only add the hash pointers if the file hashes match what was sent
+    if (job.location.plaintextHashes?.isNotEmpty ?? false) {
+      if (integrityCheckPassed) {
+        await fs.createMetadataHashEntries(
+          job.location.plaintextHashes!,
+          job.metadataId,
+        );
+      } else {
+        _log.warning('Integrity check failed for file');
+      }
+    }
+
+    final cs = GetIt.I.get<ConversationService>();
+    final conversation = (await cs.getConversationByJid(job.conversationJid))!;
     final msg = await GetIt.I.get<MessageService>().updateMessage(
           job.mId,
-        mediaUrl: downloadedPath,
-        mediaType: mime,
-        mediaWidth: mediaWidth,
-        mediaHeight: mediaHeight,
-        mediaSize: File(downloadedPath).lengthSync(),
+          fileMetadata: metadata,
           isFileUploadNotification: false,
-        warningType: integrityCheckPassed ?
-          null :
-          warningFileIntegrityCheckFailed,
-        errorType: conv.encrypted && !decryptionKeysAvailable ?
-          messageChatEncryptedButFileNot :
-          null,
+          warningType:
+              integrityCheckPassed ? null : warningFileIntegrityCheckFailed,
+          errorType: conversation.encrypted && !decryptionKeysAvailable
+              ? messageChatEncryptedButFileNot
+              : null,
           isDownloading: false,
         );
 
     sendEvent(MessageUpdatedEvent(message: msg));
 
-      final sharedMedium = await GetIt.I.get<DatabaseService>().addSharedMediumFromData(
-        downloadedPath,
-        msg.timestamp,
-        conv.id,
-        job.mId,
-        mime: mime,
+    final updatedConversation = conversation.copyWith(
+      lastMessage: conversation.lastMessage?.id == job.mId
+          ? msg
+          : conversation.lastMessage,
     );
-      final newConv = conv.copyWith(
-        lastMessage: conv.lastMessage?.id == job.mId ?
-          msg :
-          conv.lastMessage,
-        sharedMedia: [
-          sharedMedium,
-          ...conv.sharedMedia,
-        ],
-      );
-      GetIt.I.get<ConversationService>().setConversation(newConv);
+    cs.setConversation(updatedConversation);
 
     // Show a notification
-      if (notification.shouldShowNotification(msg.conversationJid) && job.shouldShowNotification) {
+    if (notification.shouldShowNotification(msg.conversationJid) &&
+        job.shouldShowNotification) {
       _log.finest('Creating notification with bigPicture $downloadedPath');
-        await notification.showNotification(newConv, msg, '');
+      await notification.showNotification(updatedConversation, msg, '');
     }
 
-      sendEvent(ConversationUpdatedEvent(conversation: newConv));
-    }
+    sendEvent(ConversationUpdatedEvent(conversation: updatedConversation));
 
     // Free the download resources for the next one
     await _pickNextDownloadTask();
   }
 
   Future<void> _pickNextDownloadTask() async {
-    if (GetIt.I.get<ConnectivityService>().currentState == ConnectivityResult.none) return;
-
     await _downloadLock.synchronized(() async {
       if (_downloadQueue.isNotEmpty) {
         _currentDownloadJob = _downloadQueue.removeFirst();
+
+        // Only download if we have a connection
+        if (GetIt.I.get<ConnectivityService>().currentState !=
+            ConnectivityResult.none) {
           unawaited(_performFileDownload(_currentDownloadJob!));
+        }
       } else {
         _currentDownloadJob = null;
       }

lib/service/httpfiletransfer/jobs.dart

@@ -6,7 +6,15 @@ import 'package:moxxyv2/shared/models/message.dart';
 /// A job describing the download of a file.
 @immutable
 class FileUploadJob {
-  const FileUploadJob(this.recipients, this.path, this.mime, this.encryptMap, this.messageMap, this.thumbnails);
+  const FileUploadJob(
+    this.recipients,
+    this.path,
+    this.mime,
+    this.encryptMap,
+    this.messageMap,
+    this.metadataId,
+    this.thumbnails,
+  );
   final List<String> recipients;
   final String path;
   final String? mime;
@@ -14,6 +22,7 @@ class FileUploadJob {
   final Map<String, bool> encryptMap;
   // Recipient -> Message
   final Map<String, Message> messageMap;
+  final String metadataId;
   final List<Thumbnail> thumbnails;
 
   @override
@@ -24,11 +33,19 @@ class FileUploadJob {
         messageMap == other.messageMap &&
         mime == other.mime &&
         thumbnails == other.thumbnails &&
-      encryptMap == other.encryptMap;
+        encryptMap == other.encryptMap &&
+        metadataId == other.metadataId;
   }
 
   @override
-  int get hashCode => path.hashCode ^ recipients.hashCode ^ messageMap.hashCode ^ mime.hashCode ^ thumbnails.hashCode ^ encryptMap.hashCode;
+  int get hashCode =>
+      path.hashCode ^
+      recipients.hashCode ^
+      messageMap.hashCode ^
+      mime.hashCode ^
+      thumbnails.hashCode ^
+      encryptMap.hashCode ^
+      metadataId.hashCode;
 }
 
 /// A job describing the upload of a file.
@@ -37,12 +54,14 @@ class FileDownloadJob {
   const FileDownloadJob(
     this.location,
     this.mId,
+    this.metadataId,
     this.conversationJid,
     this.mimeGuess, {
     this.shouldShowNotification = true,
   });
   final MediaFileLocation location;
   final int mId;
+  final String metadataId;
   final String conversationJid;
   final String? mimeGuess;
   final bool shouldShowNotification;
@@ -52,11 +71,18 @@ class FileDownloadJob {
     return other is FileDownloadJob &&
         location == other.location &&
         mId == other.mId &&
+        metadataId == other.metadataId &&
         conversationJid == other.conversationJid &&
         mimeGuess == other.mimeGuess &&
         shouldShowNotification == other.shouldShowNotification;
   }
 
   @override
-  int get hashCode => location.hashCode ^ mId.hashCode ^ conversationJid.hashCode ^ mimeGuess.hashCode ^ shouldShowNotification.hashCode;
+  int get hashCode =>
+      location.hashCode ^
+      mId.hashCode ^
+      metadataId.hashCode ^
+      conversationJid.hashCode ^
+      mimeGuess.hashCode ^
+      shouldShowNotification.hashCode;
 }

lib/service/httpfiletransfer/location.dart

@@ -1,25 +1,27 @@
 import 'dart:convert';
 import 'package:meta/meta.dart';
+import 'package:moxxmpp/moxxmpp.dart';
 
 @immutable
 class MediaFileLocation {
-
   const MediaFileLocation(
-    this.url,
+    this.urls,
     this.filename,
     this.encryptionScheme,
     this.key,
     this.iv,
     this.plaintextHashes,
     this.ciphertextHashes,
+    this.size,
   );
-  final String url;
+  final List<String> urls;
   final String filename;
   final String? encryptionScheme;
   final List<int>? key;
   final List<int>? iv;
-  final Map<String, String>? plaintextHashes;
-  final Map<String, String>? ciphertextHashes;
+  final Map<HashFunction, String>? plaintextHashes;
+  final Map<HashFunction, String>? ciphertextHashes;
+  final int? size;
 
   String? get keyBase64 {
     if (key != null) return base64Encode(key!);
@@ -34,16 +36,24 @@ class MediaFileLocation {
   }
 
   @override
-  int get hashCode => url.hashCode ^ filename.hashCode ^ encryptionScheme.hashCode ^ key.hashCode ^ iv.hashCode ^ plaintextHashes.hashCode ^ ciphertextHashes.hashCode;
+  int get hashCode =>
+      urls.hashCode ^
+      filename.hashCode ^
+      encryptionScheme.hashCode ^
+      key.hashCode ^
+      iv.hashCode ^
+      plaintextHashes.hashCode ^
+      ciphertextHashes.hashCode ^
+      size.hashCode;
 
   @override
   bool operator ==(Object other) {
     // TODO(PapaTutuWawa): Compare the Maps
     return other is MediaFileLocation &&
-           url == other.url &&
         filename == other.filename &&
         encryptionScheme == other.encryptionScheme &&
         key == other.key &&
-           iv == other.iv;
+        iv == other.iv &&
+        size == other.size;
   }
 }

lib/service/message.dart

@@ -1,36 +1,314 @@
 import 'dart:async';
-import 'dart:collection';
-import 'dart:io';
 import 'package:get_it/get_it.dart';
 import 'package:logging/logging.dart';
-import 'package:moxlib/moxlib.dart';
 import 'package:moxxmpp/moxxmpp.dart';
 import 'package:moxxyv2/service/conversation.dart';
+import 'package:moxxyv2/service/database/constants.dart';
 import 'package:moxxyv2/service/database/database.dart';
+import 'package:moxxyv2/service/database/helpers.dart';
+import 'package:moxxyv2/service/files.dart';
 import 'package:moxxyv2/service/not_specified.dart';
+import 'package:moxxyv2/service/reactions.dart';
 import 'package:moxxyv2/service/service.dart';
+import 'package:moxxyv2/shared/cache.dart';
+import 'package:moxxyv2/shared/constants.dart';
 import 'package:moxxyv2/shared/events.dart';
-import 'package:moxxyv2/shared/models/media.dart';
+import 'package:moxxyv2/shared/helpers.dart';
+import 'package:moxxyv2/shared/models/file_metadata.dart';
 import 'package:moxxyv2/shared/models/message.dart';
+import 'package:synchronized/synchronized.dart';
 
 class MessageService {
-  MessageService() : _messageCache = HashMap(), _log = Logger('MessageService');
-  final HashMap<String, List<Message>> _messageCache;
-  final Logger _log;
+  /// Logger
+  final Logger _log = Logger('MessageService');
 
-  /// Returns the messages for [jid], either from cache or from the database.
-  Future<List<Message>> getMessagesForJid(String jid) async {
-    if (!_messageCache.containsKey(jid)) {
-      _messageCache[jid] = await GetIt.I.get<DatabaseService>().loadMessagesForJid(jid);
+  final LRUCache<String, List<Message>> _messageCache =
+      LRUCache(conversationMessagePageCacheSize);
+  final Lock _cacheLock = Lock();
+
+  Future<Message?> getMessageById(
+    int id,
+    String conversationJid, {
+    bool queryReactionPreview = true,
+  }) async {
+    final db = GetIt.I.get<DatabaseService>().database;
+    final messagesRaw = await db.query(
+      messagesTable,
+      where: 'id = ? AND conversationJid = ?',
+      whereArgs: [id, conversationJid],
+      limit: 1,
+    );
+
+    if (messagesRaw.isEmpty) return null;
+
+    // TODO(PapaTutuWawa): Load the quoted message
+    final msg = messagesRaw.first;
+
+    // Load the file metadata, if available
+    FileMetadata? fm;
+    if (msg['file_metadata_id'] != null) {
+      final rawFm = (await db.query(
+        fileMetadataTable,
+        where: 'id = ?',
+        whereArgs: [msg['file_metadata_id']],
+        limit: 1,
+      ))
+          .first;
+      fm = FileMetadata.fromDatabaseJson(rawFm);
     }
 
-    final messages = _messageCache[jid];
-    if (messages == null) {
-      _log.warning('No messages found for $jid. Returning [].');
-      return [];
+    return Message.fromDatabaseJson(
+      msg,
+      null,
+      fm,
+      queryReactionPreview
+          ? await GetIt.I
+              .get<ReactionsService>()
+              .getPreviewReactionsForMessage(msg['id']! as int)
+          : [],
+    );
   }
 
-    return messages;
+  Future<Message?> getMessageByXmppId(
+    String id,
+    String conversationJid, {
+    bool includeOriginId = true,
+    bool queryReactionPreview = true,
+  }) async {
+    final db = GetIt.I.get<DatabaseService>().database;
+    final idQuery = includeOriginId ? '(sid = ? OR originId = ?)' : 'sid = ?';
+    final messagesRaw = await db.query(
+      messagesTable,
+      where: 'conversationJid = ? AND $idQuery',
+      whereArgs: [
+        conversationJid,
+        if (includeOriginId) id,
+        id,
+      ],
+      limit: 1,
+    );
+
+    if (messagesRaw.isEmpty) return null;
+
+    // TODO(PapaTutuWawa): Load the quoted message
+    final msg = messagesRaw.first;
+
+    FileMetadata? fm;
+    if (msg['file_metadata_id'] != null) {
+      final rawFm = (await db.query(
+        fileMetadataTable,
+        where: 'id = ?',
+        whereArgs: [msg['file_metadata_id']],
+        limit: 1,
+      ))
+          .first;
+      fm = FileMetadata.fromDatabaseJson(rawFm);
+    }
+
+    return Message.fromDatabaseJson(
+      msg,
+      null,
+      fm,
+      queryReactionPreview
+          ? await GetIt.I
+              .get<ReactionsService>()
+              .getPreviewReactionsForMessage(msg['id']! as int)
+          : [],
+    );
+  }
+
+  /// Return a list of messages for [jid]. If [olderThan] is true, then all messages are older than [oldestTimestamp], if
+  /// specified, or the oldest messages are returned if null. If [olderThan] is false, then message must be newer
+  /// than [oldestTimestamp], or the newest messages are returned if null.
+  Future<List<Message>> getPaginatedMessagesForJid(
+    String jid,
+    bool olderThan,
+    int? oldestTimestamp,
+  ) async {
+    if (olderThan && oldestTimestamp == null) {
+      final result = await _cacheLock.synchronized<List<Message>?>(() {
+        return _messageCache.getValue(jid);
+      });
+      if (result != null) return result;
+    }
+
+    final db = GetIt.I.get<DatabaseService>().database;
+    final comparator = olderThan ? '<' : '>';
+    final query = oldestTimestamp != null
+        ? 'conversationJid = ? AND timestamp $comparator ?'
+        : 'conversationJid = ?';
+    final rawMessages = await db.rawQuery(
+      // LEFT JOIN $messagesTable quote ON msg.quote_id = quote.id
+      '''
+SELECT
+  msg.*,
+  quote.id AS quote_id,
+  quote.sender AS quote_sender,
+  quote.body AS quote_body,
+  quote.timestamp AS quote_timestamp,
+  quote.sid AS quote_sid,
+  quote.conversationJid AS quote_conversationJid,
+  quote.isFileUploadNotification AS quote_isFileUploadNotification,
+  quote.encrypted AS quote_encrypted,
+  quote.errorType AS quote_errorType,
+  quote.warningType AS quote_warningType,
+  quote.received AS quote_received,
+  quote.displayed AS quote_displayed,
+  quote.acked AS quote_acked,
+  quote.originId AS quote_originId,
+  quote.quote_id AS quote_quote_id,
+  quote.file_metadata_id AS quote_file_metadata_id,
+  quote.isDownloading AS quote_isDownloading,
+  quote.isUploading AS quote_isUploading,
+  quote.isRetracted AS quote_isRetracted,
+  quote.isEdited AS quote_isEdited,
+  quote.containsNoStore AS quote_containsNoStore,
+  quote.stickerPackId AS quote_stickerPackId,
+  quote.pseudoMessageType AS quote_pseudoMessageType,
+  quote.pseudoMessageData AS quote_pseudoMessageData,
+  fm.id as fm_id,
+  fm.path as fm_path,
+  fm.sourceUrls as fm_sourceUrls,
+  fm.mimeType as fm_mimeType,
+  fm.thumbnailType as fm_thumbnailType,
+  fm.thumbnailData as fm_thumbnailData,
+  fm.width as fm_width,
+  fm.height as fm_height,
+  fm.plaintextHashes as fm_plaintextHashes,
+  fm.encryptionKey as fm_encryptionKey,
+  fm.encryptionIv as fm_encryptionIv,
+  fm.encryptionScheme as fm_encryptionScheme,
+  fm.cipherTextHashes as fm_cipherTextHashes,
+  fm.filename as fm_filename,
+  fm.size as fm_size
+FROM (SELECT * FROM $messagesTable WHERE $query ORDER BY timestamp DESC LIMIT $messagePaginationSize) AS msg
+  LEFT JOIN $fileMetadataTable fm ON msg.file_metadata_id = fm.id
+  LEFT JOIN $messagesTable quote ON msg.quote_id = quote.id;
+      ''',
+      [
+        jid,
+        if (oldestTimestamp != null) oldestTimestamp,
+      ],
+    );
+
+    final page = List<Message>.empty(growable: true);
+    for (final m in rawMessages) {
+      if (m.isEmpty) {
+        continue;
+      }
+
+      Message? quotes;
+      if (m['quote_id'] != null) {
+        final rawQuote = getPrefixedSubMap(m, 'quote_');
+
+        FileMetadata? quoteFm;
+        if (rawQuote['file_metadata_id'] != null) {
+          final rawQuoteFm = (await db.query(
+            fileMetadataTable,
+            where: 'id = ?',
+            whereArgs: [rawQuote['file_metadata_id']],
+            limit: 1,
+          ))
+              .first;
+          quoteFm = FileMetadata.fromDatabaseJson(rawQuoteFm);
+        }
+
+        quotes = Message.fromDatabaseJson(rawQuote, null, quoteFm, []);
+      }
+
+      FileMetadata? fm;
+      if (m['file_metadata_id'] != null) {
+        fm = FileMetadata.fromDatabaseJson(
+          getPrefixedSubMap(m, 'fm_'),
+        );
+      }
+
+      page.add(
+        Message.fromDatabaseJson(
+          m,
+          quotes,
+          fm,
+          await GetIt.I
+              .get<ReactionsService>()
+              .getPreviewReactionsForMessage(m['id']! as int),
+        ),
+      );
+    }
+
+    if (olderThan && oldestTimestamp == null) {
+      await _cacheLock.synchronized(() {
+        _messageCache.cache(
+          jid,
+          page,
+        );
+      });
+    }
+
+    return page;
+  }
+
+  /// Like getPaginatedMessagesForJid, but instead only returns messages that have file
+  /// metadata attached. This method bypasses the cache and does not load the message's
+  /// quoted message, if it exists.
+  Future<List<Message>> getPaginatedSharedMediaMessagesForJid(
+    String jid,
+    bool olderThan,
+    int? oldestTimestamp,
+  ) async {
+    final db = GetIt.I.get<DatabaseService>().database;
+    final comparator = olderThan ? '<' : '>';
+    final query = oldestTimestamp != null
+        ? 'conversationJid = ? AND file_metadata_id IS NOT NULL AND timestamp $comparator ?'
+        : 'conversationJid = ? AND file_metadata_id IS NOT NULL';
+    final rawMessages = await db.rawQuery(
+      '''
+SELECT
+  msg.*,
+  fm.id as fm_id,
+  fm.path as fm_path,
+  fm.sourceUrls as fm_sourceUrls,
+  fm.mimeType as fm_mimeType,
+  fm.thumbnailType as fm_thumbnailType,
+  fm.thumbnailData as fm_thumbnailData,
+  fm.width as fm_width,
+  fm.height as fm_height,
+  fm.plaintextHashes as fm_plaintextHashes,
+  fm.encryptionKey as fm_encryptionKey,
+  fm.encryptionIv as fm_encryptionIv,
+  fm.encryptionScheme as fm_encryptionScheme,
+  fm.cipherTextHashes as fm_cipherTextHashes,
+  fm.filename as fm_filename,
+  fm.size as fm_size
+FROM (SELECT * FROM $messagesTable WHERE $query ORDER BY timestamp DESC LIMIT $sharedMediaPaginationSize) AS msg
+  LEFT JOIN $fileMetadataTable fm ON msg.file_metadata_id = fm.id;
+      ''',
+      [
+        jid,
+        if (oldestTimestamp != null) oldestTimestamp,
+      ],
+    );
+
+    final page = List<Message>.empty(growable: true);
+    for (final m in rawMessages) {
+      if (m.isEmpty) {
+        continue;
+      }
+
+      page.add(
+        Message.fromDatabaseJson(
+          m,
+          null,
+          FileMetadata.fromDatabaseJson(
+            getPrefixedSubMap(m, 'fm_'),
+          ),
+          await GetIt.I
+              .get<ReactionsService>()
+              .getPreviewReactionsForMessage(m['id']! as int),
+        ),
+      );
+    }
+
+    return page;
   }
 
   /// Wrapper around [DatabaseService]'s addMessageFromData that updates the cache.
@@ -39,163 +317,222 @@ class MessageService {
     int timestamp,
     String sender,
     String conversationJid,
-    bool isMedia,
     String sid,
     bool isFileUploadNotification,
     bool encrypted,
-    {
-      String? srcUrl,
-      String? key,
-      String? iv,
-      String? encryptionScheme,
-      String? mediaUrl,
-      String? mediaType,
-      String? thumbnailData,
-      int? mediaWidth,
-      int? mediaHeight,
+    bool containsNoStore, {
     String? originId,
     String? quoteId,
-      String? filename,
+    FileMetadata? fileMetadata,
     int? errorType,
     int? warningType,
-      Map<String, String>? plaintextHashes,
-      Map<String, String>? ciphertextHashes,
     bool isDownloading = false,
     bool isUploading = false,
-      int? mediaSize,
-    }
-  ) async {
-    final msg = await GetIt.I.get<DatabaseService>().addMessageFromData(
+    String? stickerPackId,
+    int? pseudoMessageType,
+    Map<String, dynamic>? pseudoMessageData,
+    bool received = false,
+    bool displayed = false,
+  }) async {
+    final db = GetIt.I.get<DatabaseService>().database;
+    var m = Message(
+      sender,
       body,
       timestamp,
-      sender,
-      conversationJid,
-      isMedia,
       sid,
+      -1,
+      conversationJid,
       isFileUploadNotification,
       encrypted,
-      srcUrl: srcUrl,
-      key: key,
-      iv: iv,
-      encryptionScheme: encryptionScheme,
-      mediaUrl: mediaUrl,
-      mediaType: mediaType,
-      thumbnailData: thumbnailData,
-      mediaWidth: mediaWidth,
-      mediaHeight: mediaHeight,
-      originId: originId,
-      quoteId: quoteId,
-      filename: filename,
+      containsNoStore,
       errorType: errorType,
       warningType: warningType,
-      plaintextHashes: plaintextHashes,
-      ciphertextHashes: ciphertextHashes,
+      fileMetadata: fileMetadata,
+      received: received,
+      displayed: displayed,
+      acked: false,
+      originId: originId,
       isUploading: isUploading,
       isDownloading: isDownloading,
-      mediaSize: mediaSize,
+      stickerPackId: stickerPackId,
+      pseudoMessageType: pseudoMessageType,
+      pseudoMessageData: pseudoMessageData,
     );
 
-    // Only update the cache if the conversation already has been loaded. This prevents
-    // us from accidentally not loading the conversation afterwards.
-    if (_messageCache.containsKey(conversationJid)) {
-      _messageCache[conversationJid] = _messageCache[conversationJid]!..add(msg);
+    if (quoteId != null) {
+      final quotes = await getMessageByXmppId(quoteId, conversationJid);
+      if (quotes == null) {
+        _log.warning('Failed to add quote for message with id $quoteId');
+      } else {
+        m = m.copyWith(quotes: quotes);
+      }
     }
 
-    return msg;
+    m = m.copyWith(
+      id: await db.insert(messagesTable, m.toDatabaseJson()),
+    );
+
+    await _cacheLock.synchronized(() {
+      final cachedList = _messageCache.getValue(conversationJid);
+      if (cachedList != null) {
+        _messageCache.replaceValue(
+          conversationJid,
+          clampedListPrepend(
+            cachedList,
+            m,
+            messagePaginationSize,
+          ),
+        );
+      }
+    });
+
+    return m;
   }
 
-  Future<Message?> getMessageByStanzaId(String conversationJid, String stanzaId) async {
-    if (!_messageCache.containsKey(conversationJid)) {
-      await getMessagesForJid(conversationJid);
-    }
-    
-    return firstWhereOrNull(
-      _messageCache[conversationJid]!,
-      (message) => message.sid == stanzaId,
+  Future<Message?> getMessageByStanzaId(
+    String conversationJid,
+    String stanzaId,
+  ) async {
+    return getMessageByXmppId(
+      stanzaId,
+      conversationJid,
+      includeOriginId: false,
     );
   }
 
-  Future<Message?> getMessageById(String conversationJid, int id) async {
-    if (!_messageCache.containsKey(conversationJid)) {
-      await getMessagesForJid(conversationJid);
-    }
-
-    return firstWhereOrNull(
-      _messageCache[conversationJid]!,
-      (message) => message.id == id,
+  Future<Message?> getMessageByStanzaOrOriginId(
+    String conversationJid,
+    String id,
+  ) async {
+    return getMessageByXmppId(
+      id,
+      conversationJid,
     );
   }
 
   /// Wrapper around [DatabaseService]'s updateMessage that updates the cache
-  Future<Message> updateMessage(int id, {
+  Future<Message> updateMessage(
+    int id, {
     Object? body = notSpecified,
-    Object? mediaUrl = notSpecified,
-    Object? mediaType = notSpecified,
-    bool? isMedia,
     bool? received,
     bool? displayed,
     bool? acked,
+    Object? fileMetadata = notSpecified,
     Object? errorType = notSpecified,
     Object? warningType = notSpecified,
     bool? isFileUploadNotification,
-    Object? srcUrl = notSpecified,
-    Object? key = notSpecified,
-    Object? iv = notSpecified,
-    Object? encryptionScheme = notSpecified,
-    Object? mediaWidth = notSpecified,
-    Object? mediaHeight = notSpecified,
-    Object? mediaSize = notSpecified,
     bool? isUploading,
     bool? isDownloading,
     Object? originId = notSpecified,
     Object? sid = notSpecified,
-    Object? thumbnailData = notSpecified,
     bool? isRetracted,
     bool? isEdited,
   }) async {
-    final newMessage = await GetIt.I.get<DatabaseService>().updateMessage(
-      id,
-      body: body,
-      mediaUrl: mediaUrl,
-      mediaType: mediaType,
-      received: received,
-      displayed: displayed,
-      acked: acked,
-      errorType: errorType,
-      warningType: warningType,
-      isFileUploadNotification: isFileUploadNotification,
-      srcUrl: srcUrl,
-      key: key,
-      iv: iv,
-      encryptionScheme: encryptionScheme,
-      mediaWidth: mediaWidth,
-      mediaHeight: mediaHeight,
-      mediaSize: mediaSize,
-      isUploading: isUploading,
-      isDownloading: isDownloading,
-      originId: originId,
-      sid: sid,
-      isRetracted: isRetracted,
-      isMedia: isMedia,
-      thumbnailData: thumbnailData,
-      isEdited: isEdited,
-    );
+    final db = GetIt.I.get<DatabaseService>().database;
+    final m = <String, dynamic>{};
 
-    if (_messageCache.containsKey(newMessage.conversationJid)) {
-      _messageCache[newMessage.conversationJid] = _messageCache[newMessage.conversationJid]!.map((m) {
-        if (m.id == newMessage.id) return newMessage;
-
-        return m;
-      }).toList();
+    if (body != notSpecified) {
+      m['body'] = body as String?;
+    }
+    if (received != null) {
+      m['received'] = boolToInt(received);
+    }
+    if (displayed != null) {
+      m['displayed'] = boolToInt(displayed);
+    }
+    if (acked != null) {
+      m['acked'] = boolToInt(acked);
+    }
+    if (errorType != notSpecified) {
+      m['errorType'] = errorType as int?;
+    }
+    if (warningType != notSpecified) {
+      m['warningType'] = warningType as int?;
+    }
+    if (isFileUploadNotification != null) {
+      m['isFileUploadNotification'] = boolToInt(isFileUploadNotification);
+    }
+    if (isDownloading != null) {
+      m['isDownloading'] = boolToInt(isDownloading);
+    }
+    if (isUploading != null) {
+      m['isUploading'] = boolToInt(isUploading);
+    }
+    if (sid != notSpecified) {
+      m['sid'] = sid as String?;
+    }
+    if (originId != notSpecified) {
+      m['originId'] = originId as String?;
+    }
+    if (isRetracted != null) {
+      m['isRetracted'] = boolToInt(isRetracted);
+    }
+    if (fileMetadata != notSpecified) {
+      m['file_metadata_id'] = (fileMetadata as FileMetadata?)?.id;
+    }
+    if (isEdited != null) {
+      m['isEdited'] = boolToInt(isEdited);
     }
 
-    return newMessage;
+    final updatedMessage = await db.updateAndReturn(
+      messagesTable,
+      m,
+      where: 'id = ?',
+      whereArgs: [id],
+    );
+
+    Message? quotes;
+    if (updatedMessage['quote_id'] != null) {
+      quotes = await getMessageById(
+        updatedMessage['quote_id']! as int,
+        updatedMessage['conversationJid']! as String,
+        queryReactionPreview: false,
+      );
+    }
+
+    FileMetadata? metadata;
+    if (fileMetadata != notSpecified) {
+      metadata = fileMetadata as FileMetadata?;
+    } else if (updatedMessage['file_metadata_id'] != null) {
+      final metadataRaw = (await db.query(
+        fileMetadataTable,
+        where: 'id = ?',
+        whereArgs: [updatedMessage['file_metadata_id']],
+        limit: 1,
+      ))
+          .first;
+      metadata = FileMetadata.fromDatabaseJson(metadataRaw);
+    }
+
+    final msg = Message.fromDatabaseJson(
+      updatedMessage,
+      quotes,
+      metadata,
+      await GetIt.I.get<ReactionsService>().getPreviewReactionsForMessage(id),
+    );
+
+    await _cacheLock.synchronized(() {
+      final page = _messageCache.getValue(msg.conversationJid);
+      if (page != null) {
+        _messageCache.replaceValue(
+          msg.conversationJid,
+          page.map((m) {
+            if (m.id == msg.id) {
+              return msg;
+            }
+
+            return m;
+          }).toList(),
+        );
+      }
+    });
+
+    return msg;
   }
 
   /// Helper function that manages everything related to retracting a message. It
   /// - Replaces all metadata of the message with null values and marks it as retracted
   /// - Modified the conversation, if the retracted message was the newest message
-  /// - Remove the SharedMedium from the database, if one referenced the retracted message
   /// - Update the UI
   ///
   /// [conversationJid] is the bare JID of the conversation this message belongs to.
@@ -204,44 +541,42 @@ class MessageService {
   /// [selfRetract] indicates whether the message retraction came from the UI. If true,
   /// then the sender check (see security considerations of XEP-0424) is skipped as
   /// the UI already verifies it.
-  Future<void> retractMessage(String conversationJid, String originId, String bareSender, bool selfRetract) async {
-    final msg = await GetIt.I.get<DatabaseService>().getMessageByOriginId(
+  Future<void> retractMessage(
+    String conversationJid,
+    String originId,
+    String bareSender,
+    bool selfRetract,
+  ) async {
+    final msg = await getMessageByXmppId(
       originId,
       conversationJid,
     );
 
     if (msg == null) {
-      _log.finest('Got message retraction for origin Id $originId, but did not find the message');
+      _log.finest(
+        'Got message retraction for origin Id $originId, but did not find the message',
+      );
       return;
     }
 
     // Check if the retraction was sent by the original sender
     if (!selfRetract) {
       if (JID.fromString(msg.sender).toBare().toString() != bareSender) {
-        _log.warning('Received invalid message retraction from $bareSender but its original sender is ${msg.sender}');
+        _log.warning(
+          'Received invalid message retraction from $bareSender but its original sender is ${msg.sender}',
+        );
         return;
       }
     }
 
     final isMedia = msg.isMedia;
-    final mediaUrl = msg.mediaUrl;
     final retractedMessage = await updateMessage(
       msg.id,
-      isMedia: false,
-      mediaUrl: null,
-      mediaType: null,
       warningType: null,
       errorType: null,
-      srcUrl: null,
-      key: null,
-      iv: null,
-      encryptionScheme: null,
-      mediaWidth: null,
-      mediaHeight: null,
-      mediaSize: null,
       isRetracted: true,
-      thumbnailData: null,
       body: '',
+      fileMetadata: null,
     );
     sendEvent(MessageUpdatedEvent(message: retractedMessage));
 
@@ -249,37 +584,46 @@ class MessageService {
     final conversation = await cs.getConversationByJid(conversationJid);
     if (conversation != null) {
       if (conversation.lastMessage?.id == msg.id) {
-        var newConversation = conversation.copyWith(
+        final newConversation = conversation.copyWith(
           lastMessage: retractedMessage,
         );
 
-        if (isMedia) {
-          await GetIt.I.get<DatabaseService>().removeSharedMediumByMessageId(msg.id);
-
-          newConversation = newConversation.copyWith(
-            sharedMedia: newConversation.sharedMedia.where((SharedMedium medium) {
-              return medium.messageId != msg.id;
-            }).toList(),
-          );
-
-          // Delete the file if we downloaded it
-          if (mediaUrl != null) {
-            final file = File(mediaUrl);
-            if (file.existsSync()) {
-              unawaited(file.delete());
-            }
-          }
-        }
-
         cs.setConversation(newConversation);
         sendEvent(
           ConversationUpdatedEvent(
             conversation: newConversation,
           ),
         );
+
+        if (isMedia) {
+          // Remove the file
+          await GetIt.I.get<FilesService>().removeFileIfNotReferenced(
+                msg.fileMetadata!,
+              );
+        }
       }
     } else {
-      _log.warning('Failed to find conversation with conversationJid $conversationJid');
+      _log.warning(
+        'Failed to find conversation with conversationJid $conversationJid',
+      );
     }
   }
+
+  Future<void> replaceMessageInCache(Message message) async {
+    await _cacheLock.synchronized(() {
+      final cachedList = _messageCache.getValue(message.conversationJid);
+      if (cachedList != null) {
+        _messageCache.replaceValue(
+          message.conversationJid,
+          cachedList.map((m) {
+            if (m.id == message.id) {
+              return message;
+            }
+
+            return m;
+          }).toList(),
+        );
+      }
+    });
+  }
 }

lib/service/moxxmpp/connectivity.dart

@@ -0,0 +1,61 @@
+import 'dart:async';
+import 'package:get_it/get_it.dart';
+import 'package:logging/logging.dart';
+import 'package:moxxmpp/moxxmpp.dart';
+import 'package:moxxyv2/service/connectivity.dart';
+import 'package:synchronized/synchronized.dart';
+
+class MoxxyConnectivityManager extends ConnectivityManager {
+  MoxxyConnectivityManager() : super() {
+    GetIt.I.get<ConnectivityService>().stream.listen(_onConnectivityChanged);
+  }
+
+  final Logger _log = Logger('MoxxyConnectivityManager');
+
+  Completer<void>? _completer;
+
+  final Lock _completerLock = Lock();
+
+  Future<void> initialize() async {
+    await _completerLock.synchronized(() async {
+      final result = await GetIt.I.get<ConnectivityService>().hasConnection();
+      if (!result) {
+        _log.finest(
+          'No network connection at initialization: Creating completer',
+        );
+        _completer = Completer<void>();
+      }
+    });
+  }
+
+  Future<void> _onConnectivityChanged(ConnectivityEvent event) async {
+    if (event.regained) {
+      await _completerLock.synchronized(() {
+        _log.finest(
+          'Network regained. _completer != null: ${_completer != null}',
+        );
+        _completer?.complete();
+        _completer = null;
+      });
+    } else if (event.lost) {
+      await _completerLock.synchronized(() {
+        _log.finest('Network connection lost. Creating completer');
+        _completer ??= Completer<void>();
+      });
+    }
+  }
+
+  @override
+  Future<bool> hasConnection() async {
+    return GetIt.I.get<ConnectivityService>().hasConnection();
+  }
+
+  @override
+  Future<void> waitForConnection() async {
+    final c = await _completerLock.synchronized(() => _completer);
+    if (c != null) {
+      _log.finest('waitForConnection: Completer non-null. Waiting.');
+      await c.future;
+    }
+  }
+}

lib/service/moxxmpp/disco.dart

@@ -1,6 +0,0 @@
-import 'package:moxxmpp/moxxmpp.dart';
-
-class MoxxyDiscoManager extends DiscoManager {
-  @override
-  List<Identity> getIdentities() => const [ Identity(category: 'client', type: 'phone', name: 'Moxxy') ];
-}

lib/service/moxxmpp/omemo.dart

@@ -4,27 +4,30 @@ import 'package:moxxyv2/service/conversation.dart';
 import 'package:moxxyv2/service/omemo/omemo.dart';
 import 'package:omemo_dart/omemo_dart.dart';
 
-class MoxxyOmemoManager extends OmemoManager {
-
+class MoxxyOmemoManager extends BaseOmemoManager {
   MoxxyOmemoManager() : super();
 
   @override
-  Future<OmemoSessionManager> getSessionManager() async {
+  Future<OmemoManager> getOmemoManager() async {
     final os = GetIt.I.get<OmemoService>();
     await os.ensureInitialized();
-    return os.omemoState;
+    return os.omemoManager;
   }
 
   @override
   Future<bool> shouldEncryptStanza(JID toJid, Stanza stanza) async {
     // Never encrypt stanzas that contain PubSub elements
     if (stanza.firstTag('pubsub', xmlns: pubsubXmlns) != null ||
-        stanza.firstTag('pubsub', xmlns: pubsubOwnerXmlns) != null) {
+        stanza.firstTag('pubsub', xmlns: pubsubOwnerXmlns) != null ||
+        stanza.firstTagByXmlns(carbonsXmlns) != null ||
+        stanza.firstTagByXmlns(rosterXmlns) != null) {
       return false;
     }
 
     // Encrypt when the conversation is set to use OMEMO.
-    return GetIt.I.get<ConversationService>().shouldEncryptForConversation(toJid);
+    return GetIt.I
+        .get<ConversationService>()
+        .shouldEncryptForConversation(toJid);
   }
 }
 
@@ -33,7 +36,11 @@ class MoxxyBTBVTrustManager extends BlindTrustBeforeVerificationTrustManager {
     Map<RatchetMapKey, BTBVTrustState> trustCache,
     Map<RatchetMapKey, bool> enablementCache,
     Map<String, List<int>> devices,
-  ) : super(trustCache: trustCache, enablementCache: enablementCache, devices: devices);
+  ) : super(
+          trustCache: trustCache,
+          enablementCache: enablementCache,
+          devices: devices,
+        );
 
   @override
   Future<void> commitState() async {

lib/service/moxxmpp/reconnect.dart

@@ -1,126 +0,0 @@
-import 'dart:async';
-import 'dart:math';
-import 'package:connectivity_plus/connectivity_plus.dart';
-import 'package:get_it/get_it.dart';
-import 'package:logging/logging.dart';
-import 'package:meta/meta.dart';
-import 'package:moxxmpp/moxxmpp.dart';
-import 'package:moxxyv2/service/connectivity.dart';
-import 'package:synchronized/synchronized.dart';
-
-/// This class implements a reconnection policy that is connectivity aware with a random
-/// backoff. This means that we perform the random backoff only as long as we are
-/// connected. Otherwise, we idle until we have a connection again.
-class MoxxyReconnectionPolicy extends ReconnectionPolicy {
-
-  MoxxyReconnectionPolicy({ bool isTesting = false, this.maxBackoffTime })
-  : _isTesting = isTesting,
-    _timerLock = Lock(),
-    _log = Logger('MoxxyReconnectionPolicy'), 
-    super();
-  final Logger _log;
-
-  /// The backoff timer
-  @visibleForTesting
-  Timer? timer;
-  final Lock _timerLock;
-
-  /// Just for testing purposes
-  final bool _isTesting;
-
-  /// Maximum backoff time
-  final int? maxBackoffTime;
-  
-  /// To be called when the conectivity changes
-  Future<void> onConnectivityChanged(bool regained, bool lost) async {
-    // Do nothing if we should not reconnect
-    if (!shouldReconnect && regained) {
-      _log.finest('Connectivity changed but not attempting reconnection as shouldReconnect is false');
-      return;
-    }
-
-    if (lost) {
-      // We just lost network connectivity
-      _log.finest('Lost network connectivity. Queueing failure...');
-
-      // Cancel the timer if it was running
-      await _stopTimer();
-      await setIsReconnecting(false);
-      triggerConnectionLost!();
-    } else if (regained && shouldReconnect) {
-      // We should reconnect
-      _log.finest('Network regained. Attempting reconnection...');
-      await _attemptReconnection(true);
-    }
-  }
-
-  @override
-  Future<void> reset() async {
-    await _stopTimer();
-    await setIsReconnecting(false);
-  }
-
-  Future<void> _stopTimer() async {
-    await _timerLock.synchronized(() {
-      if (timer != null) {
-        timer!.cancel();
-        timer = null;
-        _log.finest('Destroying timer');
-      }
-    });
-  }
-  
-  @visibleForTesting
-  Future<void> onTimerElapsed() async {
-    await _stopTimer();
-
-    _log.finest('Performing reconnect');
-    await performReconnect!();
-  }
-
-  Future<void> _attemptReconnection(bool immediately) async {
-    if (await testAndSetIsReconnecting()) {
-      // Attempt reconnecting
-      int seconds;
-      if (_isTesting) {
-        seconds = 9999;
-      } else {
-        final r = Random().nextInt(15);
-        if (maxBackoffTime != null) {
-          seconds = min(maxBackoffTime!, r);
-        } else {
-          seconds = r;
-        }
-      }
-
-      await _stopTimer();
-      if (immediately) {
-        _log.finest('Immediately attempting reconnection...');
-        await onTimerElapsed();
-      } else {
-        _log.finest('Started backoff timer with ${seconds}s backoff');
-        await _timerLock.synchronized(() {
-          timer = Timer(Duration(seconds: seconds), onTimerElapsed);
-        }); 
-      }
-    } else {
-      _log.severe('_attemptReconnection called while reconnect is running!');
-    }
-  }
-  
-  @override
-  Future<void> onFailure() async {
-    final state = GetIt.I.get<ConnectivityService>().currentState;
-
-    if (state != ConnectivityResult.none) {
-      await _attemptReconnection(false);
-    } else {
-      _log.fine('Failure occurred while no network connection is available. Waiting for connection...');
-    }
-  }
-
-  @override
-  Future<void> onSuccess() async {
-    await reset();
-  }
-}

lib/service/moxxmpp/roster.dart

@@ -1,21 +1,104 @@
 import 'dart:async';
 import 'package:get_it/get_it.dart';
 import 'package:moxxmpp/moxxmpp.dart';
-import 'package:moxxyv2/service/xmpp.dart';
+import 'package:moxxyv2/service/roster.dart';
+import 'package:moxxyv2/service/service.dart';
+import 'package:moxxyv2/service/xmpp_state.dart';
+import 'package:moxxyv2/shared/events.dart';
+import 'package:moxxyv2/shared/models/roster.dart';
 
-class MoxxyRosterManager extends RosterManager {
+class MoxxyRosterStateManager extends BaseRosterStateManager {
   @override
-  Future<void> commitLastRosterVersion(String version) async {
-    await GetIt.I.get<XmppService>().modifyXmppState((state) => state.copyWith(
+  Future<RosterCacheLoadResult> loadRosterCache() async {
+    final rs = GetIt.I.get<RosterService>();
+    return RosterCacheLoadResult(
+      (await GetIt.I.get<XmppStateService>().getXmppState()).lastRosterVersion,
+      (await rs.getRoster())
+          .map(
+            (item) => XmppRosterItem(
+              jid: item.jid,
+              name: item.title,
+              subscription: item.subscription,
+              ask: item.ask.isEmpty ? null : item.ask,
+              groups: item.groups,
+            ),
+          )
+          .toList(),
+    );
+  }
+
+  @override
+  Future<void> commitRoster(
+    String? version,
+    List<String> removed,
+    List<XmppRosterItem> modified,
+    List<XmppRosterItem> added,
+  ) async {
+    final rs = GetIt.I.get<RosterService>();
+    final xss = GetIt.I.get<XmppStateService>();
+    await xss.modifyXmppState(
+      (state) => state.copyWith(
         lastRosterVersion: version,
-    ),);
+      ),
+    );
+
+    // Remove stale items
+    for (final jid in removed) {
+      await rs.removeRosterItemByJid(jid);
     }
 
-  @override
-  Future<void> loadLastRosterVersion() async {
-    final ver = (await GetIt.I.get<XmppService>().getXmppState()).lastRosterVersion;
-    if (ver != null) {
-      setRosterVersion(ver);
-    }
+    // Create new roster items
+    final rosterAdded = List<RosterItem>.empty(growable: true);
+    for (final item in added) {
+      final exists = await rs.getRosterItemByJid(item.jid) != null;
+      // Skip adding items twice
+      if (exists) continue;
+
+      rosterAdded.add(
+        await rs.addRosterItemFromData(
+          '',
+          '',
+          item.jid,
+          item.name ?? item.jid.split('@').first,
+          item.subscription,
+          item.ask ?? '',
+          false,
+          null,
+          null,
+          null,
+          groups: item.groups,
+        ),
+      );
+    }
+
+    // Update modified items
+    final rosterModified = List<RosterItem>.empty(growable: true);
+    for (final item in modified) {
+      final ritem = await rs.getRosterItemByJid(item.jid);
+      if (ritem == null) {
+        //_log.warning('Could not find roster item with JID $jid during update');
+        continue;
+      }
+
+      rosterModified.add(
+        await rs.updateRosterItem(
+          ritem.id,
+          title: item.name,
+          subscription: item.subscription,
+          ask: item.ask,
+          groups: item.groups,
+        ),
+      );
+    }
+
+    // Tell the UI
+    // TODO(Unknown): This may not be the cleanest place to put it
+    sendEvent(
+      RosterDiffEvent(
+        added: rosterAdded,
+        modified: rosterModified,
+        removed: removed,
+      ),
+    );
   }
 }

lib/service/moxxmpp/socket.dart

@@ -2,18 +2,20 @@ import 'package:moxdns/moxdns.dart';
 import 'package:moxxmpp_socket_tcp/moxxmpp_socket_tcp.dart';
 
 class MoxxyTCPSocketWrapper extends TCPSocketWrapper {
-  MoxxyTCPSocketWrapper() : super(false);
+  MoxxyTCPSocketWrapper() : super();
 
   @override
   Future<List<MoxSrvRecord>> srvQuery(String domain, bool dnssec) async {
     final records = await MoxdnsPlugin.srvQuery(domain, dnssec);
     return records
-      .map((record) => MoxSrvRecord(
+        .map(
+          (record) => MoxSrvRecord(
             record.priority,
             record.weight,
             record.target,
             record.port,
-      ),)
+          ),
+        )
         .toList();
   }
 }

lib/service/moxxmpp/stream.dart

@@ -1,31 +1,36 @@
 import 'dart:async';
 import 'package:get_it/get_it.dart';
 import 'package:moxxmpp/moxxmpp.dart';
-import 'package:moxxyv2/service/xmpp.dart';
+import 'package:moxxyv2/service/xmpp_state.dart';
 
 class MoxxyStreamManagementManager extends StreamManagementManager {
   @override
   bool shouldTriggerAckedEvent(Stanza stanza) {
     return stanza.tag == 'message' &&
-      stanza.id != null && (
-        stanza.firstTag('body') != null ||
+        stanza.id != null &&
+        (stanza.firstTag('body') != null ||
             stanza.firstTag('x', xmlns: oobDataXmlns) != null ||
             stanza.firstTag('file-sharing', xmlns: sfsXmlns) != null ||
-        stanza.firstTag('file-upload', xmlns: fileUploadNotificationXmlns) != null ||
-        stanza.firstTag('encrypted', xmlns: omemoXmlns) != null
-      );
+            stanza.firstTag(
+                  'file-upload',
+                  xmlns: fileUploadNotificationXmlns,
+                ) !=
+                null ||
+            stanza.firstTag('encrypted', xmlns: omemoXmlns) != null);
   }
 
   @override
   Future<void> commitState() async {
-    await GetIt.I.get<XmppService>().modifyXmppState((s) => s.copyWith(
+    await GetIt.I.get<XmppStateService>().modifyXmppState(
+          (s) => s.copyWith(
             smState: state,
-    ),);
+          ),
+        );
   }
 
   @override
   Future<void> loadState() async {
-    final state = await GetIt.I.get<XmppService>().getXmppState();
+    final state = await GetIt.I.get<XmppStateService>().getXmppState();
     if (state.smState != null) {
       await setState(state.smState!);
     }

lib/service/not_specified.dart

@@ -1,4 +1,6 @@
-class _NotSpecifiedValue { const _NotSpecifiedValue(); }
+class _NotSpecifiedValue {
+  const _NotSpecifiedValue();
+}
 
 /// A value used for indicating that a value is not specified.
 const notSpecified = _NotSpecifiedValue();

lib/service/notifications.dart

@@ -4,6 +4,7 @@ import 'package:flutter/foundation.dart';
 import 'package:get_it/get_it.dart';
 import 'package:logging/logging.dart';
 import 'package:moxxyv2/i18n/strings.g.dart';
+import 'package:moxxyv2/service/contacts.dart';
 import 'package:moxxyv2/service/events.dart';
 import 'package:moxxyv2/service/service.dart';
 import 'package:moxxyv2/service/xmpp.dart';
@@ -49,11 +50,13 @@ class NotificationsService {
         ),
       );
     } else {
-      logger.warning('Received unknown notification action key ${action.buttonKeyPressed}');
+      logger.warning(
+        'Received unknown notification action key ${action.buttonKeyPressed}',
+      );
     }
   }
 
-  Future<void> init() async {
+  Future<void> initialize() async {
     final an = AwesomeNotifications();
     await an.initialize(
       'resource://drawable/ic_service_icon',
@@ -61,12 +64,14 @@ class NotificationsService {
         NotificationChannel(
           channelKey: _messageChannelKey,
           channelName: t.notifications.channels.messagesChannelName,
-          channelDescription: t.notifications.channels.messagesChannelDescription,
+          channelDescription:
+              t.notifications.channels.messagesChannelDescription,
         ),
         NotificationChannel(
           channelKey: _warningChannelKey,
           channelName: t.notifications.channels.warningChannelName,
-          channelDescription: t.notifications.channels.warningChannelDescription,
+          channelDescription:
+              t.notifications.channels.warningChannelDescription,
         ),
       ],
       debug: kDebugMode,
@@ -84,32 +89,49 @@ class NotificationsService {
   /// Show a notification for a message [m] grouped by its conversationJid
   /// attribute. If the message is a media message, i.e. mediaUrl != null and isMedia == true,
   /// then Android's BigPicture will be used.
-  Future<void> showNotification(modelc.Conversation c, modelm.Message m, String title, { String? body }) async {
-    // TODO(Unknown): Keep track of notifications to create a summary notification
+  Future<void> showNotification(
+    modelc.Conversation c,
+    modelm.Message m,
+    String title, {
+    String? body,
+  }) async {
     // See https://github.com/MaikuB/flutter_local_notifications/blob/master/flutter_local_notifications/example/lib/main.dart#L1293
-    final body = m.isMedia ?
-      mimeTypeToEmoji(m.mediaType) :
-      m.body;
+    String body;
+    if (m.stickerPackId != null) {
+      body = t.messages.sticker;
+    } else if (m.isMedia) {
+      body = mimeTypeToEmoji(m.fileMetadata!.mimeType);
+    } else {
+      body = m.body;
+    }
+
+    final css = GetIt.I.get<ContactsService>();
+    final contactIntegrationEnabled = await css.isContactIntegrationEnabled();
+    final title =
+        contactIntegrationEnabled ? c.contactDisplayName ?? c.title : c.title;
+    final avatarPath = contactIntegrationEnabled
+        ? c.contactAvatarPath ?? c.avatarUrl
+        : c.avatarUrl;
 
     await AwesomeNotifications().createNotification(
       content: NotificationContent(
         id: m.id,
         groupKey: c.jid,
         channelKey: _messageChannelKey,
-        summary: c.title,
-        title: c.title,
+        summary: title,
+        title: title,
         body: body,
-        largeIcon: c.avatarUrl.isNotEmpty ? 'file://${c.avatarUrl}' : null,
-        notificationLayout: m.isThumbnailable ?
-          NotificationLayout.BigPicture :
-          NotificationLayout.Messaging,
+        largeIcon: avatarPath.isNotEmpty ? 'file://$avatarPath' : null,
+        notificationLayout: m.isThumbnailable
+            ? NotificationLayout.BigPicture
+            : NotificationLayout.Messaging,
         category: NotificationCategory.Message,
-        bigPicture: m.isThumbnailable ? 'file://${m.mediaUrl}' : null,
+        bigPicture: m.isThumbnailable ? 'file://${m.fileMetadata!.path}' : null,
         payload: <String, String>{
           'conversationJid': c.jid,
           'sid': m.sid,
-          'title': c.title,
-          'avatarUrl': c.avatarUrl,
+          'title': title,
+          'avatarUrl': avatarPath,
         },
       ),
       actionButtons: [

lib/service/omemo/implementations.dart

@@ -1,13 +1,5 @@
-import 'package:moxxyv2/service/moxxmpp/omemo.dart';
 import 'package:omemo_dart/omemo_dart.dart';
 
-Future<OmemoSessionManager> generateNewIdentityImpl(String jid) async {
-  return OmemoSessionManager.generateNewIdentity(
-    jid,
-    MoxxyBTBVTrustManager(
-      <RatchetMapKey, BTBVTrustState>{},
-      <RatchetMapKey, bool>{},
-      <String, List<int>>{},
-    ),
-  );
+Future<OmemoDevice> generateNewIdentityImpl(String jid) async {
+  return OmemoDevice.generateNewDevice(jid);
 }

lib/service/omemo/omemo.dart

@@ -1,19 +1,28 @@
 import 'dart:async';
 import 'dart:collection';
+import 'dart:convert';
 import 'package:flutter/foundation.dart';
 import 'package:get_it/get_it.dart';
 import 'package:hex/hex.dart';
 import 'package:logging/logging.dart';
-import 'package:moxxmpp/moxxmpp.dart';
+import 'package:moxxmpp/moxxmpp.dart' as moxxmpp;
+import 'package:moxxyv2/service/database/constants.dart';
 import 'package:moxxyv2/service/database/database.dart';
+import 'package:moxxyv2/service/database/helpers.dart';
+import 'package:moxxyv2/service/message.dart';
 import 'package:moxxyv2/service/moxxmpp/omemo.dart';
 import 'package:moxxyv2/service/omemo/implementations.dart';
-import 'package:moxxyv2/shared/models/omemo_device.dart';
+import 'package:moxxyv2/service/omemo/types.dart';
+import 'package:moxxyv2/service/service.dart';
+import 'package:moxxyv2/service/xmpp_state.dart';
+import 'package:moxxyv2/shared/events.dart';
+import 'package:moxxyv2/shared/models/message.dart';
+import 'package:moxxyv2/shared/models/omemo_device.dart' as model;
 import 'package:omemo_dart/omemo_dart.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
 import 'package:synchronized/synchronized.dart';
 
 class OmemoDoubleRatchetWrapper {
-
   OmemoDoubleRatchetWrapper(this.ratchet, this.id, this.jid);
   final OmemoDoubleRatchet ratchet;
   final int id;
@@ -21,59 +30,94 @@ class OmemoDoubleRatchetWrapper {
 }
 
 class OmemoService {
-
   final Logger _log = Logger('OmemoService');
 
   bool _initialized = false;
   final Lock _lock = Lock();
-  final Queue<Completer<void>> _waitingForInitialization = Queue<Completer<void>>();
+  final Queue<Completer<void>> _waitingForInitialization =
+      Queue<Completer<void>>();
+  final Map<String, Map<int, String>> _fingerprintCache = {};
 
-  late OmemoSessionManager omemoState;
+  late OmemoManager omemoManager;
 
   Future<void> initializeIfNeeded(String jid) async {
     final done = await _lock.synchronized(() => _initialized);
     if (done) return;
 
-    final db = GetIt.I.get<DatabaseService>();
-    final device = await db.loadOmemoDevice(jid);
+    final device = await _loadOmemoDevice(jid);
+    final ratchetMap = <RatchetMapKey, OmemoDoubleRatchet>{};
+    final deviceList = <String, List<int>>{};
     if (device == null) {
       _log.info('No OMEMO marker found. Generating OMEMO identity...');
-      // Generate the identity in the background
-      omemoState = await compute(generateNewIdentityImpl, jid);
-
-      await commitDevice(await omemoState.getDevice());
-      await commitDeviceMap(<String, List<int>>{});
-      await commitTrustManager(await omemoState.trustManager.toJson());
     } else {
       _log.info('OMEMO marker found. Restoring OMEMO state...');
-      final ratchetMap = <RatchetMapKey, OmemoDoubleRatchet>{};
-      for (final ratchet in await GetIt.I.get<DatabaseService>().loadRatchets()) {
+      for (final ratchet in await _loadRatchets()) {
         final key = RatchetMapKey(ratchet.jid, ratchet.id);
         ratchetMap[key] = ratchet.ratchet;
       }
 
-      final db = GetIt.I.get<DatabaseService>();
-      omemoState = OmemoSessionManager(
-        device,
-        await db.loadOmemoDeviceList(),
-        ratchetMap,
-        await loadTrustManager(),
-      );
+      deviceList.addAll(await _loadOmemoDeviceList());
     }
 
-    omemoState.eventStream.listen((event) async {
-      if (event is RatchetModifiedEvent) {
-        await GetIt.I.get<DatabaseService>().saveRatchet(
-          OmemoDoubleRatchetWrapper(event.ratchet, event.deviceId, event.jid),
+    final om = GetIt.I
+        .get<moxxmpp.XmppConnection>()
+        .getManagerById<moxxmpp.BaseOmemoManager>(moxxmpp.omemoManager)!;
+    omemoManager = OmemoManager(
+      device ?? await compute(generateNewIdentityImpl, jid),
+      await loadTrustManager(),
+      om.sendEmptyMessageImpl,
+      om.fetchDeviceList,
+      om.fetchDeviceBundle,
+      om.subscribeToDeviceListImpl,
     );
-      } else if (event is DeviceMapModifiedEvent) {
-        await commitDeviceMap(event.map);
+
+    if (device == null) {
+      await commitDevice(await omemoManager.getDevice());
+      await commitDeviceMap(<String, List<int>>{});
+      await commitTrustManager(await omemoManager.trustManager.toJson());
+    }
+
+    omemoManager.initialize(
+      ratchetMap,
+      deviceList,
+    );
+
+    omemoManager.eventStream.listen((event) async {
+      if (event is RatchetModifiedEvent) {
+        await _saveRatchet(
+          OmemoDoubleRatchetWrapper(
+            event.ratchet,
+            event.deviceId,
+            event.jid,
+          ),
+        );
+
+        if (event.added) {
+          // Cache the fingerprint
+          final fingerprint = await event.ratchet.getOmemoFingerprint();
+          await _addFingerprintsToCache([
+            OmemoCacheTriple(
+              event.jid,
+              event.deviceId,
+              fingerprint,
+            ),
+          ]);
+
+          if (_fingerprintCache.containsKey(event.jid)) {
+            _fingerprintCache[event.jid]![event.deviceId] = fingerprint;
+          }
+
+          await addNewDeviceMessage(event.jid, event.deviceId);
+        }
+      } else if (event is DeviceListModifiedEvent) {
+        await commitDeviceMap(event.list);
       } else if (event is DeviceModifiedEvent) {
         await commitDevice(event.device);
 
         // Publish it
-        await GetIt.I.get<XmppConnection>()
-          .getManagerById<OmemoManager>(omemoManager)!
+        await GetIt.I
+            .get<moxxmpp.XmppConnection>()
+            .getManagerById<moxxmpp.BaseOmemoManager>(moxxmpp.omemoManager)!
             .publishBundle(await event.device.toBundle());
       }
     });
@@ -88,32 +132,63 @@ class OmemoService {
     });
   }
 
-  Future<OmemoDevice> regenerateDevice(String jid) async {
+  /// Adds a pseudo message saying that [jid] added a new device with id [deviceId].
+  /// If, however, [jid] is our own JID, then nothing is done.
+  Future<void> addNewDeviceMessage(String jid, int deviceId) async {
+    // Add a pseudo message if it is not about our own devices
+    final xmppState = await GetIt.I.get<XmppStateService>().getXmppState();
+    if (jid == xmppState.jid) return;
+
+    final ms = GetIt.I.get<MessageService>();
+    final message = await ms.addMessageFromData(
+      '',
+      DateTime.now().millisecondsSinceEpoch,
+      '',
+      jid,
+      '',
+      false,
+      false,
+      false,
+      pseudoMessageType: pseudoMessageTypeNewDevice,
+      pseudoMessageData: <String, dynamic>{
+        'deviceId': deviceId,
+        'jid': jid,
+      },
+    );
+    sendEvent(
+      MessageAddedEvent(
+        message: message,
+      ),
+    );
+  }
+
+  Future<model.OmemoDevice> regenerateDevice(String jid) async {
     // Prevent access to the session manager as it is (mostly) guarded ensureInitialized
     await _lock.synchronized(() {
       _initialized = false;
     });
 
     _log.info('No OMEMO marker found. Generating OMEMO identity...');
-    final oldId = await omemoState.getDeviceId();
+    final oldId = await omemoManager.getDeviceId();
 
     // Clear the database
-    await GetIt.I.get<DatabaseService>().emptyOmemoSessionTables();
+    await _emptyOmemoSessionTables();
 
     // Regenerate the identity in the background
-    omemoState = await compute(generateNewIdentityImpl, jid);
-
-    await commitDevice(await omemoState.getDevice());
+    final device = await compute(generateNewIdentityImpl, jid);
+    await omemoManager.replaceDevice(device);
+    await commitDevice(device);
     await commitDeviceMap(<String, List<int>>{});
-    await commitTrustManager(await omemoState.trustManager.toJson());
+    await commitTrustManager(await omemoManager.trustManager.toJson());
 
     // Remove the old device
-    final omemo = GetIt.I.get<XmppConnection>()
-      .getManagerById<OmemoManager>(omemoManager)!;
+    final omemo = GetIt.I
+        .get<moxxmpp.XmppConnection>()
+        .getManagerById<moxxmpp.BaseOmemoManager>(moxxmpp.omemoManager)!;
     await omemo.deleteDevice(oldId);
 
     // Publish the new one
-    await omemo.publishBundle(await omemoState.getDeviceBundle());
+    await omemo.publishBundle(await omemoManager.getDeviceBundle());
 
     // Allow access again
     await _lock.synchronized(() {
@@ -126,7 +201,7 @@ class OmemoService {
     });
 
     // Return the OmemoDevice
-    return OmemoDevice(
+    return model.OmemoDevice(
       await getDeviceFingerprint(),
       true,
       true,
@@ -154,11 +229,11 @@ class OmemoService {
   }
 
   Future<void> commitDeviceMap(Map<String, List<int>> deviceMap) async {
-    await GetIt.I.get<DatabaseService>().saveOmemoDeviceList(deviceMap);
+    await _saveOmemoDeviceList(deviceMap);
   }
 
-  Future<void> commitDevice(Device device) async {
-    await GetIt.I.get<DatabaseService>().saveOmemoDevice(device);
+  Future<void> commitDevice(OmemoDevice device) async {
+    await _saveOmemoDevice(device);
   }
 
   /// Requests our device list and checks if the current device is in it. If not, then
@@ -168,55 +243,115 @@ class OmemoService {
     await ensureInitialized();
     _log.finest('publishDeviceIfNeeded: Done');
 
-    final conn = GetIt.I.get<XmppConnection>();
-    final omemo = conn.getManagerById<OmemoManager>(omemoManager)!;
-    final dm = conn.getManagerById<DiscoManager>(discoManager)!;
-    final bareJid = conn.getConnectionSettings().jid.toBare();
-    final device = await omemoState.getDevice();
+    final conn = GetIt.I.get<moxxmpp.XmppConnection>();
+    final omemo =
+        conn.getManagerById<moxxmpp.BaseOmemoManager>(moxxmpp.omemoManager)!;
+    final dm = conn.getManagerById<moxxmpp.DiscoManager>(moxxmpp.discoManager)!;
+    final bareJid = conn.connectionSettings.jid.toBare();
+    final device = await omemoManager.getDevice();
 
     final bundlesRaw = await dm.discoItemsQuery(
-      bareJid.toString(),
-      node: omemoBundlesXmlns,
+      bareJid,
+      node: moxxmpp.omemoBundlesXmlns,
     );
-    if (bundlesRaw.isType<DiscoError>()) {
+    if (bundlesRaw.isType<moxxmpp.DiscoError>()) {
       await omemo.publishBundle(await device.toBundle());
-      return bundlesRaw.get<DiscoError>();
+      return bundlesRaw.get<moxxmpp.DiscoError>();
     }
 
     final bundleIds = bundlesRaw
-      .get<List<DiscoItem>>()
+        .get<List<moxxmpp.DiscoItem>>()
         .where((item) => item.name != null)
         .map((item) => int.parse(item.name!));
     if (!bundleIds.contains(device.id)) {
       final result = await omemo.publishBundle(await device.toBundle());
-      if (result.isType<OmemoError>()) return result.get<OmemoError>();
+      if (result.isType<moxxmpp.OmemoError>()) {
+        return result.get<moxxmpp.OmemoError>();
+      }
       return null;
     }
 
     final idsRaw = await omemo.getDeviceList(bareJid);
-    final ids = idsRaw.isType<OmemoError>() ? <int>[] : idsRaw.get<List<int>>();
+    final ids =
+        idsRaw.isType<moxxmpp.OmemoError>() ? <int>[] : idsRaw.get<List<int>>();
     if (!ids.contains(device.id)) {
       final result = await omemo.publishBundle(await device.toBundle());
-      if (result.isType<OmemoError>()) return result.get<OmemoError>();
+      if (result.isType<moxxmpp.OmemoError>()) {
+        return result.get<moxxmpp.OmemoError>();
+      }
       return null;
     }
 
     return null;
   }
 
-  Future<List<OmemoDevice>> getOmemoKeysForJid(String jid) async {
+  Future<void> _fetchFingerprintsAndCache(moxxmpp.JID jid) async {
+    final bareJid = jid.toBare().toString();
+    final allDevicesRaw = await GetIt.I
+        .get<moxxmpp.XmppConnection>()
+        .getManagerById<moxxmpp.BaseOmemoManager>(moxxmpp.omemoManager)!
+        .retrieveDeviceBundles(jid);
+    if (allDevicesRaw.isType<List<OmemoBundle>>()) {
+      final allDevices = allDevicesRaw.get<List<OmemoBundle>>();
+      final map = <int, String>{};
+      final items = List<OmemoCacheTriple>.empty(growable: true);
+      for (final device in allDevices) {
+        final curveIk = await device.ik.toCurve25519();
+        final fingerprint = HEX.encode(await curveIk.getBytes());
+        map[device.id] = fingerprint;
+        items.add(OmemoCacheTriple(bareJid, device.id, fingerprint));
+      }
+
+      // Cache them in memory
+      _fingerprintCache[bareJid] = map;
+
+      // Cache them in the database
+      await _addFingerprintsToCache(items);
+    }
+  }
+
+  Future<void> _loadOrFetchFingerprints(moxxmpp.JID jid) async {
+    final bareJid = jid.toBare().toString();
+    if (!_fingerprintCache.containsKey(bareJid)) {
+      // First try to load it from the database
+      final triples = await _getFingerprintsFromCache(bareJid);
+      if (triples.isEmpty) {
+        // We found no fingerprints in the database, so try to fetch them
+        await _fetchFingerprintsAndCache(jid);
+      } else {
+        // We have fetched fingerprints from the database
+        _fingerprintCache[bareJid] = Map<int, String>.fromEntries(
+          triples.map((triple) {
+            return MapEntry<int, String>(
+              triple.deviceId,
+              triple.fingerprint,
+            );
+          }),
+        );
+      }
+    }
+  }
+
+  Future<List<model.OmemoDevice>> getOmemoKeysForJid(String jid) async {
     await ensureInitialized();
-    final fingerprints = await omemoState.getHexFingerprintsForJid(jid);
-    final keys = List<OmemoDevice>.empty(growable: true);
-    for (final fp in fingerprints) {
+
+    // Get finger prints if we have to
+    await _loadOrFetchFingerprints(moxxmpp.JID.fromString(jid));
+
+    final keys = List<model.OmemoDevice>.empty(growable: true);
+    final tm =
+        omemoManager.trustManager as BlindTrustBeforeVerificationTrustManager;
+    final trustMap = await tm.getDevicesTrust(jid);
+
+    if (!_fingerprintCache.containsKey(jid)) return [];
+    for (final deviceId in _fingerprintCache[jid]!.keys) {
       keys.add(
-        OmemoDevice(
-          fp.fingerprint,
-          await omemoState.trustManager.isTrusted(jid, fp.deviceId),
-          // TODO(Unknown): Allow verifying OMEMO keys
-          false,
-          await omemoState.trustManager.isEnabled(jid, fp.deviceId),
-          fp.deviceId,
+        model.OmemoDevice(
+          _fingerprintCache[jid]![deviceId]!,
+          await tm.isTrusted(jid, deviceId),
+          trustMap[deviceId] == BTBVTrustState.verified,
+          await tm.isEnabled(jid, deviceId),
+          deviceId,
         ),
       );
     }
@@ -225,81 +360,394 @@ class OmemoService {
   }
 
   Future<void> commitTrustManager(Map<String, dynamic> json) async {
-
-    await GetIt.I.get<DatabaseService>().saveTrustCache(
+    await _saveTrustCache(
       json['trust']! as Map<String, int>,
     );
-    await GetIt.I.get<DatabaseService>().saveTrustEnablementList(
+    await _saveTrustEnablementList(
       json['enable']! as Map<String, bool>,
     );
-    await GetIt.I.get<DatabaseService>().saveTrustDeviceList(
+    await _saveTrustDeviceList(
       json['devices']! as Map<String, List<int>>,
     );
   }
 
   Future<MoxxyBTBVTrustManager> loadTrustManager() async {
-    final db = GetIt.I.get<DatabaseService>();
     return MoxxyBTBVTrustManager(
-      await db.loadTrustCache(),
-      await db.loadTrustEnablementList(),
-      await db.loadTrustDeviceList(),
+      await _loadTrustCache(),
+      await _loadTrustEnablementList(),
+      await _loadTrustDeviceList(),
     );
   }
 
-  Future<void> setOmemoKeyEnabled(String jid, int deviceId, bool enabled) async {
+  Future<void> setOmemoKeyEnabled(
+    String jid,
+    int deviceId,
+    bool enabled,
+  ) async {
     await ensureInitialized();
-    await omemoState.trustManager.setEnabled(jid, deviceId, enabled);
+    await omemoManager.trustManager.setEnabled(jid, deviceId, enabled);
   }
 
   Future<void> removeAllSessions(String jid) async {
     await ensureInitialized();
-    await omemoState.removeAllRatchets(jid);
+    await omemoManager.removeAllRatchets(jid);
   }
 
   Future<int> getDeviceId() async {
     await ensureInitialized();
-    return omemoState.getDeviceId();
+    return omemoManager.getDeviceId();
   }
 
-  Future<String> getDeviceFingerprint() async {
-    return (await omemoState.getHexFingerprintForDevice()).fingerprint;
-  }
+  Future<String> getDeviceFingerprint() => omemoManager.getDeviceFingerprint();
 
   /// Returns a list of OmemoDevices for devices we have sessions with and other devices
   /// published on [ownJid]'s devices PubSub node.
   /// Note that the list is made so that the current device is excluded.
-  Future<List<OmemoDevice>> getOwnFingerprints(JID ownJid) async {
-    final conn = GetIt.I.get<XmppConnection>();
+  Future<List<model.OmemoDevice>> getOwnFingerprints(moxxmpp.JID ownJid) async {
     final ownId = await getDeviceId();
-    final keys = List<OmemoDevice>.from(
+    final keys = List<model.OmemoDevice>.from(
       await getOmemoKeysForJid(ownJid.toString()),
     );
+    final bareJid = ownJid.toBare().toString();
 
-    // TODO(PapaTutuWawa): This should be cached in the database and only requested if
-    //                     it's not cached.
-    final allDevicesRaw = await conn.getManagerById<OmemoManager>(omemoManager)!
-      .retrieveDeviceBundles(ownJid);
-    if (allDevicesRaw.isType<List<OmemoBundle>>()) {
-      final allDevices = allDevicesRaw.get<List<OmemoBundle>>();
+    // Get fingerprints if we have to
+    await _loadOrFetchFingerprints(ownJid);
 
-      for (final device in allDevices) {
-        // All devices that are publishes that is not the current device
-        if (device.id == ownId) continue;
-        final curveIk = await device.ik.toCurve25519();
+    final tm =
+        omemoManager.trustManager as BlindTrustBeforeVerificationTrustManager;
+    final trustMap = await tm.getDevicesTrust(bareJid);
 
+    for (final deviceId in _fingerprintCache[bareJid]!.keys) {
+      if (deviceId == ownId) continue;
+      if (keys.indexWhere((key) => key.deviceId == deviceId) != -1) continue;
+
+      final fingerprint = _fingerprintCache[bareJid]![deviceId]!;
       keys.add(
-          OmemoDevice(
-            HEX.encode(await curveIk.getBytes()),
-            false,
-            false,
-            false,
-            device.id,
+        model.OmemoDevice(
+          fingerprint,
+          await tm.isTrusted(bareJid, deviceId),
+          trustMap[deviceId] == BTBVTrustState.verified,
+          await tm.isEnabled(bareJid, deviceId),
+          deviceId,
           hasSessionWith: false,
         ),
       );
     }
-    }
 
     return keys;
   }
+
+  Future<void> verifyDevice(int deviceId, String jid) async {
+    final tm =
+        omemoManager.trustManager as BlindTrustBeforeVerificationTrustManager;
+    await tm.setDeviceTrust(
+      jid,
+      deviceId,
+      BTBVTrustState.verified,
+    );
+  }
+
+  /// Tells omemo_dart, that certain caches are to be seen as invalidated.
+  void onNewConnection() {
+    if (_initialized) {
+      omemoManager.onNewConnection();
+    }
+  }
+
+  /// Database methods
+
+  Future<List<OmemoDoubleRatchetWrapper>> _loadRatchets() async {
+    final results =
+        await GetIt.I.get<DatabaseService>().database.query(omemoRatchetsTable);
+
+    return results.map((ratchet) {
+      final json = jsonDecode(ratchet['mkskipped']! as String) as List<dynamic>;
+      final mkskipped = List<Map<String, dynamic>>.empty(growable: true);
+      for (final i in json) {
+        final element = i as Map<String, dynamic>;
+        mkskipped.add({
+          'key': element['key']! as String,
+          'public': element['public']! as String,
+          'n': element['n']! as int,
+        });
+      }
+
+      return OmemoDoubleRatchetWrapper(
+        OmemoDoubleRatchet.fromJson(
+          {
+            ...ratchet,
+            'acknowledged': intToBool(ratchet['acknowledged']! as int),
+            'mkskipped': mkskipped,
+          },
+        ),
+        ratchet['id']! as int,
+        ratchet['jid']! as String,
+      );
+    }).toList();
+  }
+
+  Future<void> _saveRatchet(OmemoDoubleRatchetWrapper ratchet) async {
+    final json = await ratchet.ratchet.toJson();
+    await GetIt.I.get<DatabaseService>().database.insert(
+          omemoRatchetsTable,
+          {
+            ...json,
+            'mkskipped': jsonEncode(json['mkskipped']),
+            'acknowledged': boolToInt(json['acknowledged']! as bool),
+            'jid': ratchet.jid,
+            'id': ratchet.id,
+          },
+          conflictAlgorithm: ConflictAlgorithm.replace,
+        );
+  }
+
+  Future<Map<RatchetMapKey, BTBVTrustState>> _loadTrustCache() async {
+    final entries = await GetIt.I
+        .get<DatabaseService>()
+        .database
+        .query(omemoTrustCacheTable);
+
+    final mapEntries =
+        entries.map<MapEntry<RatchetMapKey, BTBVTrustState>>((entry) {
+      // TODO(PapaTutuWawa): Expose this from omemo_dart
+      BTBVTrustState state;
+      final value = entry['trust']! as int;
+      if (value == 1) {
+        state = BTBVTrustState.notTrusted;
+      } else if (value == 2) {
+        state = BTBVTrustState.blindTrust;
+      } else if (value == 3) {
+        state = BTBVTrustState.verified;
+      } else {
+        state = BTBVTrustState.notTrusted;
+      }
+
+      return MapEntry(
+        RatchetMapKey.fromJsonKey(entry['key']! as String),
+        state,
+      );
+    });
+
+    return Map.fromEntries(mapEntries);
+  }
+
+  Future<void> _saveTrustCache(Map<String, int> cache) async {
+    final batch = GetIt.I.get<DatabaseService>().database.batch();
+
+    // ignore: cascade_invocations
+    batch.delete(omemoTrustCacheTable);
+    for (final entry in cache.entries) {
+      batch.insert(
+        omemoTrustCacheTable,
+        {
+          'key': entry.key,
+          'trust': entry.value,
+        },
+        conflictAlgorithm: ConflictAlgorithm.replace,
+      );
+    }
+
+    await batch.commit();
+  }
+
+  Future<Map<RatchetMapKey, bool>> _loadTrustEnablementList() async {
+    final entries = await GetIt.I
+        .get<DatabaseService>()
+        .database
+        .query(omemoTrustEnableListTable);
+
+    final mapEntries = entries.map<MapEntry<RatchetMapKey, bool>>((entry) {
+      return MapEntry(
+        RatchetMapKey.fromJsonKey(entry['key']! as String),
+        intToBool(entry['enabled']! as int),
+      );
+    });
+
+    return Map.fromEntries(mapEntries);
+  }
+
+  Future<void> _saveTrustEnablementList(Map<String, bool> list) async {
+    final batch = GetIt.I.get<DatabaseService>().database.batch();
+
+    // ignore: cascade_invocations
+    batch.delete(omemoTrustEnableListTable);
+    for (final entry in list.entries) {
+      batch.insert(
+        omemoTrustEnableListTable,
+        {
+          'key': entry.key,
+          'enabled': boolToInt(entry.value),
+        },
+        conflictAlgorithm: ConflictAlgorithm.replace,
+      );
+    }
+
+    await batch.commit();
+  }
+
+  Future<Map<String, List<int>>> _loadTrustDeviceList() async {
+    final entries = await GetIt.I
+        .get<DatabaseService>()
+        .database
+        .query(omemoTrustDeviceListTable);
+
+    final map = <String, List<int>>{};
+    for (final entry in entries) {
+      final key = entry['jid']! as String;
+      final device = entry['device']! as int;
+
+      if (map.containsKey(key)) {
+        map[key]!.add(device);
+      } else {
+        map[key] = [device];
+      }
+    }
+
+    return map;
+  }
+
+  Future<void> _saveTrustDeviceList(Map<String, List<int>> list) async {
+    final batch = GetIt.I.get<DatabaseService>().database.batch();
+
+    // ignore: cascade_invocations
+    batch.delete(omemoTrustDeviceListTable);
+    for (final entry in list.entries) {
+      for (final device in entry.value) {
+        batch.insert(
+          omemoTrustDeviceListTable,
+          {
+            'jid': entry.key,
+            'device': device,
+          },
+          conflictAlgorithm: ConflictAlgorithm.replace,
+        );
+      }
+    }
+
+    await batch.commit();
+  }
+
+  Future<void> _saveOmemoDevice(OmemoDevice device) async {
+    await GetIt.I.get<DatabaseService>().database.insert(
+          omemoDeviceTable,
+          {
+            'jid': device.jid,
+            'id': device.id,
+            'data': jsonEncode(await device.toJson()),
+          },
+          conflictAlgorithm: ConflictAlgorithm.replace,
+        );
+  }
+
+  Future<OmemoDevice?> _loadOmemoDevice(String jid) async {
+    final data = await GetIt.I.get<DatabaseService>().database.query(
+          omemoDeviceTable,
+          where: 'jid = ?',
+          whereArgs: [jid],
+          limit: 1,
+        );
+    if (data.isEmpty) return null;
+
+    final deviceJson =
+        jsonDecode(data.first['data']! as String) as Map<String, dynamic>;
+    // NOTE: We need to do this because Dart otherwise complains about not being able
+    //       to cast dynamic to List<int>.
+    final opks = List<Map<String, dynamic>>.empty(growable: true);
+    final opksIter = deviceJson['opks']! as List<dynamic>;
+    for (final tmpOpk in opksIter) {
+      final opk = tmpOpk as Map<String, dynamic>;
+      opks.add(<String, dynamic>{
+        'id': opk['id']! as int,
+        'public': opk['public']! as String,
+        'private': opk['private']! as String,
+      });
+    }
+    deviceJson['opks'] = opks;
+    return OmemoDevice.fromJson(deviceJson);
+  }
+
+  Future<Map<String, List<int>>> _loadOmemoDeviceList() async {
+    final list = await GetIt.I
+        .get<DatabaseService>()
+        .database
+        .query(omemoDeviceListTable);
+    final map = <String, List<int>>{};
+    for (final entry in list) {
+      final key = entry['jid']! as String;
+      final id = entry['id']! as int;
+
+      if (map.containsKey(key)) {
+        map[key]!.add(id);
+      } else {
+        map[key] = [id];
+      }
+    }
+
+    return map;
+  }
+
+  Future<void> _saveOmemoDeviceList(Map<String, List<int>> list) async {
+    final batch = GetIt.I.get<DatabaseService>().database.batch();
+
+    // ignore: cascade_invocations
+    batch.delete(omemoDeviceListTable);
+    for (final entry in list.entries) {
+      for (final id in entry.value) {
+        batch.insert(
+          omemoDeviceListTable,
+          {
+            'jid': entry.key,
+            'id': id,
+          },
+          conflictAlgorithm: ConflictAlgorithm.replace,
+        );
+      }
+    }
+
+    await batch.commit();
+  }
+
+  Future<void> _emptyOmemoSessionTables() async {
+    final batch = GetIt.I.get<DatabaseService>().database.batch();
+
+    // ignore: cascade_invocations
+    batch
+      ..delete(omemoRatchetsTable)
+      ..delete(omemoTrustCacheTable)
+      ..delete(omemoTrustEnableListTable);
+
+    await batch.commit();
+  }
+
+  Future<void> _addFingerprintsToCache(List<OmemoCacheTriple> items) async {
+    final batch = GetIt.I.get<DatabaseService>().database.batch();
+    for (final item in items) {
+      batch.insert(
+        omemoFingerprintCache,
+        <String, dynamic>{
+          'jid': item.jid,
+          'id': item.deviceId,
+          'fingerprint': item.fingerprint,
+        },
+        conflictAlgorithm: ConflictAlgorithm.replace,
+      );
+    }
+    await batch.commit();
+  }
+
+  Future<List<OmemoCacheTriple>> _getFingerprintsFromCache(String jid) async {
+    final rawItems = await GetIt.I.get<DatabaseService>().database.query(
+      omemoFingerprintCache,
+      where: 'jid = ?',
+      whereArgs: [jid],
+    );
+
+    return rawItems.map((item) {
+      return OmemoCacheTriple(
+        jid,
+        item['id']! as int,
+        item['fingerprint']! as String,
+      );
+    }).toList();
+  }
 }

lib/service/omemo/types.dart

@@ -0,0 +1,6 @@
+class OmemoCacheTriple {
+  const OmemoCacheTriple(this.jid, this.deviceId, this.fingerprint);
+  final String jid;
+  final int deviceId;
+  final String fingerprint;
+}

lib/service/preferences.dart

@@ -1,12 +1,37 @@
 import 'package:get_it/get_it.dart';
+import 'package:moxxyv2/service/database/constants.dart';
 import 'package:moxxyv2/service/database/database.dart';
+import 'package:moxxyv2/service/database/helpers.dart';
 import 'package:moxxyv2/shared/models/preferences.dart';
 
 class PreferencesService {
   PreferencesState? _preferences;
 
   Future<void> _loadPreferences() async {
-    _preferences = await GetIt.I.get<DatabaseService>().getPreferences();
+    final db = GetIt.I.get<DatabaseService>().database;
+    final preferencesRaw = (await db.query(preferenceTable)).map((preference) {
+      switch (preference['type']! as int) {
+        case typeInt:
+          return {
+            ...preference,
+            'value': stringToInt(preference['value']! as String),
+          };
+        case typeBool:
+          return {
+            ...preference,
+            'value': stringToBool(preference['value']! as String),
+          };
+        case typeString:
+        default:
+          return preference;
+      }
+    }).toList();
+    final json = <String, dynamic>{};
+    for (final preference in preferencesRaw) {
+      json[preference['key']! as String] = preference['value'];
+    }
+
+    _preferences = PreferencesState.fromJson(json);
   }
 
   Future<PreferencesState> getPreferences() async {
@@ -15,10 +40,44 @@ class PreferencesService {
     return _preferences!;
   }
 
-  Future<void> modifyPreferences(PreferencesState Function(PreferencesState) func) async {
+  Future<void> modifyPreferences(
+    PreferencesState Function(PreferencesState) func,
+  ) async {
     if (_preferences == null) await _loadPreferences();
 
     _preferences = func(_preferences!);
-    await GetIt.I.get<DatabaseService>().savePreferences(_preferences!);
+
+    final stateJson = _preferences!.toJson();
+    final preferences = stateJson.keys.map((key) {
+      int type;
+      String value;
+      if (stateJson[key] is int) {
+        type = typeInt;
+        value = intToString(stateJson[key]! as int);
+      } else if (stateJson[key] is bool) {
+        type = typeBool;
+        value = boolToString(stateJson[key]! as bool);
+      } else {
+        type = typeString;
+        value = stateJson[key]! as String;
+      }
+
+      return {
+        'key': key,
+        'type': type,
+        'value': value,
+      };
+    });
+
+    final batch = GetIt.I.get<DatabaseService>().database.batch();
+    for (final preference in preferences) {
+      batch.update(
+        preferenceTable,
+        preference,
+        where: 'key = ?',
+        whereArgs: [preference['key']],
+      );
+    }
+    await batch.commit();
   }
 }

lib/service/reactions.dart

@@ -0,0 +1,203 @@
+import 'package:get_it/get_it.dart';
+import 'package:logging/logging.dart';
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:moxxyv2/service/database/database.dart';
+import 'package:moxxyv2/service/message.dart';
+import 'package:moxxyv2/service/service.dart';
+import 'package:moxxyv2/service/xmpp_state.dart';
+import 'package:moxxyv2/shared/events.dart';
+import 'package:moxxyv2/shared/models/message.dart';
+import 'package:moxxyv2/shared/models/reaction.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+class ReactionWrapper {
+  const ReactionWrapper(this.emojis, this.modified);
+
+  final List<String> emojis;
+
+  final bool modified;
+}
+
+class ReactionsService {
+  final Logger _log = Logger('ReactionsService');
+
+  /// Query the database for 6 distinct emoji reactions associated with the message id
+  /// [id].
+  Future<List<String>> getPreviewReactionsForMessage(int id) async {
+    final reactions = await GetIt.I.get<DatabaseService>().database.query(
+          reactionsTable,
+          where: 'message_id = ?',
+          whereArgs: [id],
+          columns: ['emoji'],
+          distinct: true,
+          limit: 6,
+        );
+
+    return reactions.map((r) => r['emoji']! as String).toList();
+  }
+
+  Future<List<Reaction>> getReactionsForMessage(int id) async {
+    final reactions = await GetIt.I.get<DatabaseService>().database.query(
+      reactionsTable,
+      where: 'message_id = ?',
+      whereArgs: [id],
+    );
+
+    return reactions.map(Reaction.fromJson).toList();
+  }
+
+  Future<List<String>> getReactionsForMessageByJid(int id, String jid) async {
+    final reactions = await GetIt.I.get<DatabaseService>().database.query(
+      reactionsTable,
+      where: 'message_id = ? AND senderJid = ?',
+      whereArgs: [id, jid],
+    );
+
+    return reactions.map((r) => r['emoji']! as String).toList();
+  }
+
+  Future<int> _countReactions(int messageId, String emoji) async {
+    return GetIt.I.get<DatabaseService>().database.count(
+      reactionsTable,
+      'message_id = ? AND emoji = ?',
+      [messageId, emoji],
+    );
+  }
+
+  /// Adds a new reaction [emoji], if possible, to [messageId] and returns the
+  /// new message reaction preview.
+  Future<Message?> addNewReaction(
+    int messageId,
+    String conversationJid,
+    String emoji,
+  ) async {
+    final ms = GetIt.I.get<MessageService>();
+    final msg = await ms.getMessageById(messageId, conversationJid);
+    if (msg == null) {
+      _log.warning('Failed to get message $messageId');
+      return null;
+    }
+
+    if (!msg.reactionsPreview.contains(emoji) &&
+        msg.reactionsPreview.length < 6) {
+      final newPreview = [
+        ...msg.reactionsPreview,
+        emoji,
+      ];
+
+      try {
+        final jid = (await GetIt.I.get<XmppStateService>().getXmppState()).jid!;
+        await GetIt.I.get<DatabaseService>().database.insert(
+              reactionsTable,
+              Reaction(
+                messageId,
+                jid,
+                emoji,
+              ).toJson(),
+              conflictAlgorithm: ConflictAlgorithm.fail,
+            );
+
+        final newMsg = msg.copyWith(
+          reactionsPreview: newPreview,
+        );
+        await ms.replaceMessageInCache(newMsg);
+
+        sendEvent(
+          MessageUpdatedEvent(
+            message: newMsg,
+          ),
+        );
+
+        return newMsg;
+      } catch (ex) {
+        // The reaction already exists
+        return msg;
+      }
+    }
+
+    return msg;
+  }
+
+  Future<Message?> removeReaction(
+    int messageId,
+    String conversationJid,
+    String emoji,
+  ) async {
+    final ms = GetIt.I.get<MessageService>();
+    final msg = await ms.getMessageById(messageId, conversationJid);
+    if (msg == null) {
+      _log.warning('Failed to get message $messageId');
+      return null;
+    }
+
+    await GetIt.I.get<DatabaseService>().database.delete(
+      reactionsTable,
+      where: 'message_id = ? AND emoji = ? AND senderJid = ?',
+      whereArgs: [
+        messageId,
+        emoji,
+        (await GetIt.I.get<XmppStateService>().getXmppState()).jid,
+      ],
+    );
+    final count = await _countReactions(messageId, emoji);
+
+    if (count > 0) {
+      return msg;
+    }
+
+    final newPreview = List<String>.from(msg.reactionsPreview)..remove(emoji);
+    final newMsg = msg.copyWith(
+      reactionsPreview: newPreview,
+    );
+    await ms.replaceMessageInCache(newMsg);
+    sendEvent(
+      MessageUpdatedEvent(
+        message: newMsg,
+      ),
+    );
+    return newMsg;
+  }
+
+  Future<void> processNewReactions(
+    Message msg,
+    String senderJid,
+    List<String> emojis,
+  ) async {
+    // Get all reactions know for this message
+    final allReactions = await getReactionsForMessage(msg.id);
+    final userEmojis =
+        allReactions.where((r) => r.senderJid == senderJid).map((r) => r.emoji);
+    final removedReactions = userEmojis.where((e) => !emojis.contains(e));
+    final addedReactions = emojis.where((e) => !userEmojis.contains(e));
+
+    // Remove and add the new reactions
+    final db = GetIt.I.get<DatabaseService>().database;
+    for (final emoji in removedReactions) {
+      final rows = await db.delete(
+        reactionsTable,
+        where: 'message_id = ? AND senderJid = ? AND emoji = ?',
+        whereArgs: [msg.id, senderJid, emoji],
+      );
+      assert(rows == 1, 'Only one row should be removed');
+    }
+
+    for (final emoji in addedReactions) {
+      await db.insert(
+        reactionsTable,
+        Reaction(
+          msg.id,
+          senderJid,
+          emoji,
+        ).toJson(),
+      );
+    }
+
+    final newMessage = msg.copyWith(
+      reactionsPreview: await getPreviewReactionsForMessage(msg.id),
+    );
+    await GetIt.I.get<MessageService>().replaceMessageInCache(
+          newMessage,
+        );
+    sendEvent(MessageUpdatedEvent(message: newMessage));
+  }
+}

lib/service/roster.dart

@@ -1,189 +1,33 @@
 import 'dart:async';
-import 'dart:collection';
-import 'package:flutter/foundation.dart';
 import 'package:get_it/get_it.dart';
 import 'package:logging/logging.dart';
-import 'package:moxlib/moxlib.dart';
-import 'package:moxplatform/moxplatform.dart';
 import 'package:moxxmpp/moxxmpp.dart';
-import 'package:moxxyv2/service/conversation.dart';
+import 'package:moxxyv2/service/contacts.dart';
+import 'package:moxxyv2/service/database/constants.dart';
 import 'package:moxxyv2/service/database/database.dart';
+import 'package:moxxyv2/service/database/helpers.dart';
+import 'package:moxxyv2/service/not_specified.dart';
 import 'package:moxxyv2/service/service.dart';
+import 'package:moxxyv2/service/subscription.dart';
 import 'package:moxxyv2/shared/events.dart';
-import 'package:moxxyv2/shared/models/conversation.dart';
 import 'package:moxxyv2/shared/models/roster.dart';
 
-/// Closure which returns true if the jid of a [RosterItem] is equal to [jid].
-bool Function(RosterItem) _jidEqualsWrapper(String jid) {
-  return (i) => i.jid == jid;
-}
-
-typedef AddRosterItemFunction = Future<RosterItem> Function(
-  String avatarUrl,
-  String avatarHash,
-  String jid,
-  String title,
-  String subscription,
-  String ask,
-  {
-    List<String> groups,
-  }
-);
-typedef UpdateRosterItemFunction = Future<RosterItem> Function(
-  int id, {
-    String? avatarUrl,
-    String? avatarHash,
-    String? title,
-    String? subscription,
-    String? ask,
-    List<String>? groups,
-  }
-);
-typedef RemoveRosterItemFunction = Future<void> Function(String jid);
-typedef GetConversationFunction = Future<Conversation?> Function(String jid);
-typedef SendEventFunction = void Function(BackgroundEvent event, { String? id });
-
-/// Compare the local roster with the roster we received either by request or by push.
-/// Returns a diff between the roster before and after the request or the push.
-/// NOTE: This abuses the [RosterDiffEvent] type a bit.
-Future<RosterDiffEvent> processRosterDiff(
-  List<RosterItem> currentRoster,
-  List<XmppRosterItem> remoteRoster,
-  bool isRosterPush,
-  AddRosterItemFunction addRosterItemFromData,
-  UpdateRosterItemFunction updateRosterItem,
-  RemoveRosterItemFunction removeRosterItemByJid,
-  GetConversationFunction getConversationByJid,
-  SendEventFunction _sendEvent,
-) async {
-  final removed = List<String>.empty(growable: true);
-  final modified = List<RosterItem>.empty(growable: true);
-  final added = List<RosterItem>.empty(growable: true);
-
-  for (final item in remoteRoster) {
-    if (isRosterPush) {
-      final litem = firstWhereOrNull(currentRoster, _jidEqualsWrapper(item.jid));
-      if (litem != null) {
-        if (item.subscription == 'remove') {
-          // We have the item locally but it has been removed
-          await removeRosterItemByJid(item.jid);
-          removed.add(item.jid);
-          continue;
-        }
-
-        // Item has been modified
-        final newItem = await updateRosterItem(
-          litem.id,
-          subscription: item.subscription,
-          title: item.name,
-          ask: item.ask,
-          groups: item.groups,
-        );
-
-        modified.add(newItem);
-
-        // Check if we have a conversation that we need to modify
-        final conv = await getConversationByJid(item.jid);
-        if (conv != null) {
-          _sendEvent(
-            ConversationUpdatedEvent(
-              conversation: conv.copyWith(subscription: item.subscription),
-            ),
-          );
-        }
-      } else {
-        // Item does not exist locally
-        if (item.subscription == 'remove') {
-          // Item has been removed but we don't have it locally
-          removed.add(item.jid);
-        } else {
-          // Item has been added and we don't have it locally
-          final newItem = await addRosterItemFromData(
-            '',
-            '',
-            item.jid,
-            item.name ?? item.jid.split('@')[0],
-            item.subscription,
-            item.ask ?? '',
-            groups: item.groups,
-          );
-
-          added.add(newItem);
-        }
-      }
-    } else {
-      final litem = firstWhereOrNull(currentRoster, _jidEqualsWrapper(item.jid));
-      if (litem != null) {
-        // Item is modified
-        if (litem.title != item.name || litem.subscription != item.subscription || !listEquals(litem.groups, item.groups)) {
-          final modifiedItem = await updateRosterItem(
-            litem.id,
-            title: item.name,
-            subscription: item.subscription,
-            groups: item.groups,
-          );
-          modified.add(modifiedItem);
-
-          // Check if we have a conversation that we need to modify
-          final conv = await getConversationByJid(litem.jid);
-          if (conv != null) {
-            _sendEvent(
-              ConversationUpdatedEvent(
-                conversation: conv.copyWith(subscription: item.subscription),
-              ),
-            );
-          }
-        }
-      } else {
-        // Item is new
-        added.add(await addRosterItemFromData(
-            '',
-            '',
-            item.jid,
-            item.jid.split('@')[0],
-            item.subscription,
-            item.ask ?? '',
-            groups: item.groups,
-        ),);
-      }
-    }
-  }
-
-  if (!isRosterPush) {
-    for (final item in currentRoster) {
-      final ritem = firstWhereOrNull(remoteRoster, (XmppRosterItem i) => i.jid == item.jid);
-      if (ritem == null) {
-        await removeRosterItemByJid(item.jid);
-        removed.add(item.jid);
-      }
-      // We don't handle the modification case here as that is covered by the huge
-      // loop above
-    }
-  }
-  
-  return RosterDiffEvent(
-    added: added,
-    modified: modified,
-    removed: removed,
-  );
-}
-
 class RosterService {
+  /// The cached list of JID -> RosterItem. Null if not yet loaded
+  Map<String, RosterItem>? _rosterCache;
 
-  RosterService()
-    : _rosterCache = HashMap(),
-      _rosterLoaded = false,
-      _log = Logger('RosterService');
-  final HashMap<String, RosterItem> _rosterCache;
-  bool _rosterLoaded;
-  final Logger _log;
+  /// Logger.
+  final Logger _log = Logger('RosterService');
 
-  Future<bool> isInRoster(String jid) async {
-    if (!_rosterLoaded) {
+  Future<void> _loadRosterIfNeeded() async {
+    if (_rosterCache == null) {
       await loadRosterFromDatabase();
     }
+  }
 
-    return _rosterCache.containsKey(jid);
+  Future<bool> isInRoster(String jid) async {
+    await _loadRosterIfNeeded();
+    return _rosterCache!.containsKey(jid);
   }
 
   /// Wrapper around [DatabaseService]'s addRosterItemFromData that updates the cache.
@@ -194,22 +38,37 @@ class RosterService {
     String title,
     String subscription,
     String ask,
-    {
+    bool pseudoRosterItem,
+    String? contactId,
+    String? contactAvatarPath,
+    String? contactDisplayName, {
     List<String> groups = const [],
-    }
-  ) async {
-    final item = await GetIt.I.get<DatabaseService>().addRosterItemFromData(
+  }) async {
+    // TODO(PapaTutuWawa): Handle groups
+    final i = RosterItem(
+      -1,
       avatarUrl,
       avatarHash,
       jid,
       title,
       subscription,
       ask,
-      groups: groups,
+      pseudoRosterItem,
+      <String>[],
+      contactId: contactId,
+      contactAvatarPath: contactAvatarPath,
+      contactDisplayName: contactDisplayName,
+    );
+
+    final item = i.copyWith(
+      id: await GetIt.I
+          .get<DatabaseService>()
+          .database
+          .insert(rosterTable, i.toDatabaseJson()),
     );
 
     // Update the cache
-    _rosterCache[item.jid] = item;
+    _rosterCache![item.jid] = item;
 
     return item;
   }
@@ -222,40 +81,81 @@ class RosterService {
     String? title,
     String? subscription,
     String? ask,
+    Object pseudoRosterItem = notSpecified,
     List<String>? groups,
+    Object? contactId = notSpecified,
+    Object? contactAvatarPath = notSpecified,
+    Object? contactDisplayName = notSpecified,
+  }) async {
+    final i = <String, dynamic>{};
+
+    if (avatarUrl != null) {
+      i['avatarUrl'] = avatarUrl;
     }
-  ) async {
-    final newItem = await GetIt.I.get<DatabaseService>().updateRosterItem(
-      id,
-      avatarUrl: avatarUrl,
-      avatarHash: avatarHash,
-      title: title,
-      subscription: subscription,
-      ask: ask,
-      groups: groups,
+    if (avatarHash != null) {
+      i['avatarHash'] = avatarHash;
+    }
+    if (title != null) {
+      i['title'] = title;
+    }
+    /*
+    if (groups != null) {
+      i.groups = groups;
+    }
+    */
+    if (subscription != null) {
+      i['subscription'] = subscription;
+    }
+    if (ask != null) {
+      i['ask'] = ask;
+    }
+    if (contactId != notSpecified) {
+      i['contactId'] = contactId as String?;
+    }
+    if (contactAvatarPath != notSpecified) {
+      i['contactAvatarPath'] = contactAvatarPath as String?;
+    }
+    if (contactDisplayName != notSpecified) {
+      i['contactDisplayName'] = contactDisplayName as String?;
+    }
+    if (pseudoRosterItem != notSpecified) {
+      i['pseudoRosterItem'] = boolToInt(pseudoRosterItem as bool);
+    }
+
+    final result =
+        await GetIt.I.get<DatabaseService>().database.updateAndReturn(
+      rosterTable,
+      i,
+      where: 'id = ?',
+      whereArgs: [id],
     );
+    final newItem = RosterItem.fromDatabaseJson(result);
 
     // Update cache
-    _rosterCache[newItem.jid] = newItem;
+    _rosterCache![newItem.jid] = newItem;
 
     return newItem;
   }
 
-  /// Wrapper around [DatabaseService]'s removeRosterItem.
+  /// Removes a roster item from the database and cache
   Future<void> removeRosterItem(int id) async {
-    await GetIt.I.get<DatabaseService>().removeRosterItem(id);
+    // NOTE: This call ensures that _rosterCache != null
+    await GetIt.I.get<DatabaseService>().database.delete(
+      rosterTable,
+      where: 'id = ?',
+      whereArgs: [id],
+    );
+    assert(_rosterCache != null, '_rosterCache must be non-null');
 
     /// Update cache
-    _rosterCache.removeWhere((_, value) => value.id == id);
+    _rosterCache!.removeWhere((_, value) => value.id == id);
   }
 
   /// Removes a roster item from the database based on its JID.
   Future<void> removeRosterItemByJid(String jid) async {
-    if (!_rosterLoaded) {
-      await loadRosterFromDatabase();
-    }
+    await _loadRosterIfNeeded();
 
-    for (final item in _rosterCache.values) {
+    for (final item in _rosterCache!.values) {
       if (item.jid == jid) {
         await removeRosterItem(item.id);
         return;
@@ -265,17 +165,14 @@ class RosterService {
 
   /// Returns the entire roster
   Future<List<RosterItem>> getRoster() async {
-    if (!_rosterLoaded) {
-      await loadRosterFromDatabase();
-    }
-
-    return _rosterCache.values.toList();
+    await _loadRosterIfNeeded();
+    return _rosterCache!.values.toList();
   }
 
   /// Returns the roster item with jid [jid] if it exists. Null otherwise.
   Future<RosterItem?> getRosterItemByJid(String jid) async {
     if (await isInRoster(jid)) {
-      return _rosterCache[jid];
+      return _rosterCache![jid];
     }
 
     return null;
@@ -284,20 +181,29 @@ class RosterService {
   /// Load the roster from the database. This function is guarded against loading the
   /// roster multiple times and thus creating too many "RosterDiff" actions.
   Future<List<RosterItem>> loadRosterFromDatabase() async {
-    final items = await GetIt.I.get<DatabaseService>().loadRosterItems();
+    final itemsRaw =
+        await GetIt.I.get<DatabaseService>().database.query(rosterTable);
+    final items = itemsRaw.map(RosterItem.fromDatabaseJson);
 
-    _rosterLoaded = true;
+    _rosterCache = <String, RosterItem>{};
     for (final item in items) {
-      _rosterCache[item.jid] = item;
+      _rosterCache![item.jid] = item;
     }
 
-    return items;
+    return items.toList();
   }
 
   /// Attempts to add an item to the roster by first performing the roster set
   /// and, if it was successful, create the database entry. Returns the
   /// [RosterItem] model object.
-  Future<RosterItem> addToRosterWrapper(String avatarUrl, String avatarHash, String jid, String title) async {
+  Future<RosterItem> addToRosterWrapper(
+    String avatarUrl,
+    String avatarHash,
+    String jid,
+    String title,
+  ) async {
+    final css = GetIt.I.get<ContactsService>();
+    final contactId = await css.getContactIdForJid(jid);
     final item = await addRosterItemFromData(
       avatarUrl,
       avatarHash,
@@ -305,14 +211,20 @@ class RosterService {
       title,
       'none',
       '',
+      false,
+      contactId,
+      await css.getProfilePicturePathForJid(jid),
+      await css.getContactDisplayName(contactId),
     );
-    final result = await GetIt.I.get<XmppConnection>().getRosterManager().addToRoster(jid, title);
+
+    final result = await GetIt.I
+        .get<XmppConnection>()
+        .getRosterManager()!
+        .addToRoster(jid, title);
     if (!result) {
       // TODO(Unknown): Signal error?
     }
 
-    GetIt.I.get<XmppConnection>().getPresenceManager().sendSubscriptionRequest(jid);
-
     sendEvent(RosterDiffEvent(added: [item]));
     return item;
   }
@@ -320,13 +232,18 @@ class RosterService {
   /// Removes the [RosterItem] with jid [jid] from the server-side roster and, if
   /// successful, from the database. If [unsubscribe] is true, then [jid] won't receive
   /// our presence anymore.
-  Future<bool> removeFromRosterWrapper(String jid, { bool unsubscribe = true }) async {
-    final roster = GetIt.I.get<XmppConnection>().getRosterManager();
-    final presence = GetIt.I.get<XmppConnection>().getPresenceManager();
+  Future<bool> removeFromRosterWrapper(
+    String jid, {
+    bool unsubscribe = true,
+  }) async {
+    final roster = GetIt.I.get<XmppConnection>().getRosterManager()!;
     final result = await roster.removeFromRoster(jid);
-    if (result == RosterRemovalResult.okay || result == RosterRemovalResult.itemNotFound) {
+    if (result == RosterRemovalResult.okay ||
+        result == RosterRemovalResult.itemNotFound) {
       if (unsubscribe) {
-        presence.sendUnsubscriptionRequest(jid);
+        GetIt.I
+            .get<SubscriptionRequestService>()
+            .sendUnsubscriptionRequest(jid);
       }
 
       _log.finest('Removing from roster maybe worked. Removing from database');
@@ -336,73 +253,4 @@ class RosterService {
 
     return false;
   }
-
-  Future<void> requestRoster() async {
-    final roster = GetIt.I.get<XmppConnection>().getManagerById<RosterManager>(rosterManager)!;
-    Result<RosterRequestResult?, RosterError> result;
-    if (roster.rosterVersioningAvailable()) {
-      _log.fine('Stream supports roster versioning');
-      result = await roster.requestRosterPushes();
-      _log.fine('Requesting roster pushes done');
-    } else {
-      _log.fine('Stream does not support roster versioning');
-      result = await roster.requestRoster();
-    }
-
-    if (result.isType<RosterError>()) {
-      _log.warning('Failed to request roster');
-      return;
-    }
-
-    final value = result.get<RosterRequestResult?>();
-    if (value != null) {
-      final currentRoster = await getRoster();
-      sendEvent(
-        await processRosterDiff(
-          currentRoster,
-          value.items,
-          false,
-          addRosterItemFromData,
-          updateRosterItem,
-          removeRosterItemByJid,
-          GetIt.I.get<ConversationService>().getConversationByJid,
-          sendEvent,
-        ),
-      );
-    }
-  }
-
-  /// Handles a roster push.
-  Future<void> handleRosterPushEvent(RosterPushEvent event) async {
-    final item = event.item;
-    final currentRoster = await getRoster();
-    sendEvent(
-      await processRosterDiff(
-        currentRoster,
-        [ item ],
-        true,
-        addRosterItemFromData,
-        updateRosterItem,
-        removeRosterItemByJid,
-        GetIt.I.get<ConversationService>().getConversationByJid,
-        sendEvent,
-      ),
-    );
-  }
-
-  Future<void> acceptSubscriptionRequest(String jid) async {
-    GetIt.I.get<XmppConnection>().getPresenceManager().sendSubscriptionRequestApproval(jid);
-  }
-
-  Future<void> rejectSubscriptionRequest(String jid) async {
-    GetIt.I.get<XmppConnection>().getPresenceManager().sendSubscriptionRequestRejection(jid);
-  }
-
-  void sendSubscriptionRequest(String jid) {
-    GetIt.I.get<XmppConnection>().getPresenceManager().sendSubscriptionRequest(jid);
-  }
-  
-  void sendUnsubscriptionRequest(String jid) {
-    GetIt.I.get<XmppConnection>().getPresenceManager().sendUnsubscriptionRequest(jid);
-  }
 }

lib/service/service.dart

@@ -13,24 +13,29 @@ import 'package:moxxyv2/service/avatars.dart';
 import 'package:moxxyv2/service/blocking.dart';
 import 'package:moxxyv2/service/connectivity.dart';
 import 'package:moxxyv2/service/connectivity_watcher.dart';
+import 'package:moxxyv2/service/contacts.dart';
 import 'package:moxxyv2/service/conversation.dart';
 import 'package:moxxyv2/service/cryptography/cryptography.dart';
 import 'package:moxxyv2/service/database/database.dart';
 import 'package:moxxyv2/service/events.dart';
+import 'package:moxxyv2/service/files.dart';
 import 'package:moxxyv2/service/httpfiletransfer/httpfiletransfer.dart';
 import 'package:moxxyv2/service/language.dart';
 import 'package:moxxyv2/service/message.dart';
-import 'package:moxxyv2/service/moxxmpp/disco.dart';
+import 'package:moxxyv2/service/moxxmpp/connectivity.dart';
 import 'package:moxxyv2/service/moxxmpp/omemo.dart';
-import 'package:moxxyv2/service/moxxmpp/reconnect.dart';
 import 'package:moxxyv2/service/moxxmpp/roster.dart';
 import 'package:moxxyv2/service/moxxmpp/socket.dart';
 import 'package:moxxyv2/service/moxxmpp/stream.dart';
 import 'package:moxxyv2/service/notifications.dart';
 import 'package:moxxyv2/service/omemo/omemo.dart';
 import 'package:moxxyv2/service/preferences.dart';
+import 'package:moxxyv2/service/reactions.dart';
 import 'package:moxxyv2/service/roster.dart';
+import 'package:moxxyv2/service/stickers.dart';
+import 'package:moxxyv2/service/subscription.dart';
 import 'package:moxxyv2/service/xmpp.dart';
+import 'package:moxxyv2/service/xmpp_state.dart';
 import 'package:moxxyv2/shared/commands.dart';
 import 'package:moxxyv2/shared/eventhandler.dart';
 import 'package:moxxyv2/shared/events.dart';
@@ -43,7 +48,9 @@ Future<void> initializeServiceIfNeeded() async {
   final handler = MoxplatformPlugin.handler;
   if (await handler.isRunning()) {
     if (kDebugMode) {
-      logger.fine('Since kDebugMode is true, waiting 600ms before sending PreStartCommand');
+      logger.fine(
+        'Since kDebugMode is true, waiting 600ms before sending PreStartCommand',
+      );
       sleep(const Duration(milliseconds: 600));
     }
 
@@ -55,7 +62,8 @@ Future<void> initializeServiceIfNeeded() async {
     logger.info('Service is running. Sending pre start command');
     await handler.getDataSender().sendData(
           PerformPreStartCommand(
-        systemLocaleCode: WidgetsBinding.instance.platformDispatcher.locale.toLanguageTag(),
+            systemLocaleCode: WidgetsBinding.instance.platformDispatcher.locale
+                .toLanguageTag(),
           ),
           awaitable: false,
         );
@@ -80,11 +88,13 @@ void sendEvent(BackgroundEvent event, { String? id }) {
 void setupLogging() {
   Logger.root.level = kDebugMode ? Level.ALL : Level.INFO;
   Logger.root.onRecord.listen((record) {
-      final logMessageHeader = '[${record.level.name}] (${record.loggerName}) ${record.time}: ';
+    final logMessageHeader =
+        '[${record.level.name}] (${record.loggerName}) ${record.time}: ';
     var msg = record.message;
     do {
       final tooLong = logMessageHeader.length + msg.length >= 967;
-        final line = tooLong ? msg.substring(0, 967 - logMessageHeader.length) : msg;
+      final line =
+          tooLong ? msg.substring(0, 967 - logMessageHeader.length) : msg;
 
       if (tooLong) {
         msg = msg.substring(967 - logMessageHeader.length - 2);
@@ -97,7 +107,11 @@ void setupLogging() {
       if (GetIt.I.isRegistered<UDPLogger>()) {
         final udp = GetIt.I.get<UDPLogger>();
         if (udp.isEnabled()) {
-            udp.sendLog(logMessage, record.time.millisecondsSinceEpoch, record.level.name);
+          udp.sendLog(
+            logMessage,
+            record.time.millisecondsSinceEpoch,
+            record.level.name,
+          );
         }
       }
 
@@ -143,6 +157,11 @@ Future<void> entrypoint() async {
   GetIt.I.registerSingleton<DatabaseService>(DatabaseService());
   await GetIt.I.get<DatabaseService>().initialize();
 
+  // Initialize services
+  GetIt.I.registerSingleton<ConnectivityWatcherService>(
+    ConnectivityWatcherService(),
+  );
+  GetIt.I.registerSingleton<ConnectivityService>(ConnectivityService());
   GetIt.I.registerSingleton<PreferencesService>(PreferencesService());
   GetIt.I.registerSingleton<BlocklistService>(BlocklistService());
   GetIt.I.registerSingleton<NotificationsService>(NotificationsService());
@@ -153,31 +172,61 @@ Future<void> entrypoint() async {
   GetIt.I.registerSingleton<MessageService>(MessageService());
   GetIt.I.registerSingleton<OmemoService>(OmemoService());
   GetIt.I.registerSingleton<CryptographyService>(CryptographyService());
+  GetIt.I.registerSingleton<ContactsService>(ContactsService());
+  GetIt.I.registerSingleton<StickersService>(StickersService());
+  GetIt.I.registerSingleton<XmppStateService>(XmppStateService());
+  GetIt.I.registerSingleton<SubscriptionRequestService>(
+    SubscriptionRequestService(),
+  );
+  GetIt.I.registerSingleton<FilesService>(FilesService());
+  GetIt.I.registerSingleton<ReactionsService>(ReactionsService());
   final xmpp = XmppService();
   GetIt.I.registerSingleton<XmppService>(xmpp);
 
-  await GetIt.I.get<NotificationsService>().init();
+  await GetIt.I.get<NotificationsService>().initialize();
+  await GetIt.I.get<ContactsService>().initialize();
+  await GetIt.I.get<ConnectivityService>().initialize();
+  await GetIt.I.get<ConnectivityWatcherService>().initialize();
 
   if (!kDebugMode) {
-    final enableDebug = (await GetIt.I.get<PreferencesService>().getPreferences()).debugEnabled;
+    final enableDebug =
+        (await GetIt.I.get<PreferencesService>().getPreferences()).debugEnabled;
     Logger.root.level = enableDebug ? Level.ALL : Level.INFO;
   }
 
   // Init the UDPLogger
   await initUDPLogger();
 
-  GetIt.I.registerSingleton<MoxxyReconnectionPolicy>(MoxxyReconnectionPolicy());
+  final connectivityManager = MoxxyConnectivityManager();
+  await connectivityManager.initialize();
   final connection = XmppConnection(
-    GetIt.I.get<MoxxyReconnectionPolicy>(),
+    RandomBackoffReconnectionPolicy(1, 6),
+    connectivityManager,
+    ClientToServerNegotiator(),
     MoxxyTCPSocketWrapper(),
-  )..registerManagers([
+  );
+  await connection.registerFeatureNegotiators([
+    ResourceBindingNegotiator(),
+    StartTlsNegotiator(),
+    StreamManagementNegotiator(),
+    CSINegotiator(),
+    RosterFeatureNegotiator(),
+    SaslScramNegotiator(10, '', '', ScramHashType.sha512),
+    SaslScramNegotiator(9, '', '', ScramHashType.sha256),
+    SaslScramNegotiator(8, '', '', ScramHashType.sha1),
+    SaslPlainNegotiator(),
+  ]);
+  await connection.registerManagers([
     MoxxyStreamManagementManager(),
-      MoxxyDiscoManager(),
-      MoxxyRosterManager(),
+    DiscoManager([
+      const Identity(category: 'client', type: 'phone', name: 'Moxxy'),
+    ]),
+    RosterManager(MoxxyRosterStateManager()),
     MoxxyOmemoManager(),
-      PingManager(),
+    PingManager(const Duration(minutes: 3)),
     MessageManager(),
-      PresenceManager('http://moxxy.im'),
+    PresenceManager(),
+    EntityCapabilitiesManager('http://moxxy.im'),
     CSIManager(),
     CarbonsManager(),
     PubSubManager(),
@@ -198,23 +247,11 @@ Future<void> entrypoint() async {
     DelayedDeliveryManager(),
     MessageRetractionManager(),
     LastMessageCorrectionManager(),
-    ])
-    ..registerFeatureNegotiators([
-      ResourceBindingNegotiator(),
-      StartTlsNegotiator(),
-      StreamManagementNegotiator(),
-      CSINegotiator(),
-      RosterFeatureNegotiator(),
-      SaslScramNegotiator(10, '', '', ScramHashType.sha512),
-      SaslScramNegotiator(9, '', '', ScramHashType.sha256),
-      SaslScramNegotiator(8, '', '', ScramHashType.sha1),
-      SaslPlainNegotiator(),
+    MessageReactionsManager(),
+    StickersManager(),
   ]);
 
   GetIt.I.registerSingleton<XmppConnection>(connection);
-  GetIt.I.registerSingleton<ConnectivityWatcherService>(ConnectivityWatcherService());
-  GetIt.I.registerSingleton<ConnectivityService>(ConnectivityService());
-  await GetIt.I.get<ConnectivityService>().initialize();
 
   GetIt.I.get<Logger>().finest('Done with xmpp');
 
@@ -228,11 +265,17 @@ Future<void> entrypoint() async {
 
   GetIt.I.get<Logger>().finest('Got settings');
   if (settings != null) {
-    unawaited(GetIt.I.get<OmemoService>().initializeIfNeeded(settings.jid.toBare().toString()));
+    unawaited(
+      GetIt.I
+          .get<OmemoService>()
+          .initializeIfNeeded(settings.jid.toBare().toString()),
+    );
 
     // The title of the notification will be changed as soon as the connection state
     // of [XmppConnection] changes.
-    await connection.getManagerById<MoxxyStreamManagementManager>(smManager)!.loadState();
+    await connection
+        .getManagerById<MoxxyStreamManagementManager>(smManager)!
+        .loadState();
     await xmpp.connect(settings, false);
   } else {
     GetIt.I.get<BackgroundService>().setNotification(
@@ -241,10 +284,13 @@ Future<void> entrypoint() async {
         );
   }
 
-  unawaited(GetIt.I.get<SynchronizedQueue<Map<String, dynamic>?>>().removeQueueLock());
+  unawaited(
+    GetIt.I.get<SynchronizedQueue<Map<String, dynamic>?>>().removeQueueLock(),
+  );
   sendEvent(ServiceReadyEvent());
 }
 
+@pragma('vm:entry-point')
 Future<void> receiveUIEvent(Map<String, dynamic>? data) async {
   await GetIt.I.get<SynchronizedQueue<Map<String, dynamic>?>>().add(data);
 }

lib/service/stickers.dart

@@ -0,0 +1,469 @@
+import 'dart:async';
+import 'dart:convert';
+import 'dart:io';
+import 'dart:ui';
+import 'package:archive/archive.dart';
+import 'package:get_it/get_it.dart';
+import 'package:logging/logging.dart';
+import 'package:moxxmpp/moxxmpp.dart' as moxxmpp;
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:moxxyv2/service/database/database.dart';
+import 'package:moxxyv2/service/database/helpers.dart';
+import 'package:moxxyv2/service/files.dart';
+import 'package:moxxyv2/service/httpfiletransfer/client.dart';
+import 'package:moxxyv2/service/httpfiletransfer/helpers.dart';
+import 'package:moxxyv2/service/httpfiletransfer/location.dart';
+import 'package:moxxyv2/service/preferences.dart';
+import 'package:moxxyv2/service/service.dart';
+import 'package:moxxyv2/service/xmpp_state.dart';
+import 'package:moxxyv2/shared/events.dart';
+import 'package:moxxyv2/shared/helpers.dart';
+import 'package:moxxyv2/shared/models/file_metadata.dart';
+import 'package:moxxyv2/shared/models/sticker.dart';
+import 'package:moxxyv2/shared/models/sticker_pack.dart';
+import 'package:path/path.dart' as p;
+
+class StickersService {
+  final Map<String, StickerPack> _stickerPacks = {};
+  final Logger _log = Logger('StickersService');
+
+  Future<StickerPack?> getStickerPackById(String id) async {
+    if (_stickerPacks.containsKey(id)) return _stickerPacks[id];
+
+    final db = GetIt.I.get<DatabaseService>().database;
+    final rawPack = await db.query(
+      stickerPacksTable,
+      where: 'id = ?',
+      whereArgs: [id],
+      limit: 1,
+    );
+
+    if (rawPack.isEmpty) return null;
+
+    final rawStickers = await db.rawQuery(
+      '''
+SELECT
+  sticker.*,
+  fm.id AS fm_id,
+  fm.path AS fm_path,
+  fm.sourceUrls AS fm_sourceUrls,
+  fm.mimeType AS fm_mimeType,
+  fm.thumbnailType AS fm_thumbnailType,
+  fm.thumbnailData AS fm_thumbnailData,
+  fm.width AS fm_width,
+  fm.height AS fm_height,
+  fm.plaintextHashes AS fm_plaintextHashes,
+  fm.encryptionKey AS fm_encryptionKey,
+  fm.encryptionIv AS fm_encryptionIv,
+  fm.encryptionScheme AS fm_encryptionScheme,
+  fm.cipherTextHashes AS fm_cipherTextHashes,
+  fm.filename AS fm_filename,
+  fm.size AS fm_size
+FROM (SELECT * FROM $stickersTable WHERE stickerPackId = ?) AS sticker
+  JOIN $fileMetadataTable fm ON sticker.file_metadata_id = fm.id;
+      ''',
+      [id],
+    );
+
+    _stickerPacks[id] = StickerPack.fromDatabaseJson(
+      rawPack.first,
+      rawStickers.map((sticker) {
+        return Sticker.fromDatabaseJson(
+          sticker,
+          FileMetadata.fromDatabaseJson(
+            getPrefixedSubMap(sticker, 'fm_'),
+          ),
+        );
+      }).toList(),
+    );
+
+    return _stickerPacks[id]!;
+  }
+
+  Future<List<StickerPack>> getStickerPacks() async {
+    if (_stickerPacks.isEmpty) {
+      final rawPackIds = await GetIt.I.get<DatabaseService>().database.query(
+        stickerPacksTable,
+        columns: ['id'],
+      );
+      for (final rawPack in rawPackIds) {
+        final id = rawPack['id']! as String;
+        await getStickerPackById(id);
+      }
+    }
+
+    _log.finest('Got ${_stickerPacks.length} sticker packs');
+    return _stickerPacks.values.toList();
+  }
+
+  Future<void> removeStickerPack(String id) async {
+    final pack = await getStickerPackById(id);
+    assert(pack != null, 'The sticker pack must exist');
+
+    // Delete the files
+    for (final sticker in pack!.stickers) {
+      if (sticker.fileMetadata.path == null) {
+        continue;
+      }
+
+      await GetIt.I.get<FilesService>().updateFileMetadata(
+            sticker.fileMetadata.id,
+            path: null,
+          );
+      final file = File(sticker.fileMetadata.path!);
+      if (file.existsSync()) {
+        await file.delete();
+      }
+    }
+
+    // Remove from the database
+    await GetIt.I.get<DatabaseService>().database.delete(
+      stickerPacksTable,
+      where: 'id = ?',
+      whereArgs: [id],
+    );
+
+    // Remove from the cache
+    _stickerPacks.remove(id);
+
+    // Retract from PubSub
+    final state = await GetIt.I.get<XmppStateService>().getXmppState();
+    final result = await GetIt.I
+        .get<moxxmpp.XmppConnection>()
+        .getManagerById<moxxmpp.StickersManager>(moxxmpp.stickersManager)!
+        .retractStickerPack(moxxmpp.JID.fromString(state.jid!), id);
+
+    if (result.isType<moxxmpp.PubSubError>()) {
+      _log.severe('Failed to retract sticker pack');
+    }
+  }
+
+  Future<void> _publishStickerPack(moxxmpp.StickerPack pack) async {
+    final prefs = await GetIt.I.get<PreferencesService>().getPreferences();
+    final state = await GetIt.I.get<XmppStateService>().getXmppState();
+    final result = await GetIt.I
+        .get<moxxmpp.XmppConnection>()
+        .getManagerById<moxxmpp.StickersManager>(moxxmpp.stickersManager)!
+        .publishStickerPack(
+          moxxmpp.JID.fromString(state.jid!),
+          pack,
+          accessModel: prefs.isStickersNodePublic ? 'open' : null,
+        );
+
+    if (result.isType<moxxmpp.PubSubError>()) {
+      _log.severe('Failed to publish sticker pack');
+    }
+  }
+
+  Future<void> importFromPubSubWithEvent(
+    moxxmpp.JID jid,
+    String stickerPackId,
+  ) async {
+    final stickerPack = await importFromPubSub(jid, stickerPackId);
+    if (stickerPack == null) return;
+
+    sendEvent(
+      StickerPackAddedEvent(
+        stickerPack: stickerPack,
+      ),
+    );
+  }
+
+  /// Takes the jid of the host [jid] and the id [stickerPackId] of the sticker pack
+  /// and tries to fetch and install it, including publishing on our own PubSub node.
+  ///
+  /// On success, returns the installed StickerPack. On failure, returns null.
+  Future<StickerPack?> importFromPubSub(
+    moxxmpp.JID jid,
+    String stickerPackId,
+  ) async {
+    final result = await GetIt.I
+        .get<moxxmpp.XmppConnection>()
+        .getManagerById<moxxmpp.StickersManager>(moxxmpp.stickersManager)!
+        .fetchStickerPack(jid.toBare(), stickerPackId);
+
+    if (result.isType<moxxmpp.PubSubError>()) {
+      _log.warning('Failed to fetch sticker pack $jid:$stickerPackId');
+      return null;
+    }
+
+    final stickerPackRaw = StickerPack.fromMoxxmpp(
+      result.get<moxxmpp.StickerPack>(),
+      false,
+    );
+
+    // Install the sticker pack
+    return installFromPubSub(stickerPackRaw);
+  }
+
+  Future<void> _addStickerPackFromData(StickerPack pack) async {
+    await GetIt.I.get<DatabaseService>().database.insert(
+          stickerPacksTable,
+          pack.toDatabaseJson(),
+        );
+  }
+
+  Future<Sticker> _addStickerFromData(
+    String id,
+    String stickerPackId,
+    String desc,
+    Map<String, String> suggests,
+    FileMetadata fileMetadata,
+  ) async {
+    final s = Sticker(
+      id,
+      stickerPackId,
+      desc,
+      suggests,
+      fileMetadata,
+    );
+
+    await GetIt.I.get<DatabaseService>().database.insert(
+          stickersTable,
+          s.toDatabaseJson(),
+        );
+    return s;
+  }
+
+  Future<StickerPack?> installFromPubSub(StickerPack remotePack) async {
+    assert(!remotePack.local, 'Sticker pack must be remote');
+
+    var success = true;
+    final stickers = List<Sticker>.from(remotePack.stickers);
+    for (var i = 0; i < stickers.length; i++) {
+      final sticker = stickers[i];
+      final stickerPath = await computeCachedPathForFile(
+        sticker.fileMetadata.filename,
+        sticker.fileMetadata.plaintextHashes,
+      );
+
+      // Get file metadata
+      final fileMetadataRaw =
+          await GetIt.I.get<FilesService>().createFileMetadataIfRequired(
+                MediaFileLocation(
+                  sticker.fileMetadata.sourceUrls!,
+                  p.basename(stickerPath),
+                  null,
+                  null,
+                  null,
+                  sticker.fileMetadata.plaintextHashes,
+                  null,
+                  sticker.fileMetadata.size,
+                ),
+                sticker.fileMetadata.mimeType,
+                sticker.fileMetadata.size,
+                sticker.fileMetadata.width != null &&
+                        sticker.fileMetadata.height != null
+                    ? Size(
+                        sticker.fileMetadata.width!.toDouble(),
+                        sticker.fileMetadata.height!.toDouble(),
+                      )
+                    : null,
+                // TODO(Unknown): Maybe consider the thumbnails one day
+                null,
+                null,
+                path: stickerPath,
+              );
+
+      if (!fileMetadataRaw.retrieved) {
+        final downloadStatusCode = await downloadFile(
+          Uri.parse(sticker.fileMetadata.sourceUrls!.first),
+          stickerPath,
+          (_, __) {},
+        );
+
+        if (!isRequestOkay(downloadStatusCode)) {
+          _log.severe('Request not okay: $downloadStatusCode');
+          success = false;
+          break;
+        }
+      }
+
+      stickers[i] = await _addStickerFromData(
+        getStrongestHashFromMap(sticker.fileMetadata.plaintextHashes) ??
+            DateTime.now().millisecondsSinceEpoch.toString(),
+        remotePack.hashValue,
+        sticker.desc,
+        sticker.suggests,
+        fileMetadataRaw.fileMetadata,
+      );
+    }
+
+    if (!success) {
+      _log.severe('Import failed');
+      return null;
+    }
+
+    // Add the sticker pack to the database
+    await _addStickerPackFromData(remotePack);
+
+    // Publish but don't block
+    unawaited(
+      _publishStickerPack(remotePack.toMoxxmpp()),
+    );
+
+    return remotePack.copyWith(
+      stickers: stickers,
+      local: true,
+    );
+  }
+
+  /// Imports a sticker pack from [path].
+  /// The format is as follows:
+  /// - The file MUST be an uncompressed tar archive
+  /// - All files must be at the top level of the archive
+  /// - A file 'urn.xmpp.stickers.0.xml' must exist and must contain only the <pack /> element
+  /// - The File Metadata Elements must also contain a <name /> element
+  ///   - The file referenced by the <name/> element must also exist on the archive's top level
+  Future<StickerPack?> importFromFile(String path) async {
+    final archiveBytes = await File(path).readAsBytes();
+    final archive = TarDecoder().decodeBytes(archiveBytes);
+    final metadata = archive.findFile('urn.xmpp.stickers.0.xml');
+    if (metadata == null) {
+      _log.severe('Invalid sticker pack: No metadata file');
+      return null;
+    }
+
+    moxxmpp.StickerPack packRaw;
+    try {
+      final content = utf8.decode(metadata.content as List<int>);
+      final node = moxxmpp.XMLNode.fromString(content);
+      packRaw = moxxmpp.StickerPack.fromXML(
+        '',
+        node,
+        hashAvailable: false,
+      );
+    } catch (ex) {
+      _log.severe('Invalid sticker pack description: $ex');
+      return null;
+    }
+
+    if (packRaw.restricted) {
+      _log.severe('Invalid sticker pack: Restricted');
+      return null;
+    }
+
+    for (final sticker in packRaw.stickers) {
+      final filename = sticker.metadata.name;
+      if (filename == null) {
+        _log.severe('Invalid sticker pack: One sticker has no <name/>');
+        return null;
+      }
+
+      final stickerFile = archive.findFile(filename);
+      if (stickerFile == null) {
+        _log.severe(
+          'Invalid sticker pack: $filename does not exist in archive',
+        );
+        return null;
+      }
+    }
+
+    final pack = packRaw.copyWithId(
+      moxxmpp.HashFunction.sha256,
+      await packRaw.getHash(moxxmpp.HashFunction.sha256),
+    );
+    _log.finest('New sticker pack identifier: sha256:${pack.id}');
+
+    if (await getStickerPackById(pack.id) != null) {
+      _log.severe('Invalid sticker pack: Already exists');
+      return null;
+    }
+
+    final stickerDirPath = await getStickerPackPath(
+      pack.hashAlgorithm.toName(),
+      pack.hashValue,
+    );
+    final stickerDir = Directory(stickerDirPath);
+    if (!stickerDir.existsSync()) await stickerDir.create(recursive: true);
+
+    // Create the sticker pack first
+    final stickerPack = StickerPack(
+      pack.hashValue,
+      pack.name,
+      pack.summary,
+      [],
+      pack.hashAlgorithm.toName(),
+      pack.hashValue,
+      pack.restricted,
+      true,
+    );
+    await _addStickerPackFromData(stickerPack);
+
+    // Add all stickers
+    final stickers = List<Sticker>.empty(growable: true);
+    for (final sticker in pack.stickers) {
+      // Get the "path" to the sticker
+      final stickerPath = await computeCachedPathForFile(
+        sticker.metadata.name!,
+        sticker.metadata.hashes,
+      );
+
+      // Get metadata
+      final urlSources = sticker.sources
+          .whereType<moxxmpp.StatelessFileSharingUrlSource>()
+          .map((src) => src.url)
+          .toList();
+      final fileMetadataRaw = await GetIt.I
+          .get<FilesService>()
+          .createFileMetadataIfRequired(
+            MediaFileLocation(
+              urlSources,
+              p.basename(stickerPath),
+              null,
+              null,
+              null,
+              sticker.metadata.hashes,
+              null,
+              sticker.metadata.size,
+            ),
+            sticker.metadata.mediaType,
+            sticker.metadata.size,
+            sticker.metadata.width != null && sticker.metadata.height != null
+                ? Size(
+                    sticker.metadata.width!.toDouble(),
+                    sticker.metadata.height!.toDouble(),
+                  )
+                : null,
+            // TODO(Unknown): Maybe consider the thumbnails one day
+            null,
+            null,
+            path: stickerPath,
+          );
+
+      // Only copy the sticker to storage if we don't already have it
+      if (!fileMetadataRaw.retrieved) {
+        final stickerFile = archive.findFile(sticker.metadata.name!)!;
+        await File(stickerPath).writeAsBytes(
+          stickerFile.content as List<int>,
+        );
+      }
+
+      stickers.add(
+        await _addStickerFromData(
+          getStrongestHashFromMap(sticker.metadata.hashes) ??
+              DateTime.now().millisecondsSinceEpoch.toString(),
+          pack.hashValue,
+          sticker.metadata.desc!,
+          sticker.suggests,
+          fileMetadataRaw.fileMetadata,
+        ),
+      );
+    }
+
+    final stickerPackWithStickers = stickerPack.copyWith(
+      stickers: stickers,
+    );
+
+    // Add it to the cache
+    _stickerPacks[pack.hashValue] = stickerPackWithStickers;
+
+    _log.info(
+      'Sticker pack ${stickerPack.id} successfully added to the database',
+    );
+
+    // Publish but don't block
+    unawaited(_publishStickerPack(pack));
+    return stickerPackWithStickers;
+  }
+}

lib/service/subscription.dart

@@ -0,0 +1,95 @@
+import 'package:get_it/get_it.dart';
+import 'package:moxxmpp/moxxmpp.dart';
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:moxxyv2/service/database/database.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+import 'package:synchronized/synchronized.dart';
+
+class SubscriptionRequestService {
+  List<String>? _subscriptionRequests;
+
+  final Lock _lock = Lock();
+
+  /// Only load data from the database into
+  /// [SubscriptionRequestService._subscriptionRequests] when the cache has not yet
+  /// been loaded.
+  Future<void> _loadSubscriptionRequestsIfNeeded() async {
+    await _lock.synchronized(() async {
+      _subscriptionRequests ??= List<String>.from(
+        (await GetIt.I
+                .get<DatabaseService>()
+                .database
+                .query(subscriptionsTable))
+            .map((m) => m['jid']! as String)
+            .toList(),
+      );
+    });
+  }
+
+  Future<List<String>> getSubscriptionRequests() async {
+    await _loadSubscriptionRequestsIfNeeded();
+    return _subscriptionRequests!;
+  }
+
+  Future<void> addSubscriptionRequest(String jid) async {
+    await _loadSubscriptionRequestsIfNeeded();
+
+    await _lock.synchronized(() async {
+      if (!_subscriptionRequests!.contains(jid)) {
+        _subscriptionRequests!.add(jid);
+
+        await GetIt.I.get<DatabaseService>().database.insert(
+              subscriptionsTable,
+              {
+                'jid': jid,
+              },
+              conflictAlgorithm: ConflictAlgorithm.ignore,
+            );
+      }
+    });
+  }
+
+  Future<void> removeSubscriptionRequest(String jid) async {
+    await _loadSubscriptionRequestsIfNeeded();
+
+    await _lock.synchronized(() async {
+      if (_subscriptionRequests!.contains(jid)) {
+        _subscriptionRequests!.remove(jid);
+        await GetIt.I.get<DatabaseService>().database.delete(
+          subscriptionsTable,
+          where: 'jid = ?',
+          whereArgs: [jid],
+        );
+      }
+    });
+  }
+
+  Future<bool> hasPendingSubscriptionRequest(String jid) async {
+    return (await getSubscriptionRequests()).contains(jid);
+  }
+
+  PresenceManager get _presence =>
+      GetIt.I.get<XmppConnection>().getPresenceManager()!;
+
+  /// Accept a subscription request from [jid].
+  Future<void> acceptSubscriptionRequest(String jid) async {
+    _presence.sendSubscriptionRequestApproval(jid);
+    await removeSubscriptionRequest(jid);
+  }
+
+  /// Reject a subscription request from [jid].
+  Future<void> rejectSubscriptionRequest(String jid) async {
+    _presence.sendSubscriptionRequestRejection(jid);
+    await removeSubscriptionRequest(jid);
+  }
+
+  /// Send a subscription request to [jid].
+  void sendSubscriptionRequest(String jid) {
+    _presence.sendSubscriptionRequest(jid);
+  }
+
+  /// Remove a presence subscription with [jid].
+  void sendUnsubscriptionRequest(String jid) {
+    _presence.sendUnsubscriptionRequest(jid);
+  }
+}

lib/service/xmpp_state.dart

@@ -0,0 +1,39 @@
+import 'package:get_it/get_it.dart';
+import 'package:moxxyv2/service/database/constants.dart';
+import 'package:moxxyv2/service/database/database.dart';
+import 'package:moxxyv2/shared/models/xmpp_state.dart';
+import 'package:sqflite_sqlcipher/sqflite.dart';
+
+class XmppStateService {
+  /// Persistent state around the connection, like the SM token, etc.
+  XmppState? _state;
+
+  Future<XmppState> getXmppState() async {
+    if (_state != null) return _state!;
+
+    final json = <String, String?>{};
+    final rowsRaw =
+        await GetIt.I.get<DatabaseService>().database.query(xmppStateTable);
+    for (final row in rowsRaw) {
+      json[row['key']! as String] = row['value'] as String?;
+    }
+
+    _state = XmppState.fromDatabaseTuples(json);
+    return _state!;
+  }
+
+  /// A wrapper to modify the [XmppState] and commit it.
+  Future<void> modifyXmppState(XmppState Function(XmppState) func) async {
+    _state = func(_state!);
+
+    final batch = GetIt.I.get<DatabaseService>().database.batch();
+    for (final tuple in _state!.toDatabaseTuples().entries) {
+      batch.insert(
+        xmppStateTable,
+        <String, String?>{'key': tuple.key, 'value': tuple.value},
+        conflictAlgorithm: ConflictAlgorithm.replace,
+      );
+    }
+    await batch.commit();
+  }
+}

lib/shared/avatar.dart

@@ -1,12 +1,16 @@
 import 'dart:io';
-
 import 'package:path/path.dart' as pathlib;
 import 'package:path_provider/path_provider.dart';
 
 /// Save the bytes [bytes] that represent the user's avatar under
 /// the [cache directory]/users/[jid]/avatar_[hash].png.
 /// [cache directory] is provided by path_provider.
-Future<String> saveAvatarInCache(List<int> bytes, String hash, String jid, String oldPath) async {
+Future<String> saveAvatarInCache(
+  List<int> bytes,
+  String hash,
+  String jid,
+  String oldPath,
+) async {
   final cacheDir = (await getApplicationDocumentsDirectory()).path;
   final avatarsDir = Directory(pathlib.join(cacheDir, 'avatars'));
   await avatarsDir.create(recursive: true);

lib/shared/cache.dart

@@ -24,15 +24,15 @@ abstract class Cache<K, V> {
 }
 
 class _LRUCacheEntry<V> {
-
   const _LRUCacheEntry(this.value, this.t);
   final int t;
   final V value;
 }
 
 class LRUCache<K, V> extends Cache<K, V> {
-
-  LRUCache(this._maxSize) : _cache = {}, _t = 0;
+  LRUCache(this._maxSize)
+      : _cache = {},
+        _t = 0;
   final Map<K, _LRUCacheEntry<V>> _cache;
   final int _maxSize;
   int _t;
@@ -48,6 +48,13 @@ class LRUCache<K, V> extends Cache<K, V> {
   @override
   List<V> getValues() => _cache.values.map((i) => i.value).toList();
 
+  void replaceValue(K key, V newValue) {
+    _cache[key] = _LRUCacheEntry(
+      newValue,
+      _cache[key]!.t,
+    );
+  }
+
   @override
   void cache(K key, V value) {
     if (_cache.length + 1 <= _maxSize) {

lib/shared/commands.dart

@@ -2,5 +2,7 @@ import 'package:moxlib/awaitabledatasender.dart';
 import 'package:moxplatform/moxplatform.dart';
 import 'package:moxxyv2/shared/models/message.dart';
 import 'package:moxxyv2/shared/models/preferences.dart';
+import 'package:moxxyv2/shared/models/sticker.dart';
+import 'package:moxxyv2/shared/models/sticker_pack.dart';
 
 part 'commands.moxxy.dart';