mirror of
https://github.com/ankitects/anki.git
synced 2026-01-14 06:23:57 -05:00
3707e54ffa
This PR replaces the existing Python-driven sync server with a new one in Rust. The new server supports both collection and media syncing, and is compatible with both the new protocol mentioned below, and older clients. A setting has been added to the preferences screen to point Anki to a local server, and a similar setting is likely to come to AnkiMobile soon. Documentation is available here: <https://docs.ankiweb.net/sync-server.html> In addition to the new server and refactoring, this PR also makes changes to the sync protocol. The existing sync protocol places payloads and metadata inside a multipart POST body, which causes a few headaches: - Legacy clients build the request in a non-deterministic order, meaning the entire request needs to be scanned to extract the metadata. - Reqwest's multipart API directly writes the multipart body, without exposing the resulting stream to us, making it harder to track the progress of the transfer. We've been relying on a patched version of reqwest for timeouts, which is a pain to keep up to date. To address these issues, the metadata is now sent in a HTTP header, with the data payload sent directly in the body. Instead of the slower gzip, we now use zstd. The old timeout handling code has been replaced with a new implementation that wraps the request and response body streams to track progress, allowing us to drop the git dependencies for reqwest, hyper-timeout and tokio-io-timeout. The main other change to the protocol is that one-way syncs no longer need to downgrade the collection to schema 11 prior to sending.
50 lines
1.8 KiB
Rust
50 lines
1.8 KiB
Rust
// Copyright: Ankitects Pty Ltd and contributors
|
|
// License: GNU AGPL, version 3 or later; http://www.gnu.org/licenses/agpl.html
|
|
|
|
use std::{fs, io::ErrorKind};
|
|
|
|
use snafu::ResultExt;
|
|
|
|
use crate::{
|
|
error::{FileIoSnafu, FileOp},
|
|
sync::{
|
|
error::{HttpResult, OrHttpErr},
|
|
http_server::media_manager::ServerMediaManager,
|
|
media::{database::server::entry::MediaEntry, zip::zip_files_for_download},
|
|
},
|
|
};
|
|
|
|
impl ServerMediaManager {
|
|
pub fn zip_files_for_download(&mut self, files: Vec<String>) -> HttpResult<Vec<u8>> {
|
|
let entries = self.db.get_entries_for_download(&files)?;
|
|
let filenames_with_data = self.gather_file_data(&entries)?;
|
|
zip_files_for_download(filenames_with_data).or_internal_err("zip files")
|
|
}
|
|
|
|
/// Mutable for the missing file case.
|
|
fn gather_file_data(&mut self, entries: &[MediaEntry]) -> HttpResult<Vec<(String, Vec<u8>)>> {
|
|
let mut out = vec![];
|
|
for entry in entries {
|
|
let path = self.media_folder.join(&entry.nfc_filename);
|
|
match fs::read(&path) {
|
|
Ok(data) => out.push((entry.nfc_filename.clone(), data)),
|
|
Err(err) if err.kind() == ErrorKind::NotFound => {
|
|
self.db
|
|
.forget_missing_file(entry)
|
|
.or_internal_err("forget missing")?;
|
|
None.or_conflict(format!(
|
|
"requested a file that doesn't exist: {}",
|
|
entry.nfc_filename
|
|
))?;
|
|
}
|
|
Err(err) => Err(err)
|
|
.context(FileIoSnafu {
|
|
path,
|
|
op: FileOp::Read,
|
|
})
|
|
.or_internal_err("gather file data")?,
|
|
}
|
|
}
|
|
Ok(out)
|
|
}
|
|
}
|