Commit graph

8 commits

Author SHA1 Message Date
RumovZ
5a53da23ca
Deck scoped dupe check (#2372)
* Support limiting dupe check to deck

* Expose deck limiting dupe check on frontend

* Make CSV dupe options configurable with headers

* Rename duplicate file headers

* Change dupe check limit to enum
2023-02-16 17:53:36 +10:00
Damien Elmes
ded805b504
Switch Rust import style (#2330)
* Prepare to switch Rust import style

* Run nightly format

Closes #2320

* Clean up a few imports

* Enable comment wrapping

* Wrap comments
2023-01-18 21:39:55 +10:00
RumovZ
c521753057
Refactor error handling (#2136)
* Add crate snafu

* Replace all inline structs in AnkiError

* Derive Snafu on AnkiError

* Use snafu for card type errors

* Use snafu whatever error for InvalidInput

* Use snafu for NotFoundError and improve message

* Use snafu for FileIoError to attach context

Remove IoError.
Add some context-attaching helpers to replace code returning bare
io::Errors.

* Add more context-attaching io helpers

* Add message, context and backtrace to new snafus

* Utilize error context and backtrace on frontend

* Rename LocalizedError -> BackendError.
* Remove DocumentedError.
* Have all backend exceptions inherit BackendError.

* Rename localized(_description) -> message

* Remove accidentally committed experimental trait

* invalid_input_context -> ok_or_invalid

* ensure_valid_input! -> require!

* Always return `Err` from `invalid_input!`

Instead of a Result to unwrap, the macro accepts a source error now.

* new_tempfile_in_parent -> new_tempfile_in_parent_of

* ok_or_not_found -> or_not_found

* ok_or_invalid -> or_invalid

* Add crate convert_case

* Use unqualified lowercase type name

* Remove uses of snafu::ensure

* Allow public construction of InvalidInputErrors (dae)

Needed to port the AnkiDroid changes.

* Make into_protobuf() public (dae)

Also required for AnkiDroid. Not sure why it worked previously - possible
bug in older Rust version?
2022-10-21 18:02:12 +10:00
RumovZ
1f8189fe91
Some import/export features and fixes (#2038)
* Keep filtered decks when importing apkg

If all original decks exist and scheduling is included.

* Create missing decks from csv

* Export original decks if with_scheduling

* Also remap original deck ids on import

* Update imported filtered decks

* Fix meta column being mapped to tags

* Fix ids in csv deck and notetype columns

Note: This implies names which parse to an i64 will be seen as ids,
likely resulting in the intended deck/notetype not being found.

* Check for scheduling with revlog and deck configs

Might help with cases in which scheduling was included, but all cards
are new. In such a case, filtered deck should not be converted.

* Fix duplicate with same GUID being created

* Remove redundant `distinct`s from sql query

* Match notes by _either_ guid _or_ first field

* Refactor to emphasise GUID/first field distinction

* Export default deck and config if with scheduling

* Fix default deck being exported if it's a parent
2022-09-03 11:29:06 +10:00
RumovZ
79fbb6c8d8
Keep content of unmapped fields when importing (#2023)
* Keep content of unmapped fields when importing

* Test new behaviour

* Fix typo in `canonify_tags_without_resgistering`

* Log updated note instead of original one

* Revert merging imported tags

But keep old note tags if no new ones are provided.
2022-08-24 16:04:32 +10:00
RumovZ
b6a7760cb4
Restore and save last dupe resolution setting (#1930)
* Restore dupe resolution setting

* Save dupe resolution setting

* Push config logic into backend (dae)
2022-06-24 15:10:06 +10:00
RumovZ
6da5e5b042
CSV import/export fixes and features (#1898)
* Fix footer moving upwards

* Fix column detection

Was broken because escaped line breaks were not considered.
Also removes delimiter detection on `#columns:` line. User must use tabs
or set delimiter beforehand.

* Add CSV preview

* Parse `#tags column:`

* Optionally export deck and notetype with CSV

* Avoid clones in CSV export

* Prevent bottom of page appearing under footer (dae)

* Increase padding to 1em (dae)

With 0.5em, when a vertical scrollbar is shown, it sits right next to
the right edge of the content, making it look like there's no right
margin.

* Experimental changes to make table fit+scroll (dae)

- limit individual cells to 15em, and show ellipses when truncated
- limit total table width to body width, so that inner table is shown
with scrollbar
- use class rather than id - ids are bad practice in Svelte components,
as more than one may be displayed on a single page

* Skip importing foreign notes with filtered decks

Were implicitly imported into the default deck before.
Also some refactoring to fetch deck ids and names beforehand.

* Hide spacer below hidden field mapping

* Fix guid being replaced when updating note

* Fix dupe identity check

Canonify tags before checking if dupe is identical, but only add update
tags later if appropriate.

* Fix deck export for notes with missing card 1

* Fix note lines starting with `#`

csv crate doesn't support escaping a leading comment char. :(

* Support import/export of guids

* Strip HTML from preview rows

* Fix initially set deck if current is filtered

* Make isHtml toggle reactive

* Fix `html_to_text_line()` stripping sound names

* Tweak export option labels

* Switch to patched rust-csv fork

Fixes writing lines starting with `#`, so revert 5ece10ad05.

* List column options with first column field

* Fix flag for exports with HTML stripped
2022-06-09 10:28:01 +10:00
RumovZ
42cbe42f06
Plaintext import/export (#1850)
* Add crate csv

* Add start of csv importing on backend

* Add Menomosyne serializer

* Add csv and json importing on backend

* Add plaintext importing on frontend

* Add csv metadata extraction on backend

* Add csv importing with GUI

* Fix missing dfa file in build

Added compile_data_attr, then re-ran cargo/update.py.

* Don't use doubly buffered reader in csv

* Escape HTML entities if CSV is not HTML

Also use name 'is_html' consistently.

* Use decimal number as foreign ease (like '2.5')

* ForeignCard.ivl → ForeignCard.interval

* Only allow fixed set of CSV delimiters

* Map timestamp of ForeignCard to native due time

* Don't trim CSV records

* Document use of empty strings for defaults

* Avoid creating CardGenContexts for every note

This requires CardGenContext to be generic, so it works both with an
owned and borrowed notetype.

* Show all accepted file types  in import file picker

* Add import_json_file()

* factor → ease_factor

* delimter_from_value → delimiter_from_value

* Map columns to fields, not the other way around

* Fallback to current config for csv metadata

* Add start of new import csv screen

* Temporary fix for compilation issue on Linux/Mac

* Disable jest bazel action for import-csv

Jest fails with an error code if no tests are available, but this would
not be noticable on Windows as Jest is not run there.

* Fix field mapping issue

* Revert "Temporary fix for compilation issue on Linux/Mac"

This reverts commit 21f8a26140.

* Add HtmlSwitch and move Switch to components

* Fix spacing and make selectors consistent

* Fix shortcut tooltip

* Place import button at the top with path

* Fix meta column indices

* Remove NotetypeForString

* Fix queue and type of foreign cards

* Support different dupe resolution strategies

* Allow dupe resolution selection when importing CSV

* Test import of unnormalized text

Close  #1863.

* Fix logging of foreign notes

* Implement CSV exports

* Use db_scalar() in notes_table_len()

* Rework CSV metadata

- Notetypes and decks are either defined by a global id or by a column.
- If a notetype id is provided, its field map must also be specified.
- If a notetype column is provided, fields are now mapped by index
instead of name at import time. So the first non-meta column is used for
the first field of every note, regardless of notetype. This makes
importing easier and should improve compatiblity with files without a
notetype column.
- Ensure first field can be mapped to a column.
- Meta columns must be defined as `#[meta name]:[column index]` instead
of in the `#columns` tag.
- Column labels contain the raw names defined by the file and must be
prettified by the frontend.

* Adjust frontend to new backend column mapping

* Add force flags for is_html and delimiter

* Detect if CSV is HTML by field content

* Update dupe resolution labels

* Simplify selectors

* Fix coalescence of oneofs in TS

* Disable meta columns from selection

Plus a lot of refactoring.

* Make import button stick to the bottom

* Write delimiter and html flag into csv

* Refetch field map after notetype change

* Fix log labels for csv import

* Log notes whose deck/notetype was missing

* Fix hiding of empty log queues

* Implement adding tags to all notes of a csv

* Fix dupe resolution not being set in log

* Implement adding tags to updated notes of a csv

* Check first note field is not empty

* Temporary fix for build on Linux/Mac

* Fix inverted html check (dae)

* Remove unused ftl string

* Delimiter → Separator

* Remove commented-out line

* Don't accept .json files

* Tweak tag ftl strings

* Remove redundant blur call

* Strip sound and add spaces in csv export

* Export HTML by default

* Fix unset deck in Mnemosyne import

Also accept both numbers and strings for notetypes and decks in JSON.

* Make DupeResolution::Update the default

* Fix missing dot in extension

* Make column indices 1-based

* Remove StickContainer from TagEditor

Fixes line breaking, border and z index on ImportCsvPage.

* Assign different key combos to tag editors

* Log all updated duplicates

Add a log field for the true number of found notes.

* Show identical notes as skipped

* Split tag-editor into separate ts module (dae)

* Add progress for CSV export

* Add progress for text import

* Tidy-ups after tag-editor split (dae)

- import-csv no longer depends on editor
- remove some commented lines
2022-06-01 20:26:16 +10:00