Damien Elmes
b5022ad354
store config in separate DB table
...
- mtime is tracked on each key individually, which will allow
merging of config changes when syncing in the future
- added col.(get|set|remove)_config()
- in order to support existing code that was mutating returned
values (eg col.conf["something"]["another"] = 5), the returned list/dict
will be automatically wrapped so that when the value is dropped, it
will save the mutated item back to the DB if it's changed. Code that
is fetching lists/dicts from the config like so:
col.conf["foo"]["bar"] = baz
col.setMod()
will continue to work in most case, but should be gradually updated to:
conf = col.get_config("foo")
conf["bar"] = baz
col.set_config("foo", conf)
2020-04-06 15:39:47 +10:00
Arthur Milchior
c837143d31
Remove useless recursion in DeckManager.rem
...
For any deck the children of it's children are its children. So
applying rem to children of children is useless and actually slightly
costly for deep subdecks
2020-04-05 11:33:23 +02:00
Damien Elmes
1834b65714
downgrade when writing dummy collection
2020-04-04 18:43:26 +10:00
Damien Elmes
3c7eabc51e
add flag to skip downgrade on collection close
...
Disabled for now; when enabled it will allow faster collection
open and close in the normal case, while continuing to downgrade
when exporting or doing a full sync.
Also, when downgrading is disabled, the journal mode is no longer
changed back to delete.
2020-04-04 17:21:45 +10:00
Damien Elmes
ac54f85840
update tag handling
...
- tag list stored in a separate DB table
- non-wildcard searches now do full unicode case folding
(eg tag:masse matches 'Maße')
- wildcard matches do simple unicode case folding
- some functions haven't been updated yet, so ascii folding will
continue to be used in some operations
2020-04-03 19:34:46 +10:00
Damien Elmes
07c414692e
preserve mtime/usn when syncing deck config, and add snake_case names
2020-04-03 19:34:46 +10:00
Damien Elmes
296ebd9c63
set col mod explicitly in decks.select()
2020-04-03 19:34:46 +10:00
Damien Elmes
96e1eca4bb
ensure pending deck/tag/note type changes flushed before searching
2020-04-03 19:34:46 +10:00
Damien Elmes
d7edbedac0
Merge pull request #535 from evandroforks/upload_wheels_to_pypi
...
Created the GitHub Actions step Upload to PyPi
2020-04-03 08:40:17 +10:00
evandrocoan
b3ed8ebfdb
Fixed double python version check for python_requires=">=3.7"
2020-04-02 12:31:12 -03:00
evandrocoan
30107ab09d
Created the GitHub Actions step Upload to PyPi to run only on
...
tagged releases of anki:
https://anki.tenderapp.com/discussions/ankidesktop/40025-anki-in-python-package-index
2020-04-02 02:57:34 -03:00
Damien Elmes
6beb2ff1e3
return the added card id instead of mutating the backend card object
2020-04-02 12:09:46 +10:00
Damien Elmes
7068f97e1f
make sure None is returned in no-such-card case
2020-04-02 11:29:40 +10:00
Damien Elmes
cbfc0207ec
tolerate string values in deck["conf"]
2020-04-01 19:36:40 +10:00
Damien Elmes
9c09d08152
remove defaultConf
2020-04-01 17:36:46 +10:00
Damien Elmes
284ee0860b
allow aborting a media sync while http req in flight
...
The progress handling code needs a rethink, as we now have two separate
ways to flag that the media sync should abort. In the future, it may
make sense to switch to polling the backend for progress, instead of
passing a callback in.
2020-04-01 09:49:25 +10:00
Damien Elmes
6482af7361
add a temporary cache to bring deck list performance back
2020-03-30 20:27:53 +10:00
Damien Elmes
cffe146397
split deck config into separate SQL table
...
- on collection load, the schema is upgraded to 12
- on collection close, the changes are reversed so older clients
can continue to open the collection
- in the future, we could potentially skip the reversal except
when exporting/doing a full sync
- the same approach should work for decks, note types and tags in the
future too
- the deck list code needs updating to cache the deck confs for the
life of the call
2020-03-30 20:01:16 +10:00
Damien Elmes
089a68eb43
move deck conf handling to backend
2020-03-30 14:39:46 +10:00
Damien Elmes
918ec2a5a0
explicitly save deck conf after mutating it
2020-03-30 12:27:52 +10:00
Damien Elmes
d41124343d
initial work on undo support
2020-03-29 17:52:16 +10:00
Damien Elmes
db3538aae3
add an extra second so slow CI server doesn't fail
2020-03-27 15:52:35 +10:00
Damien Elmes
36314f3b6f
Merge pull request #528 from evandroforks/create_actions_for_windows_macos
...
Create actions for Windows and Mac OS
2020-03-27 15:26:22 +10:00
Damien Elmes
fc846ac4e4
use add_card()
2020-03-27 15:11:07 +10:00
evandrocoan
f1fe0209f6
Merge remote-tracking branch 'danielelmes/master' into create_actions_for_windows_macos
2020-03-26 19:31:01 -03:00
Damien Elmes
e28fdde984
handle mtime/usn bump in backend, and tweak integer sizes
2020-03-26 20:55:04 +10:00
Damien Elmes
369457a1c7
fix two cases where a float was being written into due
2020-03-26 19:05:18 +10:00
Damien Elmes
0510ab7c9c
update_card()
2020-03-26 18:54:20 +10:00
Damien Elmes
245a31a432
tweak code for mypy
2020-03-26 17:47:53 +10:00
Damien Elmes
b2fba4e1ff
use backend to get card
2020-03-26 17:47:43 +10:00
Damien Elmes
4d428f9fa8
high due numbers shouldn't force a full sync
2020-03-26 09:09:11 +10:00
evandrocoan
964cc1a11a
Merge remote-tracking branch 'danielelmes/master' into create_actions_for_windows_macos
...
# Conflicts:
# .github/scripts/trailing-newlines.sh
# .github/workflows/checks.yml
# Makefile
# pylib/Makefile
# qt/Makefile
# qt/tools/typecheck-setup.sh
# rspy/Makefile
2020-03-24 22:49:49 -03:00
Damien Elmes
1d819bd750
don't error when -l passes in language without _
2020-03-25 08:53:31 +10:00
Damien Elmes
da07298418
fix sync getting stuck in a loop when changes is a multiple of 250
2020-03-24 19:30:35 +10:00
Damien Elmes
83d1be8743
Merge pull request #510 from evandroforks/fix_tests_on_windows
...
Fix tests on windows
2020-03-24 15:33:02 +10:00
Damien Elmes
602d6b00c2
remove : check from dbproxy
2020-03-24 10:37:12 +10:00
evandrocoan
4fcecaed9f
Created the .github/workflows/windows.yml
...
# Conflicts:
# .github/scripts/trailing-newlines.sh
# Makefile
# qt/tools/typecheck-setup.sh
# react/Makefile
# rspy/Makefile
# svelte/Makefile
# tslib/Makefile
2020-03-23 19:51:03 -03:00
evandrocoan
093af85c2e
Merge remote-tracking branch 'danielelmes/master' into fix_tests_on_windows
...
# Conflicts:
# .github/scripts/trailing-newlines.sh
2020-03-23 18:44:11 -03:00
Damien Elmes
7a67f73183
convert asc to desc instead of appending desc to the end of the order
...
as the latter doesn't work when sorting on more than one column
https://anki.tenderapp.com/discussions/beta-testing/1868-anki-2124-beta#comment_48174812
2020-03-23 19:53:57 +10:00
Damien Elmes
a5d27ede6e
fix v2 timing being returned for v1 users
2020-03-23 13:53:09 +10:00
Damien Elmes
b7fdb8aeb4
fix setting of wal
2020-03-23 13:52:57 +10:00
Damien Elmes
f9f5a33e1f
simplify how the local offset is passed around
...
- no need to store it in conf
- move local_minutes_west() call to collection
2020-03-23 13:52:52 +10:00
Damien Elmes
0e9e7a84e9
release GIL during collection open/close
2020-03-23 09:27:42 +10:00
Damien Elmes
ac78699d13
add back new_timezone_enabled(), as it's used in the prefs screen
2020-03-23 07:40:50 +10:00
Damien Elmes
9d55260425
apply same list->tuple fix to v1 sched
2020-03-22 21:56:02 +10:00
zjosua
119df8c662
Fill _lrnQueue with tuples, not lists
2020-03-22 11:49:40 +01:00
Damien Elmes
71fcbc3ead
fix sync error introduced by e02f032a47
2020-03-22 17:32:34 +10:00
Damien Elmes
f44996075d
handle the two remaining timing issues
2020-03-22 14:43:29 +10:00
Damien Elmes
e02f032a47
use backend for v1 and v2 cutoff calculations
...
this should also fix the CI failures, which were happening because
the datetime module wasn't matching the shifted time.time()
2020-03-22 14:28:26 +10:00
Damien Elmes
b8aaa8fba4
add enum for controlling sort order
...
eg col.find_cards("", order=BuiltinSortKind.CARD_DUE)
2020-03-22 12:59:24 +10:00
Damien Elmes
b6ac1c0d1c
fall back on stock json if orjson unavailable
2020-03-21 21:24:21 +10:00
Damien Elmes
eab7686d2f
remove old finder code; add search hooks to browser & remove old one
2020-03-21 16:38:46 +10:00
Damien Elmes
0089f0149a
add note searching
2020-03-21 09:00:05 +10:00
Damien Elmes
f3a4552df3
don't set deck.sched.today in test
2020-03-21 08:12:32 +10:00
Damien Elmes
3cebb5b3fd
update tests to reflect what now constitutes an error
2020-03-21 08:12:01 +10:00
Damien Elmes
18b37bcb59
tests need to flush before searching
2020-03-21 08:10:52 +10:00
Damien Elmes
23717089e9
add the ability to provide a custom sort order; use backend for find
2020-03-21 07:55:21 +10:00
Damien Elmes
f36857ae27
prototype of integration
...
no ordering yet, and no tests
2020-03-20 21:15:23 +10:00
Damien Elmes
ae42885ee2
make sure we set db to None so we can tell when we've closed the DB
2020-03-20 21:15:23 +10:00
Damien Elmes
5aba5c3ab9
drop the separate i18n backend
2020-03-20 21:15:23 +10:00
Damien Elmes
e5f883d1c3
support opening and closing the DB while keeping backend alive
...
This is safer than just dropping the backend, as .close() will
block if something else is holding the mutex. Also means we can
drop the extra I18nBackend code.
Media syncing still needs fixing.
2020-03-20 21:15:23 +10:00
Damien Elmes
90a44509be
drop usage of flushSched()
2020-03-20 21:15:23 +10:00
Damien Elmes
e53ee52d2a
remove db.close()
2020-03-20 21:15:23 +10:00
Damien Elmes
258a8282d4
move .reopen() to mw; fix exporting
2020-03-20 21:15:23 +10:00
Damien Elmes
a02c4ed994
drop the DB progress handler code
...
This code was an awful hack to provide some semblance of UI
responsiveness while executing DB statements on the main thread.
Instead, we can just run DB statements in a background thread now,
keeping the UI responsive.
2020-03-20 21:15:23 +10:00
Damien Elmes
8cb3ed7435
drop lock() and setAutocommit()
...
We no longer need to worry about pysqlite implicitly beginning
transactions, and can be more explicit about beginning/ending
transactions
save() now also has a trx argument controlling whether a
transaction should be started / left open
2020-03-20 21:15:23 +10:00
Damien Elmes
3256c74409
pass weakref in from storage to fix type checking/code completion
2020-03-20 21:15:23 +10:00
Damien Elmes
116f7fa3f1
support first_row_only
2020-03-20 21:15:23 +10:00
Damien Elmes
7e5f713f05
proper implementation of executemany(); drop executescript()
2020-03-20 21:15:23 +10:00
Damien Elmes
3f3a45e2ba
add Collection struct, and get media check working again
...
- media check no longer needs collection to be closed
- use savepoints for operations initiated by Rust, so they are
atomic without forcing a commit
2020-03-20 21:15:23 +10:00
Damien Elmes
700d713f80
add begin/commit/rollback, and support creating collections
...
all but one unit test is now passing
2020-03-20 21:15:23 +10:00
Damien Elmes
0147d9e470
drop log= argument from Collection
2020-03-20 21:15:23 +10:00
Damien Elmes
70c7cde2cd
remove remaining db kwargs
2020-03-20 21:15:23 +10:00
Damien Elmes
d42df6e0f1
add a temporary executemany()
2020-03-20 21:15:23 +10:00
Damien Elmes
bd7ded18c3
drop the protobuf prototype
2020-03-20 21:15:23 +10:00
Damien Elmes
38d4ea5311
use (or)json for DB bridge
...
Some initial testing with orjson indicates performance varies from
slightly better than pysqlite to about 2x slower depending on the type
of query.
Performance could be improved by building the Python list in rspy
instead of sending back json that needs to be decoded, but it may make
more sense to rewrite the hotspots in Rust instead. More testing is
required in any case.
2020-03-20 21:15:23 +10:00
Damien Elmes
a129b8f8cf
initial work on DB handling in Rust
...
committing the Protobuf implementation for posterity, but will replace
it with json, as Protobuf measures about 6x slower for some workloads
like 'select * from notes'
2020-03-20 21:15:23 +10:00
Damien Elmes
1db876ba1c
tweak db type hints
2020-03-20 21:15:23 +10:00
Damien Elmes
7fbc229dcd
drop usage of pysqlite Cursor
2020-03-20 21:15:23 +10:00
Damien Elmes
e5a465cccb
tweak layout of db methods
2020-03-20 21:15:23 +10:00
Damien Elmes
7bf27febe6
drop .cursor()
2020-03-20 21:15:23 +10:00
Damien Elmes
fb9db72dde
use total_changes() sqlite func
2020-03-20 21:15:23 +10:00
Damien Elmes
45bc1cd73e
drop the schema <11 upgrade code
2020-03-20 21:15:23 +10:00
Damien Elmes
fcfe1bd66c
drop context manager
2020-03-20 21:15:23 +10:00
Damien Elmes
9e67ea736a
drop named sql arguments
2020-03-20 21:15:23 +10:00
Damien Elmes
6d0c84a1e9
drop progress handler and timeout arg
2020-03-20 21:15:23 +10:00
Damien Elmes
45399df30d
drop echo and text factory
2020-03-20 21:15:23 +10:00
Damien Elmes
923d85f66b
clone db.py into dbproxy.py
2020-03-20 21:15:23 +10:00
evandrocoan
5d75a0c9f1
Fixed moved comment line on qt/aqt/main.py, long variables names on
...
pylib/Makefile and qt/Makefile, and rg --version not going to /dev/null
on .github/scripts/trailing-newlines.sh
2020-03-18 11:36:50 -03:00
evandrocoan
51477a4033
Fixed mypy and pylint not building on Windows
...
1. error: unused 'type: ignore' comment
2. Module 'pywintypes' has no 'error' member
3. Module 'win32pipe' has no 'SetNamedPipeHandleState' member,
but source is unavailable. Consider adding this module to
extension-pkg-whitelist if you want to perform analysis based on
run-time introspection of living objects.
2020-03-17 19:48:16 -03:00
evandrocoan
105e2a2502
Fixed make check for pylint not working on Windows due
...
F:\Python\python.exe: can't open file '/cygdrive/f/anki/pyenv/Scripts/pylint': [Errno 2] No such file or directory
2020-03-17 19:48:16 -03:00
evandrocoan
7d976249db
Fixed test_importing.py unit tests failing on windows with
...
PermissionError: [Errno 13] Permission denied
2020-03-17 12:49:36 -03:00
evandrocoan
f5b02423c1
Fixed UnicodeEncodeError: 'charmap' codec can't encode character
...
'\u2068' in position 2724: character maps to <undefined> for
pylib/tests/test_stats.py
2020-03-17 12:49:25 -03:00
evandrocoan
d6fecea4ff
Fixed BeautifulSoup breaking string paths
...
https://anki.tenderapp.com/discussions/ankidesktop/39543-anki-is-replacing-the-character-by-when-i-exit-the-html-edit-mode-ctrlshiftx
2020-03-12 21:19:09 -03:00
Damien Elmes
a5021ee92e
Merge pull request #501 from evandroforks/fix_beautiful_soup
...
Renamed BeautifulSoup import name for consistency with other usages of it
2020-03-11 15:55:15 +10:00
Damien Elmes
9a9265f113
don't error when an add-on passes None to CardStats
...
https://anki.tenderapp.com/discussions/beta-testing/1842-syncing-message
2020-03-11 15:18:28 +10:00
Damien Elmes
a216fbedbc
Spanish is 'es' on disk
2020-03-11 14:53:19 +10:00
Damien Elmes
ea094a60da
Revert "es-ES -> es"
...
This reverts commit 7409deae34 .
This change broke Anki:
https://anki.tenderapp.com/discussions/ankidesktop/39589-lang_to_disk_lang-startup-error
2020-03-11 14:52:35 +10:00
evandrocoan
52599f364e
Renamed _decode_htmlescapes s variable
2020-03-10 13:57:09 -03:00
Damien Elmes
404a8ca3ec
add switch in prefs for new timezone handling code; bump sync version
...
AD devs: before updating the sync version to 10, please make sure
the new timezone code has been ported over. The core change to the
scheduler is in _updateCutoff():
b198ba8073/pylib/anki/schedv2.py (L1357)
with the following supporting Python functions:
b198ba8073/pylib/anki/schedv2.py (L1410)
_timing_today() is calculated in Rust (lines up to 92, the rest are
tests):
b198ba8073/rslib/src/sched/cutoff.rs (L20)
The change went through a few iterations before stabilising, so it's
probably easier to refer to the above code than the patches that got
us to that point.
2020-03-10 17:50:18 +10:00
Damien Elmes
e512199ace
add restore media action
2020-03-10 13:35:09 +10:00
Damien Elmes
cac9ccfa0d
add empty trash action in aqt
2020-03-10 12:49:40 +10:00
Damien Elmes
0dc4886558
add col.weakref() and fix more reference cycles in pylib
2020-03-05 09:47:47 +10:00
Damien Elmes
0589573dfa
fix retain cycle caused by col.tr
...
We can't use a weakref.proxy with a method, so manually proxy the
call instead.
2020-03-05 09:45:51 +10:00
Arthur Milchior
d9eb170962
hooks for limit new/rev for a single deck
2020-03-04 16:11:13 +01:00
Arthur Milchior
bf0e5f0d6e
single return in sched2 _deckRevLimitSingle
2020-03-04 16:07:15 +01:00
Damien Elmes
2343ddcb02
add diffing script
2020-03-04 21:34:18 +10:00
Damien Elmes
950d39571e
remove a few more common methods
2020-03-04 21:33:44 +10:00
Damien Elmes
27438d8625
use _deckLimit() in a few more places
2020-03-04 21:24:09 +10:00
Damien Elmes
db00452ded
reuse _leftToday()
2020-03-04 12:26:57 +10:00
Damien Elmes
7a32a0eb6f
import tidyup
2020-03-04 12:25:02 +10:00
Damien Elmes
d7c3c5d3b1
reuse _deckNewLimitSingle()
2020-03-04 12:23:56 +10:00
Damien Elmes
7bd1c59abd
reuse v2's _logRev()
2020-03-04 12:23:42 +10:00
Damien Elmes
7880227b43
remove duplicate methods from sched.py
2020-03-04 12:14:26 +10:00
Damien Elmes
966cfaec84
add some type hints that were only in the v1 scheduler
2020-03-04 12:10:38 +10:00
Damien Elmes
7ab3c4ce67
make v1 scheduler inherit from v2 so we can delete duplicate code
2020-03-04 11:41:23 +10:00
Damien Elmes
28e1e01068
use weakref for backrefs so collection doesn't need to be garbage collected
2020-03-03 21:25:32 +10:00
Damien Elmes
475f8e7f04
fix backend not deallocating due to circular reference
2020-03-03 21:12:38 +10:00
Damien Elmes
e592ac06ea
Merge pull request #478 from Arthur-Milchior/missing_constant
...
countsIdx return a queue type.
2020-03-02 15:43:31 +10:00
Alan Du
63a0b36d91
Monkeytype qt/aqt/reviewer.py
2020-03-01 10:16:08 -05:00
Arthur Milchior
80e5a97e39
countsIdx return a queue type.
...
I missed this constant before
2020-03-01 13:00:36 +01:00
Damien Elmes
ba97457e8a
add nfd tag fix to db check
...
we already normalize in the editor and importing, so perhaps these
tags were from an old version
https://anki.tenderapp.com/discussions/ankidesktop/39120-bug-filter-do-not-work-with-tags-which-contain-german-letters-like
2020-03-01 11:21:25 +10:00
Damien Elmes
19ef578556
Merge pull request #473 from evandroforks/fix_docs_urllink
...
Fixed anki website docs link on pylib/anki/consts.py
2020-02-29 21:39:11 +10:00
Damien Elmes
c97b37b7ae
make logging explicit, and support logging to a file
2020-02-29 20:44:01 +10:00
evandrocoan
bc5a9da396
Fixed anki website docs link on pylib/anki/consts.py
2020-02-28 22:18:19 -03:00
Alan Du
13a258184e
Add some more types
2020-02-27 21:27:59 -05:00
Alan Du
ab96ac324e
Monkeytype pylib/anki/importing/supermemo_xml.py
2020-02-27 19:27:32 -05:00
Alan Du
e132ef35ee
Type pylib/anki/schedv2.py
2020-02-27 19:27:32 -05:00
Alan Du
793877ce77
Monkeytype anki/rsbackend.py
2020-02-27 19:27:32 -05:00
Alan Du
3c2822ba4c
Monkeytype pylib/anki/sched.py
2020-02-27 19:27:32 -05:00
Alan Du
397afaf29b
Monkeytype pylib/anki/find.py
2020-02-27 19:27:32 -05:00
Damien Elmes
2de865d7f0
migrate strings with trailing newlines to fluent
2020-02-27 20:36:40 +10:00
Alan Du
9a5eb6c704
Fix type warning
2020-02-27 00:07:19 -05:00
Alan Du
1523412a37
Monkeytype pylib/anki/latex.py
2020-02-27 00:07:19 -05:00
Alan Du
6076e0b21a
Monkeytype pylib/anki/stdmodels.py
2020-02-27 00:07:19 -05:00
Alan Du
c74ccbe245
Monkeytype pylib/anki/stats.py
2020-02-27 00:07:19 -05:00
Alan Du
fe2d8cdea0
Monkeytype pylib/anki/template.py
2020-02-27 00:07:19 -05:00
Alan Du
4606447e4c
Monkeytype anki/tags.py
2020-02-27 00:07:19 -05:00
Alan Du
7ce81f80db
Monkeytype pylib/anki/errors.py
2020-02-27 00:07:19 -05:00
Alan Du
acf1b00cd2
Monkeytype pylib/anki/db.py
2020-02-27 00:07:19 -05:00
Alan Du
a96f1abb33
Monkeytype pylib/anki/decks.py
2020-02-27 00:07:19 -05:00
evandrocoan
59d45f309a
Merge remote-tracking branch 'danielelmes/master' into fix_windows_build
...
# Conflicts:
# CONTRIBUTORS
2020-02-27 00:54:16 -03:00
Alan Du
a095c44f40
Improve mypy config
...
- Make mypy stricter
- Warn for unnecessary workarounds
2020-02-26 22:22:40 -05:00
evandrocoan
4a2456f62f
Merge remote-tracking branch 'danielelmes/master' into fix_windows_build
...
# Conflicts:
# Makefile
2020-02-27 00:20:34 -03:00
Damien Elmes
5d09981667
add self.col.tr() shortcut
2020-02-27 12:33:15 +10:00
Damien Elmes
93c2e04267
FString -> TR
2020-02-27 12:25:19 +10:00
Damien Elmes
83d894ac87
return a localized error for all error kinds
...
some errors are not yet localized, but now the Python code
doesn't need to think about which property to use
2020-02-27 10:46:24 +10:00
Damien Elmes
59328d9ade
tweak rounding
...
- avoid rounding minutes
- round the seconds taken in the card info screen
- provide different precise and imprecise modes, since we need
to display seconds to multiple decimals in some areas
2020-02-25 17:29:06 +10:00
evandrocoan
2d316748a7
Merge remote-tracking branch 'danielelmes/master' into fix_windows_build
...
# Conflicts:
# Makefile
2020-02-25 03:16:57 -03:00
evandrocoan
9e690f80e7
Renamed FIND_EXEC to FIND, replaced IS_WINDOWS by specific commands
...
as PYTHON_BIN, ACTIVE_SCRIPT and INSTALL_PYAUDIO.
Fixed echo statements not using @ to suppress double message output.
Deprecated the usage of ECHOCMD := /bin/echo -e because it has no
effect: https://stackoverflow.com/questions/60387684/how-to-make-the-makefile-echos-to-use-bin-echo-e
# Conflicts:
# Makefile
2020-02-25 03:15:11 -03:00