Commit Graph

266 Commits

Author SHA1 Message Date
Trenton Holmes
b7577038a0 Replace usages of os.rename with shutil.move to properly handle cases where the source and dest arent't on the same filesystem 2023-05-01 07:28:52 -07:00
Trenton H
6f163111ce Upgrades black to v23, upgrades ruff 2023-04-26 09:35:27 -07:00
Trenton H
3bcbd05252 Fixes ruff not running isort against the codebase 2023-04-26 09:35:27 -07:00
Trenton H
a2d4d16867 Make the importer a little more robust against some types of errors 2023-04-26 07:08:50 -07:00
Trenton H
ce41ac9158 Configures ruff as the one stop linter and resolves warnings it raised 2023-04-01 17:03:52 -07:00
Trenton H
3c2bbf244d Creates a data model for the document consumption, allowing stronger typing of arguments and setting of some information about the file only once 2023-04-01 11:05:34 -07:00
Trenton H
4a5f21dd87 Limit the number of threads waiting for files to be ready during polling 2023-03-21 07:46:57 -07:00
shamoon
bf8ae22f3f Rename comments --> notes 2023-03-18 13:59:17 -07:00
Trenton H
c422a081bf Be sure the scratch directory exists before using it as temporary directory 2023-03-01 07:13:31 -08:00
Trenton H
c08f0054da Updates ignore path filtering so files in a folder in an ignored folder will be ignored correctly 2023-02-16 09:05:11 -08:00
Trenton H
06e2500443 Moves the mktime call into the if block where it is used, preventing exceptions during rare cases 2023-02-02 07:25:32 -08:00
Matthieu Helleboid
02a40055f5 replace --use-filename-prefix with --use-folder-prefix 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
aeecc10e45 sort exporter option by alphabetical order 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
2b3edbaa46 use pathlib glob 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
270f8677a7 add document comments to dedicated manifest file when using 'split-manifest' 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
20763e7c26 Fix split_manifest default value 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
b33ba4c902 fix json serialization bug after migration after to Pathlib) 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
fae5e834b9 fix bug on administration exporter when using -d, --delete option 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
4cb4bd13ad add split-manifest option to administration exporter 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
896304ccaa add prefix option to administration exporter 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
9ae186e6f9 add no-archive and no-thumbnail options to administration exporter and importer 2023-01-24 11:06:49 -08:00
Trenton H
b25f083687 Updates the exporter to use pathlib and add a few more tests for coverage 2023-01-14 06:33:12 -08:00
Trenton Holmes
a6b7beaf6b Adds option to allow a user to export directory to a zipfile 2022-12-04 16:38:25 -08:00
Michael Shamoon
91a2dedfec Celery JSON serializable object for override_tag_ids 2022-11-07 07:41:23 -08:00
Trenton Holmes
9247300230 Transitions the backend to celery and celery beat 2022-09-26 11:25:34 -07:00
Michael Shamoon
15d074d39c Include storage path in exporter + tests 2022-09-11 07:39:35 -07:00
Quinn Casey
44e596b0c4 Merge pull request #1451 from paperless-ngx/feature-better-redo-ocr
Feature: Even better re-do of OCR
2022-08-25 17:01:54 -07:00
shamoon
2b1c8c8d9a Merge pull request #1375 from tim-vogel/add_comments
Feature: document comments
2022-08-25 11:48:31 -07:00
shamoon
bb951ad860 Merge pull request #1421 from paperless-ngx/feature-consumer-event-driven
Feature: Event driven consumer
2022-08-25 08:16:47 -07:00
Michael Shamoon
0f4b118b61 Basic verification of Comment export & exporter comment tuple fix
From oprhaned commits b1855a4b7a
53f21574fd

Co-Authored-By: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
2022-08-24 14:24:10 -07:00
Michael Shamoon
d5018af2a3 python code style 2022-08-23 19:20:08 -07:00
tim-vogel
817882ff6f add comment function 2022-08-23 19:19:21 -07:00
Trenton Holmes
ab761e837c Implements a better re-do of OCR by making the document archiver function common. Actually creates updated file now 2022-08-21 18:20:59 -07:00
Trenton Holmes
c8e838e3a0 Adds the storage paths to the re-tagger command 2022-08-21 18:01:19 -07:00
Trenton Holmes
ad1f5ae081 Corrects the timeout's interaction with files waiting to be consumed 2022-08-14 17:47:59 -07:00
Trenton Holmes
86358d5561 Attempts to make production consumer more event driven while still allowing unit testing 2022-08-14 17:47:59 -07:00
Michael Shamoon
05feadbb7a Squashed commit of the following:
commit a4709b1175
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Thu Jul 28 15:36:13 2022 -0700

    Update stale.yml

    [skip ci]

commit 3a031084f3
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Thu Jul 28 15:24:23 2022 -0700

    Update changelog.md

commit 0c517e5351
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Thu Jul 28 15:18:49 2022 -0700

    v1.8.0 version strings

commit 5fe435048b
Merge: 278cedf3 a722bfd0
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Thu Jul 28 15:17:30 2022 -0700

    Merge pull request #1240 from paperless-ngx/beta

    [Beta] Paperless-ngx v1.8.0 Release Candidate 1

commit a722bfd099
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Thu Jul 28 07:46:12 2022 -0700

    New Crowdin updates (#1291)

    * New translations django.po (French)
    [ci skip]

    * New translations messages.xlf (French)
    [ci skip]

    * New translations django.po (French)
    [ci skip]

    * New translations messages.xlf (French)
    [ci skip]

    * New translations messages.xlf (Turkish)
    [ci skip]

    * New translations django.po (Turkish)
    [ci skip]

commit f3d99a5fdb
Merge: ca334770 79de0989
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Jul 26 11:21:42 2022 -0700

    Merge pull request #1277 from paperless-ngx/fix/redo-ocr-button-on-edit

    Fix/feature: add redo ocr button to document edit view

commit 79de0989d5
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Jul 26 09:54:05 2022 -0700

    fix button icon spacing on mobile

commit ca334770b7
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Tue Jul 26 09:45:21 2022 -0700

    New Crowdin updates (#1242)

    * New translations messages.xlf (Turkish)
    [ci skip]

    * New translations messages.xlf (German)
    [ci skip]

    * New translations django.po (German)
    [ci skip]

    * New translations messages.xlf (Italian)
    [ci skip]

    * New translations messages.xlf (Italian)
    [ci skip]

    * New translations messages.xlf (Finnish)
    [ci skip]

    * New translations messages.xlf (Finnish)
    [ci skip]

commit 1071357505
Merge: f32dfe02 ef790ca6
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Jul 26 09:44:42 2022 -0700

    Merge pull request #1268 from paperless-ngx/bugfix-db-locked

    Bugfix: Adds configuration for database timeout, fixing database locked error

commit f32dfe0278
Merge: 611707a3 4e78ca5d
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Mon Jul 25 12:59:31 2022 -0700

    Merge pull request #1261 from paperless-ngx/fix/b1.8.0-ng-select-dropdowns

    Fix: dropdown selected items not visible again

commit 278cedf3d0
Merge: b141671d ecc4553e
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Mon Jul 25 09:25:52 2022 -0700

    Merge pull request #1272 from paperless-ngx/fix-1263

    Documentation: fix occasional code block color legibility

commit 45a6b5a436
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sun Jul 24 20:15:26 2022 -0700

    Add redo OCR button to document edit

commit 611707a3d1
Merge: 2d88638d b4d20d9b
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sun Jul 24 19:59:30 2022 -0700

    Merge pull request #1276 from paperless-ngx/bugfix-webp-import

    Bugfix: Document import doesn't convert thumbnails to WebP

commit b4d20d9b9a
Author: Trenton Holmes <holmes.trenton@gmail.com>
Date:   Sun Jul 24 10:22:53 2022 -0700

    Fixes document import copying PNG files to .webp extensions without actual conversion

commit ecc4553e67
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Fri Jul 22 15:10:33 2022 -0700

    fix occasional code block color legibility

commit ef790ca6f4
Author: Trenton Holmes <holmes.trenton@gmail.com>
Date:   Fri Jul 22 11:08:52 2022 -0700

    Fixes the copy and paste of the log line

commit 2d88638da7
Merge: 0e2e5f34 91ba0bd0
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Fri Jul 22 10:45:53 2022 -0700

    Merge pull request #1269 from paperless-ngx/beta-deps-final

    Chore: Locks dependencies to the final versions for the beta

commit 91ba0bd0af
Author: Trenton Holmes <holmes.trenton@gmail.com>
Date:   Fri Jul 22 08:53:02 2022 -0700

    Locks dependencies to the final versions for the beta

commit 0e2e5f3413
Author: Trenton Holmes <holmes.trenton@gmail.com>
Date:   Tue Jul 19 13:57:00 2022 -0700

    Creates utiliy to ensure all paths in settings are normalized and absolute

commit 7a99dcf693
Author: Trenton Holmes <holmes.trenton@gmail.com>
Date:   Thu Jul 21 08:02:11 2022 -0700

    Adds configuration for database timeout, documentation and troubleshotting suggestion

commit 4e78ca5d82
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Wed Jul 20 11:15:35 2022 -0700

    remove merge error ng-select css

commit 83de38e56f
Merge: f4be2e4f b1b6d50a
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Wed Jul 20 08:26:12 2022 -0700

    Merge pull request #1247 from paperless-ngx/bugfix-pikepdf-ocrmypdf-warnings

    Bugfix: Adds pngquant and jbig2dec to Docker image

commit f4be2e4fe7
Merge: 4444925d 16b0f7f9
Author: Quinn Casey <quinn@quinncasey.com>
Date:   Tue Jul 19 21:03:16 2022 -0700

    Merge pull request #1259 from paperless-ngx/chore-add-ci-hadolint

    Chore: Add Hadolint job to CI

commit 16b0f7f9ee
Author: Trenton Holmes <holmes.trenton@gmail.com>
Date:   Tue Jul 19 14:18:47 2022 -0700

    Removes a Dockerfile I can't find referenced anywhere

commit 27721aef71
Author: Trenton Holmes <holmes.trenton@gmail.com>
Date:   Tue Jul 19 14:01:47 2022 -0700

    Fixes and updates the Hadolint action version

commit 329a317fdf
Author: Trenton Holmes <holmes.trenton@gmail.com>
Date:   Tue Jul 19 13:54:33 2022 -0700

    Configure Hadolint in a single location for both hooks and CI

commit daad634894
Author: Trenton Holmes <holmes.trenton@gmail.com>
Date:   Fri Jul 15 13:45:23 2022 -0700

    Adds a CI job for hadolint over all the Dockerfiles, fixes the minor thing it complained about

commit 4444925dea
Merge: 4c697ab5 9c1ae96d
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Mon Jul 18 15:55:29 2022 -0700

    Merge pull request #1249 from paperless-ngx/fix-generated-changelog

    [CI] Fix automatic changelog generation on release

commit 9c1ae96d33
Author: Quinn Casey <quinn@quinncasey.com>
Date:   Mon Jul 18 09:48:03 2022 -0700

    Create PR for changelog instead of direct commit

commit b1b6d50af6
Author: Trenton Holmes <holmes.trenton@gmail.com>
Date:   Mon Jul 18 09:46:31 2022 -0700

    Adds a couple packages to the Docker image for ocrmypdf and pikepdf

commit 4c697ab50e
Author: Quinn Casey <quinn@quinncasey.com>
Date:   Sun Jul 17 15:23:28 2022 -0700

    Bump version to beta

commit b141671d90
Merge: 48dfbbeb 2ab2d912
Author: Quinn Casey <quinn@quinncasey.com>
Date:   Sun Jul 17 13:18:57 2022 -0700

    Merge pull request #1237 from tooomm/patch-1

    chore: Run stale bot only on certain labels

commit 2ab2d9127d
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Fri Jul 15 20:19:28 2022 -0700

    Use cant-reproduce for stale

commit 278453451e
Author: tooomm <tooomm@users.noreply.github.com>
Date:   Fri Jul 15 21:18:38 2022 +0200

    only run on certain labels

commit 48dfbbebc6
Merge: 8efb97ef e568b300
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sun Jul 3 07:42:20 2022 -0700

    Merge pull request #1110 from paperless-ngx/update-issue-form

commit 8efb97ef4e
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sat Jul 2 19:06:32 2022 -0700

    Update stale.yml

    [ci skip]

commit d8cda7fc1b
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sat Jul 2 17:51:39 2022 -0700

    Use any-of-labels for stalebot

    [ci skip]

commit 68f0cf419b
Merge: 666b9385 26b12512
Author: Felix E <felix@eckhofer.com>
Date:   Mon Jun 20 14:25:59 2022 +0200

    Merge pull request #1148 from pReya/patch-1

    fix: update scanner capability

commit 26b12512b1
Author: Moritz Stückler <moritz.stueckler@gmail.com>
Date:   Mon Jun 20 12:06:54 2022 +0200

    fix: update scanner capability

    The Brother ADS-A1700W does indeed support SFTP. I've just bought it, and set it up like this.

commit e568b3000e
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Jun 7 15:28:49 2022 -0700

    Add lsio to issue form

commit 666b938550
Merge: de5eaf1c 163231d3
Author: Quinn Casey <quinn@quinncasey.com>
Date:   Thu May 19 17:23:23 2022 -0700

    Merge pull request #990 from tooomm/patch-2

    Docs: Fix headings and add links to PRs in changelog

commit 163231d307
Author: tooomm <tooomm@users.noreply.github.com>
Date:   Thu May 19 23:12:40 2022 +0200

    Link issues, capitalization and minor fixes

commit e530750fc6
Author: tooomm <tooomm@users.noreply.github.com>
Date:   Thu May 19 22:05:43 2022 +0200

    update heading levels for v1.7.0
2022-07-28 15:36:24 -07:00
Trenton Holmes
3aca576a0d Includes the actual OSError string in the log, instead of assuming it's a busy file 2022-07-11 11:57:02 -07:00
shamoon
186ae844bc Merge branch 'dev' into feature-redo-ocr 2022-07-02 08:41:18 -07:00
Michael Shamoon
13ffe468df Extract redo ocr to task 2022-07-02 08:39:09 -07:00
Trenton Holmes
a090cf7a10 Updates following testing of command 2022-07-02 08:39:09 -07:00
Trenton Holmes
b7250477b5 Includes the progress bar 2022-07-02 08:39:09 -07:00
Trenton Holmes
dfd16c5187 Saves work on a new management comment to re-ocr a file 2022-07-02 08:39:09 -07:00
Trenton Holmes
22e00a7080 Adds a few more test cases for coverage and adds no coverage to some others 2022-07-02 16:19:22 +02:00
Trenton Holmes
1df517afd3 Removes last vestiges of PNG from the tests, code, docs and samples 2022-06-11 14:20:50 -07:00
Trenton Holmes
cc4cea1a41 Converts the conversion into a database migration 2022-06-11 13:04:21 -07:00
Trenton Holmes
e8868d7ebf Entirely removes the optipng, updates ghostscript fall back to also use WebP. Updates the conversion to use a multiprocessing pool 2022-06-11 08:38:49 -07:00
Trenton Holmes
7d9a9033f9 Fixes the re-generation of thumbnails making webp file, but named as PNG files 2022-06-10 13:17:41 -07:00
Trenton Holmes
34192349be Corrects the functionality of the webp conversion script 2022-06-10 08:56:25 -07:00
Trenton Holmes
20092dadad Adds untested manual conversion command 2022-06-10 07:17:15 -07:00