anthrove/fuzzysearch
4
0
Fork 0
mirror of https://github.com/Syfaro/fuzzysearch.git synced 2024-11-10 08:52:41 +00:00
Code Issues Packages Projects Releases Wiki Activity
124 Commits 2 Branches 0 Tags 4.8 MiB
5773cc03f5
Commit Graph

124 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Syfaro
5773cc03f5
Break apart into multiple microservices (#10) 
* Update CI.

* Bump versions.

* Fix dependencies.

* Unify tracing and metrics export.

* Create service to hash image.

* Updates for hashing service.

* Fix missing file changes.

* Old changes I don't remember.

* Update dependencies, improve Docker images.

* Use BKApi instead of in-memory tree.

* Include health endpoint with metrics.

* Avoid some unwraps.
 2021-08-10 18:28:32 -07:00
Syfaro
097a350724 Update branch in drone file. 2021-04-30 22:34:26 -04:00
Syfaro
c64ca4a6bd Correctly parse Weasyl's explicit rating. 2021-04-30 21:34:31 -04:00
Syfaro
23affc3952 Each webhook should be own job. 2021-04-22 23:59:48 -04:00
Syfaro
f040cf3dde Revert using UNION instead of UNION ALL. 2021-04-22 22:33:18 -04:00
Syfaro
96bc22c65b Update sqlx data. 2021-04-22 21:46:56 -04:00
Syfaro
a1f28707ab Improve query performance, instrumentation. 2021-04-22 21:34:02 -04:00
Syfaro
34d32821cb Use UNION ALL for performance. 2021-04-22 12:54:41 -04:00
Syfaro
ca2247b79f Update opentelemetry. 2021-04-22 12:41:08 -04:00
Syfaro
6a0a7f2e2e Change more logging messages. 2021-04-21 22:54:08 -04:00
Syfaro
36ea876043 Other logging changes. 2021-04-21 21:59:19 -04:00
Syfaro
47703694d1 More logging updates. 2021-04-21 21:46:10 -04:00
Syfaro
32fbfe1d00 Small changes to force build. 2021-04-21 21:21:42 -04:00
Syfaro
e00f99686e Build webhook image, change base images. 2021-04-21 20:53:22 -04:00
Syfaro
3974f85ce8
JSON log output (#9) 
* Allow outputting JSON logs from ingesters.

* Skip instrument for faktory.

* Use more specific timestamp format.

* Update logger for webhooks too.
 2021-04-21 19:26:15 -04:00
Syfaro
dd7805d052
Webhooks (#8) 
* Rough initial progress on webhooks.

* Better webhook error handling, use all ingesters.

* Include hash in webhook data.

* Always pull Rust image.

* Fix missing features.
 2021-04-21 16:58:32 -04:00
Syfaro
a9b5b95350 Update dependencies. 2021-04-19 22:17:21 -04:00
Syfaro
bd1238b34a
Remove hashes table, add Weasyl lookups (#7) 2021-04-19 17:22:40 -04:00
Syfaro
bc278441b6 Remove broken test. 2021-04-12 22:27:22 -04:00
Syfaro
f8744080fd Authenticate e621 ingester. 2021-04-12 22:14:36 -04:00
Syfaro
f99d718603 Install missing dependency in Docker image. 2021-03-20 03:21:40 -04:00
Syfaro
dbfee26a40 Add looking up FA submissions by site ID. 2021-03-20 02:25:01 -04:00
Syfaro
17a4d39a75
Improve FurAffinity ingester (#5) 
* Improve FurAffinity retry logic, add timeout.

* Only build images when pushing to master.

* Use tracing instead of directly printing messages.

* Make case consistent on messages.

* Use tracing instead of panics directly.

* Record users online.

* Extract submission handling to new function.
 2021-02-27 22:26:53 -05:00
Syfaro
0e0b7e0ecb Fix typo in drone.yml file. 2021-02-21 01:59:33 -05:00
Syfaro
5f5a93c85b Update package names and Dockerfiles. 2021-02-21 01:26:29 -05:00
Syfaro
e57d932042 Derive more for types. 2021-02-21 00:38:48 -05:00
Syfaro
08f1b6ad22 Update drone.yml to build images. 2021-02-21 00:30:38 -05:00
Syfaro
f94f6a35f4 Remove panic, monitor video hashing duration. 2021-02-21 00:30:28 -05:00
Syfaro
eded80fd78 Include sample data and images. 2021-02-20 23:40:03 -05:00
Syfaro
ce3793f9a3 Add complete database migrations. 2021-02-20 22:48:03 -05:00
Syfaro
4ad577cee5 Update everything to Tokio v1. 2021-02-20 21:16:58 -05:00
Syfaro
8914227f23 Merge branch 'master' into unify 2021-02-20 18:05:04 -05:00
Syfaro
ed865ba512 Updates. 2021-02-20 17:36:48 -05:00
Syfaro
f592cc624d Add 'fuzzysearch-ingest-e621/' from commit 'd557d9bbaef52b8b86904ceade7ae04b7666b210' 
git-subtree-dir: fuzzysearch-ingest-e621
git-subtree-mainline: e1160f54db
git-subtree-split: d557d9bbae
 2021-02-20 16:46:45 -05:00
Syfaro
6fe6c8ee0d Fixes for content ratings. 2021-02-19 15:47:31 -05:00
Syfaro
274d545734 Include content rating. 2021-02-19 15:38:07 -05:00
Syfaro
c345c51a0f
Add Prometheus for request and process monitoring (#4) 
* Add Prometheus metrics.

* Set default METRICS_HOST and EXPOSE in Dockerfile.
 2021-02-17 17:41:55 -05:00
Syfaro
908cda8ce9
Use NOTIFY/LISTEN instead of polling for updates (#3) 
* Use NOTIFY/LISTEN instead of polling for updates.

* Allow different distances for multiple hashes.
 2021-02-17 16:30:05 -05:00
Syfaro
06a1c7b466 Minor formatting changes. 2021-02-17 12:28:58 -05:00
Syfaro
d557d9bbae Keep database updated. 2021-02-16 23:43:33 -05:00
Syfaro
0453a84511 Updates. 2021-02-16 13:22:29 -05:00
Syfaro
3ade5aeba9
Add ability to search by URL, rate limit headers (#2) 
* Initial progress on searching by URL.

* Avoid rejections for error messages.

* Handle some more rejections.

* Fix build issues.

* Remove detailed error messages.

* Fix issue with built Docker image.

* Add rate limit headers to all responses.

* Remove unneeded dependency.

* Limit URLs to 10MB.
 2021-01-21 21:21:16 -05:00
Syfaro
e1160f54db Update dependencies to allow building on stable. 2020-12-07 18:46:12 -06:00
Syfaro
dae97a074e Update test step. 2020-12-07 18:23:23 -06:00
Syfaro
415d030bd2 Update Weasyl ingester. 2020-12-07 18:17:38 -06:00
Syfaro
1fda51cb5d Add 'fuzzysearch-ingest-weasyl/' from commit '6baf256f09d44c7ac19dd1cdc956fbebae1ffedf' 
git-subtree-dir: fuzzysearch-ingest-weasyl
git-subtree-mainline: 348352f23e
git-subtree-split: 6baf256f09
 2020-12-07 18:07:57 -06:00
Syfaro
348352f23e Some updates for FurAffinity ingester. 2020-12-07 17:57:13 -06:00
Syfaro
3d7add5c1d Remove some unneeded files. 2020-12-07 17:49:34 -06:00
Syfaro
43e5c834a9 Add 'fuzzysearch-ingest-furaffinity/' from commit 'c44b877f70bf478e599110b10e17d152f89c8d61' 
git-subtree-dir: fuzzysearch-ingest-furaffinity
git-subtree-mainline: 59da1e99a8
git-subtree-split: c44b877f70
 2020-12-07 17:45:37 -06:00
Syfaro
59da1e99a8 Start unifying all FuzzySearch crates. 2020-12-07 17:41:32 -06:00