Commit Graph

1181 Commits

Author SHA1 Message Date
feld 53ef576739 Merge branch 'instance_rules' into 'develop'
Instance rules

See merge request pleroma/pleroma!3669
2024-05-15 20:29:04 +00:00
Mark Felder ede414094f RichMedia refactor
Rich Media parsing was previously handled on-demand with a 2 second HTTP request timeout and retained only in Cachex. Every time a Pleroma instance is restarted it will have to request and parse the data for each status with a URL detected. When fetching a batch of statuses they were processed in parallel to attempt to keep the maximum latency at 2 seconds, but often resulted in a timeline appearing to hang during loading due to a URL that could not be successfully reached. URLs which had images links that expire (Amazon AWS) were parsed and inserted with a TTL to ensure the image link would not break.

Rich Media data is now cached in the database and fetched asynchronously. Cachex is used as a read-through cache. When the data becomes available we stream an update to the clients. If the result is returned quickly the experience is almost seamless. Activities were already processed for their Rich Media data during ingestion to warm the cache, so users should not normally encounter the asynchronous loading of the Rich Media data.

Implementation notes:

- The async worker is a Task with a globally unique process name to prevent duplicate processing of the same URL
- The Task will attempt to fetch the data 3 times with increasing sleep time between attempts
- The HTTP request obeys the default HTTP request timeout value instead of 2 seconds
- URLs that cannot be successfully parsed due to an unexpected error receives a negative cache entry for 15 minutes
- URLs that fail with an expected error will receive a negative cache with no TTL
- Activities that have no detected URLs insert a nil value in the Cachex :scrubber_cache so we do not repeat parsing the object content with Floki every time the activity is rendered
- Expiring image URLs are handled with an Oban job
- There is no automatic cleanup of the Rich Media data in the database, but it is safe to delete at any time
- The post draft/preview feature makes the URL processing synchronous so the rendered post preview will have an accurate rendering

Overall performance of timelines and creating new posts which contain URLs is greatly improved.
2024-05-07 19:54:56 -04:00
marcin mikołajczak 4f5c4d79c4 FEP-2c59, add "webfinger" to user actor
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2024-04-11 17:50:11 +02:00
marcin mikołajczak ccc3ac241f Add hint to rules
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2024-04-06 11:45:19 +02:00
marcin mikołajczak 01a5f839c5 Merge remote-tracking branch 'origin/develop' into instance_rules 2024-04-06 10:42:23 +02:00
marcin mikołajczak 918c406a91 Merge remote-tracking branch 'origin/develop' into instance_rules
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2024-03-18 13:50:25 +01:00
lain 2b71f4897f Merge branch 'develop' into 'bookmark-folders'
# Conflicts:
#   docs/development/API/differences_in_mastoapi_responses.md
2024-03-18 10:35:02 +00:00
marcin mikołajczak d415686bb9 Allow to group bookmarks in folders
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2024-03-01 11:04:01 +01:00
marcin mikołajczak c592a0e58d Merge remote-tracking branch 'origin/develop' into HEAD
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2024-02-22 21:05:20 +01:00
SyoBoN 2c9fed9b73 Translated using Weblate (Japanese)
Currently translated at 63.1% (60 of 95 strings)

Translation: Pleroma/Pleroma Backend (domain errors)
Translate-URL: https://translate.pleroma.social/projects/pleroma/pleroma-backend-domain-errors/ja/
2024-02-16 17:54:01 +00:00
Haelwenn ab3f03a04a Merge branch 'develop' into 'public-polls'
# Conflicts:
#   docs/development/API/differences_in_mastoapi_responses.md
2024-01-21 12:03:29 +00:00
marcin mikołajczak 6051715a99 Merge remote-tracking branch 'origin/develop' into instance_rules
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2023-12-22 14:34:30 +01:00
Mark Felder fb3eb6e0a4 Fix more Logger warn -> warning 2023-12-20 16:24:27 -05:00
tusooa 453cb6a388
Merge remote-tracking branch 'upstream/stable' into mergeback/2.6.1 2023-12-15 19:43:07 -05:00
tusooa 948f01f19a
Bundle 2.6.1 frontend 2023-12-14 20:32:49 -05:00
lain a7f82ff82e Merge branch 'scrubbers-html4-GtS' into 'develop'
scrubbers/default: Add more formatting elements from HTML4 / GoToSocial

See merge request pleroma/pleroma!3979
2023-12-10 12:18:22 +00:00
lain ee15939d33 Skip transaction to generate the index concurrently 2023-12-09 05:54:02 +00:00
Mark Felder 003d3312fd Permit the index creation to run concurrently 2023-12-08 15:48:00 -05:00
lain 35774d44b2 Merge branch 'generate-unset-user-keys-migration' into 'develop'
Fix GenerateUnsetUserKeys migration

See merge request pleroma/pleroma!3976
2023-11-28 12:34:32 +00:00
Haelwenn (lanodan) Monnier 06c58bce00 scrubbers/default: Add more formatting elements from HTML4 / GoToSocial
Added: acronym, bdo, big, cite, dfn, ins, kbd, q, samp, s, tt, var, wbr
2023-11-16 18:27:32 +01:00
marcin mikołajczak fe776d8b33 Fix GenerateUnsetUserKeys migration
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2023-11-13 14:18:22 +01:00
lain 752bc168f6 Merge branch 'quotes-count' into 'develop'
Count and display post quotes

See merge request pleroma/pleroma!3956
2023-11-12 13:38:09 +00:00
marcin mikołajczak 9a063deacc Count and display post quotes 2023-11-12 13:38:08 +00:00
Lain Soykaf 0c5cc51983 Merge branch 'develop' of git.pleroma.social:pleroma/pleroma into pleroma-meilisearch 2023-11-12 13:53:18 +04:00
Mark Felder bf426c53b4 Fix digest email processing, consolidate Oban queues
The email related jobs can all share a single Oban queue
2023-11-07 15:14:36 -05:00
tusooa ab894d98f4
Bundle 2.6.0 frontend 2023-10-29 12:59:03 -04:00
marcin mikołajczak e5bd1ee801 Add entry to @context, tests
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2023-10-29 00:02:17 +02:00
tusooa 163e563733
Allow more flexibility in InlineQuotePolicy 2023-09-13 19:19:05 -04:00
tusooa b0a7e795e7
Unify logic for normalizing quoteUri 2023-09-13 19:19:05 -04:00
Alex Gleason 79fca39faf
Actually, don't send _misskey_quote anymore 2023-09-13 19:19:05 -04:00
Alex Gleason 817e308c0d
Handle Fedibird's new quoteUri field 2023-09-13 19:19:05 -04:00
Alex Gleason 3c8319fe9f
Transmogrifier: federate quotes with _misskey_quote field 2023-09-13 19:19:04 -04:00
Alex Gleason 36a5578d2b
Scrubber.Default: allow span.quote-inline for quote post compatibility 2023-09-13 19:19:04 -04:00
Alex Gleason db46abce47
@context: add quoteUrl 2023-09-13 19:19:04 -04:00
Haelwenn 043a00991d Merge branch 'instance-nodeinfo-metadata' into 'develop'
instances: Store some metadata based on NodeInfo

See merge request pleroma/pleroma!3853
2023-06-27 18:58:04 +00:00
Haelwenn 41f2ee69a8 Merge branch 'from/upstream-develop/tusooa/backup-status' into 'develop'
Detail backup states

Closes #3024

See merge request pleroma/pleroma!3809
2023-06-27 12:08:11 +00:00
weblate-extractor 8bf8906045 Extract translatable strings 2023-06-22 06:09:47 +00:00
Lain Soykaf 22878cf84a B Migrations: Don't remove activity_visibility_index for now. 2023-06-11 16:22:16 +04:00
Lain Soykaf 175ee9e6fe Merge branch 'develop' of git.pleroma.social:pleroma/pleroma into unused_indexes 2023-06-11 16:20:30 +04:00
lain 1f4618d64b Merge branch 'cleanup/ostatus-user-upgrade' into 'develop'
Cleanup OStatus-era user upgrades and ap_enabled indicator

See merge request pleroma/pleroma!3880
2023-06-11 11:13:57 +00:00
Haelwenn (lanodan) Monnier 313e68c180 mix: bump gettext to ~0.20
Includes https://github.com/elixir-gettext/gettext/pull/304 in 0.20.0+
Includes https://github.com/elixir-gettext/expo/issues/91 in 0.22+ via expo 0.2.0+
2023-06-02 03:06:32 +02:00
Haelwenn ce1c0f75cd Merge branch 'tusooa/3065-scopes' into 'develop'
OAuth scopes descriptions

Closes #3065

See merge request pleroma/pleroma!3848
2023-05-17 18:51:26 +00:00
tusooa 163e82bab1
Allow lang attribute 2023-05-09 19:27:32 -04:00
Haelwenn (lanodan) Monnier fcd49e3985 User: Remove ap_enabled field 2023-05-05 11:11:26 +02:00
tusooa 9283c784a3
Add English translation for oauth scopes 2023-05-02 17:03:09 -04:00
tusooa 530284e1b9
Add extracted pot 2023-05-02 16:32:33 -04:00
Mark Felder d91a823836 Remove unused indexes
These indexes were always listed as unused and several grow quite large.
The most significant impact is the activities_visibility_index which takes many hours to rebuild when restoring the server from backup even on fast hardware.
2023-04-22 16:43:33 +00:00
Haelwenn (lanodan) Monnier 9a2523a09a instances: Store some metadata based on NodeInfo 2023-03-16 09:02:20 +01:00
tusooa f33401f54b
Merge remote-tracking branch 'upstream/stable' into mergeback/2.5.1 2023-03-01 20:09:50 -05:00
tusooa 5d34fe1868
Bundle frontend 2023-02-20 12:37:44 -05:00