Compare commits

..

252 Commits

Author SHA1 Message Date
Moon Man d499aae5df Merge remote-tracking branch 'upstream/develop' into spc2 2024-05-30 15:59:59 +00:00
feld ff6f5a417f Merge branch 'mrf-nsfw-otp25' into 'develop'
Fix Logger.warn deprecation error on OTP25

See merge request pleroma/pleroma!4135
2024-05-30 15:38:18 +00:00
lain 6feb536e79 Merge branch 'missing-fks' into 'develop'
Add missing foreign key indexes

See merge request pleroma/pleroma!4134
2024-05-30 15:24:24 +00:00
Mark Felder f5065eaf99 Fix Logger.warn deprecation error on OTP25 2024-05-30 11:09:42 -04:00
Mark Felder 5f6e477eca Missing FKs changelog 2024-05-30 10:53:05 -04:00
Mark Felder c20ac6d1ad Add missing foreign key indexes 2024-05-30 10:53:00 -04:00
Mark Felder b5fcb82bff Test for missing FK indexes 2024-05-30 10:49:45 -04:00
lain bc4d6adbec Merge branch 'bandit-update' into 'develop'
Update Bandit to 1.5.2

See merge request pleroma/pleroma!4133
2024-05-30 09:35:42 +00:00
Mark Felder 36b440d9be Update Bandit to 1.5.2
Lots of fixes, also requires Websock Adapter update due to internal module changes in Bandit 1.4.0.
2024-05-29 21:59:50 -04:00
feld b4332b47d5 Merge branch 'mix-indexer' into 'develop'
Add additional flags to the Pleroma.Search.Indexer Mix task

See merge request pleroma/pleroma!4131
2024-05-29 15:04:58 +00:00
Mark Felder 14b4bd69a8 Add additional flags to the Pleroma.Search.Indexer Mix task 2024-05-29 10:44:40 -04:00
feld 3b639b467e Merge branch 'dialyzer-fixes' into 'develop'
Dialyzer fixes

See merge request pleroma/pleroma!4128
2024-05-28 17:26:58 +00:00
lain 41d3c14ba5 Merge branch 'feature/akkoma-prune-old-posts' into 'develop'
add options to mix prune_objects to delete more things

See merge request pleroma/pleroma!3952
2024-05-28 15:19:38 +00:00
Mark Felder 79c418bcb7 Dialyzer: fix invalid @spec 2024-05-28 11:07:28 -04:00
Lain Soykaf f663135724 DatabaseTest: Fix test. 2024-05-28 18:54:36 +04:00
Mark Felder 6b6a2adb07 Dialyzer: The function call will not succeed.
:idna.encode/1 expects a charlist even though it will accept a binary string. That functionality is undocumented / not part of its typespec, so we should turn it into a charlist first. Also switch to using match?/2

lib/pleroma/user.ex:2056:call
The function call will not succeed.

:idna.encode(_host :: binary())

will never return since the success typing is:
(string()) :: string()

and the contract is
(string()) :: string()
2024-05-28 10:49:43 -04:00
Mark Felder 6551ca2db7 Dialyzer: overlapping_contract
Wrong @spec name for remove_from_block/2

lib/pleroma/user.ex:2721:overlapping_contract
Overloaded contract for Pleroma.User.add_to_block/2 has
overlapping domains; such contracts are currently unsupported and
are simply ignored.
2024-05-28 10:40:54 -04:00
Mark Felder 8743c6c640 Dialyzer: The pattern can never match the type
We will never pass :plain to query_with/4, so remove that match and change it to query_with/3

lib/pleroma/search/database_search.ex:127:pattern_match
The pattern can never match the type.

Pattern:
_q, :rum, _search_query, :plain

Type:

  %Ecto.Query{
    :aliases => _,
    :assocs => _,
    :combinations => _,
    :distinct => _,
    :from => _,
    :group_bys => _,
    :havings => _,
    :joins => _,
    :limit => _,
    :lock => _,
    :offset => _,
    :order_bys => _,
    :prefix => _,
    :preloads => _,
    :select => _,
    :sources => _,
    :updates => _,
    :wheres => _,
    :windows => _,
    :with_ctes => _
  },
  :rum,
  _,
  :websearch
2024-05-28 10:36:00 -04:00
Lain Soykaf a041879eaa Linting 2024-05-28 18:26:30 +04:00
Mark Felder 1b3c84e241 Dialyzer: no_local_return
WebPushEncryption.send_web_push/4 was written to raise on erroroneus input, so we must guard against that.

lib/pleroma/web/push/impl.ex:65:no_return Function push_message/4 has no local return.
2024-05-28 10:19:35 -04:00
Mark Felder 17ebb2df84 Dialyzer: fix pattern matches preventing video thumbnailing from working
lib/pleroma/web/media_proxy/media_proxy_controller.ex:154:pattern_match
The pattern can never match the type.

Pattern:
{:ok, _thumbnail_binary}

Type:
{:error, boolean() | {:ffmpeg, :command_not_found}}
2024-05-28 10:19:22 -04:00
Mark Felder 18835bf701 Use the configured http client options for mediaproxy 2024-05-28 09:38:36 -04:00
Mark Felder f8ce639e3f Dialyzer: guard clause can never succeed
lib/pleroma/web/activity_pub/mrf/dnsrbl_policy.ex:106:guard_fail
The guard clause:

when _ ::
  [
    binary()
    | [string() | char()]
    | {string() | integer(), string()}
    | {{byte(), byte(), byte(), byte()}, integer(), binary()}
    | {integer(), integer(), integer(), string() | byte()}
    | {integer(), integer(), string(), string(), string(), string()}
    | {string(), string(), integer(), integer(), integer(), integer(), integer()}
    | {char(), char(), char(), char(), char(), char(), char(), char()}
  ] === nil

can never succeed.
2024-05-28 09:30:19 -04:00
Mark Felder 42c5f7c74e Dialyzer: fix invalid @spec
The callback already defines the @spec and these do not match it.

lib/pleroma/upload/filter/exiftool/strip_location.ex:12:callback_spec_type_mismatch
The @spec return type does not match the expected return type
for filter/1 callback in Pleroma.Upload.Filter behaviour.

Actual:
@spec filter(...) :: {:ok, _}

Expected:
@spec filter(...) :: {:error, _} | {:ok, :filtered | :noop} | {:ok, :filtered, struct()}
2024-05-28 08:55:18 -04:00
Lain Soykaf cc42b50c5b Merge branch 'develop' of git.pleroma.social:pleroma/pleroma into pleroma-feature/akkoma-prune-old-posts 2024-05-28 16:51:19 +04:00
Mark Felder 0b864c3696 Dialyzer: fix invalid @spec
lib/pleroma/notification.ex:492:invalid_contract
The @spec for the function does not match the success typing of the function.

Function:
Pleroma.Notification.get_notified_from_activity/2

Success typing:
@spec get_notified_from_activity(_, _) :: [any()]
2024-05-28 08:49:34 -04:00
lain bef15cde61 Merge branch 'secure-mode' into 'develop'
Reject requests from specified instances if `authorized_fetch_mode` is enabled

See merge request pleroma/pleroma!3711
2024-05-28 11:22:34 +00:00
Lain Soykaf 335691bae1 Add changelog 2024-05-28 14:38:44 +04:00
Lain Soykaf 8066645f71 Linting 2024-05-28 14:20:48 +04:00
Lain Soykaf f5978da676 HTTPSignaturePlugTest: Rewrite to use mox. 2024-05-28 14:00:25 +04:00
Lain Soykaf 3b4be5daa2 Merge branch 'develop' of git.pleroma.social:pleroma/pleroma into pleroma-secure-mode 2024-05-28 12:31:12 +04:00
lain 25903a4996 Merge branch 'auth-fetch-exception' into 'develop'
HTTPSignaturePlug: Add :authorized_fetch_mode_exceptions

See merge request pleroma/pleroma!4007
2024-05-28 04:42:35 +00:00
lain 8ff0c32903 Merge branch 'httpfixes' into 'develop'
Some HTTP and connection pool improvements

See merge request pleroma/pleroma!4124
2024-05-28 04:38:01 +00:00
Lain Soykaf 73d58c22d4 Linting 2024-05-28 08:09:19 +04:00
feld cdde3afb57 Merge branch 'credo' into 'develop'
Credo

See merge request pleroma/pleroma!4126
2024-05-27 19:21:14 +00:00
Mark Felder bb86a01b9b Credo 2024-05-27 15:20:47 -04:00
Lain Soykaf 687ac4a850 Merge branch 'develop' of git.pleroma.social:pleroma/pleroma into auth-fetch-exception 2024-05-27 23:09:17 +04:00
feld 38db406ce4 Merge branch 'simpler-oban-queues' into 'develop'
Oban queue simplification

See merge request pleroma/pleroma!4123
2024-05-27 19:02:53 +00:00
lain 121791882f Merge branch 'explicitly-allow-unsafe-2' into 'develop'
Explicitly allow unsafe 2

See merge request pleroma/pleroma!4125
2024-05-27 18:43:05 +00:00
lain 3316a7ab70 Merge branch 'qdrant-search-2' into 'develop'
Search: Basic Qdrant/Ollama search

See merge request pleroma/pleroma!4109
2024-05-27 18:41:20 +00:00
Lain Soykaf 81e44ced0c HTTPSecurityPlug: Fix tests 2024-05-27 22:13:20 +04:00
Mark Felder ba511a30b9 RichMedia use of ConcurrentLimiter was removed in the refactor 2024-05-27 14:12:38 -04:00
Mark Felder 6b8c15a4a1 Remove MediaProxyWarmingPolicy config for ConcurrentLimiter as we are not using it 2024-05-27 14:11:42 -04:00
feld 42150d5581 Merge branch 'logger-metadata' into 'develop'
Logger metadata

See merge request pleroma/pleroma!3990
2024-05-27 17:53:33 +00:00
Mark Felder 29eac86dc0 Logger metadata changelog 2024-05-27 13:53:22 -04:00
Mark Felder f63e44b8bc Fix Oban related tests 2024-05-27 13:48:24 -04:00
Mark Felder 0847d9ebaf Oban queue simplification 2024-05-27 13:48:17 -04:00
lain b1fec8594d Merge branch 'tusooa/extract-fix' into 'develop'
OAuth scopes translations: write out which operations are processed

See merge request pleroma/pleroma!3907
2024-05-27 17:41:33 +00:00
Lain Soykaf c67b41415b Changelog: Add changelog entry. 2024-05-27 21:28:46 +04:00
Lain Soykaf fc7ce339ed Cheatsheet: Add allow_unsafe_eval 2024-05-27 21:28:20 +04:00
Lain Soykaf 1c699144d2 HttpSecurityPlug: Don't allow unsafe-eval by default 2024-05-27 21:26:40 +04:00
lain 07b7a8d697 Merge branch 'image-description-summary' into 'develop'
Add support for Honk "summary" + "name"

See merge request pleroma/pleroma!3854
2024-05-27 16:51:07 +00:00
feld 10b7efa98c Merge branch 'anti-mention-spam-mrf' into 'develop'
Anti-mention Spam MRF

See merge request pleroma/pleroma!4072
2024-05-27 16:46:31 +00:00
feld 10713fa913 Merge branch 'feat/mrf-dnsrbl' into 'develop'
Anti-spam using an MRF DNSRBL

See merge request pleroma/pleroma!3278
2024-05-27 16:42:38 +00:00
Mark Felder cab6372d7a Make user age limit configurable
Switch to milliseconds for consistency with other configuration options in codebase
2024-05-27 12:31:29 -04:00
Mark Felder 0d092a3d4f Changelog 2024-05-27 12:26:55 -04:00
Alex Gleason 02d8ce8f0b AntiMentionSpamPolicy: remove followers check 2024-05-27 12:25:09 -04:00
Alex Gleason 64cacc3694 AntiMentionSpamPolicy: fix user age check 2024-05-27 12:25:09 -04:00
Alex Gleason 5e963736ce Add AntiMentionSpamPolicy 2024-05-27 12:25:09 -04:00
Mark Felder 0bddca361d DNSRBL in an MRF 2024-05-27 12:23:36 -04:00
feld 6291bf22bd Merge branch 'prometheus-docs' into 'develop'
Update Prometheus docs

See merge request pleroma/pleroma!4018
2024-05-27 16:20:15 +00:00
Mark Felder 7258ab1aed Changelog 2024-05-27 12:20:00 -04:00
Mark Felder f4693dc671 Update Prometheus/Grafana docs for PromEx 2024-05-27 12:18:51 -04:00
Lain Soykaf 284cd0abe5 Add changelog 2024-05-27 20:04:12 +04:00
Lain Soykaf f4c0a01f09 Merge branch 'develop' of git.pleroma.social:pleroma/pleroma into image-description-summary 2024-05-27 20:03:14 +04:00
Lain Soykaf e4f1325f78 InetHelper: Don't use deprecated function. 2024-05-27 19:44:41 +04:00
lain 7798fdc711 Merge branch 'show-reposted-replies' into 'develop'
Display reposted replies with exclude_replies: true

See merge request pleroma/pleroma!3961
2024-05-27 15:33:50 +00:00
Mark Felder 8b61d4e3e1 Changelogs 2024-05-27 11:28:31 -04:00
Lain Soykaf d3e85da0fd Merge branch 'develop' of git.pleroma.social:pleroma/pleroma into auth-fetch-exception 2024-05-27 19:27:02 +04:00
Mark Felder 37d79b76bb Use the configured http client options for mediaproxy 2024-05-27 11:26:21 -04:00
Mark Felder d272eb62cd Trust the connection pools to enforce the concurrency limitations 2024-05-27 11:25:19 -04:00
Lain Soykaf 03d0c5abfb Merge branch 'develop' of git.pleroma.social:pleroma/pleroma into tusooa/extract-fix 2024-05-27 19:21:20 +04:00
lain e93ae96e13 Merge branch 'nsfw-api-mrf' into 'develop'
NSFW API Policy

See merge request pleroma/pleroma!3471
2024-05-27 15:20:43 +00:00
Mark Felder 6708f154a4 Rework Gun connection pool sizes to make better use of the default 250 connections 2024-05-27 11:18:58 -04:00
Mark Felder a50c657427 Add a dedicated connection pool for Rich Media
Sharing this pool with regular Media is problematic as Rich Media will connect to many different
domains and thrash the pool, but regular Media will have predictable connections to the webservers
hosting media for the fediverse servers you peer with.
2024-05-27 11:17:02 -04:00
lain d11ba9e85b Merge branch 'ipfs_uploader' into 'develop'
feat: simple, but not stupid, uploader for IPFS

See merge request pleroma/pleroma!3654
2024-05-27 15:06:12 +00:00
Lain Soykaf ed93af64e1 Add changelog 2024-05-27 17:50:34 +04:00
Lain Soykaf 4325b1aec3 Merge branch 'develop' of git.pleroma.social:pleroma/pleroma into nsfw-api-mrf 2024-05-27 17:49:31 +04:00
Lain Soykaf 3055c1598b IPFSTest: Fix configuration mocking 2024-05-27 17:22:18 +04:00
Lain Soykaf 825b4122a5 Merge branch 'develop' of git.pleroma.social:pleroma/pleroma into pleroma-ipfs_uploader 2024-05-27 16:23:40 +04:00
lain 6757382abe Merge branch 'reject-replies-to-deleted' into 'develop'
Return a 422 when trying to reply to a deleted status

See merge request pleroma/pleroma!4122
2024-05-27 11:41:46 +00:00
Lain Soykaf f214c2cdac NotificationTest: Remove impossible case. 2024-05-27 15:23:33 +04:00
Lain Soykaf 4d6316b488 Merge branch 'develop' of git.pleroma.social:pleroma/pleroma into reject-replies-to-deleted 2024-05-27 15:19:53 +04:00
Lain Soykaf ddf103eca0 QdrantSearch: Fetch a post in search if possible. 2024-05-27 14:35:08 +04:00
Lain Soykaf f4c04e6b2d QdrantSearch: Add health checks. 2024-05-27 14:21:55 +04:00
Lain Soykaf ec3f3fef77 Fastembed Server: Add health check endpoint 2024-05-27 14:15:04 +04:00
Lain Soykaf 8b76f56050 QdrantSearch: Add healthcheck for qdrant 2024-05-27 14:01:17 +04:00
Lain Soykaf 08e9d995f8 Merge branch 'develop' of git.pleroma.social:pleroma/pleroma into qdrant-search-2 2024-05-27 13:50:22 +04:00
lain 5e43060128 Merge branch 'search-healthcheck' into 'develop'
Search backend healthcheck process

See merge request pleroma/pleroma!4120
2024-05-27 09:46:57 +00:00
Lain Soykaf d35b69d268 Pleroma.Search: Remove wrong (but irrelevant) results 2024-05-27 13:18:02 +04:00
Mark Felder d9b82255b9 Add an HTTP timeout for the healthcheck 2024-05-26 15:23:12 -04:00
Mark Felder d4769b076a Return a 422 when trying to reply to a deleted status 2024-05-26 15:14:48 -04:00
feld c3c804b71f Merge branch 'fix/rich-media-ttl' into 'develop'
Fix rich media parsing some Amazon URLs

See merge request pleroma/pleroma!4121
2024-05-26 18:29:46 +00:00
Mark Felder 03f4b46189 Test that healthchecks behave correctly for the expected HTTP responses 2024-05-26 14:21:24 -04:00
Mark Felder f2b0d5f1d0 Make it easier to read the state for debugging purposes and expose functions for testing 2024-05-26 14:11:41 -04:00
Mark Felder 807782b7f9 Fix rich media parsing some Amazon URLs 2024-05-26 14:02:20 -04:00
Mark Felder 354b700bed Assert that AWS URLs without query parameters do not crash 2024-05-26 14:01:12 -04:00
Mark Felder 3474b42ce3 Drop TTL to 5 seconds 2024-05-25 16:55:29 -04:00
Mark Felder 61a3b79316 Search backend healthcheck process 2024-05-25 16:07:47 -04:00
lain 895eea5c75 Merge branch 'api-docs' into 'develop'
Update pleroma_api.md

See merge request pleroma/pleroma!4119
2024-05-25 07:17:05 +00:00
marcin mikołajczak 618b77071a Update pleroma_api.md
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2024-05-25 09:12:37 +02:00
tusooa 19b2637c51 Merge branch 'bugfix/realpath-over-readlink' into 'develop'
pleroma_ctl: Use realpath(1) instead of readlink(1)

See merge request pleroma/pleroma!4118
2024-05-25 00:54:17 +00:00
lain 134f3bff67 Merge branch 'bump-elixir' into 'develop'
Bump minimum Elixir to 1.13

See merge request pleroma/pleroma!4014
2024-05-24 09:20:06 +00:00
Moon Man 2b18b2c5bd Merge remote-tracking branch 'upstream/qdrant-search-2' into spc2 2024-05-23 14:56:02 +00:00
Lain Soykaf a566ad56e1 QdrantSearch: Fix actor / author restriction 2024-05-23 18:55:16 +04:00
Moon Man 696b916c7c Merge remote-tracking branch 'upstream/qdrant-search-2' into spc2 2024-05-23 14:03:58 +00:00
Lain Soykaf 94e4f21589 QdrantSearch: Deal with actor restrictions 2024-05-23 14:38:30 +04:00
Haelwenn (lanodan) Monnier 818712f99f
pleroma_ctl: Use realpath(1) instead of readlink(1)
From realpath(1) in POSIX 202x Draft 4.1:
> If file does not name a symbolic link, readlink shall write a diagnostic
> message to standard error and exit with non-zero status.

Which also doesn't includes `-f`, in preference of `realpath`.
2024-05-23 00:39:53 +02:00
Moon Man 9d23feeb87 Merge remote-tracking branch 'upstream/develop' into spc2 2024-05-22 18:30:08 +00:00
Lain Soykaf f726e5fbbd Merge branch 'develop' of git.pleroma.social:pleroma/pleroma into qdrant-search-2 2024-05-22 20:07:43 +04:00
lain 05b9805bf9 Merge branch 'mergeback-2.6.3' into 'develop'
Mergeback 2.6.3

Closes #3245

See merge request pleroma/pleroma!4117
2024-05-22 15:48:42 +00:00
Lain Soykaf 3f0e9fd474 Merge branch 'stable' into develop 2024-05-22 19:47:15 +04:00
lain 7566b4a348 Merge branch 'release-2.6.3' into 'stable'
Release 2.6.3

See merge request pleroma/pleroma!4115
2024-05-22 15:17:36 +00:00
Lain Soykaf 53a3176d24 WebFingerControllerTest: Restore host after test. 2024-05-22 19:09:26 +04:00
marcin mikołajczak c42527dc2e changelog
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2024-05-22 19:09:16 +04:00
marcin mikołajczak 45b5e6ecd8 Fix tests
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2024-05-22 19:09:07 +04:00
marcin mikołajczak b245a5c8c2 Fix validate_webfinger when running a different domain for Webfinger
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2024-05-22 18:59:14 +04:00
marcin mikołajczak 50ffbd980e Revert "Webfinger: Allow managing account for subdomain"
This reverts commit 84bb854056.
2024-05-22 18:59:07 +04:00
lain a8e1fc0f6a Merge branch 'webfinger-validation' into 'develop'
Fix validate_webfinger when running a different domain for Webfinger

See merge request pleroma/pleroma!4116
2024-05-22 14:58:48 +00:00
Lain Soykaf 5f1f574f01 WebFingerControllerTest: Restore host after test. 2024-05-22 18:45:34 +04:00
marcin mikołajczak d536d58080 changelog
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2024-05-22 15:54:17 +02:00
marcin mikołajczak 70cabbf6dc Fix tests
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2024-05-22 15:53:16 +02:00
marcin mikołajczak d0b18e338b Fix validate_webfinger when running a different domain for Webfinger
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2024-05-22 15:52:35 +02:00
marcin mikołajczak 1f2f7e044d Revert "Webfinger: Allow managing account for subdomain"
This reverts commit 84bb854056.
2024-05-22 15:52:10 +02:00
Lain Soykaf 7b4e6d4c16 Collect changelog 2024-05-22 17:44:10 +04:00
Lain Soykaf 239c9c3f1c Mix: Update version 2024-05-22 17:40:20 +04:00
Lain Soykaf 20fa400082 Webfinger: Allow managing account for subdomain 2024-05-22 17:38:23 +04:00
Lain Soykaf 2212287b00 Changelog: Adjust changelog type 2024-05-22 17:38:16 +04:00
Lain Soykaf 275fdb26c1 Add changelog 2024-05-22 17:38:08 +04:00
Lain Soykaf eafcb7b4ec Webfinger: Fix test 2024-05-22 17:38:03 +04:00
Alex Gleason 364f6e1620 Prevent webfinger spoofing 2024-05-22 17:37:57 +04:00
Lain Soykaf 29b968ce20 Webfinger: Add test showing wrong webfinger behavior 2024-05-22 17:37:49 +04:00
lain c8e5a1f6b0 Merge branch 'fix-webfinger-spoofing' into 'develop'
Fix webfinger spoofing

See merge request pleroma/pleroma!4114
2024-05-22 12:45:24 +00:00
Lain Soykaf 84bb854056 Webfinger: Allow managing account for subdomain 2024-05-22 15:12:29 +04:00
Lain Soykaf 91c93ce3cd Changelog: Adjust changelog type 2024-05-22 13:14:59 +04:00
Lain Soykaf 4491e8c9a3 Add changelog 2024-05-22 13:01:23 +04:00
Lain Soykaf 206ea92837 Webfinger: Fix test 2024-05-22 12:59:10 +04:00
Alex Gleason b15f8b0642 Prevent webfinger spoofing 2024-05-22 12:57:45 +04:00
Lain Soykaf d1b053f3ba Webfinger: Add test showing wrong webfinger behavior 2024-05-22 12:57:30 +04:00
Moon Man 428d6e34b5 Merge branch 'streams-fixes' into spc2 2024-05-21 23:52:11 +00:00
Moon Man aee44ac25e properly get actor from key id for Streams. 2024-05-21 23:50:35 +00:00
Moon Man 3edba7943b Merge remote-tracking branch 'upstream/develop' into spc2 2024-05-21 23:47:03 +00:00
lain 7fca598268 Merge branch 'status-notification-type' into 'develop'
Add "status" notification type

See merge request pleroma/pleroma!3659
2024-05-21 05:01:45 +00:00
marcin mikołajczak 36fa0debfe Fix `get_notified_from`
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2024-05-20 23:25:50 +02:00
Lain Soykaf c67506ba68 Merge branch 'develop' of git.pleroma.social:pleroma/pleroma into auth-fetch-exception 2024-05-20 18:21:46 +04:00
Lain Soykaf f5c0295247 CI: Specify correct image name. 2024-05-20 13:43:18 +04:00
Lain Soykaf f8411a351d CI: Specify version fully in base image tag 2024-05-20 13:30:31 +04:00
Lain Soykaf 226874c9d6 CI: Add new builders for base images 2024-05-20 13:12:12 +04:00
Lain Soykaf ad26b6d593 Merge branch 'develop' of git.pleroma.social:pleroma/pleroma into bump-elixir 2024-05-20 12:30:15 +04:00
Moon Man a5b041c03b Merge remote-tracking branch 'upstream/qdrant-search-2' into spc2 2024-05-19 14:15:06 +00:00
Moon Man 9bbf4c9238 Merge remote-tracking branch 'upstream/develop' into spc2 2024-05-19 14:14:51 +00:00
lain e8cd6662eb Merge branch 'familiar-followers' into 'develop'
Implement `/api/v1/accounts/familiar_followers`

See merge request pleroma/pleroma!4098
2024-05-19 12:05:55 +00:00
Lain Soykaf 1b4f1db9b2 QdrantSearch: Support pagination. 2024-05-19 14:41:05 +04:00
Lain Soykaf dbaab6f54e Docs: Mention running the Qdrant server 2024-05-19 13:38:31 +04:00
Lain Soykaf 6ec306d068 Docs: Add more information about index memory consumption. 2024-05-19 13:24:24 +04:00
Lain Soykaf 6a3a0cc0f5 Docs: Write docs for the QdrantSearch 2024-05-19 13:20:37 +04:00
Lain Soykaf 23881842ae B FastembedAPI: Add readme 2024-05-19 13:04:27 +04:00
Lain Soykaf 8329ad5214 B FastembedAPI: Add requirements.txt 2024-05-19 12:59:03 +04:00
Lain Soykaf dd48810186 B FastembedAPI: Move to more appropriate folder 2024-05-19 12:47:08 +04:00
Lain Soykaf e142ea400a Docs: Switch docs from Ollama to OpenAI. 2024-05-19 12:42:08 +04:00
Lain Soykaf c139a9f38c B Config: Set default Qdrant embedder to our fastembed-api server 2024-05-19 12:39:54 +04:00
Lain Soykaf b9af017a4c B FastembedServer: Switch to OpenAI api, support changing models 2024-05-19 12:33:49 +04:00
Lain Soykaf 72ec261a69 B QdrantSearch: Switch to OpenAI api 2024-05-19 12:17:46 +04:00
Lain Soykaf cc1321ea2e Merge branch 'develop' of git.pleroma.social:pleroma/pleroma into qdrant-search-2 2024-05-19 11:51:33 +04:00
lain 401aca2548 Merge branch 'mark-read' into 'develop'
PleromaAPI: Simplify marking notifications as read

See merge request pleroma/pleroma!4111
2024-05-19 07:48:32 +00:00
Mark Felder d07d49227f PleromaAPI: marking notifications as read no longer returns notifications 2024-05-18 18:17:35 +00:00
Lain Soykaf 3345ddd2d4 Linting 2024-05-18 15:02:22 +04:00
Lain Soykaf 7923ede8ba Merge branch 'develop' of git.pleroma.social:pleroma/pleroma into qdrant-search-2 2024-05-18 14:45:26 +04:00
Lain Soykaf 39525bcec7 Add qdrant changelog 2024-05-18 14:07:47 +04:00
Lain Soykaf e3933a067f QdrantSearch: Implement post deletion 2024-05-18 14:04:32 +04:00
Lain Soykaf 933117785f QdrantSearch: Add basic test 2024-05-18 13:43:47 +04:00
marcin mikołajczak 2e76ceb5b4 Merge remote-tracking branch 'origin/develop' into status-notification-type
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2024-05-18 11:30:25 +02:00
Lain Soykaf 61e9027131 Add docker compose file for fastembed server 2024-05-18 12:19:42 +04:00
Lain Soykaf 769773a500 Add dockerfile 2024-05-18 12:08:42 +04:00
Lain Soykaf 069ce4448c Add basic fastembed server 2024-05-18 11:55:17 +04:00
Moon Man 4c3ef52a12 Merge remote-tracking branch 'upstream/develop' into spc2 2024-05-16 23:38:53 +00:00
feld 99eab1fa2a Merge branch 'revert-e944b152' into 'develop'
Revert "Merge branch 'strip-object-actor' into 'develop'"

See merge request pleroma/pleroma!4112
2024-05-16 23:34:00 +00:00
feld 9988dc2227 Revert "Merge branch 'strip-object-actor' into 'develop'"
This reverts merge request !4105
2024-05-16 23:33:48 +00:00
feld 7de657ac45 Merge branch 'bad-mrf' into 'develop'
Startup detection for configured MRF modules that are missing or incorrectly defined

See merge request pleroma/pleroma!4110
2024-05-16 20:37:37 +00:00
Mark Felder 7f8a9329e5 Startup detection for configured MRF modules that are missing or incorrectly defined 2024-05-16 16:13:29 -04:00
Mark Felder 43f194c434 Include one scrobble with account 2024-05-16 17:53:19 +00:00
Moon Man 1750164b2b Merge remote-tracking branch 'upstream/develop' into spc2 2024-05-16 17:52:11 +00:00
Lain Soykaf a9be4907c0 SearchBackend: Add drop_index 2024-05-16 10:47:24 +04:00
feld e944b15298 Merge branch 'strip-object-actor' into 'develop'
Strip actor from objects before federating

Closes #3269

See merge request pleroma/pleroma!4105
2024-05-15 20:51:47 +00:00
Mark Felder 2965ed47bd Changelog for stripping actor from objects 2024-05-15 16:40:31 -04:00
feld 53ef576739 Merge branch 'instance_rules' into 'develop'
Instance rules

See merge request pleroma/pleroma!3669
2024-05-15 20:29:04 +00:00
feld 8da103da57 Merge branch 'fix-muted-web-push' into 'develop'
Fix processing of Web Push and streaming notifications

See merge request pleroma/pleroma!4032
2024-05-15 20:12:41 +00:00
Moon Man d83a15e879 Merge remote-tracking branch 'upstream/qdrant-search-2' into spc2 2024-05-14 13:50:45 +00:00
Lain Soykaf 1261c43a7a SearchBackend: Add create_index 2024-05-14 17:19:36 +04:00
Lain Soykaf c50f0f31f4 Docs/Search: Add basic documentation of the qdrant search 2024-05-14 16:56:58 +04:00
Lain Soykaf 1490ff30af QdrantSearch: Add query prefix. 2024-05-14 15:09:38 +04:00
Lain Soykaf bb08a766f4 QdrantSearch: Remove debugging stuff 2024-05-14 14:26:41 +04:00
Lain Soykaf cd7e2138d1 Search: Basic Qdrant/Ollama search 2024-05-14 14:13:37 +04:00
Mark Felder 16c72d0701 Merge branch 'develop' into fix-muted-web-push 2024-05-11 08:06:04 -04:00
faried nawaz fdc3cbb8cb
add documentation for the prune_objects mix task options 2024-05-09 10:43:41 +05:00
faried nawaz 1bf3ae07b6
add options to mix pleroma.database prune_objects to delete more activities 2024-05-09 10:43:34 +05:00
Mark Felder dd03184811 Strip actor from objects before federating 2024-05-07 11:54:45 -04:00
Haelwenn 88412daf11 Apply @lanodan's suggestion
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2024-04-25 12:34:12 +02:00
marcin mikołajczak ccc3ac241f Add hint to rules
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2024-04-06 11:45:19 +02:00
marcin mikołajczak 9e6cf45906 /api/v1/accounts/familiar_followers
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2024-04-06 11:43:56 +02:00
marcin mikołajczak 01a5f839c5 Merge remote-tracking branch 'origin/develop' into instance_rules 2024-04-06 10:42:23 +02:00
Mark Felder c25fda34e7 Skip generating notifications for internal users 2024-03-19 12:11:30 -04:00
Mark Felder 291d531e4c Unify notification push and streaming events for both local and federated activities
This also removes generation of notifications for blocked/filtered/muted users and threads.
2024-03-19 12:11:30 -04:00
marcin mikołajczak 918c406a91 Merge remote-tracking branch 'origin/develop' into instance_rules
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2024-03-18 13:50:25 +01:00
Haelwenn fb4aa9f725 Merge branch 'release/2.6.2' into 'stable'
Security Release 2.6.2

See merge request pleroma/pleroma!4074
2024-02-20 08:43:07 +00:00
Haelwenn (lanodan) Monnier be075a4336
Security release 2.6.2 2024-02-20 09:16:36 +01:00
Haelwenn (lanodan) Monnier ac977bdb1c
StealEmojiPolicy: Sanitize shortcodes
Closes: https://git.pleroma.social/pleroma/pleroma/-/issues/3245
2024-02-20 09:14:02 +01:00
marcin mikołajczak 1ed8ae2d8e Add changelog
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2024-01-31 22:55:58 +01:00
marcin mikołajczak 226e53fdd7 Merge remote-tracking branch 'origin/develop' into status-notification-type
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2024-01-31 22:19:33 +01:00
marcin mikołajczak 90b442727e Update Admin API docs
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2024-01-19 17:53:37 +01:00
Mark Felder ddb9e90c40 Update minimum elixir version found in various docs 2023-12-28 15:59:25 -05:00
marcin mikołajczak f6fee39e42 Add changelog
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2023-12-27 21:24:20 +01:00
marcin mikołajczak 6051715a99 Merge remote-tracking branch 'origin/develop' into instance_rules
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2023-12-22 14:34:30 +01:00
Mark Felder 7e3bbdded5 Elixir 1.13 is the minimum required version 2023-12-20 23:39:12 +00:00
Haelwenn (lanodan) Monnier 086ba59d03 HTTPSignaturePlug: Add :authorized_fetch_mode_exceptions 2023-12-16 19:25:51 +01:00
Haelwenn (lanodan) Monnier f271ea6e43 Move Plugs.RemoteIP.maybe_add_cidr/1 to InetHelper.parse_cidr/1 2023-12-16 18:23:26 +01:00
marcin mikołajczak b6a9d87f16 Display reposted replies with exclude_replies: true
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2023-10-28 00:10:13 +02:00
tusooa 1ab4ab8d38
Extract translatable strings 2023-07-18 18:24:30 -04:00
marcin mikołajczak 9363ef53a3 Add test for 'status' notification type for NotificationView
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2023-05-14 15:02:58 +02:00
Haelwenn (lanodan) Monnier 197647a04e MastoAPI Attachment: Use "summary" for descriptions if present 2023-03-09 11:10:02 +01:00
Haelwenn (lanodan) Monnier 2ae1b802f2 AttachmentValidator: Add support for Honk "summary" + "name"
As used by Honk and supported by Mastodon
2023-03-09 10:26:33 +01:00
marcin mikołajczak 78d1105bff Fix down migration
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2023-02-19 22:02:38 +01:00
marcin mikołajczak 92592c25c2 Merge remote-tracking branch 'pleroma/develop' into status-notification-type
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2023-02-19 22:02:03 +01:00
marcin mikołajczak 6e51845d44 Merge remote-tracking branch 'pleroma/develop' into secure-mode
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2022-12-27 16:41:16 +01:00
marcin mikołajczak c899af1d6a Reject requests from specified instances if `authorized_fetch_mode` is enabled
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2022-08-05 11:06:30 +02:00
marcin mikołajczak 3ed39e3109 Add test
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2022-07-08 21:28:23 +02:00
Claudio Maradonna 21d9091f5e
ipfs: replacing single quotes with double quotes 2022-07-08 10:06:46 +02:00
Claudio Maradonna 5e097eb91d
ipfs: better tests with @ilja suggestions 2022-07-07 06:29:17 +02:00
Claudio Maradonna 254f2ea854
ipfs: remove unused alias
fix analysis job
2022-07-07 06:29:15 +02:00
Claudio Maradonna 98f268e5ec
ipfs: small refactor and more tests 2022-07-07 06:29:15 +02:00
Claudio Maradonna 7c1af86f97
ipfs: refactor final_url generation. add tests for final_url
fix lint
2022-07-07 06:29:13 +02:00
Claudio Maradonna 44659ecd65
ipfs: revert to String.replace for cid placeholder
ipfs: fix lint
2022-07-07 06:29:12 +02:00
Claudio Maradonna 43dfa58ebd
added tests for ipfs uploader. adapted changelog.md accordingly. improved ipfs uploader with external suggestions
fix lint description.exs
2022-07-07 06:29:10 +02:00
Claudio Maradonna fa2a6d5d6b
feat: simple, but not stupid, uploader for IPFS
fix: format fix with credo
2022-07-07 06:29:01 +02:00
marcin mikołajczak 5846e7d5f6 Use Repo.exists?
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2022-06-01 16:03:22 +02:00
marcin mikołajczak 0ecd6ba35e AdminAPI: Allow filtering reports by rule_id
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2022-05-30 13:24:39 +02:00
marcin mikołajczak b354d70e85 Apply, suggestions, use strings for actual Mastodon API compatibility
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2022-05-30 12:30:03 +02:00
marcin mikołajczak 5c383ada8a Correctly order rules by id/creation date
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2022-05-16 14:08:02 +02:00
marcin mikołajczak d26aadb743 Add tests
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2022-05-16 14:06:53 +02:00
marcin mikołajczak 574db5b988 Allow submitting an array of rule_ids to /api/v1/reports
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2022-05-16 14:06:49 +02:00
marcin mikołajczak bbf3bc2228 Add RuleTest
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2022-05-16 14:03:06 +02:00
marcin mikołajczak 384f8bfa78 Instance rules: Use render_many
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2022-05-16 14:02:40 +02:00
marcin mikołajczak 432599311d Add GET /api/v1/instance/rules
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2022-05-16 14:02:36 +02:00
marcin mikołajczak bd52e2aec7 Instance rules
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2022-05-16 14:02:32 +02:00
marcin mikołajczak 9423052e92 Add "status" notification type
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2022-04-25 14:08:31 +02:00
Alex Gleason 3a03d9b65f
Merge remote-tracking branch 'pleroma/develop' into nsfw-api-mrf 2021-06-17 15:38:10 -05:00
Alex Gleason a704d5499c
NsfwApiPolicy: Fall back more generously when functions don't match 2021-06-17 15:32:42 -05:00
Alex Gleason c802c3055e
NsfwApiPolicy: add systemd example file 2021-06-17 15:04:40 -05:00
Alex Gleason b293c14a1b
NsfwApiPolicy: add describe/0 and config_description/0 2021-06-17 14:52:07 -05:00
Alex Gleason 2b3dfbb42f
NsfwApiPolicy: add tests 2021-06-17 14:36:51 -05:00
Alex Gleason f15d419062
NsfwApiPolicy: raise if can't fetch user 2021-06-16 22:35:09 -05:00
Alex Gleason 718e8e1edb
Create NsfwApiPolicy 2021-06-16 22:35:04 -05:00
189 changed files with 5102 additions and 852 deletions

View File

@ -1,8 +1,8 @@
image: git.pleroma.social:5050/pleroma/pleroma/ci-base image: git.pleroma.social:5050/pleroma/pleroma/ci-base:elixir-1.13.4-otp-24
variables: &global_variables variables: &global_variables
# Only used for the release # Only used for the release
ELIXIR_VER: 1.12.3 ELIXIR_VER: 1.13.4
POSTGRES_DB: pleroma_test POSTGRES_DB: pleroma_test
POSTGRES_USER: postgres POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres POSTGRES_PASSWORD: postgres
@ -72,7 +72,7 @@ check-changelog:
tags: tags:
- amd64 - amd64
build-1.12.3: build-1.13.4:
extends: extends:
- .build_changes_policy - .build_changes_policy
- .using-ci-base - .using-ci-base
@ -85,7 +85,7 @@ build-1.15.7-otp-25:
- .build_changes_policy - .build_changes_policy
- .using-ci-base - .using-ci-base
stage: build stage: build
image: git.pleroma.social:5050/pleroma/pleroma/ci-base:elixir-1.15 image: git.pleroma.social:5050/pleroma/pleroma/ci-base:elixir-1.15-otp25
allow_failure: true allow_failure: true
script: script:
- mix compile --force - mix compile --force

View File

@ -1,7 +1,7 @@
ARG ELIXIR_IMG=hexpm/elixir ARG ELIXIR_IMG=hexpm/elixir
ARG ELIXIR_VER=1.12.3 ARG ELIXIR_VER=1.13.4
ARG ERLANG_VER=24.2.1 ARG ERLANG_VER=24.3.4.15
ARG ALPINE_VER=3.17.0 ARG ALPINE_VER=3.17.5
FROM ${ELIXIR_IMG}:${ELIXIR_VER}-erlang-${ERLANG_VER}-alpine-${ALPINE_VER} as build FROM ${ELIXIR_IMG}:${ELIXIR_VER}-erlang-${ERLANG_VER}-alpine-${ALPINE_VER} as build

View File

@ -0,0 +1 @@
HTTP Security: By default, don't allow unsafe-eval. The setting needs to be changed to allow Flash emulation.

0
changelog.d/3907.skip Normal file
View File

View File

@ -0,0 +1 @@
Uploader: Add support for uploading attachments using IPFS

View File

@ -0,0 +1 @@
Add NSFW-detecting MRF

View File

@ -0,0 +1 @@
Add DNSRBL MRF

View File

@ -0,0 +1 @@
Add options to the mix prune_objects task

View File

@ -0,0 +1 @@
Add Anti-mention Spam MRF backported from Rebased

View File

View File

@ -0,0 +1 @@
HTTPSignaturePlug: Add :authorized_fetch_mode_exceptions configuration

View File

@ -0,0 +1 @@
Add an option to reject certain domains when authorized fetch is enabled.

View File

@ -0,0 +1 @@
Update Bandit to 1.5.2

View File

@ -0,0 +1 @@
Elixir 1.13 is the minimum required version.

View File

@ -0,0 +1 @@
Implement `/api/v1/accounts/familiar_followers`

View File

@ -0,0 +1 @@
Fix webfinger spoofing.

View File

@ -0,0 +1 @@
Includes scrobble when available in the account field with statuses

View File

@ -0,0 +1 @@
Add instance rules

View File

@ -0,0 +1 @@
Logger metadata is now attached to some logs to help with troubleshooting and analysis

View File

@ -0,0 +1 @@
The query for marking notifications as read has been simplified

View File

@ -0,0 +1 @@
Ensure MediaProxy HTTP requests obey all the defined connection settings

View File

@ -0,0 +1 @@
Add missing indexes on foreign key relationships

View File

@ -0,0 +1 @@
Startup detection for configured MRF modules that are missing or incorrectly defined

View File

@ -0,0 +1 @@
Permit passing --chunk and --step values to the Pleroma.Search.Indexer Mix task

View File

@ -0,0 +1 @@
noop

View File

@ -0,0 +1 @@
Oban queues have refactored to simplify the queue design

1
changelog.d/pools.change Normal file
View File

@ -0,0 +1 @@
HTTP connection pool adjustments

View File

@ -0,0 +1 @@
Update the documentation for configuring Prometheus metrics.

View File

@ -0,0 +1 @@
PromEx documentation

View File

@ -0,0 +1 @@
Add Qdrant/OpenAI embedding search

View File

@ -0,0 +1 @@
pleroma_ctl: Use realpath(1) instead of readlink(1)

View File

@ -0,0 +1 @@
A 422 error is returned when attempting to reply to a deleted status

View File

@ -0,0 +1 @@
Parsing of RichMedia TTLs for Amazon URLs when query parameters are nil

View File

@ -0,0 +1 @@
Monitoring of search backend health to control the processing of jobs in the search indexing Oban queue

View File

@ -0,0 +1 @@
Display reposted replies with exclude_replies: true

View File

@ -0,0 +1 @@
Add "status" notification type

View File

@ -0,0 +1 @@
Support honk-style attachment summaries as alt-text.

View File

@ -0,0 +1 @@
Video thumbnails were not being generated due to a negative cache lookup logic error

View File

@ -0,0 +1 @@
Web Push notifications are no longer generated for muted/blocked threads and users.

View File

@ -0,0 +1 @@
Fix validate_webfinger when running a different domain for Webfinger

View File

@ -0,0 +1,8 @@
FROM elixir:1.13.4-otp-24
# Single RUN statement, otherwise intermediate images are created
# https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#run
RUN apt-get update &&\
apt-get install -y libmagic-dev cmake libimage-exiftool-perl ffmpeg &&\
mix local.hex --force &&\
mix local.rebar --force

View File

@ -0,0 +1 @@
docker buildx build --platform linux/amd64,linux/arm64 -t git.pleroma.social:5050/pleroma/pleroma/ci-base:elixir-1.13.4-otp-24 --push .

View File

@ -1 +1 @@
docker buildx build --platform linux/amd64 -t git.pleroma.social:5050/pleroma/pleroma/ci-base:elixir-1.15-otp25 --push . docker buildx build --platform linux/amd64,linux/arm64 -t git.pleroma.social:5050/pleroma/pleroma/ci-base:elixir-1.15-otp25 --push .

View File

@ -82,6 +82,10 @@
# region: "us-east-1", # may be required for Amazon AWS # region: "us-east-1", # may be required for Amazon AWS
scheme: "https://" scheme: "https://"
config :pleroma, Pleroma.Uploaders.IPFS,
post_gateway_url: nil,
get_gateway_url: nil
config :pleroma, :emoji, config :pleroma, :emoji,
shortcode_globs: ["/emoji/custom/**/*.png"], shortcode_globs: ["/emoji/custom/**/*.png"],
pack_extensions: [".png", ".gif"], pack_extensions: [".png", ".gif"],
@ -188,6 +192,7 @@
allow_relay: true, allow_relay: true,
public: true, public: true,
quarantined_instances: [], quarantined_instances: [],
rejected_instances: [],
static_dir: "instance/static/", static_dir: "instance/static/",
allowed_post_formats: [ allowed_post_formats: [
"text/plain", "text/plain",
@ -406,15 +411,33 @@
accept: [], accept: [],
reject: [] reject: []
config :pleroma, :mrf_dnsrbl,
nameserver: "127.0.0.1",
port: 53,
zone: "bl.pleroma.com"
# threshold of 7 days # threshold of 7 days
config :pleroma, :mrf_object_age, config :pleroma, :mrf_object_age,
threshold: 604_800, threshold: 604_800,
actions: [:delist, :strip_followers] actions: [:delist, :strip_followers]
config :pleroma, :mrf_nsfw_api,
url: "http://127.0.0.1:5000/",
threshold: 0.7,
mark_sensitive: true,
unlist: false,
reject: false
config :pleroma, :mrf_follow_bot, follower_nickname: nil config :pleroma, :mrf_follow_bot, follower_nickname: nil
config :pleroma, :mrf_inline_quote, template: "<bdi>RT:</bdi> {url}" config :pleroma, :mrf_inline_quote, template: "<bdi>RT:</bdi> {url}"
config :pleroma, :mrf_force_mention,
mention_parent: true,
mention_quoted: true
config :pleroma, :mrf_antimentionspam, user_age_limit: 30_000
config :pleroma, :rich_media, config :pleroma, :rich_media,
enabled: true, enabled: true,
ignore_hosts: [], ignore_hosts: [],
@ -497,7 +520,8 @@
sts: false, sts: false,
sts_max_age: 31_536_000, sts_max_age: 31_536_000,
ct_max_age: 2_592_000, ct_max_age: 2_592_000,
referrer_policy: "same-origin" referrer_policy: "same-origin",
allow_unsafe_eval: false
config :cors_plug, config :cors_plug,
max_age: 86_400, max_age: 86_400,
@ -559,24 +583,14 @@
log: false, log: false,
queues: [ queues: [
activity_expiration: 10, activity_expiration: 10,
token_expiration: 5,
filter_expiration: 1,
backup: 1,
federator_incoming: 5, federator_incoming: 5,
federator_outgoing: 5, federator_outgoing: 5,
ingestion_queue: 50, ingestion_queue: 50,
web_push: 50, web_push: 50,
mailer: 10,
transmogrifier: 20, transmogrifier: 20,
scheduled_activities: 10,
poll_notifications: 10,
background: 5, background: 5,
remote_fetcher: 2, search_indexing: [limit: 10, paused: true],
attachments_cleanup: 1, slow: 1
new_users_digest: 1,
mute_expire: 5,
search_indexing: 10,
rich_media_expiration: 2
], ],
plugins: [Oban.Plugins.Pruner], plugins: [Oban.Plugins.Pruner],
crontab: [ crontab: [
@ -814,22 +828,27 @@
config :pleroma, :pools, config :pleroma, :pools,
federation: [ federation: [
size: 50, size: 75,
max_waiting: 10, max_waiting: 20,
recv_timeout: 10_000 recv_timeout: 10_000
], ],
media: [ media: [
size: 50, size: 75,
max_waiting: 20,
recv_timeout: 15_000
],
rich_media: [
size: 25,
max_waiting: 20, max_waiting: 20,
recv_timeout: 15_000 recv_timeout: 15_000
], ],
upload: [ upload: [
size: 25, size: 25,
max_waiting: 5, max_waiting: 20,
recv_timeout: 15_000 recv_timeout: 15_000
], ],
default: [ default: [
size: 10, size: 50,
max_waiting: 2, max_waiting: 2,
recv_timeout: 5_000 recv_timeout: 5_000
] ]
@ -843,6 +862,10 @@
max_connections: 50, max_connections: 50,
timeout: 150_000 timeout: 150_000
], ],
rich_media: [
max_connections: 50,
timeout: 150_000
],
upload: [ upload: [
max_connections: 25, max_connections: 25,
timeout: 300_000 timeout: 300_000
@ -888,8 +911,6 @@
process_chunk_size: 100 process_chunk_size: 100
config :pleroma, ConcurrentLimiter, [ config :pleroma, ConcurrentLimiter, [
{Pleroma.Web.RichMedia.Helpers, [max_running: 5, max_waiting: 5]},
{Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy, [max_running: 5, max_waiting: 5]},
{Pleroma.Search, [max_running: 30, max_waiting: 50]} {Pleroma.Search, [max_running: 30, max_waiting: 50]}
] ]
@ -911,6 +932,19 @@
config :pleroma, Pleroma.Uploaders.Uploader, timeout: 30_000 config :pleroma, Pleroma.Uploaders.Uploader, timeout: 30_000
config :pleroma, Pleroma.Search.QdrantSearch,
qdrant_url: "http://127.0.0.1:6333/",
qdrant_api_key: "",
openai_url: "http://127.0.0.1:11345",
# The healthcheck url has to be set to nil when used with the real openai
# API, as it doesn't have a healthcheck endpoint.
openai_healthcheck_url: "http://127.0.0.1:11345/health",
openai_model: "snowflake/snowflake-arctic-embed-xs",
openai_api_key: "",
qdrant_index_configuration: %{
vectors: %{size: 384, distance: "Cosine"}
}
# Import environment specific config. This must remain at the bottom # Import environment specific config. This must remain at the bottom
# of this file so it overrides the configuration defined above. # of this file so it overrides the configuration defined above.
import_config "#{Mix.env()}.exs" import_config "#{Mix.env()}.exs"

View File

@ -136,6 +136,31 @@
} }
] ]
}, },
%{
group: :pleroma,
key: Pleroma.Uploaders.IPFS,
type: :group,
description: "IPFS uploader-related settings",
children: [
%{
key: :get_gateway_url,
type: :string,
description: "GET Gateway URL",
suggestions: [
"https://ipfs.mydomain.com/{CID}",
"https://{CID}.ipfs.mydomain.com/"
]
},
%{
key: :post_gateway_url,
type: :string,
description: "POST Gateway URL",
suggestions: [
"http://localhost:5001/"
]
}
]
},
%{ %{
group: :pleroma, group: :pleroma,
key: Pleroma.Uploaders.S3, key: Pleroma.Uploaders.S3,
@ -749,6 +774,18 @@
{"*.quarantined.com", "Reason"} {"*.quarantined.com", "Reason"}
] ]
}, },
%{
key: :rejected_instances,
type: {:list, :tuple},
key_placeholder: "instance",
value_placeholder: "reason",
description:
"List of ActivityPub instances to reject requests from if authorized_fetch_mode is enabled",
suggestions: [
{"rejected.com", "Reason"},
{"*.rejected.com", "Reason"}
]
},
%{ %{
key: :static_dir, key: :static_dir,
type: :string, type: :string,
@ -1791,6 +1828,12 @@
type: :boolean, type: :boolean,
description: "Require HTTP signatures for AP fetches" description: "Require HTTP signatures for AP fetches"
}, },
%{
key: :authorized_fetch_mode_exceptions,
type: {:list, :string},
description:
"List of IPs (CIDR format accepted) to exempt from HTTP Signatures requirement (for example to allow debugging, you shouldn't otherwise need this)"
},
%{ %{
key: :note_replies_output_limit, key: :note_replies_output_limit,
type: :integer, type: :integer,

View File

@ -153,6 +153,12 @@
config :pleroma, Pleroma.Upload, config_impl: Pleroma.UnstubbedConfigMock config :pleroma, Pleroma.Upload, config_impl: Pleroma.UnstubbedConfigMock
config :pleroma, Pleroma.ScheduledActivity, config_impl: Pleroma.UnstubbedConfigMock config :pleroma, Pleroma.ScheduledActivity, config_impl: Pleroma.UnstubbedConfigMock
config :pleroma, Pleroma.Web.RichMedia.Helpers, config_impl: Pleroma.StaticStubbedConfigMock config :pleroma, Pleroma.Web.RichMedia.Helpers, config_impl: Pleroma.StaticStubbedConfigMock
config :pleroma, Pleroma.Uploaders.IPFS, config_impl: Pleroma.UnstubbedConfigMock
config :pleroma, Pleroma.Web.Plugs.HTTPSecurityPlug, config_impl: Pleroma.StaticStubbedConfigMock
config :pleroma, Pleroma.Web.Plugs.HTTPSignaturePlug, config_impl: Pleroma.StaticStubbedConfigMock
config :pleroma, Pleroma.Web.Plugs.HTTPSignaturePlug,
http_signatures_impl: Pleroma.StubbedHTTPSignaturesMock
peer_module = peer_module =
if String.to_integer(System.otp_release()) >= 25 do if String.to_integer(System.otp_release()) >= 25 do

View File

@ -21,16 +21,18 @@ Replaces embedded objects with references to them in the `objects` table. Only n
mix pleroma.database remove_embedded_objects [option ...] mix pleroma.database remove_embedded_objects [option ...]
``` ```
### Options ### Options
- `--vacuum` - run `VACUUM FULL` after the embedded objects are replaced with their references - `--vacuum` - run `VACUUM FULL` after the embedded objects are replaced with their references
## Prune old remote posts from the database ## Prune old remote posts from the database
This will prune remote posts older than 90 days (configurable with [`config :pleroma, :instance, remote_post_retention_days`](../../configuration/cheatsheet.md#instance)) from the database, they will be refetched from source when accessed. This will prune remote posts older than 90 days (configurable with [`config :pleroma, :instance, remote_post_retention_days`](../../configuration/cheatsheet.md#instance)) from the database. Pruned posts may be refetched in some cases.
!!! note
The disk space will only be reclaimed after a proper vacuum. By default Postgresql does this for you on a regular basis, but if your instance has been running for a long time and there are many rows deleted, it may be advantageous to use `VACUUM FULL` (e.g. by using the `--vacuum` option).
!!! danger !!! danger
The disk space will only be reclaimed after `VACUUM FULL`. You may run out of disk space during the execution of the task or vacuuming if you don't have about 1/3rds of the database size free. You may run out of disk space during the execution of the task or vacuuming if you don't have about 1/3rds of the database size free. Vacuum causes a substantial increase in I/O traffic, and may lead to a degraded experience while it is running.
=== "OTP" === "OTP"
@ -45,7 +47,11 @@ This will prune remote posts older than 90 days (configurable with [`config :ple
``` ```
### Options ### Options
- `--vacuum` - run `VACUUM FULL` after the objects are pruned
- `--keep-threads` - Don't prune posts when they are part of a thread where at least one post has seen local interaction (e.g. one of the posts is a local post, or is favourited by a local user, or has been repeated by a local user...). It also won't delete posts when at least one of the posts in that thread is kept (e.g. because one of the posts has seen recent activity).
- `--keep-non-public` - Keep non-public posts like DM's and followers-only, even if they are remote.
- `--prune-orphaned-activities` - Also prune orphaned activities afterwards. Activities are things like Like, Create, Announce, Flag (aka reports). They can significantly help reduce the database size. Note: this can take a very long time.
- `--vacuum` - Run `VACUUM FULL` after the objects are pruned. This should not be used on a regular basis, but is useful if your instance has been running for a long time before pruning.
## Create a conversation for all existing DMs ## Create a conversation for all existing DMs
@ -93,6 +99,9 @@ Can be safely re-run
## Vacuum the database ## Vacuum the database
!!! note
By default Postgresql has an autovacuum deamon running. While the tasks described here can help in some cases, they shouldn't be needed on a regular basis. See [the Postgresql docs on vacuuming](https://www.postgresql.org/docs/current/sql-vacuum.html) for more information on this.
### Analyze ### Analyze
Running an `analyze` vacuum job can improve performance by updating statistics used by the query planner. **It is safe to cancel this.** Running an `analyze` vacuum job can improve performance by updating statistics used by the query planner. **It is safe to cancel this.**

View File

@ -41,6 +41,7 @@ To add configuration to your config file, you can copy it from the base config.
* `allow_relay`: Permits remote instances to subscribe to all public posts of your instance. This may increase the visibility of your instance. * `allow_relay`: Permits remote instances to subscribe to all public posts of your instance. This may increase the visibility of your instance.
* `public`: Makes the client API in authenticated mode-only except for user-profiles. Useful for disabling the Local Timeline and The Whole Known Network. Note that there is a dependent setting restricting or allowing unauthenticated access to specific resources, see `restrict_unauthenticated` for more details. * `public`: Makes the client API in authenticated mode-only except for user-profiles. Useful for disabling the Local Timeline and The Whole Known Network. Note that there is a dependent setting restricting or allowing unauthenticated access to specific resources, see `restrict_unauthenticated` for more details.
* `quarantined_instances`: ActivityPub instances where private (DMs, followers-only) activities will not be send. * `quarantined_instances`: ActivityPub instances where private (DMs, followers-only) activities will not be send.
* `rejected_instances`: ActivityPub instances to reject requests from if authorized_fetch_mode is enabled.
* `allowed_post_formats`: MIME-type list of formats allowed to be posted (transformed into HTML). * `allowed_post_formats`: MIME-type list of formats allowed to be posted (transformed into HTML).
* `extended_nickname_format`: Set to `true` to use extended local nicknames format (allows underscores/dashes). This will break federation with * `extended_nickname_format`: Set to `true` to use extended local nicknames format (allows underscores/dashes). This will break federation with
older software for theses nicknames. older software for theses nicknames.
@ -284,6 +285,7 @@ Notes:
* `deny_follow_blocked`: Whether to disallow following an account that has blocked the user in question * `deny_follow_blocked`: Whether to disallow following an account that has blocked the user in question
* `sign_object_fetches`: Sign object fetches with HTTP signatures * `sign_object_fetches`: Sign object fetches with HTTP signatures
* `authorized_fetch_mode`: Require HTTP signatures for AP fetches * `authorized_fetch_mode`: Require HTTP signatures for AP fetches
* `authorized_fetch_mode_exceptions`: List of IPs (CIDR format accepted) to exempt from HTTP Signatures requirement (for example to allow debugging, you shouldn't otherwise need this)
## Pleroma.User ## Pleroma.User
@ -472,6 +474,7 @@ This will make Pleroma listen on `127.0.0.1` port `8080` and generate urls start
* ``ct_max_age``: The maximum age for the `Expect-CT` header if sent. * ``ct_max_age``: The maximum age for the `Expect-CT` header if sent.
* ``referrer_policy``: The referrer policy to use, either `"same-origin"` or `"no-referrer"`. * ``referrer_policy``: The referrer policy to use, either `"same-origin"` or `"no-referrer"`.
* ``report_uri``: Adds the specified url to `report-uri` and `report-to` group in CSP header. * ``report_uri``: Adds the specified url to `report-uri` and `report-to` group in CSP header.
* `allow_unsafe_eval`: Adds `wasm-unsafe-eval` to the CSP header. Needed for some non-essential frontend features like Flash emulation.
### Pleroma.Web.Plugs.RemoteIp ### Pleroma.Web.Plugs.RemoteIp
@ -661,6 +664,19 @@ config :ex_aws, :s3,
host: "s3.eu-central-1.amazonaws.com" host: "s3.eu-central-1.amazonaws.com"
``` ```
#### Pleroma.Uploaders.IPFS
* `post_gateway_url`: URL with port of POST Gateway (unauthenticated)
* `get_gateway_url`: URL of public GET Gateway
Example:
```elixir
config :pleroma, Pleroma.Uploaders.IPFS,
post_gateway_url: "http://localhost:5001",
get_gateway_url: "http://{CID}.ipfs.mydomain.com"
```
### Upload filters ### Upload filters
#### Pleroma.Upload.Filter.AnonymizeFilename #### Pleroma.Upload.Filter.AnonymizeFilename

View File

@ -10,6 +10,30 @@ To use built-in search that has no external dependencies, set the search module
While it has no external dependencies, it has problems with performance and relevancy. While it has no external dependencies, it has problems with performance and relevancy.
## QdrantSearch
This uses the vector search engine [Qdrant](https://qdrant.tech) to search the posts in a vector space. This needs a way to generate embeddings and uses the [OpenAI API](https://platform.openai.com/docs/guides/embeddings/what-are-embeddings). This is implemented by several project besides OpenAI itself, including the python-based fastembed-server found in `supplemental/search/fastembed-api`.
The default settings will support a setup where both the fastembed server and Qdrant run on the same system as pleroma. To use it, set the search provider and run the fastembed server, see the README in `supplemental/search/fastembed-api`:
> config :pleroma, Pleroma.Search, module: Pleroma.Search.QdrantSearch
Then, start the Qdrant server, see [here](https://qdrant.tech/documentation/quick-start/) for instructions.
You will also need to create the Qdrant index once by running `mix pleroma.search.indexer create_index`. Running `mix pleroma.search.indexer index` will retroactively index the last 100_000 activities.
### Indexing and model options
To see the available configuration options, check out the QdrantSearch section in `config/config.exs`.
The default indexing option work for the default model (`snowflake-arctic-embed-xs`). To optimize for a low memory footprint, adjust the index configuration as described in the [Qdrant docs](https://qdrant.tech/documentation/guides/optimize/). See also [this blog post](https://qdrant.tech/articles/memory-consumption/) that goes into detail.
Different embedding models will need different vector size settings. You can see a list of the models supported by the fastembed server [here](https://qdrant.github.io/fastembed/examples/Supported_Models), including their vector dimensions. These vector dimensions need to be set in the `qdrant_index_configuration`.
E.g, If you want to use `sentence-transformers/all-MiniLM-L6-v2` as a model, you will not need to adjust things, because it and `snowflake-arctic-embed-xs` are both 384 dimensional models. If you want to use `snowflake/snowflake-arctic-embed-l`, you will need to adjust the `size` parameter in the `qdrant_index_configuration` to 1024, as it has a dimension of 1024.
When using a different model, you will need do drop the index and recreate it (`mix pleroma.search.indexer drop_index` and `mix pleroma.search.indexer create_index`), as the different embeddings are not compatible with each other.
## Meilisearch ## Meilisearch
Note that it's quite a bit more memory hungry than PostgreSQL (around 4-5G for ~1.2 million Note that it's quite a bit more memory hungry than PostgreSQL (around 4-5G for ~1.2 million

View File

@ -1751,3 +1751,53 @@ Note that this differs from the Mastodon API variant: Mastodon API only returns
```json ```json
{} {}
``` ```
## `GET /api/v1/pleroma/admin/rules`
### List rules
- Response: JSON, list of rules
```json
[
{
"id": "1",
"priority": 1,
"text": "There are no rules",
"hint": null
}
]
```
## `POST /api/v1/pleroma/admin/rules`
### Create a rule
- Params:
- `text`: string, required, rule content
- `hint`: string, optional, rule description
- `priority`: integer, optional, rule ordering priority
- Response: JSON, a single rule
## `PATCH /api/v1/pleroma/admin/rules/:id`
### Update a rule
- Params:
- `text`: string, optional, rule content
- `hint`: string, optional, rule description
- `priority`: integer, optional, rule ordering priority
- Response: JSON, a single rule
## `DELETE /api/v1/pleroma/admin/rules/:id`
### Delete a rule
- Response: JSON, empty object
```json
{}
```

View File

@ -295,9 +295,7 @@ See [Admin-API](admin_api.md)
"id": "9umDrYheeY451cQnEe", "id": "9umDrYheeY451cQnEe",
"name": "Read later", "name": "Read later",
"emoji": "🕓", "emoji": "🕓",
"source": { "emoji_url": null
"emoji": "🕓"
}
} }
] ]
``` ```

View File

@ -1,44 +1,47 @@
# Prometheus Metrics # Prometheus / OpenTelemetry Metrics
Pleroma includes support for exporting metrics via the [prometheus_ex](https://github.com/deadtrickster/prometheus.ex) library. Pleroma includes support for exporting metrics via the [prom_ex](https://github.com/akoutmos/prom_ex) library.
The metrics are exposed by a dedicated webserver/port to improve privacy and security.
Config example: Config example:
``` ```
config :prometheus, Pleroma.Web.Endpoint.MetricsExporter, config :pleroma, Pleroma.PromEx,
enabled: true, disabled: false,
auth: {:basic, "myusername", "mypassword"}, manual_metrics_start_delay: :no_delay,
ip_whitelist: ["127.0.0.1"], drop_metrics_groups: [],
path: "/api/pleroma/app_metrics", grafana: [
format: :text host: System.get_env("GRAFANA_HOST", "http://localhost:3000"),
``` auth_token: System.get_env("GRAFANA_TOKEN"),
upload_dashboards_on_start: false,
* `enabled` (Pleroma extension) enables the endpoint folder_name: "BEAM",
* `ip_whitelist` (Pleroma extension) could be used to restrict access only to specified IPs annotate_app_lifecycle: true
* `auth` sets the authentication (`false` for no auth; configurable to HTTP Basic Auth, see [prometheus-plugs](https://github.com/deadtrickster/prometheus-plugs#exporting) documentation) ],
* `format` sets the output format (`:text` or `:protobuf`) metrics_server: [
* `path` sets the path to app metrics page port: 4021,
path: "/metrics",
protocol: :http,
## `/api/pleroma/app_metrics` pool_size: 5,
cowboy_opts: [],
### Exports Prometheus application metrics auth_strategy: :none
],
* Method: `GET` datasource: "Prometheus"
* Authentication: not required by default (see configuration options above)
* Params: none
* Response: text
## Grafana
### Config example
The following is a config example to use with [Grafana](https://grafana.com)
``` ```
- job_name: 'beam'
metrics_path: /api/pleroma/app_metrics PromEx supports the ability to automatically publish dashboards to your Grafana server as well as register Annotations. If you do not wish to configure this capability you must generate the dashboard JSON files and import them directly. You can find the mix commands in the upstream [documentation](https://hexdocs.pm/prom_ex/Mix.Tasks.PromEx.Dashboard.Export.html). You can find the list of modules enabled in Pleroma for which you should generate dashboards for by examining the contents of the `lib/pleroma/prom_ex.ex` module.
scheme: https
## prometheus.yml
The following is a bare minimum config example to use with [Prometheus](https://prometheus.io) or Prometheus-compatible software like [VictoriaMetrics](https://victoriametrics.com).
```
global:
scrape_interval: 15s
scrape_configs:
- job_name: 'pleroma'
scheme: http
static_configs: static_configs:
- targets: ['pleroma.soykaf.com'] - targets: ['pleroma.soykaf.com:4021']
``` ```

View File

@ -14,7 +14,7 @@ Note: This article is potentially outdated because at this time we may not have
- PostgreSQL 11.0以上 (Ubuntu16.04では9.5しか提供されていないので,[](https://www.postgresql.org/download/linux/ubuntu/)こちらから新しいバージョンを入手してください) - PostgreSQL 11.0以上 (Ubuntu16.04では9.5しか提供されていないので,[](https://www.postgresql.org/download/linux/ubuntu/)こちらから新しいバージョンを入手してください)
- `postgresql-contrib` 11.0以上 (同上) - `postgresql-contrib` 11.0以上 (同上)
- Elixir 1.8 以上 ([Debianのリポジトリからインストールしないこと ここからインストールすること!](https://elixir-lang.org/install.html#unix-and-unix-like)。または [asdf](https://github.com/asdf-vm/asdf) をpleromaユーザーでインストールしてください) - Elixir 1.13 以上 ([Debianのリポジトリからインストールしないこと ここからインストールすること!](https://elixir-lang.org/install.html#unix-and-unix-like)。または [asdf](https://github.com/asdf-vm/asdf) をpleromaユーザーでインストールしてください)
- `erlang-dev` - `erlang-dev`
- `erlang-nox` - `erlang-nox`
- `git` - `git`

View File

@ -1,7 +1,7 @@
## Required dependencies ## Required dependencies
* PostgreSQL >=11.0 * PostgreSQL >=11.0
* Elixir >=1.11.0 <1.15 * Elixir >=1.13.0 <1.15
* Erlang OTP >=22.2.0 (supported: <27) * Erlang OTP >=22.2.0 (supported: <27)
* git * git
* file / libmagic * file / libmagic

View File

@ -0,0 +1,15 @@
[Unit]
Description=NSFW API
After=docker.service
Requires=docker.service
[Service]
TimeoutStartSec=0
Restart=always
ExecStartPre=-/usr/bin/docker stop %n
ExecStartPre=-/usr/bin/docker rm %n
ExecStartPre=/usr/bin/docker pull eugencepoi/nsfw_api:latest
ExecStart=/usr/bin/docker run --rm -p 127.0.0.1:5000:5000/tcp --env PORT=5000 --name %n eugencepoi/nsfw_api:latest
[Install]
WantedBy=multi-user.target

View File

@ -67,43 +67,168 @@ def run(["prune_objects" | args]) do
OptionParser.parse( OptionParser.parse(
args, args,
strict: [ strict: [
vacuum: :boolean vacuum: :boolean,
keep_threads: :boolean,
keep_non_public: :boolean,
prune_orphaned_activities: :boolean
] ]
) )
start_pleroma() start_pleroma()
deadline = Pleroma.Config.get([:instance, :remote_post_retention_days]) deadline = Pleroma.Config.get([:instance, :remote_post_retention_days])
time_deadline = NaiveDateTime.utc_now() |> NaiveDateTime.add(-(deadline * 86_400))
Logger.info("Pruning objects older than #{deadline} days") log_message = "Pruning objects older than #{deadline} days"
time_deadline = log_message =
NaiveDateTime.utc_now() if Keyword.get(options, :keep_non_public) do
|> NaiveDateTime.add(-(deadline * 86_400)) log_message <> ", keeping non public posts"
else
log_message
end
from(o in Object, log_message =
where: if Keyword.get(options, :keep_threads) do
fragment( log_message <> ", keeping threads intact"
"?->'to' \\? ? OR ?->'cc' \\? ?", else
o.data, log_message
^Pleroma.Constants.as_public(), end
o.data,
^Pleroma.Constants.as_public() log_message =
), if Keyword.get(options, :prune_orphaned_activities) do
where: o.inserted_at < ^time_deadline, log_message <> ", pruning orphaned activities"
where: else
log_message
end
log_message =
if Keyword.get(options, :vacuum) do
log_message <>
", doing a full vacuum (you shouldn't do this as a recurring maintanance task)"
else
log_message
end
Logger.info(log_message)
if Keyword.get(options, :keep_threads) do
# We want to delete objects from threads where
# 1. the newest post is still old
# 2. none of the activities is local
# 3. none of the activities is bookmarked
# 4. optionally none of the posts is non-public
deletable_context =
if Keyword.get(options, :keep_non_public) do
Pleroma.Activity
|> join(:left, [a], b in Pleroma.Bookmark, on: a.id == b.activity_id)
|> group_by([a], fragment("? ->> 'context'::text", a.data))
|> having(
[a],
not fragment(
# Posts (checked on Create Activity) is non-public
"bool_or((not(?->'to' \\? ? OR ?->'cc' \\? ?)) and ? ->> 'type' = 'Create')",
a.data,
^Pleroma.Constants.as_public(),
a.data,
^Pleroma.Constants.as_public(),
a.data
)
)
else
Pleroma.Activity
|> join(:left, [a], b in Pleroma.Bookmark, on: a.id == b.activity_id)
|> group_by([a], fragment("? ->> 'context'::text", a.data))
end
|> having([a], max(a.updated_at) < ^time_deadline)
|> having([a], not fragment("bool_or(?)", a.local))
|> having([_, b], fragment("max(?::text) is null", b.id))
|> select([a], fragment("? ->> 'context'::text", a.data))
Pleroma.Object
|> where([o], fragment("? ->> 'context'::text", o.data) in subquery(deletable_context))
else
if Keyword.get(options, :keep_non_public) do
Pleroma.Object
|> where(
[o],
fragment(
"?->'to' \\? ? OR ?->'cc' \\? ?",
o.data,
^Pleroma.Constants.as_public(),
o.data,
^Pleroma.Constants.as_public()
)
)
else
Pleroma.Object
end
|> where([o], o.updated_at < ^time_deadline)
|> where(
[o],
fragment("split_part(?->>'actor', '/', 3) != ?", o.data, ^Pleroma.Web.Endpoint.host()) fragment("split_part(?->>'actor', '/', 3) != ?", o.data, ^Pleroma.Web.Endpoint.host())
) )
end
|> Repo.delete_all(timeout: :infinity) |> Repo.delete_all(timeout: :infinity)
prune_hashtags_query = """ if !Keyword.get(options, :keep_threads) do
# Without the --keep-threads option, it's possible that bookmarked
# objects have been deleted. We remove the corresponding bookmarks.
"""
delete from public.bookmarks
where id in (
select b.id from public.bookmarks b
left join public.activities a on b.activity_id = a.id
left join public.objects o on a."data" ->> 'object' = o.data ->> 'id'
where o.id is null
)
"""
|> Repo.query([], timeout: :infinity)
end
if Keyword.get(options, :prune_orphaned_activities) do
# Prune activities who link to a single object
"""
delete from public.activities
where id in (
select a.id from public.activities a
left join public.objects o on a.data ->> 'object' = o.data ->> 'id'
left join public.activities a2 on a.data ->> 'object' = a2.data ->> 'id'
left join public.users u on a.data ->> 'object' = u.ap_id
where not a.local
and jsonb_typeof(a."data" -> 'object') = 'string'
and o.id is null
and a2.id is null
and u.id is null
)
"""
|> Repo.query([], timeout: :infinity)
# Prune activities who link to an array of objects
"""
delete from public.activities
where id in (
select a.id from public.activities a
join json_array_elements_text((a."data" -> 'object')::json) as j on jsonb_typeof(a."data" -> 'object') = 'array'
left join public.objects o on j.value = o.data ->> 'id'
left join public.activities a2 on j.value = a2.data ->> 'id'
left join public.users u on j.value = u.ap_id
group by a.id
having max(o.data ->> 'id') is null
and max(a2.data ->> 'id') is null
and max(u.ap_id) is null
)
"""
|> Repo.query([], timeout: :infinity)
end
"""
DELETE FROM hashtags AS ht DELETE FROM hashtags AS ht
WHERE NOT EXISTS ( WHERE NOT EXISTS (
SELECT 1 FROM hashtags_objects hto SELECT 1 FROM hashtags_objects hto
WHERE ht.id = hto.hashtag_id) WHERE ht.id = hto.hashtag_id)
""" """
|> Repo.query()
Repo.query(prune_hashtags_query)
if Keyword.get(options, :vacuum) do if Keyword.get(options, :vacuum) do
Maintenance.vacuum("full") Maintenance.vacuum("full")

View File

@ -0,0 +1,83 @@
# Pleroma: A lightweight social networking server
# Copyright © 2017-2021 Pleroma Authors <https://pleroma.social/>
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Mix.Tasks.Pleroma.Search.Indexer do
import Mix.Pleroma
import Ecto.Query
alias Pleroma.Workers.SearchIndexingWorker
def run(["create_index"]) do
start_pleroma()
with :ok <- Pleroma.Config.get([Pleroma.Search, :module]).create_index() do
IO.puts("Index created")
else
e -> IO.puts("Could not create index: #{inspect(e)}")
end
end
def run(["drop_index"]) do
start_pleroma()
with :ok <- Pleroma.Config.get([Pleroma.Search, :module]).drop_index() do
IO.puts("Index dropped")
else
e -> IO.puts("Could not drop index: #{inspect(e)}")
end
end
def run(["index" | options]) do
{options, [], []} =
OptionParser.parse(
options,
strict: [
chunk: :integer,
limit: :integer,
step: :integer
]
)
start_pleroma()
chunk_size = Keyword.get(options, :chunk, 100)
limit = Keyword.get(options, :limit, 100_000)
per_step = Keyword.get(options, :step, 1000)
chunks = max(div(limit, per_step), 1)
1..chunks
|> Enum.each(fn step ->
q =
from(a in Pleroma.Activity,
limit: ^per_step,
offset: ^per_step * (^step - 1),
select: [:id],
order_by: [desc: :id]
)
{:ok, ids} =
Pleroma.Repo.transaction(fn ->
Pleroma.Repo.stream(q, timeout: :infinity)
|> Enum.map(fn a ->
a.id
end)
end)
IO.puts("Got #{length(ids)} activities, adding to indexer")
ids
|> Enum.chunk_every(chunk_size)
|> Enum.each(fn chunk ->
IO.puts("Adding #{length(chunk)} activities to indexing queue")
chunk
|> Enum.map(fn id ->
SearchIndexingWorker.new(%{"op" => "add_to_index", "activity" => id})
end)
|> Oban.insert_all()
end)
end)
end
end

View File

@ -14,6 +14,7 @@ defmodule Pleroma.Application do
@name Mix.Project.config()[:name] @name Mix.Project.config()[:name]
@version Mix.Project.config()[:version] @version Mix.Project.config()[:version]
@repository Mix.Project.config()[:source_url] @repository Mix.Project.config()[:source_url]
@compile_env Mix.env()
def name, do: @name def name, do: @name
def version, do: @version def version, do: @version
@ -51,7 +52,11 @@ def start(_type, _args) do
Pleroma.HTML.compile_scrubbers() Pleroma.HTML.compile_scrubbers()
Pleroma.Config.Oban.warn() Pleroma.Config.Oban.warn()
Config.DeprecationWarnings.warn() Config.DeprecationWarnings.warn()
Pleroma.Web.Plugs.HTTPSecurityPlug.warn_if_disabled()
if @compile_env != :test do
Pleroma.Web.Plugs.HTTPSecurityPlug.warn_if_disabled()
end
Pleroma.ApplicationRequirements.verify!() Pleroma.ApplicationRequirements.verify!()
load_custom_modules() load_custom_modules()
Pleroma.Docs.JSON.compile() Pleroma.Docs.JSON.compile()
@ -109,7 +114,8 @@ def start(_type, _args) do
streamer_registry() ++ streamer_registry() ++
background_migrators() ++ background_migrators() ++
shout_child(shout_enabled?()) ++ shout_child(shout_enabled?()) ++
[Pleroma.Gopher.Server] [Pleroma.Gopher.Server] ++
[Pleroma.Search.Healthcheck]
# See http://elixir-lang.org/docs/stable/elixir/Supervisor.html # See http://elixir-lang.org/docs/stable/elixir/Supervisor.html
# for other strategies and supported options # for other strategies and supported options
@ -162,7 +168,8 @@ defp cachex_children do
expiration: chat_message_id_idempotency_key_expiration(), expiration: chat_message_id_idempotency_key_expiration(),
limit: 500_000 limit: 500_000
), ),
build_cachex("rel_me", limit: 2500) build_cachex("rel_me", limit: 2500),
build_cachex("host_meta", default_ttl: :timer.minutes(120), limit: 5000)
] ]
end end

View File

@ -28,6 +28,7 @@ def verify! do
|> check_welcome_message_config!() |> check_welcome_message_config!()
|> check_rum!() |> check_rum!()
|> check_repo_pool_size!() |> check_repo_pool_size!()
|> check_mrfs()
|> handle_result() |> handle_result()
end end
@ -234,4 +235,25 @@ defp check_filter(filter, command_required) do
true true
end end
end end
defp check_mrfs(:ok) do
mrfs = Config.get!([:mrf, :policies])
missing_mrfs =
Enum.reduce(mrfs, [], fn x, acc ->
if Code.ensure_compiled(x) do
acc
else
acc ++ [x]
end
end)
if Enum.empty?(missing_mrfs) do
:ok
else
{:error, "The following MRF modules are configured but missing: #{inspect(missing_mrfs)}"}
end
end
defp check_mrfs(result), do: result
end end

View File

@ -19,7 +19,8 @@ defmodule Pleroma.Constants do
"context_id", "context_id",
"deleted_activity_id", "deleted_activity_id",
"pleroma_internal", "pleroma_internal",
"generator" "generator",
"rules"
] ]
) )

View File

@ -16,4 +16,15 @@ def parse_address(ip) when is_binary(ip) do
def parse_address(ip) do def parse_address(ip) do
:inet.parse_address(ip) :inet.parse_address(ip)
end end
def parse_cidr(proxy) when is_binary(proxy) do
proxy =
cond do
"/" in String.codepoints(proxy) -> proxy
InetCidr.v4?(InetCidr.parse_address!(proxy)) -> proxy <> "/32"
InetCidr.v6?(InetCidr.parse_address!(proxy)) -> proxy <> "/128"
end
InetCidr.parse_cidr!(proxy, true)
end
end end

View File

@ -25,7 +25,7 @@ def missing_dependencies do
end end
def image_resize(url, options) do def image_resize(url, options) do
with {:ok, env} <- HTTP.get(url, [], pool: :media), with {:ok, env} <- HTTP.get(url, [], http_client_opts()),
{:ok, resized} <- {:ok, resized} <-
Operation.thumbnail_buffer(env.body, options.max_width, Operation.thumbnail_buffer(env.body, options.max_width,
height: options.max_height, height: options.max_height,
@ -45,8 +45,8 @@ def image_resize(url, options) do
@spec video_framegrab(String.t()) :: {:ok, binary()} | {:error, any()} @spec video_framegrab(String.t()) :: {:ok, binary()} | {:error, any()}
def video_framegrab(url) do def video_framegrab(url) do
with executable when is_binary(executable) <- System.find_executable("ffmpeg"), with executable when is_binary(executable) <- System.find_executable("ffmpeg"),
false <- @cachex.exists?(:failed_media_helper_cache, url), {:ok, false} <- @cachex.exists?(:failed_media_helper_cache, url),
{:ok, env} <- HTTP.get(url, [], pool: :media), {:ok, env} <- HTTP.get(url, [], http_client_opts()),
{:ok, pid} <- StringIO.open(env.body) do {:ok, pid} <- StringIO.open(env.body) do
body_stream = IO.binstream(pid, 1) body_stream = IO.binstream(pid, 1)
@ -71,17 +71,19 @@ def video_framegrab(url) do
end) end)
case Task.yield(task, 5_000) do case Task.yield(task, 5_000) do
nil -> {:ok, result} ->
{:ok, result}
_ ->
Task.shutdown(task) Task.shutdown(task)
@cachex.put(:failed_media_helper_cache, url, nil) @cachex.put(:failed_media_helper_cache, url, nil)
{:error, {:ffmpeg, :timeout}} {:error, {:ffmpeg, :timeout}}
result ->
{:ok, result}
end end
else else
nil -> {:error, {:ffmpeg, :command_not_found}} nil -> {:error, {:ffmpeg, :command_not_found}}
{:error, _} = error -> error {:error, _} = error -> error
end end
end end
defp http_client_opts, do: Pleroma.Config.get([:media_proxy, :proxy_opts, :http], pool: :media)
end end

View File

@ -0,0 +1,4 @@
defmodule Pleroma.HTTPSignaturesAPI do
@callback validate_conn(conn :: Plug.Conn.t()) :: boolean
@callback signature_for_conn(conn :: Plug.Conn.t()) :: map
end

View File

@ -73,6 +73,7 @@ def unread_notifications_count(%User{id: user_id}) do
pleroma:report pleroma:report
reblog reblog
poll poll
status
} }
def changeset(%Notification{} = notification, attrs) do def changeset(%Notification{} = notification, attrs) do
@ -280,15 +281,10 @@ def set_read_up_to(%{id: user_id} = user, id) do
select: n.id select: n.id
) )
{:ok, %{ids: {_, notification_ids}}} = Multi.new()
Multi.new() |> Multi.update_all(:ids, query, set: [seen: true, updated_at: NaiveDateTime.utc_now()])
|> Multi.update_all(:ids, query, set: [seen: true, updated_at: NaiveDateTime.utc_now()]) |> Marker.multi_set_last_read_id(user, "notifications")
|> Marker.multi_set_last_read_id(user, "notifications") |> Repo.transaction()
|> Repo.transaction()
for_user_query(user)
|> where([n], n.id in ^notification_ids)
|> Repo.all()
end end
@spec read_one(User.t(), String.t()) :: @spec read_one(User.t(), String.t()) ::
@ -299,10 +295,6 @@ def read_one(%User{} = user, notification_id) do
|> Multi.update(:update, changeset(notification, %{seen: true})) |> Multi.update(:update, changeset(notification, %{seen: true}))
|> Marker.multi_set_last_read_id(user, "notifications") |> Marker.multi_set_last_read_id(user, "notifications")
|> Repo.transaction() |> Repo.transaction()
|> case do
{:ok, %{update: notification}} -> {:ok, notification}
{:error, :update, changeset, _} -> {:error, changeset}
end
end end
end end
@ -361,37 +353,38 @@ def dismiss(%{id: user_id} = _user, id) do
end end
end end
@spec create_notifications(Activity.t(), keyword()) :: {:ok, [Notification.t()] | []} @spec create_notifications(Activity.t()) :: {:ok, [Notification.t()] | []}
def create_notifications(activity, options \\ []) def create_notifications(activity)
def create_notifications(%Activity{data: %{"to" => _, "type" => "Create"}} = activity, options) do def create_notifications(%Activity{data: %{"to" => _, "type" => "Create"}} = activity) do
object = Object.normalize(activity, fetch: false) object = Object.normalize(activity, fetch: false)
if object && object.data["type"] == "Answer" do if object && object.data["type"] == "Answer" do
{:ok, []} {:ok, []}
else else
do_create_notifications(activity, options) do_create_notifications(activity)
end end
end end
def create_notifications(%Activity{data: %{"type" => type}} = activity, options) def create_notifications(%Activity{data: %{"type" => type}} = activity)
when type in ["Follow", "Like", "Announce", "Move", "EmojiReact", "Flag", "Update"] do when type in ["Follow", "Like", "Announce", "Move", "EmojiReact", "Flag", "Update"] do
do_create_notifications(activity, options) do_create_notifications(activity)
end end
def create_notifications(_, _), do: {:ok, []} def create_notifications(_), do: {:ok, []}
defp do_create_notifications(%Activity{} = activity, options) do defp do_create_notifications(%Activity{} = activity) do
do_send = Keyword.get(options, :do_send, true) enabled_receivers = get_notified_from_activity(activity)
{enabled_receivers, disabled_receivers} = get_notified_from_activity(activity) enabled_subscribers = get_notified_subscribers_from_activity(activity)
potential_receivers = enabled_receivers ++ disabled_receivers
notifications = notifications =
Enum.map(potential_receivers, fn user -> (Enum.map(enabled_receivers, fn user ->
do_send = do_send && user in enabled_receivers create_notification(activity, user)
create_notification(activity, user, do_send: do_send) end) ++
end) Enum.map(enabled_subscribers -- enabled_receivers, fn user ->
create_notification(activity, user, type: "status")
end))
|> Enum.reject(&is_nil/1) |> Enum.reject(&is_nil/1)
{:ok, notifications} {:ok, notifications}
@ -450,7 +443,6 @@ defp type_from_activity_object(%{data: %{"type" => "Create"}} = activity) do
# TODO move to sql, too. # TODO move to sql, too.
def create_notification(%Activity{} = activity, %User{} = user, opts \\ []) do def create_notification(%Activity{} = activity, %User{} = user, opts \\ []) do
do_send = Keyword.get(opts, :do_send, true)
type = Keyword.get(opts, :type, type_from_activity(activity)) type = Keyword.get(opts, :type, type_from_activity(activity))
unless skip?(activity, user, opts) do unless skip?(activity, user, opts) do
@ -465,11 +457,6 @@ def create_notification(%Activity{} = activity, %User{} = user, opts \\ []) do
|> Marker.multi_set_last_read_id(user, "notifications") |> Marker.multi_set_last_read_id(user, "notifications")
|> Repo.transaction() |> Repo.transaction()
if do_send do
Streamer.stream(["user", "user:notification"], notification)
Push.send(notification)
end
notification notification
end end
end end
@ -502,7 +489,7 @@ def create_poll_notifications(%Activity{} = activity) do
NOTE: might be called for FAKE Activities, see ActivityPub.Utils.get_notified_from_object/1 NOTE: might be called for FAKE Activities, see ActivityPub.Utils.get_notified_from_object/1
""" """
@spec get_notified_from_activity(Activity.t(), boolean()) :: {list(User.t()), list(User.t())} @spec get_notified_from_activity(Activity.t(), boolean()) :: list(User.t())
def get_notified_from_activity(activity, local_only \\ true) def get_notified_from_activity(activity, local_only \\ true)
def get_notified_from_activity(%Activity{data: %{"type" => type}} = activity, local_only) def get_notified_from_activity(%Activity{data: %{"type" => type}} = activity, local_only)
@ -527,13 +514,28 @@ def get_notified_from_activity(%Activity{data: %{"type" => type}} = activity, lo
|> exclude_relationship_restricted_ap_ids(activity) |> exclude_relationship_restricted_ap_ids(activity)
|> exclude_thread_muter_ap_ids(activity) |> exclude_thread_muter_ap_ids(activity)
notification_enabled_users = Enum.filter(potential_receivers, fn u -> u.ap_id in notification_enabled_ap_ids end)
Enum.filter(potential_receivers, fn u -> u.ap_id in notification_enabled_ap_ids end)
{notification_enabled_users, potential_receivers -- notification_enabled_users}
end end
def get_notified_from_activity(_, _local_only), do: {[], []} def get_notified_from_activity(_, _local_only), do: []
def get_notified_subscribers_from_activity(activity, local_only \\ true)
def get_notified_subscribers_from_activity(
%Activity{data: %{"type" => "Create"}} = activity,
local_only
) do
notification_enabled_ap_ids =
[]
|> Utils.maybe_notify_subscribers(activity)
potential_receivers =
User.get_users_from_set(notification_enabled_ap_ids, local_only: local_only)
Enum.filter(potential_receivers, fn u -> u.ap_id in notification_enabled_ap_ids end)
end
def get_notified_subscribers_from_activity(_, _), do: []
# For some activities, only notify the author of the object # For some activities, only notify the author of the object
def get_potential_receiver_ap_ids(%{data: %{"type" => type, "object" => object_id}}) def get_potential_receiver_ap_ids(%{data: %{"type" => type, "object" => object_id}})
@ -576,7 +578,6 @@ def get_potential_receiver_ap_ids(activity) do
[] []
|> Utils.maybe_notify_to_recipients(activity) |> Utils.maybe_notify_to_recipients(activity)
|> Utils.maybe_notify_mentioned_recipients(activity) |> Utils.maybe_notify_mentioned_recipients(activity)
|> Utils.maybe_notify_subscribers(activity)
|> Utils.maybe_notify_followers(activity) |> Utils.maybe_notify_followers(activity)
|> Enum.uniq() |> Enum.uniq()
end end
@ -643,6 +644,7 @@ def skip?(activity, user, opts \\ [])
def skip?(%Activity{} = activity, %User{} = user, opts) do def skip?(%Activity{} = activity, %User{} = user, opts) do
[ [
:self, :self,
:internal,
:invisible, :invisible,
:block_from_strangers, :block_from_strangers,
:recently_followed, :recently_followed,
@ -662,6 +664,12 @@ def skip?(:self, %Activity{} = activity, %User{} = user, opts) do
end end
end end
def skip?(:internal, %Activity{} = activity, _user, _opts) do
actor = activity.data["actor"]
user = User.get_cached_by_ap_id(actor)
User.internal?(user)
end
def skip?(:invisible, %Activity{} = activity, _user, _opts) do def skip?(:invisible, %Activity{} = activity, _user, _opts) do
actor = activity.data["actor"] actor = activity.data["actor"]
user = User.get_cached_by_ap_id(actor) user = User.get_cached_by_ap_id(actor)
@ -748,4 +756,12 @@ def mark_context_as_read(%User{id: id}, context) do
) )
|> Repo.update_all(set: [seen: true]) |> Repo.update_all(set: [seen: true])
end end
@spec send(list(Notification.t())) :: :ok
def send(notifications) do
Enum.each(notifications, fn notification ->
Streamer.stream(["user", "user:notification"], notification)
Push.send(notification)
end)
end
end end

68
lib/pleroma/rule.ex Normal file
View File

@ -0,0 +1,68 @@
# Pleroma: A lightweight social networking server
# Copyright © 2017-2022 Pleroma Authors <https://pleroma.social/>
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.Rule do
use Ecto.Schema
import Ecto.Changeset
import Ecto.Query
alias Pleroma.Repo
alias Pleroma.Rule
schema "rules" do
field(:priority, :integer, default: 0)
field(:text, :string)
field(:hint, :string)
timestamps()
end
def changeset(%Rule{} = rule, params \\ %{}) do
rule
|> cast(params, [:priority, :text, :hint])
|> validate_required([:text])
end
def query do
Rule
|> order_by(asc: :priority)
|> order_by(asc: :id)
end
def get(ids) when is_list(ids) do
from(r in __MODULE__, where: r.id in ^ids)
|> Repo.all()
end
def get(id), do: Repo.get(__MODULE__, id)
def exists?(id) do
from(r in __MODULE__, where: r.id == ^id)
|> Repo.exists?()
end
def create(params) do
{:ok, rule} =
%Rule{}
|> changeset(params)
|> Repo.insert()
rule
end
def update(params, id) do
{:ok, rule} =
get(id)
|> changeset(params)
|> Repo.update()
rule
end
def delete(id) do
get(id)
|> Repo.delete()
end
end

View File

@ -204,7 +204,7 @@ def due_activities(offset \\ 0) do
def job_query(scheduled_activity_id) do def job_query(scheduled_activity_id) do
from(j in Oban.Job, from(j in Oban.Job,
where: j.queue == "scheduled_activities", where: j.queue == "federator_outgoing",
where: fragment("args ->> 'activity_id' = ?::text", ^to_string(scheduled_activity_id)) where: fragment("args ->> 'activity_id' = ?::text", ^to_string(scheduled_activity_id))
) )
end end

View File

@ -10,8 +10,12 @@ def remove_from_index(%Pleroma.Object{id: object_id}) do
end end
def search(query, options) do def search(query, options) do
search_module = Pleroma.Config.get([Pleroma.Search, :module], Pleroma.Activity) search_module = Pleroma.Config.get([Pleroma.Search, :module])
search_module.search(options[:for_user], query, options) search_module.search(options[:for_user], query, options)
end end
def healthcheck_endpoints do
search_module = Pleroma.Config.get([Pleroma.Search, :module])
search_module.healthcheck_endpoints
end
end end

View File

@ -28,7 +28,7 @@ def search(user, search_query, options \\ []) do
|> Activity.with_preloaded_object() |> Activity.with_preloaded_object()
|> Activity.restrict_deactivated_users() |> Activity.restrict_deactivated_users()
|> restrict_public(user) |> restrict_public(user)
|> query_with(index_type, search_query, :websearch) |> query_with(index_type, search_query)
|> maybe_restrict_local(user) |> maybe_restrict_local(user)
|> maybe_restrict_author(author) |> maybe_restrict_author(author)
|> maybe_restrict_blocked(user) |> maybe_restrict_blocked(user)
@ -48,6 +48,15 @@ def add_to_index(_activity), do: :ok
@impl true @impl true
def remove_from_index(_object), do: :ok def remove_from_index(_object), do: :ok
@impl true
def create_index, do: :ok
@impl true
def drop_index, do: :ok
@impl true
def healthcheck_endpoints, do: nil
def maybe_restrict_author(query, %User{} = author) do def maybe_restrict_author(query, %User{} = author) do
Activity.Queries.by_author(query, author) Activity.Queries.by_author(query, author)
end end
@ -79,25 +88,7 @@ defp restrict_public(q, _user) do
) )
end end
defp query_with(q, :gin, search_query, :plain) do defp query_with(q, :gin, search_query) do
%{rows: [[tsc]]} =
Ecto.Adapters.SQL.query!(
Pleroma.Repo,
"select current_setting('default_text_search_config')::regconfig::oid;"
)
from([a, o] in q,
where:
fragment(
"to_tsvector(?::oid::regconfig, ?->>'content') @@ plainto_tsquery(?)",
^tsc,
o.data,
^search_query
)
)
end
defp query_with(q, :gin, search_query, :websearch) do
%{rows: [[tsc]]} = %{rows: [[tsc]]} =
Ecto.Adapters.SQL.query!( Ecto.Adapters.SQL.query!(
Pleroma.Repo, Pleroma.Repo,
@ -115,19 +106,7 @@ defp query_with(q, :gin, search_query, :websearch) do
) )
end end
defp query_with(q, :rum, search_query, :plain) do defp query_with(q, :rum, search_query) do
from([a, o] in q,
where:
fragment(
"? @@ plainto_tsquery(?)",
o.fts_content,
^search_query
),
order_by: [fragment("? <=> now()::date", o.inserted_at)]
)
end
defp query_with(q, :rum, search_query, :websearch) do
from([a, o] in q, from([a, o] in q,
where: where:
fragment( fragment(

View File

@ -0,0 +1,86 @@
# Pleroma: A lightweight social networking server
# Copyright © 2017-2024 Pleroma Authors <https://pleroma.social/>
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.Search.Healthcheck do
@doc """
Monitors health of search backend to control processing of events based on health and availability.
"""
use GenServer
require Logger
@queue :search_indexing
@tick :timer.seconds(5)
@timeout :timer.seconds(2)
def start_link(_) do
GenServer.start_link(__MODULE__, [], name: __MODULE__)
end
@impl true
def init(_) do
state = %{healthy: false}
{:ok, state, {:continue, :start}}
end
@impl true
def handle_continue(:start, state) do
tick()
{:noreply, state}
end
@impl true
def handle_info(:check, state) do
urls = Pleroma.Search.healthcheck_endpoints()
new_state =
if check(urls) do
Oban.resume_queue(queue: @queue)
Map.put(state, :healthy, true)
else
Oban.pause_queue(queue: @queue)
Map.put(state, :healthy, false)
end
maybe_log_state_change(state, new_state)
tick()
{:noreply, new_state}
end
@impl true
def handle_call(:state, _from, state) do
{:reply, state, state, :hibernate}
end
def state, do: GenServer.call(__MODULE__, :state)
def check([]), do: true
def check(urls) when is_list(urls) do
Enum.all?(
urls,
fn url ->
case Pleroma.HTTP.get(url, [], recv_timeout: @timeout) do
{:ok, %{status: 200}} -> true
_ -> false
end
end
)
end
def check(_), do: true
defp tick do
Process.send_after(self(), :check, @tick)
end
defp maybe_log_state_change(%{healthy: true}, %{healthy: false}) do
Logger.error("Pausing Oban queue #{@queue} due to search backend healthcheck failure")
end
defp maybe_log_state_change(%{healthy: false}, %{healthy: true}) do
Logger.info("Resuming Oban queue #{@queue} due to search backend healthcheck pass")
end
defp maybe_log_state_change(_, _), do: :ok
end

View File

@ -10,6 +10,12 @@ defmodule Pleroma.Search.Meilisearch do
@behaviour Pleroma.Search.SearchBackend @behaviour Pleroma.Search.SearchBackend
@impl true
def create_index, do: :ok
@impl true
def drop_index, do: :ok
defp meili_headers do defp meili_headers do
private_key = Config.get([Pleroma.Search.Meilisearch, :private_key]) private_key = Config.get([Pleroma.Search.Meilisearch, :private_key])
@ -178,4 +184,15 @@ def add_to_index(activity) do
def remove_from_index(object) do def remove_from_index(object) do
meili_delete("/indexes/objects/documents/#{object.id}") meili_delete("/indexes/objects/documents/#{object.id}")
end end
@impl true
def healthcheck_endpoints do
endpoint =
Config.get([Pleroma.Search.Meilisearch, :url])
|> URI.parse()
|> Map.put(:path, "/health")
|> URI.to_string()
[endpoint]
end
end end

View File

@ -0,0 +1,182 @@
defmodule Pleroma.Search.QdrantSearch do
@behaviour Pleroma.Search.SearchBackend
import Ecto.Query
alias Pleroma.Activity
alias Pleroma.Config.Getting, as: Config
alias __MODULE__.OpenAIClient
alias __MODULE__.QdrantClient
import Pleroma.Search.Meilisearch, only: [object_to_search_data: 1]
import Pleroma.Search.DatabaseSearch, only: [maybe_fetch: 3]
@impl true
def create_index do
payload = Config.get([Pleroma.Search.QdrantSearch, :qdrant_index_configuration])
with {:ok, %{status: 200}} <- QdrantClient.put("/collections/posts", payload) do
:ok
else
e -> {:error, e}
end
end
@impl true
def drop_index do
with {:ok, %{status: 200}} <- QdrantClient.delete("/collections/posts") do
:ok
else
e -> {:error, e}
end
end
def get_embedding(text) do
with {:ok, %{body: %{"data" => [%{"embedding" => embedding}]}}} <-
OpenAIClient.post("/v1/embeddings", %{
input: text,
model: Config.get([Pleroma.Search.QdrantSearch, :openai_model])
}) do
{:ok, embedding}
else
_ ->
{:error, "Failed to get embedding"}
end
end
defp actor_from_activity(%{data: %{"actor" => actor}}) do
actor
end
defp actor_from_activity(_), do: nil
defp build_index_payload(activity, embedding) do
actor = actor_from_activity(activity)
published_at = activity.data["published"]
%{
points: [
%{
id: activity.id |> FlakeId.from_string() |> Ecto.UUID.cast!(),
vector: embedding,
payload: %{actor: actor, published_at: published_at}
}
]
}
end
defp build_search_payload(embedding, options) do
base = %{
vector: embedding,
limit: options[:limit] || 20,
offset: options[:offset] || 0
}
if author = options[:author] do
Map.put(base, :filter, %{
must: [%{key: "actor", match: %{value: author.ap_id}}]
})
else
base
end
end
@impl true
def add_to_index(activity) do
# This will only index public or unlisted notes
maybe_search_data = object_to_search_data(activity.object)
if activity.data["type"] == "Create" and maybe_search_data do
with {:ok, embedding} <- get_embedding(maybe_search_data.content),
{:ok, %{status: 200}} <-
QdrantClient.put(
"/collections/posts/points",
build_index_payload(activity, embedding)
) do
:ok
else
e -> {:error, e}
end
else
:ok
end
end
@impl true
def remove_from_index(object) do
activity = Activity.get_by_object_ap_id_with_object(object.data["id"])
id = activity.id |> FlakeId.from_string() |> Ecto.UUID.cast!()
with {:ok, %{status: 200}} <-
QdrantClient.post("/collections/posts/points/delete", %{"points" => [id]}) do
:ok
else
e -> {:error, e}
end
end
@impl true
def search(user, original_query, options) do
query = "Represent this sentence for searching relevant passages: #{original_query}"
with {:ok, embedding} <- get_embedding(query),
{:ok, %{body: %{"result" => result}}} <-
QdrantClient.post(
"/collections/posts/points/search",
build_search_payload(embedding, options)
) do
ids =
Enum.map(result, fn %{"id" => id} ->
Ecto.UUID.dump!(id)
end)
from(a in Activity, where: a.id in ^ids)
|> Activity.with_preloaded_object()
|> Activity.restrict_deactivated_users()
|> Ecto.Query.order_by([a], fragment("array_position(?, ?)", ^ids, a.id))
|> Pleroma.Repo.all()
|> maybe_fetch(user, original_query)
else
_ ->
[]
end
end
@impl true
def healthcheck_endpoints do
qdrant_health =
Config.get([Pleroma.Search.QdrantSearch, :qdrant_url])
|> URI.parse()
|> Map.put(:path, "/healthz")
|> URI.to_string()
openai_health = Config.get([Pleroma.Search.QdrantSearch, :openai_healthcheck_url])
[qdrant_health, openai_health] |> Enum.filter(& &1)
end
end
defmodule Pleroma.Search.QdrantSearch.OpenAIClient do
use Tesla
alias Pleroma.Config.Getting, as: Config
plug(Tesla.Middleware.BaseUrl, Config.get([Pleroma.Search.QdrantSearch, :openai_url]))
plug(Tesla.Middleware.JSON)
plug(Tesla.Middleware.Headers, [
{"Authorization",
"Bearer #{Pleroma.Config.get([Pleroma.Search.QdrantSearch, :openai_api_key])}"}
])
end
defmodule Pleroma.Search.QdrantSearch.QdrantClient do
use Tesla
alias Pleroma.Config.Getting, as: Config
plug(Tesla.Middleware.BaseUrl, Config.get([Pleroma.Search.QdrantSearch, :qdrant_url]))
plug(Tesla.Middleware.JSON)
plug(Tesla.Middleware.Headers, [
{"api-key", Pleroma.Config.get([Pleroma.Search.QdrantSearch, :qdrant_api_key])}
])
end

View File

@ -21,4 +21,22 @@ defmodule Pleroma.Search.SearchBackend do
from index. from index.
""" """
@callback remove_from_index(object :: Pleroma.Object.t()) :: :ok | {:error, any()} @callback remove_from_index(object :: Pleroma.Object.t()) :: :ok | {:error, any()}
@doc """
Create the index
"""
@callback create_index() :: :ok | {:error, any()}
@doc """
Drop the index
"""
@callback drop_index() :: :ok | {:error, any()}
@doc """
Healthcheck endpoints of search backend infrastructure to monitor for controlling
processing of jobs in the Oban queue.
It is expected a 200 response is healthy and other responses are unhealthy.
"""
@callback healthcheck_endpoints :: list() | nil
end end

View File

@ -17,6 +17,7 @@ def key_id_to_actor_id(key_id) do
key_id key_id
|> URI.parse() |> URI.parse()
|> Map.put(:fragment, nil) |> Map.put(:fragment, nil)
|> remove_query()
|> remove_suffix(@known_suffixes) |> remove_suffix(@known_suffixes)
maybe_ap_id = URI.to_string(uri) maybe_ap_id = URI.to_string(uri)
@ -33,6 +34,23 @@ def key_id_to_actor_id(key_id) do
end end
end end
defp remove_query(uri) do
if uri.query do
new_query =
URI.decode_query(uri.query)
|> Map.delete("operation")
|> URI.encode_query()
|> case do
"" -> nil
query -> query
end
Map.put(uri, :query, new_query)
else
uri
end
end
defp remove_suffix(uri, [test | rest]) do defp remove_suffix(uri, [test | rest]) do
if not is_nil(uri.path) and String.ends_with?(uri.path, test) do if not is_nil(uri.path) and String.ends_with?(uri.path, test) do
Map.put(uri, :path, String.replace(uri.path, test, "")) Map.put(uri, :path, String.replace(uri.path, test, ""))
@ -44,8 +62,7 @@ defp remove_suffix(uri, [test | rest]) do
defp remove_suffix(uri, []), do: uri defp remove_suffix(uri, []), do: uri
def fetch_public_key(conn) do def fetch_public_key(conn) do
with %{"keyId" => kid} <- HTTPSignatures.signature_for_conn(conn), with {:ok, actor_id} <- get_actor_id(conn),
{:ok, actor_id} <- key_id_to_actor_id(kid),
{:ok, public_key} <- User.get_public_key_for_ap_id(actor_id) do {:ok, public_key} <- User.get_public_key_for_ap_id(actor_id) do
{:ok, public_key} {:ok, public_key}
else else
@ -55,8 +72,7 @@ def fetch_public_key(conn) do
end end
def refetch_public_key(conn) do def refetch_public_key(conn) do
with %{"keyId" => kid} <- HTTPSignatures.signature_for_conn(conn), with {:ok, actor_id} <- get_actor_id(conn),
{:ok, actor_id} <- key_id_to_actor_id(kid),
{:ok, _user} <- ActivityPub.make_user_from_ap_id(actor_id), {:ok, _user} <- ActivityPub.make_user_from_ap_id(actor_id),
{:ok, public_key} <- User.get_public_key_for_ap_id(actor_id) do {:ok, public_key} <- User.get_public_key_for_ap_id(actor_id) do
{:ok, public_key} {:ok, public_key}
@ -66,6 +82,16 @@ def refetch_public_key(conn) do
end end
end end
def get_actor_id(conn) do
with %{"keyId" => kid} <- HTTPSignatures.signature_for_conn(conn),
{:ok, actor_id} <- key_id_to_actor_id(kid) do
{:ok, actor_id}
else
e ->
{:error, e}
end
end
def sign(%User{keys: keys} = user, headers) do def sign(%User{keys: keys} = user, headers) do
with {:ok, private_key, _} <- Keys.keys_from_pem(keys) do with {:ok, private_key, _} <- Keys.keys_from_pem(keys) do
HTTPSignatures.sign(private_key, user.ap_id <> "#main-key", headers) HTTPSignatures.sign(private_key, user.ap_id <> "#main-key", headers)

View File

@ -239,8 +239,12 @@ defp url_from_spec(%__MODULE__{name: name}, base_url, {:file, path}) do
"" ""
end end
[base_url, path] if String.contains?(base_url, Pleroma.Uploaders.IPFS.placeholder()) do
|> Path.join() String.replace(base_url, Pleroma.Uploaders.IPFS.placeholder(), path)
else
[base_url, path]
|> Path.join()
end
end end
defp url_from_spec(_upload, _base_url, {:url, url}), do: url defp url_from_spec(_upload, _base_url, {:url, url}), do: url
@ -277,6 +281,9 @@ def base_url do
Path.join([upload_base_url, bucket_with_namespace]) Path.join([upload_base_url, bucket_with_namespace])
end end
Pleroma.Uploaders.IPFS ->
@config_impl.get([Pleroma.Uploaders.IPFS, :get_gateway_url])
_ -> _ ->
public_endpoint || upload_base_url || Pleroma.Web.Endpoint.url() <> "/media/" public_endpoint || upload_base_url || Pleroma.Web.Endpoint.url() <> "/media/"
end end

View File

@ -9,8 +9,6 @@ defmodule Pleroma.Upload.Filter.Exiftool.StripLocation do
""" """
@behaviour Pleroma.Upload.Filter @behaviour Pleroma.Upload.Filter
@spec filter(Pleroma.Upload.t()) :: {:ok, any()} | {:error, String.t()}
# Formats not compatible with exiftool at this time # Formats not compatible with exiftool at this time
def filter(%Pleroma.Upload{content_type: "image/heic"}), do: {:ok, :noop} def filter(%Pleroma.Upload{content_type: "image/heic"}), do: {:ok, :noop}
def filter(%Pleroma.Upload{content_type: "image/webp"}), do: {:ok, :noop} def filter(%Pleroma.Upload{content_type: "image/webp"}), do: {:ok, :noop}

View File

@ -38,7 +38,6 @@ defmodule Pleroma.Upload.Filter.Mogrifun do
[{"fill", "yellow"}, {"tint", "40"}] [{"fill", "yellow"}, {"tint", "40"}]
] ]
@spec filter(Pleroma.Upload.t()) :: {:ok, atom()} | {:error, String.t()}
def filter(%Pleroma.Upload{tempfile: file, content_type: "image" <> _}) do def filter(%Pleroma.Upload{tempfile: file, content_type: "image" <> _}) do
try do try do
Filter.Mogrify.do_filter(file, [Enum.random(@filters)]) Filter.Mogrify.do_filter(file, [Enum.random(@filters)])

View File

@ -8,7 +8,6 @@ defmodule Pleroma.Upload.Filter.Mogrify do
@type conversion :: action :: String.t() | {action :: String.t(), opts :: String.t()} @type conversion :: action :: String.t() | {action :: String.t(), opts :: String.t()}
@type conversions :: conversion() | [conversion()] @type conversions :: conversion() | [conversion()]
@spec filter(Pleroma.Upload.t()) :: {:ok, :atom} | {:error, String.t()}
def filter(%Pleroma.Upload{tempfile: file, content_type: "image" <> _}) do def filter(%Pleroma.Upload{tempfile: file, content_type: "image" <> _}) do
try do try do
do_filter(file, Pleroma.Config.get!([__MODULE__, :args])) do_filter(file, Pleroma.Config.get!([__MODULE__, :args]))

View File

@ -0,0 +1,77 @@
# Pleroma: A lightweight social networking server
# Copyright © 2017-2022 Pleroma Authors <https://pleroma.social/>
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.Uploaders.IPFS do
@behaviour Pleroma.Uploaders.Uploader
require Logger
alias Tesla.Multipart
@config_impl Application.compile_env(:pleroma, [__MODULE__, :config_impl], Pleroma.Config)
defp get_final_url(method) do
config = @config_impl.get([__MODULE__])
post_base_url = Keyword.get(config, :post_gateway_url)
Path.join([post_base_url, method])
end
def put_file_endpoint do
get_final_url("/api/v0/add")
end
def delete_file_endpoint do
get_final_url("/api/v0/files/rm")
end
@placeholder "{CID}"
def placeholder, do: @placeholder
@impl true
def get_file(file) do
b_url = Pleroma.Upload.base_url()
if String.contains?(b_url, @placeholder) do
{:ok, {:url, String.replace(b_url, @placeholder, URI.decode(file))}}
else
{:error, "IPFS Get URL doesn't contain 'cid' placeholder"}
end
end
@impl true
def put_file(%Pleroma.Upload{} = upload) do
mp =
Multipart.new()
|> Multipart.add_content_type_param("charset=utf-8")
|> Multipart.add_file(upload.tempfile)
case Pleroma.HTTP.post(put_file_endpoint(), mp, [], params: ["cid-version": "1"]) do
{:ok, ret} ->
case Jason.decode(ret.body) do
{:ok, ret} ->
if Map.has_key?(ret, "Hash") do
{:ok, {:file, ret["Hash"]}}
else
{:error, "JSON doesn't contain Hash key"}
end
error ->
Logger.error("#{__MODULE__}: #{inspect(error)}")
{:error, "JSON decode failed"}
end
error ->
Logger.error("#{__MODULE__}: #{inspect(error)}")
{:error, "IPFS Gateway upload failed"}
end
end
@impl true
def delete_file(file) do
case Pleroma.HTTP.post(delete_file_endpoint(), "", [], params: [arg: file]) do
{:ok, %{status: 204}} -> :ok
error -> {:error, inspect(error)}
end
end
end

View File

@ -1404,6 +1404,40 @@ def get_friends_ids(%User{} = user, page \\ nil) do
|> Repo.all() |> Repo.all()
end end
@spec get_familiar_followers_query(User.t(), User.t(), pos_integer() | nil) :: Ecto.Query.t()
def get_familiar_followers_query(%User{} = user, %User{} = current_user, nil) do
friends =
get_friends_query(current_user)
|> where([u], not u.hide_follows)
|> select([u], u.id)
User.Query.build(%{is_active: true})
|> where([u], u.id not in ^[user.id, current_user.id])
|> join(:inner, [u], r in FollowingRelationship,
as: :followers_relationships,
on: r.following_id == ^user.id and r.follower_id == u.id
)
|> where([followers_relationships: r], r.state == ^:follow_accept)
|> where([followers_relationships: r], r.follower_id in subquery(friends))
end
def get_familiar_followers_query(%User{} = user, %User{} = current_user, page) do
user
|> get_familiar_followers_query(current_user, nil)
|> User.Query.paginate(page, 20)
end
@spec get_familiar_followers_query(User.t(), User.t()) :: Ecto.Query.t()
def get_familiar_followers_query(%User{} = user, %User{} = current_user),
do: get_familiar_followers_query(user, current_user, nil)
@spec get_familiar_followers(User.t(), User.t(), pos_integer() | nil) :: {:ok, list(User.t())}
def get_familiar_followers(%User{} = user, %User{} = current_user, page \\ nil) do
user
|> get_familiar_followers_query(current_user, page)
|> Repo.all()
end
def increase_note_count(%User{} = user) do def increase_note_count(%User{} = user) do
User User
|> where(id: ^user.id) |> where(id: ^user.id)
@ -2019,7 +2053,8 @@ defp verify_field_link(field, profile_urls) do
%{scheme: scheme, userinfo: nil, host: host} %{scheme: scheme, userinfo: nil, host: host}
when not_empty_string(host) and scheme in ["http", "https"] <- when not_empty_string(host) and scheme in ["http", "https"] <-
URI.parse(value), URI.parse(value),
{:not_idn, true} <- {:not_idn, to_string(:idna.encode(host)) == host}, {:not_idn, true} <-
{:not_idn, match?(^host, to_string(:idna.encode(to_charlist(host))))},
"me" <- Pleroma.Web.RelMe.maybe_put_rel_me(value, profile_urls) do "me" <- Pleroma.Web.RelMe.maybe_put_rel_me(value, profile_urls) do
CommonUtils.to_masto_date(NaiveDateTime.utc_now()) CommonUtils.to_masto_date(NaiveDateTime.utc_now())
else else
@ -2693,7 +2728,7 @@ defp add_to_block(%User{} = user, %User{} = blocked) do
end end
end end
@spec add_to_block(User.t(), User.t()) :: @spec remove_from_block(User.t(), User.t()) ::
{:ok, UserRelationship.t()} | {:ok, nil} | {:error, Ecto.Changeset.t()} {:ok, UserRelationship.t()} | {:ok, nil} | {:error, Ecto.Changeset.t()}
defp remove_from_block(%User{} = user, %User{} = blocked) do defp remove_from_block(%User{} = user, %User{} = blocked) do
with {:ok, relationship} <- UserRelationship.delete_block(user, blocked) do with {:ok, relationship} <- UserRelationship.delete_block(user, blocked) do

View File

@ -200,7 +200,8 @@ defp insert_activity_with_expiration(data, local, recipients) do
end end
def notify_and_stream(activity) do def notify_and_stream(activity) do
Notification.create_notifications(activity) {:ok, notifications} = Notification.create_notifications(activity)
Notification.send(notifications)
original_activity = original_activity =
case activity do case activity do
@ -978,8 +979,9 @@ defp restrict_media(query, _), do: query
defp restrict_replies(query, %{exclude_replies: true}) do defp restrict_replies(query, %{exclude_replies: true}) do
from( from(
[_activity, object] in query, [activity, object] in query,
where: fragment("?->>'inReplyTo' is null", object.data) where:
fragment("?->>'inReplyTo' is null or ?->>'type' = 'Announce'", object.data, activity.data)
) )
end end
@ -1259,6 +1261,15 @@ defp restrict_quote_url(query, %{quote_url: quote_url}) do
defp restrict_quote_url(query, _), do: query defp restrict_quote_url(query, _), do: query
defp restrict_rule(query, %{rule_id: rule_id}) do
from(
activity in query,
where: fragment("(?)->'rules' \\? (?)", activity.data, ^rule_id)
)
end
defp restrict_rule(query, _), do: query
defp exclude_poll_votes(query, %{include_poll_votes: true}), do: query defp exclude_poll_votes(query, %{include_poll_votes: true}), do: query
defp exclude_poll_votes(query, _) do defp exclude_poll_votes(query, _) do
@ -1421,6 +1432,7 @@ def fetch_activities_query(recipients, opts \\ %{}) do
|> restrict_instance(opts) |> restrict_instance(opts)
|> restrict_announce_object_actor(opts) |> restrict_announce_object_actor(opts)
|> restrict_filtered(opts) |> restrict_filtered(opts)
|> restrict_rule(opts)
|> restrict_quote_url(opts) |> restrict_quote_url(opts)
|> maybe_restrict_deactivated_users(opts) |> maybe_restrict_deactivated_users(opts)
|> exclude_poll_votes(opts) |> exclude_poll_votes(opts)

View File

@ -522,7 +522,7 @@ defp set_requester_reachable(%Plug.Conn{} = conn, _) do
conn conn
end end
defp log_inbox_metadata(conn = %{params: %{"actor" => actor, "type" => type}}, _) do defp log_inbox_metadata(%{params: %{"actor" => actor, "type" => type}} = conn, _) do
Logger.metadata(actor: actor, type: type) Logger.metadata(actor: actor, type: type)
conn conn
end end

View File

@ -0,0 +1,87 @@
# Pleroma: A lightweight social networking server
# Copyright © 2017-2022 Pleroma Authors <https://pleroma.social/>
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.Web.ActivityPub.MRF.AntiMentionSpamPolicy do
alias Pleroma.Config
alias Pleroma.User
require Pleroma.Constants
@behaviour Pleroma.Web.ActivityPub.MRF.Policy
defp user_has_posted?(%User{} = u), do: u.note_count > 0
defp user_has_age?(%User{} = u) do
user_age_limit = Config.get([:mrf_antimentionspam, :user_age_limit], 30_000)
diff = NaiveDateTime.utc_now() |> NaiveDateTime.diff(u.inserted_at, :millisecond)
diff >= user_age_limit
end
defp good_reputation?(%User{} = u) do
user_has_age?(u) and user_has_posted?(u)
end
# copied from HellthreadPolicy
defp get_recipient_count(message) do
recipients = (message["to"] || []) ++ (message["cc"] || [])
follower_collection =
User.get_cached_by_ap_id(message["actor"] || message["attributedTo"]).follower_address
if Enum.member?(recipients, Pleroma.Constants.as_public()) do
recipients =
recipients
|> List.delete(Pleroma.Constants.as_public())
|> List.delete(follower_collection)
{:public, length(recipients)}
else
recipients =
recipients
|> List.delete(follower_collection)
{:not_public, length(recipients)}
end
end
defp object_has_recipients?(%{"object" => object} = activity) do
{_, object_count} = get_recipient_count(object)
{_, activity_count} = get_recipient_count(activity)
object_count + activity_count > 0
end
defp object_has_recipients?(object) do
{_, count} = get_recipient_count(object)
count > 0
end
@impl true
def filter(%{"type" => "Create", "actor" => actor} = activity) do
with {:ok, %User{local: false} = u} <- User.get_or_fetch_by_ap_id(actor),
{:has_mentions, true} <- {:has_mentions, object_has_recipients?(activity)},
{:good_reputation, true} <- {:good_reputation, good_reputation?(u)} do
{:ok, activity}
else
{:ok, %User{local: true}} ->
{:ok, activity}
{:has_mentions, false} ->
{:ok, activity}
{:good_reputation, false} ->
{:reject, "[AntiMentionSpamPolicy] User rejected"}
{:error, _} ->
{:reject, "[AntiMentionSpamPolicy] Failed to get or fetch user by ap_id"}
e ->
{:reject, "[AntiMentionSpamPolicy] Unhandled error #{inspect(e)}"}
end
end
# in all other cases, pass through
def filter(message), do: {:ok, message}
@impl true
def describe, do: {:ok, %{}}
end

View File

@ -0,0 +1,146 @@
# Pleroma: A lightweight social networking server
# Copyright © 2017-2024 Pleroma Authors <https://pleroma.social/>
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.Web.ActivityPub.MRF.DNSRBLPolicy do
@moduledoc """
Dynamic activity filtering based on an RBL database
This MRF makes queries to a custom DNS server which will
respond with values indicating the classification of the domain
the activity originated from. This method has been widely used
in the email anti-spam industry for very fast reputation checks.
e.g., if the DNS response is 127.0.0.1 or empty, the domain is OK
Other values such as 127.0.0.2 may be used for specific classifications.
Information for why the host is blocked can be stored in a corresponding TXT record.
This method is fail-open so if the queries fail the activites are accepted.
An example of software meant for this purpsoe is rbldnsd which can be found
at http://www.corpit.ru/mjt/rbldnsd.html or mirrored at
https://git.pleroma.social/feld/rbldnsd
It is highly recommended that you run your own copy of rbldnsd and use an
external mechanism to sync/share the contents of the zone file. This is
important to keep the latency on the queries as low as possible and prevent
your DNS server from being attacked so it fails and content is permitted.
"""
@behaviour Pleroma.Web.ActivityPub.MRF.Policy
alias Pleroma.Config
require Logger
@query_retries 1
@query_timeout 500
@impl true
def filter(%{"actor" => actor} = object) do
actor_info = URI.parse(actor)
with {:ok, object} <- check_rbl(actor_info, object) do
{:ok, object}
else
_ -> {:reject, "[DNSRBLPolicy]"}
end
end
@impl true
def filter(object), do: {:ok, object}
@impl true
def describe do
mrf_dnsrbl =
Config.get(:mrf_dnsrbl)
|> Enum.into(%{})
{:ok, %{mrf_dnsrbl: mrf_dnsrbl}}
end
@impl true
def config_description do
%{
key: :mrf_dnsrbl,
related_policy: "Pleroma.Web.ActivityPub.MRF.DNSRBLPolicy",
label: "MRF DNSRBL",
description: "DNS RealTime Blackhole Policy",
children: [
%{
key: :nameserver,
type: {:string},
description: "DNSRBL Nameserver to Query (IP or hostame)",
suggestions: ["127.0.0.1"]
},
%{
key: :port,
type: {:string},
description: "Nameserver port",
suggestions: ["53"]
},
%{
key: :zone,
type: {:string},
description: "Root zone for querying",
suggestions: ["bl.pleroma.com"]
}
]
}
end
defp check_rbl(%{host: actor_host}, object) do
with false <- match?(^actor_host, Pleroma.Web.Endpoint.host()),
zone when not is_nil(zone) <- Keyword.get(Config.get([:mrf_dnsrbl]), :zone) do
query =
Enum.join([actor_host, zone], ".")
|> String.to_charlist()
rbl_response = rblquery(query)
if Enum.empty?(rbl_response) do
{:ok, object}
else
Task.start(fn ->
reason =
case rblquery(query, :txt) do
[[result]] -> result
_ -> "undefined"
end
Logger.warning(
"DNSRBL Rejected activity from #{actor_host} for reason: #{inspect(reason)}"
)
end)
:error
end
else
_ -> {:ok, object}
end
end
defp get_rblhost_ip(rblhost) do
case rblhost |> String.to_charlist() |> :inet_parse.address() do
{:ok, _} -> rblhost |> String.to_charlist() |> :inet_parse.address()
_ -> {:ok, rblhost |> String.to_charlist() |> :inet_res.lookup(:in, :a) |> Enum.random()}
end
end
defp rblquery(query, type \\ :a) do
config = Config.get([:mrf_dnsrbl])
case get_rblhost_ip(config[:nameserver]) do
{:ok, rblnsip} ->
:inet_res.lookup(query, :in, type,
nameservers: [{rblnsip, config[:port]}],
timeout: @query_timeout,
retry: @query_retries
)
_ ->
[]
end
end
end

View File

@ -11,11 +11,6 @@ defmodule Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy do
require Logger require Logger
@adapter_options [
pool: :media,
recv_timeout: 10_000
]
@impl true @impl true
def history_awareness, do: :auto def history_awareness, do: :auto
@ -27,17 +22,14 @@ defp prefetch(url) do
Logger.debug("Prefetching #{inspect(url)} as #{inspect(prefetch_url)}") Logger.debug("Prefetching #{inspect(url)} as #{inspect(prefetch_url)}")
if Pleroma.Config.get(:env) == :test do fetch(prefetch_url)
fetch(prefetch_url)
else
ConcurrentLimiter.limit(__MODULE__, fn ->
Task.start(fn -> fetch(prefetch_url) end)
end)
end
end end
end end
defp fetch(url), do: HTTP.get(url, [], @adapter_options) defp fetch(url) do
http_client_opts = Pleroma.Config.get([:media_proxy, :proxy_opts, :http], pool: :media)
HTTP.get(url, [], http_client_opts)
end
defp preload(%{"object" => %{"attachment" => attachments}} = _message) do defp preload(%{"object" => %{"attachment" => attachments}} = _message) do
Enum.each(attachments, fn Enum.each(attachments, fn

View File

@ -0,0 +1,265 @@
# Pleroma: A lightweight social networking server
# Copyright © 2017-2021 Pleroma Authors <https://pleroma.social/>
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.Web.ActivityPub.MRF.NsfwApiPolicy do
@moduledoc """
Hide, delete, or mark sensitive NSFW content with artificial intelligence.
Requires a NSFW API server, configured like so:
config :pleroma, Pleroma.Web.ActivityPub.MRF.NsfwMRF,
url: "http://127.0.0.1:5000/",
threshold: 0.7,
mark_sensitive: true,
unlist: false,
reject: false
The NSFW API server must implement an HTTP endpoint like this:
curl http://localhost:5000/?url=https://fedi.com/images/001.jpg
Returning a response like this:
{"score", 0.314}
Where a score is 0-1, with `1` being definitely NSFW.
A good API server is here: https://github.com/EugenCepoi/nsfw_api
You can run it with Docker with a one-liner:
docker run -it -p 127.0.0.1:5000:5000/tcp --env PORT=5000 eugencepoi/nsfw_api:latest
Options:
- `url`: Base URL of the API server. Default: "http://127.0.0.1:5000/"
- `threshold`: Lowest score to take action on. Default: `0.7`
- `mark_sensitive`: Mark sensitive all detected NSFW content? Default: `true`
- `unlist`: Unlist all detected NSFW content? Default: `false`
- `reject`: Reject all detected NSFW content (takes precedence)? Default: `false`
"""
alias Pleroma.Config
alias Pleroma.Constants
alias Pleroma.HTTP
alias Pleroma.User
require Logger
require Pleroma.Constants
@behaviour Pleroma.Web.ActivityPub.MRF.Policy
@policy :mrf_nsfw_api
def build_request_url(url) do
Config.get([@policy, :url])
|> URI.parse()
|> fix_path()
|> Map.put(:query, "url=#{url}")
|> URI.to_string()
end
def parse_url(url) do
request = build_request_url(url)
with {:ok, %Tesla.Env{body: body}} <- HTTP.get(request) do
Jason.decode(body)
else
error ->
Logger.warning("""
[NsfwApiPolicy]: The API server failed. Skipping.
#{inspect(error)}
""")
error
end
end
def check_url_nsfw(url) when is_binary(url) do
threshold = Config.get([@policy, :threshold])
case parse_url(url) do
{:ok, %{"score" => score}} when score >= threshold ->
{:nsfw, %{url: url, score: score, threshold: threshold}}
{:ok, %{"score" => score}} ->
{:sfw, %{url: url, score: score, threshold: threshold}}
_ ->
{:sfw, %{url: url, score: nil, threshold: threshold}}
end
end
def check_url_nsfw(%{"href" => url}) when is_binary(url) do
check_url_nsfw(url)
end
def check_url_nsfw(url) do
threshold = Config.get([@policy, :threshold])
{:sfw, %{url: url, score: nil, threshold: threshold}}
end
def check_attachment_nsfw(%{"url" => urls} = attachment) when is_list(urls) do
if Enum.all?(urls, &match?({:sfw, _}, check_url_nsfw(&1))) do
{:sfw, attachment}
else
{:nsfw, attachment}
end
end
def check_attachment_nsfw(%{"url" => url} = attachment) when is_binary(url) do
case check_url_nsfw(url) do
{:sfw, _} -> {:sfw, attachment}
{:nsfw, _} -> {:nsfw, attachment}
end
end
def check_attachment_nsfw(attachment), do: {:sfw, attachment}
def check_object_nsfw(%{"attachment" => attachments} = object) when is_list(attachments) do
if Enum.all?(attachments, &match?({:sfw, _}, check_attachment_nsfw(&1))) do
{:sfw, object}
else
{:nsfw, object}
end
end
def check_object_nsfw(%{"object" => %{} = child_object} = object) do
case check_object_nsfw(child_object) do
{:sfw, _} -> {:sfw, object}
{:nsfw, _} -> {:nsfw, object}
end
end
def check_object_nsfw(object), do: {:sfw, object}
@impl true
def filter(object) do
with {:sfw, object} <- check_object_nsfw(object) do
{:ok, object}
else
{:nsfw, _data} -> handle_nsfw(object)
_ -> {:reject, "NSFW: Attachment rejected"}
end
end
defp handle_nsfw(object) do
if Config.get([@policy, :reject]) do
{:reject, object}
else
{:ok,
object
|> maybe_unlist()
|> maybe_mark_sensitive()}
end
end
defp maybe_unlist(object) do
if Config.get([@policy, :unlist]) do
unlist(object)
else
object
end
end
defp maybe_mark_sensitive(object) do
if Config.get([@policy, :mark_sensitive]) do
mark_sensitive(object)
else
object
end
end
def unlist(%{"to" => to, "cc" => cc, "actor" => actor} = object) do
with %User{} = user <- User.get_cached_by_ap_id(actor) do
to =
[user.follower_address | to]
|> List.delete(Constants.as_public())
|> Enum.uniq()
cc =
[Constants.as_public() | cc]
|> List.delete(user.follower_address)
|> Enum.uniq()
object
|> Map.put("to", to)
|> Map.put("cc", cc)
else
_ -> raise "[NsfwApiPolicy]: Could not find user #{actor}"
end
end
def mark_sensitive(%{"object" => child_object} = object) when is_map(child_object) do
Map.put(object, "object", mark_sensitive(child_object))
end
def mark_sensitive(object) when is_map(object) do
tags = (object["tag"] || []) ++ ["nsfw"]
object
|> Map.put("tag", tags)
|> Map.put("sensitive", true)
end
# Hackney needs a trailing slash
defp fix_path(%URI{path: path} = uri) when is_binary(path) do
path = String.trim_trailing(path, "/") <> "/"
Map.put(uri, :path, path)
end
defp fix_path(%URI{path: nil} = uri), do: Map.put(uri, :path, "/")
@impl true
def describe do
options = %{
threshold: Config.get([@policy, :threshold]),
mark_sensitive: Config.get([@policy, :mark_sensitive]),
unlist: Config.get([@policy, :unlist]),
reject: Config.get([@policy, :reject])
}
{:ok, %{@policy => options}}
end
@impl true
def config_description do
%{
key: @policy,
related_policy: to_string(__MODULE__),
label: "NSFW API Policy",
description:
"Hide, delete, or mark sensitive NSFW content with artificial intelligence. Requires running an external API server.",
children: [
%{
key: :url,
type: :string,
description: "Base URL of the API server.",
suggestions: ["http://127.0.0.1:5000/"]
},
%{
key: :threshold,
type: :float,
description: "Lowest score to take action on. Between 0 and 1.",
suggestions: [0.7]
},
%{
key: :mark_sensitive,
type: :boolean,
description: "Mark sensitive all detected NSFW content?",
suggestions: [true]
},
%{
key: :unlist,
type: :boolean,
description: "Unlist sensitive all detected NSFW content?",
suggestions: [false]
},
%{
key: :reject,
type: :boolean,
description: "Reject sensitive all detected NSFW content (takes precedence)?",
suggestions: [false]
}
]
}
end
end

View File

@ -15,6 +15,7 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.AttachmentValidator do
field(:type, :string, default: "Link") field(:type, :string, default: "Link")
field(:mediaType, ObjectValidators.MIME, default: "application/octet-stream") field(:mediaType, ObjectValidators.MIME, default: "application/octet-stream")
field(:name, :string) field(:name, :string)
field(:summary, :string)
field(:blurhash, :string) field(:blurhash, :string)
embeds_many :url, UrlObjectValidator, primary_key: false do embeds_many :url, UrlObjectValidator, primary_key: false do
@ -44,7 +45,7 @@ def changeset(struct, data) do
|> fix_url() |> fix_url()
struct struct
|> cast(data, [:id, :type, :mediaType, :name, :blurhash]) |> cast(data, [:id, :type, :mediaType, :name, :summary, :blurhash])
|> cast_embed(:url, with: &url_changeset/2, required: true) |> cast_embed(:url, with: &url_changeset/2, required: true)
|> validate_inclusion(:type, ~w[Link Document Audio Image Video]) |> validate_inclusion(:type, ~w[Link Document Audio Image Video])
|> validate_required([:type, :mediaType]) |> validate_required([:type, :mediaType])

View File

@ -21,7 +21,6 @@ defmodule Pleroma.Web.ActivityPub.SideEffects do
alias Pleroma.Web.ActivityPub.Builder alias Pleroma.Web.ActivityPub.Builder
alias Pleroma.Web.ActivityPub.Pipeline alias Pleroma.Web.ActivityPub.Pipeline
alias Pleroma.Web.ActivityPub.Utils alias Pleroma.Web.ActivityPub.Utils
alias Pleroma.Web.Push
alias Pleroma.Web.Streamer alias Pleroma.Web.Streamer
alias Pleroma.Workers.PollWorker alias Pleroma.Workers.PollWorker
@ -125,7 +124,7 @@ def handle(
nil nil
end end
{:ok, notifications} = Notification.create_notifications(object, do_send: false) {:ok, notifications} = Notification.create_notifications(object)
meta = meta =
meta meta
@ -184,7 +183,11 @@ def handle(%{data: %{"type" => "Like"}} = object, meta) do
liked_object = Object.get_by_ap_id(object.data["object"]) liked_object = Object.get_by_ap_id(object.data["object"])
Utils.add_like_to_object(object, liked_object) Utils.add_like_to_object(object, liked_object)
Notification.create_notifications(object) {:ok, notifications} = Notification.create_notifications(object)
meta =
meta
|> add_notifications(notifications)
{:ok, object, meta} {:ok, object, meta}
end end
@ -202,7 +205,7 @@ def handle(%{data: %{"type" => "Like"}} = object, meta) do
def handle(%{data: %{"type" => "Create"}} = activity, meta) do def handle(%{data: %{"type" => "Create"}} = activity, meta) do
with {:ok, object, meta} <- handle_object_creation(meta[:object_data], activity, meta), with {:ok, object, meta} <- handle_object_creation(meta[:object_data], activity, meta),
%User{} = user <- User.get_cached_by_ap_id(activity.data["actor"]) do %User{} = user <- User.get_cached_by_ap_id(activity.data["actor"]) do
{:ok, notifications} = Notification.create_notifications(activity, do_send: false) {:ok, notifications} = Notification.create_notifications(activity)
{:ok, _user} = ActivityPub.increase_note_count_if_public(user, object) {:ok, _user} = ActivityPub.increase_note_count_if_public(user, object)
{:ok, _user} = ActivityPub.update_last_status_at_if_public(user, object) {:ok, _user} = ActivityPub.update_last_status_at_if_public(user, object)
@ -256,11 +259,13 @@ def handle(%{data: %{"type" => "Announce"}} = object, meta) do
Utils.add_announce_to_object(object, announced_object) Utils.add_announce_to_object(object, announced_object)
if !User.internal?(user) do {:ok, notifications} = Notification.create_notifications(object)
Notification.create_notifications(object)
ap_streamer().stream_out(object) if !User.internal?(user), do: ap_streamer().stream_out(object)
end
meta =
meta
|> add_notifications(notifications)
{:ok, object, meta} {:ok, object, meta}
end end
@ -281,7 +286,11 @@ def handle(%{data: %{"type" => "EmojiReact"}} = object, meta) do
reacted_object = Object.get_by_ap_id(object.data["object"]) reacted_object = Object.get_by_ap_id(object.data["object"])
Utils.add_emoji_reaction_to_object(object, reacted_object) Utils.add_emoji_reaction_to_object(object, reacted_object)
Notification.create_notifications(object) {:ok, notifications} = Notification.create_notifications(object)
meta =
meta
|> add_notifications(notifications)
{:ok, object, meta} {:ok, object, meta}
end end
@ -585,10 +594,7 @@ defp delete_object(object) do
defp send_notifications(meta) do defp send_notifications(meta) do
Keyword.get(meta, :notifications, []) Keyword.get(meta, :notifications, [])
|> Enum.each(fn notification -> |> Notification.send()
Streamer.stream(["user", "user:notification"], notification)
Push.send(notification)
end)
meta meta
end end

View File

@ -721,14 +721,18 @@ def make_listen_data(params, additional) do
#### Flag-related helpers #### Flag-related helpers
@spec make_flag_data(map(), map()) :: map() @spec make_flag_data(map(), map()) :: map()
def make_flag_data(%{actor: actor, context: context, content: content} = params, additional) do def make_flag_data(
%{actor: actor, context: context, content: content} = params,
additional
) do
%{ %{
"type" => "Flag", "type" => "Flag",
"actor" => actor.ap_id, "actor" => actor.ap_id,
"content" => content, "content" => content,
"object" => build_flag_object(params), "object" => build_flag_object(params),
"context" => context, "context" => context,
"state" => "open" "state" => "open",
"rules" => Map.get(params, :rules, nil)
} }
|> Map.merge(additional) |> Map.merge(additional)
end end

View File

@ -0,0 +1,62 @@
# Pleroma: A lightweight social networking server
# Copyright © 2017-2022 Pleroma Authors <https://pleroma.social/>
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.Web.AdminAPI.RuleController do
use Pleroma.Web, :controller
alias Pleroma.Repo
alias Pleroma.Rule
alias Pleroma.Web.Plugs.OAuthScopesPlug
import Pleroma.Web.ControllerHelper,
only: [
json_response: 3
]
plug(Pleroma.Web.ApiSpec.CastAndValidate)
plug(
OAuthScopesPlug,
%{scopes: ["admin:write"]}
when action in [:create, :update, :delete]
)
plug(OAuthScopesPlug, %{scopes: ["admin:read"]} when action == :index)
action_fallback(AdminAPI.FallbackController)
defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.Admin.RuleOperation
def index(conn, _) do
rules =
Rule.query()
|> Repo.all()
render(conn, "index.json", rules: rules)
end
def create(%{body_params: params} = conn, _) do
rule =
params
|> Rule.create()
render(conn, "show.json", rule: rule)
end
def update(%{body_params: params} = conn, %{id: id}) do
rule =
params
|> Rule.update(id)
render(conn, "show.json", rule: rule)
end
def delete(conn, %{id: id}) do
with {:ok, _} <- Rule.delete(id) do
json(conn, %{})
else
_ -> json_response(conn, :bad_request, "")
end
end
end

View File

@ -6,9 +6,11 @@ defmodule Pleroma.Web.AdminAPI.ReportView do
use Pleroma.Web, :view use Pleroma.Web, :view
alias Pleroma.HTML alias Pleroma.HTML
alias Pleroma.Rule
alias Pleroma.User alias Pleroma.User
alias Pleroma.Web.AdminAPI alias Pleroma.Web.AdminAPI
alias Pleroma.Web.AdminAPI.Report alias Pleroma.Web.AdminAPI.Report
alias Pleroma.Web.AdminAPI.RuleView
alias Pleroma.Web.CommonAPI.Utils alias Pleroma.Web.CommonAPI.Utils
alias Pleroma.Web.MastodonAPI.StatusView alias Pleroma.Web.MastodonAPI.StatusView
@ -46,7 +48,8 @@ def render("show.json", %{report: report, user: user, account: account, statuses
as: :activity as: :activity
}), }),
state: report.data["state"], state: report.data["state"],
notes: render(__MODULE__, "index_notes.json", %{notes: report.report_notes}) notes: render(__MODULE__, "index_notes.json", %{notes: report.report_notes}),
rules: rules(Map.get(report.data, "rules", nil))
} }
end end
@ -71,4 +74,16 @@ def render("show_note.json", %{
created_at: Utils.to_masto_date(inserted_at) created_at: Utils.to_masto_date(inserted_at)
} }
end end
defp rules(nil) do
[]
end
defp rules(rule_ids) do
rules =
rule_ids
|> Rule.get()
render(RuleView, "index.json", rules: rules)
end
end end

View File

@ -0,0 +1,22 @@
# Pleroma: A lightweight social networking server
# Copyright © 2017-2022 Pleroma Authors <https://pleroma.social/>
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.Web.AdminAPI.RuleView do
use Pleroma.Web, :view
require Pleroma.Constants
def render("index.json", %{rules: rules} = _opts) do
render_many(rules, __MODULE__, "show.json")
end
def render("show.json", %{rule: rule} = _opts) do
%{
id: to_string(rule.id),
priority: rule.priority,
text: rule.text,
hint: rule.hint
}
end
end

View File

@ -97,6 +97,7 @@ def spec(opts \\ []) do
"Frontend management", "Frontend management",
"Instance configuration", "Instance configuration",
"Instance documents", "Instance documents",
"Instance rule managment",
"Invites", "Invites",
"MediaProxy cache", "MediaProxy cache",
"OAuth application management", "OAuth application management",

View File

@ -11,6 +11,7 @@ defmodule Pleroma.Web.ApiSpec.AccountOperation do
alias Pleroma.Web.ApiSpec.Schemas.ActorType alias Pleroma.Web.ApiSpec.Schemas.ActorType
alias Pleroma.Web.ApiSpec.Schemas.ApiError alias Pleroma.Web.ApiSpec.Schemas.ApiError
alias Pleroma.Web.ApiSpec.Schemas.BooleanLike alias Pleroma.Web.ApiSpec.Schemas.BooleanLike
alias Pleroma.Web.ApiSpec.Schemas.FlakeID
alias Pleroma.Web.ApiSpec.Schemas.List alias Pleroma.Web.ApiSpec.Schemas.List
alias Pleroma.Web.ApiSpec.Schemas.Status alias Pleroma.Web.ApiSpec.Schemas.Status
alias Pleroma.Web.ApiSpec.Schemas.VisibilityScope alias Pleroma.Web.ApiSpec.Schemas.VisibilityScope
@ -513,6 +514,48 @@ def identity_proofs_operation do
} }
end end
def familiar_followers_operation do
%Operation{
tags: ["Retrieve account information"],
summary: "Followers that you follow",
operationId: "AccountController.familiar_followers",
description:
"Obtain a list of all accounts that follow a given account, filtered for accounts you follow.",
security: [%{"oAuth" => ["read:follows"]}],
parameters: [
Operation.parameter(
:id,
:query,
%Schema{
oneOf: [%Schema{type: :array, items: %Schema{type: :string}}, %Schema{type: :string}]
},
"Account IDs",
example: "123"
)
],
responses: %{
200 =>
Operation.response("Accounts", "application/json", %Schema{
title: "ArrayOfAccounts",
type: :array,
items: %Schema{
title: "Account",
type: :object,
properties: %{
id: FlakeID,
accounts: %Schema{
title: "ArrayOfAccounts",
type: :array,
items: Account,
example: [Account.schema().example]
}
}
}
})
}
}
end
defp create_request do defp create_request do
%Schema{ %Schema{
title: "AccountCreateRequest", title: "AccountCreateRequest",

View File

@ -30,6 +30,12 @@ def index_operation do
report_state(), report_state(),
"Filter by report state" "Filter by report state"
), ),
Operation.parameter(
:rule_id,
:query,
%Schema{type: :string},
"Filter by selected rule id"
),
Operation.parameter( Operation.parameter(
:limit, :limit,
:query, :query,
@ -169,6 +175,17 @@ defp report do
inserted_at: %Schema{type: :string, format: :"date-time"} inserted_at: %Schema{type: :string, format: :"date-time"}
} }
} }
},
rules: %Schema{
type: :array,
items: %Schema{
type: :object,
properties: %{
id: %Schema{type: :string},
text: %Schema{type: :string},
hint: %Schema{type: :string, nullable: true}
}
}
} }
} }
} }

View File

@ -0,0 +1,115 @@
# Pleroma: A lightweight social networking server
# Copyright © 2017-2022 Pleroma Authors <https://pleroma.social/>
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.Web.ApiSpec.Admin.RuleOperation do
alias OpenApiSpex.Operation
alias OpenApiSpex.Schema
alias Pleroma.Web.ApiSpec.Schemas.ApiError
import Pleroma.Web.ApiSpec.Helpers
def open_api_operation(action) do
operation = String.to_existing_atom("#{action}_operation")
apply(__MODULE__, operation, [])
end
def index_operation do
%Operation{
tags: ["Instance rule managment"],
summary: "Retrieve list of instance rules",
operationId: "AdminAPI.RuleController.index",
security: [%{"oAuth" => ["admin:read"]}],
responses: %{
200 =>
Operation.response("Response", "application/json", %Schema{
type: :array,
items: rule()
}),
403 => Operation.response("Forbidden", "application/json", ApiError)
}
}
end
def create_operation do
%Operation{
tags: ["Instance rule managment"],
summary: "Create new rule",
operationId: "AdminAPI.RuleController.create",
security: [%{"oAuth" => ["admin:write"]}],
parameters: admin_api_params(),
requestBody: request_body("Parameters", create_request(), required: true),
responses: %{
200 => Operation.response("Response", "application/json", rule()),
400 => Operation.response("Bad Request", "application/json", ApiError),
403 => Operation.response("Forbidden", "application/json", ApiError)
}
}
end
def update_operation do
%Operation{
tags: ["Instance rule managment"],
summary: "Modify existing rule",
operationId: "AdminAPI.RuleController.update",
security: [%{"oAuth" => ["admin:write"]}],
parameters: [Operation.parameter(:id, :path, :string, "Rule ID")],
requestBody: request_body("Parameters", update_request(), required: true),
responses: %{
200 => Operation.response("Response", "application/json", rule()),
400 => Operation.response("Bad Request", "application/json", ApiError),
403 => Operation.response("Forbidden", "application/json", ApiError)
}
}
end
def delete_operation do
%Operation{
tags: ["Instance rule managment"],
summary: "Delete rule",
operationId: "AdminAPI.RuleController.delete",
parameters: [Operation.parameter(:id, :path, :string, "Rule ID")],
security: [%{"oAuth" => ["admin:write"]}],
responses: %{
200 => empty_object_response(),
404 => Operation.response("Not Found", "application/json", ApiError),
403 => Operation.response("Forbidden", "application/json", ApiError)
}
}
end
defp create_request do
%Schema{
type: :object,
required: [:text],
properties: %{
priority: %Schema{type: :integer},
text: %Schema{type: :string},
hint: %Schema{type: :string}
}
}
end
defp update_request do
%Schema{
type: :object,
properties: %{
priority: %Schema{type: :integer},
text: %Schema{type: :string},
hint: %Schema{type: :string}
}
}
end
defp rule do
%Schema{
type: :object,
properties: %{
id: %Schema{type: :string},
priority: %Schema{type: :integer},
text: %Schema{type: :string},
hint: %Schema{type: :string, nullable: true}
}
}
end
end

View File

@ -46,6 +46,17 @@ def peers_operation do
} }
end end
def rules_operation do
%Operation{
tags: ["Instance misc"],
summary: "Retrieve list of instance rules",
operationId: "InstanceController.rules",
responses: %{
200 => Operation.response("Array of domains", "application/json", array_of_rules())
}
}
end
defp instance do defp instance do
%Schema{ %Schema{
type: :object, type: :object,
@ -181,7 +192,8 @@ defp instance do
"urls" => %{ "urls" => %{
"streaming_api" => "wss://lain.com" "streaming_api" => "wss://lain.com"
}, },
"version" => "2.7.2 (compatible; Pleroma 2.0.50-536-g25eec6d7-develop)" "version" => "2.7.2 (compatible; Pleroma 2.0.50-536-g25eec6d7-develop)",
"rules" => array_of_rules()
} }
} }
end end
@ -371,4 +383,18 @@ defp array_of_domains do
example: ["pleroma.site", "lain.com", "bikeshed.party"] example: ["pleroma.site", "lain.com", "bikeshed.party"]
} }
end end
defp array_of_rules do
%Schema{
type: :array,
items: %Schema{
type: :object,
properties: %{
id: %Schema{type: :string},
text: %Schema{type: :string},
hint: %Schema{type: :string}
}
}
}
end
end end

View File

@ -202,7 +202,8 @@ defp notification_type do
"pleroma:report", "pleroma:report",
"move", "move",
"follow_request", "follow_request",
"poll" "poll",
"status"
], ],
description: """ description: """
The type of event that resulted in the notification. The type of event that resulted in the notification.
@ -216,6 +217,7 @@ defp notification_type do
- `pleroma:emoji_reaction` - Someone reacted with emoji to your status - `pleroma:emoji_reaction` - Someone reacted with emoji to your status
- `pleroma:chat_mention` - Someone mentioned you in a chat message - `pleroma:chat_mention` - Someone mentioned you in a chat message
- `pleroma:report` - Someone was reported - `pleroma:report` - Someone was reported
- `status` - Someone you are subscribed to created a status
""" """
} }
end end

View File

@ -5,7 +5,6 @@
defmodule Pleroma.Web.ApiSpec.PleromaNotificationOperation do defmodule Pleroma.Web.ApiSpec.PleromaNotificationOperation do
alias OpenApiSpex.Operation alias OpenApiSpex.Operation
alias OpenApiSpex.Schema alias OpenApiSpex.Schema
alias Pleroma.Web.ApiSpec.NotificationOperation
alias Pleroma.Web.ApiSpec.Schemas.ApiError alias Pleroma.Web.ApiSpec.Schemas.ApiError
import Pleroma.Web.ApiSpec.Helpers import Pleroma.Web.ApiSpec.Helpers
@ -35,12 +34,7 @@ def mark_as_read_operation do
Operation.response( Operation.response(
"A Notification or array of Notifications", "A Notification or array of Notifications",
"application/json", "application/json",
%Schema{ %Schema{type: :string}
anyOf: [
%Schema{type: :array, items: NotificationOperation.notification()},
NotificationOperation.notification()
]
}
), ),
400 => Operation.response("Bad Request", "application/json", ApiError) 400 => Operation.response("Bad Request", "application/json", ApiError)
} }

View File

@ -53,6 +53,12 @@ defp create_request do
default: false, default: false,
description: description:
"If the account is remote, should the report be forwarded to the remote admin?" "If the account is remote, should the report be forwarded to the remote admin?"
},
rule_ids: %Schema{
type: :array,
nullable: true,
items: %Schema{type: :string},
description: "Array of rules"
} }
}, },
required: [:account_id], required: [:account_id],
@ -60,7 +66,8 @@ defp create_request do
"account_id" => "123", "account_id" => "123",
"status_ids" => ["1337"], "status_ids" => ["1337"],
"comment" => "bad status!", "comment" => "bad status!",
"forward" => "false" "forward" => "false",
"rule_ids" => ["3"]
} }
} }
end end

View File

@ -50,7 +50,11 @@ defmodule Pleroma.Web.ApiSpec.Schemas.Attachment do
pleroma: %Schema{ pleroma: %Schema{
type: :object, type: :object,
properties: %{ properties: %{
mime_type: %Schema{type: :string, description: "mime type of the attachment"} mime_type: %Schema{type: :string, description: "mime type of the attachment"},
name: %Schema{
type: :string,
description: "Name of the attachment, typically the filename"
}
} }
} }
}, },

Some files were not shown because too many files have changed in this diff Show More