moon
/
spc-pleroma
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
16,005 Commits 2 Branches 0 Tags 208 MiB
Commit Graph

526 Commits

Author SHA1 Message Date
Mark Felder 19002fd6c1 Mastodon API: Remove deprecated GET /api/v1/statuses/:id/card endpoint 
Removed back in 2019

https://github.com/mastodon/mastodon/pull/11213
 2024-05-08 01:44:58 +00:00
Mark Felder ede414094f RichMedia refactor 
Rich Media parsing was previously handled on-demand with a 2 second HTTP request timeout and retained only in Cachex. Every time a Pleroma instance is restarted it will have to request and parse the data for each status with a URL detected. When fetching a batch of statuses they were processed in parallel to attempt to keep the maximum latency at 2 seconds, but often resulted in a timeline appearing to hang during loading due to a URL that could not be successfully reached. URLs which had images links that expire (Amazon AWS) were parsed and inserted with a TTL to ensure the image link would not break.

Rich Media data is now cached in the database and fetched asynchronously. Cachex is used as a read-through cache. When the data becomes available we stream an update to the clients. If the result is returned quickly the experience is almost seamless. Activities were already processed for their Rich Media data during ingestion to warm the cache, so users should not normally encounter the asynchronous loading of the Rich Media data.

Implementation notes:

- The async worker is a Task with a globally unique process name to prevent duplicate processing of the same URL
- The Task will attempt to fetch the data 3 times with increasing sleep time between attempts
- The HTTP request obeys the default HTTP request timeout value instead of 2 seconds
- URLs that cannot be successfully parsed due to an unexpected error receives a negative cache entry for 15 minutes
- URLs that fail with an expected error will receive a negative cache with no TTL
- Activities that have no detected URLs insert a nil value in the Cachex :scrubber_cache so we do not repeat parsing the object content with Floki every time the activity is rendered
- Expiring image URLs are handled with an Oban job
- There is no automatic cleanup of the Rich Media data in the database, but it is safe to delete at any time
- The post draft/preview feature makes the URL processing synchronous so the rendered post preview will have an accurate rendering

Overall performance of timelines and creating new posts which contain URLs is greatly improved.
 2024-05-07 19:54:56 -04:00
marcin mikołajczak d415686bb9 Allow to group bookmarks in folders 
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
 2024-03-01 11:04:01 +01:00
Mark Felder ed2f8e45e9 Pleroma.Web.MastodonAPI.SearchController: fix dialyzer errors 
Add a separate Pagination.paginate_list/2 function instead of overloading paginate/4 and complicating its matching and @spec
 2024-01-31 11:12:41 -05:00
Mark Felder b8db67dafa Pleroma.Web.MastodonAPI.StatusController: fix dialzyer errors with replace_params: false 2024-01-31 10:13:02 -05:00
Mark Felder fdddba100e Pleroma.Web.MastodonAPI.SearchController: fix dialyzer errors with replace_params: false 2024-01-30 18:40:46 -05:00
Mark Felder 324fd08455 Pleroma.Web.MastodonAPI.ScheduledActivityController: fix dialyzer errors with replace_params: false 2024-01-30 18:33:23 -05:00
Mark Felder d3c6acd2f2 Pleroma.Web.MastodonAPI.PollController: fix dialyzer errors with replace_params: false 2024-01-30 18:31:13 -05:00
Mark Felder 0a6b2c9587 Pleroma.Web.MastodonAPI.NotificationController: fix dialyzer errors with replace_params: false 2024-01-30 18:27:31 -05:00
Mark Felder 90c9f38f40 Pleroma.Web.MastodonAPI.MediaController: fix dialyzer errors with replace_params: false 2024-01-30 18:09:11 -05:00
Mark Felder 9741f045e4 Pleroma.Web.MastodonAPI.ListController: fix dialyzer errors with replace_params: false 2024-01-30 18:05:28 -05:00
Mark Felder 14de8376ad Pleroma.Web.MastodonAPI.FollowRequestController: fix dialyzer errors with replace_params: false 2024-01-30 17:56:43 -05:00
Mark Felder 81c8592d69 Pleroma.Web.MastodonAPI.AccountController: fix dialyzer errors with replace_params: false 2024-01-30 16:50:00 -05:00
Mark Felder cfe7438b2f Pleroma.Web.MastodonAPI.DomainBlockController: dialyzer fixes via :replace_params 2024-01-30 14:52:15 -05:00
Mark Felder 1fa1a93cd6 Revert "Pleroma.Web.MastodonAPI.AccountController: dialyzer errors" 
This reverts commit e53c20b03c.
 2024-01-30 14:13:46 -05:00
Mark Felder 41493bd642 Revert "Pleroma.Web.MastodonAPI.DomainBlockController: dialyzer errors" 
This reverts commit 378edeaf15.
 2024-01-30 14:13:23 -05:00
Mark Felder a3426fcaf3 Revert "Pleroma.Web.MastodonAPI.MediaController: dialyzer errors" 
This reverts commit 8cd5279857.
 2024-01-30 14:13:13 -05:00
Mark Felder 8cd5279857 Pleroma.Web.MastodonAPI.MediaController: dialyzer errors 
lib/pleroma/web/mastodon_api/controllers/media_controller.ex:32:call
The function call will not succeed.

Phoenix.Controller.render(
  _conn :: %{
    :assigns => %{:user => _, _ => _},
    :body_params => %{:file => _, _ => _},
    _ => _
  },
  <<97, 116, 116, 97, 99, 104, 109, 101, 110, 116, 46, 106, 115, 111, 110>>,
  %{:attachment => map()}
)

will never return since the success typing is:
(
  %Plug.Conn{
    :adapter => {atom(), _},
    :assigns => %{atom() => _},
    :body_params => %Plug.Conn.Unfetched{:aspect => atom(), binary() => _},
    :cookies => %Plug.Conn.Unfetched{:aspect => atom(), binary() => _},
    :halted => boolean(),
    :host => binary(),
    :method => binary(),
    :owner => pid(),
    :params => %Plug.Conn.Unfetched{:aspect => atom(), binary() => _},
    :path_info => [binary()],
    :path_params => %{binary() => binary() | [any()] | map()},
    :port => char(),
    :private => %{atom() => _},
    :query_params => %Plug.Conn.Unfetched{
      :aspect => atom(),
      binary() => binary() | [any()] | map()
    },
    :query_string => binary(),
    :remote_ip =>
      {byte(), byte(), byte(), byte()}
      | {char(), char(), char(), char(), char(), char(), char(), char()},
    :req_cookies => %Plug.Conn.Unfetched{:aspect => atom(), binary() => binary()},
    :req_headers => [{_, _}],
    :request_path => binary(),
    :resp_body =>
      nil
      | binary()
      | maybe_improper_list(
          binary() | maybe_improper_list(any(), binary() | []) | byte(),
          binary() | []
        ),
    :resp_cookies => %{binary() => map()},
    :resp_headers => [{_, _}],
    :scheme => :http | :https,
    :script_name => [binary()],
    :secret_key_base => nil | binary(),
    :state =>
      :chunked | :file | :sent | :set | :set_chunked | :set_file | :unset | :upgraded,
    :status => nil | non_neg_integer()
  },
  atom() | binary(),
  atom() | binary() | [{_, _}] | map()
) :: %Plug.Conn{
  :adapter => {atom(), _},
  :assigns => %{atom() => _},
  :body_params => %Plug.Conn.Unfetched{:aspect => atom(), binary() => _},
  :cookies => %Plug.Conn.Unfetched{:aspect => atom(), binary() => _},
  :halted => boolean(),
  :host => binary(),
  :method => binary(),
  :owner => pid(),
  :params => %Plug.Conn.Unfetched{:aspect => atom(), binary() => _},
  :path_info => [binary()],
  :path_params => %{binary() => binary() | [any()] | map()},
  :port => char(),
  :private => %{atom() => _},
  :query_params => %Plug.Conn.Unfetched{
    :aspect => atom(),
    binary() => binary() | [any()] | map()
  },
  :query_string => binary(),
  :remote_ip =>
    {byte(), byte(), byte(), byte()}
    | {char(), char(), char(), char(), char(), char(), char(), char()},
  :req_cookies => %Plug.Conn.Unfetched{:aspect => atom(), binary() => binary()},
  :req_headers => [{_, _}],
  :request_path => binary(),
  :resp_body =>
    nil
    | binary()
    | maybe_improper_list(
        binary() | maybe_improper_list(any(), binary() | []) | byte(),
        binary() | []
      ),
  :resp_cookies => %{binary() => map()},
  :resp_headers => [{_, _}],
  :scheme => :http | :https,
  :script_name => [binary()],
  :secret_key_base => nil | binary(),
  :state => :sent,
  :status => nil | non_neg_integer()
}

and the contract is
(Plug.Conn.t(), binary() | atom(), Keyword.t() | map()) :: Plug.Conn.t()

________________________________________________________________________________
lib/pleroma/web/mastodon_api/controllers/media_controller.ex:48:call
The function call will not succeed.

Plug.Conn.put_status(
  _conn :: %{
    :assigns => %{:user => _, _ => _},
    :body_params => %{:file => _, _ => _},
    _ => _
  },
  202
)

breaks the contract
(t(), status()) :: t()

________________________________________________________________________________
lib/pleroma/web/mastodon_api/controllers/media_controller.ex:63:call
The function call will not succeed.

Phoenix.Controller.render(
  _conn :: %{
    :assigns => %{:user => _, _ => _},
    :body_params => %{:description => _, _ => _},
    _ => _
  },
  <<97, 116, 116, 97, 99, 104, 109, 101, 110, 116, 46, 106, 115, 111, 110>>,
  %{:attachment => map()}
)

will never return since the success typing is:
(
  %Plug.Conn{
    :adapter => {atom(), _},
    :assigns => %{atom() => _},
    :body_params => %Plug.Conn.Unfetched{:aspect => atom(), binary() => _},
    :cookies => %Plug.Conn.Unfetched{:aspect => atom(), binary() => _},
    :halted => boolean(),
    :host => binary(),
    :method => binary(),
    :owner => pid(),
    :params => %Plug.Conn.Unfetched{:aspect => atom(), binary() => _},
    :path_info => [binary()],
    :path_params => %{binary() => binary() | [any()] | map()},
    :port => char(),
    :private => %{atom() => _},
    :query_params => %Plug.Conn.Unfetched{
      :aspect => atom(),
      binary() => binary() | [any()] | map()
    },
    :query_string => binary(),
    :remote_ip =>
      {byte(), byte(), byte(), byte()}
      | {char(), char(), char(), char(), char(), char(), char(), char()},
    :req_cookies => %Plug.Conn.Unfetched{:aspect => atom(), binary() => binary()},
    :req_headers => [{_, _}],
    :request_path => binary(),
    :resp_body =>
      nil
      | binary()
      | maybe_improper_list(
          binary() | maybe_improper_list(any(), binary() | []) | byte(),
          binary() | []
        ),
    :resp_cookies => %{binary() => map()},
    :resp_headers => [{_, _}],
    :scheme => :http | :https,
    :script_name => [binary()],
    :secret_key_base => nil | binary(),
    :state =>
      :chunked | :file | :sent | :set | :set_chunked | :set_file | :unset | :upgraded,
    :status => nil | non_neg_integer()
  },
  atom() | binary(),
  atom() | binary() | [{_, _}] | map()
) :: %Plug.Conn{
  :adapter => {atom(), _},
  :assigns => %{atom() => _},
  :body_params => %Plug.Conn.Unfetched{:aspect => atom(), binary() => _},
  :cookies => %Plug.Conn.Unfetched{:aspect => atom(), binary() => _},
  :halted => boolean(),
  :host => binary(),
  :method => binary(),
  :owner => pid(),
  :params => %Plug.Conn.Unfetched{:aspect => atom(), binary() => _},
  :path_info => [binary()],
  :path_params => %{binary() => binary() | [any()] | map()},
  :port => char(),
  :private => %{atom() => _},
  :query_params => %Plug.Conn.Unfetched{
    :aspect => atom(),
    binary() => binary() | [any()] | map()
  },
  :query_string => binary(),
  :remote_ip =>
    {byte(), byte(), byte(), byte()}
    | {char(), char(), char(), char(), char(), char(), char(), char()},
  :req_cookies => %Plug.Conn.Unfetched{:aspect => atom(), binary() => binary()},
  :req_headers => [{_, _}],
  :request_path => binary(),
  :resp_body =>
    nil
    | binary()
    | maybe_improper_list(
        binary() | maybe_improper_list(any(), binary() | []) | byte(),
        binary() | []
      ),
  :resp_cookies => %{binary() => map()},
  :resp_headers => [{_, _}],
  :scheme => :http | :https,
  :script_name => [binary()],
  :secret_key_base => nil | binary(),
  :state => :sent,
  :status => nil | non_neg_integer()
}

and the contract is
(Plug.Conn.t(), binary() | atom(), Keyword.t() | map()) :: Plug.Conn.t()
 2024-01-29 18:36:46 -05:00
Mark Felder 378edeaf15 Pleroma.Web.MastodonAPI.DomainBlockController: dialyzer errors 
lib/pleroma/web/mastodon_api/controllers/domain_block_controller.ex:32:call
The function call will not succeed.

Phoenix.Controller.json(
  _conn :: %{
    :assigns => %{:user => _, _ => _},
    :body_params => %{:domain => _, _ => _},
    _ => _
  },
  %{}
)

breaks the contract
(Plug.Conn.t(), term()) :: Plug.Conn.t()

________________________________________________________________________________
lib/pleroma/web/mastodon_api/controllers/domain_block_controller.ex:43:call
The function call will not succeed.

Phoenix.Controller.json(
  _conn :: %{
    :assigns => %{:user => _, _ => _},
    :body_params => %{:domain => _, _ => _},
    _ => _
  },
  %{}
)

breaks the contract
(Plug.Conn.t(), term()) :: Plug.Conn.t()
 2024-01-29 18:36:34 -05:00
Mark Felder 6a22a80f9f Pleroma.Web.MastodonAPI.DirectoryController: dialyzer errors 
lib/pleroma/web/mastodon_api/controllers/directory_controller.ex:6:unused_fun
Function skip_auth/2 will never be called.
________________________________________________________________________________
lib/pleroma/web/mastodon_api/controllers/directory_controller.ex:6:unused_fun
Function skip_plug/2 will never be called.
________________________________________________________________________________
lib/pleroma/web/mastodon_api/controllers/directory_controller.ex:18:guard_fail
The guard clause:

when _action :: atom() == <<105, 110, 100, 101, 120>>

can never succeed.
 2024-01-28 17:41:07 -05:00
Mark Felder e53c20b03c Pleroma.Web.MastodonAPI.AccountController: dialyzer errors 
lib/pleroma/web/mastodon_api/controllers/account_controller.ex:479:call
The function call will not succeed.

Phoenix.Controller.render(
  _conn :: %{
    :assigns => %{:account => _, :user => _, _ => _},
    :body_params => %{:comment => _, _ => _},
    _ => _
  },
  <<114, 101, 108, 97, 116, 105, 111, 110, 115, 104, 105, 112, 46, 106, 115, 111, 110>>,
  [
    {:target, %Pleroma.User{:id => _, _ => _}} | {:user, %Pleroma.User{:id => _, _ => _}},
    ...
  ]
)

will never return since the success typing is:
(
  %Plug.Conn{
    :adapter => {atom(), _},
    :assigns => %{atom() => _},
    :body_params => %Plug.Conn.Unfetched{:aspect => atom(), binary() => _},
    :cookies => %Plug.Conn.Unfetched{:aspect => atom(), binary() => _},
    :halted => boolean(),
    :host => binary(),
    :method => binary(),
    :owner => pid(),
    :params => %Plug.Conn.Unfetched{:aspect => atom(), binary() => _},
    :path_info => [binary()],
    :path_params => %{binary() => binary() | [any()] | map()},
    :port => char(),
    :private => %{atom() => _},
    :query_params => %Plug.Conn.Unfetched{
      :aspect => atom(),
      binary() => binary() | [any()] | map()
    },
    :query_string => binary(),
    :remote_ip =>
      {byte(), byte(), byte(), byte()}
      | {char(), char(), char(), char(), char(), char(), char(), char()},
    :req_cookies => %Plug.Conn.Unfetched{:aspect => atom(), binary() => binary()},
    :req_headers => [{_, _}],
    :request_path => binary(),
    :resp_body =>
      nil
      | binary()
      | maybe_improper_list(
          binary() | maybe_improper_list(any(), binary() | []) | byte(),
          binary() | []
        ),
    :resp_cookies => %{binary() => map()},
    :resp_headers => [{_, _}],
    :scheme => :http | :https,
    :script_name => [binary()],
    :secret_key_base => nil | binary(),
    :state =>
      :chunked | :file | :sent | :set | :set_chunked | :set_file | :unset | :upgraded,
    :status => nil | non_neg_integer()
  },
  atom() | binary(),
  atom() | binary() | [{_, _}] | map()
) :: %Plug.Conn{
  :adapter => {atom(), _},
  :assigns => %{atom() => _},
  :body_params => %Plug.Conn.Unfetched{:aspect => atom(), binary() => _},
  :cookies => %Plug.Conn.Unfetched{:aspect => atom(), binary() => _},
  :halted => boolean(),
  :host => binary(),
  :method => binary(),
  :owner => pid(),
  :params => %Plug.Conn.Unfetched{:aspect => atom(), binary() => _},
  :path_info => [binary()],
  :path_params => %{binary() => binary() | [any()] | map()},
  :port => char(),
  :private => %{atom() => _},
  :query_params => %Plug.Conn.Unfetched{
    :aspect => atom(),
    binary() => binary() | [any()] | map()
  },
  :query_string => binary(),
  :remote_ip =>
    {byte(), byte(), byte(), byte()}
    | {char(), char(), char(), char(), char(), char(), char(), char()},
  :req_cookies => %Plug.Conn.Unfetched{:aspect => atom(), binary() => binary()},
  :req_headers => [{_, _}],
  :request_path => binary(),
  :resp_body =>
    nil
    | binary()
    | maybe_improper_list(
        binary() | maybe_improper_list(any(), binary() | []) | byte(),
        binary() | []
      ),
  :resp_cookies => %{binary() => map()},
  :resp_headers => [{_, _}],
  :scheme => :http | :https,
  :script_name => [binary()],
  :secret_key_base => nil | binary(),
  :state => :sent,
  :status => nil | non_neg_integer()
}

and the contract is
(Plug.Conn.t(), binary() | atom(), Keyword.t() | map()) :: Plug.Conn.t()

________________________________________________________________________________
lib/pleroma/web/mastodon_api/controllers/account_controller.ex:519:call
The function call will not succeed.

Plug.Conn.assign(
  _conn :: %{:body_params => %{:uri => _, _ => _}, _ => _},
  :account,
  _user :: %Pleroma.User{_ => _}
)

breaks the contract
(t(), atom(), term()) :: t()
 2024-01-28 17:41:07 -05:00
marcin mikołajczak 017e35fbf1 Fix some more typos 
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
 2023-12-28 00:17:04 +01:00
Haelwenn bd50892c25 Merge branch 'instance-v2' into 'develop' 
Implement api/v2/instance route

See merge request pleroma/pleroma!3939
 2023-12-21 21:14:46 +00:00
Lain Soykaf 0c5cc51983 Merge branch 'develop' of git.pleroma.social:pleroma/pleroma into pleroma-meilisearch 2023-11-12 13:53:18 +04:00
marcin mikołajczak c6cedbb810 InstanceV2: skip auth 
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
 2023-10-28 00:07:18 +02:00
marcin mikołajczak 9effa24f30 Implement api/v2/instance route 
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
 2023-08-11 00:08:05 +02:00
tusooa 2c66f584b5
Show more informative errors when profile exceeds char limits 2023-05-25 08:22:33 -04:00
tusooa d83f16fe44
Allow with_relationships param for blocks 2023-02-28 22:16:01 -05:00
tusooa 97f947deaf
Fix tests 2023-01-15 21:39:05 -05:00
lain e853cfe7c3 Revert "Merge branch 'copyright-bump' into 'develop'" 
This reverts merge request !3825
 2023-01-02 20:38:50 +00:00
marcin mikołajczak 10886eeaa2 Bump copyright year 
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
 2023-01-01 12:13:06 +01:00
Ekaterina Vaartis 398141da68 Merge remote-tracking branch 'upstream/develop' into meilisearch 2022-12-20 21:00:07 +03:00
duponin 9876742358 Return 413 when an actor's banner or background exceeds the size limit 2022-12-11 23:15:08 +01:00
duponin 452595baed Uploading an avatar media exceeding max size returns a 413 
Until now it was returning a 500 because the upload plug were going
through the changeset and ending in the JSON encoder, which raised
because struct has to @derive the encoder.
 2022-12-11 22:54:47 +01:00
Ekaterina Vaartis a6946048fb Rename Activity.Search to Search.DatabaseSearch 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 0fae71f88d Rename search.ex to database_search.ex and add search/2 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis e154ebbf79 Initial meilisearch implementation, doesn't delete posts yet 2022-10-10 20:19:08 +03:00
tusooa 1a7107f4a5 Merge branch 'remove_from_followers' into 'develop' 
MastoAPI: POST /api/v1/accounts/:id/remove_from_followers

See merge request pleroma/pleroma!3647
 2022-09-16 23:24:13 +00:00
Tusooa Zhu ea60c4e709
Fix wrong relationship direction 2022-09-14 20:24:04 -04:00
Haelwenn e40c221c31 Merge branch 'from/upstream-develop/tusooa/edits' into 'develop' 
Editing

Closes #1429, #2859, and #2288

See merge request pleroma/pleroma!3678
 2022-09-03 02:16:42 +00:00
Haelwenn b2ba307f4d Merge branch 'from/upstream-develop/tusooa/2871-fix-local-public' into 'develop' 
local only fixes

Closes #2871

See merge request pleroma/pleroma!3660
 2022-08-02 05:39:50 +00:00
marcin mikołajczak 5d3d6a58f7 Use `duration` param for mute expiration duration 
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
 2022-07-31 17:22:34 +02:00
Haelwenn 0f9f3d2897 Merge branch 'from/upstream-develop/tusooa/2384-pagination' into 'develop' 
Make mutes and blocks behave the same as other lists

Closes #2384

See merge request pleroma/pleroma!3693
 2022-07-28 04:37:10 +00:00
tusooa 301ce5bc62 Merge branch 'mute-expiration' into 'develop' 
MastoAPI: Show mutes expiration date

See merge request pleroma/pleroma!3682
 2022-07-23 00:34:15 +00:00
tusooa 8aba7c08d1 Merge branch 'notification_types' into 'develop' 
MastoAPI: Use `types` for filtering notifications

See merge request pleroma/pleroma!3648
 2022-07-17 03:17:43 +00:00
Tusooa Zhu c1874bc8f9
Make mutes and blocks behave the same as other lists 2022-07-12 19:03:18 -04:00
Tusooa Zhu 4edc867b87 Merge branch 'develop' into 'from/upstream-develop/tusooa/edits' 
# Conflicts:
#   lib/pleroma/constants.ex
 2022-07-03 22:24:57 +00:00
Haelwenn 6b937d1473 Merge branch 'from/upstream-develop/tusooa/server-announcements' into 'develop' 
Server announcements (1st pass)

See merge request pleroma/pleroma!3643
 2022-07-03 20:58:20 +00:00
marcin mikołajczak b0f83aea29 Store mutes expiration date 
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
 2022-06-16 20:38:37 +02:00
Tusooa Zhu 27f3d802f2
Expose history and source apis to anon users 2022-06-11 10:35:36 -04:00