Merge remote-tracking branch 'remotes/upstream/develop' into 1149-oban-job-queue

CHANGELOG.md

@@ -23,9 +23,12 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Not being able to pin unlisted posts
 - Objects being re-embedded to activities after being updated (e.g faved/reposted). Running 'mix pleroma.database prune_objects' again is advised.
 - Metadata rendering errors resulting in the entire page being inaccessible
+- `federation_incoming_replies_max_depth` option being ignored in certain cases
 - Federation/MediaProxy not working with instances that have wrong certificate order
 - Mastodon API: Handling of search timeouts (`/api/v1/search` and `/api/v2/search`)
 - Mastodon API: Embedded relationships not being properly rendered in the Account entity of Status entity
+- Mastodon API: follower/following counters not being nullified, when `hide_follows`/`hide_followers` is set
+- Mastodon API: `muted` in the Status entity, using author's account to determine if the tread was muted
 - Mastodon API: Add `account_id`, `type`, `offset`, and `limit` to search API (`/api/v1/search` and `/api/v2/search`)
 - Mastodon API, streaming: Fix filtering of notifications based on blocks/mutes/thread mutes
 - ActivityPub C2S: follower/following collection pages being inaccessible even when authentifucated if `hide_followers`/ `hide_follows` was set
@@ -38,11 +41,15 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Invalid SemVer version generation, when the current branch does not have commits ahead of tag/checked out on a tag
 - Pleroma.Upload base_url was not automatically whitelisted by MediaProxy. Now your custom CDN or file hosting will be accessed directly as expected.
 - Report email not being sent to admins when the reporter is a remote user
+- MRF: ensure that subdomain_match calls are case-insensitive
 
 ### Added
+- **Breaking:** MRF describe API, which adds support for exposing configuration information about MRF policies to NodeInfo.
+  Custom modules will need to be updated by adding, at the very least, `def describe, do: {:ok, %{}}` to the MRF policy modules.
 - MRF: Support for priming the mediaproxy cache (`Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy`)
 - MRF: Support for excluding specific domains from Transparency.
 - MRF: Support for filtering posts based on who they mention (`Pleroma.Web.ActivityPub.MRF.MentionPolicy`)
+- MRF: Support for filtering posts based on ActivityStreams vocabulary (`Pleroma.Web.ActivityPub.MRF.VocabularyPolicy`)
 - MRF (Simple Policy): Support for wildcard domains.
 - Support for wildcard domains in user domain blocks setting.
 - Configuration: `quarantined_instances` support wildcard domains.
@@ -65,6 +72,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Added synchronization of following/followers counters for external users
 - Configuration: `enabled` option for `Pleroma.Emails.Mailer`, defaulting to `false`.
 - Configuration: Pleroma.Plugs.RateLimiter `bucket_name`, `params` options.
+- Configuration: `user_bio_length` and `user_name_length` options.
 - Addressable lists
 - Twitter API: added rate limit for `/api/account/password_reset` endpoint.
 - ActivityPub: Add an internal service actor for fetching ActivityPub objects.
@@ -72,6 +80,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Admin API: Endpoint for fetching latest user's statuses
 - Pleroma API: Add `/api/v1/pleroma/accounts/confirmation_resend?email=<email>` for resending account confirmation.
 - Relays: Added a task to list relay subscriptions.
+- Mix Tasks: `mix pleroma.database fix_likes_collections`
+- Federation: Remove `likes` from objects.
 
 ### Changed
 - Configuration: Filter.AnonymizeFilename added ability to retain file extension with custom text
@@ -82,6 +92,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 ### Removed
 - Emoji: Remove longfox emojis.
 - Remove `Reply-To` header from report emails for admins.
+- ActivityPub: The `accept_blocks` configuration setting.
 
 ## [1.0.1] - 2019-07-14
 ### Security

config/config.exs

@@ -253,6 +253,8 @@
   skip_thread_containment: true,
   limit_to_local_content: :unauthenticated,
   dynamic_configuration: false,
+  user_bio_length: 5000,
+  user_name_length: 100,
   external_user_synchronization: true
 
 config :pleroma, :markup,
@@ -302,7 +304,6 @@
   default_mascot: :pleroma_fox_tan
 
 config :pleroma, :activitypub,
-  accept_blocks: true,
   unfollow_blocked: true,
   outgoing_blocks: true,
   follow_handshake_timeout: 500,
@@ -337,6 +338,10 @@
 
 config :pleroma, :mrf_subchain, match_actor: %{}
 
+config :pleroma, :mrf_vocabulary,
+  accept: [],
+  reject: []
+
 config :pleroma, :rich_media,
   enabled: true,
   ignore_hosts: [],

docs/config.md

@@ -18,6 +18,7 @@ Note: `strip_exif` has been replaced by `Pleroma.Upload.Filter.Mogrify`.
 
 ## Pleroma.Uploaders.S3
 * `bucket`: S3 bucket name
+* `bucket_namespace`: S3 bucket namespace
 * `public_endpoint`: S3 endpoint that the user finally accesses(ex. "https://s3.dualstack.ap-northeast-1.amazonaws.com")
 * `truncated_namespace`: If you use S3 compatible service such as Digital Ocean Spaces or CDN, set folder name or "" etc.
 For example, when using CDN to S3 virtual host format, set "".
@@ -102,6 +103,7 @@ config :pleroma, Pleroma.Emails.Mailer,
   * `Pleroma.Web.ActivityPub.MRF.AntiLinkSpamPolicy`: Rejects posts from likely spambots by rejecting posts from new users that contain links.
   * `Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy`: Crawls attachments using their MediaProxy URLs so that the MediaProxy cache is primed.
   * `Pleroma.Web.ActivityPub.MRF.MentionPolicy`: Drops posts mentioning configurable users. (see `:mrf_mention` section)
+  * `Pleroma.Web.ActivityPub.MRF.VocabularyPolicy`: Restricts activities to a configured set of vocabulary. (see `:mrf_vocabulary` section)
 * `public`: Makes the client API in authentificated mode-only except for user-profiles. Useful for disabling the Local Timeline and The Whole Known Network.
 * `quarantined_instances`: List of ActivityPub instances where private(DMs, followers-only) activities will not be send.
 * `managed_config`: Whenether the config for pleroma-fe is configured in this config or in ``static/config.json``
@@ -125,6 +127,8 @@ config :pleroma, Pleroma.Emails.Mailer,
 * `safe_dm_mentions`: If set to true, only mentions at the beginning of a post will be used to address people in direct messages. This is to prevent accidental mentioning of people when talking about them (e.g. "@friend hey i really don't like @enemy"). Default: `false`.
 * `healthcheck`: If set to true, system data will be shown on ``/api/pleroma/healthcheck``.
 * `remote_post_retention_days`: The default amount of days to retain remote posts when pruning the database.
+* `user_bio_length`: A user bio maximum length (default: `5000`)
+* `user_name_length`: A user name maximum length (default: `100`)
 * `skip_thread_containment`: Skip filter out broken threads. The default is `false`.
 * `limit_to_local_content`: Limit unauthenticated users to search for local statutes and users only. Possible values: `:unauthenticated`, `:all` and `false`. The default is `:unauthenticated`.
 * `dynamic_configuration`: Allow transferring configuration to DB with the subsequent customization from Admin api.
@@ -275,6 +279,10 @@ config :pleroma, :mrf_subchain,
 ## :mrf_mention
 * `actors`: A list of actors, for which to drop any posts mentioning.
 
+## :mrf_vocabulary
+* `accept`: A list of ActivityStreams terms to accept.  If empty, all supported messages are accepted.
+* `reject`: A list of ActivityStreams terms to reject.  If empty, no messages are rejected.
+
 ## :media_proxy
 * `enabled`: Enables proxying of remote media to the instance’s proxy
 * `base_url`: The base URL to access a user-uploaded file. Useful when you want to proxy the media files via another host/CDN fronts.
@@ -328,7 +336,6 @@ config :pleroma, Pleroma.Web.Endpoint,
 This will make Pleroma listen on `127.0.0.1` port `8080` and generate urls starting with `https://example.com:2020`
 
 ## :activitypub
-* ``accept_blocks``: Whether to accept incoming block activities from other instances
 * ``unfollow_blocked``: Whether blocks result in people getting unfollowed
 * ``outgoing_blocks``: Whether to federate blocks to other instances
 * ``deny_follow_blocked``: Whether to disallow following an account that has blocked the user in question

lib/mix/tasks/pleroma/database.ex

@@ -36,6 +36,10 @@ defmodule Mix.Tasks.Pleroma.Database do
   ## Remove duplicated items from following and update followers count for all users
 
       mix pleroma.database update_users_following_followers_counts
+
+  ## Fix the pre-existing "likes" collections for all objects
+
+      mix pleroma.database fix_likes_collections
   """
   def run(["remove_embedded_objects" | args]) do
     {options, [], []} =
@@ -125,4 +129,36 @@ def run(["prune_objects" | args]) do
       )
     end
   end
+
+  def run(["fix_likes_collections"]) do
+    import Ecto.Query
+
+    start_pleroma()
+
+    from(object in Object,
+      where: fragment("(?)->>'likes' is not null", object.data),
+      select: %{id: object.id, likes: fragment("(?)->>'likes'", object.data)}
+    )
+    |> Pleroma.RepoStreamer.chunk_stream(100)
+    |> Stream.each(fn objects ->
+      ids =
+        objects
+        |> Enum.filter(fn object -> object.likes |> Jason.decode!() |> is_map() end)
+        |> Enum.map(& &1.id)
+
+      Object
+      |> where([object], object.id in ^ids)
+      |> update([object],
+        set: [
+          data:
+            fragment(
+              "jsonb_set(?, '{likes}', '[]'::jsonb, true)",
+              object.data
+            )
+        ]
+      )
+      |> Repo.update_all([], timeout: :infinity)
+    end)
+    |> Stream.run()
+  end
 end

lib/pleroma/activity.ex

@@ -224,6 +224,29 @@ def get_create_by_object_ap_id(ap_id) when is_binary(ap_id) do
 
   def get_create_by_object_ap_id(_), do: nil
 
+  def create_by_object_ap_id_with_object(ap_ids) when is_list(ap_ids) do
+    from(
+      activity in Activity,
+      where:
+        fragment(
+          "coalesce((?)->'object'->>'id', (?)->>'object') = ANY(?)",
+          activity.data,
+          activity.data,
+          ^ap_ids
+        ),
+      where: fragment("(?)->>'type' = 'Create'", activity.data),
+      inner_join: o in Object,
+      on:
+        fragment(
+          "(?->>'id') = COALESCE(?->'object'->>'id', ?->>'object')",
+          o.data,
+          activity.data,
+          activity.data
+        ),
+      preload: [object: o]
+    )
+  end
+
   def create_by_object_ap_id_with_object(ap_id) when is_binary(ap_id) do
     from(
       activity in Activity,
@@ -263,8 +286,8 @@ defp get_in_reply_to_activity_from_object(%Object{data: %{"inReplyTo" => ap_id}}
 
   defp get_in_reply_to_activity_from_object(_), do: nil
 
-  def get_in_reply_to_activity(%Activity{data: %{"object" => object}}) do
+  def get_in_reply_to_activity(%Activity{} = activity) do
-    get_in_reply_to_activity_from_object(Object.normalize(object))
+    get_in_reply_to_activity_from_object(Object.normalize(activity))
   end
 
   def normalize(obj) when is_map(obj), do: get_by_ap_id_with_object(obj["id"])

lib/pleroma/uploaders/local.ex

@@ -11,7 +11,7 @@ def get_file(_) do
 
   def put_file(upload) do
     {local_path, file} =
-      case Enum.reverse(String.split(upload.path, "/", trim: true)) do
+      case Enum.reverse(Path.split(upload.path)) do
         [file] ->
           {upload_path(), file}
 
@@ -23,7 +23,7 @@ def put_file(upload) do
 
     result_file = Path.join(local_path, file)
 
-    unless File.exists?(result_file) do
+    if not File.exists?(result_file) do
       File.cp!(upload.tempfile, result_file)
     end
 

lib/pleroma/uploaders/mdii.ex

@@ -3,6 +3,8 @@
 # SPDX-License-Identifier: AGPL-3.0-only
 
 defmodule Pleroma.Uploaders.MDII do
+  @moduledoc "Represents uploader for https://github.com/hakaba-hitoyo/minimal-digital-image-infrastructure"
+
   alias Pleroma.Config
   alias Pleroma.HTTP
 

lib/pleroma/uploaders/s3.ex

@@ -6,10 +6,12 @@ defmodule Pleroma.Uploaders.S3 do
   @behaviour Pleroma.Uploaders.Uploader
   require Logger
 
+  alias Pleroma.Config
+
   # The file name is re-encoded with S3's constraints here to comply with previous
   # links with less strict filenames
   def get_file(file) do
-    config = Pleroma.Config.get([__MODULE__])
+    config = Config.get([__MODULE__])
     bucket = Keyword.fetch!(config, :bucket)
 
     bucket_with_namespace =
@@ -34,15 +36,15 @@ def get_file(file) do
   end
 
   def put_file(%Pleroma.Upload{} = upload) do
-    config = Pleroma.Config.get([__MODULE__])
+    config = Config.get([__MODULE__])
     bucket = Keyword.get(config, :bucket)
 
-    {:ok, file_data} = File.read(upload.tempfile)
-
     s3_name = strict_encode(upload.path)
 
     op =
-      ExAws.S3.put_object(bucket, s3_name, file_data, [
+      upload.tempfile
+      |> ExAws.S3.Upload.stream_file()
+      |> ExAws.S3.upload(bucket, s3_name, [
         {:acl, :public_read},
         {:content_type, upload.content_type}
       ])

lib/pleroma/user.ex

@@ -155,10 +155,10 @@ def following_count(%User{} = user) do
   end
 
   def remote_user_creation(params) do
-    params =
+    bio_limit = Pleroma.Config.get([:instance, :user_bio_length], 5000)
-      params
+    name_limit = Pleroma.Config.get([:instance, :user_name_length], 100)
-      |> Map.put(:info, params[:info] || %{})
 
+    params = Map.put(params, :info, params[:info] || %{})
     info_cng = User.Info.remote_user_creation(%User.Info{}, params[:info])
 
     changes =
@@ -167,8 +167,8 @@ def remote_user_creation(params) do
       |> validate_required([:name, :ap_id])
       |> unique_constraint(:nickname)
       |> validate_format(:nickname, @email_regex)
-      |> validate_length(:bio, max: 5000)
+      |> validate_length(:bio, max: bio_limit)
-      |> validate_length(:name, max: 100)
+      |> validate_length(:name, max: name_limit)
       |> put_change(:local, false)
       |> put_embed(:info, info_cng)
 
@@ -191,22 +191,23 @@ def remote_user_creation(params) do
   end
 
   def update_changeset(struct, params \\ %{}) do
+    bio_limit = Pleroma.Config.get([:instance, :user_bio_length], 5000)
+    name_limit = Pleroma.Config.get([:instance, :user_name_length], 100)
+
     struct
     |> cast(params, [:bio, :name, :avatar, :following])
     |> unique_constraint(:nickname)
     |> validate_format(:nickname, local_nickname_regex())
-    |> validate_length(:bio, max: 5000)
+    |> validate_length(:bio, max: bio_limit)
-    |> validate_length(:name, min: 1, max: 100)
+    |> validate_length(:name, min: 1, max: name_limit)
   end
 
   def upgrade_changeset(struct, params \\ %{}) do
-    params =
+    bio_limit = Pleroma.Config.get([:instance, :user_bio_length], 5000)
-      params
+    name_limit = Pleroma.Config.get([:instance, :user_name_length], 100)
-      |> Map.put(:last_refreshed_at, NaiveDateTime.utc_now())
 
-    info_cng =
+    params = Map.put(params, :last_refreshed_at, NaiveDateTime.utc_now())
-      struct.info
+    info_cng = User.Info.user_upgrade(struct.info, params[:info])
-      |> User.Info.user_upgrade(params[:info])
 
     struct
     |> cast(params, [
@@ -219,8 +220,8 @@ def upgrade_changeset(struct, params \\ %{}) do
     ])
     |> unique_constraint(:nickname)
     |> validate_format(:nickname, local_nickname_regex())
-    |> validate_length(:bio, max: 5000)
+    |> validate_length(:bio, max: bio_limit)
-    |> validate_length(:name, max: 100)
+    |> validate_length(:name, max: name_limit)
     |> put_embed(:info, info_cng)
   end
 
@@ -247,6 +248,9 @@ def reset_password(%User{id: user_id} = user, data) do
   end
 
   def register_changeset(struct, params \\ %{}, opts \\ []) do
+    bio_limit = Pleroma.Config.get([:instance, :user_bio_length], 5000)
+    name_limit = Pleroma.Config.get([:instance, :user_name_length], 100)
+
     need_confirmation? =
       if is_nil(opts[:need_confirmation]) do
         Pleroma.Config.get([:instance, :account_activation_required])
@@ -267,8 +271,8 @@ def register_changeset(struct, params \\ %{}, opts \\ []) do
       |> validate_exclusion(:nickname, Pleroma.Config.get([User, :restricted_nicknames]))
       |> validate_format(:nickname, local_nickname_regex())
       |> validate_format(:email, @email_regex)
-      |> validate_length(:bio, max: 1000)
+      |> validate_length(:bio, max: bio_limit)
-      |> validate_length(:name, min: 1, max: 100)
+      |> validate_length(:name, min: 1, max: name_limit)
       |> put_change(:info, info_change)
 
     changeset =

lib/pleroma/web/activity_pub/activity_pub.ex

@@ -523,6 +523,8 @@ defp fetch_activities_for_context_query(context, opts) do
 
     from(activity in Activity)
     |> maybe_preload_objects(opts)
+    |> maybe_preload_bookmarks(opts)
+    |> maybe_set_thread_muted_field(opts)
     |> restrict_blocked(opts)
     |> restrict_recipients(recipients, opts["user"])
     |> where(
@@ -536,6 +538,7 @@ defp fetch_activities_for_context_query(context, opts) do
       )
     )
     |> exclude_poll_votes(opts)
+    |> exclude_id(opts)
     |> order_by([activity], desc: activity.id)
   end
 
@@ -628,6 +631,7 @@ def fetch_user_activities(user, reading_user, params \\ %{}) do
     params =
       params
       |> Map.put("type", ["Create", "Announce"])
+      |> Map.put("user", reading_user)
       |> Map.put("actor_id", user.ap_id)
       |> Map.put("whole_db", true)
       |> Map.put("pinned_activity_ids", user.info.pinned_activities)
@@ -875,6 +879,12 @@ defp exclude_poll_votes(query, _) do
     end
   end
 
+  defp exclude_id(query, %{"exclude_id" => id}) when is_binary(id) do
+    from(activity in query, where: activity.id != ^id)
+  end
+
+  defp exclude_id(query, _), do: query
+
   defp maybe_preload_objects(query, %{"skip_preload" => true}), do: query
 
   defp maybe_preload_objects(query, _) do

lib/pleroma/web/activity_pub/mrf.ex

@@ -28,11 +28,43 @@ defp get_policies(_), do: []
 
   @spec subdomains_regex([String.t()]) :: [Regex.t()]
   def subdomains_regex(domains) when is_list(domains) do
-    for domain <- domains, do: ~r(^#{String.replace(domain, "*.", "(.*\\.)*")}$)
+    for domain <- domains, do: ~r(^#{String.replace(domain, "*.", "(.*\\.)*")}$)i
   end
 
   @spec subdomain_match?([Regex.t()], String.t()) :: boolean()
   def subdomain_match?(domains, host) do
     Enum.any?(domains, fn domain -> Regex.match?(domain, host) end)
   end
+
+  @callback describe() :: {:ok | :error, Map.t()}
+
+  def describe(policies) do
+    {:ok, policy_configs} =
+      policies
+      |> Enum.reduce({:ok, %{}}, fn
+        policy, {:ok, data} ->
+          {:ok, policy_data} = policy.describe()
+          {:ok, Map.merge(data, policy_data)}
+
+        _, error ->
+          error
+      end)
+
+    mrf_policies =
+      get_policies()
+      |> Enum.map(fn policy -> to_string(policy) |> String.split(".") |> List.last() end)
+
+    exclusions = Pleroma.Config.get([:instance, :mrf_transparency_exclusions])
+
+    base =
+      %{
+        mrf_policies: mrf_policies,
+        exclusions: length(exclusions) > 0
+      }
+      |> Map.merge(policy_configs)
+
+    {:ok, base}
+  end
+
+  def describe, do: get_policies() |> describe()
 end

lib/pleroma/web/activity_pub/mrf/anti_followbot_policy.ex

@@ -62,4 +62,7 @@ def filter(%{"type" => "Follow", "actor" => actor_id} = message) do
 
   @impl true
   def filter(message), do: {:ok, message}
+
+  @impl true
+  def describe, do: {:ok, %{}}
 end

lib/pleroma/web/activity_pub/mrf/anti_link_spam_policy.ex

@@ -5,6 +5,8 @@
 defmodule Pleroma.Web.ActivityPub.MRF.AntiLinkSpamPolicy do
   alias Pleroma.User
 
+  @behaviour Pleroma.Web.ActivityPub.MRF
+
   require Logger
 
   # has the user successfully posted before?
@@ -22,6 +24,7 @@ defp contains_links?(%{"content" => content} = _object) do
 
   defp contains_links?(_), do: false
 
+  @impl true
   def filter(%{"type" => "Create", "actor" => actor, "object" => object} = message) do
     with {:ok, %User{} = u} <- User.get_or_fetch_by_ap_id(actor),
          {:contains_links, true} <- {:contains_links, contains_links?(object)},
@@ -45,4 +48,7 @@ def filter(%{"type" => "Create", "actor" => actor, "object" => object} = message
 
   # in all other cases, pass through
   def filter(message), do: {:ok, message}
+
+  @impl true
+  def describe, do: {:ok, %{}}
 end

lib/pleroma/web/activity_pub/mrf/drop_policy.ex

@@ -12,4 +12,7 @@ def filter(object) do
     Logger.info("REJECTING #{inspect(object)}")
     {:reject, object}
   end
+
+  @impl true
+  def describe, do: {:ok, %{}}
 end

lib/pleroma/web/activity_pub/mrf/ensure_re_prepended.ex

@@ -39,4 +39,6 @@ def filter(%{"type" => "Create", "object" => child_object} = object) do
   end
 
   def filter(object), do: {:ok, object}
+
+  def describe, do: {:ok, %{}}
 end

lib/pleroma/web/activity_pub/mrf/hellthread_policy.ex

@@ -90,4 +90,7 @@ def filter(%{"type" => "Create"} = message) do
 
   @impl true
   def filter(message), do: {:ok, message}
+
+  @impl true
+  def describe, do: {:ok, %{mrf_hellthread: Pleroma.Config.get([:mrf_hellthread])}}
 end

lib/pleroma/web/activity_pub/mrf/keyword_policy.ex

@@ -96,4 +96,36 @@ def filter(%{"type" => "Create", "object" => %{"content" => _content}} = message
 
   @impl true
   def filter(message), do: {:ok, message}
+
+  @impl true
+  def describe do
+    # This horror is needed to convert regex sigils to strings
+    mrf_keyword =
+      Pleroma.Config.get(:mrf_keyword, [])
+      |> Enum.map(fn {key, value} ->
+        {key,
+         Enum.map(value, fn
+           {pattern, replacement} ->
+             %{
+               "pattern" =>
+                 if not is_binary(pattern) do
+                   inspect(pattern)
+                 else
+                   pattern
+                 end,
+               "replacement" => replacement
+             }
+
+           pattern ->
+             if not is_binary(pattern) do
+               inspect(pattern)
+             else
+               pattern
+             end
+         end)}
+      end)
+      |> Enum.into(%{})
+
+    {:ok, %{mrf_keyword: mrf_keyword}}
+  end
 end

lib/pleroma/web/activity_pub/mrf/mediaproxy_warming_policy.ex

@@ -61,4 +61,7 @@ def filter(
 
   @impl true
   def filter(message), do: {:ok, message}
+
+  @impl true
+  def describe, do: {:ok, %{}}
 end

lib/pleroma/web/activity_pub/mrf/mention_policy.ex

@@ -21,4 +21,7 @@ def filter(%{"type" => "Create"} = message) do
 
   @impl true
   def filter(message), do: {:ok, message}
+
+  @impl true
+  def describe, do: {:ok, %{}}
 end

lib/pleroma/web/activity_pub/mrf/no_placeholder_text_policy.ex

@@ -19,4 +19,7 @@ def filter(
 
   @impl true
   def filter(object), do: {:ok, object}
+
+  @impl true
+  def describe, do: {:ok, %{}}
 end

lib/pleroma/web/activity_pub/mrf/noop_policy.ex

@@ -10,4 +10,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.NoOpPolicy do
   def filter(object) do
     {:ok, object}
   end
+
+  @impl true
+  def describe, do: {:ok, %{}}
 end

lib/pleroma/web/activity_pub/mrf/normalize_markup.ex

@@ -21,4 +21,6 @@ def filter(%{"type" => "Create", "object" => child_object} = object) do
   end
 
   def filter(object), do: {:ok, object}
+
+  def describe, do: {:ok, %{}}
 end

lib/pleroma/web/activity_pub/mrf/reject_non_public.ex

@@ -44,4 +44,7 @@ def filter(%{"type" => "Create"} = object) do
 
   @impl true
   def filter(object), do: {:ok, object}
+
+  @impl true
+  def describe, do: {:ok, %{mrf_rejectnonpublic: Pleroma.Config.get([:mrf_rejectnonpublic])}}
 end

lib/pleroma/web/activity_pub/mrf/simple_policy.ex

@@ -177,4 +177,16 @@ def filter(%{"id" => actor, "type" => obj_type} = object)
   end
 
   def filter(object), do: {:ok, object}
+
+  @impl true
+  def describe do
+    exclusions = Pleroma.Config.get([:instance, :mrf_transparency_exclusions])
+
+    mrf_simple =
+      Pleroma.Config.get(:mrf_simple)
+      |> Enum.map(fn {k, v} -> {k, Enum.reject(v, fn v -> v in exclusions end)} end)
+      |> Enum.into(%{})
+
+    {:ok, %{mrf_simple: mrf_simple}}
+  end
 end

lib/pleroma/web/activity_pub/mrf/subchain_policy.ex

@@ -37,4 +37,7 @@ def filter(%{"actor" => actor} = message) do
 
   @impl true
   def filter(message), do: {:ok, message}
+
+  @impl true
+  def describe, do: {:ok, %{}}
 end

lib/pleroma/web/activity_pub/mrf/tag_policy.ex

@@ -165,4 +165,7 @@ def filter(%{"actor" => actor, "type" => "Create"} = message),
 
   @impl true
   def filter(message), do: {:ok, message}
+
+  @impl true
+  def describe, do: {:ok, %{}}
 end

lib/pleroma/web/activity_pub/mrf/user_allowlist_policy.ex

@@ -32,4 +32,13 @@ def filter(%{"actor" => actor} = object) do
   end
 
   def filter(object), do: {:ok, object}
+
+  @impl true
+  def describe do
+    mrf_user_allowlist =
+      Config.get([:mrf_user_allowlist], [])
+      |> Enum.into(%{}, fn {k, v} -> {k, length(v)} end)
+
+    {:ok, %{mrf_user_allowlist: mrf_user_allowlist}}
+  end
 end

lib/pleroma/web/activity_pub/mrf/vocabulary_policy.ex

@@ -0,0 +1,36 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ActivityPub.MRF.VocabularyPolicy do
+  @moduledoc "Filter messages which belong to certain activity vocabularies"
+
+  @behaviour Pleroma.Web.ActivityPub.MRF
+
+  def filter(%{"type" => "Undo", "object" => child_message} = message) do
+    with {:ok, _} <- filter(child_message) do
+      {:ok, message}
+    else
+      {:reject, nil} ->
+        {:reject, nil}
+    end
+  end
+
+  def filter(%{"type" => message_type} = message) do
+    with accepted_vocabulary <- Pleroma.Config.get([:mrf_vocabulary, :accept]),
+         rejected_vocabulary <- Pleroma.Config.get([:mrf_vocabulary, :reject]),
+         true <-
+           length(accepted_vocabulary) == 0 || Enum.member?(accepted_vocabulary, message_type),
+         false <-
+           length(rejected_vocabulary) > 0 && Enum.member?(rejected_vocabulary, message_type),
+         {:ok, _} <- filter(message["object"]) do
+      {:ok, message}
+    else
+      _ -> {:reject, nil}
+    end
+  end
+
+  def filter(message), do: {:ok, message}
+
+  def describe, do: {:ok, %{mrf_vocabulary: Pleroma.Config.get(:mrf_vocabulary)}}
+end

lib/pleroma/web/activity_pub/relay.ex

@@ -14,6 +14,7 @@ def get_actor do
     |> User.get_or_create_service_actor_by_ap_id()
   end
 
+  @spec follow(String.t()) :: {:ok, Activity.t()} | {:error, any()}
   def follow(target_instance) do
     with %User{} = local_user <- get_actor(),
          {:ok, %User{} = target_user} <- User.get_or_fetch_by_ap_id(target_instance),
@@ -21,12 +22,17 @@ def follow(target_instance) do
       Logger.info("relay: followed instance: #{target_instance}; id=#{activity.data["id"]}")
       {:ok, activity}
     else
+      {:error, _} = error ->
+        Logger.error("error: #{inspect(error)}")
+        error
+
       e ->
         Logger.error("error: #{inspect(e)}")
         {:error, e}
     end
   end
 
+  @spec unfollow(String.t()) :: {:ok, Activity.t()} | {:error, any()}
   def unfollow(target_instance) do
     with %User{} = local_user <- get_actor(),
          {:ok, %User{} = target_user} <- User.get_or_fetch_by_ap_id(target_instance),
@@ -34,20 +40,27 @@ def unfollow(target_instance) do
       Logger.info("relay: unfollowed instance: #{target_instance}: id=#{activity.data["id"]}")
       {:ok, activity}
     else
+      {:error, _} = error ->
+        Logger.error("error: #{inspect(error)}")
+        error
+
       e ->
         Logger.error("error: #{inspect(e)}")
         {:error, e}
     end
   end
 
+  @spec publish(any()) :: {:ok, Activity.t(), Object.t()} | {:error, any()}
   def publish(%Activity{data: %{"type" => "Create"}} = activity) do
     with %User{} = user <- get_actor(),
          %Object{} = object <- Object.normalize(activity) do
       ActivityPub.announce(user, object, nil, true, false)
     else
-      e -> Logger.error("error: #{inspect(e)}")
+      e ->
+        Logger.error("error: #{inspect(e)}")
+        {:error, inspect(e)}
     end
   end
 
-  def publish(_), do: nil
+  def publish(_), do: {:error, "Not implemented"}
 end

lib/pleroma/web/activity_pub/transmogrifier.ex

@@ -29,6 +29,7 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
   """
   def fix_object(object, options \\ []) do
     object
+    |> strip_internal_fields
     |> fix_actor
     |> fix_url
     |> fix_attachments
@@ -37,7 +38,6 @@ def fix_object(object, options \\ []) do
     |> fix_emoji
     |> fix_tag
     |> fix_content_map
-    |> fix_likes
     |> fix_addressing
     |> fix_summary
     |> fix_type(options)
@@ -154,20 +154,6 @@ def fix_actor(%{"attributedTo" => actor} = object) do
     |> Map.put("actor", Containment.get_actor(%{"actor" => actor}))
   end
 
-  # Check for standardisation
-  # This is what Peertube does
-  # curl -H 'Accept: application/activity+json' $likes | jq .totalItems
-  # Prismo returns only an integer (count) as "likes"
-  def fix_likes(%{"likes" => likes} = object) when not is_map(likes) do
-    object
-    |> Map.put("likes", [])
-    |> Map.put("like_count", 0)
-  end
-
-  def fix_likes(object) do
-    object
-  end
-
   def fix_in_reply_to(object, options \\ [])
 
   def fix_in_reply_to(%{"inReplyTo" => in_reply_to} = object, options)
@@ -350,13 +336,15 @@ def fix_content_map(object), do: object
 
   def fix_type(object, options \\ [])
 
-  def fix_type(%{"inReplyTo" => reply_id} = object, options) when is_binary(reply_id) do
+  def fix_type(%{"inReplyTo" => reply_id, "name" => _} = object, options)
+      when is_binary(reply_id) do
     reply =
-      if Federator.allowed_incoming_reply_depth?(options[:depth]) do
+      with true <- Federator.allowed_incoming_reply_depth?(options[:depth]),
-        Object.normalize(reply_id, true)
+           {:ok, object} <- get_obj_helper(reply_id, options) do
+        object
       end
 
-    if reply && (reply.data["type"] == "Question" and object["name"]) do
+    if reply && reply.data["type"] == "Question" do
       Map.put(object, "type", "Answer")
     else
       object
@@ -716,8 +704,7 @@ def handle_incoming(
         } = _data,
         _options
       ) do
-    with true <- Pleroma.Config.get([:activitypub, :accept_blocks]),
+    with %User{local: true} = blocked <- User.get_cached_by_ap_id(blocked),
-         %User{local: true} = blocked <- User.get_cached_by_ap_id(blocked),
          {:ok, %User{} = blocker} <- User.get_or_fetch_by_ap_id(blocker),
          {:ok, activity} <- ActivityPub.unblock(blocker, blocked, id, false) do
       User.unblock(blocker, blocked)
@@ -731,8 +718,7 @@ def handle_incoming(
         %{"type" => "Block", "object" => blocked, "actor" => blocker, "id" => id} = _data,
         _options
       ) do
-    with true <- Pleroma.Config.get([:activitypub, :accept_blocks]),
+    with %User{local: true} = blocked = User.get_cached_by_ap_id(blocked),
-         %User{local: true} = blocked = User.get_cached_by_ap_id(blocked),
          {:ok, %User{} = blocker} = User.get_or_fetch_by_ap_id(blocker),
          {:ok, activity} <- ActivityPub.block(blocker, blocked, id, false) do
       User.unfollow(blocker, blocked)
@@ -787,7 +773,6 @@ def prepare_object(object) do
     |> add_mention_tags
     |> add_emoji_tags
     |> add_attributed_to
-    |> add_likes
     |> prepare_attachments
     |> set_conversation
     |> set_reply_to_uri
@@ -974,22 +959,6 @@ def add_attributed_to(object) do
     |> Map.put("attributedTo", attributed_to)
   end
 
-  def add_likes(%{"id" => id, "like_count" => likes} = object) do
-    likes = %{
-      "id" => "#{id}/likes",
-      "first" => "#{id}/likes?page=1",
-      "type" => "OrderedCollection",
-      "totalItems" => likes
-    }
-
-    object
-    |> Map.put("likes", likes)
-  end
-
-  def add_likes(object) do
-    object
-  end
-
   def prepare_attachments(object) do
     attachments =
       (object["attachment"] || [])
@@ -1005,6 +974,7 @@ def prepare_attachments(object) do
   defp strip_internal_fields(object) do
     object
     |> Map.drop([
+      "likes",
       "like_count",
       "announcements",
       "announcement_count",

lib/pleroma/web/mastodon_api/mastodon_api.ex

@@ -13,10 +13,8 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPI do
   alias Pleroma.User
   alias Pleroma.Web.CommonAPI
 
+  @spec follow(User.t(), User.t(), map) :: {:ok, User.t()} | {:error, String.t()}
   def follow(follower, followed, params \\ %{}) do
-    options = cast_params(params)
-    reblogs = options[:reblogs]
-
     result =
       if not User.following?(follower, followed) do
         CommonAPI.follow(follower, followed)
@@ -24,19 +22,25 @@ def follow(follower, followed, params \\ %{}) do
         {:ok, follower, followed, nil}
       end
 
-    with {:ok, follower, followed, _} <- result do
+    with {:ok, follower, _followed, _} <- result do
-      reblogs
+      options = cast_params(params)
-      |> case do
+
-        false -> CommonAPI.hide_reblogs(follower, followed)
+      case reblogs_visibility(options[:reblogs], result) do
-        _ -> CommonAPI.show_reblogs(follower, followed)
-      end
-      |> case do
         {:ok, follower} -> {:ok, follower}
         _ -> {:ok, follower}
       end
     end
   end
 
+  defp reblogs_visibility(false, {:ok, follower, followed, _}) do
+    CommonAPI.hide_reblogs(follower, followed)
+  end
+
+  defp reblogs_visibility(_, {:ok, follower, followed, _}) do
+    CommonAPI.show_reblogs(follower, followed)
+  end
+
+  @spec get_followers(User.t(), map()) :: list(User.t())
   def get_followers(user, params \\ %{}) do
     user
     |> User.get_followers_query()

lib/pleroma/web/mastodon_api/mastodon_api_controller.ex

@@ -435,6 +435,7 @@ def public_timeline(%{assigns: %{user: user}} = conn, params) do
       |> Map.put("local_only", local_only)
       |> Map.put("blocking_user", user)
       |> Map.put("muting_user", user)
+      |> Map.put("user", user)
       |> ActivityPub.fetch_public_activities()
       |> Enum.reverse()
 
@@ -496,12 +497,9 @@ def get_context(%{assigns: %{user: user}} = conn, %{"id" => id}) do
          activities <-
            ActivityPub.fetch_activities_for_context(activity.data["context"], %{
              "blocking_user" => user,
-             "user" => user
+             "user" => user,
+             "exclude_id" => activity.id
            }),
-         activities <-
-           activities |> Enum.filter(fn %{id: aid} -> to_string(aid) != to_string(id) end),
-         activities <-
-           activities |> Enum.filter(fn %{data: %{"type" => type}} -> type == "Create" end),
          grouped_activities <- Enum.group_by(activities, fn %{id: id} -> id < activity.id end) do
       result = %{
         ancestors:
@@ -536,8 +534,8 @@ def get_poll(%{assigns: %{user: user}} = conn, %{"id" => id}) do
       |> put_view(StatusView)
       |> try_render("poll.json", %{object: object, for: user})
     else
-      nil -> render_error(conn, :not_found, "Record not found")
+      error when is_nil(error) or error == false ->
-      false -> render_error(conn, :not_found, "Record not found")
+        render_error(conn, :not_found, "Record not found")
     end
   end
 
@@ -885,8 +883,8 @@ def get_mascot(%{assigns: %{user: user}} = conn, _params) do
   end
 
   def favourited_by(%{assigns: %{user: user}} = conn, %{"id" => id}) do
-    with %Activity{data: %{"object" => object}} <- Activity.get_by_id(id),
+    with %Activity{} = activity <- Activity.get_by_id_with_object(id),
-         %Object{data: %{"likes" => likes}} <- Object.normalize(object) do
+         %Object{data: %{"likes" => likes}} <- Object.normalize(activity) do
       q = from(u in User, where: u.ap_id in ^likes)
 
       users =
@@ -902,8 +900,8 @@ def favourited_by(%{assigns: %{user: user}} = conn, %{"id" => id}) do
   end
 
   def reblogged_by(%{assigns: %{user: user}} = conn, %{"id" => id}) do
-    with %Activity{data: %{"object" => object}} <- Activity.get_by_id(id),
+    with %Activity{} = activity <- Activity.get_by_id_with_object(id),
-         %Object{data: %{"announcements" => announces}} <- Object.normalize(object) do
+         %Object{data: %{"announcements" => announces}} <- Object.normalize(activity) do
       q = from(u in User, where: u.ap_id in ^announces)
 
       users =
@@ -944,6 +942,7 @@ def hashtag_timeline(%{assigns: %{user: user}} = conn, params) do
       |> Map.put("local_only", local_only)
       |> Map.put("blocking_user", user)
       |> Map.put("muting_user", user)
+      |> Map.put("user", user)
       |> Map.put("tag", tags)
       |> Map.put("tag_all", tag_all)
       |> Map.put("tag_reject", tag_reject)
@@ -1350,6 +1349,7 @@ def list_timeline(%{assigns: %{user: user}} = conn, %{"list_id" => id} = params)
         params
         |> Map.put("type", "Create")
         |> Map.put("blocking_user", user)
+        |> Map.put("user", user)
         |> Map.put("muting_user", user)
 
       # we must filter the following list for the user to avoid leaking statuses the user
@@ -1690,45 +1690,35 @@ def suggestions(%{assigns: %{user: user}} = conn, _) do
         |> String.replace("{{user}}", user)
 
       with {:ok, %{status: 200, body: body}} <-
-             HTTP.get(
+             HTTP.get(url, [], adapter: [recv_timeout: timeout, pool: :default]),
-               url,
-               [],
-               adapter: [
-                 recv_timeout: timeout,
-                 pool: :default
-               ]
-             ),
            {:ok, data} <- Jason.decode(body) do
         data =
           data
           |> Enum.slice(0, limit)
           |> Enum.map(fn x ->
-            Map.put(
+            x
-              x,
+            |> Map.put("id", fetch_suggestion_id(x))
-              "id",
+            |> Map.put("avatar", MediaProxy.url(x["avatar"]))
-              case User.get_or_fetch(x["acct"]) do
+            |> Map.put("avatar_static", MediaProxy.url(x["avatar_static"]))
-                {:ok, %User{id: id}} -> id
-                _ -> 0
-              end
-            )
-          end)
-          |> Enum.map(fn x ->
-            Map.put(x, "avatar", MediaProxy.url(x["avatar"]))
-          end)
-          |> Enum.map(fn x ->
-            Map.put(x, "avatar_static", MediaProxy.url(x["avatar_static"]))
           end)
 
-        conn
+        json(conn, data)
-        |> json(data)
       else
-        e -> Logger.error("Could not retrieve suggestions at fetch #{url}, #{inspect(e)}")
+        e ->
+          Logger.error("Could not retrieve suggestions at fetch #{url}, #{inspect(e)}")
       end
     else
       json(conn, [])
     end
   end
 
+  defp fetch_suggestion_id(attrs) do
+    case User.get_or_fetch(attrs["acct"]) do
+      {:ok, %User{id: id}} -> id
+      _ -> 0
+    end
+  end
+
   def status_card(%{assigns: %{user: user}} = conn, %{"id" => status_id}) do
     with %Activity{} = activity <- Activity.get_by_id(status_id),
          true <- Visibility.visible_for_user?(activity, user) do

lib/pleroma/web/mastodon_api/views/account_view.ex

@@ -72,6 +72,13 @@ defp do_render("account.json", %{user: user} = opts) do
     image = User.avatar_url(user) |> MediaProxy.url()
     header = User.banner_url(user) |> MediaProxy.url()
     user_info = User.get_cached_user_info(user)
+
+    following_count =
+      ((!user.info.hide_follows or opts[:for] == user) && user_info.following_count) || 0
+
+    followers_count =
+      ((!user.info.hide_followers or opts[:for] == user) && user_info.follower_count) || 0
+
     bot = (user.info.source_data["type"] || "Person") in ["Application", "Service"]
 
     emojis =
@@ -102,8 +109,8 @@ defp do_render("account.json", %{user: user} = opts) do
       display_name: display_name,
       locked: user_info.locked,
       created_at: Utils.to_masto_date(user.inserted_at),
-      followers_count: user_info.follower_count,
+      followers_count: followers_count,
-      following_count: user_info.following_count,
+      following_count: following_count,
       statuses_count: user_info.note_count,
       note: bio || "",
       url: User.profile_url(user),

lib/pleroma/web/mastodon_api/views/status_view.ex

@@ -5,6 +5,8 @@
 defmodule Pleroma.Web.MastodonAPI.StatusView do
   use Pleroma.Web, :view
 
+  require Pleroma.Constants
+
   alias Pleroma.Activity
   alias Pleroma.HTML
   alias Pleroma.Object
@@ -24,19 +26,19 @@ defp get_replied_to_activities([]), do: %{}
   defp get_replied_to_activities(activities) do
     activities
     |> Enum.map(fn
-      %{data: %{"type" => "Create", "object" => object}} ->
+      %{data: %{"type" => "Create"}} = activity ->
-        object = Object.normalize(object)
+        object = Object.normalize(activity)
-        object.data["inReplyTo"] != "" && object.data["inReplyTo"]
+        object && object.data["inReplyTo"] != "" && object.data["inReplyTo"]
 
       _ ->
         nil
     end)
     |> Enum.filter(& &1)
-    |> Activity.create_by_object_ap_id()
+    |> Activity.create_by_object_ap_id_with_object()
     |> Repo.all()
     |> Enum.reduce(%{}, fn activity, acc ->
       object = Object.normalize(activity)
-      Map.put(acc, object.data["id"], activity)
+      if object, do: Map.put(acc, object.data["id"], activity), else: acc
     end)
   end
 
@@ -88,6 +90,7 @@ def render(
     reblogged_activity =
       Activity.create_by_object_ap_id(activity_object.data["id"])
       |> Activity.with_preloaded_bookmark(opts[:for])
+      |> Activity.with_set_thread_muted_field(opts[:for])
       |> Repo.one()
 
     reblogged = render("status.json", Map.put(opts, :activity, reblogged_activity))
@@ -142,6 +145,7 @@ def render("status.json", %{activity: %{data: %{"object" => _object}} = activity
     object = Object.normalize(activity)
 
     user = get_user(activity.data["actor"])
+    user_follower_address = user.follower_address
 
     like_count = object.data["like_count"] || 0
     announcement_count = object.data["announcement_count"] || 0
@@ -157,7 +161,11 @@ def render("status.json", %{activity: %{data: %{"object" => _object}} = activity
     mentions =
       (object.data["to"] ++ tag_mentions)
       |> Enum.uniq()
-      |> Enum.map(fn ap_id -> User.get_cached_by_ap_id(ap_id) end)
+      |> Enum.map(fn
+        Pleroma.Constants.as_public() -> nil
+        ^user_follower_address -> nil
+        ap_id -> User.get_cached_by_ap_id(ap_id)
+      end)
       |> Enum.filter(& &1)
       |> Enum.map(fn user -> AccountView.render("mention.json", %{user: user}) end)
 
@@ -168,7 +176,7 @@ def render("status.json", %{activity: %{data: %{"object" => _object}} = activity
     thread_muted? =
       case activity.thread_muted? do
         thread_muted? when is_boolean(thread_muted?) -> thread_muted?
-        nil -> CommonAPI.thread_muted?(user, activity)
+        nil -> (opts[:for] && CommonAPI.thread_muted?(opts[:for], activity)) || false
       end
 
     attachment_data = object.data["attachment"] || []

lib/pleroma/web/nodeinfo/nodeinfo_controller.ex

@@ -34,64 +34,18 @@ def schemas(conn, _params) do
   def raw_nodeinfo do
     stats = Stats.get_stats()
 
-    exclusions = Config.get([:instance, :mrf_transparency_exclusions])
-
-    mrf_simple =
-      Config.get(:mrf_simple)
-      |> Enum.map(fn {k, v} -> {k, Enum.reject(v, fn v -> v in exclusions end)} end)
-      |> Enum.into(%{})
-
-    # This horror is needed to convert regex sigils to strings
-    mrf_keyword =
-      Config.get(:mrf_keyword, [])
-      |> Enum.map(fn {key, value} ->
-        {key,
-         Enum.map(value, fn
-           {pattern, replacement} ->
-             %{
-               "pattern" =>
-                 if not is_binary(pattern) do
-                   inspect(pattern)
-                 else
-                   pattern
-                 end,
-               "replacement" => replacement
-             }
-
-           pattern ->
-             if not is_binary(pattern) do
-               inspect(pattern)
-             else
-               pattern
-             end
-         end)}
-      end)
-      |> Enum.into(%{})
-
-    mrf_policies =
-      MRF.get_policies()
-      |> Enum.map(fn policy -> to_string(policy) |> String.split(".") |> List.last() end)
-
     quarantined = Config.get([:instance, :quarantined_instances], [])
 
     staff_accounts =
       User.all_superusers()
       |> Enum.map(fn u -> u.ap_id end)
 
-    mrf_user_allowlist =
-      Config.get([:mrf_user_allowlist], [])
-      |> Enum.into(%{}, fn {k, v} -> {k, length(v)} end)
-
     federation_response =
       if Config.get([:instance, :mrf_transparency]) do
-        %{
+        {:ok, data} = MRF.describe()
-          mrf_policies: mrf_policies,
+
-          mrf_simple: mrf_simple,
+        data
-          mrf_keyword: mrf_keyword,
+        |> Map.merge(%{quarantined_instances: quarantined})
-          mrf_user_allowlist: mrf_user_allowlist,
-          quarantined_instances: quarantined,
-          exclusions: length(exclusions) > 0
-        }
       else
         %{}
       end

lib/pleroma/web/web.ex

@@ -58,10 +58,10 @@ def safe_render(view, template, assigns \\ %{}) do
       rescue
         error ->
           Logger.error(
-            "#{__MODULE__} failed to render #{inspect({view, template})}: #{inspect(error)}"
+            "#{__MODULE__} failed to render #{inspect({view, template})}\n" <>
+              Exception.format(:error, error, __STACKTRACE__)
           )
 
-          Logger.error(inspect(__STACKTRACE__))
           nil
       end
 

mix.exs

@@ -95,7 +95,7 @@ defp oauth_deps do
   defp deps do
     [
       {:phoenix, "~> 1.4.8"},
-      {:tzdata, "~> 1.0"},
+      {:tzdata, "~> 0.5.21"},
       {:plug_cowboy, "~> 2.0"},
       {:phoenix_pubsub, "~> 1.1"},
       {:phoenix_ecto, "~> 4.0"},
@@ -115,8 +115,9 @@ defp deps do
       {:tesla, "~> 1.2"},
       {:jason, "~> 1.0"},
       {:mogrify, "~> 0.6.1"},
-      {:ex_aws, "~> 2.0"},
+      {:ex_aws, "~> 2.1"},
       {:ex_aws_s3, "~> 2.0"},
+      {:sweet_xml, "~> 0.6.6"},
       {:earmark, "~> 1.3"},
       {:bbcode, "~> 0.1.1"},
       {:ex_machina, "~> 2.3", only: :test},

mix.lock

@@ -81,13 +81,14 @@
   "ranch": {:hex, :ranch, "1.7.1", "6b1fab51b49196860b733a49c07604465a47bdb78aa10c1c16a3d199f7f8c881", [:rebar3], [], "hexpm"},
   "recon": {:git, "https://github.com/ferd/recon.git", "75d70c7c08926d2f24f1ee6de14ee50fe8a52763", [tag: "2.4.0"]},
   "ssl_verify_fun": {:hex, :ssl_verify_fun, "1.1.4", "f0eafff810d2041e93f915ef59899c923f4568f4585904d010387ed74988e77b", [:make, :mix, :rebar3], [], "hexpm"},
+  "sweet_xml": {:hex, :sweet_xml, "0.6.6", "fc3e91ec5dd7c787b6195757fbcf0abc670cee1e4172687b45183032221b66b8", [:mix], [], "hexpm"},
   "swoosh": {:hex, :swoosh, "0.23.2", "7dda95ff0bf54a2298328d6899c74dae1223777b43563ccebebb4b5d2b61df38", [:mix], [{:cowboy, "~> 1.0.1 or ~> 1.1 or ~> 2.4", [hex: :cowboy, repo: "hexpm", optional: true]}, {:gen_smtp, "~> 0.13", [hex: :gen_smtp, repo: "hexpm", optional: true]}, {:hackney, "~> 1.9", [hex: :hackney, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}, {:mail, "~> 0.2", [hex: :mail, repo: "hexpm", optional: true]}, {:mime, "~> 1.1", [hex: :mime, repo: "hexpm", optional: false]}, {:plug_cowboy, ">= 1.0.0", [hex: :plug_cowboy, repo: "hexpm", optional: true]}], "hexpm"},
   "syslog": {:git, "https://github.com/Vagabond/erlang-syslog.git", "4a6c6f2c996483e86c1320e9553f91d337bcb6aa", [tag: "1.0.5"]},
   "telemetry": {:hex, :telemetry, "0.4.0", "8339bee3fa8b91cb84d14c2935f8ecf399ccd87301ad6da6b71c09553834b2ab", [:rebar3], [], "hexpm"},
   "tesla": {:hex, :tesla, "1.2.1", "864783cc27f71dd8c8969163704752476cec0f3a51eb3b06393b3971dc9733ff", [:mix], [{:exjsx, ">= 3.0.0", [hex: :exjsx, repo: "hexpm", optional: true]}, {:fuse, "~> 2.4", [hex: :fuse, repo: "hexpm", optional: true]}, {:hackney, "~> 1.6", [hex: :hackney, repo: "hexpm", optional: true]}, {:ibrowse, "~> 4.4.0", [hex: :ibrowse, repo: "hexpm", optional: true]}, {:jason, ">= 1.0.0", [hex: :jason, repo: "hexpm", optional: true]}, {:mime, "~> 1.0", [hex: :mime, repo: "hexpm", optional: false]}, {:poison, ">= 1.0.0", [hex: :poison, repo: "hexpm", optional: true]}], "hexpm"},
   "timex": {:hex, :timex, "3.6.1", "efdf56d0e67a6b956cc57774353b0329c8ab7726766a11547e529357ffdc1d56", [:mix], [{:combine, "~> 0.10", [hex: :combine, repo: "hexpm", optional: false]}, {:gettext, "~> 0.10", [hex: :gettext, repo: "hexpm", optional: false]}, {:tzdata, "~> 0.1.8 or ~> 0.5 or ~> 1.0.0", [hex: :tzdata, repo: "hexpm", optional: false]}], "hexpm"},
   "trailing_format_plug": {:hex, :trailing_format_plug, "0.0.7", "64b877f912cf7273bed03379936df39894149e35137ac9509117e59866e10e45", [:mix], [{:plug, "> 0.12.0", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
-  "tzdata": {:hex, :tzdata, "1.0.1", "f6027a331af7d837471248e62733c6ebee86a72e57c613aa071ebb1f750fc71a", [:mix], [{:hackney, "~> 1.0", [hex: :hackney, repo: "hexpm", optional: false]}], "hexpm"},
+  "tzdata": {:hex, :tzdata, "0.5.21", "8cbf3607fcce69636c672d5be2bbb08687fe26639a62bdcc283d267277db7cf0", [:mix], [{:hackney, "~> 1.0", [hex: :hackney, repo: "hexpm", optional: false]}], "hexpm"},
   "ueberauth": {:hex, :ueberauth, "0.6.1", "9e90d3337dddf38b1ca2753aca9b1e53d8a52b890191cdc55240247c89230412", [:mix], [{:plug, "~> 1.5", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
   "unicode_util_compat": {:hex, :unicode_util_compat, "0.4.1", "d869e4c68901dd9531385bb0c8c40444ebf624e60b6962d95952775cac5e90cd", [:rebar3], [], "hexpm"},
   "unsafe": {:hex, :unsafe, "1.0.1", "a27e1874f72ee49312e0a9ec2e0b27924214a05e3ddac90e91727bc76f8613d8", [:mix], [], "hexpm"},

priv/static/adminfe/index.html

@@ -1 +1 @@
-<!DOCTYPE html><html><head><meta charset=utf-8><meta http-equiv=X-UA-Compatible content="IE=edge,chrome=1"><meta name=renderer content=webkit><meta name=viewport content="width=device-width,initial-scale=1,maximum-scale=1,user-scalable=no"><title>Admin FE</title><link rel="shortcut icon" href=favicon.ico><link href=chunk-elementUI.f74c256b.css rel=stylesheet><link href=chunk-libs.4e8c4664.css rel=stylesheet><link href=app.34fc670f.css rel=stylesheet></head><body><script src=/pleroma/admin/static/tinymce4.7.5/tinymce.min.js></script><div id=app></div><script type=text/javascript src=static/js/runtime.d8d12c12.js></script><script type=text/javascript src=static/js/chunk-elementUI.1fa5434b.js></script><script type=text/javascript src=static/js/chunk-libs.d5609760.js></script><script type=text/javascript src=static/js/app.4137ad8f.js></script></body></html>
+<!DOCTYPE html><html><head><meta charset=utf-8><meta http-equiv=X-UA-Compatible content="IE=edge,chrome=1"><meta name=renderer content=webkit><meta name=viewport content="width=device-width,initial-scale=1,maximum-scale=1,user-scalable=no"><title>Admin FE</title><link rel="shortcut icon" href=favicon.ico><link href=chunk-elementUI.e5cd8da6.css rel=stylesheet><link href=chunk-libs.4e8c4664.css rel=stylesheet><link href=app.34fc670f.css rel=stylesheet></head><body><script src=/pleroma/admin/static/tinymce4.7.5/tinymce.min.js></script><div id=app></div><script type=text/javascript src=static/js/runtime.f40c8ec4.js></script><script type=text/javascript src=static/js/chunk-elementUI.1911151b.js></script><script type=text/javascript src=static/js/chunk-libs.fb0b7f4a.js></script><script type=text/javascript src=static/js/app.8e186193.js></script></body></html>

test/support/mrf_module_mock.ex

@@ -0,0 +1,13 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule MRFModuleMock do
+  @behaviour Pleroma.Web.ActivityPub.MRF
+
+  @impl true
+  def filter(message), do: {:ok, message}
+
+  @impl true
+  def describe, do: {:ok, %{mrf_module_mock: "some config data"}}
+end

test/tasks/database_test.exs

@@ -3,8 +3,11 @@
 # SPDX-License-Identifier: AGPL-3.0-only
 
 defmodule Mix.Tasks.Pleroma.DatabaseTest do
+  alias Pleroma.Object
   alias Pleroma.Repo
   alias Pleroma.User
+  alias Pleroma.Web.CommonAPI
+
   use Pleroma.DataCase
 
   import Pleroma.Factory
@@ -46,4 +49,37 @@ test "following and followers count are updated" do
       assert user.info.follower_count == 0
     end
   end
+
+  describe "running fix_likes_collections" do
+    test "it turns OrderedCollection likes into empty arrays" do
+      [user, user2] = insert_pair(:user)
+
+      {:ok, %{id: id, object: object}} = CommonAPI.post(user, %{"status" => "test"})
+      {:ok, %{object: object2}} = CommonAPI.post(user, %{"status" => "test test"})
+
+      CommonAPI.favorite(id, user2)
+
+      likes = %{
+        "first" =>
+          "http://mastodon.example.org/objects/dbdbc507-52c8-490d-9b7c-1e1d52e5c132/likes?page=1",
+        "id" => "http://mastodon.example.org/objects/dbdbc507-52c8-490d-9b7c-1e1d52e5c132/likes",
+        "totalItems" => 3,
+        "type" => "OrderedCollection"
+      }
+
+      new_data = Map.put(object2.data, "likes", likes)
+
+      object2
+      |> Ecto.Changeset.change(%{data: new_data})
+      |> Repo.update()
+
+      assert length(Object.get_by_id(object.id).data["likes"]) == 1
+      assert is_map(Object.get_by_id(object2.id).data["likes"])
+
+      assert :ok == Mix.Tasks.Pleroma.Database.run(["fix_likes_collections"])
+
+      assert length(Object.get_by_id(object.id).data["likes"]) == 1
+      assert Enum.empty?(Object.get_by_id(object2.id).data["likes"])
+    end
+  end
 end

test/uploaders/local_test.exs

@@ -0,0 +1,32 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Uploaders.LocalTest do
+  use Pleroma.DataCase
+  alias Pleroma.Uploaders.Local
+
+  describe "get_file/1" do
+    test "it returns path to local folder for files" do
+      assert Local.get_file("") == {:ok, {:static_dir, "test/uploads"}}
+    end
+  end
+
+  describe "put_file/1" do
+    test "put file to local folder" do
+      file_path = "local_upload/files/image.jpg"
+
+      file = %Pleroma.Upload{
+        name: "image.jpg",
+        content_type: "image/jpg",
+        path: file_path,
+        tempfile: Path.absname("test/fixtures/image_tmp.jpg")
+      }
+
+      assert Local.put_file(file) == :ok
+
+      assert Path.join([Local.upload_path(), file_path])
+             |> File.exists?()
+    end
+  end
+end

test/uploaders/mdii_test.exs

@@ -0,0 +1,50 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Uploaders.MDIITest do
+  use Pleroma.DataCase
+  alias Pleroma.Uploaders.MDII
+  import Tesla.Mock
+
+  describe "get_file/1" do
+    test "it returns path to local folder for files" do
+      assert MDII.get_file("") == {:ok, {:static_dir, "test/uploads"}}
+    end
+  end
+
+  describe "put_file/1" do
+    setup do
+      file_upload = %Pleroma.Upload{
+        name: "mdii-image.jpg",
+        content_type: "image/jpg",
+        path: "test_folder/mdii-image.jpg",
+        tempfile: Path.absname("test/fixtures/image_tmp.jpg")
+      }
+
+      [file_upload: file_upload]
+    end
+
+    test "save file", %{file_upload: file_upload} do
+      mock(fn
+        %{method: :post, url: "https://mdii.sakura.ne.jp/mdii-post.cgi?jpg"} ->
+          %Tesla.Env{status: 200, body: "mdii-image"}
+      end)
+
+      assert MDII.put_file(file_upload) ==
+               {:ok, {:url, "https://mdii.sakura.ne.jp/mdii-image.jpg"}}
+    end
+
+    test "save file to local if MDII  isn`t available", %{file_upload: file_upload} do
+      mock(fn
+        %{method: :post, url: "https://mdii.sakura.ne.jp/mdii-post.cgi?jpg"} ->
+          %Tesla.Env{status: 500}
+      end)
+
+      assert MDII.put_file(file_upload) == :ok
+
+      assert Path.join([Pleroma.Uploaders.Local.upload_path(), file_upload.path])
+             |> File.exists?()
+    end
+  end
+end

test/uploaders/s3_test.exs

@@ -0,0 +1,90 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Uploaders.S3Test do
+  use Pleroma.DataCase
+
+  alias Pleroma.Config
+  alias Pleroma.Uploaders.S3
+
+  import Mock
+  import ExUnit.CaptureLog
+
+  setup do
+    config = Config.get([Pleroma.Uploaders.S3])
+
+    Config.put([Pleroma.Uploaders.S3],
+      bucket: "test_bucket",
+      public_endpoint: "https://s3.amazonaws.com"
+    )
+
+    on_exit(fn ->
+      Config.put([Pleroma.Uploaders.S3], config)
+    end)
+
+    :ok
+  end
+
+  describe "get_file/1" do
+    test "it returns path to local folder for files" do
+      assert S3.get_file("test_image.jpg") == {
+               :ok,
+               {:url, "https://s3.amazonaws.com/test_bucket/test_image.jpg"}
+             }
+    end
+
+    test "it returns path without bucket when truncated_namespace set to ''" do
+      Config.put([Pleroma.Uploaders.S3],
+        bucket: "test_bucket",
+        public_endpoint: "https://s3.amazonaws.com",
+        truncated_namespace: ""
+      )
+
+      assert S3.get_file("test_image.jpg") == {
+               :ok,
+               {:url, "https://s3.amazonaws.com/test_image.jpg"}
+             }
+    end
+
+    test "it returns path with bucket namespace when namespace is set" do
+      Config.put([Pleroma.Uploaders.S3],
+        bucket: "test_bucket",
+        public_endpoint: "https://s3.amazonaws.com",
+        bucket_namespace: "family"
+      )
+
+      assert S3.get_file("test_image.jpg") == {
+               :ok,
+               {:url, "https://s3.amazonaws.com/family:test_bucket/test_image.jpg"}
+             }
+    end
+  end
+
+  describe "put_file/1" do
+    setup do
+      file_upload = %Pleroma.Upload{
+        name: "image-tet.jpg",
+        content_type: "image/jpg",
+        path: "test_folder/image-tet.jpg",
+        tempfile: Path.absname("test/fixtures/image_tmp.jpg")
+      }
+
+      [file_upload: file_upload]
+    end
+
+    test "save file", %{file_upload: file_upload} do
+      with_mock ExAws, request: fn _ -> {:ok, :ok} end do
+        assert S3.put_file(file_upload) == {:ok, {:file, "test_folder/image-tet.jpg"}}
+      end
+    end
+
+    test "returns error", %{file_upload: file_upload} do
+      with_mock ExAws, request: fn _ -> {:error, "S3 Upload failed"} end do
+        assert capture_log(fn ->
+                 assert S3.put_file(file_upload) == {:error, "S3 Upload failed"}
+               end) =~ "Elixir.Pleroma.Uploaders.S3: {:error, \"S3 Upload failed\"}"
+      end
+    end
+  end
+end

test/user_test.exs

@@ -526,7 +526,10 @@ test "it has required fields" do
     end
 
     test "it restricts some sizes" do
-      [bio: 5000, name: 100]
+      bio_limit = Pleroma.Config.get([:instance, :user_bio_length], 5000)
+      name_limit = Pleroma.Config.get([:instance, :user_name_length], 100)
+
+      [bio: bio_limit, name: name_limit]
       |> Enum.each(fn {field, size} ->
         string = String.pad_leading(".", size)
         cs = User.remote_user_creation(Map.put(@valid_remote, field, string))

test/web/activity_pub/mrf/mrf_test.exs

@@ -4,8 +4,8 @@ defmodule Pleroma.Web.ActivityPub.MRFTest do
 
   test "subdomains_regex/1" do
     assert MRF.subdomains_regex(["unsafe.tld", "*.unsafe.tld"]) == [
-             ~r/^unsafe.tld$/,
+             ~r/^unsafe.tld$/i,
-             ~r/^(.*\.)*unsafe.tld$/
+             ~r/^(.*\.)*unsafe.tld$/i
            ]
   end
 
@@ -13,7 +13,7 @@ test "subdomains_regex/1" do
     test "common domains" do
       regexes = MRF.subdomains_regex(["unsafe.tld", "unsafe2.tld"])
 
-      assert regexes == [~r/^unsafe.tld$/, ~r/^unsafe2.tld$/]
+      assert regexes == [~r/^unsafe.tld$/i, ~r/^unsafe2.tld$/i]
 
       assert MRF.subdomain_match?(regexes, "unsafe.tld")
       assert MRF.subdomain_match?(regexes, "unsafe2.tld")
@@ -24,7 +24,7 @@ test "common domains" do
     test "wildcard domains with one subdomain" do
       regexes = MRF.subdomains_regex(["*.unsafe.tld"])
 
-      assert regexes == [~r/^(.*\.)*unsafe.tld$/]
+      assert regexes == [~r/^(.*\.)*unsafe.tld$/i]
 
       assert MRF.subdomain_match?(regexes, "unsafe.tld")
       assert MRF.subdomain_match?(regexes, "sub.unsafe.tld")
@@ -35,12 +35,52 @@ test "wildcard domains with one subdomain" do
     test "wildcard domains with two subdomains" do
       regexes = MRF.subdomains_regex(["*.unsafe.tld"])
 
-      assert regexes == [~r/^(.*\.)*unsafe.tld$/]
+      assert regexes == [~r/^(.*\.)*unsafe.tld$/i]
 
       assert MRF.subdomain_match?(regexes, "unsafe.tld")
       assert MRF.subdomain_match?(regexes, "sub.sub.unsafe.tld")
       refute MRF.subdomain_match?(regexes, "sub.anotherunsafe.tld")
       refute MRF.subdomain_match?(regexes, "sub.unsafe.tldanother")
     end
+
+    test "matches are case-insensitive" do
+      regexes = MRF.subdomains_regex(["UnSafe.TLD", "UnSAFE2.Tld"])
+
+      assert regexes == [~r/^UnSafe.TLD$/i, ~r/^UnSAFE2.Tld$/i]
+
+      assert MRF.subdomain_match?(regexes, "UNSAFE.TLD")
+      assert MRF.subdomain_match?(regexes, "UNSAFE2.TLD")
+      assert MRF.subdomain_match?(regexes, "unsafe.tld")
+      assert MRF.subdomain_match?(regexes, "unsafe2.tld")
+
+      refute MRF.subdomain_match?(regexes, "EXAMPLE.COM")
+      refute MRF.subdomain_match?(regexes, "example.com")
+    end
+  end
+
+  describe "describe/0" do
+    test "it works as expected with noop policy" do
+      expected = %{
+        mrf_policies: ["NoOpPolicy"],
+        exclusions: false
+      }
+
+      {:ok, ^expected} = MRF.describe()
+    end
+
+    test "it works as expected with mock policy" do
+      config = Pleroma.Config.get([:instance, :rewrite_policy])
+      Pleroma.Config.put([:instance, :rewrite_policy], [MRFModuleMock])
+
+      expected = %{
+        mrf_policies: ["MRFModuleMock"],
+        mrf_module_mock: "some config data",
+        exclusions: false
+      }
+
+      {:ok, ^expected} = MRF.describe()
+
+      Pleroma.Config.put([:instance, :rewrite_policy], config)
+    end
   end
 end

test/web/activity_pub/mrf/vocabulary_policy_test.exs

@@ -0,0 +1,123 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ActivityPub.MRF.VocabularyPolicyTest do
+  use Pleroma.DataCase
+
+  alias Pleroma.Web.ActivityPub.MRF.VocabularyPolicy
+
+  describe "accept" do
+    test "it accepts based on parent activity type" do
+      config = Pleroma.Config.get([:mrf_vocabulary, :accept])
+      Pleroma.Config.put([:mrf_vocabulary, :accept], ["Like"])
+
+      message = %{
+        "type" => "Like",
+        "object" => "whatever"
+      }
+
+      {:ok, ^message} = VocabularyPolicy.filter(message)
+
+      Pleroma.Config.put([:mrf_vocabulary, :accept], config)
+    end
+
+    test "it accepts based on child object type" do
+      config = Pleroma.Config.get([:mrf_vocabulary, :accept])
+      Pleroma.Config.put([:mrf_vocabulary, :accept], ["Create", "Note"])
+
+      message = %{
+        "type" => "Create",
+        "object" => %{
+          "type" => "Note",
+          "content" => "whatever"
+        }
+      }
+
+      {:ok, ^message} = VocabularyPolicy.filter(message)
+
+      Pleroma.Config.put([:mrf_vocabulary, :accept], config)
+    end
+
+    test "it does not accept disallowed child objects" do
+      config = Pleroma.Config.get([:mrf_vocabulary, :accept])
+      Pleroma.Config.put([:mrf_vocabulary, :accept], ["Create", "Note"])
+
+      message = %{
+        "type" => "Create",
+        "object" => %{
+          "type" => "Article",
+          "content" => "whatever"
+        }
+      }
+
+      {:reject, nil} = VocabularyPolicy.filter(message)
+
+      Pleroma.Config.put([:mrf_vocabulary, :accept], config)
+    end
+
+    test "it does not accept disallowed parent types" do
+      config = Pleroma.Config.get([:mrf_vocabulary, :accept])
+      Pleroma.Config.put([:mrf_vocabulary, :accept], ["Announce", "Note"])
+
+      message = %{
+        "type" => "Create",
+        "object" => %{
+          "type" => "Note",
+          "content" => "whatever"
+        }
+      }
+
+      {:reject, nil} = VocabularyPolicy.filter(message)
+
+      Pleroma.Config.put([:mrf_vocabulary, :accept], config)
+    end
+  end
+
+  describe "reject" do
+    test "it rejects based on parent activity type" do
+      config = Pleroma.Config.get([:mrf_vocabulary, :reject])
+      Pleroma.Config.put([:mrf_vocabulary, :reject], ["Like"])
+
+      message = %{
+        "type" => "Like",
+        "object" => "whatever"
+      }
+
+      {:reject, nil} = VocabularyPolicy.filter(message)
+
+      Pleroma.Config.put([:mrf_vocabulary, :reject], config)
+    end
+
+    test "it rejects based on child object type" do
+      config = Pleroma.Config.get([:mrf_vocabulary, :reject])
+      Pleroma.Config.put([:mrf_vocabulary, :reject], ["Note"])
+
+      message = %{
+        "type" => "Create",
+        "object" => %{
+          "type" => "Note",
+          "content" => "whatever"
+        }
+      }
+
+      {:reject, nil} = VocabularyPolicy.filter(message)
+
+      Pleroma.Config.put([:mrf_vocabulary, :reject], config)
+    end
+
+    test "it passes through objects that aren't disallowed" do
+      config = Pleroma.Config.get([:mrf_vocabulary, :reject])
+      Pleroma.Config.put([:mrf_vocabulary, :reject], ["Like"])
+
+      message = %{
+        "type" => "Announce",
+        "object" => "whatever"
+      }
+
+      {:ok, ^message} = VocabularyPolicy.filter(message)
+
+      Pleroma.Config.put([:mrf_vocabulary, :reject], config)
+    end
+  end
+end

test/web/activity_pub/relay_test.exs

@@ -5,11 +5,71 @@
 defmodule Pleroma.Web.ActivityPub.RelayTest do
   use Pleroma.DataCase
 
+  alias Pleroma.Activity
+  alias Pleroma.Object
+  alias Pleroma.Web.ActivityPub.ActivityPub
   alias Pleroma.Web.ActivityPub.Relay
 
+  import Pleroma.Factory
+
   test "gets an actor for the relay" do
     user = Relay.get_actor()
+    assert user.ap_id == "#{Pleroma.Web.Endpoint.url()}/relay"
+  end
 
-    assert user.ap_id =~ "/relay"
+  describe "follow/1" do
+    test "returns errors when user not found" do
+      assert Relay.follow("test-ap-id") == {:error, "Could not fetch by AP id"}
+    end
+
+    test "returns activity" do
+      user = insert(:user)
+      service_actor = Relay.get_actor()
+      assert {:ok, %Activity{} = activity} = Relay.follow(user.ap_id)
+      assert activity.actor == "#{Pleroma.Web.Endpoint.url()}/relay"
+      assert user.ap_id in activity.recipients
+      assert activity.data["type"] == "Follow"
+      assert activity.data["actor"] == service_actor.ap_id
+      assert activity.data["object"] == user.ap_id
+    end
+  end
+
+  describe "unfollow/1" do
+    test "returns errors when user not found" do
+      assert Relay.unfollow("test-ap-id") == {:error, "Could not fetch by AP id"}
+    end
+
+    test "returns activity" do
+      user = insert(:user)
+      service_actor = Relay.get_actor()
+      ActivityPub.follow(service_actor, user)
+      assert {:ok, %Activity{} = activity} = Relay.unfollow(user.ap_id)
+      assert activity.actor == "#{Pleroma.Web.Endpoint.url()}/relay"
+      assert user.ap_id in activity.recipients
+      assert activity.data["type"] == "Undo"
+      assert activity.data["actor"] == service_actor.ap_id
+      assert activity.data["to"] == [user.ap_id]
+    end
+  end
+
+  describe "publish/1" do
+    test "returns error when activity not `Create` type" do
+      activity = insert(:like_activity)
+      assert Relay.publish(activity) == {:error, "Not implemented"}
+    end
+
+    test "returns error when activity not public" do
+      activity = insert(:direct_note_activity)
+      assert Relay.publish(activity) == {:error, false}
+    end
+
+    test "returns announce activity" do
+      service_actor = Relay.get_actor()
+      note = insert(:note_activity)
+      assert {:ok, %Activity{} = activity, %Object{} = obj} = Relay.publish(note)
+      assert activity.data["type"] == "Announce"
+      assert activity.data["actor"] == service_actor.ap_id
+      assert activity.data["object"] == obj.data["id"]
+    end
   end
 end

test/web/activity_pub/transmogrifier_test.exs

@@ -451,6 +451,27 @@ test "it ensures that address fields become lists" do
       assert !is_nil(data["cc"])
     end
 
+    test "it strips internal likes" do
+      data =
+        File.read!("test/fixtures/mastodon-post-activity.json")
+        |> Poison.decode!()
+
+      likes = %{
+        "first" =>
+          "http://mastodon.example.org/objects/dbdbc507-52c8-490d-9b7c-1e1d52e5c132/likes?page=1",
+        "id" => "http://mastodon.example.org/objects/dbdbc507-52c8-490d-9b7c-1e1d52e5c132/likes",
+        "totalItems" => 3,
+        "type" => "OrderedCollection"
+      }
+
+      object = Map.put(data["object"], "likes", likes)
+      data = Map.put(data, "object", object)
+
+      {:ok, %Activity{object: object}} = Transmogrifier.handle_incoming(data)
+
+      refute Map.has_key?(object.data, "likes")
+    end
+
     test "it works for incoming update activities" do
       data = File.read!("test/fixtures/mastodon-post-activity.json") |> Poison.decode!()
 
@@ -1063,14 +1084,7 @@ test "it strips internal fields of article" do
       assert is_nil(modified["object"]["announcements"])
       assert is_nil(modified["object"]["announcement_count"])
       assert is_nil(modified["object"]["context_id"])
-    end
+      assert is_nil(modified["object"]["likes"])
-
-    test "it adds like collection to object" do
-      activity = insert(:note_activity)
-      {:ok, modified} = Transmogrifier.prepare_outgoing(activity.data)
-
-      assert modified["object"]["likes"]["type"] == "OrderedCollection"
-      assert modified["object"]["likes"]["totalItems"] == 0
     end
 
     test "the directMessage flag is present" do

test/web/mastodon_api/account_view_test.exs

@@ -356,4 +356,31 @@ test "sanitizes display names" do
     result = AccountView.render("account.json", %{user: user})
     refute result.display_name == "<marquee> username </marquee>"
   end
+
+  describe "hiding follows/following" do
+    test "shows when follows/following are hidden and sets follower/following count to 0" do
+      user = insert(:user, info: %{hide_followers: true, hide_follows: true})
+      other_user = insert(:user)
+      {:ok, user, other_user, _activity} = CommonAPI.follow(user, other_user)
+      {:ok, _other_user, user, _activity} = CommonAPI.follow(other_user, user)
+
+      assert %{
+               followers_count: 0,
+               following_count: 0,
+               pleroma: %{hide_follows: true, hide_followers: true}
+             } = AccountView.render("account.json", %{user: user})
+    end
+
+    test "shows actual follower/following count to the account owner" do
+      user = insert(:user, info: %{hide_followers: true, hide_follows: true})
+      other_user = insert(:user)
+      {:ok, user, other_user, _activity} = CommonAPI.follow(user, other_user)
+      {:ok, _other_user, user, _activity} = CommonAPI.follow(other_user, user)
+
+      assert %{
+               followers_count: 1,
+               following_count: 1
+             } = AccountView.render("account.json", %{user: user, for: user})
+    end
+  end
 end

test/web/mastodon_api/mastodon_api_controller_test.exs

@@ -7,6 +7,7 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIControllerTest do
 
   alias Ecto.Changeset
   alias Pleroma.Activity
+  alias Pleroma.Config
   alias Pleroma.Notification
   alias Pleroma.Object
   alias Pleroma.Repo
@@ -86,11 +87,11 @@ test "the public timeline", %{conn: conn} do
   end
 
   test "the public timeline when public is set to false", %{conn: conn} do
-    public = Pleroma.Config.get([:instance, :public])
+    public = Config.get([:instance, :public])
-    Pleroma.Config.put([:instance, :public], false)
+    Config.put([:instance, :public], false)
 
     on_exit(fn ->
-      Pleroma.Config.put([:instance, :public], public)
+      Config.put([:instance, :public], public)
     end)
 
     assert conn
@@ -251,7 +252,7 @@ test "posting a fake status", %{conn: conn} do
     end
 
     test "posting a status with OGP link preview", %{conn: conn} do
-      Pleroma.Config.put([:rich_media, :enabled], true)
+      Config.put([:rich_media, :enabled], true)
 
       conn =
         conn
@@ -261,7 +262,7 @@ test "posting a status with OGP link preview", %{conn: conn} do
 
       assert %{"id" => id, "card" => %{"title" => "The Rock"}} = json_response(conn, 200)
       assert Activity.get_by_id(id)
-      Pleroma.Config.put([:rich_media, :enabled], false)
+      Config.put([:rich_media, :enabled], false)
     end
 
     test "posting a direct status", %{conn: conn} do
@@ -305,7 +306,7 @@ test "posting a poll", %{conn: conn} do
 
     test "option limit is enforced", %{conn: conn} do
       user = insert(:user)
-      limit = Pleroma.Config.get([:instance, :poll_limits, :max_options])
+      limit = Config.get([:instance, :poll_limits, :max_options])
 
       conn =
         conn
@@ -321,7 +322,7 @@ test "option limit is enforced", %{conn: conn} do
 
     test "option character limit is enforced", %{conn: conn} do
       user = insert(:user)
-      limit = Pleroma.Config.get([:instance, :poll_limits, :max_option_chars])
+      limit = Config.get([:instance, :poll_limits, :max_option_chars])
 
       conn =
         conn
@@ -340,7 +341,7 @@ test "option character limit is enforced", %{conn: conn} do
 
     test "minimal date limit is enforced", %{conn: conn} do
       user = insert(:user)
-      limit = Pleroma.Config.get([:instance, :poll_limits, :min_expiration])
+      limit = Config.get([:instance, :poll_limits, :min_expiration])
 
       conn =
         conn
@@ -359,7 +360,7 @@ test "minimal date limit is enforced", %{conn: conn} do
 
     test "maximum date limit is enforced", %{conn: conn} do
       user = insert(:user)
-      limit = Pleroma.Config.get([:instance, :poll_limits, :max_expiration])
+      limit = Config.get([:instance, :poll_limits, :max_expiration])
 
       conn =
         conn
@@ -1634,12 +1635,12 @@ test "returns the relationships for the current user", %{conn: conn} do
 
   describe "media upload" do
     setup do
-      upload_config = Pleroma.Config.get([Pleroma.Upload])
+      upload_config = Config.get([Pleroma.Upload])
-      proxy_config = Pleroma.Config.get([:media_proxy])
+      proxy_config = Config.get([:media_proxy])
 
       on_exit(fn ->
-        Pleroma.Config.put([Pleroma.Upload], upload_config)
+        Config.put([Pleroma.Upload], upload_config)
-        Pleroma.Config.put([:media_proxy], proxy_config)
+        Config.put([:media_proxy], proxy_config)
       end)
 
       user = insert(:user)
@@ -2582,7 +2583,7 @@ test "get instance information", %{conn: conn} do
     conn = get(conn, "/api/v1/instance")
     assert result = json_response(conn, 200)
 
-    email = Pleroma.Config.get([:instance, :email])
+    email = Config.get([:instance, :email])
     # Note: not checking for "max_toot_chars" since it's optional
     assert %{
              "uri" => _,
@@ -2667,7 +2668,7 @@ test "put settings", %{conn: conn} do
 
   describe "pinned statuses" do
     setup do
-      Pleroma.Config.put([:instance, :max_pinned_statuses], 1)
+      Config.put([:instance, :max_pinned_statuses], 1)
 
       user = insert(:user)
       {:ok, activity} = CommonAPI.post(user, %{"status" => "HI!!!"})
@@ -2767,10 +2768,10 @@ test "max pinned statuses", %{conn: conn, user: user, activity: activity_one} do
 
   describe "cards" do
     setup do
-      Pleroma.Config.put([:rich_media, :enabled], true)
+      Config.put([:rich_media, :enabled], true)
 
       on_exit(fn ->
-        Pleroma.Config.put([:rich_media, :enabled], false)
+        Config.put([:rich_media, :enabled], false)
       end)
 
       user = insert(:user)
@@ -2902,8 +2903,10 @@ test "bookmarks" do
 
   describe "conversation muting" do
     setup do
+      post_user = insert(:user)
       user = insert(:user)
-      {:ok, activity} = CommonAPI.post(user, %{"status" => "HIE"})
+
+      {:ok, activity} = CommonAPI.post(post_user, %{"status" => "HIE"})
 
       [user: user, activity: activity]
     end
@@ -2996,7 +2999,7 @@ test "comment must be up to the size specified in the config", %{
       reporter: reporter,
       target_user: target_user
     } do
-      max_size = Pleroma.Config.get([:instance, :max_report_comment_size], 1000)
+      max_size = Config.get([:instance, :max_report_comment_size], 1000)
       comment = String.pad_trailing("a", max_size + 1, "a")
 
       error = %{"error" => "Comment must be up to #{max_size} characters"}
@@ -3125,15 +3128,15 @@ test "redirects not logged-in users to the login page on private instances", %{
       conn: conn,
       path: path
     } do
-      is_public = Pleroma.Config.get([:instance, :public])
+      is_public = Config.get([:instance, :public])
-      Pleroma.Config.put([:instance, :public], false)
+      Config.put([:instance, :public], false)
 
       conn = get(conn, path)
 
       assert conn.status == 302
       assert redirected_to(conn) == "/web/login"
 
-      Pleroma.Config.put([:instance, :public], is_public)
+      Config.put([:instance, :public], is_public)
     end
 
     test "does not redirect logged in users to the login page", %{conn: conn, path: path} do
@@ -3876,8 +3879,8 @@ test "it sends an email to user", %{user: user} do
       token_record = Repo.get_by(Pleroma.PasswordResetToken, user_id: user.id)
 
       email = Pleroma.Emails.UserEmail.password_reset_email(user, token_record.token)
-      notify_email = Pleroma.Config.get([:instance, :notify_email])
+      notify_email = Config.get([:instance, :notify_email])
-      instance_name = Pleroma.Config.get([:instance, :name])
+      instance_name = Config.get([:instance, :name])
 
       assert_email_sent(
         from: {instance_name, notify_email},
@@ -3909,11 +3912,11 @@ test "it returns 400 when user is not local", %{conn: conn, user: user} do
 
   describe "POST /api/v1/pleroma/accounts/confirmation_resend" do
     setup do
-      setting = Pleroma.Config.get([:instance, :account_activation_required])
+      setting = Config.get([:instance, :account_activation_required])
 
       unless setting do
-        Pleroma.Config.put([:instance, :account_activation_required], true)
+        Config.put([:instance, :account_activation_required], true)
-        on_exit(fn -> Pleroma.Config.put([:instance, :account_activation_required], setting) end)
+        on_exit(fn -> Config.put([:instance, :account_activation_required], setting) end)
       end
 
       user = insert(:user)
@@ -3939,8 +3942,8 @@ test "resend account confirmation email", %{conn: conn, user: user} do
       ObanHelpers.perform_all()
 
       email = Pleroma.Emails.UserEmail.account_confirmation_email(user)
-      notify_email = Pleroma.Config.get([:instance, :notify_email])
+      notify_email = Config.get([:instance, :notify_email])
-      instance_name = Pleroma.Config.get([:instance, :name])
+      instance_name = Config.get([:instance, :name])
 
       assert_email_sent(
         from: {instance_name, notify_email},
@@ -3949,4 +3952,84 @@ test "resend account confirmation email", %{conn: conn, user: user} do
       )
     end
   end
+
+  describe "GET /api/v1/suggestions" do
+    setup do
+      user = insert(:user)
+      other_user = insert(:user)
+      config = Config.get(:suggestions)
+      on_exit(fn -> Config.put(:suggestions, config) end)
+
+      host = Config.get([Pleroma.Web.Endpoint, :url, :host])
+      url500 = "http://test500?#{host}&#{user.nickname}"
+      url200 = "http://test200?#{host}&#{user.nickname}"
+
+      mock(fn
+        %{method: :get, url: ^url500} ->
+          %Tesla.Env{status: 500, body: "bad request"}
+
+        %{method: :get, url: ^url200} ->
+          %Tesla.Env{
+            status: 200,
+            body:
+              ~s([{"acct":"yj455","avatar":"https://social.heldscal.la/avatar/201.jpeg","avatar_static":"https://social.heldscal.la/avatar/s/201.jpeg"}, {"acct":"#{
+                other_user.ap_id
+              }","avatar":"https://social.heldscal.la/avatar/202.jpeg","avatar_static":"https://social.heldscal.la/avatar/s/202.jpeg"}])
+          }
+      end)
+
+      [user: user, other_user: other_user]
+    end
+
+    test "returns empty result when suggestions disabled", %{conn: conn, user: user} do
+      Config.put([:suggestions, :enabled], false)
+
+      res =
+        conn
+        |> assign(:user, user)
+        |> get("/api/v1/suggestions")
+        |> json_response(200)
+
+      assert res == []
+    end
+
+    test "returns error", %{conn: conn, user: user} do
+      Config.put([:suggestions, :enabled], true)
+      Config.put([:suggestions, :third_party_engine], "http://test500?{{host}}&{{user}}")
+
+      res =
+        conn
+        |> assign(:user, user)
+        |> get("/api/v1/suggestions")
+        |> json_response(500)
+
+      assert res == "Something went wrong"
+    end
+
+    test "returns suggestions", %{conn: conn, user: user, other_user: other_user} do
+      Config.put([:suggestions, :enabled], true)
+      Config.put([:suggestions, :third_party_engine], "http://test200?{{host}}&{{user}}")
+
+      res =
+        conn
+        |> assign(:user, user)
+        |> get("/api/v1/suggestions")
+        |> json_response(200)
+
+      assert res == [
+               %{
+                 "acct" => "yj455",
+                 "avatar" => "https://social.heldscal.la/avatar/201.jpeg",
+                 "avatar_static" => "https://social.heldscal.la/avatar/s/201.jpeg",
+                 "id" => 0
+               },
+               %{
+                 "acct" => other_user.ap_id,
+                 "avatar" => "https://social.heldscal.la/avatar/202.jpeg",
+                 "avatar_static" => "https://social.heldscal.la/avatar/s/202.jpeg",
+                 "id" => other_user.id
+               }
+             ]
+    end
+  end
 end

test/web/mastodon_api/mastodon_api_test.exs

@@ -0,0 +1,103 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.MastodonAPI.MastodonAPITest do
+  use Pleroma.Web.ConnCase
+
+  alias Pleroma.Notification
+  alias Pleroma.ScheduledActivity
+  alias Pleroma.User
+  alias Pleroma.Web.MastodonAPI.MastodonAPI
+  alias Pleroma.Web.TwitterAPI.TwitterAPI
+
+  import Pleroma.Factory
+
+  describe "follow/3" do
+    test "returns error when user deactivated" do
+      follower = insert(:user)
+      user = insert(:user, local: true, info: %{deactivated: true})
+      {:error, error} = MastodonAPI.follow(follower, user)
+      assert error == "Could not follow user: You are deactivated."
+    end
+
+    test "following for user" do
+      follower = insert(:user)
+      user = insert(:user)
+      {:ok, follower} = MastodonAPI.follow(follower, user)
+      assert User.following?(follower, user)
+    end
+
+    test "returns ok if user already followed" do
+      follower = insert(:user)
+      user = insert(:user)
+      {:ok, follower} = User.follow(follower, user)
+      {:ok, follower} = MastodonAPI.follow(follower, refresh_record(user))
+      assert User.following?(follower, user)
+    end
+  end
+
+  describe "get_followers/2" do
+    test "returns user followers" do
+      follower1_user = insert(:user)
+      follower2_user = insert(:user)
+      user = insert(:user)
+      {:ok, _follower1_user} = User.follow(follower1_user, user)
+      {:ok, follower2_user} = User.follow(follower2_user, user)
+
+      assert MastodonAPI.get_followers(user, %{"limit" => 1}) == [follower2_user]
+    end
+  end
+
+  describe "get_friends/2" do
+    test "returns user friends" do
+      user = insert(:user)
+      followed_one = insert(:user)
+      followed_two = insert(:user)
+      followed_three = insert(:user)
+
+      {:ok, user} = User.follow(user, followed_one)
+      {:ok, user} = User.follow(user, followed_two)
+      {:ok, user} = User.follow(user, followed_three)
+      res = MastodonAPI.get_friends(user)
+
+      assert length(res) == 3
+      assert Enum.member?(res, refresh_record(followed_three))
+      assert Enum.member?(res, refresh_record(followed_two))
+      assert Enum.member?(res, refresh_record(followed_one))
+    end
+  end
+
+  describe "get_notifications/2" do
+    test "returns notifications for user" do
+      user = insert(:user)
+      subscriber = insert(:user)
+
+      User.subscribe(subscriber, user)
+
+      {:ok, status} = TwitterAPI.create_status(user, %{"status" => "Akariiiin"})
+      {:ok, status1} = TwitterAPI.create_status(user, %{"status" => "Magi"})
+      {:ok, [notification]} = Notification.create_notifications(status)
+      {:ok, [notification1]} = Notification.create_notifications(status1)
+      res = MastodonAPI.get_notifications(subscriber)
+
+      assert Enum.member?(Enum.map(res, & &1.id), notification.id)
+      assert Enum.member?(Enum.map(res, & &1.id), notification1.id)
+    end
+  end
+
+  describe "get_scheduled_activities/2" do
+    test "returns user scheduled activities" do
+      user = insert(:user)
+
+      today =
+        NaiveDateTime.utc_now()
+        |> NaiveDateTime.add(:timer.minutes(6), :millisecond)
+        |> NaiveDateTime.to_iso8601()
+
+      attrs = %{params: %{}, scheduled_at: today}
+      {:ok, schedule} = ScheduledActivity.create(user, attrs)
+      assert MastodonAPI.get_scheduled_activities(user) == [schedule]
+    end
+  end
+end

test/web/node_info_test.exs

@@ -85,6 +85,9 @@ test "it returns the safe_dm_mentions feature if enabled", %{conn: conn} do
   end
 
   test "it shows MRF transparency data if enabled", %{conn: conn} do
+    config = Pleroma.Config.get([:instance, :rewrite_policy])
+    Pleroma.Config.put([:instance, :rewrite_policy], [Pleroma.Web.ActivityPub.MRF.SimplePolicy])
+
     option = Pleroma.Config.get([:instance, :mrf_transparency])
     Pleroma.Config.put([:instance, :mrf_transparency], true)
 
@@ -98,11 +101,15 @@ test "it shows MRF transparency data if enabled", %{conn: conn} do
 
     assert response["metadata"]["federation"]["mrf_simple"] == simple_config
 
+    Pleroma.Config.put([:instance, :rewrite_policy], config)
     Pleroma.Config.put([:instance, :mrf_transparency], option)
     Pleroma.Config.put(:mrf_simple, %{})
   end
 
   test "it performs exclusions from MRF transparency data if configured", %{conn: conn} do
+    config = Pleroma.Config.get([:instance, :rewrite_policy])
+    Pleroma.Config.put([:instance, :rewrite_policy], [Pleroma.Web.ActivityPub.MRF.SimplePolicy])
+
     option = Pleroma.Config.get([:instance, :mrf_transparency])
     Pleroma.Config.put([:instance, :mrf_transparency], true)
 
@@ -122,6 +129,7 @@ test "it performs exclusions from MRF transparency data if configured", %{conn:
     assert response["metadata"]["federation"]["mrf_simple"] == expected_config
     assert response["metadata"]["federation"]["exclusions"] == true
 
+    Pleroma.Config.put([:instance, :rewrite_policy], config)
     Pleroma.Config.put([:instance, :mrf_transparency], option)
     Pleroma.Config.put([:instance, :mrf_transparency_exclusions], exclusions)
     Pleroma.Config.put(:mrf_simple, %{})