Merge branch 'relay-fix-admin-fe' into 'develop'

Relay fix for admin-fe

See merge request pleroma/pleroma!2902

CHANGELOG.md

@@ -6,19 +6,20 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 ## [unreleased]
 
 ### Changed
+
 - **Breaking:** The default descriptions on uploads are now empty. The old behavior (filename as default) can be configured, see the cheat sheet.
 - **Breaking:** Added the ObjectAgePolicy to the default set of MRFs. This will delist and strip the follower collection of any message received that is older than 7 days. This will stop users from seeing very old messages in the timelines. The messages can still be viewed on the user's page and in conversations. They also still trigger notifications.
 - **Breaking:** Elixir >=1.9 is now required (was >= 1.8)
 - **Breaking:** Configuration: `:auto_linker, :opts` moved to `:pleroma, Pleroma.Formatter`. Old config namespace is deprecated.
+- **Breaking:** Configuration: `:instance, welcome_user_nickname` moved to `:welcome, :direct_message, :sender_nickname`, `:instance, :welcome_message` moved to `:welcome, :direct_message, :message`. Old config namespace is deprecated.
+- **Breaking:** LDAP: Fallback to local database authentication has been removed for security reasons and lack of a mechanism to ensure the passwords are synchronized when LDAP passwords are updated.
+- **Breaking** Changed defaults for `:restrict_unauthenticated` so that when `:instance, :public` is set to `false` then all `:restrict_unauthenticated` items be effectively set to `true`. If you'd like to allow unauthenticated access to specific API endpoints on a private instance, please explicitly set `:restrict_unauthenticated` to non-default value in `config/prod.secret.exs`.
 - In Conversations, return only direct messages as `last_status`
 - Using the `only_media` filter on timelines will now exclude reblog media
 - MFR policy to set global expiration for all local Create activities
 - OGP rich media parser merged with TwitterCard
 - Configuration: `:instance, rewrite_policy` moved to `:mrf, policies`, `:instance, :mrf_transparency` moved to `:mrf, :transparency`, `:instance, :mrf_transparency_exclusions` moved to `:mrf, :transparency_exclusions`. Old config namespace is deprecated.
 - Configuration: `:media_proxy, whitelist` format changed to host with scheme (e.g. `http://example.com` instead of `example.com`). Domain format is deprecated.
-- **Breaking:** Configuration: `:instance, welcome_user_nickname` moved to `:welcome, :direct_message, :sender_nickname`, `:instance, :welcome_message` moved to `:welcome, :direct_message, :message`. Old config namespace is deprecated.
-- **Breaking:** LDAP: Fallback to local database authentication has been removed for security reasons and lack of a mechanism to ensure the passwords are synchronized when LDAP passwords are updated.
-- **Breaking** Changed defaults for `:restrict_unauthenticated` so that when `:instance, :public` is set to `false` then all `:restrict_unauthenticated` items be effectively set to `true`. If you'd like to allow unauthenticated access to specific API endpoints on a private instance, please explicitly set `:restrict_unauthenticated` to non-default value in `config/prod.secret.exs`. 
 
 <details>
   <summary>API Changes</summary>
@@ -26,29 +27,30 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - **Breaking:** Pleroma API: The routes to update avatar, banner and background have been removed.
 - **Breaking:** Image description length is limited now.
 - **Breaking:** Emoji API: changed methods and renamed routes.
+- **Breaking:** Notification Settings API for suppressing notifications has been simplified down to `block_from_strangers`.
+- **Breaking:** Notification Settings API option for hiding push notification contents has been renamed to `hide_notification_contents`.
 - MastodonAPI: Allow removal of avatar, banner and background.
 - Streaming: Repeats of a user's posts will no longer be pushed to the user's stream.
 - Mastodon API: Added `pleroma.metadata.fields_limits` to /api/v1/instance
 - Mastodon API: On deletion, returns the original post text.
 - Mastodon API: Add `pleroma.unread_count` to the Marker entity.
-- **Breaking:** Notification Settings API for suppressing notifications
-  has been simplified down to `block_from_strangers`.
-- **Breaking:** Notification Settings API option for hiding push notification
-  contents has been renamed to `hide_notification_contents`
 - Mastodon API: Added `pleroma.metadata.post_formats` to /api/v1/instance
 - Mastodon API (legacy): Allow query parameters for `/api/v1/domain_blocks`, e.g. `/api/v1/domain_blocks?domain=badposters.zone`
 - Pleroma API: `/api/pleroma/captcha` responses now include `seconds_valid` with an integer value.
+
 </details>
 
 <details>
   <summary>Admin API Changes</summary>
 
+- **Breaking** Changed relay `/api/pleroma/admin/relay` endpoints response format.
 - Status visibility stats: now can return stats per instance.
-
 - Mix task to refresh counter cache (`mix pleroma.refresh_counter_cache`)
+
 </details>
 
 ### Removed
+
 - **Breaking:** removed `with_move` parameter from notifications timeline.
 
 ### Added
@@ -108,6 +110,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Emoji Packs could not be listed when instance was set to `public: false`
 - Fix whole_word always returning false on filter get requests
 - Migrations not working on OTP releases if the database was connected over ssl
+- Fix relay following
 
 ## [Unreleased (patch)]
 

docs/API/admin_api.md

@@ -313,31 +313,53 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
   - On failure: `Not found`
   - On success: JSON array of user's latest statuses
 
+## `GET /api/pleroma/admin/relay`
+
+### List Relays
+
+Params: none
+Response:
+
+* On success: JSON array of relays
+
+```json
+[
+  {"actor": "https://example.com/relay", "followed_back": true},
+  {"actor": "https://example2.com/relay", "followed_back": false}
+]
+```
+
 ## `POST /api/pleroma/admin/relay`
 
 ### Follow a Relay
 
-- Params:
+Params:
-  - `relay_url`
+
-- Response:
+* `relay_url`
-  - On success: URL of the followed relay
+
+Response:
+
+* On success: relay json object
+
+```json
+{"actor": "https://example.com/relay", "followed_back": true}
+```
 
 ## `DELETE /api/pleroma/admin/relay`
 
 ### Unfollow a Relay
 
-- Params:
+Params:
-  - `relay_url`
-- Response:
-  - On success: URL of the unfollowed relay
 
-## `GET /api/pleroma/admin/relay`
+* `relay_url`
 
-### List Relays
+Response:
 
-- Params: none
+* On success: URL of the unfollowed relay
-- Response:
+
-  - On success: JSON array of relays
+```json
+{"https://example.com/relay"}
+```
 
 ## `POST /api/pleroma/admin/users/invite_token`
 

lib/mix/tasks/pleroma/relay.ex

@@ -35,10 +35,16 @@ def run(["unfollow", target]) do
   def run(["list"]) do
     start_pleroma()
 
-    with {:ok, list} <- Relay.list(true) do
+    with {:ok, list} <- Relay.list() do
-      list |> Enum.each(&shell_info(&1))
+      Enum.each(list, &print_relay_url/1)
     else
       {:error, e} -> shell_error("Error while fetching relay subscription list: #{inspect(e)}")
     end
   end
+
+  defp print_relay_url(%{followed_back: false} = relay) do
+    shell_info("#{relay.actor} - no Accept received (relay didn't follow back)")
+  end
+
+  defp print_relay_url(relay), do: shell_info(relay.actor)
 end

lib/pleroma/following_relationship.ex

@@ -264,4 +264,12 @@ defp validate_following_id_follower_id_inequality(%Changeset{} = changeset) do
       end
     end)
   end
+
+  @spec following_ap_ids(User.t()) :: [String.t()]
+  def following_ap_ids(%User{} = user) do
+    user
+    |> following_query()
+    |> select([r, u], u.ap_id)
+    |> Repo.all()
+  end
 end

lib/pleroma/user.ex

@@ -247,6 +247,13 @@ def unquote(:"#{outgoing_relation_target}_ap_ids")(user, restrict_deactivated? \
     end
   end
 
+  defdelegate following_count(user), to: FollowingRelationship
+  defdelegate following(user), to: FollowingRelationship
+  defdelegate following?(follower, followed), to: FollowingRelationship
+  defdelegate following_ap_ids(user), to: FollowingRelationship
+  defdelegate get_follow_requests(user), to: FollowingRelationship
+  defdelegate search(query, opts \\ []), to: User.Search
+
   @doc """
   Dumps Flake Id to SQL-compatible format (16-byte UUID).
   E.g. "9pQtDGXuq4p3VlcJEm" -> <<0, 0, 1, 110, 179, 218, 42, 92, 213, 41, 44, 227, 95, 213, 0, 0>>
@@ -372,8 +379,6 @@ def restrict_deactivated(query) do
     from(u in query, where: u.deactivated != ^true)
   end
 
-  defdelegate following_count(user), to: FollowingRelationship
-
   defp truncate_fields_param(params) do
     if Map.has_key?(params, :fields) do
       Map.put(params, :fields, Enum.map(params[:fields], &truncate_field/1))
@@ -868,8 +873,6 @@ def follow_all(follower, followeds) do
     set_cache(follower)
   end
 
-  defdelegate following(user), to: FollowingRelationship
-
   def follow(%User{} = follower, %User{} = followed, state \\ :follow_accept) do
     deny_follow_blocked = Config.get([:user, :deny_follow_blocked])
 
@@ -923,8 +926,6 @@ defp do_unfollow(%User{} = follower, %User{} = followed) do
     end
   end
 
-  defdelegate following?(follower, followed), to: FollowingRelationship
-
   @doc "Returns follow state as Pleroma.FollowingRelationship.State value"
   def get_follow_state(%User{} = follower, %User{} = following) do
     following_relationship = FollowingRelationship.get(follower, following)
@@ -1189,8 +1190,6 @@ def get_friends_ids(user, page \\ nil) do
     |> Repo.all()
   end
 
-  defdelegate get_follow_requests(user), to: FollowingRelationship
-
   def increase_note_count(%User{} = user) do
     User
     |> where(id: ^user.id)
@@ -2163,8 +2162,6 @@ def get_ap_ids_by_nicknames(nicknames) do
     |> Repo.all()
   end
 
-  defdelegate search(query, opts \\ []), to: User.Search
-
   defp put_password_hash(
          %Ecto.Changeset{valid?: true, changes: %{password: password}} = changeset
        ) do

lib/pleroma/web/activity_pub/activity_pub.ex

@@ -1344,9 +1344,8 @@ def fetch_and_prepare_user_from_ap_id(ap_id) do
   end
 
   def maybe_handle_clashing_nickname(data) do
-    nickname = data[:nickname]
+    with nickname when is_binary(nickname) <- data[:nickname],
-
+         %User{} = old_user <- User.get_by_nickname(nickname),
-    with %User{} = old_user <- User.get_by_nickname(nickname),
          {_, false} <- {:ap_id_comparison, data[:ap_id] == old_user.ap_id} do
       Logger.info(
         "Found an old user for #{nickname}, the old ap id is #{old_user.ap_id}, new one is #{
@@ -1360,7 +1359,7 @@ def maybe_handle_clashing_nickname(data) do
     else
       {:ap_id_comparison, true} ->
         Logger.info(
-          "Found an old user for #{nickname}, but the ap id #{data[:ap_id]} is the same as the new user. Race condition? Not changing anything."
+          "Found an old user for #{data[:nickname]}, but the ap id #{data[:ap_id]} is the same as the new user. Race condition? Not changing anything."
         )
 
       _ ->

lib/pleroma/web/activity_pub/builder.ex

@@ -215,7 +215,7 @@ def announce(actor, object, options \\ []) do
 
     to =
       cond do
-        actor.ap_id == Relay.relay_ap_id() ->
+        actor.ap_id == Relay.ap_id() ->
           [actor.follower_address]
 
         public? ->

lib/pleroma/web/activity_pub/relay.ex

@@ -10,19 +10,13 @@ defmodule Pleroma.Web.ActivityPub.Relay do
   alias Pleroma.Web.CommonAPI
   require Logger
 
-  @relay_nickname "relay"
+  @nickname "relay"
 
-  def get_actor do
+  @spec ap_id() :: String.t()
-    actor =
+  def ap_id, do: "#{Pleroma.Web.Endpoint.url()}/#{@nickname}"
-      relay_ap_id()
-      |> User.get_or_create_service_actor_by_ap_id(@relay_nickname)
 
-    actor
+  @spec get_actor() :: User.t() | nil
-  end
+  def get_actor, do: User.get_or_create_service_actor_by_ap_id(ap_id(), @nickname)
-
-  def relay_ap_id do
-    "#{Pleroma.Web.Endpoint.url()}/relay"
-  end
 
   @spec follow(String.t()) :: {:ok, Activity.t()} | {:error, any()}
   def follow(target_instance) do
@@ -61,34 +55,38 @@ def publish(%Activity{data: %{"type" => "Create"}} = activity) do
 
   def publish(_), do: {:error, "Not implemented"}
 
-  @spec list(boolean()) :: {:ok, [String.t()]} | {:error, any()}
+  @spec list() :: {:ok, [%{actor: String.t(), followed_back: boolean()}]} | {:error, any()}
-  def list(with_not_accepted \\ false) do
+  def list do
     with %User{} = user <- get_actor() do
       accepted =
         user
-        |> User.following()
+        |> following()
-        |> Enum.map(fn entry -> URI.parse(entry).host end)
+        |> Enum.map(fn actor -> %{actor: actor, followed_back: true} end)
-        |> Enum.uniq()
 
-      list =
-        if with_not_accepted do
       without_accept =
         user
         |> Pleroma.Activity.following_requests_for_actor()
-            |> Enum.map(fn a -> URI.parse(a.data["object"]).host <> " (no Accept received)" end)
+        |> Enum.map(fn activity -> %{actor: activity.data["object"], followed_back: false} end)
         |> Enum.uniq()
 
-          accepted ++ without_accept
+      {:ok, accepted ++ without_accept}
-        else
-          accepted
-        end
-
-      {:ok, list}
     else
       error -> format_error(error)
     end
   end
 
+  @spec following() :: [String.t()]
+  def following do
+    get_actor()
+    |> following()
+  end
+
+  defp following(user) do
+    user
+    |> User.following_ap_ids()
+    |> Enum.uniq()
+  end
+
   defp format_error({:error, error}), do: format_error(error)
 
   defp format_error(error) do

lib/pleroma/web/admin_api/controllers/relay_controller.ex

@@ -39,7 +39,7 @@ def follow(%{assigns: %{user: admin}, body_params: %{relay_url: target}} = conn,
         target: target
       })
 
-      json(conn, target)
+      json(conn, %{actor: target, followed_back: target in Relay.following()})
     else
       _ ->
         conn

lib/pleroma/web/api_spec/operations/admin/relay_operation.ex

@@ -27,8 +27,7 @@ def index_operation do
             properties: %{
               relays: %Schema{
                 type: :array,
-                items: %Schema{type: :string},
+                items: relay()
-                example: ["lain.com", "mstdn.io"]
               }
             }
           })
@@ -43,19 +42,9 @@ def follow_operation do
       operationId: "AdminAPI.RelayController.follow",
       security: [%{"oAuth" => ["write:follows"]}],
       parameters: admin_api_params(),
-      requestBody:
+      requestBody: request_body("Parameters", relay_url()),
-        request_body("Parameters", %Schema{
-          type: :object,
-          properties: %{
-            relay_url: %Schema{type: :string, format: :uri}
-          }
-        }),
       responses: %{
-        200 =>
+        200 => Operation.response("Status", "application/json", relay())
-          Operation.response("Status", "application/json", %Schema{
-            type: :string,
-            example: "http://mastodon.example.org/users/admin"
-          })
       }
     }
   end
@@ -67,13 +56,7 @@ def unfollow_operation do
       operationId: "AdminAPI.RelayController.unfollow",
       security: [%{"oAuth" => ["write:follows"]}],
       parameters: admin_api_params(),
-      requestBody:
+      requestBody: request_body("Parameters", relay_url()),
-        request_body("Parameters", %Schema{
-          type: :object,
-          properties: %{
-            relay_url: %Schema{type: :string, format: :uri}
-          }
-        }),
       responses: %{
         200 =>
           Operation.response("Status", "application/json", %Schema{
@@ -83,4 +66,29 @@ def unfollow_operation do
       }
     }
   end
+
+  defp relay do
+    %Schema{
+      type: :object,
+      properties: %{
+        actor: %Schema{
+          type: :string,
+          example: "https://example.com/relay"
+        },
+        followed_back: %Schema{
+          type: :boolean,
+          description: "Is relay followed back by this actor?"
+        }
+      }
+    }
+  end
+
+  defp relay_url do
+    %Schema{
+      type: :object,
+      properties: %{
+        relay_url: %Schema{type: :string, format: :uri}
+      }
+    }
+  end
 end

test/tasks/relay_test.exs

@@ -42,7 +42,11 @@ test "relay is followed" do
       assert activity.data["object"] == target_user.ap_id
 
       :ok = Mix.Tasks.Pleroma.Relay.run(["list"])
-      assert_receive {:mix_shell, :info, ["mastodon.example.org (no Accept received)"]}
+
+      assert_receive {:mix_shell, :info,
+                      [
+                        "http://mastodon.example.org/users/admin - no Accept received (relay didn't follow back)"
+                      ]}
     end
   end
 
@@ -95,8 +99,8 @@ test "Prints relay subscription list" do
 
       :ok = Mix.Tasks.Pleroma.Relay.run(["list"])
 
-      assert_receive {:mix_shell, :info, ["mstdn.io"]}
+      assert_receive {:mix_shell, :info, ["https://mstdn.io/users/mayuutann"]}
-      assert_receive {:mix_shell, :info, ["mastodon.example.org"]}
+      assert_receive {:mix_shell, :info, ["http://mastodon.example.org/users/admin"]}
     end
   end
 end

test/web/activity_pub/activity_pub_controller_test.exs

@@ -533,7 +533,7 @@ test "accept follow activity", %{conn: conn} do
       end)
 
       :ok = Mix.Tasks.Pleroma.Relay.run(["list"])
-      assert_receive {:mix_shell, :info, ["relay.mastodon.host"]}
+      assert_receive {:mix_shell, :info, ["https://relay.mastodon.host/actor"]}
     end
 
     @tag capture_log: true

test/web/admin_api/controllers/relay_controller_test.exs

@@ -39,8 +39,10 @@ test "POST /relay", %{conn: conn, admin: admin} do
           relay_url: "http://mastodon.example.org/users/admin"
         })
 
-      assert json_response_and_validate_schema(conn, 200) ==
+      assert json_response_and_validate_schema(conn, 200) == %{
-               "http://mastodon.example.org/users/admin"
+               "actor" => "http://mastodon.example.org/users/admin",
+               "followed_back" => false
+             }
 
       log_entry = Repo.one(ModerationLog)
 
@@ -59,8 +61,13 @@ test "GET /relay", %{conn: conn} do
 
       conn = get(conn, "/api/pleroma/admin/relay")
 
-      assert json_response_and_validate_schema(conn, 200)["relays"] --
+      assert json_response_and_validate_schema(conn, 200)["relays"] == [
-               ["mastodon.example.org", "mstdn.io"] == []
+               %{
+                 "actor" => "http://mastodon.example.org/users/admin",
+                 "followed_back" => true
+               },
+               %{"actor" => "https://mstdn.io/users/mayuutann", "followed_back" => true}
+             ]
     end
 
     test "DELETE /relay", %{conn: conn, admin: admin} do