Customer API endpoint doesn't update metadata about papercups

[jeepers3327]

Hi @CharlesMangwa,

I've found the bug and how to fix it, though I need to ask for confirmation from @reichert621 regarding the fix.

From line 131 - 142, it fetches metadata from the PUT request, merges it with extra fields and then transform some of fields after sanitizing it.

papercups/lib/chat_api_web/controllers/customer_controller.ex

Lines 127 to 144 in 7c850c4

	def update_metadata(conn, %{"id" => id, "metadata" => metadata}) do
	# TODO: include account_id
	customer = Customers.get_customer!(id)
	updates =
	metadata
	|> Map.merge(%{
	"ip" =>
	if ip_collection_enabled?() do
	conn.remote_ip |> :inet_parse.ntoa() |> to_string()
	else
	nil
	end,
	"last_seen_at" => DateTime.utc_now()
	})
	|> Customers.sanitize_metadata()
	with {:ok, %Customer{} = customer} <- Customers.update_customer_metadata(customer, updates) do

Based from the line map transformation the given map below (came from PUT request):

{
    "metadata": {
        "social handle": "mikoto"
    }
}

will become

%{
  "ip" => nil,
  "last_seen_at" => ~U[2021-12-04 11:11:38.308617Z],
  "social handle" => "mikoto"
}

and since updating a metadata calls Customers.update_customer_metadata function, the Customer.metadata_changeset will also be called, processing the merged map. Now, since metadata key is no longer present upon calling the changeset it becomes null

papercups/lib/chat_api/customers/customer.ex

Lines 121 to 142 in 7c850c4

	def metadata_changeset(customer, attrs) do
	customer
	|> cast(attrs, [
	:metadata,
	:email,
	:name,
	:phone,
	:external_id,
	:browser,
	:browser_version,
	:browser_language,
	:os,
	:ip,
	:last_seen_at,
	:current_url,
	:host,
	:pathname,
	:screen_height,
	:screen_width,
	:lib,
	:time_zone
	])

Fix

The fix is actually simple, instead of

papercups/lib/chat_api/customers.ex

Lines 284 to 290 in 7c850c4

	def sanitize_metadata(metadata) do
	metadata
	|> sanitize_metadata_email()
	|> sanitize_metadata_external_id()
	|> sanitize_metadata_current_url()
	|> sanitize_ad_hoc_metadata()
	end

I make it like this

  def sanitize_metadata(metadata) do
    metadata
    |> sanitize_metadata_email()
    |> sanitize_metadata_external_id()
    |> sanitize_metadata_current_url()
    |> sanitize_ad_hoc_metadata()
    |> prepare_metadata()
  end

  def prepare_metadata(metadata) when is_map(metadata), do: %{metadata: metadata}
  def prepare_metadata(metadata), do: metadata

After applying the change, here's what Ecto.Changeset prints out.

#Ecto.Changeset<
  action: nil,
  changes: %{
    metadata: %{
      "ip" => nil,
      "last_seen_at" => ~U[2021-12-04 11:18:05.094519Z],
      "social handle" => "mikoto"
    }
  },
  errors: [],
  data: #ChatApi.Customers.Customer<>,
  v

[jeepers3327]

I believe you should use PUT request on https://app.papercups.io/api/v1/customers/[id]/metadata as described in the router.ex file

papercups/lib/chat_api_web/router.ex

Line 52 in 7c850c4

put("/customers/:id/metadata", CustomerController, :update_metadata)

The reason https://app.papercups.io/api/v1/customers/[id]/ returns a null is because the changeset used in update_customer does not pick up the metadata key.

papercups/lib/chat_api/customers.ex

Lines 254 to 258 in 7c850c4

	def update_customer(%Customer{} = customer, attrs) do
	customer
	|> Customer.changeset(attrs)
	|> Repo.update()
	end

papercups/lib/chat_api/customers/customer.ex

Lines 91 to 119 in 7c850c4

	def changeset(customer, attrs) do
	customer
	|> cast(attrs, [
	:first_seen,
	:account_id,
	:company_id,
	:email,
	:name,
	:phone,
	:external_id,
	:profile_photo_url,
	:browser,
	:browser_version,
	:browser_language,
	:os,
	:ip,
	:last_seen_at,
	:current_url,
	:host,
	:pathname,
	:screen_height,
	:screen_width,
	:lib,
	:time_zone
	])
	|> validate_required([:first_seen, :last_seen_at, :account_id])
	|> foreign_key_constraint(:account_id)
	|> foreign_key_constraint(:company_id)
	end

But when you look at changeset used in Customers.update_customer_metadata it picks up the metadata key.

papercups/lib/chat_api/customers/customer.ex

Lines 121 to 143 in 7c850c4

	def metadata_changeset(customer, attrs) do
	customer
	|> cast(attrs, [
	:metadata,
	:email,
	:name,
	:phone,
	:external_id,
	:browser,
	:browser_version,
	:browser_language,
	:os,
	:ip,
	:last_seen_at,
	:current_url,
	:host,
	:pathname,
	:screen_height,
	:screen_width,
	:lib,
	:time_zone
	])
	end

[CharlesMangwa]

Thanks for the lead @jeepers3327! I forgot to mention in my initial description that I did try a PUT on https://app.papercups.io/api/v1/customers/[id]/metadata but it returns

{
  "errors": {
    "detail": "Not Found"
  }
}

even though the id I provided does exist and all. Am I missing something obvious?

FWIW: I also tried https://app.papercups.io/api/customers/[id]/metadata (without the v1). This one does return the data object but metadata is still null.

[jeepers3327]

Hi @CharlesMangwa,
Could you try accessing this endpoint get("/customers/:id/exists", CustomerController, :exists) with your id and tell me the result if it exist or not. Based on https://github.com/papercups-io/papercups/blob/master/lib/chat_api_web/router.ex, it shouldn't matter whether there's a v1 prefix on the api endpoint or not.

Note: I don't really launch the application on my local or the hosted version, I just give out advice based on the code.

[CharlesMangwa]

hey @jeepers3327, thanks for your help, it's much appreciated!

There seem to be different scopes actually:

papercups/lib/chat_api_web/router.ex

Line 35 in 1ad023a

scope "/api", ChatApiWeb do

papercups/lib/chat_api_web/router.ex

Line 182 in 1ad023a

scope "/api/v1", ChatApiWeb do

with only the former accepting put("/customers/:id/metadata"):

papercups/lib/chat_api_web/router.ex

Line 52 in 1ad023a

put("/customers/:id/metadata", CustomerController, :update_metadata)

That could explain why trying to access /api/v1/customers/[id]/exists only opens up the Papercups dashboard but /api/customers/[id]/exists does return

{
  "data": true
}

as /exists is defined in the /api scope only:

papercups/lib/chat_api_web/router.ex

Line 51 in 1ad023a

get("/customers/:id/exists", CustomerController, :exists)

[jeepers3327]

Have you checked if metadata field was filled when you update it or does it nullify the current metadata value

[CharlesMangwa]

There's no metadata (null) and I'm trying to add some but it always returns null

[CharlesMangwa]

Hey @jeepers3327! Do you think you could put up a PR with this fix pls?

[jeepers3327]

My apologies @CharlesMangwa for delay PR , I've been away for about a month due to unforeseen events.

[CharlesMangwa]

No worries @jeepers3327! Take care of yourself first, the PR can wait 🙌

[CharlesMangwa]

Closed by #1000 (comment).