Micro.blog isn't respecting the "dt-published" property of webmentions from brid.gy (and other oddities)

I set up brid.gy yesterday so that my Micro.blog site would get webmentions for replies to my twitter and mastodon cross-posts. So far, it’s mostly working! However, because brid.gy is built to connect indieweb sites to silos like twitter/mastodon that don’t have webmention support, they simply poll for responses and then send them to micro.blog as webmentions.

These webmentions are coming through, but it looks like Micro.blog isn’t respecting the dt-published mf2 property of the webmention that brid.gy sends. Instead, it’s just using the time of the webmention request itself (which, because brid.gy polls, can be at least 30 minutes or even hours later than the comment itself). Here’s a webmention that brid.gy recently delivered for a reply to my last post that was cross-posted to twitter:

D 2022-05-09 03:18:02.929912+00:00 Params: [('response_key', 'agdicmlk...')]
I 2022-05-09 03:18:03.256265+00:00 Source: davidcelis (Twitter) davidcelis, https://brid.gy/twitter/davidcelis
I 2022-05-09 03:18:03.339307+00:00 Created by this poll: https://brid.gy/log?start_time=1652066221&key=agdicmlkLWd5chcLEgdUd2l0dGVyIgpkYXZpZGNlbGlzDA
I 2022-05-09 03:18:03.340364+00:00 Starting Response comment tag:twitter.com,2013:1523471051123879936 https://twitter.com/wholemilk/status/1523471051123879936
I 2022-05-09 03:18:03.340771+00:00 requests.head https://davidcel.is/2022/05/08/going-out-to.html {'headers': {}}
I 2022-05-09 03:18:03.644274+00:00 Received 200
I 2022-05-09 03:18:03.646579+00:00 Webmention from https://brid.gy/comment/twitter/davidcelis/1523434725448159233/1523471051123879936 to https://davidcel.is/2022/05/08/going-out-to.html
D 2022-05-09 03:18:03.647376+00:00 Webmention discovery: attempting for https://davidcel.is/2022/05/08/going-out-to.html
I 2022-05-09 03:18:03.649502+00:00 requests.get https://davidcel.is/2022/05/08/going-out-to.html {'headers': {}}
I 2022-05-09 03:18:03.934964+00:00 Received 200
D 2022-05-09 03:18:03.945428+00:00 Webmention discovery: got endpoint in tag: https://micro.blog/webmention
I 2022-05-09 03:18:03.945777+00:00 Sending...
D 2022-05-09 03:18:03.946160+00:00 webmention send: https://brid.gy/comment/twitter/davidcelis/1523434725448159233/1523471051123879936 -> https://davidcel.is/2022/05/08/going-out-to.html
I 2022-05-09 03:18:03.946456+00:00 requests.post https://micro.blog/webmention {'data': {'source': 'https://brid.gy/comment/twitter/davidcelis/1523434725448159233/1523471051123879936', 'target': 'https://davidcel.is/2022/05/08/going-out-to.html'}, 'headers': {'User-Agent': '...', 'Accept': '...'}}
I 2022-05-09 03:18:04.327731+00:00 Received 202
D 2022-05-09 03:18:04.328064+00:00 webmention send: got HTTP 202
I 2022-05-09 03:18:04.328257+00:00 Sent! <Response [202]>
I 2022-05-09 03:18:04.440217+00:00 Setting last_webmention_sent

You don’t have to read that whole log; the important bit is this:

// I 2022-05-09 03:18:03.946456+00:00 requests.post https://micro.blog/webmention
{
  "data": {
    "source": "https://brid.gy/comment/twitter/davidcelis/1523434725448159233/1523471051123879936",
    "target": "https://davidcel.is/2022/05/08/going-out-to.html"
  },
  "headers": {"User-Agent": "…", "Accept": "…"}
}

If you grab the body of the webmention source, you’ll get this HTML document:

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <meta http-equiv="refresh" content="0;url=https://twitter.com/wholemilk/status/1523471051123879936">
    <title>An early casualty of lockdown I missed so so so much. Still have not really gotten all the way back to it!! Looks lovely. Also your public library bookmark!!! Did you have that made somewhere????</title>
    <style type="text/css">
      body { font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; }
      .p-uid { display: none; }
      .u-photo { max-width: 50px; border-radius: 4px; }
      .e-content { margin-top: 10px; font-size: 1.3em; }
    </style>
  </head>

  <article class="h-entry">
    <span class="p-uid">tag:twitter.com,2013:1523471051123879936</span>
    <time class="dt-published" datetime="2022-05-09T01:12:57+00:00">2022-05-09T01:12:57+00:00</time>
  
    <span class="p-author h-card">
      <data class="p-uid" value="tag:twitter.com,2013:wholemilk"></data>
      <data class="p-numeric-id" value="14989163"></data>
      <a class="p-name u-url" href="https://twitter.com/wholemilk">Rachel Kelly 🌈</a>
      <a class="u-url" href="http://rkode.com"></a>
      <a class="u-url" href="https://rachel.live"></a>
      <span class="p-nickname">wholemilk</span>
      <img class="u-photo" src="https://pbs.twimg.com/profile_images/1508210055086153732/IpprnDJZ.jpg" alt="" />
    </span>

    <a class="u-url" href="https://twitter.com/wholemilk/status/1523471051123879936">https://twitter.com/wholemilk/status/1523471051123879936</a>
    <div class="e-content p-name">
      An early casualty of lockdown I missed so so so much. Still have not really gotten all the way back to it!! Looks lovely. Also your public library bookmark!!! Did you have that made somewhere????
      <a class="u-mention" aria-hidden="true" href="https://davidcel.is/"></a>
      <a class="u-mention" aria-hidden="true" href="https://twitter.com/davidcelis"></a>
    </div>

    <a class="u-in-reply-to" href="https://davidcel.is/2022/05/08/going-out-to.html"></a>
    <a class="u-in-reply-to" href="https://twitter.com/davidcelis/status/1523434725448159233"></a>
  </article>
</html>

The mf2 properties are all set up really well here; you can see that the h-entry declares its dt-published here: <time class="dt-published" datetime="2022-05-09T01:12:57+00:00">2022-05-09T01:12:57+00:00</time> but it seems Micro.blog is ignoring it.

Is this an issue that could be fixed? Based on how the comments are rendering via conversations.js, it looks like there might already some special handling for brid.gy that’s happening (e.g. in my own blog post; the replies that were sent from brid.gy show up with usernames like “twitter.com/wholemilk” and “twitter.com/adfontes__” instead of just “wholemilk” and “adfontes__”, and each reply starts with “@davidcelis” despite the @mention not actually appearing in the reply’s e-content property. Maybe micro.blog just inserts that @mention at the beginning of any reply regardless of the source?

I just found the help page mentioning that you can request conversation.js with format=jf2 so I tried that for my blog post (https://micro.blog/conversation.js?url=https%3A%2F%2Fdavidcel.is%2F2022%2F05%2F08%2Fgoing-out-to.html&format=jf2) and it does indeed seem to be saving webmentions with published set to the same timestamp as received, so it seems to be an issue with receiving/saving webmentions and not just rendering them

1 Like

This is something I can fix to respect dt-published. I’ll work on it this week. And you’re right, Micro.blog has special handling for Bridgy already to show Twitter replies.

2 Likes

This is something I can fix to respect dt-published. I’ll work on it this week.

Thanks a ton, @manton!

And you’re right, Micro.blog has special handling for Bridgy already to show Twitter replies.

So, would it be possible to fix up one other oddity with regards to how these replies end up being stored in Micro.blog? I’m not actually sure if this is Micro.blog’s doing or if this is how they come in from Bridgy itself, but it looks like every webmention ends up having (in my case) @davidcelis at the very beginning of the content’s text (which then gets auto-linked to micro.blog/davidcelis because it’s parsed as a username @mention). I think this happening on Micro.blog’s end though, because double checking the HTML of the source for one of my own replies shows that the e-content property doesn’t have the @mention of my own username anywhere in it

The reason this ends up being weird is because, in the case of threaded conversations where I myself am responding to comments on my syndicated tweet, my replies come back as comments on my post prefixed with @davidcelis too, as if I’m @mentioning myself :sweat_smile: The same post mentioned above has a couple examples of this. I have no idea how others would prefer handling of webmentions from twitter, but I can think of a couple options that make sense to me…

One would be to not adding @username to the beginning of the content at all, since it technically isn’t there originally (this also matches how things look on Twitter itself). Another possibly better option would be to use the u-in-reply-to URL that links back to twitter in order to show the real @username instead of the current default

1 Like

I’m seeing the same behavior. It’s forced by M.B before it gets to Hugo. I’d love if it wasn’t… :wink:

Micro.blog does add the username for any incoming Webmention, not specifically for Bridgy. It does this because when the replies show up in the Micro.blog timeline, it’s confusing if they don’t include the @-username at the beginning. Of course, in this case the Twitter replies are only shown on your blog where it’s already obvious they are part of the conversation.

Instead of changing Micro.blog to not include the username, I think it might be better for us to just remove it when Conversation.js includes the replies to your blog post.

I’ve made a couple changes to this today:

  • Micro.blog now uses the published date for incoming Webmentions instead of the current time. Helps when the Webmentions aren’t sent right away for new replies.
  • Conversation.js (the JavaScript that includes replies on your blog post) now strips out the @-username references for your user. This cleans up the page nicely. No change is made when showing the replies in the Micro.blog timeline or when asking for the conversation in different formats like JSON Feed.

Enjoy!

1 Like

Awesome, thank you! I had a couple webmentions delivered about an hour ago and it looks like the timestamps on my site are correct :raised_hands:

Hmmm, I have another weird webmention quirk that happened yesterday; I had a webmention sent from brid.gy on a post I syndicated to Mastodon:

https://brid.gy/comment/mastodon/@davidcelis@xoxo.zone/108284396779836044/108286309856903740

On my corresponding blog post, that webmention response has mixed up authorship info. The avatar is correct, but it used my info for the name instead of the actual author’s name (jonnyborbs).

The webmention HTML has .p-author > .p-name set to jonnyborbs but further down there’s another p-name mf2 nested within a u-category. I think Micro.blog might be grabbing the wrong p-name in this case?

Hmm, thanks for letting us know. I think Micro.blog is going to need to be updated to support Mastodon replies like this… I had never seen one of those before. Because Micro.blog supports ActivityPub there may be a way to hook that in so that it works more seamlessly.

1 Like