As I did with Instagram and Twitter, I’ve spent the last couple of days importing all of my posts from Facebook to this blog. Similar to those projects, I requested my archive in JSON format and then used an Apple Shortcut to parse it and upload via the WordPress API. The Shortcut is very similar to the one I used for Instagram, but with a few more edge cases and IF statements since FB allows for more post types than just images. (I’m not going to bother sharing it. If you were clever enough to follow the other two Shortcuts, you can figure it out.)

My earliest post was from 2007, and all together I had 4,083 days worth of posts to import. I only synced images to WordPress; I haven’t touched any videos yet (but I never really uploaded many of those to FB). It took me just short of 29 hours spaced out over the course of a week, not counting the time I spent manually reviewing and cleaning things up. (I deliberately slow down the API requests to avoid DDoSing my own site.)

And it bears repeating: Facebook’s data archive sucks. A brief list of problems I encountered:

• Blank status updates. This happened a lot more in the older data.
• Missing data when I “shared a Page/post/photo/link/video/event” from Facebook itself. This happened a lot more in the older data.
• Missing data when posting from other sites/apps, like Eventbrite, Foursquare, Tweetdeck, Spotify, Meetup, Runkeeper, etc. This happened a lot more in the older data.
• Duplicated content – there would be a “Kris Howard shared a link” item with a URL, and then a matching status update where I actually shared the URL. This happened a lot more with the data in recent years.
• URLs that I’m fairly certain I shared in comments on posts, but included as top-level items with zero context. This happened exclusively with data from the past couple years.
• Inconsistent links to FB users – most of the time when I tagged someone, their name would appear like this in the data: “Hey @[1108218380:2048:Rodd Snook]”. But then in recent years, that format disappeared.
• Dead links – not Facebook’s fault, but there are so, so many.

As soon as these errors started cropping up, I had to make the call whether to stop and adjust my Shortcut to handle them, or to clean them up manually. In most cases, I decided that I’d just manually review and fix. After every couple months’ worth of import, I’d pause and page through them on the site to see if any looked weird. I’d then manually edit and tidy up any issues.

There were other oddities I noticed in the data that aren’t really errors. For example, my earliest status updates are all sentence fragments that start with a verb. This is because back in the aughts Facebook had an explicit “What are you doing right now?” prompt. Kinda funny.

The archive also included posts that I made on other people’s profiles, mostly just “Happy birthday” wishes. The data does include the name of the person I was writing to, but I couldn’t be arsed creating a special case in my Shortcut to handle that. I ended up deleting most of those and just keeping the ones that amused me or where it was a family member.

The archive didn’t include posts that I made in Groups. That may have been an option when I downloaded my archive, but I decided it wasn’t worth the effort. I’ve never been a big Group user. It also doesn’t include the comments on any of my posts. Again, that may have been an option, but I figure discussions should be ephemeral. I’m okay with not having those.

Ultimately you could argue that this import had minimal value. Most of the content is actually already on this blog, either posted natively or included already in the Twitter or Instagram imports. But there are occasional gems in there that I didn’t post anywhere else, and I’m happy I preserved those. I don’t expect anyone to ever read them, but it’s an important part of my personal data archive and I’m glad I have it.

And now I just need to finish deleting all the content over on FB…

I was cleaning up old blog posts today when I saw a mention of the widget I used to have (decades ago) that showed what I was listening to. “That would be fun to recreate,” I thought. How hard could it be?

Folks, I could find no way to easily embed my last played Apple Music song. The desktop app allows you to get iframe embed code for playlists, but there’s no playlist for your Recently Played. You can create a Smart Playlist of songs you’ve recently played, but this will only include songs in your library and not songs you’ve streamed. Also, you can’t embed a Smart Playlist anyway. There are no WordPress plugins that do this, and no third party apps that I could find. There is a Developer API for Apple Music, but to register as a developer you have to pay $99 a year. Yeah, no.

For a second I thought about switching back to Spotify. Double-plus no.

Then I remembered… scrobbling. That was a thing, right? Turns out it’s still a thing. But how to scrobble Apple Music? I mostly listen on my iPhone, and the consensus seems to be that Marvis Pro is the way to go. This app is basically a wrapper for Apple Music, but it has a massively customisable UI (the Redditors love it), it scrobbles to last.fm out of the box, and it’s only $15 AUD. I figured it was worth a shot. Installed the app, signed up for last.fm, and verified that scrobbling was happening. Now to hook it up to WordPress…

This post from RxBrad has a handy Javascript snippet that can be used in a WordPress Custom HTML widget. Too easy! I signed up for an API key and set up the script… but it wouldn’t work. In the Block Editor preview it would show the album, but when I published the widget, it would just show a broken image on the site. I noticed in the console that there were some errors about the ampersands, and I could see that WordPress was actually converting them to HTML entities. I banged my head on a wall for 10 minutes until the Snook woke up from his nap and I patiently explained the problem. Less than 2 minutes later he had solved it. Oh right! We used to always put HTML comment tags around our Javascript, back in the day. (Insert “Do not cite the Deep Magic to me, Witch” meme.) Once I added those, everything just worked!

So yeah, there it is over in the sidebar on the homepage.

Limitations:

• Marvis Pro only scrobbles while the app is open. So if my iPhone screen goes to sleep, it won’t sync again until I wake it up. It does sync the whole history then, but it does mean the sidebar isn’t really necessarily “live.” Do I care? Not at this point. If I do, I can apparently pay $10 to unlock background scrobbling via last.fm Pro.
• Marvis Pro doesn’t have a desktop app. If I want to scrobble from my Mac Mini, I’ll need to setup the last.fm desktop app. Can’t be arsed right now, but it’s an option.

Yesterday I kicked off the long gestating project to import all my old Facebook content to this website. I requested my archive several months ago, and since then I’ve been working on deleting all my content there. (It’s such a pain to delete your content without deleting the account. I’ll write up a post about that later.) Anyway, the import is now happening and you can see the posts appearing here.

The exported data is a mess though, which has made the import script a real pain. You can post to Facebook from lots of other apps (and I did, over the years), and not all of the data is in there in the same ways. There are so many cases where data is just missing from the JSON. Like, you can post to Facebook from Eventbrite, and all that’s in the export is me saying “Booked in!” and “Kris Howard posted something via Eventbrite.” but the actual event isn’t linked or listed at all. It’s just gone. I’ve found other examples too, like Instagram. (Fortunately I already imported all my Instagram posts, so I’m just skipping over those.) But I’m only 10% of the way through, and I’m doing a lot of manual cleanup work.

On the upside, this project (as well as my Instagram and Twitter import) have taught me so much about archiving and data portability. I’m happy that I will have some sort of record of this data, even if it’s not 100% complete. I’ll never end up in this situation again, and hopefully I can help a few others realise the pitfalls of entrusting your data to corporations.