Category: Meta

  • Apple Shortcut error checking

    I’m almost ready to kick off the import of my Twitter archive to this blog, and in the process I figured out how to do error checking in an Apple Shortcut in a way that doesn’t disrupt the flow! I’ve gone back to my Instagram Shortcut and updated it and the accompanying blog post to explain.

    Related note: I think maybe Apple Shortcut sharing actually does version snapshotting?! Because when I clicked the link in the post, it opened the old “pre error checking” version. I had to explicitly choose to Share again and it generated a new URL. Huh. That’s pretty cool.

  • Leaving Instagram

    Edited (6/2/25) to add: I figured out how to do error checking! I’ve included details below and updated the shared Shortcut.
    Edited (3/2/25) to add: I had the file encoding command incorrect below! It’s now correct.

    Everybody’s got a line, and Meta crossed mine a while back. It’s time to extricate my data and my attention from their greedy, fascist-enabling claws. I spent the last couple days using an Apple Shortcut to migrate all of my Instagram posts over to WordPress posts on this blog, and I’m in the process of shutting down the Insta account entirely. If you’d like to do the same, I’ll show you how.

    Why a Shortcut? Well, I did some research first to see if there was a plugin or service that could migrate the posts for me. Unfortunately, I didn’t find much. This plugin on Github looked like the perfect thing, but it hasn’t been updated in six years since Instagram got it shut down. I also found Intagrate – and would have been willing to pay for it – but spotted this alert at the top of the page: “Due to Instagram API changes, only Instagram Creator and Business accounts can be connected to the plugin.” Most likely that means I wouldn’t have any luck trying to use the API myself, either by fixing the old plugin or writing my own.

    And then I thought about the Apple Shortcuts I wrote recently for making posts straight from my phone. I already know how to upload media to WordPress and then use it to create posts. Why not use that to do the export? I just needed my Instagram data in some format that the Shortcut could parse.

    Step 1 – Request my Instagram archive

    To get to my Instagram data, I had to go to the Meta Accounts Center and click on “Your information and permissions.” Then I clicked on the “Download your information” option. Then I clicked on the “Download or transfer information” option in the box that pops up. I selected my Instagram profile and “Some of your information.” Then I selected the option to just download my “Content.” (There’s a ton of other information in there like Fundraisers and Followers and Messages and whatnot, but I only cared about my content.) Then I selected the option to “Download to device.” The final step was to select my download options:

    Instagram download settings

    I requested “All time” for the date range, “JSON” for the format, and “High” for the media quality. Once I submitted the request, it took a little while for the file to be prepared. They sent me an email once it was ready and I downloaded it.

    The archive itself is pretty straightforward. The details of all my posts were in your_instagram_activity/content/posts_1.json. All of the actual images were in media/posts and then organised in folders by month/year.

    Instagram archive

    Step 2 – Prepare the JSON

    Once thing I noticed when I looked at the JSON in a text editor was that the encoding was clearly messed up. I could see posts where I had used emojis instead looked like this:

    "title": "I was excited yesterday to find at a Mittagong antique shop a set of 6 placemats depicting artworks from the famous Australian artist Tom Roberts. Mr. Snook was similarly excited to find a red metal EAT napkin holder for $2 at the Vinnies. \u00f0\u009f\u0098\u0082\u00e2\u009d\u00a4\u00ef\u00b8\u008f",

    I’m not an expert at fixing encoding issues, so I asked Rodd to help. We tried changing the encoding in BBEdit, but nothing seemed to fix it. He went away and did some digging and figured out I needed to use iconv and jq. Here’s the Terminal command that worked:

    cat posts_1.json | jq '.|sort_by(.media[0].creation_timestamp)|reverse' | iconv -t iso-8859-1 -f utf-8 > posts_1_fix.json

    And now when I opened posts_1_fix.json I could see the emoji properly:

    "title": "I was excited yesterday to find at a Mittagong antique shop a set of 6 placemats depicting artworks from the famous Australian artist Tom Roberts. Mr. Snook was similarly excited to find a red metal EAT napkin holder for $2 at the Vinnies. 😂❤️",

    The other thing that command does is to reorder the items in the JSON. For some reason, the file was not organised by timestamp. (I thought at first some items were missing and thus did several more downloads before I realised that all the content was there, just jumbled about.) Most people probably don’t care about the order, but in my case, it’s because for several years I had a plugin that automatically pulled in my Instagram posts each day to my blog. That meant I needed to delete several years worth of posts in the middle of the JSON to avoid having duplication. I simply used Epoch Converter to get the timestamp of the first and last posts that I imported in this way, and then deleted the items that fell within that timeframe.

    Another thing – my Shortcut only handles images. It’ll skip over any videos, which come through as mp4 files. I don’t usually post these, so my approach was to handle them separately. I simply went through the file searching for them to see if they were worth moving over to YouTube and embedding on the blog, and then deleted those items. You can choose how you want to handle them.

    Lastly, I discovered that Shortcuts would only parse the JSON if it had a .txt extension. So I copied the first 1000 lines or so of it (I’ll explain why in a minute) into a separate file and called it “posts_scratch.txt”, keeping it in the same folder as “posts_1.json” and “posts_1_fix.json”.

    Step 3 – Set up the Shortcut

    You can download the Shortcut I created here. I’ll walk you through it bit by bit.

    Authorisation

    This is where you give the Shortcut the details it needs to access your WordPress blog. You’ll need to create an Application Password and then put the username and password into the Text box as shown. The Shortcut saves that in a variable and then encodes it.

    Initialise permalinks and errors

    In order to show a list of all the posts that have been created at the end of the import (as well as any errors that have occurred), I create blank variables to collect them. Nothing to do here.

    Parse the JSON

    This is things really get going. You need to click on that “content” link and ensure it points to the content folder in your downloaded Instagram archive. When you run the Shortcut, it’ll look in that folder for the “posts_scratch.txt” file and parse it, with each Instagram post becoming an item in the Dictionary. The Shortcut then loops through each of these.

    Initialise title and content

    For each Instagram post, I set a blank variable for the post_title and post_content. Nothing to do here.

    Get title

    Next the script looks to see if there is a title specified for the Instagram post in the JSON. Note: the JSON file handles this differently depending on whether it’s a single image or multi image post. (No idea why; the consistency is stupid. 🙄 ) This case is for a multi image post. If the title is specified, it gets saved to the post_title variable. Nothing to do here.

    Loop through media

    Next the script grabs the media associated to the Instagram post. It then starts looping through each media item attached. Nothing to do here.

    URL and timestamp

    For each media item, it grabs the URI specified in the JSON and saves it in the imagefile variable. It also grabs the timestamp, converts it into a number (rather than a string), and saves it in the timestamp variable. Nothing to do here. (Yes, I know that I’m doing this over and over for each image, even though the creation_timestamp is the same for every item in a post. You can fix that if it bothers you. But if such inefficiency bothers you, you’re probably not using a Shortcut for this anyway. 😂 )

    Get the media file

    Here’s where the Shortcut actually gets the media file. You need to click on that “yourinstaarchive” link and ensure it points to your downloaded Instagram archive. The URIs are relative to this parent, so it needs to be correct! Assuming the file is an image, it will resize it to 1000 pixels wide, maximum. (You can adjust that if you want, or take it out to leave them full-size.)

    Get title

    Here’s where the Shortcut handles the case where it’s a single image post. In that case, the title is included with the media item itself. So it grabs that title, and if it has value (i.e. isn’t blank), it saves it as the post_title. Nothing to do here.

    Upload image and check for errors

    Here’s where the Shortcut actually uploads the resized media file to your site. You will need to put your own domain name into the URL there. Once the image is uploaded, the Shortcut will parse the response from your site and check if there’s an error. All successful responses should have “guid” in there, so if it’s not, it’ll save the name of the file it was trying to upload into the errors variable. If the upload succeeded, it’ll instead save the ID of the newly uploaded image in the mediaid variable and progress to updating the alt text and title.

    Post content

    As part of the response, WordPress will send back the URLs for all the different resized versions of the image. The Shortcut will grab the “medium_large” version, which is the fixed width 768px wide version and save it as the src variable. (You can change the image size if you want, but that’s left as an exercise for the you. None of this stuff is well documented, I’m afraid.) The Shortcut will then create the HTML to display the image and save it in the post_content variable. Because of the way the Text item is set up, as you recursively loop through each image in the post it’ll be appended as a series of paragraphs. Feel free to change the HTML to suit your blog if you want, otherwise nothing to do here. (At a later step, it’ll inject the actual text from the Instagram post.)

    Update alt text and title, and check for errors

    You need to update the Text box with your blog domain. This step is where the Shortcut updates the title and alt text of the image it just uploaded. The title is set to “Post Pic”, but you can change that to something else if you like. The alt text is set to the post_title variable. If this update fails, the mediaid is logged to the error variable.

    Close if statements and wait 5 seconds before looping

    This closes off the two If statements (whether it was a JPG or not, and whether the image upload succeeded), waits for 5 seconds, and then loops to the next media item in the post. You can remove the Wait if you want, or experiment with how long you want it to be. I added it because I know my site struggles under heavy load, and I wanted to spread out the API requests to be safe rather than just blasting them as fast as my MacPro can generate them. Otherwise nothing to do here.

    Convert the date

    This is where the Shortcut turns the timestamp into an actual date that WordPress can work with. (No, I don’t bother doing anything tricky to handle the timezone, as I don’t need that level of precision on my posts.) Nothing to do here.

    Post caption

    Here’s where the Shortcut injects the Instagram post caption at the start of the post_content variable, which at this point is a list of HTML images. You can adjust the HTML if you want, otherwise nothing to do here.

    Upload the post and check for errors

    Here’s where the Shortcut actually creates the WordPress post. You need to adjust the URL to have your domain name. It uses the encoded username/password we created way up at the start. I set it to use “Photo Post” as the WordPress post title, but you can change that to whatever you want. The content is the post_content HTML that was created, and the date is the converted timestamp. I also assign the post to a special WordPress category. You can either remove this, or change the category ID to the appropriate one from your site.

    Also please note that I’ve set the status to “publish”. This means the posts will go live in WordPress right away, but since the publish date is in the past they won’t appear on your home page or in your RSS feeds. You may wish to change this to “draft” the first few times you run it. I found though that when I went into WordPress to check and publish those posts, it updated the published date to the current date – meaning the post no longer appeared at the correct date in the past, and it showed up in my RSS feed as a new post. So once you’re satisfied that the script is working, I suggest changing it to “publish” directly and then they’ll be saved with the correct date.

    The Shortcut will also check if there were any errors returned upon publishing the post, and if so save the caption of the post in the errors variable.

    Get post response

    If there were no errors, the Shortcut next parses the response from the WordPress API and pulls out the link to the post that’s been created. It then gets appended to the list saved within the permalinks variable. Nothing to do here.

    Final step

    Final step! Again, I have a “wait” set after the if statement finishes just to space out the post loops. You can remove this, or adjust as necessary. Then the whole thing loops back around to the next Instagram post in the JSON. Once the loop is finished, it’ll show an alert box that says “Done!”, then one with any errors that occurred, and then another with the list of permalinks. This allows you to investigate and fix any of the errors, and to spot check some of the permalinks to make sure they look correct. Note: The error checking may include some false positives, if you actually used the word “error” in the post! So if it looks fine on your site, you can disregard that error.

    Optionally, you could use the Shortcut action “Append to Text File” or “Append to Note” if you want to write those errors or permalinks to a file. I leave that as an exercise for you to work out. 🙂

    Step 4 – Run the Shortcut

    Once I had the Shortcut properly configured, I copied out just a small portion of the fixed JSON into my posts_scratch.txt so I could test it. To run it, I just clicked the “Play” button on the Shortcut. Shortcuts will highlight each step as it moves through the flow. Once I was satisfied it was doing what I wanted, I increased the number of items in my posts_scratch.txt file.

    Why not just run the whole thing? Well, I found it crashed every now and then. I’m not sure why – whether it was a memory issue with Shortcuts, or my server being flaky, or WordPress itself just throwing an error. That’s why I started adding some “wait” pauses in there, and why I found it easier to work in smaller batches. Whenever it crashed, I could look in my WordPress dashboard to see the last Media item that had been uploaded, and then find that point in the JSON and delete everything before that item. Then I’d delete the Media item in WordPress and start the Shortcut over again at that point. 1000 lines or so seemed to be the sweet spot, but you need to make sure you aren’t cutting off one of the items in the JSON, of course.

    Update: The above was written before I figured out how to add error checking to the Shortcut. Now it should run without actually crashing the Shortcut, logging errors to a variable. I would still be wary about running a really large file, as I have no idea how Shortcuts would handle that…

    One last heads up – you’ll probably get a lot of Permissions requests from your Mac at first due to the large number of files you’re transferring. I just kept clicking the option to “Always Allow” and eventually it stopped asking.

    After the import…

    As mentioned, this Shortcut only handles JPGs, so I then went through the original JSON file looking for mp4 files to decide what to do with them. Some of them I discarded, and others I uploaded to YouTube and manually created posts for them.

    I also searched through my existing WordPress posts for any places where I had previously embedded or linked to Instagram images. (Search for “instagram.com” or “instagr.am” to find these. I also had some that were done with IFTTT, so a search for “ift.tt” will find those.) I either replaced those with links to my posts, or removed them as necessary.

    Finally it was time to remove my content off Instagram. I didn’t want to nuke my account yet, which meant I needed to delete the images one by one. (There are apps that claim to do bulk delete, but I didn’t like the look of any of them.) To do this, I went to “Your Activity” -> “Photos and videos” in a browser window. Then click “Select” and start clicking away. Once you hit 100 (the maximum), click “Delete” to blast them.

    I’ll give it a few weeks or so before I delete the account entirely, to make sure folks know where to find my content going forward!

    Next up…. Facebook. *cracks knuckles*

  • Migrating Instagram Posts to WordPress

    Given, you know, everything, I’m looking to move away from Instagram and consolidate everything I’ve posted there to my WordPress blog. Has anyone come across a good way to do this? It doesn’t look like anyone’s done a plugin or anything like that. I have a rudimentary plan, but thought I’d check in case anyone has already solved this…

  • Problems with WordPress and Amazon CloudFront

    Problems with WordPress and Amazon CloudFront

    Here’s a fun story. When I originally moved this blog to Amazon Lightsail five years ago, I followed the recommended best practice and installed the AWS for WordPress plugin. I used that to set up an Amazon CloudFront distribution to manage the site’s cache. For several years, everything worked great. Then in September 2022, AWS abandoned the plugin and removed it from WordPress.org. As far as I can tell, they provided no information for users to tell them what to do without it. I continued to use the plugin for sometime, even though this is generally seen as pretty bad security since it’s no longer receiving updates. A few months back, I got tired of my Security scanner blaring at me about this discontinued plugin and deactivated it. The CloudFront distribution still existed and my site continued to work as intended, so I figured it was safe to delete.

    As you might guess, there have been consequences. I noticed recently that my site was sometimes caching things too aggressively. I’d write a new blog post and tell Rodd to check it out, but he’d still be seeing the old one for some time. I’m far from a CloudFront expert, but I’ve been looking at my distribution behaviour settings and comparing them to current best practices. This site, for example, recommends using “Origin” for your cache key. My plugin-created distro however uses “Cloudfront-Forwarded-Proto,” “CloudFront-is-Tablet-Viewer,” “CloudFront-is-Mobile-Viewer,” “CloudFront-is-Desktop-Viewer,” and “Host.” For object caching, my distribution had “Use origin cache headers” selected instead of a custom option. Without any documentation from AWS on how their plugin actually worked, all I can theorise is that it must’ve set some sort of header that CloudFront was using, and by deleting the plugin, I’ve mucked up that behaviour.

    So what to do? For the meantime, I’ve changed the default behaviour object caching to have a default TTL of 5 minutes. I’ll see if that helps the situation at all. Otherwise I’m going to either have to look at my backups and see if I can reverse-engineer what the plugin was doing, or else figure out how to modify my distribution to work properly without it. Ugh.

  • iOS Shortcuts for WordPress Bloggers

    You may have noticed I’m posting a lot more lately. It’s partly because I’m not working, plus a bit of conscious effort… plus a couple effort-saving shortcuts I’ve set up.

    In the past, I used to share images to Instagram and then had an IFTTT applet run to post those images to WordPress as blog posts. I wanted to flip that model and instead first post to my own site, and THEN have the option of sharing to Instagram or other social networks. The solution I’ve landed on is to use an iOS Shortcut as a Share Sheet action. That means I take a photo, click the Share button, and then click the Post with Pic shortcut.

    iOS Shortcuts

    The Shortcut first prompts me for a Post Title, Post Content, and Image Alt Text. It then converts the image to a JPG and uploads it to the WordPress Media API along with the title and alt text. Next, it creates a new post with the title and content, and sets the newly uploaded image as the Featured Image for the post. It also copies the Post Content to the clipboard and reopens the Share Sheet in case I want to then post the image to Instagram or any other social networks. Lastly, it opens my site so I can see the new post.

    Post with Pic ShortcutAs you can see, it’s a pretty lengthy Shortcut. You can download a blank version for yourself from here. There are a few places you’ll need to tweak and customise for your own setup. (I recommend doing that on your laptop rather than trying to do it on the phone directly.) You’ll first need to set up an Application-Specific Password for your WordPress User. There’s a Text action where you’ll need to put in your WordPress username and that application-specific password for authentication. There are also three places where you need to replace [yourdomain] with the URL to your own site. Once you’ve got it updated, click the Info icon and make sure you’ve checked “Show in Share Sheet.” Then you should be good to go! (I built this based on Chuck Grimmett’s Shortcut and helpful blog post.)

    And yes, I know that Shortcuts has a “Post to WordPress” action built in. However, I could not get the damn thing to work! I suspect it’s because it uses XML-RPC, and I have that locked down for security. This version with the Rest API works just fine.

    Upload Images to WordPress ShortcutThere’s a second Shortcut in the Share Sheet above: Upload Images to WordPress. This is for when I have a number of images I want to use in a post. You can upload through the WordPress iOS app, but I’ve found it clunky and slow. It also defaults to uploading at full resolution too. This shortcut works a lot better. I simply select several images in the Photos app, hit Share, and then the relevant Shortcut. It converts each one to JPG and resizes to 1000px wide before uploading. You can grab it here. Again, you’ll need to put in your username and application password. You’ll also need to put your domain name in for the API URL. You may also need to give it some permissions the first time you run it. Depending on how powerful your server is, you may need to limit how many photos you try to upload at once. Note: this one doesn’t do any titles or alt text. But this is for bulk uploading, and I then add in the metadata when I’m composing the post itself.

    I hope you find these useful!

  • Wrapstodon

    My Mastodon instance (aus.social) is participating in “Wrapstodon,” where the server generates some fun end-year-stats for you. My most popular post was this one featuring my niece Indie’s embroidered sneakers!

    Wrapstodon

  • Custom Bluesky and Mastodon usernames with your own domain

    I recently went through the process of setting up custom handles/aliases for myself using my domain name on Bluesky and Mastodon. I figure I’ll share the steps here in case anybody else with a similar setup (WordPress on Amazon Lightsail) wants to do the same.

    Bluesky

    Simply follow the instructions that Bluesky have provided here. In Step 5, you will need to add a TXT record to your domain DNS. If you use Amazon Lightsail, click on the Domains & DNS option and then select the relevant DNS zone (in my case, web-goddess.org).

    Lightsail console DNS zone

    Then click on the DNS records tab and click the + Add record button.

    Lightsail console DNS zone

    Change the record type to TXT record and then put in the name and response provided by you by Bluesky. (The “host” is the “Record name,” and the “value” is the “Responds with.”) Click save.

    Adding a new TXT record

    Wait a little bit — in my experience, 30 seconds is usually sufficient — and then click that Verify DNS Record button in Bluesky. And then you’re done!

    In my case, you can now find me at @web-goddess.org on Bluesky.

    Mastodon

    Mastodon is a slightly more involved process, and it doesn’t change your official server username but rather gives you an alias that you can give out. In my case, my current Mastodon account is @web_goddess@aus.social. However, if I give out an alias instead, then it doesn’t matter if I change servers in the future — you’ll still be able to find me. I was inspired by these posts by Phil Nash and Maarten Balliauw for how to do it.

    First, open up a new browser tab and go to this URL, substituting your own Mastodon account values in there:

    https://{instance}/.well-known/webfinger?resource=acct:{username}@{instance}

    So in my case, I went to https://aus.social/.well-known/webfinger?resource=acct:web_goddess@aus.social. What you’ll get back is a blob of text (JSON). Copy that and set it aside for the moment.

    Then go to the Lightsail console and click on the little terminal icon to Connect using SSH to your instance.

    Lightsail console

    A window with a terminal will appear.

    Lightsail terminal

    Now you need to go to where the WordPress files are stored for your site. Type in this command in the terminal and hit enter.

    cd stack/wordpress/

    Important note: The path above is for one of the newer WordPress Bitnami instances. If you have an older instance, the path to where your WordPress files are stored might be different.

    Then you’ll need to create a new directory called “.well-known.” (Note the dot in front of the name.) Type in this command and hit enter.

    mkdir .well-known/

    Then go into that directory. Type in this command and hit enter.

    cd .well-known

    Now you will need to make a special file called webfinger. Type in this command and hit enter to open a text editor in the terminal.

    nano webfinger

    This is where you will paste in the blob of JSON text you copied from your Mastodon server. Copy and paste it straight into the editor, and then hit CTRL-X to exit.

    The editor will ask if you want to “Save modified buffer?” Type Y for Yes, leave the name as webfinger, and hit enter to save.

    You can then exit the Lightsail terminal and close the pop-up window.

    exit

    If everything worked correctly, you should now be able to go to your own domain and receive the JSON blob back by substituting your domain in this address:

    https://{yourdomain}/.well-known/webfinger

    In my case, I went to https://www.web-goddess.org/.well-known/webfinger in a browser tab and verified that the JSON was returned.

    And that’s it! What does this actually mean? It means I can tell people that my Mastodon username is @kris@web-goddess.org, and if they type that into a search box on Mastodon, it will point them to my official account.

    Mastodon search

    If I change servers in the future, I’ll just have to update the JSON blob with the correct values from my new Mastodon server, and people will continue to be able to find me from the alias.

  • New web-goddess theme!

    I’ve been feeling like the site needed a new lick of paint, so I spent some time today installing and customising a new theme. I used the Björk theme from Anders Norén, who made my previous theme. My logo remains the awesome one that my friend Matt Hinrichs drew for me many years ago. I also decommissioned my old krishoward.org personal site, moving some of the relevant content here and then changing that domain to a redirect. Please let me know if you find any broken links or anything that looks weird!

  • A new home for w-g

    Well, that was an ordeal.

    A few weeks ago I got an email that my shared web hosting account (that I use for both this site and RoaldDahlFans) was up for renewal, and it reminded me that I didn’t like the host I was using. I was stuck on PHP5 and they were going to charge me more to move to a server with PHP7, and that pissed me off. So I asked Twitter what to do.

    Wordpress hosting tweet

    I got a lot of different answers, most of which were a LOT more expensive than I was spending. I was okay with going up a little bit, but these are both hobby sites rather than professional so anything over, say, $20/month was too much. A few folks threw out the suggestion to use Amazon Lightsail, which was intriguing. After all, it’s always a good idea to eat your own dog food. (I should also mention that these sites are and have always been hosted and paid for by me personally, so they’re not running on my employee AWS account or anything like that.)

    I thought I’d document the process that I followed for the sites, which was based on this blog post and this AWS tutorial. Truthfully, setting up a new WordPress site on Lightsail is super easy and takes less than hour. All the complexity was because I was migrating two very old and very crufty sites from PHP5 to PHP7 and trying to preserve twenty years’ worth of old URLs. So even though I encountered a few hurdles not described in the tutorials, I wouldn’t let that stop you. Read on for the details…

    (more…)

  • Linkrot diminishes us all.

    Losing the Internet You Grew Up With – The Atlantic – I think about this a lot. It’s why I maintain a blog with fifteen years of archives, as well as a fan site that’s existed for almost twenty years. We’re at the point now where that’s pretty much ancient. At what point will I have the oldest functioning blog on the Internet?