Back in January 2017, I wrote a PHP script I called MUGicalPHP that used the Meetup API to create iCal feeds (ie calendars you could subscribe to) for tech events in various Australian cities. The idea was that it would update the feeds every day with new events. It was kind of flaky though, and it often failed silently without me knowing about it. (Sometimes the Meetup API just decides to return ZERO RESULTS for some reason. 🤷♀️) Still, it was useful and mostly worked, so I shared the links with lots of folks in the community and made plenty of use of them in my job at YOW!.
Fast-forward to 2018, and now I’m working at AWS. As I was going through the hiring process, it occurred to me (not for the first time) that I could probably use “that Lambda thing” to rewrite my meetup script to be “serverless.” So… I did it!
MUGicalNode is now working and it’s up on Github here. If you were using the old links, they should point to the new version and hopefully you shouldn’t notice any change at all (beyond them being more reliable and frequently updated!). If you just want to subscribe, add these in your calendar application of choice:
- Sydney – https://krishoward.org/sydneymugs.ics
- Melbourne – https://krishoward.org/melbournemugs.ics
- Brisbane – https://krishoward.org/brisbanemugs.ics
- Perth – https://krishoward.org/perthmugs.ics
- Hobart – https://krishoward.org/hobartmugs.ics
How it works: I started the project with a straight monolithic reimplementation of the old PHP script, and I was super proud when I got it working locally. Unfortunately, the Meetup API did not like me firing off lots of requests in parallel and promptly banned my access key for a bit. As a result, I ended up breaking the script into three parts: “getgroups,” “getevents,” and “makecalendar.”
- “getgroups” runs once a day for each city. It requests all the groups in a location matching a topic list and then stores those groups in Amazon Simple Queue Service.
- “getevents” runs every minute of every day. It checks the SQS queue to see if there are any groups. If there are, it gets the top one and requests the next 10 events for it from Meetup.com. If events are returned, they’re saved in an Amazon DynamoDB database.
- “makecalendar” runs once a day for each city. It retrieves all the events for the city from the DynamoDB and then constructs an iCal feed from them. It then uploads the file to an Amazon S3 bucket.
This seems to spread the requests out enough that Meetup doesn’t complain. That said, as of three days ago SQS is now a supported event source for Lambda! That means instead of triggering “getevents” every minute, I could simply use the queue as the event source and have it trigger automatically. My only hesitation is the throttling on the Meetup side, so I’m going to have to do some testing to see if this is feasible.
Costs: I don’t have an operating cost estimate for this project, but I suspect I will be well within the free tier for all these services. I’ll let you know at the end of July.
Additionally: As part of this project, I also rebuilt my old personal site https://krishoward.org and migrated it to AWS for hosting. That code’s on Github here. I was indebted to this great post from Vicky Lai that describes how to get your custom domain and SSL certificate working with S3 buckets.