BSFC Status

Graph from BSFC admin area showing how far behind our pending posts are. You can see a steady increase in delay once the API connection goes down, and a sharp decline when the API connection was restored.

It looks like the Bluesky API went down for about 30 minutes, and we had a backlog of pending posts to clear.

The Bluesky Feed Creator system detected this and recovered automatically as soon as Bluesky's API came back online 😁.

You may have also seen other errors in this time, e.g when blocking.

And we're back up to real-time, took under 15 minutes to process 5 hours of posts! 💪💪

We're seeing similar firehose issues today, and we are currently catching up on 5 hours of posts!

Additional monitoring has now been added to alert us to this situation automatically so we can deal with it promptly going forward.

Graph showing how far behind we are currently

Shouldn't be too long 😁

Our firehose connection has been giving reduced posts for about 2 hours. This has been fixed and we are catching up now. Thanks for your patience!

This should be resolved now, but let us know if you continue to experience issues and send us a screenshot if so 🙂

Thanks Kieran:

We had a bit of downtime overnight, our cloud provider was having trouble with the physical hardware our service was on.

It looks like this has been resolved but we will keep an eye on it.

We had an outage beginning 35m ago, which was affecting some regions including the US, and which has now been resolved.

Search for posts to add to your feed. Enter multiple search terms (one per line) to search across multiple queries. You can select multiple posts to insert at once.

A bug has just been fixed that was stopping the new "Search and insert posts" dialog from actually inserting the posts in some cases.

💡🚨💡🚨💡
Free feeds now draw from a sampling of the Bluesky network rather than its entirety. If your feed is public, you'll see it picking up fewer posts.
To keep your feed content fresh, you've got some options:
1️⃣ Focus your feed by pulling posts from specific users/ communities. This is unthrottled!

We've fallen behind by up to 1 hour overnight.

Additional capacity has been added to work through the backlog, so things will be back to normal soon.

If your feed is frozen, please set "'Pending Posts' behaviour" to "Wait for labels". There's an odd issue where feed caches won't update without the pending posts service.

This has been fixed
- Josh

Heads-up, looks like some requests are occasionally failing, but this will never happen for more than 2 seconds (auto-remediation will kick-in). bsky.app/profile/did:...

One of our ingress services is having issues, this is causing some requests to occasionally fail. A temporary stop-gap fix has been applied to mitigate this.

Thankfully, this service isn't essential and can be removed if needed.

No worries!

A screenshot of the user's feed posts loading in the BSFC dashboard.

I'm able to reliably load your feed in our admin backend

Your feed is still working, it's just that 1 request failed.

When a feed loads, it goes "User -> Ingress Services -> Feed API". Feed API handles your feed, it's just Ingress is sometimes dying. Your feed is ok, just that 1 request failed.

I made a change, try loading your BSFC feed now :)

Never-mind, I see what you mean. A separate system that proxies all traffic is having issues, I will fix it when I get a moment. It's crashing and restarting, which is causing connections to temporarily drop.
- Josh.

Hey, your feed bsky.app/profile/this... loads fine for me? Which feed are you referring to?

All caught up, feeds are now operating as normal.

For future reference, we publish how far behind our indexers are (at the bottom of) blueskyfeedcreator.com/stats

A graph with a purple line and a blue line. The blue line is dramatically peaking.

blue line go brr

Hi everyone, Josh here again. Not really sure what happened…but I’ve done some things and feeds will start to move again.

And we're back to normal, thanks everyone!

Morning everyone, apologies it looks like our API connection was disabled overnight again. We are going to look further in to this today.

Pending post are currently 2 hours behind but we are reconnected and catching up quickly.

We're doing a small resilience upgrade which will mean we might miss some posts over the next 30 minutes. Apologies for any inconvenience.

Pending post delays and our API connection have now been fixed, thank you for your patience everyone

Morning everyone! Pending posts are currently about an hour behind, and it looks like our API connection was disconnected overnight. We are looking in to this now.

🫡