Mastodon Server Scheduled Maintenance: 2024-12-21

by

The Mastodon server will be taken offline for a brief maintenance, starting 2024-12-21 at 12:30 UTC, while we perform some maintenance to increase its resources:

04:30 PST | 05:30 MST | 06:30 CST | 07:30 EST | 08:30 AST

The Mastodon server should be unavailable for roughly 5 minutes while we move to a new server.

When the maintenance has concluded, the performance of the system should be greatly improved.

This topic will be updated when the maintenance has concluded, or with any other relevant details.

Thank-you for your patience while this critical maintenance is underway!

  • 2024-12-21 12:30 UTC – Maintenance has begun, the Mastodon service will be unavailable while the server migration is underway.
  • 2024-12-21 12:45 UTC – Maintenance has concluded successfully. Please alert @mick@cosocial.ca on Mastodon if you experience any issues of concern. Thank-you for your patience!

Mastodon Server Scheduled Maintenance: 2024-12-13 (POSTPONED)

by

The Mastodon server will be taken offline for up-to one hour, starting tomorrow at 11:00 UTC, while we perform some maintenance to increase its resources:

03:00 PST | 04:00 MST | 05:00 CST | 06:00 EST | 07:00 AST

When the maintenance has concluded, the performance of the system should be greatly improved.

This topic will be updated when the maintenance has concluded, or with any other relevant details.

Thank-you for your patience while this critical maintenance is underway!

NOTE: THIS MAINTENANCE HAS BEEN POSTPONED.

An announcement will be made when it has been re-scheduled.

Stay tuned.

2024 AGM held, introducing our new Board of Directors

by

On September 17 2024, we held our 2nd Annual General Meeting of the CoSocial Community Co-operative. Members can find our draft minutes on our members discourse. After the member meeting where directors were decided, we had a board meeting to figure out director roles and terms.

This year, board members move into 2 year or 1 year terms. We used a mix of previous board time and preference to determine who of the new board members would take 2 year terms, and who would take 1 year terms.

As well, the board had to elect a President and a Vice-President. Dawn Walker was acclaimed as President, and Mick Szucs acclaimed as Vice-President.

Thank you to everyone that volunteered to serve as board members, and to all members that attended and participated.

Here are your 2024 – 2025 board members:

Becoming our own Fiscal Host on Open Collective

by

Just a quick note to let everyone know that all paying members will have received a brief message saying that your membership contribution has been paused.

We’ve used Fission as a Fiscal Host on Open Collective to hold funds for us and pay expenses ahead of setting up the formal co-op and Vancity bank account.

Fission is shutting down Fiscal Hosting, and so the time is now come for us to be what is called an Independent Collective on Open Collective. We hook up our own bank account and Stripe billing, rather than having Fission handle it for us.

You’ll get a message about re-confirming your credit card details so that your annual membership will get charged automatically.

We wrote about renewals at the beginning of the year. You can, of course, choose to not renew and your membership will end on your anniversary.

We’ll update this announcement with more information as the transfer happens. Please feel free to ask questions in the comments here, or if you see some messages that are unexpected.

Annual Renewal of CoSocial Membership with Open Collective

by

Wow! What a year it has been. We are coming up on the first year of CoSocial. Auto-renewal for your membership will take place soon, from Open Collective, at the beginning of the month of your CoSo-anniversary.

For some, that will be March 1st.

We believe that, since launch, we’ve offered a reliable, responsive, and safe Fediverse environment for our members at an affordable price, and we believe we can keep this service going for the long haul.

Over the past year, your support has been instrumental in achieving significant milestones.

With your help, we successfully launched our instance, building the community and conversations. In recent months, we have experimented with offering different services Members forum, design challenge to develop our new logo.

In the coming year, we will continue to work on the directions from our AGM: create a thriving community, develop and launch member-ready services and working groups (Castopod working group), collaborate with organizations like IFTAS on trust and safety tools, for example on moderating transphobic content.

We hope you can continue your involvement – if you are new to the Members Forum, look around for a working group to share, build, do. Your expertise and input are essential as we continue to grow and improve our services.

If you have any questions or would like to make changes to your membership, leave a comment in the forum or post on Mastodon with #CoSocialCa.

Thank you for being a part of building CoSocial. Together, we’ve made it to year one, and look forward to many more achievements in the coming year.

Technology and Operations – 2023 in Review

by

Technology Update 2023
Hi all,

It’s hard to believe that we’ve not even been at this for a full year, yet.

As we approach the end of 2023 it seems like a good time to take a look at what we’ve done on the Technology and Operations side of the co-op in order to keep the lights on and provide stable technical resources for our members.

It has been an eventful year, and I am proud of what we have accomplished!

I wanted to look back and take an opportunity to share some of the key milestones and challenges we’ve encountered this year.

Starting the Journey:

@boris kickstarted our Mastodon deployment with a one-click install from Digital Ocean back on December 11th, 2022.

Aside from some initial configuration and the setup of some founding accounts, not much happened until we started opening the doors to new members in April of 2023.

Boris and @timbray put out a call for volunteers to help run this thing and a few of us answered.

It’s a tricky thing to figure out how to be useful in a volunteer organization, and I am grateful to Boris for the guidance and encouragement he offered myself and our other TechOps volunteers from the very outset.

Mastering Mastodon:

Our first task was to learn the ins and outs of Mastodon.

The one-click install provided by Digital Ocean was a useful starting point for our little server, but anyone who runs applications on the Internet will tell you that as soon as the system is live, you’re engaged in an endless battle against entropy.

Things break. Things need to be (constantly) updated.

Mastodon is built in Ruby on Rails and is a complex bit of software.

Knowing how to tell what a system needs to keep running smoothly is job one when taking on any new operations task, so we set about figuring things out.

Improving Visibility:

Shortly after I joined the team, we had our first outage.

I’m not sure what caused it, it was resolved with a quick reboot, and we weren’t down for very long, but the incident revealed that we were missing some crucial information about the operation of the server.

One of the first things we set out to do was to cobble together some monitoring and alerting tools so that we’d be able to tell if something was going on that might compromise the availability or performance of the server.

This suite of tools has grown throughout the year, providing insight into Mastodon’s internals, and giving us better ways to understand the health of the overall system.

Working Together:

In April, we were joined by Roberto, whose early involvement has been a major contributor to our ongoing success.

Roberto is a Terraform genius, and kindly integrated our GitHub, Digital Ocean and AWS environments using Terraform. We’ve been building on this work ever since, and I wanted to give him a shoutout for this significant contribution to our operations.

The core tech-ops team has remained small throughout the year, but we’ve been recently joined by Gov and Ian, and we’re continuing to learn how to build on each other’s strengths.

We come from various IT backgrounds, no-one has a ton of Ruby on Rails experience, and unless you’ve run one of these systems before you probably don’t know anything about it.

We’re all learning about what we’re running as we’re running it, which is exciting and challenging.

I’ve greatly enjoyed working with these guys, and I am excited to grow our team and and our technology in the year ahead.

If you know your way around a command prompt at all and you’re interested in helping out you are qualified and welcome!

Please reach out to Boris or myself and we’ll figure out how to get you involved.

Making Space:

Mastodon is hungry for storage!

We opened the doors to co-op members in April, and by April 15th the one-click install Droplet was bursting at the seams.

Every bit of content from everyone followed by everyone on the server gets added to our media cache.

By mid-April we were fighting against the limits of our hardware on a daily basis.

Fortunately, Roberto had been running his own Mastodon instance for a while and knew how to get us set up with AWS S3 storage and their Cloudfront CDN.

On April 15th he completed this major maintenance and we gained the room we need to allow the server to grow (thanks Roberto!)

Moving the Database to a New Data Place:

According to the docs Mastodon can (relatively) easily recover from most types of failure, but:

Mastodon stores all the most important data in the PostgreSQL database. The loss of the PostgreSQL database will result in the complete failure of the server, including all the accounts, their posts and followers.

So this seems like something to avoid!

In order to make sure that this most crucial bit of our Mastodon instance survives – whatever else might happen to the server – we set about moving the database off of the one-click install and into a Managed PostgreSQL instance with Digital Ocean.

Figuring out how to make this work took quite a bit of prep and was complicated by some … less than ideal behaviour from the Digital Ocean managed PostgreSQL service.

In the early morning on August 27th the server was offline for 23 minutes while the data was dumped and restored in its new home.

I am proud to say that this maintenance was the only “major” downtime we’ve had all year!

With this maintenance activity completed our data is replicated in real-time and we are confident that we’ll survive any major server issues with minimal loss of data, and we’re well positioned to scale up the server for years to come.

Caring for Your Mastodon:

Mastodon is in active development, and between April 2023 to the end of the year, there were 13 software releases!

Most of these have been minor bug-fix releases, but they have included one severe security issue (which was deployed with in an hour of its release) and one major upgrade.

As a team we have deployed 10 of these releases to our instance, with Gov and Ian taking the lead in promoting of a couple of them.

The most notable change to Mastodon this year was the release of version 4.2.0, released in October, which introduced full-text search.

This major release required the setup of a new Elasticsearch cluster – along with the monitoring and alerting required of every new service to make sure that everything continues to work as expected.

Conclusion:

Looking back, 2023 has been a year of learning, growth, and building community.

From our initial, small, all-in-one deployment we’ve grown to a server that supports more than 100 active members and can support hundreds more.

Our small band of volunteers have kept things up and running smoothly and I am proud of the work we have accomplished together.

Each challenge we faced was an opportunity to improve, and every milestone a testament to our team’s dedication.

As we gear up for 2024, we’re excited to continue this journey and look forward to building reliable, safe and engaging space for our members.

Strategic Priorities for 2023-24

by

We want our Association to be run by members like you, making decisions together! So, we’re figuring out our top priorities for the coming year and we asked for your input. We’ve already had a workshop, two office hours, and an online brainstorming chat.

After our Annual General Meeting (AGM), we opened a poll from October 4 to October 10, 2023. During this time, we asked you to vote for the three priorities that matter most to you.

Now, we’ll keep talking about how the Cooperative and the Board will use your chosen priorities to steer the Association’s work for the next year. Let’s do this together!

Here are the results.

Members can access the poll results and discuss in the members forum.

Server Maintenance 2023-10-01

by

Maintenance will be conducted to upgrade the cosocial.ca Mastodon instance beginning at:

11:30 PM ADT | 10:30PM EDT | 9:30 PM CST | 7:30 PM PDT

Members using the Mastodon instance may experience slowness and multiple brief (<30 second) outages during the upgrade process, which should be concluded within 30 minutes.

Following the maintenance, the server will be updated to Mastodon v.4.2.0, which includes a number of new features. More information about this release can be found here.

Follow this page for updates.

Update 11PM EDT – Maintenance has concluded successfully. Please note that full-text search features will be enabled tomorrow. Watch for future updates regarding this exciting new feature of Mastodon!