Last updated at 20th of July 2019, 15:35 (UTC)
Problem that we had two weeks ago has reappeared. We're working hard to resume normal service.
The issue is severe. Some customers might experience slow delivery of likes some mightn't get any likes on new posts at all. We temporarily don't accept any new orders.
- 19th of June 16:30 (UTC) We start noticing a rapid drop in deliveries and increased load on our system. Routine monitoring that we've been doing since abruption of deliveries in beginning of June helped us identify that something wasn't right quickly. We stopped accepting new orders within minutes to ease the load on our servers and we started looking for the cause of the problems.
- 20th of June 15:10 (UTC) We're posting this article to keep our customers updated on the issue, thus far all of our attempts to resume normal operations gave only temporary results.
- 20th of June 23:30 (UTC) Our developers are still working on it. We hope to give you better news tomorrow.
- 21st of June 20:10 (UTC) A quick update. We've been working on the issue non-stop. We still don't have an ETA on the fix. We'll keep you updated.
- 22nd of June 11:40 (UTC) The work is still ongoing. We hope to implement and test a possible solution by the end of the day.
- 23rd of June: Solution that we tested during the day didn't produce desired result. We've refunded all of the one-time orders that haven't been delivered. All of the customers that were refunded have received email notifications. If you haven't, please reach out to our support. Please keep in mind that our average reply time is longer than usual.
- 24th of June 15:00 (UTC) We're going to pause all of the subscriptions today, this change will apply to all charges starting tomorrow, 25th of June 16:00 (UTC). None of the monthly subscriptions will get renewed after that. Our developers are prioritizing working on the solution to the ongoing issue. We promise to get back to you with more in-depth information tomorrow.
- 25th of June 16:30 (UTC) We've paused all of the subscriptions and sent out email notifications.
- 26th of June 17:40 (UTC) We're finishing up a possible solution and we'll be testing it tomorrow. We hope to be back with good news.
- 27th of June 21:10 (UTC) Our developers had to postpone the test until tomorrow. We spent the day optimizing the new system and implementing certain features.
- 28th of June 21:15 (UTC) We're done optimizing the new system. There are still some minor features to implement before testing our solution - hopefully today!
- 29th of June 12:40 (UTC) Here's a bigger update since we've cleared out a lot of tasks that we wanted to implement and the finish line is in sight.
What we've done is implemented a different delivery system written from scratch. Doing it was more efficient than updating and resolving all of the problems with the previous system that were caused by Instagram updating their algorithms. That is also the reason for why it's taking us this long to return to normal state of things.
At this point we've done a great amount of work on the new platform and small-scale tests show promising results. Right now we'll focus on making it stable and resilient. This might be an unpopular decision, but we'll choose to extend the development timeline by several days in order to ensure future stability. This means that it will take a couple of days longer for us to roll out the update, but it's ultimately better to make it right the first time than to deal with the consequences of a system-wide collapse in the near future.
It's hard to give an exact timeline of when the work will be completed. We still have to implement and ensure that all of our current features such as gender targeting, country targeting and delivery speed are working. This might create a complication that will drag out the development a little bit, so we will try not to overpromise for now.
You can expect a bigger update on Monday when we'll have an even better idea of how the things are progressing.
- 1st of July 15:40 (UTC) We'll start with large-scale testing tonight and continue it throughout tomorrow. Our team will continue to work on scalability and stability in the meantime, so this isn't the last test for the week.
The work on our servers won't affect your accounts and the delivery will stay paused for all of our customers. We will post an update later tomorrow, probably at night, that will hopefully provide better insight as to when you can expect the resolution of these problems.
- 2nd of July 22:40 (UTC) Testing was delayed due to additional work and bugfixes that were required before we could continue today. We expect to be done with this iteration of tests tomorrow.
- 3rd of July 20:30 (UTC) We've done a few tests of our new implementation and it looks promising so far. We still have more tests to go through. We'll keep you updated!
- 4th of July 23:00 (UTC) We've been testing the rest of our implementation and everything seem to be working as it's supposed to.
However, we still need to make a larger test to make sure that we can deliver likes to all of our customers without any problems.
- 5th of July 23:30 (UTC) Today we've verified that our queues and delivery systems function and are able to handle all kinds of incoming orders, as well as errors and crashes if any would occur. We're closing in on the finish line and things look very promising.
We've been fixing different kinds of errors and edge-cases from earlier and we'll of course have to verify that they are completely resolved. As mentioned in yesterday's update we'll run a large-scale test once we're satisfied with the state of our system and once the roadmap has cleared up.
We expect these tests to take at most a couple of days. We wouldn't want to be overly optimistic, so we'll not say anything more than that but you can expect more updates shortly.
- 8th of July 13:00 (UTC) The larger test that we had planned and executed during the weekend was disappointing. Our solution didn't produce the expected result and we're still unable to deliver likes on a scale that is required for Stormlikes. This is due to Instagram's restrictions on likes delivery.
We're trying a different approach right now. We'll keep you updated.
- 10th of July 19:20 (UTC) Some better news at last. We've been trying several different approaches for the last couple of days and some of them have worked. We will be back with a larger update tomorrow.
- 11th of July 20:30 (UTC) We've been working on implementing our new approach to deliver the likes. There are still some major things to do on the implementation.
We'd also need to do a large-scale test of the likes delivery to see if that approach is working to serve all our customers. Fingers crossed!
- 14th of July 22:40 (UTC) We apologize for the lack of updates over the past couple of days. We've been busy testing and implementing the latest approach. We're still hesitant that it'll work well enough and our team is exploring other alternatives while the tests are being run.
- 20th of July 12:10 (UTC) This is a very short update, we wish not to make a big announcement just yet. We'll be pushing an update, hopefully today, which will enable one-time purchases to test run the new system in live mode. If everything goes smoothly during the next few days, we'll start reconnecting our subscribers to the new system.
- 20th of July 15:35 (UTC) One-time purchases were launched. We've disabled trial and as mentioned in the previous update we will not be delivering to our subscribers just yet. Our team will be monitoring the system and we might stop accepting orders without prior notice if any issues arise.
We are very sorry for the trouble that we've been having this month. We are working hard to resolve these issues and eliminate the root causes of these problems.
The Stormlikes team