[CLOSED] Issues: 19th of June

STATUS: CLOSED

Description:
Problem that we had two weeks ago has reappeared. We're working hard to resume normal service.

Current situation: 
The issue is severe. Some customers might experience slow delivery of likes some mightn't get any likes on new posts at all. We temporarily don't accept any new orders.

Timeline:

• 19th of June 16:30 (UTC) We start noticing a rapid drop in deliveries and increased load on our system. Routine monitoring that we've been doing since abruption of deliveries in beginning of June helped us identify that something wasn't right quickly. We stopped accepting new orders within minutes to ease the load on our servers and we started looking for the cause of the problems. 
  
  
• 20th of June 15:10 (UTC) We're posting this article to keep our customers updated on the issue, thus far all of our attempts to resume normal operations gave only temporary results.    
  
  
• 20th of June 23:30 (UTC) Our developers are still working on it. We hope to give you better news tomorrow. 
  
  
• 21st of June 20:10 (UTC) A quick update. We've been working on the issue non-stop. We still don't have an ETA on the fix. We'll keep you updated. 
  
  
• 22nd of June 11:40 (UTC) The work is still ongoing. We hope to implement and test a possible solution by the end of the day. 
  
  
• 23rd of June: Solution that we tested during the day didn't produce desired result. We've refunded all of the one-time orders that haven't been delivered. All of the customers that were refunded have received email notifications. If you haven't, please reach out to our support. Please keep in mind that our average reply time is longer than usual. 
  
  
• 24th of June 15:00 (UTC) We're going to pause all of the subscriptions today, this change will apply to all charges starting tomorrow, 25th of June 16:00 (UTC). None of the monthly subscriptions will get renewed after that. Our developers are prioritizing working on the solution to the ongoing issue. We promise to get back to you with more in-depth information tomorrow. 
  
  
• 25th of June 16:30 (UTC) We've paused all of the subscriptions and sent out email notifications. 
  
  
• 26th of June 17:40 (UTC) We're finishing up a possible solution and we'll be testing it tomorrow. We hope to be back with good news. 
  
  
• 27th of June 21:10 (UTC) Our developers had to postpone the test until tomorrow. We spent the day optimizing the new system and implementing certain features. 
  
  
• 28th of June 21:15 (UTC) We're done optimizing the new system. There are still some minor features to implement before testing our solution - hopefully today!
  
  
• 29th of June 12:40 (UTC) Here's a bigger update since we've cleared out a lot of tasks that we wanted to implement and the finish line is in sight.
  
  What we've done is implemented a different delivery system written from scratch. Doing it was more efficient than updating and resolving all of the problems with the previous system that were caused by Instagram updating their algorithms. That is also the reason for why it's taking us this long to return to normal state of things.
  
  At this point we've done a great amount of work on the new platform and small-scale tests show promising results. Right now we'll focus on making it stable and resilient. This might be an unpopular decision, but we'll choose to extend the development timeline by several days in order to ensure future stability. This means that it will take a couple of days longer for us to roll out the update, but it's ultimately better to make it right the first time than to deal with the consequences of a system-wide collapse in the near future.
  
  It's hard to give an exact timeline of when the work will be completed. We still have to implement and ensure that all of our current features such as gender targeting, country targeting and delivery speed are working. This might create a complication that will drag out the development a little bit, so we will try not to overpromise for now.
  
  You can expect a bigger update on Monday when we'll have an even better idea of how the things are progressing.
  
  
• 1st of July 15:40 (UTC) We'll start with large-scale testing tonight and continue it throughout tomorrow. Our team will continue to work on scalability and stability in the meantime, so this isn't the last test for the week.
  
  The work on our servers won't affect your accounts and the delivery will stay paused for all of our customers. We will post an update later tomorrow, probably at night, that will hopefully provide better insight as to when you can expect the resolution of these problems.
  
  
• 2nd of July 22:40 (UTC) Testing was delayed due to additional work and bugfixes that were required before we could continue today. We expect to be done with this iteration of tests tomorrow.
  
  
• 3rd of July 20:30 (UTC) We've done a few tests of our new implementation and it looks promising so far. We still have more tests to go through. We'll keep you updated!
  
  
• 4th of July 23:00 (UTC) We've been testing the rest of our implementation and everything seem to be working as it's supposed to.
  
  However, we still need to make a larger test to make sure that we can deliver likes to all of our customers without any problems. 
  
  
• 5th of July 23:30 (UTC) Today we've verified that our queues and delivery systems function and are able to handle all kinds of incoming orders, as well as errors and crashes if any would occur. We're closing in on the finish line and things look very promising. 
  
  We've been fixing different kinds of errors and edge-cases from earlier and we'll of course have to verify that they are completely resolved. As mentioned in yesterday's update we'll run a large-scale test once we're satisfied with the state of our system and once the roadmap has cleared up.
  
  We expect these tests to take at most a couple of days. We wouldn't want to be overly optimistic, so we'll not say anything more than that but you can expect more updates shortly. 
  
  
• 8th of July 13:00 (UTC) The larger test that we had planned and executed during the weekend was disappointing. Our solution didn't produce the expected result and we're still unable to deliver likes on a scale that is required for Stormlikes. This is due to Instagram's restrictions on likes delivery. 
  
  We're trying a different approach right now. We'll keep you updated. 
  
  
• 10th of July 19:20 (UTC) Some better news at last. We've been trying several different approaches for the last couple of days and some of them have worked. We will be back with a larger update tomorrow. 
  
  
• 11th of July 20:30 (UTC) We've been working on implementing our new approach to deliver the likes. There are still some major things to do on the implementation.
  
  We'd also need to do a large-scale test of the likes delivery to see if that approach is working to serve all our customers. Fingers crossed!
  
  
• 14th of July 22:40 (UTC) We apologize for the lack of updates over the past couple of days. We've been busy testing and implementing the latest approach. We're still hesitant that it'll work well enough and our team is exploring other alternatives while the tests are being run.
  
  
• 20th of July 12:10 (UTC) This is a very short update, we wish not to make a big announcement just yet. We'll be pushing an update, hopefully today, which will enable one-time purchases to test run the new system in live mode. If everything goes smoothly during the next few days, we'll start reconnecting our subscribers to the new system. 
  
  
• 20th of July 15:35 (UTC) One-time purchases were launched. We've disabled trial and as mentioned in the previous update we will not be delivering to our subscribers just yet. Our team will be monitoring the system and we might stop accepting orders without prior notice if any issues arise. 
  
  
• 23rd of July 18:50 (UTC) It's been a couple of days since we launched the one-time packages. Deliveries are more or less stable and all of the options are working correctly, however there's still a number of issues that we need to address before we can confidently resume subscriptions.
  
  
• 26th of July 20:10 (UTC) We'll start activating subscriptions in several waves starting next week. Our developers decided to split up the activation to ensure that our system doesn't get overwhelmed. The first emails offering a choice between activating subscription (with an additional compensation) and a refund are scheduled be sent out on Monday, 29th of July.
  
  The email will be valid for a month and as mentioned above we'll include a choice to refund the subscription. You can expect another update with more information before we send out the first emails. 
  
  
• 29th of July 16:50 (UTC) We promised to get back to you before the first emails regarding subscriptions are sent out.
  
  The first important bit of information is that our payment gateway is unable to process refunds right now, therefore we'll have to refund payments that came in before 24th of July through a bank transfer. This is far from great because it'll require a bit of extra work on your part since not everyone has easy access to their bank's SWIFT code and account's IBAN number. We apologize for that inconvenience.
  
  Second bit of information is that when you get the email with the option to activate your subscription you'll have two months to use the activation link. When you activate the subscription, we'll add two additional weeks on top of the time that you had left as a compensation for the trouble that we've had. 
  
  The first batch of emails is scheduled for today. We'll let you know when we'll activate the second wave of subscribers in a couple of days after we see how the first wave of subscription performs. 
  
  
• 2nd of August 17:10 (UTC) We've been monitoring activity of reactivated subscriptions this week, but we still need more time to predict how other upcoming waves will affect our delivery. As of now, only a quarter of all activation links that were sent out to the first wave of subscribers were clicked. This puts us in a situation where we can't estimate the load with reasonable accuracy since the first wave was a relatively small subset of our users. 
  
  We will get back to you with more updates. 
  
  
• 6th of August 15:10 (UTC) We realize that many of you wish to reactivate your accounts, we hear you and we understand you. Our devs don't want to end up in the same spot where we were two months ago, so we're taking out time and being very careful. 
  
  We don't sit still either. Our team has been working on backup systems for the platform behind Stormlikes and we've seen steady progress.
  
  As many of you might've noticed, hundreds of websites that were offering similar services to ours are either struggling, have gone silent or shut down completely. We're proud to be one of the few who are still standing, but it's a big challenge and we're doing our best to fully reactivate our services.
  
  
• 6th of August 17:40 (UTC) There's another thing that we have to mention today. 
  
  We've always striven to be the most accessible service that sells real likes. Competitive pricing is very important for us, but we have to inform you that we'll need to adjust our prices for one-time packages tomorrow. However! This will not affect registered users. Any of our customers who have ever had a subscription will still be able to log in and enjoy the same prices as before. This will probably change in the future, but we'll inform you well in advance before it happens.
  
  Our reasoning is simple. It has become much harder to deliver likes and our costs have increased significantly to a point where we no longer can keep our current prices. You can read about the increase in more detail here. It isn't great for us as the new prices will turn away many customers. Yet, we did our best to stay competitive and we will probably still be the cheapest provider of real likes on the market. 
  
  
• 13th of August 14:20 (UTC) We are sorry for the long radio silence. There are many things going on right now. We haven't sent out any further emails with subscription activation and we probably won't do it for a while. 
  
  We're disabling targeting today due to multiple complaints. We'll be investigating a number of issues and fixing up several regions.

We are very sorry for the trouble that we've been having this month. We are working hard to resolve these issues and eliminate the root causes of these problems. 

Sincerely,
The Stormlikes team