Denial of service attack [Neutralized]

2:04am PST – Criminals are attacking feedly with a distributed denial of service attack (DDoS). The attacker is trying to extort us money to make it stop. We refused to give in and are working with our network providers to mitigate the attack as best as we can.

We are working in parallel with other victims of the same group and with law enforcement.

We want to apologize for the inconvenience. Please know that you data is safe and you will be able to re-access your feedly as soon as the attack is neutralized.

We will update this blog post as soon as we have more information:

Thank you for your understanding,

/Oliv, Seb and Edwin


WAVE #1 – June 11th – NEUTRALIZED.

6:25am PST: We’re making some changes to our infrastructure that will allow us to bring feedly back online. However, these things take some time to put into place and it may still be a few more hours before service is restored. Thank you so much for your patience and for sticking with us. Remember, none of your data was compromised or lost in this attack.

15:07pm PST: We have neutralized DDoS attack that began at 2:04am PST last night. You should now be able to access your feedly from both feedly.com, mobile apps and third party applications. Our ops team is closely monitoring the situation in case the attacks resume. It might take a few hours for 40 million feeds we poll to get fully updated. We would like to re-iterate that none of your data was compromised by this attack. I would like to thank the ops team for working through the night to resume the service as quickly as possible. I would also like to thank the community for your amazing support! Note: If you are not able to access feedly.com it is most likely due to the fact that your ISP or computer is caching an old DNS entry. Please refresh your DNS cache or wait for it to expire.


WAVE #2 – June 12th – NEUTRALIZED

7:26am PST: We are currently being targeted by a second DDoS attack and are working with our service providers to mitigate the issue.

10:17am PST: The criminals have launched their second wave of DDoS attacks this morning. The ops team has reviewed the attacks and is working on building a second line of defense to neutralize this new attack. To make breaching those defenses harder, we can not communicate as frequently as we would like on the status of our infrastructure changes. But this second wave, although stronger, should be neutralized faster than the first one. We apologize for the inconvenience. Thank you for your support and understanding as we work our way through these waves of attacks.

11:30am PST: The second line of defense is up and wave #2 has been neutralized. You should be able to access feedly via feedly.com, feedly mobile apps and third party apps. These criminals are determined to try to extort some money and we are determined to say no to extortion and focus on building a stronger feedly instead. Thanks again for your overwhelmingly positive support throughout the last 2 days.


WAVE #3 – June 13th – NEUTRALIZED

3:30am PST The criminals have launched their third wave this morning. We are looking into it.

4:15am PST: Partially neutralized. You can access feedly from the US. We are working on enabling access from the rest of the world. The pollers are running a bit behind.

10:15am PST: The dev team is looking at options to work around the transatlantic issue. We will try to give you more details in 2 hours.

11:45pm PST: Pollers are back at full speed. There will be no gap in the content that gets delivered to you. The content of all the feeds should be back on schedule within the next 2 hours.

13:06am PST: Wave #3 fully neutralized. Everything is back to normal. You should be able to access your feedly on the web, via feedly mobile and via third party applications.


June 16th – ISSUE CLOSED

The service has been up and fully functional since Friday, June 13th at 13:06am PST. I would like to thanks the devops team for working really hard to build the lines of defense necessary to shield us from these DDoS attacks, I would like to thank the feedly community for sharing our determination to say no to extortion – your support was amazing. I would like to thank the Cloudflare team for working very closely with the devops team to help neutralize the attacks – your level of responsiveness was key to feedly getting back up so quickly.

Note: if you are not able to access feedly, please restart your machine/device to make sure that you have the latest DNS information regarding feedly.com.

 

Author: @feedly

Read more. Know more.

899 thoughts on “Denial of service attack [Neutralized]”

          1. Cloudflare would never work for a dynamic service. I know because I used a site that had it once. Great for static pages, but not for things you want to interact with. It’s rather annoying, really, and for feedly it would be almost entirely useless for the main purpose of keeping up with news.

            1. As others side, that is complete wrong. Feedly does use CloudFlare. And of course CloudFlare can work with a dynamic service. You seem to be under the impression that CloudFlare is simply for ‘caching static files’. That is one small feature and even that can be configured to work with dynamic sites. A better statement might be, “I don’t know how to setup CloudFlare to work with a dynamic service. Can someone help me?”

          2. To reinforce what Peterson Silva said, they *are* using cloudflare. Again though, 3rd party caching doesn’t help when an inherently dynamic site gets DDoS’d.

      1. The problem with DDos is the “Distributed” part – if the bot network involved is large enough, it’s very hard to block the bots but not the regular users.

        1. The only way I know to stop this sort of attack is to actually find a bot. DoS of any kind is petty script kiddie stuff. Lame. I say that Feedly should just wait it out.

          1. A DDoS isn’t always “Script Kiddie” stuff.

            There’s several variations of the Zeus botnet (among others!) out there with several million nodes connected to it. The botmasters are more than willing to rent these botnets out in whole or in part for any purpose, whether that purpose is email spam, phishing, keylogging, or even a simple DDOS. In this case, the person could have rented the botnet for a certain amount of time with the intended purpose of extorting Feedly to make a profit.

        2. Very true! but if you use a paid DNS service like VeriSign they can handle the DDoS attack and you won’t even know it hit you. That is the difference between free and paid DNS.

          1. Why would the DNS provider change anything to the attack? Once the IP of feedly’s server is received, why would DNS be involved at any point?

            1. I got a CloudFare error earlier, so yeah I think they are using CloudFare. Like others though, I’m really not sure how that would change anything.

          2. And how should they (who only get DNS requests) distinguish if a specific PC does calls to feedly from your browser or not from your browser? The problem with botnets is, that it’s the same computers that normal surfers use. If the http-calls are not done to stupidly, even for the server operator there is virtually no way to distinguish the bot from an (active) user _before_ the DDOS breaks the server.

            1. They should use Arbor Peakflow or a device like that from their ISP or purchase an Arbor Pravail in order to avoid this in the future. The platform will filter the real http petitions from the false ones, and of course it allows deeper countermeasures…

              1. One of the ways to distinguish the botnet behaviour is the way they do attacks, HTTP is based on TCP, the attacks the botnet usually do is open a TCP connection and let it expire without requesting anything, every single time, that will flood the TCP table of the victim and won’t be able to handle more petitions. Another type of attack is, of course, ask for a HTTP page a lot of times exhausting again the capabilities of the server.

                A legitimate user won’t have this behaviour, it’s not that hard to distinguish real users from bots.

                1. Out of interest, if Feedly had a list of the IP addresses that had accessed their service in the month prior to the start of the attack, could they simply turn that into a whitelist? Would that be useful mitigation against the attack?
                  (Not that I’m saying it would necessarily be a good idea.)

                  1. Because IPs change. When you’re in Starbucks, you have a different IP address than the one you used to access Feedly last week/month. And even if you never left home, your IP address might change.

      2. No simple solution exists. Picture 10 million or 100 million unwanted strangers showing up at your house and your legitimate friends trying to get in. There is just too much volume. the botnets can have millions of devices from around the world trying to all access the same servers, thus blocking legitimate access.

      3. There isn’t an appropriate defense for all types of DDOS. Unless you’re the size of Google, you can’t afford to mitigate it yourself, and CloudFlare-type services can only block certain types of DDOS attacks.

      1. Like the people doing this would give them their real names.

        Seriously, use some common sense.

  1. No problem guys, we will wait.
    Hope you send those idiots to jail.

    Keep up the great work on Feedly!!!

    1. +1

      Absolutely. I can wait to read my news. Better than encouraging others to engage in this bad behavior.

  2. Great to have feedback! Thanks! And good luck resolving the issue!
    Maybe you can post later how one can defend against DDoS, so we’re all better prepared against it!

      1. DDoS is not just a question of bandwidth… It’s an overload CPU for servers most of the time…

        1. No, actually, the complete opposite. CPU overload is typically a regular DoS attack, by exploiting a CPU intensive task or bug in the software. DDOS attacks are typically bandwidth-consuming attacks that can literally saturate an entire inbound pipe. That’s why they’re called distributed attacks, because it requires multiple sources to perform such an attack.

      2. That might be difficult depending on the type of attack. Using something like ntp or DNS attacks might well exceed 100GBit/s and that type of bandwidth is not cheap…

  3. Good to know. Not good news iofcourse, but good to know that there’s a reason behind the issues. Hope it gets sorted out soon and that the idiots behind the attack are punished.

  4. Ok, thx for the info.

    For now, just dont worry about users.Take your time, figth againt that cyber-criminal and don’t pay extorsions.

    And as Felipe Said, will be great if you make a post Mortem.

  5. Do you guys have CloudFlare? If not, it might be a good time to have a chat with them! :)

  6. Hang in there guys, we’re not going anywhere and are right behind you. (Just wish I could help more)

  7. Better some inconvinience in terms of feedly non-accessible than helping criminals to establish a new business model. Thanks for not giving in!

  8. Don’t let these jerks win. I’m willing to go without Feedly as long as you guys need. It’s just a minor inconvenience for me, but as long as these idiots don’t get a dime from you I am OK with that. I appreciate the update.

  9. 그들에게 굴복하지않고 feddly를 지켜주세요
    그 사실이 사용자들에게 믿음을 줄것입니다.

  10. How the fuck did you guys let this happen? There’s no need for so much downtime from a DDOS nowadays.

      1. I actually do, there’s no need to be offline for such a long time because of a DDOS, not if they would have had proper protection against it before it started.

        1. No, you really don’t. Depending on the size (bandwidth) and scope (how many bots involved), there is no way a site the size of feedly could be prepared before hand.

          The reason sites like Google can shrug off DDoS attacks is sheer bandwidth. They can take everything a big botnet like this can throw at them, chuckle and say, “Heh. That’s cute!”

        2. Please, share a bit of your expertise to summarize how a site can provide a 100% guaranteed protection against these DDoS attacks. I’m sure we’d all love to know.

          1. Contact the people at Black Lotus or ZEN Network, they (or other services of an equally high standard) could have prevented this. I just gues that Feedly didn’t really care about this enough to spend any money on it.

              1. Great argument to disprove that a DDOS attack shouldn’t have such a big impact on the uptime of a website.

            1. Just because a possible solution exists does not mean that everyone can afford it. I have no idea what Feedly’s financial position is, so I can’t comment on whether or not they should have anticipated a DDoS and paid extra for the protection. As for whether they cared enough about it to spend the money, nothing teaches so well as getting hit like this. Out here in the physical world, everyone can pay for flood insurance, too — but most don’t think about it until after they’ve suffered a flood and find out their home-owners insurance doesn’t cover floods.

              No one is perfect, so I’ll give the folks at Feedly a chance to learn from the situation and take steps to prevent it in the future. If they fail, I’ll move on to someone else.

              1. Feedly is still a free service, and I would guess that most users don’t pay.
                Protection against threats is always relative to the money you can (or want to) invest.

              2. I’ve been glancing through the comments and this one seems to sum things up for me. I see a lot of comments and critiques that are really unproductive. If someone believes they know how things should have been set up to avoid attacks like this, then by all means, please share for the benefit of everyone. Bottom line… I use Feedly as a FREE service and I suspect that many others do as well. This should be taken into account when passing judgement. I’m sure the people behind Feedly have been doing their best. LIghten up on them!

                It may be a small inconvenience for me to be without the service for a while, but I am also willing to give Feedly a chance to get it right. If Feedly gives in to the attackers and pay up, I’m sure these same type of attacks will increase. This seems to be the general consensus.. Personally, I appreciate whatever efforts are being put into the service. Good luck!

            2. Nick, you’re thinking of a DoS attack. This is a DDoS, much harder to set up, but much, much harder to stop or protect against. If the net is thrown wide enough, there is virtually know way to tell a legitimate request from an bot request. This sort of attack takes years of planning and is a bitch to settle. To the server it just appears like the customer base just increase a million fold, with requests coming from all over the world. It is nearly impossible to separate the wheat from the chaff.

              1. Not to mention that most capacity planning, especially for small companies, is based on “normal plus X percent” to account for a considerable (but not massive) bump in traffic, and when that number suddenly grows exponentially, there is no way to account for it quickly.

    1. why would a site like feedly waste the money and time to purchase the equipment and services required to harden their servers against a DDOS attack? Sure the tech exists to make a DDOS attack moot, but it isn’t cheap.

      They are politically neutral and fly well under the radar. They are not a prime candidate for a DDOS. This is a random extortion attempt. Bad luck. Chance. Roll of the dice.

      Most ordinary every day sites like Feedly DO NOT spend the time and money, which is NOT insignificant, to harden against DDOS.

      Until it happens.

      Then they sigh, write the checks, and move on.

      1. No there isn’t. There is software to protect against a DoS, but not a DDoS. It’s virtually impossible.

        1. I wonder how feasible it really is to use machine learning to classify attackers/legitimate users. Even if you have a 20% error, that could better than 100% downtime.
          Of course, making that work without amplifying the DDoS itself could be challenging and probably not even worth thinking about for sites like feedly, but I wonder if that could end up being a temporary work around in some cases.

          http://stats.stackexchange.com/questions/23488/applying-machine-learning-for-ddos-filtering

    2. I take it you are a paid subscriber to Feedly then, to have such high expectations of a small company in the face of a targeted attack.

      I am too. But I’m not sure I want to have my monthly fee increased to pay for the kind of infrastructure that is capable of withstanding something like this. So maybe you should be demanding a more expensive service level available for VIP subscribers like yourself, (with a separate domain perhaps) enjoying DDoS protection.

  11. Good you don’t give in.

    Don’t negotiate and never surrender when facing criminals and terrorists. They will give up first.

    I’m realising I’m really a feedly addict. But I can wait too!

  12. Fingers crossed guys, I hope you will deal with them soon. Good luck and hope for the best. I have found out how I depend on Feedly. Almost like being without internet connection :) Keep up the good work, appreciate you have notified all users through email and Twitter.

  13. giving them cash will result in only one thing – further demands. There is only one solution – get a ISP that stops this shit and refer all demands directly to the police.

  14. I am so sorry.. but thanks for the feedback.. I was pressing F5 and on “down right now” to see it if was only me :(

    this feels like some form of terrorism.. but I wish you the best wisdom to resolve the issue as I love feedly :)

    1. Terrorism is a bit much. The words you’re looking for are vandalism and extortion. No one is terrified.

  15. Hang in there, I can wait! Thanks for providing a great service and being so transparent!

  16. If that’s the situation, do the best you can – no harm, no foul as far as I’m concerned. Once done, harden your perimeter :-)

  17. Shit, good luck. My company has been on the receiving end of DDOS attacks before, and I know how much of a pain they are to deal with.

    Don’t give in, my RSS feeds aren’t going anywhere :-)

  18. Do you have a backup? You could create an alternative domain on alternative provider to give an access due DDoS attack.

  19. Thanks for letting us know and not giving in to the criminals. I hope these guys are caught and due justice meted out. We’ll be here when you get back up and running.

  20. Your provider sould be able to apply QoS on your public IPs. Depends of the characteristics of the attack (TCP/UDP, fixed port source or destination)

    1. dude, no. just no. QoS is applied in the egress direction, in this case traffic leaving Feedly’s “public IPs”

      The problem is that Feedly is receiving too much traffic (ingress direction), and it’s likely that their entire pipeline from the ISPs is overly saturated. From the perspective of the ISP, the only way to handle a DDoS attack is neutralizing the traffic as close to the source as possible; a feat impossible for DDoS attacks. CloudFlare’s DNS service can mitigate the attacks, but it takes hours to set up and costs up to hundreds of thousands of dollars. on top of that, it’s a service you must sign up for, can’t just say “Please fix us now”, you must sign an annual contract.

      *experience* – worked as network engineer for ISPs during DDoS attacks.

      1. +1
        Me too, it’s a real bi**h.
        To the loosers who are trying to extort this FREE service: F U.

    2. You’re assuming the routers are even getting a chance! A DDoS attack completely overloads the entire incoming pipe. You can (attempt to) filter as much as you want, the choke is further up the line, and coming from multiple sources.

  21. DDoS attacks are very common and there is almost nothing that can be done to prevent them. The basic remedy is simply to change the IP addresses of the servers receiving the attack. That takes time and effort for the re-routing. Also, as these attacks by definition come from a vast amount of computers, it’s very hard to block all of them coming in, especially that usually the computers involved are unknowingly part of the botnet.

    Just be patient guys, they’re doing everything they can. When i worked for a major webhost we saw these things all the time. You can be without your news, lolcats, and other stuff for a few hours until it is resolved.

  22. Was just able to access through my Andriod app. I assume the attack doesn’t affect it?

  23. Good on you for not giving in. Hang in there! Ignore all the twats who have no clue spouting “How can this be, it’s 2014, derp derp” ;-)

  24. For those complaining that there should be a defense for this type of thing, you must not understand the nature of a DDoS. It’s basically the same thing as what a web browser normally does, it makes a server request. It just happens to make so many so quickly that the server simply cannot respond to them. Not really much different than if a site like youtube were trying to be hosted on a $5 a month shared web host. Resources simply cannot fulfill the requests being made, and that’s no fault of feedly either. It’s just the nature of the attack.

    1. Someone should trace anonymous here. What do you mean pay him. How do you know it’s a lonely male attacker who is a complete loser and will never amount to anything outside of being a damaging jerk? Most of this is just conjecture, but who other than the culprit doesn’t feel rage at this person, and want him to shove it?

  25. Thanks for letting us know. I was wondering why it was down. Good luck with fighting off the criminals. Never give in to blackmail! I hope Feedly is back up soon, but no pressure, I can wait a couple of hours/days ;-)

  26. You guys should consider making Google your front-end. Either AppEngine or Compute Engine firewalls and load-balancing should take care of DDoS for you.

    1. I’m pretty sure it’s in the ToS that Google will suspend your account under an attack as large as this. Firewalls are meaningless, they don’t do anything for a DDOS (you can’t filter something that’s choking your inbound pipe). Load balancing doesn’t matter because they’re not even legitimate requests typically.

  27. Don’t give in guys! This outage is great for my productivity ;) No, seriously, don’t pay those saddos.

  28. Good luck awesome feedly people – your excellent communication in this matter has made me resolve to pay for pro as soon as you’re back online

  29. Sorry to see Feedly fall. Hope you sort it out soon. The UK is thinking of introducing life sentences for hackers, does anyone here agree with that?

    Network Integrity is the sort of thing I would like NSA and GCHQ spend some time on.

  30. I don’t know what type of attack you are experiencing but I do wonder why mitigation is not in place that you have to work with ISPs to mitgate after the fact. You should already have in place scrubbing facilities to filter volumetric attacks and pass clean traffic . Blackhholing if needed and defence in depth intrusion prevention. Any type of attack should be able to be mitigated fully by your isp from BGP. DNS Ntp .reflection attacks to more persistent low level trigger attacks . Unfortunately a lot don’t.

    1. you must be new to the DDoS game. explain exactly how BGP is going to mitigate an attack? By setting up an ACL to deny a route’s advertisement? DDoS can span up to several THOUSAND bots, are you going to set up the ACL for each one of those bots? And what happens when the attack shifts to new hosts, are you going to statically update that list again?

      Moreover, you’re immature to believe this is on the ISP’s list of responsibilities. Nowhere in the SLA does it list that the ISP is directly responsible for protecting the customer’s traffic from attacks, they merely provide the pipeline to the Internet.

      1. Colin is correct Darth Doodle but your replies make it obvious you have no idea what you are talking about. BGP is an integral part to most of the volumetric DDoS protection services. Shut your hole.

        1. I’ve handled three large scale DDoS attacks when working for one of the big three ISPs in America. Go home internet troll, you’re drunk.

    2. That really doesn’t help if your routers are barely able to handle the incoming traffic. Killing it as far upstream as possible helps, but also not with all kinds of traffic. Mitigating a DDOS attack isn’t easy.

  31. Good luck! Feel that we are behind you and take your time. Feedly is an indispensable tool, I’m so happy I found you after the Google Reader was discontinued, and I’m ready to wait until you sort out this incident.

  32. As said by many, we can wait, and wish you good luck to deal with these criminals. Thanks for the good work and hang on guys !

  33. I heard good things about CloudFlare as a good CDN for DDoS mitigation. Have you, guys, considered them?

  34. Reblogged this on Martie din Post and commented:
    De când Google şi-a închis reader-ul, eu am migrat către Feedly, după ce am încercat câteva readere. Probabil cel mai mult m-a atras modul de organizare a blogurilor.
    Acum câteva ore a picat Feedly, în urma a ceva atacuri de-astea de hackeri. Voi ce readere mai folosiţi?

    1. I think a good analogy for understanding DDoS attacks is to think in terms of old-fashioned postal services. You probably still use USPS / Royal Mail / Canada Post / Deutsche Post / etc. occasionally. But imagine if 1 million people from all over the world all decided to start sending you lots of junk letters every day.

      (I started posting this as a comment. Then I just made it in to a full-on blog post. Click on the “ptfrd” if you want to read the rest of it.)

  35. Damn, Im sitting at work and have to work :D can’t waste my time in Feedly ^^
    Good luck guys. Guess tomorrow I have 1000+ feeds then.. :o

  36. Here’s an idea. Redirect the requests back the to issuer. Now, they’ve DDoSed themselves. Minimal downtime for users.

    1. Unfortunately this is not going to work, as whilst you would be sending back large amounts of data, it would be from a limited number of servers to a much larger network of computers, with each only having to handle a minimal amount of the requests. DDoS attacks work by flooding a single target with information: a single machine trying to flood multiple attackers is the antithesis of this.

      1. You are correct in that there is a much larger network of computers. I’d be curious to see what the actual numbers are of the attack bandwidth. With a unique set of filtering rules, this could still be effective. In any case, the solution is a redirect. Myabe not to the issuer, but to some cloud based mitigation service…

    2. You’d never get a single request out, the entire pipe is choked. Regardless, even if it were possible, it’d just result in “mutual” DDOS, and trust me, the guys performing these attacks could not care less. Their personal home internet isn’t going to be affected, so they have nothing to lose.

    1. What is it exactly? Excuse me you need that via a translator to translate. I see these programs continue to that the whole thing is rather worse as better. Because the more entries and API is so insecure connections.

    1. Based on the error message I’m seeing when I try to view feedly, it looks like they’re working on getting CloudFlare up and running right now.

  37. Tell us the IPs and we all DDos back to the attacker! XD

    Some more informations about the DDos!? Type!? Generated traffic? Zombies/bot number? Bot name!?

  38. Thanks for keeping us updated and taking a stand against these types of attacks.

    And to those complaing about feedly not having a defense about this sort of thing, gotta be honest you probably have not idea how denial of service attacks are done or how effective they can be. They’ve affected nearly every major website at one point or another regardless of security. Feedly is taking appropriate steps in working with law enforcement and our data wasn’t compromised. Stop victim blaming and be patient for a change.

    1. Well said!

      any DDOS attack is like make a car pass through a 1 tunnel at same time when driving at 200 km/h.
      All the cars crash in the entrance, and stop the others cars back.

  39. One question – Will RSS feeds that had updates posted during this outage be updated retroactively, or will there be gaps where the outage occurred? Either way, thanks for keeping us updated.

  40. I can’t believe the amount of entitled and ungrateful posts on here.

    Anyway, sorry you’re under attack, I look forward to using your services again.

          1. It won’t. Because nobody even know it exists.
            And good luck. I didn’t like feedly from the beggining. It was too.. different and I couln’t used to it.
            I tried so many RSS and went back, because Feedly is only good service not lacking anything and without annoying bugs.

    1. This is the first time I’ve seen any down time with feedly since I’ve been using it (the day after google announced they were retiring reader). It’s just unfortunate timing.

  41. Really glad you guys aren’t giving in. I’d rather go days and days without than give those assholes more motivation to do this in the future.

  42. Reblogged this on Nookish and commented:
    I didn’t realize how addicted I’ve gotten to my Feedly account, until it went down today…

    1. if you think that’s a nice reason to pay monthly…I’ll simulate DDoS on my free services (…)

    1. I don’t really know much about DDoS attacks or how to prevent them, so I read this page partly to see if I could learn anything. So far, I’ve learned that Cloudflare is a service that can prevent some attacks, but not all, and also renders its fans COMPLETELY INCAPABLE of reading a comments page before posting to it. Come on, this is like the tenth time I’ve seen this conversation! “Could they use Cloudflare?” “I heard it would help.” “I heard it wouldn’t.” “They’re already using it.” Over and over! Can we maybe pick a different service to argue about, or something?

  43. Patintely waiting…. confident that all will be stronger to those attacks….
    And hope that police can catch them (DDoS’ers).
    Paying could seem an alternative but in reality is a real bad one.
    Happy to know feedly isn’t giving up

  44. Good work guys. Seems to be coming back online now. Don’t ever give in to those blackmailing swine. Bolster you defences and just keep your users informed if it happens again. As most have said in here, we can wait for you to fight off the attackers.

  45. For folks questioning whether or not Feedly uses Cloudflare, the answer is Yes.

    Check a ‘whois’ on feedly.com and you will see that the DNS servers point to Cloudflare.

    I also just got a Cloudflare 522 error page when attempting to go to Feedly.com.

    1. So did I. That’s more than I was getting earlier so progress of a sort. They are obviously doing their best to sort it out, so we should just give them time and encouragement. For me it’s a free service and normally really great.

  46. I concur with most of your responders, don’t give in to the criminals. Love feedly and miss the feeds but prepared to wait whatever it takes.

  47. Like several other boarders, I too will wait… While you may not be able to get the exact names, I think you should try and see if it was real “criminals” or some kind of organised corp attack (thru criminals)… More importantly, who will gain by feedly going down… It may be just a coincidence – or not..

  48. I know from experience what a DDoS attack means, so I wrote a short blog post about how this is affecting Feedly. I hope nobody will leave the service as this is most certainly a very stressful time at Feedly right now.

  49. Fight the good fight as long as it takes! Your loyal customers will support you. I hope police find these bastards! The Dos attack will cause us very little problem, especially compared to the satisfaction of them getting arrested. And as long as you hold out, they cannot use there bot net on an easier target that is wiling to pay.

  50. It would be very reasonable to give the paying customers a periodical update. Even if you still can not do anything – they will be much more relaxed.

    Beside of that: They deserve it.

        1. re-read again all this, and maybe you’ll see how many little fishes comes around the throwen hook

  51. Whew! My main daily driver for news…is down now.. Its been more than 5 hours without news!!!

  52. Who is the attackers?Does US offices like NSA CIA etc come to resolve the problem?In this case all we are compromissed!

  53. I hope the police put these guys behind bars, if they’re even in this country, which is unlikely.

  54. Good! I’m very glad you are not giving in. This is not a life or death scenario, I just can’t access my feeds for a bit. If you have any further communication with them, make sure to use obscenity and insult them as much as possible (as them to kiss your ass or suck your dick, etc). I feel obscenity should be used more in today’s world, it would be refreshing. :)

  55. The comments made me wish I could understand anything anyone is talking about here. Regardless it boils down to feedly no work now. Feedly work later.

  56. Take your time to fix it ! You have serve as well in the past months and giving us more and more features and even integrated with 16 apps including evernote.

    There is no alternative to feedly after the demise of Google reader! I’m sure newsblur, instapaper are no match!

  57. Was wondering why you guys were down. When I checked again after I got home from work I Googled the news.

    Sorry to hear about that. You provide me with such an excellent luxury. I can only assume you’ve been working solidly between updates so I hope your sleep is deep and undisturbed when you do get it.

    Good luck.

  58. Still down for me, but good luck getting it back up. Don’t give in to their demands. You guys are awesome.

  59. Good for you for not giving in to their demands. I will patiently wait for Feedly’s return.

  60. It would be nice if there existed a CDN service designed to serve DDoS requests that responds by attempting to “infect” the bots. The vector would have two purposes: patch known vulnabilities, and clean the bots. It would then remove itself. All anyone under attack need do is redirect to the CDN and the botnet self-destructs. Too bad that’s not possible, or exactly legal, or something you could trust any organization to do… but is a nice dream.

  61. This Boris here, I am demand of $1000 dollar or I press F5 as fast as I can to continue DoS attack. I must warn my finger is so fast as I am 10 year veteran Tekken champion. Do not test my will of iron. Paypal money to boris@gmail.com to prevent further outage.

  62. Gotta’ love the self-proclaimed experts and the cynics who will always complain no matter what. XD
    It happens, hackers out there with nothing better to do (or rather… refuse to do something worthwhile) often play these games. I hope these people face repercussions.

    1. there is a new game, simulating a hack attack to gain popularity, irrational adepts, and subcriptors to pay gracefully a monthly bill for “reinforcing” the infrastructure…and giving the irrational adepts money to the service :-)

  63. Make feedly come back even stronger, and faster! Let the world see feedly is the best rss reader in the world that’s is better than instapaper, newsblur and other craps.

    Feedly is colorful and has the nicest interface! And this is a tool that beat all the competitors after google reader demise! Do not let give in! Take your time and come back even stronger and invincible!

  64. Still can’t load the page. I notice that the index page attempts to load something from best-deals-products.com. Seems the attackers are trying some malware phishing. Anyone else getting this?

  65. If you pay the criminals their extortion money (which is exactly what this is), all it will do is encourage them to go after you again or some other site.

    Keep fighting the criminals and we users will be right here waiting for you (and cursing the DDOS-ers for taking Feedly from us).

  66. Pingback: Mercury rising: Feedly and Evernote crippled by DDoS attacks while hackers target TweetDeck | PandoDaily
  67. am i wrong in wishing to find those responsible for DDoS and bludgeoning their skull with a cast iron pipe?

    props to feedly for not paying!

  68. Pingback: Feedly Reader
  69. Can anyone explain why Feedly has used over 4gb of outbound data on my phone over the past 10 hours? This amount of egress traffic should not be generated by a DoS attack (especially on an end-user device), nor the site being unavailable due to such. I think you need to check your seals, you have a leak. :-/

    1. It’s because is trying to connect to the servers, but the servers are down, so it try again (and again). Just disable the app.

  70. Certainly getting a touch ridiculous now. I’d be fine with the downtime if Feedly were to be professional give some sort of update on where things stand, but instead their Twitter/Blog/etc is stale.

  71. Wow. This is like losing contact with a friend. A very talkative and scatterbrained friend with ADHD, but a friend nevertheless. Keep holding on.

    Once y’all are back, I’ll upgrade. Don’t pay the crook. I’ll pay you :-)

  72. I think we support you guys not giving in, but it’s been over 7 hours since your last update.

    You can (and should) do better than this. Give us *something*.

      1. What’s to say? They’re working as hard as they can to deal with this and we can all surely be grown-up enough not to nag them.

    1. Patience, patience. The servants remain closed to the air. Let’s wait a few hours more.

      It is necessary to say, that they are doing as sensibly as possible to avoid any type of possible attack that exists till now.

      Patiente.

      I have to keep back, because I dress the seen, there are some of them who want money attacking.

  73. Glad you’re not caving in! Otherwise they’d never go away, trying to build a regular income. They’re likely renting a Botnet for the DDOS attack, and if you don’t pay, they’ll move on instead of losing money.
    I’ll gladly wait a little longer for my news feeds, if that means you won’t give in – thanks for your great work, by the way.

  74. I know you’ve been down for a while, and I really want to be reading my feeds, but please please please don’t pay them.

  75. A hopeless situation made worse by the fact that I am one of ‘paying customers’. No updates for the last 8 hours! Some of us rely on your service to support their work. An update with a realistic and achievable timescale for resolution would be appreciated. If you can’t resolve the issue in a reasonable timeframe, then please say so, so that those of us who wish to can find an alternate service.

    1. Agreed. The lack of communication it terrible. We were told it would be a ‘few hours’ for service to be restored. I can’t fault you if you are running into more issues restoring the service, but I can fault you for not providing new updates!

      1. We have not discussed the details of the infrastructure changes because we do not want to offer information to the attacker. We are very sorry for the inconvenience this lack of access might have caused.

  76. Don’t negotiate with terrorists. Glad to see you guys fight back, and it looks like you’re back up. Hopefully the upgrades made today will be enough to fend off anything like this in the future.

  77. Pingback: Anonymous
  78. I’m trying to access feedly.com from Germany and I’m still getting the same server timeout error messages as before…

  79. Ich bin geschockt über diesen Zustand. Ich frage mich was das Unternehmen Feedly denen getan hat. Ich kenne sowas bin kein Programmierer. Doch musste mich ein Stück rein arbeiten. Bin durch meinen Unfall wie Michael Schumacher noch schlechter dran. Doch werde ich daran arbeiten. Ich bleibe auf jeden Fall Feedly treu. Und wenn man was Spenden kann bin ich gerne bereit was dabei zu tun. Ihr schaft das davon bin ich Überzeugt, eher wie meiner einer. Und bitte baut euch mehr Sicherheit ein. Damit ihr nicht alles verliert. Ich wünsche euch den Erfolg den Ihr verdient habt. Gruß Andreas http://carrabelloy.de

  80. I, too, am unable to access either the Feedly website or the backend service through the Press Android app. I hope you guys aren’t being attacked again!

  81. btw flushing the dns-cache as suggested by @feedly on twitter didn’t change a thing for me…

  82. nothing that @feedly recommended has worked. still can’t access via desktop or mobile apps.

  83. “It might take a few hours for some of the 40 million feeds we poll to be fully updated.”

    THIS IS WHAT I FEARED MOST! The feeds were NOT polled behind the scenes.. that functionality was too affected. So what, you ask?

    Well, for example, if a site which RSS you’re subscribed to publishes e.g. 40 RSS items in e.g. 8 hours _BUT_ the RSS feed only lists 20 newest items*, and Feedly outage goes over that time = PEOPLE ARE GOING TO LOSE FEED ITEMS.. FOREVER! This is a fact. This is what I feared about with this DDOS attack. :(

    * This is very common, many of the people (even large sites/companies) maintaining RSS feeds are too stupid to realize this shortcoming.

    1. Another example: this is without feedly, a person let’s say uses a separate program for fetching RSS feeds (the program will update the feeds when the program is running) and he/she opens the home PC & the RSS program around 6pm to read RSS feeds.

      If e.g. a news site RSS has only got 40 newest RSS items in its feed BUT publishes 150 news stories (RSS items), starting from around 6am to 6pm when the user is ready to read daily news (that’s 12,5 feed items published per hour), that would mean that the person would only see news items published AFTER about 3pm!! He/she would see only the 40 latest RSS items! All morning and midday news gone, missed completely!

      This is what is going to happen to many people with Feedly!

      Like I said, it’s very common that a RSS feed has a really low value for “max items” while daily published RSS items tops that max value many times over.

      This is the reason I moved to Feedly from using a RSS program; I was missing feed items and I couldn’t keep my PC & the RSS program on all the time.

      Just see, check your feed history in Feedly after you get in, I bet that many feeds have unusual gap in RSS item timestamps.

    2. So, I finally got back in and looking at my biggest (releases most RSS items per day) news feed, I see that the overall time when feeds were NOT polled by Feedly is about 10 hours. And looking at the timestamps of the items, I see that, for that site, I lost about 2,5 hours of feed items. :(

      So it did happen just like I feared. Lost feed items.

  84. Anyone have the new IP address for feedly.com. Might be able to use that to access until host providers update (which can sometimes take up to 24 hours depending)

    1. The new IP is a clouflare IP, and you won’t get the new IP, that’s the point of cloudflare protecting (hiding your backend IP behind a proxy), if feedly go down again, sadly, that means the attackers have found their backend (I doubt they will, but who know, the attackers seems to be pretty smart imo), and they can throw all the bad trafic directly to the backend servers and bypass all the ddos protection.

    1. Well, I’m not an expert, but the problem for me (and probably for you) seems to be this: feedly has a new ip-address, but your ISP will direct you to the old one until they refresh their DNS-cache. So basically you’ll have wait until they do… (Anyone feel free to correct me if I made a fool of myself here…)

  85. Looks like you spoke too soon. Cleared DNS cache and even the link you posted yourself to feedly.com is broken.

  86. Changing the whois data to point at the cloudfare NS’s is not enough to propagate out the DNS change to everyone.

    Feedly must also have the old NS’s at gandi.net return something sensible for the new Cloudfare installation, because the TTL on the NS entry is pretty big:

    $ dig feedly.com @a.gtld-servers.net.
    feedly.com. 172800 IN NS sid.ns.cloudflare.com.
    feedly.com. 172800 IN NS anna.ns.cloudflare.com.

    $ dig feedly.com @sid.ns.cloudflare.com.
    feedly.com. 300 IN A 141.101.127.247
    feedly.com. 300 IN A 108.162.200.248

    $ dig feedly.com
    feedly.com. 9 IN A 65.19.138.6
    feedly.com. 9 IN A 65.19.138.5
    feedly.com. 77131 IN NS b.dns.gandi.net.
    feedly.com. 77131 IN NS c.dns.gandi.net.
    feedly.com. 77131 IN NS a.dns.gandi.net.

    $ dig feedly.com @a.dns.gandi.net.
    feedly.com. 300 IN A 65.19.138.5
    feedly.com. 300 IN A 65.19.138.6
    feedly.com. 10800 IN NS c.dns.gandi.net.
    feedly.com. 10800 IN NS b.dns.gandi.net.
    feedly.com. 10800 IN NS a.dns.gandi.net.

    The old authoritative server isn’t even returning the new updated NS records.

    Otherwise it will be about 24 hours before everyone is updated.

  87. I flushed my DNS cache and still can’t load Feedly. I’ve tried accessing it in Firefox and Chrome on a Windows 7 laptop. Any suggestions?

    1. I can confirm this as a comcast customer. I restarted my router, and that seemed to fix the problem shortly. Feedly has once again stopped working. I should have exported the opml when i had the chance and head over to oldreader. I’m an idiot for thinking it would stay working.

  88. Still no service in germany, and it does not depend on the IP adress (I tried it from 3 different, no effect from 3 devoces…) :(

  89. Flushed my DNS on my IPAD and computer this morning, but not even the website is working. Hope you guys are still alive and kicking

  90. Thanks a lot for your efforts to bring the service back. Me considering feedly PRO now as I understood during the offline time you are my main news source today. Have a nice day.

  91. I’m sure Feedly worked around the clock to resolve this issue. There are a lot of mean spirited whinging on this forum. It’s a free service for most people so I say wind it in, it’s just a news feed I’m sure you life went on without it for a few hours. Swapping to Googles DNS in the mean time until my ISP refreshes worked for me :-)

  92. I fixed my problems by clicking ‘Force Stop’ & clear data, switching my network, previously I had my problems when connected to Wi-Fi, so I switched to my mobile network, then made sure that it was not showing up in recent apps (it was so I swiped it away), then things ran smoothly again I was able to login with my Google account and got back to reading my feedly news as normal. (By the way my specs are – Android 4.4.3, Nexus 5 – just in case that matters…). Good luck everyone! :)

  93. cleared my DNS cache (more than once), but its still not working on comcast.

    NOW I HAVE TO GO TO EACH WEBSITE INDIVIDUALLY, THIS IS AN OUTRAGE!!!!

    just kidding.

    keep us posted

  94. Still not working in Northern Virginia with Comcast as ISP. Have no idea how to clear cache on iPad. Can anyone explain?

    1. I found out how to clear the cache by googling it. It still didn’t work initially but it’s back up and running now a couple of hours later.

  95. I am a feedly pro user. How does anyone get in touch with you. Still not working on Comcast ISP. What do we need to do? Understand the difficulty you have been through in the last 24 hours but patience is wearing thin. Thank you.

    1. I’m with Virgin media and it wasn’t working. I set my DNS manually on my network card to Google’s DNS (8.8.8.8) and boom off and racing. I’m guessing the change hasn’t propagated completely yet….

  96. I can’t access the website after the attack. I’ve cleared DNS cache etc.
    My ip 213.212.109.68

  97. I refresh the DNS and I Have not change in my computer….only The server at feedly.com takes too long to respond…….pliiiisssssse…..

  98. Well done! I’m glad you were able to resolve this and not succumb to paying these criminals. I’m still unable to log into my feedly account and I’ve cleared my browser cache. Is time still needed for access for all users? Thanks.

  99. It’s back working on my phone but not my win 7 laptop or kindle fire hd. i just don’t get it. I tried clearing the dns cache. I guess I need to clear it and reboot?

  100. I know you all have been under stress, but the app on my phone isn’t working. I deleted the app, reset my network connections, restarted my phone, reinstalled, but it still doesn’t work.

    I can’t add content, and your website isn’t working either.

    Please help!

  101. Another victim of Comcast here. Anyone have any idea how often Comcast refreshes? How long before we can expect access? Feedly working fine on 3g, just can’t access over wifi (Comcast.)

  102. Same. Did manual flush. Waited overnight… the cloud.feedly.com site won’t even come up.

  103. Back from work. Still not working.

    Error message has consistently been as follows:

    Oops! Google Chrome could not connect to feedly.com

    Try reloading: feedly.­com/­index.­html

  104. wtf is going on? when is this going to be back up? maybe it is something with comcast. i did a manual dns flush and reset my modem and still nothing. cant access feedly through wifi on my phone but on the network i have no problem accessing it

  105. wtf is going on? when is this going to be back up? maybe it is something with comcast. i did a manual dns flush and reset my modem and still nothing. cant access feedly through wifi on my phone but on the network i have no problem accessing it

  106. Working with the app, but no luck with Comcast here either, but it’s also not working from work (does not use Comcast) where I’m getting the message: “Feedly is currently unavailable. We’re working to fix the problem. Thanks for your patience!”

  107. If CloudFlare allowed direct IP access then we could just use the new IP until the DNS change replicates to the different servers…

    1. If Cloudflare did that, then there would be no point in using it because it could be taking down by hackers with no abilities at all.

  108. why.why.why did I not export my feed list…….kicking myself right now. Feedly came up briefly this morning and is now down again. At least the site is up – just getting errors on the site.
    One day down is acceptable. But two days?

      1. once again…pay!, pay!, pay! surely nothing better than a supposed hack attack to gain a monthly bill :-)

      2. I use it for all my security and patch management feeds for work. So no, I’m not curing cancer but that doesn’t change the fact that it’s important to me. And while I’m not subscribed with a pro account, many people are.

    1. No, two days down is not acceptable. But it is not Feedly’s fault. It is the nature of the Internet at the moment that only the very very biggest companies can shrug off a DDoS attack. There is an analogy in my lastest blog post, if that helps you understand.

    1. Feedly IS down again. I’m getting an invalid http 552 or something like that on my phone that was working all day. Maybe they found the issue and are working on it? Would be nice if they told us though.

  109. I bet the site is down again. The site is OK for me few hours ago.
    —–
    While I am typing this, it seems it’s back for a second, then goes Error 408 again/cloudflare talked about feedly goes offline.

    1. Yes, they ‘tweeted’ about it in the last hour, but haven’t updated this blog post yet.

      1. this is simply pathetic :D and yes, I was lucky enough to export my feeds and move to competition… and what’s more surprising – their UI doesn’t suck as much and is not that bloated… Feedly was good in the days of reader shutdown, but now there are better options… (and they work :P)

        1. And you have apparently no idea how the internet works.
          And TBH I think it is their competitors who are doing this DDOS attacks, because usual hacker would target someone who actualy has money..
          So, good job in supporting those attackers. :)

          1. yes, yes… everyone that don’t sympatize with ‘poor-poor-feedly’ must be completely ignorant to how internet works and definitely works with TheBigBadCompetition… srsly, feedly fanboys are worse thatn google and apple ones combined =,=
            have you ever heard about free choice and abundance of options when it comes to rss readers? reader monopolized the area but those days are waaay behind us…

      2. It “seems” to be okay now, I’m able to access my feeds and everything (after pressing F5 many times though)

        Maybe they were able to neutralise it quicker this time…?

        1. Ah, nope, spoke too soon, it’s down again.
          Guess I’ll be using Inoreader until it’s back.

          I wish they had an option to export your saved for later items, then I could just migrate them to pocket or something and not have to worry about them.

Comments are closed.