How much network latency is between you and the load balancer, and between the load balancer and the web servers? Normally, the initial TLS connection should be around 4 * the network latency, and subsequent requests should be very close to the network latency.
I'm having issues reaching IP addresses unrelated to Cloudflare. Based on some traceroutes, it seems AS174 (Cogent) and AS3356 (Level 3) are experiencing major outages.
Is there any one place that would be a good first place to go to check on outages like this?
It would be really cool and useful to have an "public Internet health monitoring center"... this could be a foundation that gets some financing from industry that maintains a global internet health monitoring infrastructure and a central site at which all the major players announce outages. It would be pretty cheap and have a high return on investment for everybody involved.
Indeed, if we're to have a public Internet health meter, it must be distributed and hosted/served from "outside" somehow, to be resilient to all or parts of the network being down.
This is an excellent idea and simple but moderately expensive for anyone to set up.
Just have a site fetch resources from every single hosting provider everywhere. A 1x1 image would be enough, but 1K/100K/1M sized files might also be useful (they could also be crafted images)
The first step would be making the HTML page itself redundant. Strict round robin DNS might work well for that.
But yeah, moderately expensive - and... thinking about it... it'll honestly come in handy once every ten years? :/
Reddit, HN, etc. are inaccessible to me over my Spectrum fiber connection, but working on AT&T 4G. It’s not DNS, so a tier 1 ISP routing issue seems to be the most likely cause.
> Fastly is observing increased errors and latency across multiple regions due to a common IP transit provider experiencing a widespread event. Fastly is actively working on re-routing traffic in affected regions.
ip-api.com was also affected by this.
After our first alert at 10:49 (cert expired at 10:48:38) and a minute of being puzzled as to why our certificate expired, we realized the root we bundled is the issue. We finished updating our primary API servers at 10:55.
This isn't on the level of some other providers, they'll still null route you if you go over an unspecified amount of traffic. IIRC they use Juniper and Corero.
This is the reply I got from their support, just a few days ago:
>In short, our DDoS protection works by filtering out DoS-like traffic and is applied via the Linode network, so all Linodes are automatically protected. If your server were to be on the receiving end of a larger attack that impacts the Linode's host, we would need to prevent your server from receiving traffic until the attack ends. If you're concerned that you might be the target of a large DoS attack, there are a number of third-party DDoS mitigation services that you can use alongside your Linode.
>We aren't able to provide specific numbers since effects can vary depending on the attack. If you wanted to be sure your Linode is protected, we would recommend utilizing a third-party DDoS protection service overtop of your Linode's included protection. You also have the option of waiting to apply third-party protection until a null route is found to be necessary.
That's not protection, that's literally the opposite of protection lol. If you get attacked they take your service out the back and shoot it in the head.
Edit: To clarify, filter = protection. Preventing all traffic is not. Both were stated in the description above so they should be clear which one it is.
Heh, that reminds me of my first bank account. They told me I had something called "overdraft protection", which I stupidly assumed would protect me from overdrafting my account by declining transactions.
Then I forgot to deposit a check at one point and overdrafted my account. I assumed things were fine because none of my transactions were getting declined. Instead I was being charged an extra $15 fee on every transaction, so that $0.75 stick of gum? $15.75, etc. This went on for about three weeks before I got my statement and talked to my bank.
They informed me that in fact the protection was from my transactions from being declined, at the paltry expense of $15 per transaction.
I'd love a service that puts me in touch (for a fee) with an engineer/someone above level 1-2 support from Google/Amazon/huge corps
When dealing with companies that are above a certain size, it takes days or weeks to get to someone that can fix issues, but if you're lucky and you have a friend that knows someone who works there he can expedite your ticket. This service would work like that, a friend that puts you in touch with people hired at big corps.
I'm aware this service would have a number of potential issues, such as how to get employees on the platform without annoying their employers, or how to prevent abuse/bribes, but if somebody find a way..
That seems like a nice idea to develop as a side hustle. Not sure if it would be so much of an issue, as websites like https://www.teamblind.com exist. I'm more worried about the user acquisition tbh
As a small business owner, this terrifies me. Since the TTL for NS records is 48 hours, a domain takeover like this could easily bankrupt a lot of SaaS companies.
What options are there to prevent this? Would a registrar such as MarkMonitor provide at least some notice or protection?
I launched a free API 4 years ago, and in January 2015 I introduced paid plans, which offered several benefits over the free service.
I have done no actual marketing, but the website ranks somewhat okay in search queries. It keeps growing every month, and recently passed $10k/month in profits. This is from pretty much word of mouth via stackoverflow or from google searches.
From this experience I can say that if your service is competitively priced, good, and needed by a somewhat large audience, even with no marketing you can turn a profit really fast.
What percentage of your users are monthly active users? What percentage (of the monthly active users) ended up paying? How much do you charge per month? How did you figure out what to charge?
Since there is no authentication done for the free service I can't really track users, but just to give you some numbers, the free API serves over 10 billion HTTP requests/month. An educated guess would be 5% of users ended up paying.
Initially, I just wanted to make the service pay for itself (hosting, dev., data mining costs) and allow me to dedicate some time to provide support to users, so I asked myself, as a dev/company, how much would I pay for this service. I started with €80/year, and doubled the prices a few months later, to €160 (€13/month). The price increase seemed like a good business move, and surprisingly, the number of sales improved as a result.
If you were to use azure table storage and use shared access tokens for authorizing access to query results, you could run the whole thing from any individual data center for under $500/month.
Assumption is that every IP (all 4.3 billion) had about 100 bytes of data stored about them. That is ~430gb of storage runs you $30/month. The 10 billion queries to that storage costs about $360/month. ($.0036 per 100,000 transactions)
Looks like the service is an IP lookup API. The sort of thing that many sites will hit once for each visitor. Most SaaS sites will never get anywhere near this, but things that do per-user or per-pageview queries (e.g. usage analytics) can get to billions of events per month with, say, 10k customers doing 1M hits each.
That's roughly 4k qps. Assuming it's a lightweight API a single server isn't out of the question. Nginx will handle 20k qps until the end of time on a tiny box serving static content.
