Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I run https://wtfismyip.com on OVH. Egress bandwidth exceeds 8TB a month, which on AWS could cost ~$1000/month, not including the cost of the beefy EC2 instance I would need.

For some bandwidth heavy use cases, OVH makes a lot more sense than AWS. In my case it was $150/month versus $2000/month (bandwidth + EC2).



AWS bandwidth costs are obscene. They compete on price so well on most other fronts, but for some reason they just completely overcharge on bandwidth. It makes using them without a CDN cost prohibitive and drives a lot of use cases away from them.


>for some reason they just completely overcharge on bandwidth

People say this about AWS/GCP/Azure. They are so willing to compete on cost on everything else that it seems odd like this would be true, so I have a different theory:

OVH/etc are all able to massively oversell bandwidth. Most customers do not use it. This is fine, and not anything I think they're doing wrong!

But the "Big 3" are all about automatically scaling resources. With my own applications, I find that if I am scaling up/out, my network traffic is growing at a commensurate rate. Whereas back in the day when I was using SoftLayer or other places where I leased dedicated servers, I was not using anywhere near the included bandwidth.

It's easy to sell 1TB at the price of 100GB if people, on average, are only using 100GB. But then you have to sell 1TB at the price of 1TB if people, on average, are using that 1TB.


It's a distinction without a difference though. If you use 1TB of data on Amazon or on OVH, and one service costs you $x and the other costs you 20% of $x you've got the same service for less money.


Your cheap OVH bandwidth is paid for by overcharging less intensive users. Since Amazon charges exactly what you use, it’s a great deal if you’re below average, and a bad deal if you’re above average.


It's exactly the same model used by fitness clubs worldwide. And Comcast cable Internet, Google Fiber, every DSL provider, even every cellular data provider.

Current true bandwidth costs about $0.50 (50 USD cents) per Mbps, per month, sometimes even less.

Average 1 month usage of a 1Mbps link that has Internet traffic on it is about 175-210GBytes.

Literally less than 1/3rd of 1 penny, per GB of data.


So what about the cdns? They're all overselling too? Otherwise they'd obviously have to charge amazon prices?

From eg Hetzner and other less affordable hosts (and from second hand comments here on hn about bandwidth is where amazon has ridiculously high margins) - I'll venture that the "cloud" providers get away with it because on average "actual" bandwidth correlate with "actual" usage: and hence correlate with actual users that pay or are worth paying "for".

I get 30tb/month bw with my 50/month hetzner dedicated server (along with 1gps uplink). And I can actually use that.

I can't even see backing stuff up to a typical "cloud" provider, because testing restore of a single tb of data/month would be too expensive.


Hetzner is charging € 1.00 per TB after you used all the included bandwidth and they are making a profit with that. [0]

[0] https://wiki.hetzner.de/index.php/Traffic/en#Option_1


But that's how the internet works in general. An ISP doesn't have a 500,000 * 100 MBit/s connection in a city to enable simultaneous downloads from all customers at once.


I'm not claiming otherwise - just that my theory (and I have no actual evidence beyond the anecdotal) is that the big cloud providers aren't "overcharging" for bandwidth, just that they cannot offer discounts from oversubscription that others may be able to.


What's the best cdn right now? Cloudfront is just expensive as hell.


Alright, I'll go through the list.

* Cloudfront would be great for the North America and Europe they only charged for the bandwidth (those prices are reasonable), but charging for the HTTP request (and the huge premium on HTTPS). The bandwidth prices outside of those areas are absolutely awful though.

* Akamai makes Cloudfront look cheap.

* Cloudflare has ethical concerns that keep me away from them.

* Fastly is susceptible to DDoS's against the origin in some interesting ways.

* Limelight was great but then Goldman Sachs bought and destroyed them.

* EdgeCast was by far my favorite, but then they got bought by Verizon. Assuming Verizon hasn't fucked it up then they're definitely worth checking out, as their reporting tools were amazing and their performance was literally the best (my information is a few years out of date though on this one).

* If you want to serve people in China you will need to pay a China based (government owned) CDN or your traffic will get blocked for no reason once you get large enough. Conveniently enough one of their sales people will reach out to you about a day or two before the block goes into effect.

* MaxCDN isn't bad, but I haven't seen their higher tier prices (above 25TB) so I can't comment on that. Their South America and Asia coverage is pretty bad though- nothing in India, only one datacenter in Brazil for all of South America (with another being built, but also in Brazil), no Africa, no Middle East. If you're starting english only this isn't the worst, but eventually you'll need to go to a multiple CDN solution for broad coverage.


> Akamai makes Cloudfront look cheap.

Completely. I can't believe people actually pay Akamai.

> Fastly is susceptible to DDoS's against the origin in some interesting ways.

Citation requested.

> Cloudflare has ethical concerns that keep me away from them.

Not to mention their product itself isn't too good. I always get forced to Captcha on some ridiculous cloud flare "protected" site, despite not using a VPN or anything out of the ordinary.


Can you elaborate on Fastly exposing the origin servers to DDoS? Is there a link to learn more about that?


The Fastly "Purge API" by default does not require authentication. This is why you can clear the cache on Github pages directly [1].

From there you can force requests to hit the origin server by first purging the data from the cache and then requesting it.

Unrelated to DDoS I've also seen issues with the Fastly routing- it doesn't always pick the greatest end node to have a client connect to.

[1] https://stackoverflow.com/questions/26898052/how-to-force-im...


For what it’s worth I think with custom VCL you could add your own shared key / password authentication to fix this by returning a synthetic 403 if the shared key isn’t present in a header.


What are the ethical concerns with Cloudflare?


* Cloudflare has a history of promising to open source projects only to not do so. For example, they built a system to shorten SSL certificate chains that they said they would open source but then didn't for over a year. When they finally did they admitted it was due to pressure from people on this site.

* Cloudflare got some initial fame by keeping Lulzsec up during a massive DDoS. That by itself wasn't bad, but afterwards they embraced a "bulletproof hosting" mindset that involved acting as a shield for malicious activity (specifically, people were abusing their network to host malware and drive by exploits). If you reported an account for pushing malware they'd block the single instance of malware (ie, example.com/malware/djfksdjf.jar) even if the original server was configured to serve it under any random name (example.com/malware/.jar). For awhile they were a huge source of infection for users.

Their CEO directly lied and accused a security company I worked for at the time of blocking their traffic to "suppress free speech", when what we were really trying to block were active drive by exploits. We sent a ton of evidence for this to them in advance- including PCAP files showing the exact network connection required. Instead of dealing with the malware they were hosting they started a PR campaign.

* After I criticized them before on this site several of their security team followed me on twitter for some reason. That's how I learned that they have a lot of "alt-right" type people on their team (or at least did at that time).


Some people are against Cloudflare's vast internet dominance with their "free" CDN which proxies all content, but proxying content of a zone is no different from any of the other CDNs storing your content for you.


When did business success become such a problem? The basic CDN model is the same with all CDNs that are pull-first reverse proxies, but they clearly offer much better features for less cost which is why so many use them. There's nothing stopping other companies from doing the same but most are still nothing more than some nginx servers running in a few colos.


I came to pretty much the same conclusion a few years ago, Edgecast was my favourite, until Verizon bought them. I dont know if there are any changes, but I have yet to see big acquisition that dont mess up the company.

Limelight was my favourite before I discovered Edgecast, the only problem is Limelight no longer does business with small companies and no more reseller / PAYG model. Which means basically most of us cant use them. But I didn't know Goldman Sachs bought them? And destroy them?


> * Akamai makes Cloudfront look cheap.

Their list pricing does but they’ve matched or beat AWS in the past. This is by far my least favorite part of the enterprise sales model but if you can haggle it pays off.


I'm pretty sure Steam gets a good price with Akamai. At least I hope it's the case, since all their download goes though there.


Cloudflare. Cheap, fast and powerful. They seem to be doing everything right, as evidenced by all the growth and great features in a single package. It's rather surprising the other vendors aren't doing anything to even catch up.

If you need more control, than Fastly is very technical, although the recently launched Cloudflare workers give you much more programmability by running javascript.

For more traditional options, look at the quiet upstart Stackpath which is built upon the old MaxCDN network, and also CDN77 which is great value.


In terms of price I still haven't found anything better than CloudFlare. If performance is the goal all the testing I've done for work shows Akamai as the leader, though not by a lot. Fastly is making strides to compete with both, as well.


Heard good things about cloudflare


I would have said MaxCDN, but they recently got "acquired" by StackPath, so who knows what direction that's going.


They were acquired in July of 2016, so about 20 months ago. It seems to be more of a merger than an acquisition, and both brands have stayed separate but share expertise and data centers. It honestly seemed like a good way for both of them to expand their footprint while also reducing overhead.


Why the quote marks around acquired?


StackPath came out of nowhere. As far as I could tell at the time, they didn't have any public products. And all of a sudden they bought MaxCDN. I could never tell if it was actually an acquisition or more of a merger or something else?


Stackpath is from the founder of Softlayer. Raised private equity money and acquired several companies including MaxCDN to start a new service. The subbrands still exist but are slowly migrating and will refer you to the stackpath products.

https://techcrunch.com/2016/07/25/security-as-a-service-star...


Hi there. I'm DevRel at StackPath. We have an will continue to support the businesses that we have acquired, but the long term plan is to roll everything into a single platform under the StackPath umbrella. Our customers won't see much if any difference in service other then better performance and more features/products when we migrate them.



CDN77 is pretty awesome


We run a chunk on OVH too. It is over 100X cheaper than AWS.

We have had some issues though. OVH had two major outages in the last two years. Our strategy has been to build a more robust distributed architecture and use these issues to be "antifragile," which improves our product while as still saving money.

Over time I expect OVH to become more solid as they mature.


You should get some boxes from other EU vendors as well. Don't put all your eggs in the same basket. Unless your work load is very unpredictable I don't see why people use AWS, "serverless" and platform lockin.


I thought people go with serverless so they don't have to deal with ops?


It seens OVH was launched in 1999. That’s quite a long time for them to mature :)


From what I remember one of those outages wasn't connected with network hardware/their architecture but with their new electricity feed system, after which they posted post mortem and wrote about steps that they will take so this will not happen again.

Every major platform had outages in the last 2 years - Amazon many times, google cloud also.


It’s plain text, how do you get to 8TB per month! Congrats on your site!


2M hits per hour, 1M uniques. It's crazy!


I'm guessing it's mostly automated. Are you being used by a popular software package that you know of?


I wouldn't be surprised if the plain-text ip was used somewhere. Anecdotal evidence but I myself use that for most of my scripts to get my own ip.

A github search shows over 1000 code results.


Are you able to at least break even in terms of costs?


Why no gzip/brotli the content? Does it not help because html is small?


Care to explain what the business model is here?


https://wtfismyip.com/why

There is literally an entire page explaining that there is no business model other than a donation button. There are shockingly people than run websites either as a hobby or to provide a useful service for others with no expectation of making money from it.


Browsed the entire website but can't figure out how to donate. There's no link anywhere, even on the Twitter page.


When you look at https://webcache.googleusercontent.com/search?q=cache:Z2wGsh... (Google Webcache from 25th March), there was a donation button on the "Why" page, but it seems to have been removed now. Not sure why though...


$150/month seems like a lot for a hobby like this.


just curious - any particular reason to not enable gzip compression for responses?


    $ curl -s https://wtfismyip.com/text|wc -c
      16
    $ curl -s https://wtfismyip.com/text|gzip -|wc -c
      36


nginx for example can enable gzip by content-type or depending on gzip_min_length, it's very flexible. All in all, just curious if there's a deeper reason.


So..8tb vs 4tb? If paying for bw.. Seems reasonable?


The gzip version is actually the larger of the two.


Indeed. Missed it because of required side-scrolling on mobile.


I just wanted to thank you. I've used your site so many times in personal scripts.


Thank you for your service, sir!


I wonder if nearlyfreespeech would be even cheaper for you.


I've used NFS for more than 10 years. Past a certain point, it does not scale. It's really good for smaller sites and not much else I feel.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: