I have been managing email servers for over 10 years, and it has gotten to the point that I feel like blocking some of the most common ISPs. Seriously.
I am following all the best practises, hell, I even advocate them. It is just that these days it seems not to matter if you have SPF, Sender ID, DomainKey and DKIM, PTR, proper MX and even a normal to good IP reputation. There is still no guarantee what you will be able to reach the inbox of the likes of Gmail, Yahoo, Hotmail, etc.
I have been filling out huge forms for each and every major ISP for the past year because one or two users mark a newsletter as SPAM.
Conclusion: There is no common standard because every major ISP can set their own standards. This will eventually force everyone to use the same services worldwide.
I'm one of them. Those newsletters are spam. I would never sign up for a newsletter and somehow I'm getting those too. If my intent was not to get the newsletter, it's unsolicited mail by definition, i.e. spam.
Stop spamming me and I'll stop flagging you. Period.
How not to be flagged as spam:
- There should be a checkbox clearly visible and it shouldn't be pre-checked.
- Your "kind" product reminders are obnoxious too and I'll flag them as spam as well. Did I ask you to remind me of your product? Nope. Unsolicited then.
- If you ToS say I agree to receive mail, guess what? I don't agree, I just want to try your product. I'll flag you in a breeze.
- Social reminders like Twitter's trending around me or people I might know? SPAM! I don't care if I can disable these, I didn't enable them.
- You want to offer me discounts but I didn't ask for them? Flagged!
- I submitted a paper to a conference and it got published? Dozens of "calls for papers" in my inbox. Flagged, flagged, flagged, flagged!
- Calling it a newsletter or adding a tiny "unsubscribe" link won't hide the fact that it's still spam. I didn't click subscribe, I shouldn't have to unsubscribe.
--
EDIT: Woah, this seems controversial. Lots of up- and down-votes.
Dear product owners, downvoting me here won't change the fact that me (and your fellow users) will still flag the shit out of your unsolicited mail. I guess it pays if you keep doing it, but you should direct your energy far from that downvote button and closer to "ways not to annoy my users".
You are getting down voted (rightly so IMO) because you posted an irrelevant personal rant in reply to someone who had the temerity to describe a challenge in the modern email system.
You have no idea if the parent sends out a single unsolicited email. All you saw was "newsletter" and you flew off the handle.
Not that you're alone; this happens in every single HN thread about email marketing, and is in my opinion one of the worst killers of signal to noise ratio on HN. We love to talk about "growth hacking" but every mention of email marketing is a race for HN comments denouncing all email as spam.
The fact of the matter is that even the most carefully run, innocuous, double opt-in email newsletter is occasionally reported as spam. I think it is because paranoid tech folks have spent years telling people to never use an unsubscribe link because it just makes the spam worse.
But if it's an email you actually signed up for, the unsubscribe link is the correct and appropriate way to unsubscribe.
It is users such as yourself that make spam filters have the kinds of silly false positives being discussed in this thread: as a user (not as a product owner; I am a product owner, but I don't send e-mail, so I have no "skin in the game": so, you can claim all you want that the people downvoting you are all product owners, but some of us really do just feel you are abusing a shared system to carry on some kind of personal vendetta) it really sucks that I can't trust spam filters to not filter valid e-mail out of my inbox because people like you are so trigger-happy (and in some cases, simply vindictive) with the spam button... it is my sincere hope that services like Gmail are at least sometimes smart enough to realize "this person is just being annoying" and makes your spam votes meaningless :/.
Hahaha! YES! I have a personal vendetta against unsolicited email.
If you send me an email that I am not expecting, then I will flag it as "unsolicited".
I do not care if you provide other mechanisms for handling unsolicited email. I do not care if you used a dark pattern to technically ask me to solicit the email without my conscious knowledge.
The way you call it vindictive is empowering and enlightening.
It's your way of calling it unfair, of attacking the person as immoral and undermining the point of their behavior. You're trivializing them as petty. Instead of saying "I understand why someone might be unhappy at me sending them email they do not want" you say "it is vindictive and petty for someone to treat unwanted advertising from for-profit businesses this way". What a sick joke.
So you know what? GOOD! If "vindictive" means "control over your inbox" then I AM VINDICTIVE. I will PUNISH anyone who gets into my inbox without my approval. Email is WAR and I am fighting for Inbox Zero. Send those emails my way lightly ... tread carefully with that send button, because you might just find your messages are unwanted and end up clearly marked as unwanted.
Insult us users all you want, trivialize us all you want, attack our behavior all you want: I DO NOT WANT UNSOLICITED EMAIL and will HAPPILY and "vindictively" (lol) mark unwanted email as "unsolicited".
I use the term "vindictive" because the people using their spam filter in this fashion feel that they are not only training a dataset, but punishing the sender: this attitude leads to behaviors that lead to feature creep on the usage of flagging things as "spam", and if you read threads related to spam you find people using the filter to mark all kinds of things they don't like, including business models they take issue with that really have nothing to do with email at all. I thereby simply think your entire rant here is nothing more than self-rationalization of behavior you know is overstepping a boundary: email is not "war", the attitude you are describing is objectively "vindictive", and I frankly find it rather disappointing that you just don't get this.
And again: I don't send email from my service; I don't even send receipts... so most i your rant is fundamentally mistargetted. I am a user who doesn't appreciate people's emotional responses to something that isn't really fixable anyway making email fundamentally more complicated as a protocol, breaking use cases like shared mailing lists (see the recent issues with Yahoo DMARC on the IETF list), and mis-training spam filters. If you ask me about real-life "actual" criminals I would frankly have similar responses against people who prefer vigilante, vindictive punishment for behaviors they dislike :/.
None of those are false positives. Spam is unsolicited mail. Did I solicit the mail?
- Yes: not spam.
- No: spam.
And, since I never solicit promotional email to my personal address, my method is 100% accurate.
In my opinion the companies sending unsolicited promotional email are the ones gaming the system taking advantage of the "well this might be unsolicited but it's not v14gr/\" grey area.
By this definition John sending an unexpected party invitation to his friend Dave is spam.
edit: I guess spam is by definition commercial email. Still, I think it's possible to have a business relationship with a company where it's acceptable for them to send an occasional email that you didn't specifically request.
Business relationships are different from personal relationships, and "solicit," in this case, does not mean literally request the email. It is perfectly reasonably to describe unexpected email from a friend as solicited and unexpected product email from a business as unsolicited (even if you've had contact with them previously).
It is unlikely that the business is your friend ;-).
Personally, there are a select few brands from whom I enjoy reading unsolicited emails.
They have proven themselves to me and I like them on an irrationally human level.
What bothers me, however, is when the other 99.9% of the brands I interact with automatically assume that they are the 0.1%.
Yes, it's possible and acceptable to have that kind of relationship between a client/business, but the problem becomes when a brand believes that they can control that relationship, or define it themselves, or simply assume that it exists.
That's only some definitions. There are more hardcore people who'd view anything they didn't explicitly request as 'spam', just some of it originating from people they know vs companies they know.
What about solicited mail? Sure, unsolicited newsletters are annoying, but you're making it sound like NOBODY, EVER subscribes to newsletters. Sometimes, someone (who is not you) subscribes to a newsletter, and then after a period of inactivity s/he receives a newsletter but does not immediately recognize the title, and hits the Spam button out of habit. I think this is what _asciiker_ is talking about. What about cases that this?
A week or two ago I sent hundreds of messages to my customers, warning about Heartbleed and noting what measures we'd taken against it.
That mail was unsolicited. Not overtly solicited, anyway. Sure, it falls under the "we may contact you from time to time about yadda yadda" but people only know that if they actually read our Terms & Conditions. A normal person with an important mailbox and a convenient spam-button doesn't always have time for such careful consideration and nuance.
Heck, to make sure the mail reached the recipients, and to avoid getting my domain flagged (and this is for a very legitimate message!) I used a 3rd-party mail service. It sucks that I even need to do that just to communicate with my customers.
I suspect one or a few of my customers may have marked that message as spam, too. Because, whatever it was (didn't read, just glanced at subject) they didn't expect it so it must be spam.
The good news is there are plenty of ways to tune for false positives in the machine learning algorithms that underlie spam filters. I'd be surprised if email providers don't consider a flagger's flagging frequency when classifying.
As the OP said, this is just reality: people have different criteria for reporting something as spam. I could even imagine malicious users flooding false positives using bots in efforts to break spam filtering. The state of spam is so much better than it was 10-15 years ago, but it's not yet a completely solved problem.
In short, this person isn't breaking spam filtering all on his/her own--his/her use of flagging may even be arguably justified (if more liberal than average.) He/she just represents part of the problem space.
It is a false positive if someone using SES to send a password reset email is marked as spam, and you are not a victim if you sign up for something that says it will send you email that you knew you didn't want when you signed up for it: that meets neither the legal nor classic definition of spam, and is mail I actually like receiving.
I always click the "unsubscribe" link, first. If I get anything else after that, then I mark it as spam. (I don't buy into the 30-business-days bullshit, it's 2014.)
I also have catch-all addresses at my domain, so I register for things as SERVICE@mydomain, so I know exactly where spam is coming from.
Except I've been doing this for years, and 90% of garbage still just comes to my plain ol' gmail address that I've been using for over a decade, which suggests that businesses finally have wised up to the fact that sending garbage to everyone who puts their email address into a form is a bad, bad idea.
> I don't buy into the 30-business-days bullshit, it's 2014
I just went through that a few weeks ago. I ordered something and the merchant decided to start spamming me with product info on a daily basis. I thought I unchecked the "spam me" button, but I guess I was wrong.
So I unsubscribed. And got two more emails.
They were required by law to stop sending me emails within 10 business days. They carefully waited exactly 10 business days. In the mean time, they sent me two emails a day in case I might change my mind.
After two days I started marking the messages as spam. Luckily they were kind enough to provide me with an opt-out confirmation email. I was ready to sue them when they stopped right at the boundary.
Just a pure jack-ass move by the company. I'll never buy from them again.
Tanga does this. It's very easy to unsubscribe, but that's beside the point. I bought one thing from there and started getting "daily deal" mails, very annoying. I try the unsubscribe, but then if mail is still flowing, I will mark as spam, but I am very conservative with that Spam button. I manage mail for a large ISP and too know the horrors of customers complaining that our IP space is blacklisted.
Can someone help me understand why naming and shaming companies for what is in the customer's opinion poor behaviour deserves down-votes? No sarcasm, honest question.
Were the down-votes for something else?
I honestly think some naming and shaming would be a good thing in situations like this. Especially in this day and age when complaining on twitter is more and more likely to get a response from the company in question.
What company was this? Name and shame them. Ticketmaster did something similar for me. They subscribed me to three or four lists, each of which I had to individually unsubscribe from. I will never buy from Ticketmaster again.
ProFlowers did this to me. I ordered a condolence package for my manager who had a passing in the family. I immediately started receiving emails from ProFlowers and several of its brands.
It was spam, plain and simple. I purchased flowers because someone passed away and suddenly I was on the list of people who are just on the verge of buying chocolate covered strawberries, but just waiting for the right discount.
Thinking about it now, do chocolate covered strawberries and price conscious consumers even go hand in hand?
> I always click the "unsubscribe" link, first. If I get anything else after that, then I mark it as spam.
Then you're much more resilient than me :P
I used to do that but I thought, since they want to deliver their message, I'll deliver mine: "I don't want your unsolicited mail and I'll flag you without hesitation".
I click "unsubscribe" because I actually want to stop receiving the mails. Granted there are two classes of unsubscribe link, 1) I got on your list somehow through my own action and now I click here to get back out, and 2) you bought a big list of e-mails and removing my name from your list, while legitimate, is not going to stop them from reselling my name forever to V1agr4 sellers and c1al1s remailers, and I'll continue to get these spammy e-mails forever no matter what I do as long as I get this address.
The difference is more important to me because I get catch-all mails for addresses on a domain that are no longer good (former employees) and they MAY have actually signed up to some of these things on purpose. This doesn't mean I won't both unsubscribe and also mark as spam completely useless e-mails from companies I don't care for. But I do notice that the RIDICULOUS volume of spam I get actually goes down sometimes when I do spend some time unsubscribing from what I find seems to be from legitimate companies, the frequently repeating contacts in the spam buckets. YMMV.
Would be interesting to see if you would see a difference if you reported spam through spamcop.net.
It's a pretty nifty interface that will automatically look up all relevant abuse contact emails for the ip ranges and let you file and register complaints with a single click.
> I also have catch-all addresses at my domain, so I register for things as SERVICE@mydomain
This is a great technique. I was receiving Nigerian-style spam to my santander@ e-mail address, but Santander ( a big Eurobank ) denied that they had leaked the address. They blamed me, stating that I must have entered that into some web form somewhere.
So I changed my address with them to santander_2014-03-20@ and guess what.. .within a couple of weeks spam came to that one too.
No subsequent response from them as to how these are being leaked / compromised. None of my other e-mail addresses of the form companyname@ are being spammed.
I'm gradually closing my Santander accounts, I just don't trust their IT systems and processes.
In the US, the CAN-SPAM Act requires opt-outs to take effect within 10 days, so you see a lot of "you will be removed within 10 days" messages written by lawyers even when the opt-out is instant.
And to be fair, if you're horribly unlucky and an email was already queued to send to you when you hit the "unsubscribe" link, you are going to receive that message, even though you'd unsubscribed -- given some of the retry mechanisms in certain mail servers, I can certainly see the logic in being conservative and giving a ten-day window.
> If you ToS say I agree to receive mail, guess what? I don't agree, I just want to try your product. I'll flag you in a breeze.
No. You literally did agree.
That you don't like that it's a package deal, and you're exchanging getting their sales email for trying their product just makes you an asshole, but it doesn't make their message spam.
You established a business relationship with them, in which they told you ahead of time they'd do follow-up contact, and then you have the gall to complain about it in a way that damages the reputation of people who relay the messages you agreed to receive.
The way I see it, when you send me an email, there are two perspectives at play. Yours as the sender, and mine, the receiver.
You do not see the email as spam, according to you I signed up for this explicitly when agreeing to your ToS (for example), it was requested and is not spam.
I do see the email as spam. I did not make the conscious decision to receive email from you about your products or anything else: to me, I clicked a box that said I read and agree to your ToS in order to get your product.
Terms Of Service themselves are a discussion for another day, my point is we both are fully aware that nearly nobody reads the ToS and that you're leveraging that to send me emails that we both know I do not want (if I wanted them you wouldn't be resorting to these tactics, IMO the fact that you are is a tacit admission of guilt).
Here's the crucial part. When it arrives in my inbox, the choice is mine about whether or not it's spam. You have no say in the matter, ToS or not. This is a matter of perspective, and my perspective is that the email is spam.
Argue it if you want to, but understand what you're arguing against is perspective and that I don't share yours.
> I do see the email as spam. I did not make the conscious decision to receive email from you about your products or anything else: to me, I clicked a box that said I read and agree to your ToS in order to get your product.
Yes, I get that you don't want to be responsible for what you agree to with other people.
However, you punish the middle man - the mail carrier - because you regret your own decisions you admit were made in ignorance.
> Argue it if you want to, but understand what you're arguing against is perspective and that I don't share yours.
I think you're simply unreasonable: you're whining about getting a sales message from someone you proactively established a business relationship with and turned your contact information over to, and that they disclosed your information would be used that way.
In no way was that message unsolicited. You just wish you could get the product without even having to pay the meager amount of receiving sales literature in return.
I think that makes you an asshole, because you're punishing people for conducting reasonable business rather than taking some ownership of your behavior and simply unsubscribing.
> down-voted you
This bolsters my view that you're largely just an asshole: you're trying to punish my internet points or hide my comment because you don't agree with me, while you yourself admit that there's nothing in my comment but a difference of opinion.
So, really, I wish mail carriers would just ignore people like you when they submit spam reports - since you admit you're not using it how it's intended, but to flag solicited emails you agreed to receive, which damages the reputation of the mail relay, even though they're not doing anything wrong. They're just delivering requested mail.
It's like you trying to get the phone company that a second company uses to call you in trouble because they had the audacity to connect a phone call after you gave your number to that second company and told them it was okay to call you at the end of your free trial.
I really wish someone could present a argument for your view that didn't just make the person sound wildly entitled and assholish.
Evidently we don't see eye to eye on this because it seems neither of us has an argument capable of persuading the other.
It sounds to me like neither of us is willing to discontinue what the other side sees as deceitful behaviour.
On your side you assert that my agreeing to ToS is sufficient to start sending me "solicited" email, and on my side I assert that the fact that you have to hide the opt-in inside the ToS is evidence that your emails are spam.
I do concede that I could have carried on with our conversation without down-voting you, that wasn't necessary to make my point.
Other comments in the thread point out that a "Unwanted, but not spam" button could be useful, I think that's a great idea but wonder if it could be taken one step further. A spam filter that monitors who reports what email as spam and assigns them a rating based on what they report as spam.
Eg. I would have a high rating because anything I did not explicitly request is spam. You may have a low rating because you are much more lenient with your use of the Is-Spam button. This could then allow users of that service to set which rating to use when filtering spam.
Given the widely varying differences of opinion on this topic I can't help but wonder if the other commenters are correct about this being a UX issue instead of a technical one.
I just opened Gmail to look at the options for dealing with unwanted email:
It requires 2 clicks to report something as spam and 4 clicks to create a filter which automatically deletes messages from a particular sender (or routes them in a way of your choosing; can also be used to selectively stop messages, eg, receiving bills without receiving ads; option is in the drop down menu).
I can't help but feel like you're saying you should be allowed to file harassment reports against the people standing behind sample booths, since you didn't explicitly ask them to talk to you when you grabbed a sample from the table, and well, harassment reports are just so much easier to file than asking them not to talk to you! (Okay, not actually true, but would be the analogous thing.)
I suppose there isn't a lot more to say, but I just want to ask this point blank one time to be sure I really understand what you're trying to say (even if I don't agree): are you really saying that it's entirely unexpected that a company which you're getting a sample or service from sends you a sales message and that you think the best response is to report them for harassment (in the process, attacking the reputation of the middle man in the communication for enabling harassment) rather than just informing them directly that you don't want further messages?
Edit: Corrected click count to account for menu hiding; tidied up comment a bit.
To answer your question, it depends entirely on how that company presents the opt-in, or rather, does not present the opt-in.
If there is a check-box that's pre-checked and all I have to do is un-check that box as I'm signing up to opt-out, I respect the company for being up-front about the choice and will un-check the box. On the other hand, if they do anything I consider "shifty" like trying to hide the opt-in anywhere (eg in ToS), then the answer to your question is yes. I would not expect those emails so in my opinion they are unsolicited, at best.
And I bet you read all 20k words of every tos you agree to?
Just because someone shoved a statement they get to email me somewhere in the small novel I'm expected to read -- and your argument is disingenuous because you know damn well nobody reads those things -- doesn't mean I actually, you know, agreed.
A recent example: those dbags at ziprecruiter decided that, since I applied to a job at a single company that used them, they should now email me daily lists of jobs I may like. Was that buried in a tos somewhere? Probably. By any reasonable usage of the phrase, though, I in no sense opted in. And it's not my responsibility to find their unsubscribe link and figure out what username/password I used. spam
Downvotes are not for disagreement, disagreeing with somebody is a natural part of that human process that we call conversation. Disagreeing is good. This is not reddit where people upvote what they like and downvote what they don't. Use downvotes for flagging inappropriate comments that do not contribute to improving the quality of the site, not as a personal argumentative weapon.
> Seriously? I should've added that sentence to my post for dramatic effect too.
The issues is that spam filters learn. I fucking hate spammers, too, and like you I consider unsolicited emails to be spam. But when you flag mostly-legitimate, but still unwanted, emails as spam, gmail learns the wrong thing. Suddenly really legitimate emails get flagged as spam. My domain renewal emails from my registrar recently started getting filtered to my spam box, and I suspect it's due to users flagging any unwanted email as spam.
In my opinion, the best way to deal with these unsolicited emails is to use the unsubscribe link, delete the email, and then swear at them on Twitter or find their CEO's email and send them goatse or something. Fuck 'em. That way you get your revenge, and you don't muck up the spam filter for everyone.
"In my opinion, the best way to deal with these unsolicited emails is to use the unsubscribe link, delete the email, and then swear at them on Twitter or find their CEO's email and send them goatse or something. Fuck 'em. That way you get your revenge, and you don't muck up the spam filter for everyone."
This is totally absurd. My choices are "click one button" or "do a whole lot of bullshit that sounds like a lot of annoying work".
I hope you can understand why "one button" is taken more often than "raise hell on and offline".
The real solution is to separate spam and unwanted emails. Gmail and services need to add a separate button for non-spam unwanted emails, so they can categorize and learn about usage habits effectively.
But it's horrific, insanely bad UX design to create a flawed system then blame the users for using it naturally. I'm sorry but the user is not wrong, the system is wrong. The solution isn't "user training", it's "system redesign".
If the system were designed correctly, users would naturally gravitate to the correct option without training. That's good UX. Until then, it's perfectly acceptable to use whatever tools are available to achieve the desired outcome. That's software for you.
You can (from most providers) block emails from a sender or domain as easily as flagging it as spam; however, flagging it as spam has consequences for the company that relays the messages, not the one that sends them.
You're punishing the phone company because a company you had a business relationship with can't magically read your mind that you don't want further calls, after you agreed to a couple sales calls in exchange for a product demo.
Actually, the part where you invent a bad strawman solely for the purpose of labeling me an asshole makes you an asshole.
You couldn't even quote me. You couldn't even use my words. You just invented a pathetic little fantasy and then attacked ME directly based on your fantasy world.
You're describing a situation where you agreed to receive email, changed your mind, and then accused the sender of spamming you for not magically adjusting his behavior to your unstated whims. This is almost the definition of unreasonable behavior.
If you genuinely did not give permission to contact you, sure, go ahead and flag as spam. If you regret giving permission (maybe because you didn't really want to give it, maybe because the communication you're getting isn't what you'd hoped, whatever), the sensible thing to do is just unsubscribe.
Your comment is insane! "If your ToS say I agree to receive mail, guess what? I don't agree, I just want to try your product. I'll flag you in a breeze."
So you don't agree to the terms you just read before giving out your email address. That's insane! It's like smiling and saying to a guy, "Here's my number", and not saying: "But if you call me, for any reason, I will report you for harrassment", without any indication that this is how you feel. Absolutely insane.
Why would you give out your email address with that attitude? "Here is my email address, but the only thing you should use it for is to let me ruin your communications with others who agree to the same thing I just agreed to, if I receive anything I might conceivably have given it to you for."
If you don't want to get ANYTHING, EVER, then what is the purpose of giving out a means to communicate with you??? What did you think you are doing by giving out an email address??
It's just so bizarre. Do the world a favor and register an email address for what you consider spam, which is everything.
Then never look at it again, while that email address gets invitations and calls to publish in journals, updates on the product offerings you are interested in, informative newsletters, free money on things that you're already spending on, and so forth... while you get nothing.
> It's like smiling and saying to a guy, "Here's my number", and not saying: "But if you call me, for any reason, I will report you for harrassment", without any indication that this is how you feel.
Absolutely, and what's worse is that people marking ham as spam means that email providers can't take a spam report as seriously as they otherwise might, thus reducing the potential benefits of a 'report as spam' feature.
So we all get more spam because some people misuse 'report as spam'.
Not sure why you're getting downvoted. Someone giving out an email address that not only implicitly establishes a relationship of them emailing you, but explicitly even has a ToS about it, and then getting some mail on topic to that relationsihp, is absolutely not spam.
Maybe downvoter didn't get that you're using ham as meaning non-spam.
If you signed up for the ham, and it has an unsubscribe, it's absolutely immoral to flag it and deprive other people of the benefit.
I wonder what would happen to false spam reports like the GP's if every email field had a button next to it, "Don't ever email me, I don't know why I'm even giving you this." (i.e. "I'm insane.")
EDIT: cleared up that the ToS isn't what establishes the relationship - the fact that you're giving them your email does.
To be fair, if the permission to receive mail is buried in TOS, then I'd consider it spam too. So I guess I attached my point to the wrong comment.
But I know from industry experience that some people who explicitly sign up to mailing lists then go on to report those emails as spam, and this dilutes the effectiveness of 'report as spam' features.
I'm going to mention that at one point my company produced a custom product^, that cost 100's of dollars per purchase, and was only delivered via a single email. We had those emails marked as spam on numerous occasions.
To this day we have never sent a marketing email, or even non-transactional email - mainly because we suck at marketing, so it's not like they got confused about which email...
^ We do audio transcription, a customer sends an mp3 in, we have someone listen and type it up, and we emailed the customer the results. These days you can collect the transcripts from the website, but for years you couldn't.
Perhaps they signed up using a shared alias (marketing@...) and when you delivered the results, the original customer was happy but someone else on the alias marked it as spam.
The ToS is just one example. If you're giving out your email address to be an early trier of a new product, there's an implicit ToS that you can receive some news about it.
An unsubscribe button = "now stop emailing me." i.e. the opposite action of signing up and giving out an email.
flagging the kinds of things OP talks about after giving out your email address is ridiculous. Giving out your email address is opting in to communication initiated by the other side, without further requests - otherwise you wouldn't be giving out your email address, they would be giving you theirs: "Email us to get a reply with our newsletter" or whatever else the OP imagines in this bizarro-world where you opt in to a specific communication.
It's actually like I were asked for my number to get a service and then the company started calling me twice a day to offer me things I don't even want or care for because it falls within their carefully-worderd-ToS.
Do I agree to get emails sent to me from <random company> as long as they're product-related and I'm using their service? Yes I do. Will I mark these as spam? No I won't.
Do I agree to getting promotional emails or reminder emails when I stopped using their service months ago? No I don't. Will I mark these as spam? Yes I will.
The reply to that post states pretty clearly why you're wrong.
I'm curious though - after you give out your email address, which you don't consider "soliciting" in this sense but I do, how do you imagine "soliciting" any specific communication?
Nothing can possibly be 'solicited' under your definition.
I just don't get why you even give out an email address, if anything you get thereafter is still considered unsolicited.
> I just don't get why you even give out an email address
Because you pretty much have to buy anything online nowadays.
My personal solution is to use the '+' trick to give customized emails out to everyone and then to block them at the SMTP level in my /etc/mail/access file if they don't stop sending mails after the 1st unsubscribe attempt.
I can tell you from experience that some small percentage of people will mark your message as spam even if they specifically requested it -- heck, even if they want to continue receiving it in the future.
I routinely deal with people complaining about not getting a newsletter they requested after they marked it as spam (once they do that, we are required by agreement with the ISP to not send them any more messages ever).
I'm really curious about this ISP agreement. My wife hit a bug in the new Yahoo mail interface that led her to accidentally mark a few messages as spam. She immediately fixed the mistake, but it turned out that (unbeknownst to her) she was unsubscribed from an important mailing list as a result.
So she and I had two questions that we never resolved. First, is it really true that hitting the "Spam" button on a site like Yahoo or Gmail informs the sender of your email address? (Doesn't this lead to the usual concerns about confirming a valid address?) And second, what is a user supposed to do in the case of an accidental bump of the "Spam" button? Is there really no way to undo the damage (both to the sender and to the willing recipient)?
> is it really true that hitting the "Spam" button on a site like Yahoo or Gmail informs the sender of your email address?
Yes. The feedback loop shows the sender who marked the mail as spam.
> What is a user supposed to do in the case of an accidental bump of the "Spam" button? Is there really no way to undo the damage (both to the sender and to the willing recipient)?
There is no good solution, it kind of sucks for both the sender and the receiver. When you click that spam button, intentional or not, I can no longer send you email until going through a sometimes laborious process of working with the email service to get the email unblacklisted. If you really want to receive email again, send an email to the company/person and let them know you accidentally clicked the spam button so they can work it out with the email provider.
Gmail handles this the best IMO as it gives the user ~5 seconds to click "undo" before reporting the spam. As far as I can tell most other email clients are instantaneous.
That's maddening. The really frustrating part is that this effect (and especially the "can't undo" part) is completely invisible to the user. Like I said, my wife missed out on some rather important messages about an upcoming event that we were attending because of this. (After a couple of months, when the event was a few weeks away, I believe she got a message from the organizers letting her know that they hadn't been able to contact her but did she really think they were sending spam. Much frustration ensued.)
It's called a feedback loop[1] and yes they do pass email addresses, but only to whitelisted (theoretically legit) senders. This has been true for years.
I think the fear of "confirming an email address is valid" is unfounded. A list of valid email addresses of people who actively report messages they don't want as spam is not worth anything to a spammer and it would be illegal for a US company to do anything with the list of unsubscribes or spam reports anyway.
I'm not aware of any simple and universal solution to "undo" an accidental spam button click.
I guess I always assumed that spammers didn't particularly care whether what they do is legal or not (and certainly that they didn't much care whether the people they contacted wanted to get their messages). Is this a flawed impression/prejudice left over from the ancient days of the internet? Because I've been floored by the number of folks here on HN saying that they click the "unsubscribe" link within an email for any reason, let alone in preference to their email provider's internal "spam" button.
Well, let's be clear: I wouldn't advise clicking anything in a V1@GRA-type likely illegal spam message.
But if we're talking about newsletters your don't remember signing up for or marketing emails or invites to a new social network, the unsubscribe usually works and will almost never harm you. AFAIK it's a myth that there's this huge black market for "valid" email addresses. Spammers don't want a list of people who click unsubscribe; they ain't gonna buy anything. There are way easier ways to find email addresses on the internet. And it would be crazy (and possibly illegal) for any sort of semi-legit company to sell their unsubscribe list.
Anything send by an actual company -- especially anything sent through any of the major email providers -- will almost certainly have a working unsubscribe. It's almost impossible to send a message in MailChimp without a working unsubscribe.
I downvoted this message because it assumed that _asciiker_ was a bad actor who wanted to deliver spam, rather than assume that the he was perhaps misinformed about the best way to avoid delivering unwanted messages. It ignored the context of the original post (someone managing email servers for an ISP) and assumed that he was creating the spam messages directly. It took what could have been a reasonable thread about managing email and polarized it into a pro/anti spam thread.
(edit: Replaced "parent poster" with username of poster, to avoid confusion.)
I'm the parent author. I didn't use "you" referring to _asciiker_. It was plural you, not singular you, referring to "you damn spammers" as a third person, not "you _asciiker_".
You have all the right to mark it as SPAM of course. What I'm trying to get across here is that IP blocking is a bad solution. There are many cases where several domains share the same IP, specially with the shortage. So if domainA.com sends spam, why should domainB.com mail delivery on the same IP suffer from it?
Police the motherfuckingbejesus out of your users.
If it costs you when your IP is blocked, insist on a deposit that's forfeit in the event of spamming.
Sadly, I've worked for shops with poor practices. How bad? Sending thousands of emails to domains which no longer existed (let alone stale accounts at existing mail service providers). Simply not a priority.
More frequently, I've been taking to /dev/nulling entire domains for crap. Usually recruiters (no, I'm not interested in your underwater basketweaving SEO marketing position in south-west Obscuristan, it's well outside my search and skill parameters).
I haven't got around to writing a good set of whitelisting scripts, but that's next.
Figuring out if I've subscribed to something, and/or how to MAKE IT STAHHHP really isn't worth my time.
And now I have to check my spam folder to discover messages I signed up for. And I still get real spam because gmail's tools are diluted.
I think spam reporting is made too easy. I also hate over-sharing sites but not as much as I hate true spam. Unsub should be one click, spam reporting should be four clicks. Or hopefully gmail gives you an internal "spam report reliability" score so it knows you're a more trigger happy than I am.
And no, I've never sent a product mass mail out ever.
Just to illustrate a typical modern 'social' website email experience: I just signed up to MeetUp. Here's what I apparently signed up for just by giving them an email address so I could get information about a local event that someone has invited me to:
* Meetup Messenger
A fun, informative newsletter to Organizers and anyone interested in running a Meetup
* Meetup HQ Announcements
Get promotional emails from Meetup HQ
* Weekly Personal Calendar
A once-weekly email of your Meetups and top Meetups in your area
* New Meetup Group Announcements
Get email alerts about new Meetup Groups that match your interests
* Meetup Surveys
An occasional email survey asking your opinion about new or existing features, your Meetup Group(s), Meetup sponsorships, and other requests for feedback.
* Greetings
Send me an email when somebody posts a Greeting
So that's a minimum of a weekly email that I don't care about, plus promotional emails that I definitely don't care about: Joy unconfined. No mention of all these is made during the signup process that I can recall. Logging in with Facebook instead and letting them mine my FB friends graph instead is looking tempting frankly.
I can totally understand some people responding to this kind of 'dark pattern' email signup by block marking everything that isn't directly relevant to what they signed up for in the first place as spam. Marking it as spam means they never have to look at it ever again & is practically effortless on their part.
Sure, they could log back into the website in question (if they can find the password / can be bothered to log into Facebook) and faff around looking for the email subscription settings, but marking it as spam in GMail is quicker & easier: it's win all round from the user's point of view.
If you accept the terms without reading them (Not un ticking the newsletter button) then its your fault and you should be penalized for wrongly marking email as spam when you didn't even read the signup form to realise you were agreeing to receive emails.
Its people like you that make it so hard to send emails to people, even when they agree to receive emails.
Speaking as the former Sr. Internet Mail Administrator for AOL, there is none. When I was there, we actively worked very, very hard to make sure that we did everything we should do to keep our stupid AOL users from inappropriately hurting people who contact them -- Like marking that message from your own mother as spam? Really?
Problem is, there are plenty of large sites out there who aren't so conscientious, and you can keep up that level of commitment for only so long.
There is no one on this entire PLANET that hates spam more than I do, and for a time I could rightfully claim that there was no other single person on this planet who had done more to fight spam than I had. But even I have my limits. I haven't been active in CAUCE or any other anti-spam effort for many years, and I haven't even been an active mail system administrator for a few years.
As one of the members of the Postmaster Team for python.org (currently inactive), I have seriously hated some of the stupid shit that has been done by the likes of Google, hotmail, and Yahoo! I've been sorely tempted to just ban them outright, because of whatever might be their latest stupidity. But it hasn't happened -- At least, not yet.
Yes, I found the same problem with email, it's not getting spam (relatively easy to deal with) but deliverability, that's why at the end we need to use a 3rd party service like sendgrid, mailgun, mailchimp etc
The freedom of choice is actually in the hands of the end-users.
The end-users choose which email providers (Yahoo, Gmail, Hotmail) they want to use. The email providers are motivated to provide a good user experience, which includes blocking unwanted email because they make their money through user engagement.
The key to delivering into the inbox is sending mail that your recipients both want and expect. Provide a good user experience, and you'll build a good reputation. Push the limits (for example, use a "pre checked" checkbox on an order confirmation page to put people on your sales mailing list) and you'll be putting out email delivery fires all of the time.
Most of the things that you describe (SPF, SenderID, DKIM, PTR, MX) are all technical requirements, which are just the baseline for delivery. These are required, but any sender of unsolicited email can configure them. They don't earn you access to the inbox. Just like properly formatted HTML does not earn you great SEO results.
I like to break email delivery down into four areas:
* who -- send to people who requested your email and are expecting it
* what -- send something of value to these people
* technical foundation -- (SPF, SenderID, DKIM, Feedback Loops, etc.) required, but having it does not give you any points
* monitoring -- (open ratio, complaint ratio, ISP response codes) you need to know when something goes wrong
(My company, www.drh.net, has been providing email server software, services, and deliverability consulting for over 10 years.)
---
[edit; added the below]
Another way the freedom-of-choice is in the hands of the end users is this: the big ISPs (yahoo, gmail, hotmail) make most of their filtering decisions based off of end-user behavioral data.
For example:
* what percentage of your email is opened
* what percentage of your email is complained about (the "this is spam" button)
* what percentage of your email is deleted without reading
* how long is your email read
* how much is forwarded
* how much is replied to
* what percentage of your email that was placed in the Spam folder when seen by the user received a click on the "this is not Spam" button.
This is the end-users voting on if they want your email or not. This isn't the entire email deliverability equation, but it's a huge part of it.
The ISPs treat this data so importantly because: (a) it's hard to game unlike content filtering, and (b) it directly correlates to good user experience which they want to provide.
> Provide a good user experience, and you'll build a good
> reputation. Push the limits (for example, use a "pre
> checked" checkbox on an order confirmation page to put
> people on your sales mailing list) and you'll be putting
> out email delivery fires all of the time.
Believe it or not, it's possible to be a good actor, follow all of the rules and best practices, and still get flagged as spam.
Mail recipients are not perfect. They forget that they signed up for things. They accidentally click the "spam" flag on their messages. They get lazy and instead of unsubscribing they click the spam flag.
The real culprit here is that email messages rely on blacklists and not whitelists, i.e. recipients are required to give all senders full access and then block them when they misbehave, instead of giving them no access and giving them more access as they build trust.
So: What would it take to implement email whitelists across the industry?
Yes, good senders will still get messages flagged as spam. But the ISPs know this and they look at the complaint ratio. A complaint ratio of 0.5% or 1.0% is considered good. A complaint ratio of 3.0% is a problem.
We have one customer that's cleanest-of-the-clean (confirmed opt-in, valuable content, solid brand) sending 600k emails/day, and we see hundreds of spam reports. But their email gets delivered to the Inbox.
If you're a good actor with a solid technical setup you're still going to have an occasional delivery problem. This is why monitoring is so crucial. But you're not going to be putting out fires left-and-right, which is what it sounded like what _asciiker_ was saying he is doing.
> Conclusion: There is no common standard because every
> major ISP can set their own standards. This will
> eventually force everyone to use the same services
> worldwide.
>
> Where's the freedom of choice [of ESP] here?
My point being that the current blacklist-based system is broken from a "freedom of choice" perspective. The current system favors the established ESPs, as the cost of doing it yourself gets larger and larger.
Thank you for this insight, I agree with almost all of it.
As an e-mail services provider, I cannot or should inspect what my customers are sending. I can suspend them due to complaints of abuse but the damage is already done.
Same goes for tracking. I still say, block domain names, not IPs..
> As an e-mail services provider, I cannot or should inspect what my customers are sending. I can suspend them due to complaints of abuse but the damage is already done.
As an ESP, since you are letting customers send through your IP space, then a bad-apple can hurt the delivery of your other clients.
This is one of the big jobs that an ESP has. MailChimp, for example, has invested a ton of effort into detecting bad-apples as early as possible. (There are some really neat big-data techniques.) This is also why SES requires that you start with a smaller quota and build-up.
Some techniques:
* manually reviewing new clients before they send
* giving a new client a limited sending quota, so they build reputation with you over some time
* detect clients/campaigns with high complaints, high bounces, or low opens and take compliance action
* detect a partially-sent campaign with a high bounce rate and suspend it
* don't give any client an unlimited sending quota, so they can't hurt you too badly
> I still say, block domain names, not IPs..
There's a minimum amount of mail volume required to build a reputation. Many of your clients might not have this so they benefit from being lumped-in on an IP reputation.
I don't think IP blocking will ever go away, as it's an effective technique. The threat of an IP block also places some reasonable pressure on ESPs to police their client base.
Yeah, we just completed a switch to MailChimp after years of sending out our email ourselves. It's just too much hassle now for a smaller organization.
> The SES team knows about the spam cannibal listings and is in contact with them, they say it's unlikely your open rate drop from 25% to 0.15% is caused by the SC listing.
SpamCannibal can cause the originating mail server to get caught in a 'tarpit' by slowing it down. Given the AWS CUSTOMER was sending a significant amount of measurable email to a given destination server (which was running SpamCannibal) it's possible the sending servers are being slowed down. In that particular scenario, that would affect open rate over a short period of time.
Let people run rampant on your IP range, and this is what happens.
I run a fairly large website, and I block all traffic from the likes of Amazon AWS because it's full of dodgy bastards who think they're entitled to run however many HTTP requests they like. Webmasters, look at your web logs. Don't be surprised if the majority of hits are coming from bots pretending to be web browsers.
Our company offers a frontend web performance scanning SaaS product. We use EC2 for our scanning boxes. I've found many of our customers's website filter EC2 IPs. Its mainly from websites that offer a high demand product with a large secondary market. (think ticket websites for concerts/musicals/plays, airlines, hotels, etc).
bl.spamcannibal.org is notorious for a higher error rate (false positives). It'd be crazy if a popular mail service provider like Yahoo categorized incoming mail based on results from Spam Cannibal. If I added them to my MTA, I'd rank them fairly slow to diminish the effect of their false positives. See here: http://dnsbl.inps.de/analyse.cgi?type=monthly&lang=en
Engineering in a lot of reputation management and metrics would help.
There are some interesting (I'm not sure "compelling" or "strong" is necessarily the case) arguments to be made for enabling open relays and other forms of unauthenticated messaging. John Gilmore of EFF has fought that battle for a long time, and still runs an open relay on toad.com.
Signing and authentication measures (particularly on header data) have to be both standard and quick to process.
Methods which increase the costs of delivery -- pacing receipt rates from a given IP or block, can help. Being able to specify receipt priorities: high for IPs and ranges with which frequent legitimate business is transacted, very slow for most others, would also be useful. Along with a lot of built-in support for this.
Killing :80 and moving to entirely secured ports wouldn't be a bad move either.
Maybe you mean 25 or something else? Port 80 is for HTTP. And what does a secure port mean? If you want people to be able to talk through a port you have to open it. The number doesn't matter.
By "secure port", I mean forcing encryption of all over-the-wire traffic. It's happening now in many cases with STARTTLS (modulo utter brokenness of the CA and SSL/TLS systems), but that's still only opportunistic.
And of course, encrypting payloads would be vastly preferable. Headers as well other than absolutely required for delivery.
Require a small fee to send an email, e.g. $0.01. Small enough not to matter for legitimate users, large enough to make spam unprofitable.
Bitcoin or some other cryptocurrency would be ideal for facilitating micro-transactions like this. Interestingly, the Hashcash concept was originally designed to fight spam, and later became one of the important ideas that made the invention of Bitcoin possible: http://en.wikipedia.org/wiki/Hashcash
Right now I give out tagged addresses to most vendors, and I know others do it for lists. When a tag goes bad, I just route any further mail to my spam trainer.
But that means for every legitimate mailing list you're on, you get one spam. And you lose the legitimate mailing list just because their token got compromised.
Dunno, I feel like we should just charge for mailing lists too. Or use usenet!
If I can get it down to one spam each time a mailing list server is compromised, I'm ok with that.
Even with a charging scheme, some spam will still happen. Give the amount of BTC stolen so far, and given the number of compromised computers that could be used to generate cash, looks like there will be plenty of money to spend on sufficiently profitable spam.
I think the fee could be applied only to unsolicited emails. If I have a particular email address whitelisted then it will get through my gateway with no fee attached. However, if it is an unsolicited email from someone not on my whitelist it will be required to have some small fee attached to get past my filter.
I've seen some suggestions that instead of using money you could use some proof-of-work computation. This is something we could scale the difficulty factor of as computing power increased. Something that takes, say, an average computer 30 seconds (or longer) to calculate. It will attach this proof-of-work to the email.
Mailing lists wouldn't need to, or couldn't exist in this new magical email system. There's a hundred other solutions out there for what mailing lists provide.
Impose the fee only on one-way communications. A responsible mailing list would have a single instance of two-way communication when asking the user to confirm that they wanted to sign up. No I have no idea how that much state should be handled in a distributed fashion.
Separate authentication from message delivery. The problem with both spam and spam mitigation measures is that the message is considered as coming 'from' a sending system instead of the actual originator of the email.
Splitting these up converts one impossible problem into a tricky but mostly solved problem (delivery) and a hard problem that has to be addressed anyway (internet identity)
How about a PKI system were one or more authorities can sign keys and the email server admins can approve public keys based on its combined score coming from the authorities. Authorities could be universities, engineering organizations and other non-profit entities.
So now people have to register identities and can't use anonymous email addresses. And it wouldn't fix the issue anyways since spammers would just use botnets of legit sender addresses.
You are mixing up registration with registration with real data. You could still register an email address with arbitrary details and after a while you could establish a score that would grant your non-spam status. What you are confused with is that in this system there is no disposable email address. That is actually a valid concern but I don't use those so i really don't care.
Blacklists are general a very effective method in handling spam, but its kind of dangerous to use in a commercial setting. A few weeks ago SpamCop blocked gmail for a few days, causing some "mild" issues for companies that depend on email.
There is sadly not much options. Either I can accept more spam, or use blacklist and put the control of the filtering in the hands of a third-party with none of the responsibility attached.
It might help the situation more if webmail providers provided actual 'unsubscribe' or 'hide' links in their UI instead of 'spam' being the only feedback mechanism users are offered.
"unsubscribe" links vary in position, language and visibility in various clients. Making something beyond "this is spam" part of most mail clients, perhaps with reporting back to the originating sender, would help.
Problem here is that particularly such "unsubscribe" links where used in the past (and still are, I guess) to reassure spammers that somebody is there. Because one problem spammers have is the quality of the addresses they have. Many spammers use lists from dubious sources and a big number of addresses are invalid. So, if they get an "unsubscribe", they know which addresses are better and can deliver more spam to it ...
So many don't dare to use such links and rather click on spam.
The only solution could be some "trusted" functionality that goes via the own mail provider of the receiver. But of course the mail provider can not simply send information to the sender of the eMail .... So the thing gets complicated. As much I learned, for spam clicks there is something like that available ... some kind of trusted feedback chain that gives information to trusted senders, that some mails where labeled as spam. Thus those senders can (indirectly) adopt their eMail campaigns.
> Many spammers use lists from dubious sources and a big number of addresses are invalid. So, if they get an "unsubscribe", they know which addresses are better and can deliver more spam to it ...
That's only true of the dubious "viagra" style spam, where they got your name from a list. I don't think those even bother with "unsubscribe" links any more. I only see unsubscribe links from places where I've had to give my email up to buy something or sign up to a site. Those are generally legit and most of the techy/startup web sites will unsubscribe you immediately.
The next tier are the sites that unsubscribe you but take more than a week and will keep spamming their dumb newsletter in the meantime.
The final ones are either broken by stupidity (it's amazing how many web developers cannot grasp that "+" is a legit email character), or willfulness and will keep spamming no matter what. I block these at the SMTP level with 503 messages (usually containing some personal insults and swearing) as soon as they "RCPT TO" the unique email address I gave them.
Luckily I managed to block much of the spam I got before, so I don't see the current spam behavior. But some years ago, there well existed some non-viagra style spammers that just put dubious unsubscribe links inside. I never tried myself, but was warned that they use the information against you. I would guess, that verified eMails have a greater value ... but it also might be negligible now, since with bot-networks spammers don't need to care if they send 1 billion or 10 billion eMails ...
Another problem is that in some mail clients like Outlook.com, the "spam" button is actually labeled "junk". People clicking "junk" probably think they're just deleting an email.
I know there are legitimate uses for something like SES, but it has always seemed a bit like "Spam as a Service" to me. Ditto for every other service that's designed around mass email, no matter how much YC startups might depend on them to "improve their conversion rate" or whatever the buzzword is this week.
I'd call it "Sender Reputation Checking as a Service". Where said service is paid for by the sender, but provided to the email recipient. Anyone can send email directly; but knowing that email has been sent through SES and Amazon hasn't killed the account yet provides a greater degree of trustworthiness.
...and we all know how well the bond rating services performed their function. Sorry, couldn't resist.
On a more serious note, it seems like the "greater degree of trustworthiness" is only very slightly greater. SES might be better than some server in a domain nobody ever heard of, but it's still not as good as a provider with a long history of responsible email use. Many people can and do block SES and its ilk, as is the subject of this story, because the aggregate amount of spam is so great even if the individual spammers are transient (like they care).
Amazon could raise the bar, thus raising their own reputation and thus making the service more valuable to those who can still afford/qualify to use it. It's probably just not worth their while to do so. I'm not even criticizing them for that. I'm just observing that online business has a shady side, and Amazon isn't afraid to partake.
Considering Amazon themselves send spam this isn't surprising. They like to send out product spam under the guise of account notifications with no opt-out. The only way to remove yourself is to close your account. At the very least they do this to affiliates and to Student Prime members.
I had had the same issue with amazon SES. While Spamhaus PBL is not necessarily a blocklist, due to the default configuration error, all the emails we send out (from other providers like mailchimp) ended up in spam. Finally, the issue got fixed by configuring reverse PTR.
are people not relaying their newsletters/unsolicited mails/spam thru some antispam outgoing smtp gateway if they're concerned about being blocked? filter it yourself before people filter u?
I believe in IP blacklisting. Why should my business be open to countries like North-Korea or Somali if I don't have anything to do with those? Spam comes from everywhere, but you can get rid of at least 50% by simply dropping traffic from certain countries, so the expensive score based spam filtering get cheaper. Unfortunately Amazon SES is the victim of cyber warfare and spammers.
This sort of logic will hurt you with customers who are US persons living overseas. I cannot tell you how annoying it is when as a US citizen and taxpayer I am blocked from using US services because I am not physically in the US. In 2014, where we plug our computers in does not define who we are. Geolocation is not authentication.
Not even just that, but where they ignore your Accept-Language and just make up a decision on what language you should be using. Google has no way to fully switch languages - even after going to Google.com and setting English, you'll see images have tooltips in your "local" language.
Google's Play app does the same thing for a bit of their content. Certain headers get localized, despite everything else in the app being English. Netflix has the same problem, and then to further add insult, they send you to non-English phone numbers for support.
The most annoying thing is that someone probably got a raise for these "features".
It is a trade off. You can purchase a cheap VPN service from thousands of VPN providers and get routed through a US IP. For you, who wants to fill the tax return it is worth it, for those who want to spam the sh*t out of other companies it is not worth it. I think this system works perfectly...
By appending in the Posfix configuration file line smtpd_recipient_restrictions = ... spamcop and spamhaus , spam decreases in like 95% without even touching your server further (spamassassin I'm looking at you).
I you add greylisting you get rid virtually of all of spam.
I think we'd all rather "sift through a little spam than lose something I wanted" but my guess is that you've never run an email server. "a little spam" is not what you will get - it will be orders of magnitude more spam than legit emails. This is hard stuff - Gmail, supposedly one of the best, catches between 2 and 10 emails a day in my "Spam" folder that aren't actually spam. If I were to turn off the spam filter (if you could) in Gmail, I'd get 2,000 emails a day - of which 50 would be legit.
I actually have run a mail server before, and now I gladly pay someone else to do it for me :)
Point taken, but I've been on the receiving end of what I would consider false-positive blocks by Spamhaus & co. They sometimes have policies about what's considered spam that I don't think all their end users would agree with. I've been blocked for having an IP address on the same provider as someone else who allegedly advertised their website via spam. If you're running a mailing list, dealing with the anti-spam stuff is at least as big a problem as the spam it was supposed to solve.
I've never had a case of a false positive complain but yes, they can happen and I don't terribly mind because that means the sender's mail server is blacklisted and they should know.
Also in any case my recommendation is to use a 3rd party mail service, for deliverability reasons mostly.
Currently, I get a couple thousand spam messages a day, maybe 5 of wihch make it to my inbox. I haven't had a false positive in years. But if I did, I would be ok with not having to manually evaluate 50 pieces of junk for every real message I receive.
> I you add greylisting you get rid virtually of all of spam.
I don’t really like greylisting, as it takes longer for email to get through. What did help without any perceivable loss whatsoever is being extensively strict about SMTP specifications (e.g. proper hostnames in EHLO) and things like PTR records. I really like watching these ‘5xx: Client <something> rejected’ flying by in my mail.info :)
It doesn't use them to insta-spam an email, which I think is where blacklists go wrong. It gives spam points to the message if the sender is on the lists, which turns them from "blacklists" into "suspicious lists".
If an otherwise normal email happens to come from a blacklisted computer, it'll still have a chance to get through, which is the correct thing to do, in my opinion.
I still believe that everyone has the right to run their own SMTP server, and I dislike that so many places blacklist someone just because they are on a cable modem.
This sort of vague comment with no substance is just opinion stated as fact. If you have more information - hell, even a more expressed opinion, I'd be interested. As expressed, however, it's a garbage comment that waste's everyone's time. Why bother?
Not sure about "vague comment"s but anyone who has run an email server, used an rbl/rhsbl, and followed the logs <http://www.postconf.com/docs/spamrep/> would say the same. Having done so for years and run reports on dozens of servers daily it is clear that blacklists are the most effective form of spam blocking, by at least an order of magnitude.
I can't tell whether you agree with the OP or with my comment. Your first sentence argues that OP is right - "IP blacklists are a waste of everyone's time". Your second sentence though is "it is clear that blacklists are the most effective form of spam blocking, by at least an order of magnitude." Maybe a typo in your reply?
I got that part, I'm talking about your first sentence.
... anyone who has run an email server... would say the same.
This reads as though you agree that "IP blacklists are a waste of everyone's time" as OP said. And maybe you do (and that's fine) - I'm just unclear given your second sentence.
Possibly, but if Amazon isn't doing enough to secure SES against abuse by spammers(1), it's not unfair for entities intended to guard against spam to treat it as a spam source. This is one of the risks of letting third-parties run software on one's systems.
(1) keeping in mind that the definitions of "spam" can be quite subjective
Amazon SES definitely does have a spam problem, and the issue is that they don't bother doing anything about abuse reports. I've reported a spammer (who scraped our email address) to them, but the spams kept coming. Other people have reported the same thing.
If you're running a bulk mailing server you simply must respond to abuse reports, otherwise your service will get blacklisted and be essentially useless. Other providers such as mailchimp are much more proactive about getting rid of spammers.
I am following all the best practises, hell, I even advocate them. It is just that these days it seems not to matter if you have SPF, Sender ID, DomainKey and DKIM, PTR, proper MX and even a normal to good IP reputation. There is still no guarantee what you will be able to reach the inbox of the likes of Gmail, Yahoo, Hotmail, etc.
I have been filling out huge forms for each and every major ISP for the past year because one or two users mark a newsletter as SPAM.
Conclusion: There is no common standard because every major ISP can set their own standards. This will eventually force everyone to use the same services worldwide.
Where's the freedom of choice here?