Our domains, which reside in a shared hosting service, are constantly appearing in blacklists like this one: http://cbl.abuseat.org/
This results in emails being blocked and lost. We have notified our hosting provider, they have (supposedly) taken extra measures and have done several unblock requests. Only to return back to blocked a few hours or days later. They finally gave up and told us that this is a common problem with shared hosting. Apparently, some forms of mass email can be perceived as spam by these blocklists, so they end up blocking the whole server. Our only real solution is to move to dedicated hosting or VPS, and have our own unique IP.
Question… is this normal indeed? Is our provider being truthful, or simply incompetent to deal with the real problem?
Who is your host? I’ve read many, many complaints about this type of thing and, sadly, I don’t think there is much that can be done by hosts because of the way the spam marking and blacklisting process works. I think it is downright stupid to block an entire IP address because of alleged spam. And by alleged, I mean that some email providers allow customers to flag emails as spam even if they are not spam, such as someone who subscribes to an email newsletter and instead of unsubscribing hits the spam button which can lead to the provider blacklisting the IP address. You will read a lot of complaints about AOL doing this and they are not the only one.
No, I would not say it is incompetence on the part of the host. I would say it is the fault of email providers for resorting to a knee-jerk reaction of blacklisting entire IP addresses regardless of the actual sender of the alleged spam and not taking the time to determine if something is actually spam. If the IP is blacklisted, I don’t suppose something like an SPF record or [URL=“http://en.wikipedia.org/wiki/DomainKeys_Identified_Mail”]DomainKeys (DKIM) would help in that case, would it?
Some hosts offer dedicated IP addresses for a cost of $2 a month or so. Maybe you could look into that.
I don’t agree with not much can be done. You’ve already pointed to a few things that can help but here is my list:
Enforce Reverse DNS to be setup for each domain. Enforce this when provision the domain on a given shared server.
Use Sender Policy Framework/Domain Keys for a domain and smtp authentication. If the host forces smtp authentication bound to their network so that anyone that wants to route mail would need to contact the host for their unique authentication requirements. While speaking with the customer the host can request their SPF record. If the customer doesn’t have a SPF setup then the host would require them to create it.
Ensure that proper anti-virus and malware detection is enabled on the server and through their SMTP. If messages have to pass through a local network SMTP then virus and spam filtering can be done over this defined route.
Throttle the rate and the number of emails that can be sent. Block batch sends so that it becomes impractical and forces email campaigns and large newsletter distributions to be outsourced to email services that are licensed to do this.
When a host allows their IPs to get messed up then it means they can’t re-purpose them easily plus they deal with all the support headaches.
One of the shared hosts that I have used for a site makes use of all these things and the shared server public I.P. has not been blacklisted - since I started hosting there - in the last 5 years .
All hosts will run into blacklist issues from time to time. And some of the blacklists can be very difficult to work with. But any experienced and good host will have a well defined method of dealing with this, and any blocks you encounter should be rare.
Your condemnation of hosts is mildly misdirected as blacklisters (like SpamHaus) simply blacklist entire IP blocks without regard to the consequences to all the non-spammers using that host. IMHO, these people are incompetent and only serve to make matters worse, not better.
Unfortunately, that will do absolutely no good! I’d had a dedicated IP address and the host (Site5) refused to configure their EXIM to allow me to send e-mail from my dedicated IP address. I had to leave Site5 to get away from the EXIM’s IP address to get away from the blacklist problem.
[aside]I understand that Ben (a SitePoint member) took over at Site5 and improved things immensely but I can’t verify to what extent.[/aside]
This is a major problem which can only be resolved by having your own EXIM (VPS or dedicated account) unless blacklisters simply blacklist blocks (like SpamHaus).
I too had email problems because apparently another site on my server got hacked and used to send mass emails. My host switched me to a different mail server, and the result was that some of the domains that had blocked me before were now accepting my emails, but new ones were rejecting them. I guess another site that had another similar problem had at one time been using the new mail server. I think it took me three tries to find a server with a “clean” IP.
And a few weeks from now that IP may appear on another blacklist. If any of the things mentioned by ServerStorm above are real solutions to the problem, are enforceable on the hosting clients, and can be done without too much inconvenience, then I think people are going to have to start lobbying their hosting companies to put them in place. I’ve read a lot of complaints by people on many different hosts (including mine) about their outgoing emails being blocked because the IP appeared on some blacklist. It is not an isolated issue. If there is a relatively simple way to prevent this, then the hosting industry is going to have to move that direction.
With shared and reseller hosting, there are other people using your mail server. This means that one spammer can get everyone on that IP address blocked by a blacklist.
A dedicated IP does not help with emails being blacklisted. Email always uses the shared IP, even if you buy a dedicated IP.
It is obvious the blacklist is there for a reason. Your host did not have preventative measures in place to stop abuse of their services which now results in the entire server being blacklisted. Your only option is to really find a new host that is not blacklisted. Your current host’s servers are blacklisted so there is no recovery option.