Anti-spam techniques (users)

This page is about client side and other techniques. For server-side techniques read the Anti-spam techniques page.

With email spam, which involves the unwanted inundation of unsolicited bulk emails, an array of user-side anti-spam techniques have been developed. These are methods created on the client arrangement of a situation, as opposed to the server-side.

While no single method offers a foolproof solution to the spam predicament, each approach requires a delicate balance between minimizing the rejection of legitimate emails and effectively filtering out spam. This delicate equilibrium is influenced by considerations such as time, effort, and costs associated with the inadvertent blocking of non-spam emails.

Broadly, anti-spam techniques fall into four categories: those reliant on individual actions, methods administratively managed by email service providers, techniques automatically governed by email senders, and measures overseen by researchers and law enforcement agencies. Additionally, individuals can employ various strategies to mitigate the exposure of their email addresses, thereby diminishing the likelihood of receiving spam.

Attempting to balance false negatives and false positives is a critical aspect of a successful anti-spam system. Because servers are not able to block all spam, there are multiple methods for individual users to allow users to control this balance.

End-user techniques

There are several techniques that individuals can use to restrict the availability of their email addresses, reducing or preventing their attractiveness to spam.

Discretion

Sharing an email address only among a limited group of correspondents is one way to limit spam. This method relies on the discretion of all members of the group, as disclosing email addresses outside the group circumvents the trust relationship of the group. For this reason, forwarding messages to recipients who don't know one another should be avoided. When it is absolutely necessary to forward messages to recipients who don't know one another, it is good practice to list the recipient names all after "bcc:", instead of after "to:". This practice avoids the scenario where unscrupulous recipients might compile a list of email addresses for spamming purposes. This practice also reduces the risk of the address being distributed by computers affected by email address harvesting malware. However, once the privacy of the email address is lost by divulgence, it cannot likely be regained.

Address munging

Posting anonymously, or with a fake name and address, is one way to avoid email-address harvesting, but users should ensure that the fake address is not valid. Users who want to receive legitimate emails regarding their posts or websites can alter their addresses so humans can figure out but spammers cannot. For instance, joe at example.com might post as joeNOS at PAM.invalid.example.com. Address munging, however, can cause legitimate replies to be lost. If it's not the user's valid address, it has to be truly invalid, otherwise, someone or some server will still get the spam for it. Other ways use transparent address munging to avoid this by allowing users to see the actual address but obfuscate it from automated email harvesters with methods such as displaying all or part of the email address on a web page as an image, a text logo shrunken to normal size using in-line CSS, or as jumbled text with the order of characters restored using CSS.

Avoid responding to spam

Spammers often regard responses to their messages—even responses like "Don't spam me"—as confirmation that an email address is valid, including following links or addresses from the message, asking for removal from the spammer's mailing list. In several cases, spam-fighters have tested these links, confirming they do not lead to the recipient address's removal—if anything, they lead to more spam. This removal request of filing a complaint may get the address list washed, to lower complaints so the spammer can stay active without acquiring new accounts and internet providers.

Sender addresses are often forged in spam messages, including using the recipient's address as the forged sender address, so that responding to spam may result in failed deliveries, or may reach innocent email users whose addresses have been abused.

In Usenet, it is widely considered even more important to avoid responding to spam. Many ISPs have software that seeks and destroys duplicate messages. Someone may see spam and respond to it before it is canceled by their server, which can have the effect of reposting the spam for them. Since it is not a duplicate, the reposted copy will last longer. Replying may also cause the poster to be falsely linked to as part of the spam message.

Contact forms

Contact forms allow users to send emails by filling out forms in their web browser. The web server takes the form data and forwards it to an email address. Users never see the email address. Such forms, however, are sometimes inconvenient to users, as they are not able to use their preferred email client, risk entering a faulty reply address, and are typically not notified about delivery problems. Further, contact forms have the drawback that they require a website that supports server-side scripts. Finally, if the software used to run the contact forms is badly designed, it can become a spam tool in its own right. Additionally, some spammers have begun to send spam using the contact form.

Disable HTML in email

Many modern email programs incorporate Web browser functionality, such as the display of HTML, URLs, and images. This can easily expose the user to offensive images in spam. In addition, spam written in HTML can contain web bugs which allows spammers to see that the email address is valid and that the message has not been caught in spam filters. JavaScript programs can be used to direct the user's Web browser to an advertised page or to make the spam message difficult to close or delete. Spam messages have contained attacks that use security vulnerabilities in the HTML renderer to install spyware. (Some computer viruses are borne by the same mechanisms).

Mail clients which do not automatically download and display HTML, images, or attachments, have fewer risks, as do clients who have been configured to not display these by default.

Disposable email addresses

An email user may sometimes need to give an address to a site without complete assurance that the site owner will not use it for sending spam. One way to mitigate the risk is to provide a disposable email address—a temporary address that the user can disable or abandon, which forwards emails to their real account. There are several services that provide disposable email addresses, that can be manually disabled, can expire after a given time interval, or can expire after a certain number of messages have been forwarded. Disposable email addresses can be used by users to track whether a site owner has disclosed an address. This capability has resulted in legal jeopardy for sites that disclose confidential addresses without permission.

Ham passwords

Systems that use ham passwords ask unrecognized senders to include in their email a password that demonstrates that the email message is a "ham" (non-spam) message. Typically the email address and ham password would be described on a web page, and the ham password would be included in the "subject" line of an email address. Ham passwords are often combined with filtering systems, to counter the risk that a filtering system will accidentally identify a ham message as a spam message.

The "plus addressing" technique allows the appending of the password to the "username" part of the email address.

Reporting spam

Tracking down a spammer's ISP and reporting the offense can lead to the spammer's service being terminated, and criminal prosecution. Unfortunately, it can be difficult to track down the spammer—and while there are some online tools to assist, they are not always accurate. Occasionally, spammers employ their own netblocks. In this case, the abusive contact for the netblock can be the spammer itself and can confirm your address.

Examples of these online tools are SpamCop and Network Abuse Clearinghouse. They provide automated or semi-automated means to report spam to ISPs. Some spam-fighters regard them as inaccurate compared to what an expert in the email system can do; however, most email users are not experts.

A free tool called Complainterator may be used in the reporting of spam. The Complainterator will send an automatically generated complaint to the registrar of the spamming domain and the registrar of its name servers.

Historically, reporting spam in this way has not seriously abated spam, since the spammers simply move their operation to another URL, ISP, or network of IP addresses.

An alternative to contacting ISPs is to contact the registrar of a domain name that has been used in spam email. Registrars, as ICANN-accredited administrative organizations, are obliged to uphold certain rules and regulations and have the resources necessary for dealing with abuse complaints.

Responding to spam

Some people believe that responding aggressively to spam is helpful—in other words, "spamming the spammer".

The basic idea is to make spamming less attractive to the spammer by increasing the spammer's overhead. There are several ways to reach a spammer other than the caveats mentioned above, however, it may lead to retaliation by the spammer.

1. Replying directly to the spammer's email address

   Just clicking "reply" will not work in the vast majority of cases, since most of the sender addresses are forged or made up. In some cases, however, spammers do provide valid addresses, as in the case of Nigerian scams.

2. Targeting the computers used to send out spam

   In 2005, IBM announced a service to bounce spam directly to the computers that send out spam. Because the IP addresses are identified in the headers of every message, it would be possible to target those computers directly, sidestepping the problem of forged email addresses. In most cases, however, those computers do not belong to the real spammer, but to unsuspecting users with unsecured or outdated systems, hijacked through malware and controlled at a distance by the spammer; these are known as zombie computers. However, in most legal jurisdictions, ignorance is no defense, and many victims of spam regard the owners of zombie computers as willfully compliant accomplices of spammers.

1. Leaving messages on the spamvertised site

   Spammers selling their wares need a tangible point of contact so that customers can reach them. Sometimes it is a telephone number, but most often is a website containing web forms through which customers can fill out orders or inquiries, or even "unsubscribe" requests. Since the positive response to spam is probably much less than 1/10,000, if just a tiny percentage of users visit spam sites just to leave negative messages, the negative messages could easily outnumber positive ones, incurring costs for spammers to sort them out, not to mention the cost in bandwidth. An automated system, designed to respond in just such a way, was Blue Frog. Unfortunately, in doing so, you risk arousing the ire of criminals who may respond with threats or 'target' your address with even more spam.

Automated techniques for email senders

There are a variety of techniques that email senders use to try to make sure that they do not send spam. Failure to control the amount of spam sent, as judged by email receivers, can often cause even legitimate emails to be blocked and for the sender to be put on DNSBLs.

Background checks on new users and customers

Since spammer's accounts are frequently disabled due to violations of abuse policies, they are constantly trying to create new accounts. Due to the damage done to an ISP's reputation when it is the source of spam, many ISPs and web email providers use CAPTCHAs on new accounts to verify that it is a real human registering the account, and not an automated spamming system. They can also verify that credit cards are not stolen before accepting new customers, check the Spamhaus Project ROKSO list, and do other background checks.

Confirmed opt-in for mailing lists

One difficulty in implementing opt-in mailing lists is that many means of gathering user email addresses remain susceptible to forgery. For instance, if a company puts up a Web form to allow users to subscribe to a mailing list about its products, a malicious person can enter other people's email addresses—to harass them, or to make the company appear to be spamming. (To most anti-spammers, if the company sends an email to these forgery victims, it is spamming, albeit inadvertently.)

To prevent this abuse, MAPS and other anti-spam organizations encourage that all mailing lists use confirmed opt-in (also known as verified opt-in or double opt-in). That is, whenever an email address is presented for subscription to the list, the list software should send a confirmation message to that address. The confirmation message contains no advertising content, so it is not construed to be spam itself — and the address is not added to the live mail list unless the recipient responds to the confirmation message. See also the Spamhaus Mailing Lists vs. Spam Lists page.

All modern mailing list management programs (such as GNU Mailman, LISTSERV, Majordomo, and qmail's ezmlm) support confirmed opt-in by default.

Egress spam filtering

Email senders can do the same type of anti-spam checks on emails coming from their users and customers as can be done for emails coming from the rest of the Internet.

Limit email backscatter

If any sort of bounce message or anti-virus warning gets sent to a forged email address, the result will be backscatter.

Problems with sending challenges to forged email addresses can be greatly reduced by not creating a new message that contains the challenge. Instead, the challenge can be placed in the Bounce message when the receiving mail system gives a rejection code during the SMTP session. When the receiving mail system rejects an email this way, it is the sending system that actually creates the bounce message. As a result, the bounce message will almost always be sent to the real sender, and it will be in a format and language that the sender will usually recognize.

Port 25 blocking

Firewalls and routers can be programmed to not allow SMTP traffic (TCP port 25) from machines on the network that are not supposed to run Mail Transfer Agents or send email. This practice is somewhat controversial when ISPs block home users, especially if the ISPs do not allow the blocking to be turned off upon request. Email can still be sent from these computers to designated smart hosts via port 25 and to other smart hosts via the email submission port 587.

Port 25 interception

Network address translation can be used to intercept all port 25 (SMTP) traffic and direct it to a mail server that enforces rate limiting and egress spam filtering. This is commonly done in hotels, but it can cause email privacy problems, as well as making it impossible to use STARTTLS and SMTP-AUTH if the port 587 submission port isn't used.

Rate limiting

Machines that suddenly start to send large amounts of email may have become zombie computers. By limiting the rate at which email can be sent around to what is typical for the computer in question, legitimate email can still be sent, but large spam runs can be slowed until a manual investigation can be done.

Spam report feedback loops

By monitoring spam reports from places such as spamcop, AOL's feedback loop, and Network Abuse Clearinghouse, the domain's abuse@ mailbox, etc., ISPs can often learn of problems before they seriously damage the ISP's reputation and have their mail servers blacklisted.

FROM field control

Both malicious software and human spam senders often use forged FROM addresses when sending spam messages. Control may be enforced on SMTP servers to ensure senders can only use their correct email address in the FROM field of outgoing messages. In an email user's database, each user has a record with an email address. The SMTP server must check if the email address in the FROM field of an outgoing message is the same address that belongs to the user's credentials, supplied for SMTP authentication. If the FROM field is forged, an SMTP error will be returned to the email client (e.g., "You do not own the email address you are trying to send from").

Strong AUP and TOS agreements

Most ISPs and webmail providers have either an Acceptable Use Policy (AUP) or a Terms of Service (TOS) agreement that discourages spammers from using their system and allows the spammer to be terminated quickly for violations.

Techniques for researchers & law enforcement

Increasingly, anti-spam efforts have led to coordination between law enforcement, researchers, major consumer financial service companies, and Internet service providers in monitoring and tracking email spam, identity theft and phishing activities and gathering evidence for criminal cases.

Legislation and enforcement

Appropriate legislation and enforcement can have a significant impact on spamming activity.

The penalty provisions of the Australian Spam Act 2003 dropped Australia's ranking in the list of spam-relaying countries for email spam from tenth to twenty-eighth.

Legislation that provides mandates that bulk emailers must follow makes compliant spam easier to identify and filter out.

Analysis of spamvertisements

Analysis of sites being spamvertised by a given piece of spam often leads to questionable registrations of Internet domain names. Since registrars are required to maintain trustworthy WHOIS databases, digging into the registration details and complaining at the proper locations often results in site shutdowns. Uncoordinated activity may not be effective, given today's volume of spam and the rate at which criminal organizations register new domains. However, a coordinated effort, implemented with adequate infrastructure, can obtain good results.

Cost-based systems

Since spamming is facilitated by the fact that large volumes of email are very inexpensive to send, one proposed set of solutions would require that senders pay some cost in order to send email, making it prohibitively expensive for spammers. Anti-spam activist Daniel Balsam attempts to make spamming less profitable by bringing lawsuits against spammers.

Other techniques

There are a number of proposals for sideband protocols that will assist SMTP operation. The Anti-Spam Research Group (ASRG) of the Internet Research Task Force (IRTF) is working on a number of email authentication and other proposals for providing simple source authentication that is flexible, lightweight, and scalable. Recent Internet Engineering Task Force (IETF) activities include MARID (2004) leading to two approved IETF experiments in 2005, and DomainKeys Identified Mail in 2006.

DMARC, which stands for "Domain-based Message Authentication, Reporting & Conformance" standardizes how email receivers perform email authentication using the well-known Sender Policy Framework (SPF) and DKIM mechanisms.

Channel email is a new proposal for sending emails that attempt to distribute anti-spam activities by forcing verification (probably using bounce messages so back-scatter doesn't occur) when the first email is sent for new contacts.

SMTP proxy

SMTP proxies allow combating spam in real-time, combining the sender's behavior controls, providing legitimate users immediate feedback, and eliminating a need for quarantine.

Statistical content filtering

Statistical (or Bayesian) filtering once set up, requires no administrative maintenance per se: instead, users mark messages as spam or nonspam and the filtering software learns from these judgments. Thus, a statistical filter does not reflect the software author's or administrator's biases as to content, but rather the user's biases. For example, a biochemist who is researching Viagra won't have messages containing the word "Viagra" automatically flagged as spam, because "Viagra" will show up often in his or her legitimate messages. Still, spam emails containing the word "Viagra" do get filtered because the content of the rest of the spam messages differs significantly from the content of legitimate messages. A statistical filter can also respond quickly to changes in spam content, without administrative intervention, as long as users consistently designate false negative messages as spam when received in their email. Statistical filters can also look at message headers, thereby considering not just the content but also the peculiarities of the transport mechanism of the email.

Typical statistical filtering uses single words in the calculations to decide if a message should be classified as spam or not. A more powerful calculation can be made using groups of two or more words taken together. Then random "noise" words can not be used as successfully to fool the filter.

Software programs that implement statistical filtering include Bogofilter, DSPAM, SpamBayes, ASSP, the email programs Mozilla and Mozilla Thunderbird, Mailwasher, and later revisions of SpamAssassin. Another interesting project is CRM114 which hashes phrases and does Bayesian classification on the phrases.

There is also the free mail filter POPFile, which sorts mail in as many categories as the user wants (family, friends, co-workers, spam, whatever) with Bayesian filtering.

Research conferences

Spam is the subject of several research conferences, including:

• Messaging Anti-Abuse Working Group
• TREC (July 2007)
• Conference on Email and Anti-Spam, August 2007
• FTC Spam Summit, July 2007
• MIT Spam Conference, March 2007

See also

• MailChannels