Thursday, April 12, 2007

How Do Spammers Harvest Email Addresses?

Introduction

It is often difficult or impossible to tell how a spammer acquired a user's e-mail address. Is it a result of some activity the user engaged in? Did the user give his/her e-mail address to the wrong person? Is the user randomly targeted? Are there steps the user could take to avoid such spam in the future? As a result, consumers who use email are exposed to a variety of spam - including objectionable messages - no matter the source of the address.This study attempts to answer some of these questions by analyzing common activities of Internet users and looking for evidence of some activities that resulted in one e-mail address receiving more spam than others. Armed with lists of e-mail addresses, "spammers" send billions of e-mail messages every day -- messages that most users don't want.Through this investigation it is indicated that email address harvesting usually is automated, because spam can hit the addresses soon after they are used publicly the first time; the spam was not targeted; and some addresses were picked up off web pages even when they weren't visible to the eye. Still, I would say said consumers can protect their email addresses from harvesting programs.

Problem

There are many ways in which spammers can get email address. The ones commonly used are:From posts to UseNet with your email address.Spammers regularly scan UseNet for email address, using ready made programs designed to do so. Some programs just look at articles headers which contain email address (From: Reply-To: etc), while other programs check the articles' bodies, starting with programs that look at signatures, through programs that take everything that contain a '@' character.As people who where spammed frequently report that spam frequency to their mailbox dropped sharply after a period in which they did not post to UseNet, as well as evidence to spammers' chase after 'fresh' and 'live' addresses, this technique seems to be the primary source of email addresses for spammers.

1. From mailing lists.Spammers regularly attempt to get the lists of subscribers to mailing lists knowing that the email addresses are unmunged and that only a few of the addresses are invalid.When mail servers are configured to refuse such requests, another trick might be used - spammers might send an email to the mailing list with the headers Return-Receipt-To: or X-Confirm-Reading-To: . Those headers would cause some mail transfer agents and reading programs to send email back to the saying that the email was delivered to / read at a given email address, divulging it to spammers.A different technique used by spammers is to request a mailing lists server to give him the list of all mailing lists it carries (an option implemented by some mailing list servers for the convenience of legitimate users), and then send the spam to the mailing list's address, leaving the server to do the hard work of forwarding a copy to each subscribed email address.

2. From web pages.Spammers have programs which spider through web pages, looking for email addresses, e.g. email addresses contained in mailto: HTML tags [those you can click on and get a mail window opened]Some spammers even target their mail based on web pages. I have discovered a web page of mine appeared in Yahoo as some spammer harvested email addresses from each new page appearing in Yahoo and sent me a spam regarding that web page.A widely used technique to fight this technique is the 'poison' CGI script. The script creates a page with several bogus email addresses and a link to itself. Spammers' software visiting the page would harvest the bogus email addresses and follow up the link, entering an infinite loop polluting their lists with bogus email addresses.

3. From various web and paper forms.Some sites request various details via forms, e.g. guest books & registrations forms. Spammers can get email addresses from those either because the form becomes available on the World Wide Web, or because the site sells / gives the emails list to others.Some companies would sell / give email lists filled in on paper forms, e.g. organizers of conventions would make a list of participants' email addresses, and sell it when it's no longer needed.Some spammers would actually type E-mail addresses from printed material, e.g. professional directories & conference proceedings.Domain name registration forms are a favorite as well - addresses are most usually correct and updated, and people read the emails sent to them expecting important messages.

4. From a web browser.Some sites use various tricks to extract a surfer's email address from the web browser, sometimes without the surfer noticing it. Those techniques include:1. Making the browser fetch one of the page's images through an anonymous FTP connection to the site.Some browsers would give the email address the user has configured into the browser as the password for the anonymous FTP account. A surfer not aware of this technique will not notice that the email address has leaked.2. Using JavaScript to make the browser send an email to a chosen email address with the email address configured into the browser.Some browsers would allow email to be sent when the mouse passes over some part of a page. Unless the browser is properly configured, no warning will be issued.3. Using the HTTP_FROM header that browsers send to the server.Some browsers pass a header with your email address to every web server you visit. It's worth noting here that when one reads E-mail with a browser (or any mail reader that understands HTML), the reader should be aware of active content (Java applets, JavaScript, VB, etc) as well as web bugs.An E-mail containing HTML may contain a script that upon being read (or even the subject being highlighted) automatically sends E-mail to any E-mail addresses. A good example of this case is the Melissa virus.

5. From IRC and chat rooms.Some IRC clients will give a user's email address to anyone who cares to ask it. Many spammers harvest email addresses from IRC, knowing that those are 'live' addresses and send spam to those email addresses.This method is used beside the annoying IRC bots that send messages interactively to IRC and chat rooms without attempting to recognize who is participating in the first place.This is another major source of email addresses for spammers, especially as this is one of the first public activities newbie's join, making it easy for spammers to harvest 'fresh' addresses of people who might have very little experience dealing with spam.AOL chat rooms are the most popular of those - according to reports there's a utility that can get the screen names of participants in AOL chat rooms. The utility is reported to be specialized for AOL due to two main reasons - AOL makes the list of the actively participating users' screen names available and AOL users are considered prime targets by spammers due to the reputation of AOL as being the ISP of choice by newbie's.

6. From domain contact points.Every domain has one to three contact points - administration, technical, and billing. The contact point includes the email address of the contact person.As the contact points are freely available, spammers harvest the email addresses from the contact points for lists of domains (the list of domain is usually made available to the public by the domain registries). This is a tempting method for spammers, as those email addresses are most usually valid and mail sent to it is being read regularly.

7. By guessing & cleaning.Some spammers guess email addresses; send a test message (or a real spam) to a list which includes the guessed addresses. Then they wait for either an error message to return by email, indicating that the email address is correct, or for a confirmation. A confirmation could be solicited by inserting non-standard but commonly used mail headers requesting that the delivery system and/or mail client send a confirmation of delivery or reading. Specifically, the headers are -

Return-Receipt-To: which causes a delivery confirmation to be sent, and

X-Confirm-Reading-To: which causes a reading confirmation to be sent.Another method of confirming valid email addresses is sending HTML in the e-mail's body (that is sending a web page as the e-mail's content), and embedding in the HTML an image. Mail clients that decode HTML, e.g. as Outlook and Eudora do in the preview pane, will attempt fetching the image - and some spammers put the recipient's email address in the image's URL, and check the web server's log for the email addresses of recipients who viewed the spam.So it a good advice to set the mail client to *not* preview rich media emails, which would protect the recipient from both accidentally confirming their email addresses to spammers and viruses.Guessing could be done based on the fact that email addresses are based on people's names, usually in commonly used ways (first.last@domain or an initial of one name followed / preceded by the other @domain)Also, some email addresses are standard - postmaster is mandated by the RFCs for internet mail. Other common email addresses are postmaster, host master, root [for UNIX hosts], etc.

8. From white & yellow pages.There are various sites that serve as white pages, sometimes named people finders web sites. Yellow pages now have an email directory on the web.Those white/yellow pages contain addresses from various sources, e.g. from UseNet, but sometimes your E-mail address will be registered for you. Example - Hot Mail will add E-mail addresses to Bigfoot by default, making new addresses available to the public.Spammers go through those directories in order to get email addresses. Most directories prohibit email address harvesting by spammers, but as those databases have a large databases of email addresses + names, it's a tempting target for spammers.

9. By having access to the same computer.If a spammer has an access to a computer, he can usually get a list of valid usernames (and therefore email addresses) on that computer.On UNIX computers the users file (/etc/passwd) is commonly world readable, and the list of currently logged-in users is listed via the 'who' command.

10. From a previous owner of the email address.An email address might have been owned by someone else, who disposed of it. This might happen with dialup usernames at ISPs - somebody signs up for an ISP, has his/her email address harvested by spammers, and cancel the account. When somebody else signs up with the same ISP with the same username, spammers already know of it.Similar things can happen with AOL screen names - somebody uses a screen name, gets tired of it, releases it. Later on somebody else might take the same screen name.

11. Using social engineering.This method means the spammer uses a hoax to convince peopleinto giving him valid E-mail addresses.

12. A good example is Richard Douche's "Free CD's" chain letter. The letter promises a free CD for every person to whom the letter is forwarded to as long as it is CC'ed to Richard.Richard claimed to be associated with Amazon and Music blvd, among other companies, who authorized him to make this offer. Yet he supplied no references to web pages and used a free E-mail address.All Richard wanted was to get people to send him valid E-mail addresses in order to build a list of addresses to spam and/or sell.

13. From the address book and emails on other people's computers.Some viruses & worms spread by emailing themselves to all the email addresses they can find in the email address book. As some people forward jokes and other material by email to their friends, putting their friends' email addresses on either the To: or Cc: fields, rather than the BCc: field, some viruses and warms scan the mail folders for email addresses that are not in the address book, in hope to hit addresses the computer owner's friends' friends, friends' friends' friends, etc.If it wasn't already done, it's just a matter of time before such malware will not only spam copies of itself, but also send the extracted list of email addresses to it's creator.As invisible email addresses can't be harvested, it's good advice to have the email addresses of recipients of jokes & the like on BCc:, and if forwarded from somebody else remove from the e-mail's body all the email addresses inserted by the previous sender.

14. Buying lists from others.This one covers two types of trades. The first type consists of buying a list of email addresses (often on CD) that were harvested via other methods, e.g. someone harvesting email addresses from UseNet and sells the list either to a company that wishes to advertise via email (sometimes passing off the list as that of people who opted-in for emailed advertisements) or to others who resell the list.The second type consists of a company who got the email addresses legitimately (e.g. a magazine that asks subscribers for their email in order to keep in touch over the Internet) and sells the list for the extra income. This extends to selling of email addresses accompany got via other means, e.g. people who just emailed the company with inquiries in any context.

15. By hacking into sites.I've heard rumours that sites that supply free email addresses were hacked in order to get the list of email addresses, somewhat like e-commerce sites being hacked to get a list of credit cards.

My Idea

Still, we can protect their email addresses from harvesting programs. Here's how:

  • Consider "masking" your email address. Masking involves putting a word or phrase in your email address so that it will trick a harvesting computer program, but not a person. For example, if your email address is "johndoe@myisp.com," you could mask it as "johndoe@spamaway.myisp.com." Be aware that some newsgroup services or message boards won't allow you to mask your email address and some harvesting programs may be able to pick out common masks.
  • Use a separate screen name for chatting. If we use chat rooms, use a screen name that's not associated with our email address. Consider using the screen name only for online chat.
  • Set up disposable addresses. Decide if we want to use two email addresses - one for personal messages and one for posting in public. Consider using a disposable email address service that creates separate email addresses that forwards to your permanent account. If one of the disposable addresses begins to receive spam, we can shut it off without affecting your permanent address.
  • Use two email accounts. If we work for a business or organization that wants to receive email from the public, consider creating separate accounts or disposable email addresses for that purpose, rather than having an employee's address posted in public.
    • Use a unique email address, containing both letters and numbers. The choice of email address may affect the amount of spam we receive because some spammers use "dictionary attacks" to email many possible name combinations at large forwarding your spam to our ISP lets them know about the spam problem on their system and helps them to stop it. Include a copy of the spam, along with the full email header, and at the top of the message, state that I am complaining about being spammed.
    • The sender's ISP. Most ISPs want to cut off spammers who abuse their system. Include a copy of the message and header information and state that I complaining about spam.
    Conclusion

    In almost all instances, the investigators found, the spam received was not related to the address used. As a result, consumers who use email are exposed to a variety of spam - including objectionable messages - no matter the source of the address. According to research by the Federal Trade Commission (FTC) and several law enforcement partners, its harvest time for spammers. But, the consumer protection agency says, the good news for computer users is that we can minimize the amount of spam we receive.

    No comments: