Symantec.com
VERITAS.com
Partners
About Symantec
cartCart
WelcomeProducts & ServicesSecurity ResponseSupportSolutions & IndustriesLicensingTrainingStore
Enterprise
Symantec.com > Enterprise > Support > Knowledge Base


PRINT THIS PAGE PRINT THIS PAGE

Symantec Brightmail Gateway Effectiveness User’s Guide


Solution:
Introduction
This document provides an overview of antispam effectiveness issues, policies, and procedures related to Symantec Brightmail Gateway and other Symantec mail security products. It explains what messages should be captured as spam, what steps customers can take to communicate with us regarding effectiveness issues, and when those steps should be taken. Symantec always strives to improve its spam effectiveness over time, but it is to be expected that even Symantec’s industry leading antispam technology will miss some spam messages. The procedures outlined in this document explain what you should expect from Symantec technology and what to do if your expectations are not being met.

Effectiveness
Spam represents as much as 90% of all email sent across the Internet. The variance of this number is representative of different regions that are impacted more or less by spam senders, as well as the ever increasing deployment of IP-based solutions to deal with spam before it is allowed to reach an MTA. Symantec strives to maintain 97% anti-spam effectiveness for all spam. Anti-spam effectiveness is defined by the percentage of spam that is identified as spam by an anti-spam solution. This is separate from the ‘catch rate’ which is the measure of the percentage of all mail messages that have been identified as spam.

To illustrate this, consider a typical mail stream of 100 messages.
It is critical that customers do not confuse effectiveness and catch rate when considering the performance of Symantec Brightmail Gateway solutions.

Symantec uses multiple methods to measure its anti-spam effectiveness:
End User Expectations
End-user experience is typically what customers refer to when discussing spam filter effectiveness. No single inbox or small group of inboxes can by themselves be an accurate gauge for measuring overall spam filtering effectiveness. One end-user may find their experience to be poor, while another finds spam filtering to be very effective. Symantec, and other antispam vendors, cannot guarantee the same effectiveness for every end-user's experience, since different users receive different kinds and volumes of spam.

End-users also have different opinions as to what constitutes spam. The definition of spam is very subjective to most end-users. Many end-users define spam as simply unwanted email (including legitimate advertisements that they no longer wish to receive). Symantec defines spam as Unsolicited bulk email (includes Unsolicited Commercial Email).

Many end-users, customers and even analysts are actually referring to spam in a broader sense as all unwanted communication.

Symantec does not include the following in its definition of spam:
30-45% of all of missed spam reported by Symantec customer end-users is not spam according to Symantec’s definition.

Symantec’s antispam technology is focused on stopping true spam messages. Symantec also provides administrator and end-user tools to enable them to block unwanted messages. These tools include web based personal Allowed and Blocked Senders Lists.

Increased Spam Volume
If Symantec maintains the same effectiveness ratio (of spam caught vs. spam missed) but the total volume of spam increases, the end-user will experience a perceived drop in effectiveness. For example, one missed spam message out of ten total spam messages equates to 90% effectiveness. If the total volume of spam received increases from 10 spam messages to 100 spam messages, the effectiveness remains 90%. However the end-user perceives that the product is less effective, as there are now ten missed spam messages, compared to the one missed spam message previously. Therefore the volume of mail received by end-users is critical in understanding their perceived spam filtering effectiveness rate.

Symantec offers various solutions to reduce email volume, including the Symantec Mail Security 8160 appliance and the Symantec Brightmail Gateway appliances that dramatically reduce the overall mail volumes.

The unique system design of Symantec Mail Security 8160 helps to reduce the amount of unwanted email entering enterprise networks by analyzing the network's email flow and identifying the behavior of various network paths over time. The Symantec Mail Security 8160 identifies spammers by pinpointing the true source of each email and then limiting the bandwidth and resources that spamming sources can use, significantly decreasing the flow of spam. Using Transmission Control Protocol (TCP) traffic shaping at the TCP protocol level, the 8160 manages the quality of service that each email sender is given based on how likely it is that they are sending spam. Legitimate senders receive excellent quality of service and their mail flows quickly, while spammers are given very poor quality of service and their mail is slowed dramatically. Spammers have no way to force mail into your protected network, so their spam simply backs up on their own servers.

Symantec Brightmail Gateway appliances analyze incoming SMTP connections at the IP address level as well. Unlike the Symantec Mail Security 8160, Symantec Brightmail Gateway appliances consult only a local reputation database on the appliance, and then automatically defer or allow incoming connections based on the sending IP’s historical reputation. Whereas the 8160 manages connections at the TCP layer, Symantec Brightmail Gateway uses SMTP deferral to manage connections.

Steps to Follow if Seeing Increased Missed Spam
If spam effectiveness seems to have dropped, there are troubleshooting steps you can perform and information you can gather that can help determine where the issue may be. Please review your specific product documentation for details on how to investigate the following troubleshooting steps.

Use the following basic troubleshooting steps :
Installing Software Updates and Patches
Symantec mail security products have the ability to react to most new attacks via new filters that use existing technologies. However, over time, Symantec introduces new anti-spam technologies into its products to deliver new capability. It is critical that customers evaluate new versions of Symantec mail security products, since some new spam attacks can only be caught with them. If you are experiencing lower spam effectiveness, you should consider upgrading to the latest version of your Symantec technology. Customers should plan to deploy the latest release to ensure the highest levels of antispam effectiveness.

Missed Spam Submissions
If you have followed the troubleshooting and information gathering steps outlined above and determined that the increase in missed spam is not related to configuration or version issues, then you should consider making a missed spam submission. Missed spam submissions are used by Symantec for:
The Security Response Center must receive the messages within one day from the time they were initially sent. Since spammers rarely reuse old spam, Symantec does not write filters against messages older than 24 hours. The Security Response Center processes the received message using sophisticated algorithms. This process groups the message with other messages received from customers or through the extensive probe network. When a group reaches a threshold, it becomes an attack. At this point, the automation systems or a Security Response technician create a rule to respond to the attack. Adding the rule to the rule set completes the process. Your computer becomes protected when your rule set is updated.

However, due to the volume of submissions received (approximately 3 to 4 million messages per day), Security Response cannot guarantee that filters will be written for particular submissions. Because many submissions contain a forged sender address, they cannot provide feedback for submissions.

How End Users Submit Missed Spam
The customer creates an alias for the appropriate Symantec-Brightmail missed spam address:
Note: Only missed spam messages are sent to this address. If your deployment is over 50,000 users, then unique submission addresses for missed spam and FP’s can be created.
The missed spam must be sent as RFC-822 MIME encoded attachments in order for Symantec Security Response to process the mail. Information on submitting messages is available below (KB
2005012415180263):
http://service1.symantec.com/SUPPORT/ent-brightmailkb.nsf/docid/2005012415180263

Repeated Spam Attacks
Many spam messages look the same from the initial appearance, but contain many hidden characteristics to make the messages unique.

A few sample characteristics include:
If end users encounter multiple missed messages that seem to be related, they should report them to Symantec Security Response, following the procedures outlined above.

Effectiveness Escalations

Basic and Essential Support Customers
Basic and Essential Support customers should follow the steps outlined above to submit missed spam to Symantec Security Response. As stated above, there is no response to missed spam submissions.

Business Critical Customers
Symantec offers Business Critical customers an effectiveness escalation path.

Use the following procedure when a Business Critical customer experiences a significant increase in spam attacks, and requests additional analysis by the Customer Response team:
There is no Service Level Agreement for missed spam and/or effectives issues. Escalations are not handled during weekends or non-business hours outside of U.S Pacific Time.

Note: Customers of Symantec OEM's, 3rd party vendors, and/or appliance partners that are not direct Symantec Brightmail Gateway customers need to go through their vendor who can contact the appropriate support agent to assist in this process. Those customers should not contact Symantec Support directly.

Probe Accounts and the Probe Network
Symantec’s worldwide Probe Network™ is a vast collection of email accounts. The patented Probe Network is built on a base of over 2 million accounts donated by service provider and enterprise customers, as well as accounts owned by Symantec. It is one of the key reasons why Symantec Brightmail Gateway is the leading solution for accurately stopping spam.

Why the Probe Network is Important
The Probe Network is crucial to Symantec and its antispam customers for a variety of reasons:
Who should participate in the Symantec Probe Network?
Only customers exceeding 10,000 users may provide standard probes.

False Positives
Symantec Brightmail Gateway strives to maintain a false positive (FP) rate of less than one FP in one million messages scanned. Symantec utilizes several methodologies to determine our FP ratio with a conservative estimate to account for data that is not reported.

Product Evaluations
Evaluations of the product provide us with accurate lab and field results as the FP rate is monitored more closely by the prospective customers. This is also done by impartial 3rd parties that compare anti spam products against one another. Although this is anecdotal data, 95% of these evaluation tests return results that fall within the 1 in a million FP rate claimed by Symantec. The 5% that do not fall within this realm usually can be attributed to subjective differences in the definition of false positives (for example: forwarded spam messages caught at spam, spam discussion newsletters containing spam examples, 3rd party mailers sending inordinate volumes of spam, newsletters that practice spamming).

Field Data
The caveat with field data is that not 100% of end-users report FP’s. The other issue is that some customers elect to delete detected spam and therefore do not have the ability to report FP’s. Understanding this we can compare an aggregate FP ratio (used to establish a baseline) to a FP ratio of only those domains that reported FP’s. We compare the total number of reported legitimate FP’s to the total number of messages scanned. These numbers usually average to approximately 1 FP for every 20-35 million messages scanned.

How End Users Forward False Positives
The mail administrator creates an alias for the address:
Note: Only false positive messages are sent to these addresses. If the customer has more than 50,000 users, a unique submission address for missed spam and FP’s can be created End users send FULL HEADERS and BODY in the message as a RFC-822 MIME encoded attachment in order for Symantec to investigate and process the message. A copy of the message may also be forwarded to the customer’s Support Desk. Symantec investigates and adjusts filters, as necessary.

Business Critical Support Customers
Symantec offers Business Critical customers a critical false positive escalation path. Use the following procedure when a Business Critical customer experiences a critical false positive and requests additional analysis by the Security Response team:
Security Response typically responds to all False Positive submissions within 60 minutes though there might be times when a submission can take longer to process. No guarantees are made on how long it will take to have filters written or removed and available for updates.

Note: Customers of Symantec OEM's, 3rd party vendors, and/or appliance partners that are not direct Symantec Brightmail Gateway customers must go through their vendor who can contact the appropriate Technical Account Manager to assist in this process. Those customers should not contact Symantec
support directly.

Phishing
Symantec targets the largest phishing email threats with gateway email detection. The primary differences between phishing and spam are the ability of phishing attacks to be a) very small and b) difficult to distinguish from legitimate direct e-mail communications. Phishing attacks deployed using spamming techniques are readily detected and stopped, but attacks that are targeted and presumed legitimate are difficult to discern from actual communications from banks or credit card issuers. Symantec endeavors to be as effective against these threats as we are with spam by utilizing our premium antispam technology to capture them.

Contacts
Symantec has the following teams that handle different elements of our antispam offerings:


References:
http://service1.symantec.com/SUPPORT/ent-brightmailkb.nsf/docid/2005012415180263


Document ID: 2008100710321454
Last Modified: 06/18/2009
Date Created: 10/07/2008
Product(s): Brightmail Gateway, Brightmail Message Filter, Symantec Mail Security 8200 Series, Symantec Mail Security SMTP
Release(s): BG 2007 7.5, BG 2007 7.6, BG 2007 7.6.1, BG 7.7, BMF 6.0.0, Mail Security for SMTP 5.0.0, Mail Security for SMTP 5.0.1, SBAS 6.1


Site Index · Legal Notices · Privacy Policy · · Contact Us · Global Sites · License Agreements
©1995 - 2009 Symantec Corporation