email that my application sends is getting spammed: what's wrong with my headers?

Question

I'm sending out emails via my PHP application. However, they're getting marked as spam by Gmail. Here's how I'm sending the email (PHP):

$headers = "From: [email protected]\r\nReply-To: [email protected]";
$mail_sent = mail( '[email protected]', 'test mail', $message, $headers, '[email protected]' );

Gmail spams this message. So I went and clicked that handy "show original message" option. Here's what I get:

Delivered-To: [email protected]
Received: by 10.68.71.200 with SMTP id x8cs325812pbu;
        Thu, 21 Jul 2011 01:34:52 -0700 (PDT)
Received: by 10.236.114.234 with SMTP id c70mr12483739yhh.163.1311237292052;
        Thu, 21 Jul 2011 01:34:52 -0700 (PDT)
Return-Path: <[email protected]>
Received: from vps.bookmytakeout.com ([8.22.200.47])
        by mx.google.com with ESMTPS id u61si3662037yhm.119.2011.07.21.01.34.50
        (version=TLSv1/SSLv3 cipher=OTHER);
        Thu, 21 Jul 2011 01:34:51 -0700 (PDT)
Received-SPF: neutral (google.com: 8.22.200.47 is neither permitted nor denied by best guess record for domain of [email protected]) client-ip=8.22.200.47;
DomainKey-Status: bad format
Authentication-Results: mx.google.com; spf=neutral (google.com: 8.22.200.47 is neither permitted nor denied by best guess record for domain of [email protected]) [email protected]; domainkeys=neutral (bad format) [email protected]
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=default; d=bookmytakeout.com;
    h=Received:To:Subject:From:Reply-To:Message-Id:Date;
    b=WYWQ+/9+wOAvq+OUSM5KLDAueciIoNiByXaVV29HYa0XbSwm2f+89TCj8pW24G7k1vTGCvR8n64iSwgPQuoEitz6ehbszd0+75Px0WlGsvyeZGrW3PaYEpkRFmkDoaGe;
Received: from munged by vps.bookmytakeout.com with local (Exim 4.69)
    (envelope-from <[email protected]>)
    id 1QjolW-0001Vn-Us
    for [email protected]; Thu, 21 Jul 2011 14:07:31 +0530
To: [email protected]
Subject: test mail
From: [email protected]
Reply-To: [email protected]
Message-Id: <[email protected]>
Date: Thu, 21 Jul 2011 14:07:30 +0530
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname - vps.bookmytakeout.com
X-AntiAbuse: Original Domain - gmail.com
X-AntiAbuse: Originator/Caller UID/GID - [1005 1000] / [47 12]
X-AntiAbuse: Sender Address Domain - bookmytakeout.com

test

Now, I have no idea how to diagnose what could be setting off gmail's spam filters. Can someone please point out what part of this email is setting off the spam filter?

If possible, please post a solution as well. I'm more interested in learning what's wrong with the headers than in how to fix this for now.

PS: I have a few suspicions of my own:

1. the Received header says "vps.bookmytakeout.com" but the From header says "bookmytakeout.com" - but I tried sending it with From as [email protected] - same problem, still spammed.
2. The headers "Received-SPF", "DomainKey-Status" and "Authentication-Results" seem to indicate some problem. I distinctly remember NOT setting up any MX records for this domain name. Could that be the issue?

Answer 1

I guess you didn't publish any SPF / DKIM authentication record

Received-SPF: neutral (google.com: 8.22.200.47 is neither permitted nor denied by best guess record for domain of [email protected]) client-ip=8.22.200.47; DomainKey-Status: bad format

Most of the time, Gmail (as well as the other major ISPs) will place the non-authenticated messages in the junk folder.

An SPF record or a DKIM guarantees that you are allowed to use a certain domain as a sender. For example, if you don't own "paypal.com", you can't send an email from "[email protected]". If you've published the right SPF / DKIM, the ISP will consider you as a trusted sender.

SPF & DKIM were first designed to fight against phishing.

DIY SOLUTION: Publish some records by following these instructions: http://dkim.org/specs/rfc5585.html http://www.openspf.org/FAQ

EASY SOLUTION: Use a service that will do everything for you. A good ESP will usually sign your emails with DKIM / SPF by default. The problem is that you will sometimes get a "sent via ESP_NAME" mention (in Gmail).

So the best thing to do is to choose an ESP which will provide you personalized DKIM & SPF. This way, it will be 100% transparent.

I work for Mailjet and we offer this service for free. Most of our competitors offer this as an option.

Here's a useful post about this "via-mention" you get when your ESP signs "by default" and how to get rid of it. http://blog.mailjet.com/post/16922561593/personalized-spf-dkim