How do implicit joined columns work with Android contacts data?

Question

I'm querying the ContactsContract.Data table to find phone records.

I get an error when I create a new CursorLoader:

java.lang.IllegalArgumentException: Invalid column deleted

My code:

import android.provider.ContactsContract.CommonDataKinds.Phone;
import android.provider.ContactsContract.Data;

...

String[] projection = {
    Phone.DELETED,
    Phone.LOOKUP_KEY,
    Phone.NUMBER,
    Phone.TYPE,
    Phone.LABEL,
    Data.MIMETYPE,
    Data.DISPLAY_NAME_PRIMARY
};

// "mimetype = ? AND deleted = ?"
String selection = Data.MIMETYPE + " = ? AND " Phone.DELETED + " = ?";  
String[] args = {Phone.CONTENT_ITEM_TYPE, "0"};

return new CursorLoader(
    this,
    Data.CONTENT_URI,
    projection,
    selection,
    args,
    null);

Any idea why the Phone.DELETED column isn't included in the cursor? The documentation does say -

Some columns from the associated raw contact are also available through an implicit join.

Answer 1

Looks like you've found a feature that has been documented in many places, but hadn't been implemented yet. I opened a bug for tracking this issue - lets see what AOSP guys have to say on the subject (bug report).

Meanwhile, you can use the following workaround:

Uri uri = ContactsContract.RawContactsEntity.CONTENT_URI;

String[] projection = {
    Phone._ID,
    Phone.DELETED,
    //Phone.LOOKUP_KEY,
    Phone.NUMBER,
    Phone.TYPE,
    Phone.LABEL,
    Data.MIMETYPE,
    Data.DISPLAY_NAME_PRIMARY
};

String selection = Data.MIMETYPE + " = ? AND " + Data.DELETED + " = ?";
String[] args = {
    Phone.CONTENT_ITEM_TYPE, "0"
};

return new CursorLoader(
this,
uri,
projection,
selection,
args,
null);

Changes:

1. Use RawContactsEntity's URI
2. LOOKUP_KEY is not accessible via above URI - you'll have to execute additional query if you absolutely need this column
3. _ID column will be required if you are going to use the resulting Cursor in CursorAdapter.

Edit: following @MichaelAlanHuff's request I'm posting the parts of code which this answer is based upon

From com.android.providers.contacts.ContactsProvider2#queryLocal() (source code of ContactsProvider2):

protected Cursor queryLocal(final Uri uri, final String[] projection, String selection,
String[] selectionArgs, String sortOrder, final long directoryId,
final CancellationSignal cancellationSignal) {


    final SQLiteDatabase db = mDbHelper.get().getReadableDatabase();

    SQLiteQueryBuilder qb = new SQLiteQueryBuilder();
    String groupBy = null;
    String having = null;
    String limit = getLimit(uri);
    boolean snippetDeferred = false;

    // The expression used in bundleLetterCountExtras() to get count.
    String addressBookIndexerCountExpression = null;

    final int match = sUriMatcher.match(uri);
    switch (match) {


        ...

        case DATA:
        case PROFILE_DATA:
            {
                final String usageType = uri.getQueryParameter(DataUsageFeedback.USAGE_TYPE);
                final int typeInt = getDataUsageFeedbackType(usageType, USAGE_TYPE_ALL);
                setTablesAndProjectionMapForData(qb, uri, projection, false, typeInt);
                if (uri.getBooleanQueryParameter(Data.VISIBLE_CONTACTS_ONLY, false)) {
                    qb.appendWhere(" AND " + Data.CONTACT_ID + " in " + Tables.DEFAULT_DIRECTORY);
                }
                break;
            }


            ...

    }



    qb.setStrict(true);

    // Auto-rewrite SORT_KEY_{PRIMARY, ALTERNATIVE} sort orders.
    String localizedSortOrder = getLocalizedSortOrder(sortOrder);
    Cursor cursor = query(db, qb, projection, selection, selectionArgs, localizedSortOrder, groupBy,
    having, limit, cancellationSignal);

    if (readBooleanQueryParameter(uri, Contacts.EXTRA_ADDRESS_BOOK_INDEX, false)) {
        bundleFastScrollingIndexExtras(cursor, uri, db, qb, selection,
        selectionArgs, sortOrder, addressBookIndexerCountExpression,
        cancellationSignal);
    }
    if (snippetDeferred) {
        cursor = addDeferredSnippetingExtra(cursor);
    }

    return cursor;


}

As you can see, there are two additional methods where SQLiteQueryBuilder used to build the query could be changed: setTablesAndProjectionMapForData() and additional query() method.

Source of com.android.providers.contacts.ContactsProvider2#setTablesAndProjectionMapForData():

private void setTablesAndProjectionMapForData(SQLiteQueryBuilder qb, Uri uri,
        String[] projection, boolean distinct, boolean addSipLookupColumns, Integer usageType) {
    StringBuilder sb = new StringBuilder();
    sb.append(Views.DATA);
    sb.append(" data");

    appendContactPresenceJoin(sb, projection, RawContacts.CONTACT_ID);
    appendContactStatusUpdateJoin(sb, projection, ContactsColumns.LAST_STATUS_UPDATE_ID);
    appendDataPresenceJoin(sb, projection, DataColumns.CONCRETE_ID);
    appendDataStatusUpdateJoin(sb, projection, DataColumns.CONCRETE_ID);

    appendDataUsageStatJoin(
            sb, usageType == null ? USAGE_TYPE_ALL : usageType, DataColumns.CONCRETE_ID);

    qb.setTables(sb.toString());

    boolean useDistinct = distinct || !ContactsDatabaseHelper.isInProjection(
            projection, DISTINCT_DATA_PROHIBITING_COLUMNS);
    qb.setDistinct(useDistinct);

    final ProjectionMap projectionMap;
    if (addSipLookupColumns) {
        projectionMap =
                useDistinct ? sDistinctDataSipLookupProjectionMap : sDataSipLookupProjectionMap;
    } else {
        projectionMap = useDistinct ? sDistinctDataProjectionMap : sDataProjectionMap;
    }

    qb.setProjectionMap(projectionMap);
    appendAccountIdFromParameter(qb, uri);
}

Here you see the construction of table argument of the final query using StringBuilder which is being passed to several append*() methods. I'm not going to post their source code, but they really join the tables that appear in methods' names. If rawContacts table would be joined in, I'd expect to see a call to something like appendRawContactJoin() here...

For completeness: the other query() method that I mentioned does not modify table argument:

private Cursor query(final SQLiteDatabase db, SQLiteQueryBuilder qb, String[] projection,
        String selection, String[] selectionArgs, String sortOrder, String groupBy,
        String having, String limit, CancellationSignal cancellationSignal) {
    if (projection != null && projection.length == 1
            && BaseColumns._COUNT.equals(projection[0])) {
        qb.setProjectionMap(sCountProjectionMap);
    }
    final Cursor c = qb.query(db, projection, selection, selectionArgs, groupBy, having,
            sortOrder, limit, cancellationSignal);
    if (c != null) {
        c.setNotificationUri(getContext().getContentResolver(), ContactsContract.AUTHORITY_URI);
    }
    return c;
}

The inspection of the above chain of methods led me to the conclusion that there is an officially documented feature which is not implemented.