What is the correct use of $and operator in pymongo?

Question

I have this structure:

>>>test_3.find_one({"humsavar.Disease": {"$exists": True}}, 
{"humsavar":True, "_id":False})

{u'humsavar': [{u'Association': u'Polymorphism',
   u'Disease': u'-',
   u'Gene names': u'DTWD1',
   u'Mutate aa': u'Pro',
   u'Position aa': 9,
   u'Reference aa': u'Leu',
   u'Substitution': u'Leu9Pro',
   u'SwissVarID': u'VAR_036757',
   u'Uniprot': u'Q8N5C7',
   u'dbSNP': u'rs11539522'},
  {u'Association': u'Polymorphism',
   u'Disease': u'Pyruvate dehydrogenase lipoic acid synthetase deficiency',
   u'Gene names': u'DTWD1',
   u'Mutate aa': u'Lys',
   u'Position aa': 13,
   u'Reference aa': u'Glu',
   u'Substitution': u'Glu13Lys',
   u'SwissVarID': u'VAR_036758',
   u'Uniprot': u'Q8N5C7',
   u'dbSNP': u'rs11539519'}]}

Should I search with the following query to count all the documents from humsavar with a Disease and dbSNP?

test_3.find({"$and": [{"humsavar.Disease": {"$ne": u'-', "$exists":  True}},
 {"humsavar.dbSNP": {"$ne": u'-', "$ne": None, "$exists":  True}}]},
{"humsavar":True, "_id": False}).count()
# output 32

I would expect a similar number with this query:

test_3.find({"$and": [{"humsavar.Disease": {"$ne": u'-', "$ne":None, "$exists":  True}},
 {"humsavar.dbSNP": {"$ne": u'-', "$ne": None, "$exists":  True}}]},
 {"humsavar":True, "_id": False}).count()

But turn out to be 8499

Answer 1

Your last query returns too many documents, because in python you cannot really include duplicate keys in a dict, like that:

{"$ne": u'-', "$ne":None, "$exists":  True}

This causes the second occurence of "$ne" to overwrite the first, ending with a dict of

{"$ne":None, "$exists":  True}

This all happens in the python-interpreter layer, before being passed to the pymongo driver.

If you want multiple $ne conditions on a single field, you can use the $nin ("not in") operator instead.