How to use ? and ?: and : in REGEX for Python?

Question

I understand that

* = "zero or more"
? = "zero or more" ...what's the difference?

Also, ?: << my book uses this, it says its a "subtlety" but I don't know what exactly these do!

Answer 1

As Manu already said, ? means "zero or one time". It is the same as {0,1}.

And by ?:, you probably meant (?:X), where X is some other string. This is called a "non-capturing group". Normally when you wrap parenthesis around something, you group what is matched by those parenthesis. For example, the regex .(.).(.) matches any 4 characters (except line breaks) and stores the second character in group 1 and the fourth character in group 2. However, when you do: .(?:.).(.) only the fourth character is stored in group 1, everything bewteen (?:.) is matched, but not "remembered".

A little demo:

import re
m = re.search('.(.).(.)', '1234')
print m.group(1)
print m.group(2)
# output:
# 2
# 4

m = re.search('.(?:.).(.)', '1234')
print m.group(1)
# output:
# 4

You might ask yourself: "why use this non-capturing group at all?". Well, sometimes, you want to make an OR between two strings, for example, you want to match the string "www.google.com" or "www.yahoo.com", you could then do: www\.google\.com|www\.yahoo\.com, but shorter would be: www\.(google|yahoo)\.com of course. But if you're not going to do something useful with what is being captured by this group (the string "google", or "yahoo"), you mind as well use a non-capturing group: www\.(?:google|yahoo)\.com. When the regex engine does not need to "remember" the substring "google" or "yahoo" then your app/script will run faster. Of course, it wouldn't make much difference with relatively small strings, but when your regex and string(s) gets larger, it probably will.

And for a better example to use non-capturing groups, see Chris Lutz's comment below.