What is the meaning of single and double underscore before an object name?

Question

Single Underscore

Names, in a class, with a leading underscore are simply to indicate to other programmers that the attribute or method is intended to be private. However, nothing special is done with the name itself.

To quote PEP-8:

_single_leading_underscore: weak "internal use" indicator. E.g. from M import * does not import objects whose name starts with an underscore.

Double Underscore (Name Mangling)

From the Python docs:

Any identifier of the form __spam (at least two leading underscores, at most one trailing underscore) is textually replaced with _classname__spam, where classname is the current class name with leading underscore(s) stripped. This mangling is done without regard to the syntactic position of the identifier, so it can be used to define class-private instance and class variables, methods, variables stored in globals, and even variables stored in instances. private to this class on instances of other classes.

And a warning from the same page:

Name mangling is intended to give classes an easy way to define “private” instance variables and methods, without having to worry about instance variables defined by derived classes, or mucking with instance variables by code outside the class. Note that the mangling rules are designed mostly to avoid accidents; it still is possible for a determined soul to access or modify a variable that is considered private.

Example

>>> class MyClass():
...     def __init__(self):
...             self.__superprivate = "Hello"
...             self._semiprivate = ", world!"
...
>>> mc = MyClass()
>>> print mc.__superprivate
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: myClass instance has no attribute '__superprivate'
>>> print mc._semiprivate
, world!
>>> print mc.__dict__
{'_MyClass__superprivate': 'Hello', '_semiprivate': ', world!'}

__foo__: this is just a convention, a way for the Python system to use names that won't conflict with user names.

_foo: this is just a convention, a way for the programmer to indicate that the variable is private (whatever that means in Python).

__foo: this has real meaning: the interpreter replaces this name with _classname__foo as a way to ensure that the name will not overlap with a similar name in another class.

No other form of underscores have meaning in the Python world.

There's no difference between class, variable, global, etc in these conventions.

Excellent answers so far but some tidbits are missing. A single leading underscore isn't exactly just a convention: if you use from foobar import *, and module foobar does not define an __all__ list, the names imported from the module do not include those with a leading underscore. Let's say it's mostly a convention, since this case is a pretty obscure corner;-).

The leading-underscore convention is widely used not just for private names, but also for what C++ would call protected ones -- for example, names of methods that are fully intended to be overridden by subclasses (even ones that have to be overridden since in the base class they raise NotImplementedError!-) are often single-leading-underscore names to indicate to code using instances of that class (or subclasses) that said methods are not meant to be called directly.

For example, to make a thread-safe queue with a different queueing discipline than FIFO, one imports Queue, subclasses Queue.Queue, and overrides such methods as _get and _put; "client code" never calls those ("hook") methods, but rather the ("organizing") public methods such as put and get (this is known as the Template Method design pattern -- see e.g. here for an interesting presentation based on a video of a talk of mine on the subject, with the addition of synopses of the transcript).

Edit: The video links in the description of the talks are now broken. You can find the first two videos here and here.

._variable is semiprivate and meant just for convention

.__variable is often incorrectly considered superprivate, while it's actual meaning is just to namemangle to prevent accidental access[1]

.__variable__ is typically reserved for builtin methods or variables

You can still access .__mangled variables if you desperately want to. The double underscores just namemangles, or renames, the variable to something like instance._className__mangled

Example:

class Test(object):
    def __init__(self):
        self.__a = 'a'
        self._b = 'b'

>>> t = Test()
>>> t._b
'b'

t._b is accessible because it is only hidden by convention

>>> t.__a
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'Test' object has no attribute '__a'

t.__a isn't found because it no longer exists due to namemangling

>>> t._Test__a
'a'

By accessing instance._className__variable instead of just the double underscore name, you can access the hidden value

Single underscore at the beginning:

Python doesn't have real private methods. Instead, one underscore at the start of a method or attribute name means you shouldn't access this method, because it's not part of the API.

class BaseForm(StrAndUnicode):

    def _get_errors(self):
        "Returns an ErrorDict for the data provided for the form"
        if self._errors is None:
            self.full_clean()
        return self._errors

    errors = property(_get_errors)

(This code snippet was taken from django source code: django/forms/forms.py). In this code, errors is a public property, but the method this property calls, _get_errors, is "private", so you shouldn't access it.

Two underscores at the beginning:

This causes a lot of confusion. It should not be used to create a private method. It should be used to avoid your method being overridden by a subclass or accessed accidentally. Let's see an example:

class A(object):
    def __test(self):
        print "I'm a test method in class A"

    def test(self):
        self.__test()

a = A()
a.test()
# a.__test() # This fails with an AttributeError
a._A__test() # Works! We can access the mangled name directly!

Output:

$ python test.py
I'm test method in class A
I'm test method in class A

Now create a subclass B and do customization for __test method

class B(A):
    def __test(self):
        print "I'm test method in class B"

b = B()
b.test()

Output will be....

$ python test.py
I'm test method in class A

As we have seen, A.test() didn't call B.__test() methods, as we might expect. But in fact, this is the correct behavior for __. The two methods called __test() are automatically renamed (mangled) to _A__test() and _B__test(), so they do not accidentally override. When you create a method starting with __ it means that you don't want to anyone to be able to override it, and you only intend to access it from inside its own class.

Two underscores at the beginning and at the end:

When we see a method like __this__, don't call it. This is a method which python is meant to call, not you. Let's take a look:

>>> name = "test string"
>>> name.__len__()
11
>>> len(name)
11

>>> number = 10
>>> number.__add__(40)
50
>>> number + 50
60

There is always an operator or native function which calls these magic methods. Sometimes it's just a hook python calls in specific situations. For example __init__() is called when the object is created after __new__() is called to build the instance...

Let's take an example...

class FalseCalculator(object):

    def __init__(self, number):
        self.number = number

    def __add__(self, number):
        return self.number - number

    def __sub__(self, number):
        return self.number + number

number = FalseCalculator(20)
print number + 10      # 10
print number - 20      # 40

For more details, see the PEP-8 guide. For more magic methods, see this PDF.