ModelForm clean_xxxx() works for CharField, not for URLField. Django 1.5

Question

How can I remove whitespace, prior to validation of a URLField?

Using "clean_[fieldname]()" would seem to be the documented way from https://docs.djangoproject.com/en/dev/ref/forms/validation/ , but it does not work for the URLField. I've reduced it to a basic test case which can be run in the django shell:

class XXXTestModel(models.Model):
    url  = models.URLField('URL',null=True,blank=True)
    name = models.CharField(max_length=200)
class XXXTestForm(ModelForm):
    def clean_url(self):
        return self.cleaned_data['url'].strip()
    def clean_name(self):
        return self.cleaned_data['name'].strip() 
    class Meta:
        model = XXXTestModel
        fields = (
             'url',
        )

Tested from the Django shell with:

>>> django.VERSION
(1, 5, 1, 'final', 0)
>>> from xxx import XXXTestForm,XXXTestModel
>>> data = dict(url=' http://www.example.com/ ',name=' example ')
>>> f=XXXTestForm(data)
>>> f.is_valid();f.errors
False
{'url': [u'Enter a valid URL.']}
>>> f.cleaned_data
{'name': example'}

There are a number of close dupes of this question on stack overflow, but none of the answers guide toward a solution.

Answer 1

The issue here is how the django.forms.URLField works.

django.forms.Field.clean is defined as:

def clean(self, value):
    """
    Validates the given value and returns its "cleaned" value as an
    appropriate Python object.

    Raises ValidationError for any errors.
    """
    value = self.to_python(value)
    self.validate(value)
    self.run_validators(value)
    return value

Note that to_python is performed before any validation. This is the issue here - django.forms.URLField can't understand the value you're giving it, so the value it produces fails the set of validators already defined as part of django.forms.URLField (namely, django.core.validators.URLValidator).

The reason it fails is django tries to "normalize" the URL. This includes things such as adding "http://" where needed. When given your example url, " http://www.example.com ", django uses urlparse.urlsplit to get it "parts" of the url. The leading space, however, messes it up and the entire value becomes part of the path. As such, django finds no scheme, and reconstitutes the URL as "http:// http://www.example.com ". This is then given to django.core.validators.URLValidator, which obviously fails.

To avoid this, we'll need to define our own URLField for our form

from django import forms

class StrippedURLField(forms.URLField):
    def to_python(self, value):
        return super(StrippedURLField, self).to_python(value and value.strip())

Using this ensures the process will all go as expected, and we wont need a clean_url method. (note: you should use clean_* where possible, but here it is not)

class XXXTestForm(forms.ModelForm):
    url = StrippedURLField(blank=True, null=True)