Where to clean extra whitespace from form field inputs?

Question

I've just discovered that Django doesn't automatically strip out extra whitespace from form field inputs, and I think I understand the rationale ('frameworks shouldn't be altering user input').

I think I know how to remove the excess whitespace using python's re:

#data = re.sub('\A\s+|\s+\Z', '', data)
data = data.strip()
data = re.sub('\s+', ' ', data)

The question is where should I do this? Presumably this should happen in one of the form's clean stages, but which one? Ideally, I would like to clean all my fields of extra whitespace. If it should be done in the clean_field() method, that would mean I would have to have a lot of clean_field() methods that basically do the same thing, which seems like a lot of repetition.

If not the form's cleaning stages, then perhaps in the model that the form is based on?

Answer 1

My approach is borrowed from here. But instead of subclassing django.forms.Form, I use a mixin. That way I can use it with both Form and ModelForm. The method defined here overrides BaseForm's _clean_fields method.

class StripWhitespaceMixin(object):
    def _clean_fields(self):
        for name, field in self.fields.items():
            # value_from_datadict() gets the data from the data dictionaries.
            # Each widget type knows how to retrieve its own data, because some
            # widgets split data over several HTML fields.
            value = field.widget.value_from_datadict(self.data, self.files, self.add_prefix(name))

            try:
                if isinstance(field, FileField):
                    initial = self.initial.get(name, field.initial)
                    value = field.clean(value, initial)
                else:
                    if isinstance(value, basestring):
                        value = field.clean(value.strip())
                    else:
                        value = field.clean(value)
                self.cleaned_data[name] = value
                if hasattr(self, 'clean_%s' % name):
                    value = getattr(self, 'clean_%s' % name)()
                    self.cleaned_data[name] = value
            except ValidationError as e:
                self._errors[name] = self.error_class(e.messages)
                if name in self.cleaned_data:
                    del self.cleaned_data[name]

To use, simply add the mixin to your form

class MyForm(StripeWhitespaceMixin, ModelForm):
    ...

Also, if you want to trim whitespace when saving models that do not have a form you can use the following mixin. Models without forms aren't validated by default. I use this when I create objects based off of json data returned from external rest api call.

class ValidateModelMixin(object):
    def clean(self):
        for field in self._meta.fields:
            value = getattr(self, field.name)

            if value:
                # ducktyping attempt to strip whitespace
                try:
                    setattr(self, field.name, value.strip())
                except Exception:
                    pass

    def save(self, *args, **kwargs):
        self.full_clean()
        super(ValidateModelMixin, self).save(*args, **kwargs)

Then in your models.py

class MyModel(ValidateModelMixin, Model):
    ....

Answer 2

Create a custom model field so that your custom form field will be used automatically.

class TrimmedCharFormField(forms.CharField):
    def clean(self, value):
        if value:
            value = value.strip()
        return super(TrimmedCharFormField, self).clean(value)

# (If you use South) add_introspection_rules([], ["^common\.fields\.TrimmedCharField"])
class TrimmedCharField(models.CharField):
    __metaclass__ = models.SubfieldBase

    def formfield(self, **kwargs):
        return super(TrimmedCharField, self).formfield(form_class=TrimmedCharFormField, **kwargs)

Then in your models just replace django.db.models.CharField with TrimmedCharField