How to debug a Django MultiValueDictKeyError on Formset POST

Question

When I post my formset, I get a MultiValueDictKeyError. Specifically:

MultiValueDictKeyError at /core/customers/1/update/documents/
"Key u'documents-0-attachment_ptr' not found in <QueryDict: {u'documents-1-last_modified_date': [u''], u'documents-1-name': [u''], u'documents-MAX_NUM_FORMS': [u''], u'documents-0-attachment_file': [u''], u'documents-INITIAL_FORMS': [u'1'], u'documents-1-document_type': [u''], u'documents-0-notes': [u''], u'documents-1-notes': [u''], u'submit': [u'Submit changes'], u'documents-0-DELETE': [u'on'], u'documents-1-attachment_file': [u''], u'documents-0-document_type': [u''], u'documents-TOTAL_FORMS': [u'2'], u'documents-0-name': [u'test'], u'documents-1-creation_date': [u''], u'documents-0-creation_date': [u'2012-12-01 23:41:48'], u'csrfmiddlewaretoken': [u'NCQ15jA7erX5dAbx20Scr3gWxgaTn3Iq', u'NCQ15jA7erX5dAbx20Scr3gWxgaTn3Iq', u'NCQ15jA7erX5dAbx20Scr3gWxgaTn3Iq'], u'documents-0-last_modified_date': [u'2012-12-01 23:41:48']}>"

The key part is that Django is looking for the key documents-0-attachment_ptr in the post data. This is confusing -- a Document is a subclass of an Attachment. All of the other post data is as expected. Why is Django needing pointer data in my formset?

Here is the form used in the formset:

class DocumentInlineForm(forms.ModelForm):  # pylint: disable=R0924
    attachment_file = forms.FileField(widget=NoDirectoryClearableFileInput)
    notes = forms.CharField(
        required=False,
        widget=forms.Textarea(attrs={'rows': 2,}), 
    )
    helper = DocumentInlineFormHelper()

    class Meta: # pylint: disable=W0232,R0903
        fields = (
            'attachment_file', 
            'creation_date',
            'document_type',
            'last_modified_date',
            'name',
            'notes',
        )
        model = Document

And here is the Document model:

"""
Handles document model definitions.
"""
from django.db import models
from eee_core.models.attachments import Attachment
from django.db.models.signals import pre_save
from datetime import datetime
from django.utils.timezone import utc

class Document(Attachment):
    """
    A document is an attachment with additional meta data.
    """
    creation_date = models.DateTimeField(
        blank=True, 
        null=True,
    )
    document_type = models.CharField(
        blank=True, 
        choices=(
            ('CONTRACT', 'Contract'),
            ('INVOICE', 'Invoice'),
            ('FACILITY', 'Facility change form'),
            ('LOA', 'Letter of authorization'),
            ('USAGE', 'Usage history document'),
            ('OTHER', 'Other'),
        ),
        default=None, 
        null=True, 
        max_length=8, 
    )
    last_modified_date = models.DateTimeField(
        blank=True, 
        null=True,
    )
    notes = models.TextField(
        blank=True,
        null=True,
    )

    class Meta(Attachment.Meta): # pylint: disable=W0232,R0903
        """
        Sets meta fields for model.
        """
        app_label = 'core'

    def __str__(self):
        return unicode(self).encode('utf-8')

    def __unicode__(self):
        return unicode(self.name)

def pre_save_callback(sender, instance, *args, **kwargs): # pylint: disable=W0613
    if not isinstance(instance, Document):
        return

    if not instance.creation_date:
        instance.creation_date = datetime.utcnow().replace(tzinfo=utc)

    instance.last_modified_date = datetime.utcnow().replace(tzinfo=utc)

pre_save.connect(pre_save_callback, dispatch_uid='document_pre_save')

Additional info:

Curiously, the inital post of the formset works fine. It is only on update posts -- when there are initial forms in the formset -- when I get this error. It also happens when I try to delete forms from the formset.

Also, the formset is a generic inline formset using django crispy forms.

Update

There was a request for the template code used. Here is the simplified version:

{% load crispy_forms_tags %}
{% load url from future %}
<form action="" method="post" enctype="multipart/form-data">
    {{ formset.management_form }}
    {% for subform in formset.forms %}
      {{ subform.id }}
      {% crispy subform %}
    {% endfor %}
    <div class="btn-toolbar">
        <input class='btn btn-primary' type="submit" name="submit" value="Submit changes" />
    </div>
</form>

Answer 1

This isn't the case with OP, but you will encounter a MultiValueDictKeyError, if some of the hidden fields are missing in the template. It may happen when instead of quick and dirty {{form}}, fields are listed in the template one by one: {{form.field1}}, {{form.field2}}, while leaving out required hidden fields.

To include them back do something along the lines (for each form/form in formset):

{% for hidden in form.hidden_fields %}
    {{ hidden }}
{% endfor %}

or

{% for form in formset %}    
    {% for hidden in form.hidden_fields %}
        {{ hidden }}
    {% endfor %}    
{% endfor %}

Answer 2

I use also

{% for hidden in form.hidden_fields %}
    {{ hidden }}
{% endfor %}

like this one

<div role="tabpanel" class="tab-pane active" id="email">
                {% csrf_token %}
                {{ eformset.management_form}}
                    <div class="panel panel-default">
                        <div class="panel-body">
                            <div id="addemail" class="btn btn-success">
                                <span class="glyphicon glyphicon-plus" > 
                                </span>
                            </div>
                            <p><br></p>
                            {% for f in eformset %}
                                {% for hidden in f.hidden_fields %}
                                    {{ hidden }}
                                {% endfor %}
                                <div class="item_email_set">
                                    <table class="table table-condensed table-bordered">
                                    <tr>
                                        {% for field in f.visible_fields %} <!---->
                                            <td>
                                                {{ field.label }}
                                            </td>
                                        {% endfor %}

                                        <td>
                                        </td>
                                    </tr>
                                    <tr>
                                        {% for field in f.visible_fields %}
                                            <td>
                                                {{field.errors.as_ul}}
                                                {{field}}
                                            </td>
                                        {% endfor %}    
                                        <td class="btncolumn">      
                                            <p style="">
                                                <a class="delete_email_set" href="#">
                                                    <div  class="btn btn-danger">
                                                        <span class="glyphicon glyphicon-remove" > 
                                                        </span>
                                                    </div>
                                                </a>
                                            </p>
                                        </td>
                                    </tr>   

                                </table>
                            </div>  
                        {% endfor %}
                    </div>
                </div>  
            </div>

and I resolve the MultiValueDictKeyError