Ansible jenkins_plugin module returns "HTTP Error 403: No valid crumb was included in the request"

Question

I am using Ansible (v 2.8) as the provisioner behind a Packer template to build an AMI for a Jenkins master node. For previous versions, the playbook passed successfully. However, as of Jenkins version 2.176.3, the jenkins_plugin module has been throwing:

HTTP Error 403: No valid crumb was included in the request

I have retrieved the crumb and registered it in a variable. I have tried passing it to jenkins_plugin with the http_agent field, but that doesn't work. I tried using attributes, but that didn't help either. Unless I am missing something incredibly basic, I am at the end of my tether.

- name:               Get Jenkins Crumb
  uri:
    force_basic_auth: yes
    url_username:     ****
    url_password:     ****
    url:              http://localhost:8080/crumbIssuer/api/json
    return_content:   yes
  register:           jenkins_crumb
  until:              jenkins_crumb.content.find('Please wait while Jenkins is getting ready') == -1
  retries:            10
  delay:              5

- name:               Install plugin
  jenkins_plugin:
    name:             "{{ item }}"
    version:          latest
    force_basic_auth: yes
    url_username:     ****
    url_password:     ****
    http_agent:       "Jenkins-Crumb:{{ jenkins_crumb.json.crumb }}"
  with_items:         "{{ jenkins_plugins }}"

I expected installed plugins and a happily built AMI. What I got was "HTTP Error 403: No valid crumb was included in the request" and the Packer build failed.

Answer 1

Looks like a change to the crumb issuer in the 2.176 LTS release forces the inclusion of the web session id of the initial token generation call along with the crumb in subsequent calls that use said crumb.

CSRF tokens (crumbs) are now only valid for the web session they were created in to limit the impact of attackers obtaining them. Scripts that obtain a crumb using the /crumbIssuer/api URL will now fail to perform actions protected from CSRF unless the scripts retain the web session ID in subsequent requests.

In addition to the suggestion that you temporarily disable CSRF, the same doc suggests that you could only disable the new functionality, rather than CSRF as a whole, which should allow your packer/ansible to complete as it previously did, as-written.

To disable this improvement you can set the system property hudson.security.csrf.DefaultCrumbIssuer.EXCLUDE_SESSION_ID to true.

EDIT :

Adding the following line in /etc/default/jenkins cleared the CSRF issues in my own playbook (Ansible 2.8.4, Ubuntu 18.04, OpenJDK 11.0.4)

JAVA_ARGS="$JAVA_ARGS -Dhudson.security.csrf.DefaultCrumbIssuer.EXCLUDE_SESSION_ID=true"

Might be a good-enough crutch until tool maintainers catch up with the API changes.

Answer 2

It's exactly the cause @runningEagle mentioned. You need to propagate the initial session cookie value to all subsequent requests along with the crumb.

Required new Ansible code modifications:

...

# Requesting the crumb
uri:
  url: "<crumb_URL>"
register: response

...

# Actual action request
uri:
  url: "<action_URL>"
  headers: '{ ... , "Cookie": "{{ response.set_cookie }}", ... }'

...