How to solve the "remote: You are not allowed to upload code." error on GitLab CI/CD job?

Question

I am currently trying to use GitLab to run a CI/CD job that runs a Python file that makes changes to a particular repository and then commits and pushes those changes to master. I also have a role of Master in the repository. It appears that all git functions run fine except for the git push, which leads to fatal: You are not currently on a branch. and with using git push origin HEAD:master --force, that leads to fatal: unable to access 'https://gitlab-ci-token:xxx@xxx/project.git/': The requested URL returned error: 403. I've been looking over solutions online, one being this one, and another being unprotecting it, and couldn't quite find what I was looking for just yet. This is also a sub-project within the GitLab repository.

Right now, this is pretty much what my .gitlab-ci.yml looks like.

before_script:
  - apt-get update -y
  - apt-get install git -y
  - apt-get install python -y
  - apt-get python-pip -y

main:
  script:
    - git config --global user.email "xxx@xxx"
    - git config --global user.name "xxx xxx"
    - git config --global push.default simple
    - python main.py

My main.py file essentially has a function that creates a new file within an internal directory provided that it doesn't already exist. It has a looks similar to the following:

import os
import json

def createFile(strings):
    print ">>> Pushing to repo...";
    if not os.path.exists('files'):
        os.system('mkdir files');
    for s in strings:
        title = ("files/"+str(s['title'])+".json").encode('utf-8').strip();
        with open(title, 'w') as filedata:
            json.dump(s, filedata, indent=4);
    os.system('git add files/');
    os.system('git commit -m "Added a directory with a JSON file in it..."');
    os.system('git push origin HEAD:master --force');

createFile([{"title":"A"}, {"title":"B"}]);

I'm not entirely sure why this keeps happening, but I have even tried to modify the repository settings to change from protected pull and push access, but when I hit Save, it doesn't actually save. Nonetheless, this is my overall output. I would really appreciate any guidance any can offer.

 Running with gitlab-runner 10.4.0 (00000000)
      on cicd-shared-gitlab-runner (00000000)
 Using Kubernetes namespace: cicd-shared-gitlab-runner
 Using Kubernetes executor with image ubuntu:16.04 ...
 Waiting for pod cicd-shared-gitlab-runner/runner-00000000-project-00000-concurrent-000000 to be running, status is Pending
 Waiting for pod cicd-shared-gitlab-runner/runner-00000000-project-00000-concurrent-000000 to be running, status is Pending
 Running on runner-00000000-project-00000-concurrent-000000 via cicd-shared-gitlab-runner-0000000000-00000...
 Cloning repository...
 Cloning into 'project'...
 Checking out 00000000 as master...
 Skipping Git submodules setup
 $ apt-get update -y >& /dev/null
 $ apt-get install git -y >& /dev/null
 $ apt-get install python -y >& /dev/null
 $ apt-get install python-pip -y >& /dev/null
 $ git config --global user.email "xxx@xxx" >& /dev/null
 $ git config --global user.name "xxx xxx" >& /dev/null
 $ git config --global push.default simple >& /dev/null
 $ python main.py
 [detached HEAD 0000000] Added a directory with a JSON file in it...
  2 files changed, 76 insertions(+)
  create mode 100644 files/A.json
  create mode 100644 files/B.json
 remote: You are not allowed to upload code.
 fatal: unable to access 'https://gitlab-ci-token:xxx@xxx/project.git/': The requested URL returned error: 403
 HEAD detached from 000000
 Changes not staged for commit:
    modified:   otherfiles/otherstuff.txt
 no changes added to commit
 remote: You are not allowed to upload code.
 fatal: unable to access 'https://gitlab-ci-token:xxx@xxx/project.git/': The requested URL returned error: 403
 >>> Pushing to repo...
 Job succeeded

Answer 1

Here is a resource from Gitlab that describes how to make commits to the repository within the CI pipeline: https://gitlab.com/guided-explorations/gitlab-ci-yml-tips-tricks-and-hacks/commit-to-repos-during-ci/commit-to-repos-during-ci

Try configuring your gitlab-ci.yml file to push the changes rather than trying to do it from the python file.

Answer 2

I managed to do this via ssh on a runner by making sure the ssh key is added, and then using the full git url:

task_name:
  stage: some_stage
  script:
    - ssh-add -K ~/.ssh/[ssh key]
    - git push -o ci-skip [email protected]:[path to repo].git HEAD:[branch name]

If it is the same repo that triggered the job, the url could also be written as:

git@$CI_SERVER_HOST:$CI_PROJECT_PATH.git