What is resetting the PATH variable at the last second during an OpenShift v2 push hook?

Question

TL;DR: Working app, cloned it, clone doesn't start correctly from push hook (but works fine manually if I ssh in.) PATH has correct Node version added to it, but somewhere right in the last step, the incorrect Node version is prepended to the PATH again.

Path is correct here:

remote:     PATH = /var/lib/openshift/.../app-root/data//node-v4.x.x-linux-x64/bin:/var/lib/openshift/.../app-root/runtime/repo/node_modules/.bin:/var/lib/openshift/...//.node_modules/.bin:/opt/rh/nodejs010/root/usr/bin:/bin:/usr/bin:/usr/sbin

Then incorrect immediately after, somewhere in here:

remote: Starting NodeJS cartridge
remote: Tue Aug 22 2017 15:39:10 GMT-0400 (EDT): Starting application 'staging' ...

So what scripts and hooks are represented in or before these last two lines? PATH isn't just adding lines to itself...

---

I have a working OpenShift v2 app running a NodeJS version new enough to support fat arrow notation.

It appears that it was set up per Custom node.js version on Openshift because the scripts from that repo (for using a marker file) are present in .openshift.

I set up a second one using rhc create --from-app based on the working one, reset the repo, then re-deployed onto it. The second one worked great, except for the final step of starting node:

remote: npm info ok 
remote: NOTE: The .openshift/action_hooks/build hook is not executable, to make it executable:
remote:       On Windows run:   git update-index --chmod=+x .openshift/action_hooks/build
remote:       On Linux/OSX run: chmod +x .openshift/action_hooks/build
remote: Preparing build for deployment
remote: Deployment id is cedf7f51
remote: Activating deployment
remote: NOTE: The .openshift/action_hooks/deploy hook is not executable, to make it executable:
remote:       On Windows run:   git update-index --chmod=+x .openshift/action_hooks/deploy
remote:       On Linux/OSX run: chmod +x .openshift/action_hooks/deploy
remote: 
remote:   - pre_start_nodejs: Adding Node.js version 4.x.x binaries to path
remote:   - PATH set to include custom node version (4.x.x) from
remote:        /var/lib/openshift/.../app-root/data//node-v4.x.x-linux-x64/bin 
remote:     PATH = /var/lib/openshift/.../app-root/data//node-v4.x.x-linux-x64/bin:/var/lib/openshift/.../app-root/runtime/repo/node_modules/.bin:/var/lib/openshift/...//.node_modules/.bin:/opt/rh/nodejs010/root/usr/bin:/bin:/usr/bin:/usr/sbin
remote: Starting NodeJS cartridge
remote: Tue Aug 22 2017 15:39:10 GMT-0400 (EDT): Starting application 'staging' ...
remote: Waiting for application port (8080) become available ...

(Everything up to this point is exactly as it is on the working app, except for names.)

remote: Application 'staging' failed to start (port 8080 not available)
remote: -------------------------
remote: Git Post-Receive Result: failure
remote: Activation status: failure
remote: Activation failed for the following gears:
remote: ... (Error activating gear: CLIENT_ERROR: Failed to execute: 'control start' for /var/lib/openshift/.../nodejs
remote: #<IO:0x00000001cd42d0>
remote: #<IO:0x00000001cd4258>
remote: )
remote: Deployment completed with status: failure
remote: postreceive failed

rhc env and rhc app show show the settings to be identical in all the ways that they can be.

I know from other questions that the "port 8080" part above is a red herring. Plus, if I rhc ssh in and manually node www.js, it uses that port just fine and I can reach the app via browser.

So I investigated using rhc tail. I can see it failing to start, repeatedly, due to fat arrow notation:

pg.on('error', (err) => {
                      ^
SyntaxError: Unexpected token >
    at Module._compile (module.js:439:25)
    at Object.Module._extensions..js (module.js:474:10)
...
DEBUG: Program node ./www.js exited with code 8
DEBUG: Starting child process with 'node ./www.js'

Yet, if I rhc ssh into this same server and run node --version, I get the newer version (the same version as the other server, which both pull it from the marker file in the .openshift directory I deployed.)

I'm guessing somehow the final step in the push hook is using Node 0.10, since that's what the cartridge is named on both apps. However, if I check the path being added to PATH, a newer Node version does in fact live there.

However, this is where things get interesting. The PATH reported above (where 4.x.x is prepended) is no longer the path by the time Node is launched. I changed www.js to just spit out process.env.PATH and compared the two. The latter has these two paths added to the beginning!

/opt/rh/nodejs010/root/usr/bin
/opt/rh/v8314/root/usr/bin

What is doing that and how can I stop it? What even has a chance during these lines of output?

remote:     PATH = /var/lib/openshift/.../app-root/data//node-v4.x.x-linux-x64/bin:/var/lib/openshift/.../app-root/runtime/repo/node_modules/.bin:/var/lib/openshift/...//.node_modules/.bin:/opt/rh/nodejs010/root/usr/bin:/bin:/usr/bin:/usr/sbin
remote: Starting NodeJS cartridge
remote: Tue Aug 22 2017 15:39:10 GMT-0400 (EDT): Starting application 'staging' ...

(And why just on my second app, when I used --from-app to create it and all other settings seem to be equal between the two?)

Answer 1

Among one of the 129 forks of the repo used originally in the app to customize it's NodeJS version, there was a comment on an issue that solved it:

Add a line to .openshift\lib\utils:

#  Add the node binary path to the PATH.
export OPENSHIFT_NODEJS_VERSION=0.6 ### this is the new line
export PATH="$node_bin_path:${PATH}"

"It's important, like you mentioned, that it's set to 0.6, EVEN IF YOU'RE NOT RUNNING 0.6."

It's also important that it's set to 0.6, even if the original Node version you're getting reverted back to isn't 0.6, either. (I tried with 0.10, with no luck. 0.6 is the magic answer, here.)