EMR notebook fails because JupyterEnterpriseGateway application not installed on existing cluster AWS [closed]

Question

I've read the AWS documentation, created an EMR cluster high version with Hadoop SPark and Livy. Select notebooks, create one, connect it to the cluster, but when its starting it abruptly stops and the message is Cluster ########### does not have JupyterEnterpriseGateway application installed. Please retry with another cluster. I try to SSH into the master node and pip install it but that fails. Also can't find any solution online. I thought this came out of the box with these clusters. Any solutions to this?

Answer 1

I'm not sure why the change but you need to install JupyterEnterpriseGateway as one of the selected applications for emr-6-x-x, it doesn't make you (and you can't actually select) do this with emr-5-x-x/