I'm running into an issue where I seem to max out my Container Engine (Kubernetes Engine) cluster at 9 nodes (or 9 vCPUs). For context, I'm trying to run around 50 or so web scrapers as Kubernetes Cron Jobs. Here is the gcloud
command I am using to create the cluster in the us-east1-c
zone:
gcloud beta container clusters create my-example-cluster \
--cluster-version=1.8.1-gke.1 \
--machine-type=f1-micro \
--enable-autoscaling \
--min-nodes=1 \
--max-nodes=60
When I checked out IAM & Admin > Quotas, it looked like I was possibly maxing out on in-use IP addresses (max=8).
After launching all the scrapers, I'm ending up around 35 pods that are unschedulable (seemingly because I can't exceed 9 nodes?).
Has anyone run into a similar situation and come up with a solution? Is there a way, perhaps via Kubernetes config, to not count against IP address usage since these nodes do not need to be accessible externally (if that's even the issue)? Also open to other setups if anyone has recommendations.
Thanks for all the help!
P.S. I do have billing enabled
I received some help on this issue from the Google Cloud Platform Community Slack. It turns out I was hitting the In-use IP addresses quota; however, I didn't realize you could request increased quotas.
I was able to increase quotas by:
us-west1
)So, basically an oversight on my end but maybe it will help somebody else who didn't notice or wasn't aware of that option.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With