GitHub for Mac pops up "enter your password for the ssh key id

It seems to show this dialog two or three times. If I cancel it goes away and I'm still able to commit changes.

How do I prevent it from popping up each time and where do I find my passphrase if I have to reenter it and click "Remember to keychain" (which I've done before but will try again)?

How do I fix my SSH key in GitHub?

GitHub SSH configurationConfirm that your public key has been configured correctly by checking the SSH and GPG link in your account settings. If no keys are listed, add your public key and try to connect to GitHub over SSH again. The error should go away.

In the latest version of macOS - 10.12.2/Sierra - this is an easy fix. Just edit your ~/.ssh/config and enable the UseKeychain option: