How to delete something that was accidentally pushed

I wanted to do source control on my website. There were a lot of files so I just went git add -a

Unfortunately, I accidentally committed and pushed a file called credentials.txt that had my ftp and db details. I've tried deleting the file, committing then pushing but it still shows the file when I browse my repo on github in the history.

Is there anyway to get rid of this?

Github has a nice guide on this exact topic. It explains how to get it out of the repository, as well as how to deal with the data that may already be in public hands.