Remove a file that I shouldn't have committed to Git

Question

So I did a bad thing.

Somewhere during the course of making a bunch of changes, I realized that a unintended file had snuck into a commit or two. Because I didn't realize this until later, the commits that included the file have now been pushed to the remote. I want & need the commits, I just want to remove this specific file from them.

What I need to do, of course, is to reach into every nook & cranny of my tree (local and remote) and obliterate that file. I've tried a few things using filter-branch and filter-tree, but when I attempt to push the changes get rejected.

What are my options? What am I doing wrong?

Thanks.

UPDATE

At max's request, here's the message I get when attempting to push:

$ git push origin develop
To [email protected]:robwilkerson/cakephp-polyclip-plugin.git
! [rejected]        develop -> develop (non-fast-forward)
error: failed to push some refs to '[email protected]:robwilkerson/cakephp-polyclip-plugin.git'
To prevent you from losing history, non-fast-forward updates were rejected
Merge the remote changes before pushing again.  See the 'Note about fast-forwards' section of 'git push --help' for details.

Answer 1

There is a nice guide over at GitHub, that might help you.

Answer 2

Did you add -f (force) to the git push? Without that flag you can't overwrite "old" commits.

-f, --force
       Usually, the command refuses to update a remote ref that is not an
       ancestor of the local ref used to overwrite it. This flag disables
       the check. This can cause the remote repository to lose commits;
       use it with care.

It can get messy if others already pulled from those repository.