Mercurial: Permanently remove sensitive data from HG repo?

Question

I know that in Mercurial, "history is sacred".

But let's say someone accidentally commits something they shouldn't, like a settings file containing a password or something. Let's even say some time passes before anyone realizes it, so its been hanging around for several commits. Obviously, the discoverer then removes the sensitive data from the repository.

Is there any way to permanently scrub that file or the sensitive data from the commit history, as though it never existed? Or would that sensitive data just be a permanent part of the repo forever and ever?

Answer 1

There are a few methods to accomplish this. All of them require the cooperation of everybody who has cloned your repository or pulled changesets from it after the change was introduced.

Which method to use depends on the exact nature of the committed data and where it is in the history. All of them require the use of Mercurial extensions and cannot be accomplished with core Mercurial. Luckily, all the required extensions are shipped by default with Mercurial and simply have to be enabled.

I'm not going into detail about the methods here as there are several answers that give different methods in the question this is a dupe of. I just want to be clear that the accepted answer in that question is technically correct, but not useful. It is actually possible.

Answer 2

If, and only if, this repository hasn't escaped into the wild, you can remove a file from history by essentially cloning to a new repository while filtering the sensitive file in the process, using hg convert Hg Convert Extension doc here

Commonly, we find something when we audit the repository prior to publishing or delivering to a client, such as a web.config or ini file with a password.

The extension isn't enabled by default, but is included with all clients I use, you need to enable it before Mercurial will recognize the convert command.

If using Tortoise Hg or Kiln, for example:

1. Open Tortoise Hg -> Global Settings -> Extensions
2. Check the box beside "Convert"
3. Click Ok

Or edit Mercurial.ini directly:

[extensions]
convert = 

Go to the directory above your repository (in my example, my repos is HelloApp):

1. Create a file named filemap.txt

2. Add a line with the full path to the filename you want to exclude.

   exclude HelloApp/sensitive.config
   

3. Open a command prompt, cd to the same directory, containing your filemap.txt, and run the hg convert

   cd C:\projects
   
   hg convert --filemap filemap.txt HelloApp HelloApp_clean
   

4. Then get latest working copy:

   cd HelloApp_clean
   hg update
   

You will need to create a fresh clone on your server with your clean copy.