Checked a Secret into git? It’s Critical You Remove it ASAP

 

It’s easy to do. Picture this: You’ve got a local environment file with a database connection string used for testing. Or a private key file for a TLS certificate. Or perhaps you’ve got a hard-coded username and password in your Go code that you meant to delete. Seems pretty innocuous, but before you know it, you’ve inadvertently checked the sensitive information into git version control. Even worse, maybe you’ve pushed things to a public repository like GitHub for all to see.

ok, “never” may be too strong a word.

Usually a good thing, git is really good at remembering. But not so much in this case. You’ve committed sensitive information and simply put, this is bad. Fortunately, there is a solution.

Your first instinct might be to simply remove the information and check in a new version. Shazam! You’re done! But not so fast. The latest version might be “sanitized,” but your secrets remain in the version history and can be easily retrieved. So how do we clean things up properly? Here’s how.

1. Get New Credentials

Before proceeding further, change your password, get a new key, do whatever it takes to invalidate the leaked credentials.

2. Search For All Occurrences of Secret Information

Next, you’ll want to find all occurrences of sensitive information in your project. Not just the ones you currently know about.

You can use a credential scanning tool like GitLeaks or open-source TruffleHog to search your files and generate a list of credentials to be removed.

You can also manually look for leaked credentials. After cloning your repository locally search your commit history with:

git log -p | grep [password]

3. Remove Secrets Entirely

Now the step we’ve been waiting for. Drum roll please. Delete the information entirely from git.
Here git-filter-repo is your friend. It’s a Python utility that doesn’t have the potential gotcha’s of older utilities. It’s also fast and easy to use.
It’s probably easiest to install it as a Python package using pipx.
First, install pipx (I’m using Ubuntu here) with apt:

sudo apt update
sudo apt install pipx
pipx ensurepath
sudo pipx ensurepath --global # optional to allow pipx actions with --global argument
pipx install git-filter-repo

Or with brew:

brew install git-filter-repo

Next, backup your repository and then start removing the information. To remove a file called secrets.txt across all history and branches, run this command from within the repository:

git filter-repo --path secrets.txt --invert-paths --all

You can also create a file of offending mapping pairs and git filter-repo will use it accordingly. The file should look something like:

sensitive-value1==>****
sensitive-value2==>****

To remove secrets in a file secrets-map.txt with the contents above, try:

git filter-repo --replce-text ../secrets-map.txt

Verify the secrets are gone locally and note that the git remote has been removed for safety. Add it back and do a force push to the remote:

git remote add origin user@repohost:reponame.git

git push --force --all --prune

Lots more examples can be found here.

Prevent This From Happening in the First Place

To prevent future mistaken commits from happening in the first place, there are preventative measures we can take.

1. Use a .gitignore file to prevent checking in files that are named a certain name or patterned a specific way.
2. Proactively and regularly scan your git repositories for sensitive information with tools like GitLeaks added to your DevOps pipelines that run in an automated fashion, triggered by pushes and commits.

Conclusion

Here we’ve demonstrated how to find and remove credentials from git across history and branches in a safe, straightforward manner. We’ve also provided some ways to prevent having to do this in the future, reducing the likelihood in the first place.