How To Keep Your Disavow File Fresh

Chances are the disavow.txt file you sent to Google is outdated.  As Google keeps discovering and rolling out Penguin updates, its discovering links that were missed on the first go-round.  It’s important to also realize that the link data tools you used for your disavow are typically much slower at finding links than Google.  From Ahrefs to Majestic to Open Site Explorer, it could take them months to years to find a link Google knows about – thereby causing them to be absent from your first disavow.

We recently had a client who was concerned that “negative SEO” attacks were happening due to the discovery of new links in Majestic. Upon further analysis, we discovered the spammy links were from posts from 2008 that Majestic simply didn’t find until now.  There’s no need to debate whether negative SEO exists here, but there’s clearly a latency issue that few may be thinking about following a disavow campaign.  Needless to say, the disavow.txt file needed to be updated for our client.

Important Note – Don’t Lose Your Work: The version of the disavow.txt file you submitted is the master list. It’s the only list Google uses. If you replace the master list with a new list featuring 10 new URLs, Google will now only disavow those 10 URLs. They will forget about your other links from the master list unless they’re included in the most recent disavow.  This, by the way, is how you “reavow” any links, per Marie Haynes.

How To Audit Your Disavow File

To start working on your disavow file just make a copy of this Google Sheet and follow the instructions below.

First, we start by downloading the disavow list we previously uploaded (you can locate it here), and pull just the URLs we submitted. Google makes this extremely easy today with a spreadsheet download. Once this list is downloaded you may notice you have sites with https://, http://, or www. What you will want to do is find and replace those with nothing. This will help the formulas provided to find exact matches.

Once this is done, copy your list and paste it into column A in the Google Sheet:

Now that we have the sites you disavowed we want to get a fresh pull of all the backlinks going to the site. For this you will want to use all or a combination of the following:

Google Search Console
Open Site Explorer
Majestic
Ahrefs
Cognitive SEO

After you have all of the backlinks pulled, combine them all to one row on an excel sheet and remove duplicates from the list. Once the list is deduped, you will then do exactly what we did for the disavow links and remove https://, http://, and www. Once you have this finished copy and paste them into column D in the Google sheet. Column B and C will auto-populate for you based on the formulas in those cells.

And, now the sheet should look something like this:

Now, add the “CSV Converter” add-on to your Google sheets. Once it’s installed you will want to select all of your backlinks in Column D, click on “Add-Ons,” “CSV Converter,” then choose “Convert selected CSV to columns (custom separator).”

Once it loads it will ask you for the custom separator which we will make ” / ” so that we can get the root domains of all of our backlinks. Then press “OK” and it will create the following:

Everything after Column E we can ignore and delete.

The sheet is now finished and you can begin your detective work to look at which links should be disavowed. A “No” in Columns B or C means that it is not in Column A (the list of domains currently disavowed), and should be evaluated and added to disavow if necessary.

Now depending on the size of your backlink profile, you may have hundreds, maybe even thousands to go through. You can always sort by root domain and tackle the sites with the most links coming to your site first.

So there you are – a nice and easy way to start looking at your disavow file and keep it fresh. You will want to complete this every few months, or if and when you notice an unnatural link coming to your site.