Home » 6 Ways to Remove Duplicates In Google Sheets

6 Ways to Remove Duplicates In Google Sheets

  • by
  • 10 min read

A Spreadsheet is nothing but a document that arranges data in the form of rows and columns. Spreadsheets are used by almost every business organization to maintain its data records and perform operations on that data. Even schools and colleges use spreadsheet software to maintain their database. When it comes to spreadsheet software, Microsoft Excel and Google sheets are the top-ranking software that many people use. Nowadays, more users choose Google Sheets over Microsoft Excel as it stores the Spreadsheets on their Cloud Storage, i.e. Google Drive which can be accessed from any location. The only requisite is that your computer should be connected to the Internet. Another great thing about Google Sheets is that you can use it from your browser window on your PC.

When it comes to maintaining data entries, one of the common issues faced by many users are duplicates or duplicate entries. For example, imagine that you have the details of people collected from a survey. When you list them using your spreadsheet software such as Google Sheets, there is a possibility of duplicate records. That is, one person might have filled the survey more than once, and hence Google Sheets would list the entry twice. Such duplicate entries are more troublesome when it comes to businesses. Imagine if a cash transaction is entered in the records more than once. When you calculate the total expenses with that data, it would be an issue. To avoid such situations, one should ensure that there are no duplicate records in the spreadsheet. How to achieve this? Well, in this guide, you will discuss 6 different ways to remove duplicates in Google Sheets. Come on, without further introduction, let us take a peek into the topic.

6 Ways to Remove Duplicates In Google Sheets

How to Remove Duplicates in Google Sheets?

Duplicate records are really troublesome in the case of maintaining data records. But you need not worry as you can easily remove duplicate entries from your Google Sheets spreadsheet. Let us see some ways in which you can get rid of duplicates in Google Sheets.

Method 1: Using the Remove Duplicates Option

Google Sheets has a built-in option to remove entries that are repetitive (duplicate entries). To use that option, follow the illustration below.

1. For example, take a look at this (see screenshot below). Here you can see that the record “Ajit” is entered two times. This is a duplicate record.

Record “Ajit” is entered two times. This is a duplicate record

2. To remove the duplicate entry, select or highlight the rows and columns.

3. Now click on the menu option labelled Data. Scroll down then click on the Remove duplicates option.

Click on the menu labelled “Data”. Click on Remove duplicates to eliminate duplicate records

4. A pop-up box will come up, asking which columns to analyze. Choose the options as per your needs and then click on the Remove duplicates button.

Click the button labelled “Remove duplicates”

5. All duplicate records would be eliminated, and unique elements would remain. Google Sheets will prompt you with the number of duplicate records that were eliminated.

Google Sheets will prompt you with the number of duplicate records that were eliminated

6. In our case, only one duplicate entry was removed (Ajit). You can see that Google Sheets has removed the duplicate entry (refer to the screenshot that follows).

Method 2: Remove Duplicates with Formulae

Formula 1: UNIQUE

Google Sheets has a formula named UNIQUE that retains unique records and would eliminate all duplicate entries from your spreadsheet.

For Example: =UNIQUE(A2:B7)

1. This would check for duplicate entries in the specified range of cells (A2:B7).

2. Click on any empty cell on your spreadsheet and enter the above formula. Google Sheets would highlight the range of cells that you specify.

Google Sheets would highlight the range of cells that you specify

3. Google Sheets will list the unique records where you typed the formula. You can then replace the old data with the unique records.

Google Sheets would list the unique records where you typed the formula

Formula 2: COUNTIF

You can use this formula to highlight all duplicate entries in your spreadsheet.

1. For Example: Consider the following screenshot that contains one duplicate entry.

At cell C2, enter the formula

2. In the above screenshot, at cell C2, let’s enter the formula as, =COUNTIF(A$2:A2, A2)>1

3. Now, once the Enter key is pressed, it will show the result as FALSE.

As soon as hit the Enter key, it would show the result as FALSE

4. Move the mouse pointer and place it over the small square at the bottom portion of the selected cell. Now you will see a plus symbol instead of your mouse cursor. Click and hold on that box, and then drag it up to the cell where you want to find the duplicate entries. Google sheets would automatically copy the formula to the remaining cells.

Google sheets would automatically copy the formula to the remaining cells

5. Google Sheet will automatically add “TRUE” in front of duplicate entry.

NOTE: In this condition, we have specified as >1 (greater than 1). So, this condition would result TRUE in places where an entry is found more than once. In all other places, the result is FALSE.

Method 3: Remove Duplicate Entries with Conditional Formatting

You can also make use of conditional formatting to eliminate duplicate records from Google Sheets.

1. First, select the data set on which you would like to perform conditional formatting. Then, from the Menu select Format and scroll down then choose Conditional formatting.

From the Format menu, scroll down a bit to choose Conditional formatting

2. Click on the Format cells if… drop-down box, and choose the Custom Formula option.

Click on the Format cells if… drop-down box

3. Enter the formula as =COUNTIF(A$2:A2, A2)>1

Note: You need to change the row & column data according to your Google Sheet.

Choose the Custom Formula and Enter the formula as COUNTIF(A$2:A2, A2)>1

4. This formula would filter records from column A.

5. Click on the Done button. If column A contains any duplicate records, Google Sheets will highlight the repeated entries (duplicates).

Google Sheets will highlight the repeated entries (duplicates)

6. Now you can easily delete these duplicate records.

Method 4: Remove Duplicate Records with Pivot Tables

As pivot tables are fast-to-use and flexible, you can use it to find & eliminate duplicate records from your Google Sheet.

First, you will have to highlight the data in Google Sheet. Next, create a pivot table and again highlight your data. To create a pivot table with your dataset, navigate to the Data under the Google Sheet menu and click on the Pivot table option. You will be prompted with a box asking whether to create the pivot table in the existing sheet or a new sheet. Choose a suitable option and proceed.

Your pivot table will be created. From the panel on the right, choose the Add button near Rows to add the respective rows. Near the values, choose to Add a column to check for duplication of values. Your pivot table would list the values with their counts (i.e. number of times the value occurs in your sheet). You can use this to check for the duplication of entries in Google Sheet. If the count is more than one, that means the entry is repeated more than once in your spreadsheet.

Method 5: Using Apps Script

Another great way to eliminate duplicated from your document is by using the Apps Script. Given below is the apps-script to get rid of duplicate entries from your spreadsheet:

/**
* remove duplicate rows from Google Sheets data range
*/
function removeDupRows() {
  var ss = SpreadsheetApp.getActiveSpreadsheet();
  var sheet = ss.getSheetByName('Sheet1');
  // change the row number of your header row
  var startRow = 7;
  // get the data
  var range = sheet.getRange(startRow,1,sheet.getLastRow(),sheet.getLastColumn()).getValues();
  // remove duplicates with helper function
  var dedupRange = arrayUnique(range);
  Logger.log(dedupRange);
  // check if duplicate sheet exists already, if not create new one
  if (ss.getSheetByName('Sheet1 Duplicates Removed')) {
    // case when dedup sheet already exists
    var dedupSheet = ss.getSheetByName('Sheet1 Duplicates Removed');
    var lastRow = Math.max(dedupSheet.getLastRow(),1);
    var lastColumn = Math.max(dedupSheet.getLastColumn(),1);
    // clear out any previous de-duplicate data
    dedupSheet.getRange(1,1,dedupSheet.getLastRow(),dedupSheet.getLastColumn()).clear();
    // replace with new de-duplicated data
    dedupSheet.getRange(1,1,dedupRange.length,sheet.getLastColumn()).setValues(dedupRange);
  }
  else {
    // case when there is no dedup sheet
    var dedupSheet = ss.insertSheet('Sheet1 Duplicates Removed',0);
    dedupSheet.getRange(1,1,dedupRange.length,dedupRange[0].length).setValues(dedupRange);
  }
  // make the de-duplicate sheet the active one
  dedupSheet.activate();
}
/**
* helper function returns a unique array
*/
function arrayUnique(arr) {
  var tmp = [];
  // filter out duplicates
  return arr.filter(function(item, index){
    // convert row arrays to strings for comparison
    var stringItem = item.toString();
    // push string items into temporary arrays
    tmp.push(stringItem);
    // only return the first occurrence of the strings
    return tmp.indexOf(stringItem) >= index;
  });
}
You can also use the below function to add a custom menu to Google Sheets to remove duplicates so that you can easily use it.
/**
* add a menu to run a function from Sheet
*/
function onOpen() {
  var ui = SpreadsheetApp.getUi();
  ui.createMenu('Remove duplicates')
  .addItem('Highlight duplicate rows','highlightDupRows')
  .addItem('Remove duplicate rows','removeDupRows')
  .addToUi();
}

Method 6: Use Add-on to Remove Duplicates in Google Sheets

Using an add-on to eliminate duplicate entries from your spreadsheet can be beneficial. Several such extensions turn out to be helpful. One such add-on program is the add on by Ablebits named “Remove Duplicates”.

1. Open Google Sheets, then from Add-ons menu click on the Get add-ons option.

From inside Google Sheets, locate a menu named “Add-ons” and click on the “Get add-ons” options

2. Choose the Launch icon (highlighted in the screenshot) to launch the G-Suite Marketplace.

Choose the Launch icon (highlighted in the screenshot) to launch the G-Suite Marketplace

3. Now search for the Add-on you need and install it.

Search for the add-on you need and click on it

4. Go through the description of the add-on if you wish and then click on the Install option.

Click on the option to Install

Accept the necessary permissions to install the add-on. You may have to sign in with your Google account credentials. After you have installed the add-on, you can easily remove duplicates from Google Sheets.

Recommended:

We hope this information was helpful and you were able to easily remove duplicate entries from Google Sheets. If you have any suggestions or questions in your mind, make use of the comments section to ask them.

nv-author-image

Elon Decker

Elon is a tech writer at TechCult. He's been writing how-to guides for about 6 years now and has covered many topics. He loves to cover topics related to Windows, Android, and the latest tricks and tips.

Leave a Reply

Your email address will not be published. Required fields are marked *