Duplicate CSV rows are not the same as duplicate product SKUs

Suppose a product export contains these rows:

Type,SKU,Name,Regular price simple,TSHIRT-S,Shirt S,19.00 simple,TSHIRT-S,Shirt S,19.00 simple,MUG-1,Ceramic Mug,12.00 simple,MUG-1,Ceramic Mug updated,12.00

There are two different problems:

  1. Rows 1 and 2 are identical after CSV parsing.
  2. Rows 3 and 4 are different rows but share the same non-empty SKU.

A generic row deduplication tool may catch the first issue while missing the second. Before a bulk import, it helps to check both.

A small standard-library Python check 

import csv from collections import Counter from io import StringIO source = """Type,SKU,Name,Regular price simple,TSHIRT-S,Shirt S,19.00 simple,TSHIRT-S,Shirt S,19.00 simple,MUG-1,Ceramic Mug,12.00 simple,MUG-1,Ceramic Mug updated,12.00 """ reader = csv.DictReader(StringIO(source)) rows = list(reader) header = reader.fieldnames or [] exact_keys = [tuple(row.get(name) for name in header) for row in rows] sku_keys = [row["SKU"].strip() for row in rows if row.get("SKU", "").strip()] print("Duplicate row occurrences:", sum(v - 1 for v in Counter(exact_keys).values() if v > 1)) print("Duplicate SKU values:", sorted(k for k, v in Counter(sku_keys).items() if v > 1))

Expected output:

Duplicate row occurrences: 1 Duplicate SKU values: ['MUG-1', 'TSHIRT-S']

The first number counts additional repeated rows beyond the first instance. The SKU list represents duplicate identifiers, not the number of duplicates.

Import hygiene matters more than the counter

Before applying an automatic fix, consider whether identifiers are intended to be unique in your target workflow. For variations, check the expected parent relationships and the WooCommerce import schema separately. Preserve a copy of the source and review the import preview before writing to production.

This example deliberately uses fabricated data and does not access anyone's store.

A free, browser-local CSV audit is available for experimenting with a sample before importing. It does not replace testing business-specific SKU rules.

Disclosure: This tutorial was prepared with AI assistance. The Python example was run against the synthetic data shown above, and its output was verified before publication.