grep lines from one file which are in another file

A frequent task in shell scripting and data processing is filtering out lines from one text file that appear in another (e.g., subtracting a list file from a dataset). Rather than writing loops or complex awk scripts, standard GNU grep provides an efficient, one-line solution.

---

The Solution

grep -v -x -F -f list.txt dataset.txt > filtered_output.txt

(This command outputs all lines from dataset.txt that do not match any exact lines listed in list.txt.)

---

Flag Breakdown

Flag	Full Option	Function
-v	--invert-match	Selects non-matching lines (inverts the filter).
-x	--line-regexp	Forces matches to span the entire line, preventing partial substring exclusions.
-F	--fixed-strings	Treats input patterns as literal strings rather than regular expressions (much faster and avoids escaping special characters like ., *, or [).
-f FILE	--file=FILE	Reads exclusion patterns line-by-line from the specified file.

---

Regex Patterns vs. Fixed Strings

• When to omit -F: Use grep -v -x -f patterns.txt data.txt if lines inside patterns.txt contain intentional regular expression syntax (such as wildcard matchers like ^server-[0-9]+).
• When to include -F (Recommended): Use -F whenever you want a strict, exact text match (WYSIWYG). It prevents characters like dots, brackets, and dollar signs from being parsed as regular expressions, while significantly speeding up processing on large datasets.