How to remove Arabic diacritics (tashkeel) from text

What counts as tashkeel?

On a computer, a vowel mark is not part of the letter. Each mark is its own character, stored right after the letter it sits on, and Unicode calls it a combining mark. The word «كَتَبَ» is six characters: three letters and three fathas. That is why a plain search for «كتب» misses «كَتَبَ» in many apps, and why character counts jump when text is vowelled.

The Remove common Arabic diacritics option in the Cleaner removes exactly nine characters:

The first eight sit together in the Unicode chart under the heading “Tashkil from ISO/IEC 8859-6”, inherited from the old 8-bit Arabic encoding. The dagger alef is listed separately as ARABIC LETTER SUPERSCRIPT ALEF, and Unicode’s note says it is actually a vowel sign. The standard also says these marks are normally omitted in everyday writing, so plain Arabic is the norm, not a damaged copy.

What the Cleaner deliberately keeps

The point is to drop the vowels without touching the letters. So the option leaves these alone on purpose:

• Hamza and madda. أ إ آ ؤ ئ are complete letters and are never changed. Some text stores hamza or madda as a separate combining mark after the base letter instead: maddah above U+0653, hamza above U+0654 and hamza below U+0655. Those stay too, because deleting them changes the spelling: «سأل» (he asked) would become «سال» (it flowed).
• Quranic and specialist marks. Pause signs, small high letters used in mushaf typography, and the rarer combining marks from U+0656 to U+065F are not removed. That is why the option says “common” diacritics.
• Tatweel. The kashida «ـ» is not a vowel mark. It has its own checkbox, Remove tatweel.
• Letter shapes. أ does not become ا, ى does not become ي, ة does not become ه. That is search normalization, not diacritic removal, and it lives under More controls → Normalize letters for search.

هٰذَا الرَّحْمٰنُ، قَرَأَ أَحْمَدُ القُرْآنَ وَسُئِلَ عَنْ مَسْؤُولِيَّةٍ.

هذا الرحمن، قرأ أحمد القرآن وسئل عن مسؤولية.

«هٰذَا» becomes «هذا» and «الرَّحْمٰنُ» becomes «الرحمن», which is how both are normally spelled. Every hamza survives.

You cannot see whether an «أ» is one precomposed character or alef plus a combining hamza. To check, paste the word into Normalization, choose NFC and compare the code points:

سأل

سأل

U+0633 U+0627 U+0654 U+0644

U+0633 U+0623 U+0644

Before NFC the word is four characters, including the combining hamza U+0654; after it, three, with the hamza folded into U+0623. The diacritics option keeps the hamza either way.

Tatweel is separate too. Unicode describes it as a character inserted to stretch letters or to carry a mark with no base letter. Remove diacritics alone and the stretch stays:

مُبــــارَكٌ

مبــــارك

Tick Remove tatweel as well and the word goes back to its normal shape:

مُبــــارَكٌ

مبارك

Remove diacritics in the Cleaner, step by step

1. Open the Cleaner and paste your text into Original text. It takes up to 100,000 text units, and Open .txt loads a plain-text file.
2. Click the Diacritics off quick action above the editor. It turns on Remove common Arabic diacritics and resets every other option. To combine it with spacing or digit fixes, tick Remove common Arabic diacritics under Arabic marks instead and add what you need.
3. Read the Change summary. It shows how many diacritics were removed. No change means the text was not vowelled, or its marks are outside the common set.
4. Check Compare changes under the editor. Every removed mark is shown struck through in place, so anything else that changed stands out.
5. Use Copy result or Download .txt. Your original stays in the left pane, and Undo restores your previous options.

كَتَبَ الطَّالِبُ دَرْسَهُ بِخَطٍّ جَمِيلٍ.

كتب الطالب درسه بخط جميل.

Here the tool removed 18 marks, including the shadda and fatha stacked in «الطَّالِبُ» and the shadda with kasratan in «بِخَطٍّ». The full stop is untouched.

For many files, use Batch files: choose Common diacritics only under Cleaning options, add up to 20 UTF-8 .txt files and download the results as a ZIP.

Check that nothing else changed

Compare changes is enough for short text. When you have a vowelled and a plain version and need to know they are the same text, use Compare:

1. Paste the vowelled version into First text · A and the other into Second text · B.
2. Tick Ignore common diacritics. It ignores the same nine marks the Cleaner removes, and it never edits either original.
3. If the texts now match, the only difference was the marks. Anything still highlighted is a real change in letters or words.

قَرَأْتُ الكِتَابَ مَرَّتَيْنِ.

قرأت الكتاب مرتين.

قرأت الكتاب مرتين.
قرأت الكتاب مرتين.

In the next pair an alef is missing from «الكتاب». Ignoring the marks does not hide the error; it isolates it:

قَرَأْتُ الكِتَابَ مَرَّتَيْنِ.

قرأت الكتب مرتين.

قرأت الكتاب مرتين.
قرأت الكتب مرتين.

Compare takes up to 5,000 text units per side.

When stripping the marks is the right call

• Search keys and indexes. If some records are vowelled and others are not, «مُحَمَّد» will never match «محمد» in an exact search. Keep the original and store a mark-free copy for search and matching only. That copy usually needs hamza and yeh normalization too; see normalizing Arabic for search.
• Filenames and URLs. A filename with marks will not match the same name typed without them. In a URL every Arabic character, marks included, is percent-encoded, so one fatha becomes %D9%8E and links grow long and fragile.
• Subtitle files. A subtitle line is on screen for a few seconds, and full vocalization crowds short lines. Keep only the marks that prevent a misreading.
• Data entry and spreadsheets. A vowelled name in one row and a plain one in another are two different values for sorting, de-duplication and lookups between sheets. Normalize the column first; cleaning up copied Arabic text covers the other usual fixes.

When you should keep the diacritics

In some text the marks are the only thing that tells two words apart:

عِلْمٌ، عَلَمٌ، عَلِمَ، عُلِّمَ

علم، علم، علم، علم

Knowledge, flag, he knew and he was taught: four words, one spelling. Leave the marks alone in these cases:

• The Quran and other carefully vocalized texts. The vocalization is part of the transmitted text. In Uthmani script the dagger alef stands for an alef that is read aloud, so removing it changes the word.
• Poetry. Metre is built on vowels, sukun and shadda.
• Teaching material and children’s books. Learners read by the vowels.
• Names with more than one reading. «مَلَك» (Malak) and «مَلِك» (Malik) both become «ملك». On certificates, invitations and name lists, keep the marks that fix the pronunciation.
• Verbatim quotations. Legal wording or an edited manuscript must stay as published.

ذَٰلِكَ ٱلْكِتَٰبُ لَا رَيْبَ ۛ فِيهِ ۛ

ذلك ٱلكتب لا ريب ۛ فيه ۛ

«ٱلْكِتَٰبُ» turned into «ٱلكتب» once the dagger alef was gone. The alef wasla «ٱ» stayed because it is a letter, and the pause sign «ۛ» because it is outside the common set. The result is neither vocalized nor plain. If you need a bare copy of Quranic text for search, keep the original and never publish the stripped copy in its place.

A middle ground is to drop the vowels but keep the shadda. Open Replace, tick JavaScript regular expression, enter [ً-ِْٰ] in Find and leave Replacement empty. The pattern covers all nine marks except shadda, U+0651:

عَلَّمَ المُدَرِّسُ الطُّلَّابَ.

علّم المدرّس الطّلّاب.

Removing diacritics in Word and Excel

Word

In Word for Windows, people usually remove one mark at a time with Replace. Go to Home > Editing > Replace or press Ctrl+H, copy a single mark from your text (a fatha, say) and paste it into Find what. Leave Replace with empty, choose Replace All, and repeat for each mark. It is a common technique, not a documented diacritics feature, so work on a copy.

Microsoft also documents Use wildcards, which accepts a set of characters in square brackets or a range such as [r-t]. Some people put all the Arabic marks inside one pair of brackets, but Microsoft’s pages only show Latin examples.

Word’s developer reference also describes a setting that decides whether a search in a right-to-left document requires diacritics to match; it may be missing depending on your installed languages. If Word does not find a pasted mark, check that setting first. It only changes how Find behaves.

Excel

SUBSTITUTE replaces a given piece of text inside a cell, and UNICHAR returns a character from its decimal Unicode number. The fatha, U+064E, is 1614 in decimal, so this formula strips fathas from cell A2:

=SUBSTITUTE(A2,UNICHAR(1614),"")

To remove all nine marks, nest the function nine times, using 1611 to 1618 and then 1648 for the dagger alef:

=SUBSTITUTE(SUBSTITUTE(SUBSTITUTE(SUBSTITUTE(SUBSTITUTE(SUBSTITUTE(SUBSTITUTE(SUBSTITUTE(SUBSTITUTE(A2,UNICHAR(1611),""),UNICHAR(1612),""),UNICHAR(1613),""),UNICHAR(1614),""),UNICHAR(1615),""),UNICHAR(1616),""),UNICHAR(1617),""),UNICHAR(1618),""),UNICHAR(1648),"")

Microsoft lists both functions for Excel 2016 and later, including Microsoft 365. Microsoft 365 also has REGEXREPLACE, which takes a PCRE2 regular expression, but Microsoft’s examples do not cover Arabic, so test it first. Either way, write the result to a new column.