{"id":391,"date":"2023-02-10T20:44:36","date_gmt":"2023-02-10T20:44:36","guid":{"rendered":"\/blog\/?p=391"},"modified":"2025-06-23T20:04:36","modified_gmt":"2025-06-23T20:04:36","slug":"automatic-detection-mislabeled-language","status":"publish","type":"post","link":"https:\/\/evinced.com\/blog\/automatic-detection-mislabeled-language\/","title":{"rendered":"Automatic detection of mislabeled language"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">If you want to create a more accessible internet,  you must make sure that screen readers and assistive technologies speak the right \u201clang.\u201d  <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">At Evinced, we\u2019ve created a new language attribute validation that can help,  by detecting mislabeled language for all website text.  <\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Why \u201clang\u201d matters<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">In HTML, the lang attribute specifies the language of the content in a given element. Most familiarly, it\u2019s declared for the entire page right at the start:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><code>&lt;html class=\"main\" lang=\"en\"&gt;<\/code><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">But it can also be declared for an individual element, like a paragraph.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><code>&lt;p lang=\"en\"&gt;The little brown fox jumped and how.&lt;\/p&gt;<\/code><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Either way, it\u2019s important for screen readers and other assistive technologies, as it ensures content can be pronounced properly when read aloud to the user.&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For example, to specify a paragraph is in French, the lang attribute must be set to \u201cfr,\u201d and the text that follows would be presumed to be in French by screen reader software. For example:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><code>&lt;p lang=\"fr\"&gt;Bonjour, comment \u00e7a va?&lt;\/p&gt;<\/code><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">What happens when the wrong \u2018lang\u2019 attribute is assigned to text? In the example above, there\u2019s a big pronunciation difference between \u201ccomment\u201d in English and in French, and an English speaker wouldn\u2019t know a cedille (the \u201c\u00b8\u201d in \u201c\u00e7a\u201d) from a porcupine. In general, a mislabeled lang attribute can drastically and negatively impact the clarity of the read performed by the screen reader software.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Let&#8217;s show some real-world examples.  Here, we&#8217;ve recorded the output from screen reader software on the same text &#8212; both when the lang attribute is misspecified, and when it&#8217;s specified correctly:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Example 1.<\/strong>  English Text Mis-Specified as French<\/p>\n\n\n\n<figure class=\"wp-block-audio\"><audio controls src=\"\/wp-content\/uploads\/2023\/02\/fr.mp3\"><\/audio><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Example 2. <\/strong> The Same English Text Correctly Specified as English<\/p>\n\n\n\n<figure class=\"wp-block-audio\"><audio controls src=\"\/wp-content\/uploads\/2023\/02\/en.mp3\"><\/audio><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Not a small difference, right?  If accessibility is what you\u2019re after, it\u2019s essential that the lang attributes be specified correctly. It\u2019s both common sense and <a href=\"https:\/\/www.w3.org\/TR\/WCAG22\/#language-of-page\">called for directly<\/a> in the Web Content Accessibility Guidelines.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>The most common language mislabelings<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The guidelines are clear, yet there are two common things web developers need to correct when coding languages.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Often a page will be written in one language but labeled as a different one. For example, a company might have an \u201cAbout Us\u201d page in French, but the language is mistakenly declared as English in the HTML.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Another common error is hosting a page that is <em>mostly<\/em> one language but has text in another language dotted throughout. Let\u2019s say an \u201cAbout Us\u201d page is in French, and French is the language (correctly) declared for the page in the HTML, yet there are still some existing words of text and terms that are English\u2014such as \u201cTerms of Use,\u201d \u201cCookie Policy,\u201d or \u201cChat with Sales.\u201d These standard terms are often baked into the code early on and overlooked when coding pages with new languages.  It&#8217;s a recipe for screen reader confusion.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Our goal: detect WCAG language violations<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">At Evinced, we know mislabeled language attributes are common mistakes, so we\u2019ve built a way to detect these WCAG language violations.  We call it <a href=\"https:\/\/knowledge.evinced.com\/system-validations\/language-attribute-mismatch\">language-attribute-mismatch<\/a>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">That isn\u2019t to say it was easy. In fact, it was anything but.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">We had to overcome a massive technical challenge involving crawling and collecting website text data, scrubbing it for information that\u2019s irrelevant to screen readers, and labeling each text with its inherited \u201clang\u201d attribute\u2014its HTML-assigned language. The hardest part was detecting the actual language of the text, as the HTML file itself could have specified an incorrect language.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Luckily, these types of challenges aren\u2019t new to our team, as the mislabeling of HTML files is fairly common. We know how to search for both text functionality <em>and<\/em> correctness to ensure the WCAG guidelines are met.&nbsp;<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>The how: natural language processing<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">How do we do it\u2014what algorithm finds the <em>actual<\/em> language of a text?&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Detecting language is a difficult task that falls into the natural language processing (NLP) domain. NLP is a subfield of linguistics, computer science, and artificial intelligence with interactions between technology and human subject matter experts. It requires developing algorithms and models to analyze and interpret natural language data in order to build natural language processing applications. The result is the creation of systems and software that allow computers to read, understand, and generate human language. The products of NLP work are all around us, from bot detection and email spam filtration to Siri on the iPhone and smart home devices.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">So you might wonder if there were a way to leverage existing technologies for our purposes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For example, you\u2019re probably familiar with Google Translate and know that if you type &#8220;O\u00f9 es la tour Eiffel?&#8221; into that tool, it will usually recognize what you type as French. So, might there be some API from Google that could have been used here?<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The answer is yes and no.&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Yes, there are third-party APIs from Google and others for language translation, but they have significant shortcomings for a task like ours.&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>They have problems with names, addresses, and professional vocabulary.&nbsp;<\/li>\n\n\n\n<li>In our analysis, existing APIs seem skewed toward predicting \u201cEnglish\u201d more often than other languages.<\/li>\n\n\n\n<li>They can pose privacy and security challenges, since they require calls to third-party servers which transport the text in question across different servers.&nbsp;<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>The challenge of false positives<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">There have been significant technological advancements over the last few years, and today many sophisticated (often open-source) machine-learning methods can help create natural language processing applications. Still, existing language models make mistakes, and as with, say, Optical Character Recognition (OCR) systems in the past, those mistakes can add up to be costly. Even small error rates can accumulate huge numbers of problems for a typical website.&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">To understand the full scale of this issue, first, it\u2019s important to understand that NLP models scan web pages seeking errors in \u201ctexts\u201d, which is a term developers use to mean&nbsp; \u201cword,\u201d \u201csentence,\u201d or even \u201cparagraph.\u201d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Now, let\u2019s say we run a model that generates an average of one mistake \u2013 one mislabeled text \u2013 per webpage, on a webpage that averages 100 &#8220;texts.&#8221; That\u2019s 99% accuracy. But many websites contain tens of thousands of pages.  What would happen then? Even a 1% error rate would yield thousands of false positives. That\u2019s a distraction that might keep a company from solving real accessibility issues.&nbsp;<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Our solution<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">We took an unorthodox approach. To reduce false positives, we employed two different language models simultaneously. Our team leveraged language models that utilize both old and new recognition methods, from neural networks to some advanced statistical analysis. Normalizing the output from two different models requires a ton of extra work, but we felt like the upside was worth it.&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In addition to using two models, we included innovative heuristics tailor-made for our task. Essentially, we developed an entire software layer that checks all of our results and filters out potential false positives, and resolves variances generated by the two-model approach.&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">What\u2019s more, we did this without overspecifying the model. This also required a fair bit of innovation, but it means we\u2019ve created a general-purpose tool that will work for any of our customers, regardless of the type of content or website.&nbsp;<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>The result<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The result is some very good news. We\u2019ve tested our approach extensively, across thousands of websites and tens of thousands of scans.  It turns out that all of our extra refinements are effective. Accuracy is outstanding, and false positives are dramatically fewer than in traditional, single-model approaches.&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If you\u2019re a customer with content in multiple languages (whether in part or whole), our approach is already helping make your web pages much more accessible. And if you\u2019re a potential customer, <a href=\"https:\/\/www.evinced.com\/\">feel free to get in touch<\/a><strong>.<\/strong><\/p>\n","protected":false},"excerpt":{"rendered":"<p>If you want to create a more accessible internet, you must make sure that screen readers and assistive technologies speak the right \u201clang.\u201d At Evinced, we\u2019ve created a new language attribute validation that can help, by detecting mislabeled language for all website text. Why \u201clang\u201d matters In HTML, the lang attribute specifies the language of [&hellip;]<\/p>\n","protected":false},"author":13,"featured_media":1125,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"categories":[7],"tags":[],"class_list":["post-391","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology"],"acf":{"authors_to_show":[928]},"_links":{"self":[{"href":"https:\/\/evinced.com\/blog\/wp-json\/wp\/v2\/posts\/391","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/evinced.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/evinced.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/evinced.com\/blog\/wp-json\/wp\/v2\/users\/13"}],"replies":[{"embeddable":true,"href":"https:\/\/evinced.com\/blog\/wp-json\/wp\/v2\/comments?post=391"}],"version-history":[{"count":6,"href":"https:\/\/evinced.com\/blog\/wp-json\/wp\/v2\/posts\/391\/revisions"}],"predecessor-version":[{"id":1305,"href":"https:\/\/evinced.com\/blog\/wp-json\/wp\/v2\/posts\/391\/revisions\/1305"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/evinced.com\/blog\/wp-json\/wp\/v2\/media\/1125"}],"wp:attachment":[{"href":"https:\/\/evinced.com\/blog\/wp-json\/wp\/v2\/media?parent=391"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/evinced.com\/blog\/wp-json\/wp\/v2\/categories?post=391"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/evinced.com\/blog\/wp-json\/wp\/v2\/tags?post=391"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}