The problem

You have an en.json with 800 strings and you need Spanish, German and Japanese by Friday. Human review of everything is slow. Trusting a single engine with no review is risky, because one engine can be confidently wrong, and a wrong button label or error message is hard to spot later.

Why engine disagreement works

Engines are trained differently, so their mistakes do not always overlap. When three of them produce nearly the same sentence, it is usually safe. When they split, something is often ambiguous: a short string with no context, an idiom, or a term with several meanings. That split is where human attention helps most.

Step 1: protect your placeholders

Before comparing anything, check that each translation kept your variables intact. A translation that drops {name} or %s is broken, however fluent it reads.

python import re

PLACEHOLDER = re.compile(r"{{\s*\w+\s*}}|{\w+}|%(\w+)s|%s")

def placeholders(text): return sorted(PLACEHOLDER.findall(text)) Step 2: score agreement

For each engine's output, measure how similar it is to the others. The output closest to the rest is your pick, and the average similarity is your confidence signal.

python from difflib import SequenceMatcher

def similarity(a, b): return SequenceMatcher(None, a.lower(), b.lower()).ratio()

def pick(source, candidates, threshold=0.8): """candidates: {engine_name: translated_text}""" ok = {k: v for k, v in candidates.items() if placeholders(v) == placeholders(source)} if len(ok) < 2: return None, 0.0, "needs review"

scores = {
    k: sum(similarity(v, w) for j, w in ok.items() if j != k) / (len(ok) - 1)
    for k, v in ok.items()
}
best = max(scores, key=scores.get)
status = "auto" if scores[best] >= threshold else "needs review"
return ok[best], scores[best], status

The score averages each output's similarity to the other engines' outputs and does not count votes. With three engines, two that agree and one that differs can still land below the threshold and go to review. That is intended: when the engines split, a person looks.

Step 3: run it over your strings python import json

def localize(path, lang, engines): strings = json.load(open(path)) done, review = {}, {} for key, source in strings.items(): candidates = {name: fn(source, lang) for name, fn in engines.items()} text, score, status = pick(source, candidates) if status == "auto": done[key] = text else: review[key] = {"source": source, "options": candidates, "score": score} return done, review

Here engines is a dict of functions that wrap whichever translation services you already have access to. The logic does not depend on any particular one.

Ship the done strings and send the review file to a native speaker. Add their fixes to a glossary so the next run starts with better terms.

What to watch Agreement is a triage signal. Engines can agree and still be wrong, especially when they share the same weakness, so spot-check a sample of the auto-approved strings too. Short strings disagree more. "Save" and "Post" are ambiguous without context, so add context notes or keep a glossary for them. Languages with different word order and scripts score differently. Check the review list for your first batch and tune the threshold per language. Keep a human in the loop for anything legal, medical or customer-facing. FAQ How many engines should I compare?

The script needs at least two engines that kept your placeholders. Three lets you see which output sits closest to the rest.

What if every engine agrees and the translation is still wrong?

It can happen when the engines share a weakness. Spot-check a sample of the auto-approved strings, and add the terms you catch to your glossary.

Can this replace a human reviewer?

No. It decides which strings a reviewer looks at first. For legal, medical and customer-facing text, a person should still review the result.

--

MachineTranslation.com compares several AI engines side by side. The script above works with any engines. For more on comparing engines, see how consensus across engines works and what accurate translation means for a language pair.