Smaller than six tested configurations
TransHero471 MB
58.5% less storage for TransHero
63.3% less storage for TransHero
64.1% less storage for TransHero
93.8% less storage for TransHero
94.1% less storage for TransHero
95.0% less storage for TransHero
Google Gemma 4 E2B was measured using a local F16 GGUF conversion of Google’s original weights.
These figures are measured model-file sizes in decimal MB. Each row states the quantization of the file measured.
A correction is part of what was said
Consider “周三下午三点,两个人……不,周四。时间不变,就我一个。” A faithful translation is “Wednesday at 3 p.m., for two people… no, Thursday. Same time, just me.” The listener hears both the original request and the correction. Our translation target keeps that sequence, adjusts word order naturally and removes only meaningless filler. The published usability counts measure whether the listener could understand and act in context. Fidelity needs a separate assessment.
Everyday and complex text on the same Pixel 7 Pro
| Model configuration | 48 everyday texts | 240 complex texts |
|---|
| TransHero | 1.30 s | 4.56 s |
|---|
| Index-Translate-2B · Q4 | 1.78 s | 6.45 s |
|---|
| Tencent Hy-MT2-1.8B · 1.25-bit | 1.84 s | 6.85 s |
|---|
Each configuration ran in a separate session on the same Pixel 7 Pro. The source text, context and direction match for every item. All used four CPU threads and model-specific input formatting. Times are medians for a complete engine response.
Usable translations on the same inputs
| 48 everyday texts | 240 complex texts |
| TransHero | 47/48 | 191/240 |
| Index-Translate-2B · Q4 | 42/48 | 161/240 |
|---|
Separate runs on the same Pixel 7 Pro. The complex set uses 40 scenarios across six directions. An AI reviewer judged usability against the source and supplied context.
Matched everyday and medical scenarios
We compared TransHero and Index-Translate-9B Q6 on the same everyday and medical inputs. A usable translation preserved enough meaning for the listener to act in context.
| Test set | TransHero | Index 9B |
Everyday · 48 items Usable translations | 47/48 | 46/48 |
Medical scenarios · 48 items Usable translations | 43/48 | 39/48 |
Medical scenarios · 48 items Serious meaning errors | 2/48 | 3/48 |
The medical test used eight scenarios in six directions each. One AI reviewer assessed the same source text, target language and context. TransHero ran on a Pixel 7 Pro and Index ran on a Mac.
Same words. Two translations.
When can the warehouse ship?
Original · EN → ZHRelease the shipment tonight—wait, only after the client accepts it. Tonight is just our internal inspection, not completed client acceptance. Please tell the warehouse to wait for an update, and do not promise delivery tomorrow morning.
TransHero今晚就发货——等客户验收完再说。今晚只是内部检查,还没完成客户验收。请让仓库等待通知,不要承诺明天早上就送货。
Index-Translate-9B今晚发货——等等,等客户确认收货后再发。今晚只是我们内部的检查,还不算客户正式验收。请告诉仓库等待进一步通知,不要承诺明天早上就能送达。
TransHero keeps customer acceptance as the condition for shipping. Index adds “确认收货”, asking for confirmation of receipt before the goods have shipped.
Wait for the report before reassuring the family
Original · ZH → JA我说报告还没收到,不是说结果是阴性。请你告诉医生现在还不能确认,收到报告以后再核对;别先跟家里说已经没问题了。
TransHero報告はまだ受け取っていないと言っていますが、結果が陰性というわけではありません。医師には、今は確認できないと伝えてください。報告を受け取った後で確認してください。家に問題がなくなったと先に言わないでください。
Index-Translate-9B検査結果の報告書がまだ届いていないと言ったのは、結果が陰性だと言ったのとは違います。医師には、まだ確定できないこと、報告書が届いてから再確認すること、そして家族にはまだ問題ないとは伝えていないことを伝えてください。
TransHero retained the instruction to hold off telling the family. Index changed it into a statement that the family had yet to be told.
Return the earphones. Keep the charger.
Original Chinese · ZH → EN这套都退掉……不,我只想退耳机,充电器还要。它们是不是必须一起退我不知道,请你先问店员,别先替我同意把两件都交回去。
TransHeroI want to return both of these... No, I only want to return the headphones, and the charger is still needed. I don't know if they have to be returned together. Please ask the clerk first. Don't agree to return both for me.
Google Gemma 4 E2BI want to return this whole set... no, I just want to return the headphones, and the charger too. I don't know if they have to be returned together, please ask the clerk first, don't agree to return both items for me first.
TransHero preserves the request to keep the charger. Gemma’s “and the charger too” puts the charger on the return list.
Read the field notes: testing ten more languages →