The obsession with “bigger is better” AI has led many consumers to assume that more parameters mean better performance. This isn’t always true, particularly for specific use cases.
Smaller AI models that can run on the device itself, rather than in the cloud, are a better fit for use cases like translation apps.
While your laptop may handle the heavy computational requirements of multi-purpose AI models trained on models with billions of parameters, your mobile phone and smartwatch may struggle.
In the case of translation, and in more constrained use cases, some developers are building smaller, purpose-built translation models that do one thing well.
Tether is one of the companies building in this direction, with an AI team that has developed dedicated, resource-optimized translation models designed to run entirely on-device.
Your translations should stay on your device
A translation app directly gains access to your conversation. Just like your mobile chat app, it handles information, some of which may be confidential. This is the same even when the translator doesn’t communicate directly with your chat application, for example, by copying and pasting text into the translator.
However, most translation apps are connected to the cloud. That means your text, whether it’s a business contract, a medical record, or a private conversation, leaves your device every time you use it. Local translation eliminates that.
Consequently, developers are exploring online translators and embracing a local-first, offline approach as the only viable way to create effective translators without a single point of failure.
When less is better
Developing models dedicated to a single purpose allows developers to trim down the size and execution costs for the models. Prototypes designed this way are usually only a few megabytes (MB) in size and can run on regular devices.
As demonstrated by Tether’s AI team, Tether’s Bergamot-compatible models require only 21-35 MB per language pair and efficiently translate inputs at ~46ms per sentence, which is ~78 times faster than the 2-billion-parameter Salamandra model.
The modularity of these dedicated models is their biggest appeal. They are lightweight, edge-optimized, and flexible enough to fit into heterogeneous systems with a minimal footprint. This means easier integrations and more practicality.
These models are ideal for local AI due to their low compute requirements. They can be installed and run on any device, including mid-range mobile phones and IoT devices. On-premises integration for this use case offers even greater advantages for users.
Unifying efficient, local-first translation models
As considerable progress has already been made in developing local and offline translation models, Tether is leading the next stage: Application. Tether’s QVAC SDK unifies these models into a single wrapper and provides a directory of models that allows users and developers to choose a preferred model for each language pair.
Tether’s QVAC SDK simplifies Neural Machine Translation (NMT) model implementation for developers and regular users by providing prebuilt modules that enable anyone to select, deploy, and manage intelligent language translators in their applications.
It packages language pairs as dependencies that can be loaded via simple import statements and used in code to handle language translation requests.
The SDK provides primitives for intelligent language translation, supports practical scenarios (single sentences and multiple sentences via batch translation), and a fallback for the unlikely situation where the specialized, lightweight NMT models are unable to meet a developer’s or users’ needs.
The fallback is an LLM-based translation framework for training new language models or running direct translations.
Scaling efficiently to hundreds of languages
The translation component in the QVAC SDK supports multiple language pairs, reducing the number of packages required to run a translation system across hundreds of languages. For instance, a developer would normally need 2 language pairs to create a complete English-to-Chinese translation system (ENG-ZH and ZH-ENG).
To support 26 languages, this dramatically grows to 650 translation directions. The English-pivot model in QVAC SDK reduces this to just 50 language pairs for a 26-language translator.
Tether’s QVAC SDK brings resource-efficient, local, and private translation to everyone’s doorstep. It complements the progress made with the NMT technology by scaling adoption for limitless application scenarios.
Tether’s vision to support local and edge-first AI development also extends to its other technology offerings, including Brain-Computer Interfaces (BCI). Brain OS, the Brain Operating System designed by Tether AI Research engineering team and built on top of Tether’s QVAC AI platform, aims to create an open-source brain operating system that connects to the user’s personal BCI. The idea is that our most important data (our thoughts) should always remain private and owned by us.
Start building intelligent local and edge-first applications with multi-language support. Check out the QVAC repo.
