Kingston Kuan

Dictating anywhere with NVIDIA open models - Nemotron ASR + Tambourine

Unknown — Tue, 03 Feb 2026 00:00:00 +0000

I saw NVIDIA’s announcement</a> on X about their new series of open Nemotron models and ended up reading a blog post from Daily.co</a> about building voice agents with Nemotron and Pipecat.</p>

I’ve been using voice dictation daily since building Tambourine</a>, so I decided to test Nemotron with it. It’s fast for real-time use, and the accuracy is solid, unfortunately you do need a CUDA GPU. Here’s how I set it up.</p>

My Setup</h2>

Tambourine is the desktop app that ties everything together. It captures audio, sends it through the pipeline, and types the result wherever my cursor is.</p>

[Microphone] -> [Nemotron ASR] -> [LLM] -> [Insert text at cursor]</span></span></code></pre>
There are great options for AI voice dictation like Wispr Flow or Willow, but Tambourine is open source and allows you to have full control over the pipeline: swap STT/ASR and LLM, customizable formatting prompts, and import dictionaries for different domains.</p>
As Tambourine is built on top of Pipecat, adding Nemotron as an ASR provider was straightforward.</p>
I’m running this on a desktop with an RTX 4080 Super. Nemotron ASR requires a GPU with CUDA support, which means Windows or Linux only. macOS users can still use Tambourine with cloud STT/ASR providers.</p>
You can use any LLM provider compatible with Pipecat, I tend to use Cerebras</a>  with gpt-oss-120b</code> for fast cloud inference.</p>
Setting Up Nemotron ASR</h2>
To run only the ASR model via Docker, I forked PipeCat’s example</a> to strip it down to an ASR-only image</a>.</p>
git</span> clone https://github.com/kstonekuan/nemotron-january-2026</span></span>
cd</span> nemotron-january-2026</span></span>
</span>
docker</span> build -f Dockerfile.asr -t nemotron-asr .</span></span>
</span>
docker</span> run --gpus all -p 9765:9765</span> \</span></span>
  -v ~/.cache/huggingface:/root/.cache/huggingface</span> \</span></span>
  nemotron-asr</span></span></code></pre>
Once running, you can check the logs in docker.</p>
Setting Up Tambourine</h2>
See the Tambourine repo</a> for full setup instructions.</p>
To use Nemotron, set NEMOTRON_ASR_URL</code> in .env</code> and select Nemotron as the STT provider in the app settings.</p>
NEMOTRON_ASR_URL=ws://localhost:9765/</span></span></code></pre>
If running the Tambourine server in docker as well you might have to use</p>
NEMOTRON_ASR_URL=ws://host.docker.internal:9765/</span></span></code></pre>Wrap Up</h2>
I’ve really enjoyed exploring this stack, and look forward to trying out new models as they come out.
AI-powered voice dictation has changed how I interact with all my apps.
Especially when it comes to using AI tools like Claude Code or ChatGPT, I find myself reaching for the keyboard less and just thinking out loud, trusting Tambourine to capture my intent on top of the words.</p>
Quick Links:</strong></p>

Tambourine</a></li>
Pipecat Nemotron Example Fork</a></li>
NVIDIA Announcement</a></li>
Daily.co Blog Post</a></li>
PipeCat</a></li>
Cerebras</a></li>
</ul>



Why is it so hard to get rid of deepfakes?
Unknown — Sat, 07 May 2022 00:00:00 +0000
Originally published on Medium</a>.</em></p>
</p>
On April 17, 2018, the world was greeted by Former President Barack Obama saying outrageous things in a video titled “You Won’t Believe What Obama Says In This Video!”. About 30 seconds in, it is revealed that is a deepfake allowing the famous comedian Jordan Peele to use the face and voice of “Obama”. As technology advances further, higher forms of media manipulation such as video and sound have been made possible and must be addressed quickly.</p>

    
    </iframe>
</div>
Deepfake technology</a> can produce fake content indistinguishable from real content by the human eye. This technology has risen in popularity due to abuse in sensational media by the spread of misinformation and infringement of privacy. However, not only is it impossible to eradicate these misuses completely but the core technology also has other benefits and thus does not warrant an outright ban of its applications. Furthermore, regardless of the benefits or detriments, the advent of deepfake technology means media regulations must keep up before widespread use is commonplace. While technology is being developed to definitively identify Deepfakes</a>, new regulations</a> should be put in place to cater to their uses. Specialized categories should be set up on platforms to ensure users comply with the rules and technical agencies must be set up for appropriate policing.</p>
</p>
Manual manipulation of images and videos is neither new nor rare and the war on misinformation has been fought on many fronts. Ranging from digital editing in marketing to false evidence in court, a big hurdle is being faced now with deepfakes being created by artificial intelligence. Yet, it is believed that the very technology capable of detecting deepfakes</a> can also be used to improve them and thus even harder to detect. In a battle of computing power and datasets, it is imperative to not only support the detection but be able to keep this away from nefarious users. Yet the development of such powerful technology has traditionally been open source and only possible with the collaboration of many individuals made possible by the interconnectivity of the web. It is not reasonable to rely on such teams to be able to securely store their algorithms. It is better to instead consider a hybrid approach to development where the project can be managed by trusted parties such as a government body or large private institution while still allowing for open contributions.</p>
</p>
Technology evolves rapidly and where public regulations often fail to keep up, fast-moving private bodies are well-suited for this role. Given that the open-source nature of artificial intelligence development is unlikely to change significantly, regulations prove even more crucial than combative technology. Media platforms, such as Youtube or Facebook, are often in the most appropriate position to regulate deepfake technology</a>. Not only do they have access to large datasets, but they have already been dealing with similar hostile uses of their platforms and have amassed many skilled employees. The power given to these privately owned platforms by the internet has already been immense, being able to affect economics, politics, and societies on a large scale. Indeed, with this should come the responsibility to police their own platforms and be punished for not investing heavily in countermeasures.</p>
Current measures to combat deepfakes have been to put a lid on them rather than attempts at integration. Yet, completely erasing deepfakes is a fruitless endeavor. It would more productive to embrace deepfakes with their own category</a> on these platforms for creative uses. In this manner, the technology may still develop while being healthily regulated. By complying with ethical and legal standards, personal privacy and commercial rights can be protected more smoothly. Desensitisation will also play an essential role in aiding the public with understanding deepfake technology which helps them to identify and report illegal activities. With the stigma behind deepfakes shifted away from sensational topics such as politics and obscene material, the research behind combative technologies also gains an edge in terms of financial and technical support.</p>
</p>
Existing agencies still play important roles</a> in attempting to regulate deepfakes. However, their current capabilities are highly restricted. A new agency would be more suited to dealing with the impending threat of deepfakes. This agency must be technically equipped to deal with deepfakes and must also be able to administer appropriate penalties to rule breakers. As the internet has no borders, an international agency, like what the World Health Organisation is to healthcare, needs to create general guidelines that can thereafter be tailored to individual countries’ needs. This body should punish individual agencies for not being able to properly police within their jurisdiction.</p>
On the other hand, it is worth analyzing the deepfake situation from a social perspective. We need to understand the reasons behind the creation of deepfakes and their uses</a>. As advanced as technology gets, even though the intelligence being used is artificial, there is always a human aspect to how and when it is used. By focusing on the community and their technological literacy, malicious use is reduced, and misconceptions corrected. As a remedy, these measures are hard to quantify but similarly, it is known that better education contributes to a lower crime rate</a>. Targeting the root behind the misuse of deepfakes also allows greater freedom in its research which has wider benefits than just entertainment.</p>
In conclusion, deepfakes are very much a part of modern technology and therefore difficult to eradicate. Governments and platforms have important roles to play in contributing to the development of technology capable of identifying deepfakes. As fighting solely on the technical front is insufficient, laws and regulations must attempt to pull ahead by catering to expected growth in deepfakes with deliberate exposure, the appointment of responsibility, and education. If left unchecked, deepfakes may begin to rule all media and the distortion of reality may become the rule rather than the exception.</p>


Building a chatbot for Angel & Mortal
Unknown — Sun, 13 Jun 2021 00:00:00 +0000
Originally published on Medium</a>.</em></p>
</p>
I was starting an hour long commute home when my phone came abuzz. A friend was organizing the annual Angel & Mortal game for our residential college community and was interested in leveraging some online platform for anonymous communication between players. The rules of Angel & Mortal are simple:</p>

Each player is assigned a Mortal to whom they act as an Angel.</li>
The Angel must give gifts to or play pranks on their Mortal without being discovered.</li>
</ul>
Typically, players would send messages to the organizers to be manually forwarded to their Mortal or Angel, asking for their likes/dislikes or informing them of new gifts. I immediately found this project to be simple yet impactful and was compelled to help. Unfortunately the game was starting soon, in a week, so I had to work fast.</p>
Overview</h2>
This bot was developed in Python using the python-telegram-bot</a> package. This allows you to run a simple server that will poll for messages sent to your bot on telegram. Before starting, you will need to set up a bot using the BotFather</em> and generate a token (instructions here</a>). The initial player data are loaded from CSV.</p>
Original code used when I was running the bot (sorry if it’s messy!) can be found here: https://github.com/kstonekuan/angel-mortal-bot</a></strong></p>
Development</h2>
Player class</h3>
Before diving into the chatbot logic, it’s important to understand the theory of the game itself. Each player has an Angel and a Mortal. Furthermore, your Angel is another player in the game who has you as their Mortal and vice versa for your Mortal. From an OOP</em> perspective, each player can be represented as an object that references other player objects</strong>. Along with some telegram attributes, this can be represented by a Player</em> class.</p>
class</span> Player</span>:</span></span>
    def</span> __init__</span>(</span>self</span>,</span> username</span>,</span> room</span>,</span> angel</span>,</span> mortal</span>):</span></span>
        self</span>.</span>username</span> =</span> username</span></span>
        self</span>.</span>room</span> =</span> room</span></span>
        self</span>.</span>angel</span> =</span> angel</span></span>
        self</span>.</span>mortal</span> =</span> mortal</span></span>
        self</span>.</span>chat_id</span> =</span> None</span></span></code></pre>
</p>
While not crucial, you might recognise this as a graph node from graph theory. In particular, the game is designed as a directed graph</a>. Each node has two outgoing edges (to the Angel and Mortal) and two incoming edges (from the Angel and Mortal).</p>
Loading players</h3>
While it might be possible to use our system to generate Angel-Mortal mappings, organizers often have a more in-depth understanding of social dynamics within the group. I chose to load provided mappings from a CSV file</strong>, as opposed to using something more complex (but efficient at scale), as it was simpler for organizers to modify the data. After loading, players can be stored in a dictionary or hash map in memory, similar to an adjacency list.</p>
def</span> load_players</span>(</span>filename</span>):</span></span>
    players</span> =</span> {}</span></span>
    with</span> open</span>(</span>filename</span>,</span> 'r'</span>)</span> as</span> f</span>:</span></span>
        reader</span> =</span> csv</span>.</span>DictReader</span>(</span>f</span>)</span></span>
        for</span> row</span> in</span> reader</span>:</span></span>
            players</span>[</span>row</span>[</span>'</span>username</span>'</span>]]</span> =</span> Player</span>(</span></span>
                row</span>[</span>'</span>username</span>'</span>],</span></span>
                row</span>[</span>'</span>room</span>'</span>],</span></span>
                row</span>[</span>'</span>angel</span>'</span>],</span></span>
                row</span>[</span>'</span>mortal</span>'</span>]</span></span>
            )</span></span>
    # Link player objects</span></span>
    for</span> player</span> in</span> players</span>.</span>values</span>():</span></span>
        player</span>.</span>angel</span> =</span> players</span>[</span>player</span>.</span>angel</span>]</span></span>
        player</span>.</span>mortal</span> =</span> players</span>[</span>player</span>.</span>mortal</span>]</span></span>
    return</span> players</span></span></code></pre>Forwarding messages</h3>
Note: I reference parts of the python-telegram-bot API without explaining them, please refer to their tutorials</a> for this.</em></p>
You might have noticed that our Player</em> class includes a username</em> and chat_id</em>. The username</em> is used to identify players and is provided by the organizers while chat_id</em> is what is used to send messages. We cannot send messages directly to a username</em> as it would be unclear how we want the bot to communicate such as in a private message or group chat. Each user needs to start a chat with the bot first in order to get the chat_id</em> that is generated by Telegram</strong>. I also went one step further to reject users that do not appear as a key in the dictionary as they are not registered players in the game.</p>
def</span> start</span>(</span>update</span>,</span> context</span>):</span></span>
    username</span> =</span> update</span>.</span>effective_user</span>.</span>username</span></span>
    if</span> username</span> not in</span> players</span>:</span></span>
        update</span>.</span>message</span>.</span>reply_text</span>(</span>"You are not registered for this game."</span>)</span></span>
        return</span></span>
</span>
    players</span>[</span>username</span>].</span>chat_id</span> =</span> update</span>.</span>effective_chat</span>.</span>id</span></span>
    update</span>.</span>message</span>.</span>reply_text</span>(</span>f</span>"Welcome </span>{</span>username</span>}</span>! You can now send messages."</span>)</span></span></code></pre>
Conversation handlers</a> can be used to create simple menus to guide users through your bot flow. For this bot the user must first choose either their Angel or Mortal before sending their messages across. They may cancel the conversation at any time to switch between the two. Thanks to the class created in previous steps, it is not difficult to find the reference to their Mortal or Angel and get their respective chat_id</em> to forward messages</strong>.</p>
def</span> send_to_mortal</span>(</span>update</span>,</span> context</span>):</span></span>
    username</span> =</span> update</span>.</span>effective_user</span>.</span>username</span></span>
    player</span> =</span> players</span>[</span>username</span>]</span></span>
    mortal</span> =</span> player</span>.</span>mortal</span></span>
</span>
    if</span> mortal</span>.</span>chat_id</span> is</span> None</span>:</span></span>
        update</span>.</span>message</span>.</span>reply_text</span>(</span>"Your mortal hasn't started the bot yet."</span>)</span></span>
        return</span></span>
</span>
    # Forward the message</span></span>
    context</span>.</span>bot</span>.</span>send_message</span>(</span></span>
        chat_id</span>=</span>mortal</span>.</span>chat_id</span>,</span></span>
        text</span>=</span>f</span>"Message from your Angel:</span>\n\n{</span>update</span>.</span>message</span>.</span>text</span>}</span>"</span></span>
    )</span></span>
    update</span>.</span>message</span>.</span>reply_text</span>(</span>"Message sent to your mortal!"</span>)</span></span></code></pre>
The result should look something like this (with custom names for Angel and Mortal):</p>
</p>
Deployment</h3>
So far you might have been testing your bot locally but it is probably not a good idea to keep to run this on your own machine for the entire duration of such events. I personally made use of Microsoft Azure free student credits</a> to host but there are many options</a> available.</p>
Improvements</h3>
As this was built in a few days with little time for testing there are many improvements you may want to make.</p>

Common feedback from players was that they could not send pictures or emojis</strong> through the bot. I had forgotten to account for this which might have to do with the Filters</a> module.</li>
For long term games that require modifications or continuity you may also consider building a proper backend database</strong> such as using SQL or NoSQL</a> instead of a simple CSV.</li>
Using the Player</em> class and directed graph design, you can develop advanced algorithms</strong> to generate, validate or even modify the player graph for larger games.</li>
</ol>

I really enjoyed building this bot and helping out my community. This is my first article about programming so I hope you enjoyed reading and am always open to feedback about my writing!</p>